chore: import upstream snapshot with attribution
Build and push multi-arch DocsGPT Docker image / build (linux/amd64, ubuntu-latest, amd64) (push) Has been cancelled
Backend release / release (push) Has been cancelled
Bandit Security Scan / bandit_scan (push) Has been cancelled
Build and push multi-arch DocsGPT Docker image / build (linux/arm64, ubuntu-24.04-arm, arm64) (push) Has been cancelled
Build and push multi-arch DocsGPT Docker image / manifest (push) Has been cancelled
Build and push DocsGPT FE Docker image for development / build (linux/amd64, ubuntu-latest, amd64) (push) Has been cancelled
Build and push DocsGPT FE Docker image for development / build (linux/arm64, ubuntu-24.04-arm, arm64) (push) Has been cancelled
Build and push DocsGPT FE Docker image for development / manifest (push) Has been cancelled
Python linting / ruff (push) Has been cancelled
Run python tests with pytest / Run tests and count coverage (3.12) (push) Has been cancelled
React Widget Build / build (push) Has been cancelled
Build and push multi-arch DocsGPT Docker image / build (linux/amd64, ubuntu-latest, amd64) (push) Has been cancelled
Backend release / release (push) Has been cancelled
Bandit Security Scan / bandit_scan (push) Has been cancelled
Build and push multi-arch DocsGPT Docker image / build (linux/arm64, ubuntu-24.04-arm, arm64) (push) Has been cancelled
Build and push multi-arch DocsGPT Docker image / manifest (push) Has been cancelled
Build and push DocsGPT FE Docker image for development / build (linux/amd64, ubuntu-latest, amd64) (push) Has been cancelled
Build and push DocsGPT FE Docker image for development / build (linux/arm64, ubuntu-24.04-arm, arm64) (push) Has been cancelled
Build and push DocsGPT FE Docker image for development / manifest (push) Has been cancelled
Python linting / ruff (push) Has been cancelled
Run python tests with pytest / Run tests and count coverage (3.12) (push) Has been cancelled
React Widget Build / build (push) Has been cancelled
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,149 @@
|
||||
---
|
||||
title: Access Control, Roles & Teams
|
||||
description: How DocsGPT models permissions — global admin/user roles (RBAC), the admin dashboard, teams with per-team roles, and sharing agents, sources, prompts, and tools.
|
||||
---
|
||||
|
||||
import { Callout } from 'nextra/components'
|
||||
|
||||
# Access Control, Roles & Teams
|
||||
|
||||
DocsGPT has two independent permission planes:
|
||||
|
||||
- **Global RBAC** — every user holds the `admin` or `user` role for the whole instance. Admins can manage other users and see instance-wide usage and audit data.
|
||||
- **Team roles** — within a team a user is a `team_admin` or `team_member`, and resources can be shared to teams or individual members.
|
||||
|
||||
The two planes never mix: a global `admin` is a superuser over *all* teams, but a `team_admin` is **not** a global admin.
|
||||
|
||||
<Callout type="info" emoji="🔒">
|
||||
Roles are resolved **server-side on every request** and are never trusted from the JWT. The frontend route guards are cosmetic — the server-side decorators are the real boundary. Persisted roles apply under `AUTH_TYPE=oidc`; token-only modes (`simple_jwt`, `session_jwt`) can never hold the admin role.
|
||||
</Callout>
|
||||
|
||||
## Global roles (RBAC)
|
||||
|
||||
| Role | Capabilities |
|
||||
| --- | --- |
|
||||
| `user` | The default. Owns their own conversations, sources, agents, prompts, and tools. |
|
||||
| `admin` | Everything a user can do, plus the [admin dashboard](#admin-dashboard): user management, force-logout, role management, and instance-wide usage/audit. |
|
||||
|
||||
To see the roles the current request resolves to, call:
|
||||
|
||||
```text
|
||||
GET /api/user/me → { "user_id": "...", "roles": ["user"], "email": "...", "name": "...", "picture": "..." }
|
||||
```
|
||||
|
||||
This is the canonical way to check a caller's effective roles (distinct from `/api/config`, which only reports the instance `auth_type`).
|
||||
|
||||
## Granting admin
|
||||
|
||||
There are four ways an account becomes an admin:
|
||||
|
||||
1. **The `grant_admin` script** — run `python scripts/grant_admin.py <user_id>` on the server to grant (or `--revoke` / `--list`) the admin role directly. This is the canonical bootstrap for the first admin.
|
||||
2. **OIDC group mapping** — `OIDC_ADMIN_GROUPS` auto-grants admin to members of the listed IdP groups. See [below](#admin-via-oidc-groups).
|
||||
3. **Local no-auth mode** — `LOCAL_MODE_ADMIN=true` grants admin when `AUTH_TYPE=None` (self-host, no authentication).
|
||||
4. **Grant by an existing admin** — through the admin API/dashboard.
|
||||
|
||||
| Setting | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `OIDC_ADMIN_GROUPS` | — | Comma-separated IdP groups whose members are granted the global `admin` role. Re-checked at every login and silent renewal. Unset = no OIDC admin mapping. |
|
||||
| `LOCAL_MODE_ADMIN` | `false` | Grants admin in no-auth mode only (`AUTH_TYPE=None`). |
|
||||
|
||||
<Callout type="warning" emoji="⚠️">
|
||||
**Never set `LOCAL_MODE_ADMIN=true` on a networked deployment.** It only makes sense for a single-user local install with `AUTH_TYPE=None`, where there is no identity to check.
|
||||
</Callout>
|
||||
|
||||
### Bootstrapping the first admin
|
||||
|
||||
Granting admin through the dashboard itself requires *already being* an admin, which creates a chicken-and-egg problem on a fresh deployment. Break it one of these ways:
|
||||
|
||||
- **Run the script** on the server: `python scripts/grant_admin.py <user_id>` (the `user_id` is the OIDC `sub`). The grant is written to `user_roles` with `source='manual'` and takes effect on the user's next request. Use `--list` to see current admins and `--revoke` to remove a manual grant.
|
||||
- Set `OIDC_ADMIN_GROUPS` to a group you belong to, and sign in — you become an admin automatically.
|
||||
- For a no-auth local install, use `LOCAL_MODE_ADMIN=true`.
|
||||
|
||||
### Admin via OIDC groups
|
||||
|
||||
When `OIDC_ADMIN_GROUPS` is set, group membership is mapped to the admin role at every sign-in *and* every [silent renewal](/Deploying/OIDC-SSO#silent-session-renewal) — so removing a user from the admin group revokes their admin at the next renewal, just like the [sign-in allowlist](/Deploying/OIDC-SSO#restricting-sign-in-by-group). It is independent of `OIDC_ALLOWED_GROUPS` (which controls *whether* a user may sign in at all). Leaving `OIDC_ADMIN_GROUPS` unset never mass-revokes admin.
|
||||
|
||||
```env
|
||||
OIDC_ADMIN_GROUPS=platform-admins
|
||||
# OIDC_GROUPS_CLAIM=groups # only if your IdP uses a different claim name
|
||||
```
|
||||
|
||||
If your IdP only exposes groups via the userinfo endpoint, DocsGPT backfills them from there during reconciliation, the same as the allowlist.
|
||||
|
||||
## Admin dashboard
|
||||
|
||||
Admins get a dashboard backed by a REST surface under `/api/admin` (every endpoint requires the admin role). All mutating actions are written to the `auth_events` audit log with the acting admin recorded in the metadata.
|
||||
|
||||
| Method | Path | Description |
|
||||
| --- | --- | --- |
|
||||
| `GET` | `/api/admin/overview` | Instance overview counts. |
|
||||
| `GET` | `/api/admin/users` | List users (paginated; supports a `user_id` filter). |
|
||||
| `GET` | `/api/admin/users/<id>` | Drill into a single user. |
|
||||
| `PATCH` | `/api/admin/users/<id>` | Activate / deactivate a user (deactivation revokes their sessions). |
|
||||
| `POST` | `/api/admin/users/<id>/role` | Grant the admin role (guards against removing the last admin). |
|
||||
| `DELETE` | `/api/admin/users/<id>/role` | Revoke the admin role. |
|
||||
| `POST` | `/api/admin/users/<id>/revoke-sessions` | Force-logout a user. |
|
||||
| `GET` | `/api/admin/admins` | List current admins. |
|
||||
| `GET` | `/api/admin/usage` | Token-usage series and top users. |
|
||||
| `GET` | `/api/admin/audit` | Authentication/admin audit feed. |
|
||||
| `GET` | `/api/admin/devices/audit` | Remote-device audit feed. |
|
||||
| `GET` | `/api/admin/teams` | Instance-wide oversight of all teams. |
|
||||
|
||||
<Callout type="info" emoji="ℹ️">
|
||||
Deactivating a user via the dashboard works for any auth type, while OIDC deployments can also offboard through [SCIM](/Deploying/OIDC-SSO#scim-user-provisioning). Both revoke live sessions immediately.
|
||||
</Callout>
|
||||
|
||||
## Teams
|
||||
|
||||
Teams let a group of users collaborate and share resources. Teams are self-serve — any user can create one and manage its membership.
|
||||
|
||||
### Team roles
|
||||
|
||||
| Role | Capabilities |
|
||||
| --- | --- |
|
||||
| `team_member` | Belongs to the team; can use resources shared with the team. |
|
||||
| `team_admin` | Manages membership and team settings. Implies `team_member`. |
|
||||
|
||||
The team **owner** is a distinct concept from `team_admin`: ownership can be transferred, and a global `admin` is treated as a superuser over every team.
|
||||
|
||||
### Managing a team
|
||||
|
||||
| Method | Path | Description |
|
||||
| --- | --- | --- |
|
||||
| `GET` / `POST` | `/api/teams` | List your teams / create a team. |
|
||||
| `GET` / `PATCH` / `DELETE` | `/api/teams/<id>` | Read, update, or delete a team. |
|
||||
| `GET` / `POST` | `/api/teams/<id>/members` | List members / add a member. |
|
||||
| `PATCH` / `DELETE` | `/api/teams/<id>/members/<member_id>` | Change a member's role / remove (or leave). |
|
||||
| `POST` | `/api/teams/<id>/transfer_owner` | Transfer ownership. |
|
||||
|
||||
Membership and removal are guarded so a team can never be left without an admin (the last-admin guard).
|
||||
|
||||
<Callout type="info" emoji="ℹ️">
|
||||
Members are added by email (or subject id). The invitee **must have signed in at least once** so DocsGPT can resolve them to a user account.
|
||||
</Callout>
|
||||
|
||||
## Sharing resources with a team
|
||||
|
||||
Four resource types can be shared: **agents**, **sources**, **prompts**, and **tools**. Sharing is **additive** — it grants access to others without changing ownership.
|
||||
|
||||
| Method | Path | Description |
|
||||
| --- | --- | --- |
|
||||
| `GET` / `POST` / `DELETE` | `/api/teams/<id>/grants` | List, create, or revoke a share within a team. |
|
||||
| `GET` | `/api/resource_shares` | List shares visible to the caller. |
|
||||
|
||||
Sharing rules:
|
||||
|
||||
- Only the **owner** of a resource can share it.
|
||||
- A share targets either the **whole team** or a **single member**.
|
||||
- Each share carries an access level: **`viewer`** (read-only) or **`editor`** (read and modify).
|
||||
- `editor` is not the same as owner — an editor can change a resource but cannot delete it or re-share it.
|
||||
- Shared tools run server-side with the **owner's** credentials; a grantee never sees the owner's secrets.
|
||||
|
||||
## Audit log
|
||||
|
||||
Access-control actions are appended to the `auth_events` table alongside the [authentication events](/Deploying/OIDC-SSO#login-auditing). This includes admin actions — `admin_user_activated` / `admin_user_deactivated`, `admin_sessions_revoked`, `role_granted` / `role_revoked` (with `metadata.source` = `manual` or `oidc_group`) — and team events (`team.create`, `team.member_add`, `team.member_role`, `team.member_remove`, `team.share`, `team.unshare`, `team.transfer_owner`, `team.delete`). The acting admin is recorded in the event metadata.
|
||||
|
||||
## Related
|
||||
|
||||
- [SSO with OIDC](/Deploying/OIDC-SSO) — sign-in, group allowlists, and the `auth_events` table.
|
||||
- [App Configuration](/Deploying/DocsGPT-Settings) — the full settings reference.
|
||||
@@ -0,0 +1,110 @@
|
||||
---
|
||||
title: Hosting DocsGPT on Amazon Lightsail
|
||||
description:
|
||||
display: hidden
|
||||
---
|
||||
|
||||
# Self-hosting DocsGPT on Amazon Lightsail
|
||||
|
||||
Here's a step-by-step guide on how to set up an Amazon Lightsail instance to host DocsGPT.
|
||||
|
||||
## Configuring your instance
|
||||
|
||||
(If you know how to create a Lightsail instance, you can skip to the recommended configuration part by clicking [here](#connecting-to-your-newly-created-instance)).
|
||||
|
||||
### 1. Create an AWS Account:
|
||||
If you haven't already, create or log in to your AWS account at https://lightsail.aws.amazon.com.
|
||||
|
||||
### 2. Create an Instance:
|
||||
|
||||
a. Click "Create Instance."
|
||||
|
||||
b. Select the "Instance location." In most cases, the default location works fine.
|
||||
|
||||
c. Choose "Linux/Unix" as the image and "Ubuntu 20.04 LTS" as the Operating System.
|
||||
|
||||
d. Configure the instance plan based on your requirements. A "1 GB, 1vCPU, 40GB SSD, and 2TB transfer" setup is recommended for most scenarios.
|
||||
|
||||
e. Give your instance a unique name and click "Create Instance."
|
||||
|
||||
PS: It may take a few minutes for the instance setup to complete.
|
||||
|
||||
### Connecting to Your newly created Instance
|
||||
|
||||
Your instance will be ready a few minutes after creation. To access it, open the instance and click "Connect using SSH."
|
||||
|
||||
#### Clone the DocsGPT Repository
|
||||
|
||||
A terminal window will pop up, and the first step will be to clone the DocsGPT Git repository:
|
||||
|
||||
`git clone https://github.com/arc53/DocsGPT.git`
|
||||
|
||||
#### Download the package information
|
||||
|
||||
Once it has finished cloning the repository, it is time to download the package information from all sources. To do so, simply enter the following command:
|
||||
|
||||
`sudo apt update`
|
||||
|
||||
#### Install Docker and Docker Compose
|
||||
|
||||
DocsGPT backend and worker use Python, Frontend is written on React and the whole application is containerized using Docker. To install Docker and Docker Compose, enter the following commands:
|
||||
|
||||
`sudo apt install docker.io`
|
||||
|
||||
And now install docker-compose:
|
||||
|
||||
`sudo apt install docker-compose`
|
||||
|
||||
#### Access the DocsGPT Folder
|
||||
|
||||
Enter the following command to access the folder in which the DocsGPT docker-compose file is present.
|
||||
|
||||
`cd DocsGPT/`
|
||||
|
||||
#### Prepare the Environment
|
||||
|
||||
Inside the DocsGPT folder create a `.env` file and copy the contents of `.env_sample` into it.
|
||||
|
||||
`nano .env`
|
||||
|
||||
Make sure your `.env` file looks like this:
|
||||
|
||||
```
|
||||
API_KEY=<Your LLM API key>
|
||||
LLM_NAME=docsgpt
|
||||
VITE_API_STREAMING=true
|
||||
```
|
||||
|
||||
To save the file, press CTRL+X, then Y, and then ENTER.
|
||||
|
||||
Next, set the correct IP for the Backend by opening the docker-compose.yml file:
|
||||
|
||||
`nano deployment/docker-compose.yaml`
|
||||
|
||||
And Change line 7 to: `VITE_API_HOST=http://localhost:7091`
|
||||
to this `VITE_API_HOST=http://<your instance public IP>:7091`
|
||||
|
||||
This will allow the frontend to connect to the backend.
|
||||
|
||||
#### Running the Application
|
||||
|
||||
You're almost there! Now that all the necessary bits and pieces have been installed, it is time to run the application. To do so, use the following command:
|
||||
|
||||
`sudo docker compose -f deployment/docker-compose.yaml up -d`
|
||||
|
||||
Launching it for the first time will take a few minutes to download all the necessary dependencies and build.
|
||||
|
||||
Once this is done you can go ahead and close the terminal window.
|
||||
|
||||
#### Enabling Ports
|
||||
|
||||
a. Before you are able to access your live instance, you must first enable the port that it is using.
|
||||
|
||||
b. Open your Lightsail instance and head to "Networking".
|
||||
|
||||
c. Then click on "Add rule" under "IPv4 Firewall", enter `5173` as your port, and hit "Create".
|
||||
Repeat the process for port `7091`.
|
||||
|
||||
#### Access your instance
|
||||
|
||||
Your instance is now available at your Public IP Address on port 5173. Enjoy using DocsGPT!
|
||||
@@ -0,0 +1,188 @@
|
||||
---
|
||||
title: Setting Up a Development Environment
|
||||
description: Guide to setting up a development environment for DocsGPT, including backend and frontend setup.
|
||||
---
|
||||
|
||||
import { Callout } from 'nextra/components'
|
||||
|
||||
# Setting Up a Development Environment
|
||||
|
||||
This guide will walk you through setting up a development environment for DocsGPT. This setup allows you to modify and test the application's backend and frontend components.
|
||||
|
||||
## 1. Spin Up Postgres and Redis
|
||||
|
||||
For development purposes, you can quickly start Postgres and Redis containers. Postgres is the user-data store for DocsGPT (conversations, agents, prompts, sources, attachments, workflows, logs, and token usage), and Redis is used as the cache and Celery broker. We provide a dedicated Docker Compose file, `docker-compose-dev.yaml`, located in the `deployment` directory, that includes only these essential services. The backend applies the Alembic schema automatically on first boot (`AUTO_MIGRATE=true` / `AUTO_CREATE_DB=true` ship enabled), so no separate migration step is required. You can still run `python scripts/db/init_postgres.py` explicitly if you prefer.
|
||||
|
||||
You can find the `docker-compose-dev.yaml` file [here](https://github.com/arc53/DocsGPT/blob/main/deployment/docker-compose-dev.yaml).
|
||||
|
||||
**Steps to start Postgres and Redis:**
|
||||
|
||||
1. Navigate to the root directory of your DocsGPT repository in your terminal.
|
||||
|
||||
2. Run the following commands to build and start the containers defined in `docker-compose-dev.yaml`:
|
||||
|
||||
```bash
|
||||
docker compose -f deployment/docker-compose-dev.yaml build
|
||||
docker compose -f deployment/docker-compose-dev.yaml up -d
|
||||
```
|
||||
|
||||
These commands will start Postgres and Redis in detached mode, running in the background. When the Flask backend boots against the fresh Postgres instance, it will automatically create the database (if missing) and apply the current Alembic schema.
|
||||
|
||||
<Callout type="info" emoji="ℹ️">
|
||||
MongoDB is no longer required for a default DocsGPT install. If you
|
||||
specifically want to use MongoDB Atlas as the vector store
|
||||
(`VECTOR_STORE=mongodb`), start it on the side via
|
||||
`deployment/docker-compose.mongo.yaml`. For migrating an existing
|
||||
Mongo-based install to Postgres, see
|
||||
[PostgreSQL for User Data](/Deploying/Postgres-Migration).
|
||||
</Callout>
|
||||
|
||||
## 2. Run the Backend
|
||||
|
||||
To run the DocsGPT backend locally, you'll need to set up a Python environment and install the necessary dependencies.
|
||||
|
||||
**Prerequisites:**
|
||||
|
||||
* **Python 3.12:** Ensure you have Python 3.12 installed on your system. You can check your Python version by running `python --version` or `python3 --version` in your terminal.
|
||||
|
||||
**Steps to run the backend:**
|
||||
|
||||
1. **Configure Environment Variables:**
|
||||
|
||||
DocsGPT backend settings are configured using environment variables. You can set these either in a `.env` file or directly in the `settings.py` file. For a comprehensive overview of all settings, please refer to the [DocsGPT Settings Guide](/Deploying/DocsGPT-Settings).
|
||||
|
||||
* **Option 1: Using a `.env` file (Recommended):**
|
||||
* If you haven't already, create a file named `.env` in the **root directory** of your DocsGPT project.
|
||||
* Modify the `.env` file to adjust settings as needed. You can find a comprehensive list of configurable options in [`application/core/settings.py`](https://github.com/arc53/DocsGPT/blob/main/application/core/settings.py).
|
||||
|
||||
* **Option 2: Exporting Environment Variables:**
|
||||
* Alternatively, you can export environment variables directly in your terminal. However, using a `.env` file is generally more organized for development.
|
||||
|
||||
2. **Create a Python Virtual Environment (Optional but Recommended):**
|
||||
|
||||
Using a virtual environment isolates project dependencies and avoids conflicts with system-wide Python packages.
|
||||
|
||||
* **macOS and Linux:**
|
||||
|
||||
```bash
|
||||
python -m venv venv
|
||||
. venv/bin/activate
|
||||
```
|
||||
|
||||
* **Windows:**
|
||||
|
||||
```bash
|
||||
python -m venv venv
|
||||
venv/Scripts/activate
|
||||
```
|
||||
|
||||
3. **Download Embedding Model:**
|
||||
|
||||
The backend requires an embedding model. Download the `mpnet-base-v2` model and place it in the `models/` directory within the project root. You can use the following script:
|
||||
|
||||
```bash
|
||||
wget https://d3dg1063dc54p9.cloudfront.net/models/embeddings/mpnet-base-v2.zip
|
||||
unzip mpnet-base-v2.zip -d model
|
||||
rm mpnet-base-v2.zip
|
||||
```
|
||||
|
||||
Alternatively, you can manually download the zip file from [here](https://d3dg1063dc54p9.cloudfront.net/models/embeddings/mpnet-base-v2.zip), unzip it, and place the extracted folder in `models/`.
|
||||
|
||||
4. **Install Backend Dependencies:**
|
||||
|
||||
Navigate to the root of your DocsGPT repository and install the required Python packages:
|
||||
|
||||
```bash
|
||||
pip install -r application/requirements.txt
|
||||
```
|
||||
|
||||
5. **Run the Backend:**
|
||||
|
||||
For local development, run the ASGI composition under uvicorn. It serves the **whole** application, hot-reloads on source changes, and matches the production runtime:
|
||||
|
||||
```bash
|
||||
uvicorn application.asgi:asgi_app --host 0.0.0.0 --port 7091 --reload
|
||||
```
|
||||
|
||||
This makes the backend accessible on `http://localhost:7091`. Production uses `gunicorn -k uvicorn_worker.UvicornWorker` against the same `application.asgi:asgi_app` target.
|
||||
|
||||
A plain Flask run is a faster inner loop (quick startup, the Werkzeug interactive debugger):
|
||||
|
||||
```bash
|
||||
flask --app application/app.py run --host=0.0.0.0 --port=7091
|
||||
```
|
||||
|
||||
But it serves **only** the WSGI Flask app and omits the routes mounted on the ASGI shell in `application/asgi.py`: the `/mcp` FastMCP endpoint and the native-async SSE reconnect reader `GET /api/messages/<id>/events`. Under `flask run` those paths return 404 — chat still works (`POST /stream` is a Flask route), but a stream interrupted by a disconnect won't auto-resume on reconnect. Use `flask run` only when you don't need those routes.
|
||||
|
||||
6. **Start the Celery Worker:**
|
||||
|
||||
Open a new terminal window (and activate your virtual environment if you used one). Start the Celery worker to handle background tasks:
|
||||
|
||||
```bash
|
||||
celery -A application.app.celery worker -l INFO
|
||||
```
|
||||
|
||||
This command will start the Celery worker, which processes tasks such as document parsing and vector embedding.
|
||||
|
||||
**macOS note:** Due to a threading issue, start Celery with the solo pool:
|
||||
|
||||
```bash
|
||||
python -m celery -A application.app.celery worker -l INFO --pool=solo
|
||||
```
|
||||
|
||||
**Running in Debugger (VSCode):**
|
||||
|
||||
For easier debugging, you can launch the Flask app and Celery worker directly from VSCode's debugger.
|
||||
|
||||
* Press <kbd>Shift</kbd> + <kbd>Cmd</kbd> + <kbd>D</kbd> (macOS) or <kbd>Shift</kbd> + <kbd>Windows</kbd> + <kbd>D</kbd> (Windows) to open the Run and Debug view.
|
||||
* You should see configurations named "Flask" and "Celery". Select the desired configuration and click the "Start Debugging" button (green play icon).
|
||||
|
||||
## 3. Start the Frontend
|
||||
|
||||
To run the DocsGPT frontend locally, you'll need Node.js and npm (Node Package Manager).
|
||||
|
||||
**Prerequisites:**
|
||||
|
||||
* **Node.js version 16 or higher:** Ensure you have Node.js version 16 or greater installed. You can check your Node.js version by running `node -v` in your terminal. npm is usually bundled with Node.js.
|
||||
|
||||
**Steps to start the frontend:**
|
||||
|
||||
1. **Navigate to the Frontend Directory:**
|
||||
|
||||
In your terminal, change the current directory to the `frontend` folder within your DocsGPT repository:
|
||||
|
||||
```bash
|
||||
cd frontend
|
||||
```
|
||||
|
||||
2. **Install Global Packages (If Needed):**
|
||||
|
||||
If you don't have `husky` and `vite` installed globally, you can install them:
|
||||
|
||||
```bash
|
||||
npm install husky -g
|
||||
npm install vite -g
|
||||
```
|
||||
You can skip this step if you already have these packages installed or prefer to use local installations (though global installation simplifies running the commands in this guide).
|
||||
|
||||
3. **Install Frontend Dependencies:**
|
||||
|
||||
Install the project's frontend dependencies using npm:
|
||||
|
||||
```bash
|
||||
npm install --include=dev
|
||||
```
|
||||
|
||||
This command reads the `package.json` file in the `frontend` directory and installs all listed dependencies, including development dependencies.
|
||||
|
||||
4. **Run the Frontend App:**
|
||||
|
||||
Start the frontend development server:
|
||||
|
||||
```bash
|
||||
npm run dev
|
||||
```
|
||||
|
||||
This command will start the Vite development server. The frontend application will typically be accessible at [http://localhost:5173/](http://localhost:5173/). The terminal will display the exact URL where the frontend is running.
|
||||
|
||||
With both the backend and frontend running, you should now have a fully functional DocsGPT development environment. You can access the application in your browser at [http://localhost:5173/](http://localhost:5173/) and start developing!
|
||||
@@ -0,0 +1,135 @@
|
||||
---
|
||||
title: Docker Deployment of DocsGPT
|
||||
description: Deploy DocsGPT using Docker and Docker Compose for easy setup and management.
|
||||
---
|
||||
|
||||
# Docker Deployment of DocsGPT
|
||||
|
||||
Docker is the recommended method for deploying DocsGPT, providing a consistent and isolated environment for the application to run. This guide will walk you through deploying DocsGPT using Docker and Docker Compose.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
* **Docker Engine:** You need to have Docker Engine installed on your system.
|
||||
* **macOS:** [Docker Desktop for Mac](https://docs.docker.com/desktop/install/mac-install/)
|
||||
* **Linux:** [Docker Engine Installation Guide](https://docs.docker.com/engine/install/) (follow instructions for your specific distribution)
|
||||
* **Windows:** [Docker Desktop for Windows](https://docs.docker.com/desktop/install/windows-install/) (requires WSL 2 backend, see notes below)
|
||||
* **Docker Compose:** Docker Compose is usually included with Docker Desktop. If you are using Docker Engine separately, ensure you have Docker Compose V2 installed.
|
||||
|
||||
**Important Note for Windows Users:** Docker Desktop on Windows generally requires the WSL 2 backend to function correctly, especially when using features like host networking which are utilized in DocsGPT's Docker Compose setup. Ensure WSL 2 is enabled and configured in Docker Desktop settings.
|
||||
|
||||
## Quickest Setup: Using DocsGPT Public API
|
||||
|
||||
The fastest way to try out DocsGPT is by using the public API endpoint. This requires minimal configuration and no local LLM setup.
|
||||
|
||||
1. **Clone the DocsGPT Repository (if you haven't already):**
|
||||
|
||||
```bash
|
||||
git clone https://github.com/arc53/DocsGPT.git
|
||||
cd DocsGPT
|
||||
```
|
||||
|
||||
2. **Create a `.env` file:**
|
||||
|
||||
In the root directory of your DocsGPT repository, create a file named `.env`.
|
||||
|
||||
3. **Add Public API Configuration to `.env`:**
|
||||
|
||||
Open the `.env` file and add the following lines:
|
||||
|
||||
```
|
||||
LLM_PROVIDER=docsgpt
|
||||
VITE_API_STREAMING=true
|
||||
```
|
||||
|
||||
This minimal configuration tells DocsGPT to use the public API. For more advanced settings and other LLM options, refer to the [DocsGPT Settings Guide](/Deploying/DocsGPT-Settings).
|
||||
|
||||
4. **Launch DocsGPT with Docker Compose:**
|
||||
|
||||
Navigate to the root directory of the DocsGPT repository in your terminal and run:
|
||||
|
||||
```bash
|
||||
docker compose -f deployment/docker-compose.yaml up -d
|
||||
```
|
||||
|
||||
The `-d` flag runs Docker Compose in detached mode (in the background).
|
||||
|
||||
5. **Access DocsGPT in your browser:**
|
||||
|
||||
Once the containers are running, open your web browser and go to [http://localhost:5173/](http://localhost:5173/).
|
||||
|
||||
6. **Stopping DocsGPT:**
|
||||
|
||||
To stop the application, navigate to the same directory in your terminal and run:
|
||||
|
||||
```bash
|
||||
docker compose -f deployment/docker-compose.yaml down
|
||||
```
|
||||
|
||||
## Optional Ollama Setup (Local Models)
|
||||
|
||||
DocsGPT provides optional Docker Compose files to easily integrate with [Ollama](https://ollama.com/) for running local models. These files add an official Ollama container to your Docker Compose setup. These files are located in the `deployment/optional/` directory.
|
||||
|
||||
There are two Ollama optional files:
|
||||
|
||||
* **`docker-compose.optional.ollama-cpu.yaml`**: For running Ollama on CPU.
|
||||
* **`docker-compose.optional.ollama-gpu.yaml`**: For running Ollama on GPU (requires Docker to be configured for GPU usage).
|
||||
|
||||
### Launching with Ollama and Pulling a Model
|
||||
|
||||
1. **Clone the DocsGPT Repository and Create `.env` (as described above).**
|
||||
|
||||
2. **Launch DocsGPT with Ollama Docker Compose:**
|
||||
|
||||
Choose the appropriate Ollama Compose file (CPU or GPU) and launch DocsGPT:
|
||||
|
||||
**CPU:**
|
||||
```bash
|
||||
docker compose --env-file .env -f deployment/docker-compose.yaml -f deployment/optional/docker-compose.optional.ollama-cpu.yaml up -d
|
||||
```
|
||||
**GPU:**
|
||||
```bash
|
||||
docker compose --env-file .env -f deployment/docker-compose.yaml -f deployment/optional/docker-compose.optional.ollama-gpu.yaml up -d
|
||||
```
|
||||
|
||||
3. **Pull the Ollama Model:**
|
||||
|
||||
**Crucially, after launching with Ollama, you need to pull the desired model into the Ollama container.** Find the `LLM_NAME` you configured in your `.env` file (e.g., `llama3.2:1b`). Then execute the following command to pull the model *inside* the running Ollama container:
|
||||
|
||||
```bash
|
||||
docker compose -f deployment/docker-compose.yaml -f deployment/optional/docker-compose.optional.ollama-cpu.yaml exec -it ollama ollama pull <LLM_NAME>
|
||||
```
|
||||
or (for GPU):
|
||||
```bash
|
||||
docker compose -f deployment/docker-compose.yaml -f deployment/optional/docker-compose.optional.ollama-gpu.yaml exec -it ollama ollama pull <LLM_NAME>
|
||||
```
|
||||
Replace `<LLM_NAME>` with the actual model name from your `.env` file.
|
||||
|
||||
4. **Access DocsGPT in your browser:**
|
||||
|
||||
Once the model is pulled and containers are running, open your web browser and go to [http://localhost:5173/](http://localhost:5173/).
|
||||
|
||||
5. **Stopping Ollama Setup:**
|
||||
|
||||
To stop a DocsGPT setup launched with Ollama optional files, use `docker compose down` and include all the compose files used during the `up` command:
|
||||
|
||||
```bash
|
||||
docker compose -f deployment/docker-compose.yaml -f deployment/optional/docker-compose.optional.ollama-cpu.yaml down
|
||||
```
|
||||
or
|
||||
|
||||
```bash
|
||||
docker compose -f deployment/docker-compose.yaml -f deployment/optional/docker-compose.optional.ollama-gpu.yaml down
|
||||
```
|
||||
|
||||
**Important for GPU Usage:**
|
||||
|
||||
* **NVIDIA Container Toolkit (for NVIDIA GPUs):** If you are using NVIDIA GPUs, you need to have the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) installed and configured on your system for Docker to access your GPU.
|
||||
* **Docker GPU Configuration:** Ensure Docker is configured to utilize your GPU. Refer to the [Ollama Docker Hub page](https://hub.docker.com/r/ollama/ollama) and Docker documentation for GPU setup instructions specific to your GPU type (NVIDIA, AMD, Intel).
|
||||
|
||||
## Restarting After Configuration Changes
|
||||
|
||||
Whenever you modify the `.env` file or any Docker Compose files, you need to restart the Docker containers for the changes to be applied. Use the same `docker compose down` and `docker compose up -d` commands you used to launch DocsGPT, ensuring you include all relevant `-f` flags for optional files if you are using them.
|
||||
|
||||
## Further Configuration
|
||||
|
||||
This guide covers the basic Docker deployment of DocsGPT. For detailed information on configuring various aspects of DocsGPT, such as LLM providers, models, vector stores, and more, please refer to the comprehensive [DocsGPT Settings Guide](/Deploying/DocsGPT-Settings).
|
||||
@@ -0,0 +1,448 @@
|
||||
---
|
||||
title: DocsGPT Settings
|
||||
description: Configure your DocsGPT application by understanding the basic settings.
|
||||
---
|
||||
|
||||
import { Callout } from 'nextra/components'
|
||||
|
||||
# DocsGPT Settings
|
||||
|
||||
DocsGPT is highly configurable, allowing you to tailor it to your specific needs and preferences. You can control various aspects of the application, from choosing the Large Language Model (LLM) provider to selecting embedding models and vector stores.
|
||||
|
||||
This document will guide you through the basic settings you can configure in DocsGPT. These settings determine how DocsGPT interacts with LLMs and processes your data.
|
||||
|
||||
## Configuration Methods
|
||||
|
||||
There are two primary ways to configure DocsGPT settings:
|
||||
|
||||
### 1. Configuration via `.env` file (Recommended)
|
||||
|
||||
The easiest and recommended way to configure basic settings is by using a `.env` file. This file should be located in the **root directory** of your DocsGPT project (the same directory where `setup.sh` is located).
|
||||
|
||||
**Example `.env` file structure:**
|
||||
|
||||
```
|
||||
LLM_PROVIDER=openai
|
||||
API_KEY=YOUR_OPENAI_API_KEY
|
||||
LLM_NAME=gpt-4o
|
||||
```
|
||||
|
||||
### 2. Configuration via `settings.py` file (Advanced)
|
||||
|
||||
For more advanced configurations or if you prefer to manage settings directly in code, you can modify the `settings.py` file. This file is located in the `application/core` directory of your DocsGPT project.
|
||||
|
||||
While modifying `settings.py` offers more flexibility, it's generally recommended to use the `.env` file for basic settings and reserve `settings.py` for more complex adjustments or when you need to configure settings programmatically.
|
||||
|
||||
**Location of `settings.py`:** `application/core/settings.py`
|
||||
|
||||
## Basic Settings Explained
|
||||
|
||||
Here are some of the most fundamental settings you'll likely want to configure:
|
||||
|
||||
- **`LLM_PROVIDER`**: This setting determines which Large Language Model (LLM) provider DocsGPT will use. It tells DocsGPT which API to interact with.
|
||||
|
||||
- **Common values:**
|
||||
- `docsgpt`: Use the DocsGPT Public API Endpoint (simple and free, as offered in `setup.sh` option 1).
|
||||
- `openai`: Use OpenAI's API (requires an API key).
|
||||
- `google`: Use Google's Vertex AI or Gemini models.
|
||||
- `anthropic`: Use Anthropic's Claude models.
|
||||
- `groq`: Use Groq's models.
|
||||
- `huggingface`: Use HuggingFace Inference API.
|
||||
- `openai` (when using local inference engines like Ollama, Llama.cpp, TGI, etc.): This signals DocsGPT to use an OpenAI-compatible API format, even if the actual LLM is running locally.
|
||||
|
||||
- **`LLM_NAME`**: Specifies the specific model to use from the chosen LLM provider. The available models depend on the `LLM_PROVIDER` you've selected.
|
||||
|
||||
- **Examples:**
|
||||
- For `LLM_PROVIDER=openai`: `gpt-4o`
|
||||
- For `LLM_PROVIDER=google`: `gemini-3.5-flash`
|
||||
- For local models (e.g., Ollama): `llama3.2:1b` (or any model name available in your setup).
|
||||
|
||||
- **`EMBEDDINGS_NAME`**: This setting defines which embedding model DocsGPT will use to generate vector embeddings for your documents. Embeddings are numerical representations of text that allow DocsGPT to understand the semantic meaning of your documents for efficient search and retrieval.
|
||||
|
||||
- **Default value:** `huggingface_sentence-transformers/all-mpnet-base-v2` (a good general-purpose embedding model).
|
||||
- **Other options:** You can explore other embedding models from Hugging Face Sentence Transformers or other providers if needed.
|
||||
|
||||
- **`API_KEY`**: Required for most cloud-based LLM providers. This is your authentication key to access the LLM provider's API. You'll need to obtain this key from your chosen provider's platform.
|
||||
|
||||
- **`OPENAI_BASE_URL`**: Specifically used when `LLM_PROVIDER` is set to `openai` but you are connecting to a local inference engine (like Ollama, Llama.cpp, etc.) that exposes an OpenAI-compatible API. This setting tells DocsGPT where to find your local LLM server.
|
||||
|
||||
- **`STT_PROVIDER`**: Selects the speech-to-text provider used for microphone transcription in chat and for audio file ingestion through the parser pipeline.
|
||||
|
||||
## Configuration Examples
|
||||
|
||||
Let's look at some concrete examples of how to configure these settings in your `.env` file.
|
||||
|
||||
### Example for Cloud API Provider (OpenAI)
|
||||
|
||||
To use OpenAI's `gpt-4o` model, you would configure your `.env` file like this:
|
||||
|
||||
```
|
||||
LLM_PROVIDER=openai
|
||||
API_KEY=YOUR_OPENAI_API_KEY # Replace with your actual OpenAI API key
|
||||
LLM_NAME=gpt-4o
|
||||
```
|
||||
|
||||
Make sure to replace `YOUR_OPENAI_API_KEY` with your actual OpenAI API key.
|
||||
|
||||
### Example for Local Deployment
|
||||
|
||||
To use a local Ollama server with the `llama3.2:1b` model, you would configure your `.env` file like this:
|
||||
|
||||
```
|
||||
LLM_PROVIDER=openai # Using OpenAI compatible API format for local models
|
||||
API_KEY=None # API Key is not needed for local Ollama
|
||||
LLM_NAME=llama3.2:1b
|
||||
OPENAI_BASE_URL=http://host.docker.internal:11434/v1 # Default Ollama API URL within Docker
|
||||
EMBEDDINGS_NAME=huggingface_sentence-transformers/all-mpnet-base-v2 # You can also run embeddings locally if needed
|
||||
```
|
||||
|
||||
In this case, even though you are using Ollama locally, `LLM_PROVIDER` is set to `openai` because Ollama (and many other local inference engines) are designed to be API-compatible with OpenAI. `OPENAI_BASE_URL` points DocsGPT to the local Ollama server.
|
||||
|
||||
## Adding Custom Models (`MODELS_CONFIG_DIR`)
|
||||
|
||||
DocsGPT ships with a built-in catalog of models for the providers it
|
||||
supports out of the box (OpenAI, Anthropic, Google, Groq, OpenRouter,
|
||||
Novita, Hugging Face, DocsGPT). To add **your own
|
||||
models** without forking the repo — for example, a Mistral or Together
|
||||
account, a self-hosted vLLM endpoint, or any other OpenAI-compatible
|
||||
API — point `MODELS_CONFIG_DIR` at a directory of YAML files.
|
||||
|
||||
```
|
||||
MODELS_CONFIG_DIR=/etc/docsgpt/models
|
||||
MISTRAL_API_KEY=sk-...
|
||||
```
|
||||
|
||||
A minimal YAML for one provider:
|
||||
|
||||
```yaml
|
||||
# /etc/docsgpt/models/mistral.yaml
|
||||
provider: openai_compatible
|
||||
display_provider: mistral
|
||||
api_key_env: MISTRAL_API_KEY
|
||||
base_url: https://api.mistral.ai/v1
|
||||
defaults:
|
||||
supports_tools: true
|
||||
context_window: 128000
|
||||
models:
|
||||
- id: mistral-large-latest
|
||||
display_name: Mistral Large
|
||||
- id: mistral-small-latest
|
||||
display_name: Mistral Small
|
||||
```
|
||||
|
||||
After restart, those models appear in `/api/models` and are selectable
|
||||
in the UI. A working template lives at
|
||||
`application/core/models/examples/mistral.yaml.example`.
|
||||
|
||||
**What you can do:**
|
||||
|
||||
- Add new `openai_compatible` providers (Mistral, Together, Fireworks,
|
||||
Ollama, vLLM, ...) — one YAML per provider, each with its own
|
||||
`api_key_env` and `base_url`.
|
||||
- Extend an existing provider's catalog by dropping a YAML with the
|
||||
same `provider:` value as the built-in (e.g. `provider: anthropic`
|
||||
with extra models).
|
||||
- Override a built-in model's capabilities by re-declaring the same
|
||||
`id` — later wins, override is logged at `WARNING`.
|
||||
|
||||
**What you cannot do via `MODELS_CONFIG_DIR`:** add a brand-new
|
||||
non-OpenAI provider. That requires a Python plugin under
|
||||
`application/llm/providers/`. See
|
||||
`application/core/models/README.md` for the full schema reference.
|
||||
|
||||
### Docker
|
||||
|
||||
Mount the directory and set the env var:
|
||||
|
||||
```yaml
|
||||
# docker-compose.yml
|
||||
services:
|
||||
app:
|
||||
image: arc53/docsgpt
|
||||
environment:
|
||||
MODELS_CONFIG_DIR: /etc/docsgpt/models
|
||||
MISTRAL_API_KEY: ${MISTRAL_API_KEY}
|
||||
volumes:
|
||||
- ./my-models:/etc/docsgpt/models:ro
|
||||
```
|
||||
|
||||
### Misconfiguration
|
||||
|
||||
If `MODELS_CONFIG_DIR` is set but the path doesn't exist (or isn't a
|
||||
directory), the app logs a `WARNING` at boot and continues with just
|
||||
the built-in catalog — it does **not** fail to start. If a YAML
|
||||
declares an unknown provider name or has a schema error, the app
|
||||
**does** fail to start, with the offending file path in the message.
|
||||
|
||||
## Speech-to-Text Settings
|
||||
|
||||
DocsGPT can transcribe audio in two places:
|
||||
|
||||
- Voice input in the chat.
|
||||
- Audio file ingestion. Uploaded `.wav`, `.mp3`, `.m4a`, `.ogg`, and `.webm` files are transcribed first and then passed through the normal parser, chunking, embedding, and indexing pipeline.
|
||||
|
||||
The settings below control speech-to-text behaviour for both voice input and audio file ingestion.
|
||||
|
||||
| Setting | Purpose | Typical values |
|
||||
| --- | --- | --- |
|
||||
| `STT_PROVIDER` | Speech-to-text backend provider. | `openai`, `faster_whisper` |
|
||||
| `OPENAI_STT_MODEL` | OpenAI transcription model used when `STT_PROVIDER=openai`. | `gpt-4o-mini-transcribe` |
|
||||
| `STT_LANGUAGE` | Optional language hint passed to the provider. Leave unset for auto-detection when supported. | `en`, `es`, unset |
|
||||
| `STT_MAX_FILE_SIZE_MB` | Maximum file size accepted by the synchronous `/api/stt` endpoint. | `50` |
|
||||
| `STT_ENABLE_TIMESTAMPS` | Include timestamp segments in the normalized transcript response and stored parser metadata. | `true`, `false` |
|
||||
| `STT_ENABLE_DIARIZATION` | Reserved provider option for speaker diarization. Some providers may ignore it. | `true`, `false` |
|
||||
|
||||
### Example: OpenAI Speech-to-Text
|
||||
|
||||
```env
|
||||
STT_PROVIDER=openai
|
||||
OPENAI_API_KEY=YOUR_OPENAI_API_KEY
|
||||
OPENAI_STT_MODEL=gpt-4o-mini-transcribe
|
||||
STT_LANGUAGE=
|
||||
STT_MAX_FILE_SIZE_MB=50
|
||||
STT_ENABLE_TIMESTAMPS=false
|
||||
STT_ENABLE_DIARIZATION=false
|
||||
```
|
||||
|
||||
If you already use `API_KEY` for OpenAI, DocsGPT can reuse that key for transcription. Set `OPENAI_API_KEY` only when you want a dedicated key.
|
||||
|
||||
### Example: Local `faster_whisper`
|
||||
|
||||
```env
|
||||
STT_PROVIDER=faster_whisper
|
||||
STT_LANGUAGE=en
|
||||
STT_ENABLE_TIMESTAMPS=true
|
||||
STT_ENABLE_DIARIZATION=false
|
||||
```
|
||||
|
||||
`faster_whisper` is an optional backend dependency. Install it in the Python environment used by the DocsGPT API and worker before selecting this provider.
|
||||
|
||||
## Authentication Settings
|
||||
|
||||
DocsGPT includes a JWT (JSON Web Token) based authentication feature for managing sessions or securing local deployments while allowing access.
|
||||
|
||||
### `AUTH_TYPE` Overview
|
||||
|
||||
The `AUTH_TYPE` setting in your `.env` file or `settings.py` determines the authentication method used by DocsGPT. This allows you to control how users authenticate with your DocsGPT instance.
|
||||
|
||||
| Value | Description |
|
||||
| ------------- | ------------------------------------------------------------------------------------------- |
|
||||
| `None` | No authentication is used. Anyone can access the app. |
|
||||
| `simple_jwt` | A single, long-lived JWT token is generated at startup. All requests use this shared token. |
|
||||
| `session_jwt` | Unique JWT tokens are generated for each session/user. |
|
||||
| `oidc` | Users sign in through an external OpenID Connect provider (Authentik, Keycloak, Okta, ...). See [SSO with OIDC](/Deploying/OIDC-SSO). |
|
||||
|
||||
#### How to Configure
|
||||
|
||||
Add the following to your `.env` file (or set in `settings.py`):
|
||||
|
||||
```env
|
||||
# No authentication (default)
|
||||
AUTH_TYPE=None
|
||||
|
||||
# OR: Simple JWT (shared token)
|
||||
AUTH_TYPE=simple_jwt
|
||||
JWT_SECRET_KEY=your_secret_key_here
|
||||
|
||||
# OR: Session JWT (per-user/session tokens)
|
||||
AUTH_TYPE=session_jwt
|
||||
JWT_SECRET_KEY=your_secret_key_here
|
||||
|
||||
# OR: SSO via an OpenID Connect provider (Authentik, Keycloak, Okta, ...)
|
||||
AUTH_TYPE=oidc
|
||||
OIDC_ISSUER=https://auth.example.com/application/o/docsgpt/
|
||||
OIDC_CLIENT_ID=your_client_id
|
||||
OIDC_FRONTEND_URL=https://docsgpt.example.com
|
||||
JWT_SECRET_KEY=your_secret_key_here
|
||||
```
|
||||
|
||||
- If `AUTH_TYPE` is set to `simple_jwt` or `session_jwt`, a `JWT_SECRET_KEY` is required.
|
||||
- If `JWT_SECRET_KEY` is not set, DocsGPT will generate one and store it in `.jwt_secret_key` in the project root.
|
||||
|
||||
#### How Each Method Works
|
||||
|
||||
- **None**: No authentication. All API and UI access is open.
|
||||
- **simple_jwt**:
|
||||
- A single JWT token is generated at startup and printed to the console.
|
||||
- Use this token in the `Authorization` header for all API requests:
|
||||
```http
|
||||
Authorization: Bearer <SIMPLE_JWT_TOKEN>
|
||||
```
|
||||
- The frontend will prompt for this token if not already set.
|
||||
- **session_jwt**:
|
||||
- Clients can request a new token from `/api/generate_token`.
|
||||
- Use the received token in the `Authorization` header for subsequent requests.
|
||||
- Each user/session gets a unique token.
|
||||
- **oidc**:
|
||||
- The frontend redirects users to your identity provider to sign in (OAuth2 Authorization Code + PKCE).
|
||||
- After a successful sign-in, DocsGPT issues its own session JWT; API requests carry it in the `Authorization` header like the other modes.
|
||||
- Stable per-user identities come from the provider — see the full setup guide: [SSO with OIDC](/Deploying/OIDC-SSO).
|
||||
- The same guide covers the optional access controls: group allowlists, silent session renewal, back-channel logout, SCIM provisioning, and login auditing.
|
||||
|
||||
#### Security Notes
|
||||
|
||||
- Always keep your `JWT_SECRET_KEY` secure and private.
|
||||
- If you set it manually, use a strong, random string.
|
||||
- If not set, DocsGPT will generate a secure key and persist it in `.jwt_secret_key`.
|
||||
|
||||
#### Checking Current Auth Type
|
||||
|
||||
- Use the `/api/config` endpoint to check the current `auth_type` and whether authentication is required.
|
||||
|
||||
#### Frontend Token Input for `simple_jwt`
|
||||
|
||||
If you have configured `AUTH_TYPE=simple_jwt`, the DocsGPT frontend will prompt you to enter the JWT token if it's not already set or is invalid. Paste the `SIMPLE_JWT_TOKEN` (printed to your console when the backend starts) into this field to access the application.
|
||||
|
||||
<img
|
||||
src="/jwt-input.png"
|
||||
alt="Frontend prompt for JWT Token"
|
||||
style={{
|
||||
width: "500px",
|
||||
maxWidth: "100%",
|
||||
display: "block",
|
||||
margin: "1em auto",
|
||||
}}
|
||||
/>
|
||||
|
||||
## S3 Storage Backend
|
||||
|
||||
By default DocsGPT stores files locally. Set `STORAGE_TYPE=s3` to use Amazon S3 — or any S3-compatible service (MinIO, Cloudflare R2, Backblaze B2, DigitalOcean Spaces, …) — instead.
|
||||
|
||||
| Setting | Description | Default |
|
||||
| --- | --- | --- |
|
||||
| `STORAGE_TYPE` | `local` or `s3` | `local` |
|
||||
| `S3_BUCKET_NAME` | Bucket name | `docsgpt-test-bucket` |
|
||||
| `S3_ACCESS_KEY_ID` | Access key ID | — |
|
||||
| `S3_SECRET_ACCESS_KEY` | Secret access key | — |
|
||||
| `S3_REGION` | Region (use `auto` for Cloudflare R2) | — |
|
||||
| `S3_ENDPOINT_URL` | Custom endpoint for S3-compatible services; leave unset for AWS S3 | — |
|
||||
| `S3_PATH_STYLE` | Use path-style addressing (required by most non-AWS services) | `false` |
|
||||
| `URL_STRATEGY` | `backend` (proxy through API) or `s3` (direct object URLs) | `backend` |
|
||||
|
||||
### AWS S3
|
||||
|
||||
```env
|
||||
STORAGE_TYPE=s3
|
||||
S3_BUCKET_NAME=your-bucket-name
|
||||
S3_ACCESS_KEY_ID=your-access-key-id
|
||||
S3_SECRET_ACCESS_KEY=your-secret-access-key
|
||||
S3_REGION=us-east-1
|
||||
```
|
||||
|
||||
### S3-compatible services (MinIO, Cloudflare R2, …)
|
||||
|
||||
Set `S3_ENDPOINT_URL` and usually `S3_PATH_STYLE=true`:
|
||||
|
||||
```env
|
||||
STORAGE_TYPE=s3
|
||||
S3_BUCKET_NAME=your-bucket-name
|
||||
S3_ACCESS_KEY_ID=your-access-key-id
|
||||
S3_SECRET_ACCESS_KEY=your-secret-access-key
|
||||
S3_REGION=auto
|
||||
S3_ENDPOINT_URL=https://<account>.r2.cloudflarestorage.com
|
||||
S3_PATH_STYLE=true
|
||||
```
|
||||
|
||||
Your credentials need these permissions on the bucket: `s3:PutObject`, `s3:GetObject`, `s3:DeleteObject`, `s3:ListBucket`, `s3:HeadObject`.
|
||||
|
||||
> **Deprecated:** earlier versions reused the `SAGEMAKER_ACCESS_KEY`, `SAGEMAKER_SECRET_KEY`, and `SAGEMAKER_REGION` variables for S3 credentials. These are still honored as a fallback (with a deprecation warning) but you should migrate to the `S3_*` variables above.
|
||||
|
||||
## User-Data Storage (Postgres)
|
||||
|
||||
DocsGPT stores user data — conversations, agents, prompts, sources, attachments, workflows, logs, and token usage — in **PostgreSQL**. The backend connects via a single setting:
|
||||
|
||||
| Setting | Description | Default |
|
||||
| --- | --- | --- |
|
||||
| `POSTGRES_URI` | SQLAlchemy-compatible Postgres URI. Any standard `postgresql://` form works — DocsGPT normalizes it internally to the `psycopg` v3 dialect. | — |
|
||||
| `AUTO_CREATE_DB` | On startup, connect to the server's `postgres` maintenance DB and issue `CREATE DATABASE` if the target is missing. Requires `CREATEDB` or superuser. No-op when the database already exists. Disable in production. | `true` |
|
||||
| `AUTO_MIGRATE` | On startup, run `alembic upgrade head` against the target database. Idempotent and serialized across workers via `alembic_version`. Disable in production in favor of an explicit migration step. | `true` |
|
||||
|
||||
Example:
|
||||
|
||||
```env
|
||||
POSTGRES_URI=postgresql://docsgpt:docsgpt@localhost:5432/docsgpt
|
||||
# Append ?sslmode=require for managed providers that enforce SSL.
|
||||
```
|
||||
|
||||
With the defaults, the app applies the schema automatically on first
|
||||
boot. To run it explicitly instead (e.g., in CI/CD or a k8s `Job`):
|
||||
|
||||
```bash
|
||||
python scripts/db/init_postgres.py
|
||||
```
|
||||
|
||||
The default Docker Compose file bundles a `postgres` service, and the
|
||||
app auto-bootstraps the database on boot, so containerized deployments
|
||||
need no manual migration step. See
|
||||
[PostgreSQL for User Data](/Deploying/Postgres-Migration#production-hardening)
|
||||
for the recommended production flow (both flags `false`, migrations
|
||||
gated by CI/CD).
|
||||
|
||||
<Callout type="info" emoji="ℹ️">
|
||||
`MONGO_URI` is **opt-in**. It is only consulted when you select the
|
||||
MongoDB Atlas vector-store backend (`VECTOR_STORE=mongodb`) or when
|
||||
running the one-shot `scripts/db/backfill.py` migration from a legacy
|
||||
Mongo-based install. Installing the optional Mongo client libraries
|
||||
requires `pip install 'pymongo>=4.6'`. See
|
||||
[PostgreSQL for User Data](/Deploying/Postgres-Migration) for the
|
||||
migration path.
|
||||
</Callout>
|
||||
|
||||
## Retrieval & RAG Settings
|
||||
|
||||
These control how sources are retrieved and whether the advanced RAG features are available. See [Per-Source Configuration](/Sources/Per-source-configuration) and [GraphRAG](/Sources/GraphRAG) for details.
|
||||
|
||||
| Setting | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `RETRIEVERS_ENABLED` | `["classic", "default"]` | Allow-list of retrievers usable instance-wide. Valid keys: `classic`, `default`, `hybrid`, `graphrag`. A per-source `retriever` must be within this list. |
|
||||
| `PER_SOURCE_RETRIEVAL_ENABLED` | `true` | Master switch for per-source retrieval config. When `false`, all sources fall back to the classic retriever regardless of their stored config. |
|
||||
| `GRAPHRAG_ENABLED` | `false` | Enable [GraphRAG](/Sources/GraphRAG). Requires `VECTOR_STORE=pgvector`. |
|
||||
| `GRAPHRAG_EXTRACTION_MODEL` | unset | Model used for ingest-time graph extraction. Unset reuses the instance default model. |
|
||||
| `GRAPHRAG_MAX_CHUNKS_FOR_EXTRACTION` | `2000` | Hard cap on chunks extracted per source (cost control). |
|
||||
|
||||
## Embeddings Settings
|
||||
|
||||
See [Embeddings](/Models/embeddings) for full guidance.
|
||||
|
||||
| Setting | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `EMBEDDINGS_NAME` | `huggingface_sentence-transformers/all-mpnet-base-v2` | The embedding model. |
|
||||
| `EMBEDDINGS_BASE_URL` | unset | Base URL of a remote OpenAI-compatible embeddings server. Setting it routes all embedding calls there. |
|
||||
| `EMBEDDINGS_KEY` | unset | Optional bearer token for the remote embeddings server. |
|
||||
| `EMBEDDINGS_MAX_INPUT_TOKENS` | unset | Truncate each remote embedding input to N tokens (guards servers that reject oversized inputs). |
|
||||
|
||||
## Tools Settings
|
||||
|
||||
| Setting | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `DEFAULT_CHAT_TOOLS` | `["memory", "read_webpage", "scheduler"]` | Tools enabled automatically in regular (agentless) chats. See [Tools Basics](/Tools/basics#default-chat-tools). |
|
||||
|
||||
## Admin & Access Settings
|
||||
|
||||
See [Access Control, Roles & Teams](/Deploying/Access-Control) for the full model.
|
||||
|
||||
| Setting | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `OIDC_ADMIN_GROUPS` | unset | Comma-separated IdP groups granted the global `admin` role (OIDC only). |
|
||||
| `LOCAL_MODE_ADMIN` | `false` | Grants admin in no-auth mode (`AUTH_TYPE=None`) only. **Never enable on a networked deployment.** |
|
||||
|
||||
## LLM Provider Settings
|
||||
|
||||
| Setting | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `OPENAI_RESPONSES_STORE` | `false` | When `true`, allows OpenAI to persist [Responses API](/Models/cloud-providers#openai-responses-api-and-reasoning) state server-side. |
|
||||
|
||||
## Realtime Events Settings
|
||||
|
||||
The realtime notifications channel has its own settings — see [Realtime Events & Notifications](/Agents/notifications) (`ENABLE_SSE_PUSH`, `EVENTS_STREAM_MAXLEN`, `SSE_MAX_CONCURRENT_PER_USER`, and related).
|
||||
|
||||
## Exploring More Settings
|
||||
|
||||
These are just the basic settings to get you started. The `settings.py` file contains many more advanced options that you can explore to further customize DocsGPT, such as:
|
||||
|
||||
- Vector store configuration (`VECTOR_STORE`, Qdrant, Milvus, LanceDB settings) If you're looking for an easy way to set up a vector store with pgvector, try [Neon](https://get.neon.com/docsgpt).
|
||||
- Retriever settings (`RETRIEVERS_ENABLED`)
|
||||
- Cache settings (`CACHE_REDIS_URL`)
|
||||
- And many more!
|
||||
|
||||
For a complete list of available settings and their descriptions, refer to the `settings.py` file in `application/core`. Remember to restart your Docker containers after making changes to your `.env` file or `settings.py` for the changes to take effect.
|
||||
@@ -0,0 +1,33 @@
|
||||
import { DeploymentCards } from '../../components/DeploymentCards';
|
||||
|
||||
# Deployment Guides
|
||||
|
||||
<DeploymentCards
|
||||
items={[
|
||||
{
|
||||
title: 'Amazon Lightsail',
|
||||
link: 'https://docs.docsgpt.cloud/Deploying/Amazon-Lightsail',
|
||||
description: 'Self-hosting DocsGPT on Amazon Lightsail'
|
||||
},
|
||||
{
|
||||
title: 'Railway',
|
||||
link: 'https://docs.docsgpt.cloud/Deploying/Railway',
|
||||
description: 'Hosting DocsGPT on Railway'
|
||||
},
|
||||
{
|
||||
title: 'Civo Compute Cloud',
|
||||
link: 'https://dev.to/rutamhere/deploying-docsgpt-on-civo-compute-c',
|
||||
description: 'Step-by-step guide for Civo deployment'
|
||||
},
|
||||
{
|
||||
title: 'DigitalOcean Droplet',
|
||||
link: 'https://dev.to/rutamhere/deploying-docsgpt-on-digitalocean-droplet-50ea',
|
||||
description: 'Guide for DigitalOcean deployment'
|
||||
},
|
||||
{
|
||||
title: 'Kamatera Cloud',
|
||||
link: 'https://dev.to/rutamhere/deploying-docsgpt-on-kamatera-performance-cloud-1bj',
|
||||
description: 'Kamatera deployment tutorial'
|
||||
}
|
||||
]}
|
||||
/>
|
||||
@@ -0,0 +1,109 @@
|
||||
---
|
||||
title: Deploying DocsGPT on Kubernetes
|
||||
description: Learn how to self-host DocsGPT on a Kubernetes cluster for scalable and robust deployments.
|
||||
---
|
||||
|
||||
# Self-hosting DocsGPT
|
||||
on Kubernetes
|
||||
|
||||
This guide will walk you through deploying DocsGPT on Kubernetes.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Ensure you have the following installed before proceeding:
|
||||
|
||||
- [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
|
||||
- Access to a Kubernetes cluster.
|
||||
- [Neon](https://get.neon.com/docsgpt) (optional) for a quick and easy vector store setup with pgvector.
|
||||
|
||||
## Folder Structure
|
||||
|
||||
The `deployment/k8s` folder contains the necessary deployment and service configuration files:
|
||||
|
||||
- `deployments/`
|
||||
- `services/`
|
||||
- `docsgpt-secrets.yaml`
|
||||
|
||||
## Deployment Instructions
|
||||
|
||||
1. **Clone the Repository**
|
||||
|
||||
```sh
|
||||
git clone https://github.com/arc53/DocsGPT.git
|
||||
cd docsgpt/deployment/k8s
|
||||
```
|
||||
|
||||
2. **Configure Secrets (optional)**
|
||||
|
||||
Ensure that you have all the necessary secrets in `docsgpt-secrets.yaml`. Update it with your secrets before applying if you want. By default we will use qdrant as a vectorstore and public docsgpt llm as llm for inference.
|
||||
|
||||
Alternatively, you can use [Neon](https://get.neon.com/docsgpt) as an easy way to set up your vector store with pgvector, which is highly recommended for quick deployments.
|
||||
|
||||
3. **Apply Kubernetes Deployments**
|
||||
|
||||
Deploy your DocsGPT resources using the following commands:
|
||||
|
||||
```sh
|
||||
kubectl apply -f deployments/
|
||||
```
|
||||
|
||||
4. **Apply Kubernetes Services**
|
||||
|
||||
Set up your services using the following commands:
|
||||
|
||||
```sh
|
||||
kubectl apply -f services/
|
||||
```
|
||||
|
||||
5. **Apply Secrets**
|
||||
|
||||
Apply the secret configurations:
|
||||
|
||||
```sh
|
||||
kubectl apply -f docsgpt-secrets.yaml
|
||||
```
|
||||
|
||||
6. **Substitute API URL**
|
||||
|
||||
After deploying the services, you need to update the environment variable `VITE_API_HOST` in your deployment file `deployments/docsgpt-deploy.yaml` with the actual endpoint URL created by your `docsgpt-api-service`.
|
||||
|
||||
```sh
|
||||
kubectl get services/docsgpt-api-service -o jsonpath='{.status.loadBalancer.ingress[0].ip}' | xargs -I {} sed -i "s|<your-api-endpoint>|{}|g" deployments/docsgpt-deploy.yaml
|
||||
```
|
||||
|
||||
7. **Rerun Deployment**
|
||||
|
||||
After making the changes, reapply the deployment configuration to update the environment variables:
|
||||
|
||||
```sh
|
||||
kubectl apply -f deployments/
|
||||
```
|
||||
|
||||
## Verifying the Deployment
|
||||
|
||||
To verify if everything is set up correctly, you can run the following:
|
||||
|
||||
```sh
|
||||
kubectl get pods
|
||||
kubectl get services
|
||||
```
|
||||
|
||||
Ensure that the pods are running and the services are available.
|
||||
|
||||
## Accessing DocsGPT
|
||||
|
||||
To access DocsGPT, you need to find the external IP address of the frontend service. You can do this by running:
|
||||
|
||||
```sh
|
||||
kubectl get services/docsgpt-frontend-service | awk 'NR>1 {print "http://" $4}'
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
If you encounter any issues, you can check the logs of the pods for more details:
|
||||
|
||||
```sh
|
||||
kubectl logs <pod-name>
|
||||
```
|
||||
|
||||
Replace `<pod-name>` with the actual name of your DocsGPT pod.
|
||||
@@ -0,0 +1,230 @@
|
||||
---
|
||||
title: SSO with OIDC
|
||||
description: Sign users into DocsGPT through any OpenID Connect identity provider (Authentik, Keycloak, Okta, ...) — with group allowlists, silent session renewal, back-channel logout, and SCIM provisioning.
|
||||
---
|
||||
|
||||
# SSO with OIDC
|
||||
|
||||
Setting `AUTH_TYPE=oidc` makes DocsGPT delegate sign-in to an external OpenID Connect identity provider (IdP). Any spec-compliant IdP with a discovery document works; this guide uses [Authentik](https://goauthentik.io/) as the reference provider and includes a short note for Keycloak.
|
||||
|
||||
Beyond basic sign-in, this page covers the optional access controls: [group allowlists](#restricting-sign-in-by-group), [silent session renewal](#silent-session-renewal), [back-channel logout](#back-channel-logout), [SCIM user provisioning](#scim-user-provisioning), and [login auditing](#login-auditing).
|
||||
|
||||
## How the flow works
|
||||
|
||||
1. A user opens DocsGPT without a session. The frontend redirects the browser to `GET /api/auth/oidc/login` on the DocsGPT API.
|
||||
2. The backend starts an **OAuth2 Authorization Code + PKCE** flow and redirects to your IdP's sign-in page.
|
||||
3. After sign-in, the IdP redirects back to `GET /api/auth/oidc/callback`. The backend exchanges the code server-side, validates the ID token (signature via JWKS, issuer, audience, expiry, nonce), and mints a **DocsGPT session JWT** signed with `JWT_SECRET_KEY`.
|
||||
4. The browser returns to your frontend with a short-lived single-use code in the URL fragment; the frontend exchanges it for the session JWT and stores it. From here on, requests are authenticated exactly like the other `AUTH_TYPE` modes (`Authorization: Bearer <token>`).
|
||||
5. Signing out clears the local session and redirects through the IdP's end-session endpoint.
|
||||
|
||||
The user's identity (`sub` claim by default) becomes the DocsGPT `user_id`, so every user gets their own conversations, sources, agents, and settings.
|
||||
|
||||
Sessions last `OIDC_SESSION_LIFETIME_SECONDS` (8 hours by default) and renew without interrupting the user — see [Silent session renewal](#silent-session-renewal).
|
||||
|
||||
> Redis must be reachable by the API — it stores the short-lived login state, handoff codes, server-side refresh tokens, and the session revocation denylist. Redis is already a required DocsGPT dependency, so no extra infrastructure is needed.
|
||||
|
||||
### IdP compatibility notes
|
||||
|
||||
- **Token-endpoint authentication** follows the IdP's discovery document (`token_endpoint_auth_methods_supported`): `client_secret_post` when the IdP advertises it, otherwise HTTP Basic (the RFC default). Okta's default web-app configuration works without extra toggles.
|
||||
- **Userinfo fallback**: when the ID token lacks the user-id claim (`OIDC_USER_ID_CLAIM`) — or the groups claim while a group allowlist is configured — the backend fetches the IdP's userinfo endpoint and merges the missing claims. ID-token values win on conflict, and the userinfo `sub` must match the ID token's.
|
||||
|
||||
## Settings reference
|
||||
|
||||
| Setting | Required | Default | Description |
|
||||
| --- | --- | --- | --- |
|
||||
| `AUTH_TYPE` | yes | — | Set to `oidc`. |
|
||||
| `OIDC_ISSUER` | yes | — | Issuer URL of your IdP. Discovery is read from `<issuer>/.well-known/openid-configuration`. |
|
||||
| `OIDC_CLIENT_ID` | yes | — | Client ID registered at the IdP. |
|
||||
| `OIDC_FRONTEND_URL` | yes | — | Browser-facing URL of the DocsGPT frontend (where users land after login/logout), e.g. `https://docsgpt.example.com`. |
|
||||
| `OIDC_CLIENT_SECRET` | no | — | Set when the IdP client is *confidential*. PKCE is always used, so *public* clients work without a secret. |
|
||||
| `OIDC_SCOPES` | no | `openid profile email` | Scopes requested at the IdP. Add `offline_access` when your IdP requires it for refresh tokens (Authentik does). |
|
||||
| `OIDC_USER_ID_CLAIM` | no | `sub` | ID-token claim used as the DocsGPT user id. Set to `email` or `preferred_username` for human-readable ids; use `email` when provisioning over [SCIM](#scim-user-provisioning). |
|
||||
| `OIDC_REDIRECT_URI` | no | derived | Full callback URL registered at the IdP. Defaults to `<request host>/api/auth/oidc/callback`; set it explicitly when the API runs behind a reverse proxy. |
|
||||
| `OIDC_SESSION_LIFETIME_SECONDS` | no | `28800` (8h) | Lifetime of the DocsGPT session JWT. Sessions renew before expiry — see [Silent session renewal](#silent-session-renewal). |
|
||||
| `OIDC_PROVIDER_NAME` | no | — | Display name on the sign-in button: `Acme SSO` renders "Sign in with Acme SSO". Unset, the button shows a generic "SSO". |
|
||||
| `OIDC_ALLOWED_GROUPS` | no | — | Comma-separated group allowlist. Unset, any authenticated IdP user may sign in — see [Restricting sign-in by group](#restricting-sign-in-by-group). |
|
||||
| `OIDC_ADMIN_GROUPS` | no | — | Comma-separated groups whose members are granted the global `admin` role. Re-checked at every login and renewal — see [Granting admin via groups](#granting-admin-via-groups). |
|
||||
| `OIDC_GROUPS_CLAIM` | no | `groups` | ID-token/userinfo claim carrying the user's group membership. |
|
||||
| `JWT_SECRET_KEY` | recommended | auto-generated | Signs DocsGPT session tokens. Set it explicitly in production — required when running multiple API replicas. |
|
||||
|
||||
`SCIM_ENABLED` and `SCIM_TOKEN` are listed in the [SCIM section](#scim-user-provisioning).
|
||||
|
||||
## Setting up with Authentik
|
||||
|
||||
1. **Create a provider.** In the Authentik admin UI go to **Applications → Providers → Create** and pick **OAuth2/OpenID Provider**:
|
||||
- **Authorization flow**: your preferred flow (e.g. *explicit consent*).
|
||||
- **Client type**: `Public` (no secret, PKCE only) or `Confidential` (also set `OIDC_CLIENT_SECRET` in DocsGPT).
|
||||
- **Redirect URIs**: `https://<your-docsgpt-api>/api/auth/oidc/callback`
|
||||
- **Signing key**: select a certificate so ID tokens are RS256-signed.
|
||||
2. **Create an application** (Applications → Applications → Create), link it to the provider, and note its **slug**.
|
||||
3. **Find the issuer.** With Authentik's default per-provider issuer mode it is:
|
||||
```
|
||||
https://<your-authentik-host>/application/o/<application-slug>/
|
||||
```
|
||||
(the trailing slash is part of the issuer — copy it exactly; the discovery document lives at `.../<application-slug>/.well-known/openid-configuration`).
|
||||
4. **Configure DocsGPT** in `.env` and restart:
|
||||
```env
|
||||
AUTH_TYPE=oidc
|
||||
OIDC_ISSUER=https://auth.example.com/application/o/docsgpt/
|
||||
OIDC_CLIENT_ID=<client id from step 1>
|
||||
# OIDC_CLIENT_SECRET=<only for Confidential client type>
|
||||
OIDC_FRONTEND_URL=https://docsgpt.example.com
|
||||
JWT_SECRET_KEY=<long random string>
|
||||
```
|
||||
|
||||
> Planning to use [silent session renewal](#silent-session-renewal)? Authentik only issues refresh tokens when the `offline_access` scope is requested — set `OIDC_SCOPES=openid profile email offline_access`.
|
||||
|
||||
### Which claim becomes the user id?
|
||||
|
||||
Authentik's provider setting **Subject mode** controls what lands in the `sub` claim (the default is a hashed user ID — stable but opaque). If you'd rather key DocsGPT users on something readable, either change Subject mode (e.g. *based on username*) or leave Authentik alone and set `OIDC_USER_ID_CLAIM=email` in DocsGPT. Pick one strategy before going live: changing it later gives existing users fresh, empty accounts. If you plan to provision users over [SCIM](#scim-user-provisioning), use `OIDC_USER_ID_CLAIM=email` — SCIM matches users by `userName`, which IdPs typically send as the email.
|
||||
|
||||
## Keycloak (and other IdPs)
|
||||
|
||||
Any OIDC provider with discovery works the same way. For Keycloak:
|
||||
|
||||
```env
|
||||
OIDC_ISSUER=https://keycloak.example.com/realms/<realm>
|
||||
OIDC_CLIENT_ID=<client id>
|
||||
```
|
||||
|
||||
Create the client with *Standard flow* enabled and PKCE method `S256`; register the same `/api/auth/oidc/callback` redirect URI.
|
||||
|
||||
The feature sections below carry their own per-IdP notes — group claims, refresh tokens, back-channel logout, and SCIM each need one IdP-side setting.
|
||||
|
||||
## Restricting sign-in by group
|
||||
|
||||
By default any user who can authenticate at the IdP may use DocsGPT. To restrict access to specific IdP groups:
|
||||
|
||||
```env
|
||||
OIDC_ALLOWED_GROUPS=docsgpt-users,platform-admins
|
||||
# OIDC_GROUPS_CLAIM=groups # only if your IdP uses a different claim name
|
||||
```
|
||||
|
||||
At login the backend reads the `OIDC_GROUPS_CLAIM` claim (default `groups`) from the ID token, falling back to the userinfo endpoint when the claim is absent. A user whose groups share no entry with the allowlist is rejected with a clean "not authorized" screen (`oidc_error=not_authorized`), and the denial lands in the [audit log](#login-auditing).
|
||||
|
||||
Group changes take effect at the next sign-in **or** the next [silent renewal](#silent-session-renewal): whenever the IdP returns a fresh ID token during renewal, the allowlist is re-checked — so removing a user from the allowed group cuts off their session at the next renewal instead of whenever they happen to sign in again.
|
||||
|
||||
Getting groups into the token:
|
||||
|
||||
- **Authentik** includes group names in the `groups` claim through its default `profile` scope — no extra configuration needed.
|
||||
- **Keycloak** does not emit groups by default. On the client, open **Client scopes → the client's dedicated scope → Add mapper → By configuration → Group Membership**, set the claim name to `groups`, and turn **Full group path** off so the claim carries plain names (`devs`) rather than paths (`/devs`).
|
||||
|
||||
## Granting admin via groups
|
||||
|
||||
Separately from *who may sign in*, you can map an IdP group to the global **admin** role with `OIDC_ADMIN_GROUPS`:
|
||||
|
||||
```env
|
||||
OIDC_ADMIN_GROUPS=platform-admins
|
||||
```
|
||||
|
||||
Members of the listed groups are granted admin; the mapping is re-evaluated at every login **and** every [silent renewal](#silent-session-renewal), so removing a user from the admin group revokes their admin at the next renewal (exactly like the sign-in allowlist). It is independent of `OIDC_ALLOWED_GROUPS`, and leaving it unset never mass-revokes admin. On a fresh deployment this is the only way to create the first admin. For the full roles, teams, and admin-dashboard model see [Access Control, Roles & Teams](/Deploying/Access-Control).
|
||||
|
||||
## Silent session renewal
|
||||
|
||||
The DocsGPT session JWT lives for `OIDC_SESSION_LIFETIME_SECONDS` (default 8 hours). Sessions renew without user-visible interruptions, in one of two ways:
|
||||
|
||||
- **With a refresh token.** When the IdP issues one, the backend stores it server-side (in Redis — never in the browser) and the frontend calls `POST /api/auth/oidc/refresh` about 15 minutes before the session expires. The backend redeems the refresh token at the IdP, re-validates the fresh ID token (including the [group allowlist](#restricting-sign-in-by-group)), mints a new session JWT, and rotates the stored refresh token. The user notices nothing.
|
||||
- **Without a refresh token.** The frontend lets the session run to expiry and then redirects through the IdP again. While the IdP session is still alive, this round-trip is also silent; the user only sees a sign-in page once the IdP session is gone too.
|
||||
|
||||
Getting a refresh token:
|
||||
|
||||
- **Keycloak** issues refresh tokens for the authorization-code flow by default — nothing to change.
|
||||
- **Authentik** only issues refresh tokens when the `offline_access` scope is requested:
|
||||
```env
|
||||
OIDC_SCOPES=openid profile email offline_access
|
||||
```
|
||||
|
||||
Revoking the user's consent or sessions at the IdP makes the next renewal fail, and the user must sign in again. For revocation that doesn't wait for the next renewal, configure [back-channel logout](#back-channel-logout).
|
||||
|
||||
## Back-channel logout
|
||||
|
||||
DocsGPT implements [OIDC Back-Channel Logout 1.0](https://openid.net/specs/openid-connect-backchannel-1_0.html). The IdP POSTs a signed `logout_token` to:
|
||||
|
||||
```
|
||||
POST https://<your-docsgpt-api>/api/auth/oidc/backchannel-logout
|
||||
```
|
||||
|
||||
DocsGPT validates the token (signature via JWKS, issuer, audience, replay protection) and immediately revokes the user's live sessions through a Redis denylist — revoked requests get `401` with `error: token_revoked`. Signing the user out at the IdP, or an admin revoking their sessions there, takes effect on their next DocsGPT request instead of at session expiry.
|
||||
|
||||
The endpoint is called server-to-server, so it must be reachable from the IdP (it is not a browser redirect).
|
||||
|
||||
- **Keycloak**: open the client → **Settings** and set **Backchannel logout URL** to `https://<your-docsgpt-api>/api/auth/oidc/backchannel-logout`.
|
||||
- **Authentik** (2025.8.0 and later; marked Preview): on the OAuth2/OpenID provider set **Logout Method** to *Back-channel* and **Logout URI** to the same URL — see the [Authentik logout docs](https://docs.goauthentik.io/add-secure-apps/providers/oauth2/frontchannel_and_backchannel_logout/). Authentik sends the logout token when a user logs out, an admin deletes their session, the account is deactivated, or the session is revoked. On older Authentik versions back-channel logout is unavailable — revocation latency then falls back to the session lifetime, or use [SCIM deactivation](#scim-user-provisioning), which also revokes sessions instantly.
|
||||
|
||||
## SCIM user provisioning
|
||||
|
||||
DocsGPT exposes a [SCIM 2.0](https://datatracker.ietf.org/doc/html/rfc7644) endpoint so your IdP can drive the user lifecycle: create accounts ahead of first login and — more importantly — deactivate them on offboarding. Deactivating a user revokes their live sessions immediately and blocks future sign-ins (they see an "account disabled" screen); reactivating restores access.
|
||||
|
||||
| Setting | Required | Default | Description |
|
||||
| --- | --- | --- | --- |
|
||||
| `SCIM_ENABLED` | yes | `false` | Set to `true` to serve the `/scim/v2` endpoints. |
|
||||
| `SCIM_TOKEN` | yes | — | Bearer token the IdP's SCIM client must present. Use a long random string. |
|
||||
|
||||
The base URL is `https://<your-docsgpt-api>/scim/v2`; every request must carry `Authorization: Bearer <SCIM_TOKEN>`.
|
||||
|
||||
### Match the SCIM userName to the OIDC user id
|
||||
|
||||
SCIM identifies users by `userName`, which DocsGPT matches against its user id — the value of `OIDC_USER_ID_CLAIM`. With the default `sub` claim, the `userName` your IdP sends (typically the email) would never line up with the opaque `sub` of the same user signing in, and DocsGPT would treat them as two unrelated accounts. **When using SCIM, set `OIDC_USER_ID_CLAIM=email` and have the IdP send the email as the SCIM `userName`.**
|
||||
|
||||
### What the endpoint supports
|
||||
|
||||
| Operation | Support |
|
||||
| --- | --- |
|
||||
| `GET /scim/v2/ServiceProviderConfig`, `/ResourceTypes`, `/Schemas` | Discovery documents. |
|
||||
| `GET /scim/v2/Users` | List, with the exact filter `userName eq "..."` and `startIndex`/`count` pagination (1-based, max 200 per page). |
|
||||
| `POST /scim/v2/Users` | Create; returns `409` when the `userName` already exists. |
|
||||
| `GET /scim/v2/Users/<id>` | Read. |
|
||||
| `PUT` / `PATCH /scim/v2/Users/<id>` | Activate/deactivate via the `active` attribute (Okta's string `"true"`/`"false"` values are accepted). `userName` is immutable; other attributes are ignored. |
|
||||
| `DELETE /scim/v2/Users/<id>` | Soft delete — deactivates the account instead of removing data. |
|
||||
| `/scim/v2/Groups` | Group provisioning is **not** supported: listing returns an empty result so IdP probes don't fail, and mutations return `501`. Use the [group allowlist](#restricting-sign-in-by-group) for group-based access control instead. |
|
||||
|
||||
### IdP setup pointers
|
||||
|
||||
- **Okta**: add SCIM provisioning to the app integration with **SCIM connector base URL** = `https://<your-docsgpt-api>/scim/v2` and authentication mode **HTTP Header** carrying the bearer token. Enable creating and deactivating users; skip group push.
|
||||
- **Authentik**: create a **SCIM provider** with the same base URL and the token, and attach it to the application as a backchannel provider. Sync users only — leave group mappings out, since DocsGPT answers group provisioning with `501`.
|
||||
|
||||
## Login auditing
|
||||
|
||||
Authentication activity is recorded in `auth_events`, an append-only Postgres table carrying the user id, event name, IP address, user agent, a JSONB `metadata` column, and a timestamp:
|
||||
|
||||
| Event | Recorded when |
|
||||
| --- | --- |
|
||||
| `oidc_login` | A user signs in successfully. |
|
||||
| `oidc_login_denied` | A sign-in is rejected — `metadata.reason` is `not_authorized` (group allowlist) or `account_disabled`. |
|
||||
| `oidc_refresh` | A session is silently renewed. |
|
||||
| `backchannel_logout` | The IdP revokes sessions via back-channel logout. |
|
||||
| `scim_created` / `scim_deactivated` / `scim_reactivated` | SCIM lifecycle changes. |
|
||||
| `role_granted` / `role_revoked` | The admin role is granted/revoked (`metadata.source` is `manual` or `oidc_group`). |
|
||||
| `admin_user_activated` / `admin_user_deactivated` | An admin activates/deactivates a user. |
|
||||
| `admin_sessions_revoked` | An admin force-logs-out a user. |
|
||||
| `team.*` | Team management — `team.create`, `team.member_add`, `team.member_role`, `team.member_remove`, `team.share`, `team.unshare`, `team.transfer_owner`, `team.delete`. |
|
||||
|
||||
The acting admin is recorded in the event metadata. See [Access Control, Roles & Teams](/Deploying/Access-Control) for the admin and team features that emit these events.
|
||||
|
||||
There is no UI for these events yet — query the table directly:
|
||||
|
||||
```sql
|
||||
SELECT created_at, event, user_id, ip, metadata
|
||||
FROM auth_events
|
||||
ORDER BY created_at DESC
|
||||
LIMIT 50;
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
When sign-in fails, the browser lands back on the frontend with an `#oidc_error=<code>` fragment and the sign-in screen shows a matching message:
|
||||
|
||||
| Code | Cause |
|
||||
| --- | --- |
|
||||
| `invalid_state` | The login attempt expired (the state is held for 10 minutes) or was replayed. Retrying the sign-in usually fixes it. |
|
||||
| `auth_failed` | Token exchange or ID-token validation failed — check the API logs. Most common: `OIDC_ISSUER` doesn't match the issuer the discovery document reports (for Authentik this includes the application slug and trailing slash), or clock skew beyond the allowed 60 seconds. |
|
||||
| `missing_claim` | Neither the ID token nor userinfo contains `OIDC_USER_ID_CLAIM`. Make sure the matching scope is requested (`OIDC_SCOPES`) and the IdP actually emits the claim, or switch the setting back to `sub`. |
|
||||
| `not_authorized` | The user's groups don't intersect `OIDC_ALLOWED_GROUPS` — see [Restricting sign-in by group](#restricting-sign-in-by-group). |
|
||||
| `account_disabled` | The account was deactivated via [SCIM](#scim-user-provisioning) or by an operator. Reactivate it over SCIM to restore access. |
|
||||
|
||||
Other issues:
|
||||
|
||||
- **IdP shows a redirect URI error** — the callback URL registered at the IdP must match exactly. Behind a reverse proxy, set `OIDC_REDIRECT_URI` to the public callback URL instead of relying on the derived default.
|
||||
- **Revoked users can still access DocsGPT** — without back-channel logout, sessions outlive IdP-side revocation until the next renewal or expiry. Configure [back-channel logout](#back-channel-logout) for instant revocation, deactivate the user over [SCIM](#scim-user-provisioning), or lower `OIDC_SESSION_LIFETIME_SECONDS`.
|
||||
- **SCIM requests fail** — `404`: `SCIM_ENABLED` is not `true`. `503`: SCIM is enabled but `SCIM_TOKEN` is unset. `401`: the presented bearer token doesn't match `SCIM_TOKEN`.
|
||||
- **Login endpoints return 503** — Redis is unreachable or the IdP discovery document can't be fetched from the API host.
|
||||
@@ -0,0 +1,111 @@
|
||||
---
|
||||
title: Observability
|
||||
description: Send traces, metrics, and logs from DocsGPT to any OpenTelemetry-compatible backend (Axiom, Honeycomb, Grafana, Datadog, Jaeger, etc.).
|
||||
---
|
||||
|
||||
import { Callout } from 'nextra/components'
|
||||
|
||||
# Observability
|
||||
|
||||
DocsGPT bundles the OpenTelemetry SDK and auto-instrumentation packages
|
||||
in `application/requirements.txt` — they install with the rest of the
|
||||
backend deps. Telemetry is **off by default**; opt in by prefixing the
|
||||
launch command with `opentelemetry-instrument` and setting OTLP env
|
||||
vars.
|
||||
|
||||
Auto-instrumentation covers Flask, Starlette, Celery, SQLAlchemy,
|
||||
psycopg, Redis, requests, and Python logging. LLM/retriever calls are
|
||||
not captured at this layer — see *Going further* below.
|
||||
|
||||
## Enabling
|
||||
|
||||
Set these env vars in your `.env` (or compose `environment:` block):
|
||||
|
||||
```bash
|
||||
OTEL_SDK_DISABLED=false
|
||||
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
|
||||
OTEL_EXPORTER_OTLP_ENDPOINT=https://your-collector.example.com
|
||||
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer%20<token>
|
||||
OTEL_TRACES_EXPORTER=otlp
|
||||
OTEL_METRICS_EXPORTER=otlp
|
||||
OTEL_LOGS_EXPORTER=otlp
|
||||
OTEL_PYTHON_LOG_CORRELATION=true
|
||||
OTEL_RESOURCE_ATTRIBUTES=service.name=docsgpt-backend,deployment.environment=prod
|
||||
```
|
||||
|
||||
Then prefix the process command with `opentelemetry-instrument`. The
|
||||
simplest way is a compose override (no image rebuild):
|
||||
|
||||
```yaml
|
||||
# deployment/docker-compose.override.yaml
|
||||
services:
|
||||
backend:
|
||||
command: >
|
||||
opentelemetry-instrument gunicorn -w 1 -k uvicorn_worker.UvicornWorker
|
||||
--bind 0.0.0.0:7091 --config application/gunicorn_conf.py
|
||||
application.asgi:asgi_app
|
||||
environment:
|
||||
- OTEL_SERVICE_NAME=docsgpt-backend
|
||||
worker:
|
||||
command: opentelemetry-instrument celery -A application.app.celery worker -l INFO -B
|
||||
environment:
|
||||
- OTEL_SERVICE_NAME=docsgpt-celery-worker
|
||||
```
|
||||
|
||||
For local dev, prepend `dotenv run --` so the `OTEL_*` vars from `.env`
|
||||
reach `opentelemetry-instrument` before it boots the SDK:
|
||||
|
||||
```bash
|
||||
dotenv run -- opentelemetry-instrument flask --app application/app.py run --port=7091
|
||||
dotenv run -- opentelemetry-instrument celery -A application.app.celery worker -l INFO --pool=solo
|
||||
```
|
||||
|
||||
|
||||
<Callout type="info" emoji="ℹ️">
|
||||
Logs are exported in-process when `OTEL_LOGS_EXPORTER=otlp` is set —
|
||||
`application/core/logging_config.py` detects the flag and preserves
|
||||
the OTEL log handler. Without it, `logging` writes only to stdout.
|
||||
</Callout>
|
||||
|
||||
## Backend examples
|
||||
|
||||
### Axiom
|
||||
|
||||
```bash
|
||||
OTEL_EXPORTER_OTLP_ENDPOINT=https://api.axiom.co
|
||||
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer%20xaat-XXXX,X-Axiom-Dataset=docsgpt
|
||||
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
|
||||
```
|
||||
|
||||
`%20` is the URL-encoded space between `Bearer` and the token. Create
|
||||
the dataset in the Axiom UI before sending.
|
||||
|
||||
### Self-hosted OTLP collector / Jaeger / Tempo
|
||||
|
||||
```bash
|
||||
OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4317
|
||||
OTEL_EXPORTER_OTLP_PROTOCOL=grpc
|
||||
```
|
||||
|
||||
### Honeycomb / Grafana Cloud / Datadog
|
||||
|
||||
Each vendor publishes a single-line `OTEL_EXPORTER_OTLP_ENDPOINT` plus
|
||||
`OTEL_EXPORTER_OTLP_HEADERS` recipe — drop them in alongside the
|
||||
service-name override.
|
||||
|
||||
## Caveats
|
||||
|
||||
- The Dockerfile uses `gunicorn -w 1`. If you raise worker count, move
|
||||
SDK init into a `post_worker_init` hook to avoid one-thread-per-process
|
||||
exporter contention.
|
||||
- `asgi.py` wraps Flask in Starlette's `WSGIMiddleware`. Both
|
||||
instrumentors are installed, so each request produces a Starlette
|
||||
span enclosing a Flask span. Drop
|
||||
`opentelemetry-instrumentation-flask` from `requirements.txt` if the
|
||||
duplication is noisy.
|
||||
- OTEL packages add ~50 MB to the image. They install on every build —
|
||||
the runtime cost is zero unless you set `opentelemetry-instrument` on
|
||||
the command and set the OTLP env vars.
|
||||
- The OTEL exporter ecosystem currently caps `protobuf` at `<7`, so the
|
||||
backend runs on protobuf 6.x. This will catch up in a future OTEL
|
||||
release.
|
||||
@@ -0,0 +1,151 @@
|
||||
---
|
||||
title: PostgreSQL for User Data
|
||||
description: PostgreSQL is the user-data store for DocsGPT. Covers auto-bootstrap, production hardening, and the one-shot migration from legacy MongoDB deployments.
|
||||
---
|
||||
|
||||
import { Callout } from 'nextra/components'
|
||||
|
||||
# PostgreSQL for User Data
|
||||
|
||||
DocsGPT stores conversations, agents, prompts, sources, attachments,
|
||||
workflows, logs, and token usage in **PostgreSQL**. MongoDB is no longer
|
||||
required.
|
||||
|
||||
<Callout type="info" emoji="ℹ️">
|
||||
Vector stores are independent — `VECTOR_STORE` can still be `pgvector`,
|
||||
`faiss`, `qdrant`, `milvus`, `elasticsearch`, or `mongodb`.
|
||||
</Callout>
|
||||
|
||||
## Quickstart
|
||||
|
||||
Three common paths. Each assumes Postgres 13+ and the default env vars
|
||||
`AUTO_MIGRATE=true` / `AUTO_CREATE_DB=true` (both ship enabled).
|
||||
|
||||
### Docker Compose
|
||||
|
||||
The bundled compose file ships a `postgres` service. App boot handles the
|
||||
rest — no sidecar, no init job.
|
||||
|
||||
```bash
|
||||
cd deployment && docker compose up
|
||||
```
|
||||
|
||||
### Managed Postgres (Neon, RDS, Supabase, Cloud SQL)
|
||||
|
||||
Point `POSTGRES_URI` at the provider-given URI. The app applies the
|
||||
schema on first boot.
|
||||
|
||||
```bash
|
||||
export POSTGRES_URI="postgresql://user:pass@host/docsgpt?sslmode=require"
|
||||
uvicorn application.asgi:asgi_app --host 0.0.0.0 --port 7091
|
||||
```
|
||||
|
||||
### Bare-metal Postgres
|
||||
|
||||
Run Postgres locally and point `POSTGRES_URI` at the default superuser.
|
||||
First boot creates both the database and the schema.
|
||||
|
||||
```bash
|
||||
export POSTGRES_URI="postgresql://postgres@localhost/docsgpt"
|
||||
uvicorn application.asgi:asgi_app --host 0.0.0.0 --port 7091
|
||||
```
|
||||
|
||||
Prefer a dedicated non-superuser role? Create it once as superuser — the
|
||||
app never creates roles.
|
||||
|
||||
```sql
|
||||
CREATE ROLE docsgpt LOGIN PASSWORD 'docsgpt' CREATEDB;
|
||||
-- Then: POSTGRES_URI=postgresql://docsgpt:docsgpt@localhost/docsgpt
|
||||
```
|
||||
|
||||
## How auto-bootstrap works
|
||||
|
||||
Two env vars control startup behavior. Both default to `true` in the
|
||||
app and are idempotent.
|
||||
|
||||
| Setting | Effect | Requires |
|
||||
| --- | --- | --- |
|
||||
| `AUTO_CREATE_DB` | If the target database is missing, connects to the server's `postgres` maintenance DB and issues `CREATE DATABASE`. | `CREATEDB` privilege (or superuser) |
|
||||
| `AUTO_MIGRATE` | Runs `alembic upgrade head` against the target database. | Table-owner or superuser on the target DB |
|
||||
|
||||
Concurrent workers serialize through `alembic_version`, so rolling
|
||||
restarts are safe. If the role lacks the required privilege, startup
|
||||
fails fast with a clear error rather than silently skipping.
|
||||
|
||||
<Callout type="info" emoji="ℹ️">
|
||||
Convenient in dev. In production, disable both and run migrations as
|
||||
an explicit step — see [Production hardening](#production-hardening).
|
||||
</Callout>
|
||||
|
||||
## Production hardening
|
||||
|
||||
Set both flags to `false` in prod and run migrations as a gated,
|
||||
auditable step before rolling out the app.
|
||||
|
||||
```env
|
||||
AUTO_MIGRATE=false
|
||||
AUTO_CREATE_DB=false
|
||||
```
|
||||
|
||||
Run migrations from your CI/CD pipeline, a Kubernetes `Job`, or an
|
||||
init-container ahead of the app rollout:
|
||||
|
||||
```bash
|
||||
python scripts/db/init_postgres.py
|
||||
# equivalently:
|
||||
alembic -c application/alembic.ini upgrade head
|
||||
```
|
||||
|
||||
The reasoning: the app's runtime role shouldn't carry DDL privileges,
|
||||
migrations should gate each rollout, and an explicit step is
|
||||
auditable — implicit first-boot bootstrap is fine for dev but muddies
|
||||
prod deploys.
|
||||
|
||||
<Callout type="warning" emoji="⚠️">
|
||||
Migrations are not reversible by the app. Always back up production
|
||||
Postgres before running `alembic upgrade head` on a new release.
|
||||
</Callout>
|
||||
|
||||
## Migrating from MongoDB
|
||||
|
||||
One-shot, offline, app stopped. The app itself will create the
|
||||
Postgres schema when it boots — you only need to run the data copy.
|
||||
|
||||
```bash
|
||||
pip install -r application/requirements.txt
|
||||
pip install 'pymongo>=4.6'
|
||||
|
||||
export POSTGRES_URI="postgresql://docsgpt:docsgpt@localhost:5432/docsgpt"
|
||||
export MONGO_URI="mongodb://user:pass@host:27017/docsgpt"
|
||||
|
||||
python scripts/db/backfill.py --dry-run # preview
|
||||
python scripts/db/backfill.py # real run
|
||||
# or: python scripts/db/backfill.py --tables users,agents
|
||||
```
|
||||
|
||||
Then unset `MONGO_URI` and start the backend — nothing consults Mongo
|
||||
in the default path anymore. The backfill is idempotent (per-table
|
||||
`ON CONFLICT` upserts, event-log tables deduped via `mongo_id`), so
|
||||
re-running is safe and re-syncs any drifted rows. Keep Mongo online
|
||||
until you've verified Postgres is complete; decommission afterwards
|
||||
unless you still use it as a vector store.
|
||||
|
||||
<Callout type="warning" emoji="⚠️">
|
||||
No dual-write window and no runtime flag — on the current version,
|
||||
Postgres is the only user-data store the backend reads or writes.
|
||||
</Callout>
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **`relation "..." does not exist`** — schema not applied. Either let
|
||||
the app bootstrap it (`AUTO_MIGRATE=true`) or run
|
||||
`python scripts/db/init_postgres.py`.
|
||||
- **`permission denied to create database`** — the role lacks
|
||||
`CREATEDB`. As superuser: `ALTER ROLE <name> CREATEDB;`. Or create
|
||||
the database manually and set `AUTO_CREATE_DB=false`.
|
||||
- **`role "docsgpt" does not exist`** — roles are never auto-created.
|
||||
As superuser: `CREATE ROLE docsgpt LOGIN PASSWORD '...';`.
|
||||
- **SSL errors on a managed provider** — append `?sslmode=require` to
|
||||
`POSTGRES_URI`.
|
||||
- **`ModuleNotFoundError: pymongo`** — `pip install 'pymongo>=4.6'`
|
||||
(only needed for the one-shot Mongo backfill).
|
||||
@@ -0,0 +1,254 @@
|
||||
---
|
||||
title: Hosting DocsGPT on Railway
|
||||
description: Learn how to deploy your own DocsGPT instance on Railway with this step-by-step tutorial
|
||||
---
|
||||
|
||||
# Self-hosting DocsGPT on Railway
|
||||
|
||||
|
||||
|
||||
Here's a step-by-step guide on how to host DocsGPT on Railway App.
|
||||
|
||||
|
||||
|
||||
At first Clone and set up the project locally to run , test and Modify.
|
||||
|
||||
|
||||
|
||||
### 1. Clone and GitHub SetUp
|
||||
|
||||
a. Open Terminal (Windows Shell or Git bash(recommended)).
|
||||
|
||||
|
||||
|
||||
b. Type `git clone https://github.com/arc53/DocsGPT.git`
|
||||
|
||||
|
||||
|
||||
#### Download the package information
|
||||
|
||||
|
||||
|
||||
Once it has finished cloning the repository, it is time to download the package information from all sources. To do so, simply enter the following command:
|
||||
|
||||
|
||||
|
||||
`sudo apt update`
|
||||
|
||||
|
||||
|
||||
#### Install Docker and Docker Compose
|
||||
|
||||
|
||||
|
||||
DocsGPT backend and worker use Python, Frontend is written on React and the whole application is containerized using Docker. To install Docker and Docker Compose, enter the following commands:
|
||||
|
||||
|
||||
|
||||
`sudo apt install docker.io`
|
||||
|
||||
|
||||
|
||||
And now install docker-compose:
|
||||
|
||||
|
||||
|
||||
`sudo apt install docker-compose`
|
||||
|
||||
|
||||
|
||||
#### Access the DocsGPT Folder
|
||||
|
||||
|
||||
|
||||
Enter the following command to access the folder in which the DocsGPT docker-compose file is present.
|
||||
|
||||
|
||||
|
||||
`cd DocsGPT/`
|
||||
|
||||
|
||||
|
||||
#### Prepare the Environment
|
||||
|
||||
|
||||
|
||||
Inside the DocsGPT folder create a `.env` file and copy the contents of `.env_sample` into it.
|
||||
|
||||
|
||||
|
||||
`nano .env`
|
||||
|
||||
|
||||
|
||||
Make sure your `.env` file looks like this:
|
||||
|
||||
|
||||
|
||||
```
|
||||
API_KEY=<Your LLM API key>
|
||||
LLM_NAME=docsgpt
|
||||
VITE_API_STREAMING=true
|
||||
```
|
||||
|
||||
|
||||
|
||||
To save the file, press CTRL+X, then Y, and then ENTER.
|
||||
|
||||
|
||||
|
||||
Next, set the correct IP for the Backend by opening the docker-compose.yaml file:
|
||||
|
||||
|
||||
|
||||
`nano deployment/docker-compose.yaml`
|
||||
|
||||
|
||||
|
||||
And Change line 7 to: `VITE_API_HOST=http://localhost:7091`
|
||||
|
||||
to this `VITE_API_HOST=http://<your instance public IP>:7091`
|
||||
|
||||
|
||||
|
||||
This will allow the frontend to connect to the backend.
|
||||
|
||||
|
||||
|
||||
#### Running the Application
|
||||
|
||||
|
||||
|
||||
You're almost there! Now that all the necessary bits and pieces have been installed, it is time to run the application. To do so, use the following command:
|
||||
|
||||
|
||||
|
||||
`sudo docker compose -f deployment/docker-compose.yaml up -d`
|
||||
|
||||
|
||||
|
||||
Launching it for the first time will take a few minutes to download all the necessary dependencies and build.
|
||||
|
||||
|
||||
|
||||
Once this is done you can go ahead and close the terminal window.
|
||||
|
||||
|
||||
|
||||
### 2. Pushing it to your own Repository
|
||||
|
||||
|
||||
|
||||
a. Create a Repository on your GitHub.
|
||||
|
||||
|
||||
|
||||
b. Open Terminal in the same directory of the Cloned project.
|
||||
|
||||
|
||||
|
||||
c. Type `git init`
|
||||
|
||||
|
||||
|
||||
d. `git add .`
|
||||
|
||||
|
||||
|
||||
e. `git commit -m "first-commit"`
|
||||
|
||||
|
||||
|
||||
f. `git remote add origin <your repository link>`
|
||||
|
||||
|
||||
|
||||
g. `git push git push --set-upstream origin master`
|
||||
|
||||
Your local files will now be pushed to your GitHub Account. :)
|
||||
|
||||
|
||||
### 3. Create a Railway Account:
|
||||
|
||||
|
||||
|
||||
If you haven't already, create or log in to your railway account do it by visiting [Railway](https://railway.app/)
|
||||
|
||||
|
||||
|
||||
Signup via **GitHub** [Recommended].
|
||||
|
||||
|
||||
|
||||
### 4. Start New Project:
|
||||
|
||||
|
||||
|
||||
a. Open Railway app and Click on "Start New Project."
|
||||
|
||||
|
||||
|
||||
b. Choose any from the list of options available (Recommended "**Deploy from GitHub Repo**")
|
||||
|
||||
|
||||
|
||||
c. Choose the required Repository from your GitHub.
|
||||
|
||||
|
||||
|
||||
d. Configure and allow access to modify your GitHub content from the pop-up window.
|
||||
|
||||
|
||||
|
||||
e. Agree to all the terms and conditions.
|
||||
|
||||
|
||||
|
||||
PS: It may take a few minutes for the account setup to complete.
|
||||
|
||||
|
||||
|
||||
#### You will get A free trial of $5 (use it for trial and then purchase if satisfied and needed)
|
||||
|
||||
|
||||
|
||||
### 5. Connecting to Your newly Railway app with GitHub
|
||||
|
||||
|
||||
|
||||
a. Choose DocsGPT repo from the list of your GitHub repository that you want to deploy now.
|
||||
|
||||
|
||||
|
||||
b. Click on Deploy now.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
c. Select Variables Tab.
|
||||
|
||||
|
||||
|
||||
d. Upload the env file here that you used for local setup.
|
||||
|
||||
|
||||
|
||||
e. Go to Settings Tab now.
|
||||
|
||||
|
||||
|
||||
f. Go to "Networking" and click on Generate Domain Name, to get the URL of your hosted project.
|
||||
|
||||
|
||||
|
||||
g. You can update the Root directory, build command, installation command as per need.
|
||||
|
||||
*[However recommended not the disturb these options and leave them as default if not that needed.]*
|
||||
|
||||
|
||||
|
||||
|
||||
Your own DocsGPT is now available at the Generated domain URl. :)
|
||||
@@ -0,0 +1,48 @@
|
||||
export default {
|
||||
"DocsGPT-Settings": {
|
||||
"title": "⚙️ App Configuration",
|
||||
"href": "/Deploying/DocsGPT-Settings"
|
||||
},
|
||||
"OIDC-SSO": {
|
||||
"title": "🔐 SSO with OIDC",
|
||||
"href": "/Deploying/OIDC-SSO"
|
||||
},
|
||||
"Access-Control": {
|
||||
"title": "👥 Access Control & Teams",
|
||||
"href": "/Deploying/Access-Control"
|
||||
},
|
||||
"Docker-Deploying": {
|
||||
"title": "🛳️ Docker Setup",
|
||||
"href": "/Deploying/Docker-Deploying"
|
||||
},
|
||||
"Development-Environment": {
|
||||
"title": "🛠️Development Environment",
|
||||
"href": "/Deploying/Development-Environment"
|
||||
},
|
||||
"Kubernetes-Deploying": {
|
||||
"title": "☸️ Deploying on Kubernetes",
|
||||
"href": "/Deploying/Kubernetes-Deploying"
|
||||
},
|
||||
"Hosting-the-app": {
|
||||
"title": "☁️ Hosting DocsGPT",
|
||||
"href": "/Deploying/Hosting-the-app"
|
||||
},
|
||||
"Postgres-Migration": {
|
||||
"title": "🐘 PostgreSQL for User Data",
|
||||
"href": "/Deploying/Postgres-Migration"
|
||||
},
|
||||
"Observability": {
|
||||
"title": "🔭 Observability",
|
||||
"href": "/Deploying/Observability"
|
||||
},
|
||||
"Amazon-Lightsail": {
|
||||
"title": "Hosting DocsGPT on Amazon Lightsail",
|
||||
"href": "/Deploying/Amazon-Lightsail",
|
||||
"display": "hidden"
|
||||
},
|
||||
"Railway": {
|
||||
"title": "Hosting DocsGPT on Railway",
|
||||
"href": "/Deploying/Railway",
|
||||
"display": "hidden"
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user