omnigent-ai--omnigent/deploy/modal/README.md

Omnigent on Modal

Modal plays two distinct roles for Omnigent:

1. Server deploy target — run the Omnigent server itself on Modal as a single always-on web server (modal_app.py in this directory).
2. Sandbox provider — disposable cloud machines for running Omnigent hosts, so sessions execute in the cloud instead of on your laptop.

The two are independent: you can deploy the server anywhere and still use Modal sandboxes for hosts, or vice versa.

Deploying the server

Run the Omnigent server on Modal as a single always-on web server. modal_app.py pulls the standard server image and launches the same Docker entrypoint every other platform uses; Modal provides the HTTPS URL, log streaming, and a persistent Volume for the artifact store — uploaded agent bundles survive restarts and redeploys here, unlike on Heroku or Cloudflare.

Prerequisites

• A Modal account and the CLI: pip install modal && modal setup. No Docker needed locally — Modal's builders pull the image.
• A Postgres database. Modal has no managed Postgres — the fastest is Neon: create one at pg.new and copy the connection string.

Deploy

# 1. One secret bundle with the three required values. The app URL is
#    deterministic: https://<workspace>--omnigent-server.modal.run
#    (your workspace name is shown by `modal profile current`).
modal secret create omnigent-deploy \
  DATABASE_URL='postgres://…neon.tech/…' \
  OMNIGENT_ACCOUNTS_COOKIE_SECRET="$(openssl rand -hex 32)" \
  OMNIGENT_ACCOUNTS_BASE_URL='https://<workspace>--omnigent-server.modal.run'

# 2. Ship it.
modal deploy deploy/modal/modal_app.py

modal deploy prints the live URL — if it differs from what you guessed in step 1 (e.g. a non-default Modal environment adds a suffix), update the secret and redeploy.

The first boot runs DB migrations over the network (~1 minute on Neon).

Get the admin password: the first boot prints it to the app log:

modal app logs omnigent

✓ Created initial admin account (accounts auth provider).
    password: <generated>

Log in as the admin and invite teammates from Members in the web UI.

To set a known admin password instead, add OMNIGENT_ACCOUNTS_INIT_ADMIN_PASSWORD=<password> to the omnigent-deploy secret before the first deploy.

Modal-specific caveats

• 2 MiB WebSocket message cap. Modal's ingress limits WebSocket messages to 2 MiB each, well below the runner tunnel's own 100 MiB allowance. Normal streaming traffic (events, terminal frames) is far smaller, but a single very large tool payload over the tunnel can fail on this platform.
• Connections reset at the 24 h input timeout. A proxied WebSocket occupies one Modal function input, and inputs are capped at 24 hours — so a tunnel lives at most a day before being cut. Runners auto-reconnect (0.5–10 s jittered backoff).
• One always-on container by design. min_containers=1 / max_containers=1 in modal_app.py: the runner registry is in-memory, so traffic must land on a single container, and scale-to-zero would kill live tunnels. Don't raise max_containers expecting horizontal scaling.
• No SQLite tier. The artifact Volume is durable but is not a place for a SQLite database (eventual-consistency semantics); use Postgres.

Use your own IdP instead (OIDC)

Add the OIDC values to the omnigent-deploy secret (Modal secrets are key-value bundles; modal secret create with the same name replaces it) and redeploy:

modal secret create omnigent-deploy \
  DATABASE_URL='…' \
  OMNIGENT_AUTH_PROVIDER=oidc \
  OMNIGENT_OIDC_ISSUER='https://github.com' \
  OMNIGENT_OIDC_CLIENT_ID='…' \
  OMNIGENT_OIDC_CLIENT_SECRET='…' \
  OMNIGENT_OIDC_REDIRECT_URI='https://<workspace>--omnigent-server.modal.run/auth/callback' \
  OMNIGENT_OIDC_COOKIE_SECRET="$(openssl rand -hex 32)"

The IdP registration steps (GitHub / Google / Okta callback URLs, domain allow-listing) are identical to the other platforms — see deploy/render/README.md.

Custom domain

Pass custom_domains=["omnigent.example.com"] to @modal.web_server in modal_app.py (requires a paid Modal plan), point your DNS at Modal per the printed instructions, and update OMNIGENT_ACCOUNTS_BASE_URL (or the OIDC redirect URI) to match.

Upgrading

modal deploy deploy/modal/modal_app.py again — Modal re-resolves ghcr.io/omnigent-ai/omnigent-server:latest, so a redeploy is an upgrade. The rollout replaces the container; runners reconnect.

Cost

Modal bills actual usage: memory at ~$0.008/GiB-hour and CPU by the cycle (so an idle server's CPU line is small). An always-on 1 GiB instance runs roughly $6–8/month, which fits inside the Starter plan's $30/month of free credits — making this effectively free for a lightly loaded server. Rates: modal.com/pricing.

Sandboxes for runner hosts

Modal sandboxes give you disposable cloud machines for running Omnigent hosts — no laptop tethered to a session, no VM to babysit. There are two ways to use them:

1. CLI-launched sandboxes — you provision a sandbox from your terminal and register it as a host with your server. Good for development and for running your local checkout's code in the cloud.
2. Server-managed sandboxes — the server provisions a sandbox automatically when a session is created with "host_type": "managed", and terminates it when the session is deleted. Good for production deployments where users shouldn't have to think about hosts at all.

Both boot from the official prebaked host image, so startup is seconds, not minutes.

Sandbox prerequisites

pip install 'omnigent[modal]'   # installs the modal SDK extra
modal token new                  # one-time browser auth with Modal

modal token new writes ~/.modal.toml. Anywhere Omnigent needs to talk to Modal (your laptop for the CLI flow, the server for the managed flow), Modal credentials must be available — either that file or the MODAL_TOKEN_ID / MODAL_TOKEN_SECRET environment variables.

The host image

Sandboxes boot from ghcr.io/omnigent-ai/omnigent-host:latest, an image published by CI from the host target of deploy/docker/Dockerfile with Omnigent and its dependencies preinstalled — including the coding-harness CLIs (claude, codex, pi, kiro-cli), so agents on any harness run without an in-sandbox install.

To use a different image (a fork, or extra tooling baked in), build the same target and push it anywhere Modal can pull from:

docker build -f deploy/docker/Dockerfile --target host \
  -t docker.io/<you>/omnigent-host:latest .
docker push docker.io/<you>/omnigent-host:latest

Then point Omnigent at it — OMNIGENT_MODAL_HOST_IMAGE for the CLI flow, or sandbox.modal.image in the server config for the managed flow (see below). For private registries, set OMNIGENT_MODAL_REGISTRY_SECRET to the name of a Modal secret containing REGISTRY_USERNAME / REGISTRY_PASSWORD.

Note

Building on Apple Silicon? Pass --platform linux/amd64 — Modal sandboxes run x86_64.

CLI-launched sandboxes

Provision a sandbox and ship your local checkout into it:

omnigent sandbox create --provider modal

This pulls the host image, builds wheels from your local checkout, and overlays them on top — so the sandbox runs your code, not whatever the image was built from. Then register it as a host with your server:

omnigent sandbox connect --provider modal \
  --sandbox-id <id-printed-by-create> \
  --server https://your-host

connect runs omnigent host inside the sandbox and holds the connection open in your terminal — Ctrl-C tears it down. New sessions targeting that host now run in the sandbox.

Running multiple sandboxes against one server? Pass a unique --host-name <label> to each connect — the server keys hosts on (owner, name), and sandboxes that share a hostname collide.

Sandboxes are disposable. When your code changes, create a new one.

Note

Modal caps sandbox lifetime at 24 hours (a platform hard limit). Re-run create + connect to roll the host onto a fresh sandbox.

For provider-side lifecycle (list / status / terminate), use Modal's own tooling — the Modal dashboard or the modal CLI.

Connecting to an authenticated server

connect runs omnigent host inside the sandbox, and that host must present credentials when it dials back to a server that requires authentication. The interactive omnigent login browser flow can't run inside a sandbox, so inject the keys for the relevant server instead: park them in a Modal secret and name it in OMNIGENT_MODAL_SANDBOX_SECRETS (comma-separated) before running create:

modal secret create omnigent-server-auth \
  DATABRICKS_HOST=https://example.databricks.com \
  DATABRICKS_TOKEN=<your-pat>
export OMNIGENT_MODAL_SANDBOX_SECRETS=omnigent-server-auth
omnigent sandbox create --provider modal

The in-sandbox host mints a fresh bearer token from those credentials on every connect and reconnect. For a server fronted by Databricks authentication, inject DATABRICKS_HOST plus either DATABRICKS_TOKEN (a PAT) or DATABRICKS_CLIENT_ID / DATABRICKS_CLIENT_SECRET (an OAuth service principal — re-minting keeps a long-lived sandbox connected past any single token's expiry).

A server with no authentication on the host tunnel needs none of this, and neither do server-managed sandboxes — those authenticate with a server-minted per-launch token automatically.

(The same env var also carries LLM / git credentials for CLI-launched sandboxes — any secret named in OMNIGENT_MODAL_SANDBOX_SECRETS lands in the sandbox environment, exactly like sandbox.modal.secrets does for managed launches.)

Server-managed sandboxes

With managed hosts, the server does all of the above per session. Add a sandbox: section to the server config (omnigent server -c config.yaml, or <data_dir>/config.yaml):

sandbox:
  provider: modal
  server_url: https://your-host    # public URL sandboxes dial back to

server_url must be reachable from Modal's cloud — a public HTTPS URL, not localhost. The server itself needs Modal credentials in its environment (MODAL_TOKEN_ID / MODAL_TOKEN_SECRET, or a mounted ~/.modal.toml).

Now create sessions with host_type: "managed":

curl -X POST https://your-host/v1/sessions \
  -H "Content-Type: application/json" \
  -d '{"agent_id": "agent_...", "host_type": "managed"}'

The create returns immediately; the server provisions a fresh sandbox in the background, starts a host in it, and binds the session once the host comes online (host_id / workspace appear on GET /v1/sessions/{id} when it does). A message posted before then waits for the launch to settle, so you can send the first prompt right away. Deleting the session terminates the sandbox and removes the host. Each sandbox authenticates back with a server-minted, per-launch token — no user credentials ever enter the sandbox.

Optional modal: settings:

sandbox:
  provider: modal
  server_url: https://your-host
  modal:
    image: docker.io/<you>/omnigent-host:latest   # default: official image
    secrets: [omnigent-llm]                       # Modal secrets to inject

LLM credentials for managed sandboxes

A fresh sandbox has no API keys. Park your provider credentials in a Modal secret and list it under sandbox.modal.secrets — its env vars are injected into every managed sandbox, and the in-sandbox host forwards the standard harness credential vars to its runners:

modal secret create omnigent-llm \
  OMNIGENT_ANTHROPIC_API_KEY=sk-ant-… OPENAI_API_KEY=sk-…

The forwarded set covers the variables the harnesses themselves resolve — and it reaches well beyond the first-party APIs. The *_BASE_URL variables redirect each harness to any compatible endpoint, so the same mechanism covers frontier providers, gateways like OpenRouter and LiteLLM, and self-hosted open-source models:

Variable	Enables
OMNIGENT_ANTHROPIC_API_KEY or ANTHROPIC_API_KEY	Claude models on the Anthropic API (claude-sdk, pi, claude-code harnesses). Prefer the OMNIGENT_ form for Claude Code so the raw ANTHROPIC_API_KEY env var is not present in the CLI process.
ANTHROPIC_AUTH_TOKEN, ANTHROPIC_BASE_URL	Anthropic-compatible gateways — point claude-code at a LiteLLM proxy, a Bedrock/Vertex bridge, or a corporate gateway
CLAUDE_CODE_OAUTH_TOKEN	claude-code with a Claude subscription (no API key)
OPENAI_API_KEY	OpenAI models on the OpenAI API (codex, openai-agents harnesses)
OPENAI_BASE_URL	Any OpenAI-compatible endpoint — the de-facto standard API of the open-model ecosystem. Gateways (OpenRouter, LiteLLM), hosted open-weights providers (Together, Fireworks, Groq), or self-hosted vLLM / Ollama — this is how Llama, Qwen, DeepSeek, and friends plug in
CODEX_ACCESS_TOKEN	codex with a ChatGPT Business/Enterprise workspace
GEMINI_API_KEY	Gemini models on the Google AI API

Common setups:

• Claude with an API key — put OMNIGENT_ANTHROPIC_API_KEY in the secret. Omnigent resolves it into Claude Code's apiKeyHelper; do not also set ANTHROPIC_API_KEY unless you are okay with Claude Code detecting the raw custom key env var.
• Claude with a subscription — run claude setup-token on your own machine (one-time browser auth) and store the resulting long-lived token as CLAUDE_CODE_OAUTH_TOKEN.
• Codex with an API key — put OPENAI_API_KEY in the secret.
• Codex with a ChatGPT Business/Enterprise plan — mint a Codex access token in the ChatGPT admin console (a workspace admin must grant the permission) and store it as CODEX_ACCESS_TOKEN.
• Codex with a ChatGPT Plus/Pro plan — there is no headless token for personal plans. Codex stores personal-plan auth in ~/.codex/auth.json with effectively single-use refresh tokens, so copies of that file across machines invalidate each other — it can't be injected into disposable sandboxes via a shared secret. Use an API key or codex login --device-auth inside a long-lived sandbox instead (device-code login must first be enabled in ChatGPT → Settings → Security).
• Gateways and open-source models — set OPENAI_BASE_URL to the endpoint plus its key as OPENAI_API_KEY (e.g. OPENAI_BASE_URL=https://openrouter.ai/api/v1 with an OpenRouter key, or your own vLLM server's URL). Anthropic-side gateways work the same way via ANTHROPIC_BASE_URL + ANTHROPIC_AUTH_TOKEN.

For env vars beyond the standard set, add OMNIGENT_RUNNER_ENV_PASSTHROUGH=NAME1,NAME2 to the secret — the host forwards the named extras to its runners.

To check what actually landed in a sandbox, exec into it with Modal's CLI and inspect the environment:

modal shell <sandbox-id>          # interactive shell in the sandbox
env | grep -E 'ANTHROPIC|OPENAI|GIT'

Git credentials (private repositories)

Sandboxes clone repository workspaces anonymously by default, which covers public repositories only. For private repositories — both the clone the server runs at session create and the git fetch / git push the agent runs later — put an HTTPS token in a Modal secret as GIT_TOKEN:

modal secret create omnigent-git GIT_TOKEN=github_pat_…

and list the secret under sandbox.modal.secrets (multiple secrets compose, so keeping git and LLM credentials in separate secrets is fine):

sandbox:
  provider: modal
  server_url: https://your-host
  modal:
    secrets: [omnigent-llm, omnigent-git]

The host image ships a git credential helper that answers HTTPS authentication from GIT_TOKEN, so nothing is written to disk and no URL ever embeds the token. Details by provider:

• GitHub — use a fine-grained personal access token scoped to the repositories the sandbox needs (Contents: read, or read/write if the agent pushes). The default auth username (x-access-token) is already correct.
• GitLab — create a project or personal access token with read_repository / write_repository and add GIT_USERNAME=oauth2 to the secret.
• Other HTTPS remotes — any server accepting basic auth works; set GIT_USERNAME if it requires a specific username.

Use HTTPS repository URLs (https://github.com/org/repo) for private workspaces — SSH URLs (git@github.com:…) would need a key and known-hosts setup inside the sandbox, which the managed flow does not provide.

The token is forwarded host→runner (like the LLM credentials above), so the agent's own git commands authenticate the same way the launch-time clone did. If the agent should also create commits, bake or configure user.name / user.email via your agent's instructions or a custom image.

Environment variable reference

Variable	Where it's read	Purpose
MODAL_TOKEN_ID / MODAL_TOKEN_SECRET	CLI machine / server	Modal API credentials (alternative to ~/.modal.toml)
OMNIGENT_MODAL_HOST_IMAGE	CLI machine / server	Override the host image ref (sandbox.modal.image takes precedence for managed)
OMNIGENT_MODAL_REGISTRY_SECRET	CLI machine / server	Modal secret name with REGISTRY_USERNAME / REGISTRY_PASSWORD for private image pulls
OMNIGENT_MODAL_SANDBOX_SECRETS	CLI machine / server	Comma-separated Modal secret names to inject (sandbox.modal.secrets takes precedence for managed)
OMNIGENT_RUNNER_ENV_PASSTHROUGH	inside the sandbox (set via a Modal secret)	Extra env var names the host forwards to runners
GIT_TOKEN	inside the sandbox (set via a Modal secret)	HTTPS token for private repository clone / fetch / push
GIT_USERNAME	inside the sandbox (set via a Modal secret)	Auth username paired with GIT_TOKEN (default x-access-token; GitLab uses oauth2)

All of the above are supported public configuration. The variables the managed launcher itself sets inside the sandbox — OMNIGENT_HOST_TOKEN, OMNIGENT_HOST_ID, OMNIGENT_HOST_NAME — are internal plumbing (server-minted per launch) and are never set by users.

Limits and troubleshooting

• 24-hour lifetime. Modal hard-caps sandbox lifetime at 24 hours. CLI flow: re-run create + connect. Managed flow: nothing to do — when the sandbox dies, the next message to the session provisions a fresh one under the same host (the session binding survives; a repository workspace is re-cloned). Uncommitted workspace changes die with the sandbox, so push work you care about.
• Resources. Sandboxes are created with 2 CPUs and 4 GiB of memory.
• Managed launch hangs then fails. The server waits up to two minutes for the in-sandbox host to come online. If it times out, check that server_url is publicly reachable from Modal, then inspect the host log inside the sandbox: /tmp/omnigent-host.log.
• Image pull failures. Private image without OMNIGENT_MODAL_REGISTRY_SECRET set, or a secret missing REGISTRY_USERNAME / REGISTRY_PASSWORD.
• Agent has no credentials. Verify the Modal secret is listed in sandbox.modal.secrets and its var names match the forwarded set above (or are named in OMNIGENT_RUNNER_ENV_PASSTHROUGH).