chore: import upstream snapshot with attribution
Deploy Site / deploy-vercel (push) Has been skipped
Deploy Site / deploy-docs (push) Has been skipped
Build Skills Index / build-index (push) Has been skipped
CI / Deny unrelated histories (push) Has been skipped
CI / Detect affected areas (push) Successful in 27m35s
CI / OSV scan (push) Failing after 4s
CI / Build&Test Docker image (push) Successful in 9s
CI / Supply-chain scan (push) Has been skipped
CI / Lint Docker scripts (push) Failing after 5m13s
CI / Check contributors (push) Failing after 12m8s
CI / Docs Site (push) Failing after 12m8s
CI / TypeScript (push) Failing after 12m8s
CI / Python lints (push) Failing after 12m9s
CI / Python tests (push) Failing after 12m9s
CI / Check uv.lock (push) Failing after 23m22s
CI / CI timing report (push) Has been cancelled
Build Skills Index / trigger-deploy (push) Has been cancelled
CI / All required checks pass (push) Has been cancelled
Deploy Site / deploy-vercel (push) Has been skipped
Deploy Site / deploy-docs (push) Has been skipped
Build Skills Index / build-index (push) Has been skipped
CI / Deny unrelated histories (push) Has been skipped
CI / Detect affected areas (push) Successful in 27m35s
CI / OSV scan (push) Failing after 4s
CI / Build&Test Docker image (push) Successful in 9s
CI / Supply-chain scan (push) Has been skipped
CI / Lint Docker scripts (push) Failing after 5m13s
CI / Check contributors (push) Failing after 12m8s
CI / Docs Site (push) Failing after 12m8s
CI / TypeScript (push) Failing after 12m8s
CI / Python lints (push) Failing after 12m9s
CI / Python tests (push) Failing after 12m9s
CI / Check uv.lock (push) Failing after 23m22s
CI / CI timing report (push) Has been cancelled
Build Skills Index / trigger-deploy (push) Has been cancelled
CI / All required checks pass (push) Has been cancelled
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"label": "Developer Guide",
|
||||
"position": 3,
|
||||
"link": {
|
||||
"type": "generated-index",
|
||||
"description": "Contribute to Hermes Agent — architecture, tools, skills, and more."
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,184 @@
|
||||
---
|
||||
sidebar_position: 2
|
||||
title: "ACP Internals"
|
||||
description: "How the ACP adapter works: lifecycle, sessions, event bridge, approvals, and tool rendering"
|
||||
---
|
||||
|
||||
# ACP Internals
|
||||
|
||||
The ACP adapter wraps Hermes' synchronous `AIAgent` in an async JSON-RPC stdio server.
|
||||
|
||||
Key implementation files:
|
||||
|
||||
- `acp_adapter/entry.py`
|
||||
- `acp_adapter/server.py`
|
||||
- `acp_adapter/session.py`
|
||||
- `acp_adapter/events.py`
|
||||
- `acp_adapter/permissions.py`
|
||||
- `acp_adapter/tools.py`
|
||||
- `acp_adapter/auth.py`
|
||||
- `acp_registry/agent.json`
|
||||
|
||||
## Boot flow
|
||||
|
||||
```text
|
||||
hermes acp / hermes-acp / python -m acp_adapter
|
||||
-> acp_adapter.entry.main()
|
||||
-> parse --version / --check / --setup before server startup
|
||||
-> load ~/.hermes/.env
|
||||
-> configure stderr logging
|
||||
-> construct HermesACPAgent
|
||||
-> acp.run_agent(agent, use_unstable_protocol=True)
|
||||
```
|
||||
|
||||
The Zed ACP Registry path launches the same adapter through `uvx --from 'hermes-agent[acp]==<version>' hermes-acp`, pointed at the `hermes-agent` PyPI release.
|
||||
|
||||
Stdout is reserved for ACP JSON-RPC transport. Human-readable logs go to stderr.
|
||||
|
||||
## Major components
|
||||
|
||||
### `HermesACPAgent`
|
||||
|
||||
`acp_adapter/server.py` implements the ACP agent protocol.
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- initialize / authenticate
|
||||
- new/load/resume/fork/list/cancel session methods
|
||||
- prompt execution
|
||||
- session model switching
|
||||
- wiring sync AIAgent callbacks into ACP async notifications
|
||||
|
||||
### `SessionManager`
|
||||
|
||||
`acp_adapter/session.py` tracks live ACP sessions.
|
||||
|
||||
Each session stores:
|
||||
|
||||
- `session_id`
|
||||
- `agent`
|
||||
- `cwd`
|
||||
- `model`
|
||||
- `history`
|
||||
- `cancel_event`
|
||||
|
||||
The manager is thread-safe and supports:
|
||||
|
||||
- create
|
||||
- get
|
||||
- remove
|
||||
- fork
|
||||
- list
|
||||
- cleanup
|
||||
- cwd updates
|
||||
|
||||
### Event bridge
|
||||
|
||||
`acp_adapter/events.py` converts AIAgent callbacks into ACP `session_update` events.
|
||||
|
||||
Bridged callbacks:
|
||||
|
||||
- `tool_progress_callback`
|
||||
- `thinking_callback` (currently set to `None` in the ACP bridge — reasoning is forwarded through `step_callback` instead)
|
||||
- `step_callback`
|
||||
|
||||
Because `AIAgent` runs in a worker thread while ACP I/O lives on the main event loop, the bridge uses:
|
||||
|
||||
```python
|
||||
asyncio.run_coroutine_threadsafe(...)
|
||||
```
|
||||
|
||||
### Permission bridge
|
||||
|
||||
`acp_adapter/permissions.py` adapts dangerous terminal approval prompts into ACP permission requests.
|
||||
|
||||
Mapping:
|
||||
|
||||
- `allow_once` -> Hermes `once`
|
||||
- `allow_always` -> Hermes `always`
|
||||
- reject options -> Hermes `deny`
|
||||
|
||||
Timeouts and bridge failures deny by default.
|
||||
|
||||
### Tool rendering helpers
|
||||
|
||||
`acp_adapter/tools.py` maps Hermes tools to ACP tool kinds and builds editor-facing content.
|
||||
|
||||
Examples:
|
||||
|
||||
- `patch` / `write_file` -> file diffs
|
||||
- `terminal` -> shell command text
|
||||
- `read_file` / `search_files` -> text previews
|
||||
- large results -> truncated text blocks for UI safety
|
||||
|
||||
## Session lifecycle
|
||||
|
||||
```text
|
||||
new_session(cwd)
|
||||
-> create SessionState
|
||||
-> create AIAgent(platform="acp", enabled_toolsets=["hermes-acp"])
|
||||
-> bind task_id/session_id to cwd override
|
||||
|
||||
prompt(..., session_id)
|
||||
-> extract text from ACP content blocks
|
||||
-> reset cancel event
|
||||
-> install callbacks + approval bridge
|
||||
-> run AIAgent in ThreadPoolExecutor
|
||||
-> update session history
|
||||
-> emit final agent message chunk
|
||||
```
|
||||
|
||||
### Cancelation
|
||||
|
||||
`cancel(session_id)`:
|
||||
|
||||
- sets the session cancel event
|
||||
- calls `agent.interrupt()` when available
|
||||
- causes the prompt response to return `stop_reason="cancelled"`
|
||||
|
||||
### Forking
|
||||
|
||||
`fork_session()` deep-copies message history into a new live session, preserving conversation state while giving the fork its own session ID and cwd.
|
||||
|
||||
## Provider/auth behavior
|
||||
|
||||
ACP does not implement its own auth store.
|
||||
|
||||
Instead it reuses Hermes' runtime resolver:
|
||||
|
||||
- `acp_adapter/auth.py`
|
||||
- `hermes_cli/runtime_provider.py`
|
||||
|
||||
So ACP advertises and uses the currently configured Hermes provider/credentials. It also always advertises a terminal setup auth method (`hermes-setup`, args `--setup`) so first-run registry clients can open Hermes' interactive model/provider configuration before starting a normal ACP session.
|
||||
|
||||
## Working directory binding
|
||||
|
||||
ACP sessions carry an editor cwd.
|
||||
|
||||
The session manager binds that cwd to the ACP session ID via task-scoped terminal/file overrides, so file and terminal tools operate relative to the editor workspace.
|
||||
|
||||
## Duplicate same-name tool calls
|
||||
|
||||
The event bridge tracks tool IDs FIFO per tool name, not just one ID per name. This is important for:
|
||||
|
||||
- parallel same-name calls
|
||||
- repeated same-name calls in one step
|
||||
|
||||
Without FIFO queues, completion events would attach to the wrong tool invocation.
|
||||
|
||||
## Approval callback restoration
|
||||
|
||||
ACP temporarily installs an approval callback on the terminal tool during prompt execution, then restores the previous callback afterward. This avoids leaving ACP session-specific approval handlers installed globally forever.
|
||||
|
||||
## Current limitations
|
||||
|
||||
- ACP sessions are persisted to the shared `~/.hermes/state.db` (SessionDB) and transparently restored across process restarts; they appear in `session_search`
|
||||
- non-text prompt blocks are currently ignored for request text extraction
|
||||
- editor-specific UX varies by ACP client implementation
|
||||
|
||||
## Related files
|
||||
|
||||
- `tests/acp/` — ACP test suite
|
||||
- `toolsets.py` — `hermes-acp` toolset definition
|
||||
- `hermes_cli/main.py` — `hermes acp` CLI subcommand
|
||||
- `pyproject.toml` — `[acp]` optional dependency + `hermes-acp` script
|
||||
@@ -0,0 +1,692 @@
|
||||
---
|
||||
sidebar_position: 9
|
||||
---
|
||||
|
||||
# Adding a Platform Adapter
|
||||
|
||||
This guide covers adding a new messaging platform to the Hermes gateway. A platform adapter connects Hermes to an external messaging service (Telegram, Discord, WeCom, etc.) so users can interact with the agent through that service.
|
||||
|
||||
:::tip
|
||||
There are two ways to add a platform:
|
||||
- **Plugin** (recommended for community/third-party): Drop a plugin directory into `~/.hermes/plugins/` — zero core code changes needed. See [Plugin Path](#plugin-path-recommended) below.
|
||||
- **Built-in**: Modify 20+ files across code, config, and docs. Use the [Built-in Checklist](#step-by-step-checklist-built-in-path) below.
|
||||
:::
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
User ↔ Messaging Platform ↔ Platform Adapter ↔ Gateway Runner ↔ AIAgent
|
||||
```
|
||||
|
||||
Every adapter extends `BasePlatformAdapter` from `gateway/platforms/base.py` and implements:
|
||||
|
||||
- **`connect()`** — Establish connection (WebSocket, long-poll, HTTP server, etc.) *(abstract)*
|
||||
- **`disconnect()`** — Clean shutdown *(abstract)*
|
||||
- **`send()`** — Send a text message to a chat *(abstract)*
|
||||
- **`send_typing()`** — Show typing indicator (optional override)
|
||||
- **`get_chat_info()`** — Return chat metadata (optional override)
|
||||
|
||||
Inbound messages are received by the adapter and forwarded via `self.handle_message(event)`, which the base class routes to the gateway runner.
|
||||
|
||||
## Plugin Path (Recommended)
|
||||
|
||||
The plugin system lets you add a platform adapter without modifying any core Hermes code. Your plugin is a directory with two files:
|
||||
|
||||
```
|
||||
~/.hermes/plugins/my-platform/
|
||||
plugin.yaml # Plugin metadata
|
||||
adapter.py # Adapter class + register() entry point
|
||||
```
|
||||
|
||||
### plugin.yaml
|
||||
|
||||
Plugin metadata. The `requires_env` and `optional_env` blocks auto-populate `hermes config` UI entries (see [Surfacing Env Vars](#surfacing-env-vars-in-hermes-config) below).
|
||||
|
||||
```yaml
|
||||
name: my-platform
|
||||
label: My Platform
|
||||
kind: platform
|
||||
version: 1.0.0
|
||||
description: My custom messaging platform adapter
|
||||
author: Your Name
|
||||
requires_env:
|
||||
- MY_PLATFORM_TOKEN # bare string works
|
||||
- name: MY_PLATFORM_CHANNEL # or rich dict for better UX
|
||||
description: "Channel to join"
|
||||
prompt: "Channel"
|
||||
password: false
|
||||
optional_env:
|
||||
- name: MY_PLATFORM_HOME_CHANNEL
|
||||
description: "Default channel for cron delivery"
|
||||
password: false
|
||||
```
|
||||
|
||||
### adapter.py
|
||||
|
||||
```python
|
||||
import os
|
||||
from gateway.platforms.base import (
|
||||
BasePlatformAdapter, SendResult, MessageEvent, MessageType,
|
||||
)
|
||||
from gateway.config import Platform, PlatformConfig
|
||||
|
||||
|
||||
class MyPlatformAdapter(BasePlatformAdapter):
|
||||
def __init__(self, config: PlatformConfig):
|
||||
super().__init__(config, Platform("my_platform"))
|
||||
extra = config.extra or {}
|
||||
self.token = os.getenv("MY_PLATFORM_TOKEN") or extra.get("token", "")
|
||||
|
||||
async def connect(self) -> bool:
|
||||
# Connect to the platform API, start listeners
|
||||
self._mark_connected()
|
||||
return True
|
||||
|
||||
async def disconnect(self) -> None:
|
||||
self._mark_disconnected()
|
||||
|
||||
async def send(self, chat_id, content, reply_to=None, metadata=None):
|
||||
# Send message via platform API
|
||||
return SendResult(success=True, message_id="...")
|
||||
|
||||
async def get_chat_info(self, chat_id):
|
||||
return {"name": chat_id, "type": "dm"}
|
||||
|
||||
|
||||
def check_requirements() -> bool:
|
||||
return bool(os.getenv("MY_PLATFORM_TOKEN"))
|
||||
|
||||
|
||||
def validate_config(config) -> bool:
|
||||
extra = getattr(config, "extra", {}) or {}
|
||||
return bool(os.getenv("MY_PLATFORM_TOKEN") or extra.get("token"))
|
||||
|
||||
|
||||
def _env_enablement() -> dict | None:
|
||||
token = os.getenv("MY_PLATFORM_TOKEN", "").strip()
|
||||
channel = os.getenv("MY_PLATFORM_CHANNEL", "").strip()
|
||||
if not (token and channel):
|
||||
return None
|
||||
seed = {"token": token, "channel": channel}
|
||||
home = os.getenv("MY_PLATFORM_HOME_CHANNEL")
|
||||
if home:
|
||||
seed["home_channel"] = {"chat_id": home, "name": "Home"}
|
||||
return seed
|
||||
|
||||
|
||||
def register(ctx):
|
||||
"""Plugin entry point — called by the Hermes plugin system."""
|
||||
ctx.register_platform(
|
||||
name="my_platform",
|
||||
label="My Platform",
|
||||
adapter_factory=lambda cfg: MyPlatformAdapter(cfg),
|
||||
check_fn=check_requirements,
|
||||
validate_config=validate_config,
|
||||
required_env=["MY_PLATFORM_TOKEN"],
|
||||
install_hint="pip install my-platform-sdk",
|
||||
# Env-driven auto-configuration — seeds PlatformConfig.extra from
|
||||
# env vars before adapter construction. See "Env-Driven Auto-
|
||||
# Configuration" section below.
|
||||
env_enablement_fn=_env_enablement,
|
||||
# Cron home-channel delivery support. Lets deliver=my_platform cron
|
||||
# jobs route without editing cron/scheduler.py. See "Cron Delivery"
|
||||
# section below.
|
||||
cron_deliver_env_var="MY_PLATFORM_HOME_CHANNEL",
|
||||
# Per-platform user authorization env vars
|
||||
allowed_users_env="MY_PLATFORM_ALLOWED_USERS",
|
||||
allow_all_env="MY_PLATFORM_ALLOW_ALL_USERS",
|
||||
# Message length limit for smart chunking (0 = no limit)
|
||||
max_message_length=4000,
|
||||
# LLM guidance injected into system prompt
|
||||
platform_hint=(
|
||||
"You are chatting via My Platform. "
|
||||
"It supports markdown formatting."
|
||||
),
|
||||
# Display
|
||||
emoji="💬",
|
||||
)
|
||||
|
||||
# Optional: register platform-specific tools
|
||||
ctx.register_tool(
|
||||
name="my_platform_search",
|
||||
toolset="my_platform",
|
||||
schema={...},
|
||||
handler=my_search_handler,
|
||||
)
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
Users configure the platform in `config.yaml`:
|
||||
|
||||
```yaml
|
||||
gateway:
|
||||
platforms:
|
||||
my_platform:
|
||||
enabled: true
|
||||
extra:
|
||||
token: "..."
|
||||
channel: "#general"
|
||||
```
|
||||
|
||||
Or via environment variables (which the adapter reads in `__init__`).
|
||||
|
||||
### What the Plugin System Handles Automatically
|
||||
|
||||
When you call `ctx.register_platform()`, the following integration points are handled for you — no core code changes needed:
|
||||
|
||||
| Integration point | How it works |
|
||||
|---|---|
|
||||
| Gateway adapter creation | Registry checked before built-in if/elif chain |
|
||||
| Config parsing | `Platform._missing_()` accepts any platform name |
|
||||
| Connected platform validation | Registry `validate_config()` called |
|
||||
| User authorization | `allowed_users_env` / `allow_all_env` checked |
|
||||
| Env-only auto-enable | `env_enablement_fn` seeds `PlatformConfig.extra` + `home_channel` |
|
||||
| YAML config bridge | `apply_yaml_config_fn` translates `config.yaml` keys into env vars / extras |
|
||||
| Cron delivery | `cron_deliver_env_var` makes `deliver=<name>` work |
|
||||
| `hermes config` UI entries | `requires_env` / `optional_env` in `plugin.yaml` auto-populate |
|
||||
| send engine (`tools/send_message_tool.py`) | Routes through live gateway adapter |
|
||||
| Webhook cross-platform delivery | Registry checked for known platforms |
|
||||
| `/update` command access | `allow_update_command` flag |
|
||||
| Channel directory | Plugin platforms included in enumeration |
|
||||
| System prompt hints | `platform_hint` injected into LLM context |
|
||||
| Message chunking | `max_message_length` for smart splitting |
|
||||
| PII redaction | `pii_safe` flag |
|
||||
| `hermes status` | Shows plugin platforms with `(plugin)` tag |
|
||||
| `hermes gateway setup` | Plugin platforms appear in setup menu |
|
||||
| `hermes tools` / `hermes skills` | Plugin platforms in per-platform config |
|
||||
| Token lock (multi-profile) | Use `acquire_scoped_lock()` in your `connect()` |
|
||||
| Orphaned config warning | Descriptive log when plugin is missing |
|
||||
|
||||
## Env-Driven Auto-Configuration
|
||||
|
||||
Most users set up a platform by dropping env vars into `~/.hermes/.env` rather than editing `config.yaml`. The `env_enablement_fn` hook lets your plugin pick those env vars up **before** the adapter is constructed, so `hermes gateway status`, `get_connected_platforms()`, and cron delivery see the correct state without instantiating the platform SDK.
|
||||
|
||||
```python
|
||||
def _env_enablement() -> dict | None:
|
||||
"""Seed PlatformConfig.extra from env vars.
|
||||
|
||||
Called by the platform registry during load_gateway_config().
|
||||
Return None when the platform isn't minimally configured — the
|
||||
caller then skips auto-enabling. Return a dict to seed extras.
|
||||
|
||||
The special 'home_channel' key is extracted and becomes a proper
|
||||
HomeChannel dataclass on the PlatformConfig; every other key is
|
||||
merged into PlatformConfig.extra.
|
||||
"""
|
||||
token = os.getenv("MY_PLATFORM_TOKEN", "").strip()
|
||||
channel = os.getenv("MY_PLATFORM_CHANNEL", "").strip()
|
||||
if not (token and channel):
|
||||
return None
|
||||
seed = {"token": token, "channel": channel}
|
||||
home = os.getenv("MY_PLATFORM_HOME_CHANNEL")
|
||||
if home:
|
||||
seed["home_channel"] = {
|
||||
"chat_id": home,
|
||||
"name": os.getenv("MY_PLATFORM_HOME_CHANNEL_NAME", "Home"),
|
||||
}
|
||||
return seed
|
||||
|
||||
|
||||
def register(ctx):
|
||||
ctx.register_platform(
|
||||
name="my_platform",
|
||||
label="My Platform",
|
||||
adapter_factory=lambda cfg: MyPlatformAdapter(cfg),
|
||||
check_fn=check_requirements,
|
||||
validate_config=validate_config,
|
||||
env_enablement_fn=_env_enablement,
|
||||
# ... other fields
|
||||
)
|
||||
```
|
||||
|
||||
|
||||
## YAML→env Config Bridge
|
||||
|
||||
Some users prefer setting `config.yaml` keys (`my_platform.require_mention`, `my_platform.allowed_channels`, etc.) over env vars. The `apply_yaml_config_fn` hook lets your plugin own this translation instead of forcing core `gateway/config.py` to know your platform's YAML schema.
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
def _apply_yaml_config(yaml_cfg: dict, platform_cfg: dict) -> dict | None:
|
||||
"""Translate config.yaml `my_platform:` keys into env vars / extras.
|
||||
|
||||
yaml_cfg — the full top-level parsed config.yaml dict
|
||||
platform_cfg — the platform's own sub-dict (yaml_cfg.get("my_platform", {}))
|
||||
|
||||
May mutate os.environ directly (use `not os.getenv(...)` guards to
|
||||
preserve env > YAML precedence) and/or return a dict to merge into
|
||||
PlatformConfig.extra. Return None or {} for no extras.
|
||||
"""
|
||||
if "require_mention" in platform_cfg and not os.getenv("MY_PLATFORM_REQUIRE_MENTION"):
|
||||
os.environ["MY_PLATFORM_REQUIRE_MENTION"] = str(platform_cfg["require_mention"]).lower()
|
||||
allowed = platform_cfg.get("allowed_channels")
|
||||
if allowed is not None and not os.getenv("MY_PLATFORM_ALLOWED_CHANNELS"):
|
||||
if isinstance(allowed, list):
|
||||
allowed = ",".join(str(v) for v in allowed)
|
||||
os.environ["MY_PLATFORM_ALLOWED_CHANNELS"] = str(allowed)
|
||||
return None # nothing extra to merge into PlatformConfig.extra
|
||||
|
||||
def register(ctx):
|
||||
ctx.register_platform(
|
||||
name="my_platform",
|
||||
...,
|
||||
apply_yaml_config_fn=_apply_yaml_config,
|
||||
)
|
||||
```
|
||||
|
||||
The hook is invoked during `load_gateway_config()` after the generic shared-key loop (which handles common keys like `unauthorized_dm_behavior`, `notice_delivery`, `reply_prefix`, `require_mention`, etc.) and before `_apply_env_overrides()`, so your plugin only needs to bridge **platform-specific** keys.
|
||||
|
||||
Exceptions raised by the hook are swallowed and logged at debug level — a misbehaving plugin never aborts gateway config load.
|
||||
|
||||
|
||||
## Cron Delivery
|
||||
|
||||
To let `deliver=my_platform` cron jobs route to a configured home channel, set `cron_deliver_env_var` to the env var name that holds the default chat/room/channel ID:
|
||||
|
||||
```python
|
||||
ctx.register_platform(
|
||||
name="my_platform",
|
||||
...
|
||||
cron_deliver_env_var="MY_PLATFORM_HOME_CHANNEL",
|
||||
)
|
||||
```
|
||||
|
||||
The scheduler reads this env var when resolving the home target for `deliver=my_platform` jobs, and also treats the platform as a valid cron target in `_KNOWN_DELIVERY_PLATFORMS`-style checks. If your `env_enablement_fn` seeds a `home_channel` dict (see above), that takes precedence — `cron_deliver_env_var` is the fallback for cron jobs that run before env seeding.
|
||||
|
||||
### Out-of-process cron delivery
|
||||
|
||||
`cron_deliver_env_var` makes your platform a recognized `deliver=` target. To make the actual send succeed when the cron job runs in a separate process from the gateway (i.e., `hermes cron run` separate from `hermes gateway`), register a `standalone_sender_fn`:
|
||||
|
||||
```python
|
||||
async def _standalone_send(
|
||||
pconfig,
|
||||
chat_id,
|
||||
message,
|
||||
*,
|
||||
thread_id=None,
|
||||
media_files=None,
|
||||
force_document=False,
|
||||
):
|
||||
"""Open an ephemeral connection / acquire a fresh token, send, and close."""
|
||||
# ... open connection, send message, return result ...
|
||||
return {"success": True, "message_id": "..."}
|
||||
# or {"error": "..."}
|
||||
|
||||
ctx.register_platform(
|
||||
name="my_platform",
|
||||
...
|
||||
cron_deliver_env_var="MY_PLATFORM_HOME_CHANNEL",
|
||||
standalone_sender_fn=_standalone_send,
|
||||
)
|
||||
```
|
||||
|
||||
Why this hook is necessary: built-in platforms (Telegram, Discord, Slack, etc.) ship direct REST helpers in `tools/send_message_tool.py` so cron can deliver without holding the gateway in the same process. Plugin platforms historically depended on `_gateway_runner_ref()`, which returns `None` outside the gateway process, so without `standalone_sender_fn` the cron-side send fails with `No live adapter for platform '<name>'`.
|
||||
|
||||
The function receives the same `pconfig` and `chat_id` that the live adapter would, plus optional `thread_id`, `media_files`, and `force_document` keyword arguments. Returning `{"success": True, "message_id": ...}` is treated as a successful delivery; returning `{"error": "..."}` surfaces the message in cron's `delivery_errors`. Exceptions raised inside the function are caught by the dispatcher and reported as `Plugin standalone send failed: <reason>`. Reference implementations live in `plugins/platforms/{irc,teams,google_chat}/adapter.py`.
|
||||
|
||||
## Surfacing Env Vars in `hermes config`
|
||||
|
||||
`hermes_cli/config.py` scans `plugins/platforms/*/plugin.yaml` at import time and auto-populates `OPTIONAL_ENV_VARS` from `requires_env` and (optional) `optional_env` blocks. Use the rich-dict form to contribute proper descriptions, prompts, password flags, and URLs — the CLI setup UI picks them up for free.
|
||||
|
||||
```yaml
|
||||
# plugins/platforms/my_platform/plugin.yaml
|
||||
name: my_platform-platform
|
||||
label: My Platform
|
||||
kind: platform
|
||||
version: 1.0.0
|
||||
description: >
|
||||
My Platform gateway adapter for Hermes Agent.
|
||||
author: Your Name
|
||||
requires_env:
|
||||
- name: MY_PLATFORM_TOKEN
|
||||
description: "Bot API token from the My Platform console"
|
||||
prompt: "My Platform bot token"
|
||||
url: "https://my-platform.example.com/bots"
|
||||
password: true
|
||||
- name: MY_PLATFORM_CHANNEL
|
||||
description: "Channel to join (e.g. #hermes)"
|
||||
prompt: "Channel"
|
||||
password: false
|
||||
optional_env:
|
||||
- name: MY_PLATFORM_HOME_CHANNEL
|
||||
description: "Default channel for cron delivery (defaults to MY_PLATFORM_CHANNEL)"
|
||||
prompt: "Home channel (or empty)"
|
||||
password: false
|
||||
- name: MY_PLATFORM_ALLOWED_USERS
|
||||
description: "Comma-separated user IDs allowed to talk to the bot"
|
||||
prompt: "Allowed users (comma-separated)"
|
||||
password: false
|
||||
```
|
||||
|
||||
**Supported dict keys:** `name` (required), `description`, `prompt`, `url`, `password` (bool; auto-detected from `*_TOKEN` / `*_SECRET` / `*_KEY` / `*_PASSWORD` / `*_JSON` suffix when omitted), `category` (defaults to `"messaging"`).
|
||||
|
||||
Bare-string entries (`- MY_PLATFORM_TOKEN`) still work — they get a generic description auto-derived from the plugin's `label`. If a hardcoded entry for the same var already exists in `OPTIONAL_ENV_VARS`, it wins (back-compat); the plugin.yaml form acts as the fallback.
|
||||
|
||||
## Platform-Specific Slow-LLM UX
|
||||
|
||||
Some platforms have constraints that change how a slow LLM response should be presented:
|
||||
|
||||
- **LINE** issues a single-use *reply token* that expires roughly 60 seconds after the inbound event. Replying with that token is free; falling back to the metered Push API is not. If the LLM hasn't finished by the deadline, the choice is "burn paid Push quota" or "do something cleverer with the reply token before it expires."
|
||||
- **WhatsApp** marks a session inactive after 24h, after which only template messages are accepted.
|
||||
- **SMS** has no concept of typing indicators or progressive updates — long responses just look like the bot is offline.
|
||||
|
||||
These are real constraints the base `BasePlatformAdapter` can't anticipate. The plugin surface intentionally leaves the room for an adapter to layer platform-specific UX on top of the base typing loop without expanding the kwarg list.
|
||||
|
||||
### Pattern: subclass `_keep_typing` to layer mid-flight UX
|
||||
|
||||
`BasePlatformAdapter._keep_typing` is the typing-indicator heartbeat — it runs as a background task while the LLM is generating, and is cancelled when the response is delivered. To layer a platform-specific behavior at a threshold (e.g. send a "still thinking" bubble at 45s), override `_keep_typing` in your adapter, schedule your own task alongside `super()._keep_typing()`, and tear it down in `finally`:
|
||||
|
||||
```python
|
||||
class LineAdapter(BasePlatformAdapter):
|
||||
async def _keep_typing(self, chat_id: str, *args, **kwargs) -> None:
|
||||
if self.slow_response_threshold <= 0:
|
||||
await super()._keep_typing(chat_id, *args, **kwargs)
|
||||
return
|
||||
|
||||
async def _fire_at_threshold() -> None:
|
||||
try:
|
||||
await asyncio.sleep(self.slow_response_threshold)
|
||||
except asyncio.CancelledError:
|
||||
raise
|
||||
# Platform-specific work here — for LINE, send a Template
|
||||
# Buttons "Get answer" bubble using the cached reply token
|
||||
# so the user can fetch the cached response later via a
|
||||
# fresh (free) reply token from the postback callback.
|
||||
await self._send_slow_response_button(chat_id)
|
||||
|
||||
side_task = asyncio.create_task(_fire_at_threshold())
|
||||
try:
|
||||
await super()._keep_typing(chat_id, *args, **kwargs)
|
||||
finally:
|
||||
if not side_task.done():
|
||||
side_task.cancel()
|
||||
try:
|
||||
await side_task
|
||||
except (asyncio.CancelledError, Exception):
|
||||
pass
|
||||
```
|
||||
|
||||
Key points:
|
||||
|
||||
- **Always `await super()._keep_typing(...)`.** The typing heartbeat is independently useful — don't replace it, layer on top of it.
|
||||
- **Tear down the side task in `finally`.** When the LLM finishes (or `/stop` cancels the run), the gateway cancels the typing task. Your side task must observe that cancellation too, otherwise it lingers and may fire after the response was already delivered.
|
||||
- **Pair with `interrupt_session_activity`** to resolve any orphan UX state when the user issues `/stop`. For LINE, this means transitioning the postback cache entry from `PENDING` to `ERROR` so the persistent "Get answer" button delivers a "Run was interrupted" message instead of looping.
|
||||
|
||||
### Pattern: subclass `send` to route through a cache instead of sending immediately
|
||||
|
||||
If your slow-response UX caches the response for later retrieval (LINE's postback flow), your `send` override needs to recognize three modes:
|
||||
|
||||
1. **Pending postback active for this chat** → cache the response under the request_id, don't send anything visible.
|
||||
2. **System busy-ack** (`⚡ Interrupting`, `⏳ Queued`, `⏩ Steered`) → bypass the cache and send visibly so the user sees the gateway's response to their input.
|
||||
3. **Normal response** → send via reply-token-or-push as usual.
|
||||
|
||||
```python
|
||||
async def send(self, chat_id: str, content: str, **kw) -> SendResult:
|
||||
if _is_system_bypass(content):
|
||||
return await self._send_text_chunks(chat_id, content, force_push=False)
|
||||
pending_rid = self._pending_buttons.get(chat_id)
|
||||
if pending_rid:
|
||||
self._cache.set_ready(pending_rid, content)
|
||||
return SendResult(success=True, message_id=pending_rid)
|
||||
return await self._send_text_chunks(chat_id, content, force_push=False)
|
||||
```
|
||||
|
||||
`_SYSTEM_BYPASS_PREFIXES` are the gateway's own busy-acknowledgment prefixes (`⚡`, `⏳`, `⏩`, `💾`). Always let those through visibly, regardless of cached UX state.
|
||||
|
||||
### When this pattern is appropriate
|
||||
|
||||
Use the typing-loop override approach when:
|
||||
|
||||
- The platform's outbound API has a hard time-window constraint (single-use reply token, expiring sticky session, etc.) AND
|
||||
- A *visible mid-flight bubble* is acceptable UX on that platform.
|
||||
|
||||
Use the simpler `slow_response_threshold = 0` always-Push path when:
|
||||
|
||||
- The platform doesn't have a meaningful free vs. paid distinction, OR
|
||||
- The user community prefers "loading… loading… DONE" silence-then-response over an interactive intermediate bubble.
|
||||
|
||||
LINE supports both: the threshold defaults to 45s for free postback fetch, and `LINE_SLOW_RESPONSE_THRESHOLD=0` reverts to "always Push fallback."
|
||||
|
||||
### Reference Implementation
|
||||
|
||||
See `plugins/platforms/line/adapter.py` for the full LINE postback implementation — a `RequestCache` state machine (`PENDING → READY → DELIVERED`, plus `ERROR` for `/stop`), a `_keep_typing` override that fires the Template Buttons bubble at threshold, a `send` override that routes through the cache, and an `interrupt_session_activity` override that resolves orphan PENDING entries.
|
||||
|
||||
### Reference Implementations (Plugin Path)
|
||||
|
||||
See `plugins/platforms/irc/` in the repo for a complete working example — a full async IRC adapter with zero external dependencies. `plugins/platforms/teams/` covers Bot Framework / Adaptive Cards, `plugins/platforms/google_chat/` covers OAuth-based REST APIs, and `plugins/platforms/line/` covers webhook-driven Messaging APIs with platform-specific slow-LLM UX.
|
||||
|
||||
---
|
||||
|
||||
## Step-by-Step Checklist (Built-in Path)
|
||||
|
||||
:::note
|
||||
This checklist is for adding a platform directly to the Hermes core codebase — typically done by core contributors for officially supported platforms. Community/third-party platforms should use the [Plugin Path](#plugin-path-recommended) above.
|
||||
:::
|
||||
|
||||
### 1. Platform Enum
|
||||
|
||||
Add your platform to the `Platform` enum in `gateway/config.py`:
|
||||
|
||||
```python
|
||||
class Platform(str, Enum):
|
||||
# ... existing platforms ...
|
||||
NEWPLAT = "newplat"
|
||||
```
|
||||
|
||||
### 2. Adapter File
|
||||
|
||||
Create `plugins/platforms/newplat/adapter.py`:
|
||||
|
||||
```python
|
||||
from gateway.config import Platform, PlatformConfig
|
||||
from gateway.platforms.base import (
|
||||
BasePlatformAdapter, MessageEvent, MessageType, SendResult,
|
||||
)
|
||||
|
||||
def check_newplat_requirements() -> bool:
|
||||
"""Return True if dependencies are available."""
|
||||
return SOME_SDK_AVAILABLE
|
||||
|
||||
class NewPlatAdapter(BasePlatformAdapter):
|
||||
def __init__(self, config: PlatformConfig):
|
||||
super().__init__(config, Platform.NEWPLAT)
|
||||
# Read config from config.extra dict
|
||||
extra = config.extra or {}
|
||||
self._api_key = extra.get("api_key") or os.getenv("NEWPLAT_API_KEY", "")
|
||||
|
||||
async def connect(self) -> bool:
|
||||
# Set up connection, start polling/webhook
|
||||
self._mark_connected()
|
||||
return True
|
||||
|
||||
async def disconnect(self) -> None:
|
||||
self._running = False
|
||||
self._mark_disconnected()
|
||||
|
||||
async def send(self, chat_id, content, reply_to=None, metadata=None):
|
||||
# Send message via platform API
|
||||
return SendResult(success=True, message_id="...")
|
||||
|
||||
async def get_chat_info(self, chat_id):
|
||||
return {"name": chat_id, "type": "dm"}
|
||||
```
|
||||
|
||||
For inbound messages, build a `MessageEvent` and call `self.handle_message(event)`:
|
||||
|
||||
```python
|
||||
source = self.build_source(
|
||||
chat_id=chat_id,
|
||||
chat_name=name,
|
||||
chat_type="dm", # or "group"
|
||||
user_id=user_id,
|
||||
user_name=user_name,
|
||||
)
|
||||
event = MessageEvent(
|
||||
text=content,
|
||||
message_type=MessageType.TEXT,
|
||||
source=source,
|
||||
message_id=msg_id,
|
||||
)
|
||||
await self.handle_message(event)
|
||||
```
|
||||
|
||||
### 3. Gateway Config (`gateway/config.py`)
|
||||
|
||||
Three touchpoints:
|
||||
|
||||
1. **`get_connected_platforms()`** — Add a check for your platform's required credentials
|
||||
2. **`load_gateway_config()`** — Add token env map entry: `Platform.NEWPLAT: "NEWPLAT_TOKEN"`
|
||||
3. **`_apply_env_overrides()`** — Map all `NEWPLAT_*` env vars to config
|
||||
|
||||
### 4. Gateway Runner (`gateway/run.py`)
|
||||
|
||||
Five touchpoints:
|
||||
|
||||
1. **`_create_adapter()`** — Add an `elif platform == Platform.NEWPLAT:` branch
|
||||
2. **`_is_user_authorized()` allowed_users map** — `Platform.NEWPLAT: "NEWPLAT_ALLOWED_USERS"`
|
||||
3. **`_is_user_authorized()` allow_all map** — `Platform.NEWPLAT: "NEWPLAT_ALLOW_ALL_USERS"`
|
||||
4. **Early env check `_any_allowlist` tuple** — Add `"NEWPLAT_ALLOWED_USERS"`
|
||||
5. **Early env check `_allow_all` tuple** — Add `"NEWPLAT_ALLOW_ALL_USERS"`
|
||||
6. **`_UPDATE_ALLOWED_PLATFORMS` frozenset** — Add `Platform.NEWPLAT`
|
||||
|
||||
### 5. Cross-Platform Delivery
|
||||
|
||||
1. **`gateway/platforms/webhook.py`** — Add `"newplat"` to the delivery type tuple
|
||||
2. **`cron/scheduler.py`** — Add to `_KNOWN_DELIVERY_PLATFORMS` frozenset and `_deliver_result()` platform map
|
||||
|
||||
### 6. CLI Integration
|
||||
|
||||
1. **`hermes_cli/config.py`** — Add all `NEWPLAT_*` vars to `_EXTRA_ENV_KEYS`
|
||||
2. **`hermes_cli/gateway.py`** — Add entry to `_PLATFORMS` list with key, label, emoji, token_var, setup_instructions, and vars
|
||||
3. **`hermes_cli/platforms.py`** — Add `PlatformInfo` entry with label and default_toolset (used by `skills_config` and `tools_config` TUIs)
|
||||
4. **`hermes_cli/setup.py`** — Add `_setup_newplat()` function (can delegate to `gateway.py`) and add tuple to the messaging platforms list
|
||||
5. **`hermes_cli/status.py`** — Add platform detection entry: `"NewPlat": ("NEWPLAT_TOKEN", "NEWPLAT_HOME_CHANNEL")`
|
||||
6. **`hermes_cli/dump.py`** — Add `"newplat": "NEWPLAT_TOKEN"` to platform detection dict
|
||||
|
||||
### 7. Tools
|
||||
|
||||
1. **`tools/send_message_tool.py`** — Add `"newplat": Platform.NEWPLAT` to platform map
|
||||
2. **`tools/cronjob_tools.py`** — Add `newplat` to the delivery target description string
|
||||
|
||||
### 8. Toolsets
|
||||
|
||||
1. **`toolsets.py`** — Add `"hermes-newplat"` toolset definition with `_HERMES_CORE_TOOLS`
|
||||
2. **`toolsets.py`** — Add `"hermes-newplat"` to the `"hermes-gateway"` includes list
|
||||
|
||||
### 9. Optional: Platform Hints
|
||||
|
||||
**`agent/prompt_builder.py`** — If your platform has specific rendering limitations (no markdown, message length limits, etc.), add an entry to the `_PLATFORM_HINTS` dict. This injects platform-specific guidance into the system prompt:
|
||||
|
||||
```python
|
||||
_PLATFORM_HINTS = {
|
||||
# ...
|
||||
"newplat": (
|
||||
"You are chatting via NewPlat. It supports markdown formatting "
|
||||
"but has a 4000-character message limit."
|
||||
),
|
||||
}
|
||||
```
|
||||
|
||||
Not all platforms need hints — only add one if the agent's behavior should differ.
|
||||
|
||||
### 10. Tests
|
||||
|
||||
Create `tests/gateway/test_newplat.py` covering:
|
||||
|
||||
- Adapter construction from config
|
||||
- Message event building
|
||||
- Send method (mock the external API)
|
||||
- Platform-specific features (encryption, routing, etc.)
|
||||
|
||||
### 11. Documentation
|
||||
|
||||
| File | What to add |
|
||||
|------|-------------|
|
||||
| `website/docs/user-guide/messaging/newplat.md` | Full platform setup page |
|
||||
| `website/docs/user-guide/messaging/index.md` | Platform comparison table, architecture diagram, toolsets table, security section, next-steps link |
|
||||
| `website/docs/reference/environment-variables.md` | All NEWPLAT_* env vars |
|
||||
| `website/docs/reference/toolsets-reference.md` | hermes-newplat toolset |
|
||||
| `website/docs/integrations/index.md` | Platform link |
|
||||
| `website/sidebars.ts` | Sidebar entry for the docs page |
|
||||
| `website/docs/developer-guide/architecture.md` | Adapter count + listing |
|
||||
| `website/docs/developer-guide/gateway-internals.md` | Adapter file listing |
|
||||
|
||||
## Parity Audit
|
||||
|
||||
Before marking a new platform PR as complete, run a parity audit against an established platform:
|
||||
|
||||
```bash
|
||||
# Find every .py file mentioning the reference platform
|
||||
search_files "bluebubbles" output_mode="files_only" file_glob="*.py"
|
||||
|
||||
# Find every .py file mentioning the new platform
|
||||
search_files "newplat" output_mode="files_only" file_glob="*.py"
|
||||
|
||||
# Any file in the first set but not the second is a potential gap
|
||||
```
|
||||
|
||||
Repeat for `.md` and `.ts` files. Investigate each gap — is it a platform enumeration (needs updating) or a platform-specific reference (skip)?
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### Long-Poll Adapters
|
||||
|
||||
If your adapter uses long-polling (like Telegram or Weixin), use a polling loop task:
|
||||
|
||||
```python
|
||||
async def connect(self):
|
||||
self._poll_task = asyncio.create_task(self._poll_loop())
|
||||
self._mark_connected()
|
||||
|
||||
async def _poll_loop(self):
|
||||
while self._running:
|
||||
messages = await self._fetch_updates()
|
||||
for msg in messages:
|
||||
await self.handle_message(self._build_event(msg))
|
||||
```
|
||||
|
||||
### Callback/Webhook Adapters
|
||||
|
||||
If the platform pushes messages to your endpoint (like WeCom Callback), run an HTTP server:
|
||||
|
||||
```python
|
||||
async def connect(self):
|
||||
self._app = web.Application()
|
||||
self._app.router.add_post("/callback", self._handle_callback)
|
||||
# ... start aiohttp server
|
||||
self._mark_connected()
|
||||
|
||||
async def _handle_callback(self, request):
|
||||
event = self._build_event(await request.text())
|
||||
await self._message_queue.put(event)
|
||||
return web.Response(text="success") # Acknowledge immediately
|
||||
```
|
||||
|
||||
For platforms with tight response deadlines (e.g., WeCom's 5-second limit), always acknowledge immediately and deliver the agent's reply proactively via API later. Agent sessions run 3–30 minutes — inline replies within a callback response window are not feasible.
|
||||
|
||||
### Token Locks
|
||||
|
||||
If the adapter holds a persistent connection with a unique credential, add a scoped lock to prevent two profiles from using the same credential:
|
||||
|
||||
```python
|
||||
from gateway.status import acquire_scoped_lock, release_scoped_lock
|
||||
|
||||
async def connect(self):
|
||||
if not acquire_scoped_lock("newplat", self._token):
|
||||
logger.error("Token already in use by another profile")
|
||||
return False
|
||||
# ... connect
|
||||
|
||||
async def disconnect(self):
|
||||
release_scoped_lock("newplat", self._token)
|
||||
```
|
||||
|
||||
## Reference Implementations
|
||||
|
||||
| Adapter | Pattern | Complexity | Good reference for |
|
||||
|---------|---------|------------|-------------------|
|
||||
| `bluebubbles.py` | REST + webhook | Medium | Simple REST API integration |
|
||||
| `weixin.py` | Long-poll + CDN | High | Media handling, encryption |
|
||||
| `wecom_callback.py` | Callback/webhook | Medium | HTTP server, AES crypto, multi-app |
|
||||
| `plugins/platforms/irc/adapter.py` | Long-poll + IRC protocol | High | Full-featured plugin adapter with scoped token lock |
|
||||
@@ -0,0 +1,459 @@
|
||||
---
|
||||
sidebar_position: 5
|
||||
title: "Adding Providers"
|
||||
description: "How to add a new inference provider to Hermes Agent — auth, runtime resolution, CLI flows, adapters, tests, and docs"
|
||||
---
|
||||
|
||||
# Adding Providers
|
||||
|
||||
Hermes can already talk to any OpenAI-compatible endpoint through the custom provider path. Do not add a built-in provider unless you want first-class UX for that service:
|
||||
|
||||
- provider-specific auth or token refresh
|
||||
- a curated model catalog
|
||||
- setup / `hermes model` menu entries
|
||||
- provider aliases for `provider:model` syntax
|
||||
- a non-OpenAI API shape that needs an adapter
|
||||
|
||||
If the provider is just "another OpenAI-compatible base URL and API key", a named custom provider may be enough.
|
||||
|
||||
## The mental model
|
||||
|
||||
A built-in provider has to line up across a few layers:
|
||||
|
||||
1. `hermes_cli/auth.py` decides how credentials are found.
|
||||
2. `hermes_cli/runtime_provider.py` turns that into runtime data:
|
||||
- `provider`
|
||||
- `api_mode`
|
||||
- `base_url`
|
||||
- `api_key`
|
||||
- `source`
|
||||
3. `run_agent.py` uses `api_mode` to decide how requests are built and sent.
|
||||
4. `hermes_cli/models.py` and `hermes_cli/main.py` make the provider show up in the CLI. (`hermes_cli/setup.py` delegates to `main.py` automatically — no changes needed there.)
|
||||
5. `agent/auxiliary_client.py` and `agent/model_metadata.py` keep side tasks and token budgeting working.
|
||||
|
||||
The important abstraction is `api_mode`.
|
||||
|
||||
- Most providers use `chat_completions`.
|
||||
- Codex uses `codex_responses`.
|
||||
- Anthropic uses `anthropic_messages`.
|
||||
- A new non-OpenAI protocol usually means adding a new adapter and a new `api_mode` branch.
|
||||
|
||||
## Choose the implementation path first
|
||||
|
||||
### Path A — OpenAI-compatible provider
|
||||
|
||||
Use this when the provider accepts standard chat-completions style requests.
|
||||
|
||||
Typical work:
|
||||
|
||||
- add auth metadata
|
||||
- add model catalog / aliases
|
||||
- add runtime resolution
|
||||
- add CLI menu wiring
|
||||
- add aux-model defaults
|
||||
- add tests and user docs
|
||||
|
||||
You usually do not need a new adapter or a new `api_mode`.
|
||||
|
||||
### Path B — Native provider
|
||||
|
||||
Use this when the provider does not behave like OpenAI chat completions.
|
||||
|
||||
Examples in-tree today:
|
||||
|
||||
- `codex_responses`
|
||||
- `anthropic_messages`
|
||||
|
||||
This path includes everything from Path A plus:
|
||||
|
||||
- a provider adapter in `agent/`
|
||||
- `run_agent.py` branches for request building, dispatch, usage extraction, interrupt handling, and response normalization
|
||||
- adapter tests
|
||||
|
||||
## File checklist
|
||||
|
||||
### Required for every built-in provider
|
||||
|
||||
1. `hermes_cli/auth.py`
|
||||
2. `hermes_cli/models.py`
|
||||
3. `hermes_cli/runtime_provider.py`
|
||||
4. `hermes_cli/main.py`
|
||||
5. `agent/auxiliary_client.py`
|
||||
6. `agent/model_metadata.py`
|
||||
7. tests
|
||||
8. user-facing docs under `website/docs/`
|
||||
|
||||
:::tip
|
||||
`hermes_cli/setup.py` does **not** need changes. The setup wizard delegates provider/model selection to `select_provider_and_model()` in `main.py` — any provider added there is automatically available in `hermes setup`.
|
||||
:::
|
||||
|
||||
### Additional for native / non-OpenAI providers
|
||||
|
||||
10. `agent/<provider>_adapter.py`
|
||||
11. `run_agent.py`
|
||||
12. `pyproject.toml` if a provider SDK is required
|
||||
|
||||
## Fast path: Simple API-key providers
|
||||
|
||||
If your provider is just an OpenAI-compatible endpoint that authenticates with a single API key, you do not need to touch `auth.py`, `runtime_provider.py`, `main.py`, or any of the other files in the full checklist below.
|
||||
|
||||
All you need is:
|
||||
|
||||
1. A plugin directory under `plugins/model-providers/<your-provider>/` containing:
|
||||
- `__init__.py` — calls `register_provider(profile)` at module-level
|
||||
- `plugin.yaml` — manifest (name, kind: model-provider, version, description)
|
||||
2. That's it. Provider plugins auto-load the first time anything calls `get_provider_profile()` or `list_providers()` — bundled plugins (this repo) and user plugins at `$HERMES_HOME/plugins/model-providers/` both get picked up.
|
||||
|
||||
When you add a plugin and it calls `register_provider()`, the following wire up automatically:
|
||||
|
||||
1. `PROVIDER_REGISTRY` entry in `auth.py` (credential resolution, env-var lookup)
|
||||
2. `api_mode` set to `chat_completions`
|
||||
3. `base_url` sourced from the config or the declared env var
|
||||
4. `env_vars` checked in priority order for the API key
|
||||
5. `fallback_models` list registered for the provider
|
||||
6. `--provider` CLI flag accepts the provider id
|
||||
7. `hermes model` menu includes the provider
|
||||
8. `hermes setup` wizard delegates to `main.py` automatically
|
||||
9. `provider:model` alias syntax works
|
||||
10. Runtime resolver returns the correct `base_url` and `api_key`
|
||||
11. `--provider <name>` CLI flag accepts the provider id
|
||||
12. Fallback model activation can switch into the provider cleanly
|
||||
|
||||
User plugins at `$HERMES_HOME/plugins/model-providers/<name>/` override bundled plugins of the same name (last-writer-wins in `register_provider()`) — so third parties can monkey-patch or replace any built-in profile without editing the repo.
|
||||
|
||||
See `plugins/model-providers/nvidia/` or `plugins/model-providers/gmi/` as a template, and the full [Model Provider Plugin guide](/developer-guide/model-provider-plugin) for field reference, hook idioms, and end-to-end examples.
|
||||
|
||||
## Full path: OAuth and complex providers
|
||||
|
||||
Use the full checklist below when your provider needs any of the following:
|
||||
|
||||
- OAuth or token refresh (Nous Portal, Codex, Qwen Portal, Copilot)
|
||||
- A non-OpenAI API shape that requires a new adapter (Anthropic Messages, Codex Responses)
|
||||
- Custom endpoint detection or multi-region probing (z.ai, Kimi)
|
||||
- A curated static model catalog or live `/models` fetch
|
||||
- Provider-specific `hermes model` menu entries with bespoke auth flows
|
||||
|
||||
## Step 1: Pick one canonical provider id
|
||||
|
||||
Choose a single provider id and use it everywhere.
|
||||
|
||||
Examples from the repo:
|
||||
|
||||
- `openai-codex`
|
||||
- `kimi-coding`
|
||||
- `minimax-cn`
|
||||
|
||||
That same id should appear in:
|
||||
|
||||
- `PROVIDER_REGISTRY` in `hermes_cli/auth.py`
|
||||
- `_PROVIDER_LABELS` in `hermes_cli/models.py`
|
||||
- `_PROVIDER_ALIASES` in both `hermes_cli/auth.py` and `hermes_cli/models.py`
|
||||
- CLI `--provider` choices in `hermes_cli/main.py`
|
||||
- setup / model selection branches
|
||||
- auxiliary-model defaults
|
||||
- tests
|
||||
|
||||
If the id differs between those files, the provider will feel half-wired: auth may work while `/model`, setup, or runtime resolution silently misses it.
|
||||
|
||||
## Step 2: Add auth metadata in `hermes_cli/auth.py`
|
||||
|
||||
For API-key providers, add a `ProviderConfig` entry to `PROVIDER_REGISTRY` with:
|
||||
|
||||
- `id`
|
||||
- `name`
|
||||
- `auth_type="api_key"`
|
||||
- `inference_base_url`
|
||||
- `api_key_env_vars`
|
||||
- optional `base_url_env_var`
|
||||
|
||||
Also add aliases to `_PROVIDER_ALIASES`.
|
||||
|
||||
Use the existing providers as templates:
|
||||
|
||||
- simple API-key path: Z.AI, MiniMax
|
||||
- API-key path with endpoint detection: Kimi, Z.AI
|
||||
- native token resolution: Anthropic
|
||||
- OAuth / auth-store path: Nous, OpenAI Codex
|
||||
|
||||
Questions to answer here:
|
||||
|
||||
- What env vars should Hermes check, and in what priority order?
|
||||
- Does the provider need base-URL overrides?
|
||||
- Does it need endpoint probing or token refresh?
|
||||
- What should the auth error say when credentials are missing?
|
||||
|
||||
If the provider needs something more than "look up an API key", add a dedicated credential resolver instead of shoving logic into unrelated branches.
|
||||
|
||||
## Step 3: Add model catalog and aliases in `hermes_cli/models.py`
|
||||
|
||||
Update the provider catalog so the provider works in menus and in `provider:model` syntax.
|
||||
|
||||
Typical edits:
|
||||
|
||||
- `_PROVIDER_MODELS`
|
||||
- `_PROVIDER_LABELS`
|
||||
- `_PROVIDER_ALIASES`
|
||||
- provider display order inside `list_available_providers()`
|
||||
- `provider_model_ids()` if the provider supports a live `/models` fetch
|
||||
|
||||
If the provider exposes a live model list, prefer that first and keep `_PROVIDER_MODELS` as the static fallback.
|
||||
|
||||
This file is also what makes inputs like these work:
|
||||
|
||||
```text
|
||||
anthropic:claude-sonnet-4-6
|
||||
kimi:model-name
|
||||
```
|
||||
|
||||
If aliases are missing here, the provider may authenticate correctly but still fail in `/model` parsing.
|
||||
|
||||
## Step 4: Resolve runtime data in `hermes_cli/runtime_provider.py`
|
||||
|
||||
`resolve_runtime_provider()` is the shared path used by CLI, gateway, cron, ACP, and helper clients.
|
||||
|
||||
Add a branch that returns a dict with at least:
|
||||
|
||||
```python
|
||||
{
|
||||
"provider": "your-provider",
|
||||
"api_mode": "chat_completions", # or your native mode
|
||||
"base_url": "https://...",
|
||||
"api_key": "...",
|
||||
"source": "env|portal|auth-store|explicit",
|
||||
"requested_provider": requested_provider,
|
||||
}
|
||||
```
|
||||
|
||||
If the provider is OpenAI-compatible, `api_mode` should usually stay `chat_completions`.
|
||||
|
||||
Be careful with API-key precedence. Hermes already contains logic to avoid leaking an OpenRouter key to unrelated endpoints. A new provider should be equally explicit about which key goes to which base URL.
|
||||
|
||||
## Step 5: Wire the CLI in `hermes_cli/main.py`
|
||||
|
||||
A provider is not discoverable until it shows up in the interactive `hermes model` flow.
|
||||
|
||||
Update these in `hermes_cli/main.py`:
|
||||
|
||||
- `provider_labels` dict
|
||||
- `providers` list in `select_provider_and_model()`
|
||||
- provider dispatch (`if selected_provider == ...`)
|
||||
- `--provider` argument choices
|
||||
- login/logout choices if the provider supports those flows
|
||||
- a `_model_flow_<provider>()` function, or reuse `_model_flow_api_key_provider()` if it fits
|
||||
|
||||
:::tip
|
||||
`hermes_cli/setup.py` does not need changes — it calls `select_provider_and_model()` from `main.py`, so your new provider appears in both `hermes model` and `hermes setup` automatically.
|
||||
:::
|
||||
|
||||
## Step 6: Keep auxiliary calls working
|
||||
|
||||
Two files matter here:
|
||||
|
||||
### `agent/auxiliary_client.py`
|
||||
|
||||
Add a cheap / fast default aux model to `_API_KEY_PROVIDER_AUX_MODELS` if this is a direct API-key provider.
|
||||
|
||||
Auxiliary tasks include things like:
|
||||
|
||||
- vision summarization
|
||||
- web extraction summarization
|
||||
- context compression summaries
|
||||
- session-search summaries
|
||||
- memory flushes
|
||||
|
||||
If the provider has no sensible aux default, side tasks may fall back badly or use an expensive main model unexpectedly.
|
||||
|
||||
### `agent/model_metadata.py`
|
||||
|
||||
Add context lengths for the provider's models so token budgeting, compression thresholds, and limits stay sane.
|
||||
|
||||
## Step 7: If the provider is native, add an adapter and `run_agent.py` support
|
||||
|
||||
If the provider is not plain chat completions, isolate the provider-specific logic in `agent/<provider>_adapter.py`.
|
||||
|
||||
Keep `run_agent.py` focused on orchestration. It should call adapter helpers, not hand-build provider payloads inline all over the file.
|
||||
|
||||
A native provider usually needs work in these places:
|
||||
|
||||
### New adapter file
|
||||
|
||||
Typical responsibilities:
|
||||
|
||||
- build the SDK / HTTP client
|
||||
- resolve tokens
|
||||
- convert OpenAI-style conversation messages to the provider's request format
|
||||
- convert tool schemas if needed
|
||||
- normalize provider responses back into what `run_agent.py` expects
|
||||
- extract usage and finish-reason data
|
||||
|
||||
### `run_agent.py`
|
||||
|
||||
Search for `api_mode` and audit every switch point. At minimum, verify:
|
||||
|
||||
- `__init__` chooses the new `api_mode`
|
||||
- client construction works for the provider
|
||||
- `_build_api_kwargs()` knows how to format requests
|
||||
- `_interruptible_api_call()` dispatches to the right client call
|
||||
- interrupt / client rebuild paths work
|
||||
- response validation accepts the provider's shape
|
||||
- finish-reason extraction is correct
|
||||
- token-usage extraction is correct
|
||||
- fallback-model activation can switch into the new provider cleanly
|
||||
- summary-generation and memory-flush paths still work
|
||||
|
||||
Also search `run_agent.py` for `self.client.`. Any code path that assumes the standard OpenAI client exists can break when a native provider uses a different client object or `self.client = None`.
|
||||
|
||||
### Prompt caching and provider-specific request fields
|
||||
|
||||
Prompt caching and provider-specific knobs are easy to regress.
|
||||
|
||||
Examples already in-tree:
|
||||
|
||||
- Anthropic has a native prompt-caching path
|
||||
- OpenRouter gets provider-routing fields
|
||||
- not every provider should receive every request-side option
|
||||
|
||||
When you add a native provider, double-check that Hermes is only sending fields that provider actually understands.
|
||||
|
||||
## Step 8: Tests
|
||||
|
||||
At minimum, touch the tests that guard provider wiring.
|
||||
|
||||
Common places:
|
||||
|
||||
- `tests/hermes_cli/test_runtime_provider_resolution.py`
|
||||
- `tests/cli/test_cli_provider_resolution.py`
|
||||
- `tests/hermes_cli/test_model_switch_custom_providers.py` (and adjacent `tests/hermes_cli/test_model_switch_*.py`)
|
||||
- `tests/hermes_cli/test_setup_model_provider.py`
|
||||
- `tests/run_agent/test_provider_parity.py`
|
||||
- `tests/run_agent/test_run_agent.py`
|
||||
- `tests/test_<provider>_adapter.py` for a native provider
|
||||
|
||||
For docs-only examples, the exact file set may differ. The point is to cover:
|
||||
|
||||
- auth resolution
|
||||
- CLI menu / provider selection
|
||||
- runtime provider resolution
|
||||
- agent execution path
|
||||
- provider:model parsing
|
||||
- any adapter-specific message conversion
|
||||
|
||||
Run tests with xdist disabled:
|
||||
|
||||
```bash
|
||||
source venv/bin/activate
|
||||
python -m pytest tests/hermes_cli/test_runtime_provider_resolution.py tests/cli/test_cli_provider_resolution.py tests/hermes_cli/test_setup_model_provider.py tests/run_agent/test_provider_parity.py -n0 -q
|
||||
```
|
||||
|
||||
For deeper changes, run the full suite before pushing:
|
||||
|
||||
```bash
|
||||
source venv/bin/activate
|
||||
python -m pytest tests/ -n0 -q
|
||||
```
|
||||
|
||||
## Step 9: Live verification
|
||||
|
||||
After tests, run a real smoke test.
|
||||
|
||||
```bash
|
||||
source venv/bin/activate
|
||||
python -m hermes_cli.main chat -q "Say hello" --provider your-provider --model your-model
|
||||
```
|
||||
|
||||
Also test the interactive flows if you changed menus:
|
||||
|
||||
```bash
|
||||
source venv/bin/activate
|
||||
python -m hermes_cli.main model
|
||||
python -m hermes_cli.main setup
|
||||
```
|
||||
|
||||
For native providers, verify at least one tool call too, not just a plain text response.
|
||||
|
||||
## Step 10: Update user-facing docs
|
||||
|
||||
If the provider is meant to ship as a first-class option, update the user docs too:
|
||||
|
||||
- `website/docs/getting-started/quickstart.md`
|
||||
- `website/docs/user-guide/configuration.md`
|
||||
- `website/docs/reference/environment-variables.md`
|
||||
|
||||
A developer can wire the provider perfectly and still leave users unable to discover the required env vars or setup flow.
|
||||
|
||||
## OpenAI-compatible provider checklist
|
||||
|
||||
Use this if the provider is standard chat completions.
|
||||
|
||||
- [ ] `ProviderConfig` added in `hermes_cli/auth.py`
|
||||
- [ ] aliases added in `hermes_cli/auth.py` and `hermes_cli/models.py`
|
||||
- [ ] model catalog added in `hermes_cli/models.py`
|
||||
- [ ] runtime branch added in `hermes_cli/runtime_provider.py`
|
||||
- [ ] CLI wiring added in `hermes_cli/main.py` (setup.py inherits automatically)
|
||||
- [ ] aux model added in `agent/auxiliary_client.py`
|
||||
- [ ] context lengths added in `agent/model_metadata.py`
|
||||
- [ ] runtime / CLI tests updated
|
||||
- [ ] user docs updated
|
||||
|
||||
## Native provider checklist
|
||||
|
||||
Use this when the provider needs a new protocol path.
|
||||
|
||||
- [ ] everything in the OpenAI-compatible checklist
|
||||
- [ ] adapter added in `agent/<provider>_adapter.py`
|
||||
- [ ] new `api_mode` supported in `run_agent.py`
|
||||
- [ ] interrupt / rebuild path works
|
||||
- [ ] usage and finish-reason extraction works
|
||||
- [ ] fallback path works
|
||||
- [ ] adapter tests added
|
||||
- [ ] live smoke test passes
|
||||
|
||||
## Common pitfalls
|
||||
|
||||
### 1. Adding the provider to auth but not to model parsing
|
||||
|
||||
That makes credentials resolve correctly while `/model` and `provider:model` inputs fail.
|
||||
|
||||
### 2. Forgetting that `config["model"]` can be a string or a dict
|
||||
|
||||
A lot of provider-selection code has to normalize both forms.
|
||||
|
||||
### 3. Assuming a built-in provider is required
|
||||
|
||||
If the service is just OpenAI-compatible, a custom provider may already solve the user problem with less maintenance.
|
||||
|
||||
### 4. Forgetting auxiliary paths
|
||||
|
||||
The main chat path can work while summarization, memory flushes, or vision helpers fail because aux routing was never updated.
|
||||
|
||||
### 5. Native-provider branches hiding in `run_agent.py`
|
||||
|
||||
Search for `api_mode` and `self.client.`. Do not assume the obvious request path is the only one.
|
||||
|
||||
### 6. Sending OpenRouter-only knobs to other providers
|
||||
|
||||
Fields like provider routing belong only on the providers that support them.
|
||||
|
||||
### 7. Updating `hermes model` but not `hermes setup`
|
||||
|
||||
Both flows need to know about the provider.
|
||||
|
||||
## Good search targets while implementing
|
||||
|
||||
If you are hunting for all the places a provider touches, search these symbols:
|
||||
|
||||
- `PROVIDER_REGISTRY`
|
||||
- `_PROVIDER_ALIASES`
|
||||
- `_PROVIDER_MODELS`
|
||||
- `resolve_runtime_provider`
|
||||
- `_model_flow_`
|
||||
- `select_provider_and_model`
|
||||
- `api_mode`
|
||||
- `_API_KEY_PROVIDER_AUX_MODELS`
|
||||
- `self.client.`
|
||||
|
||||
## Related docs
|
||||
|
||||
- [Provider Runtime Resolution](./provider-runtime.md)
|
||||
- [Architecture](./architecture.md)
|
||||
- [Contributing](./contributing.md)
|
||||
@@ -0,0 +1,211 @@
|
||||
---
|
||||
sidebar_position: 2
|
||||
title: "Adding Tools"
|
||||
description: "How to add a new tool to Hermes Agent — schemas, handlers, registration, and toolsets"
|
||||
---
|
||||
|
||||
# Adding Tools
|
||||
|
||||
Before writing a tool, ask yourself: **should this be a [skill](creating-skills.md) instead?**
|
||||
|
||||
:::warning Built-in Core Tools Only
|
||||
This page is for adding a **built-in Hermes tool** to the repository itself.
|
||||
If you want a personal, project-local, or otherwise custom tool without
|
||||
modifying Hermes core, use the plugin route instead:
|
||||
|
||||
- [Plugins](/user-guide/features/plugins)
|
||||
- [Build a Hermes Plugin](/developer-guide/plugins)
|
||||
|
||||
Default to plugins for most custom tool creation. Only follow this page when
|
||||
you explicitly want to ship a new built-in tool in `tools/` and `toolsets.py`.
|
||||
:::
|
||||
|
||||
Make it a **Skill** when the capability can be expressed as instructions + shell commands + existing tools (arXiv search, git workflows, Docker management, PDF processing).
|
||||
|
||||
Make it a **Tool** when it requires end-to-end integration with API keys, custom processing logic, binary data handling, or streaming (browser automation, TTS, vision analysis).
|
||||
|
||||
## Overview
|
||||
|
||||
Adding a tool touches **2 files**:
|
||||
|
||||
1. **`tools/your_tool.py`** — handler, schema, check function, `registry.register()` call
|
||||
2. **`toolsets.py`** — add tool name to `_HERMES_CORE_TOOLS` (or a specific toolset)
|
||||
|
||||
Any `tools/*.py` file with a top-level `registry.register()` call is auto-discovered at startup — no manual import list required.
|
||||
|
||||
## Step 1: Create the Built-in Tool File
|
||||
|
||||
Every tool file follows the same structure:
|
||||
|
||||
```python
|
||||
# tools/weather_tool.py
|
||||
"""Weather Tool -- look up current weather for a location."""
|
||||
|
||||
import json
|
||||
import os
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# --- Availability check ---
|
||||
|
||||
def check_weather_requirements() -> bool:
|
||||
"""Return True if the tool's dependencies are available."""
|
||||
return bool(os.getenv("WEATHER_API_KEY"))
|
||||
|
||||
|
||||
# --- Handler ---
|
||||
|
||||
def weather_tool(location: str, units: str = "metric") -> str:
|
||||
"""Fetch weather for a location. Returns JSON string."""
|
||||
api_key = os.getenv("WEATHER_API_KEY")
|
||||
if not api_key:
|
||||
return json.dumps({"error": "WEATHER_API_KEY not configured"})
|
||||
try:
|
||||
# ... call weather API ...
|
||||
return json.dumps({"location": location, "temp": 22, "units": units})
|
||||
except Exception as e:
|
||||
return json.dumps({"error": str(e)})
|
||||
|
||||
|
||||
# --- Schema ---
|
||||
|
||||
WEATHER_SCHEMA = {
|
||||
"name": "weather",
|
||||
"description": "Get current weather for a location.",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"location": {
|
||||
"type": "string",
|
||||
"description": "City name or coordinates (e.g. 'London' or '51.5,-0.1')"
|
||||
},
|
||||
"units": {
|
||||
"type": "string",
|
||||
"enum": ["metric", "imperial"],
|
||||
"description": "Temperature units (default: metric)",
|
||||
"default": "metric"
|
||||
}
|
||||
},
|
||||
"required": ["location"]
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
# --- Registration ---
|
||||
|
||||
from tools.registry import registry
|
||||
|
||||
registry.register(
|
||||
name="weather",
|
||||
toolset="weather",
|
||||
schema=WEATHER_SCHEMA,
|
||||
handler=lambda args, **kw: weather_tool(
|
||||
location=args.get("location", ""),
|
||||
units=args.get("units", "metric")),
|
||||
check_fn=check_weather_requirements,
|
||||
requires_env=["WEATHER_API_KEY"],
|
||||
)
|
||||
```
|
||||
|
||||
### Key Rules
|
||||
|
||||
:::danger Important
|
||||
- Handlers **MUST** return a JSON string (via `json.dumps()`), never raw dicts
|
||||
- Errors **MUST** be returned as `{"error": "message"}`, never raised as exceptions
|
||||
- The `check_fn` is called when building tool definitions — if it returns `False`, the tool is silently excluded
|
||||
- The `handler` receives `(args: dict, **kwargs)` where `args` is the LLM's tool call arguments
|
||||
:::
|
||||
|
||||
## Step 2: Add the Built-in Tool to a Toolset
|
||||
|
||||
In `toolsets.py`, add the tool name:
|
||||
|
||||
```python
|
||||
# If it should be available on all platforms (CLI + messaging):
|
||||
_HERMES_CORE_TOOLS = [
|
||||
...
|
||||
"weather", # <-- add here
|
||||
]
|
||||
|
||||
# Or create a new standalone toolset:
|
||||
"weather": {
|
||||
"description": "Weather lookup tools",
|
||||
"tools": ["weather"],
|
||||
"includes": []
|
||||
},
|
||||
```
|
||||
|
||||
## ~~Step 3: Add Discovery Import~~ (No longer needed)
|
||||
|
||||
Tool modules with a top-level `registry.register()` call are auto-discovered by `discover_builtin_tools()` in `tools/registry.py`. No manual import list to maintain — just create your file in `tools/` and it's picked up at startup.
|
||||
|
||||
## Async Handlers
|
||||
|
||||
If your handler needs async code, mark it with `is_async=True`:
|
||||
|
||||
```python
|
||||
async def weather_tool_async(location: str) -> str:
|
||||
async with aiohttp.ClientSession() as session:
|
||||
...
|
||||
return json.dumps(result)
|
||||
|
||||
registry.register(
|
||||
name="weather",
|
||||
toolset="weather",
|
||||
schema=WEATHER_SCHEMA,
|
||||
handler=lambda args, **kw: weather_tool_async(args.get("location", "")),
|
||||
check_fn=check_weather_requirements,
|
||||
is_async=True, # registry calls _run_async() automatically
|
||||
)
|
||||
```
|
||||
|
||||
The registry handles async bridging transparently — you never call `asyncio.run()` yourself.
|
||||
|
||||
## Handlers That Need task_id
|
||||
|
||||
Tools that manage per-session state receive `task_id` via `**kwargs`:
|
||||
|
||||
```python
|
||||
def _handle_weather(args, **kw):
|
||||
task_id = kw.get("task_id")
|
||||
return weather_tool(args.get("location", ""), task_id=task_id)
|
||||
|
||||
registry.register(
|
||||
name="weather",
|
||||
...
|
||||
handler=_handle_weather,
|
||||
)
|
||||
```
|
||||
|
||||
## Agent-Loop Intercepted Tools
|
||||
|
||||
Some tools (`todo`, `memory`, `session_search`, `delegate_task`) need access to per-session agent state. These are intercepted by `run_agent.py` before reaching the registry. The registry still holds their schemas, but `dispatch()` returns a fallback error if the intercept is bypassed.
|
||||
|
||||
## Optional: Setup Wizard Integration
|
||||
|
||||
If your tool requires an API key, add it to `hermes_cli/config.py`:
|
||||
|
||||
```python
|
||||
OPTIONAL_ENV_VARS = {
|
||||
...
|
||||
"WEATHER_API_KEY": {
|
||||
"description": "Weather API key for weather lookup",
|
||||
"prompt": "Weather API key",
|
||||
"url": "https://weatherapi.com/",
|
||||
"tools": ["weather"],
|
||||
"password": True,
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] Tool file created with handler, schema, check function, and registration
|
||||
- [ ] Added to appropriate toolset in `toolsets.py`
|
||||
- [ ] Confirmed this really should be a built-in/core tool and not a plugin
|
||||
- [ ] Handler returns JSON strings, errors returned as `{"error": "..."}`
|
||||
- [ ] Optional: API key added to `OPTIONAL_ENV_VARS` in `hermes_cli/config.py`
|
||||
- [ ] Optional: Added to `toolset_distributions.py` for batch processing
|
||||
- [ ] Tested with `hermes chat -q "Use the weather tool for London"`
|
||||
@@ -0,0 +1,239 @@
|
||||
---
|
||||
sidebar_position: 3
|
||||
title: "Agent Loop Internals"
|
||||
description: "Detailed walkthrough of AIAgent execution, API modes, tools, callbacks, and fallback behavior"
|
||||
---
|
||||
|
||||
# Agent Loop Internals
|
||||
|
||||
The core orchestration engine is `run_agent.py`'s `AIAgent` class — a large file that handles everything from prompt assembly to tool dispatch to provider failover.
|
||||
|
||||
## Core Responsibilities
|
||||
|
||||
`AIAgent` is responsible for:
|
||||
|
||||
- Assembling the effective system prompt and tool schemas via `prompt_builder.py`
|
||||
- Selecting the correct provider/API mode (chat_completions, codex_responses, anthropic_messages)
|
||||
- Making interruptible model calls with cancellation support
|
||||
- Executing tool calls (sequentially or concurrently via thread pool)
|
||||
- Maintaining conversation history in OpenAI message format
|
||||
- Handling compression, retries, and fallback model switching
|
||||
- Tracking iteration budgets across parent and child agents
|
||||
- Flushing persistent memory before context is lost
|
||||
|
||||
## Two Entry Points
|
||||
|
||||
```python
|
||||
# Simple interface — returns final response string
|
||||
response = agent.chat("Fix the bug in main.py")
|
||||
|
||||
# Full interface — returns dict with messages, metadata, usage stats
|
||||
result = agent.run_conversation(
|
||||
user_message="Fix the bug in main.py",
|
||||
system_message=None, # auto-built if omitted
|
||||
conversation_history=None, # auto-loaded from session if omitted
|
||||
task_id="task_abc123"
|
||||
)
|
||||
```
|
||||
|
||||
`chat()` is a thin wrapper around `run_conversation()` that extracts the `final_response` field from the result dict.
|
||||
|
||||
## API Modes
|
||||
|
||||
Hermes supports three API execution modes, resolved from provider selection, explicit args, and base URL heuristics:
|
||||
|
||||
| API mode | Used for | Client type |
|
||||
|----------|----------|-------------|
|
||||
| `chat_completions` | OpenAI-compatible endpoints (OpenRouter, custom, most providers) | `openai.OpenAI` |
|
||||
| `codex_responses` | OpenAI Codex / Responses API | `openai.OpenAI` with Responses format |
|
||||
| `anthropic_messages` | Native Anthropic Messages API | `anthropic.Anthropic` via adapter |
|
||||
|
||||
The mode determines how messages are formatted, how tool calls are structured, how responses are parsed, and how caching/streaming works. All three converge on the same internal message format (OpenAI-style `role`/`content`/`tool_calls` dicts) before and after API calls.
|
||||
|
||||
**Mode resolution order:**
|
||||
1. Explicit `api_mode` constructor arg (highest priority)
|
||||
2. Provider-specific detection (e.g., `anthropic` provider → `anthropic_messages`)
|
||||
3. Base URL heuristics (e.g., `api.anthropic.com` → `anthropic_messages`)
|
||||
4. Default: `chat_completions`
|
||||
|
||||
## Turn Lifecycle
|
||||
|
||||
Each iteration of the agent loop follows this sequence:
|
||||
|
||||
```text
|
||||
run_conversation()
|
||||
1. Generate task_id if not provided
|
||||
2. Append user message to conversation history
|
||||
3. Build or reuse cached system prompt (prompt_builder.py)
|
||||
4. Check if preflight compression is needed (>50% context)
|
||||
5. Build API messages from conversation history
|
||||
- chat_completions: OpenAI format as-is
|
||||
- codex_responses: convert to Responses API input items
|
||||
- anthropic_messages: convert via anthropic_adapter.py
|
||||
6. Inject ephemeral prompt layers (budget warnings, context pressure)
|
||||
7. Apply prompt caching markers if on Anthropic
|
||||
8. Make interruptible API call (_interruptible_api_call)
|
||||
9. Parse response:
|
||||
- If tool_calls: execute them, append results, loop back to step 5
|
||||
- If text response: persist session, flush memory if needed, return
|
||||
```
|
||||
|
||||
### Message Format
|
||||
|
||||
All messages use OpenAI-compatible format internally:
|
||||
|
||||
```python
|
||||
{"role": "system", "content": "..."}
|
||||
{"role": "user", "content": "..."}
|
||||
{"role": "assistant", "content": "...", "tool_calls": [...]}
|
||||
{"role": "tool", "tool_call_id": "...", "content": "..."}
|
||||
```
|
||||
|
||||
Reasoning content (from models that support extended thinking) is stored in `assistant_msg["reasoning"]` and optionally displayed via the `reasoning_callback`.
|
||||
|
||||
### Message Alternation Rules
|
||||
|
||||
The agent loop enforces strict message role alternation:
|
||||
|
||||
- After the system message: `User → Assistant → User → Assistant → ...`
|
||||
- During tool calling: `Assistant (with tool_calls) → Tool → Tool → ... → Assistant`
|
||||
- **Never** two assistant messages in a row
|
||||
- **Never** two user messages in a row
|
||||
- **Only** `tool` role can have consecutive entries (parallel tool results)
|
||||
|
||||
Providers validate these sequences and will reject malformed histories.
|
||||
|
||||
## Interruptible API Calls
|
||||
|
||||
API requests are wrapped in `_interruptible_api_call()` which runs the actual HTTP call in a background thread while monitoring an interrupt event:
|
||||
|
||||
```text
|
||||
┌────────────────────────────────────────────────────┐
|
||||
│ Main thread API thread │
|
||||
│ │
|
||||
│ wait on: HTTP POST │
|
||||
│ - response ready ───▶ to provider │
|
||||
│ - interrupt event │
|
||||
│ - timeout │
|
||||
└────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
When interrupted (user sends new message, `/stop` command, or signal):
|
||||
- The API thread is abandoned (response discarded)
|
||||
- The agent can process the new input or shut down cleanly
|
||||
- No partial response is injected into conversation history
|
||||
|
||||
## Tool Execution
|
||||
|
||||
### Sequential vs Concurrent
|
||||
|
||||
When the model returns tool calls:
|
||||
|
||||
- **Single tool call** → executed directly in the main thread
|
||||
- **Multiple tool calls** → executed concurrently via `ThreadPoolExecutor`
|
||||
- Exception: tools marked as interactive (e.g., `clarify`) force sequential execution
|
||||
- Results are reinserted in the original tool call order regardless of completion order
|
||||
|
||||
### Execution Flow
|
||||
|
||||
```text
|
||||
for each tool_call in response.tool_calls:
|
||||
1. Resolve handler from tools/registry.py
|
||||
2. Fire pre_tool_call plugin hook
|
||||
3. Check if dangerous command (tools/approval.py)
|
||||
- If dangerous: invoke approval_callback, wait for user
|
||||
4. Execute handler with args + task_id
|
||||
5. Fire post_tool_call plugin hook
|
||||
6. Append {"role": "tool", "content": result} to history
|
||||
```
|
||||
|
||||
### Agent-Level Tools
|
||||
|
||||
Some tools are intercepted by `run_agent.py` *before* reaching `handle_function_call()`:
|
||||
|
||||
| Tool | Why intercepted |
|
||||
|------|--------------------|
|
||||
| `todo` | Reads/writes agent-local task state |
|
||||
| `memory` | Writes to persistent memory files with character limits |
|
||||
| `session_search` | Queries session history via the agent's session DB |
|
||||
| `delegate_task` | Spawns subagent(s) with isolated context |
|
||||
|
||||
These tools modify agent state directly and return synthetic tool results without going through the registry.
|
||||
|
||||
## Callback Surfaces
|
||||
|
||||
`AIAgent` supports platform-specific callbacks that enable real-time progress in the CLI, gateway, and ACP integrations:
|
||||
|
||||
| Callback | When fired | Used by |
|
||||
|----------|-----------|---------|
|
||||
| `tool_progress_callback` | Before/after each tool execution | CLI spinner, gateway progress messages |
|
||||
| `thinking_callback` | When model starts/stops thinking | CLI "thinking..." indicator |
|
||||
| `reasoning_callback` | When model returns reasoning content | CLI reasoning display, gateway reasoning blocks |
|
||||
| `clarify_callback` | When `clarify` tool is called | CLI input prompt, gateway interactive message |
|
||||
| `step_callback` | After each complete agent turn | Gateway step tracking, ACP progress |
|
||||
| `stream_delta_callback` | Each streaming token (when enabled) | CLI streaming display |
|
||||
| `tool_gen_callback` | When tool call is parsed from stream | CLI tool preview in spinner |
|
||||
| `status_callback` | State changes (thinking, executing, etc.) | ACP status updates |
|
||||
|
||||
## Budget and Fallback Behavior
|
||||
|
||||
### Iteration Budget
|
||||
|
||||
The agent tracks iterations via `IterationBudget`:
|
||||
|
||||
- Default: 90 iterations (configurable via `agent.max_turns`)
|
||||
- Each agent gets its own budget. Subagents get independent budgets capped at `delegation.max_iterations` (default 50) — total iterations across parent + subagents can exceed the parent's cap
|
||||
- At 100%, the agent stops and returns a summary of work done
|
||||
|
||||
### Fallback Model
|
||||
|
||||
When the primary model fails (429 rate limit, 5xx server error, 401/403 auth error):
|
||||
|
||||
1. Check `fallback_providers` list in config
|
||||
2. Try each fallback in order
|
||||
3. On success, continue the conversation with the new provider
|
||||
4. On 401/403, attempt credential refresh before failing over
|
||||
|
||||
The fallback system also covers auxiliary tasks independently — vision, compression, and web extraction each have their own fallback chain configurable via the `auxiliary.*` config section.
|
||||
|
||||
## Compression and Persistence
|
||||
|
||||
### When Compression Triggers
|
||||
|
||||
- **Preflight** (before API call): If conversation exceeds 50% of model's context window
|
||||
- **Gateway auto-compression**: If conversation exceeds 85% (more aggressive, runs between turns)
|
||||
|
||||
### What Happens During Compression
|
||||
|
||||
1. Memory is flushed to disk first (preventing data loss)
|
||||
2. Middle conversation turns are summarized into a compact summary
|
||||
3. The last N messages are preserved intact (`compression.protect_last_n`, default: 20)
|
||||
4. Tool call/result message pairs are kept together (never split)
|
||||
5. A new session lineage ID is generated (compression creates a "child" session)
|
||||
|
||||
### Session Persistence
|
||||
|
||||
After each turn:
|
||||
- Messages are saved to the session store (SQLite via `hermes_state.py`)
|
||||
- Memory changes are flushed to `MEMORY.md` / `USER.md`
|
||||
- The session can be resumed later via `/resume` or `hermes chat --resume`
|
||||
|
||||
## Key Source Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `run_agent.py` | AIAgent class — the complete agent loop |
|
||||
| `agent/prompt_builder.py` | System prompt assembly from memory, skills, context files, personality |
|
||||
| `agent/context_engine.py` | ContextEngine ABC — pluggable context management |
|
||||
| `agent/context_compressor.py` | Default engine — lossy summarization algorithm |
|
||||
| `agent/prompt_caching.py` | Anthropic prompt caching markers and cache metrics |
|
||||
| `agent/auxiliary_client.py` | Auxiliary LLM client for side tasks (vision, summarization) |
|
||||
| `model_tools.py` | Tool schema collection, `handle_function_call()` dispatch |
|
||||
|
||||
## Related Docs
|
||||
|
||||
- [Provider Runtime Resolution](./provider-runtime.md)
|
||||
- [Prompt Assembly](./prompt-assembly.md)
|
||||
- [Context Compression & Prompt Caching](./context-compression-and-caching.md)
|
||||
- [Tools Runtime](./tools-runtime.md)
|
||||
- [Architecture Overview](./architecture.md)
|
||||
@@ -0,0 +1,277 @@
|
||||
---
|
||||
sidebar_position: 1
|
||||
title: "Architecture"
|
||||
description: "Hermes Agent internals — major subsystems, execution paths, data flow, and where to read next"
|
||||
---
|
||||
|
||||
# Architecture
|
||||
|
||||
This page is the top-level map of Hermes Agent internals. Use it to orient yourself in the codebase, then dive into subsystem-specific docs for implementation details.
|
||||
|
||||
## System Overview
|
||||
|
||||
```text
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ Entry Points │
|
||||
│ │
|
||||
│ CLI (cli.py) Gateway (gateway/run.py) ACP (acp_adapter/) │
|
||||
│ Batch Runner API Server Python Library │
|
||||
└──────────┬──────────────┬───────────────────────┬───────────────────┘
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ AIAgent (run_agent.py) │
|
||||
│ │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
|
||||
│ │ Prompt │ │ Provider │ │ Tool │ │
|
||||
│ │ Builder │ │ Resolution │ │ Dispatch │ │
|
||||
│ │ (prompt_ │ │ (runtime_ │ │ (model_ │ │
|
||||
│ │ builder.py) │ │ provider.py)│ │ tools.py) │ │
|
||||
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
|
||||
│ │ │ │ │
|
||||
│ ┌──────┴───────┐ ┌──────┴───────┐ ┌──────┴───────┐ │
|
||||
│ │ Compression │ │ 3 API Modes │ │ Tool Registry│ │
|
||||
│ │ & Caching │ │ chat_compl. │ │ (registry.py)│ │
|
||||
│ │ │ │ codex_resp. │ │ 70+ tools │ │
|
||||
│ │ │ │ anthropic │ │ 28 toolsets │ │
|
||||
│ └──────────────┘ └──────────────┘ └──────────────┘ │
|
||||
└─────────┴─────────────────┴─────────────────┴───────────────────────┘
|
||||
│ │
|
||||
▼ ▼
|
||||
┌───────────────────┐ ┌──────────────────────┐
|
||||
│ Session Storage │ │ Tool Backends │
|
||||
│ (SQLite + FTS5) │ │ Terminal (6 backends) │
|
||||
│ hermes_state.py │ │ Browser (5 backends) │
|
||||
│ gateway/session.py│ │ Web (4 backends) │
|
||||
└───────────────────┘ │ MCP (dynamic) │
|
||||
│ File, Vision, etc. │
|
||||
└──────────────────────┘
|
||||
```
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```text
|
||||
hermes-agent/
|
||||
├── run_agent.py # AIAgent — core conversation loop (large file)
|
||||
├── cli.py # HermesCLI — interactive terminal UI (large file)
|
||||
├── model_tools.py # Tool discovery, schema collection, dispatch
|
||||
├── toolsets.py # Tool groupings and platform presets
|
||||
├── hermes_state.py # SQLite session/state database with FTS5
|
||||
├── hermes_constants.py # HERMES_HOME, profile-aware paths
|
||||
├── batch_runner.py # Batch trajectory generation
|
||||
│
|
||||
├── agent/ # Agent internals
|
||||
│ ├── prompt_builder.py # System prompt assembly
|
||||
│ ├── context_engine.py # ContextEngine ABC (pluggable)
|
||||
│ ├── context_compressor.py # Default engine — lossy summarization
|
||||
│ ├── prompt_caching.py # Anthropic prompt caching
|
||||
│ ├── auxiliary_client.py # Auxiliary LLM for side tasks (vision, summarization)
|
||||
│ ├── model_metadata.py # Model context lengths, token estimation
|
||||
│ ├── models_dev.py # models.dev registry integration
|
||||
│ ├── anthropic_adapter.py # Anthropic Messages API format conversion
|
||||
│ ├── display.py # KawaiiSpinner, tool preview formatting
|
||||
│ ├── skill_commands.py # Skill slash commands
|
||||
│ ├── memory_manager.py # Memory manager orchestration
|
||||
│ ├── memory_provider.py # Memory provider ABC
|
||||
│ └── trajectory.py # Trajectory saving helpers
|
||||
│
|
||||
├── hermes_cli/ # CLI subcommands and setup
|
||||
│ ├── main.py # Entry point — all `hermes` subcommands (large file)
|
||||
│ ├── config.py # DEFAULT_CONFIG, OPTIONAL_ENV_VARS, migration
|
||||
│ ├── commands.py # COMMAND_REGISTRY — central slash command definitions
|
||||
│ ├── auth.py # PROVIDER_REGISTRY, credential resolution
|
||||
│ ├── runtime_provider.py # Provider → api_mode + credentials
|
||||
│ ├── models.py # Model catalog, provider model lists
|
||||
│ ├── model_switch.py # /model command logic (CLI + gateway shared)
|
||||
│ ├── setup.py # Interactive setup wizard (large file)
|
||||
│ ├── skin_engine.py # CLI theming engine
|
||||
│ ├── skills_config.py # hermes skills — enable/disable per platform
|
||||
│ ├── skills_hub.py # /skills slash command
|
||||
│ ├── tools_config.py # hermes tools — enable/disable per platform
|
||||
│ ├── plugins.py # PluginManager — discovery, loading, hooks
|
||||
│ ├── callbacks.py # Terminal callbacks (clarify, sudo, approval)
|
||||
│ └── gateway.py # hermes gateway start/stop
|
||||
│
|
||||
├── tools/ # Tool implementations (one file per tool)
|
||||
│ ├── registry.py # Central tool registry
|
||||
│ ├── approval.py # Dangerous command detection
|
||||
│ ├── terminal_tool.py # Terminal orchestration
|
||||
│ ├── process_registry.py # Background process management
|
||||
│ ├── file_tools.py # read_file, write_file, patch, search_files
|
||||
│ ├── web_tools.py # web_search, web_extract
|
||||
│ ├── browser_tool.py # 10 browser automation tools
|
||||
│ ├── code_execution_tool.py # execute_code sandbox
|
||||
│ ├── delegate_tool.py # Subagent delegation
|
||||
│ ├── mcp_tool.py # MCP client (large file)
|
||||
│ ├── credential_files.py # File-based credential passthrough
|
||||
│ ├── env_passthrough.py # Env var passthrough for sandboxes
|
||||
│ ├── ansi_strip.py # ANSI escape stripping
|
||||
│ └── environments/ # Terminal backends (local, docker, ssh, modal, daytona, singularity)
|
||||
│
|
||||
├── gateway/ # Messaging platform gateway
|
||||
│ ├── run.py # GatewayRunner — message dispatch (large file)
|
||||
│ ├── session.py # SessionStore — conversation persistence
|
||||
│ ├── delivery.py # Outbound message delivery
|
||||
│ ├── pairing.py # DM pairing authorization
|
||||
│ ├── hooks.py # Hook discovery and lifecycle events
|
||||
│ ├── mirror.py # Cross-session message mirroring
|
||||
│ ├── status.py # Token locks, profile-scoped process tracking
|
||||
│ ├── builtin_hooks/ # Extension point for always-registered hooks (none shipped)
|
||||
│ └── platforms/ # 20 adapters: telegram, discord, slack, whatsapp,
|
||||
│ # signal, matrix, mattermost, email, sms,
|
||||
│ # dingtalk, feishu, wecom, wecom_callback, weixin,
|
||||
│ # bluebubbles, qqbot, homeassistant, webhook, api_server,
|
||||
│ # yuanbao
|
||||
│
|
||||
├── acp_adapter/ # ACP server (VS Code / Zed / JetBrains)
|
||||
├── cron/ # Scheduler (jobs.py, scheduler.py)
|
||||
├── plugins/memory/ # Memory provider plugins
|
||||
├── plugins/context_engine/ # Context engine plugins
|
||||
├── skills/ # Bundled skills (always available)
|
||||
├── optional-skills/ # Official optional skills (install explicitly)
|
||||
├── website/ # Docusaurus documentation site
|
||||
└── tests/ # Pytest suite (~25,000 tests across ~1,250 files)
|
||||
```
|
||||
|
||||
## Data Flow
|
||||
|
||||
### CLI Session
|
||||
|
||||
```text
|
||||
User input → HermesCLI.process_input()
|
||||
→ AIAgent.run_conversation()
|
||||
→ prompt_builder.build_system_prompt()
|
||||
→ runtime_provider.resolve_runtime_provider()
|
||||
→ API call (chat_completions / codex_responses / anthropic_messages)
|
||||
→ tool_calls? → model_tools.handle_function_call() → loop
|
||||
→ final response → display → save to SessionDB
|
||||
```
|
||||
|
||||
### Gateway Message
|
||||
|
||||
```text
|
||||
Platform event → Adapter.on_message() → MessageEvent
|
||||
→ GatewayRunner._handle_message()
|
||||
→ authorize user
|
||||
→ resolve session key
|
||||
→ create AIAgent with session history
|
||||
→ AIAgent.run_conversation()
|
||||
→ deliver response back through adapter
|
||||
```
|
||||
|
||||
### Cron Job
|
||||
|
||||
```text
|
||||
Scheduler tick → load due jobs from jobs.json
|
||||
→ create fresh AIAgent (no history)
|
||||
→ inject attached skills as context
|
||||
→ run job prompt
|
||||
→ deliver response to target platform
|
||||
→ update job state and next_run
|
||||
```
|
||||
|
||||
## Recommended Reading Order
|
||||
|
||||
If you are new to the codebase:
|
||||
|
||||
1. **This page** — orient yourself
|
||||
2. **[Agent Loop Internals](./agent-loop.md)** — how AIAgent works
|
||||
3. **[Prompt Assembly](./prompt-assembly.md)** — system prompt construction
|
||||
4. **[Provider Runtime Resolution](./provider-runtime.md)** — how providers are selected
|
||||
5. **[Adding Providers](./adding-providers.md)** — practical guide to adding a new provider
|
||||
6. **[Tools Runtime](./tools-runtime.md)** — tool registry, dispatch, environments
|
||||
7. **[Session Storage](./session-storage.md)** — SQLite schema, FTS5, session lineage
|
||||
8. **[Gateway Internals](./gateway-internals.md)** — messaging platform gateway
|
||||
9. **[Context Compression & Prompt Caching](./context-compression-and-caching.md)** — compression and caching
|
||||
10. **[ACP Internals](./acp-internals.md)** — IDE integration
|
||||
|
||||
## Major Subsystems
|
||||
|
||||
### Agent Loop
|
||||
|
||||
The synchronous orchestration engine (`AIAgent` in `run_agent.py`). Handles provider selection, prompt construction, tool execution, retries, fallback, callbacks, compression, and persistence. Supports three API modes for different provider backends.
|
||||
|
||||
→ [Agent Loop Internals](./agent-loop.md)
|
||||
|
||||
### Prompt System
|
||||
|
||||
Prompt construction and maintenance across the conversation lifecycle:
|
||||
|
||||
- **`system_prompt.py` + `prompt_builder.py`** — assembles the ordered system-prompt tiers (`stable` → `context` → `volatile`): identity/tool guidance/skills, context files, then memory/profile/timestamp blocks
|
||||
- **`prompt_caching.py`** — Applies Anthropic cache breakpoints for prefix caching
|
||||
- **`context_compressor.py`** — Summarizes middle conversation turns when context exceeds thresholds
|
||||
|
||||
→ [Prompt Assembly](./prompt-assembly.md), [Context Compression & Prompt Caching](./context-compression-and-caching.md)
|
||||
|
||||
### Provider Resolution
|
||||
|
||||
A shared runtime resolver used by CLI, gateway, cron, ACP, and auxiliary calls. Maps `(provider, model)` tuples to `(api_mode, api_key, base_url)`. Handles 18+ providers, OAuth flows, credential pools, and alias resolution.
|
||||
|
||||
→ [Provider Runtime Resolution](./provider-runtime.md)
|
||||
|
||||
### Tool System
|
||||
|
||||
Central tool registry (`tools/registry.py`) with 70+ registered tools across ~28 toolsets. Each tool file self-registers at import time. The registry handles schema collection, dispatch, availability checking, and error wrapping. Terminal tools support 6 backends (local, Docker, SSH, Daytona, Modal, Singularity).
|
||||
|
||||
→ [Tools Runtime](./tools-runtime.md)
|
||||
|
||||
### Session Persistence
|
||||
|
||||
SQLite-based session storage with FTS5 full-text search. Sessions have lineage tracking (parent/child across compressions), per-platform isolation, and atomic writes with contention handling.
|
||||
|
||||
→ [Session Storage](./session-storage.md)
|
||||
|
||||
### Messaging Gateway
|
||||
|
||||
Long-running process with 20 platform adapters, unified session routing, user authorization (allowlists + DM pairing), slash command dispatch, hook system, cron ticking, and background maintenance.
|
||||
|
||||
→ [Gateway Internals](./gateway-internals.md)
|
||||
|
||||
### Plugin System
|
||||
|
||||
Three discovery sources: `~/.hermes/plugins/` (user), `.hermes/plugins/` (project), and pip entry points. Plugins register tools, hooks, and CLI commands through a context API. Two specialized plugin types exist: memory providers (`plugins/memory/`) and context engines (`plugins/context_engine/`). Both are single-select — only one of each can be active at a time, configured via `hermes plugins` or `config.yaml`.
|
||||
|
||||
→ [Plugin Guide](/developer-guide/plugins), [Memory Provider Plugin](./memory-provider-plugin.md)
|
||||
|
||||
### Cron
|
||||
|
||||
First-class agent tasks (not shell tasks). Jobs store in JSON, support multiple schedule formats, can attach skills and scripts, and deliver to any platform.
|
||||
|
||||
→ [Cron Internals](./cron-internals.md)
|
||||
|
||||
### ACP Integration
|
||||
|
||||
Exposes Hermes as an editor-native agent over stdio/JSON-RPC for VS Code, Zed, and JetBrains.
|
||||
|
||||
→ [ACP Internals](./acp-internals.md)
|
||||
|
||||
### Trajectories
|
||||
|
||||
Generates ShareGPT-format trajectories from agent sessions for training data generation.
|
||||
|
||||
→ [Trajectories & Training Format](./trajectory-format.md)
|
||||
|
||||
## Design Principles
|
||||
|
||||
| Principle | What it means in practice |
|
||||
|-----------|--------------------------|
|
||||
| **Prompt stability** | System prompt doesn't change mid-conversation. No cache-breaking mutations except explicit user actions (`/model`). |
|
||||
| **Observable execution** | Every tool call is visible to the user via callbacks. Progress updates in CLI (spinner) and gateway (chat messages). |
|
||||
| **Interruptible** | API calls and tool execution can be cancelled mid-flight by user input or signals. |
|
||||
| **Platform-agnostic core** | One AIAgent class serves CLI, gateway, ACP, batch, and API server. Platform differences live in the entry point, not the agent. |
|
||||
| **Loose coupling** | Optional subsystems (MCP, plugins, memory providers, RL environments) use registry patterns and check_fn gating, not hard dependencies. |
|
||||
| **Profile isolation** | Each profile (`hermes -p <name>`) gets its own HERMES_HOME, config, memory, sessions, and gateway PID. Multiple profiles run concurrently. |
|
||||
|
||||
## File Dependency Chain
|
||||
|
||||
```text
|
||||
tools/registry.py (no deps — imported by all tool files)
|
||||
↑
|
||||
tools/*.py (each calls registry.register() at import time)
|
||||
↑
|
||||
model_tools.py (imports tools/registry + triggers tool discovery)
|
||||
↑
|
||||
run_agent.py, cli.py, batch_runner.py, environments/
|
||||
```
|
||||
|
||||
This chain means tool registration happens at import time, before any agent instance is created. Any `tools/*.py` file with a top-level `registry.register()` call is auto-discovered — no manual import list needed.
|
||||
@@ -0,0 +1,153 @@
|
||||
---
|
||||
sidebar_position: 13
|
||||
title: "Browser Provider Plugins"
|
||||
description: "How to build a cloud browser backend plugin for Hermes Agent"
|
||||
---
|
||||
|
||||
# Building a Browser Provider Plugin
|
||||
|
||||
Browser provider plugins register a **cloud browser backend** that services cloud-mode `browser_*` tool calls (navigate, click, screenshot, …). Built-in providers — Browserbase, Browser Use, and Firecrawl — all ship as plugins under `plugins/browser/<name>/`. You can add a new one, or override a bundled one, by dropping a directory next to them.
|
||||
|
||||
:::tip
|
||||
Browser backends are one of several **backend plugins** Hermes supports. The others (with their own ABCs) are [Web Search Provider Plugins](/developer-guide/web-search-provider-plugin) (which this ABC deliberately mirrors), [Image Generation](/developer-guide/image-gen-provider-plugin), [Video Generation](/developer-guide/video-gen-provider-plugin), [Memory Providers](/developer-guide/memory-provider-plugin), [Context Engines](/developer-guide/context-engine-plugin), [Secret Sources](/developer-guide/secret-source-plugin), and [Model Providers](/developer-guide/model-provider-plugin). General tool/hook/CLI plugins live in [Build a Hermes Plugin](/developer-guide/plugins).
|
||||
:::
|
||||
|
||||
## How it fits together
|
||||
|
||||
A browser provider does **not** implement browsing. It implements **session lifecycle**: create a remote browser session, hand back a CDP websocket URL, and tear the session down. Hermes' own browser stack (`agent-browser` + `tools/browser_tool.py`) connects to whatever CDP URL you return and drives the page from there — every provider gets the full `browser_*` toolset for free.
|
||||
|
||||
The active provider is selected by `browser.cloud_provider` in `config.yaml`; the dispatcher in `tools/browser_tool.py` is a pure registry lookup with no per-provider conditionals.
|
||||
|
||||
## Discovery
|
||||
|
||||
Hermes scans for browser backends in three places:
|
||||
|
||||
1. **Bundled** — `<repo>/plugins/browser/<name>/` (auto-loaded with `kind: backend`)
|
||||
2. **User** — `~/.hermes/plugins/browser/<name>/` (opt-in via `plugins.enabled` or `hermes plugins enable <name>`)
|
||||
3. **Pip** — packages declaring a `hermes_agent.plugins` entry point
|
||||
|
||||
Each plugin's `register(ctx)` calls `ctx.register_browser_provider(...)`, which puts the instance into the registry in `agent/browser_registry.py`.
|
||||
|
||||
## Directory structure
|
||||
|
||||
```
|
||||
plugins/browser/my-backend/
|
||||
├── __init__.py # register() entry point
|
||||
├── provider.py # BrowserProvider subclass
|
||||
└── plugin.yaml # Manifest with kind: backend and provides_browser_providers
|
||||
```
|
||||
|
||||
`plugin.yaml`:
|
||||
|
||||
```yaml
|
||||
name: browser-my-backend
|
||||
version: 1.0.0
|
||||
description: "My cloud browser backend. Requires MY_BACKEND_API_KEY."
|
||||
author: you
|
||||
kind: backend
|
||||
provides_browser_providers:
|
||||
- my-backend
|
||||
```
|
||||
|
||||
`__init__.py`:
|
||||
|
||||
```python
|
||||
from plugins.browser.my_backend.provider import MyBackendProvider
|
||||
|
||||
|
||||
def register(ctx) -> None:
|
||||
ctx.register_browser_provider(MyBackendProvider())
|
||||
```
|
||||
|
||||
## The BrowserProvider ABC
|
||||
|
||||
Implement `agent.browser_provider.BrowserProvider`. Three lifecycle methods plus identity:
|
||||
|
||||
```python
|
||||
from agent.browser_provider import BrowserProvider
|
||||
|
||||
|
||||
class MyBackendProvider(BrowserProvider):
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "my-backend" # the browser.cloud_provider config value
|
||||
|
||||
@property
|
||||
def display_name(self) -> str:
|
||||
return "My Backend" # shown in `hermes tools`
|
||||
|
||||
def is_available(self) -> bool:
|
||||
"""Cheap check only — env var present, dep importable.
|
||||
NO network calls: runs at tool-registration time and on every
|
||||
`hermes tools` paint."""
|
||||
return bool(os.environ.get("MY_BACKEND_API_KEY"))
|
||||
|
||||
def create_session(self, task_id: str) -> dict:
|
||||
"""Create a remote browser session; return the session-metadata contract."""
|
||||
session = my_api.create_browser(...)
|
||||
return {
|
||||
"session_name": f"my-backend-{task_id}", # unique agent-browser session name
|
||||
"bb_session_id": session.id, # provider session ID (for cleanup)
|
||||
"cdp_url": session.cdp_ws_url, # CDP websocket URL
|
||||
"features": {"stealth": True}, # feature flags you enabled
|
||||
}
|
||||
|
||||
def close_session(self, session_id: str) -> bool:
|
||||
"""Terminate by provider session ID. Log-and-return-False on error —
|
||||
never raise, so the dispatcher's cleanup loop keeps moving."""
|
||||
...
|
||||
|
||||
def emergency_cleanup(self, session_id: str) -> None:
|
||||
"""Best-effort teardown from atexit/signal handlers. Must not raise."""
|
||||
...
|
||||
```
|
||||
|
||||
### The session-metadata contract
|
||||
|
||||
`create_session()` must return at least `session_name`, `bb_session_id`, `cdp_url`, and `features`. Two quirks worth knowing:
|
||||
|
||||
- **`bb_session_id` is a legacy key name** kept verbatim for backward compatibility with `tools/browser_tool.py` — it holds *your* provider's session ID regardless of vendor. Don't rename it.
|
||||
- `create_session()` **may raise** — `ValueError` for missing credentials, `RuntimeError` for network/API failures. The dispatcher surfaces these to the user. This differs from `close_session`/`emergency_cleanup`, which must never raise.
|
||||
|
||||
An optional `external_call_id` key supports managed-gateway billing.
|
||||
|
||||
### `get_setup_schema()` — the `hermes tools` picker row
|
||||
|
||||
Override this to appear as a first-class option in the Browser Automation picker with API-key prompts and an install hook:
|
||||
|
||||
```python
|
||||
def get_setup_schema(self) -> dict:
|
||||
return {
|
||||
"name": "My Backend",
|
||||
"badge": "paid",
|
||||
"tag": "Cloud browser with stealth and proxies",
|
||||
"env_vars": [
|
||||
{"key": "MY_BACKEND_API_KEY",
|
||||
"prompt": "My Backend API key",
|
||||
"url": "https://mybackend.example"},
|
||||
],
|
||||
"post_setup": "agent_browser", # auto-installs the agent-browser npm dep
|
||||
}
|
||||
```
|
||||
|
||||
Per the project standard for tool backends: if a backend can't be selected and configured through `hermes tools`, it isn't done — "set this env var manually" is not an integration.
|
||||
|
||||
## Users configure it
|
||||
|
||||
```yaml
|
||||
browser:
|
||||
cloud_provider: my-backend
|
||||
```
|
||||
|
||||
## Reference implementations
|
||||
|
||||
The three bundled providers under `plugins/browser/` are the canonical examples, in ascending complexity: `firecrawl` (simplest), `browser_use`, and `browserbase` (stealth/proxy/keep-alive feature flags with graceful fallback when paid features are unavailable). Copy the closest one.
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] `name` is lowercase and stable (it's a config value users write)
|
||||
- [ ] `is_available()` makes zero network calls
|
||||
- [ ] `create_session()` returns the full metadata contract (`bb_session_id` key name intact)
|
||||
- [ ] `close_session()` / `emergency_cleanup()` never raise
|
||||
- [ ] `get_setup_schema()` exposes your env vars so `hermes tools` can configure the backend
|
||||
- [ ] `plugin.yaml` declares `kind: backend` + `provides_browser_providers`
|
||||
@@ -0,0 +1,198 @@
|
||||
---
|
||||
sidebar_position: 18
|
||||
title: "Browser CDP Supervisor"
|
||||
description: "How Hermes detects and responds to native JS dialogs and interacts with cross-origin iframes via a persistent CDP connection."
|
||||
---
|
||||
|
||||
# Browser CDP Supervisor
|
||||
|
||||
The CDP supervisor closes two long-standing gaps in Hermes' browser tooling:
|
||||
|
||||
1. **Native JS dialogs** (`alert`/`confirm`/`prompt`/`beforeunload`) block the
|
||||
page's JS thread. Without supervision, the agent has no way to know a
|
||||
dialog is open — subsequent tool calls hang or throw opaque errors.
|
||||
2. **Cross-origin iframes (OOPIFs)** are invisible to top-level
|
||||
`Runtime.evaluate`. The agent can see iframe nodes in the DOM snapshot but
|
||||
can't click, type, or eval inside them without a CDP session attached to
|
||||
the child target.
|
||||
|
||||
The supervisor solves both by holding a persistent WebSocket to the backend's
|
||||
CDP endpoint per browser task, surfacing pending dialogs and frame structure
|
||||
into `browser_snapshot`, and exposing a `browser_dialog` tool for explicit
|
||||
responses.
|
||||
|
||||
## Backend support
|
||||
|
||||
| Backend | Dialog detect | Dialog respond | Frame tree | OOPIF `Runtime.evaluate` via `browser_cdp(frame_id=...)` |
|
||||
|---|---|---|---|---|
|
||||
| Local Chrome (`--remote-debugging-port`) / `/browser connect` | ✓ | ✓ full workflow | ✓ | ✓ |
|
||||
| Browserbase | ✓ (via bridge) | ✓ full workflow (via bridge) | ✓ | ✓ |
|
||||
| Camofox | ✗ no CDP (REST-only) | ✗ | partial via DOM snapshot | ✗ |
|
||||
|
||||
**Browserbase quirk.** Browserbase's CDP proxy uses Playwright internally and
|
||||
auto-dismisses native dialogs within ~10ms, so `Page.handleJavaScriptDialog`
|
||||
can't keep up. The supervisor injects a bridge script via
|
||||
`Page.addScriptToEvaluateOnNewDocument` that overrides
|
||||
`window.alert`/`confirm`/`prompt` with a synchronous XHR to a magic host
|
||||
(`hermes-dialog-bridge.invalid`). `Fetch.enable` intercepts those XHRs before
|
||||
they touch the network — the dialog becomes a `Fetch.requestPaused` event the
|
||||
supervisor captures, and `respond_to_dialog` fulfills via
|
||||
`Fetch.fulfillRequest` with a JSON body the injected script decodes.
|
||||
|
||||
From the page's perspective, `prompt()` still returns the agent-supplied
|
||||
string. From the agent's perspective, it's the same `browser_dialog(action=...)`
|
||||
API either way.
|
||||
|
||||
Camofox is unsupported — no CDP surface, REST-only.
|
||||
|
||||
## Architecture
|
||||
|
||||
### CDPSupervisor
|
||||
|
||||
One `asyncio.Task` running in a background daemon thread per Hermes `task_id`.
|
||||
Holds a persistent WebSocket to the backend's CDP endpoint. Maintains:
|
||||
|
||||
- **Dialog queue** — `List[PendingDialog]` with `{id, type, message, default_prompt, session_id, opened_at}`
|
||||
- **Frame tree** — `Dict[frame_id, FrameInfo]` with parent relationships, URL, origin, whether cross-origin child session
|
||||
- **Session map** — `Dict[session_id, SessionInfo]` so interaction tools can route to the right attached session for OOPIF operations
|
||||
- **Recent console errors** — ring buffer of the last 50 for diagnostics
|
||||
|
||||
Subscribes on attach:
|
||||
|
||||
- `Page.enable` — `javascriptDialogOpening`, `frameAttached`, `frameNavigated`, `frameDetached`
|
||||
- `Runtime.enable` — `executionContextCreated`, `consoleAPICalled`, `exceptionThrown`
|
||||
- `Target.setAutoAttach {autoAttach: true, flatten: true}` — surfaces child OOPIF targets; supervisor enables `Page`+`Runtime` on each
|
||||
|
||||
Thread-safe state access via a snapshot lock; tool handlers (sync) read the
|
||||
frozen snapshot without awaiting.
|
||||
|
||||
### Lifecycle
|
||||
|
||||
- **Start:** `SupervisorRegistry.get_or_start(task_id, cdp_url)` — called by
|
||||
`browser_navigate`, Browserbase session create, `/browser connect`.
|
||||
Idempotent.
|
||||
- **Stop:** session teardown or `/browser disconnect`. Cancels the asyncio
|
||||
task, closes the WebSocket, discards state.
|
||||
- **Rebind:** if the CDP URL changes (user reconnects to a new Chrome), the
|
||||
old supervisor is stopped and a fresh one started — state is never reused
|
||||
across endpoints.
|
||||
|
||||
### Dialog policy
|
||||
|
||||
Configurable via `config.yaml` under `browser.dialog_policy`:
|
||||
|
||||
- **`must_respond`** (default) — capture, surface in `browser_snapshot`, wait
|
||||
for explicit `browser_dialog(action=...)` call. After a 300s safety timeout
|
||||
with no response, auto-dismiss and log. Prevents a buggy agent from stalling
|
||||
forever.
|
||||
- `auto_dismiss` — record and dismiss immediately; agent sees it after the
|
||||
fact via `browser_state` inside `browser_snapshot`.
|
||||
- `auto_accept` — record and accept (useful for `beforeunload` where the
|
||||
workflow wants to navigate away cleanly).
|
||||
|
||||
Policy is per-task; no per-dialog overrides.
|
||||
|
||||
## Agent surface
|
||||
|
||||
### `browser_dialog` tool
|
||||
|
||||
```
|
||||
browser_dialog(action, prompt_text=None, dialog_id=None)
|
||||
```
|
||||
|
||||
- `action="accept"` / `"dismiss"` → responds to the specified or sole pending dialog (required)
|
||||
- `prompt_text=...` → text to supply to a `prompt()` dialog
|
||||
- `dialog_id=...` → disambiguate when multiple dialogs are queued (rare)
|
||||
|
||||
Tool is response-only. The agent reads pending dialogs from `browser_snapshot`
|
||||
output before calling.
|
||||
|
||||
### `browser_snapshot` extension
|
||||
|
||||
Adds three optional fields to the existing snapshot output when a supervisor
|
||||
is attached:
|
||||
|
||||
```json
|
||||
{
|
||||
"pending_dialogs": [
|
||||
{"id": "d-1", "type": "alert", "message": "Hello", "opened_at": 1650000000.0}
|
||||
],
|
||||
"recent_dialogs": [
|
||||
{"id": "d-1", "type": "alert", "message": "...", "opened_at": 1650000000.0,
|
||||
"closed_at": 1650000000.1, "closed_by": "remote"}
|
||||
],
|
||||
"frame_tree": {
|
||||
"top": {"frame_id": "FRAME_A", "url": "https://example.com/", "origin": "https://example.com"},
|
||||
"children": [
|
||||
{"frame_id": "FRAME_B", "url": "about:srcdoc", "is_oopif": false},
|
||||
{"frame_id": "FRAME_C", "url": "https://ads.example.net/", "is_oopif": true, "session_id": "SID_C"}
|
||||
],
|
||||
"truncated": false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- **`pending_dialogs`** — dialogs currently blocking the page's JS thread.
|
||||
The agent must call `browser_dialog(action=...)` to respond. Empty on
|
||||
Browserbase because their CDP proxy auto-dismisses within ~10ms.
|
||||
|
||||
- **`recent_dialogs`** — ring buffer of up to 20 recently-closed dialogs with
|
||||
a `closed_by` tag: `"agent"` (we responded), `"auto_policy"` (local
|
||||
auto_dismiss/auto_accept), `"watchdog"` (must_respond timeout hit), or
|
||||
`"remote"` (browser/backend closed it on us, e.g. Browserbase). This is
|
||||
how agents on Browserbase still get visibility into what happened.
|
||||
|
||||
- **`frame_tree`** — frame structure including cross-origin (OOPIF) children.
|
||||
Capped at 30 entries + OOPIF depth 2 to bound snapshot size on ad-heavy
|
||||
pages. `truncated: true` surfaces when limits were hit; agents needing
|
||||
the full tree can use `browser_cdp` with `Page.getFrameTree`.
|
||||
|
||||
No new tool schema surface for any of these — the agent reads the snapshot it
|
||||
already requests.
|
||||
|
||||
### Availability gating
|
||||
|
||||
Both surfaces gate on `_browser_cdp_check` (supervisor can only run when a CDP
|
||||
endpoint is reachable). On Camofox / no-backend sessions, the dialog tool is
|
||||
hidden and the snapshot omits the new fields — no schema bloat.
|
||||
|
||||
## Cross-origin iframe interaction
|
||||
|
||||
`browser_cdp(frame_id=...)` routes CDP calls (notably `Runtime.evaluate`)
|
||||
through the supervisor's already-connected WebSocket using the OOPIF's child
|
||||
`sessionId`. Agents pick frame_ids out of
|
||||
`browser_snapshot.frame_tree.children[]` where `is_oopif=true` and pass them
|
||||
to `browser_cdp`. For same-origin iframes (no dedicated CDP session), the
|
||||
agent uses `contentWindow`/`contentDocument` from a top-level
|
||||
`Runtime.evaluate` instead — the supervisor surfaces an error pointing at that
|
||||
fallback when `frame_id` belongs to a non-OOPIF.
|
||||
|
||||
On Browserbase, this is the only reliable path for iframe interaction —
|
||||
stateless CDP connections (opened per `browser_cdp` call) hit signed-URL
|
||||
expiry, while the supervisor's long-lived connection keeps a valid session.
|
||||
|
||||
## File layout
|
||||
|
||||
- `tools/browser_supervisor.py` — `CDPSupervisor`, `SupervisorRegistry`, `PendingDialog`, `FrameInfo`
|
||||
- `tools/browser_dialog_tool.py` — `browser_dialog` tool handler
|
||||
- `tools/browser_tool.py` — `browser_navigate` start-hook, `browser_snapshot` merge, `/browser connect` reattach, `_cleanup_browser_session` teardown
|
||||
- `toolsets.py` — registers `browser_dialog` in `browser`, `hermes-acp`, `hermes-api-server`, and core toolsets (gated on CDP reachability)
|
||||
- `hermes_cli/config.py` — `browser.dialog_policy` and `browser.dialog_timeout_s` defaults
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Detection/interaction for Camofox (upstream gap; tracked separately)
|
||||
- Streaming dialog/frame events live to the user (would require gateway hooks)
|
||||
- Persisting dialog history across sessions (in-memory only)
|
||||
- Per-iframe dialog policies (agent can express this via `dialog_id`)
|
||||
- Replacing `browser_cdp` — it stays as the escape hatch for the long tail (cookies, viewport, network throttling)
|
||||
|
||||
## Testing
|
||||
|
||||
Unit tests (`tests/tools/test_browser_supervisor.py`) use an asyncio mock CDP
|
||||
server that speaks enough of the protocol to exercise all state transitions:
|
||||
attach, enable, navigate, dialog fire, dialog dismiss, frame attach/detach,
|
||||
child target attach, session teardown. Real-backend E2E (Browserbase + local
|
||||
Chromium-family browser) is manual — exercise via `/browser connect` to a
|
||||
live Chromium-family browser and run the dialog/frame test cases described
|
||||
above.
|
||||
@@ -0,0 +1,422 @@
|
||||
# Context Compression and Caching
|
||||
|
||||
Hermes Agent uses a dual compression system and Anthropic prompt caching to
|
||||
manage context window usage efficiently across long conversations.
|
||||
|
||||
Source files: `agent/context_engine.py` (ABC), `agent/context_compressor.py` (default engine),
|
||||
`agent/prompt_caching.py`, `gateway/run.py` (session hygiene), `run_agent.py` (search for `_compress_context`)
|
||||
|
||||
|
||||
## Pluggable Context Engine
|
||||
|
||||
Context management is built on the `ContextEngine` ABC (`agent/context_engine.py`). The built-in `ContextCompressor` is the default implementation, but plugins can replace it with alternative engines (e.g., Lossless Context Management).
|
||||
|
||||
```yaml
|
||||
context:
|
||||
engine: "compressor" # default — built-in lossy summarization
|
||||
engine: "lcm" # example — plugin providing lossless context
|
||||
```
|
||||
|
||||
The engine is responsible for:
|
||||
- Deciding when compaction should fire (`should_compress()`)
|
||||
- Performing compaction (`compress()`)
|
||||
- Optionally exposing tools the agent can call (e.g., `lcm_grep`)
|
||||
- Tracking token usage from API responses
|
||||
|
||||
Selection is config-driven via `context.engine` in `config.yaml`. The resolution order:
|
||||
1. Check `plugins/context_engine/<name>/` directory
|
||||
2. Check general plugin system (`register_context_engine()`)
|
||||
3. Fall back to built-in `ContextCompressor`
|
||||
|
||||
Plugin engines are **never auto-activated** — the user must explicitly set `context.engine` to the plugin's name. The default `"compressor"` always uses the built-in.
|
||||
|
||||
Configure via `hermes plugins` → Provider Plugins → Context Engine, or edit `config.yaml` directly.
|
||||
|
||||
For building a context engine plugin, see [Context Engine Plugins](/developer-guide/context-engine-plugin).
|
||||
|
||||
## Dual Compression System
|
||||
|
||||
Hermes has two separate compression layers that operate independently:
|
||||
|
||||
```
|
||||
┌──────────────────────────┐
|
||||
Incoming message │ Gateway Session Hygiene │ Fires at 85% of context
|
||||
─────────────────► │ (pre-agent, rough est.) │ Safety net for large sessions
|
||||
└─────────────┬────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────────────┐
|
||||
│ Agent ContextCompressor │ Fires at 50% of context (default)
|
||||
│ (in-loop, real tokens) │ Normal context management
|
||||
└──────────────────────────┘
|
||||
```
|
||||
|
||||
### 1. Gateway Session Hygiene (85% threshold)
|
||||
|
||||
Located in `gateway/run.py` (search for `Session hygiene: auto-compress`). This is a **safety net** that
|
||||
runs before the agent processes a message. It prevents API failures when sessions
|
||||
grow too large between turns (e.g., overnight accumulation in Telegram/Discord).
|
||||
|
||||
- **Threshold**: Fixed at 85% of model context length
|
||||
- **Token source**: Prefers actual API-reported tokens from last turn; falls back
|
||||
to rough character-based estimate (`estimate_messages_tokens_rough`)
|
||||
- **Fires**: Only when `len(history) >= 4` and compression is enabled
|
||||
- **Purpose**: Catch sessions that escaped the agent's own compressor
|
||||
|
||||
The gateway hygiene threshold is intentionally higher than the agent's compressor.
|
||||
Setting it at 50% (same as the agent) caused premature compression on every turn
|
||||
in long gateway sessions.
|
||||
|
||||
### 2. Agent ContextCompressor (50% threshold, configurable)
|
||||
|
||||
Located in `agent/context_compressor.py`. This is the **primary compression
|
||||
system** that runs inside the agent's tool loop with access to accurate,
|
||||
API-reported token counts.
|
||||
|
||||
|
||||
## Configuration
|
||||
|
||||
All compression settings are read from `config.yaml` under the `compression` key:
|
||||
|
||||
```yaml
|
||||
compression:
|
||||
enabled: true # Enable/disable compression (default: true)
|
||||
threshold: 0.50 # Fraction of context window (default: 0.50 = 50%)
|
||||
target_ratio: 0.20 # How much of threshold to keep as tail (default: 0.20)
|
||||
protect_last_n: 20 # Minimum protected tail messages (default: 20)
|
||||
codex_gpt55_autoraise: true # gpt-5.5 on Codex OAuth: raise trigger to 85% (default: true)
|
||||
codex_gpt55_autoraise_notice: true # Show the one-time autoraise notice (default: true)
|
||||
codex_app_server_auto: native # native|hermes|off for Codex app-server thread compaction
|
||||
|
||||
# Summarization model/provider configured under auxiliary:
|
||||
auxiliary:
|
||||
compression:
|
||||
model: null # Override model for summaries (default: auto-detect)
|
||||
provider: auto # Provider: "auto", "openrouter", "nous", "main", etc.
|
||||
base_url: null # Custom OpenAI-compatible endpoint
|
||||
```
|
||||
|
||||
### Parameter Details
|
||||
|
||||
| Parameter | Default | Range | Description |
|
||||
|-----------|---------|-------|-------------|
|
||||
| `threshold` | `0.50` | 0.0-1.0 | Compression triggers when prompt tokens ≥ `threshold × context_length` |
|
||||
| `target_ratio` | `0.20` | 0.10-0.80 | Controls tail protection token budget: `threshold_tokens × target_ratio` |
|
||||
| `protect_last_n` | `20` | ≥1 | Minimum number of recent messages always preserved |
|
||||
| `protect_first_n` | `3` | (hardcoded) | System prompt + first exchange always preserved |
|
||||
| `codex_gpt55_autoraise` | `true` | bool | Raise the trigger to 85% for gpt-5.5 on the ChatGPT Codex OAuth route (see below). Set `false` to keep the global `threshold` |
|
||||
| `codex_gpt55_autoraise_notice` | `true` | bool | Show the one-time Codex gpt-5.5 autoraise notice. Set `false` to keep the 85% autoraise but suppress the banner |
|
||||
| `codex_app_server_auto` | `native` | `native`, `hermes`, `off` | Thread-compaction mode for Codex app-server sessions (see below) |
|
||||
|
||||
### Codex gpt-5.5 threshold autoraise
|
||||
|
||||
The ChatGPT Codex OAuth backend hard-caps gpt-5.5 at a **272K** context window
|
||||
(the same slug exposes 1.05M on OpenAI's direct API and OpenRouter, and 400K on
|
||||
GitHub Copilot). At the default 50% trigger, compaction would fire at ~136K —
|
||||
half the window the model can actually use. When the active route is Codex
|
||||
OAuth (`provider: openai-codex`) and the model is gpt-5.5, Hermes raises the
|
||||
trigger to **85%** (~231K) and shows a notice with the opt-out command. The
|
||||
notice is shown once per profile — a marker under `$HERMES_HOME`
|
||||
(`.codex_gpt55_autoraise_notice`) records that it ran, so repeated agent/session
|
||||
inits (e.g. every inbound gateway message) don't re-emit it; if the raised
|
||||
threshold later changes it re-notifies once. Only this exact route is affected;
|
||||
gpt-5.5 on any other provider keeps your global `threshold`. To opt back down to
|
||||
the global value:
|
||||
|
||||
```bash
|
||||
hermes config set compression.codex_gpt55_autoraise false
|
||||
```
|
||||
|
||||
To keep the 85% autoraise but hide only the one-time notice:
|
||||
|
||||
```bash
|
||||
hermes config set compression.codex_gpt55_autoraise_notice false
|
||||
```
|
||||
|
||||
### Codex app-server thread compaction
|
||||
|
||||
Codex app-server sessions (`api_mode: codex_app_server` — the codex CLI/agent
|
||||
runtime) are different from every other route: the codex agent owns the backing
|
||||
thread context, so Hermes' auxiliary summarizer cannot shrink it — rewriting the
|
||||
local transcript mirror leaves the real thread growing unbounded until a hard
|
||||
context reset. For this runtime, compaction goes through the app-server's own
|
||||
mechanism instead:
|
||||
|
||||
- Manual compaction (`/compress`) asks the app-server to compact the thread
|
||||
(`thread/compact/start`) and waits for the compaction turn to complete.
|
||||
- Automatic compaction is controlled by `compression.codex_app_server_auto`:
|
||||
the default `native` lets the app-server decide when to compact and Hermes
|
||||
records the resulting compaction events (compression counters, session
|
||||
events). Set `hermes` to let Hermes' compression threshold initiate
|
||||
app-server compaction, or `off` to disable Hermes-initiated automatic
|
||||
compaction entirely (codex may still compact natively).
|
||||
|
||||
Hermes' local transcript is never rewritten on this runtime — state.db records
|
||||
the compaction boundary while the visible transcript stays intact. All other
|
||||
routes (including Codex OAuth chat sessions) keep Hermes' summary compressor.
|
||||
|
||||
### Computed Values (for a 200K context model at defaults)
|
||||
|
||||
```
|
||||
context_length = 200,000
|
||||
threshold_tokens = 200,000 × 0.50 = 100,000
|
||||
tail_token_budget = 100,000 × 0.20 = 20,000
|
||||
max_summary_tokens = min(200,000 × 0.05, 12,000) = 10,000
|
||||
```
|
||||
|
||||
:::note Threshold is derived from the MAIN model's context window
|
||||
`threshold_tokens` is always `threshold × context_length`, where `context_length`
|
||||
is the **main agent model's** context window — never the auxiliary/summary
|
||||
model's. On a 262,144-token model at the default `0.50`, the threshold is
|
||||
`262,144 × 0.50 = 131,072`. That number being close to a common "128K context"
|
||||
is a coincidence of the percentage, not a sign that the auxiliary model's window
|
||||
is the trigger. The auxiliary model's context window is a separate concern — see
|
||||
the "Summary model context length" warning below for how it affects whether a
|
||||
summary can be produced, not when compression fires.
|
||||
:::
|
||||
|
||||
|
||||
## Compression Algorithm
|
||||
|
||||
The `ContextCompressor.compress()` method follows a 4-phase algorithm:
|
||||
|
||||
### Phase 1: Prune Old Tool Results (cheap, no LLM call)
|
||||
|
||||
Old tool results (>200 chars) outside the protected tail are replaced with:
|
||||
```
|
||||
[Old tool output cleared to save context space]
|
||||
```
|
||||
|
||||
This is a cheap pre-pass that saves significant tokens from verbose tool
|
||||
outputs (file contents, terminal output, search results).
|
||||
|
||||
### Phase 2: Determine Boundaries
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Message list │
|
||||
│ │
|
||||
│ [0..2] ← protect_first_n (system + first exchange) │
|
||||
│ [3..N] ← middle turns → SUMMARIZED │
|
||||
│ [N..end] ← tail (by token budget OR protect_last_n) │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
Tail protection is **token-budget based**: walks backward from the end,
|
||||
accumulating tokens until the budget is exhausted. Falls back to the fixed
|
||||
`protect_last_n` count if the budget would protect fewer messages.
|
||||
|
||||
Boundaries are aligned to avoid splitting tool_call/tool_result groups.
|
||||
The `_align_boundary_backward()` method walks past consecutive tool results
|
||||
to find the parent assistant message, keeping groups intact.
|
||||
|
||||
### Phase 3: Generate Structured Summary
|
||||
|
||||
:::warning Summary model context length
|
||||
The summary model must have a context window **at least as large** as the main agent model's. The entire middle section is sent to the summary model in a single `call_llm(task="compression")` call. If the summary model's context is smaller, the API returns a context-length error — `_generate_summary()` catches it, logs a warning, and returns `None`. The compressor then drops the middle turns **without a summary**, silently losing conversation context. This is the most common cause of degraded compaction quality.
|
||||
:::
|
||||
|
||||
The middle turns are summarized using the auxiliary LLM with a structured
|
||||
template:
|
||||
|
||||
```
|
||||
## Goal
|
||||
[What the user is trying to accomplish]
|
||||
|
||||
## Constraints & Preferences
|
||||
[User preferences, coding style, constraints, important decisions]
|
||||
|
||||
## Progress
|
||||
### Done
|
||||
[Completed work — specific file paths, commands run, results]
|
||||
### In Progress
|
||||
[Work currently underway]
|
||||
### Blocked
|
||||
[Any blockers or issues encountered]
|
||||
|
||||
## Key Decisions
|
||||
[Important technical decisions and why]
|
||||
|
||||
## Relevant Files
|
||||
[Files read, modified, or created — with brief note on each]
|
||||
|
||||
## Next Steps
|
||||
[What needs to happen next]
|
||||
|
||||
## Critical Context
|
||||
[Specific values, error messages, configuration details]
|
||||
```
|
||||
|
||||
Summary budget scales with the amount of content being compressed:
|
||||
- Formula: `content_tokens × 0.20` (the `_SUMMARY_RATIO` constant)
|
||||
- Minimum: 2,000 tokens
|
||||
- Maximum: `min(context_length × 0.05, 12,000)` tokens
|
||||
|
||||
### Phase 4: Assemble Compressed Messages
|
||||
|
||||
The compressed message list is:
|
||||
1. Head messages (with a note appended to system prompt on first compression)
|
||||
2. Summary message (role chosen to avoid consecutive same-role violations)
|
||||
3. Tail messages (unmodified)
|
||||
|
||||
Orphaned tool_call/tool_result pairs are cleaned up by `_sanitize_tool_pairs()`:
|
||||
- Tool results referencing removed calls → removed
|
||||
- Tool calls whose results were removed → stub result injected
|
||||
|
||||
### Iterative Re-compression
|
||||
|
||||
On subsequent compressions, the previous summary is passed to the LLM with
|
||||
instructions to **update** it rather than summarize from scratch. This preserves
|
||||
information across multiple compactions — items move from "In Progress" to "Done",
|
||||
new progress is added, and obsolete information is removed.
|
||||
|
||||
The `_previous_summary` field on the compressor instance stores the last summary
|
||||
text for this purpose.
|
||||
|
||||
|
||||
## Before/After Example
|
||||
|
||||
### Before Compression (45 messages, ~95K tokens)
|
||||
|
||||
```
|
||||
[0] system: "You are a helpful assistant..." (system prompt)
|
||||
[1] user: "Help me set up a FastAPI project"
|
||||
[2] assistant: <tool_call> terminal: mkdir project </tool_call>
|
||||
[3] tool: "directory created"
|
||||
[4] assistant: <tool_call> write_file: main.py </tool_call>
|
||||
[5] tool: "file written (2.3KB)"
|
||||
... 30 more turns of file editing, testing, debugging ...
|
||||
[38] assistant: <tool_call> terminal: pytest </tool_call>
|
||||
[39] tool: "8 passed, 2 failed\n..." (5KB output)
|
||||
[40] user: "Fix the failing tests"
|
||||
[41] assistant: <tool_call> read_file: tests/test_api.py </tool_call>
|
||||
[42] tool: "import pytest\n..." (3KB)
|
||||
[43] assistant: "I see the issue with the test fixtures..."
|
||||
[44] user: "Great, also add error handling"
|
||||
```
|
||||
|
||||
### After Compression (25 messages, ~45K tokens)
|
||||
|
||||
```
|
||||
[0] system: "You are a helpful assistant...
|
||||
[Note: Some earlier conversation turns have been compacted...]"
|
||||
[1] user: "Help me set up a FastAPI project"
|
||||
[2] assistant: "[CONTEXT COMPACTION] Earlier turns were compacted...
|
||||
|
||||
## Goal
|
||||
Set up a FastAPI project with tests and error handling
|
||||
|
||||
## Progress
|
||||
### Done
|
||||
- Created project structure: main.py, tests/, requirements.txt
|
||||
- Implemented 5 API endpoints in main.py
|
||||
- Wrote 10 test cases in tests/test_api.py
|
||||
- 8/10 tests passing
|
||||
|
||||
### In Progress
|
||||
- Fixing 2 failing tests (test_create_user, test_delete_user)
|
||||
|
||||
## Relevant Files
|
||||
- main.py — FastAPI app with 5 endpoints
|
||||
- tests/test_api.py — 10 test cases
|
||||
- requirements.txt — fastapi, pytest, httpx
|
||||
|
||||
## Next Steps
|
||||
- Fix failing test fixtures
|
||||
- Add error handling"
|
||||
[3] user: "Fix the failing tests"
|
||||
[4] assistant: <tool_call> read_file: tests/test_api.py </tool_call>
|
||||
[5] tool: "import pytest\n..."
|
||||
[6] assistant: "I see the issue with the test fixtures..."
|
||||
[7] user: "Great, also add error handling"
|
||||
```
|
||||
|
||||
|
||||
## Prompt Caching (Anthropic)
|
||||
|
||||
Source: `agent/prompt_caching.py`
|
||||
|
||||
Reduces input token costs by ~75% on multi-turn conversations by caching the
|
||||
conversation prefix. Uses Anthropic's `cache_control` breakpoints.
|
||||
|
||||
### Strategy: system_and_3
|
||||
|
||||
Anthropic allows a maximum of 4 `cache_control` breakpoints per request. Hermes
|
||||
uses the "system_and_3" strategy:
|
||||
|
||||
```
|
||||
Breakpoint 1: System prompt (stable across all turns)
|
||||
Breakpoint 2: 3rd-to-last non-system message ─┐
|
||||
Breakpoint 3: 2nd-to-last non-system message ├─ Rolling window
|
||||
Breakpoint 4: Last non-system message ─┘
|
||||
```
|
||||
|
||||
### How It Works
|
||||
|
||||
`apply_anthropic_cache_control()` deep-copies the messages and injects
|
||||
`cache_control` markers:
|
||||
|
||||
```python
|
||||
# Cache marker format
|
||||
marker = {"type": "ephemeral"}
|
||||
# Or for 1-hour TTL:
|
||||
marker = {"type": "ephemeral", "ttl": "1h"}
|
||||
```
|
||||
|
||||
The marker is applied differently based on content type:
|
||||
|
||||
| Content Type | Where Marker Goes |
|
||||
|-------------|-------------------|
|
||||
| String content | Converted to `[{"type": "text", "text": ..., "cache_control": ...}]` |
|
||||
| List content | Added to the last element's dict |
|
||||
| None/empty | Added as `msg["cache_control"]` |
|
||||
| Tool messages | Added as `msg["cache_control"]` (native Anthropic only) |
|
||||
|
||||
### Cache-Aware Design Patterns
|
||||
|
||||
1. **Stable system prompt**: The system prompt is breakpoint 1 and cached across
|
||||
all turns. Avoid mutating it mid-conversation (compression appends a note
|
||||
only on the first compaction).
|
||||
|
||||
2. **Message ordering matters**: Cache hits require prefix matching. Adding or
|
||||
removing messages in the middle invalidates the cache for everything after.
|
||||
|
||||
3. **Compression cache interaction**: After compression, the cache is invalidated
|
||||
for the compressed region but the system prompt cache survives. The rolling
|
||||
3-message window re-establishes caching within 1-2 turns.
|
||||
|
||||
4. **TTL selection**: Default is `5m` (5 minutes). Use `1h` for long-running
|
||||
sessions where the user takes breaks between turns.
|
||||
|
||||
5. **Model identity is part of the cache key**: Provider-side caches are scoped
|
||||
to the model (and account/API key) serving the request. Any mid-conversation
|
||||
model change — an explicit `/model` switch, primary-model fallback, or a
|
||||
credential-pool rotation onto a different account — means the next request
|
||||
gets zero cache hits and re-reads the full conversation at undiscounted
|
||||
input price. This is inherent to how provider caches work, not something
|
||||
Hermes can avoid; user-facing docs for `/model`, fallback providers, and
|
||||
credential pools carry cost warnings for this reason. Don't add features
|
||||
that silently swap the model or credentials mid-session.
|
||||
|
||||
### Enabling Prompt Caching
|
||||
|
||||
Prompt caching is automatically enabled when:
|
||||
- The model is an Anthropic Claude model (detected by model name)
|
||||
- The provider supports `cache_control` (native Anthropic API or OpenRouter)
|
||||
|
||||
```yaml
|
||||
# config.yaml — TTL is configurable (must be "5m" or "1h")
|
||||
prompt_caching:
|
||||
cache_ttl: "5m"
|
||||
```
|
||||
|
||||
The CLI shows caching status at startup:
|
||||
```
|
||||
💾 Prompt caching: ENABLED (Claude via OpenRouter, 5m TTL)
|
||||
```
|
||||
|
||||
|
||||
## Context Pressure Warnings
|
||||
|
||||
Intermediate context-pressure warnings have been removed (see the iteration-budget block in `run_agent.py`, which notes: "No intermediate pressure warnings — they caused models to 'give up' prematurely on complex tasks"). Compression fires when prompt tokens reach the configured `compression.threshold` (default 50%) with no prior warning step; gateway session hygiene fires as the secondary safety net at 85% of the model's context window.
|
||||
@@ -0,0 +1,194 @@
|
||||
---
|
||||
sidebar_position: 9
|
||||
title: "Context Engine Plugins"
|
||||
description: "How to build a context engine plugin that replaces the built-in ContextCompressor"
|
||||
---
|
||||
|
||||
# Building a Context Engine Plugin
|
||||
|
||||
Context engine plugins replace the built-in `ContextCompressor` with an alternative strategy for managing conversation context. For example, a Lossless Context Management (LCM) engine that builds a knowledge DAG instead of lossy summarization.
|
||||
|
||||
## How it works
|
||||
|
||||
The agent's context management is built on the `ContextEngine` ABC (`agent/context_engine.py`). The built-in `ContextCompressor` is the default implementation. Plugin engines must implement the same interface.
|
||||
|
||||
Only **one** context engine can be active at a time. Selection is config-driven:
|
||||
|
||||
```yaml
|
||||
# config.yaml
|
||||
context:
|
||||
engine: "compressor" # default built-in
|
||||
engine: "lcm" # activates a plugin engine named "lcm"
|
||||
```
|
||||
|
||||
Plugin engines are **never auto-activated** — the user must explicitly set `context.engine` to the plugin's name.
|
||||
|
||||
## Directory structure
|
||||
|
||||
Each context engine lives in `plugins/context_engine/<name>/`:
|
||||
|
||||
```
|
||||
plugins/context_engine/lcm/
|
||||
├── __init__.py # exports the ContextEngine subclass
|
||||
├── plugin.yaml # metadata (name, description, version)
|
||||
└── ... # any other modules your engine needs
|
||||
```
|
||||
|
||||
## The ContextEngine ABC
|
||||
|
||||
Your engine must implement these **required** methods:
|
||||
|
||||
```python
|
||||
from agent.context_engine import ContextEngine
|
||||
|
||||
class LCMEngine(ContextEngine):
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
"""Short identifier, e.g. 'lcm'. Must match config.yaml value."""
|
||||
return "lcm"
|
||||
|
||||
def update_from_response(self, usage: dict) -> None:
|
||||
"""Called after every LLM call with the usage dict.
|
||||
|
||||
Update self.last_prompt_tokens, self.last_completion_tokens,
|
||||
self.last_total_tokens from the response.
|
||||
"""
|
||||
|
||||
def should_compress(self, prompt_tokens: int = None) -> bool:
|
||||
"""Return True if compaction should fire this turn."""
|
||||
|
||||
def compress(self, messages: list, current_tokens: int = None,
|
||||
focus_topic: str = None) -> list:
|
||||
"""Compact the message list and return a new (possibly shorter) list.
|
||||
|
||||
The returned list must be a valid OpenAI-format message sequence.
|
||||
|
||||
``focus_topic`` is an optional topic string from manual
|
||||
``/compress <focus>``; engines that support guided compression should
|
||||
prioritise preserving information related to it, others may ignore it.
|
||||
"""
|
||||
```
|
||||
|
||||
### Class attributes your engine must maintain
|
||||
|
||||
The agent reads these directly for display and logging:
|
||||
|
||||
```python
|
||||
last_prompt_tokens: int = 0
|
||||
last_completion_tokens: int = 0
|
||||
last_total_tokens: int = 0
|
||||
threshold_tokens: int = 0 # when compression triggers
|
||||
context_length: int = 0 # model's full context window
|
||||
compression_count: int = 0 # how many times compress() has run
|
||||
```
|
||||
|
||||
### Optional methods
|
||||
|
||||
These have sensible defaults in the ABC. Override as needed:
|
||||
|
||||
| Method | Default | Override when |
|
||||
|--------|---------|--------------|
|
||||
| `on_session_start(session_id, **kwargs)` | No-op | You need to load persisted state (DAG, DB) |
|
||||
| `on_session_end(session_id, messages)` | No-op | You need to flush state, close connections |
|
||||
| `on_session_reset()` | Resets token counters | You have per-session state to clear |
|
||||
| `update_model(model, context_length, ...)` | Updates context_length + threshold | You need to recalculate budgets on model switch |
|
||||
| `get_tool_schemas()` | Returns `[]` | Your engine provides agent-callable tools (e.g., `lcm_grep`) |
|
||||
| `handle_tool_call(name, args, **kwargs)` | Returns error JSON | You implement tool handlers |
|
||||
| `should_compress_preflight(messages)` | Returns `False` | You can do a cheap pre-API-call estimate |
|
||||
| `get_status()` | Standard token/threshold dict | You have custom metrics to expose |
|
||||
|
||||
## Engine tools
|
||||
|
||||
Context engines can expose tools the agent calls directly. Return schemas from `get_tool_schemas()` and handle calls in `handle_tool_call()`:
|
||||
|
||||
```python
|
||||
def get_tool_schemas(self):
|
||||
return [{
|
||||
"name": "lcm_grep",
|
||||
"description": "Search the context knowledge graph",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"query": {"type": "string", "description": "Search query"}
|
||||
},
|
||||
"required": ["query"],
|
||||
},
|
||||
}]
|
||||
|
||||
def handle_tool_call(self, name, args, **kwargs):
|
||||
if name == "lcm_grep":
|
||||
results = self._search_dag(args["query"])
|
||||
return json.dumps({"results": results})
|
||||
return json.dumps({"error": f"Unknown tool: {name}"})
|
||||
```
|
||||
|
||||
Engine tools are injected into the agent's tool list at startup and dispatched automatically — no registry registration needed.
|
||||
|
||||
## Registration
|
||||
|
||||
### Via directory (recommended)
|
||||
|
||||
Place your engine in `plugins/context_engine/<name>/`. The `__init__.py` must export a `ContextEngine` subclass. The discovery system finds and instantiates it automatically.
|
||||
|
||||
### Via general plugin system
|
||||
|
||||
A general plugin can also register a context engine:
|
||||
|
||||
```python
|
||||
def register(ctx):
|
||||
engine = LCMEngine(context_length=200000)
|
||||
ctx.register_context_engine(engine)
|
||||
```
|
||||
|
||||
Only one engine can be registered. A second plugin attempting to register is rejected with a warning.
|
||||
|
||||
## Lifecycle
|
||||
|
||||
```
|
||||
1. Engine instantiated (plugin load or directory discovery)
|
||||
2. on_session_start() — conversation begins
|
||||
3. update_from_response() — after each API call
|
||||
4. should_compress() — checked each turn
|
||||
5. compress() — called when should_compress() returns True
|
||||
6. on_session_end() — session boundary (CLI exit, /reset, gateway expiry)
|
||||
```
|
||||
|
||||
`on_session_reset()` is called on `/new` or `/reset` to clear per-session state without a full shutdown.
|
||||
|
||||
## Configuration
|
||||
|
||||
Users select your engine via `hermes plugins` → Provider Plugins → Context Engine, or by editing `config.yaml`:
|
||||
|
||||
```yaml
|
||||
context:
|
||||
engine: "lcm" # must match your engine's name property
|
||||
```
|
||||
|
||||
The `compression` config block (`compression.threshold`, `compression.protect_last_n`, etc.) is specific to the built-in `ContextCompressor`. Your engine should define its own config format if needed, reading from `config.yaml` during initialization.
|
||||
|
||||
## Testing
|
||||
|
||||
```python
|
||||
from agent.context_engine import ContextEngine
|
||||
|
||||
def test_engine_satisfies_abc():
|
||||
engine = YourEngine(context_length=200000)
|
||||
assert isinstance(engine, ContextEngine)
|
||||
assert engine.name == "your-name"
|
||||
|
||||
def test_compress_returns_valid_messages():
|
||||
engine = YourEngine(context_length=200000)
|
||||
msgs = [{"role": "user", "content": "hello"}]
|
||||
result = engine.compress(msgs)
|
||||
assert isinstance(result, list)
|
||||
assert all("role" in m for m in result)
|
||||
```
|
||||
|
||||
See `tests/agent/test_context_engine.py` for the full ABC contract test suite.
|
||||
|
||||
## See also
|
||||
|
||||
- [Context Compression and Caching](/developer-guide/context-compression-and-caching) — how the built-in compressor works
|
||||
- [Memory Provider Plugins](/developer-guide/memory-provider-plugin) — analogous single-select plugin system for memory
|
||||
- [Plugins](/user-guide/features/plugins) — general plugin system overview
|
||||
@@ -0,0 +1,298 @@
|
||||
---
|
||||
sidebar_position: 4
|
||||
title: "Contributing"
|
||||
description: "How to contribute to Hermes Agent — dev setup, code style, PR process"
|
||||
---
|
||||
|
||||
# Contributing
|
||||
|
||||
Thank you for contributing to Hermes Agent! This guide covers setting up your dev environment, understanding the codebase, and getting your PR merged.
|
||||
|
||||
## Contribution Priorities
|
||||
|
||||
We value contributions in this order:
|
||||
|
||||
1. **Bug fixes** — crashes, incorrect behavior, data loss
|
||||
2. **Cross-platform compatibility** — macOS, different Linux distros, WSL2
|
||||
3. **Security hardening** — shell injection, prompt injection, path traversal
|
||||
4. **Performance and robustness** — retry logic, error handling, graceful degradation
|
||||
5. **New skills** — broadly useful ones (see [Creating Skills](creating-skills.md))
|
||||
6. **New tools** — rarely needed; most capabilities should be skills
|
||||
7. **Documentation** — fixes, clarifications, new examples
|
||||
|
||||
## Common contribution paths
|
||||
|
||||
- Building a custom/local tool without modifying Hermes core? Start with [Build a Hermes Plugin](../developer-guide/plugins/index.md)
|
||||
- Building a new built-in core tool for Hermes itself? Start with [Adding Tools](./adding-tools.md)
|
||||
- Building a new skill? Start with [Creating Skills](./creating-skills.md)
|
||||
- Building a new inference provider? Start with [Adding Providers](./adding-providers.md)
|
||||
|
||||
## Development Setup
|
||||
|
||||
### Prerequisites
|
||||
|
||||
| Requirement | Notes |
|
||||
| -------------------- | --------------------------------------------------------------------------------------------- |
|
||||
| **Git** | With the `git-lfs` extension installed |
|
||||
| **Python 3.11–3.13** | uv will install it if missing |
|
||||
| **uv** | Fast Python package manager ([install](https://docs.astral.sh/uv/)) |
|
||||
| **Node.js 20+** | Optional — needed for browser tools and WhatsApp bridge (matches root `package.json` engines) |
|
||||
|
||||
### Install with the standard installer
|
||||
|
||||
For most contributors, the best development bootstrap is the same path users
|
||||
take: run the standard installer, then work inside the repository it cloned.
|
||||
The installer creates the Hermes venv, wires the `hermes` command, stamps the
|
||||
install method for `hermes update`, and clones the full git project into
|
||||
`$HERMES_HOME/hermes-agent` (usually `~/.hermes/hermes-agent`). That keeps your
|
||||
development environment on the same layout the CLI, updater, lazy dependency
|
||||
installer, gateway, and docs assume.
|
||||
|
||||
```bash
|
||||
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
|
||||
cd "${HERMES_HOME:-$HOME/.hermes}/hermes-agent"
|
||||
|
||||
# Add dev/test extras on top of the standard install.
|
||||
uv pip install -e ".[all,dev]"
|
||||
|
||||
# Optional: browser tools / docs site dependencies.
|
||||
npm install
|
||||
```
|
||||
|
||||
After that, create branches and run tests from that checkout:
|
||||
|
||||
```bash
|
||||
git checkout -b fix/description
|
||||
scripts/run_tests.sh
|
||||
```
|
||||
|
||||
You can also run a fully isolated Hermes instance (throwaway HERMES_HOME, separate Electron
|
||||
userData, distinct Electron app name to avoid the single-instance lock):
|
||||
|
||||
```bash
|
||||
scripts/dev-sandbox.sh python -m hermes_cli.main
|
||||
scripts/dev-sandbox.sh --persistent python -m hermes_cli.main desktop # state survives restarts, but lives in the worktree :)
|
||||
```
|
||||
|
||||
### Manual clone fallback
|
||||
|
||||
Use this only if you intentionally do not want Hermes' managed install layout
|
||||
(for example, a throwaway clone inside a container or CI job). If you install
|
||||
this way, make sure you run the `hermes` entrypoint from this venv; running the
|
||||
system `python3 -m hermes_cli.main` can pick up unrelated system Python
|
||||
packages.
|
||||
|
||||
Create the venv **outside** the cloned source tree. A venv that lives inside
|
||||
the directory the agent operates from can be wiped by a relative-path command
|
||||
the agent runs against its own checkout (`rm -rf venv`, `uv venv venv`, etc.),
|
||||
which silently destroys the running runtime mid-session. Keeping it outside the
|
||||
tree means no relative path from the workspace resolves to it.
|
||||
|
||||
```bash
|
||||
git clone https://github.com/NousResearch/hermes-agent.git
|
||||
cd hermes-agent
|
||||
|
||||
# Create venv with Python 3.11, OUTSIDE the source tree
|
||||
uv venv ~/.hermes/venvs/hermes-dev --python 3.11
|
||||
export VIRTUAL_ENV="$HOME/.hermes/venvs/hermes-dev"
|
||||
export PATH="$VIRTUAL_ENV/bin:$PATH"
|
||||
|
||||
# Install with all extras (messaging, cron, CLI menus, dev tools)
|
||||
uv pip install -e ".[all,dev]"
|
||||
|
||||
# Optional: browser tools
|
||||
npm install
|
||||
```
|
||||
|
||||
### Configure for Development
|
||||
|
||||
```bash
|
||||
mkdir -p ~/.hermes/{cron,sessions,logs,memories,skills}
|
||||
cp cli-config.yaml.example ~/.hermes/config.yaml
|
||||
touch ~/.hermes/.env
|
||||
|
||||
# Add at minimum an LLM provider key:
|
||||
echo 'OPENROUTER_API_KEY=sk-or-v1-your-key' >> ~/.hermes/.env
|
||||
```
|
||||
|
||||
### Run
|
||||
|
||||
```bash
|
||||
# The standard installer already put `hermes` on PATH.
|
||||
hermes doctor
|
||||
hermes chat -q "Hello"
|
||||
```
|
||||
|
||||
If you used the manual clone fallback, run `./hermes` from the checkout or
|
||||
symlink this clone's venv explicitly:
|
||||
|
||||
```bash
|
||||
mkdir -p ~/.local/bin
|
||||
ln -sf "$(pwd)/venv/bin/hermes" ~/.local/bin/hermes
|
||||
```
|
||||
|
||||
### Run Tests
|
||||
|
||||
```bash
|
||||
scripts/run_tests.sh
|
||||
```
|
||||
|
||||
## Code Style
|
||||
|
||||
- **PEP 8** with practical exceptions (no strict line length enforcement)
|
||||
- **Comments**: Only when explaining non-obvious intent, trade-offs, or API quirks
|
||||
- **Error handling**: Catch specific exceptions. Use `logger.warning()`/`logger.error()` with `exc_info=True` for unexpected errors
|
||||
- **Cross-platform**: Never assume Unix (see below)
|
||||
- **Profile-safe paths**: Never hardcode `~/.hermes` — use `get_hermes_home()` from `hermes_constants` for code paths and `display_hermes_home()` for user-facing messages. See [AGENTS.md](https://github.com/NousResearch/hermes-agent/blob/main/AGENTS.md#profiles-multi-instance-support) for full rules.
|
||||
|
||||
## Cross-Platform Compatibility
|
||||
|
||||
See **[Platform Support](../getting-started/platform-support.md)**. Native Windows uses Git Bash (from [Git for Windows](https://git-scm.com/download/win)) for shell commands. A few features require POSIX kernel primitives and are gated: the dashboard's embedded PTY terminal pane (`/chat` tab) needs a POSIX PTY (Linux, macOS, or WSL2). If you're doing Windows-heavy dev, run the Windows-footgun lint (`scripts/check-windows-footguns.py`) before pushing.
|
||||
|
||||
When contributing code, keep these rules in mind:
|
||||
|
||||
- **Don't add unguarded `signal.SIGKILL` references.** It's not defined on Windows. Either route through `gateway.status.terminate_pid(pid, force=True)` (the centralized primitive that does `taskkill /T /F` on Windows and SIGKILL on POSIX), or fall back with `getattr(signal, "SIGKILL", signal.SIGTERM)`.
|
||||
- **Catch `OSError` alongside `ProcessLookupError` on `os.kill(pid, 0)` probes.** Windows raises `OSError` (WinError 87, "parameter is incorrect") for an already-gone PID instead of `ProcessLookupError`.
|
||||
- **Don't force the terminal to POSIX semantics.** `os.setsid`, `os.killpg`, `os.getpgid`, `os.fork` all raise on Windows — gate them with `if sys.platform != "win32":` or `if os.name != "nt":`.
|
||||
- **Open files with an explicit `encoding="utf-8"`.** The Python default on Windows is the system locale (often cp1252), which mojibakes or crashes on non-Latin text.
|
||||
- **Use `pathlib.Path` / `os.path.join` — never manually concat with `/`.** This matters less for strings the OS gives us back and more for strings we construct to hand to subprocesses.
|
||||
|
||||
Key patterns:
|
||||
|
||||
### 1. `termios` and `fcntl` are Unix-only
|
||||
|
||||
Always catch both `ImportError` and `NotImplementedError`:
|
||||
|
||||
```python
|
||||
try:
|
||||
from simple_term_menu import TerminalMenu
|
||||
menu = TerminalMenu(options)
|
||||
idx = menu.show()
|
||||
except (ImportError, NotImplementedError):
|
||||
# Fallback: numbered menu
|
||||
for i, opt in enumerate(options):
|
||||
print(f" {i+1}. {opt}")
|
||||
idx = int(input("Choice: ")) - 1
|
||||
```
|
||||
|
||||
### 2. File encoding
|
||||
|
||||
Some environments may save `.env` files in non-UTF-8 encodings:
|
||||
|
||||
```python
|
||||
try:
|
||||
load_dotenv(env_path)
|
||||
except UnicodeDecodeError:
|
||||
load_dotenv(env_path, encoding="latin-1")
|
||||
```
|
||||
|
||||
### 3. Process management
|
||||
|
||||
`os.setsid()`, `os.killpg()`, and signal handling differ across platforms:
|
||||
|
||||
```python
|
||||
import platform
|
||||
if platform.system() != "Windows":
|
||||
kwargs["preexec_fn"] = os.setsid
|
||||
```
|
||||
|
||||
### 4. Path separators
|
||||
|
||||
Use `pathlib.Path` instead of string concatenation with `/`.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
Hermes has terminal access. Security matters.
|
||||
|
||||
### Existing Protections
|
||||
|
||||
| Layer | Implementation |
|
||||
| ------------------------------- | --------------------------------------------------------------------------- |
|
||||
| **Sudo password piping** | Uses `shlex.quote()` to prevent shell injection |
|
||||
| **Dangerous command detection** | Regex patterns in `tools/approval.py` with user approval flow |
|
||||
| **Cron prompt injection** | Scanner blocks instruction-override patterns |
|
||||
| **Write deny list** | Protected paths resolved via `os.path.realpath()` to prevent symlink bypass |
|
||||
| **Skills guard** | Security scanner for hub-installed skills |
|
||||
| **Code execution sandbox** | Child process runs with API keys stripped |
|
||||
| **Container hardening** | Docker: all capabilities dropped, no privilege escalation, PID limits |
|
||||
|
||||
### Contributing Security-Sensitive Code
|
||||
|
||||
- Always use `shlex.quote()` when interpolating user input into shell commands
|
||||
- Resolve symlinks with `os.path.realpath()` before access control checks
|
||||
- Don't log secrets
|
||||
- Catch broad exceptions around tool execution
|
||||
- Test on all platforms if your change touches file paths or processes
|
||||
|
||||
## Pull Request Process
|
||||
|
||||
### Branch Naming
|
||||
|
||||
```
|
||||
fix/description # Bug fixes
|
||||
feat/description # New features
|
||||
docs/description # Documentation
|
||||
test/description # Tests
|
||||
refactor/description # Code restructuring
|
||||
```
|
||||
|
||||
### Before Submitting
|
||||
|
||||
1. **Run tests**: `scripts/run_tests.sh` for CI-parity. Use direct `python -m pytest ...` only when the wrapper is unavailable or you are intentionally debugging outside the wrapper.
|
||||
2. **Test manually**: Run `hermes` and exercise the code path you changed
|
||||
3. **Check cross-platform impact**: Consider macOS, Linux, WSL2, and native Windows. If you touch file I/O, process management, terminal handling, subprocesses, or signals, run `scripts/check-windows-footguns.py`.
|
||||
4. **Keep PRs focused**: One logical change per PR
|
||||
|
||||
### PR Description
|
||||
|
||||
Include:
|
||||
|
||||
- **What** changed and **why**
|
||||
- **How to test** it
|
||||
- **What platforms** you tested on
|
||||
- Reference any related issues
|
||||
|
||||
### Commit Messages
|
||||
|
||||
We use [Conventional Commits](https://www.conventionalcommits.org/):
|
||||
|
||||
```
|
||||
<type>(<scope>): <description>
|
||||
```
|
||||
|
||||
| Type | Use for |
|
||||
| ---------- | ----------------------------- |
|
||||
| `fix` | Bug fixes |
|
||||
| `feat` | New features |
|
||||
| `docs` | Documentation |
|
||||
| `test` | Tests |
|
||||
| `refactor` | Code restructuring |
|
||||
| `chore` | Build, CI, dependency updates |
|
||||
|
||||
Scopes: `cli`, `gateway`, `tools`, `skills`, `agent`, `install`, `whatsapp`, `security`
|
||||
|
||||
Examples:
|
||||
|
||||
```
|
||||
fix(cli): prevent crash in save_config_value when model is a string
|
||||
feat(gateway): add WhatsApp multi-user session isolation
|
||||
fix(security): prevent shell injection in sudo password piping
|
||||
```
|
||||
|
||||
## Reporting Issues
|
||||
|
||||
- Use [GitHub Issues](https://github.com/NousResearch/hermes-agent/issues)
|
||||
- Include: OS, Python version, Hermes version (`hermes version`), full error traceback
|
||||
- Include steps to reproduce
|
||||
- Check existing issues before creating duplicates
|
||||
- For security vulnerabilities, please report privately
|
||||
|
||||
## Community
|
||||
|
||||
- **Discord**: [discord.gg/NousResearch](https://discord.gg/NousResearch)
|
||||
- **GitHub Discussions**: For design proposals and architecture discussions
|
||||
- **Skills Hub**: Upload specialized skills and share with the community
|
||||
|
||||
## License
|
||||
|
||||
By contributing, you agree that your contributions will be licensed under the [MIT License](https://github.com/NousResearch/hermes-agent/blob/main/LICENSE).
|
||||
@@ -0,0 +1,438 @@
|
||||
---
|
||||
sidebar_position: 3
|
||||
title: "Creating Skills"
|
||||
description: "How to create skills for Hermes Agent — SKILL.md format, guidelines, and publishing"
|
||||
---
|
||||
|
||||
# Creating Skills
|
||||
|
||||
Skills are the preferred way to add new capabilities to Hermes Agent. They're easier to create than tools, require no code changes to the agent, and can be shared with the community.
|
||||
|
||||
## Should it be a Skill or a Tool?
|
||||
|
||||
Make it a **Skill** when:
|
||||
- The capability can be expressed as instructions + shell commands + existing tools
|
||||
- It wraps an external CLI or API that the agent can call via `terminal` or `web_extract`
|
||||
- It doesn't need custom Python integration or API key management baked into the agent
|
||||
- Examples: arXiv search, git workflows, Docker management, PDF processing, email via CLI tools
|
||||
|
||||
Make it a **Tool** when:
|
||||
- It requires end-to-end integration with API keys, auth flows, or multi-component configuration
|
||||
- It needs custom processing logic that must execute precisely every time
|
||||
- It handles binary data, streaming, or real-time events
|
||||
- Examples: browser automation, TTS, vision analysis
|
||||
|
||||
## Skill Directory Structure
|
||||
|
||||
Bundled skills live in `skills/` organized by category. Official optional skills use the same structure in `optional-skills/`:
|
||||
|
||||
```text
|
||||
skills/
|
||||
├── research/
|
||||
│ └── arxiv/
|
||||
│ ├── SKILL.md # Required: main instructions
|
||||
│ └── scripts/ # Optional: helper scripts
|
||||
│ └── search_arxiv.py
|
||||
├── productivity/
|
||||
│ └── ocr-and-documents/
|
||||
│ ├── SKILL.md
|
||||
│ ├── scripts/
|
||||
│ └── references/
|
||||
└── ...
|
||||
```
|
||||
|
||||
## SKILL.md Format
|
||||
|
||||
```markdown
|
||||
---
|
||||
name: my-skill
|
||||
description: Brief description (shown in skill search results)
|
||||
version: 1.0.0
|
||||
author: Your Name
|
||||
license: MIT
|
||||
platforms: [macos, linux] # Optional — restrict to specific OS platforms
|
||||
# Valid: macos, linux, windows
|
||||
# Omit to load on all platforms (default)
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Category, Subcategory, Keywords]
|
||||
related_skills: [other-skill-name]
|
||||
requires_toolsets: [web] # Optional — only show when these toolsets are active
|
||||
requires_tools: [web_search] # Optional — only show when these tools are available
|
||||
fallback_for_toolsets: [browser] # Optional — hide when these toolsets are active
|
||||
fallback_for_tools: [browser_navigate] # Optional — hide when these tools exist
|
||||
config: # Optional — config.yaml settings the skill needs
|
||||
- key: my.setting
|
||||
description: "What this setting controls"
|
||||
default: "sensible-default"
|
||||
prompt: "Display prompt for setup"
|
||||
blueprint: # Optional — marks this skill a runnable automation
|
||||
schedule: "0 9 * * *" # cron expr / "every 2h" / ISO timestamp
|
||||
deliver: origin # optional (default origin)
|
||||
prompt: "Task instruction for each run" # optional
|
||||
no_agent: false # optional
|
||||
required_environment_variables: # Optional — env vars the skill needs
|
||||
- name: MY_API_KEY
|
||||
prompt: "Enter your API key"
|
||||
help: "Get one at https://example.com"
|
||||
required_for: "API access"
|
||||
---
|
||||
|
||||
# Skill Title
|
||||
|
||||
Brief intro.
|
||||
|
||||
## When to Use
|
||||
Trigger conditions — when should the agent load this skill?
|
||||
|
||||
## Quick Reference
|
||||
Table of common commands or API calls.
|
||||
|
||||
## Procedure
|
||||
Step-by-step instructions the agent follows.
|
||||
|
||||
## Pitfalls
|
||||
Known failure modes and how to handle them.
|
||||
|
||||
## Verification
|
||||
How the agent confirms it worked.
|
||||
```
|
||||
|
||||
### Platform-Specific Skills
|
||||
|
||||
Skills can restrict themselves to specific operating systems using the `platforms` field:
|
||||
|
||||
```yaml
|
||||
platforms: [macos] # macOS only (e.g., iMessage, Apple Reminders)
|
||||
platforms: [macos, linux] # macOS and Linux
|
||||
platforms: [windows] # Windows only
|
||||
```
|
||||
|
||||
When set, the skill is automatically hidden from the system prompt, `skills_list()`, and slash commands on incompatible platforms. If omitted or empty, the skill loads on all platforms (backward compatible).
|
||||
|
||||
### Conditional Skill Activation
|
||||
|
||||
Skills can declare dependencies on specific tools or toolsets. This controls whether the skill appears in the system prompt for a given session.
|
||||
|
||||
```yaml
|
||||
metadata:
|
||||
hermes:
|
||||
requires_toolsets: [web] # Hide if the web toolset is NOT active
|
||||
requires_tools: [web_search] # Hide if web_search tool is NOT available
|
||||
fallback_for_toolsets: [browser] # Hide if the browser toolset IS active
|
||||
fallback_for_tools: [browser_navigate] # Hide if browser_navigate IS available
|
||||
```
|
||||
|
||||
| Field | Behavior |
|
||||
|-------|----------|
|
||||
| `requires_toolsets` | Skill is **hidden** when ANY listed toolset is **not** available |
|
||||
| `requires_tools` | Skill is **hidden** when ANY listed tool is **not** available |
|
||||
| `fallback_for_toolsets` | Skill is **hidden** when ANY listed toolset **is** available |
|
||||
| `fallback_for_tools` | Skill is **hidden** when ANY listed tool **is** available |
|
||||
|
||||
**Use case for `fallback_for_*`:** Create a skill that serves as a workaround when a primary tool isn't available. For example, a `duckduckgo-search` skill with `fallback_for_tools: [web_search]` only shows when the web search tool (which requires an API key) is not configured.
|
||||
|
||||
**Use case for `requires_*`:** Create a skill that only makes sense when certain tools are present. For example, a web scraping workflow skill with `requires_toolsets: [web]` won't clutter the prompt when web tools are disabled.
|
||||
|
||||
### Environment Variable Requirements
|
||||
|
||||
Skills can declare environment variables they need. When a skill is loaded via `skill_view`, its required vars are automatically registered for passthrough into sandboxed execution environments (terminal, execute_code).
|
||||
|
||||
```yaml
|
||||
required_environment_variables:
|
||||
- name: TENOR_API_KEY
|
||||
prompt: "Tenor API key" # Shown when prompting user
|
||||
help: "Get your key at https://tenor.com" # Help text or URL
|
||||
required_for: "GIF search functionality" # What needs this var
|
||||
```
|
||||
|
||||
Each entry supports:
|
||||
- `name` (required) — the environment variable name
|
||||
- `prompt` (optional) — prompt text when asking the user for the value
|
||||
- `help` (optional) — help text or URL for obtaining the value
|
||||
- `required_for` (optional) — describes which feature needs this variable
|
||||
|
||||
Users can also manually configure passthrough variables in `config.yaml`:
|
||||
|
||||
```yaml
|
||||
terminal:
|
||||
env_passthrough:
|
||||
- MY_CUSTOM_VAR
|
||||
- ANOTHER_VAR
|
||||
```
|
||||
|
||||
See `skills/apple/` for examples of macOS-only skills.
|
||||
|
||||
## Secure Setup on Load
|
||||
|
||||
Use `required_environment_variables` when a skill needs an API key or token. Missing values do **not** hide the skill from discovery. Instead, Hermes prompts for them securely when the skill is loaded in the local CLI.
|
||||
|
||||
```yaml
|
||||
required_environment_variables:
|
||||
- name: TENOR_API_KEY
|
||||
prompt: Tenor API key
|
||||
help: Get a key from https://developers.google.com/tenor
|
||||
required_for: full functionality
|
||||
```
|
||||
|
||||
The user can skip setup and keep loading the skill. Hermes never exposes the raw secret value to the model. Gateway and messaging sessions show local setup guidance instead of collecting secrets in-band.
|
||||
|
||||
:::tip Sandbox Passthrough
|
||||
When your skill is loaded, any declared `required_environment_variables` that are set are **automatically passed through** to `execute_code` and `terminal` sandboxes — including remote backends like Docker and Modal. Your skill's scripts can access `$TENOR_API_KEY` (or `os.environ["TENOR_API_KEY"]` in Python) without the user needing to configure anything extra. See [Environment Variable Passthrough](/user-guide/security#environment-variable-passthrough) for details.
|
||||
:::
|
||||
|
||||
Legacy `prerequisites.env_vars` remains supported as a backward-compatible alias.
|
||||
|
||||
### Config Settings (config.yaml)
|
||||
|
||||
Skills can declare non-secret settings that are stored in `config.yaml` under the `skills.config` namespace. Unlike environment variables (which are secrets stored in `.env`), config settings are for paths, preferences, and other non-sensitive values.
|
||||
|
||||
```yaml
|
||||
metadata:
|
||||
hermes:
|
||||
config:
|
||||
- key: myplugin.path
|
||||
description: Path to the plugin data directory
|
||||
default: "~/myplugin-data"
|
||||
prompt: Plugin data directory path
|
||||
- key: myplugin.domain
|
||||
description: Domain the plugin operates on
|
||||
default: ""
|
||||
prompt: Plugin domain (e.g., AI/ML research)
|
||||
```
|
||||
|
||||
Each entry supports:
|
||||
- `key` (required) — dotpath for the setting (e.g., `myplugin.path`)
|
||||
- `description` (required) — explains what the setting controls
|
||||
- `default` (optional) — default value if the user doesn't configure it
|
||||
- `prompt` (optional) — prompt text shown during `hermes config migrate`; falls back to `description`
|
||||
|
||||
**How it works:**
|
||||
|
||||
1. **Storage:** Values are written to `config.yaml` under `skills.config.<key>`:
|
||||
```yaml
|
||||
skills:
|
||||
config:
|
||||
myplugin:
|
||||
path: ~/my-data
|
||||
```
|
||||
|
||||
2. **Discovery:** `hermes config migrate` scans all enabled skills, finds unconfigured settings, and prompts the user. Settings also appear in `hermes config show` under "Skill Settings."
|
||||
|
||||
3. **Runtime injection:** When a skill loads, its config values are resolved and appended to the skill message:
|
||||
```
|
||||
[Skill config (from ~/.hermes/config.yaml):
|
||||
myplugin.path = /home/user/my-data
|
||||
]
|
||||
```
|
||||
The agent sees the configured values without needing to read `config.yaml` itself.
|
||||
|
||||
4. **Manual setup:** Users can also set values directly:
|
||||
```bash
|
||||
hermes config set skills.config.myplugin.path ~/my-data
|
||||
```
|
||||
|
||||
:::tip When to use which
|
||||
Use `required_environment_variables` for API keys, tokens, and other **secrets** (stored in `~/.hermes/.env`, never shown to the model). Use `config` for **paths, preferences, and non-sensitive settings** (stored in `config.yaml`, visible in config show).
|
||||
:::
|
||||
|
||||
### Credential File Requirements (OAuth tokens, etc.)
|
||||
|
||||
Skills that use OAuth or file-based credentials can declare files that need to be mounted into remote sandboxes. This is for credentials stored as **files** (not env vars) — typically OAuth token files produced by a setup script.
|
||||
|
||||
```yaml
|
||||
required_credential_files:
|
||||
- path: google_token.json
|
||||
description: Google OAuth2 token (created by setup script)
|
||||
- path: google_client_secret.json
|
||||
description: Google OAuth2 client credentials
|
||||
```
|
||||
|
||||
Each entry supports:
|
||||
- `path` (required) — file path relative to `~/.hermes/`
|
||||
- `description` (optional) — explains what the file is and how it's created
|
||||
|
||||
When loaded, Hermes checks if these files exist. Missing files trigger `setup_needed`. Existing files are automatically:
|
||||
- **Mounted into Docker** containers as read-only bind mounts
|
||||
- **Synced into Modal** sandboxes (at creation + before each command, so mid-session OAuth works)
|
||||
- Available on **local** backend without any special handling
|
||||
|
||||
:::tip When to use which
|
||||
Use `required_environment_variables` for simple API keys and tokens (strings stored in `~/.hermes/.env`). Use `required_credential_files` for OAuth token files, client secrets, service account JSON, certificates, or any credential that's a file on disk.
|
||||
:::
|
||||
|
||||
See the `skills/productivity/google-workspace/SKILL.md` for a complete example using both.
|
||||
|
||||
## Skill Guidelines
|
||||
|
||||
### No External Dependencies
|
||||
|
||||
Prefer stdlib Python, curl, and existing Hermes tools (`web_extract`, `terminal`, `read_file`). If a dependency is needed, document installation steps in the skill.
|
||||
|
||||
### Progressive Disclosure
|
||||
|
||||
Put the most common workflow first. Edge cases and advanced usage go at the bottom. This keeps token usage low for common tasks.
|
||||
|
||||
### Include Helper Scripts
|
||||
|
||||
For XML/JSON parsing or complex logic, include helper scripts in `scripts/` — don't expect the LLM to write parsers inline every time.
|
||||
|
||||
### Deliver media as documents (`[[as_document]]`)
|
||||
|
||||
If your skill produces a high-resolution screenshot, chart, or any image where lossy preview compression would hurt — emit the literal directive `[[as_document]]` somewhere in the response (commonly the last line). The gateway strips the directive and delivers every extracted media path in that response as a downloadable file attachment instead of an inline image bubble. See [Skill output and media delivery](../user-guide/features/skills.md#skill-output-and-media-delivery) for the full semantics.
|
||||
|
||||
#### Referencing bundled scripts from SKILL.md
|
||||
|
||||
When a skill is loaded, the activation message exposes the absolute skill directory as `[Skill directory: /abs/path]` and also substitutes two template tokens anywhere in the SKILL.md body:
|
||||
|
||||
| Token | Replaced with |
|
||||
|---|---|
|
||||
| `${HERMES_SKILL_DIR}` | Absolute path to the skill's directory |
|
||||
| `${HERMES_SESSION_ID}` | The active session id (left in place if there is no session) |
|
||||
|
||||
So a SKILL.md can tell the agent to run a bundled script directly with:
|
||||
|
||||
```markdown
|
||||
To analyse the input, run:
|
||||
|
||||
node ${HERMES_SKILL_DIR}/scripts/analyse.js <input>
|
||||
```
|
||||
|
||||
The agent sees the substituted absolute path and invokes the `terminal` tool with a ready-to-run command — no path math, no extra `skill_view` round-trip. Disable substitution globally with `skills.template_vars: false` in `config.yaml`.
|
||||
|
||||
#### Inline shell snippets (opt-in)
|
||||
|
||||
Skills can also embed inline shell snippets written as `` !`cmd` `` in the SKILL.md body. When enabled, each snippet's stdout is inlined into the message before the agent reads it, so skills can inject dynamic context:
|
||||
|
||||
```markdown
|
||||
Current date: !`date -u +%Y-%m-%d`
|
||||
Git branch: !`git -C ${HERMES_SKILL_DIR} rev-parse --abbrev-ref HEAD`
|
||||
```
|
||||
|
||||
This is **off by default** — any snippet in a SKILL.md runs on the host without approval, so only enable it for skill sources you trust:
|
||||
|
||||
```yaml
|
||||
# config.yaml
|
||||
skills:
|
||||
inline_shell: true
|
||||
inline_shell_timeout: 10 # seconds per snippet
|
||||
```
|
||||
|
||||
Snippets run with the skill directory as their working directory, and output is capped at 4000 characters. Failures (timeouts, non-zero exits) show up as a short `[inline-shell error: ...]` marker instead of breaking the whole skill.
|
||||
|
||||
### Test It
|
||||
|
||||
Run the skill and verify the agent follows the instructions correctly:
|
||||
|
||||
```bash
|
||||
hermes chat --toolsets skills -q "Use the X skill to do Y"
|
||||
```
|
||||
|
||||
## Where Should the Skill Live?
|
||||
|
||||
Bundled skills (in `skills/`) ship with every Hermes install. They should be **broadly useful to most users**:
|
||||
|
||||
- Document handling, web research, common dev workflows, system administration
|
||||
- Used regularly by a wide range of people
|
||||
|
||||
If your skill is official and useful but not universally needed (e.g., a paid service integration, a heavyweight dependency), put it in **`optional-skills/`** — it ships with the repo, is discoverable via `hermes skills browse` (labeled "official"), and installs with built-in trust.
|
||||
|
||||
If your skill is specialized, community-contributed, or niche, it's better suited for a **Skills Hub** — upload it to a registry and share it via `hermes skills install`.
|
||||
|
||||
## Blueprints: skills that are also automations
|
||||
|
||||
A **blueprint** is an ordinary skill that additionally declares a schedule in its frontmatter. Add a `metadata.hermes.blueprint` block and the skill becomes a shareable, runnable automation:
|
||||
|
||||
```yaml
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [blueprint, email]
|
||||
blueprint:
|
||||
schedule: "0 8 * * *" # presence of `blueprint:` marks it runnable
|
||||
deliver: telegram # optional (default: origin)
|
||||
prompt: "Summarize my unread email and today's calendar." # optional
|
||||
no_agent: false # optional
|
||||
```
|
||||
|
||||
Because a blueprint **is** a skill, it flows through the entire skills pipeline unchanged — search, inspect, install, security scan, provenance, taps, the centralized index, and `hermes skills publish` for sharing. Nothing new to learn.
|
||||
|
||||
**Installing a blueprint.** When you install a skill that carries a `blueprint:` block, Hermes registers it as a **suggested cron job** rather than scheduling it. Scheduling is **opt-in** — installing never silently creates a recurring job. You review and accept it via `/suggestions`:
|
||||
|
||||
```bash
|
||||
hermes skills install owner/morning-brief
|
||||
# → Blueprint: 'morning-brief' is an automation (schedule 0 8 * * *).
|
||||
# Added to your suggestions — run /suggestions to schedule or dismiss it.
|
||||
|
||||
# then, in a session:
|
||||
/suggestions # lists pending suggestions, numbered
|
||||
/suggestions accept 1 # creates the cron job
|
||||
/suggestions dismiss 1 # never offer it again
|
||||
```
|
||||
|
||||
Blueprints are one **source** of the unified Suggested Cron Jobs surface — the same place curated starter automations and (later) usage-pattern and integration suggestions appear. See [Suggested Cron Jobs](#suggested-cron-jobs) below.
|
||||
|
||||
**Sharing an automation you built.** A blueprint loaded by a cron job (`hermes cron create --skill <name> ...`) can be exported back to a SKILL.md and published like any other skill, so an automation you tuned for yourself becomes a one-command install for someone else.
|
||||
|
||||
The blueprint layer adds no new object type, store, or transport — the blueprint is a skill, the schedule is a cron job, and sharing is the existing publish/tap/index path.
|
||||
|
||||
## Suggested Cron Jobs
|
||||
|
||||
Hermes can *propose* automations and let you accept them with one tap, instead of making you assemble cron jobs by hand. Every proposal flows through one surface — the `/suggestions` command — regardless of where it came from:
|
||||
|
||||
| Source | Trigger |
|
||||
|--------|---------|
|
||||
| `catalog` | Curated starter automations (`/suggestions catalog`) — daily briefing, important-mail monitor, weekly review, workday-start reminder |
|
||||
| `blueprint` | You installed a skill carrying a `blueprint:` block |
|
||||
| `usage` | The background review noticed a recurring ask a schedule would serve |
|
||||
| `integration` | You connected an account (Gmail, GitHub, ...) and the obvious automations are offered |
|
||||
|
||||
```bash
|
||||
/suggestions # list pending
|
||||
/suggestions accept N # schedule suggestion N (creates the cron job)
|
||||
/suggestions dismiss N # dismiss it — latched, never re-offered
|
||||
/suggestions catalog # add the curated starter automations
|
||||
```
|
||||
|
||||
Accepting a suggestion calls the same `cron.jobs.create_job` the `cronjob` tool uses — there is no second job engine. Suggestions **never** auto-create jobs; acceptance is always explicit. Dismissed suggestions latch by a stable key so the same proposal is never re-offered. The pending list is capped so it never becomes a nag wall.
|
||||
|
||||
The **important-mail monitor** catalog entry is the poll→classify→surface pattern: it scores inbox items with a cheap classifier model (`auxiliary.monitor` in `config.yaml`) and delivers only the ones above an urgency threshold, staying silent otherwise.
|
||||
|
||||
## Publishing Skills
|
||||
|
||||
### To the Skills Hub
|
||||
|
||||
```bash
|
||||
hermes skills publish skills/my-skill --to github --repo owner/repo
|
||||
```
|
||||
|
||||
### To a Custom Repository
|
||||
|
||||
Add your repo as a tap:
|
||||
|
||||
```bash
|
||||
hermes skills tap add owner/repo
|
||||
```
|
||||
|
||||
Users can then search and install from your repository.
|
||||
|
||||
## Security Scanning
|
||||
|
||||
All hub-installed skills go through a security scanner that checks for:
|
||||
|
||||
- Data exfiltration patterns
|
||||
- Prompt injection attempts
|
||||
- Destructive commands
|
||||
- Shell injection
|
||||
|
||||
Trust levels:
|
||||
- `builtin` — ships with Hermes (always trusted)
|
||||
- `official` — from `optional-skills/` in the repo (built-in trust, no third-party warning)
|
||||
- `trusted` — from openai/skills, anthropics/skills, huggingface/skills
|
||||
- `community` — non-dangerous findings can be overridden with `--force`; `dangerous` verdicts remain blocked
|
||||
|
||||
Hermes can now consume third-party skills from multiple external discovery models:
|
||||
- direct GitHub identifiers (for example `openai/skills/k8s`)
|
||||
- `skills.sh` identifiers (for example `skills-sh/vercel-labs/json-render/json-render-react`)
|
||||
- well-known endpoints served from `/.well-known/skills/index.json`
|
||||
|
||||
If you want your skills to be discoverable without a GitHub-specific installer, consider serving them from a well-known endpoint in addition to publishing them in a repo or marketplace.
|
||||
@@ -0,0 +1,303 @@
|
||||
---
|
||||
sidebar_position: 11
|
||||
title: "Cron Internals"
|
||||
description: "How Hermes stores, schedules, edits, pauses, skill-loads, and delivers cron jobs"
|
||||
---
|
||||
|
||||
# Cron Internals
|
||||
|
||||
The cron subsystem provides scheduled task execution — from simple one-shot delays to recurring cron-expression jobs with skill injection and cross-platform delivery.
|
||||
|
||||
## Key Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `cron/jobs.py` | Job model, storage, atomic read/write to `jobs.json` |
|
||||
| `cron/scheduler.py` | Scheduler loop — due-job detection, execution, repeat tracking |
|
||||
| `tools/cronjob_tools.py` | Model-facing `cronjob` tool registration and handler |
|
||||
| `gateway/run.py` | Gateway integration — cron ticking in the long-running loop |
|
||||
| `hermes_cli/cron.py` | CLI `hermes cron` subcommands |
|
||||
|
||||
## Scheduling Model
|
||||
|
||||
Four schedule formats are supported:
|
||||
|
||||
| Format | Example | Behavior |
|
||||
|--------|---------|----------|
|
||||
| **Relative delay** | `30m`, `2h`, `1d` | One-shot, fires after the specified duration |
|
||||
| **Interval** | `every 2h`, `every 30m` | Recurring, fires at regular intervals |
|
||||
| **Cron expression** | `0 9 * * *` | Standard 5-field cron syntax (minute, hour, day, month, weekday) |
|
||||
| **ISO timestamp** | `2025-01-15T09:00:00` | One-shot, fires at the exact time |
|
||||
|
||||
The model-facing surface is a single `cronjob` tool with action-style operations: `create`, `list`, `update`, `pause`, `resume`, `run`, `remove`.
|
||||
|
||||
## Job Storage
|
||||
|
||||
Jobs are stored in `~/.hermes/cron/jobs.json` with atomic write semantics (write to temp file, then rename). Each job record contains:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "a1b2c3d4e5f6",
|
||||
"name": "Daily briefing",
|
||||
"prompt": "Summarize today's AI news and funding rounds",
|
||||
"schedule": {
|
||||
"kind": "cron",
|
||||
"expr": "0 9 * * *",
|
||||
"display": "0 9 * * *"
|
||||
},
|
||||
"skills": ["ai-funding-daily-report"],
|
||||
"deliver": "telegram:-1001234567890",
|
||||
"repeat": {
|
||||
"times": null,
|
||||
"completed": 42
|
||||
},
|
||||
"state": "scheduled",
|
||||
"enabled": true,
|
||||
"next_run_at": "2025-01-16T09:00:00Z",
|
||||
"last_run_at": "2025-01-15T09:00:00Z",
|
||||
"last_status": "ok",
|
||||
"created_at": "2025-01-01T00:00:00Z",
|
||||
"model": null,
|
||||
"provider": null,
|
||||
"script": null
|
||||
}
|
||||
```
|
||||
|
||||
### Job Lifecycle States
|
||||
|
||||
| State | Meaning |
|
||||
|-------|---------|
|
||||
| `scheduled` | Active, will fire at next scheduled time |
|
||||
| `paused` | Suspended — won't fire until resumed |
|
||||
| `completed` | Repeat count exhausted or one-shot that has fired |
|
||||
| `running` | Currently executing (transient state) |
|
||||
|
||||
### Backward Compatibility
|
||||
|
||||
Older jobs may have a single `skill` field instead of the `skills` array. The scheduler normalizes this at load time — single `skill` is promoted to `skills: [skill]`.
|
||||
|
||||
## Scheduler Runtime
|
||||
|
||||
### Tick Cycle
|
||||
|
||||
The scheduler runs on a periodic tick (default: every 60 seconds):
|
||||
|
||||
```text
|
||||
tick()
|
||||
1. Acquire scheduler lock (prevents overlapping ticks)
|
||||
2. Load all jobs from jobs.json
|
||||
3. Filter to due jobs (next_run <= now AND state == "scheduled")
|
||||
4. For each due job:
|
||||
a. Set state to "running"
|
||||
b. Create fresh AIAgent session (no conversation history)
|
||||
c. Load attached skills in order (injected as user messages)
|
||||
d. Run the job prompt through the agent
|
||||
e. Deliver the response to the configured target
|
||||
f. Update run_count, compute next_run
|
||||
g. If repeat count exhausted → state = "completed"
|
||||
h. Otherwise → state = "scheduled"
|
||||
5. Write updated jobs back to jobs.json
|
||||
6. Release scheduler lock
|
||||
```
|
||||
|
||||
### Gateway Integration
|
||||
|
||||
In gateway mode, the cron **trigger** (the part that decides *when* a due job
|
||||
fires — "Axis B") is selected through a pluggable `CronScheduler` provider. The
|
||||
gateway calls `resolve_cron_scheduler()` (`cron/scheduler_provider.py`) and runs
|
||||
the resolved provider's `start()` in a dedicated background thread, alongside a
|
||||
separate gateway-housekeeping thread.
|
||||
|
||||
The active provider is chosen by the `cron.provider` config key:
|
||||
|
||||
- **empty (default)** → the built-in `InProcessCronScheduler`, which runs the
|
||||
historical in-process loop calling `scheduler.tick()` every 60 seconds. This
|
||||
is byte-identical to the pre-provider behavior.
|
||||
- **a named provider** (e.g. `chronos`, a managed-cron provider for
|
||||
scale-to-zero deployments) → discovered from `plugins/cron/<name>/` or
|
||||
`$HERMES_HOME/plugins/<name>/`.
|
||||
|
||||
If a named provider is missing, fails to load, or reports `is_available() ==
|
||||
False`, the resolver falls back to the built-in with a warning — **cron is
|
||||
never left without a trigger.** The built-in provider lives in core
|
||||
(`cron/scheduler_provider.py`), not in `plugins/`, so the fallback can't be
|
||||
accidentally removed.
|
||||
|
||||
What "firing" *means* (job execution + delivery) is unchanged and shared by all
|
||||
providers — it stays in `scheduler.run_job()` / `scheduler._deliver_result()`.
|
||||
A provider only controls the trigger, never execution.
|
||||
|
||||
In CLI mode, cron jobs only fire when `hermes cron` commands are run or during active CLI sessions.
|
||||
|
||||
### Managed cron (Chronos) for scale-to-zero
|
||||
|
||||
Hosted gateways can run the **Chronos** provider (`cron.provider: chronos`)
|
||||
instead of the built-in ticker. Chronos lets an idle gateway **scale to zero**
|
||||
and still fire cron jobs: rather than a 60-second in-process loop (which would
|
||||
keep the process awake), it asks Nous infrastructure to arm exactly **one
|
||||
managed one-shot per job at that job's real next-fire time**. At fire time Nous
|
||||
calls the gateway back over an authenticated webhook (`POST /api/cron/fire`);
|
||||
the gateway runs the job through the same `run_one_job` path as the built-in,
|
||||
then re-arms the next one-shot. Between fires the process can be fully stopped —
|
||||
it wakes only on a genuine fire, never on a periodic timer.
|
||||
|
||||
The flow (the managed scheduler is provided by Nous; the agent holds no
|
||||
scheduler credentials):
|
||||
|
||||
```
|
||||
create/update a cron job
|
||||
→ Chronos asks Nous to arm a one-shot at the job's next_run_at
|
||||
(authenticated with the agent's existing Nous token)
|
||||
→ at fire time Nous calls the gateway: POST {callback_url}/api/cron/fire
|
||||
(authenticated with a short-lived, purpose-scoped Nous-minted JWT)
|
||||
→ the gateway verifies the token, claims the job (store compare-and-set so
|
||||
multi-replica deployments fire at-most-once), runs it, and re-arms the next
|
||||
one-shot
|
||||
```
|
||||
|
||||
Config (all non-secret; on hosted agents Nous sets these at provision time):
|
||||
|
||||
| key | meaning |
|
||||
|---|---|
|
||||
| `cron.provider` | `chronos` to activate (empty = built-in ticker) |
|
||||
| `cron.chronos.portal_url` | Nous base URL (arming + the fire-token issuer) |
|
||||
| `cron.chronos.callback_url` | the gateway's own public base URL for inbound fires |
|
||||
| `cron.chronos.expected_audience` | this agent's fire-token audience |
|
||||
| `cron.chronos.nas_jwks_url` | key set for verifying the inbound fire token |
|
||||
|
||||
If Chronos is misconfigured or the agent isn't logged into Nous,
|
||||
`resolve_cron_scheduler()` falls back to the built-in ticker (logged warning) —
|
||||
cron never loses its trigger. Recurring jobs re-arm after each fire; `repeat`-N
|
||||
jobs stop cleanly when the count is exhausted (no orphaned one-shot). The full
|
||||
agent↔Nous wire contract lives in `docs/chronos-managed-cron-contract.md`.
|
||||
|
||||
### Fresh Session Isolation
|
||||
|
||||
Each cron job runs in a completely fresh agent session:
|
||||
|
||||
- No conversation history from previous runs
|
||||
- No memory of previous cron executions (unless persisted to memory/files)
|
||||
- The prompt must be self-contained — cron jobs cannot ask clarifying questions
|
||||
- The `cronjob` toolset is disabled (recursion guard)
|
||||
|
||||
## Skill-Backed Jobs
|
||||
|
||||
A cron job can attach one or more skills via the `skills` field. At execution time:
|
||||
|
||||
1. Skills are loaded in the specified order
|
||||
2. Each skill's SKILL.md content is injected as context
|
||||
3. The job's prompt is appended as the task instruction
|
||||
4. The agent processes the combined skill context + prompt
|
||||
|
||||
This enables reusable, tested workflows without pasting full instructions into cron prompts. For example:
|
||||
|
||||
```
|
||||
Create a daily funding report → attach "ai-funding-daily-report" skill
|
||||
```
|
||||
|
||||
### Script-Backed Jobs
|
||||
|
||||
Jobs can also attach a Python script via the `script` field. The script runs *before* each agent turn, and its stdout is injected into the prompt as context. This enables data collection and change detection patterns:
|
||||
|
||||
```python
|
||||
# ~/.hermes/scripts/check_competitors.py
|
||||
import requests, json
|
||||
# Fetch competitor release notes, diff against last run
|
||||
# Print summary to stdout — agent analyzes and reports
|
||||
```
|
||||
|
||||
The script timeout defaults to 3600 seconds (1 hour). `_get_script_timeout()` resolves the limit through a three-layer chain:
|
||||
|
||||
1. **Module-level override** — `_SCRIPT_TIMEOUT` (for tests/monkeypatching). Only used when it differs from the default.
|
||||
2. **Environment variable** — `HERMES_CRON_SCRIPT_TIMEOUT`
|
||||
3. **Config** — `cron.script_timeout_seconds` in `config.yaml` (read via `load_config()`)
|
||||
4. **Default** — 3600 seconds (1 hour)
|
||||
|
||||
This timeout bounds the **pre-run script only**, not the agent. Skill-based / LLM-driven jobs run on a separate *inactivity*-based budget (`HERMES_CRON_TIMEOUT`, default 600s of idle time, `0` = unlimited) — they can run for hours as long as they keep calling tools or streaming tokens, and are only killed after the configured idle period with no activity. Scripts are dispatched to a persistent thread pool (not held under the tick lock), so a long-running script does not block other due jobs from firing.
|
||||
|
||||
### Provider Recovery
|
||||
|
||||
`run_job()` passes the user's configured fallback providers and credential pool into the `AIAgent` instance:
|
||||
|
||||
- **Fallback providers** — reads `fallback_providers` (list) or `fallback_model` (legacy dict) from `config.yaml`, matching the gateway's `_load_fallback_model()` pattern. Passed as `fallback_model=` to `AIAgent.__init__`, which normalizes both formats into a fallback chain.
|
||||
- **Credential pool** — loads via `load_pool(provider)` from `agent.credential_pool` using the resolved runtime provider name. Only passed when the pool has credentials (`pool.has_credentials()`). Enables same-provider key rotation on 429/rate-limit errors.
|
||||
|
||||
This mirrors the gateway's behavior — without it, cron agents would fail on rate limits without attempting recovery.
|
||||
|
||||
## Delivery Model
|
||||
|
||||
Cron job results can be delivered to any supported platform.
|
||||
|
||||
A bare platform name (`slack`, `telegram`, …) delivers to that platform's configured **home channel**. To target a **specific** destination instead, append a target after a colon: `platform:<target>`. The target is resolved at fire time (not when the job is created), so a job can name a destination on a platform that isn't connected yet and start delivering once it comes online.
|
||||
|
||||
Most platforms also accept an optional thread/topic as a third segment: `platform:<chat_id>:<thread_id>`.
|
||||
|
||||
| Target | Syntax | Example |
|
||||
|--------|--------|---------|
|
||||
| Origin chat | `origin` | Deliver to the chat where the job was created |
|
||||
| Local file | `local` | Save to `~/.hermes/cron/output/` |
|
||||
| Telegram | `telegram`, `telegram:<chat_id>`, `telegram:<chat_id>:<thread_id>`, `telegram:@username` | `telegram:-1001234567890:17585` |
|
||||
| Discord | `discord`, `discord:#channel`, `discord:<channel_id>`, `discord:<channel_id>:<thread_id>` | `discord:#engineering` |
|
||||
| Slack | `slack`, `slack:#channel`, `slack:<channel_id>`, `slack:<channel_id>:<thread_ts>` | `slack:#engineering` |
|
||||
| Matrix | `matrix`, `matrix:<!room_id:server>`, `matrix:<@user:server>` | `matrix:!abc123:example.org` |
|
||||
| Feishu | `feishu`, `feishu:<chat_id>`, `feishu:<chat_id>:<thread_id>` | `feishu:oc_abc123def` |
|
||||
| WhatsApp | `whatsapp`, `whatsapp:<jid>`, `whatsapp:+<E.164>` | `whatsapp:123456@g.us` |
|
||||
| Signal | `signal`, `signal:group:<id>`, `signal:+<E.164>` | `signal:group:aBcD==` |
|
||||
| SMS | `sms`, `sms:+<E.164>` | `sms:+<E.164 number>` |
|
||||
| Email | `email`, `email:<address>` | `email:alerts@example.com` |
|
||||
| Weixin | `weixin`, `weixin:<wxid>` | `weixin:wxid_abc123` |
|
||||
| Mattermost | `mattermost` or `mattermost:<channel_id>` | Bare name delivers to Mattermost home |
|
||||
| Home Assistant | `homeassistant` or `homeassistant:<conversation>` | Bare name delivers to HA conversation |
|
||||
| DingTalk | `dingtalk` or `dingtalk:<chat_id>` | Bare name delivers to DingTalk |
|
||||
| WeCom | `wecom` or `wecom:<chat_id>` | Bare name delivers to WeCom |
|
||||
| BlueBubbles | `bluebubbles` or `bluebubbles:<chat_guid>` | Bare name delivers to iMessage via BlueBubbles |
|
||||
| QQ Bot | `qqbot` or `qqbot:<chat_id>` | Bare name delivers to QQ (Tencent) via Official API v2 |
|
||||
|
||||
Platforms in the first group have explicit, validated target syntax — named channels (`#channel`), topics/threads, room/user IDs, group IDs, or phone numbers. The remaining platforms accept the generic `platform:<chat_id>` form (the value after the colon is used verbatim as the destination ID); a bare platform name always delivers to the home channel.
|
||||
|
||||
**Named channels** (`slack:#engineering`, `discord:#engineering`, or a friendly name like `slack:engineering`) are resolved against the channel directory the gateway builds from connected adapters, so the gateway must have discovered the channel for name resolution to succeed; raw IDs (`slack:C0123ABCD45`) always work.
|
||||
|
||||
For **Telegram topics**, use `telegram:<chat_id>:<thread_id>` (e.g., `telegram:-1001234567890:17585`). For **Slack threads**, the third segment is the parent message's `thread_ts` (e.g., `slack:C0123ABCD45:1700000000.000100`), so it only applies when replying under an existing message.
|
||||
|
||||
### Response Wrapping
|
||||
|
||||
By default (`cron.wrap_response: true`), cron deliveries are wrapped with:
|
||||
- A header identifying the cron job name and task
|
||||
- A footer noting the agent cannot see the delivered message in conversation
|
||||
|
||||
The `[SILENT]` prefix in a cron response suppresses delivery entirely — useful for jobs that only need to write to files or perform side effects.
|
||||
|
||||
### Session Isolation
|
||||
|
||||
Cron deliveries are NOT mirrored into gateway session conversation history. They exist only in the cron job's own session. This prevents message alternation violations in the target chat's conversation.
|
||||
|
||||
## Recursion Guard
|
||||
|
||||
Cron-run sessions have the `cronjob` toolset disabled. This prevents:
|
||||
- A scheduled job from creating new cron jobs
|
||||
- Recursive scheduling that could explode token usage
|
||||
- Accidental mutation of the job schedule from within a job
|
||||
|
||||
## Locking
|
||||
|
||||
The scheduler uses cross-process file-based locking (`fcntl.flock` on Unix, `msvcrt.locking` on Windows) to prevent overlapping ticks from executing the same due-job batch twice — even between the gateway's in-process ticker and a standalone `hermes cron` / manual `tick()` call. If the lock cannot be acquired, `tick()` returns 0 immediately.
|
||||
|
||||
## CLI Interface
|
||||
|
||||
The `hermes cron` CLI provides direct job management:
|
||||
|
||||
```bash
|
||||
hermes cron list # Show all jobs
|
||||
hermes cron create # Interactive job creation (alias: add)
|
||||
hermes cron edit <job_id> # Edit job configuration
|
||||
hermes cron pause <job_id> # Pause a running job
|
||||
hermes cron resume <job_id> # Resume a paused job
|
||||
hermes cron run <job_id> # Trigger immediate execution
|
||||
hermes cron remove <job_id> # Delete a job
|
||||
```
|
||||
|
||||
## Related Docs
|
||||
|
||||
- [Cron Feature Guide](/user-guide/features/cron)
|
||||
- [Gateway Internals](./gateway-internals.md)
|
||||
- [Agent Loop Internals](./agent-loop.md)
|
||||
@@ -0,0 +1,192 @@
|
||||
---
|
||||
sidebar_position: 8
|
||||
title: "Extending the CLI"
|
||||
description: "Build wrapper CLIs that extend the Hermes TUI with custom widgets, keybindings, and layout changes"
|
||||
---
|
||||
|
||||
# Extending the CLI
|
||||
|
||||
Hermes exposes protected extension hooks on `HermesCLI` so wrapper CLIs can add widgets, keybindings, and layout customizations without overriding the 1000+ line `run()` method. This keeps your extension decoupled from internal changes.
|
||||
|
||||
## Extension points
|
||||
|
||||
There are five extension seams available:
|
||||
|
||||
| Hook | Purpose | Override when... |
|
||||
|------|---------|------------------|
|
||||
| `_get_extra_tui_widgets()` | Inject widgets into the layout | You need a persistent UI element (panel, status line, mini-player) |
|
||||
| `_register_extra_tui_keybindings(kb, *, input_area)` | Add keyboard shortcuts | You need hotkeys (toggle panels, transport controls, modal shortcuts) |
|
||||
| `_build_tui_layout_children(**widgets)` | Full control over widget ordering | You need to reorder or wrap existing widgets (rare) |
|
||||
| `process_command()` | Add custom slash commands | You need `/mycommand` handling (pre-existing hook) |
|
||||
| `_build_tui_style_dict()` | Custom prompt_toolkit styles | You need custom colors or styling (pre-existing hook) |
|
||||
|
||||
The first three are new protected hooks. The last two already existed.
|
||||
|
||||
## Quick start: a wrapper CLI
|
||||
|
||||
```python
|
||||
#!/usr/bin/env python3
|
||||
"""my_cli.py — Example wrapper CLI that extends Hermes."""
|
||||
|
||||
from cli import HermesCLI
|
||||
from prompt_toolkit.layout import FormattedTextControl, Window
|
||||
from prompt_toolkit.filters import Condition
|
||||
|
||||
|
||||
class MyCLI(HermesCLI):
|
||||
|
||||
def __init__(self, **kwargs):
|
||||
super().__init__(**kwargs)
|
||||
self._panel_visible = False
|
||||
|
||||
def _get_extra_tui_widgets(self):
|
||||
"""Add a toggleable info panel above the status bar."""
|
||||
cli_ref = self
|
||||
return [
|
||||
Window(
|
||||
FormattedTextControl(lambda: "📊 My custom panel content"),
|
||||
height=1,
|
||||
filter=Condition(lambda: cli_ref._panel_visible),
|
||||
),
|
||||
]
|
||||
|
||||
def _register_extra_tui_keybindings(self, kb, *, input_area):
|
||||
"""F2 toggles the custom panel."""
|
||||
cli_ref = self
|
||||
|
||||
@kb.add("f2")
|
||||
def _toggle_panel(event):
|
||||
cli_ref._panel_visible = not cli_ref._panel_visible
|
||||
|
||||
def process_command(self, cmd: str) -> bool:
|
||||
"""Add a /panel slash command."""
|
||||
if cmd.strip().lower() == "/panel":
|
||||
self._panel_visible = not self._panel_visible
|
||||
state = "visible" if self._panel_visible else "hidden"
|
||||
print(f"Panel is now {state}")
|
||||
return True
|
||||
return super().process_command(cmd)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
cli = MyCLI()
|
||||
cli.run()
|
||||
```
|
||||
|
||||
Run it:
|
||||
|
||||
```bash
|
||||
cd ~/.hermes/hermes-agent
|
||||
source .venv/bin/activate
|
||||
python my_cli.py
|
||||
```
|
||||
|
||||
## Hook reference
|
||||
|
||||
### `_get_extra_tui_widgets()`
|
||||
|
||||
Returns a list of prompt_toolkit widgets to insert into the TUI layout. Widgets appear **between the spacer and the status bar** — above the input area but below the main output.
|
||||
|
||||
```python
|
||||
def _get_extra_tui_widgets(self) -> list:
|
||||
return [] # default: no extra widgets
|
||||
```
|
||||
|
||||
Each widget should be a prompt_toolkit container (e.g., `Window`, `ConditionalContainer`, `HSplit`). Use `ConditionalContainer` or `filter=Condition(...)` to make widgets toggleable.
|
||||
|
||||
```python
|
||||
from prompt_toolkit.layout import ConditionalContainer, Window, FormattedTextControl
|
||||
from prompt_toolkit.filters import Condition
|
||||
|
||||
def _get_extra_tui_widgets(self):
|
||||
return [
|
||||
ConditionalContainer(
|
||||
Window(FormattedTextControl("Status: connected"), height=1),
|
||||
filter=Condition(lambda: self._show_status),
|
||||
),
|
||||
]
|
||||
```
|
||||
|
||||
### `_register_extra_tui_keybindings(kb, *, input_area)`
|
||||
|
||||
Called after Hermes registers its own keybindings and before the layout is built. Add your keybindings to `kb`.
|
||||
|
||||
```python
|
||||
def _register_extra_tui_keybindings(self, kb, *, input_area):
|
||||
pass # default: no extra keybindings
|
||||
```
|
||||
|
||||
Parameters:
|
||||
- **`kb`** — The `KeyBindings` instance for the prompt_toolkit application
|
||||
- **`input_area`** — The main `TextArea` widget, if you need to read or manipulate user input
|
||||
|
||||
```python
|
||||
def _register_extra_tui_keybindings(self, kb, *, input_area):
|
||||
cli_ref = self
|
||||
|
||||
@kb.add("f3")
|
||||
def _clear_input(event):
|
||||
input_area.text = ""
|
||||
|
||||
@kb.add("f4")
|
||||
def _insert_template(event):
|
||||
input_area.text = "/search "
|
||||
```
|
||||
|
||||
**Avoid conflicts** with built-in keybindings: `Enter` (submit), `Escape Enter` (newline), `Ctrl-C` (interrupt), `Ctrl-D` (exit), `Tab` (auto-suggest accept). Function keys F2+ and Ctrl-combinations are generally safe.
|
||||
|
||||
### `_build_tui_layout_children(**widgets)`
|
||||
|
||||
Override this only when you need full control over widget ordering. Most extensions should use `_get_extra_tui_widgets()` instead.
|
||||
|
||||
```python
|
||||
def _build_tui_layout_children(self, *, sudo_widget, secret_widget,
|
||||
approval_widget, clarify_widget, model_picker_widget=None,
|
||||
spinner_widget=None, spacer, status_bar, input_rule_top,
|
||||
image_bar, input_area, input_rule_bot, voice_status_bar,
|
||||
completions_menu) -> list:
|
||||
```
|
||||
|
||||
The default implementation returns (any `None` widgets are filtered out):
|
||||
|
||||
```python
|
||||
[
|
||||
Window(height=0), # anchor
|
||||
sudo_widget, # sudo password prompt (conditional)
|
||||
secret_widget, # secret input prompt (conditional)
|
||||
approval_widget, # dangerous command approval (conditional)
|
||||
clarify_widget, # clarify question UI (conditional)
|
||||
model_picker_widget, # model picker overlay (conditional)
|
||||
spinner_widget, # thinking spinner (conditional)
|
||||
spacer, # fills remaining vertical space
|
||||
*self._get_extra_tui_widgets(), # YOUR WIDGETS GO HERE
|
||||
status_bar, # model/token/context status line
|
||||
input_rule_top, # ─── border above input
|
||||
image_bar, # attached images indicator
|
||||
input_area, # user text input
|
||||
input_rule_bot, # ─── border below input
|
||||
voice_status_bar, # voice mode status (conditional)
|
||||
completions_menu, # autocomplete dropdown
|
||||
]
|
||||
```
|
||||
|
||||
## Layout diagram
|
||||
|
||||
The default layout from top to bottom:
|
||||
|
||||
1. **Output area** — scrolling conversation history
|
||||
2. **Spacer**
|
||||
3. **Extra widgets** — from `_get_extra_tui_widgets()`
|
||||
4. **Status bar** — model, context %, elapsed time
|
||||
5. **Image bar** — attached image count
|
||||
6. **Input area** — user prompt
|
||||
7. **Voice status** — recording indicator
|
||||
8. **Completions menu** — autocomplete suggestions
|
||||
|
||||
## Tips
|
||||
|
||||
- **Invalidate the display** after state changes: call `self._invalidate()` to trigger a prompt_toolkit redraw.
|
||||
- **Access agent state**: `self.agent`, `self.model`, `self.conversation_history` are all available.
|
||||
- **Custom styles**: Override `_build_tui_style_dict()` and add entries for your custom style classes.
|
||||
- **Slash commands**: Override `process_command()`, handle your commands, and call `super().process_command(cmd)` for everything else.
|
||||
- **Don't override `run()`** unless absolutely necessary — the extension hooks exist specifically to avoid that coupling.
|
||||
@@ -0,0 +1,269 @@
|
||||
---
|
||||
sidebar_position: 7
|
||||
title: "Gateway Internals"
|
||||
description: "How the messaging gateway boots, authorizes users, routes sessions, and delivers messages"
|
||||
---
|
||||
|
||||
# Gateway Internals
|
||||
|
||||
The messaging gateway is the long-running process that connects Hermes to 20+ external messaging platforms through a unified architecture.
|
||||
|
||||
## Key Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `gateway/run.py` | `GatewayRunner` — main loop, slash commands, message dispatch (large file; check git for current LOC) |
|
||||
| `gateway/session.py` | `SessionStore` — conversation persistence and session key construction |
|
||||
| `gateway/delivery.py` | Outbound message delivery to target platforms/channels |
|
||||
| `gateway/pairing.py` | DM pairing flow for user authorization |
|
||||
| `gateway/channel_directory.py` | Maps chat IDs to human-readable names for cron delivery |
|
||||
| `gateway/hooks.py` | Hook discovery, loading, and lifecycle event dispatch |
|
||||
| `gateway/mirror.py` | Cross-session message mirroring for `send_message` |
|
||||
| `gateway/status.py` | Token lock management for profile-scoped gateway instances |
|
||||
| `gateway/builtin_hooks/` | Extension point for always-registered hooks (none shipped) |
|
||||
| `gateway/platforms/` | Platform adapters (one per messaging platform) |
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```text
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ GatewayRunner │
|
||||
│ │
|
||||
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
|
||||
│ │ Telegram │ │ Discord │ │ Slack │ │
|
||||
│ │ Adapter │ │ Adapter │ │ Adapter │ │
|
||||
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
|
||||
│ │ │ │ │
|
||||
│ └─────────────┼─────────────┘ │
|
||||
│ ▼ │
|
||||
│ _handle_message() │
|
||||
│ │ │
|
||||
│ ┌───────────┼───────────┐ │
|
||||
│ ▼ ▼ ▼ │
|
||||
│ Slash command AIAgent Queue/BG │
|
||||
│ dispatch creation sessions │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ SessionStore │
|
||||
│ (SQLite persistence) │
|
||||
└───────┴─────────────┴─────────────┴─────────────┘
|
||||
```
|
||||
|
||||
## Message Flow
|
||||
|
||||
When a message arrives from any platform:
|
||||
|
||||
1. **Platform adapter** receives raw event, normalizes it into a `MessageEvent`
|
||||
2. **Base adapter** checks active session guard:
|
||||
- If agent is running for this session → queue message, set interrupt event
|
||||
- If `/approve`, `/deny`, `/stop` → bypass guard (dispatched inline)
|
||||
3. **GatewayRunner._handle_message()** receives the event:
|
||||
- Resolve session key via `_session_key_for_source()` (format: `agent:main:{platform}:{chat_type}:{chat_id}`)
|
||||
- Check authorization (see Authorization below)
|
||||
- Check if it's a slash command → dispatch to command handler
|
||||
- Check if agent is already running → intercept commands like `/stop`, `/status`
|
||||
- Otherwise → create `AIAgent` instance and run conversation
|
||||
4. **Response** is sent back through the platform adapter
|
||||
|
||||
### Session Key Format
|
||||
|
||||
Session keys encode the full routing context:
|
||||
|
||||
```
|
||||
agent:main:{platform}:{chat_type}:{chat_id}
|
||||
```
|
||||
|
||||
For example: `agent:main:telegram:private:123456789`
|
||||
|
||||
Thread-aware platforms (Telegram forum topics, Discord threads, Slack threads) may include thread IDs in the chat_id portion. **Never construct session keys manually** — always use `build_session_key()` from `gateway/session.py`.
|
||||
|
||||
### Two-Level Message Guard
|
||||
|
||||
When an agent is actively running, incoming messages pass through two sequential guards:
|
||||
|
||||
1. **Level 1 — Base adapter** (`gateway/platforms/base.py`): Checks `_active_sessions`. If the session is active, queues the message in `_pending_messages` and sets an interrupt event. This catches messages *before* they reach the gateway runner.
|
||||
|
||||
2. **Level 2 — Gateway runner** (`gateway/run.py`): Checks `_running_agents`. Intercepts specific commands (`/stop`, `/new`, `/queue`, `/status`, `/approve`, `/deny`) and routes them appropriately. Everything else triggers `running_agent.interrupt()`.
|
||||
|
||||
Commands that must reach the runner while the agent is blocked (like `/approve`) are dispatched **inline** via `await self._message_handler(event)` — they bypass the background task system to avoid race conditions.
|
||||
|
||||
## Authorization
|
||||
|
||||
The gateway uses a multi-layer authorization check, evaluated in order:
|
||||
|
||||
1. **Per-platform allow-all flag** (e.g., `TELEGRAM_ALLOW_ALL_USERS`) — if set, all users on that platform are authorized
|
||||
2. **Platform allowlist** (e.g., `TELEGRAM_ALLOWED_USERS`) — comma-separated user IDs
|
||||
3. **DM pairing** — authenticated users can pair new users via a pairing code
|
||||
4. **Global allow-all** (`GATEWAY_ALLOW_ALL_USERS`) — if set, all users across all platforms are authorized
|
||||
5. **Default: deny** — unauthorized users are rejected
|
||||
|
||||
### DM Pairing Flow
|
||||
|
||||
```text
|
||||
Admin: /pair
|
||||
Gateway: "Pairing code: ABC123. Share with the user."
|
||||
New user: ABC123
|
||||
Gateway: "Paired! You're now authorized."
|
||||
```
|
||||
|
||||
Pairing state is persisted in `gateway/pairing.py` and survives restarts.
|
||||
|
||||
## Slash Command Dispatch
|
||||
|
||||
All slash commands in the gateway flow through the same resolution pipeline:
|
||||
|
||||
1. `resolve_command()` from `hermes_cli/commands.py` maps input to canonical name (handles aliases, prefix matching)
|
||||
2. The canonical name is checked against `GATEWAY_KNOWN_COMMANDS`
|
||||
3. Handler in `_handle_message()` dispatches based on canonical name
|
||||
4. Some commands are gated on config (`gateway_config_gate` on `CommandDef`)
|
||||
|
||||
### Running-Agent Guard
|
||||
|
||||
Commands that must NOT execute while the agent is processing are rejected early:
|
||||
|
||||
```python
|
||||
if _quick_key in self._running_agents:
|
||||
if canonical == "model":
|
||||
return "⏳ Agent is running — wait for it to finish or /stop first."
|
||||
```
|
||||
|
||||
Bypass commands (`/stop`, `/new`, `/approve`, `/deny`, `/queue`, `/status`) have special handling.
|
||||
|
||||
## Config Sources
|
||||
|
||||
The gateway reads configuration from multiple sources:
|
||||
|
||||
| Source | What it provides |
|
||||
|--------|-----------------|
|
||||
| `~/.hermes/.env` | API keys, bot tokens, platform credentials |
|
||||
| `~/.hermes/config.yaml` | Model settings, tool configuration, display options |
|
||||
| Environment variables | Override any of the above |
|
||||
|
||||
Unlike the CLI (which uses `load_cli_config()` with hardcoded defaults), the gateway reads `config.yaml` directly via YAML loader. This means config keys that exist in the CLI's defaults dict but not in the user's config file may behave differently between CLI and gateway.
|
||||
|
||||
## Platform Adapters
|
||||
|
||||
Most messaging platforms ship as plugin adapters under `plugins/platforms/<name>/adapter.py`; a few legacy adapters still live directly in `gateway/platforms/`. All extend `BasePlatformAdapter` from `gateway/platforms/base.py`:
|
||||
|
||||
```text
|
||||
plugins/platforms/ # plugin-packaged adapters (one dir each)
|
||||
├── telegram/adapter.py # Telegram Bot API (long polling or webhook)
|
||||
├── discord/adapter.py # Discord bot via discord.py
|
||||
├── slack/adapter.py # Slack Socket Mode
|
||||
├── whatsapp/adapter.py # WhatsApp Business Cloud API
|
||||
├── matrix/adapter.py # Matrix via mautrix (optional E2EE)
|
||||
├── mattermost/adapter.py # Mattermost WebSocket API
|
||||
├── email/adapter.py # Email via IMAP/SMTP
|
||||
├── sms/adapter.py # SMS via Twilio
|
||||
├── dingtalk/adapter.py # DingTalk WebSocket
|
||||
├── feishu/adapter.py # Feishu/Lark WebSocket or webhook
|
||||
├── wecom/adapter.py # WeCom (WeChat Work) callback
|
||||
├── line/adapter.py # LINE Messaging API
|
||||
├── teams/adapter.py # Microsoft Teams
|
||||
├── irc/adapter.py # IRC (canonical scoped-lock example)
|
||||
├── homeassistant/adapter.py # Home Assistant conversation integration
|
||||
└── … # google_chat, ntfy, photon, raft, simplex, …
|
||||
|
||||
gateway/platforms/ # core base + legacy direct adapters
|
||||
├── base.py # BasePlatformAdapter — shared logic for all platforms
|
||||
├── signal.py # Signal via signal-cli REST API
|
||||
├── weixin.py # Weixin (personal WeChat) via iLink Bot API
|
||||
├── bluebubbles.py # Apple iMessage via BlueBubbles macOS server
|
||||
├── qqbot/ # QQ Bot (Tencent QQ) via Official API v2 (sub-package)
|
||||
├── yuanbao.py # Yuanbao (Tencent) DM/group adapter
|
||||
├── msgraph_webhook.py # Microsoft Graph change-notification webhook (Teams, Outlook, etc.)
|
||||
├── webhook.py # Inbound/outbound webhook adapter
|
||||
└── api_server.py # REST API server adapter
|
||||
```
|
||||
|
||||
Experimental connector-backed platforms use the generic relay adapter in `gateway/relay/` instead of a direct platform module. When `GATEWAY_RELAY_URL` or `gateway.relay_url` is configured, the gateway registers the `relay` platform, dials the connector over an outbound WebSocket, and receives `descriptor`, `inbound`, and `interrupt_inbound` frames on that same socket. The connector advertises a `CapabilityDescriptor`; Hermes can send normal outbound replies, token-less `follow_up` operations, and interrupt frames back through the relay. The source-grounded wire contract lives in [`docs/relay-connector-contract.md`](https://github.com/NousResearch/hermes-agent/blob/main/docs/relay-connector-contract.md).
|
||||
|
||||
Adapters implement a common interface:
|
||||
- `connect()` / `disconnect()` — lifecycle management
|
||||
- `send_message()` — outbound message delivery
|
||||
- `on_message()` — inbound message normalization → `MessageEvent`
|
||||
|
||||
### Token Locks
|
||||
|
||||
Adapters that connect with unique credentials call `acquire_scoped_lock()` in `connect()` and `release_scoped_lock()` in `disconnect()`. This prevents two profiles from using the same bot token simultaneously.
|
||||
|
||||
## Delivery Path
|
||||
|
||||
Outgoing deliveries (`gateway/delivery.py`) handle:
|
||||
|
||||
- **Direct reply** — send response back to the originating chat
|
||||
- **Home channel delivery** — route cron job outputs and background results to a configured home channel
|
||||
- **Explicit target delivery** — the send engine specifying `telegram:-1001234567890`, exposed via the [`hermes send` CLI](/guides/pipe-script-output) for shell scripts and via cron `deliver:` targets
|
||||
- **Cross-platform delivery** — deliver to a different platform than the originating message
|
||||
|
||||
Cron job deliveries are NOT mirrored into gateway session history — they live in their own cron session only. This is a deliberate design choice to avoid message alternation violations.
|
||||
|
||||
## Hooks
|
||||
|
||||
Gateway hooks are Python modules that respond to lifecycle events:
|
||||
|
||||
### Gateway Hook Events
|
||||
|
||||
| Event | When fired |
|
||||
|-------|-----------|
|
||||
| `gateway:startup` | Gateway process starts |
|
||||
| `session:start` | New conversation session begins |
|
||||
| `session:end` | Session completes or times out |
|
||||
| `session:reset` | User resets session with `/new` |
|
||||
| `agent:start` | Agent begins processing a message |
|
||||
| `agent:step` | Agent completes one tool-calling iteration |
|
||||
| `agent:end` | Agent finishes and returns response |
|
||||
| `command:*` | Any slash command is executed |
|
||||
|
||||
Hooks are discovered from `gateway/builtin_hooks/` (an extension point — currently empty in the shipped distribution; `_register_builtin_hooks()` is a no-op stub) and `~/.hermes/hooks/` (user-installed). Each hook is a directory with a `HOOK.yaml` manifest and `handler.py`.
|
||||
|
||||
## Memory Provider Integration
|
||||
|
||||
When a memory provider plugin (e.g., Honcho) is enabled:
|
||||
|
||||
1. Gateway creates an `AIAgent` per message with the session ID
|
||||
2. The `MemoryManager` initializes the provider with the session context
|
||||
3. Provider tools (e.g., `honcho_profile`, `viking_search`) are routed through:
|
||||
|
||||
```text
|
||||
AIAgent._invoke_tool()
|
||||
→ self._memory_manager.handle_tool_call(name, args)
|
||||
→ provider.handle_tool_call(name, args)
|
||||
```
|
||||
|
||||
4. On session end/reset, `on_session_end()` fires for cleanup and final data flush
|
||||
|
||||
### Memory Flush Lifecycle
|
||||
|
||||
When a session is reset, resumed, or expires:
|
||||
1. Built-in memories are flushed to disk
|
||||
2. Memory provider's `on_session_end()` hook fires
|
||||
3. A temporary `AIAgent` runs a memory-only conversation turn
|
||||
4. Context is then discarded or archived
|
||||
|
||||
## Background Maintenance
|
||||
|
||||
The gateway runs periodic maintenance alongside message handling:
|
||||
|
||||
- **Cron ticking** — checks job schedules and fires due jobs
|
||||
- **Session expiry** — cleans up abandoned sessions after timeout
|
||||
- **Memory flush** — proactively flushes memory before session expiry
|
||||
- **Cache refresh** — refreshes model lists and provider status
|
||||
|
||||
## Process Management
|
||||
|
||||
The gateway runs as a long-lived process, managed via:
|
||||
|
||||
- `hermes gateway start` / `hermes gateway stop` — manual control
|
||||
- `systemctl` (Linux) or `launchctl` (macOS) — service management
|
||||
- PID file at `~/.hermes/gateway.pid` — profile-scoped process tracking
|
||||
|
||||
**Profile-scoped vs global**: `start_gateway()` uses profile-scoped PID files. `hermes gateway stop` stops only the current profile's gateway. `hermes gateway stop --all` uses global `ps aux` scanning to kill all gateway processes (used during updates).
|
||||
|
||||
## Related Docs
|
||||
|
||||
- [Session Storage](./session-storage.md)
|
||||
- [Cron Internals](./cron-internals.md)
|
||||
- [ACP Internals](./acp-internals.md)
|
||||
- [Agent Loop Internals](./agent-loop.md)
|
||||
- [Messaging Gateway (User Guide)](/user-guide/messaging)
|
||||
@@ -0,0 +1,316 @@
|
||||
---
|
||||
sidebar_position: 11
|
||||
title: "Image Generation Provider Plugins"
|
||||
description: "How to build an image-generation backend plugin for Hermes Agent"
|
||||
---
|
||||
|
||||
# Building an Image Generation Provider Plugin
|
||||
|
||||
Image-gen provider plugins register a backend that services every `image_generate` tool call — DALL·E, gpt-image, Grok, Flux, Imagen, Stable Diffusion, fal, Replicate, a local ComfyUI rig, anything. Built-in providers (OpenAI, OpenAI-Codex, xAI) all ship as plugins. You can add a new one, or override a bundled one, by dropping a directory into `plugins/image_gen/<name>/`.
|
||||
|
||||
:::tip
|
||||
Image-gen is one of several **backend plugins** Hermes supports. The others (with more specialized ABCs) are [Memory Provider Plugins](/developer-guide/memory-provider-plugin), [Context Engine Plugins](/developer-guide/context-engine-plugin), and [Model Provider Plugins](/developer-guide/model-provider-plugin). General tool/hook/CLI plugins live in [Build a Hermes Plugin](/developer-guide/plugins).
|
||||
:::
|
||||
|
||||
## How discovery works
|
||||
|
||||
Hermes scans for image-gen backends in three places:
|
||||
|
||||
1. **Bundled** — `<repo>/plugins/image_gen/<name>/` (auto-loaded with `kind: backend`, always available)
|
||||
2. **User** — `~/.hermes/plugins/image_gen/<name>/` (opt-in via `plugins.enabled`)
|
||||
3. **Pip** — packages declaring a `hermes_agent.plugins` entry point
|
||||
|
||||
Each plugin's `register(ctx)` function calls `ctx.register_image_gen_provider(...)` — that puts it into the registry in `agent/image_gen_registry.py`. The active provider is picked by `image_gen.provider` in `config.yaml`; `hermes tools` walks users through selection.
|
||||
|
||||
The `image_generate` tool wrapper asks the registry for the active provider and dispatches there. If no provider is registered, the tool surfaces a helpful error pointing at `hermes tools`.
|
||||
|
||||
## Directory structure
|
||||
|
||||
```
|
||||
plugins/image_gen/my-backend/
|
||||
├── __init__.py # ImageGenProvider subclass + register()
|
||||
└── plugin.yaml # Manifest with kind: backend
|
||||
```
|
||||
|
||||
A bundled plugin is complete at this point. User plugins at `~/.hermes/plugins/image_gen/<name>/` need to be added to `plugins.enabled` in `config.yaml` (or run `hermes plugins enable <name>`).
|
||||
|
||||
## The ImageGenProvider ABC
|
||||
|
||||
Subclass `agent.image_gen_provider.ImageGenProvider`. The only required members are the `name` property and the `generate()` method — everything else has sane defaults:
|
||||
|
||||
```python
|
||||
# plugins/image_gen/my-backend/__init__.py
|
||||
from typing import Any, Dict, List, Optional
|
||||
import os
|
||||
|
||||
from agent.image_gen_provider import (
|
||||
DEFAULT_ASPECT_RATIO,
|
||||
ImageGenProvider,
|
||||
error_response,
|
||||
normalize_reference_images,
|
||||
resolve_aspect_ratio,
|
||||
save_b64_image,
|
||||
success_response,
|
||||
)
|
||||
|
||||
|
||||
class MyBackendImageGenProvider(ImageGenProvider):
|
||||
@property
|
||||
def name(self) -> str:
|
||||
# Stable id used in image_gen.provider config. Lowercase, no spaces.
|
||||
return "my-backend"
|
||||
|
||||
@property
|
||||
def display_name(self) -> str:
|
||||
# Human label shown in `hermes tools`. Defaults to name.title() if omitted.
|
||||
return "My Backend"
|
||||
|
||||
def is_available(self) -> bool:
|
||||
# Return False if credentials or deps are missing.
|
||||
# The tool's availability gate calls this before dispatch.
|
||||
if not os.environ.get("MY_BACKEND_API_KEY"):
|
||||
return False
|
||||
try:
|
||||
import my_backend_sdk # noqa: F401
|
||||
except ImportError:
|
||||
return False
|
||||
return True
|
||||
|
||||
def list_models(self) -> List[Dict[str, Any]]:
|
||||
# Catalog shown in `hermes tools` model picker.
|
||||
return [
|
||||
{
|
||||
"id": "my-model-fast",
|
||||
"display": "My Model (Fast)",
|
||||
"speed": "~5s",
|
||||
"strengths": "Quick iteration",
|
||||
"price": "$0.01/image",
|
||||
},
|
||||
{
|
||||
"id": "my-model-hq",
|
||||
"display": "My Model (HQ)",
|
||||
"speed": "~30s",
|
||||
"strengths": "Highest fidelity",
|
||||
"price": "$0.04/image",
|
||||
},
|
||||
]
|
||||
|
||||
def default_model(self) -> Optional[str]:
|
||||
return "my-model-fast"
|
||||
|
||||
def get_setup_schema(self) -> Dict[str, Any]:
|
||||
# Metadata for the `hermes tools` picker — keys to prompt for at setup.
|
||||
return {
|
||||
"name": "My Backend",
|
||||
"badge": "paid", # optional; shown as a short tag in the picker
|
||||
"tag": "One-line description shown under the name",
|
||||
"env_vars": [
|
||||
{
|
||||
"key": "MY_BACKEND_API_KEY",
|
||||
"prompt": "My Backend API key",
|
||||
"url": "https://my-backend.example.com/api-keys",
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
def capabilities(self) -> Dict[str, Any]:
|
||||
# Declare whether this backend supports image-to-image / editing.
|
||||
# The tool layer surfaces this in the dynamic schema so the model
|
||||
# knows when `image_url` is honored. Default (if you omit this) is
|
||||
# text-only: {"modalities": ["text"], "max_reference_images": 0}.
|
||||
return {"modalities": ["text", "image"], "max_reference_images": 4}
|
||||
|
||||
def generate(
|
||||
self,
|
||||
prompt: str,
|
||||
aspect_ratio: str = DEFAULT_ASPECT_RATIO,
|
||||
*,
|
||||
image_url: Optional[str] = None,
|
||||
reference_image_urls: Optional[List[str]] = None,
|
||||
**kwargs: Any,
|
||||
) -> Dict[str, Any]:
|
||||
prompt = (prompt or "").strip()
|
||||
aspect_ratio = resolve_aspect_ratio(aspect_ratio)
|
||||
|
||||
if not prompt:
|
||||
return error_response(
|
||||
error="Prompt is required",
|
||||
error_type="invalid_input",
|
||||
provider=self.name,
|
||||
prompt="",
|
||||
aspect_ratio=aspect_ratio,
|
||||
)
|
||||
|
||||
# Routing: if image_url (or reference_image_urls) is set, the call is
|
||||
# an image-to-image / edit request; otherwise text-to-image. Report
|
||||
# which path you took via the `modality` field of success_response.
|
||||
sources = []
|
||||
if image_url:
|
||||
sources.append(image_url)
|
||||
sources.extend(normalize_reference_images(reference_image_urls) or [])
|
||||
modality = "image" if sources else "text"
|
||||
|
||||
# Model selection precedence: env var → config → default. The helper
|
||||
# _resolve_model() in the built-in openai plugin is a good reference.
|
||||
model_id = kwargs.get("model") or self.default_model() or "my-model-fast"
|
||||
|
||||
try:
|
||||
import my_backend_sdk
|
||||
client = my_backend_sdk.Client(api_key=os.environ["MY_BACKEND_API_KEY"])
|
||||
if modality == "image":
|
||||
result = client.edit(
|
||||
prompt=prompt,
|
||||
model=model_id,
|
||||
image_urls=sources,
|
||||
)
|
||||
else:
|
||||
result = client.generate(
|
||||
prompt=prompt,
|
||||
model=model_id,
|
||||
aspect_ratio=aspect_ratio,
|
||||
)
|
||||
|
||||
# Two shapes supported:
|
||||
# - URL string: return it as `image`
|
||||
# - base64 data: save under $HERMES_HOME/cache/images/ via save_b64_image()
|
||||
if result.get("image_b64"):
|
||||
path = save_b64_image(
|
||||
result["image_b64"],
|
||||
prefix=self.name,
|
||||
extension="png",
|
||||
)
|
||||
image = str(path)
|
||||
else:
|
||||
image = result["image_url"]
|
||||
|
||||
return success_response(
|
||||
image=image,
|
||||
model=model_id,
|
||||
prompt=prompt,
|
||||
aspect_ratio=aspect_ratio,
|
||||
provider=self.name,
|
||||
modality=modality,
|
||||
)
|
||||
except Exception as exc:
|
||||
return error_response(
|
||||
error=str(exc),
|
||||
error_type=type(exc).__name__,
|
||||
provider=self.name,
|
||||
model=model_id,
|
||||
prompt=prompt,
|
||||
aspect_ratio=aspect_ratio,
|
||||
)
|
||||
|
||||
|
||||
def register(ctx) -> None:
|
||||
"""Plugin entry point — called once at load time."""
|
||||
ctx.register_image_gen_provider(MyBackendImageGenProvider())
|
||||
```
|
||||
|
||||
## plugin.yaml
|
||||
|
||||
```yaml
|
||||
name: my-backend
|
||||
version: 1.0.0
|
||||
description: My image backend — text-to-image via My Backend SDK
|
||||
author: Your Name
|
||||
kind: backend
|
||||
requires_env:
|
||||
- MY_BACKEND_API_KEY
|
||||
```
|
||||
|
||||
`kind: backend` is what routes the plugin to the image-gen registration path. `requires_env` is prompted during `hermes plugins install`.
|
||||
|
||||
## ABC reference
|
||||
|
||||
Full contract in `agent/image_gen_provider.py`. The methods you'll typically override:
|
||||
|
||||
| Member | Required | Default | Purpose |
|
||||
|---|---|---|---|
|
||||
| `name` | ✅ | — | Stable id used in `image_gen.provider` config |
|
||||
| `display_name` | — | `name.title()` | Label shown in `hermes tools` |
|
||||
| `is_available()` | — | `True` | Gate for missing creds/deps |
|
||||
| `list_models()` | — | `[]` | Catalog for `hermes tools` model picker |
|
||||
| `default_model()` | — | first from `list_models()` | Fallback when no model is configured |
|
||||
| `get_setup_schema()` | — | minimal | Picker metadata + env-var prompts |
|
||||
| `generate(prompt, aspect_ratio, **kwargs)` | ✅ | — | The call |
|
||||
|
||||
## Response format
|
||||
|
||||
`generate()` must return a dict built via `success_response()` or `error_response()`. Both live in `agent/image_gen_provider.py`.
|
||||
|
||||
**Success:**
|
||||
```python
|
||||
success_response(
|
||||
image=<url-or-absolute-path>,
|
||||
model=<model-id>,
|
||||
prompt=<echoed-prompt>,
|
||||
aspect_ratio="landscape" | "square" | "portrait",
|
||||
provider=<your-provider-name>,
|
||||
extra={...}, # optional backend-specific fields
|
||||
)
|
||||
```
|
||||
|
||||
**Error:**
|
||||
```python
|
||||
error_response(
|
||||
error="human-readable message",
|
||||
error_type="provider_error" | "invalid_input" | "<exception class name>",
|
||||
provider=<your-provider-name>,
|
||||
model=<model-id>,
|
||||
prompt=<prompt>,
|
||||
aspect_ratio=<resolved aspect>,
|
||||
)
|
||||
```
|
||||
|
||||
The tool wrapper JSON-serializes the dict and hands it to the LLM. Errors are surfaced as the tool result; the LLM decides how to explain them to the user.
|
||||
|
||||
## Handling base64 vs URL output
|
||||
|
||||
Some backends return image URLs (fal, Replicate); others return base64 payloads (OpenAI gpt-image-2). For the base64 case, use `save_b64_image()` — it writes to `$HERMES_HOME/cache/images/<prefix>_<timestamp>_<uuid>.<ext>` and returns the absolute `Path`. Pass that path (as `str`) as `image=` in `success_response()`. Gateway delivery (Telegram photo bubble, Discord attachment) recognizes both URLs and absolute paths.
|
||||
|
||||
## User overrides
|
||||
|
||||
Drop a user plugin at `~/.hermes/plugins/image_gen/<name>/` with the same `name` property as a bundled one and enable it via `hermes plugins enable <name>` — the registry is last-writer-wins, so your version replaces the built-in. Useful for pointing an `openai` plugin at a private proxy, or swapping in a custom model catalog.
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
export HERMES_HOME=/tmp/hermes-imggen-test
|
||||
mkdir -p $HERMES_HOME/plugins/image_gen/my-backend
|
||||
# …copy __init__.py + plugin.yaml into that dir…
|
||||
|
||||
export MY_BACKEND_API_KEY=your-test-key
|
||||
hermes plugins enable my-backend
|
||||
|
||||
# Pick it as the active provider
|
||||
echo "image_gen:" >> $HERMES_HOME/config.yaml
|
||||
echo " provider: my-backend" >> $HERMES_HOME/config.yaml
|
||||
|
||||
# Exercise it
|
||||
hermes -z "Generate an image of a corgi in a spacesuit"
|
||||
```
|
||||
|
||||
Or interactively: `hermes tools` → "Image Generation" → select `my-backend` → enter API key if prompted.
|
||||
|
||||
## Reference implementations
|
||||
|
||||
- **`plugins/image_gen/openai/__init__.py`** — gpt-image-2 at low/medium/high tiers as three virtual model IDs sharing one API model with different `quality` params. Good example of tiered models under a single backend + config.yaml precedence chain.
|
||||
- **`plugins/image_gen/xai/__init__.py`** — Grok Imagine via xAI. Different shape (URL output, simpler catalog).
|
||||
- **`plugins/image_gen/openai-codex/__init__.py`** — Codex-style Responses API variant reusing the OpenAI SDK with a different routing base URL.
|
||||
|
||||
## Distribute via pip
|
||||
|
||||
```toml
|
||||
# pyproject.toml
|
||||
[project.entry-points."hermes_agent.plugins"]
|
||||
my-backend-imggen = "my_backend_imggen_package"
|
||||
```
|
||||
|
||||
`my_backend_imggen_package` must expose a top-level `register` function. See [Distribute via pip](/developer-guide/plugins#distribute-via-pip) in the general plugin guide for the full setup.
|
||||
|
||||
## Related pages
|
||||
|
||||
- [Image Generation](/user-guide/features/image-generation) — user-facing feature documentation
|
||||
- [Plugins overview](/user-guide/features/plugins) — all plugin types at a glance
|
||||
- [Build a Hermes Plugin](/developer-guide/plugins) — general tools/hooks/slash commands guide
|
||||
@@ -0,0 +1,268 @@
|
||||
---
|
||||
sidebar_position: 8
|
||||
title: "Memory Provider Plugins"
|
||||
description: "How to build a memory provider plugin for Hermes Agent"
|
||||
---
|
||||
|
||||
# Building a Memory Provider Plugin
|
||||
|
||||
Memory provider plugins give Hermes Agent persistent, cross-session knowledge beyond the built-in MEMORY.md and USER.md. This guide covers how to build one.
|
||||
|
||||
:::tip
|
||||
Memory providers are one of two **provider plugin** types. The other is [Context Engine Plugins](/developer-guide/context-engine-plugin), which replace the built-in context compressor. Both follow the same pattern: single-select, config-driven, managed via `hermes plugins`.
|
||||
:::
|
||||
|
||||
## Directory Structure
|
||||
|
||||
Each memory provider lives in `plugins/memory/<name>/`:
|
||||
|
||||
```
|
||||
plugins/memory/my-provider/
|
||||
├── __init__.py # MemoryProvider implementation + register() entry point
|
||||
├── plugin.yaml # Metadata (name, description, hooks)
|
||||
└── README.md # Setup instructions, config reference, tools
|
||||
```
|
||||
|
||||
## The MemoryProvider ABC
|
||||
|
||||
Your plugin implements the `MemoryProvider` abstract base class from `agent/memory_provider.py`:
|
||||
|
||||
```python
|
||||
from agent.memory_provider import MemoryProvider
|
||||
|
||||
class MyMemoryProvider(MemoryProvider):
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "my-provider"
|
||||
|
||||
def is_available(self) -> bool:
|
||||
"""Check if this provider can activate. NO network calls."""
|
||||
return bool(os.environ.get("MY_API_KEY"))
|
||||
|
||||
def initialize(self, session_id: str, **kwargs) -> None:
|
||||
"""Called once at agent startup.
|
||||
|
||||
kwargs always includes:
|
||||
hermes_home (str): Active HERMES_HOME path. Use for storage.
|
||||
"""
|
||||
self._api_key = os.environ.get("MY_API_KEY", "")
|
||||
self._session_id = session_id
|
||||
|
||||
# ... implement remaining methods
|
||||
```
|
||||
|
||||
## Required Methods
|
||||
|
||||
### Core Lifecycle
|
||||
|
||||
| Method | When Called | Must Implement? |
|
||||
|--------|-----------|-----------------|
|
||||
| `name` (property) | Always | **Yes** |
|
||||
| `is_available()` | Agent init, before activation | **Yes** — no network calls |
|
||||
| `initialize(session_id, **kwargs)` | Agent startup | **Yes** |
|
||||
| `get_tool_schemas()` | After init, for tool injection | **Yes** |
|
||||
| `handle_tool_call(tool_name, args, **kwargs)` | When agent uses your tools | **Yes** (if you have tools) |
|
||||
|
||||
### Config
|
||||
|
||||
| Method | Purpose | Must Implement? |
|
||||
|--------|---------|-----------------|
|
||||
| `get_config_schema()` | Declare config fields for `hermes memory setup` | **Yes** |
|
||||
| `save_config(values, hermes_home)` | Write non-secret config to native location | **Yes** (unless env-var-only) |
|
||||
|
||||
### Optional Hooks
|
||||
|
||||
| Method | When Called | Use Case |
|
||||
|--------|-----------|----------|
|
||||
| `system_prompt_block()` | System prompt assembly | Static provider info |
|
||||
| `prefetch(query, *, session_id="")` | Before each API call | Return recalled context |
|
||||
| `queue_prefetch(query)` | After each turn | Pre-warm for next turn |
|
||||
| `sync_turn(user, assistant, *, session_id="")` | After each completed turn | Persist conversation |
|
||||
| `on_session_end(messages)` | Conversation ends | Final extraction/flush |
|
||||
| `on_pre_compress(messages)` | Before context compression | Save insights before discard |
|
||||
| `on_memory_write(action, target, content)` | Built-in memory writes | Mirror to your backend |
|
||||
| `shutdown()` | Process exit | Clean up connections |
|
||||
|
||||
## Config Schema
|
||||
|
||||
`get_config_schema()` returns a list of field descriptors used by `hermes memory setup`:
|
||||
|
||||
```python
|
||||
def get_config_schema(self):
|
||||
return [
|
||||
{
|
||||
"key": "api_key",
|
||||
"description": "My Provider API key",
|
||||
"secret": True, # → written to .env
|
||||
"required": True,
|
||||
"env_var": "MY_API_KEY", # explicit env var name
|
||||
"url": "https://my-provider.com/keys", # where to get it
|
||||
},
|
||||
{
|
||||
"key": "region",
|
||||
"description": "Server region",
|
||||
"default": "us-east",
|
||||
"choices": ["us-east", "eu-west", "ap-south"],
|
||||
},
|
||||
{
|
||||
"key": "project",
|
||||
"description": "Project identifier",
|
||||
"default": "hermes",
|
||||
},
|
||||
]
|
||||
```
|
||||
|
||||
Fields with `secret: True` and `env_var` go to `.env`. Non-secret fields are passed to `save_config()`.
|
||||
|
||||
:::tip Minimal vs Full Schema
|
||||
Every field in `get_config_schema()` is prompted during `hermes memory setup`. Providers with many options should keep the schema minimal — only include fields the user **must** configure (API key, required credentials). Document optional settings in a config file reference (e.g. `$HERMES_HOME/myprovider.json`) rather than prompting for them all during setup. This keeps the setup wizard fast while still supporting advanced configuration. See the Supermemory provider for an example — it only prompts for the API key; all other options live in `supermemory.json`.
|
||||
:::
|
||||
|
||||
## Save Config
|
||||
|
||||
```python
|
||||
def save_config(self, values: dict, hermes_home: str) -> None:
|
||||
"""Write non-secret config to your native location."""
|
||||
import json
|
||||
from pathlib import Path
|
||||
config_path = Path(hermes_home) / "my-provider.json"
|
||||
config_path.write_text(json.dumps(values, indent=2))
|
||||
```
|
||||
|
||||
For env-var-only providers, leave the default no-op.
|
||||
|
||||
## Plugin Entry Point
|
||||
|
||||
```python
|
||||
def register(ctx) -> None:
|
||||
"""Called by the memory plugin discovery system."""
|
||||
ctx.register_memory_provider(MyMemoryProvider())
|
||||
```
|
||||
|
||||
## plugin.yaml
|
||||
|
||||
```yaml
|
||||
name: my-provider
|
||||
version: 1.0.0
|
||||
description: "Short description of what this provider does."
|
||||
hooks:
|
||||
- on_session_end # list hooks you implement
|
||||
```
|
||||
|
||||
## Threading Contract
|
||||
|
||||
**`sync_turn()` MUST be non-blocking.** If your backend has latency (API calls, LLM processing), run the work in a daemon thread:
|
||||
|
||||
```python
|
||||
def sync_turn(self, user_content, assistant_content, *, session_id="", messages=None):
|
||||
def _sync():
|
||||
try:
|
||||
self._api.ingest(user_content, assistant_content, session_id=session_id, messages=messages)
|
||||
except Exception as e:
|
||||
logger.warning("Sync failed: %s", e)
|
||||
|
||||
if self._sync_thread and self._sync_thread.is_alive():
|
||||
self._sync_thread.join(timeout=5.0)
|
||||
self._sync_thread = threading.Thread(target=_sync, daemon=True)
|
||||
self._sync_thread.start()
|
||||
```
|
||||
|
||||
`messages` is optional OpenAI-style conversation context as of the completed
|
||||
turn. When present, it includes user/assistant messages, assistant tool calls,
|
||||
and tool result messages. Providers that do not need raw turn context can omit
|
||||
the `messages` parameter; Hermes will continue calling them with the legacy
|
||||
signature.
|
||||
|
||||
Cloud providers should document what parts of `messages` are sent off-device.
|
||||
Tool calls and tool results may contain file paths, command output, or other
|
||||
workspace data.
|
||||
|
||||
## Profile Isolation
|
||||
|
||||
All storage paths **must** use the `hermes_home` kwarg from `initialize()`, not hardcoded `~/.hermes`:
|
||||
|
||||
```python
|
||||
# CORRECT — profile-scoped
|
||||
from hermes_constants import get_hermes_home
|
||||
data_dir = get_hermes_home() / "my-provider"
|
||||
|
||||
# WRONG — shared across all profiles
|
||||
data_dir = Path("~/.hermes/my-provider").expanduser()
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
See `tests/agent/test_memory_provider.py` and adjacent memory tests (`tests/agent/test_memory_session_switch.py`, `tests/agent/test_memory_user_id.py`, `tests/run_agent/test_memory_provider_init.py`) for end-to-end patterns.
|
||||
|
||||
```python
|
||||
from agent.memory_manager import MemoryManager
|
||||
|
||||
mgr = MemoryManager()
|
||||
mgr.add_provider(my_provider)
|
||||
mgr.initialize_all(session_id="test-1", platform="cli")
|
||||
|
||||
# Test tool routing
|
||||
result = mgr.handle_tool_call("my_tool", {"action": "add", "content": "test"})
|
||||
|
||||
# Test lifecycle
|
||||
mgr.sync_all("user msg", "assistant msg")
|
||||
mgr.on_session_end([])
|
||||
mgr.shutdown_all()
|
||||
```
|
||||
|
||||
## Adding CLI Commands
|
||||
|
||||
Memory provider plugins can register their own CLI subcommand tree (e.g. `hermes my-provider status`, `hermes my-provider config`). This uses a convention-based discovery system — no changes to core files needed.
|
||||
|
||||
### How it works
|
||||
|
||||
1. Add a `cli.py` file to your plugin directory
|
||||
2. Define a `register_cli(subparser)` function that builds the argparse tree
|
||||
3. The memory plugin system discovers it at startup via `discover_plugin_cli_commands()`
|
||||
4. Your commands appear under `hermes <provider-name> <subcommand>`
|
||||
|
||||
**Active-provider gating:** Your CLI commands only appear when your provider is the active `memory.provider` in config. If a user hasn't configured your provider, your commands won't show in `hermes --help`.
|
||||
|
||||
### Example
|
||||
|
||||
```python
|
||||
# plugins/memory/my-provider/cli.py
|
||||
|
||||
def my_command(args):
|
||||
"""Handler dispatched by argparse."""
|
||||
sub = getattr(args, "my_command", None)
|
||||
if sub == "status":
|
||||
print("Provider is active and connected.")
|
||||
elif sub == "config":
|
||||
print("Showing config...")
|
||||
else:
|
||||
print("Usage: hermes my-provider <status|config>")
|
||||
|
||||
def register_cli(subparser) -> None:
|
||||
"""Build the hermes my-provider argparse tree.
|
||||
|
||||
Called by discover_plugin_cli_commands() at argparse setup time.
|
||||
"""
|
||||
subs = subparser.add_subparsers(dest="my_command")
|
||||
subs.add_parser("status", help="Show provider status")
|
||||
subs.add_parser("config", help="Show provider config")
|
||||
subparser.set_defaults(func=my_command)
|
||||
```
|
||||
|
||||
### Reference implementation
|
||||
|
||||
See `plugins/memory/honcho/cli.py` for a full example with 13 subcommands, cross-profile management (`--target-profile`), and config read/write.
|
||||
|
||||
### Directory structure with CLI
|
||||
|
||||
```
|
||||
plugins/memory/my-provider/
|
||||
├── __init__.py # MemoryProvider implementation + register()
|
||||
├── plugin.yaml # Metadata
|
||||
├── cli.py # register_cli(subparser) — CLI commands
|
||||
└── README.md # Setup instructions
|
||||
```
|
||||
|
||||
## Single Provider Rule
|
||||
|
||||
Only **one** external memory provider can be active at a time. If a user tries to register a second, the MemoryManager rejects it with a warning. This prevents tool schema bloat and conflicting backends.
|
||||
@@ -0,0 +1,268 @@
|
||||
---
|
||||
sidebar_position: 10
|
||||
title: "Model Provider Plugins"
|
||||
description: "How to build a model provider (inference backend) plugin for Hermes Agent"
|
||||
---
|
||||
|
||||
# Building a Model Provider Plugin
|
||||
|
||||
Model provider plugins declare an inference backend — an OpenAI-compatible endpoint, an Anthropic Messages server, a Codex-style Responses API, or a Bedrock-native surface — that Hermes can route `AIAgent` calls through. Every built-in provider (OpenRouter, Anthropic, GMI, DeepSeek, Nvidia, …) ships as one of these plugins. Third parties can add their own by dropping a directory under `$HERMES_HOME/plugins/model-providers/` with zero changes to the repo.
|
||||
|
||||
:::tip
|
||||
Model provider plugins are the third kind of **provider plugin**. The others are [Memory Provider Plugins](/developer-guide/memory-provider-plugin) (cross-session knowledge) and [Context Engine Plugins](/developer-guide/context-engine-plugin) (context compression strategies). All three follow the same "drop a directory, declare a profile, no repo edits" pattern.
|
||||
:::
|
||||
|
||||
## How discovery works
|
||||
|
||||
`providers/__init__.py._discover_providers()` runs lazily the first time any code calls `get_provider_profile()` or `list_providers()`. Discovery order:
|
||||
|
||||
1. **Bundled plugins** — `<repo>/plugins/model-providers/<name>/` — ship with Hermes
|
||||
2. **User plugins** — `$HERMES_HOME/plugins/model-providers/<name>/` — drop in any directory; no restart required for subsequent sessions
|
||||
3. **Legacy single-file** — `<repo>/providers/<name>.py` — back-compat for out-of-tree editable installs
|
||||
|
||||
**User plugins override bundled plugins of the same name** because `register_provider()` is last-writer-wins. Drop a `$HERMES_HOME/plugins/model-providers/gmi/` directory to replace the built-in GMI profile without touching the repo.
|
||||
|
||||
## Directory structure
|
||||
|
||||
```
|
||||
plugins/model-providers/my-provider/
|
||||
├── __init__.py # Calls register_provider(profile) at module-level
|
||||
├── plugin.yaml # kind: model-provider + metadata (optional but recommended)
|
||||
└── README.md # Setup instructions (optional)
|
||||
```
|
||||
|
||||
The only required file is `__init__.py`. `plugin.yaml` is used by `hermes plugins` for introspection and by the general PluginManager to route the plugin to the right loader; without it, the general loader falls back to a source-text heuristic.
|
||||
|
||||
## Minimal example — a simple API-key provider
|
||||
|
||||
```python
|
||||
# plugins/model-providers/acme-inference/__init__.py
|
||||
from providers import register_provider
|
||||
from providers.base import ProviderProfile
|
||||
|
||||
acme = ProviderProfile(
|
||||
name="acme-inference",
|
||||
aliases=("acme",),
|
||||
display_name="Acme Inference",
|
||||
description="Acme — OpenAI-compatible direct API",
|
||||
signup_url="https://acme.example.com/keys",
|
||||
env_vars=("ACME_API_KEY", "ACME_BASE_URL"),
|
||||
base_url="https://api.acme.example.com/v1",
|
||||
auth_type="api_key",
|
||||
default_aux_model="acme-small-fast",
|
||||
fallback_models=(
|
||||
"acme-large-v3",
|
||||
"acme-medium-v3",
|
||||
"acme-small-fast",
|
||||
),
|
||||
)
|
||||
|
||||
register_provider(acme)
|
||||
```
|
||||
|
||||
```yaml
|
||||
# plugins/model-providers/acme-inference/plugin.yaml
|
||||
name: acme-inference
|
||||
kind: model-provider
|
||||
version: 1.0.0
|
||||
description: Acme Inference — OpenAI-compatible direct API
|
||||
author: Your Name
|
||||
```
|
||||
|
||||
That's it. After dropping these two files, the following **auto-wire** with no other edits:
|
||||
|
||||
| Integration | Where | What it gets |
|
||||
|---|---|---|
|
||||
| Credential resolution | `hermes_cli/auth.py` | `PROVIDER_REGISTRY["acme-inference"]` populated from profile |
|
||||
| `--provider` CLI flag | `hermes_cli/main.py` | Accepts `acme-inference` |
|
||||
| `hermes model` picker | `hermes_cli/models.py` | Appears in `CANONICAL_PROVIDERS`, model list fetched from `{base_url}/models` |
|
||||
| `hermes doctor` | `hermes_cli/doctor.py` | Health check for `ACME_API_KEY` + `{base_url}/models` probe |
|
||||
| `hermes setup` | `hermes_cli/config.py` | `ACME_API_KEY` appears in `OPTIONAL_ENV_VARS` and the setup wizard |
|
||||
| URL reverse-mapping | `agent/model_metadata.py` | Hostname → provider name for auto-detection |
|
||||
| Auxiliary model | `agent/auxiliary_client.py` | Uses `default_aux_model` for compression / summarization |
|
||||
| Runtime resolution | `hermes_cli/runtime_provider.py` | Returns correct `base_url`, `api_key`, `api_mode` |
|
||||
| Transport | `agent/transports/chat_completions.py` | Profile path generates kwargs via `prepare_messages` / `build_extra_body` / `build_api_kwargs_extras` |
|
||||
|
||||
## ProviderProfile fields
|
||||
|
||||
Full definition in `providers/base.py`. The most useful ones:
|
||||
|
||||
| Field | Type | Purpose |
|
||||
|---|---|---|
|
||||
| `name` | str | Canonical id — matches `model.provider` in `config.yaml` and the `--provider` flag |
|
||||
| `aliases` | `tuple[str, ...]` | Alternative names resolved by `get_provider_profile()` (e.g. `grok` → `xai`) |
|
||||
| `api_mode` | str | `chat_completions` \| `codex_responses` \| `anthropic_messages` \| `bedrock_converse` |
|
||||
| `display_name` | str | Human label shown in `hermes model` picker |
|
||||
| `description` | str | Picker subtitle |
|
||||
| `signup_url` | str | Shown during first-run setup ("get an API key here") |
|
||||
| `env_vars` | `tuple[str, ...]` | API-key env vars in priority order; a final `*_BASE_URL` entry is used as the user base-URL override |
|
||||
| `base_url` | str | Default inference endpoint |
|
||||
| `models_url` | str | Explicit catalog URL (falls back to `{base_url}/models`) |
|
||||
| `auth_type` | str | `api_key` \| `oauth_device_code` \| `oauth_external` \| `copilot` \| `aws_sdk` \| `external_process` |
|
||||
| `fallback_models` | `tuple[str, ...]` | Curated list shown when live catalog fetch fails |
|
||||
| `default_headers` | `dict[str, str]` | Sent on every request (e.g. Copilot's `Editor-Version`) |
|
||||
| `fixed_temperature` | Any | `None` = use caller's value; `OMIT_TEMPERATURE` sentinel = don't send temperature at all (Kimi) |
|
||||
| `default_max_tokens` | `int \| None` | Provider-level max_tokens cap (Nvidia: 16384) |
|
||||
| `default_aux_model` | str | Cheap model for auxiliary tasks (compression, vision, summarization) |
|
||||
|
||||
## Overridable hooks
|
||||
|
||||
Subclass `ProviderProfile` for non-trivial quirks:
|
||||
|
||||
```python
|
||||
from typing import Any
|
||||
from providers.base import ProviderProfile
|
||||
|
||||
class AcmeProfile(ProviderProfile):
|
||||
def prepare_messages(self, messages: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
||||
"""Provider-specific message preprocessing. Runs after codex
|
||||
sanitization, before developer-role swap. Default: pass-through."""
|
||||
# Example: Qwen normalizes plain-text content to a list-of-parts
|
||||
# array and injects cache_control; Kimi rewrites tool-call JSON
|
||||
return messages
|
||||
|
||||
def build_extra_body(self, *, session_id=None, **context) -> dict:
|
||||
"""Provider-specific extra_body fields merged into the API call.
|
||||
Context includes: session_id, provider_preferences, model, base_url,
|
||||
reasoning_config. Default: empty dict."""
|
||||
# Example: OpenRouter's provider-preferences block,
|
||||
# Gemini's thinking_config translation.
|
||||
return {}
|
||||
|
||||
def build_api_kwargs_extras(self, *, reasoning_config=None, **context):
|
||||
"""Returns (extra_body_additions, top_level_kwargs). Needed when some
|
||||
fields go top-level (Kimi's reasoning_effort, OpenRouter's verbosity for
|
||||
adaptive Anthropic models) and some go in extra_body (OpenRouter's
|
||||
reasoning dict). Default: ({}, {})."""
|
||||
return {}, {}
|
||||
|
||||
def fetch_models(self, *, api_key=None, timeout=8.0) -> list[str] | None:
|
||||
"""Live catalog fetch. Default hits {models_url or base_url}/models with
|
||||
Bearer auth. Override for: custom auth (Anthropic), no REST endpoint
|
||||
(Bedrock → None), or public/unauthenticated catalogs (OpenRouter)."""
|
||||
return super().fetch_models(api_key=api_key, timeout=timeout)
|
||||
```
|
||||
|
||||
## Hook reference examples
|
||||
|
||||
Look at these bundled plugins for idioms:
|
||||
|
||||
| Plugin | Why look |
|
||||
|---|---|
|
||||
| `plugins/model-providers/openrouter/` | Aggregator with provider preferences, public model catalog |
|
||||
| `plugins/model-providers/gemini/` | `thinking_config` translation (native + OpenAI-compat nested forms) |
|
||||
| `plugins/model-providers/kimi-coding/` | `OMIT_TEMPERATURE`, `extra_body.thinking`, top-level `reasoning_effort` |
|
||||
| `plugins/model-providers/qwen-oauth/` | Message normalization, `cache_control` injection, VL high-res |
|
||||
| `plugins/model-providers/nous/` | Attribution tags, "omit reasoning when disabled" |
|
||||
| `plugins/model-providers/custom/` | Ollama `num_ctx` + `think: false` quirks |
|
||||
| `plugins/model-providers/bedrock/` | `api_mode="bedrock_converse"`, `fetch_models` returns None (no REST endpoint) |
|
||||
|
||||
## User overrides — replace a built-in without editing the repo
|
||||
|
||||
Say you want to point `gmi` at your private staging endpoint for testing. Create `~/.hermes/plugins/model-providers/gmi/__init__.py`:
|
||||
|
||||
```python
|
||||
from providers import register_provider
|
||||
from providers.base import ProviderProfile
|
||||
|
||||
register_provider(ProviderProfile(
|
||||
name="gmi",
|
||||
aliases=("gmi-cloud", "gmicloud"),
|
||||
env_vars=("GMI_API_KEY",),
|
||||
base_url="https://gmi-staging.internal.example.com/v1",
|
||||
auth_type="api_key",
|
||||
default_aux_model="google/gemini-3.1-flash-lite-preview",
|
||||
))
|
||||
```
|
||||
|
||||
Next session, `get_provider_profile("gmi").base_url` returns the staging URL. No repo patch, no rebuild. Because user plugins are discovered after bundled ones, the user `register_provider()` call wins.
|
||||
|
||||
## api_mode selection
|
||||
|
||||
Four values are recognized. Hermes picks one based on:
|
||||
|
||||
1. User explicit override (`config.yaml` `model.api_mode` when set)
|
||||
2. OpenCode's per-model dispatch (`opencode_model_api_mode` for Zen and Go)
|
||||
3. URL auto-detection — `/anthropic` suffix → `anthropic_messages`, `api.openai.com` → `codex_responses`, `api.x.ai` → `codex_responses`, `/coding` on Kimi domains → `chat_completions`
|
||||
4. **Profile `api_mode`** as a fallback when URL detection finds nothing
|
||||
5. Default `chat_completions`
|
||||
|
||||
Set `profile.api_mode` to match the default your provider ships — it acts as a hint. User URL overrides still win.
|
||||
|
||||
## Auth types
|
||||
|
||||
| `auth_type` | Meaning | Who uses it |
|
||||
|---|---|---|
|
||||
| `api_key` | Single env var carries a static API key | Most providers |
|
||||
| `oauth_device_code` | Device-code OAuth flow | — |
|
||||
| `oauth_external` | User signs in elsewhere, tokens land in `auth.json` | Anthropic OAuth, MiniMax OAuth, Qwen Portal, Nous Portal |
|
||||
| `copilot` | GitHub Copilot token refresh cycle | `copilot` plugin only |
|
||||
| `aws_sdk` | AWS SDK credential chain (IAM role, profile, env) | `bedrock` plugin only |
|
||||
| `external_process` | Auth handled by a subprocess the agent spawns | `copilot-acp` plugin only |
|
||||
|
||||
`auth_type` gates which codepaths treat your provider as a "simple api-key provider" — if it's not `api_key`, the PluginManager still records the manifest but Hermes' CLI-level automation (doctor checks, `--provider` flag, setup wizard delegation) may skip over it.
|
||||
|
||||
## Discovery timing
|
||||
|
||||
Provider discovery is **lazy** — triggered by the first `get_provider_profile()` or `list_providers()` call in the process. In practice this happens early at startup (`auth.py` module load extends `PROVIDER_REGISTRY` eagerly). If you need to verify your plugin loaded, run:
|
||||
|
||||
```bash
|
||||
hermes doctor
|
||||
```
|
||||
|
||||
— a successful `auth_type="api_key"` profile appears under the Provider Connectivity section with a `/models` probe.
|
||||
|
||||
For programmatic inspection:
|
||||
|
||||
```python
|
||||
from providers import list_providers
|
||||
for p in list_providers():
|
||||
print(p.name, p.base_url, p.api_mode)
|
||||
```
|
||||
|
||||
## Testing your plugin
|
||||
|
||||
Point `HERMES_HOME` at a temp directory so you don't pollute your real config:
|
||||
|
||||
```bash
|
||||
export HERMES_HOME=/tmp/hermes-plugin-test
|
||||
mkdir -p $HERMES_HOME/plugins/model-providers/my-provider
|
||||
cat > $HERMES_HOME/plugins/model-providers/my-provider/__init__.py <<'EOF'
|
||||
from providers import register_provider
|
||||
from providers.base import ProviderProfile
|
||||
register_provider(ProviderProfile(
|
||||
name="my-provider",
|
||||
env_vars=("MY_API_KEY",),
|
||||
base_url="https://api.my-provider.example.com/v1",
|
||||
auth_type="api_key",
|
||||
))
|
||||
EOF
|
||||
|
||||
export MY_API_KEY=your-test-key
|
||||
hermes -z "hello" --provider my-provider -m some-model
|
||||
```
|
||||
|
||||
## General PluginManager integration
|
||||
|
||||
The general `PluginManager` (the thing `hermes plugins` operates on) **sees** model-provider plugins but does not import them — `providers/__init__.py` owns their lifecycle. The manager records the manifest for introspection and categorizes by `kind: model-provider`. When you drop an unlabeled user plugin into `$HERMES_HOME/plugins/` that happens to call `register_provider` with a `ProviderProfile`, the manager auto-coerces it to `kind: model-provider` via a source-text heuristic — so the plugin still routes correctly even without `plugin.yaml`.
|
||||
|
||||
## Distribute via pip
|
||||
|
||||
Like any Hermes plugin, model providers can ship as a pip package. Add an entry point to your `pyproject.toml`:
|
||||
|
||||
```toml
|
||||
[project.entry-points."hermes_agent.plugins"]
|
||||
acme-inference = "acme_hermes_plugin:register"
|
||||
```
|
||||
|
||||
…where `acme_hermes_plugin:register` is a function that calls `register_provider(profile)`. The general PluginManager picks up entry-point plugins during `discover_and_load()`. For `kind: model-provider` pip plugins, you still need to declare the kind in your manifest (or rely on the source-text heuristic).
|
||||
|
||||
See [Building a Hermes Plugin](/developer-guide/plugins#distribute-via-pip) for the full entry-points setup.
|
||||
|
||||
## Related pages
|
||||
|
||||
- [Provider Runtime](/developer-guide/provider-runtime) — resolution precedence + where each layer reads the profile
|
||||
- [Adding Providers](/developer-guide/adding-providers) — end-to-end checklist for new inference backends (covers both the fast plugin path and the full CLI/auth integration)
|
||||
- [Memory Provider Plugins](/developer-guide/memory-provider-plugin)
|
||||
- [Context Engine Plugins](/developer-guide/context-engine-plugin)
|
||||
- [Building a Hermes Plugin](/developer-guide/plugins) — general plugin authoring
|
||||
@@ -0,0 +1,465 @@
|
||||
---
|
||||
sidebar_position: 11
|
||||
title: "Plugin LLM Access"
|
||||
description: "Run any LLM call from inside a plugin via ctx.llm — chat or structured, sync or async. Host-owned auth, fail-closed trust gate, optional JSON Schema validation."
|
||||
---
|
||||
|
||||
# Plugin LLM Access
|
||||
|
||||
`ctx.llm` is the supported way for a plugin to make an LLM call.
|
||||
Chat completion, structured extraction, sync, async, with or without
|
||||
images — same surface, same trust gate, same host-owned credentials.
|
||||
|
||||
Plugins reach for this when they need to do something that involves
|
||||
the model but isn't part of the agent's conversation. A hook that
|
||||
rewrites a tool error into something a non-engineer can read. A
|
||||
gateway adapter that translates an inbound message before queuing
|
||||
it. A slash command that summarises a long paste. A scheduled job
|
||||
that scores yesterday's activity and writes one line to a status
|
||||
board. A pre-filter that decides whether a message is worth waking
|
||||
the agent up for at all.
|
||||
|
||||
These are jobs the agent shouldn't be in the loop on. They want one
|
||||
LLM call, a typed answer, and to be done.
|
||||
|
||||
## The smallest possible call
|
||||
|
||||
```python
|
||||
result = ctx.llm.complete(messages=[{"role": "user", "content": "ping"}])
|
||||
return result.text
|
||||
```
|
||||
|
||||
That's the whole API in one line. No keys, no provider config, no
|
||||
SDK initialisation. The plugin runs against whatever provider and
|
||||
model the user is currently using — when they switch providers, the
|
||||
plugin follows them automatically.
|
||||
|
||||
## A more complete chat example
|
||||
|
||||
```python
|
||||
result = ctx.llm.complete(
|
||||
messages=[
|
||||
{"role": "system", "content": "Rewrite errors as one short sentence a non-engineer can act on."},
|
||||
{"role": "user", "content": traceback_text},
|
||||
],
|
||||
max_tokens=64,
|
||||
purpose="hooks.error-rewrite",
|
||||
)
|
||||
return result.text
|
||||
```
|
||||
|
||||
`purpose` is a free-form audit string — it shows up in `agent.log`
|
||||
and in `result.audit` so operators can see which plugin made which
|
||||
call. Optional but recommended for anything that fires often.
|
||||
|
||||
## Structured output
|
||||
|
||||
When the plugin needs a typed answer, switch to the structured lane:
|
||||
|
||||
```python
|
||||
result = ctx.llm.complete_structured(
|
||||
instructions="Score this support reply for urgency (0–1) and pick a category.",
|
||||
input=[{"type": "text", "text": message_body}],
|
||||
json_schema=TRIAGE_SCHEMA,
|
||||
purpose="support.triage",
|
||||
temperature=0.0,
|
||||
max_tokens=128,
|
||||
)
|
||||
|
||||
if result.parsed["urgency"] > 0.8:
|
||||
await dispatch_to_oncall(result.parsed["category"], message_body)
|
||||
```
|
||||
|
||||
The host requests JSON output from the provider, parses it locally
|
||||
as a fallback, validates against your schema if `jsonschema` is
|
||||
installed, and hands back a Python object on `result.parsed`. If the
|
||||
model couldn't produce valid JSON, `result.parsed` is `None` and
|
||||
`result.text` carries the raw response.
|
||||
|
||||
## What this lane gives you
|
||||
|
||||
* **One call, four shapes.** `complete()` for chat,
|
||||
`complete_structured()` for typed JSON, `acomplete()` and
|
||||
`acomplete_structured()` for asyncio. Same arguments, same result
|
||||
objects.
|
||||
* **Host-owned credentials.** OAuth tokens, refresh flows, the
|
||||
credential pool, per-task aux overrides — every credential
|
||||
concept Hermes already has applies. The plugin never sees a
|
||||
token; the host attributes the call back through `result.audit`.
|
||||
* **Bounded.** Single sync or async call. No streaming, no tool
|
||||
loops, no conversation state to manage. State the input, get the
|
||||
result, return.
|
||||
* **Fail-closed trust.** A plugin you've never configured cannot
|
||||
pick its own provider, model, agent, or stored credential. The
|
||||
default posture is "use what the user is using." Operators opt in
|
||||
to specific overrides, per plugin, in `config.yaml`.
|
||||
|
||||
## Quick start
|
||||
|
||||
Two complete plugins below — one chat, one structured. Both ship
|
||||
inside a single `register(ctx)` function and need zero outside
|
||||
configuration to run against whatever model the user has active.
|
||||
|
||||
### Chat completion — `/tldr`
|
||||
|
||||
```python
|
||||
def register(ctx):
|
||||
ctx.register_command(
|
||||
name="tldr",
|
||||
handler=lambda raw: _tldr(ctx, raw),
|
||||
description="Summarise the supplied text in one paragraph.",
|
||||
args_hint="<text>",
|
||||
)
|
||||
|
||||
|
||||
def _tldr(ctx, raw_args: str) -> str:
|
||||
text = raw_args.strip()
|
||||
if not text:
|
||||
return "Usage: /tldr <text to summarise>"
|
||||
result = ctx.llm.complete(
|
||||
messages=[
|
||||
{"role": "system",
|
||||
"content": "Summarise the user's text in one tight paragraph. No preamble."},
|
||||
{"role": "user", "content": text},
|
||||
],
|
||||
max_tokens=256,
|
||||
temperature=0.3,
|
||||
purpose="tldr",
|
||||
)
|
||||
return result.text
|
||||
```
|
||||
|
||||
`result.text` is the model's response; `result.usage` carries token
|
||||
counts; `result.provider` and `result.model` carry attribution.
|
||||
|
||||
### Structured extraction — `/paste-to-tasks`
|
||||
|
||||
```python
|
||||
def register(ctx):
|
||||
ctx.register_command(
|
||||
name="paste-to-tasks",
|
||||
handler=lambda raw: _paste_to_tasks(ctx, raw),
|
||||
description="Turn freeform meeting notes into structured tasks.",
|
||||
args_hint="<text>",
|
||||
)
|
||||
|
||||
|
||||
_TASKS_SCHEMA = {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"tasks": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"owner": {"type": "string"},
|
||||
"action": {"type": "string"},
|
||||
"due": {"type": "string", "description": "ISO date or empty"},
|
||||
},
|
||||
"required": ["action"],
|
||||
},
|
||||
},
|
||||
},
|
||||
"required": ["tasks"],
|
||||
}
|
||||
|
||||
|
||||
def _paste_to_tasks(ctx, raw_args: str) -> str:
|
||||
if not raw_args.strip():
|
||||
return "Usage: /paste-to-tasks <meeting notes>"
|
||||
result = ctx.llm.complete_structured(
|
||||
instructions=(
|
||||
"Extract concrete action items from these meeting notes. "
|
||||
"One task per actionable line. If no owner is named, leave 'owner' blank."
|
||||
),
|
||||
input=[{"type": "text", "text": raw_args}],
|
||||
json_schema=_TASKS_SCHEMA,
|
||||
schema_name="meeting.tasks",
|
||||
purpose="paste-to-tasks",
|
||||
temperature=0.0,
|
||||
max_tokens=512,
|
||||
)
|
||||
if result.parsed is None:
|
||||
return f"Couldn't parse a response. Raw output:\n{result.text}"
|
||||
lines = [f"- [{t.get('owner') or '?'}] {t['action']}" for t in result.parsed["tasks"]]
|
||||
return "\n".join(lines) or "(no tasks found)"
|
||||
```
|
||||
|
||||
A third worked example, this time with image input, lives in the
|
||||
[`hermes-example-plugins`](https://github.com/NousResearch/hermes-example-plugins/tree/main/plugin-llm-example)
|
||||
repo (companion repo for reference plugins — not bundled with
|
||||
hermes-agent itself). For the async surface (`acomplete()` /
|
||||
`acomplete_structured()` with `asyncio.gather()`), see
|
||||
[`plugin-llm-async-example`](https://github.com/NousResearch/hermes-example-plugins/tree/main/plugin-llm-async-example)
|
||||
in the same repo.
|
||||
|
||||
## When to use which
|
||||
|
||||
| You want… | Reach for |
|
||||
|---|---|
|
||||
| A free-form text response (translation, summary, rewrite, generation) | `complete()` |
|
||||
| A multi-turn prompt (system + few-shot examples + user) | `complete()` |
|
||||
| A typed dict back, validated against a schema | `complete_structured()` |
|
||||
| Image-or-text input with a typed dict back | `complete_structured()` |
|
||||
| The same call from async code (gateway adapters, async hooks) | `acomplete()` / `acomplete_structured()` |
|
||||
|
||||
Everything else — provider selection, model resolution, auth, fallback,
|
||||
timeout, vision routing — is the same across all four.
|
||||
|
||||
## API surface
|
||||
|
||||
`ctx.llm` is an instance of `agent.plugin_llm.PluginLlm`.
|
||||
|
||||
### `complete()`
|
||||
|
||||
```python
|
||||
result = ctx.llm.complete(
|
||||
messages=[{"role": "user", "content": "Hi"}],
|
||||
provider=None, # optional, gated — Hermes provider id (e.g. "openrouter")
|
||||
model=None, # optional, gated — whatever string that provider expects
|
||||
temperature=None,
|
||||
max_tokens=None,
|
||||
timeout=None, # seconds
|
||||
agent_id=None, # optional, gated
|
||||
profile=None, # optional, gated — explicit auth-profile name
|
||||
purpose="optional-audit-string",
|
||||
)
|
||||
# → PluginLlmCompleteResult(text, provider, model, agent_id, usage, audit)
|
||||
```
|
||||
|
||||
Plain chat completion. `messages` is the standard OpenAI shape — a
|
||||
list of `{"role": "...", "content": "..."}` dicts. Multi-turn
|
||||
prompts (system + few-shot user/assistant pairs + final user) work
|
||||
exactly as they would with the OpenAI SDK.
|
||||
|
||||
`provider=` and `model=` are independent and follow the same shape
|
||||
as the host's main config (`model.provider` + `model.model`). Set
|
||||
just `model=` to use the user's active provider with a different
|
||||
model on it. Set both to switch providers entirely. Either argument
|
||||
without operator opt-in raises `PluginLlmTrustError`.
|
||||
|
||||
### `complete_structured()`
|
||||
|
||||
```python
|
||||
result = ctx.llm.complete_structured(
|
||||
instructions="What you want extracted.",
|
||||
input=[
|
||||
{"type": "text", "text": "..."},
|
||||
{"type": "image", "data": b"...", "mime_type": "image/png"},
|
||||
{"type": "image", "url": "https://..."},
|
||||
],
|
||||
json_schema={...}, # optional — triggers parsed result + validation
|
||||
json_mode=False, # set True without a schema to ask for JSON anyway
|
||||
schema_name=None, # optional human-readable schema name
|
||||
system_prompt=None,
|
||||
provider=None, # optional, gated
|
||||
model=None, # optional, gated
|
||||
temperature=None,
|
||||
max_tokens=None,
|
||||
timeout=None,
|
||||
agent_id=None,
|
||||
profile=None,
|
||||
purpose=None,
|
||||
)
|
||||
# → PluginLlmStructuredResult(text, provider, model, agent_id,
|
||||
# usage, parsed, content_type, audit)
|
||||
```
|
||||
|
||||
Inputs are typed text or image blocks (raw bytes get base64 encoded
|
||||
as a `data:` URL automatically). When `json_schema` or
|
||||
`json_mode=True` is supplied, the host requests JSON output via
|
||||
`response_format`, parses it locally as a fallback, and validates
|
||||
against your schema if `jsonschema` is installed.
|
||||
|
||||
* `result.content_type == "json"` — `result.parsed` is a Python
|
||||
object that matches your schema.
|
||||
* `result.content_type == "text"` — parsing or validation failed;
|
||||
inspect `result.text` for the raw model response.
|
||||
|
||||
### Async
|
||||
|
||||
```python
|
||||
result = await ctx.llm.acomplete(messages=...)
|
||||
result = await ctx.llm.acomplete_structured(instructions=..., input=...)
|
||||
```
|
||||
|
||||
Same arguments and result types as their sync counterparts. Use
|
||||
these from gateway adapters, async hooks, or any plugin code
|
||||
already running on an asyncio loop.
|
||||
|
||||
### Result attributes
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class PluginLlmCompleteResult:
|
||||
text: str # the assistant's response
|
||||
provider: str # e.g. "openrouter", "anthropic"
|
||||
model: str # whatever the provider returned for this call
|
||||
agent_id: str # whose model/auth was used
|
||||
usage: PluginLlmUsage # tokens + cache + cost estimate
|
||||
audit: Dict[str, Any] # plugin_id, purpose, profile
|
||||
|
||||
@dataclass
|
||||
class PluginLlmStructuredResult(PluginLlmCompleteResult):
|
||||
parsed: Optional[Any] # JSON object when content_type == "json"
|
||||
content_type: str # "json" or "text"
|
||||
# audit also carries schema_name when supplied
|
||||
```
|
||||
|
||||
`usage` carries `input_tokens`, `output_tokens`, `total_tokens`,
|
||||
`cache_read_tokens`, `cache_write_tokens`, and `cost_usd` when the
|
||||
provider returns those fields.
|
||||
|
||||
## Trust gate
|
||||
|
||||
The default behaviour is fail-closed. With no `plugins.entries`
|
||||
config block, a plugin can:
|
||||
|
||||
* run any of the four methods against the user's active provider
|
||||
and model,
|
||||
* set request-shaping arguments (`temperature`, `max_tokens`,
|
||||
`timeout`, `system_prompt`, `purpose`, `messages`, `instructions`,
|
||||
`input`, `json_schema`),
|
||||
|
||||
…and that's it. `provider=`, `model=`, `agent_id=`, and `profile=`
|
||||
arguments raise `PluginLlmTrustError` until the operator opts in.
|
||||
|
||||
**Most plugins never need this section.** A plugin that just calls
|
||||
`ctx.llm.complete(messages=...)` with no overrides runs against
|
||||
whatever the user has active and works zero-config. The block below
|
||||
is only relevant when a plugin specifically wants to pin to a
|
||||
different model or provider than the user.
|
||||
|
||||
```yaml
|
||||
plugins:
|
||||
entries:
|
||||
my-plugin:
|
||||
llm:
|
||||
# Allow this plugin to choose a different Hermes provider
|
||||
# (must be one Hermes already knows about — same names as
|
||||
# `hermes model` and config.yaml model.provider).
|
||||
allow_provider_override: true
|
||||
|
||||
# Optionally restrict which providers. Use ["*"] for any.
|
||||
allowed_providers:
|
||||
- openrouter
|
||||
- anthropic
|
||||
|
||||
# Allow this plugin to ask for a specific model.
|
||||
allow_model_override: true
|
||||
|
||||
# Optionally restrict which models. Use ["*"] for any.
|
||||
# Models are matched literally against whatever string the
|
||||
# plugin sends — Hermes does not look anything up.
|
||||
allowed_models:
|
||||
- openai/gpt-4o-mini
|
||||
- anthropic/claude-3-5-haiku
|
||||
|
||||
# Allow cross-agent calls (rare).
|
||||
allow_agent_id_override: false
|
||||
|
||||
# Allow the plugin to request a specific stored auth profile
|
||||
# (e.g. a different OAuth account on the same provider).
|
||||
allow_profile_override: false
|
||||
```
|
||||
|
||||
The plugin id is the manifest `name:` field for flat plugins, or the
|
||||
path-derived key for nested plugins (`image_gen/openai`,
|
||||
`memory/honcho`, etc.).
|
||||
|
||||
### What the gate enforces
|
||||
|
||||
| Override | Default | Config key |
|
||||
| --------------- | ------- | -------------------------------- |
|
||||
| `provider=` | denied | `allow_provider_override: true` |
|
||||
| ↳ allowlist | — | `allowed_providers: [...]` |
|
||||
| `model=` | denied | `allow_model_override: true` |
|
||||
| ↳ allowlist | — | `allowed_models: [...]` |
|
||||
| `agent_id=` | denied | `allow_agent_id_override: true` |
|
||||
| `profile=` | denied | `allow_profile_override: true` |
|
||||
|
||||
Each override is independently gated. Granting `allow_model_override`
|
||||
does **not** also grant `allow_provider_override` — a plugin trusted
|
||||
to pick a model is still pinned to the user's active provider unless
|
||||
it gets the provider gate as well.
|
||||
|
||||
### What the gate does NOT need to enforce
|
||||
|
||||
* Request-shaping arguments — `temperature`, `max_tokens`,
|
||||
`timeout`, `system_prompt`, `purpose`, `messages`, `instructions`,
|
||||
`input`, `json_schema`, `schema_name`, `json_mode` — are always
|
||||
allowed; they don't pick credentials or routes.
|
||||
* The default deny posture means an unconfigured plugin can still do
|
||||
useful work — it just runs against the active provider and model.
|
||||
Operators only need to think about `plugins.entries` for plugins
|
||||
that want finer routing.
|
||||
|
||||
## What the host owns
|
||||
|
||||
A complete list of the things `ctx.llm` does for the plugin so you
|
||||
don't have to:
|
||||
|
||||
* **Provider resolution.** Reads `model.provider` + `model.model`
|
||||
from the user's config (or the explicit overrides when trusted).
|
||||
* **Auth.** Pulls API keys, OAuth tokens, or refresh tokens from
|
||||
`~/.hermes/auth.json` / env, including the credential pool when
|
||||
one is configured. The plugin never sees them.
|
||||
* **Vision routing.** When image input is supplied and the user's
|
||||
active text model is text-only, the host falls back to the
|
||||
configured vision model automatically.
|
||||
* **Fallback chain.** If the user's primary provider 5xxs or 429s,
|
||||
the request goes through Hermes' usual aggregator-aware fallback
|
||||
before it returns an error to the plugin.
|
||||
* **Timeout.** Honours your `timeout=` argument, falling back to
|
||||
`auxiliary.<task>.timeout` config or the global aux default.
|
||||
* **JSON shaping.** Sends `response_format` to the provider when
|
||||
you ask for JSON, then re-parses locally from a code-fenced
|
||||
response if the provider returned one.
|
||||
* **Schema validation.** Validates against your `json_schema` when
|
||||
`jsonschema` is installed; logs a debug line and skips strict
|
||||
validation otherwise.
|
||||
* **Audit log.** Each call writes one INFO line to `agent.log` with
|
||||
the plugin id, provider/model, purpose, and token totals.
|
||||
|
||||
## What the plugin owns
|
||||
|
||||
* **Request shape.** `messages` for chat, `instructions` + `input`
|
||||
for structured. The plugin builds the prompt; the host runs it.
|
||||
* **Schema.** Whatever shape you want back. The host doesn't infer
|
||||
it for you.
|
||||
* **Error handling.** `complete_structured()` raises `ValueError` on
|
||||
empty inputs and on schema-validation failure. `PluginLlmTrustError`
|
||||
fires when the trust gate denies an override. Anything else
|
||||
(provider 5xx, no credentials configured, timeout) raises whatever
|
||||
`auxiliary_client.call_llm()` raises.
|
||||
* **Cost.** Every call runs against the user's paid provider. Don't
|
||||
loop on `complete()` for every gateway message without thinking
|
||||
about token spend.
|
||||
|
||||
## Where this fits in the plugin surface
|
||||
|
||||
Existing `ctx.*` methods extend an existing Hermes subsystem:
|
||||
|
||||
| `ctx.register_tool` | adds a tool the agent can call |
|
||||
| `ctx.register_platform` | wires a new gateway adapter |
|
||||
| `ctx.register_image_gen_provider` | replaces an image-gen backend |
|
||||
| `ctx.register_memory_provider` | replaces the memory backend |
|
||||
| `ctx.register_context_engine` | replaces the context compressor |
|
||||
| `ctx.register_hook` | observes a lifecycle event |
|
||||
|
||||
`ctx.llm` is the first surface that lets a plugin run the same
|
||||
model the user is talking to, *out of band*, without any of the
|
||||
above. That's its only job. If your plugin needs to register a
|
||||
tool the agent invokes, use `register_tool`. If it needs to react
|
||||
to a lifecycle event, use `register_hook`. If it needs to make its
|
||||
own model call — for any reason, structured or not — `ctx.llm`.
|
||||
|
||||
## Reference
|
||||
|
||||
* Implementation: [`agent/plugin_llm.py`](https://github.com/NousResearch/hermes-agent/blob/main/agent/plugin_llm.py)
|
||||
* Tests: [`tests/agent/test_plugin_llm.py`](https://github.com/NousResearch/hermes-agent/blob/main/tests/agent/test_plugin_llm.py)
|
||||
* Reference plugins (companion repo):
|
||||
* [`plugin-llm-example`](https://github.com/NousResearch/hermes-example-plugins/tree/main/plugin-llm-example) — sync structured extraction with image input
|
||||
* [`plugin-llm-async-example`](https://github.com/NousResearch/hermes-example-plugins/tree/main/plugin-llm-async-example) — async with `asyncio.gather()`
|
||||
* Auxiliary client (the engine under the hood): see
|
||||
[Provider Runtime](/developer-guide/provider-runtime).
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,129 @@
|
||||
---
|
||||
sidebar_position: 8
|
||||
title: "Programmatic Integration"
|
||||
description: "Three protocols for driving hermes-agent from external programs: ACP, the TUI gateway JSON-RPC, and the OpenAI-compatible HTTP API"
|
||||
---
|
||||
|
||||
# Programmatic Integration
|
||||
|
||||
Hermes ships three protocols for driving the agent from external programs — IDE plugins, custom UIs, CI pipelines, embedded sub-agents. Pick the one that matches your transport and consumer.
|
||||
|
||||
| Protocol | Transport | Best for | Defined by |
|
||||
|----------|-----------|----------|------------|
|
||||
| **ACP** | JSON-RPC over stdio | IDE clients (VS Code, Zed, JetBrains) that already speak the [Agent Client Protocol](https://github.com/zed-industries/agent-client-protocol) | `acp_adapter/` |
|
||||
| **TUI gateway** | JSON-RPC over stdio (or WebSocket) | Custom hosts that want fine-grained control of sessions, slash commands, approvals, and streaming events | `tui_gateway/server.py` |
|
||||
| **API server** | HTTP + Server-Sent Events | OpenAI-compatible frontends (Open WebUI, LobeChat, LibreChat…) and language-agnostic web clients | `gateway/platforms/api_server.py` |
|
||||
|
||||
All three drive the same `AIAgent` core. They differ only in wire format and which set of features they expose.
|
||||
|
||||
---
|
||||
|
||||
## ACP (Agent Client Protocol)
|
||||
|
||||
`hermes acp` starts a stdio JSON-RPC server speaking ACP. Used in production by VS Code (Zed Industries' ACP extension), Zed, and any JetBrains IDE with an ACP plugin.
|
||||
|
||||
Capabilities exposed: session creation, prompt submission, streaming agent message chunks, tool-call events, permission requests, session fork, cancel, and authentication. Tool output is rendered into ACP `Diff`/`ToolCall` content blocks the IDE understands.
|
||||
|
||||
Full lifecycle, event bridge, and approval flow: [ACP Internals](./acp-internals).
|
||||
|
||||
```bash
|
||||
hermes acp # serve ACP on stdio
|
||||
hermes acp --bootstrap # print install snippet for an ACP-capable IDE
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## TUI Gateway JSON-RPC
|
||||
|
||||
`tui_gateway/server.py` is the protocol the Ink TUI (`hermes --tui`) and the embedded dashboard PTY bridge talk to. Any external host can speak the same protocol over stdio (or WebSocket via `tui_gateway/ws.py`).
|
||||
|
||||
### Method catalog (selected)
|
||||
|
||||
```
|
||||
prompt.submit prompt.background session.steer
|
||||
session.create session.list session.active_list
|
||||
session.activate session.close session.interrupt
|
||||
session.history session.compress session.branch
|
||||
session.title session.usage session.status
|
||||
clarify.respond sudo.respond secret.respond
|
||||
approval.respond config.set / config.get commands.catalog
|
||||
command.resolve command.dispatch cli.exec
|
||||
reload.mcp reload.env process.stop
|
||||
delegation.status subagent.interrupt spawn_tree.save / list / load
|
||||
terminal.resize clipboard.paste image.attach
|
||||
```
|
||||
|
||||
`session.active_list`, `session.activate`, and `session.close` are the process-local live-session controls used by the TUI session switcher. Use `session.list` / `/resume` for saved transcript discovery; use the active-session methods only for sessions that are currently open in the TUI gateway process.
|
||||
|
||||
### Events streamed back
|
||||
|
||||
`message.delta`, `message.complete`, `tool.start`, `tool.progress`, `tool.complete`, `approval.request`, `clarify.request`, `sudo.request`, `sudo.expire`, `secret.request`, `secret.expire`, `gateway.ready`, plus session lifecycle and error events. Expiry events carry the original `{ request_id }`; external hosts should clear only the matching pending prompt.
|
||||
|
||||
### Pi-style RPC mapping
|
||||
|
||||
Every command in the Pi-mono RPC spec ([issue #360](https://github.com/NousResearch/hermes-agent/issues/360)) has a TUI-gateway equivalent:
|
||||
|
||||
| Pi command | Hermes equivalent |
|
||||
|------------|-------------------|
|
||||
| `prompt` | `prompt.submit` (or ACP `session/prompt`) |
|
||||
| `steer` | `session.steer` |
|
||||
| `follow_up` | `prompt.submit` queued after current turn |
|
||||
| `abort` | `session.interrupt` |
|
||||
| `set_model` | `command.dispatch` for `/model <provider:model>` (mid-session, persistent) |
|
||||
| `compact` | `session.compress` |
|
||||
| `get_state` | `session.status` |
|
||||
| `get_messages` | `session.history` |
|
||||
| `switch_session` | `session.resume` |
|
||||
| `fork` | `session.branch` |
|
||||
| `ui_request` / `ui_response` | `clarify.respond` / `sudo.respond` / `secret.respond` / `approval.respond` |
|
||||
|
||||
---
|
||||
|
||||
## OpenAI-Compatible API Server
|
||||
|
||||
`gateway/platforms/api_server.py` exposes hermes over HTTP for any client that already speaks the OpenAI format. Useful when you want a web frontend, a curl-driven CI runner, or a non-Python consumer.
|
||||
|
||||
Endpoints:
|
||||
|
||||
```
|
||||
POST /v1/chat/completions OpenAI Chat Completions (streaming via SSE)
|
||||
POST /v1/responses OpenAI Responses API (stateful)
|
||||
POST /v1/runs Start a run, returns run_id (202)
|
||||
GET /v1/runs/{id} Run status
|
||||
GET /v1/runs/{id}/events SSE stream of lifecycle events
|
||||
POST /v1/runs/{id}/approval Resolve a pending approval
|
||||
POST /v1/runs/{id}/stop Interrupt the run
|
||||
GET /v1/capabilities Machine-readable feature flags
|
||||
GET /v1/models Lists hermes-agent
|
||||
GET /health, /health/detailed
|
||||
```
|
||||
|
||||
Setup, headers (`X-Hermes-Session-Id`, `X-Hermes-Session-Key`), and frontend wiring: [API Server](../user-guide/features/api-server).
|
||||
|
||||
---
|
||||
|
||||
## Which one should I use?
|
||||
|
||||
- **You're writing an IDE plugin and the IDE already speaks ACP** → ACP. Zero protocol work on the IDE side.
|
||||
- **You're writing a custom desktop / web / TUI host and want every Hermes feature** (slash commands, approvals, clarify, multi-agent, session branching) → TUI gateway JSON-RPC.
|
||||
- **You want any OpenAI-compatible frontend, a language-agnostic HTTP client, or curl-driven automation** → API server.
|
||||
- **You want a Python in-process embed without a subprocess** → import `run_agent.AIAgent` directly. See [Agent Loop](./agent-loop).
|
||||
|
||||
---
|
||||
|
||||
## Model hot-swapping
|
||||
|
||||
Mid-session model switching works on every surface — it's the `/model` slash command under the hood.
|
||||
|
||||
- **CLI / TUI:** `/model claude-sonnet-4` or `/model openrouter:anthropic/claude-sonnet-4.6`
|
||||
- **TUI gateway RPC:** `command.dispatch` with `{"command": "/model claude-sonnet-4"}`
|
||||
- **ACP:** the IDE sends the slash command as a prompt; the agent dispatches it
|
||||
- **API server:** include a `model` field in the request body or set `X-Hermes-Model`
|
||||
|
||||
Provider-aware resolution (the same model name picks the right format for whatever provider you're on) is built in. See `hermes_cli/model_switch.py`.
|
||||
|
||||
---
|
||||
|
||||
## A note on `--mode rpc`
|
||||
|
||||
Hermes does not have a `--mode rpc` flag. The three protocols above already cover the use cases — ACP for IDE-protocol clients, the TUI gateway for stdio JSON-RPC hosts, and the API server for HTTP. If you find a real gap that none of them fill, open an issue with the concrete consumer you're building.
|
||||
@@ -0,0 +1,309 @@
|
||||
---
|
||||
sidebar_position: 5
|
||||
title: "Prompt Assembly"
|
||||
description: "How Hermes builds the system prompt, preserves cache stability, and injects ephemeral layers"
|
||||
---
|
||||
|
||||
# Prompt Assembly
|
||||
|
||||
Hermes deliberately separates:
|
||||
|
||||
- **cached system prompt state**
|
||||
- **ephemeral API-call-time additions**
|
||||
|
||||
This is one of the most important design choices in the project because it affects:
|
||||
|
||||
- token usage
|
||||
- prompt caching effectiveness
|
||||
- session continuity
|
||||
- memory correctness
|
||||
|
||||
Primary files:
|
||||
|
||||
- `run_agent.py`
|
||||
- `agent/prompt_builder.py`
|
||||
- `tools/memory_tool.py`
|
||||
|
||||
## Cached system prompt layers
|
||||
|
||||
The cached system prompt is assembled as three ordered tiers (see `agent/system_prompt.py`):
|
||||
|
||||
1. **stable** — identity (`SOUL.md` or fallback), tool/model guidance, skills prompt, environment hints, platform hints
|
||||
2. **context** — caller-supplied `system_message` plus project context files (`.hermes.md` / `AGENTS.md` / `CLAUDE.md` / `.cursorrules`)
|
||||
3. **volatile** — built-in memory snapshot (`MEMORY.md`), user profile snapshot (`USER.md`), external memory-provider block, timestamp/session/model/provider line
|
||||
|
||||
The final system prompt is then joined as: `stable` → `context` → `volatile`.
|
||||
|
||||
This ordering matters for precedence discussions:
|
||||
- skills are part of the **stable** tier
|
||||
- memory/profile snapshots are part of the **volatile** tier
|
||||
- both are still in the cached system prompt (they are not injected as ad-hoc mid-turn overlays)
|
||||
|
||||
When `skip_context_files` is set (e.g., subagent delegation), SOUL.md is not loaded and the hardcoded `DEFAULT_AGENT_IDENTITY` is used instead.
|
||||
|
||||
### Concrete example: assembled system prompt
|
||||
|
||||
Here is a simplified view of what the final system prompt looks like when all layers are present (comments show the source of each section):
|
||||
|
||||
```
|
||||
# Layer 1: Agent Identity (from ~/.hermes/SOUL.md)
|
||||
You are Hermes, an AI assistant created by Nous Research.
|
||||
You are an expert software engineer and researcher.
|
||||
You value correctness, clarity, and efficiency.
|
||||
...
|
||||
|
||||
# Layer 2: Tool-aware behavior guidance
|
||||
You have persistent memory across sessions. Save durable facts using
|
||||
the memory tool: user preferences, environment details, tool quirks,
|
||||
and stable conventions. Memory is injected into every turn, so keep
|
||||
it compact and focused on facts that will still matter later.
|
||||
...
|
||||
When the user references something from a past conversation or you
|
||||
suspect relevant cross-session context exists, use session_search
|
||||
to recall it before asking them to repeat themselves.
|
||||
|
||||
# Tool-use enforcement (for GPT/Codex models only)
|
||||
You MUST use your tools to take action — do not describe what you
|
||||
would do or plan to do without actually doing it.
|
||||
...
|
||||
|
||||
# Layer 3: Honcho static block (when active)
|
||||
[Honcho personality/context data]
|
||||
|
||||
# Layer 4: Optional system message (from config or API)
|
||||
[User-configured system message override]
|
||||
|
||||
# Layer 5: Frozen MEMORY snapshot
|
||||
## Persistent Memory
|
||||
- User prefers Python 3.12, uses pyproject.toml
|
||||
- Default editor is nvim
|
||||
- Working on project "atlas" in ~/code/atlas
|
||||
- Timezone: US/Pacific
|
||||
|
||||
# Layer 6: Frozen USER profile snapshot
|
||||
## User Profile
|
||||
- Name: Alice
|
||||
- GitHub: alice-dev
|
||||
|
||||
# Layer 7: Skills index
|
||||
## Skills (mandatory)
|
||||
Before replying, scan the skills below. If one clearly matches
|
||||
your task, load it with skill_view(name) and follow its instructions.
|
||||
...
|
||||
<available_skills>
|
||||
software-development:
|
||||
- code-review: Structured code review workflow
|
||||
- test-driven-development: TDD methodology
|
||||
research:
|
||||
- arxiv: Search and summarize arXiv papers
|
||||
</available_skills>
|
||||
|
||||
# Layer 8: Context files (from project directory)
|
||||
# Project Context
|
||||
The following project context files have been loaded and should be followed:
|
||||
|
||||
## AGENTS.md
|
||||
This is the atlas project. Use pytest for testing. The main
|
||||
entry point is src/atlas/main.py. Always run `make lint` before
|
||||
committing.
|
||||
|
||||
# Layer 9: Timestamp + session
|
||||
Current time: 2026-03-30T14:30:00-07:00
|
||||
Session: abc123
|
||||
|
||||
# Layer 10: Platform hint
|
||||
You are a CLI AI Agent. Try not to use markdown but simple text
|
||||
renderable inside a terminal.
|
||||
```
|
||||
|
||||
## Customizing platform hints
|
||||
|
||||
The platform hint (Layer 10 above) is the per-surface guidance Hermes
|
||||
injects for Telegram, WhatsApp, Slack, CLI, and other platforms — for
|
||||
example "you are on a terminal, avoid Markdown." The built-in defaults
|
||||
live in `PLATFORM_HINTS` (`agent/system_prompt.py`); plugin-provided
|
||||
platforms supply theirs through the platform registry.
|
||||
|
||||
An administrator can append to or replace a single platform's hint from
|
||||
`config.yaml` via the top-level `platform_hints` key, without touching
|
||||
any other platform:
|
||||
|
||||
```yaml
|
||||
platform_hints:
|
||||
whatsapp:
|
||||
append: >
|
||||
When tabular output would be useful, invoke the table_formatting
|
||||
skill instead of emitting a Markdown table.
|
||||
slack:
|
||||
replace: "You are on Slack. Keep responses tight and avoid wide tables."
|
||||
telegram: "Prefer short messages; split long answers." # shorthand = append
|
||||
```
|
||||
|
||||
- `append` — keep the built-in hint and add the extra text after it.
|
||||
- `replace` — substitute the built-in hint entirely.
|
||||
- A bare string — shorthand for `append`.
|
||||
- `replace` wins over `append` when both are present.
|
||||
- A malformed entry is ignored defensively and falls back to the
|
||||
unmodified default, so a bad config value can never break prompt
|
||||
assembly or leak across platforms.
|
||||
|
||||
The override is resolved when the system prompt is built (session start,
|
||||
and again on compaction since that rebuilds the prompt). It produces a
|
||||
byte-stable hint for a fixed config, so it lives in the **stable** tier
|
||||
alongside the built-in hint and does not break prompt caching — it is
|
||||
not a live mid-session mutation of a frozen prompt.
|
||||
|
||||
## How SOUL.md appears in the prompt
|
||||
|
||||
`SOUL.md` lives at `~/.hermes/SOUL.md` and serves as the agent's identity — the very first section of the system prompt. The loading logic in `prompt_builder.py` works as follows:
|
||||
|
||||
```python
|
||||
# From agent/prompt_builder.py (simplified)
|
||||
def load_soul_md() -> Optional[str]:
|
||||
soul_path = get_hermes_home() / "SOUL.md"
|
||||
if not soul_path.exists():
|
||||
return None
|
||||
content = soul_path.read_text(encoding="utf-8").strip()
|
||||
content = _scan_context_content(content, "SOUL.md") # Security scan
|
||||
content = _truncate_content(content, "SOUL.md") # Cap defaults to 20k chars, configurable
|
||||
return content
|
||||
```
|
||||
|
||||
When `load_soul_md()` returns content, it replaces the hardcoded `DEFAULT_AGENT_IDENTITY`. The `build_context_files_prompt()` function is then called with `skip_soul=True` to prevent SOUL.md from appearing twice (once as identity, once as a context file).
|
||||
|
||||
If `SOUL.md` doesn't exist, the system falls back to:
|
||||
|
||||
```
|
||||
You are Hermes Agent, an intelligent AI assistant created by Nous Research.
|
||||
You are helpful, knowledgeable, and direct. You assist users with a wide
|
||||
range of tasks including answering questions, writing and editing code,
|
||||
analyzing information, creative work, and executing actions via your tools.
|
||||
You communicate clearly, admit uncertainty when appropriate, and prioritize
|
||||
being genuinely useful over being verbose unless otherwise directed below.
|
||||
Be targeted and efficient in your exploration and investigations.
|
||||
```
|
||||
|
||||
## How context files are injected
|
||||
|
||||
`build_context_files_prompt()` uses a **priority system** — only one project context type is loaded (first match wins):
|
||||
|
||||
```python
|
||||
# From agent/prompt_builder.py (simplified)
|
||||
def build_context_files_prompt(cwd=None, skip_soul=False):
|
||||
cwd_path = Path(cwd).resolve()
|
||||
|
||||
# Priority: first match wins — only ONE project context loaded
|
||||
project_context = (
|
||||
_load_hermes_md(cwd_path) # 1. .hermes.md / HERMES.md (walks to git root)
|
||||
or _load_agents_md(cwd_path) # 2. AGENTS.md (cwd only)
|
||||
or _load_claude_md(cwd_path) # 3. CLAUDE.md (cwd only)
|
||||
or _load_cursorrules(cwd_path) # 4. .cursorrules / .cursor/rules/*.mdc
|
||||
)
|
||||
|
||||
sections = []
|
||||
if project_context:
|
||||
sections.append(project_context)
|
||||
|
||||
# SOUL.md from HERMES_HOME (independent of project context)
|
||||
if not skip_soul:
|
||||
soul_content = load_soul_md()
|
||||
if soul_content:
|
||||
sections.append(soul_content)
|
||||
|
||||
if not sections:
|
||||
return ""
|
||||
|
||||
return (
|
||||
"# Project Context\n\n"
|
||||
"The following project context files have been loaded "
|
||||
"and should be followed:\n\n"
|
||||
+ "\n".join(sections)
|
||||
)
|
||||
```
|
||||
|
||||
### Context file discovery details
|
||||
|
||||
| Priority | Files | Search scope | Notes |
|
||||
|----------|-------|-------------|-------|
|
||||
| 1 | `.hermes.md`, `HERMES.md` | CWD up to git root | Hermes-native project config |
|
||||
| 2 | `AGENTS.md` | CWD only | Common agent instruction file |
|
||||
| 3 | `CLAUDE.md` | CWD only | Claude Code compatibility |
|
||||
| 4 | `.cursorrules`, `.cursor/rules/*.mdc` | CWD only | Cursor compatibility |
|
||||
|
||||
All context files are:
|
||||
- **Security scanned** — checked for prompt injection patterns (invisible unicode, "ignore previous instructions", credential exfiltration attempts)
|
||||
- **Truncated** — capped at `context_file_max_chars` characters (default 20,000) using 70/20 head/tail ratio with a truncation marker
|
||||
- **YAML frontmatter stripped** — `.hermes.md` frontmatter is removed (reserved for future config overrides)
|
||||
|
||||
## API-call-time-only layers
|
||||
|
||||
These are intentionally *not* persisted as part of the cached system prompt:
|
||||
|
||||
- `ephemeral_system_prompt`
|
||||
- prefill messages
|
||||
- gateway-derived session context overlays
|
||||
- later-turn Honcho/external recall injected into the current-turn user message
|
||||
|
||||
`pre_llm_call` plugin context also lands in this API-call-time path: it is appended to the current turn's **user message**, not written into the cached system prompt. When multiple plugins return context, Hermes concatenates those context blocks (see [Hooks → `pre_llm_call`](../user-guide/features/hooks.md#pre_llm_call)).
|
||||
|
||||
This separation keeps the stable prefix stable for caching.
|
||||
|
||||
## Memory snapshots
|
||||
|
||||
Local memory and user profile data are captured in the system prompt's **volatile tier**. Mid-session writes update disk state but do not mutate the already-built cached system prompt until a rebuild path runs (new session, or explicit invalidation/rebuild flow such as compression-triggered rebuild).
|
||||
|
||||
## Context files
|
||||
|
||||
`agent/prompt_builder.py` scans and sanitizes project context files using a **priority system** — only one type is loaded (first match wins):
|
||||
|
||||
1. `.hermes.md` / `HERMES.md` (walks to git root)
|
||||
2. `AGENTS.md` (CWD at startup; subdirectories discovered progressively during the session via `agent/subdirectory_hints.py`)
|
||||
3. `CLAUDE.md` (CWD only)
|
||||
4. `.cursorrules` / `.cursor/rules/*.mdc` (CWD only)
|
||||
|
||||
`SOUL.md` is loaded separately via `load_soul_md()` for the identity slot. When it loads successfully, `build_context_files_prompt(skip_soul=True)` prevents it from appearing twice.
|
||||
|
||||
Long files are truncated before injection.
|
||||
|
||||
## Skills index
|
||||
|
||||
The skills system contributes a compact skills index to the prompt when skills tooling is available.
|
||||
|
||||
## Supported prompt customization surfaces
|
||||
|
||||
Most users should treat `agent/prompt_builder.py` as implementation code, not a configuration surface. The supported customization path is to change the prompt inputs Hermes already loads, rather than editing Python templates in place.
|
||||
|
||||
### Use these surfaces first
|
||||
|
||||
- `~/.hermes/SOUL.md` — replace the built-in default identity block with your own agent persona and standing behavior.
|
||||
- `~/.hermes/MEMORY.md` and `~/.hermes/USER.md` — provide durable cross-session facts and user profile data that should be snapshotted into new sessions.
|
||||
- Project context files such as `.hermes.md`, `HERMES.md`, `AGENTS.md`, `CLAUDE.md`, or `.cursorrules` — inject repo-specific working rules.
|
||||
- Skills — package reusable workflows and references without editing core prompt code.
|
||||
- Optional system prompt config / API overrides — add deployment-specific instruction text without forking Hermes.
|
||||
- Ephemeral overlays such as `HERMES_EPHEMERAL_SYSTEM_PROMPT` or prefill messages — add turn-scoped guidance that should not become part of the cached prompt prefix.
|
||||
|
||||
### When to edit code instead
|
||||
|
||||
Edit `agent/prompt_builder.py` only if you are intentionally maintaining a fork or contributing upstream behavior changes. That file assembles the prompt plumbing, cache boundaries, and injection order for every session. Direct edits there are global product changes, not per-user prompt customization.
|
||||
|
||||
In other words:
|
||||
|
||||
- if you want a different assistant identity, edit `SOUL.md`
|
||||
- if you want different repo rules, edit project context files
|
||||
- if you want reusable operating procedures, add or modify skills
|
||||
- if you want to change how Hermes assembles prompts for everyone, change Python and treat it as a code contribution
|
||||
|
||||
## Why prompt assembly is split this way
|
||||
|
||||
The architecture is intentionally optimized to:
|
||||
|
||||
- preserve provider-side prompt caching
|
||||
- avoid mutating history unnecessarily
|
||||
- keep memory semantics understandable
|
||||
- let gateway/ACP/CLI add context without poisoning persistent prompt state
|
||||
|
||||
## Related docs
|
||||
|
||||
- [Context Compression & Prompt Caching](./context-compression-and-caching.md)
|
||||
- [Session Storage](./session-storage.md)
|
||||
- [Gateway Internals](./gateway-internals.md)
|
||||
@@ -0,0 +1,206 @@
|
||||
---
|
||||
sidebar_position: 4
|
||||
title: "Provider Runtime Resolution"
|
||||
description: "How Hermes resolves providers, credentials, API modes, and auxiliary models at runtime"
|
||||
---
|
||||
|
||||
# Provider Runtime Resolution
|
||||
|
||||
Hermes has a shared provider runtime resolver used across:
|
||||
|
||||
- CLI
|
||||
- gateway
|
||||
- cron jobs
|
||||
- ACP
|
||||
- auxiliary model calls
|
||||
|
||||
Primary implementation:
|
||||
|
||||
- `hermes_cli/runtime_provider.py` — credential resolution, `_resolve_custom_runtime()`
|
||||
- `hermes_cli/auth.py` — provider registry, `resolve_provider()`
|
||||
- `hermes_cli/model_switch.py` — shared `/model` switch pipeline (CLI + gateway)
|
||||
- `agent/auxiliary_client.py` — auxiliary model routing
|
||||
- `providers/` — ABC + registry entry points (`ProviderProfile`, `register_provider`, `get_provider_profile`, `list_providers`)
|
||||
- `plugins/model-providers/<name>/` — per-provider plugins (bundled) that declare `api_mode`, `base_url`, `env_vars`, `fallback_models` and register themselves into the registry on first access. User plugins at `$HERMES_HOME/plugins/model-providers/<name>/` override bundled ones of the same name.
|
||||
|
||||
`get_provider_profile()` in `providers/` returns a `ProviderProfile` for a given provider id. `runtime_provider.py` calls this at resolution time to get the canonical `base_url`, `env_vars` priority list, `api_mode`, and `fallback_models` without needing to duplicate that data in multiple files. Adding a new plugin under `plugins/model-providers/<your-provider>/` (or `$HERMES_HOME/plugins/model-providers/<your-provider>/`) that calls `register_provider()` is enough for `runtime_provider.py` to pick it up — no branch needed in the resolver itself.
|
||||
|
||||
If you are trying to add a new first-class inference provider, read [Adding Providers](./adding-providers.md) and the [Model Provider Plugin guide](./model-provider-plugin.md) alongside this page.
|
||||
|
||||
## Resolution precedence
|
||||
|
||||
At a high level, provider resolution uses:
|
||||
|
||||
1. explicit CLI/runtime request
|
||||
2. `config.yaml` model/provider config
|
||||
3. environment variables
|
||||
4. provider-specific defaults or auto resolution
|
||||
|
||||
That ordering matters because Hermes treats the saved model/provider choice as the source of truth for normal runs. This prevents a stale shell export from silently overriding the endpoint a user last selected in `hermes model`.
|
||||
|
||||
## Providers
|
||||
|
||||
Current provider families include (see `plugins/model-providers/` for the complete bundled set):
|
||||
|
||||
- OpenRouter
|
||||
- Nous Portal
|
||||
- OpenAI Codex
|
||||
- Copilot / Copilot ACP
|
||||
- Anthropic (native)
|
||||
- Google / Gemini (`gemini`)
|
||||
- Alibaba / DashScope (`alibaba`, `alibaba-coding-plan`)
|
||||
- DeepSeek
|
||||
- Z.AI
|
||||
- Kimi / Moonshot (`kimi-coding`, `kimi-coding-cn`)
|
||||
- MiniMax (`minimax`, `minimax-cn`, `minimax-oauth`)
|
||||
- Kilo Code
|
||||
- Hugging Face
|
||||
- OpenCode Zen / OpenCode Go
|
||||
- AWS Bedrock
|
||||
- Azure Foundry
|
||||
- NVIDIA NIM
|
||||
- xAI (Grok)
|
||||
- Arcee
|
||||
- GMI Cloud
|
||||
- StepFun
|
||||
- Qwen OAuth
|
||||
- Xiaomi
|
||||
- Ollama Cloud
|
||||
- LM Studio
|
||||
- Tencent TokenHub
|
||||
- Custom (`provider: custom`) — first-class provider for any OpenAI-compatible endpoint
|
||||
- Named custom providers (`custom_providers` list in config.yaml)
|
||||
|
||||
## Output of runtime resolution
|
||||
|
||||
The runtime resolver returns data such as:
|
||||
|
||||
- `provider`
|
||||
- `api_mode`
|
||||
- `base_url`
|
||||
- `api_key`
|
||||
- `source`
|
||||
- provider-specific metadata like expiry/refresh info
|
||||
|
||||
## Why this matters
|
||||
|
||||
This resolver is the main reason Hermes can share auth/runtime logic between:
|
||||
|
||||
- `hermes chat`
|
||||
- gateway message handling
|
||||
- cron jobs running in fresh sessions
|
||||
- ACP editor sessions
|
||||
- auxiliary model tasks
|
||||
|
||||
## OpenRouter and custom OpenAI-compatible base URLs
|
||||
|
||||
Hermes contains logic to avoid leaking the wrong API key to a custom endpoint when multiple provider keys exist (e.g. `OPENROUTER_API_KEY` and `OPENAI_API_KEY`).
|
||||
|
||||
Each provider's API key is scoped to its own base URL:
|
||||
|
||||
- `OPENROUTER_API_KEY` is only sent to `openrouter.ai` endpoints
|
||||
- `OPENAI_API_KEY` is used for custom endpoints and as a fallback
|
||||
|
||||
Hermes also distinguishes between:
|
||||
|
||||
- a real custom endpoint selected by the user
|
||||
- the OpenRouter fallback path used when no custom endpoint is configured
|
||||
|
||||
That distinction is especially important for:
|
||||
|
||||
- local model servers
|
||||
- non-OpenRouter OpenAI-compatible APIs
|
||||
- switching providers without re-running setup
|
||||
- config-saved custom endpoints that should keep working even when `OPENAI_BASE_URL` is not exported in the current shell
|
||||
|
||||
## Native Anthropic path
|
||||
|
||||
Anthropic is not just "via OpenRouter" anymore.
|
||||
|
||||
When provider resolution selects `anthropic`, Hermes uses:
|
||||
|
||||
- `api_mode = anthropic_messages`
|
||||
- the native Anthropic Messages API
|
||||
- `agent/anthropic_adapter.py` for translation
|
||||
|
||||
Credential resolution for native Anthropic now prefers refreshable Claude Code credentials over copied env tokens when both are present. In practice that means:
|
||||
|
||||
- Claude Code credential files are treated as the preferred source when they include refreshable auth
|
||||
- manual `ANTHROPIC_TOKEN` / `CLAUDE_CODE_OAUTH_TOKEN` values still work as explicit overrides
|
||||
- Hermes preflights Anthropic credential refresh before native Messages API calls
|
||||
- Hermes still retries once on a 401 after rebuilding the Anthropic client, as a fallback path
|
||||
|
||||
## OpenAI Codex path
|
||||
|
||||
Codex uses a separate Responses API path:
|
||||
|
||||
- `api_mode = codex_responses`
|
||||
- dedicated credential resolution and auth store support
|
||||
|
||||
## Auxiliary model routing
|
||||
|
||||
Auxiliary tasks such as:
|
||||
|
||||
- vision
|
||||
- web extraction summarization
|
||||
- context compression summaries
|
||||
- skills hub operations
|
||||
- MCP helper operations
|
||||
- memory flushes
|
||||
|
||||
can use their own provider/model routing rather than the main conversational model.
|
||||
|
||||
When an auxiliary task is configured with provider `main`, Hermes resolves that through the same shared runtime path as normal chat. In practice that means:
|
||||
|
||||
- env-driven custom endpoints still work
|
||||
- custom endpoints saved via `hermes model` / `config.yaml` also work
|
||||
- auxiliary routing can tell the difference between a real saved custom endpoint and the OpenRouter fallback
|
||||
|
||||
## Fallback models
|
||||
|
||||
Hermes supports a configured fallback provider chain — a list of `(provider, model)` entries tried in order when the primary model encounters errors. The legacy single-pair `fallback_model` dict is still accepted for back-compat (and migrated on first write).
|
||||
|
||||
### How it works internally
|
||||
|
||||
1. **Storage**: `AIAgent.__init__` stores the `fallback_model` dict and sets `_fallback_activated = False`.
|
||||
|
||||
2. **Trigger points**: `_try_activate_fallback()` is called from three places in the main retry loop in `run_agent.py`:
|
||||
- After max retries on invalid API responses (None choices, missing content)
|
||||
- On non-retryable client errors (HTTP 401, 403, 404)
|
||||
- After max retries on transient errors (HTTP 429, 500, 502, 503)
|
||||
|
||||
3. **Activation flow** (`_try_activate_fallback`):
|
||||
- Returns `False` immediately if already activated or not configured
|
||||
- Calls `resolve_provider_client()` from `auxiliary_client.py` to build a new client with proper auth
|
||||
- Determines `api_mode`: `codex_responses` for openai-codex, `anthropic_messages` for anthropic, `chat_completions` for everything else
|
||||
- Swaps in-place: `self.model`, `self.provider`, `self.base_url`, `self.api_mode`, `self.client`, `self._client_kwargs`
|
||||
- For anthropic fallback: builds a native Anthropic client instead of OpenAI-compatible
|
||||
- Re-evaluates prompt caching (enabled for Claude models on OpenRouter)
|
||||
- Sets `_fallback_activated = True` — prevents firing again
|
||||
- Resets retry count to 0 and continues the loop
|
||||
|
||||
4. **Config flow**:
|
||||
- CLI: `cli.py` reads `CLI_CONFIG["fallback_model"]` → passes to `AIAgent(fallback_model=...)`
|
||||
- Gateway: `gateway/run.py._load_fallback_model()` reads `config.yaml` → passes to `AIAgent`
|
||||
- Validation: both `provider` and `model` keys must be non-empty, or fallback is disabled
|
||||
|
||||
### What does NOT support fallback
|
||||
|
||||
- **Subagent delegation** (`tools/delegate_tool.py`): subagents inherit the parent's provider but not the fallback config
|
||||
- **Auxiliary tasks**: use their own independent provider auto-detection chain (see Auxiliary model routing above)
|
||||
|
||||
Cron jobs **do** support fallback: `run_job()` reads `fallback_providers` (or legacy `fallback_model`) from `config.yaml` and passes it to `AIAgent(fallback_model=...)`, matching the gateway's `_load_fallback_model()` pattern. See [Cron Internals](./cron-internals.md).
|
||||
|
||||
### Test coverage
|
||||
|
||||
Fallback behavior is exercised across several suites:
|
||||
|
||||
- `tests/run_agent/test_fallback_credential_isolation.py` — credential isolation between primary and fallback
|
||||
- `tests/hermes_cli/test_fallback_cmd.py` — the `/fallback` CLI command
|
||||
- `tests/gateway/test_fallback_eviction.py` — gateway eviction of failed providers
|
||||
|
||||
## Related docs
|
||||
|
||||
- [Agent Loop Internals](./agent-loop.md)
|
||||
- [ACP Internals](./acp-internals.md)
|
||||
- [Context Compression & Prompt Caching](./context-compression-and-caching.md)
|
||||
@@ -0,0 +1,171 @@
|
||||
---
|
||||
sidebar_position: 9
|
||||
title: "Secret Source Plugins"
|
||||
description: "How to build a secret-manager backend plugin for Hermes Agent"
|
||||
---
|
||||
|
||||
# Building a Secret Source Plugin
|
||||
|
||||
Secret sources resolve provider credentials from an external secret manager (a vault, a password manager, an OS keystore, a custom script) into environment variables at process startup — after `~/.hermes/.env` loads, before Hermes reads credentials. Bitwarden and 1Password ship in-tree; **every other backend is a plugin**. This guide covers building one.
|
||||
|
||||
:::tip
|
||||
The bundled set is deliberately closed, same policy as [memory providers](/developer-guide/memory-provider-plugin): PRs adding new vault backends under `agent/secret_sources/` are closed with a pointer to this guide. Publish your backend as a standalone plugin repo and share it in the Nous Research Discord (`#plugins-skills-and-skins`).
|
||||
:::
|
||||
|
||||
## What the framework owns vs. what you own
|
||||
|
||||
The orchestrator (`agent.secret_sources.registry.apply_all`) owns everything security- and precedence-sensitive, so a backend cannot get it wrong:
|
||||
|
||||
| Framework owns | You own |
|
||||
|---|---|
|
||||
| Source ordering, mapped-vs-bulk precedence | Fetching values from your backend |
|
||||
| First-claim-wins conflict handling + warnings | Validating your reference format |
|
||||
| `override_existing` semantics (never crosses sources) | Talking to your CLI/SDK/API |
|
||||
| Protected bootstrap tokens | Declaring which env var IS your bootstrap token |
|
||||
| Per-source wall-clock timeout | Keeping `fetch()` reasonably fast |
|
||||
| Per-var provenance + `(from X)` labels | A human-readable `label` |
|
||||
| `os.environ` writes | Nothing — you never touch the environment |
|
||||
|
||||
## Directory structure
|
||||
|
||||
```
|
||||
~/.hermes/plugins/my-vault/
|
||||
├── plugin.yaml # name, description
|
||||
└── __init__.py # SecretSource subclass + register(ctx)
|
||||
```
|
||||
|
||||
## The SecretSource ABC
|
||||
|
||||
Implement `agent.secret_sources.base.SecretSource`. One method is required:
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
|
||||
from agent.secret_sources.base import (
|
||||
ErrorKind,
|
||||
FetchResult,
|
||||
SecretSource,
|
||||
run_secret_cli,
|
||||
)
|
||||
|
||||
|
||||
class MyVaultSource(SecretSource):
|
||||
name = "myvault" # config section key: secrets.myvault
|
||||
label = "My Vault" # used in startup lines + provenance labels
|
||||
shape = "mapped" # "mapped" (explicit VAR→ref map) or "bulk" (project dump)
|
||||
scheme = "mv" # optional: unique URI scheme you own (mv://...)
|
||||
|
||||
def fetch(self, cfg: dict, home_path: Path) -> FetchResult:
|
||||
"""Resolve secrets. MUST NOT raise. MUST NOT prompt."""
|
||||
result = FetchResult()
|
||||
token = os.environ.get("MYVAULT_TOKEN", "").strip()
|
||||
if not token:
|
||||
result.error = "secrets.myvault.enabled is true but MYVAULT_TOKEN is not set."
|
||||
result.error_kind = ErrorKind.NOT_CONFIGURED
|
||||
return result
|
||||
|
||||
try:
|
||||
proc = run_secret_cli(
|
||||
["myvault-cli", "export", "--json"],
|
||||
allow_env=["MYVAULT_TOKEN"], # ONLY your auth vars — never full os.environ
|
||||
timeout=30,
|
||||
)
|
||||
except RuntimeError as exc: # spawn failure / timeout
|
||||
result.error = str(exc)
|
||||
result.error_kind = ErrorKind.BINARY_MISSING
|
||||
return result
|
||||
|
||||
if proc.returncode != 0:
|
||||
result.error = f"myvault-cli exited {proc.returncode}: {proc.stderr[:200]}"
|
||||
result.error_kind = ErrorKind.AUTH_FAILED
|
||||
return result
|
||||
|
||||
result.secrets = parse_your_output(proc.stdout) # {ENV_VAR: value}
|
||||
return result
|
||||
|
||||
def protected_env_vars(self, cfg: dict):
|
||||
# Your bootstrap token — no source (including yours) may ever overwrite it.
|
||||
return frozenset({"MYVAULT_TOKEN"})
|
||||
```
|
||||
|
||||
### Contract rules (enforced, not suggestions)
|
||||
|
||||
- **`fetch()` never raises.** Errors go in `result.error` + `result.error_kind`. A raising fetch is contained by the orchestrator and reported as `INTERNAL` — a contract violation, not a feature.
|
||||
- **`fetch()` never prompts.** Startup runs in non-TTY contexts (gateway, cron, Docker). `run_secret_cli()` closes stdin so a prompting helper fails fast. Interactive auth belongs in your CLI setup flow, never on the startup path.
|
||||
- **Sync, within budget.** The orchestrator enforces a wall-clock timeout (default 120s, user-tunable via `secrets.<name>.timeout_seconds`). Exceeding it reports `TIMEOUT` and your result is discarded.
|
||||
- **You fetch; the orchestrator applies.** Return the mapping you *would* contribute. Never write `os.environ` yourself — you'd bypass precedence, conflict detection, and provenance.
|
||||
- **API versioning.** `SecretSource.api_version` defaults to the current `SECRET_SOURCE_API_VERSION`. The registry skips (with a warning) sources built against a different version instead of crashing startup.
|
||||
|
||||
### Choosing your `shape`
|
||||
|
||||
- `mapped` — the user explicitly binds env-var names to references in config (like 1Password's `env:` map). Strongest intent: mapped claims beat bulk claims on contested vars.
|
||||
- `bulk` — you inject a whole project/folder of secrets implicitly (like Bitwarden BSM). Yields to mapped sources.
|
||||
|
||||
### Optional hooks
|
||||
|
||||
| Method | Default | Override when |
|
||||
|---|---|---|
|
||||
| `is_enabled(cfg)` | `cfg.get("enabled")` | Custom activation logic |
|
||||
| `override_existing(cfg)` | `cfg.get("override_existing", False)` | You want a different default (both bundled sources default `True` for rotation) |
|
||||
| `protected_env_vars(cfg)` | empty | You have a bootstrap token (you almost certainly do) |
|
||||
| `fetch_timeout_seconds(cfg)` | 120s | Your backend needs a different budget |
|
||||
| `config_schema()` | `{}` | Declare config keys for setup surfaces |
|
||||
|
||||
## Subprocess safety: use `run_secret_cli()`
|
||||
|
||||
If your backend shells out to a CLI, use the shared helper instead of `subprocess.run` directly. It gives you the audited posture for free: argv-only (no `shell=True`), a **minimal allowlisted child environment** (by the time sources run, `os.environ` holds every credential Hermes knows — never hand that to a child process), `NO_COLOR` + ANSI-scrubbed stderr, stdin closed, timeout → clean `RuntimeError`. Pass user-supplied reference strings after a `--` terminator in your argv so they can never parse as flags.
|
||||
|
||||
## Registering
|
||||
|
||||
```python
|
||||
# __init__.py
|
||||
def register(ctx):
|
||||
ctx.register_secret_source(MyVaultSource())
|
||||
```
|
||||
|
||||
Registration is rejected (with a log warning, never a crash) for: non-`SecretSource` instances, invalid/duplicate names, a `scheme` another source owns, wrong `api_version`, or a `shape` outside `mapped`/`bulk`.
|
||||
|
||||
:::note Timing
|
||||
Plugin discovery runs later in startup than the first `load_hermes_dotenv()` call, so a plugin source is not consulted by the very first env load of the process that discovers it. It IS consulted by every subsequently spawned Hermes process (gateway children, cron sessions, subagents). Bundled sources cover first-process bootstrap.
|
||||
:::
|
||||
|
||||
## Users configure it like any other source
|
||||
|
||||
```yaml
|
||||
secrets:
|
||||
sources: [myvault, bitwarden] # optional ordering
|
||||
myvault:
|
||||
enabled: true
|
||||
# ... your config_schema keys
|
||||
```
|
||||
|
||||
Multi-source precedence, conflict warnings, and `(from My Vault)` provenance labels all work automatically — see the [user-facing secrets docs](/user-guide/secrets/) for the precedence ladder.
|
||||
|
||||
## Validate with the conformance kit
|
||||
|
||||
Subclass the kit from the Hermes repo (`tests/secret_sources/conformance.py`) in your plugin's tests:
|
||||
|
||||
```python
|
||||
import pytest
|
||||
from tests.secret_sources.conformance import SecretSourceConformance
|
||||
|
||||
class TestMyVaultConformance(SecretSourceConformance):
|
||||
@pytest.fixture
|
||||
def source(self):
|
||||
return MyVaultSource()
|
||||
```
|
||||
|
||||
It checks the rules that break other people when violated: never-raises on malformed config, machine-readable error kinds, disabled-by-default, positive timeouts, valid protected-var names, and a full `apply_all()` round trip. Green conformance is the review bar for calling a backend contract-compliant.
|
||||
|
||||
## ErrorKind reference
|
||||
|
||||
| Kind | Meaning |
|
||||
|---|---|
|
||||
| `NOT_CONFIGURED` | Enabled but missing token / project / map |
|
||||
| `BINARY_MISSING` | Helper CLI not found or not executable |
|
||||
| `AUTH_FAILED` / `AUTH_EXPIRED` | Bad / expired credentials |
|
||||
| `REF_INVALID` | A secret reference failed validation |
|
||||
| `NETWORK` | Transport-level failure |
|
||||
| `EMPTY_VALUE` | Backend returned nothing for a ref — never apply `""` over a good credential |
|
||||
| `TIMEOUT` | Fetch exceeded its budget |
|
||||
| `INTERNAL` | Anything else (bug, unexpected shape) |
|
||||
@@ -0,0 +1,395 @@
|
||||
# Session Storage
|
||||
|
||||
Hermes Agent uses a SQLite database (`~/.hermes/state.db`) to persist session
|
||||
metadata, full message history, and model configuration across CLI and gateway
|
||||
sessions. This replaces the earlier per-session JSONL file approach.
|
||||
|
||||
Source file: `hermes_state.py`
|
||||
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
~/.hermes/state.db (SQLite, WAL mode)
|
||||
├── sessions — Session metadata, token counts, billing
|
||||
├── messages — Full message history per session
|
||||
├── messages_fts — FTS5 virtual table (content + tool_name + tool_calls)
|
||||
├── messages_fts_trigram — FTS5 virtual table with trigram tokenizer (CJK / substring search)
|
||||
├── state_meta — Key/value metadata table
|
||||
└── schema_version — Single-row table tracking migration state
|
||||
```
|
||||
|
||||
Key design decisions:
|
||||
- **WAL mode** for concurrent readers + one writer (gateway multi-platform)
|
||||
- **FTS5 virtual table** for fast text search across all session messages
|
||||
- **Session lineage** via `parent_session_id` chains (compression-triggered splits)
|
||||
- **Source tagging** (`cli`, `telegram`, `discord`, etc.) for platform filtering
|
||||
- Batch runner and RL trajectories are NOT stored here (separate systems)
|
||||
|
||||
|
||||
## SQLite Schema
|
||||
|
||||
### Sessions Table
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS sessions (
|
||||
id TEXT PRIMARY KEY,
|
||||
source TEXT NOT NULL,
|
||||
user_id TEXT,
|
||||
model TEXT,
|
||||
model_config TEXT,
|
||||
system_prompt TEXT,
|
||||
parent_session_id TEXT,
|
||||
started_at REAL NOT NULL,
|
||||
ended_at REAL,
|
||||
end_reason TEXT,
|
||||
message_count INTEGER DEFAULT 0,
|
||||
tool_call_count INTEGER DEFAULT 0,
|
||||
input_tokens INTEGER DEFAULT 0,
|
||||
output_tokens INTEGER DEFAULT 0,
|
||||
cache_read_tokens INTEGER DEFAULT 0,
|
||||
cache_write_tokens INTEGER DEFAULT 0,
|
||||
reasoning_tokens INTEGER DEFAULT 0,
|
||||
billing_provider TEXT,
|
||||
billing_base_url TEXT,
|
||||
billing_mode TEXT,
|
||||
estimated_cost_usd REAL,
|
||||
actual_cost_usd REAL,
|
||||
cost_status TEXT,
|
||||
cost_source TEXT,
|
||||
pricing_version TEXT,
|
||||
title TEXT,
|
||||
api_call_count INTEGER DEFAULT 0,
|
||||
FOREIGN KEY (parent_session_id) REFERENCES sessions(id)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_sessions_source ON sessions(source);
|
||||
CREATE INDEX IF NOT EXISTS idx_sessions_parent ON sessions(parent_session_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_sessions_started ON sessions(started_at DESC);
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_sessions_title_unique
|
||||
ON sessions(title) WHERE title IS NOT NULL;
|
||||
```
|
||||
|
||||
### Messages Table
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS messages (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
session_id TEXT NOT NULL REFERENCES sessions(id),
|
||||
role TEXT NOT NULL,
|
||||
content TEXT,
|
||||
tool_call_id TEXT,
|
||||
tool_calls TEXT,
|
||||
tool_name TEXT,
|
||||
timestamp REAL NOT NULL,
|
||||
token_count INTEGER,
|
||||
finish_reason TEXT,
|
||||
reasoning TEXT,
|
||||
reasoning_content TEXT,
|
||||
reasoning_details TEXT,
|
||||
codex_reasoning_items TEXT,
|
||||
codex_message_items TEXT
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_messages_session ON messages(session_id, timestamp);
|
||||
```
|
||||
|
||||
Notes:
|
||||
- `tool_calls` is stored as a JSON string (serialized list of tool call objects)
|
||||
- `reasoning_details`, `codex_reasoning_items`, and `codex_message_items` are stored as JSON strings
|
||||
- `reasoning` stores the raw reasoning text for providers that expose it
|
||||
- Timestamps are Unix epoch floats (`time.time()`)
|
||||
|
||||
### FTS5 Full-Text Search
|
||||
|
||||
```sql
|
||||
CREATE VIRTUAL TABLE IF NOT EXISTS messages_fts USING fts5(
|
||||
content,
|
||||
content=messages,
|
||||
content_rowid=id
|
||||
);
|
||||
```
|
||||
|
||||
The FTS5 table is kept in sync via three triggers that fire on INSERT, UPDATE,
|
||||
and DELETE of the `messages` table:
|
||||
|
||||
```sql
|
||||
CREATE TRIGGER IF NOT EXISTS messages_fts_insert AFTER INSERT ON messages BEGIN
|
||||
INSERT INTO messages_fts(rowid, content) VALUES (new.id, new.content);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS messages_fts_delete AFTER DELETE ON messages BEGIN
|
||||
INSERT INTO messages_fts(messages_fts, rowid, content)
|
||||
VALUES('delete', old.id, old.content);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS messages_fts_update AFTER UPDATE ON messages BEGIN
|
||||
INSERT INTO messages_fts(messages_fts, rowid, content)
|
||||
VALUES('delete', old.id, old.content);
|
||||
INSERT INTO messages_fts(rowid, content) VALUES (new.id, new.content);
|
||||
END;
|
||||
```
|
||||
|
||||
|
||||
## Schema Version and Migrations
|
||||
|
||||
Current schema version: **11**
|
||||
|
||||
The `schema_version` table stores a single integer. Simple column additions are handled declaratively by `_reconcile_columns()` (which diffs live columns against `SCHEMA_SQL` and ADDs any missing ones). The version-gated chain is reserved for data migrations and index/FTS changes that can't be expressed declaratively:
|
||||
|
||||
| Version | Change |
|
||||
|---------|--------|
|
||||
| 1 | Initial schema (sessions, messages, FTS5) |
|
||||
| 2 | Add `finish_reason` column to messages |
|
||||
| 3 | Add `title` column to sessions |
|
||||
| 4 | Add unique index on `title` (NULLs allowed, non-NULL must be unique) |
|
||||
| 5 | Add billing columns: `cache_read_tokens`, `cache_write_tokens`, `reasoning_tokens`, `billing_provider`, `billing_base_url`, `billing_mode`, `estimated_cost_usd`, `actual_cost_usd`, `cost_status`, `cost_source`, `pricing_version` |
|
||||
| 6 | Add reasoning columns to messages: `reasoning`, `reasoning_details`, `codex_reasoning_items` |
|
||||
| 7 | Add `reasoning_content` column to messages |
|
||||
| 8 | Add `api_call_count` column to sessions |
|
||||
| 9 | Add `codex_message_items` column to messages for Codex Responses message id/phase replay |
|
||||
| 10 | Add `messages_fts_trigram` virtual table (trigram tokenizer for CJK / substring search) and backfill existing rows |
|
||||
| 11 | Re-index `messages_fts` and `messages_fts_trigram` to cover `tool_name` + `tool_calls` and switch from external-content to inline mode; drop old triggers and backfill every message row |
|
||||
|
||||
Declarative column adds use `ALTER TABLE ADD COLUMN` wrapped in try/except to handle the column-already-exists case (idempotent). The version number is bumped after each successful migration block.
|
||||
|
||||
|
||||
## Write Contention Handling
|
||||
|
||||
Multiple hermes processes (gateway + CLI sessions + worktree agents) share one
|
||||
`state.db`. The `SessionDB` class handles write contention with:
|
||||
|
||||
- **Short SQLite timeout** (1 second) instead of the default 30s
|
||||
- **Application-level retry** with random jitter (20-150ms, up to 15 retries)
|
||||
- **BEGIN IMMEDIATE** transactions to surface lock contention at transaction start
|
||||
- **Periodic WAL checkpoints** every 50 successful writes (PASSIVE mode)
|
||||
|
||||
This avoids the "convoy effect" where SQLite's deterministic internal backoff
|
||||
causes all competing writers to retry at the same intervals.
|
||||
|
||||
```
|
||||
_WRITE_MAX_RETRIES = 15
|
||||
_WRITE_RETRY_MIN_S = 0.020 # 20ms
|
||||
_WRITE_RETRY_MAX_S = 0.150 # 150ms
|
||||
_CHECKPOINT_EVERY_N_WRITES = 50
|
||||
```
|
||||
|
||||
|
||||
## Common Operations
|
||||
|
||||
### Initialize
|
||||
|
||||
```python
|
||||
from hermes_state import SessionDB
|
||||
|
||||
db = SessionDB() # Default: ~/.hermes/state.db
|
||||
db = SessionDB(db_path=Path("/tmp/test.db")) # Custom path
|
||||
```
|
||||
|
||||
### Create and Manage Sessions
|
||||
|
||||
```python
|
||||
# Create a new session
|
||||
db.create_session(
|
||||
session_id="sess_abc123",
|
||||
source="cli",
|
||||
model="anthropic/claude-sonnet-4.6",
|
||||
user_id="user_1",
|
||||
parent_session_id=None, # or previous session ID for lineage
|
||||
)
|
||||
|
||||
# End a session
|
||||
db.end_session("sess_abc123", end_reason="user_exit")
|
||||
|
||||
# Reopen a session (clear ended_at/end_reason)
|
||||
db.reopen_session("sess_abc123")
|
||||
```
|
||||
|
||||
### Store Messages
|
||||
|
||||
```python
|
||||
msg_id = db.append_message(
|
||||
session_id="sess_abc123",
|
||||
role="assistant",
|
||||
content="Here's the answer...",
|
||||
tool_calls=[{"id": "call_1", "function": {"name": "terminal", "arguments": "{}"}}],
|
||||
token_count=150,
|
||||
finish_reason="stop",
|
||||
reasoning="Let me think about this...",
|
||||
)
|
||||
```
|
||||
|
||||
### Retrieve Messages
|
||||
|
||||
```python
|
||||
# Raw messages with all metadata
|
||||
messages = db.get_messages("sess_abc123")
|
||||
|
||||
# OpenAI conversation format (for API replay)
|
||||
conversation = db.get_messages_as_conversation("sess_abc123")
|
||||
# Returns: [{"role": "user", "content": "..."}, {"role": "assistant", ...}]
|
||||
```
|
||||
|
||||
### Session Titles
|
||||
|
||||
```python
|
||||
# Set a title (must be unique among non-NULL titles)
|
||||
db.set_session_title("sess_abc123", "Fix Docker Build")
|
||||
|
||||
# Resolve by title (returns most recent in lineage)
|
||||
session_id = db.resolve_session_by_title("Fix Docker Build")
|
||||
|
||||
# Auto-generate next title in lineage
|
||||
next_title = db.get_next_title_in_lineage("Fix Docker Build")
|
||||
# Returns: "Fix Docker Build #2"
|
||||
```
|
||||
|
||||
|
||||
## Full-Text Search
|
||||
|
||||
The `search_messages()` method supports FTS5 query syntax with automatic
|
||||
sanitization of user input.
|
||||
|
||||
### Basic Search
|
||||
|
||||
```python
|
||||
results = db.search_messages("docker deployment")
|
||||
```
|
||||
|
||||
### FTS5 Query Syntax
|
||||
|
||||
| Syntax | Example | Meaning |
|
||||
|--------|---------|---------|
|
||||
| Keywords | `docker deployment` | Both terms (implicit AND) |
|
||||
| Quoted phrase | `"exact phrase"` | Exact phrase match |
|
||||
| Boolean OR | `docker OR kubernetes` | Either term |
|
||||
| Boolean NOT | `python NOT java` | Exclude term |
|
||||
| Prefix | `deploy*` | Prefix match |
|
||||
|
||||
### Filtered Search
|
||||
|
||||
```python
|
||||
# Search only CLI sessions
|
||||
results = db.search_messages("error", source_filter=["cli"])
|
||||
|
||||
# Exclude gateway sessions
|
||||
results = db.search_messages("bug", exclude_sources=["telegram", "discord"])
|
||||
|
||||
# Search only user messages
|
||||
results = db.search_messages("help", role_filter=["user"])
|
||||
```
|
||||
|
||||
### Search Results Format
|
||||
|
||||
Each result includes:
|
||||
- `id`, `session_id`, `role`, `timestamp`
|
||||
- `snippet` — FTS5-generated snippet with `>>>match<<<` markers
|
||||
- `context` — 1 message before and after the match (content truncated to 200 chars)
|
||||
- `source`, `model`, `session_started` — from the parent session
|
||||
|
||||
The `_sanitize_fts5_query()` method handles edge cases:
|
||||
- Strips unmatched quotes and special characters
|
||||
- Wraps hyphenated terms in quotes (`chat-send` → `"chat-send"`)
|
||||
- Removes dangling boolean operators (`hello AND` → `hello`)
|
||||
|
||||
|
||||
## Session Lineage
|
||||
|
||||
Sessions can form chains via `parent_session_id`. This happens when context
|
||||
compression triggers a session split in the gateway.
|
||||
|
||||
### Query: Find Session Lineage
|
||||
|
||||
```sql
|
||||
-- Find all ancestors of a session
|
||||
WITH RECURSIVE lineage AS (
|
||||
SELECT * FROM sessions WHERE id = ?
|
||||
UNION ALL
|
||||
SELECT s.* FROM sessions s
|
||||
JOIN lineage l ON s.id = l.parent_session_id
|
||||
)
|
||||
SELECT id, title, started_at, parent_session_id FROM lineage;
|
||||
|
||||
-- Find all descendants of a session
|
||||
WITH RECURSIVE descendants AS (
|
||||
SELECT * FROM sessions WHERE id = ?
|
||||
UNION ALL
|
||||
SELECT s.* FROM sessions s
|
||||
JOIN descendants d ON s.parent_session_id = d.id
|
||||
)
|
||||
SELECT id, title, started_at FROM descendants;
|
||||
```
|
||||
|
||||
### Query: Recent Sessions with Preview
|
||||
|
||||
```sql
|
||||
SELECT s.*,
|
||||
COALESCE(
|
||||
(SELECT SUBSTR(m.content, 1, 63)
|
||||
FROM messages m
|
||||
WHERE m.session_id = s.id AND m.role = 'user' AND m.content IS NOT NULL
|
||||
ORDER BY m.timestamp, m.id LIMIT 1),
|
||||
''
|
||||
) AS preview,
|
||||
COALESCE(
|
||||
(SELECT MAX(m2.timestamp) FROM messages m2 WHERE m2.session_id = s.id),
|
||||
s.started_at
|
||||
) AS last_active
|
||||
FROM sessions s
|
||||
ORDER BY s.started_at DESC
|
||||
LIMIT 20;
|
||||
```
|
||||
|
||||
### Query: Token Usage Statistics
|
||||
|
||||
```sql
|
||||
-- Total tokens by model
|
||||
SELECT model,
|
||||
COUNT(*) as session_count,
|
||||
SUM(input_tokens) as total_input,
|
||||
SUM(output_tokens) as total_output,
|
||||
SUM(estimated_cost_usd) as total_cost
|
||||
FROM sessions
|
||||
WHERE model IS NOT NULL
|
||||
GROUP BY model
|
||||
ORDER BY total_cost DESC;
|
||||
|
||||
-- Sessions with highest token usage
|
||||
SELECT id, title, model, input_tokens + output_tokens AS total_tokens,
|
||||
estimated_cost_usd
|
||||
FROM sessions
|
||||
ORDER BY total_tokens DESC
|
||||
LIMIT 10;
|
||||
```
|
||||
|
||||
|
||||
## Export and Cleanup
|
||||
|
||||
```python
|
||||
# Export a single session with messages
|
||||
data = db.export_session("sess_abc123")
|
||||
|
||||
# Export all sessions (with messages) as list of dicts
|
||||
all_data = db.export_all(source="cli")
|
||||
|
||||
# Delete old sessions (only ended sessions)
|
||||
deleted_count = db.prune_sessions(older_than_days=90)
|
||||
deleted_count = db.prune_sessions(older_than_days=30, source="telegram")
|
||||
|
||||
# Clear messages but keep the session record
|
||||
db.clear_messages("sess_abc123")
|
||||
|
||||
# Delete session and all messages
|
||||
db.delete_session("sess_abc123")
|
||||
```
|
||||
|
||||
|
||||
## Database Location
|
||||
|
||||
Default path: `~/.hermes/state.db`
|
||||
|
||||
This is derived from `hermes_constants.get_hermes_home()` which resolves to
|
||||
`~/.hermes/` by default, or the value of `HERMES_HOME` environment variable.
|
||||
|
||||
The database file, WAL file (`state.db-wal`), and shared-memory file
|
||||
(`state.db-shm`) are all created in the same directory.
|
||||
@@ -0,0 +1,233 @@
|
||||
---
|
||||
sidebar_position: 9
|
||||
title: "Tools Runtime"
|
||||
description: "Runtime behavior of the tool registry, toolsets, dispatch, and terminal environments"
|
||||
---
|
||||
|
||||
# Tools Runtime
|
||||
|
||||
Hermes tools are self-registering functions grouped into toolsets and executed through a central registry/dispatch system.
|
||||
|
||||
Primary files:
|
||||
|
||||
- `tools/registry.py`
|
||||
- `model_tools.py`
|
||||
- `toolsets.py`
|
||||
- `tools/terminal_tool.py`
|
||||
- `tools/environments/*`
|
||||
|
||||
## Tool registration model
|
||||
|
||||
Each tool module calls `registry.register(...)` at import time.
|
||||
|
||||
`model_tools.py` is responsible for importing/discovering tool modules and building the schema list used by the model.
|
||||
|
||||
### How `registry.register()` works
|
||||
|
||||
Every tool file in `tools/` calls `registry.register()` at module level to declare itself. The function signature is:
|
||||
|
||||
```python
|
||||
registry.register(
|
||||
name="terminal", # Unique tool name (used in API schemas)
|
||||
toolset="terminal", # Toolset this tool belongs to
|
||||
schema={...}, # OpenAI function-calling schema (description, parameters)
|
||||
handler=handle_terminal, # The function that executes when the tool is called
|
||||
check_fn=check_terminal, # Optional: returns True/False for availability
|
||||
requires_env=["SOME_VAR"], # Optional: env vars needed (for UI display)
|
||||
is_async=False, # Whether the handler is an async coroutine
|
||||
description="Run commands", # Human-readable description
|
||||
emoji="💻", # Emoji for spinner/progress display
|
||||
)
|
||||
```
|
||||
|
||||
Each call creates a `ToolEntry` stored in the singleton `ToolRegistry._tools` dict keyed by tool name. If a name collision occurs across toolsets, a warning is logged and the later registration wins.
|
||||
|
||||
### Discovery: `discover_builtin_tools()`
|
||||
|
||||
When `model_tools.py` is imported, it calls `discover_builtin_tools()` from `tools/registry.py`. This function scans every `tools/*.py` file using AST parsing to find modules that contain top-level `registry.register()` calls, then imports them:
|
||||
|
||||
```python
|
||||
# tools/registry.py (simplified)
|
||||
def discover_builtin_tools(tools_dir=None):
|
||||
tools_path = Path(tools_dir) if tools_dir else Path(__file__).parent
|
||||
for path in sorted(tools_path.glob("*.py")):
|
||||
if path.name in {"__init__.py", "registry.py", "mcp_tool.py"}:
|
||||
continue
|
||||
if _module_registers_tools(path): # AST check for top-level registry.register()
|
||||
importlib.import_module(f"tools.{path.stem}")
|
||||
```
|
||||
|
||||
This auto-discovery means new tool files are picked up automatically — no manual list to maintain. The AST check only matches top-level `registry.register()` calls (not calls inside functions), so helper modules in `tools/` are not imported.
|
||||
|
||||
Each import triggers the module's `registry.register()` calls. Errors in optional tools (e.g., missing `fal_client` for image generation) are caught and logged — they don't prevent other tools from loading.
|
||||
|
||||
After core tool discovery, MCP tools and plugin tools are also discovered:
|
||||
|
||||
1. **MCP tools** — `tools.mcp_tool.discover_mcp_tools()` reads MCP server config and registers tools from external servers.
|
||||
2. **Plugin tools** — `hermes_cli.plugins.discover_plugins()` loads user/project/pip plugins that may register additional tools.
|
||||
|
||||
## Tool availability checking (`check_fn`)
|
||||
|
||||
Each tool can optionally provide a `check_fn` — a callable that returns `True` when the tool is available and `False` otherwise. Typical checks include:
|
||||
|
||||
- **API key present** — e.g., `lambda: bool(os.environ.get("SERP_API_KEY"))` for web search
|
||||
- **Service running** — e.g., checking if the Honcho server is configured
|
||||
- **Binary installed** — e.g., verifying `playwright` is available for browser tools
|
||||
|
||||
When `registry.get_definitions()` builds the schema list for the model, it runs each tool's `check_fn()`:
|
||||
|
||||
```python
|
||||
# Simplified from registry.py
|
||||
if entry.check_fn:
|
||||
try:
|
||||
available = bool(entry.check_fn())
|
||||
except Exception:
|
||||
available = False # Exceptions = unavailable
|
||||
if not available:
|
||||
continue # Skip this tool entirely
|
||||
```
|
||||
|
||||
Key behaviors:
|
||||
- Check results are **cached per-call** — if multiple tools share the same `check_fn`, it only runs once.
|
||||
- Exceptions in `check_fn()` are treated as "unavailable" (fail-safe).
|
||||
- The `is_toolset_available()` method checks whether a toolset's `check_fn` passes, used for UI display and toolset resolution.
|
||||
|
||||
## Toolset resolution
|
||||
|
||||
Toolsets are named bundles of tools. Hermes resolves them through:
|
||||
|
||||
- explicit enabled/disabled toolset lists
|
||||
- platform presets (`hermes-cli`, `hermes-telegram`, etc.)
|
||||
- dynamic MCP toolsets
|
||||
- curated special-purpose sets like `hermes-acp`
|
||||
|
||||
### How `get_tool_definitions()` filters tools
|
||||
|
||||
The main entry point is `model_tools.get_tool_definitions(enabled_toolsets, disabled_toolsets, quiet_mode)`:
|
||||
|
||||
1. **If `enabled_toolsets` is provided** — only tools from those toolsets are included. Each toolset name is resolved via `resolve_toolset()` which expands composite toolsets into individual tool names.
|
||||
|
||||
2. **If `disabled_toolsets` is provided** — start with ALL toolsets, then subtract the disabled ones.
|
||||
|
||||
3. **If neither** — include all known toolsets.
|
||||
|
||||
4. **Registry filtering** — the resolved tool name set is passed to `registry.get_definitions()`, which applies `check_fn` filtering and returns OpenAI-format schemas.
|
||||
|
||||
5. **Dynamic schema patching** — after filtering, `execute_code` and `browser_navigate` schemas are dynamically adjusted to only reference tools that actually passed filtering (prevents model hallucination of unavailable tools).
|
||||
|
||||
### Legacy toolset names
|
||||
|
||||
Old toolset names with `_tools` suffixes (e.g., `web_tools`, `terminal_tools`) are mapped to their modern tool names via `_LEGACY_TOOLSET_MAP` for backward compatibility.
|
||||
|
||||
## Dispatch
|
||||
|
||||
At runtime, tools are dispatched through the central registry, with agent-loop exceptions for some agent-level tools such as memory/todo/session-search handling.
|
||||
|
||||
### Dispatch flow: model tool_call → handler execution
|
||||
|
||||
When the model returns a `tool_call`, the flow is:
|
||||
|
||||
```
|
||||
Model response with tool_call
|
||||
↓
|
||||
run_agent.py agent loop
|
||||
↓
|
||||
model_tools.handle_function_call(name, args, task_id, user_task)
|
||||
↓
|
||||
[Agent-loop tools?] → handled directly by agent loop (todo, memory, session_search, delegate_task)
|
||||
↓
|
||||
[Plugin pre-hook] → invoke_hook("pre_tool_call", ...)
|
||||
↓
|
||||
registry.dispatch(name, args, **kwargs)
|
||||
↓
|
||||
Look up ToolEntry by name
|
||||
↓
|
||||
[Async handler?] → bridge via _run_async()
|
||||
[Sync handler?] → call directly
|
||||
↓
|
||||
Return result string (or JSON error)
|
||||
↓
|
||||
[Plugin post-hook] → invoke_hook("post_tool_call", ...)
|
||||
```
|
||||
|
||||
### Error wrapping
|
||||
|
||||
All tool execution is wrapped in error handling at two levels:
|
||||
|
||||
1. **`registry.dispatch()`** — catches any exception from the handler and returns `{"error": "Tool execution failed: ExceptionType: message"}` as JSON.
|
||||
|
||||
2. **`handle_function_call()`** — wraps the entire dispatch in a secondary try/except that returns `{"error": "Error executing tool_name: message"}`.
|
||||
|
||||
This ensures the model always receives a well-formed JSON string, never an unhandled exception.
|
||||
|
||||
### Agent-loop tools
|
||||
|
||||
Four tools are intercepted before registry dispatch because they need agent-level state (TodoStore, MemoryStore, etc.):
|
||||
|
||||
- `todo` — planning/task tracking
|
||||
- `memory` — persistent memory writes
|
||||
- `session_search` — cross-session recall
|
||||
- `delegate_task` — spawns subagent sessions
|
||||
|
||||
These tools' schemas are still registered in the registry (for `get_tool_definitions`), but their handlers return a stub error if dispatch somehow reaches them directly.
|
||||
|
||||
### Async bridging
|
||||
|
||||
When a tool handler is async, `_run_async()` bridges it to the sync dispatch path:
|
||||
|
||||
- **CLI path (no running loop)** — uses a persistent event loop to keep cached async clients alive
|
||||
- **Gateway path (running loop)** — spins up a disposable thread with `asyncio.run()`
|
||||
- **Worker threads (parallel tools)** — uses per-thread persistent loops stored in thread-local storage
|
||||
|
||||
## The DANGEROUS_PATTERNS approval flow
|
||||
|
||||
The terminal tool integrates a dangerous-command approval system defined in `tools/approval.py`:
|
||||
|
||||
1. **Pattern detection** — `DANGEROUS_PATTERNS` is a list of `(regex, description)` tuples covering destructive operations:
|
||||
- Recursive deletes (`rm -rf`)
|
||||
- Filesystem formatting (`mkfs`, `dd`)
|
||||
- SQL destructive operations (`DROP TABLE`, `DELETE FROM` without `WHERE`)
|
||||
- System config overwrites (`> /etc/`)
|
||||
- Service manipulation (`systemctl stop`)
|
||||
- Remote code execution (`curl | sh`)
|
||||
- Fork bombs, process kills, etc.
|
||||
|
||||
2. **Detection** — before executing any terminal command, `detect_dangerous_command(command)` checks against all patterns.
|
||||
|
||||
3. **Approval prompt** — if a match is found:
|
||||
- **CLI mode** — an interactive prompt asks the user to approve, deny, or allow permanently
|
||||
- **Gateway mode** — an async approval callback sends the request to the messaging platform
|
||||
- **Smart approval** — optionally, an auxiliary LLM can auto-approve low-risk commands that match patterns (e.g., `rm -rf node_modules/` is safe but matches "recursive delete")
|
||||
|
||||
4. **Session state** — approvals are tracked per-session. Once you approve "recursive delete" for a session, subsequent `rm -rf` commands don't re-prompt.
|
||||
|
||||
5. **Permanent allowlist** — the "allow permanently" option writes the pattern to `config.yaml`'s `command_allowlist`, persisting across sessions.
|
||||
|
||||
## Terminal/runtime environments
|
||||
|
||||
The terminal system supports multiple backends:
|
||||
|
||||
- local
|
||||
- docker
|
||||
- ssh
|
||||
- singularity
|
||||
- modal
|
||||
- daytona
|
||||
|
||||
It also supports:
|
||||
|
||||
- per-task cwd overrides
|
||||
- background process management
|
||||
- PTY mode
|
||||
- approval callbacks for dangerous commands
|
||||
|
||||
## Concurrency
|
||||
|
||||
Tool calls may execute sequentially or concurrently depending on the tool mix and interaction requirements.
|
||||
|
||||
## Related docs
|
||||
|
||||
- [Toolsets Reference](../reference/toolsets-reference.md)
|
||||
- [Built-in Tools Reference](../reference/tools-reference.md)
|
||||
- [Agent Loop Internals](./agent-loop.md)
|
||||
- [ACP Internals](./acp-internals.md)
|
||||
@@ -0,0 +1,233 @@
|
||||
# Trajectory Format
|
||||
|
||||
Hermes Agent saves conversation trajectories in ShareGPT-compatible JSONL format
|
||||
for use as training data, debugging artifacts, and reinforcement learning datasets.
|
||||
|
||||
Source files: `agent/trajectory.py`, `run_agent.py` (search for `_save_trajectory`), `batch_runner.py`
|
||||
|
||||
|
||||
## File Naming Convention
|
||||
|
||||
Trajectories are written to files in the current working directory:
|
||||
|
||||
| File | When |
|
||||
|------|------|
|
||||
| `trajectory_samples.jsonl` | Conversations that completed successfully (`completed=True`) |
|
||||
| `failed_trajectories.jsonl` | Conversations that failed or were interrupted (`completed=False`) |
|
||||
|
||||
The batch runner (`batch_runner.py`) writes to a custom output file per batch
|
||||
(e.g., `batch_001_output.jsonl`) with additional metadata fields.
|
||||
|
||||
You can override the filename via the `filename` parameter in `save_trajectory()`.
|
||||
|
||||
|
||||
## JSONL Entry Format
|
||||
|
||||
Each line in the file is a self-contained JSON object. There are two variants:
|
||||
|
||||
### CLI/Interactive Format (from `_save_trajectory`)
|
||||
|
||||
```json
|
||||
{
|
||||
"conversations": [ ... ],
|
||||
"timestamp": "2026-03-30T14:22:31.456789",
|
||||
"model": "anthropic/claude-sonnet-4.6",
|
||||
"completed": true
|
||||
}
|
||||
```
|
||||
|
||||
### Batch Runner Format (from `batch_runner.py`)
|
||||
|
||||
```json
|
||||
{
|
||||
"prompt_index": 42,
|
||||
"conversations": [ ... ],
|
||||
"metadata": { "prompt_source": "gsm8k", "difficulty": "hard" },
|
||||
"completed": true,
|
||||
"partial": false,
|
||||
"api_calls": 7,
|
||||
"toolsets_used": ["code_tools", "file_tools"],
|
||||
"tool_stats": {
|
||||
"terminal": {"count": 3, "success": 3, "failure": 0},
|
||||
"read_file": {"count": 2, "success": 2, "failure": 0},
|
||||
"write_file": {"count": 0, "success": 0, "failure": 0}
|
||||
},
|
||||
"tool_error_counts": {
|
||||
"terminal": 0,
|
||||
"read_file": 0,
|
||||
"write_file": 0
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The `tool_stats` and `tool_error_counts` dictionaries are normalized to include
|
||||
ALL possible tools (from `model_tools.TOOL_TO_TOOLSET_MAP`) with zero defaults,
|
||||
ensuring consistent schema across entries for HuggingFace dataset loading.
|
||||
|
||||
|
||||
## Conversations Array (ShareGPT Format)
|
||||
|
||||
The `conversations` array uses ShareGPT role conventions:
|
||||
|
||||
| API Role | ShareGPT `from` |
|
||||
|----------|-----------------|
|
||||
| system | `"system"` |
|
||||
| user | `"human"` |
|
||||
| assistant | `"gpt"` |
|
||||
| tool | `"tool"` |
|
||||
|
||||
### Complete Example
|
||||
|
||||
```json
|
||||
{
|
||||
"conversations": [
|
||||
{
|
||||
"from": "system",
|
||||
"value": "You are a function calling AI model. You are provided with function signatures within <tools> </tools> XML tags. You may call one or more functions to assist with the user query. If available tools are not relevant in assisting with user query, just respond in natural conversational language. Don't make assumptions about what values to plug into functions. After calling & executing the functions, you will be provided with function results within <tool_response> </tool_response> XML tags. Here are the available tools:\n<tools>\n[{\"name\": \"terminal\", \"description\": \"Execute shell commands\", \"parameters\": {\"type\": \"object\", \"properties\": {\"command\": {\"type\": \"string\"}}}, \"required\": null}]\n</tools>\nFor each function call return a JSON object, with the following pydantic model json schema for each:\n{'title': 'FunctionCall', 'type': 'object', 'properties': {'name': {'title': 'Name', 'type': 'string'}, 'arguments': {'title': 'Arguments', 'type': 'object'}}, 'required': ['name', 'arguments']}\nEach function call should be enclosed within <tool_call> </tool_call> XML tags.\nExample:\n<tool_call>\n{'name': <function-name>,'arguments': <args-dict>}\n</tool_call>"
|
||||
},
|
||||
{
|
||||
"from": "human",
|
||||
"value": "What Python version is installed?"
|
||||
},
|
||||
{
|
||||
"from": "gpt",
|
||||
"value": "<think>\nThe user wants to know the Python version. I should run python3 --version.\n</think>\n<tool_call>\n{\"name\": \"terminal\", \"arguments\": {\"command\": \"python3 --version\"}}\n</tool_call>"
|
||||
},
|
||||
{
|
||||
"from": "tool",
|
||||
"value": "<tool_response>\n{\"tool_call_id\": \"call_abc123\", \"name\": \"terminal\", \"content\": \"Python 3.11.6\"}\n</tool_response>"
|
||||
},
|
||||
{
|
||||
"from": "gpt",
|
||||
"value": "<think>\nGot the version. I can now answer the user.\n</think>\nPython 3.11.6 is installed on this system."
|
||||
}
|
||||
],
|
||||
"timestamp": "2026-03-30T14:22:31.456789",
|
||||
"model": "anthropic/claude-sonnet-4.6",
|
||||
"completed": true
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
## Normalization Rules
|
||||
|
||||
### Reasoning Content Markup
|
||||
|
||||
The trajectory converter normalizes ALL reasoning into `<think>` tags, regardless
|
||||
of how the model originally produced it:
|
||||
|
||||
1. **Native thinking tokens** (`msg["reasoning"]` field from providers like
|
||||
Anthropic, OpenAI o-series): Wrapped as `<think>\n{reasoning}\n</think>\n`
|
||||
and prepended before the content.
|
||||
|
||||
2. **REASONING_SCRATCHPAD XML** (when native thinking is disabled and the model
|
||||
reasons via system-prompt-instructed XML): `<REASONING_SCRATCHPAD>` tags are
|
||||
converted to `<think>` via `convert_scratchpad_to_think()`.
|
||||
|
||||
3. **Empty think blocks**: Every `gpt` turn is guaranteed to have a `<think>`
|
||||
block. If no reasoning was produced, an empty block is inserted:
|
||||
`<think>\n</think>\n` — this ensures consistent format for training data.
|
||||
|
||||
### Tool Call Normalization
|
||||
|
||||
Tool calls from the API format (with `tool_call_id`, function name, arguments as
|
||||
JSON string) are converted to XML-wrapped JSON:
|
||||
|
||||
```
|
||||
<tool_call>
|
||||
{"name": "terminal", "arguments": {"command": "ls -la"}}
|
||||
</tool_call>
|
||||
```
|
||||
|
||||
- Arguments are parsed from JSON strings back to objects (not double-encoded)
|
||||
- If JSON parsing fails (shouldn't happen — validated during conversation),
|
||||
an empty `{}` is used with a warning logged
|
||||
- Multiple tool calls in one assistant turn produce multiple `<tool_call>` blocks
|
||||
in a single `gpt` message
|
||||
|
||||
### Tool Response Normalization
|
||||
|
||||
All tool results following an assistant message are grouped into a single `tool`
|
||||
turn with XML-wrapped JSON responses:
|
||||
|
||||
```
|
||||
<tool_response>
|
||||
{"tool_call_id": "call_abc123", "name": "terminal", "content": "output here"}
|
||||
</tool_response>
|
||||
```
|
||||
|
||||
- If tool content looks like JSON (starts with `{` or `[`), it's parsed so the
|
||||
content field contains a JSON object/array rather than a string
|
||||
- Multiple tool results are joined with newlines in one message
|
||||
- The tool name is matched by position against the parent assistant's `tool_calls`
|
||||
array
|
||||
|
||||
### System Message
|
||||
|
||||
The system message is generated at save time (not taken from the conversation).
|
||||
It follows the Hermes function-calling prompt template with:
|
||||
|
||||
- Preamble explaining the function-calling protocol
|
||||
- `<tools>` XML block containing the JSON tool definitions
|
||||
- Schema reference for `FunctionCall` objects
|
||||
- `<tool_call>` example
|
||||
|
||||
Tool definitions include `name`, `description`, `parameters`, and `required`
|
||||
(set to `null` to match the canonical format).
|
||||
|
||||
|
||||
## Loading Trajectories
|
||||
|
||||
Trajectories are standard JSONL — load with any JSON-lines reader:
|
||||
|
||||
```python
|
||||
import json
|
||||
|
||||
def load_trajectories(path: str):
|
||||
"""Load trajectory entries from a JSONL file."""
|
||||
entries = []
|
||||
with open(path, "r", encoding="utf-8") as f:
|
||||
for line in f:
|
||||
line = line.strip()
|
||||
if line:
|
||||
entries.append(json.loads(line))
|
||||
return entries
|
||||
|
||||
# Filter to successful completions only
|
||||
successful = [e for e in load_trajectories("trajectory_samples.jsonl")
|
||||
if e.get("completed")]
|
||||
|
||||
# Extract just the conversations for training
|
||||
training_data = [e["conversations"] for e in successful]
|
||||
```
|
||||
|
||||
### Loading for HuggingFace Datasets
|
||||
|
||||
```python
|
||||
from datasets import load_dataset
|
||||
|
||||
ds = load_dataset("json", data_files="trajectory_samples.jsonl")
|
||||
```
|
||||
|
||||
The normalized `tool_stats` schema ensures all entries have the same columns,
|
||||
preventing Arrow schema mismatch errors during dataset loading.
|
||||
|
||||
|
||||
## Controlling Trajectory Saving
|
||||
|
||||
In the CLI, trajectory saving is controlled by:
|
||||
|
||||
```yaml
|
||||
# config.yaml
|
||||
agent:
|
||||
save_trajectories: true # default: false
|
||||
```
|
||||
|
||||
Or via the `--save-trajectories` flag. When the agent initializes with
|
||||
`save_trajectories=True`, the `_save_trajectory()` method is called at the end
|
||||
of each conversation turn.
|
||||
|
||||
The batch runner always saves trajectories (that's its primary purpose).
|
||||
|
||||
Samples with zero reasoning across all turns are automatically discarded by the
|
||||
batch runner to avoid polluting training data with non-reasoning examples.
|
||||
@@ -0,0 +1,231 @@
|
||||
---
|
||||
sidebar_position: 12
|
||||
title: "Video Generation Provider Plugins"
|
||||
description: "How to build a video-generation backend plugin for Hermes Agent"
|
||||
---
|
||||
|
||||
# Building a Video Generation Provider Plugin
|
||||
|
||||
Video-gen provider plugins register a backend that services every `video_generate` tool call. Built-in providers (xAI, FAL) ship as plugins. Add a new one, or override a bundled one, by dropping a directory into `plugins/video_gen/<name>/`.
|
||||
|
||||
:::tip
|
||||
Video-gen mirrors [Image Generation Provider Plugins](/developer-guide/image-gen-provider-plugin) almost line-for-line — if you've built an image-gen backend, you already know the shape. The main differences: a `capabilities()` method advertising modalities/aspect-ratios/durations, and a routing convention (pass `image_url` to use image-to-video, omit it to use text-to-video — the provider picks the right endpoint internally).
|
||||
:::
|
||||
|
||||
## The unified surface (one tool, two modalities)
|
||||
|
||||
The `video_generate` tool exposes two modalities through one parameter:
|
||||
|
||||
- **Text-to-video** — call with `prompt` only. The provider routes to its text-to-video endpoint.
|
||||
- **Image-to-video** — call with `prompt` + `image_url`. The provider routes to its image-to-video endpoint.
|
||||
|
||||
Edit and extend are intentionally out of scope. Most backends don't support them and the inconsistency would force per-backend prose into the agent's tool description.
|
||||
|
||||
## How discovery works
|
||||
|
||||
Hermes scans for video-gen backends in three places:
|
||||
|
||||
1. **Bundled** — `<repo>/plugins/video_gen/<name>/` (auto-loaded with `kind: backend`)
|
||||
2. **User** — `~/.hermes/plugins/video_gen/<name>/` (opt-in via `plugins.enabled`)
|
||||
3. **Pip** — packages declaring a `hermes_agent.plugins` entry point
|
||||
|
||||
Each plugin's `register(ctx)` function calls `ctx.register_video_gen_provider(...)`. The active provider is picked by `video_gen.provider` in `config.yaml`; `hermes tools` → Video Generation walks users through selection. Unlike `image_generate`, there is no in-tree legacy backend — every provider is a plugin.
|
||||
|
||||
## Directory structure
|
||||
|
||||
```
|
||||
plugins/video_gen/my-backend/
|
||||
├── __init__.py # VideoGenProvider subclass + register()
|
||||
└── plugin.yaml # Manifest with kind: backend
|
||||
```
|
||||
|
||||
## The VideoGenProvider ABC
|
||||
|
||||
Subclass `agent.video_gen_provider.VideoGenProvider`. Required: `name` property and `generate()` method.
|
||||
|
||||
```python
|
||||
# plugins/video_gen/my-backend/__init__.py
|
||||
from typing import Any, Dict, List, Optional
|
||||
import os
|
||||
|
||||
from agent.video_gen_provider import (
|
||||
VideoGenProvider,
|
||||
error_response,
|
||||
success_response,
|
||||
)
|
||||
|
||||
|
||||
class MyVideoGenProvider(VideoGenProvider):
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "my-backend"
|
||||
|
||||
@property
|
||||
def display_name(self) -> str:
|
||||
return "My Backend"
|
||||
|
||||
def is_available(self) -> bool:
|
||||
return bool(os.environ.get("MY_API_KEY"))
|
||||
|
||||
def list_models(self) -> List[Dict[str, Any]]:
|
||||
# Each entry is a model FAMILY — a name the user picks once.
|
||||
# Your provider's generate() routes within the family based on
|
||||
# whether image_url was passed.
|
||||
return [
|
||||
{
|
||||
"id": "fast",
|
||||
"display": "Fast",
|
||||
"speed": "~30s",
|
||||
"strengths": "Cheapest tier",
|
||||
"price": "$0.05/s",
|
||||
"modalities": ["text", "image"], # advisory
|
||||
},
|
||||
]
|
||||
|
||||
def default_model(self) -> Optional[str]:
|
||||
return "fast"
|
||||
|
||||
def capabilities(self) -> Dict[str, Any]:
|
||||
return {
|
||||
"modalities": ["text", "image"],
|
||||
"aspect_ratios": ["16:9", "9:16"],
|
||||
"resolutions": ["720p", "1080p"],
|
||||
"min_duration": 1,
|
||||
"max_duration": 10,
|
||||
"supports_audio": False,
|
||||
"supports_negative_prompt": True,
|
||||
"max_reference_images": 0,
|
||||
}
|
||||
|
||||
def get_setup_schema(self) -> Dict[str, Any]:
|
||||
return {
|
||||
"name": "My Backend",
|
||||
"badge": "paid",
|
||||
"tag": "Short description shown in `hermes tools`",
|
||||
"env_vars": [
|
||||
{
|
||||
"key": "MY_API_KEY",
|
||||
"prompt": "My Backend API key",
|
||||
"url": "https://mybackend.example.com/keys",
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
def generate(
|
||||
self,
|
||||
prompt: str,
|
||||
*,
|
||||
model: Optional[str] = None,
|
||||
image_url: Optional[str] = None,
|
||||
reference_image_urls: Optional[List[str]] = None,
|
||||
duration: Optional[int] = None,
|
||||
aspect_ratio: str = "16:9",
|
||||
resolution: str = "720p",
|
||||
negative_prompt: Optional[str] = None,
|
||||
audio: Optional[bool] = None,
|
||||
seed: Optional[int] = None,
|
||||
**kwargs: Any, # always ignore unknown kwargs for forward-compat
|
||||
) -> Dict[str, Any]:
|
||||
# ROUTE: image_url presence picks the endpoint.
|
||||
if image_url:
|
||||
endpoint = "my-backend/image-to-video"
|
||||
modality_used = "image"
|
||||
else:
|
||||
endpoint = "my-backend/text-to-video"
|
||||
modality_used = "text"
|
||||
|
||||
# ... call your API ...
|
||||
|
||||
return success_response(
|
||||
video="https://your-cdn/output.mp4",
|
||||
model=model or "fast",
|
||||
prompt=prompt,
|
||||
modality=modality_used,
|
||||
aspect_ratio=aspect_ratio,
|
||||
duration=duration or 5,
|
||||
provider=self.name,
|
||||
)
|
||||
|
||||
|
||||
def register(ctx) -> None:
|
||||
ctx.register_video_gen_provider(MyVideoGenProvider())
|
||||
```
|
||||
|
||||
## The plugin manifest
|
||||
|
||||
```yaml
|
||||
# plugins/video_gen/my-backend/plugin.yaml
|
||||
name: my-backend
|
||||
version: 1.0.0
|
||||
description: "My video generation backend"
|
||||
author: Your Name
|
||||
kind: backend
|
||||
requires_env:
|
||||
- MY_API_KEY
|
||||
```
|
||||
|
||||
## The `video_generate` schema
|
||||
|
||||
The tool exposes one schema across every backend. Providers ignore parameters they don't support.
|
||||
|
||||
| Parameter | What it does |
|
||||
|---|---|
|
||||
| `prompt` | Text instruction (required) |
|
||||
| `image_url` | When set → image-to-video; when omitted → text-to-video |
|
||||
| `reference_image_urls` | Style/character refs (provider-dependent) |
|
||||
| `duration` | Seconds — provider clamps |
|
||||
| `aspect_ratio` | `"16:9"`, `"9:16"`, `"1:1"`, ... — provider clamps |
|
||||
| `resolution` | `"480p"` / `"540p"` / `"720p"` / `"1080p"` — provider clamps |
|
||||
| `negative_prompt` | Content to avoid (Pixverse/Kling only) |
|
||||
| `audio` | Native audio (Veo3 / Pixverse pricing tier) |
|
||||
| `seed` | Reproducibility |
|
||||
| `model` | Override the active model/family |
|
||||
|
||||
The provider's `capabilities()` advertises which of these are honored. The agent sees the active backend's capabilities in the tool description, dynamically rebuilt when the user changes backend via `hermes tools`.
|
||||
|
||||
## Model families and endpoint routing (the FAL pattern)
|
||||
|
||||
When your backend has multiple endpoints per "model" — like FAL, where every family (Veo 3.1, Pixverse v6, Kling O3) has both a `/text-to-video` and an `/image-to-video` URL — represent each **family** as one catalog entry. Your `generate()` picks the right endpoint based on whether `image_url` was passed:
|
||||
|
||||
```python
|
||||
FAMILIES = {
|
||||
"veo3.1": {
|
||||
"text_endpoint": "fal-ai/veo3.1",
|
||||
"image_endpoint": "fal-ai/veo3.1/image-to-video",
|
||||
# ... family-specific capability flags ...
|
||||
},
|
||||
}
|
||||
|
||||
def generate(self, prompt, *, image_url=None, model=None, **kwargs):
|
||||
family_id, family = _resolve_family(model)
|
||||
endpoint = family["image_endpoint"] if image_url else family["text_endpoint"]
|
||||
# ... build payload from family's declared capability flags, call endpoint ...
|
||||
```
|
||||
|
||||
The user picks `veo3.1` once in `hermes tools`. The agent never thinks about endpoints — it just passes (or doesn't pass) `image_url`.
|
||||
|
||||
## Selection precedence
|
||||
|
||||
For per-instance model knobs (see `plugins/video_gen/fal/__init__.py`):
|
||||
|
||||
1. `model=` keyword from the tool call
|
||||
2. `<PROVIDER>_VIDEO_MODEL` env var
|
||||
3. `video_gen.<provider>.model` in `config.yaml`
|
||||
4. `video_gen.model` in `config.yaml` (when it's one of your IDs)
|
||||
5. Provider's `default_model()`
|
||||
|
||||
## Response shape
|
||||
|
||||
`success_response()` and `error_response()` produce the dict shape every backend returns. Use them — don't hand-roll the dict.
|
||||
|
||||
Success keys: `success`, `video` (URL or absolute path), `model`, `prompt`, `modality` (`"text"` or `"image"`), `aspect_ratio`, `duration`, `provider`, plus `extra`.
|
||||
|
||||
Error keys: `success`, `video` (None), `error`, `error_type`, `model`, `prompt`, `aspect_ratio`, `provider`.
|
||||
|
||||
## Where to save artifacts
|
||||
|
||||
If your backend returns base64, use `save_b64_video()` to write under `$HERMES_HOME/cache/videos/`. For raw bytes from a follow-up HTTP fetch, use `save_bytes_video()`. Otherwise return the upstream URL directly — the gateway resolves remote URLs on delivery.
|
||||
|
||||
## Testing
|
||||
|
||||
Drop a smoke test under `tests/plugins/video_gen/test_<name>_plugin.py`. The xAI and FAL tests show the pattern — register, verify catalog, exercise routing both with and without `image_url`, assert clean error responses on missing auth.
|
||||
@@ -0,0 +1,260 @@
|
||||
---
|
||||
sidebar_position: 12
|
||||
title: "Web Search Provider Plugins"
|
||||
description: "How to build a web-search/extract/crawl backend plugin for Hermes Agent"
|
||||
---
|
||||
|
||||
# Building a Web Search Provider Plugin
|
||||
|
||||
Web-search provider plugins register a backend that services `web_search`, `web_extract`, and (optionally) deep-crawl tool calls. Built-in providers — Firecrawl, SearXNG, Tavily, Exa, Parallel, Brave Search (free tier), xAI, and DDGS — all ship as plugins under `plugins/web/<name>/`. You can add a new one, or override a bundled one, by dropping a directory next to them.
|
||||
|
||||
:::tip
|
||||
Web search is one of several **backend plugins** Hermes supports. The others (with their own ABCs) are [Image Generation Provider Plugins](/developer-guide/image-gen-provider-plugin), [Video Generation Provider Plugins](/developer-guide/video-gen-provider-plugin), [Memory Provider Plugins](/developer-guide/memory-provider-plugin), [Context Engine Plugins](/developer-guide/context-engine-plugin), and [Model Provider Plugins](/developer-guide/model-provider-plugin). General tool/hook/CLI plugins live in [Build a Hermes Plugin](/developer-guide/plugins).
|
||||
:::
|
||||
|
||||
## How discovery works
|
||||
|
||||
Hermes scans for web-search backends in three places:
|
||||
|
||||
1. **Bundled** — `<repo>/plugins/web/<name>/` (auto-loaded with `kind: backend`, always available)
|
||||
2. **User** — `~/.hermes/plugins/web/<name>/` (opt-in via `plugins.enabled` or `hermes plugins enable <name>`)
|
||||
3. **Pip** — packages declaring a `hermes_agent.plugins` entry point
|
||||
|
||||
Each plugin's `register(ctx)` function calls `ctx.register_web_search_provider(...)` — that puts the instance into the registry in `agent/web_search_registry.py`. The active provider for each capability is picked by config:
|
||||
|
||||
| Capability | Config key | Falls back to |
|
||||
|---|---|---|
|
||||
| `web_search` | `web.search_backend` | `web.backend` |
|
||||
| `web_extract` | `web.extract_backend` | `web.backend` |
|
||||
| Deep crawl modes inside `web_extract` | `web.extract_backend` | `web.backend` |
|
||||
|
||||
When neither key is set, Hermes auto-detects the backend from whichever API key/URL is present in the environment. `hermes tools` walks users through selection.
|
||||
|
||||
## Directory structure
|
||||
|
||||
```
|
||||
plugins/web/my-backend/
|
||||
├── __init__.py # register() entry point
|
||||
├── provider.py # WebSearchProvider subclass
|
||||
└── plugin.yaml # Manifest with kind: backend and provides_web_providers
|
||||
```
|
||||
|
||||
`brave_free/` and `ddgs/` are the smallest in-tree references — `brave_free` for an API-key-gated search-only provider, `ddgs` for a no-key provider that lazy-installs its SDK.
|
||||
|
||||
## The WebSearchProvider ABC
|
||||
|
||||
Subclass `agent.web_search_provider.WebSearchProvider`. The only required members are `name`, `is_available()`, and whichever of `search()` / `extract()` you implement. (Deep crawling is not a separate method — it's a mode of `extract()`.)
|
||||
|
||||
```python
|
||||
# plugins/web/my-backend/provider.py
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from typing import Any, Dict, List
|
||||
|
||||
from agent.web_search_provider import WebSearchProvider
|
||||
|
||||
|
||||
class MyBackendWebSearchProvider(WebSearchProvider):
|
||||
"""Minimal search-only provider against the My Backend HTTP API."""
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
# Stable id used in web.search_backend / web.extract_backend / web.backend
|
||||
# config keys. Lowercase, no spaces; hyphens permitted.
|
||||
return "my-backend"
|
||||
|
||||
@property
|
||||
def display_name(self) -> str:
|
||||
# Human label shown in `hermes tools`. Defaults to `name`.
|
||||
return "My Backend"
|
||||
|
||||
def is_available(self) -> bool:
|
||||
# Cheap check — env var present, optional dep importable, etc.
|
||||
# MUST NOT make network calls (runs on every `hermes tools` paint).
|
||||
return bool(os.getenv("MY_BACKEND_API_KEY", "").strip())
|
||||
|
||||
def supports_search(self) -> bool:
|
||||
return True
|
||||
|
||||
def supports_extract(self) -> bool:
|
||||
return False
|
||||
|
||||
def search(self, query: str, limit: int = 5) -> Dict[str, Any]:
|
||||
import httpx
|
||||
|
||||
api_key = os.environ["MY_BACKEND_API_KEY"]
|
||||
try:
|
||||
resp = httpx.get(
|
||||
"https://api.example.com/search",
|
||||
params={"q": query, "count": max(1, min(int(limit), 20))},
|
||||
headers={"Authorization": f"Bearer {api_key}"},
|
||||
timeout=15,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
except httpx.HTTPError as exc:
|
||||
return {"success": False, "error": str(exc)}
|
||||
|
||||
# Response shape is fixed — see "Response shape" below.
|
||||
return {
|
||||
"success": True,
|
||||
"data": {
|
||||
"web": [
|
||||
{
|
||||
"title": item.get("title", ""),
|
||||
"url": item.get("url", ""),
|
||||
"description": item.get("snippet", ""),
|
||||
"position": idx + 1,
|
||||
}
|
||||
for idx, item in enumerate(data.get("results", []))
|
||||
],
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
```python
|
||||
# plugins/web/my-backend/__init__.py
|
||||
from plugins.web.my_backend.provider import MyBackendWebSearchProvider
|
||||
|
||||
|
||||
def register(ctx) -> None:
|
||||
"""Plugin entry point — called once at load time."""
|
||||
ctx.register_web_search_provider(MyBackendWebSearchProvider())
|
||||
```
|
||||
|
||||
## plugin.yaml
|
||||
|
||||
```yaml
|
||||
name: web-my-backend
|
||||
version: 1.0.0
|
||||
description: "My Backend web search — Bearer-auth REST API"
|
||||
author: Your Name
|
||||
kind: backend
|
||||
provides_web_providers:
|
||||
- my-backend
|
||||
requires_env:
|
||||
- MY_BACKEND_API_KEY
|
||||
```
|
||||
|
||||
| Key | Purpose |
|
||||
|---|---|
|
||||
| `kind: backend` | Routes the plugin through the backend-loading path |
|
||||
| `provides_web_providers` | List of provider `name`s this plugin registers — used by the loader to advertise the plugin in `hermes tools` even before `register()` runs |
|
||||
| `requires_env` | Interactive credential prompt during `hermes plugins install` (see [Build a Hermes Plugin](/developer-guide/plugins#gate-on-environment-variables) for the rich format) |
|
||||
|
||||
## ABC reference
|
||||
|
||||
Full contract in `agent/web_search_provider.py`. Methods you may override:
|
||||
|
||||
| Member | Required | Default | Purpose |
|
||||
|---|---|---|---|
|
||||
| `name` | ✅ | — | Stable id used in `web.*_backend` config |
|
||||
| `display_name` | — | `name` | Label shown in `hermes tools` |
|
||||
| `is_available()` | ✅ | — | Cheap availability gate — env vars, optional deps |
|
||||
| `supports_search()` | — | `True` | Capability flag for `web_search` routing |
|
||||
| `supports_extract()` | — | `False` | Capability flag for `web_extract` routing |
|
||||
| `search(query, limit)` | conditional | raises | Required when `supports_search()` returns `True` |
|
||||
| `extract(urls, **kwargs)` | conditional | raises | Required when `supports_extract()` returns `True` |
|
||||
|
||||
Providers can advertise multiple capabilities from a single class — Firecrawl, Tavily, Exa, and Parallel all implement both search and extract. Brave Search and DDGS are search-only; SearXNG is search-only with a documented "pair me with an extract provider" workflow.
|
||||
|
||||
## Response shape
|
||||
|
||||
The tool wrapper expects a fixed envelope so it doesn't have to translate between backends.
|
||||
|
||||
**Search success:**
|
||||
|
||||
```python
|
||||
{
|
||||
"success": True,
|
||||
"data": {
|
||||
"web": [
|
||||
{"title": str, "url": str, "description": str, "position": int},
|
||||
...
|
||||
],
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
**Extract success:**
|
||||
|
||||
```python
|
||||
{
|
||||
"success": True,
|
||||
"data": [
|
||||
{
|
||||
"url": str,
|
||||
"title": str,
|
||||
"content": str,
|
||||
"raw_content": str,
|
||||
"metadata": dict, # optional
|
||||
"error": str, # optional, only on per-URL failure
|
||||
},
|
||||
...
|
||||
],
|
||||
}
|
||||
```
|
||||
|
||||
**Either capability, on failure:**
|
||||
|
||||
```python
|
||||
{"success": False, "error": "human-readable message"}
|
||||
```
|
||||
|
||||
Both `search()` and `extract()` may be `async def` — the dispatcher detects coroutine functions via `inspect.iscoroutinefunction` and awaits accordingly. Sync implementations that do blocking I/O (HTTP, SDK calls) are fine for small backends; the dispatcher handles threading.
|
||||
|
||||
## Capability flags
|
||||
|
||||
Hermes routes calls to the right provider based on the `supports_*` flags. A common multi-provider setup:
|
||||
|
||||
```yaml
|
||||
# ~/.hermes/config.yaml
|
||||
web:
|
||||
search_backend: "brave-free" # search-only, fast, free 2k/mo
|
||||
extract_backend: "firecrawl" # extract + crawl, paid quota
|
||||
```
|
||||
|
||||
When `web.search_backend` or `web.extract_backend` aren't set, both fall through to `web.backend`. When that's also unset, Hermes picks the first available provider that supports the requested capability based on env-var presence.
|
||||
|
||||
If your provider only supports one capability, leave the other flags at their default (`False`) and the registry will skip it for that tool — users won't see misleading "provider X failed" errors when they're using X only for search and asking the agent to extract.
|
||||
|
||||
## How Hermes wires it into the tools
|
||||
|
||||
The `web_search` and `web_extract` tools live in `tools/web_tools.py`. At call time they:
|
||||
|
||||
1. Read the relevant config key (`web.search_backend` for `web_search`, `web.extract_backend` for `web_extract`)
|
||||
2. Ask the registry for the provider with that `name`
|
||||
3. Check `is_available()` and the matching `supports_*()` flag
|
||||
4. Dispatch to `search()` / `extract()` (deep crawl runs as a mode inside `extract()`), awaiting if the method is a coroutine
|
||||
5. JSON-serialize the response envelope and hand it back to the LLM
|
||||
|
||||
Errors surface as the tool result; the LLM decides how to explain them. If no provider is registered (or every available one fails the capability gate), the tool returns a helpful error pointing at `hermes tools`.
|
||||
|
||||
## Lazy-installing optional dependencies
|
||||
|
||||
If your provider wraps a third-party SDK (like DDGS does with the `ddgs` package), don't `import` it at module top level. Use `tools.lazy_deps.ensure(...)` inside `is_available()` or `search()` — Hermes will install the package on first use, gated by `security.allow_lazy_installs`. See [Build a Hermes Plugin → Lazy-install](/developer-guide/plugins#lazy-install-optional-python-dependencies) for the security model.
|
||||
|
||||
## Reference implementations
|
||||
|
||||
- **`plugins/web/brave_free/`** — small, API-key-gated, search-only HTTP provider. Good starting template.
|
||||
- **`plugins/web/ddgs/`** — no-key provider that lazy-installs its SDK. Useful pattern for backends that wrap a Python package.
|
||||
- **`plugins/web/firecrawl/`** — full multi-capability provider (search + extract + crawl) with multiple format modes.
|
||||
- **`plugins/web/searxng/`** — self-hosted, URL-configured backend with no auth.
|
||||
- **`plugins/web/xai/`** — LLM-backed search via Grok's server-side `web_search` tool. Shows how to reuse an existing OAuth/env-var credential surface (`tools/xai_http.py`) without adding new env vars, and how to write a cheap `is_available()` that honors the no-network contract.
|
||||
|
||||
## Distribute via pip
|
||||
|
||||
```toml
|
||||
# pyproject.toml
|
||||
[project.entry-points."hermes_agent.plugins"]
|
||||
my-backend-web = "my_backend_web_package"
|
||||
```
|
||||
|
||||
`my_backend_web_package` must expose a top-level `register` function. See [Distribute via pip](/developer-guide/plugins#distribute-via-pip) in the general plugin guide for the full setup.
|
||||
|
||||
## Related pages
|
||||
|
||||
- [Web Search](/user-guide/features/web-search) — user-facing feature documentation and per-backend configuration
|
||||
- [Plugins overview](/user-guide/features/plugins) — all plugin types at a glance
|
||||
- [Build a Hermes Plugin](/developer-guide/plugins) — general tools/hooks/slash commands guide
|
||||
Reference in New Issue
Block a user