--- title: 'Local Agent Fleet' description: 'Slash-command surface for monitoring, coordinating, and exchanging context between local AI agents (Claude Code, Cursor, Aider, ...)' --- ## Overview OpenSRE treats every other AI agent running on your machine — Claude Code, Cursor, Aider, Codex CLI, Gemini, and friends — as a microservice and applies normal SRE practice: golden signals, SLOs, and incident response. The whole fleet view lives behind one slash command in the interactive shell: ```text > /fleet ``` Subcommands drill into specific surfaces. Below: how the agent fleet is discovered (`opensre fleet scan`), the dashboard itself (`/fleet`), then live tail of agent stdout (`/fleet trace`), then the cross-agent context bus (`/fleet bus`). ## `opensre fleet scan` — discover running agent sessions `opensre fleet scan` enumerates running AI-coding-agent sessions visible to the current user. The classifier inspects the process table (via `ps -axo pid,ppid,args`) and labels each candidate by executable plus known argv shapes — no PID is registered until you ask for it with `--register`. ### Strict mode (default) Recognizes the typical CLI invocations for Claude Code, Cursor, Aider, Codex, Gemini, and Antigravity (`agy`). For Claude Code specifically, all of the following process shapes are treated as a session: ```text claude claude --resume claude -r claude --prefill "" claude --print "" claude -p "" claude --continue claude -c claude code … claude --input-format stream-json … claude --output-format stream-json … ``` Equals-form flags (e.g. `claude --resume=`, `claude --print=`) are accepted for any flag in the list above. The Claude Desktop GUI and its Electron helpers are filtered out cross-platform — `Claude.app/Contents/`, `Claude Helper (Renderer)`, `/snap/claude/`, `/usr/lib/claude-desktop`, AppImage mount points (`/.mount_Claude…`), and the Windows `Program Files\Claude\` / `AppData\Local\Programs\Claude\` install locations are recognized as desktop artifacts and never labeled as `claude-code`. The negative filter matches against `argv[0]` only, so a CLI installed under a Claude-flavored prefix (for example `/opt/Claude/claude`) is still surfaced. The same cross-platform desktop filter also rejects **Codex Desktop** (`Codex.app/Contents/MacOS/Codex`, `/snap/codex/`, `/.mount_Codex…`, Flatpak `com.openai.codex`, Windows `Program Files\Codex\` and `AppData\Local\Programs\codex\`) and **Cursor Desktop** (`Cursor.app/Contents/MacOS/Cursor`, `/snap/cursor/`, `/.mount_Cursor…`, Flatpak `com.cursor.cursor`, Windows `Program Files\Cursor\` and `AppData\Local\Programs\cursor\`), so neither GUI is mislabeled as the `codex` or `cursor` CLI in strict or `--all` mode. The macOS hints deliberately target only the main bundle binary so Electron helper subprocesses (e.g. `Cursor Helper (Plugin)` running Cursor's AI agent) remain eligible for the loose `--all` matcher. ### Loose mode (`--all`) `opensre fleet scan --all` relaxes the argv requirements so that helper and broker processes whose argv contains agent-shaped tokens are also surfaced. The Claude / Codex / Cursor Desktop negative filter is still applied — `--all` never mislabels desktop processes as `claude-code`, `codex`, or `cursor`. ### Registering discovered sessions ```text opensre fleet scan --register ``` writes every discovered session into the local agent registry so that the rest of the fleet surface (`/fleet`, `/fleet trace`, `/fleet bus`) can target it by PID. Without `--register`, the command is read-only. ## `/fleet` — fleet dashboard The dashboard renders a seven-column table of every registered or discovered local AI agent. Run from the interactive REPL: ```text > /fleet agents agent pid uptime cpu% tokens/min $/hr status claude-code-8421 8421 2h12m 18.4 320 $0.08 running codex-13442 13442 11m 4.2 175 $0.04 running cursor-agent-9999 9999 47m 0.6 - - running ``` ### Column data sources | Column | Source | Notes | | --- | --- | --- | | `agent` | `AgentRecord.name` | Registered name or discovery-generated `-`. | | `pid` | `AgentRecord.pid` | OS process id. | | `uptime` | sampler probe (psutil `create_time`) | Compact form: `45s` / `12m` / `2h12m` / `3d4h`. | | `cpu%` | sampler probe (psutil) | Trailing 100 ms `cpu_percent`. | | `tokens/min` | per-PID 60 s rolling window | Real for `claude-code` and `codex`; `-` for the rest. | | `$/hr` | observed cost from token rate × model pricing | Renders `-` when the model is unknown. | | `status` | sampler probe (psutil) | `running` / `sleeping` / `zombie` / etc. | ### `tokens/min` semantics The cell shows the **sum of tokens emitted in the trailing 60 seconds**, scaled to a per-minute figure. Three states: - **Real value** (`320`, `1.2k`): the agent's provider has a working meter and the on-disk session log was readable. Today this is `claude-code` (reads `~/.claude/projects//.jsonl`) and `codex` (reads `$CODEX_HOME/sessions/YYYY/MM/DD/rollout-*.jsonl`, default `$CODEX_HOME=~/.codex`). - **`0`**: the agent was observed at least once but emitted nothing in the last 60 s. Honest UX — distinguishes "idle session" from "not observable". - **`-`**: never observed. Either the provider has no meter yet (cursor, aider, gemini-cli, antigravity-cli, opencode, kimi, copilot in this PR), the session log is unreadable (rare; macOS hardened-runtime processes can deny `psutil.cwd()`), or the interactive REPL is not running (non-interactive `opensre fleet list` never starts the sampler). Claude Code cache-read/cache-creation tokens are included in the visible activity count because they are separate input work. Codex `cached_input_tokens` are treated as a discounted subset of `input_tokens`, so they affect `$/hr` but are not added again to the visible `tokens/min` total. ### `$/hr` semantics `$/hr` is a **projected hourly burn rate** derived from the same trailing 60 s window the `tokens/min` column reports — not the actual spend over the last hour: ```text $/hr = cost_of_usage_buckets_in_the_trailing_60s_window(model) × 60 ``` Reads as "if the agent sustains the current ritmo for one hour, it will cost this much". Useful as an operational signal: it tracks `cpu%` in spirit, reacts immediately when an agent goes idle or switches model, and keeps memory bounded (a ~12-entry deque per PID at the default 5 s tick instead of an unbounded hour-long history). If you need actual spend over the last hour, that's a different metric — open a follow-up issue. Pricing uses per-bucket rates for input, output, cached input, cache read, and cache creation where the provider emits those counters. The local pricing table is a vendored `models.dev` snapshot for the Claude Code and Codex models supported here, with optional `agents.yaml` input/output overrides. Pricing returns `None` for unknown models and the cell falls back to `-`; the dashboard never invents a rate. The yaml `hourly_budget_usd` field is **not** the cell content — it's reserved for a future budget-alarm feature. Model resolution order (highest to lowest): 1. **NDJSON model hint** from the meter (most accurate — reflects the model the running session is currently using). 2. **`agents.yaml` override** (`AgentBudget.model`): ```yaml agents: claude-code-8421: model: claude-sonnet-4-5 codex-13442: model: gpt-5-codex ``` 3. **Provider env var** (`CLAUDE_CODE_MODEL`, `CODEX_MODEL`) read via psutil. May fail on macOS hardened-runtime processes, in which case the cell falls back to `-` unless one of the two higher priority sources resolved. ### Provider coverage today | Provider | `tokens/min` | `$/hr` | | --- | --- | --- | | `claude-code` | real | real | | `codex` | real | real | | `cursor`, `aider`, `gemini-cli`, `antigravity-cli`, `opencode`, `kimi`, `copilot` | `-` | `-` | Adding a real meter for another provider is a self-contained follow-up PR: implement the `TokenMeter` and `TokenSource` protocols, register both instances, add tests. No changes to the sampler, tracker, view, or pricing layer are needed. ## `/fleet trace` — live stdout tail `/fleet trace ` opens a live tail of an agent's stdout inside the OpenSRE interactive shell — the equivalent of `kubectl logs -f` for the local AI agent fleet. Use it when the `/fleet` dashboard shows an agent that looks **stuck**, **looping**, or **noisy** and you want to see what it's actually printing without leaving the REPL. ```text > /fleet trace 8421 trace claude-code (pid 8421) Ctrl+C to stop … live agent output … ^C · trace ended ``` ### Trace usage ```text /fleet trace ``` `` is the operating-system process id of the agent to attach to. The pid does **not** have to be in the OpenSRE registry; if it is, the agent's registered name is shown in the header. Otherwise the header falls back to `pid `. ### Platform support Only **regular files** backing fd 1 of the target process are supported. TTY/PTY/pipe/socket/anon-inode targets are rejected at attach time with a precise reason — tailing those would compete with the legitimate consumer for bytes and produce corrupted output. | Platform | Resolver | Supported targets | | --- | --- | --- | | **Linux** | `os.readlink("/proc//fd/1")` | regular files | | **macOS** (best-effort) | `lsof -F ftn -p `, only `t REG` blocks | regular files | | **Windows** | not supported | — | The most common useful case is an agent whose stdout was redirected to a log file (for example `claude > ~/.claude/log` or `nohup`-launched agents). TTY-bound foreground processes cannot be tailed. If a target cannot be tailed, `/fleet trace` exits with one of: - `cannot trace …: stdout is on a terminal; live tail not supported` - `cannot trace …: stdout is a pipe; live tail not supported` - `cannot trace …: stdout is a socket; live tail not supported` - `cannot trace …: no such pid ` - `cannot trace …: stdout target /path no longer exists` - `cannot trace …: cannot inspect pid (permission denied)` ### Memory The live view is bounded by a **4 MiB ring buffer per session**. When the buffer fills, the oldest whole chunks are dropped first, so the visible tail always reflects the latest output. Internally the reader thread also publishes through a bounded queue and drops the oldest chunk on overflow — burst writers cannot blow up memory. There is **no backlog replay**: only output emitted *after* attach is shown. The reader seeks the file to EOF on attach. ### Stopping a trace A single **Ctrl+C** returns to the REPL prompt. The session is closed, the reader thread joins, and the file descriptor is released. This is deliberately different from the LLM-streaming surface (`/fleet`, `/investigate` and friends), where a Ctrl+C double-press is required so a stray keypress doesn't abort an in-flight response. ### Trace limitations - **stdout only.** Stderr (fd 2) is not tailed in this version. - **No backlog replay.** Pre-attach bytes are not visible. - **Not for TTY/PTY targets.** Foreground processes whose stdout is the controlling terminal cannot be tailed; a future change may add PTY interception for OpenSRE-spawned agents. - **Log rotation is not detected.** If the underlying file is rotated or replaced (logrotate-style), the tail keeps following the original inode until the process exits. - **No secret redaction.** Output is rendered as raw bytes (with UTF-8 decoded under `errors="replace"`). Redaction of secrets in the live tail is tracked separately under the `monitor-local-agents` Phase 3 hygiene work. - **Quiet stdout while the PID is still alive.** The reader follows file EOF like `tail -f`: if the process stops writing while it remains alive, the last chunk stays on screen and nothing new appears until more bytes land or you detach. That is normal idling — not necessarily a exited or stuck agent reader. - **ANSI and terminal sequences.** Trace output passes through Rich with ANSI interpretation, same trust model as dumping `kubectl logs` into a TTY: buggy or hostile agents can emit control sequences affecting the viewer. Only trace processes you trust; there is no sandboxing step. ## `/fleet bus` — shared context channel The bus is an opt-in, local-only pub/sub channel that carries findings between agents. One agent publishes a finding (e.g. *"the auth bug is in `services/auth.py:42`"*) and every attached subscriber sees it live. The inspector is the REPL itself: ```text > /fleet bus tailing /fleet bus — Ctrl-C to exit [claude-code:8421] services/auth.py:42 — null deref on missing token [cursor:9133] services/auth.py:42 — confirmed, repro on commit abc123 ^C (detached) > ``` Ctrl-C returns to the prompt. Messages already published are not replayed to late subscribers. The bus provides **at-most-once delivery with no ordering guarantees** — a frame may be dropped if a subscriber's socket is slow or disconnected, and two publishers writing concurrently may be interleaved in different orders at different subscribers. Do not assume per-publisher or global FIFO ordering. ### Transport - **Socket**: Unix-domain stream socket at `~/.opensre/agents-bus.sock`. - **PID sidecar**: `~/.opensre/agents-bus.sock.pid` (mode `0600`). The broker writes its PID here on `start()` and removes it on `stop()`. The liveness probe used by every `publish()` / `subscribe()` reads this file rather than connecting to the socket — connection probing would otherwise register a short-lived phantom subscriber on every call. **The directory must be writable.** If the PID file write fails (disk full, permission denied, ...), the broker refuses to start and the `OSError` propagates to the caller. This is intentional: silently running without a sidecar would let peers see the broker as dead, unlink its socket, and silently split the bus. - **Permissions**: `0600` — only the user who started the broker can read or write it. - **Wire format**: JSON Lines (one JSON object per `\n`-terminated frame). - **Topology**: self-electing broker. The first `publish()` or `subscribe()` call that finds no live socket binds it and runs an in-process daemon thread that fans frames out. Other processes attach as plain clients. If the broker dies, the next operation re-elects — agents can publish and subscribe even when OpenSRE itself is not running. ### Message schema The wire payload mirrors the shape of `evidence` records in `core/state/models.py` so a finding can later be lifted into an investigation without renaming fields. | Field | Type | Required | Notes | | --- | --- | --- | --- | | `agent` | string | yes | `":"`, e.g. `"claude-code:8421"`. Same convention as `WriteEvent.agent`. | | `topic` | string | yes | `"finding"` is the canonical value; other topics are reserved for future phases. | | `summary` | string | yes | One-line human-readable description. | | `source` | string | no | One of the `EvidenceSource` literals (`github`, `datadog`, ...) or free-form. | | `path` | string | no | `"file.py:42"` style location. Optional. | | `data` | object | no | Free-form payload. Default `{}`. | | `id` | string | no | UUID. Generated if omitted. | | `timestamp` | string | no | ISO-8601 UTC. Generated if omitted. | | `schema_version` | int | no | Currently `1`. | Example frame on the wire (single line, broken here for readability): ```json { "agent": "claude-code:8421", "topic": "finding", "summary": "null deref on missing token", "source": "github", "path": "services/auth.py:42", "data": {"commit": "abc123"}, "id": "f4c4...", "timestamp": "2026-05-09T15:04:42+00:00", "schema_version": 1 } ``` ### Publishing from another agent Any process that can speak Unix-domain sockets can publish. The simplest path is to import the helper: ```python from tools.system.fleet_monitoring import BusMessage, publish publish(BusMessage( agent="claude-code:8421", topic="finding", summary="null deref on missing token", source="github", path="services/auth.py:42", data={"commit": "abc123"}, )) ``` Publishers without a Python dependency on OpenSRE can connect directly: ```bash python - <<'EOF' import json, os, socket, uuid, datetime sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) sock.connect(os.path.expanduser("~/.opensre/agents-bus.sock")) sock.sendall((json.dumps({ "agent": "claude-code:8421", "topic": "finding", "summary": "null deref on missing token", "path": "services/auth.py:42", "id": str(uuid.uuid4()), "timestamp": datetime.datetime.now(datetime.UTC).isoformat(), "schema_version": 1, }) + "\n").encode()) sock.close() EOF ``` ### Limits and trust boundary - **Local-only.** The bus never leaves the machine. The socket has no network binding. - **Trusted-peer channel — treat findings as unverified input.** The bus has no authentication beyond filesystem permissions: any process running as your user can publish arbitrary findings. This is intentional — the bus is designed for cooperative agents, not adversarial ones. Downstream consumers (agents, the REPL, investigation state) **must not act on a bus finding without independent confirmation**; treat it as a hint or lead, not a verified fact. A compromised or misbehaving agent on the same user account can inject any payload it likes. - **Frame cap.** Frames over 64 KiB are dropped with a warning — a finding payload that big is almost certainly a bug. - **At-most-once, unordered delivery.** A frame is dropped silently if a subscriber is slow or disconnected at broadcast time. Two publishers writing concurrently may arrive in different orders at different subscribers. Do not build logic that depends on delivery guarantees or ordering. - **No replay buffer.** Subscribers see only what is published *after* they attach. A persistent ring buffer is a candidate for a follow-up phase. ## Related - `/fleet` — the registered fleet dashboard. - `/fleet budget` — per-agent hourly budgets. - `/fleet conflicts` — file-write conflicts between local AI agents.