tracer-cloud--opensre/surfaces/interactive_shell/AGENTS.md

Interactive shell package

These instructions apply to interactive_shell/ and all of its subdirectories. The repo-root AGENTS.md still applies.

Purpose

interactive_shell/ owns the interactive OpenSRE terminal surface: the REPL loop, slash-command surface, local alert ingestion, shell execution, and Rich / prompt-toolkit UI. Reusable agent session state, prompt history, grounding, and prompt construction live under core.agent.

Design for a terminal user who may be in the middle of an incident: behavior should be predictable, interruptible, explainable, and safe by default.

Package map and ownership

Area	Owns	Keep out
main.py	process/bootstrap boundary for starting the REPL	per-turn dispatch/runtime logic
controller.py	top-level REPL wiring, alert listener lifecycle, prompt loop, background workers, and shutdown	feature-specific business logic or compatibility-only forwarding
runtime/core/turn_accounting.py	shell turn accounting (ShellTurnAccounting) for analytics, telemetry, recorder flush, turn persistence, and intent stamps	turn-flow control (owned by core.agent_harness) or tool-calling turn execution
command_registry/	slash-command definitions, argument validation, command dispatch	long-running implementation details better placed in services/runtime modules
runtime/	background task workers, lifecycle/ReplState, runtime context assembly, semantic shell-turn execution, and core harness adapters	prompt text, reusable session persistence, or compatibility shims
tools/interactive_shell/shell/	shell command parsing, shell execution policy, subprocess execution, and the run_shell_command/run_cd/run_pwd runner (next to the shell_run tool in tools/interactive_shell/actions/shell.py)	slash-command execution
references/	CLI/docs/source/AGENTS reference loading and caching	generated model prose
config/	interactive-shell config loading and tool catalog metadata	global app config unrelated to the REPL
ui/	Rich/prompt-toolkit rendering, theme, menus, streaming output, and domain views such as incoming_alerts.py (receiver/queue/listener lifecycle lives in core.domain.alerts.inbox)	business logic or network calls

When a change crosses these boundaries, prefer extracting a small helper in the owning area rather than adding more logic to the caller.

Cross-cutting rules

• Treat every external input as untrusted: user prompt text, slash-command args, alert payloads, files read into prompts, history, subprocess output, model output, and integration metadata.
• Keep the interactive path responsive. Long-running work must be cancellable, timeout-bounded, moved off the input path, or surfaced with clear progress.
• Preserve import-time lightness. Do not start threads, call LLMs, read large files, or contact networks at module import time.
• Prefer explicit data models and typed helpers over loosely shaped dictionaries when data crosses submodule boundaries.
• Keep user-visible strings intentional. Slash-command names, flags, output labels, prompts, response bodies, and error wording are user-facing API.
• Avoid new module-level mutable globals. If global coordination is unavoidable, provide deterministic reset/cleanup hooks and test isolation.
• Do not keep compatibility-only forwarding modules after moving code. Migrate callers/tests to the canonical owner and remove the old import path in the same change.

Slash commands

• Add commands as SlashCommand entries in the relevant command_registry/* module. Keep handlers small: parse args, call focused helpers, render result.
• REPL + CLI parity (required): Every command in SLASH_COMMANDS must have a matching _MCP_BY_COMMAND entry in command_registry/slash_catalog.py. That catalog feeds the LLM planner (slash_invoke), planner tool specs, and compact help text. Without it, CI fails (test_slash_catalog_covers_all_registered_commands).
  • New REPL-only slash command: add SlashCommand in the owning command_registry/* module and _mcp(...) in slash_catalog.py (keep keys sorted alphabetically in _MCP_BY_COMMAND).
  • New CLI with REPL parity: add the Click command under surfaces/cli/commands/, register a SlashCommand in command_registry/cli_parity.py (subprocess to opensre …), and add _MCP_BY_COMMAND in slash_catalog.py with llm_description, use_cases, and anti_examples aligned to the command’s usage tuple.
  • Verify before push: uv run python -m pytest tests/interactive_shell/command_registry/test_slash_catalog.py -q
• Use validate_args for cheap pre-policy validation so bad arguments do not trigger confirmations or side effects.
• Send command execution through the central dispatch and execution-policy helpers. Do not bypass execution_policy.py for new commands.
• Alpha allow-all execution policy (current behavior): the REPL runs with no command guardrails. execution_policy.py resolves every action to allow with no confirmation prompt — all slash/opensre commands, investigations, synthetic tests, code-agent launches, LLM runtime switches, and all shell commands run immediately, in any context (TTY or not, trust mode or not). There is no shell-command safety policy: the read-only/mutating/restricted classification and the deny floor were removed (shell_policy.py deleted; parsing/policy/execution live under tools/shell/). Mutating commands (rm/mv/docker), restricted commands (sudo, systemctl, kill, dd, …), shell operators (| && ; > <), and command substitution all run; the ! prefix is honored but optional. The only shell input still rejected is genuinely empty input (a bare ! or whitespace). Do not re-add a shell allowlist or deny floor while in alpha — see docs/interactive-shell-action-policy.md. The former ExecutionTier classification was removed because it gated nothing under default-allow; if an opt-in stricter policy is reintroduced after alpha, gate it in execution_policy.py (the ask verdict, confirmation UX, and trust_mode are retained as the hook), not via a planner-stage denial.
• Non-TTY behavior under default-allow: actions no longer fail closed on non-interactive stdin (there is nothing to confirm). The fail-closed path only applies if a verdict is explicitly ask, which the default policy does not emit.
• CPR / exclusive-stdin registration (required for table-outputting commands): Under patch_stdout(raw=True), the REPL runs dispatch concurrently with the next prompt_async(). When a command emits Rich table output, prompt_toolkit redraws the prompt mid-flight, sending an ESC[6n DSR query; the terminal's CPR response (ESC[<row>;<col>R) arrives as literal keystrokes in the incoming prompt buffer, causing garbage like ^[[60;1R to appear. Any command that calls print_repl_table (directly or via render_table / render_integrations_table / render_models_table / etc.) must be added to _EXCLUSIVE_STDIN_MENU_COMMANDS in runtime/utils/input_policy.py. That makes the main loop call await state.queue.join(), blocking the next prompt until dispatch completes and both drain cycles clean up stale CPR bytes before the next prompt_async() starts.
  • How to check: after adding a command, run it in the REPL and type a few characters in the next prompt. If no ^[[…R garbage appears, the registration is correct.
  • Agent-selected interactive commands: _EXCLUSIVE_STDIN_MENU_COMMANDS only reserves stdin for literal /slash command text that _literal_slash_command_text recognizes. When free text like "remove github" is resolved by the action agent into an inline-picker command (/integrations remove, /integrations setup, /mcp connect, /mcp disconnect, or a bare /integrations / /mcp menu), the loop has not reserved stdin, so tools/interactive_shell/actions/slash.py must NOT run the picker inline. It defers via session.queue_auto_command(...), which re-submits the command as literal command text so the loop can reserve exclusive stdin before the agent path runs it. New raw-stdin picker/wizard commands the action agent can emit must be added to _INTERACTIVE_PICKER_MENUS / _INTERACTIVE_PICKER_SUBCOMMANDS in tools/interactive_shell/actions/slash.py.

Action Selection And Execution

• Hard boundary: do not add regex/keyword/fuzzy intent routing for natural language, or any deterministic mapping from non-/-prefixed prose to an action (e.g. "show integrations" -> /integrations). Engineers have been fired before for reintroducing intent heuristics that compete with the action agent.
  • Sanctioned exception: input the user types as a literal /slash command is dispatched deterministically (a static slash_invoke call in core/agent_harness/turns/action_driver.py), so slash commands keep working when the action-agent LLM is unavailable. This is an explicit-command bypass, not intent inference — it fires only when the message is a /command, and free-form text is still LLM-selected. See docs/interactive-shell-action-policy.md ("Deterministic literal-/slash dispatch").
• No planning-stage fail-closed safeguard (v0.1 decision). The second-phase action agent never denies a turn. Because every terminal action is read-only, an unmatched/ambiguous/chatty clause is not a safety risk — the agent executes the clauses it can map and lets the rest fall through to the conversational assistant. We removed the denied decision path, the mark_unhandled planner tool, the UNHANDLED: convention, and the "I couldn't safely decide actions" message because they caused frequent false denials (e.g. a conversational question that embedded a quoted, list-style directive) with no safety upside. Details and rationale live in core/agent_harness/AGENTS.md. If mutating actions are ever introduced, gate them with the execution-stage confirmation policy (tools/shared/execution_policy.py), not a planner-stage denial.
• Keep deterministic command detection in orchestration/ for terminal UI policy only; use the action agent for slash/tool action selection.
• Send uncertainty to a safe surface: help/chat or a clarification, not direct mutation or shell execution.
• LLM-generated text must never execute directly. Convert proposed actions into explicit planned actions, show them to the user when appropriate, then execute through orchestration/ and policy gates.
• Keep action summaries human-readable and specific enough for confirmation UX and audit logs.
• When adding a new action type, test allowed, denied, and confirmation-required paths.

LLM prompts, grounding, and references

• Keep prompts bounded. Enforce size caps for docs, source chunks, histories, observations, alert text, and command output included in model context.
• Ground procedural/help answers in maintained references (docs/, CLI help, AGENTS files, source snippets). If references do not support an answer, say so rather than inventing steps.
• Do not include secrets in prompts. Redact or omit tokens, auth headers, env values, local credentials, and raw integration config.
• Keep prompt rules reusable in core.agent_harness.prompts so chat/help/action surfaces use consistent terminology and formatting.
• Reference caches should be deterministic, invalidatable when source files change, and cheap to rebuild in tests.

Terminal UI and rendering

• Escape user-controlled content before passing it to Rich markup (rich.markup.escape): alerts, command output, file paths, integration names, model/provider labels, errors, docs snippets, and model text that is not already intentionally rendered as Markdown.
• Use semantic tokens from ui/theme.py. Do not introduce raw hex colors, Rich named colors, or raw ANSI escapes outside ui/theme.py unless a narrow prompt-toolkit compatibility path requires it.
• Keep rendering helpers as pure as practical: accept data, return/render Rich objects, avoid reading config or mutating session state from UI modules.
• Any raw terminal-mode code must check TTY support and restore terminal state in finally.
• Be careful mixing prompt_toolkit.patch_stdout, Rich live rendering, and background output. Prefer append-only, paragraph-buffered, or throttled rendering paths that do not corrupt the editable prompt.
• UI changes should handle narrow terminals, non-ASCII fallback where relevant, long text, empty states, and non-TTY automation.

Shell, subprocesses, and local system effects

• Shell execution changes belong under shell/ and must preserve parsing, quoting, timeout, redaction, and policy behavior.
• Treat subprocess output as untrusted display text; escape it before Rich markup and cap what is retained or sent to prompts.
• Use explicit timeouts and clear cancellation behavior for subprocesses. Avoid waits that can hang the REPL indefinitely.
• Keep allow/deny decisions explainable. If a command is blocked, return a user-facing reason and a safe alternative when possible.

State, history, config, and background work

• Prefer explicit Session fields for session state. Keep ownership clear: runtime owns lifecycle, history owns persistence, config owns shell-specific settings.
• Background threads/tasks/listeners must have deterministic shutdown. Tests should stop handles and workers in fixtures or finally blocks.
• Protect shared queues and mutable session data with locks or single-owner discipline. Avoid check-then-act races around queues, cancellation flags, current tasks, and listener handles.
• History should avoid storing secrets or excessive payloads. Apply truncation and privacy policy consistently.
• Config loading should degrade gracefully with actionable errors; do not make the REPL unusable because an optional config or catalog source is missing.

External input and local listener safety

• Network-ish local surfaces such as core.domain.alerts.inbox (started by the REPL entrypoint) must validate cheap request metadata before blocking reads or expensive parsing.
• Never perform unbounded request-body reads. For alert POSTs specifically, validate Content-Length first, and only then read the bounded body:
  • non-numeric Content-Length values make int(...) raise ValueError; catch this and return 400.
  • negative lengths must return 400; rfile.read(-1) reads until EOF rather than zero bytes, which can stall the single-threaded handler.
  • oversized positive lengths must return 413 without attempting to read the advertised body.
• Preserve clean unauthorized responses for real POST bodies by draining only a bounded body before returning 401; this avoids close-with-unread-data resets on some platforms without allowing oversized pre-auth reads.
• Keep request-size and malformed-header checks effective for both authenticated and unauthenticated callers.
• Keep non-loopback listener binding protected by a token. Use constant-time token comparison and never log bearer tokens, raw auth headers, or full alert payloads.

Testing expectations

• Put tests under tests/interactive_shell/, mirroring the package area when useful (orchestration/, ui/, etc.). Never add tests under source packages.
• For focused changes, run the closest tests, for example:
  • uv run python -m pytest tests/core/domain/alerts/test_inbox.py
  • uv run python -m pytest tests/interactive_shell/<area>/
  • uv run python -m pytest tests/interactive_shell/
• Add regression tests for incident-prone edges: platform socket behavior, malformed input, non-TTY execution, cancellation, policy denial, prompt-size caps, Rich escaping, and background cleanup.
• Prefer deterministic tests over sleep-heavy tests. Use fake classifiers, fake sessions, fake consoles, monkeypatched subprocesses, and small fixtures.
• For UI work, test pure formatting/rendering helpers where possible and keep full REPL-loop tests minimal.
• For action-planning or execution-policy changes, test both safe fallback behavior and the intended positive path.

Change checklist

Before considering an interactive-shell change complete, check:

1. Is the logic in the right submodule, with import-time side effects avoided?
2. Is user-facing behavior preserved or intentionally documented?
3. Are unsafe actions sent through execution policy with the correct tier?
4. Are external inputs bounded, escaped, redacted, and timeout-protected?
5. Do background resources shut down deterministically?
6. Are focused tests added or updated under tests/interactive_shell/?
7. If SLASH_COMMANDS changed, does slash_catalog.py include every command (REPL and cli_parity)? Run test_slash_catalog.py.