tracer-cloud--opensre/docs/interactive-shell-action-policy.md

Interactive Shell Action Policy (ADR)

Status

Superseded — Jun 18, 2026. The declarative-rule-pack deterministic mapper and the regex-based planner postprocessing overrides described in the original decision have been removed. See "Decision (current): LLM is the sole tool selector" below. The original decision is retained for historical context.

Context

The interactive-shell action policy had grown through layered heuristics in single modules: a regex/keyword deterministic mapper inferred tools from free-form text, and planner postprocessing rewrote the model's chosen actions with more regex. These heuristics competed with the LLM and caused misclassifications (e.g. "investigate a sample test alert?" being treated as an informational question instead of running the sample alert), and they were a recurring source of precedence drift.

Decision (current): The shell action agent is the sole tool selector

1. There is no regex/keyword intent inference. Non-command turns are selected entirely by the shell action agent via native tool-calling.
2. Tool selection is driven by the action-agent system prompt (core/agent_harness/prompts/action_agent_prompt.py) and the per-tool descriptions in the tool catalog (tools/interactive_shell/*). Keep both precise — they are the only selection signal.
3. The action path does not post-hoc rewrite the model's tool calls. Tool calls execute as first-class AgentTools through the shared core tool-calling loop; argument shape and availability are enforced by the AgentTool runtime contract and per-tool gates.
4. When the action-agent prompt overflows the context window, the turn falls through to a conversational reply rather than guessing an action. When the action-agent LLM itself is unavailable, the REPL renders and persists a failed assistant turn so /resume can show the outage.
5. Literal /slash command text the user types verbatim is dispatched deterministically, without the action-agent LLM (see the "Deterministic literal-/slash dispatch" addendum below). This is an explicit-command bypass, not natural-language intent inference: free-form text is still selected entirely by the action agent. The runtime's literal-/slash detection in runtime/utils/input_policy._literal_slash_command_text remains terminal-UI policy (spinner suppression and exclusive-stdin gating); the execution-side deterministic dispatch lives in core/agent_harness/turns/action_driver.py.

What this means for changes

• To change how a phrasing maps to a tool, edit the action-agent system prompt and/or the relevant tool description — never add a regex.
• To add a new tool, add it to the tool catalog with a clear, self-describing description and input_schema; the action agent selects it from that text and receives it as an AgentTool.
• Live turn scenarios under tests/core/agent/scenarios/ are the regression surface for action-agent behavior. Deterministic scenarios (intent_class: deterministic) assert literal command dispatch only.

Original decision (historical, superseded)

1. Deterministic mapping was split into declarative rule packs with one explicit precedence table.
2. Rule matching windows were named typed strategies instead of inline numeric slices.
3. Planner postprocessing ran as pure transforms over a typed PlannerState.
4. Fail-closed policy transforms and normalization transforms were registered separately and executed in one ordered list.
5. Legacy planner-result tuple compatibility was collapsed behind a single adapter.
6. Planner contracts included policy-trace artifacts to detect silent precedence drift.

Integration awareness and LLM-driven read-only discovery

Addendum — Jun 18, 2026.

Factual questions about live state (for example "is sentry installed?") are answered without adding keyword/regex rules. Two complementary mechanisms:

1. Context grounding (not action planning). At REPL boot, run_repl_async (surfaces/interactive_shell/main.py) hydrates session.configured_integrations from the shared configured_integration_services() helper in integrations/catalog.py (the same source the welcome banner uses, so they never diverge). The chat assistant prompt (build_environment_block in core/agent_harness/prompts/assistant.py) lists the configured set as facts, letting the model answer directly when state is already known.
2. LLM-driven discovery. The action-agent system prompt (core/agent_harness/prompts/action_agent_prompt.py) lets the model, at its own discretion, emit a read-only discovery action (for example slash_invoke("/integrations", ["list"]) or ["verify"]) to discover the answer instead of deflecting. There is no keyword mapping for this — the LLM decides. Under the alpha allow-all policy every discovery action runs without confirmation (execution_policy.allow_tool("slash") returns allow); the former ExecutionTier/resolve_slash_execution_tier classification was removed because it gated nothing. No fail-closed regex rule is involved; the action agent decides whether to emit a discovery action.

Observe→answer summary loop

Addendum — Jun 18, 2026.

When the action agent runs a read-only discovery command to answer a question (e.g. the user asks "is sentry installed?" and the model runs /integrations), the raw command output (a verification table) is not a direct answer on its own. The pipeline now follows up with a short assistant pass that summarizes that output:

1. Read-only discovery slash commands stash a compact text view of what they found on session.agent.last_observation (_record_integrations_observation in surfaces/interactive_shell/command_registry/integrations.py).
2. run_agent_prompt resets that field at the start of every action-agent turn and, when a discovery command produced an observation and succeeded, calls the conversational assistant with tool_observation=... (inside the handled-turn observation branch in pipeline.py). The assistant summarizes the output into a direct answer and is instructed not to emit further actions.

This only fires when the action-agent tool path executes a read-only discovery command and records an observation. The pipeline no longer has a pre-agent deterministic dispatch branch.

Discovery commands also no longer dump validator stack traces into the REPL: a vendor/config failure during verification (for example a GitHub MCP 401) is logged as a one-line warning instead of a full traceback, because report_validation_failure now defaults to include_traceback=False while still capturing the exception to Sentry.

Auto-launching interactive setup ("can you configure X?")

Addendum — Jun 18, 2026.

When the user asks to configure, connect, set up, or add an integration ("can you configure sentry?", "connect datadog"), the action agent does not just hand off to the conversational assistant — it launches the setup wizard for them. The action agent emits a slash_invoke tool call for /integrations setup <service> or /mcp connect <server>. The model chooses the service; there is no per-vendor hardcoding.

The setup wizard is a child process that needs exclusive stdin, so it cannot run inline mid-turn (the live prompt is competing for stdin). Instead tools/interactive_shell/actions/slash.py queues the command via session.queue_auto_command(...), which prefills the next prompt and marks it for auto-submit. The prompt refresh hook (wire_prompt_refresh in surfaces/interactive_shell/ui/input_prompt/refresh.py) then submits it, so the command flows through the normal exclusive-stdin turn path of the REPL (turn_needs_exclusive_stdin recognizes /integrations setup) — the only place an interactive child process gets clean stdin. In a non-TTY/scripted context (no prompt to submit into), the slash command path degrades to normal non-interactive slash behavior.

Removal of the planning-stage fail-closed safeguard (v0.1)

Addendum — Jun 18, 2026.

The action agent does not deny a turn. Previously, any clause the old planner could not map to an executable tool — flagged via the mark_unhandled tool, an UNHANDLED: text marker, or an unavailable tool call — collapsed the whole turn into a hard denial that printed "I couldn't safely decide actions for that request." In practice this fired on legitimate input (most often a conversational question that embedded a quoted, list-style directive such as figure out why X is crashing by querying (a) sentry, (b) github, (c) posthog), producing a dead end with no safety benefit.

Every terminal action in v0.1 is read-only, so an unmatched, ambiguous, or chatty clause is not a safety risk. The action agent now:

• runs every clause it can map to an executable action, and
• lets everything else fall through to the conversational assistant (or simply drops a chatty clause in a compound request).

Removed as part of this change: the denied field on ActionPlanningDecision, enforce_plan_fail_closed_policy, normalize_terminal_plan, render_plan_denied, the mark_unhandled planner tool, and the UNHANDLED: convention. The fail_closed, has_unhandled_clause, and turn.expected_signals fields were also removed from turn scenario fixtures, since the oracle never asserted on them; the fixture policy block now carries a single executes_terminal_action boolean (true only when a shell action AgentTool is expected to run).

If write/mutating actions are introduced later, gate them with the execution-stage confirmation policy (tools/interactive_shell/shared/execution_policy.py), not an action-selection denial.

Removal of the shell-command safety policy (alpha)

Addendum — Jun 27, 2026.

Decision: while OpenSRE is in alpha, the interactive REPL runs every shell command with no guardrails. The shell-command safety policy — the read-only / mutating / restricted classification, the command allowlist, and the hard deny floor — has been removed. This is a deliberate trade-off: alpha prioritizes developer velocity over command sandboxing, and the REPL already runs on the developer's own machine with their own privileges.

What changed:

• shell_policy.py (classification, allowlists, classify_command, evaluate_policy, PolicyDecision) was deleted. The pure parsing helpers it also contained moved to tools/shell/parsing.py (parse_shell_command, argv_for_repl_builtin_detection, ParsedShellCommand), alongside the shell execution policy in tools/shell/policy.py.
• tools.shell.policy.evaluate_shell_from_parsed now returns allow for every command — read-only, mutating, restricted (sudo, systemctl, kill, dd, …), shell operators (| && ; > <), and command substitution (`/$(...)). Commands that need a shell run through one automatically; the ! prefix is still honored but no longer required to escape the old operator block.
• The only remaining non-execution outcome is genuinely empty input (a bare ! or whitespace), which is rejected as input validation, not as a guardrail.

The ask/confirmation machinery (trust_mode plus the confirmation UX) is retained as an unused hook, split across two layers: the pure decision lives in tools/interactive_shell/shared/execution_policy.py (resolve_confirmation), and the terminal interaction (execution_allowed — console output, the Proceed? [Y/n] prompt, analytics) lives in surfaces/interactive_shell/ui/execution_confirm.py. If command guardrails are reintroduced after alpha, gate them here at the execution stage — never with an action-selection denial in the planner.

Deterministic literal-/slash dispatch (no LLM)

Addendum — Jun 28, 2026.

Decision: input the user types as a literal /slash command is dispatched deterministically, without consulting the action-agent LLM. This supersedes the earlier "the literal-/slash detection must never become an action execution shortcut" wording.

Why: all REPL turns previously routed through the action-agent LLM, so when that LLM was unavailable (a provider with no credit, a failed auth, an outage) every slash command failed — including the exact commands needed to recover (/login, /auth, /onboard, /model). That is a deadlock: you could not log in because logging in required the LLM you were trying to fix. Typed commands should not depend on a funded LLM.

Scope — the line that keeps the original concern intact. The original ADR removed regex/keyword heuristics because they inferred intent from natural language and competed with the LLM. This bypass does the opposite: it fires only when the message text itself is a literal /command the user typed verbatim. There is no inference. Free-form natural language ("log me in", "show my integrations") is still selected entirely by the action agent. The line is "explicit typed command" vs "natural language", identical to the line the terminal-UI policy (_literal_slash_command_text) already draws.

How it works. core/agent_harness/turns/action_driver.run_action_agent_turn recognizes literal /slash input and emits a deterministic slash_invoke tool call through the same static-LLM path as the explicit !cmd shell escape (_StaticToolCallLLM). Execution then flows through the normal slash_invoke AgentTool → dispatch_slash, so recording, execution policy, exclusive-stdin gating, pickers, and exit behavior are identical to an LLM-selected slash call — the only difference is the tool selection is deterministic instead of LLM-driven. The bypass no-ops (falls back to the LLM path) when slash_invoke is not an available tool that turn.

Consequences.

• Literal slash commands are faster, free, and reliable even with no LLM credit.
• Compound requests that start with a literal slash are dispatched as that one command (a single slash_invoke); compound phrasing that does not start with a slash (e.g. run /health and then investigate …) is unaffected and still LLM-routed. No turn scenario uses a prompt beginning with a literal /.
• A literal discovery command (e.g. /integrations list) shows its raw output directly; the optional LLM summary pass only runs if the action-agent LLM is available, and degrades cleanly (no summary) when it is not.

Still forbidden: regex/keyword/fuzzy intent routing for natural language, post-hoc rewriting of LLM-selected tool calls, and any deterministic mapping from non-/-prefixed text to an action. Those compete with the LLM and were removed for good reason.