chore: import upstream snapshot with attribution
CI (OpenClaw E2E) / openclaw test (push) Has been cancelled
CI / coverage-report (push) Has been cancelled
CI / test-kubernetes (push) Has been cancelled
CI / should-run-thorough (push) Has been cancelled
CI / test-thorough (cloudwatch-demo) (push) Has been cancelled
CI / test-thorough (flink-ecs) (push) Has been cancelled
CI / test-thorough (upstream-lambda) (push) Has been cancelled
CI / test-thorough (prefect-ecs-fargate) (push) Has been cancelled
Release / build-binaries (zip, opensre.exe, onefile, windows-latest, windows-x64) (push) Has been cancelled
Benchmark image — build + push to ECR (any adapter) / build + push (push) Has been cancelled
CI / quality (ubuntu-latest) (push) Has been cancelled
CI / test (tools-runtime) (push) Has been cancelled
CI / test (e2e-general) (push) Has been cancelled
CI / test (cli-runtime) (push) Has been cancelled
CI / test (e2e-provider-and-openclaw) (push) Has been cancelled
CI / test (integrations-and-misc) (push) Has been cancelled
Release / verify (push) Has been cancelled
Release / build-python-dist (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, macos-15-intel, darwin-x64) (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, macos-latest, darwin-arm64) (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, ubuntu-22.04, linux-x64) (push) Has been cancelled
Release / publish-release (push) Has been cancelled
Release / publish-main-release (push) Has been cancelled
Interactive Shell Live (PR + post-merge) / turn-checks (no-LLM) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Interactive Shell Live (PR + post-merge) / turn-live shard ${{ matrix.shard_index }} (push) Has been cancelled
Release / prepare (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, ubuntu-22.04-arm, linux-arm64) (push) Has been cancelled
Synthetic Deterministic Tests / Synthetic offline (deterministic) (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 13:10:45 +08:00
commit 4b6817381b
3933 changed files with 525247 additions and 0 deletions
@@ -0,0 +1,252 @@
# Runtime package rules
## Human summary
The `runtime` package holds the focused support modules for the interactive
shell runtime. The top-level bootstrap and controller live one level up in
`interactive_shell/`.
In simple terms:
- `../main.py` starts the interactive session and handles process/bootstrap gates.
- `startup/first_launch_github.py` owns the first-launch GitHub sign-in gate.
- `../controller.py` owns the `InteractiveShellController` orchestration class,
including prompt input, submitted prompt handling, queued turn consumption,
prompt-mediated confirmation waits, one-turn pipeline handoff, background output draining, and
shutdown.
- `core/prompt_manager.py` owns prompt-toolkit setup and prompt rendering.
- `input/` owns prompt input event conversion: EOF, Ctrl-C, CPR cleanup, and
resume hints.
- `utils/input_policy.py` owns prompt stdin/spinner decisions for turns.
- `startup/initial_input.py` owns non-interactive initial-input replay.
- `background/workers.py` owns alert watching, spinner ticking, sampler startup,
and turn-start background output drains.
- `background/` also owns background investigation records, launchers, and
completion notification delivery.
- `core/` holds the core runtime engine:
- `state.py` — shared runtime state (`ReplState`, `SpinnerState`)
- `turn_detection.py` — pure text classifiers for cancel, confirm, and correction detection
- `core.agent_harness.session.tasks` owns the cross-session task registry surfaced via
`/tasks` and `/cancel`.
- Reusable per-agent session state (`Session`) lives in
`core.agent_harness.session`. Terminal runtime context assembly
(`ReplRuntimeContext`, `create_repl_runtime_context`) lives in
`interactive_shell.runtime.context`.
These instructions apply to `interactive_shell/runtime/` and all
subdirectories. Parent `AGENTS.md` files still apply.
## Architectural intent (locked)
The runtime package is intentionally split into focused concerns:
- `core/state.py` — runtime state and transition helpers only.
- `core/turn_detection.py` — pure prompt text classification only.
- `utils/input_policy.py` — terminal stdin/spinner gating decisions only.
- `agent_presentation.py` — terminal presentation for the agent prompt only (agent
lifecycle events, presentation-state reducer/renderer, `ConsoleAgentEventSink`).
- `turn_host.py` — terminal/runtime host for `ShellAgent` prompts only.
- `../controller.py` — stable async entrypoint and async prompt runtime/event loop
orchestration, submitted prompt handling, queued-turn consumption,
prompt-mediated confirmation waits, turn telemetry, one-turn pipeline
handoff, background output draining, and shutdown only.
- `core/prompt_manager.py` — prompt-toolkit setup and prompt rendering only.
- `input/` — prompt input event conversion and terminal-input cleanup only.
- `background/workers.py` — background worker startup and turn-start drain hooks
only.
- `core.agent_harness.session.background` — background investigation record and
preferences only.
- `background/runner.py` — session-local background investigation launchers only.
- `background/notifications.py` — background RCA completion notification delivery only.
- `../main.py` — process/bootstrap boundary only.
- `startup/initial_input.py` — scripted initial-input replay only.
- `startup/first_launch_github.py` — first-launch GitHub sign-in gate only.
- `core.agent_harness.session.tasks` — task registry + persistence only.
- `core.agent_harness.accounting.token_accounting` — session-scoped LLM token accounting and run metadata only.
Keep these boundaries strict. If a change crosses concerns, move code to the
owner module instead of broadening module responsibilities.
## Data flow contract (locked)
The interactive runtime must keep this shape:
1. `interactive_shell.main.run_repl` (synchronous entrypoint) sets up
process-level concerns and calls `run_repl_async`.
2. `interactive_shell.main.run_repl_async` (async body) creates `InteractiveShellController`.
3. `InteractiveShellController.start_interactive_shell` owns prompt lifecycle,
submitted input handling, queued-turn consumption, and per-turn task
scheduling.
4. `runtime.turn_host.run_agent_turn_queue` consumes queued prompts through an
injected `run_turn` callable, which in production is
`lambda text: run_agent_turn(self.turn_runtime, text)`.
5. `runtime.turn_host.run_agent_turn(runtime, text)` drives one submitted turn.
`AgentTurnRuntime` is the immutable dependency bundle it operates on
(`session`, `state`, `spinner`, `invalidate_prompt`, `request_exit`);
`run_agent_turn` owns presentation setup, prompt-mediated confirmation,
dispatch state, and per-turn execution.
6. `interactive_shell.runtime.shell_turn_execution.execute_shell_turn` binds shell adapters
around `core.agent_harness.turns.orchestrator.run_turn`.
7. `core.agent_harness` owns one prompt's action/answer mechanics and accounting
finalization. The terminal presentation for `AgentEvent` emissions lives in
`runtime/agent_presentation.py`.
Do not invert this dependency direction.
### Architecture diagram
```mermaid
flowchart TD
runRepl["interactive_shell.main.run_repl"] --> replMain["interactive_shell.main.run_repl_async"]
replMain --> controller["interactive_shell.controller.InteractiveShellController"]
controller --> turnHost["runtime.turn_host.run_agent_turn(turn_runtime, text)"]
turnHost --> turnEntry["interactive_shell.runtime.shell_turn_execution.execute_shell_turn"]
turnEntry --> coreHarness["core.agent_harness.turns.orchestrator.run_turn"]
coreHarness --> sideEffects["slash/help/agent/follow-up/investigation side effects"]
controller --> replState["core.state.ReplState"]
controller --> spinnerState["core.state.SpinnerState"]
controller --> inputReader["input.PromptInputReader"]
```
## State ownership rules
- `ReplState` is the single source of truth for:
- active dispatch task
- cancellation event
- confirmation event/response lifecycle
- exit requests
- the explicit turn `phase` (`TurnPhase`: `IDLE`, `DISPATCHING`,
`AWAITING_CONFIRMATION`, `CANCELLING`)
- Mutate turn state through the `ReplState` transition methods, never by poking
raw fields from other modules:
- `start_dispatch` / `attach_turn_task` / `attach_cancel_event` -> `DISPATCHING`
- `begin_confirmation` -> `AWAITING_CONFIRMATION`; `clear_confirmation` returns
to `DISPATCHING`/`IDLE` (and never clobbers an in-progress `CANCELLING`)
- `cancel_current_dispatch` -> `CANCELLING` (only when there is something to
cancel) then signals the cancel/confirm events and `task.cancel`
- `finish_dispatch` / `clear_current_task` -> `IDLE`
- `phase` is authoritative for confirmation and cancelling. `is_dispatch_running()`
stays derived from the asyncio task (the runtime truth of the in-flight turn);
`is_awaiting_confirmation()` and `is_cancelling()` are derived from `phase`.
- Do not reorder the signaling inside `cancel_current_dispatch` or move the
`confirm_response` reset after the `confirm_event` publish in
`begin_confirmation`; both orderings are load-bearing for cancellation and
confirmation race-safety.
- `SpinnerState` owns spinner rendering state only; it must not depend on
runtime task management.
## Turn execution rules
- Do not reintroduce `dispatch.py`, an `AgentTurnRunner`-style wrapper class, or
any compatibility-only forwarding module.
- The terminal host lives in `runtime/turn_host.py`: `run_agent_turn(runtime,
text)` owns shell presentation (StreamingConsole, spinner, recorder, progress
scope), constructs a `ConsoleAgentEventSink`, owns dispatch state, and
lazily imports + calls
`interactive_shell.runtime.shell_turn_execution.execute_shell_turn`.
`AgentTurnRuntime` is the immutable dependency bundle it operates on; the
controller constructs it and passes a bound `run_agent_turn` coroutine into
`run_agent_turn_queue`.
- The shell adapter entry lives in `runtime/shell_turn_execution.py`: `execute_shell_turn`
composes the action-turn (`runtime/action_turn.py`), gather (`runtime/integration_tool_gathering.py`),
and answer (`runtime/answer_turn.py`) adapters plus accounting around
`core.agent_harness.turns.orchestrator.run_turn`. Each adapter owns its own
binding; tests import them from their owning module (not `shell_turn_execution`).
- The reusable per-prompt loop lives in `core.agent_harness`: turn snapshots,
observation reset, action/response routing, and core result construction stay
surface-agnostic.
- Keep terminal side effects (spinner, prompt suppression, `console.print`, CPR
drain) in `ConsoleAgentEventSink` — defined in `runtime/agent_presentation.py`
— not in the turn-entry adapter or the core harness.
- Put cancel/confirm/correction text classifiers in `core/turn_detection.py`.
- Put stdin blocking and spinner decisions in `utils/input_policy.py`.
- Keep prompt-mediated confirmation waiting in `runtime/core/confirmation.py`.
- Turn accounting is consolidated behind `ShellTurnAccounting` in
`interactive_shell/runtime/core/turn_accounting.py`, invoked from
`interactive_shell.runtime.shell_turn_execution.execute_shell_turn`. It owns action-agent analytics, terminal-turn aggregate
telemetry, prompt-recorder flush, conversational-turn persistence, and the final
assistant-intent stamp. `interactive_shell.runtime.action_turn.run_action_tool_turn` returns facts
only (`ToolCallingTurnResult` with `accounting_status` of `completed` / `not_run`)
and emits no analytics itself. Do not re-scatter accounting back into
`run_action_tool_turn` or standalone `_record_*` helpers.
## Controller rules
- `../controller.py` owns:
- `InteractiveShellController`
- `start_interactive_shell` shell lifecycle orchestration
- alert listener setup/teardown
- `AgentTurnRuntime` construction and shutdown
- queued prompt submission
- `turn_host.py` owns:
- `run_input_loop` (module-level) — read and handle user input until exit
- `run_agent_turn_queue` (module-level) — consume queued prompts until exit (runs an injected `run_turn`)
- `run_agent_turn(runtime, text)` (module-level) — presentation, dispatch state, prompt-mediated confirmation, and turn-entry invocation over an `AgentTurnRuntime`
- `ConsoleAgentEventSink` (in `runtime/agent_presentation.py`) — terminal presentation for agent lifecycle events over `_reduce_agent_presentation` / `_render_agent_presentation_transition`
- queued turn consumption
- per-turn task lifecycle
- dispatch start/finish state transitions
- prompt-mediated confirmation waiting
- turn telemetry and `execute_shell_turn` invocation
- current turn cancellation helpers
- coordination between prompt, background, and shutdown helpers
- `core/prompt_manager.py` owns:
- prompt-toolkit wiring
- prompt rendering callbacks
- pending prompt defaults and autosubmit handling
- `input/` owns:
- prompt input event types
- terminal EOF and Ctrl-C conversion
- CPR cleanup for submitted prompt text
- session resume hints when prompt input closes
- `background/workers.py` owns:
- alert watcher lifecycle
- spinner ticker lifecycle
- sampler startup
- background notice drains at turn start
- Keep prompt rendering concerns in runtime/prompting modules, not in
dispatch/execution.
## Main/bootstrap rules
- `../main.py` owns:
- startup sweep
- TTY/non-TTY gate
- banner display for interactive runs
- async boundary (`asyncio.run`)
- `../controller.py` owns alert listener setup/teardown because the inbox is
part of the running shell lifecycle.
- Do not move per-turn dispatch/runtime logic back into startup bootstrap.
## Compatibility surface policy
- `runtime/__init__.py` should be a thin export layer.
- Do not duplicate business logic in `__init__.py`.
- `runtime/__init__.py` exports `Session` from `core.agent_harness.session` and
runtime-context helpers from `interactive_shell.runtime.context`. New shared
runtime code should import session names directly from `core.agent_harness.session`.
- Do not re-add `_xxx` underscore aliases or wrapper functions for
compatibility. Tests and callers should import canonical names from their
owning submodule.
## Test seam policy
- Prefer patching canonical module seams:
- `interactive_shell.controller.*` for prompt-loop, queued-turn, confirmation behavior,
one-turn pipeline execution, and side effects
- `interactive_shell.main.*` for process/bootstrap behavior
- `runtime.core.state.*` for state-specific behavior
- Avoid adding new tests that monkeypatch package-root internals in
`runtime.__init__` unless there is no stable canonical seam.
## Refactor guardrails
- No behavior changes to action-planning policy should be introduced from
`runtime/` refactors.
- Keep interruption semantics unchanged:
- Esc or bare cancel commands interrupt active dispatch
- cancellation moves the turn to `TurnPhase.CANCELLING` and signals both the
cancel event and any pending confirmation event before `task.cancel`
- confirmation prompts are cancel-safe and never silently auto-confirm; a
cancel during confirmation must not be downgraded back to `DISPATCHING`
- Preserve observability semantics (turn telemetry and turn summaries).
@@ -0,0 +1,29 @@
from __future__ import annotations
from platform.common.task_registry import TaskRegistry
from platform.common.task_types import TaskKind, TaskRecord, TaskStatus
from surfaces.interactive_shell.runtime.context import (
ReplRuntimeContext,
SessionBootstrapSpec,
create_repl_runtime_context,
prepare_repl_session,
)
from surfaces.interactive_shell.session.background_investigations import (
BackgroundInvestigationRecord,
BackgroundNotificationPreferences,
)
from surfaces.interactive_shell.session.session import Session
__all__ = [
"BackgroundInvestigationRecord",
"BackgroundNotificationPreferences",
"ReplRuntimeContext",
"Session",
"SessionBootstrapSpec",
"TaskKind",
"TaskRecord",
"TaskRegistry",
"TaskStatus",
"create_repl_runtime_context",
"prepare_repl_session",
]
@@ -0,0 +1,113 @@
"""Shell adapter for one action-selection turn.
Binds the interactive shell's console, session, and default providers around
core ``run_action_agent_turn``.
"""
from __future__ import annotations
from collections.abc import Callable
from rich.console import Console
from core.agent_harness.error_reporting import DefaultErrorReporter
from core.agent_harness.llm_resolution import default_llm_factory
from core.agent_harness.ports import OutputSink
from core.agent_harness.tools.tool_provider import DefaultToolProvider
from core.agent_harness.turns.action_driver import ToolCallingDeps, run_action_agent_turn
from core.agent_harness.turns.turn_plan import TurnPlan
from core.agent_harness.turns.turn_results import ToolCallingTurnResult
from core.execution import ToolExecutionHooks
from surfaces.interactive_shell.command_registry import SLASH_COMMANDS
from surfaces.interactive_shell.command_registry.suggestions import resolve_literal_slash_typo
from surfaces.interactive_shell.runtime.agent_harness_adapters import resolve_output_sink
from surfaces.interactive_shell.runtime.subprocess_runner.repl_presenter import (
ReplSubprocessPresenter,
)
from surfaces.interactive_shell.session import Session
from surfaces.interactive_shell.ui.action_rendering import ActionRenderObserver
def _complete_literal_slash_typo_turn(
message: str,
session: Session,
output: OutputSink,
) -> ToolCallingTurnResult | None:
"""Handle unknown slash roots and invalid subcommands before tool validation."""
typo = resolve_literal_slash_typo(message, SLASH_COMMANDS)
if typo is None:
return None
output.print()
output.print(typo.message)
session.record(
"slash",
message.strip(),
ok=False,
response_text=typo.message,
slash_outcome=typo.outcome,
)
return ToolCallingTurnResult(0, 1, 0, False, True, response_text=typo.message)
def _subprocess_presenter_factory(
session: Session,
console: Console,
confirm_fn: Callable[[str], str] | None,
is_tty: bool | None,
action_already_listed: bool,
) -> ReplSubprocessPresenter:
return ReplSubprocessPresenter(
session,
console,
confirm_fn=confirm_fn,
is_tty=is_tty,
action_already_listed=action_already_listed,
)
def run_action_tool_turn(
message: str,
session: Session,
console: Console,
*,
confirm_fn: Callable[[str], str] | None = None,
is_tty: bool | None = None,
request_exit: Callable[[], None] | None = None,
deps: ToolCallingDeps | None = None,
turn_plan: TurnPlan | None = None,
output: OutputSink | None = None,
tool_hooks: ToolExecutionHooks | None = None,
) -> ToolCallingTurnResult:
"""Run one action-selection turn through core with shell adapters bound."""
resolved_output = resolve_output_sink(console, output)
typo_result = _complete_literal_slash_typo_turn(message, session, resolved_output)
if typo_result is not None:
return typo_result
effective_deps = (
deps
if deps is not None and deps.llm_factory is not None
else ToolCallingDeps(llm_factory=default_llm_factory)
)
return run_action_agent_turn(
message,
session,
output=resolved_output,
tools=DefaultToolProvider(
session,
console,
request_exit=request_exit,
observer_factory=lambda msg: ActionRenderObserver(
session=session, console=console, message=msg
),
subprocess_presenter_factory=_subprocess_presenter_factory,
),
confirm_fn=confirm_fn,
is_tty=is_tty,
deps=effective_deps,
turn_plan=turn_plan,
error_reporter=DefaultErrorReporter(),
tool_hooks=tool_hooks,
)
__all__ = ["run_action_tool_turn"]
@@ -0,0 +1,66 @@
"""Interactive-shell output adapter implementing :mod:`core.agent_harness.ports`.
This module owns terminal rendering only. Shared action-tool, reasoning-client,
run-record, and error-reporting providers live in :mod:`core.agent_harness`.
"""
from __future__ import annotations
from collections.abc import Iterable
from rich.console import Console
from rich.markup import escape
from core.agent_harness.ports import OutputSink
from core.llm.shared.llm_retry import CREDIT_EXHAUSTED_MARKER
from surfaces.interactive_shell.ui import (
stream_to_console,
)
from surfaces.interactive_shell.ui.streaming import render_response_header
class ShellOutputSink:
""":class:`core.agent_harness.ports.OutputSink` over a Rich console."""
def __init__(self, console: Console) -> None:
self._console = console
def print(self, message: str = "") -> None:
self._console.print(message)
def render_response_header(self, label: str) -> None:
render_response_header(self._console, label)
def render_error(self, message: str) -> None:
self._console.print(f"[yellow]{escape(message)}[/]")
# On a credit/billing wall, add the in-tool recovery hint.
if CREDIT_EXHAUSTED_MARKER in message:
self._console.print("[dim]Run /model to switch to another provider.[/]")
self._console.print(
"[dim]Or run /auth login <provider> to re-authenticate "
"or add a different provider.[/]"
)
def stream(
self,
*,
label: str,
chunks: Iterable[str],
suppress_if_starts_with: str | None = None,
) -> str:
return stream_to_console(
self._console,
label=label,
chunks=iter(chunks),
suppress_if_starts_with=suppress_if_starts_with,
)
def resolve_output_sink(console: Console, output: OutputSink | None) -> OutputSink:
"""Return the caller's sink, or a shell sink bound to ``console``."""
if output is not None:
return output
return ShellOutputSink(console)
__all__ = ["ShellOutputSink", "resolve_output_sink"]
@@ -0,0 +1,151 @@
"""Terminal presentation for the interactive shell agent prompt.
This module owns the **UI / presentation** side of one submitted shell prompt:
the pure presentation-state reducer, the effectful terminal transition renderer,
and the ``ConsoleAgentEventSink`` imperative shell that wires them together.
Keeping this separate from ``runtime/shell_turn_execution.py`` isolates spinner
lifecycle, prompt suppression, interruption/error messages, and stale CPR
draining from the turn's action-routing and prompt-construction logic.
"""
from __future__ import annotations
import asyncio
from collections.abc import Awaitable, Callable
from dataclasses import dataclass
from typing import Literal
from rich.markup import escape
from surfaces.interactive_shell.runtime.core.state import SpinnerState
from surfaces.interactive_shell.runtime.utils.input_policy import turn_should_show_spinner
from surfaces.interactive_shell.session import Session
from surfaces.interactive_shell.ui import (
DIM,
ERROR,
WARNING,
)
from surfaces.interactive_shell.ui.components.cpr_stdin import drain_stale_cpr_bytes
from surfaces.interactive_shell.ui.streaming.console import StreamingConsole
@dataclass(frozen=True)
class AgentEvent:
"""Agent lifecycle event emitted during one submitted shell turn."""
type: Literal["turn_start", "turn_interrupted", "turn_error", "turn_end"]
text: str | None = None
error: Exception | None = None
AgentEventSink = Callable[[AgentEvent], Awaitable[None]]
@dataclass(frozen=True)
class AgentPresentationState:
"""Immutable presentation state evolved across lifecycle events."""
show_spinner: bool = False
prompt_suppressed: bool = False
def _reduce_agent_presentation(
state: AgentPresentationState,
event: AgentEvent,
*,
should_show_spinner: bool,
) -> AgentPresentationState:
"""Compute the next presentation state for *event* (pure)."""
if event.type == "turn_start":
return AgentPresentationState(
show_spinner=should_show_spinner,
prompt_suppressed=should_show_spinner,
)
if event.type == "turn_end":
return AgentPresentationState()
if event.type in {"turn_interrupted", "turn_error"}:
return state
raise ValueError(f"Unknown agent event type: {event.type!r}")
async def _render_agent_presentation_transition(
*,
previous: AgentPresentationState,
current: AgentPresentationState,
event: AgentEvent,
console: StreamingConsole,
spinner: SpinnerState,
) -> None:
"""Perform the terminal side effects for one presentation transition."""
match event.type:
case "turn_start":
if current.show_spinner:
spinner.start()
case "turn_interrupted":
console.print(f"[{WARNING}]· interrupted[/]")
case "turn_error":
exc = event.error
if exc is None:
raise ValueError("turn_error event requires an error")
console.print(f"[{ERROR}]turn error:[/] {escape(str(exc))}")
# On a credit/billing wall, add the in-tool recovery hint.
from core.llm.shared.llm_retry import LLMCreditExhaustedError
if isinstance(exc, LLMCreditExhaustedError):
console.print(f"[{DIM}]Run /model to switch to another provider.[/]")
console.print(
f"[{DIM}]Or run /auth login <provider> to re-authenticate "
f"or add a different provider.[/]"
)
case "turn_end":
if previous.show_spinner:
spinner.stop()
await asyncio.sleep(0.05)
drain_stale_cpr_bytes()
case _:
raise ValueError(f"Unknown agent event type: {event.type!r}")
class ConsoleAgentEventSink:
"""Render agent lifecycle events to the terminal console.
Imperative shell: it holds the evolving ``AgentPresentationState`` and routes
each event through the pure ``_reduce_agent_presentation`` reducer and the
effectful ``_render_agent_presentation_transition`` renderer.
"""
def __init__(
self,
*,
session: Session,
spinner: SpinnerState,
console: StreamingConsole,
) -> None:
self.session = session
self.spinner = spinner
self.console = console
self.state = AgentPresentationState()
async def __call__(self, event: AgentEvent) -> None:
previous = self.state
self.state = _reduce_agent_presentation(
previous,
event,
should_show_spinner=turn_should_show_spinner(event.text or "", self.session),
)
await _render_agent_presentation_transition(
previous=previous,
current=self.state,
event=event,
console=self.console,
spinner=self.spinner,
)
__all__ = [
"AgentEvent",
"AgentPresentationState",
"ConsoleAgentEventSink",
"AgentEventSink",
]
@@ -0,0 +1,63 @@
"""Shell adapter for one conversational answer turn.
Binds the interactive shell's Rich output, grounding caches, reasoning client,
and telemetry around core ``stream_answer``.
"""
from __future__ import annotations
from collections.abc import Callable
from rich.console import Console
from core.agent_harness.accounting.run_record import DefaultRunRecordFactory
from core.agent_harness.error_reporting import DefaultErrorReporter
from core.agent_harness.ports import OutputSink
from core.agent_harness.turns.default_reasoning_client import DefaultReasoningClientProvider
from core.agent_harness.turns.orchestrator import (
stream_answer as core_stream_answer,
)
from core.agent_harness.turns.turn_plan import TurnPlan
from surfaces.interactive_shell.grounding.cli_reference import shell_prompt_context_provider
from surfaces.interactive_shell.runtime.agent_harness_adapters import resolve_output_sink
from surfaces.interactive_shell.session import Session
from surfaces.interactive_shell.utils.telemetry import LlmRunInfo
def answer_shell_question(
message: str,
session: Session,
console: Console,
*,
confirm_fn: Callable[[str], str] | None = None,
is_tty: bool | None = None,
tool_observation: str | None = None,
tool_observation_on_screen: bool = True,
handoff_contents: tuple[str, ...] = (),
turn_plan: TurnPlan | None = None,
output: OutputSink | None = None,
) -> LlmRunInfo | None:
"""Answer one shell question through the grounded conversational assistant."""
resolved_output = resolve_output_sink(console, output)
return core_stream_answer(
message,
session,
resolved_output,
prompts=shell_prompt_context_provider(session),
reasoning=DefaultReasoningClientProvider(
output=resolved_output,
error_reporter=DefaultErrorReporter(),
session=session,
),
run_factory=DefaultRunRecordFactory(session),
error_reporter=DefaultErrorReporter(),
confirm_fn=confirm_fn,
is_tty=is_tty,
tool_observation=tool_observation,
tool_observation_on_screen=tool_observation_on_screen,
handoff_contents=handoff_contents,
turn_plan=turn_plan,
)
__all__ = ["answer_shell_question"]
@@ -0,0 +1 @@
"""Background runtime support for interactive shell."""
@@ -0,0 +1,47 @@
"""Background RCA notification helpers."""
from __future__ import annotations
from surfaces.interactive_shell.session.background_investigations import (
BackgroundInvestigationRecord,
)
def deliver_background_notifications(
*,
record: BackgroundInvestigationRecord,
channels: tuple[str, ...],
) -> dict[str, str]:
"""Send configured notifications for a completed background RCA."""
# Imported lazily: email delivery only fires on background-RCA completion, so
# the SMTP client must not load into the base REPL boot import path.
from integrations.smtp.delivery import format_background_rca_email, send_smtp_report
results: dict[str, str] = {}
from integrations.catalog import resolve_effective_integrations
effective_integrations = resolve_effective_integrations()
for channel in channels:
if channel != "email":
results[channel] = "unsupported"
continue
smtp_integration = effective_integrations.get("smtp")
smtp_config = smtp_integration.get("config") if isinstance(smtp_integration, dict) else None
if not isinstance(smtp_config, dict):
results["email"] = "missing smtp integration"
continue
subject, body = format_background_rca_email(
task_id=record.task_id,
command=record.command,
root_cause=record.root_cause,
top_analysis=record.top_analysis,
next_steps=record.next_steps,
stats=record.stats,
)
ok, error = send_smtp_report(report=body, subject=subject, smtp_ctx=smtp_config)
results["email"] = "sent" if ok else f"failed: {error}"
return results
@@ -0,0 +1,236 @@
"""Helpers for launching session-local background investigations."""
from __future__ import annotations
import threading
from collections.abc import Callable
from contextlib import nullcontext
from typing import Any
from uuid import uuid4
from prompt_toolkit.patch_stdout import patch_stdout
from rich.console import Console
from rich.markup import escape
from platform.analytics.cli import track_investigation
from platform.analytics.source import EntrypointSource, TriggerMode
from platform.common.errors import OpenSREError
from surfaces.interactive_shell.runtime import (
BackgroundInvestigationRecord,
Session,
TaskKind,
)
from surfaces.interactive_shell.runtime.background.notifications import (
deliver_background_notifications,
)
from surfaces.interactive_shell.ui import DIM, ERROR, HIGHLIGHT, WARNING
from surfaces.interactive_shell.utils.error_handling.exception_reporting import report_exception
BackgroundRunFn = Callable[..., dict[str, Any]]
def _safe_console_print(console: Console, message: str) -> None:
isatty = getattr(console.file, "isatty", None)
stdout_context = patch_stdout(raw=True) if callable(isatty) and isatty() else nullcontext()
with stdout_context:
console.print(message)
def drain_background_notices(session: Session, console: Console) -> None:
"""Print queued background investigation status lines on the main REPL thread."""
for message in session.terminal.drain_background_notices():
_safe_console_print(console, message)
def _build_record(
*,
task_id: str,
command: str,
investigation_id: str,
) -> BackgroundInvestigationRecord:
return BackgroundInvestigationRecord(
task_id=task_id,
status="running",
command=command,
investigation_id=investigation_id,
)
def _top_analysis(final_state: dict[str, Any]) -> tuple[str, ...]:
claims = final_state.get("validated_claims", [])
if not isinstance(claims, list):
return ()
lines: list[str] = []
for entry in claims:
if not isinstance(entry, dict):
continue
claim = str(entry.get("claim") or "").strip()
if claim:
lines.append(claim)
if len(lines) >= 3:
break
return tuple(lines)
def _next_steps(final_state: dict[str, Any]) -> tuple[str, ...]:
steps = final_state.get("remediation_steps", [])
if not isinstance(steps, list):
return ()
values: list[str] = []
for step in steps[:3]:
text = str(step).strip()
if text:
values.append(text)
return tuple(values)
def _stats(final_state: dict[str, Any]) -> dict[str, Any]:
tool_calls = final_state.get("evidence_entries", [])
loops = final_state.get("investigation_loop_count", 0)
validity = final_state.get("validity_score", 0.0)
return {
"tool_call_count": len(tool_calls) if isinstance(tool_calls, list) else 0,
"investigation_loop_count": int(loops) if isinstance(loops, int | float) else 0,
"validity_score": float(validity) if isinstance(validity, int | float) else 0.0,
}
def _start_background_investigation(
*,
session: Session,
console: Console,
display_command: str,
run_fn: BackgroundRunFn,
kwargs: dict[str, Any],
investigation_target: str = "",
input_path: str | None = None,
) -> str:
investigation_id = str(uuid4())
session.last_investigation_id = investigation_id
task = session.task_registry.create(TaskKind.INVESTIGATION, command=display_command)
task.mark_running()
record = _build_record(
task_id=task.task_id,
command=display_command,
investigation_id=investigation_id,
)
session.terminal.background_investigations[task.task_id] = record
def _worker() -> None:
try:
with track_investigation(
entrypoint=EntrypointSource.CLI_REPL_FILE,
trigger_mode=TriggerMode.FILE,
input_path=input_path,
interactive=True,
investigation_id=investigation_id,
investigation_target=investigation_target or None,
session=session,
) as tracker:
final_state = run_fn(cancel_requested=task.cancel_requested, **kwargs)
tracker.record_loop_metrics_from_state(final_state)
root = str(final_state.get("root_cause") or "")
record.status = "completed"
record.root_cause = root
record.top_analysis = _top_analysis(final_state)
record.next_steps = _next_steps(final_state)
record.stats = _stats(final_state)
record.final_state = dict(final_state)
record.notification_results = deliver_background_notifications(
record=record,
channels=session.terminal.background_notification_preferences.channels,
)
task.mark_completed(result=root)
session.terminal.enqueue_background_notice(
f"[{HIGHLIGHT}]background investigation complete[/] "
f"[{DIM}]— task {escape(task.task_id)} ready; "
f"use[/] [{HIGHLIGHT}]/background show {escape(task.task_id)}[/]",
)
except KeyboardInterrupt:
record.status = "cancelled"
task.mark_cancelled()
session.terminal.enqueue_background_notice(
f"[{WARNING}]background investigation cancelled[/] "
f"[{DIM}]for task {escape(task.task_id)}.[/]",
)
except OpenSREError as exc:
record.status = "failed"
task.mark_failed(str(exc))
session.terminal.enqueue_background_notice(
f"[{ERROR}]background investigation failed[/] "
f"[{DIM}]for task {escape(task.task_id)}:[/] {escape(str(exc))}",
)
except Exception as exc: # noqa: BLE001
record.status = "failed"
task.mark_failed(str(exc))
report_exception(exc, context="surfaces.interactive_shell.background_investigation")
session.terminal.enqueue_background_notice(
f"[{ERROR}]background investigation failed[/] "
f"[{DIM}]for task {escape(task.task_id)}:[/] {escape(str(exc))}",
)
thread = threading.Thread(
target=_worker,
daemon=True,
name=f"background-investigation-{task.task_id}",
)
thread.start()
_safe_console_print(
console,
f"[{DIM}]background investigation started — task[/] [bold]{escape(task.task_id)}[/bold]. "
f"[{HIGHLIGHT}]/background list[/] [{DIM}]to monitor, "
f"[/][{HIGHLIGHT}]/cancel {escape(task.task_id)}[/] [{DIM}]to stop.[/]",
)
return task.task_id
def start_background_text_investigation(
*,
alert_text: str,
session: Session,
console: Console,
display_command: str = "background free-text investigation",
investigation_target: str = "",
) -> str:
from surfaces.interactive_shell.runtime.investigation_adapter import (
run_investigation_for_session_background,
)
return _start_background_investigation(
session=session,
console=console,
display_command=display_command,
run_fn=run_investigation_for_session_background,
kwargs={
"alert_text": alert_text,
"context_overrides": session.accumulated_context or None,
},
investigation_target=investigation_target,
input_path=display_command,
)
def start_background_template_investigation(
*,
template_name: str,
session: Session,
console: Console,
display_command: str,
investigation_target: str = "",
) -> str:
from surfaces.interactive_shell.runtime.investigation_adapter import (
run_sample_alert_for_session_background,
)
return _start_background_investigation(
session=session,
console=console,
display_command=display_command,
run_fn=run_sample_alert_for_session_background,
kwargs={
"template_name": template_name,
"context_overrides": session.accumulated_context or None,
},
investigation_target=investigation_target,
input_path=f"template:{template_name}",
)
@@ -0,0 +1,127 @@
"""Background task lifecycle for the interactive REPL runtime."""
from __future__ import annotations
import asyncio
import logging
from collections.abc import Callable, Coroutine
from typing import Any
from rich.console import Console
from core.domain.alerts import inbox as _alert_inbox
from surfaces.interactive_shell.runtime.background.runner import drain_background_notices
from surfaces.interactive_shell.runtime.core.state import ReplState, SpinnerState
from surfaces.interactive_shell.session import Session
from surfaces.interactive_shell.ui.alerts import drain_and_render_incoming
log = logging.getLogger(__name__)
class BackgroundTaskManager:
"""Start background workers and drain their user-visible output."""
def __init__(
self,
session: Session,
state: ReplState,
spinner: SpinnerState,
inbox: _alert_inbox.AlertInbox | None,
prompt_invalidator: Callable[[], None],
) -> None:
self.session = session
self.state = state
self.spinner = spinner
self.inbox = inbox
self.prompt_invalidator = prompt_invalidator
self.tasks: list[tuple[str, asyncio.Task[None]]] = []
self._loop: asyncio.AbstractEventLoop | None = None
self._sampler_started = False
def start_all(
self,
processor_coro: Callable[[], Coroutine[Any, Any, None]],
) -> list[tuple[str, asyncio.Task[None]]]:
# The fleet sampler (and its psutil dependency) is intentionally NOT
# started here: local-agent monitoring only runs once the user actually
# opens /fleet, via ``ensure_fleet_sampler_started``.
self._loop = asyncio.get_running_loop()
self.tasks = [
("processor", asyncio.create_task(processor_coro())),
("alert watcher", asyncio.create_task(self._alert_watcher())),
("spinner ticker", asyncio.create_task(self._spinner_ticker())),
]
return self.tasks
def ensure_fleet_sampler_started(self) -> None:
"""Start the fleet sampler on demand (first live ``/fleet`` use).
Safe to call from any thread: a shell turn (and thus the ``/fleet``
handler) runs in a worker thread via ``asyncio.to_thread``, so sampler
task creation is marshalled back onto the REPL event loop. Idempotent.
"""
loop = self._loop
if loop is None:
return
loop.call_soon_threadsafe(self._start_sampler_task)
def _start_sampler_task(self) -> None:
if self._sampler_started:
return
# Imported lazily so base REPL startup does not pull the sampler +
# psutil into the import path.
from tools.system.fleet_monitoring.sampler import start_sampler
self._sampler_started = True
self.tasks.append(("sampler", start_sampler()))
def drain_turn_start_output(self, console: Console) -> None:
if self.inbox is not None:
try:
drain_and_render_incoming(self.session, console, self.inbox)
except Exception as exc:
log.warning("Error draining alerts at turn start: %s", exc)
try:
drain_background_notices(self.session, console)
except Exception as exc:
log.warning("Error draining background notices at turn start: %s", exc)
async def _alert_watcher(self) -> None:
if self.inbox is None:
return
alert_console = Console(
highlight=False,
force_terminal=True,
color_system="truecolor",
legacy_windows=False,
)
drain_and_render_incoming(self.session, alert_console, self.inbox)
while not self.state.exit_requested:
try:
await asyncio.to_thread(self.inbox.pending_event.wait, timeout=1)
except asyncio.CancelledError:
return
try:
drain_and_render_incoming(self.session, alert_console, self.inbox)
except Exception as exc:
log.warning("Error draining incoming alerts: %s", exc)
async def _spinner_ticker(self) -> None:
# prompt_async's refresh_interval alone is not guaranteed to drive
# visible prompt redraws while patch_stdout(raw=True) is active and
# the LLM stream is writing rapidly. This task explicitly invalidates
# the prompt at 100 ms intervals so the braille glyph cycles smoothly.
tick_s = 0.1
was_streaming = False
while not self.state.exit_requested:
try:
await asyncio.sleep(tick_s)
except asyncio.CancelledError:
return
streaming = self.spinner.streaming
# Invalidate while streaming, plus one extra tick on the
# streaming->idle edge so the prompt repaints without the stale
# spinner/phase label instead of waiting for unrelated I/O.
if streaming or was_streaming:
self.prompt_invalidator()
was_streaming = streaming
@@ -0,0 +1,160 @@
"""Validated runtime context for interactive shell sessions."""
from __future__ import annotations
from typing import Self
from prompt_toolkit import PromptSession
from pydantic import BaseModel, ConfigDict, Field, InstanceOf, field_validator, model_validator
from core.agent_harness.session import SessionManager
from core.domain.alerts import inbox as _alert_inbox
from platform.observability.trace.spans import set_session_trace_sink
from surfaces.interactive_shell.runtime.core.state import (
ReplState,
SpinnerState,
create_repl_mutable_state,
)
from surfaces.interactive_shell.session.session import Session
from surfaces.interactive_shell.session.trace_sink import jsonl_trace_sink_for_session
class SessionBootstrapSpec(BaseModel):
"""Pydantic-enforced inputs for preparing a REPL session."""
model_config = ConfigDict(arbitrary_types_allowed=True, extra="forbid")
session: InstanceOf[Session] = Field(default_factory=Session)
pt_session: PromptSession[str] | None = None
active_theme_name: str | None = None
hydrate_integrations: bool = True
persistent_tasks: bool = True
@field_validator("active_theme_name")
@classmethod
def _active_theme_name_must_not_be_blank(cls, value: str | None) -> str | None:
if value is not None and not value.strip():
raise ValueError("active_theme_name must not be blank")
return value
@model_validator(mode="after")
def apply_to_session(self) -> Self:
"""Apply the canonical startup mutations to the validated session.
Core bootstrap (persistent task registry + integration hydration) is
delegated to :class:`SessionManager`; the shell layers its own UI
concerns (theme, grounding providers, prompt history) on top.
"""
SessionManager().bootstrap(
self.session,
hydrate_integrations=self.hydrate_integrations,
persistent_tasks=self.persistent_tasks,
)
self.session.terminal.active_theme_name = self.active_theme_name or _current_theme_name()
if self.pt_session is not None:
self.session.terminal.prompt_history_backend = self.pt_session.history
return self
class ReplRuntimeContext(BaseModel):
"""Validated bundle shared by REPL entrypoints and the controller."""
model_config = ConfigDict(
arbitrary_types_allowed=True,
extra="forbid",
validate_assignment=True,
)
session: InstanceOf[Session]
state: InstanceOf[ReplState]
spinner: InstanceOf[SpinnerState]
pt_session: PromptSession[str] | None = None
inbox: _alert_inbox.AlertInbox | None = None
@model_validator(mode="before")
@classmethod
def apply_initial_mutable_state(cls, data: object) -> object:
"""Set the paired mutable state defaults through one canonical factory."""
if not isinstance(data, dict):
return data
if "state" in data and "spinner" in data:
return data
mutable_state = create_repl_mutable_state(
state=data.get("state"),
spinner=data.get("spinner"),
)
return {
**data,
"state": mutable_state.state,
"spinner": mutable_state.spinner,
}
@model_validator(mode="after")
def bind_prompt_history_backend(self) -> Self:
"""Keep session prompt-history state aligned with the prompt session."""
if self.pt_session is not None:
self.session.terminal.prompt_history_backend = self.pt_session.history
return self
def _current_theme_name() -> str:
from platform.terminal.theme import get_active_theme_name
return get_active_theme_name()
def prepare_repl_session(
session: Session | None = None,
*,
pt_session: PromptSession[str] | None = None,
active_theme_name: str | None = None,
hydrate_integrations: bool = True,
persistent_tasks: bool = True,
) -> Session:
"""Return a session with the same defaults used by REPL boot."""
spec = SessionBootstrapSpec(
session=session or Session(),
pt_session=pt_session,
active_theme_name=active_theme_name,
hydrate_integrations=hydrate_integrations,
persistent_tasks=persistent_tasks,
)
return spec.session
def create_repl_runtime_context(
session: Session | None = None,
*,
state: ReplState | None = None,
spinner: SpinnerState | None = None,
pt_session: PromptSession[str] | None = None,
inbox: _alert_inbox.AlertInbox | None = None,
active_theme_name: str | None = None,
hydrate_integrations: bool = True,
persistent_tasks: bool = True,
) -> ReplRuntimeContext:
"""Create the canonical validated context for a REPL controller."""
prepared_session = prepare_repl_session(
session,
pt_session=pt_session,
active_theme_name=active_theme_name,
hydrate_integrations=hydrate_integrations,
persistent_tasks=persistent_tasks,
)
set_session_trace_sink(jsonl_trace_sink_for_session(prepared_session))
mutable_state = create_repl_mutable_state(state=state, spinner=spinner)
return ReplRuntimeContext(
session=prepared_session,
state=mutable_state.state,
spinner=mutable_state.spinner,
pt_session=pt_session,
inbox=inbox,
)
__all__ = [
"ReplRuntimeContext",
"SessionBootstrapSpec",
"create_repl_runtime_context",
"prepare_repl_session",
]
@@ -0,0 +1,13 @@
"""Core runtime engine for the interactive shell.
Reusable session state lives in ``core.agent_harness.session`` and terminal runtime
context lives in ``interactive_shell.runtime.context``. This package owns the
remaining runtime engine concerns (mutable runtime state, prompt manager,
turn detection).
"""
from __future__ import annotations
from platform.common.task_registry import TaskRegistry
__all__ = ["TaskRegistry"]
@@ -0,0 +1,39 @@
"""Prompt-mediated confirmation waiting for interactive-shell turns.
A turn worker thread parks here while it waits for the user to answer a
confirmation prompt rendered by the REPL prompt loop. The wait is
cancel-safe: it polls the dispatch cancel event and raises
:class:`DispatchCancelled` instead of silently auto-confirming.
"""
from __future__ import annotations
import threading
from surfaces.interactive_shell.runtime.core.state import PROMPT_REFRESH_INTERVAL_S, ReplState
class DispatchCancelled(Exception):
"""Raised when in-flight dispatch is cancelled during confirmation."""
def request_confirmation_via_prompt(state: ReplState, prompt_text: str) -> str:
response_event = threading.Event()
state.begin_confirmation(response_event, prompt_text)
try:
while not response_event.is_set():
cancel = state.current_cancel_event
if cancel is not None and cancel.is_set():
raise DispatchCancelled("cancelled while awaiting confirmation")
response_event.wait(timeout=PROMPT_REFRESH_INTERVAL_S)
if not state.confirm_response:
raise DispatchCancelled("cancelled while awaiting confirmation")
return state.confirm_response[0]
finally:
state.clear_confirmation()
__all__ = [
"DispatchCancelled",
"request_confirmation_via_prompt",
]
@@ -0,0 +1,128 @@
"""Prompt lifecycle and rendering glue for the interactive REPL loop."""
from __future__ import annotations
import asyncio
from collections.abc import Callable
from prompt_toolkit import PromptSession
from prompt_toolkit.application import Application
from prompt_toolkit.formatted_text import ANSI
from rich.console import Console
from surfaces.interactive_shell.runtime.core.state import (
PROMPT_REFRESH_INTERVAL_S,
ReplState,
SpinnerState,
)
from surfaces.interactive_shell.session import Session
from surfaces.interactive_shell.ui import input_prompt
from surfaces.interactive_shell.ui.components.cpr_stdin import (
drain_stale_cpr_bytes,
strip_cpr_sequences,
)
from surfaces.interactive_shell.ui.input_prompt import rendering as prompt_rendering
from surfaces.interactive_shell.ui.input_prompt.key_bindings import (
build_cancel_key_bindings,
install_session_key_bindings,
)
from surfaces.interactive_shell.ui.input_prompt.refresh import wire_prompt_refresh
from surfaces.interactive_shell.ui.input_prompt.style import refresh_prompt_theme
# Brief pause so a CPR reply still in flight lands in the stdin buffer before the
# non-blocking drain runs; without it the reply leaks into this prompt as literal bytes.
_CPR_SETTLE_SECONDS = 0.05
class PromptManager:
"""Own prompt-toolkit setup, prompt rendering, and prompt redraw hooks."""
def __init__(
self,
session: Session,
state: ReplState,
spinner: SpinnerState,
pt_session: PromptSession[str] | None = None,
) -> None:
self.session = session
self.state = state
self.spinner = spinner
self.pt_session = pt_session
self.pt_app: Application[str] | None = None
self.loop: asyncio.AbstractEventLoop | None = None
self._invalidate_prompt: Callable[[], None] | None = None
def setup(self) -> None:
if self.pt_session is None:
self.pt_session = input_prompt.build_prompt_session(self.session)
self.session.terminal.prompt_history_backend = self.pt_session.history
cancel_kb = build_cancel_key_bindings(self.state)
install_session_key_bindings(self.pt_session, cancel_kb)
self.pt_app = self.pt_session.app
self.loop = asyncio.get_running_loop()
self.session.terminal.prompt_app = self.pt_app
self.session.terminal.main_loop = self.loop
self.state.bind_loop(self.loop)
self._invalidate_prompt = wire_prompt_refresh(self.session, self.pt_app, self.loop)
@property
def invalidate_prompt(self) -> Callable[[], None]:
if self._invalidate_prompt is None:
raise RuntimeError("PromptManager.setup() must run before prompt invalidation")
return self._invalidate_prompt
def request_exit(self) -> None:
if self.pt_app is None or self.loop is None:
self.state.request_exit()
return
self.state.request_exit()
def _exit_prompt_app(attempts_left: int = 5) -> None:
if self.pt_app is not None and self.pt_app.is_running:
self.pt_app.exit()
return
if attempts_left > 0 and self.loop is not None:
self.loop.call_later(0.02, _exit_prompt_app, attempts_left - 1)
self.loop.call_soon_threadsafe(_exit_prompt_app)
def message_with_spinner(self) -> ANSI:
base = prompt_rendering._prompt_message(self.session).value
if self.state.is_awaiting_confirmation():
confirm_text = self.state.confirm_prompt_text
return ANSI(f"{confirm_text}\n{base}")
prefix = strip_cpr_sequences(
prompt_rendering.resolve_prompt_prefix_ansi(
inline_spinner=self.spinner.inline_spinner_ansi(),
idle_hint=prompt_rendering.resolve_idle_hint_ansi(self.session),
)
)
return ANSI(f"{prefix}\n{base}")
async def read_prompt_text(self) -> str:
if self.pt_session is None:
raise RuntimeError("PromptManager.setup() must run before reading prompts")
if self.session.terminal.pending_theme_refresh:
self.session.terminal.pending_theme_refresh = False
refresh_prompt_theme(self.session)
await asyncio.sleep(_CPR_SETTLE_SECONDS)
drain_stale_cpr_bytes()
prefilled = self.session.terminal.pop_pending_prompt_default()
if prefilled and self.session.terminal.pop_pending_autosubmit():
return prefilled
return await self.pt_session.prompt_async(
message=self.message_with_spinner,
bottom_toolbar=self.spinner.toolbar_ansi,
refresh_interval=PROMPT_REFRESH_INTERVAL_S,
placeholder=lambda: prompt_rendering.resolve_prompt_placeholder(self.session),
default=prefilled,
)
def render_submitted_prompt(self, console: Console, text: str) -> None:
prompt_rendering.render_submitted_prompt(console, self.session, text)
@@ -0,0 +1,268 @@
"""State models for the interactive shell UI runtime."""
from __future__ import annotations
import asyncio
import enum
import random
import threading
import time
from dataclasses import dataclass, field
from prompt_toolkit.application.current import get_app_or_none
from platform.terminal import theme as ui_theme
from surfaces.interactive_shell.ui.components.token_format import (
_CHARS_PER_TOKEN,
format_token_count_short,
)
# How often prompt-toolkit refreshes prompt callbacks and confirmation polling.
PROMPT_REFRESH_INTERVAL_S = 0.25
class TurnPhase(enum.Enum):
"""Explicit lifecycle phase of the current interactive-shell turn.
``phase`` is the declared turn intent and is authoritative for the
confirmation and cancelling states. ``is_dispatch_running()`` remains
derived from the asyncio task (the runtime truth of the in-flight turn),
because a task can settle on its own without an explicit transition.
"""
IDLE = "idle"
DISPATCHING = "dispatching"
AWAITING_CONFIRMATION = "awaiting_confirmation"
CANCELLING = "cancelling"
@dataclass
class ReplState:
"""Shared runtime state for prompt loop, queue worker, and cancel handlers.
Single source of truth for the active dispatch task, cancellation event,
confirmation lifecycle, exit request, and the explicit ``TurnPhase``.
Mutate turn state through the transition methods below rather than poking
raw fields, so ``phase`` stays consistent with the cancellation primitives.
"""
queue: asyncio.Queue[str] = field(default_factory=asyncio.Queue)
current_task: asyncio.Task[None] | None = None
current_cancel_event: threading.Event | None = None
loop: asyncio.AbstractEventLoop | None = None
exit_requested: bool = False
confirm_event: threading.Event | None = None
confirm_response: list[str] = field(default_factory=list)
confirm_prompt_text: str = ""
phase: TurnPhase = TurnPhase.IDLE
def is_dispatch_running(self) -> bool:
return self.current_task is not None and not self.current_task.done()
def is_awaiting_confirmation(self) -> bool:
return self.phase is TurnPhase.AWAITING_CONFIRMATION
def is_cancelling(self) -> bool:
return self.phase is TurnPhase.CANCELLING
def deliver_confirmation(self, answer: str) -> None:
if self.confirm_event is None:
return
self.confirm_response.append(answer)
self.confirm_event.set()
def bind_loop(self, loop: asyncio.AbstractEventLoop) -> None:
self.loop = loop
def request_exit(self) -> None:
self.exit_requested = True
def begin_confirmation(self, event: threading.Event, prompt_text: str = "") -> None:
# Reset the response list BEFORE publishing ``confirm_event`` so a
# concurrent ``deliver_confirmation`` cannot have its answer clobbered.
# ``phase`` is set before the publish so a parked worker is observable
# as awaiting confirmation the instant the event is visible.
self.confirm_response = []
self.confirm_prompt_text = prompt_text
self.phase = TurnPhase.AWAITING_CONFIRMATION
self.confirm_event = event
def clear_confirmation(self) -> None:
self.confirm_event = None
self.confirm_response = []
self.confirm_prompt_text = ""
# Only a normal confirmation completion returns to dispatching/idle; a
# cancel in progress must keep its CANCELLING phase.
if self.phase is TurnPhase.AWAITING_CONFIRMATION:
self.phase = TurnPhase.DISPATCHING if self.is_dispatch_running() else TurnPhase.IDLE
def start_dispatch(self, *, task: asyncio.Task[None], cancel_event: threading.Event) -> None:
self.current_task = task
self.current_cancel_event = cancel_event
self.phase = TurnPhase.DISPATCHING
def attach_turn_task(self, task: asyncio.Task[None]) -> None:
"""Mark a queued turn task as the active dispatch (queue worker entry)."""
self.current_task = task
self.phase = TurnPhase.DISPATCHING
def attach_cancel_event(self, cancel_event: threading.Event) -> None:
"""Park a cancel event for a dispatch that has no asyncio task."""
self.current_cancel_event = cancel_event
self.phase = TurnPhase.DISPATCHING
def clear_current_task(self, task: asyncio.Task[None] | None = None) -> None:
if task is None or self.current_task is task:
self.current_task = None
self.phase = TurnPhase.IDLE
def finish_dispatch(self, cancel_event: threading.Event) -> None:
if self.current_cancel_event is cancel_event:
self.current_cancel_event = None
self.phase = TurnPhase.IDLE
def cancel_current_dispatch(self) -> None:
# Mark the cancel intent first, but only when there is something to
# cancel, so an idle no-op call does not leave a stale CANCELLING phase.
if (
self.current_cancel_event is not None
or self.confirm_event is not None
or self.is_dispatch_running()
):
self.phase = TurnPhase.CANCELLING
if self.current_cancel_event is not None:
self.current_cancel_event.set()
if self.confirm_event is not None:
self.confirm_event.set()
task = self.current_task
if task is not None and not task.done():
if self.loop is not None:
self.loop.call_soon_threadsafe(task.cancel)
else:
task.cancel()
class SpinnerState:
"""Mutable state read by prompt callbacks for toolbar + inline spinner."""
_SPINNER_FRAMES = ("", "", "", "", "", "", "", "", "", "")
# One glyph advance per interval of *elapsed time*. The frame must be a
# pure function of the clock, never of how often the prompt message
# callback runs: prompt_toolkit evaluates the message several times per
# render pass (layout measurement + paint), so a per-call counter can land
# on the same frame every visible render and freeze the animation.
_FRAME_INTERVAL_S = 0.1
_THINKING_VERBS = (
"thinking",
"pondering",
"exploring",
"reasoning",
"considering",
"analysing",
"investigating",
"deliberating",
"ruminating",
"deducing",
"noodling",
)
def __init__(self) -> None:
self.streaming: bool = False
self.started_at: float = 0.0
self.bytes_in: int = 0
self._verb: str = self._THINKING_VERBS[0]
self.phase: str = ""
def start(self) -> None:
self.streaming = True
self.started_at = time.monotonic()
self.bytes_in = 0
self._verb = random.choice(self._THINKING_VERBS)
self.phase = ""
def set_phase(self, label: str) -> None:
"""Animate a caller-supplied phase label instead of a thinking verb.
Investigation stages (``/investigate``) dispatch deterministically, so
the turn-level "thinking" spinner never starts. The progress display
calls this to keep the prompt spinner cycling with the active pipeline
stage; it can be called repeatedly to advance the phase.
"""
if not self.streaming:
self.started_at = time.monotonic()
self._frame_idx = 0
self.streaming = True
self.phase = label
def stop(self) -> None:
self.streaming = False
self.phase = ""
def toolbar_ansi(self) -> str:
# Always return an empty string so prompt_toolkit's ConditionalContainer
# collapses the toolbar in every state. A visible toolbar causes
# prompt_toolkit to emit \033[6n (CPR) cursor-position queries on every
# refresh_interval; those responses leak into the vt100 input parser as
# literal keystrokes, corrupting the input field. Hiding the toolbar
# unconditionally also keeps its height at zero in both streaming and
# idle states, which prevents the one-row height delta that would cause
# prompt_toolkit to misplace the cursor and leave stale spinner lines on
# screen. Idle hints are surfaced through idle_hint_ansi() instead,
# which is rendered in the prompt message's reserved first line.
return ""
def idle_hint_ansi(self) -> str:
"""Dim hint line shown above the rule when no dispatch is running."""
hint = "/ for commands · ↑↓ history"
app = get_app_or_none()
if app is not None and app.current_buffer.text:
hint += " · esc to clear"
return f"{ui_theme.DIM_ANSI}{hint}{ui_theme.ANSI_RESET}"
def inline_spinner_ansi(self) -> str:
if not self.streaming:
return ""
elapsed = time.monotonic() - self.started_at
token_count = self.bytes_in // _CHARS_PER_TOKEN
frame_idx = int(elapsed / self._FRAME_INTERVAL_S)
glyph = self._SPINNER_FRAMES[frame_idx % len(self._SPINNER_FRAMES)]
if token_count > 0:
tokens_str = format_token_count_short(token_count)
suffix = f" ({elapsed:.0f}s · ↓ {tokens_str} tokens)"
else:
suffix = f" ({elapsed:.0f}s)"
label = self.phase or f"{self._verb}"
return (
f"{ui_theme.PROMPT_ACCENT_ANSI}{glyph} {label}{ui_theme.ANSI_RESET}"
f"{ui_theme.ANSI_DIM}{suffix} esc to cancel{ui_theme.ANSI_RESET}"
)
@dataclass(frozen=True)
class ReplMutableState:
"""Initial mutable state bundle shared by the interactive runtime."""
state: ReplState
spinner: SpinnerState
def create_repl_mutable_state(
*,
state: ReplState | None = None,
spinner: SpinnerState | None = None,
) -> ReplMutableState:
"""Return the canonical initial mutable state objects for a REPL runtime."""
return ReplMutableState(
state=state if state is not None else ReplState(),
spinner=spinner if spinner is not None else SpinnerState(),
)
__all__ = [
"PROMPT_REFRESH_INTERVAL_S",
"ReplMutableState",
"ReplState",
"SpinnerState",
"TurnPhase",
"create_repl_mutable_state",
]
@@ -0,0 +1,126 @@
"""Turn-result data model and the single owner of shell-turn accounting.
This module holds the shell's accounting side effects around the core
"facts only" turn-result models: action-agent analytics, terminal-turn
aggregate telemetry, prompt-recorder flushing, conversational-turn
persistence, and the final assistant-intent stamp.
"""
from __future__ import annotations
from dataclasses import dataclass
# The neutral "facts only" turn-result models live in the decoupled agent
# package; this module owns only the shell's accounting side effects over them.
from core.agent_harness.turns.turn_results import (
ShellTurnResult,
ToolCallingAccountingStatus,
ToolCallingTurnResult,
)
from platform.analytics.cli import capture_terminal_turn_summarized
from surfaces.interactive_shell.session import Session
from surfaces.interactive_shell.utils.telemetry import PromptRecorder
@dataclass
class ShellTurnAccounting:
"""Single owner of a shell turn's accounting side effects.
Separates "what happened" (decided by the turn flow) from "how it is
accounted for": action-agent analytics, terminal-turn aggregate telemetry,
prompt-recorder flushing, conversational-turn persistence, and the final
assistant-intent stamp.
"""
session: Session
text: str
recorder: PromptRecorder | None
def record_action_result(self, action_result: ToolCallingTurnResult) -> None:
"""Emit action-agent analytics and update terminal-turn aggregates."""
self._record_action_analytics(action_result)
self._record_terminal_turn(action_result)
def finalize(self, result: ShellTurnResult) -> ShellTurnResult:
"""Flush the recorder, persist the turn, and stamp the session intent."""
self._flush_prompt_recorder(result)
if result.llm_run is not None:
self.session.record("cli_agent", self.text)
self.session.last_assistant_intent = result.final_intent
return result
def _record_action_analytics(self, action_result: ToolCallingTurnResult) -> None:
from platform.analytics.cli import (
capture_repl_execution_policy_decision,
capture_terminal_actions_executed,
capture_terminal_actions_planned,
)
if action_result.accounting_status == "not_run":
capture_terminal_actions_executed(
planned_count=0,
executed_count=0,
executed_success_count=0,
)
return
capture_terminal_actions_planned(
planned_count=action_result.planned_count,
has_unhandled_clause=action_result.has_unhandled_clause,
)
capture_repl_execution_policy_decision(
{
"policy_stage": "shell_action_agent",
"policy_trace": (
"agent_tool_calls" if action_result.planned_count else "assistant_handoff"
),
"planned_count": action_result.planned_count,
"has_unhandled_clause": action_result.has_unhandled_clause,
}
)
capture_terminal_actions_executed(
planned_count=action_result.planned_count,
executed_count=action_result.executed_count,
executed_success_count=action_result.executed_success_count,
)
def _record_terminal_turn(self, action_result: ToolCallingTurnResult) -> None:
fallback_to_llm = not action_result.handled
snapshot = self.session.terminal.metrics.record_turn(
executed_count=action_result.executed_count,
executed_success_count=action_result.executed_success_count,
fallback_to_llm=fallback_to_llm,
)
capture_terminal_turn_summarized(
planned_count=action_result.planned_count,
executed_count=action_result.executed_count,
executed_success_count=action_result.executed_success_count,
fallback_to_llm=fallback_to_llm,
session_turn_index=snapshot.turn_index,
session_fallback_count=snapshot.fallback_count,
session_action_success_percent=snapshot.action_success_percent,
session_fallback_rate_percent=snapshot.fallback_rate_percent,
)
def _flush_prompt_recorder(self, result: ShellTurnResult) -> None:
# Pending turn LLM/error state is consumed unconditionally so a turn
# that stages it can never leak it into a later turn's flush.
pending_run = self.session.terminal.pop_pending_turn_llm()
pending_error = self.session.terminal.pop_pending_turn_error()
if self.recorder is None:
return
if pending_error is not None:
self.recorder.set_error(pending_error[0], pending_error[1])
self.recorder.set_response(
result.assistant_response_text,
result.llm_run if result.llm_run is not None else pending_run,
)
self.recorder.flush()
__all__ = [
"ShellTurnAccounting",
"ShellTurnResult",
"ToolCallingAccountingStatus",
"ToolCallingTurnResult",
]
@@ -0,0 +1,47 @@
"""Pure text classifiers for interactive-shell prompt turns."""
from __future__ import annotations
import re
_INTERVENTION_CORRECTION_RE = re.compile(
r"("
r"no(?=[,.!?]|$)"
r"|nope\b"
r"|nvm\b"
r"|nevermind\b|never\s*mind\b"
r"|wrong\b"
r"|wait(?=[,.!?]|$)"
r"|stop(?=[,.!?]|$)"
r"|actually\b"
r"|scratch\s+that\b"
r"|instead(?=[,.!?]|$)"
r"|(?:let'?s\s+)?do\s+[^.\n]{1,60}\s+instead\b"
r"|try\s+[^.\n]{1,60}\s+instead\b"
r")",
re.IGNORECASE,
)
_CONFIRMATION_TOKENS: frozenset[str] = frozenset({"", "y", "yes", "n", "no"})
_CANCEL_REQUEST_TOKENS: frozenset[str] = frozenset({"/cancel", "/stop", "/abort"})
def looks_like_confirmation_answer(text: str | None) -> bool:
return (text or "").strip().lower() in _CONFIRMATION_TOKENS
def looks_like_cancel_request(text: str | None) -> bool:
return (text or "").strip().lower() in _CANCEL_REQUEST_TOKENS
def looks_like_correction(text: str) -> bool:
stripped = text.lstrip()
if not stripped or stripped.startswith("```"):
return False
return _INTERVENTION_CORRECTION_RE.match(stripped[:80]) is not None
__all__ = [
"looks_like_cancel_request",
"looks_like_confirmation_answer",
"looks_like_correction",
]
@@ -0,0 +1,37 @@
"""Prompt input event reader for the interactive shell runtime."""
from surfaces.interactive_shell.runtime.input.actions import (
QUEUE_DURING_CONFIRMATION_WARNING,
CancelTurn,
CloseShell,
DeliverConfirmation,
IgnoreInput,
InputAction,
ShellInputSnapshot,
SubmitTurn,
decide_input_action,
)
from surfaces.interactive_shell.runtime.input.events import (
InputCancelled,
InputClosed,
InputEvent,
InputSubmitted,
)
from surfaces.interactive_shell.runtime.input.prompt_input_reader import PromptInputReader
__all__ = [
"CancelTurn",
"CloseShell",
"DeliverConfirmation",
"IgnoreInput",
"InputAction",
"InputCancelled",
"InputClosed",
"InputEvent",
"InputSubmitted",
"PromptInputReader",
"QUEUE_DURING_CONFIRMATION_WARNING",
"ShellInputSnapshot",
"SubmitTurn",
"decide_input_action",
]
@@ -0,0 +1,109 @@
"""Pure input-action decisions for the interactive shell controller."""
from __future__ import annotations
from collections.abc import Callable
from dataclasses import dataclass
from surfaces.interactive_shell.runtime.core.turn_detection import (
looks_like_cancel_request,
looks_like_confirmation_answer,
)
from surfaces.interactive_shell.runtime.input.events import (
InputCancelled,
InputClosed,
InputEvent,
InputSubmitted,
)
QUEUE_DURING_CONFIRMATION_WARNING = (
"[dim](type y/N to confirm the pending action; your input has been queued for after)[/]"
)
@dataclass(frozen=True)
class ShellInputSnapshot:
exit_requested: bool
dispatch_running: bool
awaiting_confirmation: bool
@dataclass(frozen=True)
class IgnoreInput:
pass
@dataclass(frozen=True)
class CloseShell:
pass
@dataclass(frozen=True)
class CancelTurn:
submitted_text: str | None = None
@dataclass(frozen=True)
class DeliverConfirmation:
text: str
@dataclass(frozen=True)
class SubmitTurn:
text: str
wait_until_idle: bool = False
warning: str | None = None
InputAction = IgnoreInput | CloseShell | CancelTurn | DeliverConfirmation | SubmitTurn
def decide_input_action(
event: InputEvent,
snapshot: ShellInputSnapshot,
*,
needs_exclusive_stdin: Callable[[str], bool],
) -> InputAction:
"""Interpret one prompt event without mutating runtime state."""
match event:
case InputClosed():
return CloseShell()
case InputCancelled():
return CancelTurn()
case InputSubmitted(text):
if snapshot.exit_requested or not text:
return IgnoreInput()
stripped = text.strip()
if not stripped:
return IgnoreInput()
if snapshot.dispatch_running and looks_like_cancel_request(stripped):
return CancelTurn(submitted_text=stripped)
if snapshot.awaiting_confirmation:
if looks_like_confirmation_answer(stripped):
return DeliverConfirmation(text=stripped)
return SubmitTurn(
text=stripped,
warning=QUEUE_DURING_CONFIRMATION_WARNING,
)
return SubmitTurn(
text=stripped,
wait_until_idle=needs_exclusive_stdin(stripped),
)
raise AssertionError(f"Unhandled input event: {event!r}")
__all__ = [
"CancelTurn",
"CloseShell",
"DeliverConfirmation",
"IgnoreInput",
"InputAction",
"QUEUE_DURING_CONFIRMATION_WARNING",
"ShellInputSnapshot",
"SubmitTurn",
"decide_input_action",
]
@@ -0,0 +1,26 @@
"""Explicit input events consumed by the interactive shell loop."""
from __future__ import annotations
from dataclasses import dataclass
@dataclass(frozen=True)
class InputSubmitted:
text: str
@dataclass(frozen=True)
class InputCancelled:
pass
@dataclass(frozen=True)
class InputClosed:
pass
InputEvent = InputSubmitted | InputCancelled | InputClosed
__all__ = ["InputCancelled", "InputClosed", "InputEvent", "InputSubmitted"]
@@ -0,0 +1,74 @@
"""Convert prompt-toolkit terminal behavior into shell input events."""
from __future__ import annotations
from rich.console import Console
from platform.terminal.prompt_support import (
print_session_resume_hint,
repl_prompt_note_ctrl_c,
repl_reset_ctrl_c_gate,
)
from surfaces.interactive_shell.runtime.core.prompt_manager import PromptManager
from surfaces.interactive_shell.runtime.core.state import ReplState
from surfaces.interactive_shell.runtime.input.events import (
InputCancelled,
InputClosed,
InputEvent,
InputSubmitted,
)
from surfaces.interactive_shell.session import Session
from surfaces.interactive_shell.ui import DIM
from surfaces.interactive_shell.ui.components.cpr_stdin import (
contains_cpr_sequence,
strip_cpr_sequences,
)
class PromptInputReader:
"""Read prompt text and hide terminal-specific control flow from the loop."""
def __init__(
self,
prompt: PromptManager,
state: ReplState,
session: Session,
console: Console,
) -> None:
self.prompt = prompt
self.state = state
self.session = session
self.console = console
async def read(self) -> InputEvent:
while True:
try:
text = await self.prompt.read_prompt_text()
except EOFError:
if self.state.is_dispatch_running():
return InputCancelled()
self._render_session_resume_hint()
return InputClosed()
except KeyboardInterrupt:
if self.state.is_dispatch_running():
return InputCancelled()
if repl_prompt_note_ctrl_c(self.console, self.session.session_id):
return InputClosed()
return InputCancelled()
repl_reset_ctrl_c_gate()
raw_text = text
text = strip_cpr_sequences(text)
if not text.strip() and contains_cpr_sequence(raw_text):
continue
return InputSubmitted(text)
def _render_session_resume_hint(self) -> None:
if not self.session.session_id:
return
self.console.print()
print_session_resume_hint(self.console, self.session.session_id)
self.console.print(f"[{DIM}]Goodbye![/]")
__all__ = ["PromptInputReader"]
@@ -0,0 +1,197 @@
"""Gather integration evidence for a conversational shell answer.
The bounded think -> call-tools -> observe loop lives in the decoupled
:func:`core.agent_harness.turns.evidence_driver.gather_tool_evidence`. This module is the terminal adapter:
it renders each gathering step to the console and persists the gathered tool
calls into the shell's session storage, then hands the collected observation back
to :func:`interactive_shell.runtime.answer_turn.answer_shell_question`.
"""
from __future__ import annotations
import contextlib
import json
from typing import Any
from rich.console import Console
from rich.markup import escape
from core.agent_harness.turns import evidence_driver
from core.agent_harness.turns.evidence_driver import GatherAgentFactory
from surfaces.interactive_shell.session import Session
from surfaces.interactive_shell.ui import DIM
from surfaces.interactive_shell.utils.error_handling.exception_reporting import report_exception
from surfaces.shared.tool_labels import tool_short_label, tool_source_label
# Cap so a chatty tool result can't blow up persistence writes.
_MAX_PER_TOOL_CHARS = 4_000
# Keys most likely to distinguish back-to-back calls to the same tool.
_GATHER_INPUT_HINT_KEYS: tuple[str, ...] = (
"metric_name",
"query",
"search",
"filter",
"expression",
"promql",
"service_name",
"owner",
"repo",
"log_group",
"monitor_id",
"alert_id",
"issue_id",
"trace_id",
"span_id",
"dashboard_uid",
"panel_id",
"from",
"to",
"time_range",
)
class _ShellGatherErrorReporter:
"""Minimal :class:`core.agent_harness.ports.ErrorReporter` over ``report_exception``."""
def report(self, exc: BaseException, *, context: str, expected: bool = False) -> None:
report_exception(exc, context=context, expected=expected)
def _truncate_hint(text: str, *, max_len: int = 48) -> str:
compact = " ".join(text.split())
if len(compact) <= max_len:
return compact
return f"{compact[: max_len - 1]}"
def _tool_input_hint(tool_input: Any) -> str:
if not isinstance(tool_input, dict):
return ""
hints: list[str] = []
seen: set[str] = set()
for key in _GATHER_INPUT_HINT_KEYS:
value = tool_input.get(key)
if value in (None, "", [], {}):
continue
rendered = _truncate_hint(str(value))
if not rendered or rendered in seen:
continue
seen.add(rendered)
hints.append(rendered)
if len(hints) >= 2:
break
return " · ".join(hints)
def _format_gathering_progress_line(
tool_name: str,
tool_input: Any,
*,
repeat_index: int,
) -> str:
source = tool_source_label(tool_name)
label = tool_short_label(tool_name, source)
call_display = f"{source} · {label}" if label else source
if repeat_index > 1:
call_display = f"{call_display} ({repeat_index})"
safe_display = escape(call_display)
hint = _tool_input_hint(tool_input)
if hint:
return f"· gathering via {safe_display}{escape(hint)}"
return f"· gathering via {safe_display}"
def _resolve_gather_integrations(
session: Session,
message: str,
resolved_integrations: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""Resolve gather integrations through the decoupled agent helper."""
return evidence_driver._resolve_gather_integrations( # noqa: SLF001
session, message, resolved_integrations=resolved_integrations
)
def _truncate(text: str, limit: int) -> str:
if len(text) <= limit:
return text
return text[:limit] + f"\n…[truncated, {len(text)} chars total]"
def _persist_tool_calls(session: Session, executed: list[tuple[Any, Any]]) -> None:
"""Record each gathered tool-call result into the session log.
Arguments and results are redacted and bounded before writing; failures are
swallowed so logging never breaks the turn.
"""
from core.agent_harness.session import default_session_storage
from platform.observability.trace.redaction import redact_sensitive
storage = default_session_storage()
for tc, output in executed:
with contextlib.suppress(Exception):
body = (
output
if isinstance(output, str)
else json.dumps(redact_sensitive(output), default=str)
)
arguments = (
redact_sensitive(tc.input) if isinstance(tc.input, dict) else {"value": tc.input}
)
storage.append_tool_call(
session.session_id,
tool=str(tc.name),
arguments=arguments,
result=_truncate(body, _MAX_PER_TOOL_CHARS),
ok=not (isinstance(output, dict) and "error" in output),
)
def gather_integration_tool_evidence(
message: str,
session: Session,
console: Console,
*,
is_tty: bool | None = None,
agent_factory: GatherAgentFactory | None = None,
resolved_integrations: dict[str, Any] | None = None,
) -> str | None:
"""Run a bounded tool-calling loop and return collected evidence, or None.
Returns a formatted observation block when at least one tool was executed;
otherwise ``None`` so the caller falls back to the normal text-only answer.
``resolved_integrations`` is the turn's resolved view; it is forwarded so the
gather phase reuses it instead of resolving again.
"""
tool_call_counts: dict[str, int] = {}
def on_progress(kind: str, data: dict[str, Any]) -> None:
if kind == "tool_start":
name = str(data.get("name", "")).strip() or "tool"
tool_call_counts[name] = tool_call_counts.get(name, 0) + 1
line = _format_gathering_progress_line(
name,
data.get("input"),
repeat_index=tool_call_counts[name],
)
console.print(f"[{DIM}]{line}[/]")
elif kind == "gather_cancelled":
console.print(f"[{DIM}]· gathering cancelled[/]")
def persist(executed: list[tuple[Any, Any]]) -> None:
_persist_tool_calls(session, executed)
return evidence_driver.gather_tool_evidence(
message,
session,
on_progress=on_progress,
persist=persist,
error_reporter=_ShellGatherErrorReporter(),
is_tty=is_tty,
agent_factory=agent_factory,
resolved_integrations=resolved_integrations,
)
__all__ = ["gather_integration_tool_evidence"]
@@ -0,0 +1,170 @@
"""REPL adapters for session investigation streaming and action-tool launch."""
from __future__ import annotations
import threading
from collections.abc import Callable, Iterator
from typing import Any
from rich.console import Console
from core.domain.stream import StreamEvent
from platform.common.task_types import TaskRecord
from surfaces.interactive_shell.session import Session
from surfaces.interactive_shell.ui.execution_confirm import execution_allowed
from surfaces.interactive_shell.ui.foreground_investigation import run_foreground_investigation
from tools.interactive_shell.shared.execution_policy import ExecutionPolicyResult
from tools.interactive_shell.shared.investigation_launch import (
ForegroundInvestigationResult,
InvestigationLaunchPorts,
)
from tools.investigation import session_runner
def repl_foreground_renderer() -> session_runner.StreamRendererFn:
"""Return a renderer that streams investigation progress to the REPL terminal."""
from surfaces.cli.ui.renderer import StreamRenderer
def _render(events: Iterator[StreamEvent]) -> dict[str, Any]:
renderer = StreamRenderer(local=True)
return dict(renderer.render_stream(events))
return _render
def repl_background_renderer() -> session_runner.StreamRendererFn:
"""Return a silent renderer for background investigations."""
from surfaces.cli.ui.renderer import StreamRenderer
from surfaces.interactive_shell.ui.output import reset_tracker, set_silent_tracker
def _render(events: Iterator[StreamEvent]) -> dict[str, Any]:
set_silent_tracker()
try:
renderer = StreamRenderer(local=True, display=False)
return dict(renderer.render_stream(events))
finally:
reset_tracker()
return _render
def run_investigation_for_session(
*,
alert_text: str,
context_overrides: dict[str, Any] | None = None,
cancel_requested: threading.Event | None = None,
) -> dict[str, Any]:
"""Run a foreground streaming investigation in the REPL."""
return session_runner.run_investigation_for_session(
alert_text=alert_text,
context_overrides=context_overrides,
cancel_requested=cancel_requested,
render_stream=repl_foreground_renderer(),
)
def run_sample_alert_for_session(
*,
template_name: str = "generic",
context_overrides: dict[str, Any] | None = None,
cancel_requested: threading.Event | None = None,
) -> dict[str, Any]:
"""Run a foreground sample-alert investigation in the REPL."""
return session_runner.run_sample_alert_for_session(
template_name=template_name,
context_overrides=context_overrides,
cancel_requested=cancel_requested,
render_stream=repl_foreground_renderer(),
)
def run_investigation_for_session_background(
*,
alert_text: str,
context_overrides: dict[str, Any] | None = None,
cancel_requested: threading.Event | None = None,
) -> dict[str, Any]:
"""Run a silent background investigation in the REPL."""
return session_runner.run_investigation_for_session_background(
alert_text=alert_text,
context_overrides=context_overrides,
cancel_requested=cancel_requested,
render_stream=repl_background_renderer(),
)
def run_sample_alert_for_session_background(
*,
template_name: str = "generic",
context_overrides: dict[str, Any] | None = None,
cancel_requested: threading.Event | None = None,
) -> dict[str, Any]:
"""Run a silent background sample-alert investigation in the REPL."""
return session_runner.run_sample_alert_for_session_background(
template_name=template_name,
context_overrides=context_overrides,
cancel_requested=cancel_requested,
render_stream=repl_background_renderer(),
)
class ReplInvestigationLaunchPorts:
"""Default REPL ports for investigation-style action tools."""
def execution_allowed(
self,
*,
policy: ExecutionPolicyResult,
session: Session,
console: Console,
action_summary: str,
confirm_fn: Callable[[str], str] | None,
is_tty: bool | None,
action_already_listed: bool,
) -> bool:
return execution_allowed(
policy,
session=session,
console=console,
action_summary=action_summary,
confirm_fn=confirm_fn,
is_tty=is_tty,
action_already_listed=action_already_listed,
)
def run_foreground_investigation(
self,
*,
session: Session,
console: Console,
task_command: str,
run: Callable[[TaskRecord], dict[str, object]],
exception_context: str,
target: str,
) -> ForegroundInvestigationResult:
outcome = run_foreground_investigation(
session=session,
console=console,
task_command=task_command,
run=run,
exception_context=exception_context,
target=target,
)
return ForegroundInvestigationResult(status=outcome.status)
def repl_investigation_launch_ports() -> InvestigationLaunchPorts:
"""Return REPL investigation launch ports for action tools."""
return ReplInvestigationLaunchPorts()
__all__ = [
"ReplInvestigationLaunchPorts",
"repl_background_renderer",
"repl_foreground_renderer",
"repl_investigation_launch_ports",
"run_investigation_for_session",
"run_investigation_for_session_background",
"run_sample_alert_for_session",
"run_sample_alert_for_session_background",
]
@@ -0,0 +1,114 @@
"""Compose one interactive-shell turn from its action/gather/answer adapters.
Adapter-only: binds the shell's action-turn (``action_turn``), gather pass
(``integration_tool_gathering``), and answer (``answer_turn``) adapters to the
surface-agnostic ``run_turn`` engine. Each adapter owns its own binding; this
file only composes them and attaches turn accounting. The injection contracts
live in ``turn_seams``.
"""
from __future__ import annotations
from collections.abc import Callable
from typing import Unpack
from rich.console import Console
from core.agent_harness.ports import OutputSink
from core.agent_harness.turns.orchestrator import run_turn
from core.agent_harness.turns.turn_plan import TurnPlan
from core.agent_harness.turns.turn_results import ShellTurnResult, ToolCallingTurnResult
from core.execution import ToolExecutionHooks
from surfaces.interactive_shell.runtime.action_turn import run_action_tool_turn
from surfaces.interactive_shell.runtime.agent_harness_adapters import resolve_output_sink
from surfaces.interactive_shell.runtime.answer_turn import answer_shell_question
from surfaces.interactive_shell.runtime.core.turn_accounting import ShellTurnAccounting
from surfaces.interactive_shell.runtime.integration_tool_gathering import (
gather_integration_tool_evidence,
)
from surfaces.interactive_shell.runtime.turn_seams import (
AnswerKwargs,
AnswerShellQuestion,
GatherEvidence,
RunActionToolTurn,
)
from surfaces.interactive_shell.session import Session
from surfaces.interactive_shell.utils.telemetry import LlmRunInfo, PromptRecorder
def execute_shell_turn(
text: str,
session: Session,
console: Console,
*,
recorder: PromptRecorder | None,
confirm_fn: Callable[[str], str] | None = None,
is_tty: bool | None = None,
request_exit: Callable[[], None] | None = None,
execute_actions: RunActionToolTurn | None = None,
gather_evidence: GatherEvidence | None = None,
answer_agent: AnswerShellQuestion | None = None,
output: OutputSink | None = None,
tool_hooks: ToolExecutionHooks | None = None,
) -> ShellTurnResult:
"""Execute one submitted interactive-shell turn.
The action driver, gather pass, and conversational assistant default to the
shell adapters but are overridable via ``execute_actions`` / ``gather_evidence``
/ ``answer_agent`` (the test injection seams, typed in ``turn_seams``). They are
bound to the live ``session``/``console`` here and handed to
:func:`core.agent_harness.turns.orchestrator.run_turn`, which performs
the pure path routing.
"""
_execute = execute_actions or run_action_tool_turn
_gather = gather_evidence or gather_integration_tool_evidence
_answer = answer_agent or answer_shell_question
accounting = ShellTurnAccounting(session=session, text=text, recorder=recorder)
resolved_output = resolve_output_sink(console, output)
def execute_bound(
t: str,
*,
confirm_fn: Callable[[str], str] | None = None,
is_tty: bool | None = None,
turn_plan: TurnPlan | None = None,
) -> ToolCallingTurnResult:
return _execute(
t,
session,
console,
confirm_fn=confirm_fn,
is_tty=is_tty,
request_exit=request_exit,
turn_plan=turn_plan,
output=resolved_output,
tool_hooks=tool_hooks,
)
def answer_bound(t: str, **kwargs: Unpack[AnswerKwargs]) -> LlmRunInfo | None:
# run_turn controls which keys are present (it omits tool_observation_on_screen
# on the plain path); AnswerKwargs types them without forcing presence.
return _answer(t, session, console, output=resolved_output, **kwargs)
def gather_bound(
t: str,
*,
is_tty: bool | None = None,
turn_plan: TurnPlan | None = None,
) -> str | None:
resolved = turn_plan.resolved_integrations if turn_plan is not None else None
return _gather(t, session, console, is_tty=is_tty, resolved_integrations=resolved)
return run_turn(
text,
session,
execute_actions=execute_bound,
answer=answer_bound,
gather=gather_bound,
accounting=accounting,
confirm_fn=confirm_fn,
is_tty=is_tty,
)
__all__ = ["execute_shell_turn"]
@@ -0,0 +1 @@
"""Interactive shell startup and first-launch gates."""
@@ -0,0 +1,303 @@
"""First-launch GitHub login gate.
On the first interactive launch of ``opensre`` (all platforms), the user is
prompted to sign in to GitHub via device flow unless they skip, are in CI/CD, a
test harness, or a non-interactive session. The sign-in runs the hosted GitHub
MCP setup, persists the integration, and propagates the authenticated GitHub
username to PostHog.
Escape hatch: ``OPENSRE_SKIP_GITHUB_LOGIN=1`` bypasses the gate so a GitHub
outage or a disabled device flow can never permanently lock anyone out. The gate
is also auto-bypassed in CI/test environments and when stdin is not a TTY.
"""
from __future__ import annotations
import logging
import os
import sys
import time
from rich.console import Console
from rich.markup import escape
from config.repl_config import read_github_login_deferred, write_github_login_deferred
from platform.analytics.cli import capture_github_login_completed
from platform.analytics.source import is_test_run
from platform.terminal.theme import DEVICE_CODE
from surfaces.interactive_shell.ui import repl_tty_interactive
_SKIP_ENV_VAR = "OPENSRE_SKIP_GITHUB_LOGIN"
_TRUTHY = frozenset({"1", "true", "yes", "on"})
_SIGN_IN_CHOICE = "sign_in"
_SKIP_CHOICE = "skip"
def _skip_requested() -> bool:
return os.getenv(_SKIP_ENV_VAR, "").strip().lower() in _TRUTHY
def _github_login_explicitly_bypassed() -> bool:
"""Cheap check for contexts where gate errors should not block startup."""
if _skip_requested():
return True
if os.getenv("OPENSRE_INVESTIGATION_SOURCE", "").strip().lower() == "test":
return True
if os.getenv("OPENSRE_IS_TEST", "0").strip() == "1":
return True
if os.getenv("PYTEST_CURRENT_TEST"):
return True
if os.getenv("GITHUB_ACTIONS", "").strip().lower() == "true":
return True
ci_value = os.getenv("CI", "").strip().lower()
if ci_value in {"1", "true", "yes"}:
return True
try:
return not sys.stdin.isatty()
except Exception:
return True
def _github_already_configured() -> bool:
from integrations.github.mcp import github_integration_is_configured
return github_integration_is_configured()
def should_require_github_login() -> bool:
"""Return True when the first-launch GitHub login prompt must run now."""
if _skip_requested():
return False
if read_github_login_deferred():
return False
if is_test_run():
return False
if not repl_tty_interactive():
return False
# GitHub being configured is the authoritative bypass. We intentionally do
# NOT consult a first-launch "completion" marker here: a stale marker must
# never let the REPL start once the GitHub integration has been removed
# (e.g. via ``/integrations remove github``). Re-checking the store is cheap,
# so the gate always re-runs when GitHub is not currently configured.
return not _github_already_configured()
def clear_github_login_deferral() -> None:
"""Clear a saved skip so removing GitHub can re-prompt on the next launch."""
if not read_github_login_deferred():
return
write_github_login_deferred(False)
def _propagate_username(username: str) -> None:
if not username:
return
# ``authenticate_and_configure_github`` already calls identify_github_username;
# only emit the one-time login lifecycle event here.
capture_github_login_completed(username)
def _print_intro(console: Console) -> None:
console.print()
console.print("[bold]Connect GitHub to get started[/bold]")
console.print(
"OpenSRE needs read access to your GitHub repositories to investigate "
"incidents against your source. Sign in once with your browser."
)
console.print(
"[dim](Escape to skip for now, or set "
f"{_SKIP_ENV_VAR}=1 if GitHub sign-in is unavailable.)[/dim]"
)
def _show_device_code(console: Console, code: object) -> None:
from integrations.github.mcp_oauth import GitHubDeviceCode
if not isinstance(code, GitHubDeviceCode):
return
user_code = escape(code.user_code)
console.print()
console.print(f" 1. Your browser will open [underline]{code.verification_uri}[/underline]")
console.print(" (if it doesn't open automatically, visit that URL yourself).")
console.print(f" 2. Enter this one-time code when GitHub asks: [{DEVICE_CODE}]{user_code}[/]")
console.print(" 3. Approve the request for OpenSRE.")
console.print()
console.print(
" [dim]Waiting for you to approve in the browser… (Escape or Ctrl-C to skip)[/dim]"
)
def _print_skip_guidance(console: Console) -> None:
console.print()
console.print(
"[dim]Skipped GitHub sign-in. Connect later with "
"[bold]/integrations setup[/bold] or [bold]/mcp connect github[/bold].[/dim]"
)
def _defer_github_login() -> None:
write_github_login_deferred(True)
def _sleep_until_or_cancel(seconds: float) -> None:
"""Sleep up to ``seconds``, raising ``KeyboardInterrupt`` when the user skips."""
if seconds <= 0 or not sys.stdin.isatty():
time.sleep(seconds)
return
if os.name == "nt":
import msvcrt
from surfaces.interactive_shell.ui.components.key_reader import read_key_windows
deadline = time.monotonic() + seconds
while time.monotonic() < deadline:
if msvcrt.kbhit() and read_key_windows() == "cancel": # type: ignore[attr-defined]
raise KeyboardInterrupt
time.sleep(0.05)
return
import select
import termios
import tty
from surfaces.interactive_shell.ui.components.key_reader import read_key_unix
fd = sys.stdin.fileno()
old_attrs = termios.tcgetattr(fd) # type: ignore[attr-defined]
try:
tty.setraw(fd) # type: ignore[attr-defined]
deadline = time.monotonic() + seconds
while time.monotonic() < deadline:
remaining = deadline - time.monotonic()
if remaining <= 0:
return
ready, _, _ = select.select([fd], [], [], min(remaining, 0.15))
if not ready:
continue
if read_key_unix() == "cancel":
raise KeyboardInterrupt
finally:
termios.tcsetattr(fd, termios.TCSADRAIN, old_attrs) # type: ignore[attr-defined]
def _offer_github_login(_console: Console) -> bool:
"""Return True when the user wants to start browser sign-in."""
import questionary
try:
choice = questionary.select(
"Connect GitHub now?",
choices=[
questionary.Choice(
"Sign in with GitHub (opens browser)",
value=_SIGN_IN_CHOICE,
),
questionary.Choice("Skip for now", value=_SKIP_CHOICE),
],
default=_SIGN_IN_CHOICE,
).ask()
except (EOFError, KeyboardInterrupt):
return False
if choice is None:
return False
return bool(choice == _SIGN_IN_CHOICE)
def _ask_retry(_console: Console) -> bool:
import questionary
try:
answer = questionary.confirm("Try GitHub sign-in again?", default=True).ask()
except (EOFError, KeyboardInterrupt):
return False
if answer is None:
return False
return bool(answer)
def _attempt_login(console: Console) -> str:
"""Run one login attempt. Returns ``"success"``, ``"failed"``, or ``"skipped"``."""
from integrations.github.login import authenticate_and_configure_github
from integrations.github.mcp_oauth import GitHubDeviceFlowError
try:
result = authenticate_and_configure_github(
on_prompt=lambda code: _show_device_code(console, code),
poll_sleep=_sleep_until_or_cancel,
)
except (EOFError, KeyboardInterrupt):
console.print("\nSkipped GitHub sign-in.")
return "skipped"
except GitHubDeviceFlowError as err:
console.print(f"[yellow]GitHub sign-in is unavailable:[/yellow] {err}")
return "failed"
except Exception as err: # network/transport issues
console.print(f"[yellow]GitHub sign-in failed:[/yellow] {err}")
return "failed"
if result.ok:
clear_github_login_deferral()
# Persisting the GitHub integration (done inside
# ``authenticate_and_configure_github``) is what suppresses the gate on
# subsequent launches — there is no separate completion marker to write.
_propagate_username(result.username)
who = f"@{result.username}" if result.username else "your GitHub account"
console.print(f"[bold]Connected.[/bold] Signed in as {who}.")
return "success"
console.print(f"[yellow]Could not verify GitHub access:[/yellow] {result.detail}")
return "failed"
def require_github_login_on_first_launch(console: Console | None = None) -> bool:
"""Run the first-launch GitHub login prompt.
Returns True when the caller should proceed into the REPL (login succeeded or
the user skipped), and False only when startup must abort.
"""
con = console or Console(highlight=False)
_print_intro(con)
if not _offer_github_login(con):
_defer_github_login()
_print_skip_guidance(con)
return True
while True:
outcome = _attempt_login(con)
if outcome == "success":
return True
if outcome == "skipped":
_defer_github_login()
_print_skip_guidance(con)
return True
if not _ask_retry(con):
_defer_github_login()
_print_skip_guidance(con)
return True
def require_startup_github_login(console: Console) -> bool:
"""Return True when startup may proceed past the GitHub login gate.
On an unexpected gate error we deliberately do NOT fail open into the REPL:
that would let a gate bug silently skip sign-in. Instead we only allow
startup when an explicit, documented bypass applies.
"""
try:
if not should_require_github_login():
return True
return require_github_login_on_first_launch(console)
except Exception:
logging.getLogger(__name__).warning(
"First-launch GitHub login gate failed.",
exc_info=True,
)
if _github_login_explicitly_bypassed():
return True
console.print(
"GitHub sign-in could not run. "
f"Set [bold]{_SKIP_ENV_VAR}=1[/bold] to bypass this, then relaunch "
"[bold]opensre[/bold]."
)
return False
@@ -0,0 +1,54 @@
"""Non-interactive initial-input replay for REPL startup."""
from __future__ import annotations
from rich.console import Console
from platform.analytics.repl_context import bound_repl_turn_context
from surfaces.interactive_shell.session import Session
from surfaces.interactive_shell.ui.banner import render_banner
from surfaces.interactive_shell.ui.input_prompt.rendering import render_submitted_prompt
from surfaces.interactive_shell.utils.telemetry import PromptRecorder
_TURN_KIND = "agent"
def run_initial_input(
initial_input: str,
session: Session,
) -> int:
# Imported lazily so importing this module during REPL boot (main.py imports
# ``run_initial_input`` at top) does not pull the harness/turn-execution
# stack into the base import path when there is no initial input to replay.
from surfaces.interactive_shell.runtime.shell_turn_execution import execute_shell_turn
console = Console(
highlight=False,
force_terminal=True,
color_system="truecolor",
legacy_windows=False,
)
render_banner(console)
for line in initial_input.splitlines():
stripped = line.strip()
if not stripped:
continue
render_submitted_prompt(console, session, stripped)
recorder = PromptRecorder.start(session=session, text=stripped, turn_kind=_TURN_KIND)
with bound_repl_turn_context(
session_id=session.session_id,
turn_kind=_TURN_KIND,
prompt_turn_id=recorder.turn_id if recorder is not None else None,
):
execute_shell_turn(
stripped,
session,
console,
recorder=recorder,
confirm_fn=None,
is_tty=False,
)
return 0
__all__ = ["run_initial_input"]
@@ -0,0 +1,132 @@
"""Execute planned opensre CLI actions.
Pure subprocess execution, planning, and orchestration for shell / synthetic /
Claude Code / opensre CLI action tools live under ``tools.interactive_shell``
(``subprocess/``, ``shell/``, ``synthetic/``, ``implementation/``, ``cli/``).
This package keeps Rich relay helpers in ``task_streaming``, the REPL presenter
in ``repl_presenter``, and backward-compatible surface adapters such as
``opensre_cli_runner`` for slash parity and legacy test monkeypatch paths.
Shell command execution lives in ``tools.interactive_shell.shell`` (parsing,
policy, ``execute_shell_command``, and the ``run_shell_command`` / ``run_cd`` /
``run_pwd`` runner); it is intentionally not re-exported here. Synthetic test
execution lives in ``tools.interactive_shell.synthetic`` (the
``run_synthetic_test`` / ``watch_synthetic_subprocess`` runner), Claude Code
implementation execution lives in ``tools.interactive_shell.implementation.claude_code_executor``
(``run_claude_code_implementation``), and sample-alert / free-text investigation
execution lives in ``tools.interactive_shell.actions.sample_alert`` /
``tools.interactive_shell.actions.investigation`` (``run_sample_alert`` /
``run_text_investigation``); none are re-exported here. Shared stdlib subprocess
primitives live in ``tools.interactive_shell.subprocess``; Rich stream relay
remains in ``task_streaming``.
Public API is stable: all names exported below are importable directly from
``subprocess_runner`` and will remain so regardless of internal submodule changes.
Stdlib modules ``os``, ``subprocess``, and ``threading`` are re-imported here so
that tests can patch them via the full ``subprocess_runner.<module>.<attr>`` path
(e.g. ``subprocess_runner.subprocess.Popen``). Since these are module singletons in
``sys.modules``, patching via this attribute also affects the actual call sites
inside the submodules.
"""
from __future__ import annotations
# Stdlib singletons — imported so that monkeypatch paths resolve correctly in tests:
# ``"…subprocess_runner.os.chdir"``, ``"…subprocess_runner.subprocess.Popen"``,
# ``"…subprocess_runner.threading.Thread"``, ``"…subprocess_runner.time.sleep"``,
# ``"…subprocess_runner.Path.cwd"``.
import os
import subprocess
import threading
import time
from pathlib import Path
from .background_task_executor import start_background_cli_task
from .opensre_cli_runner import (
_INTERACTIVE_OPENSRE_COMMAND_PATHS,
_OPENSRE_BLOCKED_SUBCOMMANDS,
OpensreCommandClass,
OpensreExecutionMode,
OpensreExecutionPlan,
OpensreRunOutcome,
OpensreRunResult,
_build_opensre_execution_plan,
_classify_opensre_command,
_is_interactive_wizard,
_opensre_confirmation_reason,
_run_opensre_foreground,
_run_opensre_foreground_streaming,
build_opensre_cli_argv,
print_interactive_wizard_handoff,
run_opensre_cli_command,
run_opensre_cli_command_result,
)
from .task_streaming import (
_MAX_COMMAND_OUTPUT_CHARS,
_MIN_SUBPROCESS_TERMINAL_WIDTH,
_SYNTHETIC_DIAG_CHARS,
_SYNTHETIC_POLL_SECONDS,
_TASK_OUTPUT_JOIN_TIMEOUT_SECONDS,
_TASK_OUTPUT_PREFIX_WIDTH,
CLAUDE_CODE_IMPLEMENTATION_TIMEOUT_SECONDS,
SHELL_COMMAND_TIMEOUT_SECONDS,
SYNTHETIC_TEST_TIMEOUT_SECONDS,
_console_file_is_tty,
_join_task_output_streams,
_print_task_output_line,
_pump_task_pty,
_pump_task_stream,
_should_use_pty,
_start_task_output_streams,
_subprocess_env_with_aligned_width,
read_diag,
read_task_output,
terminate_child_process,
)
__all__ = [
"CLAUDE_CODE_IMPLEMENTATION_TIMEOUT_SECONDS",
"SHELL_COMMAND_TIMEOUT_SECONDS",
"SYNTHETIC_TEST_TIMEOUT_SECONDS",
"OpensreCommandClass",
"OpensreExecutionMode",
"OpensreExecutionPlan",
"OpensreRunOutcome",
"OpensreRunResult",
"Path",
"_INTERACTIVE_OPENSRE_COMMAND_PATHS",
"_MAX_COMMAND_OUTPUT_CHARS",
"_MIN_SUBPROCESS_TERMINAL_WIDTH",
"_OPENSRE_BLOCKED_SUBCOMMANDS",
"_SYNTHETIC_DIAG_CHARS",
"_SYNTHETIC_POLL_SECONDS",
"_TASK_OUTPUT_JOIN_TIMEOUT_SECONDS",
"_TASK_OUTPUT_PREFIX_WIDTH",
"_classify_opensre_command",
"_build_opensre_execution_plan",
"_console_file_is_tty",
"_is_interactive_wizard",
"_join_task_output_streams",
"_opensre_confirmation_reason",
"_print_task_output_line",
"_pump_task_pty",
"_pump_task_stream",
"_run_opensre_foreground",
"_run_opensre_foreground_streaming",
"_should_use_pty",
"_start_task_output_streams",
"_subprocess_env_with_aligned_width",
"build_opensre_cli_argv",
"os",
"print_interactive_wizard_handoff",
"read_diag",
"read_task_output",
"run_opensre_cli_command",
"run_opensre_cli_command_result",
"start_background_cli_task",
"subprocess",
"terminate_child_process",
"threading",
"time",
]
@@ -0,0 +1,252 @@
"""Background CLI task launcher — runs subprocesses with streamed output above the prompt."""
from __future__ import annotations
import contextlib
import os
import subprocess
import tempfile
import threading
import time
from typing import Any
from rich.console import Console
from rich.markup import escape
from surfaces.interactive_shell.runtime import Session, TaskKind, TaskRecord
from surfaces.interactive_shell.ui import DIM, ERROR, HIGHLIGHT
from surfaces.interactive_shell.utils.error_handling.exception_reporting import report_exception
from surfaces.interactive_shell.utils.telemetry import PromptRecorder
from .task_streaming import (
_MAX_COMMAND_OUTPUT_CHARS,
_SYNTHETIC_DIAG_CHARS,
_SYNTHETIC_POLL_SECONDS,
SHELL_COMMAND_TIMEOUT_SECONDS,
_join_task_output_streams,
_pump_task_pty,
_should_use_pty,
_sr_resolve,
_start_task_output_streams,
_subprocess_env_with_aligned_width,
read_diag,
read_task_output,
terminate_child_process,
)
def _compose_task_log_response(
*,
headline: str,
stdout_buf: tempfile.SpooledTemporaryFile[bytes] | None, # type: ignore[type-arg]
stderr_buf: tempfile.SpooledTemporaryFile[bytes], # type: ignore[type-arg]
) -> str:
"""Build the prompt-log response text for a finished background task.
Combines a status headline with the captured stdout and stderr so the
flushed ``background_task`` event carries the real command output (the
error text the user sees in the terminal), not an empty assistant reply.
"""
parts = [headline]
stdout_text = read_task_output(stdout_buf, limit=_MAX_COMMAND_OUTPUT_CHARS)
stderr_text = read_task_output(stderr_buf, limit=_MAX_COMMAND_OUTPUT_CHARS)
if stdout_text:
parts.append(stdout_text)
if stderr_text:
parts.append(stderr_text)
return "\n".join(parts)
def start_background_cli_task(
*,
display_command: str,
argv_list: list[str],
session: Session,
console: Console,
timeout_seconds: int = SHELL_COMMAND_TIMEOUT_SECONDS,
kind: TaskKind = TaskKind.CLI_COMMAND,
use_pty: bool = False,
) -> TaskRecord | None:
"""Start a subprocess as a REPL task while streaming output above the prompt."""
console.print(f"[bold]$ {display_command}[/bold]")
task = session.task_registry.create(kind, command=display_command)
task.mark_running()
# Created at launch so the flushed prompt-log latency spans the full task
# duration; the watcher sets the response and flushes once the outcome
# (including any error text) is known. See for_background_task() docstring.
recorder = PromptRecorder.for_background_task(
session=session, command=display_command, task_id=task.task_id
)
stderr_buf: tempfile.SpooledTemporaryFile[bytes] = tempfile.SpooledTemporaryFile( # type: ignore[type-arg]
max_size=_SYNTHETIC_DIAG_CHARS * 2
)
pty_fds: tuple[int, int] | None = None
if _should_use_pty(console, use_pty):
try:
pty_fds = os.openpty()
except OSError:
pty_fds = None
stdout_buf: tempfile.SpooledTemporaryFile[bytes] | None = None # type: ignore[type-arg]
if pty_fds is None:
stdout_buf = tempfile.SpooledTemporaryFile( # type: ignore[type-arg]
max_size=_MAX_COMMAND_OUTPUT_CHARS
)
subprocess_env = _subprocess_env_with_aligned_width(console)
proc: subprocess.Popen[Any]
try:
if pty_fds is None:
proc = subprocess.Popen(
argv_list,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
encoding="utf-8",
errors="replace",
start_new_session=True,
env=subprocess_env,
)
else:
_master_fd, slave_fd = pty_fds
proc = subprocess.Popen(
argv_list,
stdin=subprocess.DEVNULL,
stdout=slave_fd,
stderr=slave_fd,
close_fds=True,
start_new_session=True,
env=subprocess_env,
)
except Exception as exc: # noqa: BLE001
if pty_fds is not None:
for fd in pty_fds:
with contextlib.suppress(OSError):
os.close(fd)
task.mark_failed(str(exc))
if recorder is not None:
with contextlib.suppress(Exception):
recorder.set_error("spawn_failed", str(exc))
recorder.set_response(
_compose_task_log_response(
headline=f"command failed to start: {exc}",
stdout_buf=stdout_buf,
stderr_buf=stderr_buf,
)
)
recorder.flush()
if stdout_buf is not None:
stdout_buf.close()
stderr_buf.close()
report_exception(exc, context="surfaces.interactive_shell.background_cli_task.start")
console.print(f"[{ERROR}]failed to start:[/] {escape(str(exc))}")
return None
task.attach_process(proc)
started_at = time.monotonic()
if pty_fds is None:
output_threads = _start_task_output_streams(
task=task,
proc=proc,
console=console,
stdout_capture=stdout_buf,
stderr_capture=stderr_buf,
)
else:
master_fd, slave_fd = pty_fds
with contextlib.suppress(OSError):
os.close(slave_fd)
output_thread = threading.Thread(
target=_pump_task_pty,
kwargs={"master_fd": master_fd, "console": console, "capture": stderr_buf},
daemon=True,
name=f"task-terminal-{task.task_id}",
)
output_thread.start()
output_threads = [output_thread]
history_gen_when_watch_started = session.terminal.history_generation
def _watch() -> None:
terminated_by_watcher = False
timed_out = False
suggest_follow_up = False
outcome_headline = "command completed (exit 0)"
outcome_error_kind = ""
while proc.poll() is None:
if time.monotonic() - started_at > timeout_seconds:
timed_out = True
task.request_cancel()
terminate_child_process(proc)
terminated_by_watcher = True
break
if task.cancel_requested.is_set():
terminate_child_process(proc)
terminated_by_watcher = True
break
time.sleep(_SYNTHETIC_POLL_SECONDS)
try:
if timed_out:
outcome_headline = f"command timed out after {timeout_seconds} seconds"
outcome_error_kind = "timeout"
task.mark_failed(f"timed out after {timeout_seconds}s")
suggest_follow_up = kind is TaskKind.SYNTHETIC_TEST
return
if terminated_by_watcher and task.cancel_requested.is_set():
outcome_headline = "command cancelled"
task.mark_cancelled()
return
_join_task_output_streams(output_threads)
code = proc.returncode
if code == 0:
task.mark_completed()
else:
diag = _sr_resolve("read_diag", read_diag)(stderr_buf)
error_msg = f"exit code {code}" + (f": {diag}" if diag else "")
outcome_headline = f"command failed (exit {code})"
outcome_error_kind = "cli_exit_nonzero"
task.mark_failed(error_msg)
console.print(f"[{ERROR}]command failed (exit {code}):[/]")
suggest_follow_up = kind is TaskKind.SYNTHETIC_TEST
except Exception as exc: # noqa: BLE001
outcome_headline = f"command error: {exc}"
outcome_error_kind = "watcher_error"
task.mark_failed(str(exc))
report_exception(exc, context="surfaces.interactive_shell.background_cli_task.watch")
console.print(f"[{ERROR}]error:[/] {escape(str(exc))}")
suggest_follow_up = kind is TaskKind.SYNTHETIC_TEST
finally:
_join_task_output_streams(output_threads)
# Flush the prompt-log/PostHog event with the real outcome (stdout,
# stderr, exit/timeout/cancel) before the capture buffers are closed.
if recorder is not None:
with contextlib.suppress(Exception):
if outcome_error_kind:
recorder.set_error(outcome_error_kind, outcome_headline)
recorder.set_response(
_compose_task_log_response(
headline=outcome_headline,
stdout_buf=stdout_buf,
stderr_buf=stderr_buf,
)
)
recorder.flush()
if stdout_buf is not None:
stdout_buf.close()
stderr_buf.close()
if (
suggest_follow_up
and session.terminal.history_generation == history_gen_when_watch_started
):
session.suggest_synthetic_failure_follow_up(label=display_command)
else:
session.terminal.notify_prompt_changed()
thread = threading.Thread(target=_watch, daemon=True)
thread.start()
console.print(
f"[{DIM}]started — task[/] [bold]{escape(task.task_id)}[/bold]. "
f"[{HIGHLIGHT}]/tasks[/] [{DIM}]to monitor,[/] "
f"[{HIGHLIGHT}]/cancel {escape(task.task_id)}[/] [{DIM}]to stop.[/]"
)
return task
@@ -0,0 +1,151 @@
"""OpenSRE CLI command runner — surface adapter over tools.interactive_shell.cli."""
from __future__ import annotations
from collections.abc import Callable
from rich.console import Console
from surfaces.interactive_shell.runtime.subprocess_runner.repl_presenter import make_repl_presenter
from surfaces.interactive_shell.session import Session
from surfaces.interactive_shell.ui import DIM, WARNING
from tools.interactive_shell.cli import (
INTERACTIVE_OPENSRE_COMMAND_PATHS,
OPENSRE_BLOCKED_SUBCOMMANDS,
OpensreCommandClass,
OpensreExecutionMode,
OpensreExecutionPlan,
OpensreRunOutcome,
OpensreRunResult,
_run_foreground_via_presenter,
_run_streaming_via_presenter,
build_opensre_cli_argv,
build_opensre_execution_plan,
classify_opensre_command,
interactive_wizard_handoff_response_text,
is_interactive_wizard,
opensre_confirmation_reason,
)
from tools.interactive_shell.cli import (
run_opensre_cli_command as _run_opensre_cli_command,
)
from tools.interactive_shell.cli import (
run_opensre_cli_command_result as _run_opensre_cli_command_result,
)
# Backward-compatible aliases for tests and slash parity.
_INTERACTIVE_OPENSRE_COMMAND_PATHS = INTERACTIVE_OPENSRE_COMMAND_PATHS
_OPENSRE_BLOCKED_SUBCOMMANDS = OPENSRE_BLOCKED_SUBCOMMANDS
def _is_interactive_wizard(tokens: list[str]) -> bool:
return is_interactive_wizard(tokens)
def _classify_opensre_command(tokens: list[str]) -> str:
return classify_opensre_command(tokens)
def _opensre_confirmation_reason(tokens: list[str]) -> str:
return opensre_confirmation_reason(tokens)
def _build_opensre_execution_plan(tokens: list[str]) -> OpensreExecutionPlan:
return build_opensre_execution_plan(tokens)
def print_interactive_wizard_handoff(console: Console, command_str: str) -> None:
console.print(
f"[{WARNING}]`opensre {command_str}` is an interactive wizard "
"that needs a full terminal.[/]"
)
console.print(
f"[{DIM}]Type [bold]/{command_str}[/bold] directly in this shell to launch it.[/]"
)
def run_opensre_cli_command(
args: str,
session: Session,
console: Console,
*,
confirm_fn: Callable[[str], str] | None = None,
is_tty: bool | None = None,
) -> bool:
presenter = make_repl_presenter(
session,
console,
confirm_fn=confirm_fn,
is_tty=is_tty,
action_already_listed=True,
)
return _run_opensre_cli_command(args, presenter)
def run_opensre_cli_command_result(
args: str,
session: Session,
console: Console,
*,
confirm_fn: Callable[[str], str] | None = None,
is_tty: bool | None = None,
) -> OpensreRunResult:
presenter = make_repl_presenter(
session,
console,
confirm_fn=confirm_fn,
is_tty=is_tty,
action_already_listed=True,
)
return _run_opensre_cli_command_result(args, presenter)
# Foreground helpers kept for monkeypatch tests that patch subprocess_runner paths.
def _run_opensre_foreground(
argv_list: list[str],
display_command: str,
session: Session,
console: Console,
) -> None:
presenter = make_repl_presenter(session, console, action_already_listed=True)
_run_foreground_via_presenter(
presenter,
argv_list=argv_list,
display_command=display_command,
)
def _run_opensre_foreground_streaming(
argv_list: list[str],
display_command: str,
session: Session,
console: Console,
) -> None:
presenter = make_repl_presenter(session, console, action_already_listed=True)
_run_streaming_via_presenter(
presenter,
argv_list=argv_list,
display_command=display_command,
)
__all__ = [
"OpensreCommandClass",
"OpensreExecutionMode",
"OpensreExecutionPlan",
"OpensreRunOutcome",
"OpensreRunResult",
"_INTERACTIVE_OPENSRE_COMMAND_PATHS",
"_OPENSRE_BLOCKED_SUBCOMMANDS",
"_build_opensre_execution_plan",
"_classify_opensre_command",
"_is_interactive_wizard",
"_opensre_confirmation_reason",
"_run_opensre_foreground",
"_run_opensre_foreground_streaming",
"build_opensre_cli_argv",
"interactive_wizard_handoff_response_text",
"print_interactive_wizard_handoff",
"run_opensre_cli_command",
"run_opensre_cli_command_result",
]
@@ -0,0 +1,226 @@
"""REPL subprocess presenter — Rich UI + session hooks for action tools."""
from __future__ import annotations
import re
import subprocess
import tempfile
import threading
from collections.abc import Callable
from typing import Any
from rich.console import Console
from rich.markup import escape
from rich.text import Text
from platform.common.task_types import TaskKind
from surfaces.interactive_shell.session import Session
from surfaces.interactive_shell.ui import DIM, ERROR, HIGHLIGHT, WARNING, print_command_output
from surfaces.interactive_shell.ui.execution_confirm import execution_allowed
from surfaces.interactive_shell.utils.error_handling.exception_reporting import report_exception
from tools.interactive_shell.shared import ExecutionPolicyResult
from tools.interactive_shell.subprocess import SubprocessPresenter, subprocess_env_with_width
from .background_task_executor import (
start_background_cli_task as _start_background_cli_task_default,
)
from .task_streaming import (
_join_task_output_streams,
_sr_resolve,
_start_task_output_streams,
)
_MARKUP_STYLE_ALIASES: dict[str, str] = {
"error": str(ERROR),
"dim": str(DIM),
"highlight": str(HIGHLIGHT),
"warning": str(WARNING),
}
# Intentional Rich markup tags used by subprocess presenters and action tools.
_ALLOWED_MARKUP_TAG = re.compile(
r"(\["
r"(?:"
r"#[0-9A-Fa-f]{6}|"
r"bold|dim|highlight|warning|error|"
r"/(?:bold|dim|highlight|warning|error)?"
r")"
r"\])"
)
_MARKUP_HINT = re.compile(r"\[(?:/?(?:error|dim|highlight|warning|bold)|/)\]")
def _expand_markup_aliases(message: str) -> str:
for alias, token in _MARKUP_STYLE_ALIASES.items():
message = message.replace(f"[{alias}]", f"[{token}]")
message = message.replace(f"[/{alias}]", f"[/{token}]")
return message
def _message_uses_intentional_markup(message: str) -> bool:
if _MARKUP_HINT.search(message):
return True
return any(
f"[{token}]" in message or f"[/{token}]" in message
for token in _MARKUP_STYLE_ALIASES.values()
)
def _escape_markup_message(message: str) -> str:
"""Escape plain-text segments while preserving intentional Rich markup tags."""
expanded = _expand_markup_aliases(message)
if not _message_uses_intentional_markup(expanded):
return escape(expanded)
parts = _ALLOWED_MARKUP_TAG.split(expanded)
if len(parts) == 1:
return escape(expanded)
rendered: list[str] = []
for index, part in enumerate(parts):
if index % 2 == 1:
rendered.append(part)
elif part:
rendered.append(escape(part))
return "".join(rendered)
class ReplSubprocessPresenter:
"""Surface implementation of :class:`SubprocessPresenter` for the interactive REPL."""
def __init__(
self,
session: Session,
console: Console,
*,
confirm_fn: Callable[[str], str] | None = None,
is_tty: bool | None = None,
action_already_listed: bool = False,
) -> None:
self._session = session
self._console = console
self._confirm_fn = confirm_fn
self._is_tty = is_tty
self._action_already_listed = action_already_listed
@property
def session(self) -> Session:
return self._session
@property
def console(self) -> Console:
return self._console
def execution_allowed(
self,
policy: ExecutionPolicyResult,
*,
action_summary: str,
) -> bool:
return execution_allowed(
policy,
session=self._session,
console=self._console,
action_summary=action_summary,
confirm_fn=self._confirm_fn,
is_tty=self._is_tty,
action_already_listed=self._action_already_listed,
)
def print(self, message: str = "") -> None:
self._console.print(_escape_markup_message(message))
def print_bold_command(self, display_command: str) -> None:
self._console.print(f"[bold]$ {escape(display_command)}[/bold]")
def print_command_output(self, text: str, *, style: str | None = None) -> None:
resolved: str | None
if style in _MARKUP_STYLE_ALIASES:
resolved = _MARKUP_STYLE_ALIASES[style]
elif style is None:
resolved = None
else:
resolved = style
print_command_output(self._console, text, style=resolved)
def print_plain(self, text: str) -> None:
self._console.print(Text(text))
def report_exception(self, exc: BaseException, *, context: str) -> None:
report_exception(exc, context=context)
def subprocess_env(self) -> dict[str, str]:
return subprocess_env_with_width(
columns=self._console.size.width or 80,
lines=self._console.size.height,
)
def start_task_output_streams(
self,
*,
task: Any,
proc: subprocess.Popen[Any],
stdout_capture: tempfile.SpooledTemporaryFile[bytes] | None = None, # type: ignore[type-arg]
stderr_capture: tempfile.SpooledTemporaryFile[bytes] | None = None, # type: ignore[type-arg]
) -> list[threading.Thread]:
return _start_task_output_streams(
task=task,
proc=proc,
console=self._console,
stdout_capture=stdout_capture,
stderr_capture=stderr_capture,
)
def join_task_output_streams(self, threads: list[threading.Thread]) -> None:
_join_task_output_streams(threads)
def start_background_cli_task(
self,
*,
display_command: str,
argv_list: list[str],
timeout_seconds: int,
kind: TaskKind = TaskKind.CLI_COMMAND,
use_pty: bool = False,
) -> Any:
starter = _sr_resolve("start_background_cli_task", _start_background_cli_task_default)
return starter(
display_command=display_command,
argv_list=argv_list,
session=self._session,
console=self._console,
timeout_seconds=timeout_seconds,
kind=kind,
use_pty=use_pty,
)
def print_error(self, message: str) -> None:
self._console.print(f"[{ERROR}]{escape(message)}[/]")
def print_dim(self, message: str) -> None:
self._console.print(f"[{DIM}]{escape(message)}[/]")
def print_highlight(self, message: str) -> None:
self._console.print(f"[{HIGHLIGHT}]{escape(message)}[/]")
def print_warning(self, message: str) -> None:
self._console.print(f"[{WARNING}]{escape(message)}[/]")
def make_repl_presenter(
session: Session,
console: Console,
*,
confirm_fn: Callable[[str], str] | None = None,
is_tty: bool | None = None,
action_already_listed: bool = False,
) -> SubprocessPresenter:
"""Construct a :class:`ReplSubprocessPresenter` for runners and tests."""
return ReplSubprocessPresenter(
session,
console,
confirm_fn=confirm_fn,
is_tty=is_tty,
action_already_listed=action_already_listed,
)
__all__ = ["ReplSubprocessPresenter", "make_repl_presenter"]
@@ -0,0 +1,202 @@
"""Shared subprocess-streaming primitives, PTY helpers, and module-wide constants."""
from __future__ import annotations
import contextlib
import errno
import os
import subprocess
import sys
import tempfile
import threading
from typing import IO, Any
from rich.console import Console
from rich.markup import escape
from rich.text import Text
from platform.common.task_types import TaskRecord
from surfaces.interactive_shell.ui import DIM, ERROR
from surfaces.interactive_shell.utils.error_handling.exception_reporting import report_exception
from tools.interactive_shell.subprocess import (
CLAUDE_CODE_IMPLEMENTATION_TIMEOUT_SECONDS,
MAX_COMMAND_OUTPUT_CHARS,
MIN_SUBPROCESS_TERMINAL_WIDTH,
SHELL_COMMAND_TIMEOUT_SECONDS,
SYNTHETIC_DIAG_CHARS,
SYNTHETIC_POLL_SECONDS,
SYNTHETIC_TEST_TIMEOUT_SECONDS,
TASK_OUTPUT_JOIN_TIMEOUT_SECONDS,
TASK_OUTPUT_PREFIX_WIDTH,
read_diag,
read_task_output,
subprocess_env_with_width,
terminate_child_process,
)
# Full dotted name of the ``subprocess_runner`` package. Submodules use this to
# look up patchable names from the parent namespace at call time so that tests
# using ``monkeypatch.setattr("…subprocess_runner.X", fake)`` take effect even
# when the implementation lives in a submodule.
_SUBPROCESS_RUNNER_MODULE = "surfaces.interactive_shell.runtime.subprocess_runner"
# Backward-compatible aliases for tests and callers using underscore-prefixed names.
_MAX_COMMAND_OUTPUT_CHARS = MAX_COMMAND_OUTPUT_CHARS
_SYNTHETIC_POLL_SECONDS = SYNTHETIC_POLL_SECONDS
_SYNTHETIC_DIAG_CHARS = SYNTHETIC_DIAG_CHARS
_MIN_SUBPROCESS_TERMINAL_WIDTH = MIN_SUBPROCESS_TERMINAL_WIDTH
_TASK_OUTPUT_PREFIX_WIDTH = TASK_OUTPUT_PREFIX_WIDTH
_TASK_OUTPUT_JOIN_TIMEOUT_SECONDS = TASK_OUTPUT_JOIN_TIMEOUT_SECONDS
def _sr_resolve(name: str, default: Any) -> Any:
"""Return ``subprocess_runner.<name>`` if the package is loaded, else ``default``.
Used by submodules to honour monkeypatches applied to the parent package
namespace (e.g. ``monkeypatch.setattr("…subprocess_runner.read_diag", …)``).
"""
sr = sys.modules.get(_SUBPROCESS_RUNNER_MODULE)
return getattr(sr, name, default) if sr is not None else default
def _print_task_output_line(
console: Console,
task: TaskRecord,
stream_name: str,
line: str,
*,
style: str | None = None,
) -> None:
text = Text()
text.append(f"{task.task_id} {stream_name}", style=DIM)
text.append(line.rstrip("\r\n"), style=style)
console.print(text)
def _subprocess_env_with_aligned_width(console: Console) -> dict[str, str]:
"""Return ``os.environ`` patched so a piped Rich subprocess wraps to fit."""
user_width = console.size.width or _MIN_SUBPROCESS_TERMINAL_WIDTH + _TASK_OUTPUT_PREFIX_WIDTH
return subprocess_env_with_width(
columns=user_width,
lines=console.size.height,
)
def _pump_task_stream(
*,
task: TaskRecord,
stream_name: str,
stream: IO[str],
console: Console,
style: str | None = None,
capture: tempfile.SpooledTemporaryFile[bytes] | None = None, # type: ignore[type-arg]
) -> None:
try:
for line in stream:
if capture is not None:
capture.write(line.encode("utf-8", errors="replace"))
if line.strip():
_print_task_output_line(console, task, stream_name, line, style=style)
task.update_progress(line)
except Exception as exc: # noqa: BLE001
report_exception(exc, context=f"surfaces.interactive_shell.task_stream.{stream_name}")
console.print(f"[{DIM}]task output stream ended unexpectedly:[/] {escape(str(exc))}")
def _start_task_output_streams(
*,
task: TaskRecord,
proc: subprocess.Popen[Any],
console: Console,
stdout_capture: tempfile.SpooledTemporaryFile[bytes] | None = None, # type: ignore[type-arg]
stderr_capture: tempfile.SpooledTemporaryFile[bytes] | None = None, # type: ignore[type-arg]
) -> list[threading.Thread]:
threads: list[threading.Thread] = []
streams: tuple[tuple[str, IO[str] | None, str | None, Any], ...] = (
("stdout", proc.stdout, None, stdout_capture),
("stderr", proc.stderr, ERROR, stderr_capture),
)
for stream_name, stream, style, capture in streams:
if stream is None:
continue
thread = threading.Thread(
target=_pump_task_stream,
kwargs={
"task": task,
"stream_name": stream_name,
"stream": stream,
"console": console,
"style": style,
"capture": capture,
},
daemon=True,
name=f"task-output-{task.task_id}-{stream_name}",
)
thread.start()
threads.append(thread)
return threads
def _join_task_output_streams(threads: list[threading.Thread]) -> None:
for thread in threads:
thread.join(timeout=TASK_OUTPUT_JOIN_TIMEOUT_SECONDS)
def _console_file_is_tty(console: Console) -> bool:
isatty = getattr(console.file, "isatty", None)
return bool(isatty and isatty())
def _should_use_pty(console: Console, requested: bool) -> bool:
return requested and hasattr(os, "openpty") and _console_file_is_tty(console)
def _pump_task_pty(
*,
master_fd: int,
console: Console,
capture: tempfile.SpooledTemporaryFile[bytes], # type: ignore[type-arg]
) -> None:
try:
while True:
try:
chunk = os.read(master_fd, 4096)
except OSError as exc:
if exc.errno == errno.EIO:
break
raise
if not chunk:
break
capture.write(chunk)
console.file.write(chunk.decode("utf-8", errors="replace"))
console.file.flush()
except Exception as exc: # noqa: BLE001
report_exception(exc, context="surfaces.interactive_shell.task_pty_stream")
console.print(f"[{DIM}]task terminal stream ended unexpectedly:[/] {escape(str(exc))}")
finally:
with contextlib.suppress(OSError):
os.close(master_fd)
__all__ = [
"SHELL_COMMAND_TIMEOUT_SECONDS",
"SYNTHETIC_TEST_TIMEOUT_SECONDS",
"CLAUDE_CODE_IMPLEMENTATION_TIMEOUT_SECONDS",
"_SYNTHETIC_POLL_SECONDS",
"_MAX_COMMAND_OUTPUT_CHARS",
"_SYNTHETIC_DIAG_CHARS",
"_TASK_OUTPUT_PREFIX_WIDTH",
"_MIN_SUBPROCESS_TERMINAL_WIDTH",
"_TASK_OUTPUT_JOIN_TIMEOUT_SECONDS",
"terminate_child_process",
"read_diag",
"read_task_output",
"_print_task_output_line",
"_subprocess_env_with_aligned_width",
"_pump_task_stream",
"_start_task_output_streams",
"_join_task_output_streams",
"_console_file_is_tty",
"_should_use_pty",
"_pump_task_pty",
]
@@ -0,0 +1,250 @@
"""Runtime turn host for submitted interactive-shell prompts.
Three public runtime functions live here:
- ``run_agent_turn`` — set up shell presentation for one submitted turn and drive
its lifecycle (the injected ``run_turn`` callable for the queue).
- ``run_input_loop`` — read prompt input events and dispatch them until exit.
- ``run_agent_turn_queue`` — consume queued turns and run each one until exit.
"""
from __future__ import annotations
import asyncio
import contextlib
import logging
import threading
from collections.abc import Awaitable, Callable, Coroutine
from dataclasses import dataclass
from typing import Any
from rich.console import Console
from platform.analytics.repl_context import bound_repl_turn_context
from platform.observability.trace.spans import bind_session_trace, emit_thread_boundary
from surfaces.interactive_shell.runtime.agent_presentation import (
AgentEvent,
AgentEventSink,
ConsoleAgentEventSink,
)
from surfaces.interactive_shell.runtime.background.workers import BackgroundTaskManager
from surfaces.interactive_shell.runtime.core.confirmation import (
DispatchCancelled,
request_confirmation_via_prompt,
)
from surfaces.interactive_shell.runtime.core.state import ReplState, SpinnerState
from surfaces.interactive_shell.runtime.input import PromptInputReader
from surfaces.interactive_shell.runtime.input.actions import (
InputAction,
ShellInputSnapshot,
decide_input_action,
)
from surfaces.interactive_shell.runtime.utils.input_policy import (
turn_needs_exclusive_stdin,
)
from surfaces.interactive_shell.session import Session
from surfaces.interactive_shell.ui.output.console_state import set_investigation_spinner
from surfaces.interactive_shell.ui.output.repl_progress import repl_safe_progress_scope
from surfaces.interactive_shell.ui.streaming.console import StreamingConsole
from surfaces.interactive_shell.utils.error_handling.exception_reporting import report_exception
from surfaces.interactive_shell.utils.telemetry import PromptRecorder
_logger = logging.getLogger(__name__)
_AGENT_TURN_KIND = "agent"
@dataclass(frozen=True)
class AgentTurnRuntime:
"""Immutable dependencies for running one submitted shell turn."""
session: Session
state: ReplState
spinner: SpinnerState
invalidate_prompt: Callable[[], None]
request_exit: Callable[[], None] | None = None
async def run_agent_turn(runtime: AgentTurnRuntime, text: str) -> None:
"""Set up shell presentation for one turn and drive its lifecycle."""
dispatch_cancel = threading.Event()
console = StreamingConsole(
runtime.spinner,
dispatch_cancel,
highlight=False,
force_terminal=True,
color_system="truecolor",
legacy_windows=False,
)
emit = ConsoleAgentEventSink(
session=runtime.session,
spinner=runtime.spinner,
console=console,
)
recorder = PromptRecorder.start(
session=runtime.session,
text=text,
turn_kind=_AGENT_TURN_KIND,
)
exclusive_stdin = turn_needs_exclusive_stdin(text, runtime.session)
progress_scope = contextlib.nullcontext() if exclusive_stdin else repl_safe_progress_scope()
runtime.session.terminal.exclusive_stdin_active = exclusive_stdin
# Expose this turn's spinner so investigation stages can animate phase labels.
set_investigation_spinner(runtime.spinner)
emit_thread_boundary(
runtime.session.session_id,
name="turn_boundary",
phase="turn_start",
)
try:
with (
bind_session_trace(runtime.session.session_id),
progress_scope,
):
await _run_agent_turn_loop(
runtime=runtime,
text=text,
output=console,
recorder=recorder,
confirm=lambda prompt: request_confirmation_via_prompt(runtime.state, prompt),
emit=emit,
dispatch_cancel=dispatch_cancel,
)
finally:
set_investigation_spinner(None)
runtime.session.terminal.exclusive_stdin_active = False
emit_thread_boundary(
runtime.session.session_id,
name="turn_boundary",
phase="turn_end",
)
async def _run_agent_turn_loop(
*,
runtime: AgentTurnRuntime,
text: str,
output: StreamingConsole,
recorder: PromptRecorder | None,
confirm: Callable[[str], str],
emit: AgentEventSink,
dispatch_cancel: threading.Event,
) -> None:
current_task = asyncio.current_task()
if current_task is not None:
runtime.state.start_dispatch(task=current_task, cancel_event=dispatch_cancel)
else:
runtime.state.attach_cancel_event(dispatch_cancel)
await emit(AgentEvent(type="turn_start", text=text))
try:
# Imported lazily so constructing the controller (and importing this
# module) does not pull the harness/turn-execution stack
# (``action_agent -> core.agent``) before the first turn is queued.
from surfaces.interactive_shell.runtime.shell_turn_execution import execute_shell_turn
with bound_repl_turn_context(
session_id=runtime.session.session_id,
turn_kind=_AGENT_TURN_KIND,
prompt_turn_id=recorder.turn_id if recorder is not None else None,
):
await asyncio.to_thread(
execute_shell_turn,
text,
runtime.session,
output,
recorder=recorder,
confirm_fn=confirm,
is_tty=None,
request_exit=runtime.request_exit,
)
except asyncio.CancelledError:
await emit(AgentEvent(type="turn_interrupted"))
raise
except DispatchCancelled:
await emit(AgentEvent(type="turn_interrupted"))
except Exception as exc:
report_exception(exc, context="surfaces.interactive_shell.turn")
await emit(AgentEvent(type="turn_error", error=exc))
finally:
runtime.state.finish_dispatch(dispatch_cancel)
await emit(AgentEvent(type="turn_end"))
async def run_input_loop(
*,
state: ReplState,
session: Session,
background: BackgroundTaskManager | None,
input_reader: PromptInputReader,
echo_console: Console,
handle_input_action: Callable[[InputAction], Awaitable[bool]],
) -> None:
"""Run the interactive session's main input loop until exit or close.
This loop reads input; it does not run agent turns itself. Each raw input
event is classified into an ``InputAction`` by ``decide_input_action`` and
handed to ``handle_input_action``. For a submitted prompt that handler pushes
the text onto ``state.queue``; the queued text is then consumed
asynchronously by ``run_agent_turn_queue`` (started in the controller's
``_start_runtime_services``), which runs each turn via ``run_agent_turn``.
Keeping input reading and turn execution as two separate loops joined only by
``state.queue`` is deliberate: it lets the user keep typing, cancel, or
answer a confirmation while a turn is still in flight.
"""
while not state.exit_requested:
if background is not None:
background.drain_turn_start_output(echo_console)
event = await input_reader.read()
action = decide_input_action(
event,
ShellInputSnapshot(
exit_requested=state.exit_requested,
dispatch_running=state.is_dispatch_running(),
awaiting_confirmation=state.is_awaiting_confirmation(),
),
needs_exclusive_stdin=lambda text: turn_needs_exclusive_stdin(
text,
session,
),
)
should_continue = await handle_input_action(action)
if not should_continue:
return
async def run_agent_turn_queue(
*,
state: ReplState,
run_turn: Callable[[str], Coroutine[Any, Any, None]],
) -> None:
"""Consume queued turns and run each one until exit."""
while not state.exit_requested:
try:
text = await state.queue.get()
except asyncio.CancelledError:
return
if state.exit_requested:
state.queue.task_done()
return
turn_task = asyncio.create_task(run_turn(text))
state.attach_turn_task(turn_task)
try:
await turn_task
except asyncio.CancelledError:
_logger.debug("Queued turn task was cancelled")
except Exception as exc:
_logger.debug("Queued turn task ended with exception: %s", exc)
finally:
state.clear_current_task()
state.queue.task_done()
__all__ = [
"AgentTurnRuntime",
"run_agent_turn",
"run_agent_turn_queue",
"run_input_loop",
]
@@ -0,0 +1,102 @@
"""Injection contracts for the interactive-shell turn seams.
These protocols describe exactly what ``execute_shell_turn`` requires from the
action / gather / answer adapters it composes, so an injected test double is
checked at type-time rather than at runtime. The default adapters
(``action_turn``, ``answer_turn``, ``integration_tool_gathering``) satisfy them.
"""
from __future__ import annotations
from collections.abc import Callable
from typing import Any, Protocol, TypedDict
from rich.console import Console
from core.agent_harness.ports import OutputSink
from core.agent_harness.turns.turn_plan import TurnPlan
from core.agent_harness.turns.turn_results import ToolCallingTurnResult
from core.execution import ToolExecutionHooks
from surfaces.interactive_shell.session import Session
from surfaces.interactive_shell.utils.telemetry import LlmRunInfo
class RunActionToolTurn(Protocol):
"""Action-selection seam driven by ``execute_shell_turn``.
``deps`` is intentionally not part of the contract: ``execute_shell_turn``
never injects it, and the default adapter supplies its own LLM factory.
"""
def __call__(
self,
message: str,
session: Session,
console: Console,
*,
confirm_fn: Callable[[str], str] | None = None,
is_tty: bool | None = None,
request_exit: Callable[[], None] | None = None,
turn_plan: TurnPlan | None = None,
output: OutputSink | None = None,
tool_hooks: ToolExecutionHooks | None = None,
) -> ToolCallingTurnResult:
"""Run one action turn and return its facts."""
class GatherEvidence(Protocol):
"""Gather seam: collect read-only integration evidence, or None."""
def __call__(
self,
message: str,
session: Session,
console: Console,
*,
is_tty: bool | None = None,
resolved_integrations: dict[str, Any] | None = None,
) -> str | None:
"""Gather evidence for the message, or return None when nothing applies."""
class AnswerKwargs(TypedDict, total=False):
"""Keyword args ``run_turn`` forwards to the answer seam (all optional).
``total=False`` mirrors ``run_turn`` omitting ``tool_observation_on_screen``
on the plain (no-evidence) path.
"""
confirm_fn: Callable[[str], str] | None
is_tty: bool | None
tool_observation: str | None
tool_observation_on_screen: bool
handoff_contents: tuple[str, ...]
turn_plan: TurnPlan | None
class AnswerShellQuestion(Protocol):
"""Answer seam: respond via the grounded conversational assistant."""
def __call__(
self,
message: str,
session: Session,
console: Console,
*,
confirm_fn: Callable[[str], str] | None = None,
is_tty: bool | None = None,
tool_observation: str | None = None,
tool_observation_on_screen: bool = True,
handoff_contents: tuple[str, ...] = (),
turn_plan: TurnPlan | None = None,
output: OutputSink | None = None,
) -> LlmRunInfo | None:
"""Answer the question, returning the LLM run info or None."""
__all__ = [
"AnswerKwargs",
"AnswerShellQuestion",
"GatherEvidence",
"RunActionToolTurn",
]
@@ -0,0 +1 @@
"""Runtime utility helpers for interactive shell orchestration."""
@@ -0,0 +1,120 @@
"""Prompt input and stdin coordination policy for runtime turns."""
from __future__ import annotations
from surfaces.interactive_shell.session import Session
from surfaces.interactive_shell.ui.components.choice_menu import repl_tty_interactive
def _literal_slash_command_text(text: str) -> str | None:
"""Return literal ``/slash`` command text for command-shaped input, else ``None``.
Terminal-UI policy only (spinner suppression and exclusive-stdin gating). The
matching execution-side deterministic dispatch lives separately in
``core/agent_harness/turns/action_driver.py``; keep this function UI-only and do not
grow natural-language intent inference here.
"""
stripped = text.strip()
return stripped if stripped.startswith("/") else None
_EXCLUSIVE_STDIN_MENU_COMMANDS: frozenset[str] = frozenset(
{
"/history",
"/auth",
"/help",
"/integrations",
"/investigate",
"/mcp",
"/model",
"/tools",
"/template",
"/trust",
"/verbose",
"/?",
# Table-outputting commands must complete before the next prompt_async()
# starts, otherwise patch_stdout redraws trigger ESC[6n DSR queries whose
# CPR responses land as literal keystrokes in the incoming prompt buffer.
"/doctor",
"/version",
"/verify",
"/status",
"/cost",
"/tasks",
"/watches",
"/alerts",
"/privacy",
"/context",
"/fleet",
"/compact",
"/welcome",
"/sessions",
"/resume",
"/new",
"/rca",
}
)
_EXCLUSIVE_STDIN_SUBCOMMANDS: frozenset[tuple[str, str]] = frozenset(
{
("/integrations", "setup"),
# ``remove`` drives a native inline arrow-key picker (raw os.read on
# stdin). Without exclusive stdin the concurrent prompt_async() steals
# keystrokes and CPR responses leak into the next prompt buffer.
("/integrations", "remove"),
("/mcp", "connect"),
("/mcp", "disconnect"),
("/rca", "history"),
("/rca", "list"),
("/rca", "ls"),
("/rca", "show"),
("/rca", "save"),
}
)
_WAIT_FOR_COMPLETION_COMMANDS: frozenset[str] = frozenset(
{"/exit", "/quit", "/update", "/onboard", "/config", "/auth", "/login"}
)
def turn_should_show_spinner(text: str, _session: Session) -> bool:
# UI-only: suppress the "thinking" spinner for literal slash commands, which
# dispatch deterministically (no LLM) and would otherwise show a misleading
# spinner. Natural-language turns still go through the action-agent LLM.
return _literal_slash_command_text(text.strip()) is None
def turn_needs_exclusive_stdin(text: str, _session: Session) -> bool:
if not repl_tty_interactive():
return False
t = text.strip()
if not t:
return False
# Reserve stdin early for literal command-shaped input, but do not dispatch
# here. This stays UI-only; deterministic slash execution lives in the turn
# engine (core/agent_harness/turns/action_driver.py), not in this gating layer.
dispatch_text = _literal_slash_command_text(t)
if dispatch_text is None:
return False
parts = dispatch_text.split()
if not parts:
return False
name = parts[0].lower()
args = [arg.lower() for arg in parts[1:]]
if name in _WAIT_FOR_COMPLETION_COMMANDS:
return True
if name == "/theme":
return True
if name in _EXCLUSIVE_STDIN_MENU_COMMANDS and not args:
return True
if name == "/tests" and not args:
return True
return bool(args and (name, args[0]) in _EXCLUSIVE_STDIN_SUBCOMMANDS)
__all__ = [
"turn_needs_exclusive_stdin",
"turn_should_show_spinner",
]