chore: import upstream snapshot with attribution

plugins/plugin-computeruse/AGENTS.md

@@ -0,0 +1,301 @@
+# @elizaos/plugin-computeruse
+
+Desktop automation for elizaOS agents — screenshots, mouse/keyboard control, browser CDP automation, window management, and a multi-display scene model.
+
+Ported from [`coasty-ai/open-computer-use`](https://github.com/coasty-ai/open-computer-use) (Apache 2.0).
+
+## Purpose / role
+
+Adds real desktop control to an Eliza agent: taking screenshots, clicking/typing/scrolling, managing windows, automating web browsers via CDP (puppeteer-core), and building a structured `Scene` from displays, accessibility tree, OCR, and process list so the agent can reason about what is visible. Opt-in: the plugin auto-enables only when `COMPUTER_USE_ENABLED=1` is set (see `autoEnable` in `src/index.ts`). Requires a headful display session on macOS/Linux; headless browser mode is supported.
+
+File operations belong to the FILE action; shell/terminal access belongs to the SHELL action — this plugin does not expose them.
+
+## Plugin surface
+
+### Actions
+
+| Name | File | What it does |
+|------|------|--------------|
+| `COMPUTER_USE` | `src/actions/use-computer.ts` | Umbrella desktop action: `screenshot`, `click`, `click_with_modifiers`, `double_click`, `right_click`, `mouse_move`, `middle_click`, `mouse_down`, `mouse_up`, `type`, `key`, `key_combo`, `key_down`, `key_up`, `scroll`, `drag` (single segment or multi-point `path`), `get_cursor_position`, `detect_elements`, `ocr`, `open` (file/URL/folder via the OS default handler), `launch` (start an app → returns pid). `mouse_down/up` + `key_down/up` are real press-and-hold primitives (nutjs `pressButton`/`pressKey` without release — back hold-drags, marquee, held modifiers); they require the nutjs driver. Requires `OWNER` role. Subactions are promoted to virtual top-level actions (`COMPUTER_USE_CLICK`, etc.) via `promoteSubactionsToActions`. |
+| `WINDOW` | `src/actions/window.ts` | Window management: `list`, `focus`, `switch`, `arrange`, `move`, `minimize`, `maximize`, `restore`, `close`, `get_current_window_id`, `get_application_windows`, `set_bounds` (position + size). Also promoted to `WINDOW_FOCUS`, etc. |
+| `CLIPBOARD` | `src/actions/clipboard.ts` | Host clipboard read/write (`read`, `write`) — trycua/cua parity. Promoted to `CLIPBOARD_READ` / `CLIPBOARD_WRITE`. macOS pbcopy/pbpaste, Linux wl-clipboard/xclip, Windows PowerShell `Get-Clipboard` / `Set-Clipboard`. |
+| `COMPUTER_USE_AGENT` | `src/actions/use-computer-agent.ts` | High-level "give me a goal, click my way there" loop (WS7). Selects an agent loop by model string (#9170 M10, default `local-grounder` = Brain → Cascade) and runs predictStep → dispatch up to `maxSteps`, through a callback-middleware pipeline (#9170 M11: operator-normalizer + trajectory by default; `maxDurationMs` budget cap + `imageRetentionLast` window opt-in). Emits trajectory events as structured log lines and on `report.trajectory`. |
+
+### Providers
+
+| Name | File | What it injects |
+|------|------|----------------|
+| `computerState` | `src/providers/computer-state.ts` | Platform info, screen dimensions, available capabilities, recent actions, approval queue. Gate: `browser/files/terminal/automation/admin` contexts. |
+| `scene` | `src/providers/scene.ts` | Live desktop scene (displays, focused window, apps, OCR boxes, AX nodes, VLM annotations) via `SceneBuilder`. Refreshed once per turn; serialized as token-efficient JSON fence. Gate: `browser/automation/admin` contexts. |
+
+### Services
+
+| Name | `serviceType` | File | What it does |
+|------|--------------|------|--------------|
+| `ComputerUseService` | `"computeruse"` | `src/services/computer-use-service.ts` | Central service: input dispatch, screenshot capture, browser CDP session, window ops, approval-manager wiring, `SceneBuilder` lifecycle. |
+| `VisionContextProvider` | `"vision-context"` | `src/services/vision-context-provider.ts` | Surfaces a `VisionContext` snapshot (open apps, focused window, recent actions, current task goal) for downstream consumers (e.g. plugin-vision). |
+
+### Routes
+
+All paths are under `/api/computer-use/` and implemented in `src/routes/computer-use-compat-routes.ts`:
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/computer-use/approvals` | List pending approval requests |
+| GET | `/api/computer-use/approvals/stream` | SSE stream of approval events (public) |
+| POST | `/api/computer-use/approval-mode` | Change the active `COMPUTER_USE_APPROVAL_MODE` |
+| POST | `/api/computer-use/approvals/:id` | Approve or deny a pending action |
+
+## Layout
+
+```
+src/
+  index.ts                   Plugin entry — assembles and exports computerUsePlugin
+  types.ts                   All shared types (ApprovalMode, ComputerUseConfig, DesktopActionParams, …)
+  approval-manager.ts        ComputerUseApprovalManager — queues pending actions, applies approval mode
+  register-routes.ts         Route registration helper
+
+  actions/
+    use-computer.ts          COMPUTER_USE parent action + subaction table
+    use-computer-agent.ts    COMPUTER_USE_AGENT (WS7 autonomous loop)
+    window.ts                WINDOW parent action
+    window-handlers.ts       Per-verb handlers called by window.ts
+    clipboard.ts             clipboardAction (CLIPBOARD parent action) — registered in index.ts; promoted to CLIPBOARD_READ / CLIPBOARD_WRITE
+    helpers.ts               resolveActionParams, buildScreenshotAttachment, …
+
+  actor/                     WS7 autonomous desktop loop
+    brain.ts                 Brain — sends display PNGs to IMAGE_DESCRIPTION model, produces BrainOutput
+    cascade.ts               Cascade (ScreenSeekeR) — Brain → Actor → ProposedAction
+    actor.ts                 Actor interface + OsAtlasProActor + OcrCoordinateGroundingActor
+    dispatch.ts              dispatch() — executes a ProposedAction via ComputerInterface
+    computer-interface.ts    ComputerInterface abstraction + makeComputerInterface()
+    aosp-input-actor.ts      AOSP-specific actor
+    types.ts                 BrainOutput, ProposedAction, …
+    index.ts                 Public re-exports
+
+  platform/
+    browser.ts               Puppeteer-core CDP browser automation
+    capture.ts               captureDisplay / captureAllDisplays
+    displays.ts              listDisplays / getPrimaryDisplay
+    driver.ts                driverClick / driverType / … (nutjs or legacy shell)
+    nut-driver.ts            @nut-tree-fork/nut-js implementation
+    windows-list.ts          listWindows / focusWindow / arrangeWindows / getActiveWindow / resizeWindow / …
+    launch.ts                openTarget (open file/URL/folder) / launchApp (spawn app → pid)
+    clipboard.ts             OS clipboard read/write
+    a11y.ts                  Accessibility tree query
+    coords.ts                localToGlobal coordinate translation
+    wayland-portal.ts        xdg-desktop-portal screenshot sidecar for Wayland
+    capabilities.ts          detectPlatformCapabilities
+    desktop.ts               High-level desktop helpers
+    helpers.ts               Shared platform utilities
+    permissions.ts           classifyPermissionDeniedError
+    process-list.ts          listProcesses / parsePsOutput
+    screenshot-quality.ts    Quality / compression settings for screenshots
+    terminal.ts              Terminal session management (internal; not exposed as action)
+    file-ops.ts              File primitives (internal; not exposed as action)
+    screenshot.ts            Low-level screencapture wrappers
+    security.ts              Path + command security checks
+
+  providers/
+    computer-state.ts        computerStateProvider
+    scene.ts                 sceneProvider
+
+  routes/
+    computer-use-routes.ts       handleComputerUseRoutes (full route table)
+    computer-use-compat-routes.ts  computerUseRouteHandler() (compat wrapper used by plugin entry)
+    sandbox-routes.ts            handleSandboxRoute
+
+  scene/
+    scene-builder.ts         SceneBuilder — composes displays + a11y + OCR into Scene
+    scene-types.ts           Scene, SceneApp, SceneAppWindow, SceneAxNode, …
+    a11y-provider.ts         DarwinAccessibilityProvider / LinuxAccessibilityProvider / …
+    apps.ts                  enumerateApps / joinAppsAndWindows
+    dhash.ts                 Perceptual hash / dirty-block diffing for change detection
+    screen-state.ts          ScreenState + ScreenStateStore — one shared capture/turn (dHash/blockGrid + change events)
+    ocr-adapter.ts           OcrProvider / CoordOcrProvider adapter seam
+    serialize.ts             serializeSceneForPrompt — token-efficient JSON fence
+
+  services/
+    computer-use-service.ts  ComputerUseService (serviceType = "computeruse")
+    vision-context-provider.ts  VisionContextProvider (serviceType = "vision-context")
+    desktop-control.ts       Low-level desktop control primitives + DesktopControl* types
+    index.ts                 Barrel re-exports for services/
+
+  mobile/
+    ocr-provider.ts          OcrProvider / CoordOcrProvider interfaces (plugin-vision contributes impls)
+    ios-bridge.ts            iOS computer-use bridge
+    ios-computer-interface.ts  iOS-specific ComputerInterface implementation
+    ios-app-intent-registry.ts  Registry of iOS app intents for automation
+    android-bridge.ts        AOSP input bridge
+    android-scene.ts         Android scene capture and representation
+    android-trajectory.ts    Android action trajectory recording
+    mobile-computer-interface.ts  MobileComputerInterface
+    mobile-screen-capture.ts Screen capture abstraction for mobile targets
+    index.ts                 Mobile public surface
+
+  osworld/
+    adapter.ts               OSWorld benchmark adapter
+    action-converter.ts      OSWorld action → ComputerInterface translation
+    types.ts                 OSWorld-specific type definitions
+
+  parity/                    trycua/cua parity tooling (#9170 M14)
+    parity-matrix.ts         machine-checkable capability matrix + validateParityMatrix (guards drift vs the live action surface)
+    screenspot.ts            ScreenSpot grounding harness (point-in-bbox scorer + score fold)
+    index.ts                 Public re-exports
+
+  sandbox/
+    sandbox-driver.ts        Sandbox driver (backend-agnostic)
+    docker-backend.ts        Docker backend
+    remote-guest.ts          Generic {command,params}→{success,result} RPC seam + RemoteGuestBackend base (#9170 M13)
+    wsb-backend.ts           Windows Sandbox provider (RemoteGuestBackend)
+    qemu-backend.ts          QEMU provider (RemoteGuestBackend)
+    surface-types.ts         Shared surface type definitions
+    types.ts                 Sandbox-specific types
+    index.ts                 Public re-exports
+
+  security/
+    browser-script-policy.ts GHSA-rcvr-766c-4phv — browser_execute disabled by default
+```
+
+## Commands
+
+Scripts are defined in `package.json`; run them from the repo root with `bun run --cwd`:
+
+```bash
+bun run --cwd plugins/plugin-computeruse build                              # build package artifacts
+bun run --cwd plugins/plugin-computeruse typecheck                          # TypeScript typecheck
+bun run --cwd plugins/plugin-computeruse lint                               # mutating Biome check
+bun run --cwd plugins/plugin-computeruse lint:check                         # read-only Biome check
+bun run --cwd plugins/plugin-computeruse format                             # write formatting
+bun run --cwd plugins/plugin-computeruse format:check                       # read-only formatting check
+bun run --cwd plugins/plugin-computeruse test                               # run package tests
+bun run --cwd plugins/plugin-computeruse capture:macos-desktop-evidence     # macos-desktop-evidence evidence capture
+bun run --cwd plugins/plugin-computeruse capture:windows-desktop-evidence   # windows-desktop-evidence evidence capture
+bun run --cwd plugins/plugin-computeruse postinstall                        # postinstall setup
+bun run --cwd plugins/plugin-computeruse record:windows-cua-input           # bun scripts/record-windows-cua-input.mjs
+bun run --cwd plugins/plugin-computeruse validate:android-aosp-evidence     # android-aosp-evidence evidence validation
+bun run --cwd plugins/plugin-computeruse validate:android-device-evidence   # android-device-evidence evidence validation
+bun run --cwd plugins/plugin-computeruse validate:ios-device-evidence       # ios-device-evidence evidence validation
+bun run --cwd plugins/plugin-computeruse validate:linux-desktop-evidence    # linux-desktop-evidence evidence validation
+bun run --cwd plugins/plugin-computeruse validate:macos-desktop-evidence    # macos-desktop-evidence evidence validation
+bun run --cwd plugins/plugin-computeruse validate:platform-evidence         # platform-evidence evidence validation
+bun run --cwd plugins/plugin-computeruse validate:windows-desktop-evidence  # windows-desktop-evidence evidence validation
+```
+
+## Config / env vars
+
+All read via `runtime.getSetting()` / `process.env`. Core vars are declared in `package.json#agentConfig.pluginParameters`; sandbox vars are read directly.
+
+| Env var | Type | Default | Required | Description |
+|---------|------|---------|----------|-------------|
+| `COMPUTER_USE_ENABLED` | boolean | `false` | No | Master toggle; also controls `autoEnable` |
+| `COMPUTER_USE_SCREENSHOT_AFTER_ACTION` | boolean | `true` | No | Auto-capture screenshot after each desktop action |
+| `COMPUTER_USE_ACTION_TIMEOUT_MS` | number | `10000` | No | Per-action timeout in ms |
+| `COMPUTER_USE_APPROVAL_MODE` | enum | `"smart_approve"` | No | `full_control` / `smart_approve` / `approve_all` / `off` |
+| `COMPUTER_USE_BROWSER_HEADLESS` | boolean | `false` | No | Headless browser (useful in CI) |
+| `ELIZA_COMPUTERUSE_DRIVER` | enum | `"nutjs"` | No | Input driver: `nutjs` (@nut-tree-fork/nut-js) or `legacy` (cliclick/xdotool/PowerShell) |
+| `COMPUTER_USE_MODE` | enum | `"yolo"` | No | Runtime mode: `yolo` (direct desktop) or `sandbox` (Docker-isolated). Alias: `COMPUTERUSE_MODE` |
+| `COMPUTER_USE_SANDBOX_BACKEND` | enum | — | No | Sandbox backend when `COMPUTER_USE_MODE=sandbox`: `"docker"`, `"wsb"` (Windows Sandbox), or `"qemu"`. `docker`/`qemu` need `COMPUTER_USE_SANDBOX_IMAGE`; `wsb` is imageless. Alias: `COMPUTERUSE_SANDBOX_BACKEND` |
+| `COMPUTER_USE_SANDBOX_RPC_URL` | string | — | No | VM-provider (wsb/qemu) in-guest computer-server RPC URL (#9170 M13). Default `http://127.0.0.1:<rpcPort>/cua`. Alias: `COMPUTERUSE_SANDBOX_RPC_URL` |
+| `COMPUTER_USE_SANDBOX_RPC_PORT` | number | `8000` | No | Host-forwarded guest RPC port for wsb/qemu. Alias: `COMPUTERUSE_SANDBOX_RPC_PORT` |
+| `COMPUTER_USE_SANDBOX_IMAGE` | string | — | No | Docker image to use for sandbox mode. Alias: `COMPUTERUSE_SANDBOX_IMAGE` |
+
+Power-user escape hatches read directly via `process.env` (not declared as plugin parameters):
+
+| Env var | Type | Default | Description |
+|---------|------|---------|-------------|
+| `COMPUTERUSE_PS_HOST` | `0` to disable | enabled | Windows-only warm PowerShell host (`ps-host.ts`) that amortizes the cold-spawn AV tax. Set `0` to force every spawn through one-shot `powershell.exe`. |
+| `ELIZA_COMPUTERUSE_PS_TIMEOUT_MS` | number | unset | Windows-only **floor** for every PowerShell/WinRT spawn budget (capture, clipboard, ps-host startup, window enumeration/ops in `windows-list.ts`). Applied via `psSpawnTimeoutMs` — only ever RAISES a call site's default, never lowers it. Raise it on Defender-heavy hosts where cold `powershell.exe` spawns exceed the default budgets and false-fail with `ETIMEDOUT` (#9581). Mirrors `ELIZA_WAYLAND_PORTAL_TIMEOUT_MS`. |
+
+`BROWSER_EXECUTE_DISABLED` is declared in `package.json#agentConfig.pluginParameters` but is **inert**: `browser_execute` is unconditionally disabled in `src/security/browser-script-policy.ts` (`isBrowserExecuteAllowed()` always returns `false`, GHSA-rcvr-766c-4phv). No setting re-enables it.
+
+## How to extend
+
+### Add a new desktop action verb to COMPUTER_USE
+
+1. Add the verb string to the `action` enum in `src/types.ts` (`DesktopActionType`).
+2. Add a handler branch in the `ComputerUseService` dispatch switch in `src/services/computer-use-service.ts`.
+3. Add the low-level platform function in the appropriate `src/platform/*.ts` file.
+4. Update the `parameters[].schema.enum` in `src/actions/use-computer.ts` so the planner sees it.
+5. `promoteSubactionsToActions` will auto-promote `COMPUTER_USE_<VERB>` — no extra registration needed.
+
+### Add a new window operation
+
+Follow the same pattern in `src/actions/window.ts` / `src/actions/window-handlers.ts` / `src/platform/windows-list.ts`.
+
+### Add a new provider
+
+1. Create `src/providers/<name>.ts` exporting a `Provider` object.
+2. Import and add it to the `providers` array in `src/index.ts`.
+
+### Add a route
+
+Add a `Route` object to `computerUseRoutes` in `src/index.ts`, implement the handler in `src/routes/`.
+
+### Register an OCR provider (from another plugin)
+
+Call the module-level `registerCoordOcrProvider(provider)` exported from `src/mobile/ocr-provider.ts` (not a method on `ComputerUseService`). plugin-vision does this at boot to contribute its hierarchical OCR adapter.
+
+### Register a Set-of-Marks provider (from another plugin)
+
+Call `registerSetOfMarksProvider(provider)` (same module, `src/mobile/ocr-provider.ts`). When registered, `detect_elements` prefers it: it returns a deduplicated, **1-indexed** set of numbered marks (GGUF YOLO icons + OCR text fused, icon-over-text suppression + NMS) plus an optional numbered-overlay PNG under `data.setOfMarks.overlay`. Each mark carries a `center` click target the VLM's chosen number resolves to. plugin-vision contributes the implementation (`som.ts` + `set-of-marks-provider.ts`); with no provider registered, `detect_elements` falls back to OCR-only text elements.
+
+### Register an agent loop (model-string → loop)
+
+`COMPUTER_USE_AGENT` selects its loop from a model string via the registry in `src/actor/agent-loop.ts` (#9170 M10) — trycua/cua parity. Every loop implements the same `predictStep` (observe + plan the next action) / `predictClick` (ground a target to a coordinate) seam. The built-in `local-grounder` wraps the existing Brain → Cascade (ScreenSeekeR) and exposes the M5 grounding cache through `predictClick`; it is the match-anything fallback. Register a provider-specific loop (Anthropic / OpenAI `computer-use-preview` / a remote grounder) with `registerAgentLoop({ name, matches: matchesModelFamily("anthropic"), create, priority })`; the runner reads the active model string from the `COMPUTER_USE_AGENT_LOOP` setting/env (default `local-grounder`).
+
+## Conventions / gotchas
+
+- **Approval flow**: every destructive action passes through `ComputerUseApprovalManager`. The default mode is `smart_approve` — only read-only `SAFE_COMMANDS` auto-approve, and destructive verbs (terminal execute, file write/delete) require explicit human approval. Switch to `full_control` (auto-approve everything), `approve_all`, or `off` (deny all) via env or the `/api/computer-use/approval-mode` route.
+- **`browser_execute` is always disabled** (GHSA-rcvr-766c-4phv) — `isBrowserExecuteAllowed()` returns `false` unconditionally; no setting re-enables it. Use `dom`, `clickables`, `click`, `type`, `navigate`, `screenshot` browser subactions instead.
+- **Coordinate system**: each display has its own local coordinate space. `src/platform/coords.ts` translates local→global when needed. Always pass `displayId` when targeting a specific monitor.
+- **nutjs native bindings**: `@nut-tree-fork/nut-js` requires native compilation. If the build fails, set `ELIZA_COMPUTERUSE_DRIVER=legacy` to fall back to shell tools.
+- **Scene is per-turn**: `sceneProvider` calls `SceneBuilder.onAgentTurn()` once per turn. Code that needs the scene outside a provider turn should call `ComputerUseService.getCurrentScene()` or `refreshScene("active")`.
+- **WS7 trajectory events**: `COMPUTER_USE_AGENT` emits `logger.info` lines with `evt: "computeruse.agent.step"`. These are picked up by plugin-trajectory-logger via log capture — no direct dependency.
+- **Platform evidence**: `docs/ios-device-validation.json`, `docs/android-device-validation.json`, `docs/android-aosp-validation.json`, `docs/macos-desktop-validation.json`, `docs/linux-desktop-validation.json`, and `docs/windows-desktop-validation.json` are the release evidence manifests. Keep incomplete live-device checks in `requires_device_evidence`; use `validate:platform-evidence -- --require-complete` only for release gates that truly have artifacts for every required platform check.
+- **Mobile surface**: `src/mobile/` is real but constrained. Read `docs/IOS_CONSTRAINTS.md` and `docs/ANDROID_CONSTRAINTS.md` before touching mobile code.
+- **OSWorld benchmark**: `src/osworld/` adapts the plugin to the OSWorld desktop benchmark format. Not part of normal agent runtime.
+- **Parity matrix (#9170 M14)**: `src/parity/parity-matrix.ts` is the machine-checkable trycua/cua capability map. `validateParityMatrix(actionNames)` is asserted in the test suite — adding a verb to the matrix without registering it (or renaming a promoted action) fails CI, so the map can't silently drift. `src/parity/screenspot.ts` scores click-grounding (point-in-bbox) for any grounder.
+- **Further reading**: `docs/MULTI_MONITOR.md`, `docs/SCENE_BUILDER.md`, `docs/MOBILE_ASSISTANT_ROUTING.md`, `docs/AOSP_SYSTEM_APP.md`, `docs/TEST_LANES_COMPUTERUSE_VISION.md` (unit vs real-driver lanes, per-OS reqs, and the Windows non-interactive-session input gotcha).
+
+<!-- BEGIN: evidence-and-e2e-mandate (managed; canonical standard = repo-root AGENTS.md) -->
+## ⛔ NON-NEGOTIABLE — evidence, trajectories & real end-to-end tests
+
+> The binding, repo-wide standard is **[AGENTS.md](../../AGENTS.md)**. Read it.
+> Nothing in this package is *done* until it is *proven* done — a reviewer must confirm it
+> works **without reading the code**, from the artifacts you attach. This applies to **every**
+> feature, fix, refactor, and chore here. "Tests pass" is not proof; "CI is green" is not proof.
+
+- **Record AND read model trajectories.** Capture the *actual* inputs and outputs of the model
+  from a **live** LLM — not the deterministic proxy, not a mock: the prompt, the
+  providers/context, the raw model output, every tool/action call, and the result. Then **open
+  the trajectory and review it by hand.** A captured-but-unread trajectory is not evidence
+  (`packages/scenario-runner/bin/eliza-scenarios run <scenario> --report <out>`).
+- **Real, full-featured E2E — no larp.** Every feature ships detailed end-to-end tests that
+  drive the *real* path end to end. Not the happy "front door" only: cover error paths,
+  edge/empty/invalid input, concurrency, roles/permissions, and adversarial input. A test that
+  asserts against a mock/stub/fixture standing in for the thing under test **does not count**.
+  If the real model/device/chain/connector/account is hard to reach, **make it reachable — that
+  is the work**, not an excuse to mock. If the existing tests here are shallow or mocked, fixing
+  them is part of your change.
+- **Screenshots + logs at every phase**, plus a **complete walkthrough video/run-through** of
+  the entire feature or view, start to finish (`bun run test:e2e:record`).
+- **Manually review every artifact the change touches** — never just the green check: client
+  logs (console + network), server logs (`[ClassName] …`), the model trajectories in and out,
+  before/after full-page screenshots, **and the domain artifacts listed below for this package.**
+- **No residuals. No shortcuts.** The goal is not "done" — it is *everything* done. Clear every
+  blocker by the **hard path**: build the real architecture, stand up the real
+  model/device/service, actually test it. Never leave a TODO, a stub, a stepping-stone, or a
+  "follow-up." When unsure, research thoroughly, weigh the options, and ship the best,
+  highest-effort, production-ready version. Keep going until every possibility is exhausted.
+
+Artifacts → attached inline in the PR (MP4 video, JPG screenshots, logs in `<details>`); attach each evidence type **or**
+explicitly mark it N/A with a reason — never leave it blank. If `develop` moved and changed
+behavior, **re-capture** evidence; stale proof is worse than none.
+
+**Capture & manually review for this package — CLI / tooling:**
+- The real command/flow invocation transcript (args in, stdout/stderr, exit code) and the artifacts it generated (files, scaffolds, manifests, screenshots/recordings).
+- Failure paths: bad args, missing deps, partial state, permission/network errors.
+- A recording/log of the actual run end to end — not a unit test of one helper.
+- Any model interaction captured as a live trajectory and reviewed.
+<!-- END: evidence-and-e2e-mandate -->

plugins/plugin-computeruse/CLAUDE.md

@@ -0,0 +1,301 @@
+# @elizaos/plugin-computeruse
+
+Desktop automation for elizaOS agents — screenshots, mouse/keyboard control, browser CDP automation, window management, and a multi-display scene model.
+
+Ported from [`coasty-ai/open-computer-use`](https://github.com/coasty-ai/open-computer-use) (Apache 2.0).
+
+## Purpose / role
+
+Adds real desktop control to an Eliza agent: taking screenshots, clicking/typing/scrolling, managing windows, automating web browsers via CDP (puppeteer-core), and building a structured `Scene` from displays, accessibility tree, OCR, and process list so the agent can reason about what is visible. Opt-in: the plugin auto-enables only when `COMPUTER_USE_ENABLED=1` is set (see `autoEnable` in `src/index.ts`). Requires a headful display session on macOS/Linux; headless browser mode is supported.
+
+File operations belong to the FILE action; shell/terminal access belongs to the SHELL action — this plugin does not expose them.
+
+## Plugin surface
+
+### Actions
+
+| Name | File | What it does |
+|------|------|--------------|
+| `COMPUTER_USE` | `src/actions/use-computer.ts` | Umbrella desktop action: `screenshot`, `click`, `click_with_modifiers`, `double_click`, `right_click`, `mouse_move`, `middle_click`, `mouse_down`, `mouse_up`, `type`, `key`, `key_combo`, `key_down`, `key_up`, `scroll`, `drag` (single segment or multi-point `path`), `get_cursor_position`, `detect_elements`, `ocr`, `open` (file/URL/folder via the OS default handler), `launch` (start an app → returns pid). `mouse_down/up` + `key_down/up` are real press-and-hold primitives (nutjs `pressButton`/`pressKey` without release — back hold-drags, marquee, held modifiers); they require the nutjs driver. Requires `OWNER` role. Subactions are promoted to virtual top-level actions (`COMPUTER_USE_CLICK`, etc.) via `promoteSubactionsToActions`. |
+| `WINDOW` | `src/actions/window.ts` | Window management: `list`, `focus`, `switch`, `arrange`, `move`, `minimize`, `maximize`, `restore`, `close`, `get_current_window_id`, `get_application_windows`, `set_bounds` (position + size). Also promoted to `WINDOW_FOCUS`, etc. |
+| `CLIPBOARD` | `src/actions/clipboard.ts` | Host clipboard read/write (`read`, `write`) — trycua/cua parity. Promoted to `CLIPBOARD_READ` / `CLIPBOARD_WRITE`. macOS pbcopy/pbpaste, Linux wl-clipboard/xclip, Windows PowerShell `Get-Clipboard` / `Set-Clipboard`. |
+| `COMPUTER_USE_AGENT` | `src/actions/use-computer-agent.ts` | High-level "give me a goal, click my way there" loop (WS7). Selects an agent loop by model string (#9170 M10, default `local-grounder` = Brain → Cascade) and runs predictStep → dispatch up to `maxSteps`, through a callback-middleware pipeline (#9170 M11: operator-normalizer + trajectory by default; `maxDurationMs` budget cap + `imageRetentionLast` window opt-in). Emits trajectory events as structured log lines and on `report.trajectory`. |
+
+### Providers
+
+| Name | File | What it injects |
+|------|------|----------------|
+| `computerState` | `src/providers/computer-state.ts` | Platform info, screen dimensions, available capabilities, recent actions, approval queue. Gate: `browser/files/terminal/automation/admin` contexts. |
+| `scene` | `src/providers/scene.ts` | Live desktop scene (displays, focused window, apps, OCR boxes, AX nodes, VLM annotations) via `SceneBuilder`. Refreshed once per turn; serialized as token-efficient JSON fence. Gate: `browser/automation/admin` contexts. |
+
+### Services
+
+| Name | `serviceType` | File | What it does |
+|------|--------------|------|--------------|
+| `ComputerUseService` | `"computeruse"` | `src/services/computer-use-service.ts` | Central service: input dispatch, screenshot capture, browser CDP session, window ops, approval-manager wiring, `SceneBuilder` lifecycle. |
+| `VisionContextProvider` | `"vision-context"` | `src/services/vision-context-provider.ts` | Surfaces a `VisionContext` snapshot (open apps, focused window, recent actions, current task goal) for downstream consumers (e.g. plugin-vision). |
+
+### Routes
+
+All paths are under `/api/computer-use/` and implemented in `src/routes/computer-use-compat-routes.ts`:
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/computer-use/approvals` | List pending approval requests |
+| GET | `/api/computer-use/approvals/stream` | SSE stream of approval events (public) |
+| POST | `/api/computer-use/approval-mode` | Change the active `COMPUTER_USE_APPROVAL_MODE` |
+| POST | `/api/computer-use/approvals/:id` | Approve or deny a pending action |
+
+## Layout
+
+```
+src/
+  index.ts                   Plugin entry — assembles and exports computerUsePlugin
+  types.ts                   All shared types (ApprovalMode, ComputerUseConfig, DesktopActionParams, …)
+  approval-manager.ts        ComputerUseApprovalManager — queues pending actions, applies approval mode
+  register-routes.ts         Route registration helper
+
+  actions/
+    use-computer.ts          COMPUTER_USE parent action + subaction table
+    use-computer-agent.ts    COMPUTER_USE_AGENT (WS7 autonomous loop)
+    window.ts                WINDOW parent action
+    window-handlers.ts       Per-verb handlers called by window.ts
+    clipboard.ts             clipboardAction (CLIPBOARD parent action) — registered in index.ts; promoted to CLIPBOARD_READ / CLIPBOARD_WRITE
+    helpers.ts               resolveActionParams, buildScreenshotAttachment, …
+
+  actor/                     WS7 autonomous desktop loop
+    brain.ts                 Brain — sends display PNGs to IMAGE_DESCRIPTION model, produces BrainOutput
+    cascade.ts               Cascade (ScreenSeekeR) — Brain → Actor → ProposedAction
+    actor.ts                 Actor interface + OsAtlasProActor + OcrCoordinateGroundingActor
+    dispatch.ts              dispatch() — executes a ProposedAction via ComputerInterface
+    computer-interface.ts    ComputerInterface abstraction + makeComputerInterface()
+    aosp-input-actor.ts      AOSP-specific actor
+    types.ts                 BrainOutput, ProposedAction, …
+    index.ts                 Public re-exports
+
+  platform/
+    browser.ts               Puppeteer-core CDP browser automation
+    capture.ts               captureDisplay / captureAllDisplays
+    displays.ts              listDisplays / getPrimaryDisplay
+    driver.ts                driverClick / driverType / … (nutjs or legacy shell)
+    nut-driver.ts            @nut-tree-fork/nut-js implementation
+    windows-list.ts          listWindows / focusWindow / arrangeWindows / getActiveWindow / resizeWindow / …
+    launch.ts                openTarget (open file/URL/folder) / launchApp (spawn app → pid)
+    clipboard.ts             OS clipboard read/write
+    a11y.ts                  Accessibility tree query
+    coords.ts                localToGlobal coordinate translation
+    wayland-portal.ts        xdg-desktop-portal screenshot sidecar for Wayland
+    capabilities.ts          detectPlatformCapabilities
+    desktop.ts               High-level desktop helpers
+    helpers.ts               Shared platform utilities
+    permissions.ts           classifyPermissionDeniedError
+    process-list.ts          listProcesses / parsePsOutput
+    screenshot-quality.ts    Quality / compression settings for screenshots
+    terminal.ts              Terminal session management (internal; not exposed as action)
+    file-ops.ts              File primitives (internal; not exposed as action)
+    screenshot.ts            Low-level screencapture wrappers
+    security.ts              Path + command security checks
+
+  providers/
+    computer-state.ts        computerStateProvider
+    scene.ts                 sceneProvider
+
+  routes/
+    computer-use-routes.ts       handleComputerUseRoutes (full route table)
+    computer-use-compat-routes.ts  computerUseRouteHandler() (compat wrapper used by plugin entry)
+    sandbox-routes.ts            handleSandboxRoute
+
+  scene/
+    scene-builder.ts         SceneBuilder — composes displays + a11y + OCR into Scene
+    scene-types.ts           Scene, SceneApp, SceneAppWindow, SceneAxNode, …
+    a11y-provider.ts         DarwinAccessibilityProvider / LinuxAccessibilityProvider / …
+    apps.ts                  enumerateApps / joinAppsAndWindows
+    dhash.ts                 Perceptual hash / dirty-block diffing for change detection
+    screen-state.ts          ScreenState + ScreenStateStore — one shared capture/turn (dHash/blockGrid + change events)
+    ocr-adapter.ts           OcrProvider / CoordOcrProvider adapter seam
+    serialize.ts             serializeSceneForPrompt — token-efficient JSON fence
+
+  services/
+    computer-use-service.ts  ComputerUseService (serviceType = "computeruse")
+    vision-context-provider.ts  VisionContextProvider (serviceType = "vision-context")
+    desktop-control.ts       Low-level desktop control primitives + DesktopControl* types
+    index.ts                 Barrel re-exports for services/
+
+  mobile/
+    ocr-provider.ts          OcrProvider / CoordOcrProvider interfaces (plugin-vision contributes impls)
+    ios-bridge.ts            iOS computer-use bridge
+    ios-computer-interface.ts  iOS-specific ComputerInterface implementation
+    ios-app-intent-registry.ts  Registry of iOS app intents for automation
+    android-bridge.ts        AOSP input bridge
+    android-scene.ts         Android scene capture and representation
+    android-trajectory.ts    Android action trajectory recording
+    mobile-computer-interface.ts  MobileComputerInterface
+    mobile-screen-capture.ts Screen capture abstraction for mobile targets
+    index.ts                 Mobile public surface
+
+  osworld/
+    adapter.ts               OSWorld benchmark adapter
+    action-converter.ts      OSWorld action → ComputerInterface translation
+    types.ts                 OSWorld-specific type definitions
+
+  parity/                    trycua/cua parity tooling (#9170 M14)
+    parity-matrix.ts         machine-checkable capability matrix + validateParityMatrix (guards drift vs the live action surface)
+    screenspot.ts            ScreenSpot grounding harness (point-in-bbox scorer + score fold)
+    index.ts                 Public re-exports
+
+  sandbox/
+    sandbox-driver.ts        Sandbox driver (backend-agnostic)
+    docker-backend.ts        Docker backend
+    remote-guest.ts          Generic {command,params}→{success,result} RPC seam + RemoteGuestBackend base (#9170 M13)
+    wsb-backend.ts           Windows Sandbox provider (RemoteGuestBackend)
+    qemu-backend.ts          QEMU provider (RemoteGuestBackend)
+    surface-types.ts         Shared surface type definitions
+    types.ts                 Sandbox-specific types
+    index.ts                 Public re-exports
+
+  security/
+    browser-script-policy.ts GHSA-rcvr-766c-4phv — browser_execute disabled by default
+```
+
+## Commands
+
+Scripts are defined in `package.json`; run them from the repo root with `bun run --cwd`:
+
+```bash
+bun run --cwd plugins/plugin-computeruse build                              # build package artifacts
+bun run --cwd plugins/plugin-computeruse typecheck                          # TypeScript typecheck
+bun run --cwd plugins/plugin-computeruse lint                               # mutating Biome check
+bun run --cwd plugins/plugin-computeruse lint:check                         # read-only Biome check
+bun run --cwd plugins/plugin-computeruse format                             # write formatting
+bun run --cwd plugins/plugin-computeruse format:check                       # read-only formatting check
+bun run --cwd plugins/plugin-computeruse test                               # run package tests
+bun run --cwd plugins/plugin-computeruse capture:macos-desktop-evidence     # macos-desktop-evidence evidence capture
+bun run --cwd plugins/plugin-computeruse capture:windows-desktop-evidence   # windows-desktop-evidence evidence capture
+bun run --cwd plugins/plugin-computeruse postinstall                        # postinstall setup
+bun run --cwd plugins/plugin-computeruse record:windows-cua-input           # bun scripts/record-windows-cua-input.mjs
+bun run --cwd plugins/plugin-computeruse validate:android-aosp-evidence     # android-aosp-evidence evidence validation
+bun run --cwd plugins/plugin-computeruse validate:android-device-evidence   # android-device-evidence evidence validation
+bun run --cwd plugins/plugin-computeruse validate:ios-device-evidence       # ios-device-evidence evidence validation
+bun run --cwd plugins/plugin-computeruse validate:linux-desktop-evidence    # linux-desktop-evidence evidence validation
+bun run --cwd plugins/plugin-computeruse validate:macos-desktop-evidence    # macos-desktop-evidence evidence validation
+bun run --cwd plugins/plugin-computeruse validate:platform-evidence         # platform-evidence evidence validation
+bun run --cwd plugins/plugin-computeruse validate:windows-desktop-evidence  # windows-desktop-evidence evidence validation
+```
+
+## Config / env vars
+
+All read via `runtime.getSetting()` / `process.env`. Core vars are declared in `package.json#agentConfig.pluginParameters`; sandbox vars are read directly.
+
+| Env var | Type | Default | Required | Description |
+|---------|------|---------|----------|-------------|
+| `COMPUTER_USE_ENABLED` | boolean | `false` | No | Master toggle; also controls `autoEnable` |
+| `COMPUTER_USE_SCREENSHOT_AFTER_ACTION` | boolean | `true` | No | Auto-capture screenshot after each desktop action |
+| `COMPUTER_USE_ACTION_TIMEOUT_MS` | number | `10000` | No | Per-action timeout in ms |
+| `COMPUTER_USE_APPROVAL_MODE` | enum | `"smart_approve"` | No | `full_control` / `smart_approve` / `approve_all` / `off` |
+| `COMPUTER_USE_BROWSER_HEADLESS` | boolean | `false` | No | Headless browser (useful in CI) |
+| `ELIZA_COMPUTERUSE_DRIVER` | enum | `"nutjs"` | No | Input driver: `nutjs` (@nut-tree-fork/nut-js) or `legacy` (cliclick/xdotool/PowerShell) |
+| `COMPUTER_USE_MODE` | enum | `"yolo"` | No | Runtime mode: `yolo` (direct desktop) or `sandbox` (Docker-isolated). Alias: `COMPUTERUSE_MODE` |
+| `COMPUTER_USE_SANDBOX_BACKEND` | enum | — | No | Sandbox backend when `COMPUTER_USE_MODE=sandbox`: `"docker"`, `"wsb"` (Windows Sandbox), or `"qemu"`. `docker`/`qemu` need `COMPUTER_USE_SANDBOX_IMAGE`; `wsb` is imageless. Alias: `COMPUTERUSE_SANDBOX_BACKEND` |
+| `COMPUTER_USE_SANDBOX_RPC_URL` | string | — | No | VM-provider (wsb/qemu) in-guest computer-server RPC URL (#9170 M13). Default `http://127.0.0.1:<rpcPort>/cua`. Alias: `COMPUTERUSE_SANDBOX_RPC_URL` |
+| `COMPUTER_USE_SANDBOX_RPC_PORT` | number | `8000` | No | Host-forwarded guest RPC port for wsb/qemu. Alias: `COMPUTERUSE_SANDBOX_RPC_PORT` |
+| `COMPUTER_USE_SANDBOX_IMAGE` | string | — | No | Docker image to use for sandbox mode. Alias: `COMPUTERUSE_SANDBOX_IMAGE` |
+
+Power-user escape hatches read directly via `process.env` (not declared as plugin parameters):
+
+| Env var | Type | Default | Description |
+|---------|------|---------|-------------|
+| `COMPUTERUSE_PS_HOST` | `0` to disable | enabled | Windows-only warm PowerShell host (`ps-host.ts`) that amortizes the cold-spawn AV tax. Set `0` to force every spawn through one-shot `powershell.exe`. |
+| `ELIZA_COMPUTERUSE_PS_TIMEOUT_MS` | number | unset | Windows-only **floor** for every PowerShell/WinRT spawn budget (capture, clipboard, ps-host startup, window enumeration/ops in `windows-list.ts`). Applied via `psSpawnTimeoutMs` — only ever RAISES a call site's default, never lowers it. Raise it on Defender-heavy hosts where cold `powershell.exe` spawns exceed the default budgets and false-fail with `ETIMEDOUT` (#9581). Mirrors `ELIZA_WAYLAND_PORTAL_TIMEOUT_MS`. |
+
+`BROWSER_EXECUTE_DISABLED` is declared in `package.json#agentConfig.pluginParameters` but is **inert**: `browser_execute` is unconditionally disabled in `src/security/browser-script-policy.ts` (`isBrowserExecuteAllowed()` always returns `false`, GHSA-rcvr-766c-4phv). No setting re-enables it.
+
+## How to extend
+
+### Add a new desktop action verb to COMPUTER_USE
+
+1. Add the verb string to the `action` enum in `src/types.ts` (`DesktopActionType`).
+2. Add a handler branch in the `ComputerUseService` dispatch switch in `src/services/computer-use-service.ts`.
+3. Add the low-level platform function in the appropriate `src/platform/*.ts` file.
+4. Update the `parameters[].schema.enum` in `src/actions/use-computer.ts` so the planner sees it.
+5. `promoteSubactionsToActions` will auto-promote `COMPUTER_USE_<VERB>` — no extra registration needed.
+
+### Add a new window operation
+
+Follow the same pattern in `src/actions/window.ts` / `src/actions/window-handlers.ts` / `src/platform/windows-list.ts`.
+
+### Add a new provider
+
+1. Create `src/providers/<name>.ts` exporting a `Provider` object.
+2. Import and add it to the `providers` array in `src/index.ts`.
+
+### Add a route
+
+Add a `Route` object to `computerUseRoutes` in `src/index.ts`, implement the handler in `src/routes/`.
+
+### Register an OCR provider (from another plugin)
+
+Call the module-level `registerCoordOcrProvider(provider)` exported from `src/mobile/ocr-provider.ts` (not a method on `ComputerUseService`). plugin-vision does this at boot to contribute its hierarchical OCR adapter.
+
+### Register a Set-of-Marks provider (from another plugin)
+
+Call `registerSetOfMarksProvider(provider)` (same module, `src/mobile/ocr-provider.ts`). When registered, `detect_elements` prefers it: it returns a deduplicated, **1-indexed** set of numbered marks (GGUF YOLO icons + OCR text fused, icon-over-text suppression + NMS) plus an optional numbered-overlay PNG under `data.setOfMarks.overlay`. Each mark carries a `center` click target the VLM's chosen number resolves to. plugin-vision contributes the implementation (`som.ts` + `set-of-marks-provider.ts`); with no provider registered, `detect_elements` falls back to OCR-only text elements.
+
+### Register an agent loop (model-string → loop)
+
+`COMPUTER_USE_AGENT` selects its loop from a model string via the registry in `src/actor/agent-loop.ts` (#9170 M10) — trycua/cua parity. Every loop implements the same `predictStep` (observe + plan the next action) / `predictClick` (ground a target to a coordinate) seam. The built-in `local-grounder` wraps the existing Brain → Cascade (ScreenSeekeR) and exposes the M5 grounding cache through `predictClick`; it is the match-anything fallback. Register a provider-specific loop (Anthropic / OpenAI `computer-use-preview` / a remote grounder) with `registerAgentLoop({ name, matches: matchesModelFamily("anthropic"), create, priority })`; the runner reads the active model string from the `COMPUTER_USE_AGENT_LOOP` setting/env (default `local-grounder`).
+
+## Conventions / gotchas
+
+- **Approval flow**: every destructive action passes through `ComputerUseApprovalManager`. The default mode is `smart_approve` — only read-only `SAFE_COMMANDS` auto-approve, and destructive verbs (terminal execute, file write/delete) require explicit human approval. Switch to `full_control` (auto-approve everything), `approve_all`, or `off` (deny all) via env or the `/api/computer-use/approval-mode` route.
+- **`browser_execute` is always disabled** (GHSA-rcvr-766c-4phv) — `isBrowserExecuteAllowed()` returns `false` unconditionally; no setting re-enables it. Use `dom`, `clickables`, `click`, `type`, `navigate`, `screenshot` browser subactions instead.
+- **Coordinate system**: each display has its own local coordinate space. `src/platform/coords.ts` translates local→global when needed. Always pass `displayId` when targeting a specific monitor.
+- **nutjs native bindings**: `@nut-tree-fork/nut-js` requires native compilation. If the build fails, set `ELIZA_COMPUTERUSE_DRIVER=legacy` to fall back to shell tools.
+- **Scene is per-turn**: `sceneProvider` calls `SceneBuilder.onAgentTurn()` once per turn. Code that needs the scene outside a provider turn should call `ComputerUseService.getCurrentScene()` or `refreshScene("active")`.
+- **WS7 trajectory events**: `COMPUTER_USE_AGENT` emits `logger.info` lines with `evt: "computeruse.agent.step"`. These are picked up by plugin-trajectory-logger via log capture — no direct dependency.
+- **Platform evidence**: `docs/ios-device-validation.json`, `docs/android-device-validation.json`, `docs/android-aosp-validation.json`, `docs/macos-desktop-validation.json`, `docs/linux-desktop-validation.json`, and `docs/windows-desktop-validation.json` are the release evidence manifests. Keep incomplete live-device checks in `requires_device_evidence`; use `validate:platform-evidence -- --require-complete` only for release gates that truly have artifacts for every required platform check.
+- **Mobile surface**: `src/mobile/` is real but constrained. Read `docs/IOS_CONSTRAINTS.md` and `docs/ANDROID_CONSTRAINTS.md` before touching mobile code.
+- **OSWorld benchmark**: `src/osworld/` adapts the plugin to the OSWorld desktop benchmark format. Not part of normal agent runtime.
+- **Parity matrix (#9170 M14)**: `src/parity/parity-matrix.ts` is the machine-checkable trycua/cua capability map. `validateParityMatrix(actionNames)` is asserted in the test suite — adding a verb to the matrix without registering it (or renaming a promoted action) fails CI, so the map can't silently drift. `src/parity/screenspot.ts` scores click-grounding (point-in-bbox) for any grounder.
+- **Further reading**: `docs/MULTI_MONITOR.md`, `docs/SCENE_BUILDER.md`, `docs/MOBILE_ASSISTANT_ROUTING.md`, `docs/AOSP_SYSTEM_APP.md`, `docs/TEST_LANES_COMPUTERUSE_VISION.md` (unit vs real-driver lanes, per-OS reqs, and the Windows non-interactive-session input gotcha).
+
+<!-- BEGIN: evidence-and-e2e-mandate (managed; canonical standard = repo-root AGENTS.md) -->
+## ⛔ NON-NEGOTIABLE — evidence, trajectories & real end-to-end tests
+
+> The binding, repo-wide standard is **[AGENTS.md](../../AGENTS.md)**. Read it.
+> Nothing in this package is *done* until it is *proven* done — a reviewer must confirm it
+> works **without reading the code**, from the artifacts you attach. This applies to **every**
+> feature, fix, refactor, and chore here. "Tests pass" is not proof; "CI is green" is not proof.
+
+- **Record AND read model trajectories.** Capture the *actual* inputs and outputs of the model
+  from a **live** LLM — not the deterministic proxy, not a mock: the prompt, the
+  providers/context, the raw model output, every tool/action call, and the result. Then **open
+  the trajectory and review it by hand.** A captured-but-unread trajectory is not evidence
+  (`packages/scenario-runner/bin/eliza-scenarios run <scenario> --report <out>`).
+- **Real, full-featured E2E — no larp.** Every feature ships detailed end-to-end tests that
+  drive the *real* path end to end. Not the happy "front door" only: cover error paths,
+  edge/empty/invalid input, concurrency, roles/permissions, and adversarial input. A test that
+  asserts against a mock/stub/fixture standing in for the thing under test **does not count**.
+  If the real model/device/chain/connector/account is hard to reach, **make it reachable — that
+  is the work**, not an excuse to mock. If the existing tests here are shallow or mocked, fixing
+  them is part of your change.
+- **Screenshots + logs at every phase**, plus a **complete walkthrough video/run-through** of
+  the entire feature or view, start to finish (`bun run test:e2e:record`).
+- **Manually review every artifact the change touches** — never just the green check: client
+  logs (console + network), server logs (`[ClassName] …`), the model trajectories in and out,
+  before/after full-page screenshots, **and the domain artifacts listed below for this package.**
+- **No residuals. No shortcuts.** The goal is not "done" — it is *everything* done. Clear every
+  blocker by the **hard path**: build the real architecture, stand up the real
+  model/device/service, actually test it. Never leave a TODO, a stub, a stepping-stone, or a
+  "follow-up." When unsure, research thoroughly, weigh the options, and ship the best,
+  highest-effort, production-ready version. Keep going until every possibility is exhausted.
+
+Artifacts → attached inline in the PR (MP4 video, JPG screenshots, logs in `<details>`); attach each evidence type **or**
+explicitly mark it N/A with a reason — never leave it blank. If `develop` moved and changed
+behavior, **re-capture** evidence; stale proof is worse than none.
+
+**Capture & manually review for this package — CLI / tooling:**
+- The real command/flow invocation transcript (args in, stdout/stderr, exit code) and the artifacts it generated (files, scaffolds, manifests, screenshots/recordings).
+- Failure paths: bad args, missing deps, partial state, permission/network errors.
+- A recording/log of the actual run end to end — not a unit test of one helper.
+- Any model interaction captured as a live trajectory and reviewed.
+<!-- END: evidence-and-e2e-mandate -->

plugins/plugin-computeruse/README.md

@@ -0,0 +1,80 @@
+# `@elizaos/plugin-computeruse`
+
+Desktop automation plugin for elizaOS agents — screenshots, mouse /
+keyboard control, browser CDP automation, window management, clipboard,
+and the OCR provider registry that other plugins contribute to.
+
+Ported from
+[`coasty-ai/open-computer-use`](https://github.com/coasty-ai/open-computer-use)
+(Apache 2.0).
+
+## Boundary with `@elizaos/plugin-vision`
+
+This plugin owns the OS surfaces:
+
+- screen / display capture (`src/platform/capture.ts`,
+  `src/platform/displays.ts`,
+  `ComputerUseService.captureScreen()`),
+- input + windows + clipboard + accessibility,
+- the OCR provider registries — `OcrProvider` (line-level) and
+  `CoordOcrProvider` (hierarchical with absolute coords), defined in
+  `src/mobile/ocr-provider.ts`.
+
+`@elizaos/plugin-vision` owns the camera pipeline, scene description
+via `runtime.useModel(IMAGE_DESCRIPTION)`, the screen tiler, the
+detector pipeline (faces / people / objects), and the OCR
+implementations themselves. plugin-vision *consumes* capture from this
+plugin via `runtime.getService("computeruse")` and *contributes* the
+hierarchical OCR adapter into this plugin's `registerCoordOcrProvider`
+seam at boot.
+
+Both seams are runtime feature-detected — neither package depends on
+the other.
+
+## Enabling
+
+- Config: `features.computeruse: true`
+- Env: `COMPUTER_USE_ENABLED=1`
+
+## Platform requirements
+
+| OS | Capture | Input |
+|----|---------|-------|
+| macOS | `screencapture` (built-in) | `cliclick` (`brew install cliclick`), AppleScript |
+| Linux | `import` (ImageMagick) / `scrot` | `xdotool` (`sudo apt install xdotool`) |
+| Windows | PowerShell + `System.Drawing` | PowerShell |
+| Browser | — | `puppeteer-core` + Chrome / Edge / Brave |
+
+## Surface
+
+- **Actions** — `COMPUTER_USE` (canonical screenshot / click / key /
+  scroll / etc.), `WINDOW` (list / focus / arrange / move /...), and
+  `COMPUTER_USE_AGENT` (high-level goal-driven autonomous desktop loop:
+  Brain → Cascade → dispatch up to `maxSteps` iterations).
+  Subactions of `COMPUTER_USE` and `WINDOW` are promoted to virtual
+  top-level actions (e.g. `COMPUTER_USE_CLICK`, `WINDOW_FOCUS`) so the
+  planner picks a specific verb directly from the catalogue.
+- **Services** — `ComputerUseService` (`serviceType = "computeruse"`)
+  and `VisionContextProvider`.
+- **Providers** — `computerStateProvider`, `sceneProvider`.
+- **Routes** — approval inbox + SSE stream + approval-mode toggle under
+  `/api/computer-use/...`.
+
+## File operations + shell
+
+File operations live on the FILE action; shell / terminal access lives
+on the SHELL action. They are **not** exposed by this plugin.
+
+## Further reading
+
+- [`docs/MULTI_MONITOR.md`](./docs/MULTI_MONITOR.md) — multi-display
+  capture and coordinate translation.
+- [`docs/SCENE_BUILDER.md`](./docs/SCENE_BUILDER.md) — how windows,
+  a11y, screen, and OCR are composed into a single `Scene`.
+- [`docs/IOS_CONSTRAINTS.md`](./docs/IOS_CONSTRAINTS.md) /
+  [`docs/ANDROID_CONSTRAINTS.md`](./docs/ANDROID_CONSTRAINTS.md) —
+  honest scope on mobile.
+- [`docs/MOBILE_ASSISTANT_ROUTING.md`](./docs/MOBILE_ASSISTANT_ROUTING.md)
+  — mobile request routing.
+- [`docs/AOSP_SYSTEM_APP.md`](./docs/AOSP_SYSTEM_APP.md) — AOSP
+  system-app deployment notes.

plugins/plugin-computeruse/build.ts

@@ -0,0 +1,51 @@
+#!/usr/bin/env bun
+/**
+ * Build script for @elizaos/plugin-computeruse. Orchestration lives in the
+ * shared driver (plugins/plugin-build.ts); this lists only what differs.
+ *
+ * Three Node/ESM entrypoints are bundled with linked sourcemaps and flat
+ * `[name].[ext]` naming (index + register-routes at the dist root, the mobile
+ * OCR provider under dist/mobile). Declarations are emitted declaration-only
+ * from tsconfig.build.json, preserving the package's established `dist/`
+ * layout for downstream imports.
+ */
+import { buildPlugin } from "../plugin-build";
+
+const naming = { entry: "[name].[ext]" };
+
+await buildPlugin({
+  name: "plugin-computeruse",
+  clean: true,
+  externalsOptions: { extra: ["node:*"] },
+  targets: [
+    {
+      label: "index",
+      entry: "./src/index.ts",
+      outSubdir: "",
+      target: "node",
+      format: "esm",
+      sourcemap: "linked",
+      naming,
+    },
+    {
+      label: "register-routes",
+      entry: "./src/register-routes.ts",
+      outSubdir: "",
+      target: "node",
+      format: "esm",
+      sourcemap: "linked",
+      naming,
+    },
+    {
+      label: "mobile/ocr-provider",
+      entry: "./src/mobile/ocr-provider.ts",
+      outSubdir: "mobile",
+      target: "node",
+      format: "esm",
+      sourcemap: "linked",
+      naming,
+    },
+  ],
+  dtsProject: "tsconfig.build.json",
+  dtsEmitDeclarationOnly: true,
+});

plugins/plugin-computeruse/ci-templates/README.md

@@ -0,0 +1,31 @@
+# CI templates — computeruse
+
+Workflow files that need `workflow` OAuth scope to commit, parked here so they're
+version-controlled and one `git mv` away from active. The bot that authored them
+cannot write under `.github/workflows/`.
+
+## `cua-parity-real.yml` — the last step for "tested on Linux + macOS"
+
+The trycua/cua parity verbs (#9170) are implemented and **machine-checked on all
+four platforms in the default lane** (`src/parity/parity-matrix.ts` +
+`__tests__/cua-parity-coverage.test.ts` run on Windows/Linux/macOS/AOSP-Node).
+The **real-driver actuation** lane (`__tests__/cua-parity-input.real.test.ts` —
+real mouse/keyboard/window/clipboard against a live desktop) is currently only
+exercised on Windows. The test gate is already OS-agnostic
+(`win32 || darwin || (linux && DISPLAY)`); the only missing piece is a CI lane
+that provides a display on Linux (Xvfb) and runs it on macOS.
+
+`cua-parity-real.yml` is that lane (3-OS matrix, `workflow_dispatch`). To enable:
+
+```bash
+git mv plugins/plugin-computeruse/ci-templates/cua-parity-real.yml \
+       .github/workflows/cua-parity-real.yml
+git commit -m "ci(computeruse): enable 3-OS cua-parity real-driver lane (#9170)"
+```
+
+Then trigger it from the Actions tab (or add `push`/`pull_request` triggers if you
+want it on every change). The recipe bakes in the empirically-verified build
+prerequisites (run the i18n codegen, then build `@elizaos/core` to `dist` before
+vitest — `bun` does not auto-run `prebuild`, and the source export condition trips
+an `@opentelemetry` ESM error). Once green, the verb-by-verb actuation that is
+Windows-verified today extends to Linux and macOS automatically.

plugins/plugin-computeruse/ci-templates/cua-parity-real.yml

@@ -0,0 +1,125 @@
+# cua-parity-real.yml — 3-OS real-driver lane for the cua-parity input verbs.
+#
+# ADOPT-ME: the committing bot token lacks `workflow` scope, so this file is
+# parked at a non-.github path (plugins/plugin-computeruse/ci-templates/).
+# A human must move it to .github/workflows/cua-parity-real.yml to enable it:
+#     git mv plugins/plugin-computeruse/ci-templates/cua-parity-real.yml \
+#            .github/workflows/cua-parity-real.yml
+#
+# Runs ONLY:
+#   plugins/plugin-computeruse/src/__tests__/cua-parity-input.real.test.ts
+# through the shared real config (packages/test/vitest/real.config.ts) on
+# Linux (Xvfb), macOS (headful), and Windows (interactive desktop). This is the
+# one remaining step to satisfy "tested on windows, linux, mac os" for the
+# trycua/cua parity verbs (#9170): today only Windows real-driver actuation is
+# CI-exercised; this lane adds Linux + macOS.
+#
+# Verified facts baked in (see #9170 / the parity PRs' evidence):
+#   - codegen MUST run before building core (bun does NOT auto-run prebuild)
+#   - core MUST be built to dist (the eliza-source export condition otherwise
+#     resolves @elizaos/core to src and trips an @opentelemetry extensionless
+#     ESM error under vitest)
+#   - the granular verbs are nutjs-only; the @nut-tree-fork/libnut-* prebuilt
+#     .node ships in the npm package, so --ignore-scripts installs are fine
+#   - Linux needs DISPLAY (Xvfb) or the lane is skipped by the test gate
+#     (win32 || darwin || (linux && DISPLAY)); the real config's
+#     fail-on-silent-skip guard turns an unexpected skip into a failure
+#   - Windows input-effect tests need an interactive desktop (GH-hosted = OK)
+
+name: CUA Parity Real Driver (3-OS)
+
+on:
+  workflow_dispatch:
+
+concurrency:
+  group: cua-parity-real-${{ github.ref }}
+  cancel-in-progress: true
+
+permissions:
+  contents: read
+
+env:
+  BUN_VERSION: "canary"
+  NODE_VERSION: "24.15.0"
+  NODE_NO_WARNINGS: "1"
+  NODE_OPTIONS: "--max-old-space-size=8192"
+  ELIZA_CI_REAL: "1"
+  ELIZA_COMPUTERUSE_DRIVER: "nutjs"
+  TEST_FILE: "plugins/plugin-computeruse/src/__tests__/cua-parity-input.real.test.ts"
+  REAL_CONFIG: "packages/test/vitest/real.config.ts"
+
+jobs:
+  cua-parity-real:
+    name: cua-parity-real (${{ matrix.os }})
+    runs-on: ${{ matrix.os }}
+    timeout-minutes: 45
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-24.04, macos-14, windows-latest]
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          submodules: false
+
+      # ---- OS-specific real-driver host deps -----------------------------
+      - name: Install Linux desktop deps (Xvfb + nutjs/clipboard runtime)
+        if: runner.os == 'Linux'
+        shell: bash
+        run: |
+          sudo apt-get update
+          # xvfb -> DISPLAY for the linux gate; xclip -> clipboard test;
+          # xdotool -> legacy safety net; libxtst6/libxkbcommon0 -> libnut runtime.
+          sudo apt-get install -y --no-install-recommends \
+            xvfb xdotool xclip libxtst6 libxkbcommon0
+
+      - name: Install macOS legacy safety-net (cliclick)
+        if: runner.os == 'macOS'
+        shell: bash
+        run: brew install cliclick || echo "cliclick install best-effort; nutjs is the driver"
+
+      # ---- Toolchain ------------------------------------------------------
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: ${{ env.NODE_VERSION }}
+
+      - name: Setup Bun
+        uses: oven-sh/setup-bun@v2
+        with:
+          bun-version: ${{ env.BUN_VERSION }}
+
+      # ---- Install (prebuilt libnut ships in the package; scripts not needed) ----
+      - name: Install dependencies
+        shell: bash
+        run: bun install --frozen-lockfile --ignore-scripts || bun install --ignore-scripts
+
+      # ---- REQUIRED: codegen before core build (bun does not run prebuild) ----
+      - name: Generate core i18n keyword data
+        shell: bash
+        run: node packages/shared/scripts/generate-keywords.mjs
+
+      # ---- REQUIRED: build core to dist (fixes eliza-source/@opentelemetry ESM) ----
+      - name: Build @elizaos/core (Node target)
+        shell: bash
+        run: bun run --cwd packages/core build:node
+
+      # ---- Run the single real test through the real config ---------------
+      - name: Run cua-parity real test (Linux / Xvfb)
+        if: runner.os == 'Linux'
+        shell: bash
+        run: |
+          xvfb-run -a bunx vitest run "$TEST_FILE" --config "$REAL_CONFIG"
+
+      - name: Run cua-parity real test (macOS / headful)
+        if: runner.os == 'macOS'
+        shell: bash
+        run: |
+          bunx vitest run "$TEST_FILE" --config "$REAL_CONFIG"
+
+      - name: Run cua-parity real test (Windows / interactive desktop)
+        if: runner.os == 'Windows'
+        shell: bash
+        run: |
+          bunx vitest run "$TEST_FILE" --config "$REAL_CONFIG"

plugins/plugin-computeruse/docs/ANDROID_CONSTRAINTS.md

@@ -0,0 +1,215 @@
+# Android computer-use — constraints, capabilities, and validation checklist
+
+## What works in a consumer APK
+
+| Capability | Permission | Notes |
+|---|---|---|
+| AccessibilityService — view tree + gesture dispatch | `BIND_ACCESSIBILITY_SERVICE` (user must enable in Settings) | Survives Advanced Protection Mode in most OEM ROMs |
+| MediaProjection — screen frame capture | `FOREGROUND_SERVICE_MEDIA_PROJECTION` + user consent dialog | 1 Hz default; foreground service required |
+| UsageStatsManager — app history + foreground app | `PACKAGE_USAGE_STATS` (user must enable in Settings > Usage Access, no runtime prompt) | 24-hour window; 5-min scan for foreground app |
+| Camera2 — JPEG/RGBA frames | `CAMERA` (runtime permission) | Service-friendly; no Activity or SurfaceView needed |
+| onTrimMemory → MemoryArbiter pressure | None (ComponentCallbacks2) | Fires at TRIM_MEMORY_RUNNING_LOW and TRIM_MEMORY_RUNNING_CRITICAL |
+
+## Pixel/Google Android assistant entry points
+
+For normal Google Android and Play distribution, Eliza uses Google App
+Actions and Android shortcuts only:
+
+- `packages/app-core/platforms/android/app/src/main/res/xml/shortcuts.xml`
+  declares `actions.intent.OPEN_APP_FEATURE`, `CREATE_MESSAGE`,
+  and `GET_THING`.
+- `OPEN_APP_FEATURE` is the static feature surface for chat/ask, voice,
+  LifeOps daily brief, LifeOps task creation, and LifeOps tasks.
+- `CREATE_MESSAGE` and `GET_THING` route free-form ask/chat text to the
+  app's chat deep link.
+- Static shortcuts use source-tagged deep links and are bound to
+  `OPEN_APP_FEATURE` inline inventory.
+- Each App Actions capability keeps a fallback fulfillment without a
+  required parameter. The build validator rejects regressions because
+  Assistant may invoke vague feature/search/message requests.
+Unsupported BIIs such as `actions.intent.CREATE_THING` are intentionally
+not declared; LifeOps task creation is a feature-open flow and any
+mutation must still go through runtime confirmation and the ScheduledTask
+path.
+
+Mapping by flow:
+
+| Flow | Supported App Actions / shortcut entry |
+|---|---|
+| Ask/chat | `CREATE_MESSAGE`, `GET_THING`, `eliza_app_action_chat` |
+| Voice chat | `OPEN_APP_FEATURE` inline inventory, `eliza_app_action_voice` |
+| LifeOps daily brief | `OPEN_APP_FEATURE` inline inventory, `eliza_app_action_daily_brief` |
+| Create LifeOps task | `OPEN_APP_FEATURE` inline inventory, `eliza_app_action_new_task` |
+| View LifeOps tasks | `OPEN_APP_FEATURE` inline inventory, `eliza_app_action_tasks` |
+
+There is no Play-compatible default-assistant handoff for the Pixel build.
+Normal Google Android entry goes through App Actions/static shortcuts/deep
+links. AOSP-only `ROLE_ASSISTANT` and `ACTION_ASSIST` behavior stays out of
+the Play build.
+
+The Play-compatible `android-cloud` build must not request or expose
+default-assistant/system-only powers. Keep `ACTION_ASSIST`,
+`VOICE_COMMAND`, `ROLE_ASSISTANT`, `BIND_VOICE_INTERACTION`,
+usage-stats appop permissions, SMS/call default-role components, boot
+receivers, battery-optimization exemption, MediaProjection foreground
+services, and special-use foreground services out of that build.
+AOSP/default-assistant behavior belongs only to `android-system` or
+sideload-only validation builds.
+
+Current Android docs describe App Actions as `shortcuts.xml`
+capabilities registered on the launcher activity; Gemini/Assistant
+interoperability for general apps is through those App Actions and
+shortcuts. The navigation-app Gemini/Assistant intent formats are
+navigation-specific and are not a general personal-assistant integration
+surface for this app.
+
+## What requires a system-app build (AOSP flavor)
+
+| Capability | Mechanism | Permission |
+|---|---|---|
+| High-fidelity screen capture | `SurfaceControl.captureDisplay()` | `READ_FRAME_BUFFER` (`signature|privileged`) |
+| High-fidelity input injection | `InputManager.injectInputEvent()` | `INJECT_EVENTS` (`signature|privileged`) |
+| Full process enumeration | `IActivityManager.getRunningAppProcesses()` via AIDL | `REAL_GET_TASKS` (`signature`) |
+
+See `AOSP_SYSTEM_APP.md` for the privileged build path.
+
+## Advanced Protection Mode caveat
+
+When Advanced Protection Mode (APM) is active on Pixel 9+ and some OEM variants:
+
+- AccessibilityService registered with `featureAccessibility` (not `featureGeneric`) survives.
+  `ElizaAccessibilityService` is registered correctly.
+- Third-party AccessibilityServices with broad event masks may be killed on APM devices
+  even when re-enabled in Settings. If the service is repeatedly stopped, check `adb logcat`
+  for `AccessibilityManagerService` or `android.safetycenter` entries.
+
+## lmkd survival strategy
+
+The Linux low-memory killer daemon (lmkd) uses oom_score_adj to prioritize kills.
+Two mitigations are active:
+
+1. `ScreenCaptureService` is a foreground service — lmkd ranks foreground services
+   below cached apps; they survive until memory is critically exhausted.
+2. `onTrimMemory` → `capacitorPressureSource.dispatch()` — WS1 MemoryArbiter receives
+   the pressure signal and proactively unloads lower-priority model handles
+   (transcribe, vision-describe) before the OOM killer fires.
+
+## Manual on-device validation checklist
+
+Run this against a physical Android device (API 24+ for gesture dispatch; API 29+ for
+`FOREGROUND_SERVICE_MEDIA_PROJECTION`). Cuttlefish x86_64 emulator is acceptable for
+smoke-testing, but the x86_64 JNI patch must be present (see WS4 llama-cpp-capacitor patch).
+
+### 1. Permissions setup
+
+- [ ] Install the APK and open the app.
+- [ ] Grant `CAMERA` runtime permission when prompted.
+- [ ] Navigate to Settings > Accessibility > Eliza > enable the service.
+  Verify `ElizaAccessibilityService.instance` is non-null via:
+  `adb shell dumpsys accessibility | grep -i eliza`
+- [ ] Navigate to Settings > Digital Wellbeing (or Settings > Security > Usage Access)
+  > Eliza > enable Usage Access.
+
+### 2. AccessibilityService — view tree
+
+```
+adb shell am start com.example.app   # open any app
+curl -X POST http://localhost:1337/api/computer-use/getAccessibilityTree
+```
+Expected: JSON array with `[{id, role, label, bbox, actions}]` entries.
+Verify `role` values are Android class names (e.g. `android.widget.Button`).
+
+### 3. Gesture dispatch
+
+```
+curl -X POST http://localhost:1337/api/computer-use/dispatchGesture \
+  -d '{"type":"tap","x":540,"y":960}'
+```
+Expected: `{"ok":true}` and the tap is visible on screen.
+
+```
+curl -X POST http://localhost:1337/api/computer-use/dispatchGesture \
+  -d '{"type":"swipe","x":540,"y":1600,"x2":540,"y2":400,"durationMs":400}'
+```
+Expected: `{"ok":true}` and the list scrolls up.
+
+### 4. Global actions
+
+```
+curl -X POST http://localhost:1337/api/computer-use/performGlobalAction -d '{"action":"home"}'
+curl -X POST http://localhost:1337/api/computer-use/performGlobalAction -d '{"action":"recents"}'
+curl -X POST http://localhost:1337/api/computer-use/performGlobalAction -d '{"action":"back"}'
+curl -X POST http://localhost:1337/api/computer-use/performGlobalAction -d '{"action":"notifications"}'
+```
+Expected: each action is visually confirmed on device.
+
+### 5. MediaProjection — screen capture
+
+```
+curl -X POST http://localhost:1337/api/computer-use/startMediaProjection -d '{"fps":1}'
+```
+Expected: system consent dialog appears. Accept it.
+
+```
+curl -X GET http://localhost:1337/api/computer-use/captureFrame
+```
+Expected: `{ok:true, data:{jpegBase64:"...", width:..., height:..., timestampMs:...}}`.
+Verify `jpegBase64` decodes to a valid JPEG of the current screen.
+
+```
+curl -X POST http://localhost:1337/api/computer-use/stopMediaProjection
+```
+Expected: `{ok:true, data:{stopped:true}}`.
+
+### 6. UsageStats — app enumeration
+
+```
+curl -X GET http://localhost:1337/api/computer-use/enumerateApps
+```
+Expected: JSON array of `{packageName, label, lastUsedMs, totalForegroundMs, isForeground}`.
+Verify `isForeground:true` for the frontmost app.
+If you receive `{ok:false, code:"permission_denied"}`, confirm Usage Access is enabled.
+
+### 7. Camera capture
+
+```
+curl -X POST http://localhost:1337/api/computer-use/startCamera -d '{"fps":1}'
+```
+Expected: `{ok:true, data:{cameras:"[...]"}}` with at least one camera entry.
+
+```
+curl -X GET http://localhost:1337/api/computer-use/captureFrameCamera
+```
+Expected: `{ok:true, data:{jpegBase64:"..."}}`.
+
+```
+curl -X POST http://localhost:1337/api/computer-use/stopCamera
+```
+
+### 8. Memory pressure dispatch
+
+Open a memory-intensive app or use `adb shell am send-trim-memory $(pidof ai.eliza.eliza) 80`
+to simulate TRIM_MEMORY_RUNNING_CRITICAL.
+
+Expected: the JS console (or logcat for bridge events) shows:
+`[capacitorPressureSource] dispatching pressure: critical`
+followed by MemoryArbiter eviction log entries.
+
+Verify via `GET /api/training/auto/config` that arbiter pressure state transitions to `critical`.
+
+### 9. App Actions / static shortcuts
+
+Use a Play/Assistant-capable Pixel or Google Android device signed into
+the same account used by the App Actions test tool.
+
+- [ ] Confirm the launcher activity registers `@xml/shortcuts`:
+  `aapt dump xmltree app-release.aab base/manifest/AndroidManifest.xml`.
+- [ ] Confirm `shortcuts.xml` contains `OPEN_APP_FEATURE`,
+  `CREATE_MESSAGE`, and `GET_THING`, with no unsupported
+  `actions.intent.CREATE_THING`.
+- [ ] Confirm generated shortcuts use the app package and URL scheme for
+  the current brand; no `ai.elizaos.app`, `app.eliza`, or stale `eliza://`
+  value should remain after rewriting.
+- [ ] Trigger chat/ask, voice, new task, daily brief, and tasks from the Assistant
+  preview. Expected: Eliza opens via a source-tagged deep link and any
+  LifeOps mutation goes through the app/runtime `ScheduledTask` path.

plugins/plugin-computeruse/docs/AOSP_SYSTEM_APP.md

@@ -0,0 +1,112 @@
+# AOSP system-app deployment (computer-use on the Eliza AOSP fork)
+
+> Status: **design + contract**. The desktop CUA surface (capture, input, scene,
+> Brain→Cascade→dispatch) is production; the AOSP path is partially built —
+> `createAospInputActor()` returns `null` in consumer builds today
+> (`src/actor/aosp-input-actor.ts`), and the privileged bridge below is the
+> remaining work. This doc is the contract that wiring must satisfy. See
+> [`ANDROID_CONSTRAINTS.md`](./ANDROID_CONSTRAINTS.md) for the honest scope on
+> stock (non-system) Android, and `docs/android-aosp-validation.json` for the
+> release evidence manifest this path is gated on.
+
+## Why a system app
+
+On stock Android an app cannot freely read the framebuffer of *other* apps or
+inject input into them — those are `signature|privileged` permissions. The Eliza
+AOSP fork ships the agent as a **platform-signed system app** (or a privileged
+companion service), which unlocks the two capabilities a computer-use agent
+needs:
+
+| Capability | Stock Android | AOSP system app |
+|---|---|---|
+| Full-screen capture | `MediaProjection` (per-session user consent, own-app-biased) | `SurfaceControl.captureDisplay` / `READ_FRAME_BUFFER` (no prompt, any display) |
+| Global input injection | none (only own UI / a11y gestures) | `InputManager.injectInputEvent` with `INJECT_EVENTS` |
+| Element grounding | `AccessibilityService` tree (opt-in) | `AccessibilityService` tree + OCR, always available |
+
+## Manifest / signing
+
+```xml
+<!-- Platform-signed; placed on the system image (priv-app). -->
+<uses-permission android:name="android.permission.READ_FRAME_BUFFER" />
+<uses-permission android:name="android.permission.INJECT_EVENTS" />
+<uses-permission android:name="android.permission.ACCESS_SURFACE_FLINGER" />
+<application android:sharedUserId="android.uid.system" ... />
+```
+
+- Sign with the platform key; install under `/system/priv-app` (or
+  `/product/priv-app`) with a matching `privapp-permissions` allowlist entry for
+  the two signature permissions.
+- Non-system builds **must** degrade to the `MediaProjection` + `AccessibilityService`
+  path (see ANDROID_CONSTRAINTS.md). The plugin feature-detects at runtime and
+  never assumes the privileged path.
+
+## Capture — `ScreenState` source
+
+The AOSP capture feeds the same per-display frame contract the desktop scene
+uses (`DisplayCapture { display, frame: PNG }`, see `src/platform/capture.ts`):
+
+- **Privileged**: `SurfaceControl.captureDisplay(DisplayCaptureArgs)` →
+  `HardwareBuffer` → PNG. No user prompt; supports secondary/virtual displays.
+- **Fallback**: `MediaProjection` + `ImageReader` (one-time user consent), wired
+  in `src/mobile/android-scene.ts` / `mobile-screen-capture.ts`.
+
+Both paths populate the Android scene (`android-scene.ts`) with display bounds so
+multi-display coordinate translation (`src/platform/coords.ts`) works unchanged.
+
+## Input — `AospPrivilegedInputBridge`
+
+`src/actor/aosp-input-actor.ts` is the seam. `createAospInputActor()` returns
+`null` today; the privileged bridge must implement the `ComputerInterface` verbs
+via `InputManager.injectInputEvent`:
+
+| Verb | Implementation |
+|---|---|
+| `click` / `double_click` / `long_press` | `MotionEvent` ACTION_DOWN→UP (→DOWN→UP) at (x,y), `INJECT_INPUT_EVENT_MODE_ASYNC` |
+| `drag` | ACTION_DOWN → interpolated ACTION_MOVE waypoints → ACTION_UP (mirror the desktop manual-drag in `nut-driver.ts`) |
+| `scroll` | per-notch ACTION_SCROLL or MOVE deltas (mirror the desktop per-notch split) |
+| `type` | `KeyEvent` stream or `InputConnection` commit |
+| `key` / `key_combo` | `KeyEvent` with meta-state |
+
+Coordinates are **logical display pixels** (same contract the empirical Windows
+probe established for nutjs); the bridge applies the display's density only if
+`injectInputEvent` is found to operate in physical pixels on the target ROM —
+verify empirically before adding any scale, exactly as documented for the
+desktop driver.
+
+## Grounding — Brain-less, Actor-only (cheap by construction)
+
+There is **no mobile Brain** (no full-frame VLM reasoning loop on device). AOSP
+uses the **Actor-only** path: `OcrCoordinateGroundingActor` resolves a target by
+matching `Scene.ocr` (OCR boxes) and `Scene.ax` (`AccessibilityNodeInfo`
+clickables) — exactly the structured readout `GET_SCREEN` produces (#9105 M2).
+This means AOSP gets computer-use **without a remote VLM**:
+
+1. `android-scene.ts` builds `Scene` from MediaProjection/SurfaceControl capture
+   + the `AccessibilityNodeInfo` tree + OCR.
+2. OCR provider: register **PaddleOCR PP-OCRv5 mobile (Paddle-Lite)** as the
+   AOSP `CoordOcrProvider` via `registerCoordOcrProvider` (the same seam
+   plugin-vision uses on desktop, #9105 M1) — on-device, ARM-optimized, zero LLM
+   tokens.
+3. `Cascade` grounds via `resolveReference` (OCR/AX text-match) with the
+   per-Scene grounding cache (#9105 M5); `dispatch` routes the concrete coords to
+   the `AospPrivilegedInputBridge`.
+
+## Remaining work (tracked in #9105 M6)
+
+- [ ] Implement `AospPrivilegedInputBridge` and return it from
+      `createAospInputActor()` when the privileged permissions are held.
+- [ ] `SurfaceControl.captureDisplay` capture path + `MediaProjection` fallback
+      feeding `android-scene.ts`.
+- [ ] Register Paddle-Lite PP-OCRv5 as the AOSP `CoordOcrProvider`.
+- [ ] Empirically verify `injectInputEvent` coordinate space (logical vs
+      physical) on the target ROM before any density scaling.
+- [ ] Complete `docs/android-aosp-validation.json` evidence (capture + a click
+      landed via injection + an OCR-grounded tap) and gate the release on it.
+
+## Security
+
+All destructive actions still pass through `ComputerUseApprovalManager`
+(`smart_approve` by default). A platform-signed agent with `INJECT_EVENTS` is a
+high-privilege component — keep approval gating on, scope the priv-app allowlist
+to exactly the two signature permissions, and never expose the injection bridge
+over an unauthenticated IPC surface.

plugins/plugin-computeruse/docs/IOS_CONSTRAINTS.md

@@ -0,0 +1,233 @@
+# iOS Computer-Use — Honest Scope
+
+Apple does not let third-party apps drive other apps. There is no equivalent
+to Android's `MediaProjection` for full-screen capture without a user prompt,
+no `ScreenCaptureKit` (that is macOS only), and no system-wide accessibility
+API that a sandboxed iOS app can use to inject events into other apps.
+
+This document spells out what *is* possible on iOS, exactly how each surface
+behaves, and the manual validation checklist that has to be run on a real
+device before any of this ships.
+
+## What works
+
+### 1. ReplayKit foreground capture (own app)
+
+`RPScreenRecorder.shared().startCapture(handler:completionHandler:)` returns
+`CMSampleBuffer`s of *the host app's* window. Useful for "show me what's on
+the Eliza screen right now."
+
+```swift
+RPScreenRecorder.shared().startCapture(handler: { sampleBuffer, type, error in
+    // Receive frames for own-app capture only.
+}, completionHandler: { error in
+    // Setup result; ReplayKit shows a system permission prompt on first use.
+})
+```
+
+Constraints:
+
+- Frame delivery rate is the display refresh rate; the bridge throttles to a
+  caller-supplied `frameRate` (default 1Hz).
+- Hard cap of 30 seconds per session. Past that, the work has to move to a
+  `BGProcessingTask`.
+
+### 2. ReplayKit broadcast extension (system-wide capture)
+
+A separate target with a ~50MB memory ceiling. Streams frames over an App
+Group shared container into the main app. The user must start the broadcast
+themselves via the system share-sheet picker — apps cannot programmatically
+launch the extension.
+
+```swift
+override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer,
+                                  with sampleBufferType: RPSampleBufferType) {
+    // Compress + dump into the App Group container.
+}
+```
+
+**iOS 26 / 26.1 beta regression**: extensions are killed within ~3 seconds
+even when memory headroom is fine. We surface this as
+`extension_died` from `broadcastExtensionHandshake` so callers can fall back
+to foreground capture. Track Apple's feedback status before shipping this
+target on iOS 26.
+
+### 3. Apple Vision OCR
+
+`VNRecognizeTextRequest` is on-device, free, supports ~30 languages, and
+typically runs sub-300ms on modern devices.
+
+```swift
+let request = VNRecognizeTextRequest { request, error in
+    let observations = request.results as? [VNRecognizedTextObservation]
+    // Top candidate per observation; each has bounding box, text, confidence.
+}
+request.recognitionLevel = .accurate
+request.recognitionLanguages = ["en-US"]
+try VNImageRequestHandler(cgImage: cg).perform([request])
+```
+
+This is the OCR provider the WS6 scene-builder will pick up on iOS. The
+provider interface is in
+`eliza/plugins/plugin-computeruse/src/mobile/ocr-provider.ts`.
+
+### 4. App Intents
+
+The only sanctioned way to drive other apps. Each target app must expose
+intents (Shortcuts-style). We support invocation via x-callback URL schemes
+for the system apps in the static registry
+(`ios-app-intent-registry.ts`):
+
+- Mail — `mailto:` with `subject` / `body` / `cc` / `bcc`
+- Messages — `sms:` with `body`
+- Maps — `http://maps.apple.com/?daddr=...&dirflg=...`
+- Safari — open URL
+
+For richer intents (Notes append, Reminders add, Music play with a query)
+the user has to donate the action via Shortcuts; we can then invoke via
+`AppIntent` on iOS 16+. The bridge's `appIntentList` returns the runtime
+list of donated intents this app sees.
+
+### 5. UIAccessibility (own-app reading only)
+
+`accessibilitySnapshot` walks the key window's view hierarchy and returns
+`accessibilityLabel` / `accessibilityValue` / role. iOS gives us no way to
+read another app's UIAccessibility tree.
+
+### 6. Apple Foundation Models (iOS 26+)
+
+Apple ships an on-device LLM under the `FoundationModels` framework when
+Apple Intelligence is enabled. We expose this as an *opportunistic*
+fast-path. If unavailable, the existing llama-cpp-capacitor local Eliza-1
+vision path stays as the default.
+
+Entitlement and Info.plist updates are listed below.
+
+## What does NOT work
+
+Stock iOS does not allow any of the following from a third-party app, and
+we do not pretend otherwise:
+
+- **Driving other apps' UI**. No cross-app input synthesis, no
+  `MediaProjection` / `ScreenCaptureKit` equivalent, no system-wide
+  accessibility event injection.
+- **Listing other running apps' processes**. The kernel hides this from
+  third-party apps; there is no `ps`-equivalent.
+- **Persistent background inference past ~30s**, except via
+  `BGProcessingTask` (opportunistic, OS-scheduled, not guaranteed).
+
+If a feature spec calls for any of the above, escalate it as not feasible
+on iOS rather than inventing a workaround that will get the app rejected
+or silently broken by Apple in the next OS update.
+
+## Entitlements + Info.plist
+
+Add the following to `apps/app/ios/App/App/App.entitlements`:
+
+```xml
+<key>com.apple.developer.kernel.increased-memory-limit</key>
+<true/>
+<key>com.apple.developer.kernel.extended-virtual-addressing</key>
+<true/>
+```
+
+Add the following to `apps/app/ios/App/App/Info.plist`:
+
+```xml
+<key>NSScreenCaptureDescription</key>
+<string>Captures the Eliza app window when you ask it to see the screen.</string>
+<key>NSAppleEventsUsageDescription</key>
+<string>Sends Shortcuts to other apps when you authorize an action.</string>
+```
+
+The screen capture string is shown on ReplayKit's first-launch system
+prompt. The Apple Events string is shown only if the host (this app) ever
+needs to drive AppleScript via the Catalyst surface; iOS apps that only
+use `UIApplication.shared.open(url:)` to invoke x-callback intents do not
+need it, but it costs nothing to include for forward compatibility with
+Mac Catalyst builds.
+
+## Validation checklist
+
+The iOS bridge contract is covered by TypeScript tests, but physical iOS
+device behavior is not proven by this repository alone. The required evidence
+manifest is `docs/ios-device-validation.json`; keep it in
+`requires_device_evidence` until a real device run records device/build
+metadata, artifacts, and per-method results. Release gates that require
+physical proof should run:
+
+```bash
+bun run --cwd plugins/plugin-computeruse validate:ios-device-evidence -- --require-complete
+```
+
+Without `--require-complete`, the same command validates that the manifest
+still tracks every required method and evidence field. Before shipping any
+iOS bridge release, complete the manifest for:
+
+### Simulator vs device
+
+| Surface                          | Simulator | Real device |
+| -------------------------------- | --------- | ----------- |
+| ReplayKit foreground capture     | Partial   | Required    |
+| Broadcast extension              | No        | Required    |
+| Vision OCR                       | Yes       | Required    |
+| App Intents (x-callback)         | Partial   | Required    |
+| Accessibility snapshot           | Yes       | Required    |
+| Foundation Models                | No        | Required    |
+| Memory pressure probe            | Limited   | Required    |
+
+### Per-method checklist
+
+For each method below, run on at least one A14-or-later iPhone running the
+target iOS (currently iOS 26.1, with iOS 17.6 as the floor):
+
+1. **`probe()`** — assert `data.platform === "ios"`, `osVersion` matches the
+   device, all six capability bits are present.
+2. **`replayKitForegroundStart`** — assert the system permission prompt is
+   shown on first use; on accept, `replayKitForegroundDrain` returns frames
+   with non-empty `jpegBase64` and the device's screen resolution.
+3. **`broadcastExtensionHandshake`** — bundle the extension target via Xcode,
+   tap the share-sheet broadcast picker, and assert `broadcastActive`
+   transitions to `true`. On iOS 26 betas, verify `regression.observed`
+   eventually flips to `true` after the extension dies.
+4. **`visionOcr`** — render a known PNG with the string `"WS9 OCR SMOKE"`
+   into the app, run OCR with `recognitionLevel: "accurate"`, and assert
+   `fullText` contains the source string (case-insensitive).
+5. **`appIntentList`** — verify the returned list is empty for a fresh
+   install and grows after the user donates intents via Shortcuts.
+6. **`appIntentInvoke`** with `com.apple.mobilesafari.open-url` — assert
+   Safari opens to the provided URL. With `com.apple.MobileSMS.send-message`
+   — assert Messages opens with the recipient and body pre-filled.
+7. **`accessibilitySnapshot`** — assert the returned tree's top-level node
+   has `role !== "labeled"` and `children.length > 0` on a populated screen.
+8. **`foundationModelGenerate`** — on a device with Apple Intelligence
+   enabled, assert a short prompt returns non-empty text. With AI disabled,
+   assert the call resolves with `foundation_model_unavailable`.
+9. **`memoryPressureProbe`** — invoke `UIApplication.shared.performMemoryWarning()`
+   (debug-only) and assert the next probe call returns
+   `severity >= 0.7` with `lastWarningAt` set.
+
+## Hand-off to other workstreams
+
+- **WS1 (memory-pressure arbiter)** consumes `MemoryPressureSample` via the
+  shared `IPressureSignal` contract in `ios-bridge.ts`. The bridge is the
+  producer; the arbiter is the consumer. WS1 will subscribe to push events
+  via the bridge once that channel lands; for now it polls `memoryPressureProbe`.
+- **WS6 (scene-builder OCR)** picks up the iOS Vision provider through
+  `selectOcrProvider()` in `ocr-provider.ts`. Register the provider at app
+  boot when running on iOS:
+  ```ts
+  registerOcrProvider(
+    createIosVisionOcrProvider(() => Capacitor.Plugins.ComputerUse),
+  );
+  ```
+
+## Apple documentation references
+
+- ReplayKit — https://developer.apple.com/documentation/replaykit
+- Vision Text Recognition — https://developer.apple.com/documentation/vision/recognizing_text_in_images
+- App Intents — https://developer.apple.com/documentation/appintents
+- Foundation Models — https://developer.apple.com/documentation/foundationmodels (iOS 26)
+- App Groups — https://developer.apple.com/documentation/xcode/configuring-app-groups
+- `os_proc_available_memory()` — https://developer.apple.com/documentation/foundation/process_info/3743117-os_proc_available_memory
+- Background Tasks — https://developer.apple.com/documentation/backgroundtasks

plugins/plugin-computeruse/docs/MOBILE_ASSISTANT_ROUTING.md

@@ -0,0 +1,174 @@
+# Siri, App Actions, and LifeOps Routing
+
+Assistant entry surfaces are launch and capture surfaces. They are not a
+second task engine.
+
+Siri/App Shortcuts, Android App Actions, Android assistant-role intents, and
+desktop shortcut surfaces should route the user's utterance or tap into the
+Eliza app/runtime. If the user asks for a reminder, check-in, follow-up,
+watcher, recap, or approval, the runtime must create or update a LifeOps
+`ScheduledTask`. Native mobile code may deliver notifications for tasks that
+already exist, but it must not create native-only reminder state that bypasses
+LifeOps.
+
+## Intended Flow
+
+1. The OS entry surface opens Eliza with a source-tagged deep link or assistant
+   intent payload.
+2. The app/runtime normalizes the payload into the same planner path used by
+   chat and device-bus requests.
+3. LifeOps creates or updates a `ScheduledTask` through the app/runtime.
+4. Native notification APIs only deliver output for that task or wake the
+   runtime so the scheduled-task runner can decide what fires.
+
+## Platform Contracts
+
+| Platform | Entry surface | Required routing contract |
+|---|---|---|
+| Mac | Shortcuts/deep link/menu-bar voice entry | Opens the app/runtime; LifeOps mutations are persisted as `ScheduledTask` records. |
+| iOS | Siri/App Shortcuts/App Intents | Uses App Intents or app deep links to hand off to the app/runtime. No cross-app UI driving, and no native-only reminder store. |
+| Android consumer | Static shortcuts + App Actions | `shortcuts.xml` and assistant intents open the app custom scheme; LifeOps scheduling happens after runtime routing. |
+| Android AOSP | `ROLE_ASSISTANT` + privileged system app | Assistant role may wake Eliza more directly; AOSP-only accessibility/notification-listener services are available, but scheduled behavior still goes through the same `ScheduledTask` runner. |
+
+Android consumer App Actions coverage:
+
+| Flow | BII/static shortcut |
+|---|---|
+| Ask/chat | `actions.intent.CREATE_MESSAGE`, `actions.intent.GET_THING`, `eliza_app_action_chat` |
+| Voice chat | `actions.intent.OPEN_APP_FEATURE` inline inventory, `eliza_app_action_voice` |
+| LifeOps daily brief | `actions.intent.OPEN_APP_FEATURE` inline inventory, `eliza_app_action_daily_brief` |
+| LifeOps task creation | `actions.intent.OPEN_APP_FEATURE` inline inventory, `eliza_app_action_new_task` |
+| LifeOps task list | `actions.intent.OPEN_APP_FEATURE` inline inventory, `eliza_app_action_tasks` |
+
+Each Android App Actions capability keeps a fallback fulfillment without a
+required parameter. Feature-specific requests route through
+`elizaos://feature/open?...`; vague requests fall back to the chat route so
+the runtime can disambiguate instead of executing native-only behavior.
+
+The Play/Pixel build does not request `ROLE_ASSISTANT`,
+`ACTION_ASSIST`, `VOICE_COMMAND`, or `BIND_VOICE_INTERACTION`; those are
+reserved for AOSP/system validation builds. It also avoids unsupported BIIs
+such as `actions.intent.CREATE_THING`; task creation is modeled as opening
+the LifeOps task feature and letting the app/runtime confirm any mutation.
+
+## External Assistant Landscape
+
+Checked 2026-05-14 against primary vendor docs:
+
+- OpenAI/ChatGPT on Apple platforms is exposed through Apple's ChatGPT
+  extension inside Apple Intelligence and Siri settings, not as a third-party
+  Siri API pattern for other apps to copy. See
+  <https://help.openai.com/en/articles/10263570-apple-intelligence-siri-faq>
+  and
+  <https://help.openai.com/en/articles/10269382-setting-up-chatgpt-with-apple-intelligence>.
+- Apple's supported third-party integration path is App Intents/App Shortcuts.
+  App Intents expose app capabilities to Shortcuts, Spotlight, Siri, widgets,
+  controls, and Apple Intelligence experiences. See
+  <https://developer.apple.com/documentation/appintents/app-intents>,
+  <https://developer.apple.com/shortcuts/>, and
+  <https://developer.apple.com/documentation/appintents/app-intent-domains>.
+- Claude uses the same Apple-native pattern: an "Ask Claude" App Intent
+  available from Siri, Spotlight, the Share menu, and Shortcuts. See
+  <https://support.anthropic.com/en/articles/10263469-using-claude-app-intents-shortcuts-and-widgets-on-ios>.
+- Perplexity exposes iOS voice assistant entry through Shortcuts, Lock Screen
+  controls, and the Action Button, including "Ask Perplexity" and "Start voice
+  conversation" shortcut actions. See
+  <https://www.perplexity.ai/help-center/en/articles/11132456-how-to-use-the-perplexity-voice-assistant-for-ios>.
+- Grok/X official docs currently document Grok availability on iOS, Android,
+  and web, including voice and text interactions, but do not document a public
+  Siri/App Intent shortcut contract comparable to Claude's. See
+  <https://help.x.com/en/using-x/about-grok>.
+- Google Assistant/App Actions are driven by capabilities in `shortcuts.xml`.
+  Assistant matches a built-in intent declared there and launches the app to
+  the requested screen. See
+  <https://developer.android.com/develop/devices/assistant/action-schema>.
+
+## Validation Checklist
+
+### Mac
+
+- [ ] Trigger the Mac shortcut/voice entry surface with a plain chat request
+  and confirm it opens the app/runtime instead of executing native-only logic.
+- [ ] Ask for a one-off reminder and verify a LifeOps `ScheduledTask` record is
+  created with `kind: "reminder"`.
+- [ ] Ask for a recurring check-in or follow-up and verify cadence lives in the
+  `ScheduledTask.trigger`/relationship edge, not in a platform notification.
+
+### iOS
+
+- [ ] Verify Siri/App Shortcuts open Eliza or invoke the App Intent handoff.
+- [ ] Verify `appIntentList` only reports local app/donated intents and does
+  not claim cross-app enumeration.
+- [ ] Ask Siri for a LifeOps reminder/check-in/follow-up and verify the app
+  creates a `ScheduledTask` through the runtime.
+- [ ] Confirm native `UNUserNotificationCenter` entries, if any, reference an
+  existing task or deep link back into the app.
+
+### Android Consumer
+
+- [ ] Verify `AndroidManifest.xml` registers `@xml/shortcuts` on `MainActivity`.
+- [ ] Verify the `eliza_app_action_chat` static shortcut opens the app chat
+  route.
+- [ ] Trigger an App Action and confirm the app/runtime receives the request.
+- [ ] Ask for a reminder/check-in/follow-up and verify LifeOps stores a
+  `ScheduledTask`; no Java/Kotlin-only reminder table or notification-only
+  schedule is created.
+
+### Android AOSP Assistant Role
+
+- [ ] On the system image, confirm `ROLE_ASSISTANT` resolves to Eliza.
+- [ ] Trigger `android.intent.action.ASSIST` and `VOICE_COMMAND`; both should
+  reach Eliza's app/runtime.
+- [ ] Confirm `ElizaAccessibilityService` and
+  `ElizaNotificationListenerService` are present only on the AOSP/system APK,
+  not the Play/Pixel cloud APK.
+- [ ] Confirm privileged capture/input capabilities do not change LifeOps
+  persistence: reminders/check-ins/follow-ups still use `ScheduledTask`.
+
+## Static Coverage
+
+- `ios-bridge.test.ts` checks that the TypeScript iOS AppIntent registry stays
+  aligned with the native x-callback switch.
+- `android-bridge.test.ts` checks that Android assistant/App Actions source
+  files route into app deep links rather than native notification creation.
+
+## Current Status and Remaining Work
+
+Implemented static/build coverage:
+
+- iOS App Shortcuts/App Intents are present in the native iOS target and hand
+  off source-tagged deep links to the app runtime.
+- Android App Actions metadata is present in `shortcuts.xml`, is registered on
+  `MainActivity`, and is rewritten by `run-mobile-build.mjs` to the configured
+  package and URL scheme.
+- Android assistant-role launch uses the same template deep-link scheme as the
+  other Android bridge activities, so white-label builds are rewritten
+  consistently.
+- Android AOSP system builds declare `ElizaAccessibilityService` and
+  `ElizaNotificationListenerService`; `android-cloud` strips those services,
+  their Java sources, and the accessibility-service XML resource.
+- Assistant launch payloads with captured text are consumed by the chat surface
+  and sent through the normal chat/planner path with source metadata. LifeOps
+  task/reminder deep links with text route to chat first so `ScheduledTask`
+  creation remains owned by the LifeOps runtime.
+
+Live-device validation is tracked through evidence manifests and platform
+release gates:
+
+- macOS shortcut/voice entry has installer/verifier scripts, but the final Siri
+  phrase creation and spoken-phrase result are still per-device manual checks.
+- iOS Siri/App Shortcuts must be installed on a physical device and recorded
+  in `docs/ios-device-validation.json` alongside the spoken phrases in
+  `ElizaAppShortcuts.swift`.
+- Android consumer App Actions require a Play/Assistant-capable device or test
+  environment to confirm Assistant fulfillment, not just static XML validity.
+- Android AOSP assistant-role behavior still needs a system-image validation
+  pass.
+
+Known remaining product test:
+
+- Static tests prove source routing and build-time wiring, but they do not prove
+  the planner produced the intended `ScheduledTask`. The remaining product test
+  is live end-to-end validation: speak or invoke a LifeOps reminder/check-in,
+  confirm the assistant launch payload sends through chat/planner, and verify a
+  LifeOps `ScheduledTask` record exists.

plugins/plugin-computeruse/docs/MULTI_MONITOR.md

@@ -0,0 +1,123 @@
+# Multi-monitor capture & coordinate contract (WS5)
+
+This plugin treats every physical display as an independent scene. There is
+NO virtual-desktop coordinate space exposed to the agent or the model.
+
+## Display enumeration
+
+`platform/displays.ts` returns the live attached set:
+
+```ts
+{
+  id: number,                          // OS-stable handle or 0-based index
+  bounds: [x, y, w, h],                // OS-global pixel space
+  scaleFactor: number,                 // 1 on Linux/Win, >1 on retina
+  primary: boolean,
+  name: string                         // e.g. "eDP-1", "DISPLAY1"
+}
+```
+
+Per-OS source:
+
+| OS              | Source                                                                  |
+| --------------- | ----------------------------------------------------------------------- |
+| Linux X11       | `xrandr --listmonitors`                                                 |
+| Linux Wayland   | `hyprctl monitors -j` / `swaymsg -t get_outputs` (else falls back to X) |
+| macOS           | `system_profiler SPDisplaysDataType -json`, then JXA `CGMainDisplayID`  |
+| Windows         | PowerShell `[Screen]::AllScreens`                                       |
+
+Native sidecars (Swift ScreenCaptureKit, DXGI/WGC, Rust libdisplay) are a
+follow-up — the interface is shaped to absorb them without changing callers.
+
+## Per-display capture
+
+`platform/capture.ts` is the canonical capture entry point:
+
+```ts
+captureDisplay(id):       Promise<{ display, frame: PNG-Buffer }>
+captureAllDisplays():     Promise<DisplayCapture[]>
+captureDisplayRegion():   Promise<DisplayCapture>     // local-to-display region
+```
+
+`frame` is at backing-store resolution. On a 2× retina display reporting a
+2560×1440 logical bounds with `scaleFactor: 2`, the PNG is 5120×2880.
+
+The legacy single-display `captureScreenshot()` from `screenshot.ts` is still
+exported for back-compat, but new code should prefer the per-display path.
+
+## Coordinate contract
+
+Every coordinate-bearing action accepts:
+
+```ts
+{
+  displayId: number,           // which display the coords belong to
+  coordinate: [x, y],          // LOCAL to that display
+  coordSource?: "logical"|"backing"  // default "logical"
+}
+```
+
+`platform/coords.ts::localToGlobal` translates to OS-global before the input
+driver fires. The model never sees OS-global coords.
+
+### Per-OS translation
+
+- **Linux/X11**: global = display.x + local.x. Pixels-to-pixels, no DPI.
+- **Windows (PerMonitorV2)**: same. Process MUST be PerMonitorV2 DPI-aware
+  (manifest entry — Electrobun handles this for the desktop app).
+- **macOS**: Quartz event coords are in *points* (logical pixels), not the
+  backing-store resolution of the capture. If the model used the raw retina
+  capture to choose coords, it must declare `coordSource: "backing"` so the
+  translator divides by `scaleFactor` before adding the display origin.
+
+### Why local-first
+
+1. Local coords match the model's actual visual context — it analyzed one
+   capture, not a stitched virtual desktop.
+2. Virtual-desktop coords are a perpetual bug source: negative origins
+   when secondary is left-of-primary, mixed DPI tiles, X11 vs CG origin
+   conventions.
+3. `displayId` is opaque to the model — it just echoes whatever the
+   `displays[]` provider gave it.
+
+### Backwards compatibility
+
+If `displayId` is omitted, the service:
+
+1. On single-display hosts: silently defaults to primary (debug-log only).
+2. On multi-display hosts: warns once per process, then defaults to
+   primary. Plan: this fallback will be removed once all in-tree callers
+   are migrated.
+
+## Ultrawide
+
+> 21:9 displays are NOT sliced. Each `displayId` is one scene. The
+aspect-aware patcher in WS6/WS7 sends the whole frame at the model's
+`max_pixels` budget; M-RoPE preserves aspect inside the model.
+
+## Provider surface
+
+`computerState` provider includes `data.displays: DisplayDescriptor[]` and
+in-text `computer_use.displays`. The planner reads this to pick a target
+display before issuing any coordinate-bearing COMPUTER_USE action.
+
+## What's manually validated
+
+This Linux test host has a single display. Automated tests cover:
+
+- `parseXrandrMonitors()` — string → DisplayInfo[] with golden fixtures.
+- `parseHyprlandMonitors()`, `parseSwayOutputs()` — JSON → DisplayInfo[].
+- `parseSystemProfilerDisplays()` — macOS JSON → DisplayInfo[].
+- `parseWindowsScreens()` — PowerShell JSON → DisplayInfo[].
+- `localToGlobal` / `globalToLocal` round-trip.
+
+Real multi-monitor capture & input injection on each OS still needs a
+manual rig:
+
+| What                                  | OS      | Manual check                                         |
+| ------------------------------------- | ------- | ---------------------------------------------------- |
+| `screencapture -D 2`                  | macOS   | dual display rig, capture each independently         |
+| Retina backing-store scale            | macOS   | scaleFactor=2, capture is 2× logical bounds          |
+| WGC / DXGI vs PerMonitorV2 DPI        | Windows | per-monitor DPI mix, click lands on correct pixel    |
+| Wayland portal capture                | Linux   | GNOME 45+ / KDE 6 — `platform/wayland-portal.ts` captures through xdg-desktop-portal; per-display crop still depends on compositor support |
+| Hyprland / Sway parser                | Linux   | run `hyprctl monitors -j` against live compositor    |

plugins/plugin-computeruse/docs/TEST_LANES_COMPUTERUSE_VISION.md

@@ -0,0 +1,148 @@
+# Test lanes — computeruse × vision (per-OS)
+
+How the `@elizaos/plugin-computeruse` + `@elizaos/plugin-vision` test surface is
+organized across the unit lane and the real-driver lane, what runs where, the
+per-OS host requirements, and the **Windows non-interactive-session gotcha** that
+makes input-effect tests fail outside a logged-in desktop. Companion to the
+CUA × Vision EPIC (#9105) and the trycua/cua parity tracker (#9170).
+
+## Lanes at a glance
+
+| Lane | Selects | Runs the real OS? | Command |
+|------|---------|-------------------|---------|
+| **Unit / component** | every `*.test.ts` **except** `*.real`/`*.live`/`*.e2e` | No (mocked) — but platform-gated `*.test.ts` self-skip on the wrong OS | `bun run --cwd plugins/plugin-computeruse test` · `bun run --cwd plugins/plugin-vision test` |
+| **Real-driver / live** | `*.real.test.ts` / `*.live.test.ts` (repo-wide) | **Yes** — nutjs / PowerShell / WinRT / Apple Vision / xdotool against the host | shared config `packages/test/vitest/real.config.ts` (post-merge lane via `packages/scripts/run-all-tests.mjs`) |
+| **Standalone probe** | a hand-written `.mts` importing `src/platform/*.js` | **Yes** — fastest Windows real smoke | `bun plugins/plugin-computeruse/<probe>.mts` |
+
+### Unit lane
+
+Each plugin's `vitest.config.ts` **excludes** `**/*.real.test.{ts,tsx}`,
+`**/*.live.test.{ts,tsx}`, and `**/*.e2e.*`. So `bun run --cwd plugins/plugin-computeruse test`
+never touches the real OS.
+
+Platform-specific behavior that *can* be exercised without a real desktop still
+lives in the unit lane as a regular `*.test.ts` gated with
+`it.skipIf(platform() !== "win32")(...)`. These **run on Windows in the normal
+lane** and self-skip elsewhere. Examples:
+
+- `plugin-vision/src/ocr-service-windows.test.ts` — renders a PNG and OCRs it
+  through the real **`Windows.Media.Ocr`** WinRT engine (works headless).
+- `plugin-computeruse/src/__tests__/cua-parity-surface.test.ts` — Windows
+  clipboard-write command regression (`Set-Clipboard -Value
+  [Console]::In.ReadToEnd()`, not `$input | Set-Clipboard`).
+
+### Real-driver / live lane
+
+`*.real.test.ts` files are **only** picked up by the shared real config
+`packages/test/vitest/real.config.ts` (include globs `**/*.real.test.ts`,
+`**/*.live.test.ts`). They drive the real input/capture/OCR stack. A
+`fail-on-silent-skip` setup (`packages/test/vitest/fail-on-silent-skip.setup.ts`)
+**fails any test that silently skips**, so a `.real` test must either run or be
+explicitly excluded — it cannot quietly no-op.
+
+Run a single real test against the host (the lane's include globs are repo-wide,
+so pass a file path to scope it):
+
+```bash
+bunx vitest run plugins/plugin-computeruse/src/__tests__/cua-parity-input.real.test.ts \
+  --config packages/test/vitest/real.config.ts
+```
+
+`ELIZA_CI_REAL=1` additionally drops credential/upstream-gated reals
+(e.g. `computeruse.real.test.ts`, whose headless-browser path needs a display).
+
+### Standalone probe (proven Windows smoke)
+
+For a fast Windows real-driver check, a standalone `.mts` that imports the
+platform modules directly and is run with `bun` beats the vitest real lane
+(no config graph, instant feedback). Pattern:
+
+```ts
+// parity-probe.mts — bun plugins/plugin-computeruse/parity-probe.mts
+import { legacyGetCursorPosition } from "./src/platform/desktop.js";
+import { readClipboard, writeClipboard } from "./src/platform/clipboard.js";
+console.log(legacyGetCursorPosition());          // live WinForms read
+await writeClipboard("hi"); console.log(await readClipboard());
+```
+
+Keep probes out of commits (delete after use) — they are smoke checks, not the
+suite.
+
+## Input driver selection
+
+`ELIZA_COMPUTERUSE_DRIVER` selects the input backend:
+
+- `nutjs` (default) — `@nut-tree-fork/nut-js` native bindings.
+- `legacy` — per-OS shell tools: PowerShell (Windows), `cliclick` (macOS),
+  `xdotool` (Linux).
+
+nutjs auto-falls-back to `legacy` when the native module fails to load. Note the
+read path is asymmetric on Windows: `driverGetCursorPosition` always uses the
+WinForms OS query on Windows because nutjs `mouse.getPosition()` returns a stale
+constant there (see #9165).
+
+## Per-OS host requirements
+
+| OS | Capture | Input | OCR | Notes |
+|----|---------|-------|-----|-------|
+| **Windows** | PowerShell / nutjs | nutjs (SendInput) or legacy PowerShell | **`Windows.Media.Ocr`** (WinRT via `powershell` 5.1, **not** pwsh 7) — 0 tokens | **Input needs an interactive desktop** (see gotcha). Read/clipboard/OCR work headless. |
+| **macOS** | `screencapture -D` (retina 2× backing store) | nutjs or `cliclick` | Apple Vision | Needs Accessibility + Screen Recording grants; headful. |
+| **Linux** | X11 `import`/`scrot` (Xvfb headful); Wayland `xdg-desktop-portal` screenshot sidecar | nutjs or `xdotool` on X11 | docTR / PaddleOCR | Wayland capture uses `python3` + `gdbus`; Wayland still lacks AT-SPI in many compositors, so grounding can be OCR-only. Clipboard needs `wl-clipboard`/`xclip`. |
+| **AOSP** | MediaProjection | privileged input bridge | Paddle-Lite | Emulator + system-app path. See `AOSP_SYSTEM_APP.md`. |
+
+## ⚠️ Windows non-interactive-session gotcha (verified 2026-06-23)
+
+In a **Session-0 / RDP-disconnected / service** Windows session, programmatic
+cursor **movement and button/key presses are silent no-ops**, while reads,
+clipboard, and OCR work. Verified directly on the Windows backend:
+
+| Operation | Non-interactive session |
+|-----------|-------------------------|
+| `legacyGetCursorPosition()` (WinForms read) | ✅ live, accurate |
+| nutjs `mouse.setPosition` / `mouse.move` | ❌ no-op (cursor unchanged) |
+| PowerShell `SetCursorPos` | ❌ no-op |
+| nutjs `pressButton`/`releaseButton`, `pressKey`/`releaseKey` (SendInput) | ❌ no-op (no foreground desktop to receive input) |
+| clipboard read/write round-trip | ✅ works headless |
+| `Windows.Media.Ocr` (rendered PNG → text + word boxes) | ✅ works headless |
+
+**Consequence for the real lane:** any `*.real.test.ts` that asserts an *input
+effect* — e.g. `cua-parity-input.real.test.ts`'s `move → get_cursor_position`
+round-trip — will **fail** (not skip) on a non-interactive Windows runner. Run
+against this backend it produces:
+
+```
+× get_cursor_position reflects driverMouseMove
+  AssertionError: expected 388 to be less than or equal to 2   # cursor never moved
+✓ clipboard write/read round-trips
+```
+
+This is an **environment** failure, not a code defect. The Windows real-driver
+CI lane therefore MUST run on an **interactive, logged-in desktop** (autologon +
+unlocked session; the agent on a real desktop), not as a service / in Session 0.
+Read-only reals (cursor read, clipboard, OCR) are safe headless and can run on
+any Windows runner.
+
+## What is validated on Windows today
+
+Confirmed working on the Windows backend (2026-06-23):
+
+- **Cursor read** — `driverGetCursorPosition` returns the live position (WinForms).
+- **Clipboard** — read/write round-trips a newline/unicode/quote payload
+  (the `Set-Clipboard -Value [Console]::In.ReadToEnd()` fix; #9165).
+- **OCR** — native `Windows.Media.Ocr` extracts rendered text with per-word boxes
+  (`ocr-service-windows.test.ts`, 5/5; #9121 M4a, wired to `COMPUTER_USE ocr` /
+  `detect_elements` via #9173 M7).
+
+Input-effect verification (move/click/drag/press fidelity, DPI, multi-monitor)
+requires an interactive Windows desktop and is gated to that lane.
+
+## Release evidence
+
+Per-OS evidence manifests live next to this doc
+(`windows-desktop-validation.json`, `macos-desktop-validation.json`,
+`linux-desktop-validation.json`, `android-*-validation.json`,
+`ios-device-validation.json`) and are checked by
+`bun run --cwd plugins/plugin-computeruse validate:<os>-desktop-evidence`.
+Keep incomplete live-device checks under `requires_device_evidence`; only use
+`validate:platform-evidence -- --require-complete` for release gates that truly
+have artifacts for every required platform check.

plugins/plugin-computeruse/docs/TRYCUA_PARITY_AUDIT.md

@@ -0,0 +1,176 @@
+# Computer-Use × Vision — trycua/cua parity audit (current state)
+
+Authoritative, **current-state** review of where elizaOS `plugin-computeruse` +
+`plugin-vision` stand against [trycua/cua](https://github.com/trycua/cua) and the
+six other computer-use / OCR references, refreshed **after M3 / M3.5 / M4a / M5 /
+M6 / M7 / M8** landed on `develop` (the older `reports/trycua-*.txt` snapshot
+predates M7+M8 and is stale).
+
+- Tracking issues: **#9105** (EPIC — integration) and **#9170** (parity + tests
+  on Windows/Linux/macOS/AOSP).
+- Test-lane reference: [`TEST_LANES_COMPUTERUSE_VISION.md`](./TEST_LANES_COMPUTERUSE_VISION.md).
+- Raw research snapshots: `reports/cua-external-summary.txt`,
+  `reports/trycua-matrix.txt`, `reports/trycua-gaps.txt`.
+
+---
+
+## 1. External-repo review (what to adopt, what is N/A)
+
+| Repo | What it is | OCR | Local vision model | Continuous/low-token screen | Net takeaway for us |
+|------|-----------|-----|--------------------|------------------------------|---------------------|
+| **trycua/cua** | ~50-verb computer-server (WS/HTTP/MCP) over 6 VM/sandbox providers + agent SDK (loop registry, predict_step/predict_click, Set-of-Marks/OmniParser grounding, callback middleware) | model-side (hosted) | no | discrete capture→act loop | **The parity target.** Verb gaps are small/additive; the real work is the architecture: loop registry (M10), SoM overlay (M9), provider/RPC matrix (M13), eval harness (M14). |
+| **coasty-ai/open-computer-use** (Apache-2.0) | OSS+SaaS CUA; web + Electron + MCP | classical (pytesseract, Linux VM only) | no (hosted "Coasty CUA") | discrete loop, JPEG q70, HD price penalty | **Adopt:** cross-platform input-correctness (DPI scale, off-primary cursor) — *already taken in M3.5*; predict/ground separation — *M5*; stateful session to cut tokens; avoid runtime-compiled inline C# in the PS fallback (AMSI fingerprint). |
+| **injaneity/pi-computer-use** (MIT) | macOS-only; AX-tree grounding, **no vision/OCR** | none (AX tree + CDP) | none | pull-based AX snapshots | **Adopt:** accessibility-tree grounding tier to skip OCR for most clicks — *partially in scene a11y-provider*; stale-ref reacquisition; batched multi-action → one post-state; GET_SCREEN envelope with explicit scale/coordinateSpace/stateId. |
+| **domdomegg/computer-use-mcp** (MIT) | thin local MCP mirror of Anthropic's computer-use tool | none (model does it) | none | pull-based, downscaled | **Adopt:** cursor-crosshair drawn into the frame; hard-cap to 1568px/~1.15MP + re-encode before any cloud model; validated bidirectional image↔logical scale; `xdotool --clearmodifiers`. |
+| **vercel-labs/ai-sdk-computer-use** (no license — reference only) | Next.js demo, Claude drives a Vercel Sandbox | none | none (cloud Claude) | noVNC live for human; pull screenshots for model | **Adopt (patterns, not code):** `prunedMessages()` token-frugal history (redact prior frames, keep latest) — *aligns with M3 dHash + M11*; pull-based, skip re-screenshot after blind type/key; typed image tool-results; Anthropic `cacheControl` on the stable preamble; provider-defined-tool path when the model is Anthropic. |
+| **bracesproul/gen-ui-computer-use** (MIT) | LangGraph Generative-UI showcase over OpenAI CUA + Scrapybara | none | none (OpenAI CUA) | hosted VNC iframe + per-action audit shots | **Adopt:** before/after action hooks streaming typed UI cards; clean live-view vs audit-trail split; content-addressed screenshot store (we already have the better sha256 media-store) + collapsed-by-default frames; live pause/approve/abort controls. |
+| **baidu/Unlimited-OCR** (MIT) | one-shot long-horizon **document** OCR (DeepSeek-OCR + DeepSeek-V2 MoE, GGUF-able) | VLM end-to-end | yes (3B/0.5B-active) | n/a (batch doc parser) | **Adopt later:** a candidate **local** OCR/VLM for `plugin-vision` once exposed as GGUF — fits the "local, token-frugal" goal. **Not** a screen/CUA tool; document-parsing only. |
+
+**Cross-cutting design conclusions already encoded in our roadmap:** OCR + element
+detection belong in `plugin-vision` and feed `plugin-computeruse` through the
+`CoordOcrProvider` seam (M1); screen understanding must be **local + token-frugal**
+(GGUF only, no ONNX) with dHash-gated re-description (M3); a `GET_SCREEN` action
+returns a typed image + structured elements (M2).
+
+---
+
+## 2. Verb / capability parity matrix (vs trycua/cua, on `develop` today)
+
+Legend: ✅ have · 🟡 partial · ❌ missing · ⛔ N/A (deliberately not chased)
+
+### Pointer / keyboard / screen (host drivers)
+| trycua capability | elizaOS status | Where |
+|---|---|---|
+| screenshot (per-display, native res, quality/downscale) | ✅ | `platform/capture.ts`, `screenshot-quality.ts` |
+| click / right_click / double_click | ✅ | `COMPUTER_USE` |
+| **middle_click** | ✅ (M8) | `nut-driver.ts` `nutMiddleClick` |
+| click_with_modifiers | ✅ | |
+| mouse_move | ✅ | |
+| **mouse_down / mouse_up** (button-param press-hold) | ✅ (M8) | `driver.ts` `driverMouseDown/Up(x,y,button)` |
+| **key_down / key_up** (press-hold; bare modifiers) | ✅ (M8 + #9189) | `resolveKeyCode` now maps `MODIFIER_KEYS` |
+| type / key / key_combo | ✅ | |
+| scroll (per-notch + raw) | ✅ (M3.5 per-notch pacing = parity-plus) | |
+| drag (start→end) | ✅ | |
+| **drag (multi-point polyline path)** | ✅ (M8) | `driver.ts` `driverDragPath` → `nutDragPath`/`densifyDragPath` |
+| get_cursor_position | ✅ (M3.5) | win32 uses `System.Windows.Forms.Cursor` (nutjs getPosition is stale) |
+| get_screen_size / to_screen(shot)_coordinates | ✅ | multi-display + DPI transforms |
+| get_environment / desktop info | 🟡 | `get_environment` yes; `get_desktop_environment` (for driver selection) ❌ minor |
+
+### Vision / grounding
+| trycua capability | elizaOS status | Where |
+|---|---|---|
+| **ocr** (full-screen, coord-bearing) | ✅ (M7) | `COMPUTER_USE ocr` → `getCoordOcrProvider()` (plugin-vision OCR) |
+| **detect_elements** | ✅ (M7) | routes through scene element registry + CoordOcr |
+| **GET_SCREEN action** (typed image + elements, token-frugal) | ✅ (M2) | `plugin-vision/src/get-screen.ts`, `get-screen-elements.ts` |
+| low-token continuous description | ✅ (M3) | dHash-gated `DirtyTileDescriber` + frame-dHash skip + token counters |
+| predict / ground split (cheap ground vs full predict) | 🟡 (M5) | `OcrCoordinateGroundingActor` + per-Scene grounding cache; no model-string registry yet |
+| accessibility-tree grounding tier | ✅ | scene `a11y-provider` (Win UIA / macOS AX / Linux AT-SPI) |
+| OmniParser/icon detection (GGUF, no ONNX) | 🟡 | `yolo-detector.ts` (YOLOv8n COCO GGUF via `native/yolo.cpp`) |
+| **Set-of-Marks (numbered overlay) grounding** | ✅ (M9) | `plugin-vision/src/som.ts` + `set-of-marks-provider.ts`; registered via `registerSetOfMarksProvider` seam in `mobile/ocr-provider.ts` |
+
+### Windows / files / process / shell
+| trycua capability | elizaOS status | Where |
+|---|---|---|
+| window list/focus/switch/arrange/move/min/max/restore/close | ✅ | `WINDOW` action |
+| get_current_window_id / get_application_windows(app) | ❌ (M12) | `scene.focused_window` is adjacent but no discrete getter verb |
+| get_window_size / get_window_position / set_window_size | ❌ (M12) | `WINDOW_MOVE` sets position only |
+| open(target) / launch(app,args)→pid | ❌ (M12) | |
+| filesystem verbs (exists/list/read_text/write_text/read_bytes/write_bytes/delete/create_dir/get_file_size) | 🟡 (M12) | internal `file-ops.ts` + docker-backend wire ops; **not** exposed as `COMPUTER_USE` verbs |
+| run_command (CommandResult) | 🟡 (M12) | `terminal.ts` one-shot + docker-backend; not a host `COMPUTER_USE` verb by design |
+| PTY / interactive terminal | ❌ low-pri | one-shot only |
+| clipboard read/write | ✅ (M3.5) | `CLIPBOARD` action |
+
+### Architecture / platform
+| trycua capability | elizaOS status | Notes |
+|---|---|---|
+| agent-loop registry (model-string → loop) | 🟡 (M10) | hardcoded Brain→Cascade→Actor→dispatch |
+| callback middleware (budget/image-retention/trajectory/operator-normalize) | 🟡 (M11) | approval-manager + token counters + log-line trajectory events |
+| VM/sandbox provider matrix | 🟡 (M13) | Docker backend only; WSB + QEMU are the opt-in targets worth adding |
+| daemon/RPC seam (`{command,params}→{success,result}`) | ❌ (M13) | in-process service; routes only for approvals |
+| MCP server seam | ❌ | one optional MCP seam is the relevant subset of cua's 2 servers / 35 tools |
+| eval harness (ScreenSpot / OSWorld / GET-commands matrix) | 🟡 (M14) | `osworld/` adapter + `benchmarks/eliza-1/vision-cua-e2e`; not yet per-OS scenario-wired |
+| AOSP touch + hardware keys + multitouch | 🟡 | Capacitor bridge tap/swipe + globalAction(back/home/recents); `multitouch_gesture` ❌ |
+
+### Explicitly N/A (do not chase)
+`set_wallpaper` (niche) · `playwright_exec` / `browser_execute` (**unconditionally
+disabled**, GHSA-rcvr-766c-4phv) · cloud managed sandbox · the full 2-MCP-server /
+35-driver-tool surface · PII anonymization · AOSP gRPC `:8554` backend (CI-only).
+
+---
+
+## 3. Test-coverage matrix (every-verb, by lane and platform)
+
+Lanes: **unit** = default Vitest (runs on all OSes), **real** = `*.real.test.ts`
+gated `platform()==="win32"` and excluded from the default run (Windows only),
+**e2e**, **scenario** (scenario-runner / issue-evidence), **probe** = standalone
+`bun` driver check.
+
+| Capability group | unit (all OS) | Windows real/probe | Linux | macOS | AOSP |
+|---|---|---|---|---|---|
+| pointer click family + modifiers | `cua-parity-surface`, `computer-interface`, `dispatch`, `use-computer-action` | `cua-parity-input.real`, `computeruse.real`, `service.real`, probes | ⚠️ no real lane | ⚠️ no real lane | n/a |
+| M8 middle/down-up + key down-up + drag-path | `cua-parity-surface`, `nut-driver-input` (incl. modifier guard) | `cua-parity-input.real` + this-session probe (**verified**) | ⚠️ no real lane | ⚠️ no real lane | n/a |
+| ocr / detect_elements (M7) | `ocr-adapter-coord-seam`, `ocr-with-coords`, `computeruse-ocr-bridge` | `service.real`, OCR bridge | unit only | `ocr-service-apple-vision` (unit) | n/a |
+| GET_SCREEN / scene / dHash (M2/M3) | `get-screen`, `get-screen-elements`, `dirty-tile-describer`, `screen-state`, `dhash`, `scene-*` | scene multimon/probe | unit only | unit only | `android-scene` |
+| windows / clipboard / cursor | `platform-capabilities`, `windows-powershell-safety` (static) | `windows-list.real`, `clipboard` (win32), cursor probe | unit only | unit only | n/a |
+| files / terminal | `security-file-target` | `file-ops.real`, `terminal.real` | unit only | unit only | n/a |
+| OCR engines | `ocr-service`, `ocr-service-windows`, `yolo-detector` | `ocr-service-windows` (win32) | `ocr-service` (docTR) | `ocr-service-apple-vision` | n/a |
+| autonomous loop | `cascade`, `brain`, `dispatch`, `computer-use-agent` | `computer-use-agent.real`, `runtime.live.e2e` | unit only | unit only | `mobile-cascade`, `aosp-input-actor` |
+| benchmarks | `osworld-action-converter` | `benchmark/osworld-*.real` | — | — | — |
+| approvals / progress | `computer-use-approval-relay` | `routes-e2e` | scenario `8912-computeruse-progress-approvals-*` | — | — |
+| mobile/AOSP | — | — | — | — | `android-bridge`, `android-trajectory`, `ios-*`, device-evidence manifests |
+
+### The single biggest "tests on all platforms" hole
+The real-driver lane is **`win32`-only** (`const RUN = platform()==="win32"`). The
+nutjs + legacy `xdotool`/`cliclick` code paths exist for Linux/macOS but **no
+equivalent real-driver lane executes them** — and because the gate skips silently,
+a Linux/macOS regression would not turn a lane red. This is exactly how the M8
+`key_down("shift")` bug (fixed in #9189) reached `develop`: M8's real test was
+authored on a host without a Windows machine, so the gated case never ran.
+
+**Recommended (M14):** a Linux Xvfb headful lane and a macOS headful lane that run
+the parity verbs against the real driver, plus a machine-checkable matrix derived
+from the action enum so a newly-added trycua verb that lacks a per-OS test fails CI.
+
+---
+
+## 4. Per-platform verification status
+
+| OS | Driver | Status |
+|----|--------|--------|
+| **Windows 11** | nutjs (default) + legacy PowerShell | **Primary, verified.** All M8 verbs + key_down modifier fix dispatched against the live desktop this cycle (cursor read back via WinForms). OCR via `Windows.Media.Ocr` (M4a). |
+| **Linux** | nutjs + legacy `xdotool` (X11; AT-SPI/Wayland fallback) | Code paths present + unit-tested; **no real-driver lane runs** → needs Xvfb headful CI. Clipboard needs `wl-clipboard`/`xclip` (asserts clear error when absent). |
+| **macOS** | nutjs + legacy `cliclick`; retina backing-store 2× transforms | Code paths present + unit-tested (incl. Apple Vision OCR); **no real-driver lane runs** → needs headful macOS CI; assert retina coordinate transforms. |
+| **AOSP / Android** | Capacitor bridge + `MobileComputerInterface` | tap/swipe + globalAction(back/home/recents) implemented + unit-tested; device-evidence manifests exist; `multitouch_gesture` missing; system-app path is CI-only. |
+
+---
+
+## 5. Remaining roadmap (prioritized)
+
+1. **M14 cross-platform real lanes** *(highest leverage for the stated goal)* —
+   The enum-derived parity coverage guard (`cua-parity-coverage.test.ts`) is
+   **landed** and runs on all platforms in the default lane, and the real lane is
+   no longer win32-hardcoded. The remaining piece is the **Linux Xvfb + macOS
+   headful CI lanes** that actually invoke the real config (requires a
+   `.github/workflows/**` change / `workflow` token scope). Until those exist,
+   real-driver *actuation* is verified on Windows only.
+2. **M12 additive verbs** — window getters (`get_current_window_id`,
+   `get_application_windows`, `get/set_window_size`, `get_window_position`),
+   `open`/`launch`, and the filesystem/`run_command` verbs (wrap the existing
+   internal `file-ops.ts`/`terminal.ts` as gated `COMPUTER_USE` verbs).
+3. **M10 agent-loop registry** + predict_step/predict_click split (extend the Actor
+   registry; keep approval gating).
+4. **M11 callback middleware** — budget cap, image-retention (only-N-recent, aligns
+   with M3), operator-normalizer, trajectory.
+5. **M13 provider matrix + RPC seam** — Windows Sandbox (WSB) + QEMU behind a
+   remote-guest `{command,params}→{success,result}` RPC over the route/WS seam.
+6. **AOSP `multitouch_gesture`** (MT-Protocol-B) + an optional MCP server seam.
+
+**Done since this audit was first written:** M9 Set-of-Marks (numbered-overlay
+grounding) landed — `plugin-vision/src/som.ts` + `set-of-marks-provider.ts`,
+registered via the `registerSetOfMarksProvider` seam; 23/23 unit tests green on
+Windows.
+
+> **Process note for contributors:** the `*.real.test.ts` lanes only execute on a
+> Windows host. Before trusting any newly-landed computeruse milestone, run its
+> verbs against the real driver on Windows (or the future Linux/macOS lanes) — a
+> green default-lane run does **not** mean the platform paths work. See §3.

plugins/plugin-computeruse/docs/android-aosp-validation.json

@@ -0,0 +1,107 @@
+{
+  "schemaVersion": 1,
+  "platform": "android-aosp",
+  "status": "requires_device_evidence",
+  "target": {
+    "minimumApi": "system image capable of ROLE_ASSISTANT and privileged permission grants",
+    "imageType": "AOSP/system validation image",
+    "requiredBuildFlavor": "android-system or sideload-only system APK",
+    "privilegedPermissions": [
+      "READ_FRAME_BUFFER",
+      "INJECT_EVENTS",
+      "REAL_GET_TASKS"
+    ]
+  },
+  "evidence": {
+    "imageName": null,
+    "androidVersion": null,
+    "apiLevel": null,
+    "buildId": null,
+    "validatedAt": null,
+    "validator": null,
+    "artifacts": []
+  },
+  "checks": [
+    {
+      "id": "assistantRole",
+      "method": "ROLE_ASSISTANT resolution",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "Eliza is installed as or granted the assistant role",
+        "role manager resolves Eliza for assistant requests",
+        "consumer/Play build remains free of ROLE_ASSISTANT requirements"
+      ]
+    },
+    {
+      "id": "assistVoiceCommand",
+      "method": "ACTION_ASSIST/VOICE_COMMAND",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "android.intent.action.ASSIST reaches Eliza runtime",
+        "android.intent.action.VOICE_COMMAND reaches Eliza runtime",
+        "payload source metadata is preserved"
+      ]
+    },
+    {
+      "id": "privilegedCapture",
+      "method": "SurfaceControl.captureDisplay",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "READ_FRAME_BUFFER is granted only to the system build",
+        "capture returns a non-empty frame",
+        "frame visually matches the current display"
+      ]
+    },
+    {
+      "id": "privilegedInput",
+      "method": "InputManager.injectInputEvent",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "INJECT_EVENTS is granted only to the system build",
+        "tap injection returns success",
+        "swipe injection returns success",
+        "unsupported text/key actions are rejected with explicit errors"
+      ]
+    },
+    {
+      "id": "processEnumeration",
+      "method": "IActivityManager.getRunningAppProcesses",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "REAL_GET_TASKS is granted only to the system build",
+        "running processes are enumerated",
+        "foreground process is identifiable"
+      ]
+    },
+    {
+      "id": "serviceFlavorSeparation",
+      "method": "build flavor inspection",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "ElizaAccessibilityService is present only where expected",
+        "ElizaNotificationListenerService is present only where expected",
+        "system-only Java/Kotlin sources are excluded from android-cloud"
+      ]
+    },
+    {
+      "id": "consumerBuildStripping",
+      "method": "consumer artifact inspection",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "android-cloud strips privileged services",
+        "android-cloud strips accessibility-service XML resource",
+        "android-cloud has no privileged permission declarations"
+      ]
+    },
+    {
+      "id": "lifeOpsPersistence",
+      "method": "assistant-to-runtime handoff",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "AOSP assistant requests enter the app/runtime planner",
+        "reminders/check-ins/follow-ups persist as ScheduledTask records",
+        "privileged capture/input does not create native-only task state"
+      ]
+    }
+  ]
+}

plugins/plugin-computeruse/docs/android-device-validation.json

@@ -0,0 +1,126 @@
+{
+  "schemaVersion": 1,
+  "platform": "android-consumer",
+  "status": "requires_device_evidence",
+  "target": {
+    "minimumApi": "24 for Accessibility gesture dispatch; 29 for MediaProjection foreground service",
+    "minimumAndroid": "Android 10 for full capture validation; Android 7+ for basic bridge validation",
+    "requiredBuildFlavor": "android-cloud or Play-compatible consumer APK",
+    "distribution": "Google Android / Pixel / Play-compatible build; no privileged system permissions"
+  },
+  "evidence": {
+    "deviceModel": null,
+    "androidVersion": null,
+    "apiLevel": null,
+    "buildId": null,
+    "validatedAt": null,
+    "validator": null,
+    "artifacts": []
+  },
+  "checks": [
+    {
+      "id": "permissionsSetup",
+      "method": "manual-device-setup",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "CAMERA runtime permission is granted",
+        "Eliza AccessibilityService is enabled and visible in dumpsys accessibility",
+        "Usage Access is enabled for Eliza",
+        "consumer build does not request privileged assistant/system permissions"
+      ]
+    },
+    {
+      "id": "accessibilityTree",
+      "method": "getAccessibilityTree",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "foreground app has a populated accessibility tree",
+        "nodes include id, role, label, bbox, and actions",
+        "role values are Android class names"
+      ]
+    },
+    {
+      "id": "gestureDispatch",
+      "method": "dispatchGesture",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "tap returns ok:true and is visibly applied",
+        "swipe returns ok:true and scrolls the target content",
+        "coordinates are screen-pixel coordinates"
+      ]
+    },
+    {
+      "id": "globalActions",
+      "method": "performGlobalAction",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "back action is visually confirmed",
+        "home action is visually confirmed",
+        "recents action is visually confirmed",
+        "notifications action is visually confirmed"
+      ]
+    },
+    {
+      "id": "mediaProjectionCapture",
+      "method": "startMediaProjection/captureFrame/stopMediaProjection",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "system consent dialog appears and is accepted",
+        "captureFrame returns ok:true with non-empty jpegBase64",
+        "decoded JPEG matches the current screen",
+        "stopMediaProjection returns stopped:true"
+      ]
+    },
+    {
+      "id": "usageStatsEnumeration",
+      "method": "enumerateApps",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "Usage Access grant is honored",
+        "enumerateApps returns packageName, label, lastUsedMs, totalForegroundMs, and isForeground",
+        "frontmost app is marked isForeground:true"
+      ]
+    },
+    {
+      "id": "cameraCapture",
+      "method": "startCamera/captureFrameCamera/stopCamera",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "startCamera returns at least one camera",
+        "captureFrameCamera returns non-empty jpegBase64",
+        "stopCamera succeeds"
+      ]
+    },
+    {
+      "id": "memoryPressureDispatch",
+      "method": "onTrimMemory/dispatchMemoryPressure",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "TRIM_MEMORY_RUNNING_LOW or CRITICAL is triggered",
+        "bridge dispatches a pressure sample",
+        "MemoryArbiter pressure state transitions to low or critical"
+      ]
+    },
+    {
+      "id": "appActionsShortcuts",
+      "method": "Assistant/App Actions/static shortcuts",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "launcher activity registers @xml/shortcuts",
+        "OPEN_APP_FEATURE, CREATE_MESSAGE, and GET_THING are present",
+        "Assistant preview triggers chat, voice, daily brief, new task, and tasks routes",
+        "deep links are source-tagged and brand-rewritten"
+      ]
+    },
+    {
+      "id": "lifeOpsScheduledTaskHandoff",
+      "method": "assistant-to-runtime handoff",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "reminder/check-in/follow-up request enters the app/runtime planner",
+        "LifeOps creates or updates a ScheduledTask record",
+        "no native-only reminder table or notification-only schedule is created"
+      ]
+    }
+  ]
+}

plugins/plugin-computeruse/docs/ios-device-validation.json

@@ -0,0 +1,113 @@
+{
+  "schemaVersion": 1,
+  "platform": "ios",
+  "status": "requires_device_evidence",
+  "target": {
+    "minimumIos": "17.6",
+    "primaryIos": "26.1",
+    "minimumDeviceClass": "A14-or-later iPhone"
+  },
+  "evidence": {
+    "deviceModel": null,
+    "iosVersion": null,
+    "buildId": null,
+    "validatedAt": null,
+    "validator": null,
+    "artifacts": []
+  },
+  "checks": [
+    {
+      "id": "probe",
+      "method": "probe",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "platform is ios",
+        "osVersion matches the device",
+        "all six capability bits are present"
+      ]
+    },
+    {
+      "id": "replayKitForegroundStart",
+      "method": "replayKitForegroundStart",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "first-use permission prompt is shown",
+        "accepted capture returns non-empty jpegBase64 frames",
+        "frame resolution matches the device screen"
+      ]
+    },
+    {
+      "id": "broadcastExtensionHandshake",
+      "method": "broadcastExtensionHandshake",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "extension target is bundled",
+        "share-sheet broadcast picker starts a session",
+        "broadcastActive transitions to true",
+        "iOS 26 beta extension-death regression status is recorded when applicable"
+      ]
+    },
+    {
+      "id": "visionOcr",
+      "method": "visionOcr",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "known PNG with WS9 OCR SMOKE is rendered",
+        "accurate OCR returns fullText containing the source string"
+      ]
+    },
+    {
+      "id": "appIntentList",
+      "method": "appIntentList",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "fresh install returns an empty donated-intent list",
+        "list grows after donation through Shortcuts"
+      ]
+    },
+    {
+      "id": "appIntentInvokeSafari",
+      "method": "appIntentInvoke",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "com.apple.mobilesafari.open-url opens Safari to the provided URL"
+      ]
+    },
+    {
+      "id": "appIntentInvokeMessages",
+      "method": "appIntentInvoke",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "com.apple.MobileSMS.send-message opens Messages with recipient and body pre-filled"
+      ]
+    },
+    {
+      "id": "accessibilitySnapshot",
+      "method": "accessibilitySnapshot",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "top-level node role is not labeled",
+        "top-level node has children on a populated screen"
+      ]
+    },
+    {
+      "id": "foundationModelGenerate",
+      "method": "foundationModelGenerate",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "Apple Intelligence enabled returns non-empty generated text",
+        "Apple Intelligence disabled returns foundation_model_unavailable"
+      ]
+    },
+    {
+      "id": "memoryPressureProbe",
+      "method": "memoryPressureProbe",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "debug memory warning is invoked",
+        "next probe returns severity >= 0.7",
+        "lastWarningAt is set"
+      ]
+    }
+  ]
+}

plugins/plugin-computeruse/docs/linux-desktop-validation.json

@@ -0,0 +1,114 @@
+{
+  "schemaVersion": 1,
+  "platform": "linux-desktop",
+  "status": "requires_device_evidence",
+  "target": {
+    "minimumDistribution": "Ubuntu 22.04 or equivalent modern desktop Linux",
+    "displayServer": "X11 required for xdotool/wmctrl path; Wayland portal capture remains a separate manual check",
+    "driver": "nutjs preferred; legacy xdotool/scrot/wmctrl fallback allowed"
+  },
+  "evidence": {
+    "machineId": null,
+    "distribution": null,
+    "kernelVersion": null,
+    "displayServer": null,
+    "buildId": null,
+    "validatedAt": null,
+    "validator": null,
+    "artifacts": []
+  },
+  "checks": [
+    {
+      "id": "capabilityProbe",
+      "method": "detectPlatformCapabilities/getCapabilities",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "platform is linux",
+        "screenshot, computerUse, windowList, browser, terminal, fileSystem, and clipboard capabilities are reported",
+        "missing optional tools produce explicit unavailable/tool messages"
+      ]
+    },
+    {
+      "id": "dependencyProbe",
+      "method": "ensure-platform-deps / commandExists",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "xdotool or nutjs driver availability is recorded",
+        "screenshot backend availability is recorded",
+        "wmctrl or xdotool window-list backend availability is recorded",
+        "wl-clipboard, xclip, or xsel clipboard backend availability is recorded"
+      ]
+    },
+    {
+      "id": "screenshotCapture",
+      "method": "captureDisplay/capturePrimaryDisplay",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "primary display capture returns non-empty PNG bytes",
+        "captured dimensions match display metadata",
+        "screenshot artifact is attached"
+      ]
+    },
+    {
+      "id": "mouseKeyboardInput",
+      "method": "ComputerUseService desktop actions",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "mouse_move succeeds",
+        "click or double_click succeeds on a harmless target",
+        "type and key actions succeed in a controlled text field",
+        "post-action screenshot is captured when configured"
+      ]
+    },
+    {
+      "id": "windowListFocus",
+      "method": "listWindows/focusWindow",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "listWindows returns visible windows with app/process metadata",
+        "focusWindow focuses a safe target window",
+        "window operation failures include dependency guidance"
+      ]
+    },
+    {
+      "id": "browserAutomation",
+      "method": "browser open/navigate/get/screenshot/close",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "browser target opens or attaches to a supported Chromium browser",
+        "navigate/get/screenshot succeed",
+        "browser cleanup closes the test tab"
+      ]
+    },
+    {
+      "id": "clipboardRoundTrip",
+      "method": "readClipboard/writeClipboard",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "configured Linux clipboard backend writes a test token",
+        "configured Linux clipboard backend reads the same test token",
+        "missing clipboard backend returns ClipboardUnavailableError"
+      ]
+    },
+    {
+      "id": "terminalSafety",
+      "method": "terminal execution safety",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "allowed harmless shell command succeeds",
+        "dangerous or blocked command is rejected",
+        "timeout behavior is recorded"
+      ]
+    },
+    {
+      "id": "approvalMode",
+      "method": "ComputerUseApprovalManager",
+      "status": "requires_device_evidence",
+      "requiredEvidence": [
+        "smart_approve auto-approves read-only actions",
+        "destructive actions require approval",
+        "full_control/approve_all/off modes behave as configured"
+      ]
+    }
+  ]
+}

plugins/plugin-computeruse/docs/macos-desktop-validation.json

@@ -0,0 +1,123 @@
+{
+  "schemaVersion": 1,
+  "platform": "macos-desktop",
+  "status": "passed",
+  "target": {
+    "minimumMacos": "macOS 13 or newer",
+    "requiredPermissions": ["Screen Recording", "Accessibility"],
+    "driver": "nutjs preferred; legacy cliclick/AppleScript fallback allowed"
+  },
+  "evidence": {
+    "machineModel": "Mac16,5",
+    "macosVersion": "26.2",
+    "buildId": "25C56",
+    "validatedAt": "2026-06-26T22:47:17.507Z",
+    "validator": "bun scripts/capture-macos-desktop-evidence.mjs (08255e4731)",
+    "artifacts": [
+      ".github/issue-evidence/9581-macos-desktop-cua/screenshot-primary.png",
+      ".github/issue-evidence/9581-macos-desktop-cua/browser-evidence.png",
+      ".github/issue-evidence/9581-macos-desktop-cua/approval-full-control.txt",
+      ".github/issue-evidence/9581-macos-desktop-cua/macos-desktop-validation.json",
+      ".github/issue-evidence/9581-macos-desktop-cua/report.json",
+      ".github/issue-evidence/9581-macos-desktop-cua/README.md",
+      ".github/issue-evidence/9581-macos-desktop-cua/manifest.json"
+    ]
+  },
+  "checks": [
+    {
+      "id": "capabilityProbe",
+      "method": "detectPlatformCapabilities/getCapabilities",
+      "status": "passed",
+      "requiredEvidence": [
+        "platform is darwin",
+        "reported capabilities: screenshot=available:screencapture (built-in); computerUse=available:cliclick; windowList=available:AppleScript System Events; browser=available:puppeteer-core (Chromium detected); terminal=available:/bin/zsh; fileSystem=available:node:fs; clipboard=available:pbpaste / pbcopy (built-in)",
+        "service capabilities match darwin probe for screenshot=true, windowList=true"
+      ]
+    },
+    {
+      "id": "screenRecordingPermission",
+      "method": "screencapture permission gate",
+      "status": "passed",
+      "requiredEvidence": [
+        "granted Screen Recording permission allowed screenshot capture",
+        "permission-denied screenshot errors are classified by the service when TCC blocks capture",
+        "captured nonblank artifact .github/issue-evidence/9581-macos-desktop-cua/screenshot-primary.png"
+      ]
+    },
+    {
+      "id": "screenshotCapture",
+      "method": "captureDisplay/capturePrimaryDisplay",
+      "status": "passed",
+      "requiredEvidence": [
+        "primary display capture returned 2502814 PNG bytes",
+        "captured dimensions 3456x2234 match display 0",
+        "screenshot artifact .github/issue-evidence/9581-macos-desktop-cua/screenshot-primary.png"
+      ]
+    },
+    {
+      "id": "accessibilityPermission",
+      "method": "Accessibility/TCC gate",
+      "status": "passed",
+      "requiredEvidence": [
+        "granted Accessibility permission allowed TextEdit focus, window bounds, click, key_combo, and type operations",
+        "missing Accessibility permission is classified by desktop service errors when TCC blocks input or System Events access"
+      ]
+    },
+    {
+      "id": "mouseKeyboardInput",
+      "method": "ComputerUseService desktop actions",
+      "status": "passed",
+      "requiredEvidence": [
+        "mouse_move succeeded at 1348,581 on display 0",
+        "click succeeded on a controlled TextEdit document",
+        "key_combo cmd+a succeeded in the controlled text field",
+        "type wrote and verified marker macos-cua-1782514041177",
+        "post-action screenshots were requested by service configuration"
+      ]
+    },
+    {
+      "id": "windowListFocus",
+      "method": "listWindows/focusWindow",
+      "status": "passed",
+      "requiredEvidence": [
+        "listWindows returned 15 visible window(s)",
+        "focusWindow/switchWindow succeeded for Simulator:iPhone 16 Pro",
+        "window operation errors retain permission guidance through service failure results"
+      ]
+    },
+    {
+      "id": "browserAutomation",
+      "method": "browser open/navigate/get/screenshot/close",
+      "status": "passed",
+      "requiredEvidence": [
+        "browser target opened data:text/html,<html><head><title>macOS CUA Evidence</title></head><body><main><h1>macOS CUA Evidence</h1><button id='go'>Ready</button><input id='field' value=''></main></body></html>",
+        "browser_get_dom returned the local evidence page",
+        "browser_get_clickables returned 2 element(s)",
+        "browser screenshot artifact .github/issue-evidence/9581-macos-desktop-cua/browser-evidence.png (1280x900)",
+        "browser cleanup closed the test browser"
+      ]
+    },
+    {
+      "id": "clipboardRoundTrip",
+      "method": "readClipboard/writeClipboard",
+      "status": "passed",
+      "requiredEvidence": [
+        "pbcopy wrote test marker macos-clipboard-1782514118884",
+        "pbpaste read the same test marker",
+        "large payload cap and command failures remain covered by unit tests"
+      ]
+    },
+    {
+      "id": "approvalMode",
+      "method": "ComputerUseApprovalManager",
+      "status": "passed",
+      "requiredEvidence": [
+        "smart_approve auto-approves read-only actions",
+        "smart_approve queues destructive file_write until approval",
+        "full_control auto-approves destructive actions",
+        "approve_all does not auto-approve read-only actions",
+        "off mode denies actions"
+      ]
+    }
+  ]
+}

plugins/plugin-computeruse/docs/windows-desktop-validation.json

@@ -0,0 +1,127 @@
+{
+  "schemaVersion": 1,
+  "platform": "windows-desktop",
+  "status": "passed",
+  "target": {
+    "minimumWindows": "Windows 10 or newer",
+    "driver": "nutjs preferred; legacy PowerShell/user32 fallback allowed",
+    "shell": "PowerShell -NoProfile"
+  },
+  "evidence": {
+    "machineModel": "QEMU Standard PC (Q35 + ICH9, 2009)",
+    "windowsVersion": "Microsoft Windows 11 Pro 10.0.26200",
+    "buildId": "26200",
+    "validatedAt": "2026-06-29T16:56:32.389Z",
+    "validator": "bun scripts/capture-windows-desktop-evidence.mjs (03e0b1fc94)",
+    "artifacts": [
+      ".github/issue-evidence/9581-windows-cua/screenshot-primary.png",
+      ".github/issue-evidence/9581-windows-cua/browser-evidence.png",
+      ".github/issue-evidence/9581-windows-cua/approval-full-control.txt",
+      ".github/issue-evidence/9581-windows-cua/cua-windows-desktop-capture.jpg",
+      ".github/issue-evidence/9581-windows-cua/windows-desktop-validation.json",
+      ".github/issue-evidence/9581-windows-cua/report.json",
+      ".github/issue-evidence/9581-windows-cua/CAPTURE_README.md",
+      ".github/issue-evidence/9581-windows-cua/manifest.json"
+    ]
+  },
+  "checks": [
+    {
+      "id": "capabilityProbe",
+      "method": "detectPlatformCapabilities/getCapabilities",
+      "status": "passed",
+      "requiredEvidence": [
+        "platform is win32",
+        "reported capabilities: screenshot=available:PowerShell System.Drawing; computerUse=available:PowerShell user32.dll; windowList=available:PowerShell Get-Process; browser=available:puppeteer-core (Chromium detected); terminal=available:powershell.exe; fileSystem=available:node:fs; clipboard=available:PowerShell Get-Clipboard",
+        "PowerShell-backed tools identified (screenshot=PowerShell System.Drawing, windowList=PowerShell Get-Process, clipboard=PowerShell Get-Clipboard)"
+      ]
+    },
+    {
+      "id": "screenshotCapture",
+      "method": "captureDisplay/capturePrimaryDisplay",
+      "status": "passed",
+      "requiredEvidence": [
+        "primary display capture returned 845567 PNG bytes",
+        "captured dimensions 1728x1052 match display 0",
+        "screenshot artifact .github/issue-evidence/9581-windows-cua/screenshot-primary.png"
+      ]
+    },
+    {
+      "id": "mouseKeyboardInput",
+      "method": "ComputerUseService desktop actions",
+      "status": "passed",
+      "requiredEvidence": [
+        "launched controlled Notepad (pid 14796); resolved window [46656] \"*RESEARCH-LIST.txt - Notepad\"",
+        "mouse_move succeeded at 758,648 on display 0",
+        "click succeeded inside the controlled Notepad text area",
+        "key_combo ctrl+a succeeded in the controlled text field",
+        "type wrote and verified marker eliza-win-cua-1782752194983 (read back via select-all/copy)",
+        "post-action screenshots were requested by service configuration"
+      ]
+    },
+    {
+      "id": "windowListFocus",
+      "method": "listWindows/focusWindow",
+      "status": "passed",
+      "requiredEvidence": [
+        "listWindows returned 3 visible window(s) with id + title metadata (finding #2 resolved — not 0)",
+        "3 window(s) had a usable id + title; focusWindow/switchWindow succeeded for [52064] \"Claude\"",
+        "window operation failures surface actionable diagnostics through the service result"
+      ]
+    },
+    {
+      "id": "browserAutomation",
+      "method": "browser open/navigate/get/screenshot/close",
+      "status": "passed",
+      "requiredEvidence": [
+        "browser target opened the evidence data URL",
+        "browser_get_dom returned the local evidence page",
+        "browser_get_clickables returned 2 element(s)",
+        "browser screenshot artifact .github/issue-evidence/9581-windows-cua/browser-evidence.png (1280x900)",
+        "browser cleanup closed the test browser"
+      ]
+    },
+    {
+      "id": "clipboardRoundTrip",
+      "method": "readClipboard/writeClipboard",
+      "status": "passed",
+      "requiredEvidence": [
+        "Set-Clipboard wrote test marker windows-clipboard-1782752210096",
+        "Get-Clipboard -Raw read the same test marker (trailing newline normalized)",
+        "large payload cap and command failures remain covered by unit tests"
+      ]
+    },
+    {
+      "id": "terminalSafety",
+      "method": "PowerShell terminal execution safety",
+      "status": "passed",
+      "requiredEvidence": [
+        "allowed harmless PowerShell command 'echo eliza-term-1782752210157' succeeded and echoed the token",
+        "dangerous 'Remove-Item -Recurse 'C:\\'' was rejected: Command blocked: PowerShell recursive deletion of drive root.",
+        "a 1s timeout killed a 5s Start-Sleep in 1045ms (timeout enforced)"
+      ]
+    },
+    {
+      "id": "approvalMode",
+      "method": "ComputerUseApprovalManager",
+      "status": "passed",
+      "requiredEvidence": [
+        "smart_approve auto-approves read-only actions",
+        "smart_approve queues destructive file_write until approval",
+        "full_control auto-approves destructive actions",
+        "approve_all does not auto-approve read-only actions",
+        "off mode denies actions"
+      ]
+    },
+    {
+      "id": "windowsHardeningRegression",
+      "method": "Windows hardening evidence replay",
+      "status": "passed",
+      "requiredEvidence": [
+        "primary-screen-size PowerShell command runs Add-Type System.Windows.Forms before using [System.Windows.Forms.Screen]",
+        "degenerate 0x0 capture region rejected with: Invalid capture region: width and height must be positive (got 0x0).",
+        "UNC network path blocked and a safe local temp path allowed by validateFilePath",
+        "linked real-driver screenshot evidence: .github/issue-evidence/9581-windows-cua/cua-windows-desktop-capture.jpg, .github/issue-evidence/9581-windows-cua/screenshot-primary.png"
+      ]
+    }
+  ]
+}

plugins/plugin-computeruse/package.json

@@ -0,0 +1,144 @@
+{
+  "name": "@elizaos/plugin-computeruse",
+  "version": "2.0.3-beta.7",
+  "description": "Desktop automation plugin — screenshots, mouse/keyboard control, browser CDP, and window management for elizaOS agents. Ported from open-computer-use (Apache 2.0).",
+  "type": "module",
+  "main": "dist/index.js",
+  "module": "dist/index.js",
+  "types": "dist/index.d.ts",
+  "exports": {
+    "./package.json": "./package.json",
+    ".": {
+      "types": "./dist/index.d.ts",
+      "eliza-source": {
+        "types": "./src/index.ts",
+        "import": "./src/index.ts",
+        "default": "./src/index.ts"
+      },
+      "import": "./dist/index.js",
+      "default": "./dist/index.js"
+    },
+    "./*.css": "./dist/*.css",
+    "./*": {
+      "types": "./dist/*.d.ts",
+      "eliza-source": {
+        "types": "./src/*.ts",
+        "import": "./src/*.ts",
+        "default": "./src/*.ts"
+      },
+      "import": "./dist/*.js",
+      "default": "./dist/*.js"
+    }
+  },
+  "files": [
+    "registry-entry.json",
+    "dist",
+    "README.md",
+    "package.json"
+  ],
+  "scripts": {
+    "postinstall": "node scripts/ensure-platform-deps.mjs",
+    "build": "bun run build.ts",
+    "test": "vitest run --config vitest.config.ts",
+    "typecheck": "tsgo --noEmit -p tsconfig.json",
+    "lint": "bunx @biomejs/biome check --write --unsafe .",
+    "lint:check": "bunx @biomejs/biome check .",
+    "format": "bunx @biomejs/biome format --write .",
+    "format:check": "bunx @biomejs/biome format .",
+    "validate:ios-device-evidence": "node scripts/validate-ios-device-evidence.mjs",
+    "validate:platform-evidence": "node scripts/validate-platform-evidence.mjs",
+    "validate:android-device-evidence": "node scripts/validate-platform-evidence.mjs docs/android-device-validation.json",
+    "validate:android-aosp-evidence": "node scripts/validate-platform-evidence.mjs docs/android-aosp-validation.json",
+    "capture:macos-desktop-evidence": "bun scripts/capture-macos-desktop-evidence.mjs",
+    "capture:windows-desktop-evidence": "bun scripts/capture-windows-desktop-evidence.mjs",
+    "record:windows-cua-input": "bun scripts/record-windows-cua-input.mjs",
+    "validate:macos-desktop-evidence": "node scripts/validate-platform-evidence.mjs docs/macos-desktop-validation.json",
+    "validate:linux-desktop-evidence": "node scripts/validate-platform-evidence.mjs docs/linux-desktop-validation.json",
+    "validate:windows-desktop-evidence": "node scripts/validate-platform-evidence.mjs docs/windows-desktop-validation.json"
+  },
+  "peerDependencies": {
+    "@elizaos/core": "workspace:*"
+  },
+  "devDependencies": {
+    "@elizaos/core": "workspace:*",
+    "@types/node": "^25.2.3",
+    "bun-types": "^1.3.14",
+    "typescript": "^6.0.3",
+    "vitest": "4.1.10"
+  },
+  "dependencies": {
+    "@nut-tree-fork/nut-js": "^4.2.6",
+    "puppeteer-core": "^24.9.0"
+  },
+  "optionalDependencies": {
+    "@modelcontextprotocol/sdk": "^1.29.0"
+  },
+  "license": "Apache-2.0",
+  "agentConfig": {
+    "pluginType": "elizaos:plugin:1.0.0",
+    "pluginParameters": {
+      "COMPUTER_USE_ENABLED": {
+        "type": "boolean",
+        "description": "Enable computer use capabilities (desktop automation, screenshots, browser control)",
+        "required": false,
+        "default": false,
+        "sensitive": false
+      },
+      "COMPUTER_USE_SCREENSHOT_AFTER_ACTION": {
+        "type": "boolean",
+        "description": "Automatically capture a screenshot after each desktop action",
+        "required": false,
+        "default": true,
+        "sensitive": false
+      },
+      "COMPUTER_USE_ACTION_TIMEOUT_MS": {
+        "type": "number",
+        "description": "Maximum action execution timeout in milliseconds",
+        "required": false,
+        "default": 10000,
+        "sensitive": false
+      },
+      "COMPUTER_USE_APPROVAL_MODE": {
+        "type": "string",
+        "description": "Approval mode for computer-use commands: full_control auto-approves everything, smart_approve auto-approves safe read-only commands, approve_all requires approval for every command, off denies all commands.",
+        "required": false,
+        "default": "smart_approve",
+        "enum": [
+          "full_control",
+          "smart_approve",
+          "approve_all",
+          "off"
+        ],
+        "sensitive": false
+      },
+      "COMPUTER_USE_BROWSER_HEADLESS": {
+        "type": "boolean",
+        "description": "Run browser automation headlessly. Useful for CI and test environments without a visible desktop session.",
+        "required": false,
+        "default": false,
+        "sensitive": false
+      },
+      "ELIZA_COMPUTERUSE_DRIVER": {
+        "type": "string",
+        "description": "Cross-platform input driver: 'nutjs' (default; uses @nut-tree-fork/nut-js native bindings, works on macOS/Linux/Windows) or 'legacy' (per-OS shell tools: cliclick/xdotool/PowerShell).",
+        "required": false,
+        "default": "nutjs",
+        "enum": [
+          "nutjs",
+          "legacy"
+        ],
+        "sensitive": false
+      },
+      "BROWSER_EXECUTE_DISABLED": {
+        "type": "boolean",
+        "description": "browser_execute / executeBrowser always fail closed (GHSA-rcvr-766c-4phv). Use DOM, clickables, click, type, navigate, and screenshot instead.",
+        "required": false,
+        "default": true,
+        "sensitive": false
+      }
+    }
+  },
+  "publishConfig": {
+    "access": "public"
+  }
+}

plugins/plugin-computeruse/registry-entry.json

@@ -0,0 +1,75 @@
+{
+  "id": "computeruse",
+  "name": "Computeruse",
+  "description": "Computer automation plugin for ElizaOS - enables AI agents to control the local machine (when supported) or a remote ComputerUse MCP server",
+  "npmName": "@elizaos/plugin-computeruse",
+  "shortIds": ["computeruse"],
+  "version": "2.0.0-beta.0",
+  "source": "bundled",
+  "tags": ["computeruse", "automation", "desktop", "gui", "mcp"],
+  "config": {
+    "COMPUTER_USE_ENABLED": {
+      "type": "boolean",
+      "required": false,
+      "sensitive": false,
+      "default": false,
+      "label": "Use Enabled",
+      "help": "Enable computer use capabilities (desktop automation, screenshots, browser control)",
+      "advanced": false
+    },
+    "COMPUTER_USE_SCREENSHOT_AFTER_ACTION": {
+      "type": "boolean",
+      "required": false,
+      "sensitive": false,
+      "default": true,
+      "label": "Use Screenshot After Action",
+      "help": "Automatically capture a screenshot after each desktop action",
+      "advanced": false
+    },
+    "COMPUTER_USE_ACTION_TIMEOUT_MS": {
+      "type": "number",
+      "required": false,
+      "sensitive": false,
+      "default": 10000,
+      "label": "Use Action Timeout Ms",
+      "help": "Maximum action execution timeout in milliseconds",
+      "advanced": false,
+      "min": 0,
+      "unit": "ms"
+    },
+    "COMPUTER_USE_APPROVAL_MODE": {
+      "type": "textarea",
+      "required": false,
+      "sensitive": false,
+      "default": "full_control",
+      "label": "Use Approval Mode",
+      "help": "Approval mode for computer-use commands: full_control auto-approves everything, smart_approve auto-approves safe read-only commands, approve_all requires approval for every command, off denies all commands.",
+      "advanced": false
+    },
+    "COMPUTER_USE_BROWSER_HEADLESS": {
+      "type": "boolean",
+      "required": false,
+      "sensitive": false,
+      "default": false,
+      "label": "Use Browser Headless",
+      "help": "Run browser automation headlessly. Useful for CI and test environments without a visible desktop session.",
+      "advanced": false
+    }
+  },
+  "render": {
+    "visible": true,
+    "pinTo": [],
+    "style": "card",
+    "icon": "MousePointer2",
+    "group": "devtools",
+    "groupOrder": 5,
+    "actions": ["enable", "configure"]
+  },
+  "resources": {
+    "homepage": "https://elizaos.ai",
+    "repository": "https://github.com/elizaos/eliza"
+  },
+  "dependsOn": [],
+  "kind": "plugin",
+  "subtype": "devtools"
+}

plugins/plugin-computeruse/scripts/ensure-platform-deps.mjs

@@ -0,0 +1,114 @@
+#!/usr/bin/env node
+/**
+ * Ensures platform-specific dependencies are installed for computer-use.
+ *
+ * Default driver = nutjs (cross-platform native bindings). With nutjs, no
+ * shell tools are required — this script exits early after confirming the
+ * native module loads. Set ELIZA_COMPUTERUSE_DRIVER=legacy to switch back
+ * to the per-OS shell drivers (cliclick / xdotool / PowerShell), in which
+ * case this script ensures those tools are present.
+ *
+ * macOS:  installs cliclick via Homebrew (fast, reliable mouse/keyboard control)
+ * Linux:  checks for xdotool, suggests install command
+ * Windows: no extra deps needed (PowerShell built-in)
+ *
+ * This runs as a postinstall hook — it's best-effort and never fails the install.
+ */
+
+import { execSync } from "node:child_process";
+import { platform } from "node:os";
+
+const driver = (process.env.ELIZA_COMPUTERUSE_DRIVER ?? "nutjs")
+  .trim()
+  .toLowerCase();
+
+if (driver === "nutjs") {
+  console.log(
+    "[computeruse] Driver = nutjs (cross-platform). Shell-tool checks are only required for the legacy driver.",
+  );
+  console.log(
+    "[computeruse] Set ELIZA_COMPUTERUSE_DRIVER=legacy to switch to per-OS shell drivers.",
+  );
+  process.exit(0);
+}
+
+function commandExists(cmd) {
+  try {
+    const which = platform() === "win32" ? "where" : "which";
+    execSync(`${which} ${cmd}`, { stdio: "ignore", timeout: 5000 });
+    return true;
+  } catch {
+    return false;
+  }
+}
+
+function run(cmd) {
+  try {
+    execSync(cmd, { stdio: "inherit", timeout: 120000 });
+    return true;
+  } catch {
+    return false;
+  }
+}
+
+const os = platform();
+
+if (os === "darwin") {
+  // macOS: install cliclick for fast, reliable mouse/keyboard control
+  if (!commandExists("cliclick")) {
+    if (commandExists("brew")) {
+      console.log(
+        "[computeruse] Installing cliclick via Homebrew (fast mouse/keyboard control)...",
+      );
+      if (run("brew install cliclick")) {
+        console.log("[computeruse] cliclick installed successfully.");
+      } else {
+        console.log(
+          "[computeruse] cliclick install failed — falling back to AppleScript (slower).",
+        );
+        console.log("[computeruse] To install manually: brew install cliclick");
+      }
+    } else {
+      console.log(
+        "[computeruse] Homebrew not found. For best performance, install cliclick:",
+      );
+      console.log(
+        '[computeruse]   /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"',
+      );
+      console.log("[computeruse]   brew install cliclick");
+      console.log(
+        "[computeruse] Falling back to AppleScript for mouse/keyboard control.",
+      );
+    }
+  } else {
+    console.log("[computeruse] cliclick already installed.");
+  }
+} else if (os === "linux") {
+  // Linux: check for xdotool
+  if (!commandExists("xdotool")) {
+    console.log(
+      "[computeruse] xdotool not found — required for mouse/keyboard control on Linux.",
+    );
+    console.log("[computeruse] Install via: sudo apt install xdotool");
+    console.log("[computeruse] Or: sudo dnf install xdotool");
+    console.log("[computeruse] Or: sudo pacman -S xdotool");
+  }
+
+  // Check for screenshot tool
+  if (
+    !commandExists("import") &&
+    !commandExists("scrot") &&
+    !commandExists("gnome-screenshot")
+  ) {
+    console.log("[computeruse] No screenshot tool found. Install one of:");
+    console.log(
+      "[computeruse]   sudo apt install imagemagick   (provides 'import')",
+    );
+    console.log("[computeruse]   sudo apt install scrot");
+    console.log("[computeruse]   sudo apt install gnome-screenshot");
+  }
+} else if (os === "win32") {
+  console.log(
+    "[computeruse] Windows detected — using built-in PowerShell for automation.",
+  );
+}

plugins/plugin-computeruse/scripts/record-windows-cua-input.mjs

@@ -0,0 +1,554 @@
+#!/usr/bin/env bun
+/**
+ * #9581 — Windows non-disruptive mouse/keyboard *effect* screen RECORDING.
+ *
+ * The capture harness (`capture-windows-desktop-evidence.mjs`) proves the input
+ * lands by reading the typed marker back; this companion produces the moving
+ * picture the issue asks for — a real screen recording of CUA input taking
+ * effect on a controlled Windows text-input window.
+ *
+ * gdigrab is blocked in this RDP session (BitBlt error 5), but the computeruse
+ * capture path (WinRT/.NET) works, so we capture a dense frame burst through it
+ * WHILE driving mouse_move → click → paste (progressive, chunked) → save →
+ * select-all, verify the marker via a saved file + clipboard/UIA read-back,
+ * then ffmpeg the frames into an mp4 + gif. Non-disruptive: it drives a
+ * freshly-launched controlled window and kills it by window id, never the
+ * user's apps.
+ *
+ * Run: bun scripts/record-windows-cua-input.mjs
+ */
+
+import { execFileSync } from "node:child_process";
+import { copyFile, mkdir, readFile, rm, writeFile } from "node:fs/promises";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+import { readClipboard, writeClipboard } from "../src/platform/clipboard.ts";
+import { ComputerUseService } from "../src/services/computer-use-service.ts";
+
+const here = path.dirname(fileURLToPath(import.meta.url));
+const repoRoot = path.resolve(here, "../../..");
+const outDir = path.join(
+  repoRoot,
+  "test-results/evidence/9581-windows-cua/input-recording",
+);
+const framesDir = path.join(outDir, "frames");
+
+const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
+
+function createRuntime(settings = {}) {
+  return {
+    character: {},
+    getSetting: (k) => settings[k],
+    getService: () => null,
+  };
+}
+
+function displayForPoint(displays, x, y) {
+  return (
+    displays.find((d) => {
+      const [dx, dy, w, h] = d.bounds;
+      return x >= dx && x < dx + w && y >= dy && y < dy + h;
+    }) ??
+    displays.find((d) => d.primary) ??
+    displays[0]
+  );
+}
+
+function windowHaystack(w) {
+  return `${w?.app ?? ""} ${w?.title ?? ""}`.toLowerCase();
+}
+
+function looksLikeTargetWindow(w, targetTitle) {
+  return windowHaystack(w).includes(targetTitle.toLowerCase());
+}
+
+function psSingleQuote(value) {
+  return `'${String(value).replace(/'/g, "''")}'`;
+}
+
+function createInputTargetScript(title, savePath) {
+  const savedTitle = `${title} (saved)`;
+  return `
+Add-Type -AssemblyName System.Windows.Forms
+Add-Type -AssemblyName System.Drawing
+$savePath = ${psSingleQuote(savePath)}
+$form = New-Object System.Windows.Forms.Form
+$form.Text = ${psSingleQuote(title)}
+$form.Width = 920
+$form.Height = 680
+$form.StartPosition = 'CenterScreen'
+$form.KeyPreview = $true
+$box = New-Object System.Windows.Forms.TextBox
+$box.Multiline = $true
+$box.AcceptsReturn = $true
+$box.AcceptsTab = $true
+$box.ScrollBars = 'Both'
+$box.WordWrap = $false
+$box.Dock = 'Fill'
+$box.Font = New-Object System.Drawing.Font('Consolas', 16)
+$form.Controls.Add($box)
+$save = {
+  [System.IO.File]::WriteAllText($savePath, $box.Text)
+  $form.Text = ${psSingleQuote(savedTitle)}
+}
+$box.Add_KeyDown({
+  if ($_.Control -and $_.KeyCode -eq [System.Windows.Forms.Keys]::S) {
+    & $save
+    $_.SuppressKeyPress = $true
+  }
+})
+$form.Add_KeyDown({
+  if ($_.Control -and $_.KeyCode -eq [System.Windows.Forms.Keys]::S) {
+    & $save
+    $_.SuppressKeyPress = $true
+  }
+})
+$form.Add_Shown({ $box.Focus() })
+[void]$form.ShowDialog()
+`;
+}
+
+function readWindowTextViaUia(windowId) {
+  const pid = Number.parseInt(String(windowId), 10);
+  if (!Number.isFinite(pid) || pid <= 0) return "";
+  const ps = `
+Add-Type -AssemblyName UIAutomationClient
+Add-Type -AssemblyName UIAutomationTypes
+$proc = Get-Process -Id ${pid} -ErrorAction Stop
+$root = [System.Windows.Automation.AutomationElement]::FromHandle($proc.MainWindowHandle)
+if ($null -eq $root) { exit 0 }
+$nodes = $root.FindAll([System.Windows.Automation.TreeScope]::Descendants, [System.Windows.Automation.Condition]::TrueCondition)
+$out = New-Object System.Collections.Generic.List[string]
+for ($i = 0; $i -lt $nodes.Count; $i++) {
+  $el = $nodes.Item($i)
+  try {
+    $pattern = $null
+    if ($el.TryGetCurrentPattern([System.Windows.Automation.TextPattern]::Pattern, [ref]$pattern)) {
+      $text = $pattern.DocumentRange.GetText(-1)
+      if ($text) { [void]$out.Add($text) }
+    }
+  } catch {}
+  try {
+    $pattern = $null
+    if ($el.TryGetCurrentPattern([System.Windows.Automation.ValuePattern]::Pattern, [ref]$pattern)) {
+      $value = $pattern.Current.Value
+      if ($value) { [void]$out.Add($value) }
+    }
+  } catch {}
+  try {
+    $name = $el.Current.Name
+    if ($name) { [void]$out.Add($name) }
+  } catch {}
+}
+$out -join "\`n"
+`;
+  try {
+    return execFileSync("powershell", ["-NoProfile", "-Command", ps], {
+      encoding: "utf8",
+      timeout: 15_000,
+      stdio: ["ignore", "pipe", "ignore"],
+    });
+  } catch {
+    return "";
+  }
+}
+
+let frameIndex = 0;
+async function captureFrame(service, label, frames) {
+  const shot = await service.executeCommand("screenshot");
+  if (!shot.success || !shot.screenshot) {
+    throw new Error(`screenshot failed: ${shot.error ?? "no payload"}`);
+  }
+  const buf = Buffer.from(shot.screenshot, "base64");
+  const name = `frame-${String(frameIndex).padStart(3, "0")}.png`;
+  await writeFile(path.join(framesDir, name), buf);
+  frames.push({ index: frameIndex, name, label, bytes: buf.length });
+  frameIndex++;
+  return buf.length;
+}
+
+async function main() {
+  await rm(outDir, { recursive: true, force: true }).catch(() => {});
+  await mkdir(framesDir, { recursive: true });
+
+  const token = `eliza-win-cua-${Date.now()}`;
+  const phrase = token;
+  const chunks = phrase.match(/.{1,12}/g) ?? [phrase];
+  const targetTitle = `elizaOS CUA Input Target ${process.pid}`;
+  const targetTextFile = path.join(outDir, "windows-cua-input-target.txt");
+  const targetScript = path.join(outDir, "windows-cua-input-target.ps1");
+  await writeFile(targetTextFile, "", "utf8");
+  await writeFile(
+    targetScript,
+    createInputTargetScript(targetTitle, targetTextFile),
+    "utf8",
+  );
+
+  const service = await ComputerUseService.start(
+    createRuntime({
+      COMPUTER_USE_APPROVAL_MODE: "full_control",
+      // Explicit captures only — no implicit post-action screenshots (faster,
+      // and we control exactly which frames make the recording).
+      COMPUTER_USE_SCREENSHOT_AFTER_ACTION: "false",
+      COMPUTER_USE_BROWSER_HEADLESS: "true",
+    }),
+  );
+
+  const frames = [];
+  const originalClipboard = await readClipboard().catch(() => "");
+  let targetWindow = null;
+  let verified = false;
+  let verificationMethod = "none";
+  let verificationReadback = "";
+
+  try {
+    const displays = service.getDisplays();
+
+    const launched = await service.executeCommand("launch", {
+      app: "cmd.exe",
+      appArgs: [
+        "/c",
+        "start",
+        "",
+        "powershell.exe",
+        "-NoProfile",
+        "-STA",
+        "-ExecutionPolicy",
+        "Bypass",
+        "-File",
+        targetScript,
+      ],
+    });
+    if (!launched.success) {
+      throw new Error(
+        `launch input target failed: ${launched.error ?? "unknown"}`,
+      );
+    }
+
+    // Resolve the real target window by the generated title. That keeps this
+    // run isolated from whatever user apps or restored editor sessions are open.
+    for (let attempt = 0; attempt < 20; attempt++) {
+      const found = await service.executeWindowAction({ action: "list" });
+      targetWindow =
+        (found?.windows ?? []).find((window) =>
+          looksLikeTargetWindow(window, targetTitle),
+        ) ?? null;
+      if (targetWindow?.id) break;
+      await sleep(250);
+    }
+    if (!targetWindow?.id) {
+      throw new Error(
+        `could not resolve controlled input window titled ${targetTitle}`,
+      );
+    }
+    await service
+      .executeWindowAction({ action: "focus", windowId: targetWindow.id })
+      .catch(() => {});
+    await sleep(500);
+    await captureFrame(
+      service,
+      "controlled text target launched (empty)",
+      frames,
+    );
+
+    // Maximize the target, then read its actual bounds. Windows can preserve a
+    // snapped/tiled layout even after a maximize request in this RDP session, so
+    // display-center clicks can land in a neighboring Edge window.
+    await service
+      .executeWindowAction({ action: "maximize", windowId: targetWindow.id })
+      .catch(() => {});
+    await sleep(700);
+    await captureFrame(
+      service,
+      "controlled text target maximized (empty)",
+      frames,
+    );
+
+    const bounds = await service.executeWindowAction({
+      action: "get_window_position",
+      windowId: targetWindow.id,
+    });
+    if (!bounds.success || !bounds.bounds) {
+      throw new Error(
+        `could not read target bounds: ${bounds.error ?? "unknown"}`,
+      );
+    }
+    const b = bounds.bounds;
+    // Aim at the center of the multiline text box.
+    const globalX = Math.round(b.x + b.width / 2);
+    const globalY = Math.round(b.y + b.height / 2);
+    const display = displayForPoint(displays, globalX, globalY);
+    if (!display) {
+      throw new Error("no display available for input target");
+    }
+    const [displayX, displayY] = display.bounds;
+    const coordinate = [globalX - displayX, globalY - displayY];
+
+    // Click to focus + place the caret; Ctrl+End re-homes the caret so re-clicks
+    // between chunks never split the text.
+    const focusControlledTarget = async () => {
+      if (!targetWindow?.id) {
+        throw new Error("Input target window is not resolved");
+      }
+      await service
+        .executeWindowAction({ action: "focus", windowId: targetWindow.id })
+        .catch(() => {});
+      await sleep(200);
+    };
+    const clickFocus = async () => {
+      await focusControlledTarget();
+      await service.executeCommand("click", {
+        coordinate,
+        displayId: display.id,
+      });
+      await sleep(250);
+    };
+    const keyComboInTarget = async (key) => {
+      await clickFocus();
+      const result = await service.executeCommand("key_combo", { key });
+      if (!result.success) {
+        throw new Error(`key_combo ${key} failed: ${result.error}`);
+      }
+      await sleep(250);
+    };
+
+    await service
+      .executeCommand("mouse_move", { coordinate, displayId: display.id })
+      .catch(() => {});
+    await clickFocus();
+    await captureFrame(
+      service,
+      "clicked into the text area (caret active)",
+      frames,
+    );
+
+    // Progressive, chunked typing so the recording shows text appearing.
+    let typedSoFar = "";
+    for (const chunk of chunks) {
+      await keyComboInTarget("ctrl+End");
+      await writeClipboard(chunk);
+      await keyComboInTarget("ctrl+v");
+      typedSoFar += chunk;
+      await captureFrame(
+        service,
+        `pasted ${typedSoFar.length}/${phrase.length} chars via ctrl+v`,
+        frames,
+      );
+    }
+    await keyComboInTarget("ctrl+s");
+    await sleep(700);
+    await captureFrame(service, "ctrl+s saved the typed text file", frames);
+
+    // Ask for select-all (visible when the legacy SendKeys driver manages to
+    // keep focus), then verify by clipboard. On this Windows/Edge split-screen
+    // session SendKeys can leave focus on the text area without a visible
+    // selection, so fall back to UI Automation text readback from the controlled
+    // target window. These methods read the real app/window, not a mock.
+    await keyComboInTarget("ctrl+a");
+    await captureFrame(service, "ctrl+a requested for the typed text", frames);
+    await sleep(300);
+    await keyComboInTarget("ctrl+a");
+    await keyComboInTarget("ctrl+c");
+    await sleep(350);
+    const fileReadBack = await readFile(targetTextFile, "utf8").catch(() => "");
+    const readBack = await readClipboard().catch(() => "");
+    const uiaReadBack = readWindowTextViaUia(targetWindow.id);
+    const freshWindows = await service.executeCommand("list_windows");
+    const titleReadBack = Array.isArray(freshWindows.windows)
+      ? freshWindows.windows
+          .filter((window) => looksLikeTargetWindow(window, targetTitle))
+          .map((window) => `${window.title ?? ""} ${window.app ?? ""}`.trim())
+          .join("\n")
+      : "";
+    if (fileReadBack.includes(token)) {
+      verificationReadback = fileReadBack;
+      verificationMethod = "saved-file";
+    } else if (readBack.includes(token)) {
+      verificationReadback = readBack;
+      verificationMethod = "clipboard";
+    } else if (uiaReadBack.includes(token)) {
+      verificationReadback = uiaReadBack;
+      verificationMethod = "uia";
+    } else {
+      verificationReadback = titleReadBack;
+      verificationMethod = "window-title";
+    }
+    verified = verificationReadback.includes(token);
+    await captureFrame(
+      service,
+      verified
+        ? `verified: marker read back from target via ${verificationMethod}`
+        : "read-back did NOT contain the marker",
+      frames,
+    );
+
+    if (!verified) {
+      throw new Error(
+        `read-back missing marker; clipboard=${JSON.stringify(
+          readBack.slice(0, 80),
+        )}; savedFile=${JSON.stringify(
+          fileReadBack.slice(0, 120),
+        )}; uia=${JSON.stringify(
+          uiaReadBack.slice(0, 120),
+        )}; title=${JSON.stringify(titleReadBack.slice(0, 120))}`,
+      );
+    }
+  } finally {
+    if (targetWindow?.id) {
+      const killed = await service
+        .executeCommand("kill_app", { target: String(targetWindow.id) })
+        .catch(() => ({ success: false }));
+      if (!killed.success) {
+        await service
+          .executeCommand("close_window", { windowId: targetWindow.id })
+          .catch(() => {});
+      }
+    }
+    await writeClipboard(originalClipboard).catch(() => {});
+    await service.stop().catch(() => {});
+  }
+
+  // Assemble the frame burst into an mp4 + gif (ffmpeg reads PNG files, not the
+  // blocked gdigrab device). ~2.5 fps reads as a clear step-by-step recording.
+  const fps = "5/2";
+  const pattern = path.join(framesDir, "frame-%03d.png");
+  const mp4 = path.join(outDir, "windows-cua-input.mp4");
+  const gif = path.join(outDir, "windows-cua-input.gif");
+  const palette = path.join(outDir, "palette.png");
+
+  execFileSync(
+    "ffmpeg",
+    [
+      "-y",
+      "-framerate",
+      fps,
+      "-i",
+      pattern,
+      "-vf",
+      "scale=1100:-2:flags=lanczos,format=yuv420p",
+      "-r",
+      "12",
+      mp4,
+    ],
+    { stdio: "pipe" },
+  );
+  execFileSync(
+    "ffmpeg",
+    [
+      "-y",
+      "-framerate",
+      fps,
+      "-i",
+      pattern,
+      "-vf",
+      "scale=1000:-1:flags=lanczos,palettegen",
+      palette,
+    ],
+    { stdio: "pipe" },
+  );
+  execFileSync(
+    "ffmpeg",
+    [
+      "-y",
+      "-framerate",
+      fps,
+      "-i",
+      pattern,
+      "-i",
+      palette,
+      "-lavfi",
+      "scale=1000:-1:flags=lanczos[x];[x][1:v]paletteuse",
+      "-r",
+      "6",
+      gif,
+    ],
+    { stdio: "pipe" },
+  );
+  await rm(palette, { force: true }).catch(() => {});
+
+  const initialFrame = frames[1] ?? frames[0];
+  const finalFrame = frames[frames.length - 1];
+  const initialStill = path.join(outDir, "initial-empty-input-window.png");
+  const finalStill = path.join(outDir, "final-typed-selected.png");
+  if (initialFrame) {
+    await copyFile(path.join(framesDir, initialFrame.name), initialStill);
+  }
+  if (finalFrame) {
+    await copyFile(path.join(framesDir, finalFrame.name), finalStill);
+  }
+
+  const readme = `# #9581 — Windows non-disruptive mouse/keyboard effect screen recording
+
+The capture harness proves Windows CUA input lands by reading the typed marker
+back. This is the moving-picture companion: a real screen recording of CUA mouse
+and keyboard input taking effect on a controlled Windows text-input window.
+
+Captured on a real Windows 11 Pro host (QEMU), 1728x1052, via
+\`plugins/plugin-computeruse/scripts/record-windows-cua-input.mjs\`:
+
+1. launch a generated Windows Forms text target through computeruse
+2. click inside the controlled text box bounds
+3. progressively paste a marker with \`Ctrl+V\` while capturing frames
+4. save with \`Ctrl+S\` and verify the marker from the real saved file/window
+
+Frames are captured through the computeruse WinRT/.NET capture path and
+assembled into MP4/GIF with ffmpeg.
+
+| File | What it is |
+|------|------------|
+| \`windows-cua-input.gif\` | Inline recording of the run. |
+| \`windows-cua-input.mp4\` | Same recording, H.264. |
+| \`final-typed-selected.png\` | Final frame after verification; the typed marker is visible in the target. |
+| \`initial-empty-input-window.png\` | Early frame before typing. |
+| \`windows-cua-input-target.txt\` | The real text file saved by the \`Ctrl+S\` step. |
+| \`windows-cua-input-target.ps1\` | The generated controlled Windows Forms target used for the run. |
+| \`recording-summary.json\` | Run metadata, including \`verified: true\` and verification method. |
+
+Verification method for this run: \`${verificationMethod}\`.
+`;
+  await writeFile(path.join(outDir, "README.md"), readme);
+
+  const summary = {
+    issue: 9581,
+    capturedAt: new Date().toISOString(),
+    host: "Windows 11 Pro (QEMU)",
+    verified,
+    verificationMethod,
+    verificationReadbackPreview: verificationReadback.slice(0, 240),
+    marker: token,
+    phrase,
+    frameCount: frames.length,
+    frames,
+    artifacts: {
+      mp4: path.relative(repoRoot, mp4),
+      gif: path.relative(repoRoot, gif),
+      initialStill: path.relative(repoRoot, initialStill),
+      finalStill: path.relative(repoRoot, finalStill),
+      savedTextFile: path.relative(repoRoot, targetTextFile),
+      targetScript: path.relative(repoRoot, targetScript),
+      readme: path.relative(repoRoot, path.join(outDir, "README.md")),
+      framesDir: path.relative(repoRoot, framesDir),
+    },
+  };
+  await writeFile(
+    path.join(outDir, "recording-summary.json"),
+    `${JSON.stringify(summary, null, 2)}\n`,
+  );
+
+  console.log(
+    JSON.stringify(
+      {
+        status: verified ? "passed" : "failed",
+        ...summary.artifacts,
+        frameCount: frames.length,
+      },
+      null,
+      2,
+    ),
+  );
+}
+
+main().catch((err) => {
+  console.error(err);
+  process.exit(1);
+});

plugins/plugin-computeruse/scripts/validate-ios-device-evidence.mjs

@@ -0,0 +1,160 @@
+#!/usr/bin/env node
+/**
+ * Validates an iOS device-evidence manifest against the required check-id
+ * contract (ReplayKit foreground start, broadcast-extension handshake, Vision
+ * OCR, app-intent invocations, Foundation-model generation, memory-pressure
+ * probe). Fails when a required check is missing or carries a status outside
+ * the allowed set.
+ */
+import { readFile } from "node:fs/promises";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+
+const REQUIRED_CHECK_IDS = new Set([
+  "probe",
+  "replayKitForegroundStart",
+  "broadcastExtensionHandshake",
+  "visionOcr",
+  "appIntentList",
+  "appIntentInvokeSafari",
+  "appIntentInvokeMessages",
+  "accessibilitySnapshot",
+  "foundationModelGenerate",
+  "memoryPressureProbe",
+]);
+
+const COMPLETE_STATUSES = new Set(["passed", "blocked_by_platform"]);
+const ALLOWED_STATUSES = new Set([
+  "requires_device_evidence",
+  "passed",
+  "failed",
+  "blocked_by_platform",
+]);
+
+function usage() {
+  return [
+    "Usage: node scripts/validate-ios-device-evidence.mjs [--require-complete] [manifest]",
+    "",
+    "Validates docs/ios-device-validation.json. With --require-complete, every",
+    "check must have passed or be explicitly blocked by platform behavior, and",
+    "top-level device/build/validator evidence must be present.",
+  ].join("\n");
+}
+
+function fail(message) {
+  console.error(`[ios-device-evidence] ${message}`);
+  process.exitCode = 1;
+}
+
+function isRecord(value) {
+  return typeof value === "object" && value !== null && !Array.isArray(value);
+}
+
+function nonEmptyString(value) {
+  return typeof value === "string" && value.trim().length > 0;
+}
+
+const args = process.argv.slice(2);
+let requireComplete = false;
+let manifestArg;
+
+for (const arg of args) {
+  if (arg === "--help" || arg === "-h") {
+    console.log(usage());
+    process.exit(0);
+  }
+  if (arg === "--require-complete") {
+    requireComplete = true;
+    continue;
+  }
+  if (manifestArg) {
+    fail(`unexpected argument: ${arg}`);
+  } else {
+    manifestArg = arg;
+  }
+}
+
+const here = path.dirname(fileURLToPath(import.meta.url));
+const manifestPath =
+  manifestArg ?? path.resolve(here, "../docs/ios-device-validation.json");
+
+const manifest = JSON.parse(await readFile(manifestPath, "utf8"));
+if (!isRecord(manifest)) fail("manifest must be an object");
+if (manifest.schemaVersion !== 1) fail("schemaVersion must be 1");
+if (manifest.platform !== "ios") fail('platform must be "ios"');
+if (!ALLOWED_STATUSES.has(manifest.status)) {
+  fail(`invalid top-level status: ${manifest.status}`);
+}
+if (!isRecord(manifest.target)) fail("target must be an object");
+for (const key of ["minimumIos", "primaryIos", "minimumDeviceClass"]) {
+  if (!nonEmptyString(manifest.target[key])) {
+    fail(`target.${key} must be a non-empty string`);
+  }
+}
+if (!isRecord(manifest.evidence)) fail("evidence must be an object");
+if (!Array.isArray(manifest.evidence.artifacts)) {
+  fail("evidence.artifacts must be an array");
+}
+if (!Array.isArray(manifest.checks)) fail("checks must be an array");
+
+const seen = new Set();
+for (const check of manifest.checks) {
+  if (!isRecord(check)) {
+    fail("each check must be an object");
+    continue;
+  }
+  if (!REQUIRED_CHECK_IDS.has(check.id)) {
+    fail(`unexpected check id: ${check.id}`);
+  }
+  if (seen.has(check.id)) {
+    fail(`duplicate check id: ${check.id}`);
+  }
+  seen.add(check.id);
+  if (!nonEmptyString(check.method)) {
+    fail(`check ${check.id} missing method`);
+  }
+  if (!ALLOWED_STATUSES.has(check.status)) {
+    fail(`check ${check.id} has invalid status: ${check.status}`);
+  }
+  if (
+    !Array.isArray(check.requiredEvidence) ||
+    check.requiredEvidence.length === 0 ||
+    !check.requiredEvidence.every(nonEmptyString)
+  ) {
+    fail(`check ${check.id} must list requiredEvidence strings`);
+  }
+}
+
+for (const id of REQUIRED_CHECK_IDS) {
+  if (!seen.has(id)) fail(`missing check id: ${id}`);
+}
+
+if (requireComplete) {
+  for (const key of [
+    "deviceModel",
+    "iosVersion",
+    "buildId",
+    "validatedAt",
+    "validator",
+  ]) {
+    if (!nonEmptyString(manifest.evidence[key])) {
+      fail(`--require-complete needs evidence.${key}`);
+    }
+  }
+  if (manifest.evidence.artifacts.length === 0) {
+    fail("--require-complete needs at least one evidence artifact");
+  }
+  for (const check of manifest.checks) {
+    if (!COMPLETE_STATUSES.has(check.status)) {
+      fail(`--require-complete: check ${check.id} is ${check.status}`);
+    }
+  }
+}
+
+if (process.exitCode) {
+  process.exit(process.exitCode);
+}
+
+console.log(
+  `[ios-device-evidence] ${manifest.checks.length} checks validated (${manifest.status})`,
+);

plugins/plugin-computeruse/scripts/validate-platform-evidence.mjs

@@ -0,0 +1,351 @@
+#!/usr/bin/env node
+/**
+ * Validates a desktop/mobile platform evidence manifest against its per-platform
+ * contract (required target keys and complete-evidence keys), enforcing that
+ * every check reports an allowed status and that passed platforms carry the
+ * device metadata the contract demands.
+ */
+import { readFile } from "node:fs/promises";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+
+const COMPLETE_STATUSES = new Set(["passed", "blocked_by_platform"]);
+const ALLOWED_STATUSES = new Set([
+  "requires_device_evidence",
+  "passed",
+  "failed",
+  "blocked_by_platform",
+]);
+
+const PLATFORM_CONTRACTS = {
+  ios: {
+    label: "ios-device-evidence",
+    defaultManifest: "docs/ios-device-validation.json",
+    targetKeys: ["minimumIos", "primaryIos", "minimumDeviceClass"],
+    completeEvidenceKeys: [
+      "deviceModel",
+      "iosVersion",
+      "buildId",
+      "validatedAt",
+      "validator",
+    ],
+    checkIds: [
+      "probe",
+      "replayKitForegroundStart",
+      "broadcastExtensionHandshake",
+      "visionOcr",
+      "appIntentList",
+      "appIntentInvokeSafari",
+      "appIntentInvokeMessages",
+      "accessibilitySnapshot",
+      "foundationModelGenerate",
+      "memoryPressureProbe",
+    ],
+  },
+  "android-consumer": {
+    label: "android-device-evidence",
+    defaultManifest: "docs/android-device-validation.json",
+    targetKeys: [
+      "minimumApi",
+      "minimumAndroid",
+      "requiredBuildFlavor",
+      "distribution",
+    ],
+    completeEvidenceKeys: [
+      "deviceModel",
+      "androidVersion",
+      "apiLevel",
+      "buildId",
+      "validatedAt",
+      "validator",
+    ],
+    checkIds: [
+      "permissionsSetup",
+      "accessibilityTree",
+      "gestureDispatch",
+      "globalActions",
+      "mediaProjectionCapture",
+      "usageStatsEnumeration",
+      "cameraCapture",
+      "memoryPressureDispatch",
+      "appActionsShortcuts",
+      "lifeOpsScheduledTaskHandoff",
+    ],
+  },
+  "android-aosp": {
+    label: "android-aosp-evidence",
+    defaultManifest: "docs/android-aosp-validation.json",
+    targetKeys: [
+      "minimumApi",
+      "imageType",
+      "requiredBuildFlavor",
+      "privilegedPermissions",
+    ],
+    completeEvidenceKeys: [
+      "imageName",
+      "androidVersion",
+      "apiLevel",
+      "buildId",
+      "validatedAt",
+      "validator",
+    ],
+    checkIds: [
+      "assistantRole",
+      "assistVoiceCommand",
+      "privilegedCapture",
+      "privilegedInput",
+      "processEnumeration",
+      "serviceFlavorSeparation",
+      "consumerBuildStripping",
+      "lifeOpsPersistence",
+    ],
+  },
+  "macos-desktop": {
+    label: "macos-desktop-evidence",
+    defaultManifest: "docs/macos-desktop-validation.json",
+    targetKeys: ["minimumMacos", "requiredPermissions", "driver"],
+    completeEvidenceKeys: [
+      "machineModel",
+      "macosVersion",
+      "buildId",
+      "validatedAt",
+      "validator",
+    ],
+    checkIds: [
+      "capabilityProbe",
+      "screenRecordingPermission",
+      "screenshotCapture",
+      "accessibilityPermission",
+      "mouseKeyboardInput",
+      "windowListFocus",
+      "browserAutomation",
+      "clipboardRoundTrip",
+      "approvalMode",
+    ],
+  },
+  "linux-desktop": {
+    label: "linux-desktop-evidence",
+    defaultManifest: "docs/linux-desktop-validation.json",
+    targetKeys: ["minimumDistribution", "displayServer", "driver"],
+    completeEvidenceKeys: [
+      "machineId",
+      "distribution",
+      "kernelVersion",
+      "displayServer",
+      "buildId",
+      "validatedAt",
+      "validator",
+    ],
+    checkIds: [
+      "capabilityProbe",
+      "dependencyProbe",
+      "screenshotCapture",
+      "mouseKeyboardInput",
+      "windowListFocus",
+      "browserAutomation",
+      "clipboardRoundTrip",
+      "terminalSafety",
+      "approvalMode",
+    ],
+  },
+  "windows-desktop": {
+    label: "windows-desktop-evidence",
+    defaultManifest: "docs/windows-desktop-validation.json",
+    targetKeys: ["minimumWindows", "driver", "shell"],
+    completeEvidenceKeys: [
+      "machineModel",
+      "windowsVersion",
+      "buildId",
+      "validatedAt",
+      "validator",
+    ],
+    checkIds: [
+      "capabilityProbe",
+      "screenshotCapture",
+      "mouseKeyboardInput",
+      "windowListFocus",
+      "browserAutomation",
+      "clipboardRoundTrip",
+      "terminalSafety",
+      "approvalMode",
+      "windowsHardeningRegression",
+    ],
+  },
+};
+
+function usage() {
+  return [
+    "Usage: node scripts/validate-platform-evidence.mjs [--require-complete] [manifest ...]",
+    "",
+    "Validates browser/computer-use platform evidence manifests. With no",
+    "manifest arguments, validates all default manifests. With --require-complete,",
+    "every check must have passed or be explicitly blocked by platform behavior,",
+    "and top-level device/build/validator evidence must be present.",
+  ].join("\n");
+}
+
+function isRecord(value) {
+  return typeof value === "object" && value !== null && !Array.isArray(value);
+}
+
+function nonEmptyString(value) {
+  return typeof value === "string" && value.trim().length > 0;
+}
+
+function nonEmptyStringArray(value) {
+  return (
+    Array.isArray(value) && value.length > 0 && value.every(nonEmptyString)
+  );
+}
+
+function hasEvidenceValue(value) {
+  return nonEmptyString(value) || nonEmptyStringArray(value);
+}
+
+function relativeFromHere(here, value) {
+  return path.isAbsolute(value) ? value : path.resolve(here, "..", value);
+}
+
+function parseArgs(argv) {
+  const manifests = [];
+  let requireComplete = false;
+  for (const arg of argv) {
+    if (arg === "--help" || arg === "-h") {
+      console.log(usage());
+      process.exit(0);
+    }
+    if (arg === "--require-complete") {
+      requireComplete = true;
+      continue;
+    }
+    manifests.push(arg);
+  }
+  return { manifests, requireComplete };
+}
+
+function validateManifest(manifest, manifestPath, requireComplete) {
+  const failures = [];
+  const fail = (message) => failures.push(`${manifestPath}: ${message}`);
+
+  if (!isRecord(manifest)) {
+    fail("manifest must be an object");
+    return failures;
+  }
+  if (manifest.schemaVersion !== 1) fail("schemaVersion must be 1");
+  if (!nonEmptyString(manifest.platform)) fail("platform must be a string");
+  const contract = PLATFORM_CONTRACTS[manifest.platform];
+  if (!contract) {
+    fail(`unsupported platform: ${manifest.platform}`);
+    return failures;
+  }
+  if (!ALLOWED_STATUSES.has(manifest.status)) {
+    fail(`invalid top-level status: ${manifest.status}`);
+  }
+  if (!isRecord(manifest.target)) {
+    fail("target must be an object");
+  } else {
+    for (const key of contract.targetKeys) {
+      if (!hasEvidenceValue(manifest.target[key])) {
+        fail(`target.${key} must be a non-empty string or string array`);
+      }
+    }
+  }
+  if (!isRecord(manifest.evidence)) {
+    fail("evidence must be an object");
+  } else if (!Array.isArray(manifest.evidence.artifacts)) {
+    fail("evidence.artifacts must be an array");
+  }
+  if (!Array.isArray(manifest.checks)) {
+    fail("checks must be an array");
+    return failures;
+  }
+
+  const requiredCheckIds = new Set(contract.checkIds);
+  const seen = new Set();
+  for (const check of manifest.checks) {
+    if (!isRecord(check)) {
+      fail("each check must be an object");
+      continue;
+    }
+    if (!requiredCheckIds.has(check.id))
+      fail(`unexpected check id: ${check.id}`);
+    if (seen.has(check.id)) fail(`duplicate check id: ${check.id}`);
+    seen.add(check.id);
+    if (!nonEmptyString(check.method)) fail(`check ${check.id} missing method`);
+    if (!ALLOWED_STATUSES.has(check.status)) {
+      fail(`check ${check.id} has invalid status: ${check.status}`);
+    }
+    if (
+      !Array.isArray(check.requiredEvidence) ||
+      check.requiredEvidence.length === 0 ||
+      !check.requiredEvidence.every(nonEmptyString)
+    ) {
+      fail(`check ${check.id} must list requiredEvidence strings`);
+    }
+  }
+
+  for (const id of requiredCheckIds) {
+    if (!seen.has(id)) fail(`missing check id: ${id}`);
+  }
+
+  if (requireComplete) {
+    for (const key of contract.completeEvidenceKeys) {
+      if (!hasEvidenceValue(manifest.evidence?.[key])) {
+        fail(`--require-complete needs evidence.${key}`);
+      }
+    }
+    if (
+      !Array.isArray(manifest.evidence?.artifacts) ||
+      manifest.evidence.artifacts.length === 0
+    ) {
+      fail("--require-complete needs at least one evidence artifact");
+    }
+    for (const check of manifest.checks) {
+      if (!COMPLETE_STATUSES.has(check.status)) {
+        fail(`--require-complete: check ${check.id} is ${check.status}`);
+      }
+    }
+  }
+
+  return failures;
+}
+
+const here = path.dirname(fileURLToPath(import.meta.url));
+const { manifests, requireComplete } = parseArgs(process.argv.slice(2));
+const manifestPaths =
+  manifests.length > 0
+    ? manifests.map((manifest) => relativeFromHere(here, manifest))
+    : Object.values(PLATFORM_CONTRACTS).map((contract) =>
+        path.resolve(here, "..", contract.defaultManifest),
+      );
+
+const failures = [];
+const summaries = [];
+
+for (const manifestPath of manifestPaths) {
+  let manifest;
+  try {
+    manifest = JSON.parse(await readFile(manifestPath, "utf8"));
+  } catch (error) {
+    failures.push(
+      `${manifestPath}: failed to read/parse JSON: ${
+        error instanceof Error ? error.message : String(error)
+      }`,
+    );
+    continue;
+  }
+  failures.push(...validateManifest(manifest, manifestPath, requireComplete));
+  const contract = PLATFORM_CONTRACTS[manifest?.platform];
+  if (contract && Array.isArray(manifest.checks)) {
+    summaries.push(
+      `[${contract.label}] ${manifest.checks.length} checks validated (${manifest.status})`,
+    );
+  }
+}
+
+if (failures.length > 0) {
+  for (const failure of failures) console.error(failure);
+  process.exit(1);
+}
+
+for (const summary of summaries) console.log(summary);

plugins/plugin-computeruse/src/__tests__/action-error-path.test.ts

@@ -0,0 +1,77 @@
+/**
+ * Real-error-path test for the COMPUTER_USE action boundary (#12273): a
+ * genuinely failing platform operation (spawning a binary that does not
+ * exist — no mocks of the failing dependency) must surface to the caller as
+ * `success:false` + `result.error`, which is what the planner loop shows the
+ * model. Guards against any regression toward catch-and-return-success.
+ */
+
+import type { IAgentRuntime, Memory } from "@elizaos/core";
+import { afterAll, describe, expect, it } from "vitest";
+import { useComputerAction } from "../actions/use-computer.js";
+import { ComputerUseService } from "../services/computer-use-service.js";
+
+const MISSING_BINARY =
+  process.platform === "win32"
+    ? "C:\\\\eliza-definitely-missing\\\\no-such-binary-12273.exe"
+    : "/eliza-definitely-missing/no-such-binary-12273";
+
+function makeRuntime(): {
+  runtime: IAgentRuntime;
+  attach: (service: ComputerUseService) => void;
+} {
+  let service: ComputerUseService | null = null;
+  const runtime = {
+    character: {},
+    getSetting: (key: string) =>
+      key === "COMPUTER_USE_APPROVAL_MODE" ? "full_control" : undefined,
+    getService: (name: string) => (name === "computeruse" ? service : null),
+    reportError: () => undefined,
+  } as unknown as IAgentRuntime;
+  return {
+    runtime,
+    attach: (s) => {
+      service = s;
+    },
+  };
+}
+
+const message = {
+  id: "00000000-0000-0000-0000-000000000001",
+  entityId: "00000000-0000-0000-0000-000000000002",
+  agentId: "00000000-0000-0000-0000-000000000003",
+  roomId: "00000000-0000-0000-0000-000000000004",
+  content: { action: "launch", app: MISSING_BINARY },
+} as unknown as Memory;
+
+describe("COMPUTER_USE surfaces real platform failures as success:false + error", () => {
+  let stopService: (() => Promise<void>) | null = null;
+
+  afterAll(async () => {
+    await stopService?.();
+  });
+
+  it("launching a nonexistent binary reaches the caller as a structured failure", async () => {
+    const { runtime, attach } = makeRuntime();
+    const service = (await ComputerUseService.start(
+      runtime,
+    )) as ComputerUseService;
+    attach(service);
+    stopService = () => service.stop();
+
+    const handler = useComputerAction.handler;
+    if (!handler) throw new Error("COMPUTER_USE handler missing");
+
+    // The spawn of MISSING_BINARY really fails (ENOENT) — the failing
+    // dependency is the OS itself, not a mock.
+    const result = await handler(runtime, message, undefined, undefined);
+
+    expect(result?.success).toBe(false);
+    expect(typeof result?.error === "string" ? result.error : "").not.toBe("");
+    // The old fabricated-success shape must never come back.
+    expect(result?.data).toMatchObject({
+      source: "computeruse",
+      computerUseAction: "launch",
+    });
+  });
+});

plugins/plugin-computeruse/src/__tests__/action-helpers.test.ts

@@ -0,0 +1,112 @@
+/**
+ * Unit coverage for the action param/result helpers (#9170).
+ *
+ * `resolveActionParams` merges planner-supplied `options.parameters` over the
+ * message content; `toComputerUseActionResult` shapes the ActionResult and
+ * **redacts raw screenshot base64 to a boolean** so multi-MB frames never leak
+ * into the action-result data; `buildScreenshotAttachment` builds the data-URI
+ * attachment. None were tested.
+ */
+
+import type { HandlerOptions, Memory } from "@elizaos/core";
+import { describe, expect, it } from "vitest";
+import {
+  buildScreenshotAttachment,
+  resolveActionParams,
+  toComputerUseActionResult,
+} from "../actions/helpers.js";
+
+describe("resolveActionParams", () => {
+  it("prefers options.parameters over message.content, filling the rest from content", () => {
+    const out = resolveActionParams<{ x: number; y: number }>(
+      { content: { x: 1, y: 2 } } as unknown as Memory,
+      { parameters: { x: 99 } } as unknown as HandlerOptions,
+    );
+    expect(out).toEqual({ x: 99, y: 2 });
+  });
+
+  it("falls back entirely to message.content when there are no options", () => {
+    const out = resolveActionParams<{ a: string }>({
+      content: { a: "hi" },
+    } as unknown as Memory);
+    expect(out).toEqual({ a: "hi" });
+  });
+
+  it("keeps a defined option even when it is falsy (does not get overwritten by content)", () => {
+    const out = resolveActionParams<{ flag: boolean }>(
+      { content: { flag: true } } as unknown as Memory,
+      { parameters: { flag: false } } as unknown as HandlerOptions,
+    );
+    expect(out.flag).toBe(false);
+  });
+});
+
+describe("buildScreenshotAttachment", () => {
+  it("builds a base64 PNG data-uri attachment with a prefixed id", () => {
+    const a = buildScreenshotAttachment({
+      idPrefix: "shot",
+      screenshot: "AAAA",
+      title: "Title",
+      description: "Desc",
+    });
+    expect(a.id.startsWith("shot-")).toBe(true);
+    expect(a.url).toBe("data:image/png;base64,AAAA");
+    expect(a.title).toBe("Title");
+    expect(a.description).toBe("Desc");
+    expect(a.source).toBe("computeruse");
+    expect(a.contentType).toBe("image");
+  });
+});
+
+describe("toComputerUseActionResult", () => {
+  it("maps a success result and redacts screenshot base64 to a boolean", () => {
+    const r = toComputerUseActionResult({
+      action: "click",
+      result: {
+        success: true,
+        message: "ok",
+        screenshot: "BBBB",
+        frontendScreenshot: "",
+      },
+      text: "Clicked",
+    });
+    expect(r.success).toBe(true);
+    expect(r.text).toBe("Clicked");
+    expect(r.error).toBeUndefined();
+    expect(r.data).toMatchObject({
+      source: "computeruse",
+      computerUseAction: "click",
+      result: {
+        success: true,
+        message: "ok",
+        hasScreenshot: true,
+        hasFrontendScreenshot: false,
+      },
+    });
+    // The raw base64 must NOT survive into the action-result data.
+    const data = r.data as { result: Record<string, unknown> };
+    expect(data.result.screenshot).toBeUndefined();
+    expect(data.result.frontendScreenshot).toBeUndefined();
+  });
+
+  it("surfaces an error on failure and the clipboard-suppression flag on request", () => {
+    const r = toComputerUseActionResult({
+      action: "ocr",
+      result: { success: false, error: "boom" },
+      text: "failed",
+      suppressClipboard: true,
+    });
+    expect(r.success).toBe(false);
+    expect(r.error).toBe("boom");
+    expect(r.data).toMatchObject({ suppressActionResultClipboard: true });
+  });
+
+  it("defaults the error message when a failure carries none", () => {
+    const r = toComputerUseActionResult({
+      action: "x",
+      result: { success: false },
+      text: "t",
+    });
+    expect(r.error).toBe("Computer-use failed");
+  });
+});

plugins/plugin-computeruse/src/__tests__/actor-registry.test.ts

@@ -0,0 +1,36 @@
+/**
+ * Unit coverage for the cascade Actor registry (#9170 M5/M10 grounding seam).
+ *
+ * `setActor` / `getRegisteredActor` is the single-slot seam through which a
+ * grounding Actor (OCR/AX coordinate grounding, or a remote grounder) is wired
+ * into the cascade loop. Its lifecycle was untested.
+ */
+
+import { afterEach, describe, expect, it } from "vitest";
+import type { Actor } from "../actor/actor.js";
+import { getRegisteredActor, setActor } from "../actor/cascade.js";
+
+const fakeActor = (name: string): Actor => ({
+  name,
+  ground: async () => {
+    throw new Error("ground() is not exercised by the registry test");
+  },
+});
+
+afterEach(() => setActor(null));
+
+describe("cascade Actor registry", () => {
+  it("is empty until an actor is registered", () => {
+    setActor(null);
+    expect(getRegisteredActor()).toBeNull();
+  });
+
+  it("returns the registered actor; last call wins; null clears", () => {
+    setActor(fakeActor("primary"));
+    expect(getRegisteredActor()?.name).toBe("primary");
+    setActor(fakeActor("secondary"));
+    expect(getRegisteredActor()?.name).toBe("secondary");
+    setActor(null);
+    expect(getRegisteredActor()).toBeNull();
+  });
+});

plugins/plugin-computeruse/src/__tests__/agent-callbacks.test.ts

@@ -0,0 +1,177 @@
+/**
+ * Agent-loop callback middleware (#9170 M11).
+ *
+ * Pure middlewares (budget cap, image-retention, operator-normalizer,
+ * trajectory) plus the fold helpers are asserted directly; the runner
+ * integration (default pipeline, budget abort, trajectory on the report) is
+ * covered in computer-use-agent.test.ts.
+ */
+
+import { describe, expect, it } from "vitest";
+import {
+  type AgentMiddleware,
+  createBudgetCapMiddleware,
+  createImageRetentionMiddleware,
+  createOperatorNormalizerMiddleware,
+  createTrajectoryMiddleware,
+  normalizeProposedAction,
+  runBeforeStep,
+  runTransformProposed,
+} from "../actor/agent-callbacks.js";
+import type { CascadeResult, ProposedAction } from "../actor/types.js";
+
+function stepCtx(over: Partial<{ step: number; elapsedMs: number }> = {}) {
+  return {
+    step: over.step ?? 1,
+    maxSteps: 5,
+    goal: "g",
+    elapsedMs: over.elapsedMs ?? 0,
+  };
+}
+
+function cascade(action: Partial<ProposedAction>): CascadeResult {
+  return {
+    scene_summary: "s",
+    rois: [],
+    proposed: {
+      kind: "click",
+      displayId: 0,
+      rationale: "r",
+      ...action,
+    } as ProposedAction,
+  };
+}
+
+describe("budget-cap middleware", () => {
+  it("aborts once the step budget is exceeded", async () => {
+    const mw = createBudgetCapMiddleware({ maxSteps: 2 });
+    expect((await runBeforeStep([mw], stepCtx({ step: 2 }))).abort).toBeFalsy();
+    const d = await runBeforeStep([mw], stepCtx({ step: 3 }));
+    expect(d.abort).toBe(true);
+    expect(d.reason).toContain("step budget");
+  });
+
+  it("aborts once the time budget is exceeded", async () => {
+    const mw = createBudgetCapMiddleware({ maxDurationMs: 1000 });
+    expect(
+      (await runBeforeStep([mw], stepCtx({ elapsedMs: 500 }))).abort,
+    ).toBeFalsy();
+    const d = await runBeforeStep([mw], stepCtx({ elapsedMs: 1500 }));
+    expect(d.abort).toBe(true);
+    expect(d.reason).toContain("time budget");
+  });
+});
+
+describe("image-retention middleware", () => {
+  it("keeps only the N most-recent steps' captures", async () => {
+    const mw = createImageRetentionMiddleware({ keepLast: 2 });
+    const cap = (id: number) => new Map([[id, { display: { id } } as never]]);
+    await mw.onCaptures?.(cap(0), stepCtx({ step: 1 }));
+    await mw.onCaptures?.(cap(0), stepCtx({ step: 2 }));
+    await mw.onCaptures?.(cap(1), stepCtx({ step: 3 }));
+    const retained = mw.retained();
+    expect(retained.map((r) => r.step)).toEqual([2, 3]);
+    expect(retained[1]?.displayIds).toEqual([1]);
+  });
+});
+
+describe("operator-normalizer", () => {
+  it("rounds coordinates, normalizes newlines, dedupes hotkey keys", () => {
+    const action: ProposedAction = {
+      kind: "drag",
+      displayId: 0,
+      x: 10.6,
+      y: 20.4,
+      startX: 1.5,
+      startY: 2.5,
+      dx: 3.2,
+      dy: -4.8,
+      text: "a\r\nb",
+      keys: ["ctrl", "ctrl", " shift "],
+      rationale: "r",
+    };
+    const out = normalizeProposedAction(action);
+    expect(out.x).toBe(11);
+    expect(out.y).toBe(20);
+    expect(out.startX).toBe(2);
+    expect(out.dy).toBe(-5);
+    expect(out.text).toBe("a\nb");
+    expect(out.keys).toEqual(["ctrl", "shift"]);
+  });
+
+  it("is idempotent on already-clean input", () => {
+    const clean: ProposedAction = {
+      kind: "click",
+      displayId: 0,
+      x: 5,
+      y: 5,
+      rationale: "r",
+    };
+    expect(normalizeProposedAction(normalizeProposedAction(clean))).toEqual(
+      clean,
+    );
+  });
+
+  it("transforms the proposed action through the pipeline fold", async () => {
+    const mw = createOperatorNormalizerMiddleware();
+    const result = await runTransformProposed(
+      [mw],
+      cascade({ x: 9.9, y: 0.1 }),
+      stepCtx(),
+    );
+    expect(result.proposed.x).toBe(10);
+    expect(result.proposed.y).toBe(0);
+  });
+});
+
+describe("trajectory middleware", () => {
+  it("records one entry per dispatched step", async () => {
+    const mw = createTrajectoryMiddleware();
+    await mw.afterStep?.({
+      step: 1,
+      goal: "g",
+      proposed: cascade({ kind: "click", rationale: "click save" }),
+      dispatchSuccess: true,
+    });
+    await mw.afterStep?.({
+      step: 2,
+      goal: "g",
+      proposed: cascade({ kind: "type", rationale: "type hello" }),
+      dispatchSuccess: false,
+      error: "boom",
+    });
+    const entries = mw.entries();
+    expect(entries).toHaveLength(2);
+    expect(entries[0]).toMatchObject({
+      step: 1,
+      actionKind: "click",
+      success: true,
+    });
+    expect(entries[1]).toMatchObject({
+      step: 2,
+      actionKind: "type",
+      success: false,
+      error: "boom",
+    });
+  });
+});
+
+describe("runBeforeStep fold", () => {
+  it("returns the first aborting middleware's reason", async () => {
+    const a: AgentMiddleware = {
+      name: "a",
+      beforeStep: () => ({ abort: false }),
+    };
+    const b: AgentMiddleware = {
+      name: "b",
+      beforeStep: () => ({ abort: true, reason: "stop here" }),
+    };
+    const c: AgentMiddleware = {
+      name: "c",
+      beforeStep: () => ({ abort: true, reason: "too late" }),
+    };
+    const d = await runBeforeStep([a, b, c], stepCtx());
+    expect(d.abort).toBe(true);
+    expect(d.reason).toBe("stop here");
+  });
+});

plugins/plugin-computeruse/src/__tests__/agent-loop.test.ts

@@ -0,0 +1,160 @@
+/**
+ * Agent-loop registry (#9170 M10).
+ *
+ * The registry maps a model string → loop, with the built-in local grounder as
+ * the match-anything fallback. These tests pin selection precedence, the
+ * model-family matcher, and the predictStep/predictClick seam of the default
+ * LocalGrounderLoop.
+ */
+
+import { afterEach, describe, expect, it } from "vitest";
+import {
+  _resetAgentLoopsForTests,
+  type AgentLoop,
+  createAgentLoop,
+  DEFAULT_AGENT_LOOP_MODEL,
+  LocalGrounderLoop,
+  listAgentLoops,
+  matchesModelFamily,
+  registerAgentLoop,
+  selectAgentLoopRegistration,
+  unregisterAgentLoop,
+} from "../actor/agent-loop.js";
+import type { CascadeResult } from "../actor/types.js";
+
+function noopLoop(name: string): AgentLoop {
+  return {
+    name,
+    async predictStep(): Promise<CascadeResult> {
+      return {
+        scene_summary: "",
+        rois: [],
+        proposed: { kind: "wait", displayId: 0, rationale: "" },
+      };
+    },
+    async predictClick() {
+      return null;
+    },
+  };
+}
+
+describe("agent-loop registry", () => {
+  afterEach(() => _resetAgentLoopsForTests());
+
+  it("falls back to the local grounder for any model string", () => {
+    const reg = selectAgentLoopRegistration("some/unknown-model");
+    expect(reg.name).toBe(DEFAULT_AGENT_LOOP_MODEL);
+  });
+
+  it("creates a LocalGrounderLoop by default", () => {
+    const loop = createAgentLoop("local-grounder", {
+      runtime: null,
+      getScene: () => null,
+    });
+    expect(loop).toBeInstanceOf(LocalGrounderLoop);
+    expect(loop.name).toBe(DEFAULT_AGENT_LOOP_MODEL);
+  });
+
+  it("prefers a registered loop over the local fallback when it matches", () => {
+    registerAgentLoop({
+      name: "anthropic-cua",
+      matches: matchesModelFamily("anthropic"),
+      create: () => noopLoop("anthropic-cua"),
+      priority: 10,
+    });
+    expect(selectAgentLoopRegistration("anthropic/claude-opus").name).toBe(
+      "anthropic-cua",
+    );
+    // A non-matching string still falls back to the local grounder.
+    expect(
+      selectAgentLoopRegistration("openai/computer-use-preview").name,
+    ).toBe(DEFAULT_AGENT_LOOP_MODEL);
+  });
+
+  it("ranks higher-priority registrations first when several match", () => {
+    registerAgentLoop({
+      name: "low",
+      matches: () => true,
+      create: () => noopLoop("low"),
+      priority: 1,
+    });
+    registerAgentLoop({
+      name: "high",
+      matches: () => true,
+      create: () => noopLoop("high"),
+      priority: 5,
+    });
+    expect(selectAgentLoopRegistration("anything").name).toBe("high");
+  });
+
+  it("unregisters a loop, reverting to the fallback", () => {
+    registerAgentLoop({
+      name: "temp",
+      matches: matchesModelFamily("openai"),
+      create: () => noopLoop("temp"),
+      priority: 3,
+    });
+    expect(selectAgentLoopRegistration("openai/x").name).toBe("temp");
+    unregisterAgentLoop("temp");
+    expect(selectAgentLoopRegistration("openai/x").name).toBe(
+      DEFAULT_AGENT_LOOP_MODEL,
+    );
+  });
+
+  it("listAgentLoops returns registrations sorted by descending priority", () => {
+    registerAgentLoop({
+      name: "p2",
+      matches: () => false,
+      create: () => noopLoop("p2"),
+      priority: 2,
+    });
+    const names = listAgentLoops().map((r) => r.name);
+    // The local grounder sits at -Infinity, so it is last.
+    expect(names[0]).toBe("p2");
+    expect(names[names.length - 1]).toBe(DEFAULT_AGENT_LOOP_MODEL);
+  });
+});
+
+describe("matchesModelFamily", () => {
+  const m = matchesModelFamily("anthropic");
+  it("matches family slash/dash/equals/substring forms", () => {
+    expect(m("anthropic")).toBe(true);
+    expect(m("anthropic/claude-3-7")).toBe(true);
+    expect(m("anthropic-claude")).toBe(true);
+    expect(m("provider/anthropic/model")).toBe(true);
+    expect(m("ANTHROPIC/Claude")).toBe(true);
+  });
+  it("does not match unrelated families", () => {
+    expect(matchesModelFamily("openai")("anthropic/claude")).toBe(false);
+  });
+});
+
+describe("LocalGrounderLoop.predictClick", () => {
+  it("returns null when there is no ref/roi to ground", async () => {
+    const loop = new LocalGrounderLoop({ runtime: null, getScene: () => null });
+    const result = await loop.predictClick({
+      scene: {
+        timestamp: 1,
+        displays: [
+          {
+            id: 0,
+            bounds: [0, 0, 1920, 1080],
+            scaleFactor: 1,
+            primary: true,
+            name: "d0",
+          },
+        ],
+        focused_window: null,
+        apps: [],
+        ocr: [],
+        ax: [],
+        vlm_scene: null,
+        vlm_elements: null,
+      },
+      captures: new Map(),
+      targetDisplayId: 0,
+      instruction: "click something",
+    });
+    expect(result).toBeNull();
+  });
+});

plugins/plugin-computeruse/src/__tests__/android-bridge.test.ts

@@ -0,0 +1,444 @@
+/**
+ * Pure-JS tests for the Android bridge type contract, the AndroidBridgeResult
+ * discriminated union, error propagation, and constant values.
+ *
+ * These do NOT exercise any Kotlin code — that is unverified on this host
+ * and lives behind the on-device validation checklist in
+ * docs/ANDROID_CONSTRAINTS.md.
+ */
+
+import fs from "node:fs";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+import { describe, expect, it } from "vitest";
+import {
+  ANDROID_BRIDGE_JS_NAME,
+  ANDROID_DEFAULT_FPS,
+  type AndroidAxNode,
+  type AndroidBridgeErrorCode,
+  type AndroidBridgeProbe,
+  type AndroidBridgeResult,
+  type AndroidComputerUseBridge,
+  type AndroidMemoryPressureSnapshot,
+  type AppUsageEntry,
+  type CapturedScreenFrame,
+  type GestureArgs,
+  type GlobalAction,
+  type SwipeGestureArgs,
+} from "../mobile/android-bridge.js";
+
+// ── Constants ─────────────────────────────────────────────────────────────────
+
+describe("Android bridge constants", () => {
+  it("uses the same jsName as iOS so the planner needs no platform branch", () => {
+    expect(ANDROID_BRIDGE_JS_NAME).toBe("ComputerUse");
+  });
+
+  it("default FPS is conservative for battery life", () => {
+    expect(ANDROID_DEFAULT_FPS).toBe(1);
+  });
+});
+
+// ── Android assistant/App Actions static source checks ───────────────────────
+
+const repoRoot = path.resolve(
+  path.dirname(fileURLToPath(import.meta.url)),
+  "../../../..",
+);
+const readRepoFile = (relativePath: string) =>
+  fs.readFileSync(path.join(repoRoot, relativePath), "utf8");
+
+describe("Android Assistant and App Actions routing source", () => {
+  it("registers shortcuts.xml on the launcher activity", () => {
+    const manifest = readRepoFile(
+      "packages/app-core/platforms/android/app/src/main/AndroidManifest.xml",
+    );
+
+    expect(manifest).toContain('android:name="android.app.shortcuts"');
+    expect(manifest).toContain('android:resource="@xml/shortcuts"');
+  });
+
+  it("declares default-assistant and voice-command surfaces without privileged voice permissions", () => {
+    const manifest = readRepoFile(
+      "packages/app-core/platforms/android/app/src/main/AndroidManifest.xml",
+    );
+
+    expect(manifest).toContain("ElizaAssistActivity");
+    expect(manifest).toContain("android.intent.action.ASSIST");
+    expect(manifest).toContain("android.intent.action.VOICE_COMMAND");
+    expect(manifest).not.toContain("android.app.role.ASSISTANT");
+    // BIND_VOICE_INTERACTION is the standard binder guard the framework
+    // REQUIRES on the exported VoiceInteractionService/session services
+    // (`android:permission=` on the <service>) so only the system can bind them
+    // — not a privileged capability the app requests. "Without privileged voice
+    // permissions" means the app must never DECLARE it as a granted
+    // <uses-permission>; the service-protection attribute is correct and stays.
+    expect(manifest).not.toMatch(
+      /<uses-permission[^>]*android:name="android\.permission\.BIND_VOICE_INTERACTION"/,
+    );
+  });
+
+  it("declares Play-compatible App Actions BIIs and static shortcuts", () => {
+    const shortcuts = readRepoFile(
+      "packages/app-core/platforms/android/app/src/main/res/xml/shortcuts.xml",
+    );
+
+    expect(shortcuts).toContain(
+      'android:name="actions.intent.OPEN_APP_FEATURE"',
+    );
+    expect(shortcuts).toContain('android:name="actions.intent.CREATE_MESSAGE"');
+    expect(shortcuts).toContain('android:name="actions.intent.GET_THING"');
+    expect(shortcuts).toContain('android:name="feature"');
+    expect(shortcuts).toContain('android:required="true"');
+    expect(shortcuts).not.toContain("actions.intent.CREATE_THING");
+    expect(shortcuts).toContain('android:shortcutId="eliza_app_action_chat"');
+    expect(shortcuts).toContain('android:shortcutId="eliza_app_action_voice"');
+    expect(shortcuts).toContain(
+      'android:shortcutId="eliza_app_action_daily_brief"',
+    );
+    expect(shortcuts).toContain(
+      'android:shortcutId="eliza_app_action_new_task"',
+    );
+    expect(shortcuts).toContain('android:shortcutId="eliza_app_action_tasks"');
+    expect(shortcuts).toContain("source=android-app-actions");
+    expect(shortcuts).toContain("source=android-static-shortcut");
+    expect(shortcuts).toContain(
+      "elizaos://feature/open?source=android-app-actions",
+    );
+    expect(shortcuts).toContain(
+      "elizaos://chat?source=android-app-actions&amp;action=chat",
+    );
+    expect(shortcuts).toContain(
+      "elizaos://lifeops/task/new?source=android-static-shortcut",
+    );
+    expect(shortcuts).not.toContain("example://");
+    expect(shortcuts.toLowerCase()).not.toContain("notification");
+    expect(shortcuts).not.toContain("assistant/open");
+    expect(shortcuts).not.toContain("android.intent.action.ASSIST");
+    expect(shortcuts).not.toContain("android.intent.action.VOICE_COMMAND");
+    expect(shortcuts).not.toContain("ScheduledTask");
+  });
+});
+
+// ── AndroidBridgeResult discriminated union ───────────────────────────────────
+
+describe("AndroidBridgeResult<T> discriminated union", () => {
+  it("ok=true narrows to data", () => {
+    const result: AndroidBridgeResult<number> = { ok: true, data: 42 };
+    if (result.ok) {
+      expect(result.data).toBe(42);
+    } else {
+      throw new Error("unreachable");
+    }
+  });
+
+  it("ok=false narrows to code + message", () => {
+    const result: AndroidBridgeResult<number> = {
+      ok: false,
+      code: "permission_denied",
+      message: "CAMERA not granted",
+    };
+    if (!result.ok) {
+      expect(result.code).toBe("permission_denied");
+      expect(result.message).toBe("CAMERA not granted");
+    } else {
+      throw new Error("unreachable");
+    }
+  });
+
+  it("all error codes are string literals", () => {
+    const codes: AndroidBridgeErrorCode[] = [
+      "unsupported_platform",
+      "permission_denied",
+      "permission_pending",
+      "accessibility_unavailable",
+      "capture_unavailable",
+      "camera_not_open",
+      "invalid_argument",
+      "internal_error",
+    ];
+    for (const code of codes) {
+      expect(typeof code).toBe("string");
+    }
+  });
+});
+
+// ── AndroidAxNode (WS6 Scene.ax shape) ────────────────────────────────────────
+
+describe("AndroidAxNode shape", () => {
+  it("has all required fields matching WS6 Scene.ax contract", () => {
+    const node: AndroidAxNode = {
+      id: "42",
+      role: "android.widget.Button",
+      label: "OK",
+      bbox: { x: 100, y: 200, w: 80, h: 40 },
+      actions: ["click", "focus"],
+    };
+    expect(node.id).toBe("42");
+    expect(node.role).toBe("android.widget.Button");
+    expect(node.bbox.w).toBe(80);
+    expect(node.bbox.h).toBe(40);
+    expect(node.actions).toContain("click");
+  });
+
+  it("label is nullable (nodes without text or contentDescription)", () => {
+    const node: AndroidAxNode = {
+      id: "0",
+      role: "android.view.View",
+      label: null,
+      bbox: { x: 0, y: 0, w: 100, h: 100 },
+      actions: [],
+    };
+    expect(node.label).toBeNull();
+  });
+});
+
+// ── GestureArgs discriminated union ───────────────────────────────────────────
+
+describe("GestureArgs discriminated union", () => {
+  it("tap gesture has type='tap', x, y", () => {
+    const g: GestureArgs = { type: "tap", x: 300, y: 500 };
+    if (g.type === "tap") {
+      expect(g.x).toBe(300);
+      expect(g.y).toBe(500);
+    }
+  });
+
+  it("swipe gesture has type='swipe' and target coords", () => {
+    const g: GestureArgs = {
+      type: "swipe",
+      x: 100,
+      y: 800,
+      x2: 100,
+      y2: 200,
+      durationMs: 400,
+    };
+    if (g.type === "swipe") {
+      expect(g.x2).toBe(100);
+      expect(g.y2).toBe(200);
+      expect(g.durationMs).toBe(400);
+    }
+  });
+
+  it("swipe durationMs is optional", () => {
+    const g: SwipeGestureArgs = { type: "swipe", x: 0, y: 0, x2: 100, y2: 100 };
+    expect(g.durationMs).toBeUndefined();
+  });
+});
+
+// ── GlobalAction union ────────────────────────────────────────────────────────
+
+describe("GlobalAction", () => {
+  it("covers the four required system actions", () => {
+    const actions: GlobalAction[] = [
+      "back",
+      "home",
+      "recents",
+      "notifications",
+    ];
+    expect(actions).toHaveLength(4);
+  });
+});
+
+// ── CapturedScreenFrame ───────────────────────────────────────────────────────
+
+describe("CapturedScreenFrame", () => {
+  it("has the expected shape", () => {
+    const frame: CapturedScreenFrame = {
+      jpegBase64: "base64string",
+      width: 1080,
+      height: 1920,
+      timestampMs: Date.now(),
+    };
+    expect(frame.jpegBase64).toBe("base64string");
+    expect(frame.width).toBe(1080);
+    expect(frame.height).toBe(1920);
+    expect(typeof frame.timestampMs).toBe("number");
+  });
+});
+
+// ── AndroidMemoryPressureSnapshot ────────────────────────────────────────────
+
+describe("AndroidMemoryPressureSnapshot", () => {
+  it("has all required pressure fields", () => {
+    const snap: AndroidMemoryPressureSnapshot = {
+      level: "low",
+      freeMb: 512,
+      maxMb: 4096,
+      usedMb: 3584,
+      source: "android-runtime",
+    };
+    expect(snap.level).toBe("low");
+    expect(snap.freeMb).toBe(512);
+    expect(snap.source).toBe("android-runtime");
+  });
+
+  it("pressure levels are the expected string literals", () => {
+    const valid = ["nominal", "low", "critical"];
+    for (const level of valid) {
+      const snap: AndroidMemoryPressureSnapshot = {
+        level: level as AndroidMemoryPressureSnapshot["level"],
+        freeMb: 0,
+        maxMb: 0,
+        usedMb: 0,
+        source: "android-runtime",
+      };
+      expect(valid).toContain(snap.level);
+    }
+  });
+});
+
+// ── AndroidBridgeProbe ───────────────────────────────────────────────────────
+
+describe("AndroidBridgeProbe", () => {
+  it("reports Android platform metadata and capability booleans", async () => {
+    const fakeBridge = buildFakeAndroidBridge({
+      probe: async () => ({
+        ok: true,
+        data: {
+          platform: "android",
+          osVersion: "14",
+          sdkInt: 34,
+          capabilities: {
+            mediaProjection: true,
+            accessibilityService: false,
+            usageStats: true,
+            camera: false,
+            aospPrivileged: false,
+          },
+        },
+      }),
+    });
+
+    const result = await fakeBridge.probe();
+    expect(result.ok).toBe(true);
+    if (!result.ok) throw new Error("expected probe success");
+
+    const probe: AndroidBridgeProbe = result.data;
+    expect(probe.platform).toBe("android");
+    expect(probe.osVersion).toBe("14");
+    expect(probe.sdkInt).toBe(34);
+    expect(typeof probe.capabilities.mediaProjection).toBe("boolean");
+    expect(typeof probe.capabilities.accessibilityService).toBe("boolean");
+    expect(typeof probe.capabilities.usageStats).toBe("boolean");
+    expect(typeof probe.capabilities.camera).toBe("boolean");
+    expect(typeof probe.capabilities.aospPrivileged).toBe("boolean");
+  });
+});
+
+// ── AppUsageEntry (enumerateApps) ─────────────────────────────────────────────
+
+describe("AppUsageEntry", () => {
+  it("has the expected shape matching WS6 enumerateApps interface", () => {
+    const entry: AppUsageEntry = {
+      packageName: "com.android.chrome",
+      label: "Chrome",
+      lastUsedMs: Date.now() - 60_000,
+      totalForegroundMs: 3_600_000,
+      isForeground: false,
+    };
+    expect(entry.packageName).toBe("com.android.chrome");
+    expect(entry.totalForegroundMs).toBe(3_600_000);
+    expect(typeof entry.isForeground).toBe("boolean");
+  });
+});
+
+// ── onTrimMemory → MemoryArbiter pressure call chain (structural) ─────────────
+
+describe("onTrimMemory → dispatchMemoryPressure call chain", () => {
+  /**
+   * Structural verification of the call chain described in android-bridge.ts:
+   *
+   *   Kotlin ComponentCallbacks2.onTrimMemory(level)
+   *   → notifyListeners("memoryPressure", { level, freeMb })
+   *   → JS: AndroidComputerUseBridge.dispatchMemoryPressure({ level, freeMb })
+   *   → capacitorPressureSource.dispatch(level, freeMb) [WS1]
+   *   → MemoryArbiter pressure listener
+   *
+   * This test verifies the JS signature accepts the expected payload.
+   */
+  it("dispatchMemoryPressure accepts nominal level", async () => {
+    const fakeBridge = buildFakeAndroidBridge({
+      dispatchMemoryPressure: async ({ level, freeMb }) => {
+        expect(level).toBe("nominal");
+        expect(freeMb).toBe(2048);
+        return { ok: true, data: { ok: true } };
+      },
+    });
+    const result = await fakeBridge.dispatchMemoryPressure({
+      level: "nominal",
+      freeMb: 2048,
+    });
+    expect(result.ok).toBe(true);
+  });
+
+  it("dispatchMemoryPressure accepts critical level without freeMb", async () => {
+    const fakeBridge = buildFakeAndroidBridge({
+      dispatchMemoryPressure: async ({ level }) => {
+        expect(level).toBe("critical");
+        return { ok: true, data: { ok: true } };
+      },
+    });
+    const result = await fakeBridge.dispatchMemoryPressure({
+      level: "critical",
+    });
+    expect(result.ok).toBe(true);
+  });
+
+  it("getAccessibilityTree error propagates as ok=false", async () => {
+    const fakeBridge = buildFakeAndroidBridge({
+      getAccessibilityTree: async () => ({
+        ok: false,
+        code: "accessibility_unavailable",
+        message: "Service not running",
+      }),
+    });
+    const result = await fakeBridge.getAccessibilityTree();
+    if (!result.ok) {
+      expect(result.code).toBe("accessibility_unavailable");
+    } else {
+      throw new Error("expected failure");
+    }
+  });
+});
+
+describe("Android bridge featureCheck", () => {
+  it("reports unsupported when Capacitor is absent (Node test host)", async () => {
+    const { featureCheck, getAndroidBridge } = await import(
+      "../mobile/android-bridge.js"
+    );
+    const result = featureCheck();
+    expect(result.supported).toBe(false);
+    expect(result.reason).toMatch(/Capacitor/i);
+    expect(getAndroidBridge()).toBeNull();
+  });
+});
+
+// ── Fake bridge builder ───────────────────────────────────────────────────────
+
+function buildFakeAndroidBridge(
+  overrides: Partial<AndroidComputerUseBridge> = {},
+): AndroidComputerUseBridge {
+  const unavailable = <T>(
+    code: AndroidBridgeErrorCode = "internal_error",
+  ): Promise<AndroidBridgeResult<T>> =>
+    Promise.resolve({ ok: false, code, message: "unavailable" });
+
+  return {
+    probe: () => unavailable("internal_error"),
+    startMediaProjection: () => unavailable("internal_error"),
+    stopMediaProjection: () => unavailable("internal_error"),
+    captureFrame: () => unavailable("capture_unavailable"),
+    getAccessibilityTree: () => unavailable("accessibility_unavailable"),
+    dispatchGesture: () => unavailable("accessibility_unavailable"),
+    performGlobalAction: () => unavailable("accessibility_unavailable"),
+    setText: () => unavailable("accessibility_unavailable"),
+    enumerateApps: () => unavailable("permission_denied"),
+    getMemoryPressureSnapshot: () => unavailable("internal_error"),
+    dispatchMemoryPressure: () => unavailable("internal_error"),
+    startCamera: () => unavailable("permission_denied"),
+    stopCamera: () => Promise.resolve({ ok: true, data: { ok: true } }),
+    captureFrameCamera: () => unavailable("camera_not_open"),
+    ...overrides,
+  };
+}

plugins/plugin-computeruse/src/__tests__/android-scene.test.ts

@@ -0,0 +1,200 @@
+/**
+ * WS8 — Android AX tree normalization tests.
+ *
+ * Verifies the JSON emitted by `ElizaAccessibilityService.getAccessibilityTreeJson()`
+ * (Kotlin) round-trips through `parseAndroidAxTree` into the WS6 `Scene.ax`
+ * shape that WS7's `OcrCoordinateGroundingActor.resolveReference` expects.
+ *
+ * The Kotlin shape (from ElizaAccessibilityService.kt walkNode):
+ *
+ *   {
+ *     "id": "0",
+ *     "role": "android.widget.Button",
+ *     "label": "OK"        // or null
+ *     "bbox": { "x": 100, "y": 200, "w": 80, "h": 40 },
+ *     "actions": ["click", "focus"]
+ *   }
+ *
+ * The TS-side scene shape (SceneAxNode):
+ *
+ *   { id: "a0-0", role, label, bbox: [100, 200, 80, 40], actions, displayId }
+ */
+
+import { describe, expect, it } from "vitest";
+import { resolveReference } from "../actor/actor.js";
+import {
+  androidAxIdToSceneId,
+  normalizeAndroidAxNode,
+  parseAndroidAxTree,
+  sceneAxToAndroidAxNode,
+} from "../mobile/android-scene.js";
+import { ANDROID_LOGICAL_DISPLAY_ID } from "../mobile/mobile-screen-capture.js";
+import type { Scene } from "../scene/scene-types.js";
+
+/**
+ * Fixture mirroring exactly what ElizaAccessibilityService.kt would emit.
+ * We verify the parser handles every documented quirk:
+ *   - integer ids stringified
+ *   - label nullable
+ *   - bbox object → tuple
+ *   - unknown actions filtered out is not the contract — we keep them verbatim
+ *     as the Kotlin side enumerates only the documented set.
+ */
+const KOTLIN_AX_JSON = JSON.stringify([
+  {
+    id: "0",
+    role: "android.widget.FrameLayout",
+    label: null,
+    bbox: { x: 0, y: 0, w: 1080, h: 1920 },
+    actions: ["focus"],
+  },
+  {
+    id: "1",
+    role: "android.widget.Button",
+    label: "Save",
+    bbox: { x: 100, y: 200, w: 80, h: 40 },
+    actions: ["click", "focus"],
+  },
+  {
+    id: "2",
+    role: "android.widget.EditText",
+    label: "Title",
+    bbox: { x: 100, y: 400, w: 880, h: 120 },
+    actions: ["click", "type", "focus"],
+  },
+]);
+
+describe("parseAndroidAxTree", () => {
+  it("normalizes the Kotlin shape into the WS6 SceneAxNode shape", () => {
+    const nodes = parseAndroidAxTree(
+      KOTLIN_AX_JSON,
+      ANDROID_LOGICAL_DISPLAY_ID,
+    );
+    expect(nodes).toHaveLength(3);
+    expect(nodes[0]).toMatchObject({
+      id: "a0-0",
+      role: "android.widget.FrameLayout",
+      bbox: [0, 0, 1080, 1920],
+      actions: ["focus"],
+      displayId: 0,
+    });
+    expect(nodes[0]?.label).toBeUndefined();
+    expect(nodes[1]).toMatchObject({
+      id: "a0-1",
+      role: "android.widget.Button",
+      label: "Save",
+      bbox: [100, 200, 80, 40],
+      actions: ["click", "focus"],
+    });
+    expect(nodes[2]?.actions).toEqual(["click", "type", "focus"]);
+  });
+
+  it("uses the supplied displayId in the scene-ax id", () => {
+    const nodes = parseAndroidAxTree(KOTLIN_AX_JSON, 7);
+    expect(nodes[1]?.id).toBe("a7-1");
+    expect(nodes[1]?.displayId).toBe(7);
+  });
+
+  it("returns an empty array for the Kotlin error sentinel `[]`", () => {
+    expect(parseAndroidAxTree("[]", ANDROID_LOGICAL_DISPLAY_ID)).toEqual([]);
+  });
+
+  it("drops malformed entries instead of throwing", () => {
+    const bad = JSON.stringify([
+      {
+        id: "1",
+        role: "Button",
+        bbox: { x: 0, y: 0, w: 10, h: 10 },
+        actions: [],
+      },
+      { id: 99, role: "no-string-id" }, // dropped
+      {
+        id: "2",
+        role: "X",
+        bbox: { x: 0, y: 0, w: "bad" as unknown as number, h: 10 },
+      }, // dropped
+      "not an object", // dropped
+    ]);
+    const nodes = parseAndroidAxTree(bad, 0);
+    expect(nodes).toHaveLength(1);
+    expect(nodes[0]?.id).toBe("a0-1");
+  });
+
+  it("throws when the top-level payload is not JSON or not an array", () => {
+    expect(() => parseAndroidAxTree("nope", 0)).toThrow(/parse failed/);
+    expect(() => parseAndroidAxTree(JSON.stringify({ a: 1 }), 0)).toThrow(
+      /must be an array/,
+    );
+  });
+});
+
+describe("androidAxIdToSceneId + sceneAxToAndroidAxNode round-trip", () => {
+  it("round-trips through normalize + inverse", () => {
+    const original = JSON.parse(KOTLIN_AX_JSON)[1];
+    const normalized = normalizeAndroidAxNode(original, 0)!;
+    const inverted = sceneAxToAndroidAxNode(normalized);
+    expect(inverted.id).toBe(original.id);
+    expect(inverted.role).toBe(original.role);
+    expect(inverted.label).toBe(original.label);
+    expect(inverted.bbox).toEqual(original.bbox);
+    expect(inverted.actions).toEqual(original.actions);
+  });
+
+  it("scene-id prefix is `a<displayId>-`", () => {
+    expect(androidAxIdToSceneId("42", 0)).toBe("a0-42");
+    expect(androidAxIdToSceneId("42", 3)).toBe("a3-42");
+  });
+});
+
+describe("Android AX nodes feed WS7 resolveReference", () => {
+  it("Brain `ref:a0-1` from a parsed Android tree resolves to the Save button bbox center", () => {
+    const ax = parseAndroidAxTree(KOTLIN_AX_JSON, ANDROID_LOGICAL_DISPLAY_ID);
+    const scene: Scene = {
+      timestamp: 1,
+      displays: [
+        {
+          id: 0,
+          bounds: [0, 0, 1080, 1920],
+          scaleFactor: 1,
+          primary: true,
+          name: "android-screen",
+        },
+      ],
+      focused_window: null,
+      apps: [],
+      ocr: [],
+      ax,
+      vlm_scene: null,
+      vlm_elements: null,
+    };
+    const target = resolveReference(scene, "a0-1", "save", 0);
+    expect(target).not.toBeNull();
+    expect(target?.kind).toBe("ax");
+    expect(target?.bbox).toEqual([100, 200, 80, 40]);
+    expect(target?.displayId).toBe(0);
+  });
+
+  it("hint-based label match works without an explicit ref", () => {
+    const ax = parseAndroidAxTree(KOTLIN_AX_JSON, ANDROID_LOGICAL_DISPLAY_ID);
+    const scene: Scene = {
+      timestamp: 1,
+      displays: [
+        {
+          id: 0,
+          bounds: [0, 0, 1080, 1920],
+          scaleFactor: 1,
+          primary: true,
+          name: "android-screen",
+        },
+      ],
+      focused_window: null,
+      apps: [],
+      ocr: [],
+      ax,
+      vlm_scene: null,
+      vlm_elements: null,
+    };
+    const target = resolveReference(scene, undefined, "Title", 0);
+    expect(target?.label).toBe("Title");
+  });
+});

plugins/plugin-computeruse/src/__tests__/android-trajectory.test.ts

@@ -0,0 +1,138 @@
+/**
+ * WS8 — Android trajectory event emission tests.
+ *
+ * The trajectory logger reads `computeruse.agent.step` and
+ * `computeruse.android.action` events from the structured log stream. We
+ * verify both emitters publish the expected shape so the logger contract
+ * stays in lock-step with the desktop emitter in `use-computer-agent.ts`.
+ */
+
+import { logger } from "@elizaos/core";
+import { describe, expect, it, vi } from "vitest";
+import {
+  emitAndroidAction,
+  emitAndroidAgentStep,
+} from "../mobile/android-trajectory.js";
+
+describe("emitAndroidAction", () => {
+  it("emits a structured `computeruse.android.action` log entry with platform=android", () => {
+    const spy = vi.spyOn(logger, "info").mockImplementation(() => logger);
+    try {
+      const payload = emitAndroidAction({
+        kind: "tap",
+        success: true,
+        x: 540,
+        y: 960,
+        ref: "a0-1",
+        rationale: "tap save",
+      });
+      expect(payload.kind).toBe("tap");
+      expect(spy).toHaveBeenCalledTimes(1);
+      const [first] = spy.mock.calls;
+      const obj = first?.[0] as Record<string, unknown>;
+      expect(obj.evt).toBe("computeruse.android.action");
+      expect(obj.platform).toBe("android");
+      expect(obj.kind).toBe("tap");
+      expect(obj.x).toBe(540);
+      expect(obj.ref).toBe("a0-1");
+    } finally {
+      spy.mockRestore();
+    }
+  });
+
+  it("trims error messages over 256 chars", () => {
+    const spy = vi.spyOn(logger, "info").mockImplementation(() => logger);
+    try {
+      const long = "x".repeat(2_000);
+      const payload = emitAndroidAction({
+        kind: "tap",
+        success: false,
+        errorCode: "accessibility_unavailable",
+        errorMessage: long,
+      });
+      expect(payload.errorMessage?.length).toBe(256);
+      expect(spy.mock.calls[0]?.[0]).toMatchObject({
+        evt: "computeruse.android.action",
+        platform: "android",
+        kind: "tap",
+        success: false,
+        errorCode: "accessibility_unavailable",
+      });
+    } finally {
+      spy.mockRestore();
+    }
+  });
+
+  it("does not emit fields that were not supplied", () => {
+    const spy = vi.spyOn(logger, "info").mockImplementation(() => logger);
+    try {
+      emitAndroidAction({ kind: "back", success: true });
+      const obj = spy.mock.calls[0]?.[0] as Record<string, unknown>;
+      expect(obj.x).toBeUndefined();
+      expect(obj.y).toBeUndefined();
+      expect(obj.ref).toBeUndefined();
+    } finally {
+      spy.mockRestore();
+    }
+  });
+});
+
+describe("emitAndroidAgentStep", () => {
+  it("emits a `computeruse.agent.step` entry with platform=android", () => {
+    const spy = vi.spyOn(logger, "info").mockImplementation(() => logger);
+    try {
+      emitAndroidAgentStep({
+        step: 3,
+        goal: "save the document",
+        actionKind: "click",
+        displayId: 0,
+        rois: 1,
+        success: true,
+        rationale: "tap save",
+      });
+      const obj = spy.mock.calls[0]?.[0] as Record<string, unknown>;
+      expect(obj.evt).toBe("computeruse.agent.step");
+      expect(obj.platform).toBe("android");
+      expect(obj.step).toBe(3);
+      expect(obj.actionKind).toBe("click");
+      expect(obj.success).toBe(true);
+    } finally {
+      spy.mockRestore();
+    }
+  });
+
+  it("shape matches the desktop emitter in use-computer-agent.ts (same evt key)", () => {
+    const spy = vi.spyOn(logger, "info").mockImplementation(() => logger);
+    try {
+      emitAndroidAgentStep({
+        step: 1,
+        goal: "g",
+        actionKind: "finish",
+        displayId: 0,
+        rois: 0,
+        success: true,
+        rationale: "done",
+      });
+      const obj = spy.mock.calls[0]?.[0] as Record<string, unknown>;
+      // The desktop emitter publishes: evt, step, goal, actionKind,
+      // displayId, rois, success, error?, rationale. We add platform.
+      const expectedKeys = [
+        "evt",
+        "platform",
+        "step",
+        "goal",
+        "actionKind",
+        "displayId",
+        "rois",
+        "success",
+        "error",
+        "rationale",
+      ];
+      for (const k of expectedKeys) {
+        expect(k in obj).toBe(true);
+      }
+    } finally {
+      spy.mockRestore();
+    }
+  });
+});

plugins/plugin-computeruse/src/__tests__/aosp-input-actor.test.ts

@@ -0,0 +1,248 @@
+/**
+ * WS7 — AospInputActor fake-bridge tests.
+ *
+ * Exercises the privileged-build adapter that maps cascade-resolved
+ * `ProposedAction`s into `injectMotionEvent` calls. We don't have a real
+ * AOSP build on this Linux host, so this is pure-JS — a fake
+ * `AospPrivilegedInputBridge` records the calls and we assert the call
+ * sequence + the `ActionResult` envelope shape (matching `dispatch.ts`).
+ */
+
+import { describe, expect, it } from "vitest";
+import {
+  AospInputActor,
+  type AospPrivilegedInputBridge,
+  MOTION_EVENT_ACTION_DOWN,
+  MOTION_EVENT_ACTION_MOVE,
+  MOTION_EVENT_ACTION_UP,
+} from "../actor/aosp-input-actor.js";
+import type { ProposedAction } from "../actor/types.js";
+
+interface BridgeCall {
+  x: number;
+  y: number;
+  action: number;
+  downTimeMs: number;
+}
+
+function fakeBridge(opts: { failAt?: number } = {}): {
+  bridge: AospPrivilegedInputBridge;
+  calls: BridgeCall[];
+} {
+  const calls: BridgeCall[] = [];
+  let i = 0;
+  const bridge: AospPrivilegedInputBridge = {
+    async injectMotionEvent(args) {
+      calls.push(args);
+      i += 1;
+      if (opts.failAt !== undefined && i === opts.failAt) {
+        return { ok: false };
+      }
+      return { ok: true };
+    },
+  };
+  return { bridge, calls };
+}
+
+describe("AospInputActor — happy paths", () => {
+  it("click → DOWN then UP at the same point", async () => {
+    const { bridge, calls } = fakeBridge();
+    const actor = new AospInputActor({
+      getBridge: () => bridge,
+      now: () => 1000,
+    });
+    const action: ProposedAction = {
+      kind: "click",
+      displayId: 0,
+      x: 100,
+      y: 200,
+      rationale: "tap save",
+    };
+    const result = await actor.execute(action);
+    expect(result.success).toBe(true);
+    expect(calls).toHaveLength(2);
+    expect(calls[0]).toMatchObject({
+      x: 100,
+      y: 200,
+      action: MOTION_EVENT_ACTION_DOWN,
+      downTimeMs: 1000,
+    });
+    expect(calls[1]).toMatchObject({
+      x: 100,
+      y: 200,
+      action: MOTION_EVENT_ACTION_UP,
+    });
+    expect(calls[1]?.downTimeMs).toBeGreaterThan(1000);
+  });
+
+  it("double_click → two DOWN/UP pairs", async () => {
+    const { bridge, calls } = fakeBridge();
+    const actor = new AospInputActor({ getBridge: () => bridge });
+    await actor.execute({
+      kind: "double_click",
+      displayId: 0,
+      x: 50,
+      y: 60,
+      rationale: "",
+    });
+    expect(calls).toHaveLength(4);
+    expect(calls.map((c) => c.action)).toEqual([
+      MOTION_EVENT_ACTION_DOWN,
+      MOTION_EVENT_ACTION_UP,
+      MOTION_EVENT_ACTION_DOWN,
+      MOTION_EVENT_ACTION_UP,
+    ]);
+  });
+
+  it("drag → DOWN/MOVE/UP across path endpoints", async () => {
+    const { bridge, calls } = fakeBridge();
+    const actor = new AospInputActor({
+      getBridge: () => bridge,
+      now: () => 5_000,
+    });
+    const action: ProposedAction = {
+      kind: "drag",
+      displayId: 0,
+      startX: 10,
+      startY: 20,
+      x: 500,
+      y: 800,
+      rationale: "drag",
+    };
+    const result = await actor.execute(action);
+    expect(result.success).toBe(true);
+    expect(calls).toHaveLength(3);
+    expect(calls[0]?.action).toBe(MOTION_EVENT_ACTION_DOWN);
+    expect(calls[1]?.action).toBe(MOTION_EVENT_ACTION_MOVE);
+    expect(calls[2]?.action).toBe(MOTION_EVENT_ACTION_UP);
+    expect(calls[2]?.x).toBe(500);
+    expect(calls[2]?.y).toBe(800);
+  });
+
+  it("scroll → DOWN/MOVE/UP with inverted sign convention", async () => {
+    const { bridge, calls } = fakeBridge();
+    const actor = new AospInputActor({
+      getBridge: () => bridge,
+      now: () => 5_000,
+    });
+    await actor.execute({
+      kind: "scroll",
+      displayId: 0,
+      x: 540,
+      y: 960,
+      dx: 0,
+      dy: 2,
+      rationale: "scroll",
+    });
+    expect(calls).toHaveLength(3);
+    expect(calls[0]?.y).toBe(960);
+    // dy=2 → endY = 960 - 2*200 = 560 (swipe upward)
+    expect(calls[2]?.y).toBe(560);
+  });
+
+  it("wait / finish are no-ops with success:true", async () => {
+    const { bridge, calls } = fakeBridge();
+    const actor = new AospInputActor({ getBridge: () => bridge });
+    const r1 = await actor.execute({
+      kind: "wait",
+      displayId: 0,
+      rationale: "",
+    });
+    const r2 = await actor.execute({
+      kind: "finish",
+      displayId: 0,
+      rationale: "",
+    });
+    expect(r1.success).toBe(true);
+    expect(r2.success).toBe(true);
+    expect(calls).toHaveLength(0);
+  });
+});
+
+describe("AospInputActor — invalid args + driver_error", () => {
+  it("type/key/hotkey return invalid_args (privileged-input doesn't handle text)", async () => {
+    const { bridge } = fakeBridge();
+    const actor = new AospInputActor({ getBridge: () => bridge });
+    for (const kind of ["type", "key", "hotkey"] as const) {
+      const r = await actor.execute({
+        kind,
+        displayId: 0,
+        rationale: "",
+      } as ProposedAction);
+      expect(r.success).toBe(false);
+      expect(r.error?.code).toBe("invalid_args");
+    }
+  });
+
+  it("returns driver_error when the bridge is null (consumer build)", async () => {
+    const actor = new AospInputActor({ getBridge: () => null });
+    const r = await actor.execute({
+      kind: "click",
+      displayId: 0,
+      x: 10,
+      y: 10,
+      rationale: "",
+    });
+    expect(r.success).toBe(false);
+    expect(r.error?.code).toBe("driver_error");
+    expect(r.error?.message).toMatch(/not available/);
+  });
+
+  it("missing click coords → invalid_args", async () => {
+    const { bridge } = fakeBridge();
+    const actor = new AospInputActor({ getBridge: () => bridge });
+    const r = await actor.execute({
+      kind: "click",
+      displayId: 0,
+      rationale: "",
+    });
+    expect(r.error?.code).toBe("invalid_args");
+  });
+
+  it("missing scroll dx/dy → invalid_args", async () => {
+    const { bridge } = fakeBridge();
+    const actor = new AospInputActor({ getBridge: () => bridge });
+    const r = await actor.execute({
+      kind: "scroll",
+      displayId: 0,
+      x: 10,
+      y: 10,
+      rationale: "",
+    });
+    expect(r.error?.code).toBe("invalid_args");
+  });
+
+  it("bridge throwing → driver_error, action message surfaces", async () => {
+    const bridge: AospPrivilegedInputBridge = {
+      injectMotionEvent: async () => {
+        throw new Error("INJECT_EVENTS denied");
+      },
+    };
+    const actor = new AospInputActor({ getBridge: () => bridge });
+    const r = await actor.execute({
+      kind: "click",
+      displayId: 0,
+      x: 10,
+      y: 10,
+      rationale: "",
+    });
+    expect(r.success).toBe(false);
+    expect(r.error?.code).toBe("driver_error");
+    expect(r.error?.message).toContain("INJECT_EVENTS denied");
+  });
+
+  it("bridge returning ok:false partway through tap → driver_error", async () => {
+    const { bridge } = fakeBridge({ failAt: 2 });
+    const actor = new AospInputActor({ getBridge: () => bridge });
+    const r = await actor.execute({
+      kind: "click",
+      displayId: 0,
+      x: 10,
+      y: 10,
+      rationale: "",
+    });
+    expect(r.success).toBe(false);
+    expect(r.error?.code).toBe("driver_error");
+    expect(r.error?.message).toContain("UP");
+  });
+});

plugins/plugin-computeruse/src/__tests__/apps-join.test.ts

@@ -0,0 +1,53 @@
+/**
+ * Unit coverage for joinAppsAndWindows on Windows (#9170 / #9105 scene model).
+ *
+ * SceneBuilder joins the process list and the window list into a per-app
+ * `SceneApp[]`. On Windows the join key is the window id (which IS the owning
+ * pid). The linux path shells out to wmctrl, so this is win32-gated — and it
+ * therefore runs on this Windows box, validating the pure win32 join. Untested
+ * until now.
+ */
+
+import { platform } from "node:os";
+import { describe, expect, it } from "vitest";
+import { joinAppsAndWindows } from "../scene/apps.js";
+import type { WindowInfo } from "../types.js";
+
+const IS_WIN = platform() === "win32";
+
+const procs = [
+  { pid: 100, name: "Notepad" },
+  { pid: 200, name: "Code" },
+  { pid: 300, name: "Idle" },
+];
+const windows: WindowInfo[] = [
+  { id: "100", title: "untitled", app: "Notepad" },
+  { id: "200", title: "project", app: "Code" },
+  { id: "999", title: "Orphan", app: "Ghost" },
+];
+
+describe.skipIf(!IS_WIN)("joinAppsAndWindows (win32)", () => {
+  it("joins each window to its owning process by pid (win.id === pid)", () => {
+    const apps = joinAppsAndWindows(procs, windows, "win32");
+    const byPid = new Map(apps.map((a) => [a.pid, a]));
+    expect(byPid.get(100)?.windows.map((w) => w.id)).toEqual(["100"]);
+    expect(byPid.get(100)?.windows[0].title).toBe("untitled");
+    expect(byPid.get(200)?.windows.map((w) => w.id)).toEqual(["200"]);
+  });
+
+  it("keeps a process that owns no windows", () => {
+    const apps = joinAppsAndWindows(procs, windows, "win32");
+    expect(apps.find((a) => a.pid === 300)?.windows).toEqual([]);
+  });
+
+  it("creates an app bucket for a window whose pid has no process", () => {
+    const apps = joinAppsAndWindows(procs, windows, "win32");
+    const ghost = apps.find((a) => a.pid === 999);
+    expect(ghost?.name).toBe("Ghost");
+    expect(ghost?.windows.map((w) => w.id)).toEqual(["999"]);
+  });
+
+  it("returns exactly one bucket per distinct pid", () => {
+    expect(joinAppsAndWindows(procs, windows, "win32")).toHaveLength(4);
+  });
+});

plugins/plugin-computeruse/src/__tests__/benchmark/osworld-local.real.test.ts

@@ -0,0 +1,411 @@
+/**
+ * Local OSWorld-style benchmark tests.
+ *
+ * These tests exercise the full OSWorld adapter loop (observe → act → observe)
+ * against the local desktop. They verify that:
+ *
+ * 1. The observation pipeline works (screenshots, a11y)
+ * 2. All OSWorld action types execute without error
+ * 3. The step() loop advances correctly
+ * 4. Action conversion round-trips work end-to-end
+ * 5. Trajectory recording captures everything
+ *
+ * Requires: desktop access (screen recording + accessibility permissions on macOS).
+ * Skipped on CI or headless environments.
+ */
+
+import type { IAgentRuntime } from "@elizaos/core";
+import { afterAll, beforeAll, describe, expect, it } from "vitest";
+import { assertScreenshotBase64NotBlank } from "../../../test/helpers/screenshot-quality.ts";
+import { OSWorldAdapter } from "../../osworld/adapter.js";
+import type { OSWorldAction } from "../../osworld/types.js";
+import { captureScreenshot } from "../../platform/screenshot.js";
+import { ComputerUseService } from "../../services/computer-use-service.js";
+
+// ── Environment detection ───────────────────────────────────────────────
+
+let canCapture = false;
+try {
+  await captureScreenshot();
+  canCapture = true;
+} catch {
+  // No screen recording permission
+}
+
+const forceRun = process.env.FORCE_OSWORLD_BENCHMARK === "1";
+const canRun = (canCapture || forceRun) && !process.env.CI;
+const describeIfDesktop = canRun ? describe : describe.skip;
+
+// ── Setup ───────────────────────────────────────────────────────────────
+
+function createMockRuntime(): IAgentRuntime {
+  return {
+    character: {},
+    getSetting: (key: string) => {
+      if (key === "COMPUTER_USE_APPROVAL_MODE") return "full_control";
+      if (key === "COMPUTER_USE_SCREENSHOT_AFTER_ACTION") return "false";
+      return undefined;
+    },
+    getService: () => null,
+  } as IAgentRuntime;
+}
+
+describeIfDesktop("OSWorld local benchmark", () => {
+  let service: ComputerUseService;
+  let adapter: OSWorldAdapter;
+
+  beforeAll(async () => {
+    service = (await ComputerUseService.start(
+      createMockRuntime(),
+    )) as ComputerUseService;
+    adapter = new OSWorldAdapter(service, {
+      actionSpace: "computer_13",
+      observationType: "screenshot",
+      includeA11yTree: false,
+      maxTrajectoryLength: 20,
+      screenshotDelayMs: 200, // Fast for testing
+    });
+  });
+
+  afterAll(async () => {
+    if (service) await service.stop();
+  });
+
+  // ── Observation Pipeline ────────────────────────────────────────────
+
+  describe("observation pipeline", () => {
+    it("captures a screenshot observation", async () => {
+      const obs = await adapter.getObservation("Test task");
+      expect(obs.screenshot).toBeDefined();
+      expect(obs.instruction).toBe("Test task");
+
+      if (canCapture) {
+        assertScreenshotBase64NotBlank(
+          obs.screenshot,
+          "OSWorld observation screenshot",
+        );
+      } else {
+        // Without screen recording permission, screenshot is empty string
+        expect(typeof obs.screenshot).toBe("string");
+      }
+    });
+
+    it("includes a11y tree when configured", async () => {
+      const a11yAdapter = new OSWorldAdapter(service, {
+        actionSpace: "computer_13",
+        observationType: "screenshot_a11y_tree",
+        includeA11yTree: true,
+        maxTrajectoryLength: 5,
+        screenshotDelayMs: 200,
+      });
+
+      const obs = await a11yAdapter.getObservation("Test with a11y");
+      expect(obs.screenshot).toBeDefined();
+      if (canCapture) {
+        assertScreenshotBase64NotBlank(
+          obs.screenshot,
+          "OSWorld a11y observation screenshot",
+        );
+      }
+      // a11y tree may or may not be available depending on permissions
+      expect(
+        obs.accessibility_tree === null ||
+          typeof obs.accessibility_tree === "string",
+      ).toBe(true);
+    });
+  });
+
+  // ── computer_13 Action Execution ────────────────────────────────────
+
+  describe("computer_13 action execution", () => {
+    it("executes CLICK action", async () => {
+      const result = await adapter.executeAction({
+        action_type: "CLICK",
+        x: 200,
+        y: 200,
+      });
+      expect(result.executed).toBe(true);
+      expect(result.done).toBe(false);
+    });
+
+    it("executes MOVE_TO action", async () => {
+      const result = await adapter.executeAction({
+        action_type: "MOVE_TO",
+        x: 300,
+        y: 300,
+      });
+      expect(result.executed).toBe(true);
+    });
+
+    it("executes TYPING action", async () => {
+      const result = await adapter.executeAction({
+        action_type: "TYPING",
+        text: " ", // Safe character
+      });
+      expect(result.executed).toBe(true);
+    });
+
+    it("executes PRESS action", async () => {
+      const result = await adapter.executeAction({
+        action_type: "PRESS",
+        key: "Escape",
+      });
+      expect(result.executed).toBe(true);
+    });
+
+    it("executes HOTKEY action", async () => {
+      const result = await adapter.executeAction({
+        action_type: "HOTKEY",
+        keys: ["shift", "Escape"], // Harmless combo
+      });
+      expect(result.executed).toBe(true);
+    });
+
+    it("executes SCROLL action", async () => {
+      const result = await adapter.executeAction({
+        action_type: "SCROLL",
+        x: 400,
+        y: 400,
+        direction: "down",
+        amount: 2,
+      });
+      expect(result.executed).toBe(true);
+    });
+
+    it("handles WAIT action", async () => {
+      const result = await adapter.executeAction({
+        action_type: "WAIT",
+      });
+      expect(result.executed).toBe(true);
+      expect(result.done).toBe(false);
+    });
+
+    it("handles DONE action", async () => {
+      const result = await adapter.executeAction({
+        action_type: "DONE",
+      });
+      expect(result.executed).toBe(false);
+      expect(result.done).toBe(true);
+      expect(result.failed).toBe(false);
+    });
+
+    it("handles FAIL action", async () => {
+      const result = await adapter.executeAction({
+        action_type: "FAIL",
+      });
+      expect(result.done).toBe(true);
+      expect(result.failed).toBe(true);
+    });
+  });
+
+  // ── pyautogui Action Execution ──────────────────────────────────────
+
+  describe("pyautogui action execution", () => {
+    it("executes pyautogui.click()", async () => {
+      const result = await adapter.executePyAutoGUI(
+        "pyautogui.click(200, 200)",
+      );
+      expect(result.executed).toBe(true);
+    });
+
+    it("executes pyautogui.press()", async () => {
+      const result = await adapter.executePyAutoGUI(
+        "pyautogui.press('escape')",
+      );
+      expect(result.executed).toBe(true);
+    });
+
+    it("handles DONE string", async () => {
+      const result = await adapter.executePyAutoGUI("DONE");
+      expect(result.done).toBe(true);
+    });
+
+    it("handles WAIT string", async () => {
+      const result = await adapter.executePyAutoGUI("WAIT");
+      expect(result.executed).toBe(true);
+      expect(result.done).toBe(false);
+    });
+  });
+
+  // ── Gymnasium-style Step Loop ───────────────────────────────────────
+
+  describe("step loop", () => {
+    it("runs a multi-step trajectory", async () => {
+      adapter.reset();
+
+      const instruction = "Move the mouse and press escape";
+
+      // Step 1: Move mouse
+      const step1 = await adapter.step(
+        { action_type: "MOVE_TO", x: 100, y: 100 } as OSWorldAction,
+        instruction,
+      );
+      expect(step1.done).toBe(false);
+      expect(typeof step1.observation.screenshot).toBe("string");
+      expect(step1.observation.instruction).toBe(instruction);
+      expect(adapter.getStepCount()).toBe(1);
+
+      // Step 2: Press escape
+      const step2 = await adapter.step(
+        { action_type: "PRESS", key: "Escape" } as OSWorldAction,
+        instruction,
+      );
+      expect(step2.done).toBe(false);
+      expect(adapter.getStepCount()).toBe(2);
+
+      // Step 3: Done
+      const step3 = await adapter.step(
+        { action_type: "DONE" } as OSWorldAction,
+        instruction,
+      );
+      expect(step3.done).toBe(true);
+      expect(adapter.getStepCount()).toBe(3);
+
+      // Verify trajectory
+      const trajectory = adapter.getTrajectory();
+      expect(trajectory.length).toBe(3);
+      expect(trajectory[0].action).toEqual({
+        action_type: "MOVE_TO",
+        x: 100,
+        y: 100,
+      });
+      expect(trajectory[2].action).toEqual({ action_type: "DONE" });
+    });
+
+    it("enforces max trajectory length", async () => {
+      const shortAdapter = new OSWorldAdapter(service, {
+        actionSpace: "computer_13",
+        observationType: "screenshot",
+        maxTrajectoryLength: 3,
+        includeA11yTree: false,
+        screenshotDelayMs: 50,
+      });
+
+      shortAdapter.reset();
+
+      // Step until limit
+      for (let i = 0; i < 3; i++) {
+        const result = await shortAdapter.step(
+          { action_type: "WAIT" } as OSWorldAction,
+          "Test",
+        );
+        if (i < 2) {
+          expect(result.done).toBe(false);
+        } else {
+          // On the 3rd step, trajectory limit reached
+          expect(result.done).toBe(true);
+          expect(result.info.trajectoryExceeded).toBe(true);
+        }
+      }
+    });
+
+    it("runs with pyautogui strings", async () => {
+      adapter.reset();
+
+      const step1 = await adapter.step(
+        "pyautogui.press('escape')",
+        "Press escape",
+      );
+      expect(step1.done).toBe(false);
+      expect(typeof step1.observation.screenshot).toBe("string");
+
+      const step2 = await adapter.step("DONE", "Press escape");
+      expect(step2.done).toBe(true);
+    });
+
+    it("reset clears trajectory and step count", () => {
+      adapter.reset();
+      expect(adapter.getStepCount()).toBe(0);
+      expect(adapter.getTrajectory().length).toBe(0);
+    });
+  });
+
+  // ── Metrics Report ──────────────────────────────────────────────────
+
+  describe("benchmark metrics", () => {
+    it("reports capabilities and timing", async () => {
+      const startTime = Date.now();
+
+      // Capture observation
+      const obs = await adapter.getObservation("Benchmark test");
+      const obsTime = Date.now() - startTime;
+
+      // Execute a set of actions and measure timing
+      const actions: OSWorldAction[] = [
+        { action_type: "MOVE_TO", x: 100, y: 100 },
+        { action_type: "CLICK", x: 200, y: 200 },
+        { action_type: "PRESS", key: "Escape" },
+        { action_type: "SCROLL", x: 300, y: 300, direction: "down", amount: 2 },
+      ];
+
+      const timings: number[] = [];
+      for (const action of actions) {
+        const t0 = Date.now();
+        await adapter.executeAction(action);
+        timings.push(Date.now() - t0);
+      }
+
+      const totalTime = Date.now() - startTime;
+
+      // Report metrics
+      const caps = service.getCapabilities();
+      const screen = service.getScreenDimensions();
+
+      console.log("\n╔══════════════════════════════════════════════╗");
+      console.log("║     OSWorld Local Benchmark Results          ║");
+      console.log("╠══════════════════════════════════════════════╣");
+      console.log(`║ Platform:        ${process.platform.padEnd(27)}║`);
+      console.log(
+        `║ Screen:          ${`${screen.width}x${screen.height}`.padEnd(27)}║`,
+      );
+      console.log(
+        `║ Screenshot tool: ${caps.screenshot.tool.slice(0, 27).padEnd(27)}║`,
+      );
+      console.log(
+        `║ Mouse/KB tool:   ${caps.computerUse.tool.slice(0, 27).padEnd(27)}║`,
+      );
+      console.log(
+        `║ Browser:         ${caps.browser.tool.slice(0, 27).padEnd(27)}║`,
+      );
+      console.log(
+        `║ A11y available:  ${String(adapter.isA11yAvailable()).padEnd(27)}║`,
+      );
+      console.log("╠──────────────────────────────────────────────╣");
+      console.log(
+        `║ Observation time:  ${String(obsTime).padStart(5)}ms                  ║`,
+      );
+      for (let i = 0; i < actions.length; i++) {
+        const label = actions[i].action_type.padEnd(15);
+        console.log(
+          `║ ${label}    ${String(timings[i]).padStart(5)}ms                  ║`,
+        );
+      }
+      console.log(
+        `║ Total time:        ${String(totalTime).padStart(5)}ms                  ║`,
+      );
+      console.log(
+        `║ Avg action time:   ${String(Math.round(timings.reduce((a, b) => a + b, 0) / timings.length)).padStart(5)}ms                  ║`,
+      );
+      console.log("╠──────────────────────────────────────────────╣");
+      const ssSize =
+        obs.screenshot.length > 0
+          ? `${Math.round(obs.screenshot.length / 1024)}KB base64`
+          : "N/A (no permission)";
+      console.log(`║ Screenshot size:   ${ssSize.padEnd(27)}║`);
+      console.log(`║ Actions tested:    ${String(actions.length).padEnd(27)}║`);
+      console.log(`║ All actions OK:    ${"✓ PASS".padEnd(27)}║`);
+      console.log("╚══════════════════════════════════════════════╝\n");
+
+      // Assertions for the report
+      expect(typeof obs.screenshot).toBe("string");
+      if (canCapture) {
+        assertScreenshotBase64NotBlank(
+          obs.screenshot,
+          "OSWorld benchmark metrics screenshot",
+        );
+      }
+      expect(timings.every((t) => t < 10000)).toBe(true); // Each action under 10s
+      expect(totalTime).toBeLessThan(60000); // Total under 60s
+    });
+  });
+});

plugins/plugin-computeruse/src/__tests__/benchmark/osworld-tasks.real.test.ts

@@ -0,0 +1,452 @@
+/**
+ * OSWorld task subset benchmark.
+ *
+ * Self-contained benchmark that exercises all computer-use capabilities
+ * using the OSWorld action format (computer_13 structured actions).
+ *
+ * Runs 15 tasks across 5 domains:
+ *   - file: file create/read/edit/list
+ *   - term: terminal execute/session/blocking
+ *   - win:  window listing
+ *   - desk: desktop mouse/keyboard (OSWorld actions)
+ *   - sec:  approval mode enforcement
+ *
+ * Run: FORCE_OSWORLD_BENCHMARK=1 bun run test:live
+ */
+
+import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import type { IAgentRuntime } from "@elizaos/core";
+import { afterAll, beforeAll, describe, expect, it } from "vitest";
+import { ComputerUseService } from "../../services/computer-use-service.js";
+
+// ── Environment detection ───────────────────────────────────────────────
+
+const forceRun = process.env.FORCE_OSWORLD_BENCHMARK === "1";
+const describeIfBenchmark = forceRun ? describe : describe.skip;
+
+// ── Helpers ─────────────────────────────────────────────────────────────
+
+interface TaskResult {
+  name: string;
+  domain: string;
+  steps: number;
+  passed: boolean;
+  timeMs: number;
+}
+
+function mockRuntime(overrides: Record<string, string> = {}): IAgentRuntime {
+  return {
+    character: {},
+    getSetting: (key: string) => {
+      if (key === "COMPUTER_USE_APPROVAL_MODE")
+        return overrides.COMPUTER_USE_APPROVAL_MODE ?? "full_control";
+      if (key === "COMPUTER_USE_SCREENSHOT_AFTER_ACTION") return "false";
+      return overrides[key] ?? undefined;
+    },
+    getService: () => null,
+  } as IAgentRuntime;
+}
+
+// ── Test Suite ──────────────────────────────────────────────────────────
+
+describeIfBenchmark("OSWorld task subset benchmark", () => {
+  let service: ComputerUseService;
+  let tempDir: string;
+  const results: TaskResult[] = [];
+
+  beforeAll(async () => {
+    service = (await ComputerUseService.start(
+      mockRuntime(),
+    )) as ComputerUseService;
+    tempDir = mkdtempSync(join(tmpdir(), "osworld-benchmark-"));
+  });
+
+  afterAll(async () => {
+    if (service) await service.stop();
+    if (tempDir) rmSync(tempDir, { recursive: true, force: true });
+
+    // Print results table
+    console.log(
+      "\n╔══════════════════════════════════════════════════════════════════╗",
+    );
+    console.log(
+      "║                OSWorld Task Benchmark Results                   ║",
+    );
+    console.log(
+      "╠════════════════════════════╦═══════╦═══════╦═════════╦══════════╣",
+    );
+    console.log(
+      "║ Task                       ║ Domn  ║ Steps ║ Time    ║ Result   ║",
+    );
+    console.log(
+      "╠════════════════════════════╬═══════╬═══════╬═════════╬══════════╣",
+    );
+    for (const r of results) {
+      const name = r.name.slice(0, 26).padEnd(26);
+      const domain = r.domain.slice(0, 5).padEnd(5);
+      const steps = String(r.steps).padStart(3);
+      const time = `${r.timeMs}ms`.padStart(7);
+      const status = r.passed ? "  PASS  " : "  FAIL  ";
+      console.log(`║ ${name} ║ ${domain} ║  ${steps}  ║ ${time} ║ ${status} ║`);
+    }
+    console.log(
+      "╠════════════════════════════╩═══════╩═══════╩═════════╩══════════╣",
+    );
+    const passed = results.filter((r) => r.passed).length;
+    const total = results.length;
+    const rate = total > 0 ? Math.round((passed / total) * 100) : 0;
+    const avgMs =
+      total > 0
+        ? Math.round(results.reduce((s, r) => s + r.timeMs, 0) / total)
+        : 0;
+    console.log(
+      `${`║ Passed: ${passed}/${total} (${rate}%)  |  Avg: ${avgMs}ms/task`.padEnd(
+        66,
+      )}║`,
+    );
+
+    // Domain breakdown
+    const domains = [...new Set(results.map((r) => r.domain))];
+    for (const d of domains) {
+      const dTasks = results.filter((r) => r.domain === d);
+      const dPassed = dTasks.filter((r) => r.passed).length;
+      const dRate = Math.round((dPassed / dTasks.length) * 100);
+      console.log(
+        `${`║   ${d.padEnd(6)}: ${dPassed}/${dTasks.length} (${dRate}%)`.padEnd(
+          66,
+        )}║`,
+      );
+    }
+    console.log(`╚${"═".repeat(66)}╝\n`);
+  });
+
+  // ── FILE DOMAIN ─────────────────────────────────────────────────────
+
+  it("T01: File create & read", async () => {
+    const t0 = Date.now();
+    const fp = join(tempDir, "hello.txt");
+    const content = "Hello from OSWorld!";
+
+    const w = await service.executeFileAction({
+      action: "write",
+      path: fp,
+      content,
+    });
+    const r = await service.executeFileAction({ action: "read", path: fp });
+
+    const passed = w.success && r.success && r.content === content;
+    results.push({
+      name: "File Create & Read",
+      domain: "file",
+      steps: 2,
+      passed,
+      timeMs: Date.now() - t0,
+    });
+    expect(passed).toBe(true);
+  });
+
+  it("T02: File edit (find & replace)", async () => {
+    const t0 = Date.now();
+    const fp = join(tempDir, "edit.txt");
+    writeFileSync(fp, "The quick brown fox jumps over the lazy dog.");
+
+    const e = await service.executeFileAction({
+      action: "edit",
+      path: fp,
+      old_text: "brown fox",
+      new_text: "red fox",
+    });
+    const verify = readFileSync(fp, "utf-8");
+
+    const passed =
+      e.success && verify === "The quick red fox jumps over the lazy dog.";
+    results.push({
+      name: "File Edit",
+      domain: "file",
+      steps: 2,
+      passed,
+      timeMs: Date.now() - t0,
+    });
+    expect(passed).toBe(true);
+  });
+
+  it("T03: File append", async () => {
+    const t0 = Date.now();
+    const fp = join(tempDir, "append.txt");
+    writeFileSync(fp, "line1\n");
+
+    await service.executeFileAction({
+      action: "append",
+      path: fp,
+      content: "line2\n",
+    });
+    const verify = readFileSync(fp, "utf-8");
+
+    const passed = verify === "line1\nline2\n";
+    results.push({
+      name: "File Append",
+      domain: "file",
+      steps: 2,
+      passed,
+      timeMs: Date.now() - t0,
+    });
+    expect(passed).toBe(true);
+  });
+
+  it("T04: Directory listing", async () => {
+    const t0 = Date.now();
+    writeFileSync(join(tempDir, "x.txt"), "x");
+
+    const r = await service.executeFileAction({
+      action: "list",
+      path: tempDir,
+    });
+    const names = (r.items ?? []).map((i) => i.name);
+
+    const passed = r.success && names.includes("x.txt");
+    results.push({
+      name: "Directory Listing",
+      domain: "file",
+      steps: 1,
+      passed,
+      timeMs: Date.now() - t0,
+    });
+    expect(passed).toBe(true);
+  });
+
+  it("T05: File exists check", async () => {
+    const t0 = Date.now();
+    const fp = join(tempDir, "exists-check.txt");
+    writeFileSync(fp, "hi");
+
+    const yes = await service.executeFileAction({ action: "exists", path: fp });
+    const no = await service.executeFileAction({
+      action: "exists",
+      path: join(tempDir, "nope.txt"),
+    });
+
+    const passed = yes.exists === true && no.exists === false;
+    results.push({
+      name: "File Exists Check",
+      domain: "file",
+      steps: 2,
+      passed,
+      timeMs: Date.now() - t0,
+    });
+    expect(passed).toBe(true);
+  });
+
+  // ── TERMINAL DOMAIN ─────────────────────────────────────────────────
+
+  it("T06: Terminal echo", async () => {
+    const t0 = Date.now();
+    const r = await service.executeTerminalAction({
+      action: "execute",
+      command: "echo benchmark-ok",
+    });
+
+    const passed = r.success && (r.output ?? "").includes("benchmark-ok");
+    results.push({
+      name: "Terminal Echo",
+      domain: "term",
+      steps: 1,
+      passed,
+      timeMs: Date.now() - t0,
+    });
+    expect(passed).toBe(true);
+  });
+
+  it("T07: Terminal pwd in /tmp", async () => {
+    const t0 = Date.now();
+    const r = await service.executeTerminalAction({
+      action: "execute",
+      command: "pwd",
+      cwd: "/tmp",
+    });
+
+    const passed = r.success && (r.output ?? "").includes("/tmp");
+    results.push({
+      name: "Terminal PWD",
+      domain: "term",
+      steps: 1,
+      passed,
+      timeMs: Date.now() - t0,
+    });
+    expect(passed).toBe(true);
+  });
+
+  it("T08: Dangerous command blocked", async () => {
+    const t0 = Date.now();
+    const r = await service.executeTerminalAction({
+      action: "execute",
+      command: "rm -rf /",
+    });
+
+    const passed = !r.success;
+    results.push({
+      name: "Dangerous Cmd Block",
+      domain: "term",
+      steps: 1,
+      passed,
+      timeMs: Date.now() - t0,
+    });
+    expect(passed).toBe(true);
+  });
+
+  it("T09: Terminal session lifecycle", async () => {
+    const t0 = Date.now();
+    const c = await service.executeTerminalAction({
+      action: "connect",
+      cwd: tempDir,
+    });
+    const e = await service.executeTerminalAction({
+      action: "execute",
+      command: "ls",
+      sessionId: c.sessionId,
+    });
+    const x = await service.executeTerminalAction({
+      action: "close",
+      sessionId: c.sessionId,
+    });
+
+    const passed = c.success && e.success && x.success;
+    results.push({
+      name: "Terminal Session",
+      domain: "term",
+      steps: 3,
+      passed,
+      timeMs: Date.now() - t0,
+    });
+    expect(passed).toBe(true);
+  });
+
+  // ── WINDOW DOMAIN ──────────────────────────────────────────────────
+
+  it("T10: Window listing", async () => {
+    const t0 = Date.now();
+    const r = await service.executeWindowAction({ action: "list" });
+
+    const passed = r.success && Array.isArray(r.windows);
+    results.push({
+      name: "Window List",
+      domain: "win",
+      steps: 1,
+      passed,
+      timeMs: Date.now() - t0,
+    });
+    expect(passed).toBe(true);
+  });
+
+  // ── DESKTOP DOMAIN (OSWorld actions) ────────────────────────────────
+
+  it("T11: Desktop mouse move", async () => {
+    const t0 = Date.now();
+    const r = await service.executeDesktopAction({
+      action: "mouse_move",
+      coordinate: [200, 200],
+    });
+
+    // Mouse move may fail if Accessibility permission is not granted
+    if (!r.success) {
+      console.log(`[T11 mouse_move error]: ${r.error}`);
+    }
+    // Mouse move without cliclick uses AppleScript or Python Quartz — may fail without accessibility permission.
+    // Record truthfully but don't fail the test (permission issues are environment-specific).
+    results.push({
+      name: r.success ? "Desktop Mouse Move" : "Desktop Mouse Move (⚠ perm)",
+      domain: "desk",
+      steps: 1,
+      passed: r.success,
+      timeMs: Date.now() - t0,
+    });
+    // Don't assert — this is environment-dependent
+    expect(typeof r.success).toBe("boolean");
+  });
+
+  it("T12: Desktop key press", async () => {
+    const t0 = Date.now();
+    const r = await service.executeDesktopAction({
+      action: "key",
+      key: "Escape",
+    });
+
+    const passed = r.success;
+    results.push({
+      name: "Desktop Key Press",
+      domain: "desk",
+      steps: 1,
+      passed,
+      timeMs: Date.now() - t0,
+    });
+    expect(passed).toBe(true);
+  });
+
+  it("T13: Desktop key combo", async () => {
+    const t0 = Date.now();
+    const r = await service.executeDesktopAction({
+      action: "key_combo",
+      key: "shift+Escape",
+    });
+
+    const passed = r.success;
+    results.push({
+      name: "Desktop Key Combo",
+      domain: "desk",
+      steps: 1,
+      passed,
+      timeMs: Date.now() - t0,
+    });
+    expect(passed).toBe(true);
+  });
+
+  // ── CROSS-DOMAIN ──────────────────────────────────────────────────
+
+  it("T14: Terminal→File round-trip", async () => {
+    const t0 = Date.now();
+    const fp = join(tempDir, "from-terminal.txt");
+
+    await service.executeTerminalAction({
+      action: "execute",
+      command: `echo "written by terminal" > "${fp}"`,
+    });
+    const r = await service.executeFileAction({ action: "read", path: fp });
+
+    const passed =
+      r.success && (r.content ?? "").includes("written by terminal");
+    results.push({
+      name: "Terminal→File Trip",
+      domain: "multi",
+      steps: 2,
+      passed,
+      timeMs: Date.now() - t0,
+    });
+    expect(passed).toBe(true);
+  });
+
+  // ── SECURITY DOMAIN ────────────────────────────────────────────────
+
+  it("T15: Approval mode off blocks actions", async () => {
+    const t0 = Date.now();
+    const offService = (await ComputerUseService.start(
+      mockRuntime({ COMPUTER_USE_APPROVAL_MODE: "off" }),
+    )) as ComputerUseService;
+
+    const r = await offService.executeDesktopAction({
+      action: "click",
+      coordinate: [100, 100],
+    });
+    await offService.stop();
+
+    const passed = !r.success && (r.error ?? "").includes("blocked");
+    results.push({
+      name: "Approval Off Block",
+      domain: "sec",
+      steps: 1,
+      passed,
+      timeMs: Date.now() - t0,
+    });
+    expect(passed).toBe(true);
+  });
+});

plugins/plugin-computeruse/src/__tests__/brain-dhash-cache.test.ts

@@ -0,0 +1,322 @@
+/**
+ * Brain frame-dHash describe-cache (#9105 M3).
+ *
+ * The WS2 MemoryArbiter only dedups IMAGE_DESCRIPTION for local backends; the
+ * remote path re-burns tokens on an identical screen every step. The Brain's
+ * call-site dHash cache skips the model entirely for the same frame + goal.
+ * Uses real synthesized PNGs so frameDhash() is meaningful.
+ */
+
+import { deflateSync } from "node:zlib";
+import { describe, expect, it } from "vitest";
+import { BRAIN_DHASH_HAMMING_THRESHOLD, Brain } from "../actor/brain.js";
+import type { DisplayCapture } from "../platform/capture.js";
+import { frameDhash, hamming } from "../scene/dhash.js";
+import type { Scene } from "../scene/scene-types.js";
+
+// ── minimal PNG synthesizer (16x16 RGB) ──────────────────────────────────────
+function crc32(buf: Buffer): number {
+  let crc = 0xffffffff;
+  for (let i = 0; i < buf.length; i += 1) {
+    crc ^= buf[i];
+    for (let k = 0; k < 8; k += 1) {
+      crc = crc & 1 ? (crc >>> 1) ^ 0xedb88320 : crc >>> 1;
+    }
+  }
+  return (crc ^ 0xffffffff) >>> 0;
+}
+function pngChunk(type: string, data: Buffer): Buffer {
+  const len = Buffer.alloc(4);
+  len.writeUInt32BE(data.length);
+  const t = Buffer.from(type, "ascii");
+  const crc = Buffer.alloc(4);
+  crc.writeUInt32BE(crc32(Buffer.concat([t, data])));
+  return Buffer.concat([len, t, data, crc]);
+}
+function makeTinyPng(seed = 0): Buffer {
+  const w = 16;
+  const h = 16;
+  const rows: number[] = [];
+  for (let y = 0; y < h; y += 1) {
+    rows.push(0);
+    for (let x = 0; x < w; x += 1) {
+      const v = ((x + seed) * 16) % 255;
+      rows.push(v, v, v);
+    }
+  }
+  const idat = deflateSync(Buffer.from(rows));
+  const ihdr = Buffer.alloc(13);
+  ihdr.writeUInt32BE(w, 0);
+  ihdr.writeUInt32BE(h, 4);
+  ihdr[8] = 8;
+  ihdr[9] = 2;
+  const sig = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
+  return Buffer.concat([
+    sig,
+    pngChunk("IHDR", ihdr),
+    pngChunk("IDAT", idat),
+    pngChunk("IEND", Buffer.alloc(0)),
+  ]);
+}
+
+/** 32×32 2-D pattern PNG with an optional rectangular perturbation, so a small
+ * localized change flips only a few dHash bits (unlike the flat 16×16 gradient,
+ * where a change flips a whole bit-column across every identical row). */
+function make2dPng(
+  seed: number,
+  rect?: { x: number; y: number; w: number; h: number; delta: number },
+): Buffer {
+  const w = 32;
+  const h = 32;
+  const rows: number[] = [];
+  for (let y = 0; y < h; y += 1) {
+    rows.push(0);
+    for (let x = 0; x < w; x += 1) {
+      let v = (x * 37 + y * 101 + seed * 53) % 256;
+      if (
+        rect &&
+        x >= rect.x &&
+        x < rect.x + rect.w &&
+        y >= rect.y &&
+        y < rect.y + rect.h
+      ) {
+        v = (v + rect.delta) % 256;
+      }
+      rows.push(v, v, v);
+    }
+  }
+  const idat = deflateSync(Buffer.from(rows));
+  const ihdr = Buffer.alloc(13);
+  ihdr.writeUInt32BE(w, 0);
+  ihdr.writeUInt32BE(h, 4);
+  ihdr[8] = 8;
+  ihdr[9] = 2;
+  const sig = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
+  return Buffer.concat([
+    sig,
+    pngChunk("IHDR", ihdr),
+    pngChunk("IDAT", idat),
+    pngChunk("IEND", Buffer.alloc(0)),
+  ]);
+}
+
+function dummyScene(overrides: Partial<Scene> = {}): Scene {
+  return {
+    timestamp: 1,
+    displays: [
+      {
+        id: 0,
+        bounds: [0, 0, 16, 16],
+        scaleFactor: 1,
+        primary: true,
+        name: "f",
+      },
+    ],
+    focused_window: null,
+    apps: [],
+    ocr: [],
+    ax: [],
+    vlm_scene: null,
+    vlm_elements: null,
+    ...overrides,
+  };
+}
+function captures(frame: Buffer): Map<number, DisplayCapture> {
+  const m = new Map<number, DisplayCapture>();
+  m.set(0, {
+    display: {
+      id: 0,
+      bounds: [0, 0, 16, 16],
+      scaleFactor: 1,
+      primary: true,
+      name: "f",
+    },
+    frame,
+  });
+  return m;
+}
+
+const VALID = JSON.stringify({
+  scene_summary: "OK",
+  target_display_id: 0,
+  roi: [],
+  proposed_action: { kind: "click", ref: "t0-1", rationale: "Save" },
+});
+
+describe("Brain frame-dHash describe cache (M3)", () => {
+  it("serves an identical frame+goal from cache without re-invoking the model", async () => {
+    let calls = 0;
+    const brain = new Brain(null, {
+      // Pin "always" so this dHash-cache test isolates frame dedup from the
+      // imageless image-policy (#9105), exercised separately.
+      imagePolicy: "always",
+      invokeModel: async () => {
+        calls += 1;
+        return VALID;
+      },
+    });
+    const frame = makeTinyPng(7);
+    const first = await brain.observeAndPlan({
+      scene: dummyScene(),
+      goal: "click save",
+      captures: captures(frame),
+    });
+    const second = await brain.observeAndPlan({
+      scene: dummyScene(),
+      goal: "click save",
+      captures: captures(makeTinyPng(7)), // identical pixels → identical dHash
+    });
+    expect(calls).toBe(1);
+    expect(second).toEqual(first);
+    expect(brain.getStats()).toEqual({
+      invocations: 1,
+      cacheHits: 1,
+      imagelessCalls: 0,
+      estImageTokensSaved: 0,
+    });
+  });
+
+  it("serves a NEAR-identical frame (cosmetic churn) from cache (#9581 tuning)", async () => {
+    // Two frames differing by a small rect → a few dHash bits, the way a cursor
+    // blink / caret / anti-aliasing perturbs a real screen. Self-validate the
+    // fixture is genuinely near-identical-but-not-equal before asserting the hit.
+    const base = make2dPng(5);
+    const nudged = make2dPng(5, { x: 24, y: 24, w: 8, h: 8, delta: 128 });
+    const dhA = frameDhash(base);
+    const dhB = frameDhash(nudged);
+    expect(dhA).not.toBeNull();
+    expect(dhB).not.toBeNull();
+    const dist = hamming(dhA as bigint, dhB as bigint);
+    expect(dist).toBeGreaterThan(0); // not byte-identical (exact cache would still hit)
+    expect(dist).toBeLessThan(BRAIN_DHASH_HAMMING_THRESHOLD);
+
+    let calls = 0;
+    const brain = new Brain(null, {
+      imagePolicy: "always",
+      invokeModel: async () => {
+        calls += 1;
+        return VALID;
+      },
+    });
+    const first = await brain.observeAndPlan({
+      scene: dummyScene(),
+      goal: "click save",
+      captures: captures(base),
+    });
+    const second = await brain.observeAndPlan({
+      scene: dummyScene({ timestamp: 2 }),
+      goal: "click save",
+      captures: captures(nudged), // near-identical → reuse the plan, skip the model
+    });
+    expect(calls).toBe(1);
+    expect(second).toEqual(first);
+    expect(brain.getStats().cacheHits).toBe(1);
+  });
+
+  it("re-invokes for a semantic scene change even when the frame remains near-identical", async () => {
+    const base = make2dPng(5);
+    const nudged = make2dPng(5, { x: 24, y: 24, w: 8, h: 8, delta: 128 });
+    const dhA = frameDhash(base);
+    const dhB = frameDhash(nudged);
+    expect(dhA).not.toBeNull();
+    expect(dhB).not.toBeNull();
+    expect(hamming(dhA as bigint, dhB as bigint)).toBeLessThan(
+      BRAIN_DHASH_HAMMING_THRESHOLD,
+    );
+
+    let calls = 0;
+    const brain = new Brain(null, {
+      imagePolicy: "always",
+      invokeModel: async () => {
+        calls += 1;
+        return VALID;
+      },
+    });
+    const saveScene = dummyScene({
+      ocr: [
+        {
+          id: "t0-1",
+          text: "Save",
+          bbox: [1, 1, 8, 3],
+          conf: 0.99,
+          displayId: 0,
+        },
+      ],
+    });
+    const deleteScene = dummyScene({
+      timestamp: 2,
+      ocr: [
+        {
+          id: "t0-1",
+          text: "Delete",
+          bbox: [1, 1, 8, 3],
+          conf: 0.99,
+          displayId: 0,
+        },
+      ],
+    });
+
+    await brain.observeAndPlan({
+      scene: saveScene,
+      goal: "click save",
+      captures: captures(base),
+    });
+    await brain.observeAndPlan({
+      scene: deleteScene,
+      goal: "click save",
+      captures: captures(nudged),
+    });
+    expect(calls).toBe(2);
+    expect(brain.getStats().cacheHits).toBe(0);
+  });
+
+  it("re-invokes for a visually different frame", async () => {
+    let calls = 0;
+    const brain = new Brain(null, {
+      // Pin "always" so this dHash-cache test isolates frame dedup from the
+      // imageless image-policy (#9105), exercised separately.
+      imagePolicy: "always",
+      invokeModel: async () => {
+        calls += 1;
+        return VALID;
+      },
+    });
+    await brain.observeAndPlan({
+      scene: dummyScene(),
+      goal: "g",
+      captures: captures(makeTinyPng(0)),
+    });
+    await brain.observeAndPlan({
+      scene: dummyScene(),
+      goal: "g",
+      captures: captures(makeTinyPng(60)),
+    });
+    expect(calls).toBe(2);
+    expect(brain.getStats().cacheHits).toBe(0);
+  });
+
+  it("re-invokes when the goal changes even for the same frame", async () => {
+    let calls = 0;
+    const brain = new Brain(null, {
+      // Pin "always" so this dHash-cache test isolates frame dedup from the
+      // imageless image-policy (#9105), exercised separately.
+      imagePolicy: "always",
+      invokeModel: async () => {
+        calls += 1;
+        return VALID;
+      },
+    });
+    const frame = makeTinyPng(3);
+    await brain.observeAndPlan({
+      scene: dummyScene(),
+      goal: "goal A",
+      captures: captures(frame),
+    });
+    await brain.observeAndPlan({
+      scene: dummyScene(),
+      goal: "goal B",
+      captures: captures(makeTinyPng(3)),
+    });
+    expect(calls).toBe(2);
+  });
+});

plugins/plugin-computeruse/src/__tests__/brain.test.ts

@@ -0,0 +1,337 @@
+/**
+ * WS7 — Brain unit tests.
+ *
+ * Validates:
+ *   - `parseBrainOutput` accepts raw JSON, fenced JSON, and prose-then-JSON
+ *     forms, while rejecting structurally invalid bodies.
+ *   - `Brain.observeAndPlan` calls the injected model once when the first
+ *     payload parses, and retries exactly once on a parse failure.
+ *   - The retry uses the *strict* prompt variant; on a second failure a
+ *     `BrainParseError` surfaces so the cascade can return a structured
+ *     error result instead of crashing.
+ *   - ROI extraction is preserved through enforcement of the cap.
+ *   - The model receives a `data:image/png;base64,...` URL (no resizing
+ *     happens client-side — adapters do that downstream).
+ */
+
+import { describe, expect, it } from "vitest";
+import {
+  BRAIN_MAX_ROIS,
+  Brain,
+  BrainParseError,
+  brainPromptFor,
+  parseBrainOutput,
+} from "../actor/brain.js";
+import type { DisplayCapture } from "../platform/capture.js";
+import type { Scene } from "../scene/scene-types.js";
+
+function dummyScene(): Scene {
+  return {
+    timestamp: 1,
+    displays: [
+      {
+        id: 0,
+        bounds: [0, 0, 1920, 1080],
+        scaleFactor: 1,
+        primary: true,
+        name: "fake",
+      },
+    ],
+    focused_window: {
+      app: "Test",
+      pid: 1,
+      bounds: [0, 0, 1920, 1080],
+      title: "T",
+      displayId: 0,
+    },
+    apps: [],
+    ocr: [
+      {
+        id: "t0-1",
+        text: "Save",
+        bbox: [100, 100, 80, 32],
+        conf: 0.97,
+        displayId: 0,
+      },
+    ],
+    ax: [],
+    vlm_scene: null,
+    vlm_elements: null,
+  };
+}
+
+function pngBuffer(seed: number): Buffer {
+  // Anything starting with the PNG signature is fine — `encodeForBrain`
+  // base64-encodes the bytes as-is.
+  const sig = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
+  return Buffer.concat([sig, Buffer.from([seed & 0xff])]);
+}
+
+function captures(): Map<number, DisplayCapture> {
+  const m = new Map<number, DisplayCapture>();
+  m.set(0, {
+    display: {
+      id: 0,
+      bounds: [0, 0, 1920, 1080],
+      scaleFactor: 1,
+      primary: true,
+      name: "fake",
+    },
+    frame: pngBuffer(1),
+  });
+  return m;
+}
+
+describe("parseBrainOutput", () => {
+  it("accepts a raw JSON object", () => {
+    const out = parseBrainOutput(
+      JSON.stringify({
+        scene_summary: "S",
+        target_display_id: 0,
+        roi: [{ displayId: 0, bbox: [10, 10, 20, 20], reason: "r" }],
+        proposed_action: {
+          kind: "click",
+          ref: "t0-1",
+          args: {},
+          rationale: "y",
+        },
+      }),
+    );
+    expect(out.scene_summary).toBe("S");
+    expect(out.target_display_id).toBe(0);
+    expect(out.roi).toHaveLength(1);
+    expect(out.proposed_action.kind).toBe("click");
+    expect(out.proposed_action.ref).toBe("t0-1");
+  });
+
+  it("strips ```json fences", () => {
+    const fenced =
+      "```json\n" +
+      JSON.stringify({
+        scene_summary: "fenced",
+        target_display_id: 0,
+        roi: [],
+        proposed_action: { kind: "wait", rationale: "r" },
+      }) +
+      "\n```";
+    const out = parseBrainOutput(fenced);
+    expect(out.scene_summary).toBe("fenced");
+    expect(out.proposed_action.kind).toBe("wait");
+  });
+
+  it("tolerates leading prose before the first brace", () => {
+    const raw =
+      "Sure! Here's the JSON: " +
+      JSON.stringify({
+        scene_summary: "with-prose",
+        target_display_id: 0,
+        roi: [],
+        proposed_action: { kind: "finish", rationale: "done" },
+      });
+    const out = parseBrainOutput(raw);
+    expect(out.scene_summary).toBe("with-prose");
+    expect(out.proposed_action.kind).toBe("finish");
+  });
+
+  it("throws BrainParseError on non-JSON", () => {
+    expect(() => parseBrainOutput("totally not json")).toThrow(BrainParseError);
+  });
+
+  it("throws BrainParseError when proposed_action is missing", () => {
+    expect(() =>
+      parseBrainOutput(
+        JSON.stringify({ scene_summary: "x", target_display_id: 0, roi: [] }),
+      ),
+    ).toThrow(/proposed_action/);
+  });
+
+  it("drops malformed ROIs without failing the whole parse", () => {
+    const out = parseBrainOutput(
+      JSON.stringify({
+        scene_summary: "x",
+        target_display_id: 0,
+        roi: [
+          { displayId: 0, bbox: [1, 2, 3, 4], reason: "ok" },
+          { displayId: 0, bbox: "not-an-array", reason: "bad" },
+          { displayId: 0, bbox: [1, 2, 3], reason: "short" },
+        ],
+        proposed_action: { kind: "click", rationale: "r" },
+      }),
+    );
+    expect(out.roi).toHaveLength(1);
+    expect(out.roi[0]?.bbox).toEqual([1, 2, 3, 4]);
+  });
+});
+
+describe("brainPromptFor", () => {
+  it("includes the goal and switches header on strict mode", () => {
+    const a = brainPromptFor("{}", "click save", false);
+    const b = brainPromptFor("{}", "click save", true);
+    expect(a).toContain("click save");
+    expect(b).toContain("click save");
+    expect(a).not.toContain("MUST emit ONLY a JSON");
+    expect(b).toContain("MUST emit ONLY a JSON");
+  });
+
+  it("documents the ROI cap in the prompt", () => {
+    const p = brainPromptFor("{}", "g", false);
+    expect(p).toContain(`Cap ROIs to ${BRAIN_MAX_ROIS}`);
+  });
+});
+
+describe("Brain.observeAndPlan", () => {
+  it("invokes the model once and returns the parsed BrainOutput", async () => {
+    let calls = 0;
+    const lastArgs: Array<{ imageUrl: string; prompt: string }> = [];
+    const brain = new Brain(null, {
+      // This test exercises the original always-attach plan flow.
+      imagePolicy: "always",
+      invokeModel: async (args) => {
+        calls += 1;
+        lastArgs.push({ imageUrl: args.imageUrl, prompt: args.prompt });
+        return JSON.stringify({
+          scene_summary: "OK",
+          target_display_id: 0,
+          roi: [{ displayId: 0, bbox: [0, 0, 10, 10], reason: "a" }],
+          proposed_action: {
+            kind: "click",
+            ref: "t0-1",
+            rationale: "Save button",
+          },
+        });
+      },
+    });
+    const out = await brain.observeAndPlan({
+      scene: dummyScene(),
+      goal: "click save",
+      captures: captures(),
+    });
+    expect(calls).toBe(1);
+    expect(lastArgs[0]?.imageUrl.startsWith("data:image/png;base64,")).toBe(
+      true,
+    );
+    expect(out.scene_summary).toBe("OK");
+    expect(out.target_display_id).toBe(0);
+    expect(out.roi).toHaveLength(1);
+    expect(out.proposed_action.ref).toBe("t0-1");
+  });
+
+  it("retries once with the strict prompt on parse failure", async () => {
+    let calls = 0;
+    const seenPrompts: string[] = [];
+    const brain = new Brain(null, {
+      // The light→strict retry is the always-attach path; pin it explicitly.
+      imagePolicy: "always",
+      invokeModel: async (args) => {
+        calls += 1;
+        seenPrompts.push(args.prompt);
+        if (calls === 1) return "not json at all";
+        return JSON.stringify({
+          scene_summary: "retry-good",
+          target_display_id: 0,
+          roi: [],
+          proposed_action: { kind: "finish", rationale: "done" },
+        });
+      },
+    });
+    const out = await brain.observeAndPlan({
+      scene: dummyScene(),
+      goal: "g",
+      captures: captures(),
+    });
+    expect(calls).toBe(2);
+    expect(seenPrompts[0]).not.toContain("MUST emit ONLY a JSON");
+    expect(seenPrompts[1]).toContain("MUST emit ONLY a JSON");
+    expect(out.proposed_action.kind).toBe("finish");
+  });
+
+  it("throws BrainParseError after the retry also fails", async () => {
+    const brain = new Brain(null, {
+      invokeModel: async () => "broken",
+    });
+    await expect(
+      brain.observeAndPlan({
+        scene: dummyScene(),
+        goal: "g",
+        captures: captures(),
+      }),
+    ).rejects.toBeInstanceOf(BrainParseError);
+  });
+
+  it("enforces the ROI cap", async () => {
+    const tooMany = Array.from({ length: BRAIN_MAX_ROIS + 3 }, (_, i) => ({
+      displayId: 0,
+      bbox: [i, i, 1, 1] as [number, number, number, number],
+      reason: `r${i}`,
+    }));
+    const brain = new Brain(null, {
+      invokeModel: async () =>
+        JSON.stringify({
+          scene_summary: "S",
+          target_display_id: 0,
+          roi: tooMany,
+          proposed_action: { kind: "wait", rationale: "" },
+        }),
+    });
+    const out = await brain.observeAndPlan({
+      scene: dummyScene(),
+      goal: "g",
+      captures: captures(),
+    });
+    expect(out.roi.length).toBe(BRAIN_MAX_ROIS);
+  });
+
+  it("accepts an ImageDescriptionResult with `description` payload", async () => {
+    const brain = new Brain(null, {
+      invokeModel: async () => ({
+        title: "ignored",
+        description: JSON.stringify({
+          scene_summary: "from-desc",
+          target_display_id: 0,
+          roi: [],
+          proposed_action: { kind: "wait", rationale: "" },
+        }),
+      }),
+    });
+    const out = await brain.observeAndPlan({
+      scene: dummyScene(),
+      goal: "g",
+      captures: captures(),
+    });
+    expect(out.scene_summary).toBe("from-desc");
+  });
+
+  it("fails when no captures are supplied", async () => {
+    const brain = new Brain(null, {
+      invokeModel: async () => "{}",
+    });
+    await expect(
+      brain.observeAndPlan({
+        scene: dummyScene(),
+        goal: "g",
+        captures: new Map(),
+      }),
+    ).rejects.toThrow(/no captures/);
+  });
+
+  it("picks the focused display capture when present", async () => {
+    let receivedDisplay = -1;
+    const brain = new Brain(null, {
+      invokeModel: async (args) => {
+        receivedDisplay = args.displayId;
+        return JSON.stringify({
+          scene_summary: "S",
+          target_display_id: args.displayId,
+          roi: [],
+          proposed_action: { kind: "wait", rationale: "" },
+        });
+      },
+    });
+    const scene = dummyScene();
+    // Force the focused window onto a synthetic display id 0 — same as
+    // captures map key.
+    scene.focused_window!.displayId = 0;
+    await brain.observeAndPlan({ scene, goal: "g", captures: captures() });
+    expect(receivedDisplay).toBe(0);
+  });
+});

plugins/plugin-computeruse/src/__tests__/browser-auto-open.test.ts

@@ -0,0 +1,184 @@
+/**
+ * Auto-open browser behavior of the computer-use flow, driven against a mocked
+ * platform/browser module (deterministic, no real CDP).
+ */
+import type { IAgentRuntime } from "@elizaos/core";
+import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
+
+vi.mock("../platform/browser.js", () => {
+  const state = (url = "about:blank") => ({
+    url,
+    title: "Example",
+    isOpen: true,
+    is_open: true,
+  });
+
+  return {
+    clickBrowser: vi.fn(async () => {}),
+    closeBrowser: vi.fn(async () => {}),
+    closeBrowserTab: vi.fn(async () => {}),
+    executeBrowser: vi.fn(async () => "ok"),
+    getBrowserClickables: vi.fn(async () => []),
+    getBrowserContext: vi.fn(async () => state()),
+    getBrowserDom: vi.fn(async () => "<html></html>"),
+    getBrowserInfo: vi.fn(async () => ({
+      success: true,
+      ...state(),
+    })),
+    getBrowserState: vi.fn(async () => state()),
+    isBrowserAvailable: vi.fn(() => true),
+    listBrowserTabs: vi.fn(async () => []),
+    navigateBrowser: vi.fn(async (url: string) => state(url)),
+    openBrowser: vi.fn(async (url?: string) => state(url)),
+    openBrowserTab: vi.fn(async (url?: string) => ({
+      id: "1",
+      url: url ?? "about:blank",
+      title: "Example",
+      active: true,
+    })),
+    screenshotBrowser: vi.fn(async () => "base64"),
+    scrollBrowser: vi.fn(async () => {}),
+    setBrowserRuntimeOptions: vi.fn(),
+    switchBrowserTab: vi.fn(async () => state()),
+    typeBrowser: vi.fn(async () => {}),
+    waitBrowser: vi.fn(async () => {}),
+  };
+});
+
+const browser = await import("../platform/browser.js");
+const { ComputerUseService } = await import(
+  "../services/computer-use-service.js"
+);
+
+function createRuntime(): IAgentRuntime {
+  return {
+    character: {},
+    getSetting(key: string) {
+      return key === "COMPUTER_USE_APPROVAL_MODE" ? "full_control" : undefined;
+    },
+    getService() {
+      return null;
+    },
+  } as IAgentRuntime;
+}
+
+describe("ComputerUseService browser auto-open recovery", () => {
+  let service: ComputerUseService;
+
+  beforeEach(async () => {
+    vi.clearAllMocks();
+    vi.mocked(browser.openBrowser).mockImplementation(async (url?: string) => ({
+      url: url ?? "about:blank",
+      title: "Example",
+      isOpen: true,
+      is_open: true,
+    }));
+    vi.mocked(browser.navigateBrowser).mockImplementation(
+      async (url: string) => ({
+        url,
+        title: "Example",
+        isOpen: true,
+        is_open: true,
+      }),
+    );
+    vi.mocked(browser.getBrowserInfo).mockResolvedValue({
+      success: true,
+      url: "about:blank",
+      title: "Example",
+      isOpen: true,
+      is_open: true,
+    });
+    vi.mocked(browser.screenshotBrowser).mockResolvedValue("base64");
+    service = (await ComputerUseService.start(
+      createRuntime(),
+    )) as ComputerUseService;
+  });
+
+  afterEach(async () => {
+    await service.stop();
+  });
+
+  it("opens the browser and retries a first-call navigate failure once", async () => {
+    vi.mocked(browser.navigateBrowser)
+      .mockRejectedValueOnce(
+        new Error("Browser not open. Use the open action first."),
+      )
+      .mockResolvedValueOnce({
+        url: "https://example.com",
+        title: "Example",
+        isOpen: true,
+        is_open: true,
+      });
+
+    const result = await service.executeBrowserAction({
+      action: "navigate",
+      url: "https://example.com",
+    });
+
+    expect(result).toMatchObject({
+      success: true,
+      url: "https://example.com",
+    });
+    expect(browser.openBrowser).toHaveBeenCalledTimes(1);
+    expect(browser.openBrowser).toHaveBeenCalledWith("https://example.com");
+    expect(browser.navigateBrowser).toHaveBeenCalledTimes(2);
+  });
+
+  it("also recovers when a non-lifecycle action returns a Browser not open result", async () => {
+    vi.mocked(browser.getBrowserInfo)
+      .mockResolvedValueOnce({
+        success: false,
+        url: "",
+        title: "",
+        isOpen: false,
+        is_open: false,
+        error: "Browser not open.",
+      })
+      .mockResolvedValueOnce({
+        success: true,
+        url: "about:blank",
+        title: "Example",
+        isOpen: true,
+        is_open: true,
+      });
+
+    const result = await service.executeBrowserAction({ action: "info" });
+
+    expect(result).toMatchObject({
+      success: true,
+      isOpen: true,
+    });
+    expect(browser.openBrowser).toHaveBeenCalledTimes(1);
+    expect(browser.getBrowserInfo).toHaveBeenCalledTimes(2);
+  });
+
+  it("does not auto-open for lifecycle close failures", async () => {
+    vi.mocked(browser.closeBrowser).mockRejectedValueOnce(
+      new Error("Browser not open."),
+    );
+
+    const result = await service.executeBrowserAction({ action: "close" });
+
+    expect(result).toMatchObject({
+      success: false,
+      error: "Browser not open.",
+    });
+    expect(browser.openBrowser).not.toHaveBeenCalled();
+  });
+
+  it("returns a failed command result when browser screenshot quality fails", async () => {
+    vi.mocked(browser.screenshotBrowser).mockRejectedValueOnce(
+      new Error(
+        'browser screenshot: screenshot quality failed: browser screenshot: screenshot is one color; metrics={"colorBuckets":1}',
+      ),
+    );
+
+    const result = await service.executeBrowserAction({ action: "screenshot" });
+
+    expect(result.success).toBe(false);
+    expect(result.error).toContain("screenshot quality failed");
+    expect(result.error).toContain("screenshot is one color");
+    expect(result.screenshot).toBeUndefined();
+    expect(browser.screenshotBrowser).toHaveBeenCalledTimes(1);
+  });
+});

plugins/plugin-computeruse/src/__tests__/browser-execute-security.test.ts

@@ -0,0 +1,15 @@
+/**
+ * Guards that executeBrowser refuses arbitrary agent-supplied script and never
+ * opens a page (GHSA-rcvr-766c-4phv). Deterministic unit test.
+ */
+import { describe, expect, it } from "vitest";
+import { executeBrowser } from "../platform/browser.js";
+import { BrowserExecuteDisabledError } from "../security/browser-script-policy.js";
+
+describe("executeBrowser security", () => {
+  it("rejects arbitrary script without opening a browser page", async () => {
+    await expect(executeBrowser("document.cookie")).rejects.toThrow(
+      BrowserExecuteDisabledError,
+    );
+  });
+});

plugins/plugin-computeruse/src/__tests__/browser-script-policy.test.ts

@@ -0,0 +1,23 @@
+/**
+ * Pins the browser-script policy: arbitrary page.evaluate is never allowed.
+ * Deterministic unit test over the policy predicate.
+ */
+import { describe, expect, it } from "vitest";
+import {
+  assertBrowserExecuteAllowed,
+  BROWSER_EXECUTE_DISABLED_MESSAGE,
+  BrowserExecuteDisabledError,
+  isBrowserExecuteAllowed,
+} from "../security/browser-script-policy.js";
+
+describe("browser-script-policy", () => {
+  it("never allows arbitrary browser script execution", () => {
+    expect(isBrowserExecuteAllowed()).toBe(false);
+    expect(() => assertBrowserExecuteAllowed()).toThrow(
+      BrowserExecuteDisabledError,
+    );
+    expect(() => assertBrowserExecuteAllowed()).toThrow(
+      BROWSER_EXECUTE_DISABLED_MESSAGE,
+    );
+  });
+});

plugins/plugin-computeruse/src/__tests__/capture-region.test.ts

@@ -0,0 +1,66 @@
+/**
+ * Unit tests for normalizeCaptureRegion (capture.ts).
+ *
+ * Guards the GDI+ screenshot path: zero/negative/non-integer regions throw a
+ * clear error instead of an opaque "Parameter is not valid" from
+ * `New-Object System.Drawing.Bitmap`. `x`/`y` may be negative (secondary
+ * monitors in the negative virtual-desktop quadrant); only dimensions are
+ * bounds-checked. Pure + cross-platform.
+ */
+
+import { describe, expect, it } from "vitest";
+import { normalizeCaptureRegion } from "../platform/capture.js";
+
+describe("normalizeCaptureRegion", () => {
+  it("passes a valid integer region through unchanged", () => {
+    expect(
+      normalizeCaptureRegion({ x: 0, y: 0, width: 1920, height: 1080 }),
+    ).toEqual({ x: 0, y: 0, width: 1920, height: 1080 });
+  });
+
+  it("rounds fractional coordinates and dimensions to integers", () => {
+    expect(
+      normalizeCaptureRegion({ x: 10.4, y: 10.6, width: 99.5, height: 50.2 }),
+    ).toEqual({ x: 10, y: 11, width: 100, height: 50 });
+  });
+
+  it("allows negative x/y (secondary monitors left of / above primary)", () => {
+    expect(
+      normalizeCaptureRegion({ x: -1920, y: -100, width: 1280, height: 1024 }),
+    ).toEqual({ x: -1920, y: -100, width: 1280, height: 1024 });
+  });
+
+  it("rejects zero width or height", () => {
+    expect(() =>
+      normalizeCaptureRegion({ x: 0, y: 0, width: 0, height: 100 }),
+    ).toThrow(/positive/i);
+    expect(() =>
+      normalizeCaptureRegion({ x: 0, y: 0, width: 100, height: 0 }),
+    ).toThrow(/positive/i);
+  });
+
+  it("rejects negative width or height", () => {
+    expect(() =>
+      normalizeCaptureRegion({ x: 0, y: 0, width: -10, height: 100 }),
+    ).toThrow(/positive/i);
+  });
+
+  it("rejects non-finite dimensions", () => {
+    expect(() =>
+      normalizeCaptureRegion({
+        x: 0,
+        y: 0,
+        width: Number.NaN,
+        height: 100,
+      }),
+    ).toThrow(/finite/i);
+    expect(() =>
+      normalizeCaptureRegion({
+        x: 0,
+        y: 0,
+        width: 100,
+        height: Number.POSITIVE_INFINITY,
+      }),
+    ).toThrow(/finite/i);
+  });
+});

plugins/plugin-computeruse/src/__tests__/cascade-grounding-cache.test.ts

@@ -0,0 +1,120 @@
+/**
+ * Cascade per-Scene grounding cache (#9105 M5 — predict/ground split).
+ *
+ * Grounding the same target on the same Scene is deterministic, so the cheap
+ * GROUND step is memoized: a repeat ground on the same Scene reuses the coords
+ * without re-running OCR/AX resolution or the (possibly model-backed) actor. A
+ * new Scene (new timestamp) invalidates the cache.
+ */
+
+import { describe, expect, it } from "vitest";
+import type { Actor } from "../actor/actor.js";
+import { Brain } from "../actor/brain.js";
+import { Cascade } from "../actor/cascade.js";
+import type { DisplayCapture } from "../platform/capture.js";
+import type { Scene } from "../scene/scene-types.js";
+
+function scene(timestamp = 1): Scene {
+  return {
+    timestamp,
+    displays: [
+      {
+        id: 0,
+        bounds: [0, 0, 1920, 1080],
+        scaleFactor: 1,
+        primary: true,
+        name: "f",
+      },
+    ],
+    focused_window: null,
+    apps: [],
+    ocr: [
+      {
+        id: "t0-1",
+        text: "Save",
+        bbox: [100, 200, 80, 32],
+        conf: 0.97,
+        displayId: 0,
+      },
+    ],
+    ax: [],
+    vlm_scene: null,
+    vlm_elements: null,
+  };
+}
+
+function captures(): Map<number, DisplayCapture> {
+  const sig = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
+  const m = new Map<number, DisplayCapture>();
+  m.set(0, {
+    display: {
+      id: 0,
+      bounds: [0, 0, 1920, 1080],
+      scaleFactor: 1,
+      primary: true,
+      name: "f",
+    },
+    frame: Buffer.concat([sig, Buffer.alloc(56, 1)]),
+  });
+  return m;
+}
+
+const BRAIN_OUT = {
+  scene_summary: "s",
+  target_display_id: 0,
+  roi: [{ displayId: 0, bbox: [100, 200, 80, 32], reason: "r" }],
+  proposed_action: { kind: "click", ref: "t0-1", rationale: "Save" },
+};
+
+function fakeBrain(): Brain {
+  return new Brain(null, {
+    invokeModel: async () => JSON.stringify(BRAIN_OUT),
+  });
+}
+
+function countingActor(counter: { calls: number }): Actor {
+  return {
+    name: "counting",
+    async ground() {
+      counter.calls += 1;
+      return { displayId: 0, x: 140, y: 216, confidence: 1, reason: "refined" };
+    },
+  };
+}
+
+describe("Cascade grounding cache (M5)", () => {
+  it("reuses grounding for the same target on the same Scene", async () => {
+    const counter = { calls: 0 };
+    const cascade = new Cascade({
+      brain: fakeBrain(),
+      actor: countingActor(counter),
+    });
+    const s = scene(1);
+    const r1 = await cascade.run({
+      scene: s,
+      goal: "save",
+      captures: captures(),
+    });
+    const r2 = await cascade.run({
+      scene: s,
+      goal: "save",
+      captures: captures(),
+    });
+    expect(counter.calls).toBe(1); // 2nd ground served from cache
+    expect(r2.proposed.x).toBe(r1.proposed.x);
+    expect(r2.proposed.y).toBe(r1.proposed.y);
+    expect(cascade.getGroundStats()).toEqual({ hits: 1, misses: 1 });
+  });
+
+  it("re-grounds when the Scene changes (cache invalidated by timestamp)", async () => {
+    const counter = { calls: 0 };
+    const cascade = new Cascade({
+      brain: fakeBrain(),
+      actor: countingActor(counter),
+    });
+    await cascade.run({ scene: scene(1), goal: "save", captures: captures() });
+    await cascade.run({ scene: scene(2), goal: "save", captures: captures() });
+    expect(counter.calls).toBe(2);
+    expect(cascade.getGroundStats()).toEqual({ hits: 0, misses: 2 });
+  });
+});

plugins/plugin-computeruse/src/__tests__/cascade.test.ts

@@ -0,0 +1,462 @@
+/**
+ * WS7 — Cascade (ScreenSeekeR) tests.
+ *
+ * Validates the orchestrator that takes a fake `Brain` and turns its
+ * BrainOutput into a concrete `ProposedAction`:
+ *
+ *   - Non-coordinate actions (`wait`, `finish`, `type`, `hotkey`, `key`)
+ *     short-circuit grounding and don't invoke the Actor.
+ *   - `click` with a `ref` resolves via OCR/AX deterministic grounding.
+ *   - `click` with a ROI and a registered Actor passes the *cropped* PNG
+ *     (native resolution, exact bbox slice) to the Actor, and uses its
+ *     returned coords (rounded).
+ *   - `click` with no ref / no actor falls back to ROI center.
+ *   - `scroll` resolves an anchor (ROI center) and forwards dx/dy intact.
+ *   - `drag` enforces both endpoints.
+ *
+ * Memory-arbiter pass-through:
+ *   - Repeated identical inputs to `runtime.useModel(IMAGE_DESCRIPTION, ...)`
+ *     produce identical `imageUrl` payloads (deterministic base64 of the
+ *     same PNG bytes), which is what the WS2 MemoryArbiter content-hashes
+ *     on. The cache hit on the arbiter side is its own job; here we just
+ *     prove that two identical frames produce two identical model inputs.
+ */
+
+import { describe, expect, it } from "vitest";
+import { type Actor, OcrCoordinateGroundingActor } from "../actor/actor.js";
+import { Brain } from "../actor/brain.js";
+import { Cascade } from "../actor/cascade.js";
+import type { DisplayCapture } from "../platform/capture.js";
+import type { Scene } from "../scene/scene-types.js";
+
+function scene(): Scene {
+  return {
+    timestamp: 1,
+    displays: [
+      {
+        id: 0,
+        bounds: [0, 0, 1920, 1080],
+        scaleFactor: 1,
+        primary: true,
+        name: "fake",
+      },
+    ],
+    focused_window: {
+      app: "Editor",
+      pid: 1,
+      bounds: [0, 0, 1920, 1080],
+      title: "Untitled",
+      displayId: 0,
+    },
+    apps: [],
+    ocr: [
+      {
+        id: "t0-1",
+        text: "Save",
+        bbox: [100, 200, 80, 32],
+        conf: 0.97,
+        displayId: 0,
+      },
+      {
+        id: "t0-2",
+        text: "Cancel",
+        bbox: [200, 200, 80, 32],
+        conf: 0.95,
+        displayId: 0,
+      },
+      {
+        id: "t0-3",
+        text: "File",
+        bbox: [50, 10, 40, 20],
+        conf: 0.99,
+        displayId: 0,
+      },
+    ],
+    ax: [
+      {
+        id: "a0-1",
+        role: "button",
+        label: "Open",
+        bbox: [400, 500, 60, 30],
+        actions: ["press"],
+        displayId: 0,
+      },
+    ],
+    vlm_scene: null,
+    vlm_elements: null,
+  };
+}
+
+function pngOf(seed: number, sizeBytes = 64): Buffer {
+  const sig = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
+  const body = Buffer.alloc(sizeBytes - sig.length, seed & 0xff);
+  return Buffer.concat([sig, body]);
+}
+
+function captures(seed = 1): Map<number, DisplayCapture> {
+  const m = new Map<number, DisplayCapture>();
+  m.set(0, {
+    display: {
+      id: 0,
+      bounds: [0, 0, 1920, 1080],
+      scaleFactor: 1,
+      primary: true,
+      name: "fake",
+    },
+    frame: pngOf(seed),
+  });
+  return m;
+}
+
+function fakeBrain(out: unknown): Brain {
+  // These memory-arbiter tests content-hash the attached image payload, so they
+  // require the always-attach path (the new default plans imageless-first).
+  return new Brain(null, {
+    imagePolicy: "always",
+    invokeModel: async () => JSON.stringify(out),
+  });
+}
+
+describe("Cascade — non-coordinate actions short-circuit grounding", () => {
+  it("wait", async () => {
+    const cascade = new Cascade({
+      brain: fakeBrain({
+        scene_summary: "S",
+        target_display_id: 0,
+        roi: [],
+        proposed_action: { kind: "wait", rationale: "loading" },
+      }),
+    });
+    const res = await cascade.run({
+      scene: scene(),
+      goal: "g",
+      captures: captures(),
+    });
+    expect(res.proposed.kind).toBe("wait");
+    expect(res.proposed.displayId).toBe(0);
+    expect(res.proposed.x).toBeUndefined();
+  });
+
+  it("finish", async () => {
+    const cascade = new Cascade({
+      brain: fakeBrain({
+        scene_summary: "done",
+        target_display_id: 0,
+        roi: [],
+        proposed_action: { kind: "finish", rationale: "" },
+      }),
+    });
+    const res = await cascade.run({
+      scene: scene(),
+      goal: "g",
+      captures: captures(),
+    });
+    expect(res.proposed.kind).toBe("finish");
+  });
+
+  it("type forwards args.text", async () => {
+    const cascade = new Cascade({
+      brain: fakeBrain({
+        scene_summary: "S",
+        target_display_id: 0,
+        roi: [],
+        proposed_action: {
+          kind: "type",
+          args: { text: "hello" },
+          rationale: "",
+        },
+      }),
+    });
+    const res = await cascade.run({
+      scene: scene(),
+      goal: "g",
+      captures: captures(),
+    });
+    expect(res.proposed.kind).toBe("type");
+    expect(res.proposed.text).toBe("hello");
+  });
+
+  it("hotkey forwards args.keys", async () => {
+    const cascade = new Cascade({
+      brain: fakeBrain({
+        scene_summary: "S",
+        target_display_id: 0,
+        roi: [],
+        proposed_action: {
+          kind: "hotkey",
+          args: { keys: ["ctrl", "s"] },
+          rationale: "",
+        },
+      }),
+    });
+    const res = await cascade.run({
+      scene: scene(),
+      goal: "g",
+      captures: captures(),
+    });
+    expect(res.proposed.kind).toBe("hotkey");
+    expect(res.proposed.keys).toEqual(["ctrl", "s"]);
+  });
+
+  it("key forwards args.key", async () => {
+    const cascade = new Cascade({
+      brain: fakeBrain({
+        scene_summary: "S",
+        target_display_id: 0,
+        roi: [],
+        proposed_action: { kind: "key", args: { key: "Enter" }, rationale: "" },
+      }),
+    });
+    const res = await cascade.run({
+      scene: scene(),
+      goal: "g",
+      captures: captures(),
+    });
+    expect(res.proposed.kind).toBe("key");
+    expect(res.proposed.key).toBe("Enter");
+  });
+
+  it("type without args.text throws", async () => {
+    const cascade = new Cascade({
+      brain: fakeBrain({
+        scene_summary: "S",
+        target_display_id: 0,
+        roi: [],
+        proposed_action: { kind: "type", rationale: "" },
+      }),
+    });
+    await expect(
+      cascade.run({ scene: scene(), goal: "g", captures: captures() }),
+    ).rejects.toThrow(/args.text/);
+  });
+});
+
+describe("Cascade — click grounding paths", () => {
+  it("ref → OCR/AX deterministic grounding resolves to bbox center", async () => {
+    const cascade = new Cascade({
+      brain: fakeBrain({
+        scene_summary: "S",
+        target_display_id: 0,
+        roi: [],
+        proposed_action: { kind: "click", ref: "t0-1", rationale: "save" },
+      }),
+    });
+    const res = await cascade.run({
+      scene: scene(),
+      goal: "g",
+      captures: captures(),
+    });
+    // Save bbox = [100, 200, 80, 32] -> center = (140, 216)
+    expect(res.proposed.kind).toBe("click");
+    expect(res.proposed.x).toBe(140);
+    expect(res.proposed.y).toBe(216);
+  });
+
+  it("ROI + registered Actor: Actor receives the cropped native-resolution PNG", async () => {
+    const seenCrops: Buffer[] = [];
+    const fakeActor: Actor = {
+      name: "fake",
+      ground: async (args) => {
+        seenCrops.push(args.croppedImage);
+        return {
+          displayId: args.displayId,
+          x: 555.4,
+          y: 666.6,
+          confidence: 0.9,
+          reason: "fake",
+        };
+      },
+    };
+    const fakeCrop = (
+      frame: Buffer,
+      bbox: [number, number, number, number],
+    ): Buffer => {
+      // Concatenate a deterministic tag with bbox so we can assert the bbox
+      // we cropped from is the one the Brain asked for.
+      return Buffer.concat([
+        Buffer.from(`crop:[${bbox.join(",")}]:`, "utf8"),
+        frame,
+      ]);
+    };
+    const cascade = new Cascade({
+      brain: fakeBrain({
+        scene_summary: "S",
+        target_display_id: 0,
+        roi: [{ displayId: 0, bbox: [10, 20, 200, 300], reason: "btn-area" }],
+        proposed_action: { kind: "click", rationale: "click btn" },
+      }),
+      actor: fakeActor,
+      crop: fakeCrop,
+    });
+    const res = await cascade.run({
+      scene: scene(),
+      goal: "g",
+      captures: captures(42),
+    });
+    expect(seenCrops).toHaveLength(1);
+    // Crop carries the exact bbox the Brain emitted.
+    expect(seenCrops[0]?.toString("utf8", 0, 20)).toBe("crop:[10,20,200,300]");
+    // Actor coords are rounded by the cascade.
+    expect(res.proposed.x).toBe(555);
+    expect(res.proposed.y).toBe(667);
+    expect(res.proposed.displayId).toBe(0);
+  });
+
+  it("ROI without Actor falls back to ROI center", async () => {
+    const cascade = new Cascade({
+      brain: fakeBrain({
+        scene_summary: "S",
+        target_display_id: 0,
+        roi: [{ displayId: 0, bbox: [100, 200, 80, 40], reason: "r" }],
+        proposed_action: { kind: "click", rationale: "click me" },
+      }),
+    });
+    const res = await cascade.run({
+      scene: scene(),
+      goal: "g",
+      captures: captures(),
+    });
+    expect(res.proposed.x).toBe(140);
+    expect(res.proposed.y).toBe(220);
+  });
+
+  it("rois are truncated to the cascade cap in the result", async () => {
+    const cascade = new Cascade({
+      brain: fakeBrain({
+        scene_summary: "S",
+        target_display_id: 0,
+        roi: [
+          { displayId: 0, bbox: [100, 200, 80, 40], reason: "r1" },
+          { displayId: 0, bbox: [100, 300, 80, 40], reason: "r2" },
+          { displayId: 0, bbox: [100, 400, 80, 40], reason: "r3" },
+        ],
+        proposed_action: { kind: "click", rationale: "x" },
+      }),
+    });
+    const res = await cascade.run({
+      scene: scene(),
+      goal: "g",
+      captures: captures(),
+    });
+    expect(res.rois.length).toBeLessThanOrEqual(2);
+  });
+
+  it("no ref and no roi → can't resolve a click", async () => {
+    const cascade = new Cascade({
+      brain: fakeBrain({
+        scene_summary: "S",
+        target_display_id: 0,
+        roi: [],
+        proposed_action: { kind: "click", rationale: "where?" },
+      }),
+    });
+    await expect(
+      cascade.run({ scene: scene(), goal: "g", captures: captures() }),
+    ).rejects.toThrow(/could not resolve coordinates/);
+  });
+
+  it("scroll forwards (dx, dy) and anchors on ROI center", async () => {
+    const cascade = new Cascade({
+      brain: fakeBrain({
+        scene_summary: "S",
+        target_display_id: 0,
+        roi: [
+          { displayId: 0, bbox: [100, 100, 200, 200], reason: "scroll-area" },
+        ],
+        proposed_action: {
+          kind: "scroll",
+          args: { dx: 0, dy: 3 },
+          rationale: "scroll",
+        },
+      }),
+    });
+    const res = await cascade.run({
+      scene: scene(),
+      goal: "g",
+      captures: captures(),
+    });
+    expect(res.proposed.kind).toBe("scroll");
+    expect(res.proposed.x).toBe(200);
+    expect(res.proposed.y).toBe(200);
+    expect(res.proposed.dx).toBe(0);
+    expect(res.proposed.dy).toBe(3);
+  });
+
+  it("drag requires start+end endpoints", async () => {
+    const cascade = new Cascade({
+      brain: fakeBrain({
+        scene_summary: "S",
+        target_display_id: 0,
+        roi: [],
+        proposed_action: {
+          kind: "drag",
+          args: { from: { x: 10, y: 20 }, to: { x: 100, y: 200 } },
+          rationale: "drag",
+        },
+      }),
+    });
+    const res = await cascade.run({
+      scene: scene(),
+      goal: "g",
+      captures: captures(),
+    });
+    expect(res.proposed.kind).toBe("drag");
+    expect(res.proposed.startX).toBe(10);
+    expect(res.proposed.startY).toBe(20);
+    expect(res.proposed.x).toBe(100);
+    expect(res.proposed.y).toBe(200);
+  });
+});
+
+describe("Cascade — registered OCR actor end-to-end", () => {
+  it("uses OcrCoordinateGroundingActor when the cascade receives one", async () => {
+    const actor = new OcrCoordinateGroundingActor(() => scene());
+    const cascade = new Cascade({
+      brain: fakeBrain({
+        scene_summary: "S",
+        target_display_id: 0,
+        roi: [],
+        proposed_action: { kind: "click", ref: "t0-2", rationale: "cancel" },
+      }),
+      actor,
+    });
+    const res = await cascade.run({
+      scene: scene(),
+      goal: "g",
+      captures: captures(),
+    });
+    // Cancel bbox = [200, 200, 80, 32] -> center (240, 216)
+    expect(res.proposed.x).toBe(240);
+    expect(res.proposed.y).toBe(216);
+  });
+});
+
+describe("Cascade — memory-arbiter pass-through", () => {
+  it("sends the same imageUrl payload for two identical frames", async () => {
+    // The cascade asks the Brain to call the model; we inspect the args.
+    const seenImageUrls: string[] = [];
+    const brain = new Brain(null, {
+      // This asserts the attached PNG payload is stable across identical
+      // frames, so it requires the always-attach path.
+      imagePolicy: "always",
+      invokeModel: async (args) => {
+        seenImageUrls.push(args.imageUrl);
+        return JSON.stringify({
+          scene_summary: "S",
+          target_display_id: 0,
+          roi: [],
+          proposed_action: { kind: "wait", rationale: "" },
+        });
+      },
+    });
+    const cascade = new Cascade({ brain });
+    const caps = captures(7);
+    await cascade.run({ scene: scene(), goal: "g", captures: caps });
+    await cascade.run({ scene: scene(), goal: "g", captures: caps });
+    expect(seenImageUrls).toHaveLength(2);
+    // Same input bytes → identical base64. The WS2 MemoryArbiter content-
+    // hashes on this exact payload, which is what enables its cache.
+    expect(seenImageUrls[0]).toBe(seenImageUrls[1]);
+    expect(seenImageUrls[0]?.startsWith("data:image/png;base64,")).toBe(true);
+  });
+});

plugins/plugin-computeruse/src/__tests__/computer-interface.test.ts

@@ -0,0 +1,376 @@
+/**
+ * WS7 — ComputerInterface tests.
+ *
+ * Validates:
+ *   - `toScreenCoordinates` / `toScreenshotCoordinates` round-trip cleanly
+ *     for the image-space ↔ display-local mapping the VLM cascade relies on.
+ *   - The thin-facade `DefaultComputerInterface` delegates each primitive
+ *     (click / drag / scroll / type / hotkey / key) through the injected
+ *     driver overrides — proving the cascade can swap in a fake driver for
+ *     deterministic tests without touching the real platform layer.
+ *   - Unknown display ids throw; the dispatcher relies on that contract.
+ */
+
+import { describe, expect, it } from "vitest";
+import {
+  type ComputerInterfaceDeps,
+  DefaultComputerInterface,
+  makeComputerInterface,
+} from "../actor/computer-interface.js";
+import type { DisplayDescriptor } from "../types.js";
+
+function fakeDisplays(): DisplayDescriptor[] {
+  return [
+    {
+      id: 0,
+      bounds: [0, 0, 1920, 1080],
+      scaleFactor: 1,
+      primary: true,
+      name: "fake-1",
+    },
+    {
+      id: 1,
+      bounds: [1920, 0, 2560, 1440],
+      scaleFactor: 1,
+      primary: false,
+      name: "fake-2",
+    },
+  ];
+}
+
+function makeIface(extra: Partial<ComputerInterfaceDeps> = {}): {
+  iface: DefaultComputerInterface;
+  calls: Record<string, unknown[][]>;
+} {
+  const calls: Record<string, unknown[][]> = {
+    click: [],
+    doubleClick: [],
+    rightClick: [],
+    mouseMove: [],
+    mouseDown: [],
+    mouseUp: [],
+    drag: [],
+    dragPath: [],
+    scroll: [],
+    type: [],
+    keyPress: [],
+    keyCombo: [],
+    keyDown: [],
+    keyUp: [],
+  };
+  const iface = new DefaultComputerInterface({
+    listDisplays: () => fakeDisplays(),
+    driver: {
+      click: async (x, y) => {
+        calls.click?.push([x, y]);
+      },
+      doubleClick: async (x, y) => {
+        calls.doubleClick?.push([x, y]);
+      },
+      rightClick: async (x, y) => {
+        calls.rightClick?.push([x, y]);
+      },
+      mouseMove: async (x, y) => {
+        calls.mouseMove?.push([x, y]);
+      },
+      mouseDown: async (x, y, button) => {
+        calls.mouseDown?.push([x, y, button]);
+      },
+      mouseUp: async (x, y, button) => {
+        calls.mouseUp?.push([x, y, button]);
+      },
+      drag: async (x1, y1, x2, y2) => {
+        calls.drag?.push([x1, y1, x2, y2]);
+      },
+      dragPath: async (path) => {
+        calls.dragPath?.push([path]);
+      },
+      scroll: async (x, y, dir, amt) => {
+        calls.scroll?.push([x, y, dir, amt]);
+      },
+      type: async (t) => {
+        calls.type?.push([t]);
+      },
+      keyPress: async (k) => {
+        calls.keyPress?.push([k]);
+      },
+      keyCombo: async (c) => {
+        calls.keyCombo?.push([c]);
+      },
+      keyDown: async (k) => {
+        calls.keyDown?.push([k]);
+      },
+      keyUp: async (k) => {
+        calls.keyUp?.push([k]);
+      },
+    },
+    ...extra,
+  });
+  return { iface, calls };
+}
+
+describe("DefaultComputerInterface — coord conversions", () => {
+  it("round-trips toScreenCoordinates → toScreenshotCoordinates", () => {
+    const { iface } = makeIface();
+    // Image-space 640x360 represents the 1920x1080 display.
+    const screen = iface.toScreenCoordinates({
+      displayId: 0,
+      imgX: 320,
+      imgY: 180,
+      imgW: 640,
+      imgH: 360,
+    });
+    // 320/640 * 1920 = 960; 180/360 * 1080 = 540.
+    expect(screen).toEqual({ x: 960, y: 540 });
+    const back = iface.toScreenshotCoordinates({
+      displayId: 0,
+      x: screen.x,
+      y: screen.y,
+      imgW: 640,
+      imgH: 360,
+    });
+    expect(back).toEqual({ imgX: 320, imgY: 180 });
+  });
+
+  it("handles non-unity image dimensions for both directions", () => {
+    const { iface } = makeIface();
+    // Asymmetric image — 800x600 against the 2560x1440 second display.
+    const a = iface.toScreenCoordinates({
+      displayId: 1,
+      imgX: 400,
+      imgY: 300,
+      imgW: 800,
+      imgH: 600,
+    });
+    // 400/800 * 2560 = 1280; 300/600 * 1440 = 720.
+    expect(a).toEqual({ x: 1280, y: 720 });
+    const b = iface.toScreenshotCoordinates({
+      displayId: 1,
+      x: 1280,
+      y: 720,
+      imgW: 800,
+      imgH: 600,
+    });
+    expect(b).toEqual({ imgX: 400, imgY: 300 });
+  });
+
+  it("rejects zero image dimensions", () => {
+    const { iface } = makeIface();
+    expect(() =>
+      iface.toScreenCoordinates({
+        displayId: 0,
+        imgX: 1,
+        imgY: 1,
+        imgW: 0,
+        imgH: 100,
+      }),
+    ).toThrow(/positive image dimensions/);
+  });
+
+  it("throws on unknown display id for coord conversions", () => {
+    const { iface } = makeIface();
+    expect(() =>
+      iface.toScreenCoordinates({
+        displayId: 99,
+        imgX: 1,
+        imgY: 1,
+        imgW: 100,
+        imgH: 100,
+      }),
+    ).toThrow(/unknown displayId/);
+  });
+});
+
+describe("DefaultComputerInterface — driver delegation", () => {
+  it("leftClick / doubleClick / rightClick route through injected driver", async () => {
+    // The driver-delegation path goes through the central
+    // `localToGlobal` (coords.ts), which uses the live OS-level
+    // `findDisplay`, not the injected `listDisplays`. Drive only against
+    // display 0 (the host's primary) here; the global-translation math is
+    // exercised in `coords` tests.
+    const { iface, calls } = makeIface();
+    await iface.leftClick({ displayId: 0, x: 100, y: 200 });
+    await iface.doubleClick({ displayId: 0, x: 300, y: 400 });
+    await iface.rightClick({ displayId: 0, x: 50, y: 60 });
+    expect(calls.click).toHaveLength(1);
+    expect(calls.doubleClick).toHaveLength(1);
+    expect(calls.rightClick).toHaveLength(1);
+    // Each entry is [globalX, globalY]; on this host primary origin is 0,0,
+    // so the local coords pass through unchanged.
+    expect(calls.click[0]?.[0]).toBe(100);
+    expect(calls.click[0]?.[1]).toBe(200);
+  });
+
+  it("typeText routes through injected driver.type", async () => {
+    const { iface, calls } = makeIface();
+    await iface.typeText({ text: "hello world" });
+    expect(calls.type).toEqual([["hello world"]]);
+  });
+
+  it("hotkey joins keys with + and routes through driver.keyCombo", async () => {
+    const { iface, calls } = makeIface();
+    await iface.hotkey({ keys: ["ctrl", "shift", "p"] });
+    expect(calls.keyCombo).toEqual([["ctrl+shift+p"]]);
+  });
+
+  it("pressKey routes through driver.keyPress", async () => {
+    const { iface, calls } = makeIface();
+    await iface.pressKey({ key: "Enter" });
+    expect(calls.keyPress).toEqual([["Enter"]]);
+  });
+
+  it("scroll splits vertical + horizontal deltas into driver calls", async () => {
+    const { iface, calls } = makeIface();
+    await iface.scroll({ displayId: 0, x: 100, y: 100, dx: 3, dy: -2 });
+    // dy first (up 2), then dx (right 3).
+    expect(calls.scroll).toEqual([
+      [100, 100, "up", 2],
+      [100, 100, "right", 3],
+    ]);
+  });
+
+  it("drag routes the full polyline through driver.dragPath (M8)", async () => {
+    const { iface, calls } = makeIface();
+    await iface.drag({
+      displayId: 0,
+      path: [
+        { x: 10, y: 10 },
+        { x: 100, y: 100 },
+        { x: 100, y: 200 },
+      ],
+    });
+    // Every vertex is forwarded (global coords; primary origin is 0,0 here).
+    expect(calls.dragPath).toEqual([
+      [
+        [
+          { x: 10, y: 10 },
+          { x: 100, y: 100 },
+          { x: 100, y: 200 },
+        ],
+      ],
+    ]);
+    // A multi-point path does not collapse into a single start→end driver.drag.
+    expect(calls.drag).toHaveLength(0);
+  });
+
+  it("dragTo still routes start + end through driver.drag", async () => {
+    const { iface, calls } = makeIface();
+    await iface.moveCursor({ displayId: 0, x: 10, y: 10 });
+    await iface.dragTo({ displayId: 0, x: 100, y: 100 });
+    expect(calls.drag).toEqual([[10, 10, 100, 100]]);
+  });
+
+  it("mouseDown / mouseUp route real press-hold through the driver (M8)", async () => {
+    const { iface, calls } = makeIface();
+    await iface.mouseDown({ displayId: 0, x: 30, y: 40 });
+    await iface.mouseUp({ displayId: 0, x: 30, y: 40, button: "middle" });
+    expect(calls.mouseDown).toEqual([[30, 40, "left"]]);
+    expect(calls.mouseUp).toEqual([[30, 40, "middle"]]);
+    // Cursor state tracks the press location.
+    expect(iface.getCursorPosition()).toMatchObject({ x: 30, y: 40 });
+  });
+
+  it("keyDown / keyUp route real key press-hold through the driver (M8)", async () => {
+    const { iface, calls } = makeIface();
+    await iface.keyDown({ key: "shift" });
+    await iface.keyUp({ key: "shift" });
+    expect(calls.keyDown).toEqual([["shift"]]);
+    expect(calls.keyUp).toEqual([["shift"]]);
+  });
+
+  it("drag fails on cross-display dragTo", async () => {
+    // Drag-across-displays check fires before the coord translation.
+    // Mutate cursorState to a synthetic non-zero display so we can hit it
+    // without needing a real second display on the host.
+    const cursorState = { current: { displayId: 0, x: 5, y: 5 } };
+    const { iface } = makeIface({ cursorState });
+    cursorState.current = { displayId: 7, x: 5, y: 5 };
+    await expect(iface.dragTo({ displayId: 0, x: 5, y: 5 })).rejects.toThrow(
+      /across displays/,
+    );
+  });
+
+  it("drag rejects single-point paths", async () => {
+    const { iface } = makeIface();
+    await expect(
+      iface.drag({ displayId: 0, path: [{ x: 1, y: 1 }] }),
+    ).rejects.toThrow(/at least two points/);
+  });
+
+  it("rejects unknown displayId on click", async () => {
+    const { iface } = makeIface();
+    await expect(
+      iface.leftClick({ displayId: 99, x: 0, y: 0 }),
+    ).rejects.toThrow(/unknown displayId 99/);
+  });
+
+  it("rejects non-finite coords on click", async () => {
+    const { iface } = makeIface();
+    await expect(
+      iface.leftClick({ displayId: 0, x: Number.NaN, y: 0 }),
+    ).rejects.toThrow(/non-finite coords/);
+  });
+
+  it("getScreenSize returns the display's logical bounds", () => {
+    const { iface } = makeIface();
+    expect(iface.getScreenSize({ displayId: 0 })).toEqual({ w: 1920, h: 1080 });
+    expect(iface.getScreenSize({ displayId: 1 })).toEqual({ w: 2560, h: 1440 });
+  });
+
+  it("getCursorPosition tracks the last successful movement", async () => {
+    const { iface } = makeIface();
+    await iface.moveCursor({ displayId: 0, x: 42, y: 84 });
+    expect(iface.getCursorPosition()).toMatchObject({
+      displayId: 0,
+      x: 42,
+      y: 84,
+    });
+  });
+
+  it("hotkey requires at least one key", async () => {
+    const { iface } = makeIface();
+    await expect(iface.hotkey({ keys: [] })).rejects.toThrow(
+      /at least one key/,
+    );
+  });
+
+  it("getAccessibilityTree returns the current scene's AX nodes, filterable by display", () => {
+    const { iface } = makeIface({
+      getScene: () => ({
+        timestamp: 0,
+        displays: fakeDisplays(),
+        focused_window: null,
+        apps: [],
+        ocr: [],
+        ax: [
+          {
+            id: "a0-1",
+            role: "button",
+            bbox: [0, 0, 10, 10],
+            actions: [],
+            displayId: 0,
+          },
+          {
+            id: "a1-1",
+            role: "button",
+            bbox: [0, 0, 10, 10],
+            actions: [],
+            displayId: 1,
+          },
+        ],
+        vlm_scene: null,
+        vlm_elements: null,
+      }),
+    });
+    expect(iface.getAccessibilityTree({})).toHaveLength(2);
+    expect(iface.getAccessibilityTree({ displayId: 0 })).toHaveLength(1);
+    expect(iface.getAccessibilityTree({ displayId: 0 })[0]?.id).toBe("a0-1");
+  });
+});
+
+describe("makeComputerInterface factory", () => {
+  it("returns a DefaultComputerInterface instance", () => {
+    const iface = makeComputerInterface({ listDisplays: () => fakeDisplays() });
+    expect(iface).toBeInstanceOf(DefaultComputerInterface);
+  });
+});

plugins/plugin-computeruse/src/__tests__/computer-use-agent.real.test.ts

@@ -0,0 +1,134 @@
+/**
+ * WS7 — Linux end-to-end agent loop test.
+ *
+ * Skipped by default (matches `*.real.test.ts` in `vitest.config.ts`). Run
+ * explicitly with:
+ *
+ *   bunx vitest run src/__tests__/computer-use-agent.real.test.ts --reporter=default \
+ *     --config=<(echo "export default { test: { include: ['src/__tests__/computer-use-agent.real.test.ts'] } }")
+ *
+ * Or via the simpler form:
+ *
+ *   bun x vitest run --reporter=default --testPathIgnorePatterns='[]' \
+ *     src/__tests__/computer-use-agent.real.test.ts
+ *
+ * What it asserts:
+ *   - On a Linux host with `xrandr` / `import` (ImageMagick) or `scrot`
+ *     installed, `runComputerUseAgentLoop` captures the live screen,
+ *     marshals it through a fake Brain, walks two steps (`wait` →
+ *     `finish`), and terminates cleanly with `reason: "finish"`.
+ *   - No NVIDIA GPU required — the Brain is replaced via the
+ *     `Brain.invokeModel` injection point.
+ */
+
+import { describe, expect, it } from "vitest";
+import {
+  type ComputerUseAgentReport,
+  runComputerUseAgentLoop,
+} from "../actions/use-computer-agent.js";
+import { Brain } from "../actor/brain.js";
+import type { DisplayCapture } from "../platform/capture.js";
+import { captureAllDisplays } from "../platform/capture.js";
+import { listDisplays } from "../platform/displays.js";
+import type { Scene, SceneOcrBox } from "../scene/scene-types.js";
+import type { ComputerUseService } from "../services/computer-use-service.js";
+
+function fakeScene(): Scene {
+  const ds = listDisplays();
+  const ocr: SceneOcrBox[] = [
+    {
+      id: "t0-1",
+      text: "Hello",
+      bbox: [100, 100, 80, 24],
+      conf: 0.9,
+      displayId: ds[0]?.id ?? 0,
+    },
+  ];
+  return {
+    timestamp: Date.now(),
+    displays: ds.map((d) => ({
+      id: d.id,
+      bounds: d.bounds,
+      scaleFactor: d.scaleFactor,
+      primary: d.primary,
+      name: d.name,
+    })),
+    focused_window: null,
+    apps: [],
+    ocr,
+    ax: [],
+    vlm_scene: null,
+    vlm_elements: null,
+  };
+}
+
+function fixtureService(): ComputerUseService {
+  return {
+    getCurrentScene: () => fakeScene(),
+    refreshScene: async () => fakeScene(),
+    getDisplays: () =>
+      listDisplays().map((d) => ({
+        id: d.id,
+        bounds: d.bounds,
+        scaleFactor: d.scaleFactor,
+        primary: d.primary,
+        name: d.name,
+      })),
+  } as unknown as ComputerUseService;
+}
+
+describe("Computer-use agent — real Linux end-to-end with fixture VLM", () => {
+  it("captures live screen frames, walks wait→finish, terminates cleanly", async () => {
+    // Try a real capture once. If the host has no screen tools this is unavailable,
+    // since the env is the constraint, not the cascade logic.
+    let captures: DisplayCapture[] = [];
+    try {
+      captures = await captureAllDisplays();
+    } catch (err) {
+      console.warn(
+        `[computer-use-agent.real] live capture failed (${String(err)}); real-screen E2E unavailable`,
+      );
+      return;
+    }
+    expect(captures.length).toBeGreaterThan(0);
+    // PNG signature on the first capture proves it's a real frame.
+    expect(captures[0]?.frame.subarray(0, 4)).toEqual(
+      Buffer.from([0x89, 0x50, 0x4e, 0x47]),
+    );
+
+    let step = 0;
+    const brain = new Brain(null, {
+      invokeModel: async () => {
+        step += 1;
+        if (step === 1) {
+          return JSON.stringify({
+            scene_summary: "screen captured",
+            target_display_id: captures[0]?.display.id,
+            roi: [],
+            proposed_action: { kind: "wait", rationale: "wait one tick" },
+          });
+        }
+        return JSON.stringify({
+          scene_summary: "done",
+          target_display_id: captures[0]?.display.id,
+          roi: [],
+          proposed_action: { kind: "finish", rationale: "goal reached" },
+        });
+      },
+    });
+
+    const report: ComputerUseAgentReport = await runComputerUseAgentLoop(
+      null,
+      { goal: "wait then finish", maxSteps: 5 },
+      fixtureService(),
+      { brain, captureAll: async () => captures },
+    );
+    expect(report.reason).toBe("finish");
+    expect(report.finished).toBe(true);
+    expect(report.steps.length).toBe(2);
+    expect(report.steps[0]?.actionKind).toBe("wait");
+    expect(report.steps[1]?.actionKind).toBe("finish");
+    expect(report.steps[0]?.result.success).toBe(true);
+    expect(report.steps[1]?.result.success).toBe(true);
+  }, 60_000);
+});

plugins/plugin-computeruse/src/__tests__/computer-use-agent.test.ts

@@ -0,0 +1,536 @@
+/**
+ * WS7 — Agent loop integration test (no live screen).
+ *
+ * Drives `runComputerUseAgentLoop` with fully synthetic deps: a fake Brain,
+ * a fake `captureAll` that emits a hand-rolled PNG, and a fake service that
+ * returns a deterministic Scene. Asserts:
+ *
+ *   - Loop terminates on `finish` and reports `reason: "finish"`.
+ *   - Loop terminates on `maxSteps` after that many turns when the Brain
+ *     keeps emitting `wait`.
+ *   - Cascade errors surface as `reason: "error"`, not exceptions.
+ *   - Dispatch failures (out-of-bounds, etc.) abort the loop.
+ *   - Each step's `result.success` mirrors the dispatch outcome.
+ *
+ * This is the in-suite counterpart to `computer-use-agent.real.test.ts`,
+ * which exercises the live capture path on a Linux host (skipped by
+ * default).
+ */
+
+import type { Content } from "@elizaos/core";
+import { describe, expect, it } from "vitest";
+import {
+  type ComputerUseAgentReport,
+  type ComputerUseAgentStepProgress,
+  formatComputerUseAgentProgress,
+  runComputerUseAgentLoop,
+} from "../actions/use-computer-agent.js";
+import { Brain } from "../actor/brain.js";
+import type { DisplayCapture } from "../platform/capture.js";
+import type { Scene } from "../scene/scene-types.js";
+import type { ComputerUseService } from "../services/computer-use-service.js";
+import type { DisplayDescriptor } from "../types.js";
+
+function display(): DisplayDescriptor {
+  return {
+    id: 0,
+    bounds: [0, 0, 1920, 1080],
+    scaleFactor: 1,
+    primary: true,
+    name: "fake",
+  };
+}
+
+function syntheticScene(): Scene {
+  return {
+    timestamp: Date.now(),
+    displays: [display()],
+    focused_window: null,
+    apps: [],
+    ocr: [
+      {
+        id: "t0-1",
+        text: "Save",
+        bbox: [100, 100, 80, 32],
+        conf: 0.97,
+        displayId: 0,
+      },
+    ],
+    ax: [],
+    vlm_scene: null,
+    vlm_elements: null,
+  };
+}
+
+function fakeService(refresh?: () => Promise<Scene>): ComputerUseService {
+  return {
+    getCurrentScene: () => syntheticScene(),
+    refreshScene: refresh ?? (async () => syntheticScene()),
+    getDisplays: () => [display()],
+    setSceneVlmAnnotations: () => {},
+  } as unknown as ComputerUseService;
+}
+
+function fakeCaptures(): Map<number, DisplayCapture> {
+  const m = new Map<number, DisplayCapture>();
+  m.set(0, {
+    display: { ...display(), id: 0 },
+    frame: Buffer.concat([
+      Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
+      Buffer.alloc(64, 0),
+    ]),
+  });
+  return m;
+}
+
+async function captureAll(): Promise<DisplayCapture[]> {
+  return Array.from(fakeCaptures().values());
+}
+
+describe("runComputerUseAgentLoop — fake Brain", () => {
+  it("terminates cleanly on `finish`", async () => {
+    const brain = new Brain(null, {
+      invokeModel: async () =>
+        JSON.stringify({
+          scene_summary: "done",
+          target_display_id: 0,
+          roi: [],
+          proposed_action: { kind: "finish", rationale: "ok" },
+        }),
+    });
+    const report = await runComputerUseAgentLoop(
+      null,
+      { goal: "g" },
+      fakeService(),
+      { brain, captureAll },
+    );
+    expect(report.reason).toBe("finish");
+    expect(report.finished).toBe(true);
+    expect(report.steps.length).toBe(1);
+    expect(report.steps[0]?.actionKind).toBe("finish");
+  });
+
+  it("records a trajectory on the report via the default middleware (#9170 M11)", async () => {
+    const brain = new Brain(null, {
+      invokeModel: async () =>
+        JSON.stringify({
+          scene_summary: "done",
+          target_display_id: 0,
+          roi: [],
+          proposed_action: { kind: "finish", rationale: "all set" },
+        }),
+    });
+    const report = await runComputerUseAgentLoop(
+      null,
+      { goal: "g" },
+      fakeService(),
+      { brain, captureAll },
+    );
+    expect(report.trajectory).toHaveLength(1);
+    expect(report.trajectory?.[0]).toMatchObject({
+      step: 1,
+      actionKind: "finish",
+      success: true,
+    });
+  });
+
+  it("aborts on the wall-clock budget before any step (#9170 M11)", async () => {
+    const brain = new Brain(null, {
+      invokeModel: async () => {
+        throw new Error("brain should not be called once budget is blown");
+      },
+    });
+    // now(): first call = run start (0), second = beforeStep elapsed (5000ms).
+    let t = 0;
+    const ticks = [0, 5000];
+    const now = () => ticks[Math.min(t++, ticks.length - 1)] ?? 5000;
+    const report = await runComputerUseAgentLoop(
+      null,
+      { goal: "g", maxDurationMs: 100 },
+      fakeService(),
+      { brain, captureAll, now },
+    );
+    expect(report.reason).toBe("budget");
+    expect(report.steps.length).toBe(0);
+    expect(report.error).toContain("time budget");
+  });
+
+  it("emits per-step callback content when streamProgress is true", async () => {
+    const progress: Content[] = [];
+    const brain = new Brain(null, {
+      invokeModel: async () =>
+        JSON.stringify({
+          scene_summary: "done",
+          target_display_id: 0,
+          roi: [],
+          proposed_action: { kind: "finish", rationale: "ok" },
+        }),
+    });
+
+    const report = await runComputerUseAgentLoop(
+      null,
+      { goal: "g", streamProgress: true },
+      fakeService(),
+      {
+        brain,
+        captureAll,
+        onCompactStepProgress: (content) => {
+          progress.push(content);
+        },
+      },
+    );
+
+    expect(report.reason).toBe("finish");
+    expect(progress).toHaveLength(1);
+    expect(progress[0]).toMatchObject({
+      text: "Step 1: finish — ok",
+      source: "action_progress",
+      merge: "replace",
+      metadata: {
+        transient: true,
+        compactProgress: true,
+        progress: {
+          source: "computeruse",
+          actionName: "COMPUTER_USE_AGENT",
+          step: 1,
+          kind: "finish",
+          rationale: "ok",
+          success: true,
+        },
+      },
+    });
+  });
+
+  it("keeps the loop result when per-step callback delivery fails", async () => {
+    const brain = new Brain(null, {
+      invokeModel: async () =>
+        JSON.stringify({
+          scene_summary: "done",
+          target_display_id: 0,
+          roi: [],
+          proposed_action: { kind: "finish", rationale: "ok" },
+        }),
+    });
+
+    const report = await runComputerUseAgentLoop(
+      null,
+      { goal: "g", streamProgress: true },
+      fakeService(),
+      {
+        brain,
+        captureAll,
+        onCompactStepProgress: () => {
+          throw new Error("send failed");
+        },
+      },
+    );
+
+    expect(report.reason).toBe("finish");
+    expect(report.steps[0]?.actionKind).toBe("finish");
+  });
+
+  it("keeps per-step callbacks behind the streamProgress flag", async () => {
+    const progress: Content[] = [];
+    const brain = new Brain(null, {
+      invokeModel: async () =>
+        JSON.stringify({
+          scene_summary: "done",
+          target_display_id: 0,
+          roi: [],
+          proposed_action: { kind: "finish", rationale: "ok" },
+        }),
+    });
+
+    await runComputerUseAgentLoop(null, { goal: "g" }, fakeService(), {
+      brain,
+      captureAll,
+      onCompactStepProgress: (content) => {
+        progress.push(content);
+      },
+    });
+
+    expect(progress).toEqual([]);
+  });
+
+  it("hits maxSteps when the Brain keeps emitting wait", async () => {
+    const brain = new Brain(null, {
+      invokeModel: async () =>
+        JSON.stringify({
+          scene_summary: "still loading",
+          target_display_id: 0,
+          roi: [],
+          proposed_action: { kind: "wait", rationale: "..." },
+        }),
+    });
+    const report: ComputerUseAgentReport = await runComputerUseAgentLoop(
+      null,
+      { goal: "g", maxSteps: 3 },
+      fakeService(),
+      { brain, captureAll },
+    );
+    expect(report.reason).toBe("max_steps");
+    expect(report.finished).toBe(false);
+    expect(report.steps.length).toBe(3);
+  });
+
+  it("emits a per-step progress callback when streamProgress is set (#8912)", async () => {
+    const brain = new Brain(null, {
+      invokeModel: async () =>
+        JSON.stringify({
+          scene_summary: "still loading",
+          target_display_id: 0,
+          roi: [],
+          proposed_action: { kind: "wait", rationale: "waiting for page" },
+        }),
+    });
+    const progress: ComputerUseAgentStepProgress[] = [];
+    const report = await runComputerUseAgentLoop(
+      null,
+      { goal: "g", maxSteps: 3, streamProgress: true },
+      fakeService(),
+      { brain, captureAll, onStepProgress: (p) => void progress.push(p) },
+    );
+    // One callback per dispatched step, in order, carrying kind + rationale.
+    expect(progress.length).toBe(report.steps.length);
+    expect(progress.length).toBeGreaterThanOrEqual(1);
+    expect(progress[0]).toMatchObject({
+      step: 1,
+      maxSteps: 3,
+      actionKind: "wait",
+      rationale: "waiting for page",
+      result: { success: true },
+    });
+  });
+
+  it("does not call progress callback when streamProgress is unset", async () => {
+    const brain = new Brain(null, {
+      invokeModel: async () =>
+        JSON.stringify({
+          scene_summary: "done",
+          target_display_id: 0,
+          roi: [],
+          proposed_action: { kind: "finish", rationale: "ok" },
+        }),
+    });
+    let calls = 0;
+    await runComputerUseAgentLoop(null, { goal: "g" }, fakeService(), {
+      brain,
+      captureAll,
+      onStepProgress: () => {
+        calls += 1;
+      },
+    });
+    expect(calls).toBe(0);
+  });
+
+  it("surfaces cascade failures as `reason: error` instead of throwing", async () => {
+    // Brain emits a click with no ref + no roi → cascade can't resolve it.
+    const brain = new Brain(null, {
+      invokeModel: async () =>
+        JSON.stringify({
+          scene_summary: "S",
+          target_display_id: 0,
+          roi: [],
+          proposed_action: { kind: "click", rationale: "where?" },
+        }),
+    });
+    const report = await runComputerUseAgentLoop(
+      null,
+      { goal: "g" },
+      fakeService(),
+      { brain, captureAll },
+    );
+    expect(report.reason).toBe("error");
+    // The loop wraps Brain→Cascade as the default "local-grounder" loop
+    // (#9170 M10); planning/grounding failures surface under that loop name.
+    expect(report.error).toContain('agent loop "local-grounder" failed');
+  });
+
+  it("aborts on dispatch error (out-of-bounds)", async () => {
+    const brain = new Brain(null, {
+      invokeModel: async () =>
+        JSON.stringify({
+          scene_summary: "S",
+          target_display_id: 0,
+          roi: [
+            {
+              displayId: 0,
+              bbox: [9_000, 9_000, 10, 10],
+              reason: "off-screen",
+            },
+          ],
+          proposed_action: { kind: "click", rationale: "click out-of-bounds" },
+        }),
+    });
+    const report = await runComputerUseAgentLoop(
+      null,
+      { goal: "g" },
+      fakeService(),
+      { brain, captureAll },
+    );
+    expect(report.reason).toBe("error");
+    expect(report.steps.length).toBe(1);
+    expect(report.steps[0]?.result.success).toBe(false);
+    expect(report.error).toMatch(/outside display/);
+  });
+
+  it("aborts on scene refresh error", async () => {
+    const brain = new Brain(null, {
+      invokeModel: async () => "{}",
+    });
+    const report = await runComputerUseAgentLoop(
+      null,
+      { goal: "g" },
+      fakeService(async () => {
+        throw new Error("scene-broken");
+      }),
+      { brain, captureAll },
+    );
+    expect(report.reason).toBe("error");
+    expect(report.error).toContain("scene-broken");
+  });
+
+  it("aborts when no displays can be captured", async () => {
+    const brain = new Brain(null, {
+      invokeModel: async () => "{}",
+    });
+    const report = await runComputerUseAgentLoop(
+      null,
+      { goal: "g" },
+      fakeService(),
+      {
+        brain,
+        captureAll: async () => [],
+      },
+    );
+    expect(report.reason).toBe("error");
+    expect(report.error).toBe("no displays captured");
+  });
+
+  it("clamps maxSteps to [1, 20]", async () => {
+    const brain = new Brain(null, {
+      invokeModel: async () =>
+        JSON.stringify({
+          scene_summary: "S",
+          target_display_id: 0,
+          roi: [],
+          proposed_action: { kind: "wait", rationale: "" },
+        }),
+    });
+    const r1 = await runComputerUseAgentLoop(
+      null,
+      { goal: "g", maxSteps: 0 },
+      fakeService(),
+      { brain, captureAll },
+    );
+    expect(r1.steps.length).toBe(1);
+    const r2 = await runComputerUseAgentLoop(
+      null,
+      { goal: "g", maxSteps: 100 },
+      fakeService(),
+      { brain, captureAll },
+    );
+    expect(r2.steps.length).toBe(20);
+  });
+
+  it("emits opt-in step progress after each dispatched step", async () => {
+    let step = 0;
+    const brain = new Brain(null, {
+      invokeModel: async () => {
+        step += 1;
+        if (step === 1) {
+          return JSON.stringify({
+            scene_summary: "waiting for modal",
+            target_display_id: 0,
+            roi: [],
+            proposed_action: {
+              kind: "wait",
+              rationale: "wait for the modal to settle",
+            },
+          });
+        }
+        return JSON.stringify({
+          scene_summary: "done",
+          target_display_id: 0,
+          roi: [],
+          proposed_action: { kind: "finish", rationale: "goal reached" },
+        });
+      },
+    });
+    const progress: ComputerUseAgentStepProgress[] = [];
+    const report = await runComputerUseAgentLoop(
+      null,
+      { goal: "watch the screen", maxSteps: 5, streamProgress: true },
+      fakeService(),
+      {
+        brain,
+        captureAll,
+        onStepProgress: (event) => {
+          progress.push(event);
+        },
+      },
+    );
+
+    expect(report.reason).toBe("finish");
+    expect(progress).toHaveLength(2);
+    expect(progress[0]).toMatchObject({
+      goal: "watch the screen",
+      step: 1,
+      maxSteps: 5,
+      actionKind: "wait",
+      rationale: "wait for the modal to settle",
+      result: { success: true },
+    });
+    const firstProgress = progress[0];
+    expect(firstProgress).toBeDefined();
+    if (!firstProgress) {
+      throw new Error("expected first progress event");
+    }
+    expect(formatComputerUseAgentProgress(firstProgress)).toBe(
+      "Step 1/5: wait - wait for the modal to settle",
+    );
+  });
+
+  it("emits a failed step progress event before aborting", async () => {
+    const brain = new Brain(null, {
+      invokeModel: async () =>
+        JSON.stringify({
+          scene_summary: "S",
+          target_display_id: 0,
+          roi: [
+            {
+              displayId: 0,
+              bbox: [9_000, 9_000, 10, 10],
+              reason: "off-screen",
+            },
+          ],
+          proposed_action: { kind: "click", rationale: "click out-of-bounds" },
+        }),
+    });
+    const progress: ComputerUseAgentStepProgress[] = [];
+    const report = await runComputerUseAgentLoop(
+      null,
+      { goal: "click", maxSteps: 2, streamProgress: true },
+      fakeService(),
+      {
+        brain,
+        captureAll,
+        onStepProgress: (event) => {
+          progress.push(event);
+        },
+      },
+    );
+
+    expect(report.reason).toBe("error");
+    expect(progress).toHaveLength(1);
+    expect(progress[0]?.result.success).toBe(false);
+    const firstProgress = progress[0];
+    expect(firstProgress).toBeDefined();
+    if (!firstProgress) {
+      throw new Error("expected failed progress event");
+    }
+    expect(formatComputerUseAgentProgress(firstProgress)).toContain(
+      "failed: Coordinates",
+    );
+  });
+});

plugins/plugin-computeruse/src/__tests__/computer-use-approval-relay.test.ts

@@ -0,0 +1,222 @@
+/**
+ * COMPUTER_USE action's approval-relay wiring — pending-approval snapshots reach
+ * the message callback. Deterministic; no real desktop.
+ */
+import { describe, expect, it, vi } from "vitest";
+import { useComputerAction } from "../actions/use-computer.js";
+import type { ApprovalSnapshot } from "../types.js";
+
+function emptySnapshot(): ApprovalSnapshot {
+  return {
+    mode: "approve_all",
+    pendingCount: 0,
+    pendingApprovals: [],
+  };
+}
+
+describe("COMPUTER_USE approval relay", () => {
+  it("posts approve/deny inline choices to the action callback", async () => {
+    const listeners: Array<(snapshot: ApprovalSnapshot) => void> = [];
+    const unsubscribe = vi.fn();
+    const approval = {
+      id: "approval_123",
+      command: "desktop_click",
+      parameters: { action: "click", coordinate: [10, 20] },
+      requestedAt: "2026-06-22T12:00:00.000Z",
+    };
+    const service = {
+      getApprovalSnapshot: vi.fn(() => emptySnapshot()),
+      subscribeApprovals: vi.fn(
+        (listener: (snapshot: ApprovalSnapshot) => void) => {
+          listeners.push(listener);
+          listener(emptySnapshot());
+          return unsubscribe;
+        },
+      ),
+      executeDesktopAction: vi.fn(async () => {
+        listeners[0]?.({
+          mode: "approve_all",
+          pendingCount: 1,
+          pendingApprovals: [approval],
+        });
+        return { success: true, message: "Clicked." };
+      }),
+    };
+    const runtime = {
+      getService: vi.fn((name: string) =>
+        name === "computeruse" ? service : null,
+      ),
+    };
+    const callback = vi.fn(async () => []);
+
+    const result = await useComputerAction.handler?.(
+      runtime as never,
+      { content: { text: "" } } as never,
+      undefined,
+      {
+        parameters: {
+          action: "click",
+          coordinate: [10, 20],
+          displayId: 0,
+        },
+      } as never,
+      callback,
+    );
+
+    expect(result?.success).toBe(true);
+    expect(unsubscribe).toHaveBeenCalledTimes(1);
+    const approvalCall = callback.mock.calls.find(
+      ([, actionName]) => actionName === "COMPUTER_USE_APPROVAL",
+    );
+    expect(approvalCall?.[0]).toMatchObject({
+      source: "computeruse_approval",
+      text: expect.stringContaining(
+        "[CHOICE:computeruse-approval id=approval_123]",
+      ),
+    });
+    expect(approvalCall?.[0].text).toContain(
+      "cua:approval_123:approve=Approve",
+    );
+    expect(approvalCall?.[0].text).toContain("cua:approval_123:deny=Deny");
+    expect(callback).toHaveBeenLastCalledWith({ text: "Clicked." });
+  });
+
+  it("includes the Telegram requester id in approval callbacks when available", async () => {
+    const listeners: Array<(snapshot: ApprovalSnapshot) => void> = [];
+    const approval = {
+      id: "approval_123",
+      command: "desktop_click",
+      parameters: { action: "click", coordinate: [10, 20] },
+      requestedAt: "2026-06-22T12:00:00.000Z",
+    };
+    const service = {
+      getApprovalSnapshot: vi.fn(() => emptySnapshot()),
+      subscribeApprovals: vi.fn(
+        (listener: (snapshot: ApprovalSnapshot) => void) => {
+          listeners.push(listener);
+          return vi.fn();
+        },
+      ),
+      executeDesktopAction: vi.fn(async () => {
+        listeners[0]?.({
+          mode: "approve_all",
+          pendingCount: 1,
+          pendingApprovals: [approval],
+        });
+        return { success: true, message: "Clicked." };
+      }),
+    };
+    const runtime = {
+      getService: vi.fn((name: string) =>
+        name === "computeruse" ? service : null,
+      ),
+    };
+    const callback = vi.fn(async () => []);
+
+    await useComputerAction.handler?.(
+      runtime as never,
+      { content: { text: "" }, metadata: { telegramUserId: "42" } } as never,
+      undefined,
+      {
+        parameters: {
+          action: "click",
+          coordinate: [10, 20],
+          displayId: 0,
+        },
+      } as never,
+      callback,
+    );
+
+    const approvalCall = callback.mock.calls.find(
+      ([, actionName]) => actionName === "COMPUTER_USE_APPROVAL",
+    );
+    expect(approvalCall?.[0].text).toContain(
+      "cua:approval_123:approve:u42=Approve",
+    );
+    expect(approvalCall?.[0].text).toContain("cua:approval_123:deny:u42=Deny");
+  });
+
+  it("relays approval prompts while the desktop action is still pending", async () => {
+    const listeners: Array<(snapshot: ApprovalSnapshot) => void> = [];
+    const approval = {
+      id: "approval_123",
+      command: "desktop_click",
+      parameters: { action: "click", coordinate: [10, 20] },
+      requestedAt: "2026-06-22T12:00:00.000Z",
+    };
+    let finishAction:
+      | ((result: {
+          success: boolean;
+          message?: string;
+          error?: string;
+        }) => void)
+      | undefined;
+    const service = {
+      getApprovalSnapshot: vi.fn(() => emptySnapshot()),
+      subscribeApprovals: vi.fn(
+        (listener: (snapshot: ApprovalSnapshot) => void) => {
+          listeners.push(listener);
+          return vi.fn();
+        },
+      ),
+      executeDesktopAction: vi.fn(async () => {
+        listeners[0]?.({
+          mode: "approve_all",
+          pendingCount: 1,
+          pendingApprovals: [approval],
+        });
+        return await new Promise<{
+          success: boolean;
+          message?: string;
+          error?: string;
+        }>((resolve) => {
+          finishAction = resolve;
+        });
+      }),
+    };
+    const runtime = {
+      getService: vi.fn((name: string) =>
+        name === "computeruse" ? service : null,
+      ),
+    };
+    const callback = vi.fn(async () => []);
+
+    let completed = false;
+    const resultPromise = useComputerAction
+      .handler?.(
+        runtime as never,
+        { content: { text: "" } } as never,
+        undefined,
+        {
+          parameters: {
+            action: "click",
+            coordinate: [10, 20],
+            displayId: 0,
+          },
+        } as never,
+        callback,
+      )
+      .then((result) => {
+        completed = true;
+        return result;
+      });
+
+    await Promise.resolve();
+    await Promise.resolve();
+
+    expect(completed).toBe(false);
+    expect(
+      callback.mock.calls.some(
+        ([content, actionName]) =>
+          actionName === "COMPUTER_USE_APPROVAL" &&
+          String(content.text).includes("approval_123"),
+      ),
+    ).toBe(true);
+
+    finishAction?.({ success: true, message: "Clicked." });
+    const result = await resultPromise;
+
+    expect(result?.success).toBe(true);
+    expect(callback).toHaveBeenLastCalledWith({ text: "Clicked." });
+  });
+});

plugins/plugin-computeruse/src/__tests__/computeruse.real.test.ts

@@ -0,0 +1,430 @@
+/**
+ * Real computer-use lane: drives useComputerAction/windowAction through a live
+ * ComputerUseService and the real platform driver, asserting non-blank
+ * screenshots. Skips where no headful display is available.
+ */
+import { mkdtemp, readFile, rm } from "node:fs/promises";
+import os from "node:os";
+import path from "node:path";
+import type { IAgentRuntime } from "@elizaos/core";
+import { afterEach, beforeEach, describe, expect, it } from "vitest";
+import { assertScreenshotBase64NotBlank } from "../../test/helpers/screenshot-quality.ts";
+import { useComputerAction } from "../actions/use-computer.js";
+import { windowAction } from "../actions/window.js";
+import computerUsePlugin from "../index.js";
+import { ComputerUseService } from "../services/computer-use-service.js";
+
+function createRuntime(settings: Record<string, string> = {}): IAgentRuntime {
+  return {
+    character: {},
+    getSetting(key: string) {
+      return settings[key];
+    },
+  } as IAgentRuntime;
+}
+
+async function waitForPendingApproval(
+  service: ComputerUseService,
+): Promise<string> {
+  const startedAt = Date.now();
+  while (Date.now() - startedAt < 5000) {
+    const snapshot = service.getApprovalSnapshot();
+    if (snapshot.pendingApprovals[0]) {
+      return snapshot.pendingApprovals[0].id;
+    }
+    await new Promise((resolve) => setTimeout(resolve, 50));
+  }
+  throw new Error("Timed out waiting for a pending approval.");
+}
+
+describe("computer-use live parity", () => {
+  let workspaceDir = "";
+  let service: ComputerUseService;
+
+  beforeEach(async () => {
+    workspaceDir = await mkdtemp(path.join(os.tmpdir(), "computeruse-live-"));
+    service = (await ComputerUseService.start(
+      createRuntime({
+        COMPUTER_USE_APPROVAL_MODE: "full_control",
+      }),
+    )) as ComputerUseService;
+  });
+
+  afterEach(async () => {
+    await service.stop();
+    await rm(workspaceDir, { recursive: true, force: true });
+  });
+
+  it("exports the full public action surface", () => {
+    const actionNames = computerUsePlugin.actions?.map((action) => action.name);
+    expect(actionNames).toContain("COMPUTER_USE");
+    expect(actionNames).toContain("WINDOW");
+    expect(actionNames).not.toContain("DESKTOP");
+    expect(useComputerAction.similes).toContain("USE_COMPUTER");
+    expect(useComputerAction.roleGate).toMatchObject({ minRole: "OWNER" });
+    expect(windowAction.similes).toEqual(
+      expect.arrayContaining(["MANAGE_WINDOW", "WINDOW", "USE_WINDOW"]),
+    );
+  });
+
+  it("publishes the COMPUTER_USE desktop action surface and WINDOW action enum", () => {
+    const desktopActions = useComputerAction.parameters?.find(
+      (parameter) => parameter.name === "action",
+    );
+    expect(desktopActions?.schema).toMatchObject({
+      enum: expect.arrayContaining([
+        "screenshot",
+        "click",
+        "click_with_modifiers",
+        "double_click",
+        "right_click",
+        "mouse_move",
+        "type",
+        "key",
+        "key_combo",
+        "scroll",
+        "drag",
+        "detect_elements",
+        "ocr",
+      ]),
+    });
+
+    const windowOp = windowAction.parameters?.find(
+      (parameter) => parameter.name === "action",
+    );
+    expect(windowOp?.schema).toMatchObject({
+      enum: expect.arrayContaining([
+        "list",
+        "focus",
+        "switch",
+        "arrange",
+        "move",
+        "minimize",
+        "maximize",
+        "restore",
+        "close",
+      ]),
+    });
+  });
+
+  it("supports file commands and upstream path/edit aliases live", async () => {
+    const filePath = path.join(workspaceDir, "notes.txt");
+    const subdir = path.join(workspaceDir, "subdir");
+    const nestedFilePath = path.join(subdir, "nested.txt");
+
+    const writeResult = await service.executeCommand("file_write", {
+      filepath: filePath,
+      content: "hello",
+    });
+    expect(writeResult.success).toBe(true);
+
+    const editResult = await service.executeCommand("file_edit", {
+      filepath: filePath,
+      find: "hello",
+      replace: "world",
+    });
+    expect(editResult.success).toBe(true);
+
+    const appendResult = await service.executeCommand("file_append", {
+      path: filePath,
+      content: "!",
+    });
+    expect(appendResult.success).toBe(true);
+
+    const readResult = await service.executeCommand("file_read", {
+      path: filePath,
+    });
+    expect(readResult.success).toBe(true);
+    expect(readResult).toMatchObject({ content: "world!" });
+
+    const existsResult = await service.executeCommand("file_exists", {
+      path: filePath,
+    });
+    expect(existsResult).toMatchObject({
+      success: true,
+      exists: true,
+      isFile: true,
+    });
+
+    const listResult = await service.executeCommand("directory_list", {
+      dirpath: workspaceDir,
+    });
+    expect(listResult.success).toBe(true);
+    expect(listResult.items?.some((entry) => entry.name === "notes.txt")).toBe(
+      true,
+    );
+
+    const uploadPath = path.join(workspaceDir, "upload.txt");
+    const uploadResult = await service.executeCommand("file_upload", {
+      filepath: uploadPath,
+      content: "upload",
+    });
+    expect(uploadResult.success).toBe(true);
+
+    const downloadResult = await service.executeCommand("file_download", {
+      filepath: uploadPath,
+    });
+    expect(downloadResult).toMatchObject({
+      success: true,
+      content: "upload",
+    });
+
+    const nestedWriteResult = await service.executeCommand("file_write", {
+      filepath: nestedFilePath,
+      content: "nested",
+    });
+    expect(nestedWriteResult.success).toBe(true);
+
+    const deleteFileResult = await service.executeCommand("file_delete", {
+      path: uploadPath,
+    });
+    expect(deleteFileResult.success).toBe(true);
+
+    const deleteDirectoryResult = await service.executeCommand(
+      "directory_delete",
+      {
+        dirpath: subdir,
+      },
+    );
+    expect(deleteDirectoryResult.success).toBe(true);
+  });
+
+  it("supports terminal commands and blocks catastrophic commands live", async () => {
+    const connectResult = await service.executeCommand("terminal_connect", {
+      cwd: workspaceDir,
+    });
+    expect(connectResult.success).toBe(true);
+    const sessionId = connectResult.sessionId;
+    expect(sessionId).toBeTruthy();
+
+    const executeResult = await service.executeCommand("execute_command", {
+      session_id: sessionId,
+      command: "printf 'live-terminal' > terminal.txt",
+    });
+    expect(executeResult.success).toBe(true);
+
+    expect(
+      await readFile(path.join(workspaceDir, "terminal.txt"), "utf8"),
+    ).toBe("live-terminal");
+
+    const readResult = await service.executeCommand("terminal_read", {
+      session_id: sessionId,
+    });
+    expect(readResult.success).toBe(true);
+
+    const typeResult = await service.executeCommand("terminal_type", {
+      text: "echo from parity",
+    });
+    expect(typeResult.success).toBe(true);
+
+    const clearResult = await service.executeCommand("terminal_clear", {
+      session_id: sessionId,
+    });
+    expect(clearResult.success).toBe(true);
+
+    const blockedResult = await service.executeCommand("terminal_execute", {
+      command: "rm -rf /",
+    });
+    expect(blockedResult.success).toBe(false);
+    expect(blockedResult.error).toContain("Command blocked");
+
+    const closeResult = await service.executeCommand("terminal_close", {
+      session_id: sessionId,
+    });
+    expect(closeResult.success).toBe(true);
+  });
+
+  it("supports browser commands, aliases, and tab management live", async () => {
+    const capabilities = service.getCapabilities();
+    if (!capabilities.browser.available) {
+      return;
+    }
+
+    // GitHub Actions ubuntu runners detect Chrome on PATH but Chromium fails
+    // to actually launch under the runner sandbox despite --no-sandbox args,
+    // so skip this live browser flow on CI until a headed-or-docker strategy
+    // is in place. Local dev and dev desktop still exercise this path.
+    if (process.env.CI === "true" || process.env.GITHUB_ACTIONS === "true") {
+      return;
+    }
+
+    const pageUrl =
+      "data:text/html,<html><body data-ready='yes'><button id='go'>Go</button><a href='#next'>Next</a><div>Ready</div></body></html>";
+
+    const openResult = await service.executeCommand("browser_connect", {
+      url: pageUrl,
+    });
+    if (!openResult.success) {
+      throw new Error(
+        `browser_connect failed locally: ${String(openResult.error ?? "unknown")}`,
+      );
+    }
+    expect(openResult.success).toBe(true);
+
+    const domResult = await service.executeCommand("browser_get_dom");
+    expect(domResult.success).toBe(true);
+    expect(domResult.content).toContain("data-ready");
+
+    const clickablesResult = await service.executeCommand(
+      "browser_get_clickables",
+    );
+    expect(clickablesResult.success).toBe(true);
+    expect(Array.isArray(clickablesResult.data)).toBe(true);
+
+    const stateResult = await service.executeCommand("browser_state");
+    expect(stateResult.success).toBe(true);
+
+    const infoResult = await service.executeCommand("browser_info");
+    expect(infoResult.success).toBe(true);
+
+    const contextResult = await service.executeCommand("browser_get_context");
+    expect(contextResult.success).toBe(true);
+
+    const waitResult = await service.executeCommand("browser_wait", {
+      selector: "#go",
+      timeout: 2000,
+    });
+    expect(waitResult.success).toBe(true);
+
+    const openTabResult = await service.executeCommand("browser_open_tab", {
+      url: "data:text/html,<html><body>Tab 2</body></html>",
+    });
+    expect(openTabResult.success).toBe(true);
+
+    const tabsResult = await service.executeCommand("browser_list_tabs");
+    expect(tabsResult.success).toBe(true);
+    expect(Array.isArray(tabsResult.data)).toBe(true);
+    expect((tabsResult.data as Array<unknown>).length).toBeGreaterThanOrEqual(
+      2,
+    );
+
+    const switchTabResult = await service.executeCommand("browser_switch_tab", {
+      tab_index: 0,
+    });
+    expect(switchTabResult.success).toBe(true);
+
+    const closeTabResult = await service.executeCommand("browser_close_tab", {
+      tab_index: 1,
+    });
+    expect(closeTabResult.success).toBe(true);
+
+    const screenshotResult = await service.executeCommand("browser_screenshot");
+    expect(screenshotResult.success).toBe(true);
+    expect(typeof screenshotResult.screenshot).toBe("string");
+
+    const closeResult = await service.executeCommand("browser_close");
+    expect(closeResult.success).toBe(true);
+  });
+
+  it("reports screenshot and desktop permission outcomes live", async () => {
+    if (!service.getCapabilities().screenshot.available) {
+      return;
+    }
+
+    const screenshotResult = await service.executeDesktopAction({
+      action: "screenshot",
+    });
+    if (screenshotResult.success) {
+      assertScreenshotBase64NotBlank(
+        screenshotResult.screenshot,
+        "computer-use permission outcome screenshot",
+      );
+    } else {
+      expect(screenshotResult.permissionDenied).toBe(true);
+      expect(screenshotResult.permissionType).toBe("screen_recording");
+    }
+
+    const detectResult = await service.executeDesktopAction({
+      action: "detect_elements",
+    });
+    expect(detectResult.success).toBe(false);
+    expect(detectResult.error).toContain("not available");
+
+    const ocrResult = await service.executeDesktopAction({ action: "ocr" });
+    expect(ocrResult.success).toBe(false);
+    expect(ocrResult.error).toContain("not available");
+
+    const computerUseCapability = service.getCapabilities().computerUse;
+    if (!computerUseCapability.available) {
+      return;
+    }
+
+    const moveResult = await service.executeDesktopAction({
+      action: "mouse_move",
+      coordinate: [1, 1],
+    });
+    if (moveResult.success) {
+      expect(moveResult.success).toBe(true);
+    } else if (
+      process.platform === "darwin" &&
+      !computerUseCapability.tool.includes("cliclick")
+    ) {
+      expect(moveResult.error).toContain("mouse_move requires cliclick");
+    } else if (moveResult.permissionDenied) {
+      expect(moveResult.permissionType).toBe("accessibility");
+    } else {
+      expect(
+        (typeof moveResult.error === "string" && moveResult.error.length > 0) ||
+          moveResult.permissionDenied === true ||
+          moveResult.permissionType === "accessibility",
+      ).toBe(true);
+    }
+  });
+
+  it("supports window listing and rejects underspecified window move commands live", async () => {
+    const listResult = await service.executeCommand("list_windows");
+    expect(listResult.success).toBe(true);
+    expect(Array.isArray(listResult.windows)).toBe(true);
+
+    const moveResult = await service.executeCommand("move_window", {
+      x: 10,
+      y: 20,
+    });
+    expect(moveResult.success).toBe(false);
+    expect(moveResult.error).toMatch(/windowId or windowTitle is required/i);
+  });
+
+  it("honors approval mode and safe-command auto-approval live", async () => {
+    const filePath = path.join(workspaceDir, "approval.txt");
+    await service.executeCommand("file_write", {
+      path: filePath,
+      content: "approval",
+    });
+
+    const approveAllService = (await ComputerUseService.start(
+      createRuntime({
+        COMPUTER_USE_APPROVAL_MODE: "approve_all",
+      }),
+    )) as ComputerUseService;
+
+    try {
+      const pendingRead = approveAllService.executeCommand("file_read", {
+        path: filePath,
+      });
+      const approvalId = await waitForPendingApproval(approveAllService);
+      const snapshot = approveAllService.getApprovalSnapshot();
+      expect(snapshot.pendingApprovals[0]?.command).toBe("file_read");
+      approveAllService.resolveApproval(approvalId, true);
+      const approvedResult = await pendingRead;
+      expect(approvedResult.success).toBe(true);
+    } finally {
+      await approveAllService.stop();
+    }
+
+    const smartApproveService = (await ComputerUseService.start(
+      createRuntime({
+        COMPUTER_USE_APPROVAL_MODE: "smart_approve",
+      }),
+    )) as ComputerUseService;
+
+    try {
+      const safeRead = await smartApproveService.executeCommand("file_read", {
+        path: filePath,
+      });
+      expect(safeRead.success).toBe(true);
+      expect(smartApproveService.getApprovalSnapshot().pendingCount).toBe(0);
+    } finally {
+      await smartApproveService.stop();
+    }
+  });
+});

plugins/plugin-computeruse/src/__tests__/cua-parity-coverage.test.ts

@@ -0,0 +1,420 @@
+/**
+ * trycua/cua parity COVERAGE GUARD (#9170, M14).
+ *
+ * A single machine-checkable registry of every trycua/cua capability, each tagged
+ * `have` | `partial` | `missing` | `na`. This is the executable form of
+ * `docs/TRYCUA_PARITY_AUDIT.md`: it fails CI when a `have` verb is dropped from the
+ * live action surface, when a host input verb loses its test coverage, or when an
+ * `na` decision is silently reverted (e.g. browser_execute re-enabled).
+ *
+ * Crucially this runs in the DEFAULT lane — i.e. on Windows, Linux, macOS and the
+ * AOSP/Node test runner alike — so "we have parity, and every verb is tested" is
+ * verified on ALL platforms, not just the win32-gated real-driver lane. The real
+ * actuation of each verb on a live desktop is the `*.real.test.ts` lane (Windows
+ * today; Linux/macOS once their headful CI lanes exist — see audit §3/§5).
+ */
+
+import { readdirSync, readFileSync } from "node:fs";
+import { dirname, join } from "node:path";
+import { fileURLToPath } from "node:url";
+import { describe, expect, it } from "vitest";
+import { computerUsePlugin } from "../index.js";
+import { isBrowserExecuteAllowed } from "../security/browser-script-policy.js";
+
+type Domain =
+  | "input" // host pointer/keyboard/screen verb on COMPUTER_USE
+  | "vision" // OCR / detect / get_screen (computeruse enum or plugin-vision)
+  | "window" // WINDOW action verb
+  | "clipboard" // CLIPBOARD action verb
+  | "architecture" // loop/provider/RPC/MCP/eval — not a single verb
+  | "mobile";
+
+type Status = "have" | "partial" | "missing" | "na";
+
+interface Capability {
+  /** trycua/cua capability name. */
+  cua: string;
+  domain: Domain;
+  status: Status;
+  /** elizaOS COMPUTER_USE/WINDOW/CLIPBOARD verb that satisfies it, when applicable. */
+  verb?: string;
+  /** Required when status === "na": why we deliberately don't chase it. */
+  na?: string;
+  /** Milestone tracking the remaining work, when partial/missing. */
+  milestone?: string;
+}
+
+// The full surface. Keep in sync with docs/TRYCUA_PARITY_AUDIT.md.
+const CAPABILITIES: Capability[] = [
+  // ── host input ────────────────────────────────────────────────────────────
+  { cua: "screenshot", domain: "input", status: "have", verb: "screenshot" },
+  { cua: "left_click", domain: "input", status: "have", verb: "click" },
+  { cua: "right_click", domain: "input", status: "have", verb: "right_click" },
+  {
+    cua: "double_click",
+    domain: "input",
+    status: "have",
+    verb: "double_click",
+  },
+  {
+    cua: "middle_click",
+    domain: "input",
+    status: "have",
+    verb: "middle_click",
+  },
+  {
+    cua: "click_with_modifiers",
+    domain: "input",
+    status: "have",
+    verb: "click_with_modifiers",
+  },
+  { cua: "move_cursor", domain: "input", status: "have", verb: "mouse_move" },
+  {
+    cua: "left_mouse_down",
+    domain: "input",
+    status: "have",
+    verb: "mouse_down",
+  },
+  { cua: "left_mouse_up", domain: "input", status: "have", verb: "mouse_up" },
+  { cua: "type_text", domain: "input", status: "have", verb: "type" },
+  { cua: "press_key", domain: "input", status: "have", verb: "key" },
+  { cua: "hotkey", domain: "input", status: "have", verb: "key_combo" },
+  { cua: "key_down", domain: "input", status: "have", verb: "key_down" },
+  { cua: "key_up", domain: "input", status: "have", verb: "key_up" },
+  { cua: "scroll", domain: "input", status: "have", verb: "scroll" },
+  { cua: "drag", domain: "input", status: "have", verb: "drag" },
+  {
+    cua: "get_cursor_position",
+    domain: "input",
+    status: "have",
+    verb: "get_cursor_position",
+  },
+  // ── vision ────────────────────────────────────────────────────────────────
+  { cua: "ocr", domain: "vision", status: "have", verb: "ocr" },
+  {
+    cua: "detect_elements",
+    domain: "vision",
+    status: "have",
+    verb: "detect_elements",
+  },
+  {
+    cua: "get_screen (typed image + elements, token-frugal)",
+    domain: "vision",
+    status: "have", // plugin-vision GET_SCREEN / GET_SCREEN_ELEMENTS (M2)
+  },
+  {
+    cua: "set_of_marks_overlay",
+    domain: "vision",
+    status: "have", // M9: plugin-vision som.ts + set-of-marks-provider via registerSetOfMarksProvider seam
+  },
+  // ── window ────────────────────────────────────────────────────────────────
+  { cua: "window_list", domain: "window", status: "have", verb: "list" },
+  { cua: "activate_window", domain: "window", status: "have", verb: "focus" },
+  {
+    cua: "minimize_window",
+    domain: "window",
+    status: "have",
+    verb: "minimize",
+  },
+  {
+    cua: "maximize_window",
+    domain: "window",
+    status: "have",
+    verb: "maximize",
+  },
+  { cua: "restore_window", domain: "window", status: "have", verb: "restore" },
+  { cua: "close_window", domain: "window", status: "have", verb: "close" },
+  {
+    cua: "set_window_position",
+    domain: "window",
+    status: "have",
+    verb: "move",
+  },
+  // M12 window getters/setters — all landed.
+  {
+    cua: "get_current_window_id",
+    domain: "window",
+    status: "have",
+    verb: "get_current_window_id",
+    milestone: "M12",
+  },
+  {
+    cua: "get_application_windows",
+    domain: "window",
+    status: "have",
+    verb: "get_application_windows",
+    milestone: "M12",
+  },
+  {
+    cua: "set_window_size (set_bounds)",
+    domain: "window",
+    status: "have",
+    verb: "set_bounds",
+    milestone: "M12",
+  },
+  {
+    cua: "get_window_size",
+    domain: "window",
+    status: "have",
+    verb: "get_window_size",
+    milestone: "M12",
+  },
+  {
+    cua: "get_window_position",
+    domain: "window",
+    status: "have",
+    verb: "get_window_position",
+    milestone: "M12",
+  },
+  // open/launch are COMPUTER_USE verbs (implemented; launch returns a pid).
+  {
+    cua: "open(target)",
+    domain: "input",
+    status: "have",
+    verb: "open",
+    milestone: "M12",
+  },
+  {
+    cua: "launch(app,args)->pid",
+    domain: "input",
+    status: "have",
+    verb: "launch",
+    milestone: "M12",
+  },
+  // ── clipboard ───────────────────────────────────────────────────────────────
+  {
+    cua: "copy_to_clipboard",
+    domain: "clipboard",
+    status: "have",
+    verb: "read",
+  },
+  { cua: "set_clipboard", domain: "clipboard", status: "have", verb: "write" },
+  // ── filesystem / shell (gated/internal today) ───────────────────────────────
+  // Host file/shell route through the FILE / SHELL actions; sandbox guests get
+  // run_command + basic fs over the M13 RPC. Basic ops exist; the binary/dir
+  // transfer verbs below are genuine gaps (cua exposes them for guest I/O).
+  {
+    cua: "filesystem basic (exists/list/read_text/write_text/delete)",
+    domain: "architecture",
+    status: "partial",
+    milestone: "M13",
+  },
+  {
+    cua: "run_command (CommandResult)",
+    domain: "architecture",
+    status: "partial",
+    milestone: "M13",
+  },
+  {
+    cua: "read_bytes/write_bytes (base64 binary), create_dir, directory_exists, get_file_size",
+    domain: "architecture",
+    status: "have",
+    milestone: "M13 — file-ops binary I/O + executeFileAction",
+  },
+  // ── trycua blind spots — tracked decisions, not silent gaps (workflow audit) ──
+  {
+    cua: "set_value (a11y element value write)",
+    domain: "input",
+    status: "have",
+    verb: "set_value",
+    milestone: "M12",
+  },
+  {
+    cua: "kill_app (process terminate)",
+    domain: "input",
+    status: "have",
+    verb: "kill_app",
+    milestone: "M12",
+  },
+  {
+    cua: "zoom (region magnify for grounding)",
+    domain: "architecture",
+    status: "partial",
+    milestone: "covered by M5 ROI + M9 Set-of-Marks",
+  },
+  {
+    cua: "screen recording / replay_trajectory (MP4)",
+    domain: "architecture",
+    status: "na",
+    na: "evidence concern owned by test:e2e:record + scenario-runner viewer, not an agent verb",
+  },
+  {
+    cua: "agent-cursor overlay (set_agent_cursor_*)",
+    domain: "architecture",
+    status: "na",
+    na: "UX overlay tied to cua's recording; no agent-decision value",
+  },
+  // ── architecture ────────────────────────────────────────────────────────────
+  {
+    cua: "agent_loop_registry",
+    domain: "architecture",
+    status: "partial",
+    milestone: "M10",
+  },
+  {
+    cua: "predict_step/predict_click split",
+    domain: "architecture",
+    status: "partial",
+    milestone: "M10",
+  },
+  {
+    cua: "callback_middleware (budget/retention/trajectory)",
+    domain: "architecture",
+    status: "partial",
+    milestone: "M11",
+  },
+  {
+    cua: "vm/sandbox provider matrix",
+    domain: "architecture",
+    status: "partial",
+    milestone: "M13",
+  },
+  {
+    cua: "daemon/RPC seam",
+    domain: "architecture",
+    status: "missing",
+    milestone: "M13",
+  },
+  {
+    cua: "MCP server seam",
+    domain: "architecture",
+    status: "have",
+    milestone: "src/mcp — tool catalog + dispatch + optional-SDK server",
+  },
+  {
+    cua: "eval harness (ScreenSpot/OSWorld/per-OS scenarios)",
+    domain: "architecture",
+    status: "partial",
+    milestone: "M14",
+  },
+  {
+    cua: "low-token continuous screen description",
+    domain: "architecture",
+    status: "have",
+  },
+  {
+    cua: "accessibility tree grounding",
+    domain: "architecture",
+    status: "have",
+  },
+  // ── mobile / AOSP ─────────────────────────────────────────────────────────────
+  {
+    cua: "android tap/swipe + hardware keys",
+    domain: "mobile",
+    status: "have",
+  },
+  { cua: "android multitouch_gesture", domain: "mobile", status: "missing" },
+  // ── deliberately not chased ──────────────────────────────────────────────────
+  {
+    cua: "browser_execute / playwright_exec",
+    domain: "architecture",
+    status: "na",
+    na: "Unconditionally disabled by security policy (GHSA-rcvr-766c-4phv).",
+  },
+  {
+    cua: "set_wallpaper",
+    domain: "architecture",
+    status: "na",
+    na: "Niche; no agent value.",
+  },
+  {
+    cua: "cloud managed sandbox",
+    domain: "architecture",
+    status: "na",
+    na: "We control the local machine in-process; cloud VM fleet out of scope.",
+  },
+];
+
+const INPUT_VERBS = CAPABILITIES.filter(
+  (c) =>
+    (c.domain === "input" || (c.domain === "vision" && c.verb)) &&
+    c.status === "have" &&
+    c.verb,
+).map((c) => c.verb as string);
+
+// Live action surface.
+const actions = computerUsePlugin.actions ?? [];
+const actionNames = actions.map((a) => a.name);
+const computerUse = actions.find((a) => a.name === "COMPUTER_USE") as
+  | { parameters?: Array<{ name: string; schema?: { enum?: string[] } }> }
+  | undefined;
+const enumVerbs =
+  computerUse?.parameters?.find((p) => p.name === "action")?.schema?.enum ?? [];
+
+// Test corpus (every *.test.ts under __tests__, minus this file).
+const testDir = dirname(fileURLToPath(import.meta.url));
+const SELF = "cua-parity-coverage.test.ts";
+const corpus = readdirSync(testDir)
+  .filter((f) => f.endsWith(".test.ts") && f !== SELF)
+  .map((f) => readFileSync(join(testDir, f), "utf8"))
+  .join("\n");
+
+describe("cua parity registry integrity", () => {
+  it("every capability has a known status; na entries justify themselves", () => {
+    for (const c of CAPABILITIES) {
+      expect(["have", "partial", "missing", "na"]).toContain(c.status);
+      if (c.status === "na") {
+        expect(c.na, `na capability "${c.cua}" needs a reason`).toBeTruthy();
+      }
+    }
+  });
+
+  it("has no duplicate verb mappings", () => {
+    const verbs = CAPABILITIES.map((c) => c.verb).filter(Boolean) as string[];
+    expect(new Set(verbs).size).toBe(verbs.length);
+  });
+});
+
+describe("cua parity surface (all platforms)", () => {
+  it("every have/input + ocr/detect verb is in the COMPUTER_USE action enum", () => {
+    for (const v of INPUT_VERBS) {
+      expect(enumVerbs, `enum: ${enumVerbs.join(", ")}`).toContain(v);
+    }
+  });
+
+  it("every have/window verb is a promoted WINDOW action", () => {
+    for (const c of CAPABILITIES.filter(
+      (c) => c.domain === "window" && c.status === "have" && c.verb,
+    )) {
+      const promoted = `WINDOW_${(c.verb as string).toUpperCase()}`;
+      expect(actionNames, actionNames.join(", ")).toContain(promoted);
+    }
+  });
+
+  it("clipboard read/write are promoted CLIPBOARD actions", () => {
+    expect(actionNames).toContain("CLIPBOARD_READ");
+    expect(actionNames).toContain("CLIPBOARD_WRITE");
+  });
+
+  it("Set-of-Marks (M9) registry seam exists for plugin-vision to wire", () => {
+    // M9 is a grounding capability (no discrete verb): plugin-vision builds the
+    // numbered overlay (som.ts) and registers it through this seam. Assert the
+    // seam is present so the `have` status above can't silently regress.
+    const ocrProviderSrc = readFileSync(
+      join(testDir, "..", "mobile", "ocr-provider.ts"),
+      "utf8",
+    );
+    expect(ocrProviderSrc).toContain("registerSetOfMarksProvider");
+    expect(ocrProviderSrc).toContain("SetOfMarksProvider");
+  });
+});
+
+describe("cua parity test-coverage (every input/vision verb is tested)", () => {
+  it.each(INPUT_VERBS)("verb %s is referenced by at least one test", (verb) => {
+    const upper = `COMPUTER_USE_${verb.toUpperCase()}`;
+    const referenced =
+      corpus.includes(`"${verb}"`) ||
+      corpus.includes(`'${verb}'`) ||
+      corpus.includes(upper);
+    expect(
+      referenced,
+      `no test references verb "${verb}" (snake or ${upper})`,
+    ).toBe(true);
+  });
+});
+
+describe("cua parity N/A decisions are honored", () => {
+  it("browser_execute stays disabled (GHSA-rcvr-766c-4phv)", () => {
+    expect(isBrowserExecuteAllowed()).toBe(false);
+  });
+});

plugins/plugin-computeruse/src/__tests__/cua-parity-input.real.test.ts

@@ -0,0 +1,124 @@
+/**
+ * Real-driver verification of the cua-parity input verbs (#9105).
+ * Gated: runs only on a Windows host with a desktop session (and in Windows CI).
+ *
+ * - get_cursor_position: move then read back the OS cursor (uses the native
+ *   System.Windows.Forms.Cursor query on Windows because nutjs getPosition()
+ *   returns a stale constant there).
+ * - clipboard: write then read round-trips (Set-Clipboard via
+ *   [Console]::In.ReadToEnd()).
+ */
+
+import { platform } from "node:os";
+import { describe, expect, it } from "vitest";
+import { readClipboard, writeClipboard } from "../platform/clipboard.js";
+import {
+  driverDragPath,
+  driverGetCursorPosition,
+  driverKeyDown,
+  driverKeyUp,
+  driverMiddleClick,
+  driverMouseDown,
+  driverMouseMove,
+  driverMouseUp,
+} from "../platform/driver.js";
+
+// Run the real-driver lane wherever a real host driver + display session is
+// available — not just Windows. macOS runners are headful; Linux needs an X
+// display (Xvfb in CI). The previous win32-only gate silently skipped Linux/macOS
+// and let the M8 key_down("shift") bug (#9189) ship. See TRYCUA_PARITY_AUDIT.md
+// §3 — the matching CI lanes (Xvfb on Linux, headful macOS) are the remaining M14
+// step.
+const os = platform();
+const RUN =
+  os === "win32" ||
+  os === "darwin" ||
+  (os === "linux" && Boolean(process.env.DISPLAY));
+
+describe("cua parity input (real driver, Windows)", () => {
+  it.skipIf(!RUN)(
+    "get_cursor_position reflects driverMouseMove",
+    async () => {
+      for (const [x, y] of [
+        [320, 240],
+        [640, 480],
+      ] as const) {
+        await driverMouseMove(x, y);
+        await new Promise((r) => setTimeout(r, 150));
+        const pos = await driverGetCursorPosition();
+        expect(Math.abs(pos.x - x)).toBeLessThanOrEqual(2);
+        expect(Math.abs(pos.y - y)).toBeLessThanOrEqual(2);
+      }
+    },
+    20000,
+  );
+
+  it.skipIf(!RUN)(
+    "clipboard write/read round-trips",
+    async () => {
+      const token = `eliza-clip-${Date.now()}`;
+      await writeClipboard(token);
+      const back = (await readClipboard()).trim();
+      expect(back).toBe(token);
+    },
+    20000,
+  );
+
+  // ── M8 verb-parity pack (real driver) ───────────────────────────────────
+
+  it.skipIf(!RUN)(
+    "middle_click fires without throwing",
+    async () => {
+      await driverMouseMove(400, 300);
+      await expect(driverMiddleClick(400, 300)).resolves.toBeUndefined();
+    },
+    20000,
+  );
+
+  it.skipIf(!RUN)(
+    "mouse_down/mouse_up press-hold round-trips (always releases)",
+    async () => {
+      await driverMouseMove(420, 320);
+      await driverMouseDown(420, 320, "left");
+      try {
+        const pos = await driverGetCursorPosition();
+        expect(Math.abs(pos.x - 420)).toBeLessThanOrEqual(2);
+      } finally {
+        // Never leave a button held, even on assertion failure.
+        await driverMouseUp(420, 320, "left");
+      }
+    },
+    20000,
+  );
+
+  it.skipIf(!RUN)(
+    "key_down/key_up press-hold round-trips (always releases)",
+    async () => {
+      await driverKeyDown("shift");
+      try {
+        // No throw is the contract; the held modifier has no observable side
+        // effect on its own here.
+        expect(true).toBe(true);
+      } finally {
+        await driverKeyUp("shift");
+      }
+    },
+    20000,
+  );
+
+  it.skipIf(!RUN)(
+    "multi-point drag path lands the cursor on the final vertex",
+    async () => {
+      await driverDragPath([
+        { x: 300, y: 300 },
+        { x: 500, y: 300 },
+        { x: 500, y: 450 },
+      ]);
+      await new Promise((r) => setTimeout(r, 150));
+      const pos = await driverGetCursorPosition();
+      expect(Math.abs(pos.x - 500)).toBeLessThanOrEqual(3);
+      expect(Math.abs(pos.y - 450)).toBeLessThanOrEqual(3);
+    },
+    20000,
+  );
+});

plugins/plugin-computeruse/src/__tests__/cua-parity-surface.test.ts

@@ -0,0 +1,53 @@
+/**
+ * trycua/cua parity surface (#9105) — verify the newly-exposed computer-use
+ * capabilities are actually registered and wired:
+ *   - CLIPBOARD (read/write) is registered (was defined-but-unregistered).
+ *   - COMPUTER_USE exposes a get_cursor_position verb.
+ * Plus a regression guard on the Windows clipboard-write command (the previous
+ * `$input | Set-Clipboard` hung; it must read stdin via [Console]::In.ReadToEnd()).
+ */
+
+import { platform } from "node:os";
+import { describe, expect, it } from "vitest";
+import { computerUsePlugin } from "../index.js";
+import { __testing } from "../platform/clipboard.js";
+
+const actionNames = (computerUsePlugin.actions ?? []).map((a) => a.name);
+
+describe("cua parity surface", () => {
+  it("registers the CLIPBOARD action (read/write)", () => {
+    const hasClipboard = actionNames.some((n) => /CLIPBOARD/i.test(n));
+    expect(hasClipboard, `actions: ${actionNames.join(", ")}`).toBe(true);
+    // Promoted subactions present.
+    expect(actionNames).toContain("CLIPBOARD_READ");
+    expect(actionNames).toContain("CLIPBOARD_WRITE");
+  });
+
+  it("exposes a get_cursor_position computer-use verb", () => {
+    expect(actionNames).toContain("COMPUTER_USE_GET_CURSOR_POSITION");
+  });
+
+  it("promotes the M8 verb-parity pack (middle_click, mouse/key down-up)", () => {
+    for (const verb of [
+      "COMPUTER_USE_MIDDLE_CLICK",
+      "COMPUTER_USE_MOUSE_DOWN",
+      "COMPUTER_USE_MOUSE_UP",
+      "COMPUTER_USE_KEY_DOWN",
+      "COMPUTER_USE_KEY_UP",
+    ]) {
+      expect(actionNames, `actions: ${actionNames.join(", ")}`).toContain(verb);
+    }
+  });
+});
+
+describe("clipboard write command (Windows regression)", () => {
+  it.skipIf(platform() !== "win32")(
+    "reads stdin via [Console]::In.ReadToEnd(), not $input",
+    () => {
+      const plan = __testing.pickPlan();
+      const joined = plan.write.args.join(" ");
+      expect(joined).toContain("ReadToEnd");
+      expect(joined).not.toContain("$input | Set-Clipboard");
+    },
+  );
+});

plugins/plugin-computeruse/src/__tests__/dhash.test.ts

@@ -0,0 +1,157 @@
+/**
+ * Pure-function tests for the dHash + block-grid implementation.
+ *
+ * We synthesize tiny PNGs via `node:zlib` and the PNG chunk format so we
+ * don't need any decoder dependency to exercise both code paths.
+ */
+
+import { deflateSync } from "node:zlib";
+import { describe, expect, it } from "vitest";
+import {
+  blockGrid,
+  decodePng,
+  diffBlocks,
+  frameDhash,
+  hamming,
+} from "../scene/dhash.js";
+
+function crc32(bytes: Buffer): number {
+  let crc = 0xffffffff >>> 0;
+  for (let i = 0; i < bytes.length; i += 1) {
+    crc = (crc ^ bytes[i]!) >>> 0;
+    for (let j = 0; j < 8; j += 1) {
+      const lsb = crc & 1;
+      crc = (crc >>> 1) ^ (lsb ? 0xedb88320 : 0);
+    }
+  }
+  return (crc ^ 0xffffffff) >>> 0;
+}
+
+function pngChunk(type: string, data: Buffer): Buffer {
+  const len = Buffer.alloc(4);
+  len.writeUInt32BE(data.length);
+  const t = Buffer.from(type, "ascii");
+  const crc = Buffer.alloc(4);
+  crc.writeUInt32BE(crc32(Buffer.concat([t, data])));
+  return Buffer.concat([len, t, data, crc]);
+}
+
+/**
+ * Build a 16×16 8-bit RGB PNG painted in a horizontal gradient unless
+ * `solid` is true (in which case every pixel is the same color).
+ */
+function makeTinyPng(seed = 0, solid = false): Buffer {
+  const w = 16;
+  const h = 16;
+  const _stride = w * 3;
+  const rows: number[] = [];
+  for (let y = 0; y < h; y += 1) {
+    rows.push(0); // filter = None
+    for (let x = 0; x < w; x += 1) {
+      const v = solid ? (seed * 7) % 255 : ((x + seed) * 16) % 255;
+      rows.push(v, v, v);
+    }
+  }
+  const raw = Buffer.from(rows);
+  const idat = deflateSync(raw);
+  const ihdr = Buffer.alloc(13);
+  ihdr.writeUInt32BE(w, 0);
+  ihdr.writeUInt32BE(h, 4);
+  ihdr[8] = 8; // bit depth
+  ihdr[9] = 2; // color type RGB
+  ihdr[10] = 0;
+  ihdr[11] = 0;
+  ihdr[12] = 0;
+  const signature = Buffer.from([
+    0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a,
+  ]);
+  return Buffer.concat([
+    signature,
+    pngChunk("IHDR", ihdr),
+    pngChunk("IDAT", idat),
+    pngChunk("IEND", Buffer.alloc(0)),
+  ]);
+}
+
+describe("dhash — pure functions", () => {
+  it("decodes a minimal RGB PNG", () => {
+    const png = makeTinyPng();
+    const decoded = decodePng(png);
+    expect(decoded).not.toBeNull();
+    expect(decoded?.width).toBe(16);
+    expect(decoded?.height).toBe(16);
+    expect(decoded?.rgba.length).toBe(16 * 16 * 4);
+  });
+
+  it("returns null for non-PNG input", () => {
+    expect(decodePng(Buffer.from("not a png"))).toBeNull();
+  });
+
+  it("frameDhash is stable for identical frames", () => {
+    const a = frameDhash(makeTinyPng(7));
+    const b = frameDhash(makeTinyPng(7));
+    expect(a).not.toBeNull();
+    expect(b).not.toBeNull();
+    expect(a).toBe(b);
+  });
+
+  it("frameDhash differs for visually different frames", () => {
+    const a = frameDhash(makeTinyPng(0));
+    const b = frameDhash(makeTinyPng(50));
+    expect(a).not.toBeNull();
+    expect(b).not.toBeNull();
+    expect(hamming(a!, b!)).toBeGreaterThan(0);
+  });
+
+  it("hamming(x, x) == 0", () => {
+    expect(hamming(0xdeadbeefcafe0001n, 0xdeadbeefcafe0001n)).toBe(0);
+  });
+
+  it("hamming(a, b) counts changed bits", () => {
+    expect(hamming(0n, 0xffffffffffffffffn)).toBe(64);
+  });
+});
+
+describe("dhash — block grid", () => {
+  it("produces a cols*rows grid", () => {
+    const grid = blockGrid(makeTinyPng(0), 4, 4);
+    expect(grid).not.toBeNull();
+    expect(grid?.cols).toBe(4);
+    expect(grid?.rows).toBe(4);
+    expect(grid?.hashes.length).toBe(16);
+  });
+
+  it("identical frames produce identical block grids and zero dirty blocks", () => {
+    const a = blockGrid(makeTinyPng(11), 4, 4)!;
+    const b = blockGrid(makeTinyPng(11), 4, 4)!;
+    const dirty = diffBlocks(a, b);
+    expect(dirty.length).toBe(0);
+  });
+
+  it("first frame (prev=null) marks every block dirty", () => {
+    const grid = blockGrid(makeTinyPng(0), 4, 4)!;
+    const dirty = diffBlocks(null, grid);
+    expect(dirty.length).toBe(grid.cols * grid.rows);
+  });
+
+  it("changed frames produce a non-zero dirty list", () => {
+    const a = blockGrid(makeTinyPng(0), 4, 4)!;
+    const b = blockGrid(makeTinyPng(120), 4, 4)!;
+    const dirty = diffBlocks(a, b);
+    expect(dirty.length).toBeGreaterThan(0);
+  });
+
+  it("dirty-block bboxes are translated to image pixel space when dims are known", () => {
+    const a = blockGrid(makeTinyPng(0), 4, 4)!;
+    const b = blockGrid(makeTinyPng(120), 4, 4)!;
+    const dirty = diffBlocks(a, b, 16, 16);
+    for (const d of dirty) {
+      expect(d.bbox[2]).toBeGreaterThan(0);
+      expect(d.bbox[3]).toBeGreaterThan(0);
+      expect(d.bbox[0]).toBeGreaterThanOrEqual(0);
+      expect(d.bbox[1]).toBeGreaterThanOrEqual(0);
+      expect(d.bbox[0] + d.bbox[2]).toBeLessThanOrEqual(16);
+      expect(d.bbox[1] + d.bbox[3]).toBeLessThanOrEqual(16);
+    }
+  });
+});

plugins/plugin-computeruse/src/__tests__/dispatch.test.ts

@@ -0,0 +1,326 @@
+/**
+ * WS7 — Dispatch tests.
+ *
+ * Validates the `dispatch()` function — the single entry point that takes a
+ * resolved `ProposedAction` and routes it to a `ComputerInterface`.
+ *
+ *   - unknown displayId → `unknown_display`
+ *   - out-of-bounds coords → `out_of_bounds`
+ *   - kind-specific arg validation → `invalid_args`
+ *   - driver throw → `driver_error`
+ *   - wait/finish short-circuit cleanly with `success: true`
+ *   - happy path calls the right ComputerInterface method
+ */
+
+import { describe, expect, it } from "vitest";
+import type {
+  ComputerInterface,
+  CursorPosition,
+  DisplayPoint,
+  DragPath,
+  ScrollDelta,
+} from "../actor/computer-interface.js";
+import { dispatch } from "../actor/dispatch.js";
+import type { ProposedAction } from "../actor/types.js";
+import type { DisplayDescriptor } from "../types.js";
+
+function fakeDisplays(): DisplayDescriptor[] {
+  return [
+    {
+      id: 0,
+      bounds: [0, 0, 1920, 1080],
+      scaleFactor: 1,
+      primary: true,
+      name: "primary",
+    },
+  ];
+}
+
+interface DriverCalls {
+  leftClick: DisplayPoint[];
+  rightClick: DisplayPoint[];
+  doubleClick: DisplayPoint[];
+  typed: string[];
+  pressed: string[];
+  hotkeys: string[][];
+  scrolls: ScrollDelta[];
+  drags: DragPath[];
+}
+
+function makeIface(throwOn?: keyof DriverCalls): {
+  iface: ComputerInterface;
+  calls: DriverCalls;
+} {
+  const calls: DriverCalls = {
+    leftClick: [],
+    rightClick: [],
+    doubleClick: [],
+    typed: [],
+    pressed: [],
+    hotkeys: [],
+    scrolls: [],
+    drags: [],
+  };
+  const fail = async (label: keyof DriverCalls): Promise<void> => {
+    if (throwOn === label) {
+      throw new Error(`driver-failure:${label}`);
+    }
+  };
+  const iface: ComputerInterface = {
+    screenshot: async () => {
+      throw new Error("not used");
+    },
+    mouseDown: async () => {},
+    mouseUp: async () => {},
+    leftClick: async (p) => {
+      calls.leftClick.push(p);
+      await fail("leftClick");
+    },
+    rightClick: async (p) => {
+      calls.rightClick.push(p);
+      await fail("rightClick");
+    },
+    doubleClick: async (p) => {
+      calls.doubleClick.push(p);
+      await fail("doubleClick");
+    },
+    moveCursor: async () => {},
+    dragTo: async () => {},
+    drag: async (path) => {
+      calls.drags.push(path);
+      await fail("drags");
+    },
+    keyDown: async () => {},
+    keyUp: async () => {},
+    typeText: async ({ text }) => {
+      calls.typed.push(text);
+      await fail("typed");
+    },
+    pressKey: async ({ key }) => {
+      calls.pressed.push(key);
+      await fail("pressed");
+    },
+    hotkey: async ({ keys }) => {
+      calls.hotkeys.push([...keys]);
+      await fail("hotkeys");
+    },
+    scroll: async (delta) => {
+      calls.scrolls.push(delta);
+      await fail("scrolls");
+    },
+    scrollUp: async () => {},
+    scrollDown: async () => {},
+    getScreenSize: () => ({ w: 1920, h: 1080 }),
+    getCursorPosition: (): CursorPosition => ({ displayId: 0, x: 0, y: 0 }),
+    toScreenCoordinates: () => ({ x: 0, y: 0 }),
+    toScreenshotCoordinates: () => ({ imgX: 0, imgY: 0 }),
+    getAccessibilityTree: () => [],
+  };
+  return { iface, calls };
+}
+
+describe("dispatch — display + bounds validation", () => {
+  it("rejects unknown displayId for coord-bearing actions", async () => {
+    const { iface } = makeIface();
+    const action: ProposedAction = {
+      kind: "click",
+      displayId: 99,
+      x: 10,
+      y: 20,
+      rationale: "",
+    };
+    const res = await dispatch(action, {
+      interface: iface,
+      listDisplays: () => fakeDisplays(),
+    });
+    expect(res.success).toBe(false);
+    expect(res.error?.code).toBe("unknown_display");
+  });
+
+  it("permits unknown displayId for wait / finish", async () => {
+    const { iface } = makeIface();
+    const res = await dispatch(
+      { kind: "wait", displayId: 99, rationale: "" },
+      { interface: iface, listDisplays: () => fakeDisplays() },
+    );
+    expect(res.success).toBe(true);
+  });
+
+  it("rejects out-of-bounds click coords", async () => {
+    const { iface } = makeIface();
+    const res = await dispatch(
+      { kind: "click", displayId: 0, x: 5000, y: 10, rationale: "" },
+      { interface: iface, listDisplays: () => fakeDisplays() },
+    );
+    expect(res.success).toBe(false);
+    expect(res.error?.code).toBe("out_of_bounds");
+  });
+
+  it("rejects negative click coords", async () => {
+    const { iface } = makeIface();
+    const res = await dispatch(
+      { kind: "click", displayId: 0, x: -1, y: 10, rationale: "" },
+      { interface: iface, listDisplays: () => fakeDisplays() },
+    );
+    expect(res.success).toBe(false);
+    expect(res.error?.code).toBe("out_of_bounds");
+  });
+});
+
+describe("dispatch — happy paths", () => {
+  it("click routes to leftClick", async () => {
+    const { iface, calls } = makeIface();
+    const res = await dispatch(
+      { kind: "click", displayId: 0, x: 100, y: 200, rationale: "" },
+      { interface: iface, listDisplays: () => fakeDisplays() },
+    );
+    expect(res.success).toBe(true);
+    expect(calls.leftClick).toHaveLength(1);
+    expect(calls.leftClick[0]).toMatchObject({ displayId: 0, x: 100, y: 200 });
+  });
+
+  it("double_click routes to doubleClick", async () => {
+    const { iface, calls } = makeIface();
+    await dispatch(
+      { kind: "double_click", displayId: 0, x: 50, y: 50, rationale: "" },
+      { interface: iface, listDisplays: () => fakeDisplays() },
+    );
+    expect(calls.doubleClick).toHaveLength(1);
+  });
+
+  it("right_click routes to rightClick", async () => {
+    const { iface, calls } = makeIface();
+    await dispatch(
+      { kind: "right_click", displayId: 0, x: 50, y: 50, rationale: "" },
+      { interface: iface, listDisplays: () => fakeDisplays() },
+    );
+    expect(calls.rightClick).toHaveLength(1);
+  });
+
+  it("type routes to typeText", async () => {
+    const { iface, calls } = makeIface();
+    await dispatch(
+      { kind: "type", displayId: 0, text: "hi", rationale: "" },
+      { interface: iface, listDisplays: () => fakeDisplays() },
+    );
+    expect(calls.typed).toEqual(["hi"]);
+  });
+
+  it("key routes to pressKey", async () => {
+    const { iface, calls } = makeIface();
+    await dispatch(
+      { kind: "key", displayId: 0, key: "Enter", rationale: "" },
+      { interface: iface, listDisplays: () => fakeDisplays() },
+    );
+    expect(calls.pressed).toEqual(["Enter"]);
+  });
+
+  it("hotkey routes to hotkey", async () => {
+    const { iface, calls } = makeIface();
+    await dispatch(
+      { kind: "hotkey", displayId: 0, keys: ["ctrl", "s"], rationale: "" },
+      { interface: iface, listDisplays: () => fakeDisplays() },
+    );
+    expect(calls.hotkeys).toEqual([["ctrl", "s"]]);
+  });
+
+  it("scroll routes to scroll with delta", async () => {
+    const { iface, calls } = makeIface();
+    await dispatch(
+      {
+        kind: "scroll",
+        displayId: 0,
+        x: 50,
+        y: 50,
+        dx: 0,
+        dy: 3,
+        rationale: "",
+      },
+      { interface: iface, listDisplays: () => fakeDisplays() },
+    );
+    expect(calls.scrolls).toHaveLength(1);
+    expect(calls.scrolls[0]).toMatchObject({ displayId: 0, dx: 0, dy: 3 });
+  });
+
+  it("drag routes to drag with start + end", async () => {
+    const { iface, calls } = makeIface();
+    await dispatch(
+      {
+        kind: "drag",
+        displayId: 0,
+        startX: 10,
+        startY: 20,
+        x: 100,
+        y: 200,
+        rationale: "",
+      },
+      { interface: iface, listDisplays: () => fakeDisplays() },
+    );
+    expect(calls.drags).toHaveLength(1);
+    expect(calls.drags[0]?.path).toEqual([
+      { x: 10, y: 20 },
+      { x: 100, y: 200 },
+    ]);
+  });
+});
+
+describe("dispatch — invalid_args", () => {
+  it("type without text", async () => {
+    const { iface } = makeIface();
+    const res = await dispatch(
+      { kind: "type", displayId: 0, rationale: "" },
+      { interface: iface, listDisplays: () => fakeDisplays() },
+    );
+    expect(res.success).toBe(false);
+    expect(res.error?.code).toBe("invalid_args");
+  });
+
+  it("key without key", async () => {
+    const { iface } = makeIface();
+    const res = await dispatch(
+      { kind: "key", displayId: 0, rationale: "" },
+      { interface: iface, listDisplays: () => fakeDisplays() },
+    );
+    expect(res.error?.code).toBe("invalid_args");
+  });
+
+  it("hotkey with empty keys", async () => {
+    const { iface } = makeIface();
+    const res = await dispatch(
+      { kind: "hotkey", displayId: 0, keys: [], rationale: "" },
+      { interface: iface, listDisplays: () => fakeDisplays() },
+    );
+    expect(res.error?.code).toBe("invalid_args");
+  });
+
+  it("click without coords", async () => {
+    const { iface } = makeIface();
+    const res = await dispatch(
+      { kind: "click", displayId: 0, rationale: "" },
+      { interface: iface, listDisplays: () => fakeDisplays() },
+    );
+    expect(res.error?.code).toBe("invalid_args");
+  });
+
+  it("scroll without dx/dy", async () => {
+    const { iface } = makeIface();
+    const res = await dispatch(
+      { kind: "scroll", displayId: 0, x: 10, y: 10, rationale: "" },
+      { interface: iface, listDisplays: () => fakeDisplays() },
+    );
+    expect(res.error?.code).toBe("invalid_args");
+  });
+});
+
+describe("dispatch — driver_error wrap", () => {
+  it("wraps an interface throw in driver_error", async () => {
+    const { iface } = makeIface("leftClick");
+    const res = await dispatch(
+      { kind: "click", displayId: 0, x: 10, y: 10, rationale: "" },
+      { interface: iface, listDisplays: () => fakeDisplays() },
+    );
+    expect(res.success).toBe(false);
+    expect(res.error?.code).toBe("driver_error");
+    expect(res.error?.message).toContain("leftClick");
+  });
+});

plugins/plugin-computeruse/src/__tests__/displays.real.test.ts

@@ -0,0 +1,396 @@
+/**
+ * Real-host display-enumeration tests.
+ *
+ * - Linux/X11 path is exercised against `xrandr --listmonitors` on the live
+ *   host when it's available. Otherwise the parser is still validated on
+ *   curated fixtures.
+ * - macOS / Windows paths are validated structurally against fixtures only.
+ */
+
+import { execFileSync } from "node:child_process";
+import { describe, expect, it } from "vitest";
+import {
+  clampToDisplay,
+  globalToLocal,
+  localToGlobal,
+  localToGlobalDefault,
+} from "../platform/coords.js";
+import {
+  findDisplay,
+  getPrimaryDisplay,
+  listDisplays,
+  parseDarwinDisplays,
+  parseHyprlandMonitors,
+  parseSwayOutputs,
+  parseSystemProfilerDisplays,
+  parseWindowsScreens,
+  parseXrandrMonitors,
+  refreshDisplays,
+} from "../platform/displays.js";
+
+function tryXrandr(): string | null {
+  try {
+    return execFileSync("xrandr", ["--listmonitors"], {
+      timeout: 3000,
+      encoding: "utf-8",
+      stdio: ["ignore", "pipe", "pipe"],
+    });
+  } catch {
+    return null;
+  }
+}
+
+describe("displays — xrandr parser", () => {
+  it("parses the canonical single-display layout", () => {
+    const fixture = `Monitors: 1\n 0: +*eDP-1 2560/390x1600/240+0+0  eDP-1\n`;
+    const parsed = parseXrandrMonitors(fixture);
+    expect(parsed).toHaveLength(1);
+    expect(parsed[0]).toMatchObject({
+      id: 0,
+      bounds: [0, 0, 2560, 1600],
+      scaleFactor: 1,
+      primary: true,
+      name: "eDP-1",
+    });
+  });
+
+  it("parses a multi-display side-by-side layout", () => {
+    const fixture = [
+      "Monitors: 2",
+      " 0: +*eDP-1 2560/390x1600/240+0+0  eDP-1",
+      " 1: +HDMI-0 3840/600x2160/340+2560+0  HDMI-0",
+      "",
+    ].join("\n");
+    const parsed = parseXrandrMonitors(fixture);
+    expect(parsed).toHaveLength(2);
+    expect(parsed[0]).toMatchObject({
+      id: 0,
+      bounds: [0, 0, 2560, 1600],
+      primary: true,
+      name: "eDP-1",
+    });
+    expect(parsed[1]).toMatchObject({
+      id: 1,
+      bounds: [2560, 0, 3840, 2160],
+      primary: false,
+      name: "HDMI-0",
+    });
+  });
+
+  it("handles a secondary-primary layout (no asterisk on first)", () => {
+    const fixture = [
+      "Monitors: 2",
+      " 0: +HDMI-0 1920/600x1080/340+0+0  HDMI-0",
+      " 1: +*eDP-1 2560/390x1600/240+1920+0  eDP-1",
+      "",
+    ].join("\n");
+    const parsed = parseXrandrMonitors(fixture);
+    expect(parsed).toHaveLength(2);
+    expect(parsed.filter((d) => d.primary)).toHaveLength(1);
+    expect(parsed.find((d) => d.primary)?.name).toBe("eDP-1");
+  });
+
+  it("handles negative origins (left-of-primary monitor)", () => {
+    const fixture = [
+      "Monitors: 2",
+      " 0: +HDMI-0 1920/600x1080/340-1920+0  HDMI-0",
+      " 1: +*eDP-1 2560/390x1600/240+0+0  eDP-1",
+      "",
+    ].join("\n");
+    const parsed = parseXrandrMonitors(fixture);
+    expect(parsed[0]?.bounds).toEqual([-1920, 0, 1920, 1080]);
+    expect(parsed[1]?.bounds).toEqual([0, 0, 2560, 1600]);
+  });
+
+  it("ignores garbage and empty input", () => {
+    expect(parseXrandrMonitors("")).toEqual([]);
+    expect(parseXrandrMonitors("Monitors: 0\n")).toEqual([]);
+    expect(parseXrandrMonitors("Not the expected format")).toEqual([]);
+  });
+
+  it("validates against the live host when xrandr is available", () => {
+    const live = tryXrandr();
+    if (!live) return;
+    const parsed = parseXrandrMonitors(live);
+    expect(parsed.length).toBeGreaterThan(0);
+    expect(
+      parsed.every((d) => Number.isFinite(d.bounds[2]) && d.bounds[2] > 0),
+    ).toBe(true);
+    expect(parsed.filter((d) => d.primary)).toHaveLength(1);
+  });
+});
+
+describe("displays — Hyprland parser", () => {
+  it("parses a single-monitor Hyprland JSON snapshot", () => {
+    const fixture = JSON.stringify([
+      {
+        id: 0,
+        name: "eDP-1",
+        x: 0,
+        y: 0,
+        width: 2560,
+        height: 1600,
+        scale: 1.5,
+        focused: true,
+      },
+    ]);
+    const parsed = parseHyprlandMonitors(fixture);
+    expect(parsed).toEqual([
+      {
+        id: 0,
+        bounds: [0, 0, 2560, 1600],
+        scaleFactor: 1.5,
+        primary: true,
+        name: "eDP-1",
+      },
+    ]);
+  });
+
+  it("parses a dual-monitor Hyprland snapshot", () => {
+    const fixture = JSON.stringify([
+      {
+        id: 0,
+        name: "eDP-1",
+        x: 0,
+        y: 0,
+        width: 2560,
+        height: 1600,
+        scale: 1.5,
+        focused: true,
+      },
+      {
+        id: 1,
+        name: "DP-2",
+        x: 2560,
+        y: 0,
+        width: 3840,
+        height: 2160,
+        scale: 1,
+        focused: false,
+      },
+    ]);
+    const parsed = parseHyprlandMonitors(fixture);
+    expect(parsed).toHaveLength(2);
+    expect(parsed[0]?.primary).toBe(true);
+    expect(parsed[1]?.bounds).toEqual([2560, 0, 3840, 2160]);
+  });
+
+  it("rejects malformed JSON cleanly", () => {
+    expect(parseHyprlandMonitors("not json")).toEqual([]);
+    expect(parseHyprlandMonitors("{}")).toEqual([]);
+  });
+});
+
+describe("displays — Sway parser", () => {
+  it("parses sway get_outputs JSON", () => {
+    const fixture = JSON.stringify([
+      {
+        name: "eDP-1",
+        focused: true,
+        rect: { x: 0, y: 0, width: 2560, height: 1600 },
+        scale: 2,
+      },
+      {
+        name: "HDMI-A-1",
+        focused: false,
+        rect: { x: 2560, y: 0, width: 1920, height: 1080 },
+        scale: 1,
+      },
+    ]);
+    const parsed = parseSwayOutputs(fixture);
+    expect(parsed).toHaveLength(2);
+    expect(parsed[0]?.primary).toBe(true);
+    expect(parsed[0]?.scaleFactor).toBe(2);
+    expect(parsed[1]?.bounds).toEqual([2560, 0, 1920, 1080]);
+  });
+});
+
+describe("displays — macOS system_profiler parser", () => {
+  it("parses a single-display macOS report", () => {
+    const fixture = JSON.stringify({
+      SPDisplaysDataType: [
+        {
+          spdisplays_ndrvs: [
+            {
+              _name: "Built-in Retina Display",
+              spdisplays_resolution: "1512 x 982",
+              spdisplays_pixelresolution: "3024 x 1964",
+              spdisplays_main: "spdisplays_yes",
+            },
+          ],
+        },
+      ],
+    });
+    const parsed = parseSystemProfilerDisplays(fixture);
+    expect(parsed).toHaveLength(1);
+    expect(parsed[0]).toMatchObject({
+      id: 0,
+      bounds: [0, 0, 1512, 982],
+      scaleFactor: 2,
+      primary: true,
+      name: "Built-in Retina Display",
+    });
+  });
+
+  it("parses a dual-display retina + external 4K layout", () => {
+    const fixture = JSON.stringify({
+      SPDisplaysDataType: [
+        {
+          spdisplays_ndrvs: [
+            {
+              _name: "Built-in Retina Display",
+              spdisplays_resolution: "1512 x 982",
+              spdisplays_pixelresolution: "3024 x 1964",
+              spdisplays_main: "spdisplays_yes",
+            },
+            {
+              _name: "LG UltraFine",
+              spdisplays_resolution: "3840 x 2160",
+              spdisplays_pixelresolution: "3840 x 2160",
+            },
+          ],
+        },
+      ],
+    });
+    const parsed = parseSystemProfilerDisplays(fixture);
+    expect(parsed).toHaveLength(2);
+    expect(parsed[0]?.scaleFactor).toBe(2);
+    expect(parsed[1]?.scaleFactor).toBe(1);
+    // Cursor-style layout: each display gets x offset by previous widths.
+    expect(parsed[1]?.bounds[0]).toBe(1512);
+  });
+});
+
+describe("displays — parseDarwinDisplays JXA round-trip", () => {
+  it("parses a JXA-style JSON envelope", () => {
+    const fixture = JSON.stringify([
+      {
+        id: 69733382,
+        bounds: { x: 0, y: 0, width: 2560, height: 1440 },
+        pixelWidth: 5120,
+        pixelHeight: 2880,
+        primary: true,
+        name: "main",
+      },
+    ]);
+    const parsed = parseDarwinDisplays(fixture);
+    expect(parsed[0]).toMatchObject({
+      id: 69733382,
+      bounds: [0, 0, 2560, 1440],
+      scaleFactor: 2,
+      primary: true,
+      name: "main",
+    });
+  });
+});
+
+describe("displays — Windows PowerShell parser", () => {
+  it("parses a single-display object (PowerShell collapses arrays of length 1)", () => {
+    const fixture = JSON.stringify({
+      DeviceName: "\\\\.\\DISPLAY1",
+      Primary: true,
+      Bounds: { X: 0, Y: 0, Width: 2560, Height: 1440 },
+    });
+    const parsed = parseWindowsScreens(fixture);
+    expect(parsed).toEqual([
+      {
+        id: 0,
+        bounds: [0, 0, 2560, 1440],
+        scaleFactor: 1,
+        primary: true,
+        name: "\\\\.\\DISPLAY1",
+      },
+    ]);
+  });
+
+  it("parses a multi-display array", () => {
+    const fixture = JSON.stringify([
+      {
+        DeviceName: "\\\\.\\DISPLAY1",
+        Primary: true,
+        Bounds: { X: 0, Y: 0, Width: 2560, Height: 1440 },
+      },
+      {
+        DeviceName: "\\\\.\\DISPLAY2",
+        Primary: false,
+        Bounds: { X: 2560, Y: 0, Width: 1920, Height: 1080 },
+      },
+    ]);
+    const parsed = parseWindowsScreens(fixture);
+    expect(parsed).toHaveLength(2);
+    expect(parsed[0]?.primary).toBe(true);
+    expect(parsed[1]?.bounds).toEqual([2560, 0, 1920, 1080]);
+  });
+});
+
+describe("displays — live enumeration on this host", () => {
+  it("returns at least one display from listDisplays()", () => {
+    const displays = refreshDisplays();
+    expect(displays.length).toBeGreaterThan(0);
+    expect(displays.filter((d) => d.primary)).toHaveLength(1);
+    expect(displays.every((d) => d.bounds[2] > 0 && d.bounds[3] > 0)).toBe(
+      true,
+    );
+  });
+
+  it("getPrimaryDisplay matches a listDisplays entry", () => {
+    const primary = getPrimaryDisplay();
+    const all = listDisplays();
+    expect(all.some((d) => d.id === primary.id)).toBe(true);
+    expect(primary.primary).toBe(true);
+  });
+
+  it("findDisplay returns null for unknown id", () => {
+    expect(findDisplay(987654321)).toBeNull();
+  });
+});
+
+describe("coords — translation", () => {
+  it("localToGlobal adds display origin (logical coords)", () => {
+    // Force fresh enumeration so listDisplays is current.
+    const displays = refreshDisplays();
+    const primary = displays.find((d) => d.primary)!;
+    const result = localToGlobal({ displayId: primary.id, x: 100, y: 200 });
+    expect(result.x).toBe(primary.bounds[0] + 100);
+    expect(result.y).toBe(primary.bounds[1] + 200);
+  });
+
+  it("localToGlobalDefault falls back to primary on missing displayId", () => {
+    const primary = getPrimaryDisplay();
+    const result = localToGlobalDefault({ x: 10, y: 20 });
+    expect(result.x).toBe(primary.bounds[0] + 10);
+    expect(result.y).toBe(primary.bounds[1] + 20);
+  });
+
+  it("globalToLocal inverts localToGlobal for in-bounds points", () => {
+    const primary = getPrimaryDisplay();
+    const global = { x: primary.bounds[0] + 50, y: primary.bounds[1] + 60 };
+    const local = globalToLocal(global);
+    expect(local).not.toBeNull();
+    expect(local?.displayId).toBe(primary.id);
+    expect(local?.x).toBe(50);
+    expect(local?.y).toBe(60);
+  });
+
+  it("globalToLocal returns null for points outside any display", () => {
+    const result = globalToLocal({ x: -99999, y: -99999 });
+    expect(result).toBeNull();
+  });
+
+  it("clampToDisplay clamps within bounds", () => {
+    const primary = getPrimaryDisplay();
+    const clamped = clampToDisplay({
+      displayId: primary.id,
+      x: 99999,
+      y: -50,
+    });
+    expect(clamped.x).toBe(primary.bounds[2] - 1);
+    expect(clamped.y).toBe(0);
+  });
+
+  it("rejects unknown displayId", () => {
+    expect(() => localToGlobal({ displayId: 999_999_999, x: 0, y: 0 })).toThrow(
+      /Unknown displayId/,
+    );
+  });
+});

plugins/plugin-computeruse/src/__tests__/file-bytes.test.ts

@@ -0,0 +1,105 @@
+/**
+ * Binary filesystem primitives (#9170 — trycua/cua read_bytes / write_bytes /
+ * create_dir / directory_exists / get_file_size). Pure Node fs over the safe-path
+ * guard, so this runs in the DEFAULT lane on Windows / Linux / macOS alike.
+ */
+
+import { mkdtempSync, rmSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { afterEach, beforeEach, describe, expect, it } from "vitest";
+import {
+  createDirectory,
+  directoryExists,
+  getFileSize,
+  readBytes,
+  writeBytes,
+} from "../platform/file-ops.js";
+
+describe("binary file ops (read_bytes / write_bytes)", () => {
+  let dir: string;
+  beforeEach(() => {
+    dir = mkdtempSync(join(tmpdir(), "cu-bytes-"));
+  });
+  afterEach(() => {
+    rmSync(dir, { recursive: true, force: true });
+  });
+
+  it("round-trips arbitrary bytes (incl. non-UTF8) via base64", async () => {
+    const raw = Buffer.from([0x00, 0xff, 0x10, 0x7f, 0x80, 0xfe, 0x01]);
+    const b64 = raw.toString("base64");
+    const file = join(dir, "blob.bin");
+
+    const w = await writeBytes(file, b64);
+    expect(w.success).toBe(true);
+    expect(w.size).toBe(raw.length);
+
+    const r = await readBytes(file);
+    expect(r.success).toBe(true);
+    expect(r.size).toBe(raw.length);
+    expect(Buffer.from(r.bytes ?? "", "base64").equals(raw)).toBe(true);
+  });
+
+  it("reads a byte window with offset + length", async () => {
+    const raw = Buffer.from([10, 11, 12, 13, 14, 15]);
+    const file = join(dir, "win.bin");
+    await writeBytes(file, raw.toString("base64"));
+
+    const r = await readBytes(file, 2, 3);
+    expect(r.success).toBe(true);
+    expect(
+      Buffer.from(r.bytes ?? "", "base64").equals(Buffer.from([12, 13, 14])),
+    ).toBe(true);
+  });
+
+  it("write_bytes creates parent directories", async () => {
+    const file = join(dir, "a", "b", "c.bin");
+    const w = await writeBytes(file, Buffer.from([1, 2, 3]).toString("base64"));
+    expect(w.success).toBe(true);
+    const r = await readBytes(file);
+    expect(r.size).toBe(3);
+  });
+});
+
+describe("create_dir / directory_exists / get_file_size", () => {
+  let dir: string;
+  beforeEach(() => {
+    dir = mkdtempSync(join(tmpdir(), "cu-fs-"));
+  });
+  afterEach(() => {
+    rmSync(dir, { recursive: true, force: true });
+  });
+
+  it("create_dir makes a (recursive) directory; directory_exists confirms it", async () => {
+    const sub = join(dir, "x", "y", "z");
+    const c = await createDirectory(sub);
+    expect(c.success).toBe(true);
+    expect(c.is_directory).toBe(true);
+
+    const d = await directoryExists(sub);
+    expect(d.success).toBe(true);
+    expect(d.exists).toBe(true);
+    expect(d.is_directory).toBe(true);
+  });
+
+  it("directory_exists is false for a missing path and for a regular file", async () => {
+    const missing = await directoryExists(join(dir, "nope"));
+    expect(missing.exists).toBe(false);
+
+    const file = join(dir, "f.bin");
+    await writeBytes(file, Buffer.from([9]).toString("base64"));
+    const onFile = await directoryExists(file);
+    expect(onFile.exists).toBe(false);
+    expect(onFile.is_directory).toBe(false);
+  });
+
+  it("get_file_size returns the byte length of a file", async () => {
+    const file = join(dir, "sz.bin");
+    const raw = Buffer.from([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]);
+    await writeBytes(file, raw.toString("base64"));
+    const s = await getFileSize(file);
+    expect(s.success).toBe(true);
+    expect(s.size).toBe(raw.length);
+    expect(s.is_file).toBe(true);
+  });
+});

plugins/plugin-computeruse/src/__tests__/file-ops.real.test.ts

@@ -0,0 +1,167 @@
+/**
+ * Real integration tests for file operations.
+ * Uses temp directories — no mocks needed.
+ */
+
+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { afterEach, beforeEach, describe, expect, it } from "vitest";
+import {
+  appendFile,
+  deleteFile,
+  editFile,
+  fileExists,
+  listDirectory,
+  readFile,
+  writeFile,
+} from "../platform/file-ops.js";
+
+describe("file operations (real)", () => {
+  let tempDir: string;
+
+  beforeEach(() => {
+    tempDir = mkdtempSync(join(tmpdir(), "computeruse-fileops-test-"));
+  });
+
+  afterEach(() => {
+    rmSync(tempDir, { recursive: true, force: true });
+  });
+
+  describe("readFile", () => {
+    it("reads an existing file", async () => {
+      const filePath = join(tempDir, "test.txt");
+      writeFileSync(filePath, "hello world");
+
+      const result = await readFile(filePath);
+      expect(result.success).toBe(true);
+      expect(result.content).toBe("hello world");
+    });
+
+    it("returns error for non-existent file", async () => {
+      const result = await readFile(join(tempDir, "nope.txt"));
+      expect(result.success).toBe(false);
+      expect(result.error).toBeDefined();
+    });
+  });
+
+  describe("writeFile", () => {
+    it("creates a new file", async () => {
+      const filePath = join(tempDir, "new.txt");
+      const result = await writeFile(filePath, "new content");
+      expect(result.success).toBe(true);
+
+      const read = await readFile(filePath);
+      expect(read.content).toBe("new content");
+    });
+
+    it("creates parent directories", async () => {
+      const filePath = join(tempDir, "sub", "dir", "deep.txt");
+      const result = await writeFile(filePath, "deep");
+      expect(result.success).toBe(true);
+    });
+
+    it("overwrites existing file", async () => {
+      const filePath = join(tempDir, "overwrite.txt");
+      writeFileSync(filePath, "old");
+      await writeFile(filePath, "new");
+
+      const read = await readFile(filePath);
+      expect(read.content).toBe("new");
+    });
+  });
+
+  describe("editFile", () => {
+    it("replaces text in a file", async () => {
+      const filePath = join(tempDir, "edit.txt");
+      writeFileSync(filePath, "hello world");
+
+      const result = await editFile(filePath, "world", "universe");
+      expect(result.success).toBe(true);
+
+      const read = await readFile(filePath);
+      expect(read.content).toBe("hello universe");
+    });
+
+    it("returns error when old text not found", async () => {
+      const filePath = join(tempDir, "edit2.txt");
+      writeFileSync(filePath, "hello");
+
+      const result = await editFile(filePath, "missing", "replacement");
+      expect(result.success).toBe(false);
+    });
+  });
+
+  describe("appendFile", () => {
+    it("appends content to a file", async () => {
+      const filePath = join(tempDir, "append.txt");
+      writeFileSync(filePath, "line1\n");
+
+      const result = await appendFile(filePath, "line2\n");
+      expect(result.success).toBe(true);
+
+      const read = await readFile(filePath);
+      expect(read.content).toBe("line1\nline2\n");
+    });
+  });
+
+  describe("deleteFile", () => {
+    it("deletes an existing file", async () => {
+      const filePath = join(tempDir, "delete.txt");
+      writeFileSync(filePath, "delete me");
+
+      const result = await deleteFile(filePath);
+      expect(result.success).toBe(true);
+
+      const exists = await fileExists(filePath);
+      expect(exists.exists).toBe(false);
+    });
+  });
+
+  describe("fileExists", () => {
+    it("detects existing file", async () => {
+      const filePath = join(tempDir, "exists.txt");
+      writeFileSync(filePath, "hi");
+
+      const result = await fileExists(filePath);
+      expect(result.exists).toBe(true);
+      expect(result.isFile).toBe(true);
+      expect(result.isDirectory).toBe(false);
+    });
+
+    it("detects existing directory", async () => {
+      const dirPath = join(tempDir, "subdir");
+      mkdirSync(dirPath);
+
+      const result = await fileExists(dirPath);
+      expect(result.exists).toBe(true);
+      expect(result.isDirectory).toBe(true);
+    });
+
+    it("returns exists=false for non-existent path", async () => {
+      const result = await fileExists(join(tempDir, "nope"));
+      expect(result.exists).toBe(false);
+    });
+  });
+
+  describe("listDirectory", () => {
+    it("lists directory contents", async () => {
+      writeFileSync(join(tempDir, "a.txt"), "a");
+      writeFileSync(join(tempDir, "b.txt"), "b");
+      mkdirSync(join(tempDir, "subdir"));
+
+      const result = await listDirectory(tempDir);
+      expect(result.success).toBe(true);
+      expect(result.items).toBeDefined();
+      expect(result.items?.length).toBeGreaterThanOrEqual(3);
+
+      const names = result.items?.map((e) => e.name);
+      expect(names).toContain("a.txt");
+      expect(names).toContain("b.txt");
+      expect(names).toContain("subdir");
+
+      const subdir = result.items?.find((e) => e.name === "subdir");
+      expect(subdir?.type).toBe("directory");
+    });
+  });
+});

plugins/plugin-computeruse/src/__tests__/helpers-ps-timeout-floor.test.ts

@@ -0,0 +1,77 @@
+/**
+ * Behavioral guard for the central PowerShell spawn-timeout floor in
+ * `helpers.ts#runCommand` / `runCommandBuffer`.
+ *
+ * Regression coverage for #9581: the legacy-driver desktop paths in
+ * `desktop.ts` spawn `powershell` through `runCommand` with small hardcoded
+ * budgets (e.g. the 5s `System.Windows.Forms.Cursor` position query that
+ * `driverGetCursorPosition` always uses on Windows). On a Defender-heavy host a
+ * cold `powershell.exe` spawn pays a ~10-16s AV-scan tax, so 5s ETIMEDOUTs and
+ * the capability false-fails (`cua-parity-input.real` died at
+ * `legacyGetCursorPosition`). #10100 floored the `windows-list.ts` spawns but
+ * not these; flooring `runCommand` centrally covers every legacy-driver
+ * PowerShell spawn at once. The mock keeps this a normal cross-platform unit
+ * test (no real spawn), so it fails loudly if the floor is ever dropped.
+ */
+
+import { afterEach, describe, expect, it, vi } from "vitest";
+
+const { execFileSyncMock } = vi.hoisted(() => ({
+  execFileSyncMock: vi.fn(() => ""),
+}));
+
+vi.mock("node:child_process", () => ({
+  execFileSync: execFileSyncMock,
+  execSync: vi.fn(() => ""),
+}));
+
+import { runCommand, runCommandBuffer } from "../platform/helpers.js";
+
+const ENV = "ELIZA_COMPUTERUSE_PS_TIMEOUT_MS";
+
+afterEach(() => {
+  delete process.env[ENV];
+  execFileSyncMock.mockClear();
+});
+
+function lastTimeout(): number {
+  const call = execFileSyncMock.mock.calls.at(-1);
+  return (call?.[2] as { timeout: number }).timeout;
+}
+
+describe("runCommand PowerShell spawn-timeout floor", () => {
+  it("raises a powershell timeout to the ELIZA_COMPUTERUSE_PS_TIMEOUT_MS floor", () => {
+    process.env[ENV] = "30000";
+    runCommand("powershell", ["-Command", "x"], 5000);
+    expect(lastTimeout()).toBe(30000);
+  });
+
+  it("also raises pwsh spawns", () => {
+    process.env[ENV] = "30000";
+    runCommand("pwsh", ["-Command", "x"], 5000);
+    expect(lastTimeout()).toBe(30000);
+  });
+
+  it("only ever raises — never lowers a base above the floor", () => {
+    process.env[ENV] = "1000";
+    runCommand("powershell", ["-Command", "x"], 5000);
+    expect(lastTimeout()).toBe(5000);
+  });
+
+  it("does not touch non-powershell commands", () => {
+    process.env[ENV] = "30000";
+    runCommand("cliclick", ["p:."], 5000);
+    expect(lastTimeout()).toBe(5000);
+  });
+
+  it("is a no-op when the env var is unset", () => {
+    runCommand("powershell", ["-Command", "x"], 5000);
+    expect(lastTimeout()).toBe(5000);
+  });
+
+  it("floors runCommandBuffer powershell spawns too", () => {
+    process.env[ENV] = "30000";
+    runCommandBuffer("powershell", ["-Command", "x"], 5000);
+    expect(lastTimeout()).toBe(30000);
+  });
+});

plugins/plugin-computeruse/src/__tests__/helpers.test.ts

@@ -0,0 +1,212 @@
+/**
+ * Unit coverage for the pure platform helpers (#9170 / #9105).
+ *
+ * These functions are the input-sanitization + key-normalization layer that
+ * sits between agent-supplied params and the per-OS shell drivers
+ * (AppleScript / xdotool / PowerShell SendKeys). Several are security
+ * boundaries — `validateInt`, `validateWindowId`, `escapeAppleScript`,
+ * `safeXdotoolKey`, and `validateKeypress` exist specifically to stop shell /
+ * AppleScript injection — and shipped with no regression lock. Pure +
+ * deterministic, so they run in the default vitest lane on every OS.
+ */
+
+import { platform } from "node:os";
+import { describe, expect, it } from "vitest";
+import {
+  canonicalKeyName,
+  currentPlatform,
+  escapeAppleScript,
+  safeXdotoolKey,
+  toCliclickKeyName,
+  toWindowsSendKey,
+  toXdotoolKeyName,
+  validateCoordinate,
+  validateInt,
+  validateKeypress,
+  validateText,
+  validateWindowId,
+} from "../platform/helpers.js";
+
+describe("validateInt", () => {
+  it("rounds finite numbers and numeric strings", () => {
+    expect(validateInt(42)).toBe(42);
+    expect(validateInt(42.7)).toBe(43);
+    expect(validateInt(-3.2)).toBe(-3);
+    expect(validateInt("100")).toBe(100);
+    expect(validateInt("1e3")).toBe(1000);
+  });
+
+  it("rejects non-numeric / non-finite / empty input (injection guard)", () => {
+    expect(() => validateInt("12; rm -rf /")).toThrow(/Invalid numeric/);
+    expect(() => validateInt("$(whoami)")).toThrow(/Invalid numeric/);
+    expect(() => validateInt("12abc")).toThrow(/Invalid numeric/);
+    expect(() => validateInt("")).toThrow(/Invalid numeric/);
+    expect(() => validateInt("   ")).toThrow(/Invalid numeric/);
+    expect(() => validateInt(Number.NaN)).toThrow(/Invalid numeric/);
+    expect(() => validateInt(Number.POSITIVE_INFINITY)).toThrow(
+      /Invalid numeric/,
+    );
+    expect(() => validateInt(null)).toThrow(/Invalid numeric/);
+    expect(() => validateInt(undefined)).toThrow(/Invalid numeric/);
+  });
+});
+
+describe("validateCoordinate", () => {
+  it("clamps into [0, max] on each axis", () => {
+    expect(validateCoordinate(50, 60, 100, 100)).toEqual([50, 60]);
+    expect(validateCoordinate(-10, 5, 100, 100)).toEqual([0, 5]);
+    expect(validateCoordinate(5, 999, 100, 100)).toEqual([5, 100]);
+    expect(validateCoordinate(150, 150, 100, 80)).toEqual([100, 80]);
+  });
+
+  it("rounds through validateInt and rejects junk", () => {
+    expect(validateCoordinate(10.6, 20.4, 100, 100)).toEqual([11, 20]);
+    expect(() => validateCoordinate(Number.NaN, 0, 100, 100)).toThrow();
+  });
+});
+
+describe("validateWindowId (shell-literal injection guard)", () => {
+  it("accepts decimal and 0x-hex ids, trimming whitespace", () => {
+    expect(validateWindowId("12345")).toBe("12345");
+    expect(validateWindowId("  42 ")).toBe("42");
+    expect(validateWindowId("0xAF09")).toBe("0xAF09");
+    expect(validateWindowId("0xff")).toBe("0xff");
+  });
+
+  it("rejects anything that could escape the surrounding script literal", () => {
+    for (const bad of [
+      "12; ls",
+      "$(whoami)",
+      "`id`",
+      '1" or "1',
+      "0x",
+      "12 34",
+      "abc",
+      "",
+      "-1",
+      "1.5",
+    ]) {
+      expect(() => validateWindowId(bad), bad).toThrow(/Invalid windowId/);
+    }
+  });
+});
+
+describe("validateText", () => {
+  it("passes strings within the length budget", () => {
+    expect(validateText("hello")).toBe("hello");
+    expect(validateText("x".repeat(4096))).toHaveLength(4096);
+    expect(validateText("ab", 2)).toBe("ab");
+  });
+
+  it("rejects over-long input and non-strings", () => {
+    expect(() => validateText("x".repeat(4097))).toThrow(/too long/);
+    expect(() => validateText("abc", 2)).toThrow(/too long/);
+    // @ts-expect-error — runtime guard for non-string callers
+    expect(() => validateText(123)).toThrow(/must be a string/);
+  });
+});
+
+describe("escapeAppleScript (AppleScript-literal injection guard)", () => {
+  it("wraps in quotes and escapes backslashes then double-quotes", () => {
+    expect(escapeAppleScript("hello")).toBe('"hello"');
+    expect(escapeAppleScript('say "hi"')).toBe('"say \\"hi\\""');
+    expect(escapeAppleScript("C:\\path")).toBe('"C:\\\\path"');
+  });
+
+  it("neutralizes an attempt to break out of the literal", () => {
+    const escaped = escapeAppleScript('"; do shell script "rm -rf /');
+    // The whole payload stays inside one quoted literal: it begins and ends
+    // with an unescaped quote, and every interior quote is backslash-escaped.
+    expect(escaped.startsWith('"')).toBe(true);
+    expect(escaped.endsWith('"')).toBe(true);
+    expect(escaped.slice(1, -1)).not.toMatch(/(?<!\\)"/); // no bare interior quote
+  });
+});
+
+describe("safeXdotoolKey (key whitelist)", () => {
+  it("accepts known key names and single printable ASCII chars", () => {
+    expect(safeXdotoolKey("Return")).toBe("Return");
+    expect(safeXdotoolKey(" F5 ")).toBe("F5");
+    expect(safeXdotoolKey("a")).toBe("a");
+    expect(safeXdotoolKey("5")).toBe("5");
+    expect(safeXdotoolKey("+")).toBe("+");
+  });
+
+  it("rejects unknown multi-char names and non-ASCII (injection guard)", () => {
+    expect(() => safeXdotoolKey("Return; rm -rf /")).toThrow(/Invalid key/);
+    expect(() => safeXdotoolKey("notakey")).toThrow(/Invalid key/);
+    expect(() => safeXdotoolKey("")).toThrow(/Invalid key/);
+    expect(() => safeXdotoolKey("é")).toThrow(/Invalid key/); // > 126
+  });
+});
+
+describe("validateKeypress (charset guard)", () => {
+  it("accepts combos of alphanumerics + the allowed punctuation", () => {
+    expect(validateKeypress("ctrl+a")).toBe("ctrl+a");
+    expect(validateKeypress("shift+F5")).toBe("shift+F5");
+    expect(validateKeypress("A-Z 0-9 .,:_+-")).toBe("A-Z 0-9 .,:_+-");
+  });
+
+  it("rejects shell metacharacters, empty, and over-long input", () => {
+    for (const bad of ["a;b", "a|b", "a$b", "a`b", "a/b", "a(b)", "a&b"]) {
+      expect(() => validateKeypress(bad), bad).toThrow(/invalid characters/);
+    }
+    expect(() => validateKeypress("")).toThrow(/non-empty/);
+    expect(() => validateKeypress("a".repeat(129))).toThrow(/too long/);
+  });
+});
+
+describe("canonicalKeyName", () => {
+  it("normalizes aliases, casing, arrow-prefix, and separators", () => {
+    expect(canonicalKeyName("Esc")).toBe("escape");
+    expect(canonicalKeyName("RETURN")).toBe("enter");
+    expect(canonicalKeyName("ArrowUp")).toBe("up");
+    expect(canonicalKeyName("Page_Up")).toBe("pageup");
+    expect(canonicalKeyName("spacebar")).toBe("space");
+    expect(canonicalKeyName(" del ")).toBe("delete");
+  });
+
+  it("passes function keys through and leaves unknown keys normalized", () => {
+    expect(canonicalKeyName("F5")).toBe("f5");
+    expect(canonicalKeyName("ctrl")).toBe("ctrl");
+    expect(canonicalKeyName("a")).toBe("a");
+  });
+
+  it("rejects empty input", () => {
+    expect(() => canonicalKeyName("  ")).toThrow(/non-empty/);
+  });
+});
+
+describe("per-OS key name mappers", () => {
+  it("toCliclickKeyName maps canonical names, else passes through", () => {
+    expect(toCliclickKeyName("escape")).toBe("esc");
+    expect(toCliclickKeyName("Enter")).toBe("return");
+    expect(toCliclickKeyName("ArrowDown")).toBe("arrow-down");
+    expect(toCliclickKeyName("f7")).toBe("f7");
+    expect(toCliclickKeyName("a")).toBe("a");
+  });
+
+  it("toXdotoolKeyName maps names and uppercases function keys", () => {
+    expect(toXdotoolKeyName("escape")).toBe("Escape");
+    expect(toXdotoolKeyName("enter")).toBe("Return");
+    expect(toXdotoolKeyName("f9")).toBe("F9");
+    expect(toXdotoolKeyName("a")).toBe("a");
+  });
+
+  it("toWindowsSendKey wraps special + function keys, passes chars through", () => {
+    expect(toWindowsSendKey("escape")).toBe("{ESC}");
+    expect(toWindowsSendKey("Enter")).toBe("{ENTER}");
+    expect(toWindowsSendKey("pageup")).toBe("{PGUP}");
+    expect(toWindowsSendKey("f12")).toBe("{F12}");
+    expect(toWindowsSendKey("space")).toBe(" ");
+    expect(toWindowsSendKey("a")).toBe("a");
+  });
+});
+
+describe("currentPlatform", () => {
+  it("reports the running OS as a typed union member", () => {
+    const os = currentPlatform();
+    expect(["darwin", "linux", "win32"]).toContain(os);
+    expect(os).toBe(platform());
+  });
+});

plugins/plugin-computeruse/src/__tests__/ios-bridge.test.ts

@@ -0,0 +1,420 @@
+/**
+ * Pure-JS tests for the iOS bridge contract, the AppIntent registry, and the
+ * OCR provider chain. These do not exercise any Swift code — that is unverified
+ * on this host and lives behind the on-device validation checklist in
+ * `docs/IOS_CONSTRAINTS.md`.
+ */
+
+import { beforeEach, describe, expect, it } from "vitest";
+import {
+  _resetOcrProvidersForTests,
+  createIosVisionOcrProvider,
+  findIosAppIntent,
+  findIosAppIntentsForBundle,
+  type IntentSpec,
+  IOS_APP_GROUP_ID,
+  IOS_APP_INTENT_BUNDLE_IDS,
+  IOS_APP_INTENT_REGISTRY,
+  IOS_BRIDGE_JS_NAME,
+  type IosBridgeResult,
+  type IosComputerUseBridge,
+  listIosAppIntents,
+  listOcrProviders,
+  REPLAYKIT_FOREGROUND_MAX_BUFFER,
+  REPLAYKIT_FOREGROUND_MAX_SESSION_SEC,
+  registerOcrProvider,
+  selectOcrProvider,
+  unregisterOcrProvider,
+  type VisionOcrResult,
+} from "../mobile/index.js";
+
+// ── Bridge constants ─────────────────────────────────────────────────────────
+
+describe("iOS bridge constants", () => {
+  it("uses the App Group id that matches App.entitlements", () => {
+    expect(IOS_APP_GROUP_ID).toBe("group.ai.elizaos.app");
+  });
+  it("exposes the Capacitor plugin jsName", () => {
+    expect(IOS_BRIDGE_JS_NAME).toBe("ComputerUse");
+  });
+  it("caps ReplayKit foreground buffering at 30 frames and 30 seconds", () => {
+    expect(REPLAYKIT_FOREGROUND_MAX_BUFFER).toBe(30);
+    expect(REPLAYKIT_FOREGROUND_MAX_SESSION_SEC).toBe(30);
+  });
+});
+
+// ── AppIntent registry ───────────────────────────────────────────────────────
+
+describe("iOS AppIntent registry", () => {
+  it("covers all the apps WS9 requires", () => {
+    expect(IOS_APP_INTENT_BUNDLE_IDS).toEqual(
+      expect.arrayContaining([
+        "com.apple.mobilemail",
+        "com.apple.MobileSMS",
+        "com.apple.mobilenotes",
+        "com.apple.reminders",
+        "com.apple.Music",
+        "com.apple.Maps",
+        "com.apple.mobilesafari",
+      ]),
+    );
+  });
+
+  it("returns frozen registry entries", () => {
+    const registry = IOS_APP_INTENT_REGISTRY as Record<
+      string,
+      readonly IntentSpec[]
+    >;
+    for (const bundleId of IOS_APP_INTENT_BUNDLE_IDS) {
+      expect(Array.isArray(registry[bundleId])).toBe(true);
+      expect(registry[bundleId].length).toBeGreaterThan(0);
+    }
+  });
+
+  it("each intent declares a stable id, displayName, and parameter list", () => {
+    for (const intent of listIosAppIntents()) {
+      expect(intent.id).toMatch(/^com\.apple\./);
+      expect(intent.bundleId).toMatch(/^com\.apple\./);
+      expect(intent.displayName.length).toBeGreaterThan(0);
+      expect(intent.summary.length).toBeGreaterThan(0);
+      expect(Array.isArray(intent.parameters)).toBe(true);
+      for (const param of intent.parameters) {
+        expect(typeof param.name).toBe("string");
+        expect([
+          "string",
+          "number",
+          "boolean",
+          "date",
+          "url",
+          "enum",
+        ]).toContain(param.type);
+        expect(typeof param.required).toBe("boolean");
+        if (param.type === "enum") {
+          expect(Array.isArray(param.enumValues)).toBe(true);
+          expect(param.enumValues?.length).toBeGreaterThan(0);
+        }
+      }
+      expect(["donated", "system"]).toContain(intent.source);
+    }
+  });
+
+  it("intent ids are unique across all bundles", () => {
+    const ids = listIosAppIntents().map((i) => i.id);
+    expect(new Set(ids).size).toBe(ids.length);
+  });
+
+  it("findIosAppIntent finds a known system intent", () => {
+    const intent = findIosAppIntent("com.apple.mobilenotes.create-note");
+    expect(intent).toBeDefined();
+    expect(intent?.bundleId).toBe("com.apple.mobilenotes");
+    expect(intent?.parameters.find((p) => p.name === "body")).toBeDefined();
+  });
+
+  it("findIosAppIntent returns undefined for unknown ids", () => {
+    expect(findIosAppIntent("com.example.does-not-exist")).toBeUndefined();
+  });
+
+  it("findIosAppIntentsForBundle returns only that bundle's intents", () => {
+    const mail = findIosAppIntentsForBundle("com.apple.mobilemail");
+    expect(mail.length).toBeGreaterThan(0);
+    for (const intent of mail) {
+      expect(intent.bundleId).toBe("com.apple.mobilemail");
+    }
+    expect(findIosAppIntentsForBundle("com.example.does-not-exist")).toEqual(
+      [],
+    );
+  });
+
+  it("Maps directions intent declares the expected transport enum", () => {
+    const maps = findIosAppIntent("com.apple.Maps.directions");
+    expect(maps).toBeDefined();
+    const transport = maps?.parameters.find((p) => p.name === "transport");
+    expect(transport?.type).toBe("enum");
+    expect(transport?.enumValues).toEqual(
+      expect.arrayContaining(["driving", "walking", "transit", "cycling"]),
+    );
+  });
+
+  it("intents with at least one parameter either declare a required one or are list/control verbs", () => {
+    const listOrControlSuffixes = [".pause", ".next-track", ".list-reminders"];
+    for (const intent of listIosAppIntents()) {
+      const isListOrControl = listOrControlSuffixes.some((suffix) =>
+        intent.id.endsWith(suffix),
+      );
+      if (isListOrControl) continue;
+      if (intent.parameters.length === 0) continue;
+      const required = intent.parameters.filter((p) => p.required);
+      expect(required.length).toBeGreaterThan(0);
+    }
+  });
+});
+
+// ── OCR provider chain ───────────────────────────────────────────────────────
+
+function fakeBridge(
+  visionImpl: (args: {
+    imageBase64: string;
+    options?: unknown;
+  }) => Promise<IosBridgeResult<VisionOcrResult>>,
+): IosComputerUseBridge {
+  // Minimal mock: only `visionOcr` is exercised here.
+  return {
+    probe: () =>
+      Promise.resolve({
+        ok: true,
+        data: {
+          platform: "ios",
+          osVersion: "26.1",
+          capabilities: {
+            replayKitForeground: true,
+            broadcastExtension: false,
+            visionOcr: true,
+            appIntents: true,
+            accessibilityRead: true,
+            foundationModel: false,
+          },
+        },
+      }),
+    replayKitForegroundStart: () =>
+      Promise.resolve({
+        ok: false,
+        code: "internal_error",
+        message: "unavailable",
+      }),
+    replayKitForegroundStop: () =>
+      Promise.resolve({
+        ok: false,
+        code: "internal_error",
+        message: "unavailable",
+      }),
+    replayKitForegroundDrain: () =>
+      Promise.resolve({
+        ok: false,
+        code: "internal_error",
+        message: "unavailable",
+      }),
+    broadcastExtensionHandshake: () =>
+      Promise.resolve({
+        ok: false,
+        code: "internal_error",
+        message: "unavailable",
+      }),
+    visionOcr: visionImpl,
+    appIntentList: () => Promise.resolve({ ok: true, data: { intents: [] } }),
+    appIntentInvoke: () =>
+      Promise.resolve({
+        ok: false,
+        code: "internal_error",
+        message: "unavailable",
+      }),
+    accessibilitySnapshot: () =>
+      Promise.resolve({
+        ok: false,
+        code: "internal_error",
+        message: "unavailable",
+      }),
+    foundationModelGenerate: () =>
+      Promise.resolve({
+        ok: false,
+        code: "foundation_model_unavailable",
+        message: "unavailable",
+      }),
+    memoryPressureProbe: () =>
+      Promise.resolve({
+        ok: true,
+        data: {
+          source: "ios-uikit",
+          capturedAt: Date.now(),
+          severity: 0,
+          availableMb: 1024,
+          broadcastActive: false,
+        },
+      }),
+  };
+}
+
+describe("OCR provider chain", () => {
+  beforeEach(() => {
+    _resetOcrProvidersForTests();
+  });
+
+  it("selectOcrProvider throws when nothing is registered", () => {
+    expect(() => selectOcrProvider()).toThrow(/No OCR provider available/);
+  });
+
+  it("listOcrProviders is sorted by priority descending", () => {
+    registerOcrProvider({
+      name: "low",
+      priority: 10,
+      available: () => true,
+      recognize: async () => ({
+        lines: [],
+        fullText: "",
+        elapsedMs: 0,
+        providerName: "low",
+        languagesUsed: [],
+      }),
+    });
+    registerOcrProvider({
+      name: "high",
+      priority: 100,
+      available: () => true,
+      recognize: async () => ({
+        lines: [],
+        fullText: "",
+        elapsedMs: 0,
+        providerName: "high",
+        languagesUsed: [],
+      }),
+    });
+    const ordered = listOcrProviders().map((p) => p.name);
+    expect(ordered).toEqual(["high", "low"]);
+  });
+
+  it("selectOcrProvider skips unavailable providers", () => {
+    registerOcrProvider({
+      name: "high-unavail",
+      priority: 100,
+      available: () => false,
+      recognize: async () => {
+        throw new Error("should not be called");
+      },
+    });
+    registerOcrProvider({
+      name: "low-avail",
+      priority: 10,
+      available: () => true,
+      recognize: async () => ({
+        lines: [],
+        fullText: "",
+        elapsedMs: 0,
+        providerName: "low-avail",
+        languagesUsed: [],
+      }),
+    });
+    expect(selectOcrProvider().name).toBe("low-avail");
+  });
+
+  it("createIosVisionOcrProvider reports unavailable when bridge is null", () => {
+    const provider = createIosVisionOcrProvider(() => null);
+    expect(provider.available()).toBe(false);
+  });
+
+  it("createIosVisionOcrProvider passes through Vision results", async () => {
+    const bridge = fakeBridge(async ({ imageBase64 }) => {
+      expect(imageBase64).toBe("aGVsbG8="); // "hello"
+      return {
+        ok: true,
+        data: {
+          lines: [
+            {
+              text: "WS9 OCR SMOKE",
+              confidence: 0.99,
+              boundingBox: { x: 0.1, y: 0.2, width: 0.3, height: 0.05 },
+            },
+          ],
+          fullText: "WS9 OCR SMOKE",
+          elapsedMs: 42,
+          languagesUsed: ["en-US"],
+        },
+      };
+    });
+    const provider = createIosVisionOcrProvider(() => bridge);
+    expect(provider.available()).toBe(true);
+    const result = await provider.recognize(
+      { kind: "base64", data: "aGVsbG8=" },
+      { recognitionLevel: "accurate", languages: ["en-US"] },
+    );
+    expect(result.fullText).toBe("WS9 OCR SMOKE");
+    expect(result.lines).toHaveLength(1);
+    expect(result.providerName).toBe("ios-apple-vision");
+    expect(result.languagesUsed).toEqual(["en-US"]);
+  });
+
+  it("createIosVisionOcrProvider throws on bridge error", async () => {
+    const bridge = fakeBridge(async () => ({
+      ok: false,
+      code: "permission_denied",
+      message: "user said no",
+    }));
+    const provider = createIosVisionOcrProvider(() => bridge);
+    await expect(
+      provider.recognize({ kind: "base64", data: "aGVsbG8=" }),
+    ).rejects.toThrow(/permission_denied/);
+  });
+
+  it("createIosVisionOcrProvider converts byte input to base64", async () => {
+    let captured: string | undefined;
+    const bridge = fakeBridge(async ({ imageBase64 }) => {
+      captured = imageBase64;
+      return {
+        ok: true,
+        data: {
+          lines: [],
+          fullText: "",
+          elapsedMs: 1,
+          languagesUsed: ["auto"],
+        },
+      };
+    });
+    const provider = createIosVisionOcrProvider(() => bridge);
+    const bytes = new Uint8Array([104, 101, 108, 108, 111]); // "hello"
+    await provider.recognize({ kind: "bytes", data: bytes });
+    expect(captured).toBe("aGVsbG8=");
+  });
+
+  it("unregisterOcrProvider removes the provider", () => {
+    registerOcrProvider({
+      name: "temp",
+      priority: 5,
+      available: () => true,
+      recognize: async () => ({
+        lines: [],
+        fullText: "",
+        elapsedMs: 0,
+        providerName: "temp",
+        languagesUsed: [],
+      }),
+    });
+    expect(listOcrProviders()).toHaveLength(1);
+    unregisterOcrProvider("temp");
+    expect(listOcrProviders()).toHaveLength(0);
+  });
+});
+
+// ── Type contract sanity ─────────────────────────────────────────────────────
+
+describe("Bridge type contract", () => {
+  it("IosBridgeResult discriminates on ok flag", () => {
+    const success: IosBridgeResult<number> = { ok: true, data: 7 };
+    const failure: IosBridgeResult<number> = {
+      ok: false,
+      code: "internal_error",
+      message: "x",
+    };
+    if (success.ok) {
+      expect(success.data).toBe(7);
+    } else {
+      throw new Error("unreachable");
+    }
+    if (!failure.ok) {
+      expect(failure.code).toBe("internal_error");
+    } else {
+      throw new Error("unreachable");
+    }
+  });
+});
+
+// ── Runtime feature-detect ───────────────────────────────────────────────────
+
+describe("iOS bridge featureCheck", () => {
+  it("reports unsupported when Capacitor is absent (Node test host)", async () => {
+    const { featureCheck, getIosBridge } = await import(
+      "../mobile/ios-bridge.js"
+    );
+    const result = featureCheck();
+    expect(result.supported).toBe(false);
+    expect(result.reason).toMatch(/Capacitor/i);
+    expect(getIosBridge()).toBeNull();
+  });
+});

plugins/plugin-computeruse/src/__tests__/ios-computer-interface.test.ts

@@ -0,0 +1,338 @@
+/**
+ * WS9 cross-review — IosComputerInterface.
+ *
+ * Validates:
+ *   1. screenshot drains a frame from `replayKitForegroundDrain`.
+ *   2. input-bearing methods throw with a "use invokeAppIntent" message.
+ *   3. invokeAppIntent forwards to `bridge.appIntentInvoke`.
+ *   4. metadata helpers (coord conversion, getScreenSize) work without a bridge call.
+ *   5. AccessibilityTree comes from the scene accessor (own-app AX only).
+ */
+
+import { describe, expect, it } from "vitest";
+import type {
+  IntentInvocationRequest,
+  IosBridgeResult,
+  IosComputerUseBridge,
+} from "../mobile/ios-bridge.js";
+import {
+  IOS_LOGICAL_DISPLAY_ID,
+  IosComputerInterface,
+  makeIosComputerInterface,
+} from "../mobile/ios-computer-interface.js";
+import type { Scene } from "../scene/scene-types.js";
+
+function fakeBridge(
+  overrides: Partial<IosComputerUseBridge> = {},
+): IosComputerUseBridge {
+  const generic = <T>(): Promise<IosBridgeResult<T>> =>
+    Promise.resolve({
+      ok: false,
+      code: "internal_error",
+      message: "unavailable",
+    });
+  return {
+    probe: () =>
+      Promise.resolve({
+        ok: true,
+        data: {
+          platform: "ios",
+          osVersion: "26.1",
+          capabilities: {
+            replayKitForeground: true,
+            broadcastExtension: false,
+            visionOcr: true,
+            appIntents: true,
+            accessibilityRead: true,
+            foundationModel: true,
+          },
+        },
+      }),
+    replayKitForegroundStart: generic,
+    replayKitForegroundStop: generic,
+    replayKitForegroundDrain: () =>
+      Promise.resolve({
+        ok: true,
+        data: {
+          frames: [
+            {
+              timestampNs: 1,
+              width: 1170,
+              height: 2532,
+              jpegBase64: Buffer.from([0xff, 0xd8, 0xff]).toString("base64"),
+            },
+          ],
+        },
+      }),
+    broadcastExtensionHandshake: generic,
+    visionOcr: generic,
+    appIntentList: () => Promise.resolve({ ok: true, data: { intents: [] } }),
+    appIntentInvoke: (req: IntentInvocationRequest) =>
+      Promise.resolve({
+        ok: true,
+        data: {
+          intentId: req.intentId,
+          success: true,
+          response: { echo: req.parameters },
+          elapsedMs: 12,
+        },
+      }),
+    accessibilitySnapshot: generic,
+    foundationModelGenerate: generic,
+    memoryPressureProbe: generic,
+    ...overrides,
+  };
+}
+
+describe("IosComputerInterface — screenshot", () => {
+  it("drains the first frame from replayKitForegroundDrain", async () => {
+    const bridge = fakeBridge();
+    const iface = new IosComputerInterface({
+      getBridge: () => bridge,
+      getReplayKitSessionId: () => "rk-session-1",
+    });
+    const shot = await iface.screenshot();
+    expect(shot.displayId).toBe(IOS_LOGICAL_DISPLAY_ID);
+    expect(shot.bounds).toEqual([0, 0, 1170, 2532]);
+    expect(shot.scaleFactor).toBe(1);
+    expect(shot.frame[0]).toBe(0xff);
+    expect(shot.frame[1]).toBe(0xd8);
+  });
+
+  it("throws when no ReplayKit session is active", async () => {
+    const bridge = fakeBridge();
+    const iface = new IosComputerInterface({
+      getBridge: () => bridge,
+      getReplayKitSessionId: () => null,
+    });
+    await expect(iface.screenshot()).rejects.toThrow(
+      /requires an active ReplayKit session/,
+    );
+  });
+
+  it("throws when the bridge returns no frames", async () => {
+    const bridge = fakeBridge({
+      replayKitForegroundDrain: () =>
+        Promise.resolve({ ok: true, data: { frames: [] } }),
+    });
+    const iface = new IosComputerInterface({
+      getBridge: () => bridge,
+      getReplayKitSessionId: () => "rk-session-1",
+    });
+    await expect(iface.screenshot()).rejects.toThrow(/returned no frames/);
+  });
+
+  it("surfaces bridge errors with code + message", async () => {
+    const bridge = fakeBridge({
+      replayKitForegroundDrain: () =>
+        Promise.resolve({
+          ok: false,
+          code: "extension_died",
+          message: "iOS-26 beta regression",
+        }),
+    });
+    const iface = new IosComputerInterface({
+      getBridge: () => bridge,
+      getReplayKitSessionId: () => "rk-session-1",
+    });
+    await expect(iface.screenshot()).rejects.toThrow(/extension_died.*iOS-26/);
+  });
+});
+
+describe("IosComputerInterface — input refused with redirect", () => {
+  const cases: ReadonlyArray<
+    readonly [string, (i: IosComputerInterface) => Promise<unknown>]
+  > = [
+    ["leftClick", (i) => i.leftClick({ displayId: 0, x: 10, y: 20 })],
+    ["rightClick", (i) => i.rightClick({ displayId: 0, x: 10, y: 20 })],
+    ["doubleClick", (i) => i.doubleClick({ displayId: 0, x: 10, y: 20 })],
+    ["dragTo", (i) => i.dragTo({ displayId: 0, x: 10, y: 20 })],
+    [
+      "drag",
+      (i) =>
+        i.drag({
+          displayId: 0,
+          path: [
+            { x: 0, y: 0 },
+            { x: 100, y: 100 },
+          ],
+        }),
+    ],
+    ["scroll", (i) => i.scroll({ displayId: 0, x: 10, y: 20, dx: 0, dy: 1 })],
+    ["scrollUp", (i) => i.scrollUp({ displayId: 0, clicks: 1 })],
+    ["typeText", (i) => i.typeText({ text: "hi" })],
+    ["pressKey", (i) => i.pressKey({ key: "a" })],
+    ["hotkey", (i) => i.hotkey({ keys: ["cmd", "a"] })],
+  ];
+  for (const [name, fn] of cases) {
+    it(`${name} throws with "not supported on iOS" message`, async () => {
+      const iface = new IosComputerInterface({ getBridge: () => fakeBridge() });
+      // Pointer-input refusal redirects to App Intents; keyboard-input
+      // refusal points at "keyboards are app-local only". Either message
+      // is a valid refusal — both point the caller at IOS_CONSTRAINTS.md.
+      await expect(fn(iface)).rejects.toThrow(
+        /not supported on iOS.*IOS_CONSTRAINTS\.md/i,
+      );
+    });
+  }
+});
+
+describe("IosComputerInterface — invokeAppIntent", () => {
+  it("forwards intent id and parameters to the bridge", async () => {
+    let received: IntentInvocationRequest | null = null;
+    const bridge = fakeBridge({
+      appIntentInvoke: (req) => {
+        received = req;
+        return Promise.resolve({
+          ok: true,
+          data: {
+            intentId: req.intentId,
+            success: true,
+            elapsedMs: 5,
+          },
+        });
+      },
+    });
+    const iface = new IosComputerInterface({ getBridge: () => bridge });
+    const result = await iface.invokeAppIntent({
+      intentId: "com.apple.mobilenotes.create-note",
+      parameters: { body: "hello", title: "test" },
+    });
+    expect(result.success).toBe(true);
+    expect(result.intentId).toBe("com.apple.mobilenotes.create-note");
+    expect(received).not.toBeNull();
+    expect(received?.parameters).toEqual({ body: "hello", title: "test" });
+  });
+
+  it("throws with structured code + message on bridge failure", async () => {
+    const bridge = fakeBridge({
+      appIntentInvoke: () =>
+        Promise.resolve({
+          ok: false,
+          code: "intent_not_found",
+          message: "no such intent on this device",
+        }),
+    });
+    const iface = new IosComputerInterface({ getBridge: () => bridge });
+    await expect(
+      iface.invokeAppIntent({
+        intentId: "com.unknown.app.do-thing",
+        parameters: {},
+      }),
+    ).rejects.toThrow(/intent_not_found.*no such intent/);
+  });
+
+  it("throws when the bridge is not registered", async () => {
+    const iface = new IosComputerInterface({ getBridge: () => null });
+    await expect(
+      iface.invokeAppIntent({ intentId: "x", parameters: {} }),
+    ).rejects.toThrow(/bridge is not registered/);
+  });
+});
+
+describe("IosComputerInterface — metadata + AX", () => {
+  it("getScreenSize returns the display bounds without a bridge call", () => {
+    const iface = new IosComputerInterface({
+      getBridge: () => null,
+      getDisplay: () => ({
+        id: IOS_LOGICAL_DISPLAY_ID,
+        bounds: [0, 0, 1290, 2796],
+        scaleFactor: 3,
+        primary: true,
+        name: "iphone-15-pro-max",
+      }),
+    });
+    expect(iface.getScreenSize({ displayId: IOS_LOGICAL_DISPLAY_ID })).toEqual({
+      w: 1290,
+      h: 2796,
+    });
+  });
+
+  it("toScreenCoordinates scales from image space into display space", () => {
+    const iface = new IosComputerInterface({
+      getBridge: () => null,
+      getDisplay: () => ({
+        id: IOS_LOGICAL_DISPLAY_ID,
+        bounds: [0, 0, 1000, 2000],
+        scaleFactor: 1,
+        primary: true,
+        name: "fake",
+      }),
+    });
+    const p = iface.toScreenCoordinates({
+      displayId: IOS_LOGICAL_DISPLAY_ID,
+      imgX: 100,
+      imgY: 200,
+      imgW: 500,
+      imgH: 1000,
+    });
+    expect(p).toEqual({ x: 200, y: 400 });
+  });
+
+  it("getAccessibilityTree returns scene AX scoped to the iOS display", () => {
+    const scene: Scene = {
+      timestamp: 1,
+      displays: [
+        {
+          id: IOS_LOGICAL_DISPLAY_ID,
+          bounds: [0, 0, 1170, 2532],
+          scaleFactor: 1,
+          primary: true,
+          name: "ios",
+        },
+      ],
+      focused_window: null,
+      apps: [],
+      ocr: [],
+      ax: [
+        {
+          id: "a1",
+          role: "button",
+          label: "OK",
+          bbox: [0, 0, 100, 40],
+          displayId: IOS_LOGICAL_DISPLAY_ID,
+        },
+        {
+          id: "a2",
+          role: "label",
+          label: "Other",
+          bbox: [0, 0, 100, 40],
+          displayId: 99,
+        },
+      ],
+      vlm_scene: null,
+      vlm_elements: null,
+    };
+    const iface = new IosComputerInterface({
+      getBridge: () => null,
+      getScene: () => scene,
+    });
+    const allNodes = iface.getAccessibilityTree({});
+    expect(allNodes).toHaveLength(2);
+    const onlyIos = iface.getAccessibilityTree({
+      displayId: IOS_LOGICAL_DISPLAY_ID,
+    });
+    expect(onlyIos).toHaveLength(1);
+    expect(onlyIos[0]?.label).toBe("OK");
+  });
+
+  it("rejects unknown displayId", () => {
+    const iface = new IosComputerInterface({ getBridge: () => null });
+    expect(() =>
+      iface.toScreenCoordinates({
+        displayId: 99,
+        imgX: 0,
+        imgY: 0,
+        imgW: 100,
+        imgH: 100,
+      }),
+    ).toThrow(/unknown iOS displayId 99/);
+  });
+});
+
+describe("makeIosComputerInterface factory", () => {
+  it("returns an IosComputerInterface instance", () => {
+    const iface = makeIosComputerInterface({ getBridge: () => fakeBridge() });
+    expect(iface).toBeInstanceOf(IosComputerInterface);
+  });
+});

plugins/plugin-computeruse/src/__tests__/m12-verbs.test.ts

@@ -0,0 +1,121 @@
+/**
+ * M12 verb parity (#9170): open / launch + window getters.
+ *
+ * The pure guards (empty target/app, set_bounds param validation, application
+ * filter) run in the default lane; a real benign `launch` proves a pid comes
+ * back. The action-surface promotion is asserted against the plugin. Platform
+ * window queries (active window, real resize) are exercised by the gated
+ * real-driver evidence lane.
+ */
+
+import { describe, expect, it } from "vitest";
+import { computerUsePlugin } from "../index.js";
+import { killApp, launchApp, openTarget } from "../platform/launch.js";
+import {
+  getApplicationWindows,
+  resizeWindow,
+} from "../platform/windows-list.js";
+
+const actionNames = (computerUsePlugin.actions ?? []).map((a) => a.name);
+
+describe("open / launch guards", () => {
+  it("openTarget rejects an empty target", async () => {
+    await expect(openTarget("")).rejects.toThrow(/non-empty target/);
+    await expect(openTarget("   ")).rejects.toThrow(/non-empty target/);
+  });
+
+  it("launchApp rejects an empty app", async () => {
+    await expect(launchApp("")).rejects.toThrow(/non-empty app/);
+  });
+
+  it("launchApp returns a pid for a real executable", async () => {
+    // Spawn the current JS runtime as a benign, immediately-exiting child.
+    const result = await launchApp(process.execPath, ["-e", "0"]);
+    expect(typeof result.pid).toBe("number");
+    expect(result.pid).toBeGreaterThan(0);
+    expect(result.command).toBe(process.execPath);
+  });
+
+  it("killApp rejects an empty target", async () => {
+    await expect(killApp("")).rejects.toThrow(/non-empty target/);
+    await expect(killApp("   ")).rejects.toThrow(/non-empty target/);
+  });
+
+  it("launch then kill_app round-trips by pid (real process)", async () => {
+    // Spawn a benign long-lived child, then terminate it by pid.
+    const launched = await launchApp(process.execPath, [
+      "-e",
+      "setTimeout(()=>{}, 60000)",
+    ]);
+    expect(launched.pid).toBeGreaterThan(0);
+    const killed = await killApp(String(launched.pid));
+    expect(killed.killed).toBe(true);
+    expect(killed.pid).toBe(launched.pid);
+  });
+
+  it("launchApp rejects a non-existent executable path", async () => {
+    await expect(
+      launchApp("/nonexistent/definitely-not-a-real-binary-xyz"),
+    ).rejects.toThrow();
+  });
+});
+
+describe("window getters", () => {
+  it("getApplicationWindows returns [] for an empty app name", () => {
+    expect(getApplicationWindows("")).toEqual([]);
+    expect(getApplicationWindows("   ")).toEqual([]);
+  });
+
+  it("getApplicationWindows always returns an array", () => {
+    expect(Array.isArray(getApplicationWindows("definitely-no-such-app"))).toBe(
+      true,
+    );
+  });
+
+  it("resizeWindow validates x/y before touching the OS", async () => {
+    // resizeWindow is async (it may route through the warm PowerShell host), so
+    // the validation surfaces as a rejection rather than a synchronous throw.
+    await expect(
+      resizeWindow("w1", undefined as unknown as number, 5),
+    ).rejects.toThrow(/x and y are required/);
+    await expect(
+      resizeWindow("w1", 5, undefined as unknown as number),
+    ).rejects.toThrow(/x and y are required/);
+  });
+});
+
+describe("action-surface promotion (M12)", () => {
+  it("promotes open / launch / kill_app under COMPUTER_USE", () => {
+    expect(actionNames, `actions: ${actionNames.join(", ")}`).toContain(
+      "COMPUTER_USE_OPEN",
+    );
+    expect(actionNames).toContain("COMPUTER_USE_LAUNCH");
+    expect(actionNames).toContain("COMPUTER_USE_KILL_APP");
+  });
+
+  it("promotes the new window getters under WINDOW", () => {
+    for (const verb of [
+      "WINDOW_GET_CURRENT_WINDOW_ID",
+      "WINDOW_GET_APPLICATION_WINDOWS",
+      "WINDOW_SET_BOUNDS",
+      "WINDOW_GET_WINDOW_SIZE",
+      "WINDOW_GET_WINDOW_POSITION",
+    ]) {
+      expect(actionNames, `actions: ${actionNames.join(", ")}`).toContain(verb);
+    }
+  });
+});
+
+describe("window bounds getter (get_window_size / get_window_position)", () => {
+  it("exposes get_window_size + get_window_position as window verbs", () => {
+    // Real bounds read against a live window is exercised by the gated
+    // real-driver lane; here we assert the verbs are wired into the surface.
+    expect(actionNames).toContain("WINDOW_GET_WINDOW_SIZE");
+    expect(actionNames).toContain("WINDOW_GET_WINDOW_POSITION");
+  });
+
+  it("getWindowBounds is exported by the window platform module", async () => {
+    const mod = await import("../platform/windows-list.js");
+    expect(typeof mod.getWindowBounds).toBe("function");
+  });
+});

plugins/plugin-computeruse/src/__tests__/macos-desktop-evidence-capture.test.ts

@@ -0,0 +1,37 @@
+/**
+ * Classifier that maps macOS Accessibility/TCC blocker messages to
+ * requires-device-evidence in the capture harness. Deterministic unit test.
+ */
+import { describe, expect, it } from "vitest";
+import { isMacosAccessibilityEvidenceBlocker } from "../../scripts/capture-macos-desktop-evidence.mjs";
+
+describe("macOS desktop evidence capture", () => {
+  it("classifies known Accessibility/TCC blockers as missing device evidence", () => {
+    expect(
+      isMacosAccessibilityEvidenceBlocker(
+        "list_windows returned only placeholder window metadata; grant Accessibility permission in System Settings > Privacy & Security > Accessibility, then retry",
+      ),
+    ).toBe(true);
+    expect(
+      isMacosAccessibilityEvidenceBlocker("spawnSync osascript ETIMEDOUT"),
+    ).toBe(true);
+    expect(
+      isMacosAccessibilityEvidenceBlocker(
+        "could not read TextEdit bounds: Window not found; listWindows could not resolve the TextEdit window",
+      ),
+    ).toBe(true);
+  });
+
+  it("keeps unrelated capture failures as hard failures", () => {
+    expect(
+      isMacosAccessibilityEvidenceBlocker(
+        "browser_screenshot failed: target closed unexpectedly",
+      ),
+    ).toBe(false);
+    expect(
+      isMacosAccessibilityEvidenceBlocker(
+        "primary display screenshot: screenshot quality failed",
+      ),
+    ).toBe(false);
+  });
+});

plugins/plugin-computeruse/src/__tests__/mcp-tools.test.ts

@@ -0,0 +1,117 @@
+/**
+ * Computer-use MCP server seam (#9170) — pure catalog + dispatch tests. Run in
+ * the DEFAULT lane on all platforms (no MCP SDK, no live desktop): they verify
+ * the MCP tool surface and that every tool routes to the right
+ * ComputerUseService.executeCommand command.
+ */
+
+import { describe, expect, it, vi } from "vitest";
+import {
+  COMPUTERUSE_MCP_TOOLS,
+  type ComputerUseCommandRunner,
+  dispatchComputerUseMcpTool,
+  findComputerUseMcpTool,
+} from "../mcp/tools.js";
+
+function fakeRunner(): ComputerUseCommandRunner & {
+  calls: Array<{ command: string; params?: Record<string, unknown> }>;
+} {
+  const calls: Array<{ command: string; params?: Record<string, unknown> }> =
+    [];
+  return {
+    calls,
+    executeCommand: vi.fn(async (command: string, params = {}) => {
+      calls.push({ command, params });
+      return { success: true, message: `ran ${command}` };
+    }),
+  };
+}
+
+describe("computer-use MCP tool catalog", () => {
+  it("exposes the core CUA verbs with unique names + commands", () => {
+    const names = COMPUTERUSE_MCP_TOOLS.map((t) => t.name);
+    expect(new Set(names).size).toBe(names.length);
+    // A representative slice of the surface must be present.
+    for (const n of [
+      "computer_screenshot",
+      "computer_left_click",
+      "computer_type",
+      "computer_scroll",
+      "computer_drag",
+      "computer_ocr",
+      "computer_set_value",
+      "computer_kill_app",
+    ]) {
+      expect(names, names.join(", ")).toContain(n);
+    }
+  });
+
+  it("every tool maps to a non-empty executeCommand command", () => {
+    for (const t of COMPUTERUSE_MCP_TOOLS) {
+      expect(t.command, t.name).toBeTruthy();
+      expect(typeof t.destructive).toBe("boolean");
+    }
+  });
+
+  it("read-only tools are non-destructive; input tools are destructive", () => {
+    expect(findComputerUseMcpTool("computer_screenshot")?.destructive).toBe(
+      false,
+    );
+    expect(findComputerUseMcpTool("computer_ocr")?.destructive).toBe(false);
+    expect(
+      findComputerUseMcpTool("computer_get_cursor_position")?.destructive,
+    ).toBe(false);
+    expect(findComputerUseMcpTool("computer_left_click")?.destructive).toBe(
+      true,
+    );
+    expect(findComputerUseMcpTool("computer_set_value")?.destructive).toBe(
+      true,
+    );
+  });
+});
+
+describe("dispatchComputerUseMcpTool", () => {
+  it("routes each tool to its executeCommand command with the given args", async () => {
+    const runner = fakeRunner();
+    await dispatchComputerUseMcpTool(runner, "computer_left_click", {
+      coordinate: [12, 34],
+      displayId: 0,
+    });
+    expect(runner.calls).toEqual([
+      { command: "click", params: { coordinate: [12, 34], displayId: 0 } },
+    ]);
+  });
+
+  it("routes set_value / kill_app / ocr to their commands", async () => {
+    const runner = fakeRunner();
+    await dispatchComputerUseMcpTool(runner, "computer_set_value", {
+      coordinate: [1, 2],
+      text: "hi",
+    });
+    await dispatchComputerUseMcpTool(runner, "computer_kill_app", {
+      target: "1234",
+    });
+    await dispatchComputerUseMcpTool(runner, "computer_ocr", { displayId: 1 });
+    expect(runner.calls.map((c) => c.command)).toEqual([
+      "set_value",
+      "kill_app",
+      "ocr",
+    ]);
+  });
+
+  it("every catalog tool dispatches to its declared command", async () => {
+    for (const tool of COMPUTERUSE_MCP_TOOLS) {
+      const runner = fakeRunner();
+      await dispatchComputerUseMcpTool(runner, tool.name, {});
+      expect(runner.calls[0]?.command, tool.name).toBe(tool.command);
+    }
+  });
+
+  it("throws on an unknown tool", async () => {
+    const runner = fakeRunner();
+    await expect(
+      dispatchComputerUseMcpTool(runner, "computer_nonexistent", {}),
+    ).rejects.toThrow(/Unknown computer-use MCP tool/);
+    expect(runner.calls).toHaveLength(0);
+  });
+});

plugins/plugin-computeruse/src/__tests__/mobile-cascade.test.ts

@@ -0,0 +1,399 @@
+/**
+ * WS7 ↔ WS8 — Mobile cascade integration.
+ *
+ * Drives the full agent loop on a fake Android bridge:
+ *   Scene (parsed from Kotlin AX JSON)
+ *     → MobileScreenCaptureSource.captureAllDisplays() (JPEG bytes)
+ *     → Brain (fake runtime)
+ *     → Cascade (`ScreenSeekeR`)
+ *     → dispatch → MobileComputerInterface.leftClick
+ *     → AndroidComputerUseBridge.dispatchGesture(tap)
+ *
+ * Also covers the WS1 ↔ WS2 ↔ WS7 memory-pressure path: when the bridge
+ * dispatches a `critical` pressure level mid-cascade, the test verifies the
+ * pressure listener fires (which the WS1 MemoryArbiter wires to model
+ * eviction) and that the next Brain call goes back through the model
+ * (loaded again from the runtime side).
+ */
+
+import { describe, expect, it } from "vitest";
+import {
+  type ComputerUseAgentReport,
+  runComputerUseAgentLoop,
+} from "../actions/use-computer-agent.js";
+import { OcrCoordinateGroundingActor } from "../actor/actor.js";
+import { Brain } from "../actor/brain.js";
+import { Cascade } from "../actor/cascade.js";
+import { dispatch } from "../actor/dispatch.js";
+import type {
+  AndroidComputerUseBridge,
+  AndroidPressureLevel,
+  CapturedScreenFrame,
+  GestureArgs,
+} from "../mobile/android-bridge.js";
+import { parseAndroidAxTree } from "../mobile/android-scene.js";
+import { MobileComputerInterface } from "../mobile/mobile-computer-interface.js";
+import {
+  ANDROID_LOGICAL_DISPLAY_ID,
+  MobileScreenCaptureSource,
+} from "../mobile/mobile-screen-capture.js";
+import type { Scene } from "../scene/scene-types.js";
+import type { ComputerUseService } from "../services/computer-use-service.js";
+import type { DisplayDescriptor } from "../types.js";
+
+type PressureListener = (level: AndroidPressureLevel, freeMb?: number) => void;
+
+interface FakeAndroidBridgeOpts {
+  axTreeJson?: string;
+  frame?: CapturedScreenFrame;
+  onGesture?: (g: GestureArgs) => void;
+  onMemoryPressure?: PressureListener;
+}
+
+function defaultFrame(): CapturedScreenFrame {
+  return {
+    jpegBase64: Buffer.from([0xff, 0xd8, 0xff, 0xe0, 0xaa, 0xbb]).toString(
+      "base64",
+    ),
+    width: 1080,
+    height: 1920,
+    timestampMs: 0,
+  };
+}
+
+const DEFAULT_AX_JSON = JSON.stringify([
+  {
+    id: "1",
+    role: "android.widget.Button",
+    label: "Save",
+    bbox: { x: 200, y: 400, w: 200, h: 80 },
+    actions: ["click", "focus"],
+  },
+]);
+
+function makeFakeBridge(opts: FakeAndroidBridgeOpts = {}): {
+  bridge: AndroidComputerUseBridge;
+  taps: GestureArgs[];
+  pressureEvents: Array<{ level: AndroidPressureLevel; freeMb?: number }>;
+} {
+  const taps: GestureArgs[] = [];
+  const pressureEvents: Array<{
+    level: AndroidPressureLevel;
+    freeMb?: number;
+  }> = [];
+  const ax = opts.axTreeJson ?? DEFAULT_AX_JSON;
+  const frame = opts.frame ?? defaultFrame();
+  const bridgeFailure = <T>(code = "internal_error" as const) =>
+    Promise.resolve({
+      ok: false as const,
+      code,
+      message: "fake bridge failure",
+    }) as Promise<
+      { ok: false; code: typeof code; message: string } & { data?: T }
+    >;
+  const bridge = {
+    startMediaProjection: () => bridgeFailure(),
+    stopMediaProjection: () => bridgeFailure(),
+    captureFrame: async () => ({ ok: true as const, data: frame }),
+    getAccessibilityTree: async () => ({
+      ok: true as const,
+      data: { nodes: ax },
+    }),
+    dispatchGesture: async (g: GestureArgs) => {
+      taps.push(g);
+      opts.onGesture?.(g);
+      return { ok: true as const, data: { ok: true } };
+    },
+    performGlobalAction: async () => ({
+      ok: true as const,
+      data: { ok: true },
+    }),
+    setText: async () => ({ ok: true as const, data: { ok: true } }),
+    enumerateApps: () => bridgeFailure(),
+    getMemoryPressureSnapshot: () => bridgeFailure(),
+    dispatchMemoryPressure: async ({
+      level,
+      freeMb,
+    }: {
+      level: AndroidPressureLevel;
+      freeMb?: number;
+    }) => {
+      const event = { level, freeMb };
+      pressureEvents.push(event);
+      opts.onMemoryPressure?.(level, freeMb);
+      return { ok: true as const, data: { ok: true } };
+    },
+    startCamera: () => bridgeFailure(),
+    stopCamera: () =>
+      Promise.resolve({ ok: true as const, data: { ok: true } }),
+    captureFrameCamera: () => bridgeFailure(),
+  } as unknown as AndroidComputerUseBridge;
+  return { bridge, taps, pressureEvents };
+}
+
+function androidDisplay(): DisplayDescriptor {
+  return {
+    id: ANDROID_LOGICAL_DISPLAY_ID,
+    bounds: [0, 0, 1080, 1920],
+    scaleFactor: 1,
+    primary: true,
+    name: "android-screen",
+  };
+}
+
+function buildScene(axJson: string = DEFAULT_AX_JSON): Scene {
+  const ax = parseAndroidAxTree(axJson, ANDROID_LOGICAL_DISPLAY_ID);
+  return {
+    timestamp: Date.now(),
+    displays: [androidDisplay()],
+    focused_window: {
+      app: "com.example.app",
+      pid: null,
+      bounds: [0, 0, 1080, 1920],
+      title: "Example",
+      displayId: ANDROID_LOGICAL_DISPLAY_ID,
+    },
+    apps: [],
+    ocr: [],
+    ax,
+    vlm_scene: null,
+    vlm_elements: null,
+  };
+}
+
+function fakeService(scene: Scene): ComputerUseService {
+  return {
+    getCurrentScene: () => scene,
+    refreshScene: async () => scene,
+    getDisplays: () => [androidDisplay()],
+    setSceneVlmAnnotations: () => {},
+  } as unknown as ComputerUseService;
+}
+
+describe("Mobile cascade — end-to-end scene → brain → dispatch → tap", () => {
+  it("Brain `ref:a0-1` resolves to the AX node center and fires a tap", async () => {
+    const { bridge, taps } = makeFakeBridge();
+    const scene = buildScene();
+    const capture = new MobileScreenCaptureSource({ getBridge: () => bridge });
+    const computer = new MobileComputerInterface({
+      getBridge: () => bridge,
+      getDisplay: () => androidDisplay(),
+      getScene: () => scene,
+    });
+    const actor = new OcrCoordinateGroundingActor(() => scene);
+    const brain = new Brain(null, {
+      invokeModel: async () =>
+        JSON.stringify({
+          scene_summary: "Save button visible",
+          target_display_id: ANDROID_LOGICAL_DISPLAY_ID,
+          roi: [],
+          proposed_action: {
+            kind: "click",
+            ref: "a0-1",
+            rationale: "tap save",
+          },
+        }),
+    });
+    const cascade = new Cascade({ brain, actor });
+    const captures = new Map();
+    for (const c of await capture.captureAllDisplays())
+      captures.set(c.display.id, c);
+    const cascadeResult = await cascade.run({
+      scene,
+      goal: "save the document",
+      captures,
+    });
+    expect(cascadeResult.proposed.kind).toBe("click");
+    // bbox [200, 400, 200, 80] → center (300, 440)
+    expect(cascadeResult.proposed.x).toBe(300);
+    expect(cascadeResult.proposed.y).toBe(440);
+    const dispatched = await dispatch(cascadeResult.proposed, {
+      interface: computer,
+      listDisplays: () => [androidDisplay()],
+    });
+    expect(dispatched.success).toBe(true);
+    expect(taps).toEqual([{ type: "tap", x: 300, y: 440 }]);
+  });
+
+  it("runComputerUseAgentLoop walks scene→cascade→tap on the Android stack", async () => {
+    const { bridge, taps } = makeFakeBridge();
+    const scene = buildScene();
+    const capture = new MobileScreenCaptureSource({ getBridge: () => bridge });
+    const computer = new MobileComputerInterface({
+      getBridge: () => bridge,
+      getDisplay: () => androidDisplay(),
+      getScene: () => scene,
+    });
+    let step = 0;
+    const brain = new Brain(null, {
+      invokeModel: async () => {
+        step += 1;
+        if (step === 1) {
+          return JSON.stringify({
+            scene_summary: "Save button visible",
+            target_display_id: ANDROID_LOGICAL_DISPLAY_ID,
+            roi: [],
+            proposed_action: {
+              kind: "click",
+              ref: "a0-1",
+              rationale: "tap save",
+            },
+          });
+        }
+        return JSON.stringify({
+          scene_summary: "saved",
+          target_display_id: ANDROID_LOGICAL_DISPLAY_ID,
+          roi: [],
+          proposed_action: { kind: "finish", rationale: "done" },
+        });
+      },
+    });
+    const report: ComputerUseAgentReport = await runComputerUseAgentLoop(
+      null,
+      { goal: "save the document", maxSteps: 5 },
+      fakeService(scene),
+      {
+        brain,
+        computerInterface: computer,
+        captureAll: () => capture.captureAllDisplays(),
+      },
+    );
+    expect(report.reason).toBe("finish");
+    expect(report.finished).toBe(true);
+    expect(report.steps.length).toBe(2);
+    expect(report.steps[0]?.actionKind).toBe("click");
+    expect(report.steps[1]?.actionKind).toBe("finish");
+    expect(taps).toHaveLength(1);
+    expect(taps[0]?.type).toBe("tap");
+  });
+
+  it("propagates a tap-failed bridge result as `reason: error`", async () => {
+    const baseScene = buildScene();
+    const failingBridge: AndroidComputerUseBridge = {
+      ...makeFakeBridge().bridge,
+      dispatchGesture: async () => ({
+        ok: false,
+        code: "accessibility_unavailable",
+        message: "service off",
+      }),
+    } as AndroidComputerUseBridge;
+    const capture = new MobileScreenCaptureSource({
+      getBridge: () => failingBridge,
+    });
+    const computer = new MobileComputerInterface({
+      getBridge: () => failingBridge,
+      getDisplay: () => androidDisplay(),
+      getScene: () => baseScene,
+    });
+    const brain = new Brain(null, {
+      invokeModel: async () =>
+        JSON.stringify({
+          scene_summary: "S",
+          target_display_id: ANDROID_LOGICAL_DISPLAY_ID,
+          roi: [],
+          proposed_action: {
+            kind: "click",
+            ref: "a0-1",
+            rationale: "tap save",
+          },
+        }),
+    });
+    const report = await runComputerUseAgentLoop(
+      null,
+      { goal: "save", maxSteps: 2 },
+      fakeService(baseScene),
+      {
+        brain,
+        computerInterface: computer,
+        captureAll: () => capture.captureAllDisplays(),
+      },
+    );
+    expect(report.reason).toBe("error");
+    expect(report.error).toMatch(/accessibility_unavailable/);
+  });
+});
+
+describe("Mobile cascade — onTrimMemory → arbiter eviction → Brain reload", () => {
+  it("dispatches `critical` pressure mid-cascade; Brain is re-invoked on the next call", async () => {
+    let pressureListener: PressureListener | null = null;
+    const { bridge } = makeFakeBridge({
+      onMemoryPressure: (level, freeMb) => pressureListener?.(level, freeMb),
+    });
+    const scene = buildScene();
+    const capture = new MobileScreenCaptureSource({ getBridge: () => bridge });
+    const _computer = new MobileComputerInterface({
+      getBridge: () => bridge,
+      getDisplay: () => androidDisplay(),
+      getScene: () => scene,
+    });
+
+    // Track Brain calls + arbiter eviction effects.
+    const brainCallTimes: number[] = [];
+    let visionDescribeLoaded = true; // simulates the resident vision-describe model
+    const evictionLog: Array<{ at: number; reason: string }> = [];
+
+    // Stand-in arbiter: when pressure fires `critical`, evict vision-describe
+    // and mark the next inference as a cold-load.
+    pressureListener = (level) => {
+      if (level === "critical" && visionDescribeLoaded) {
+        visionDescribeLoaded = false;
+        evictionLog.push({ at: Date.now(), reason: "memory-critical" });
+      }
+    };
+
+    let pressureTriggered = false;
+    const brain = new Brain(null, {
+      invokeModel: async () => {
+        brainCallTimes.push(Date.now());
+        // After the first model call, simulate the system firing a memory
+        // trim event from the bridge → JS layer.
+        if (!pressureTriggered) {
+          pressureTriggered = true;
+          await bridge.dispatchMemoryPressure({
+            level: "critical",
+            freeMb: 32,
+          });
+        }
+        // If the vision-describe model was evicted, the next Brain call
+        // must reload it before responding — we model that as a Promise
+        // delay so the order is observable.
+        if (!visionDescribeLoaded) {
+          await new Promise((r) => setTimeout(r, 1));
+          visionDescribeLoaded = true; // re-loaded for this call
+        }
+        return JSON.stringify({
+          scene_summary: "S",
+          target_display_id: ANDROID_LOGICAL_DISPLAY_ID,
+          roi: [],
+          proposed_action: { kind: "wait", rationale: "wait" },
+        });
+      },
+    });
+
+    const cascade = new Cascade({ brain });
+    const captures = new Map();
+    for (const c of await capture.captureAllDisplays())
+      captures.set(c.display.id, c);
+    // First run: triggers the trim event after the model call.
+    await cascade.run({ scene, goal: "g", captures });
+    // Second run: must re-invoke the Brain (reload path).
+    await cascade.run({ scene, goal: "g", captures });
+    expect(brainCallTimes).toHaveLength(2);
+    expect(evictionLog).toHaveLength(1);
+    expect(evictionLog[0]?.reason).toBe("memory-critical");
+    expect(visionDescribeLoaded).toBe(true);
+  });
+
+  it("dispatchMemoryPressure with `nominal` level does not trigger eviction", async () => {
+    let evicted = 0;
+    const { bridge } = makeFakeBridge({
+      onMemoryPressure: (level) => {
+        if (level === "critical") evicted += 1;
+      },
+    });
+    await bridge.dispatchMemoryPressure({ level: "nominal", freeMb: 2048 });
+    expect(evicted).toBe(0);
+    await bridge.dispatchMemoryPressure({ level: "critical" });
+    expect(evicted).toBe(1);
+  });
+});

plugins/plugin-computeruse/src/__tests__/mobile-computer-interface.test.ts

@@ -0,0 +1,459 @@
+/**
+ * WS7 ↔ WS8 — MobileComputerInterface tests.
+ *
+ * Exercises the WS7 `ComputerInterface` port that adapts to Android via
+ * `AndroidComputerUseBridge`. Pure JS — no Capacitor at runtime.
+ *
+ * What we cover:
+ *   - leftClick / doubleClick / rightClick → dispatchGesture(tap)
+ *   - dragTo / drag → dispatchGesture(swipe)
+ *   - scroll → swipe with inverted sign convention
+ *   - pressKey routes the supported `back`/`home`/`recents`/`notifications`
+ *     into performGlobalAction; unsupported keys throw
+ *   - hotkey throws; typeText routes to Android setText
+ *   - screenshot drains captureFrame and decodes JPEG bytes
+ *   - bridge ok:false propagates as a thrown Error
+ *   - dispatch.ts converts those throws to ActionResult.driver_error
+ *     (integration with the desktop dispatcher)
+ */
+
+import { describe, expect, it } from "vitest";
+import { dispatch } from "../actor/dispatch.js";
+import type {
+  AndroidComputerUseBridge,
+  GestureArgs,
+} from "../mobile/android-bridge.js";
+import {
+  MobileComputerInterface,
+  makeMobileComputerInterface,
+} from "../mobile/mobile-computer-interface.js";
+import type { DisplayDescriptor } from "../types.js";
+
+interface BridgeCalls {
+  gestures: GestureArgs[];
+  globalActions: string[];
+  setTexts: string[];
+  capturedFrames: number;
+}
+
+function fakeDisplay(): DisplayDescriptor {
+  return {
+    id: 0,
+    bounds: [0, 0, 1080, 1920],
+    scaleFactor: 1,
+    primary: true,
+    name: "android-screen",
+  };
+}
+
+function makeFakeBridge(opts: {
+  gestureOk?: boolean;
+  gestureError?: {
+    code: "accessibility_unavailable" | "internal_error";
+    message: string;
+  };
+  captureError?: { code: "capture_unavailable"; message: string };
+  globalActionOk?: boolean;
+}): { bridge: AndroidComputerUseBridge; calls: BridgeCalls } {
+  const calls: BridgeCalls = {
+    gestures: [],
+    globalActions: [],
+    setTexts: [],
+    capturedFrames: 0,
+  };
+  const unavailable = <T>(
+    code = "internal_error" as const,
+  ): Promise<
+    { ok: false; code: typeof code; message: string } & { data?: T }
+  > => Promise.resolve({ ok: false, code, message: "unavailable" });
+  const bridge = {
+    startMediaProjection: () => unavailable(),
+    stopMediaProjection: () => unavailable(),
+    captureFrame: async () => {
+      calls.capturedFrames += 1;
+      if (opts.captureError) {
+        return {
+          ok: false as const,
+          code: opts.captureError.code,
+          message: opts.captureError.message,
+        };
+      }
+      return {
+        ok: true as const,
+        data: {
+          jpegBase64: Buffer.from([0xff, 0xd8, 0xff, 0xe0]).toString("base64"),
+          width: 1080,
+          height: 1920,
+          timestampMs: 0,
+        },
+      };
+    },
+    getAccessibilityTree: () => unavailable(),
+    dispatchGesture: async (g: GestureArgs) => {
+      calls.gestures.push(g);
+      if (opts.gestureError) {
+        return {
+          ok: false as const,
+          code: opts.gestureError.code,
+          message: opts.gestureError.message,
+        };
+      }
+      return { ok: true as const, data: { ok: opts.gestureOk ?? true } };
+    },
+    performGlobalAction: async ({ action }: { action: string }) => {
+      calls.globalActions.push(action);
+      return { ok: true as const, data: { ok: opts.globalActionOk ?? true } };
+    },
+    setText: async ({ text }: { text: string }) => {
+      calls.setTexts.push(text);
+      return { ok: true as const, data: { ok: true } };
+    },
+    enumerateApps: () => unavailable(),
+    getMemoryPressureSnapshot: () => unavailable(),
+    dispatchMemoryPressure: () => unavailable(),
+    startCamera: () => unavailable(),
+    stopCamera: () =>
+      Promise.resolve({ ok: true as const, data: { ok: true } }),
+    captureFrameCamera: () => unavailable(),
+  } as unknown as AndroidComputerUseBridge;
+  return { bridge, calls };
+}
+
+function makeIface(opts: Parameters<typeof makeFakeBridge>[0] = {}): {
+  iface: MobileComputerInterface;
+  calls: BridgeCalls;
+} {
+  const { bridge, calls } = makeFakeBridge(opts);
+  const iface = new MobileComputerInterface({
+    getBridge: () => bridge,
+    getDisplay: () => fakeDisplay(),
+  });
+  return { iface, calls };
+}
+
+describe("MobileComputerInterface — gestures", () => {
+  it("leftClick dispatches a tap with the local coords", async () => {
+    const { iface, calls } = makeIface();
+    await iface.leftClick({ displayId: 0, x: 540, y: 960 });
+    expect(calls.gestures).toEqual([{ type: "tap", x: 540, y: 960 }]);
+  });
+
+  it("doubleClick dispatches two taps back-to-back", async () => {
+    const { iface, calls } = makeIface();
+    await iface.doubleClick({ displayId: 0, x: 100, y: 200 });
+    expect(calls.gestures).toHaveLength(2);
+    expect(calls.gestures[0]).toEqual({ type: "tap", x: 100, y: 200 });
+    expect(calls.gestures[1]).toEqual({ type: "tap", x: 100, y: 200 });
+  });
+
+  it("rightClick falls back to a tap (Android has no right-click)", async () => {
+    const { iface, calls } = makeIface();
+    await iface.rightClick({ displayId: 0, x: 50, y: 60 });
+    expect(calls.gestures).toEqual([{ type: "tap", x: 50, y: 60 }]);
+  });
+
+  it("dragTo uses cursorState start + new end → swipe", async () => {
+    const cursorState = { current: { displayId: 0, x: 100, y: 200 } };
+    const { bridge, calls } = makeFakeBridge({});
+    const iface = new MobileComputerInterface({
+      getBridge: () => bridge,
+      getDisplay: () => fakeDisplay(),
+      cursorState,
+    });
+    await iface.dragTo({ displayId: 0, x: 500, y: 800 });
+    expect(calls.gestures).toHaveLength(1);
+    const g = calls.gestures[0];
+    expect(g?.type).toBe("swipe");
+    if (g?.type === "swipe") {
+      expect(g.x).toBe(100);
+      expect(g.y).toBe(200);
+      expect(g.x2).toBe(500);
+      expect(g.y2).toBe(800);
+      expect(g.durationMs).toBe(300);
+    }
+    expect(cursorState.current).toMatchObject({ displayId: 0, x: 500, y: 800 });
+  });
+
+  it("drag (full path) uses first + last points", async () => {
+    const { iface, calls } = makeIface();
+    await iface.drag({
+      displayId: 0,
+      path: [
+        { x: 10, y: 10 },
+        { x: 100, y: 200 },
+        { x: 500, y: 800 },
+      ],
+    });
+    expect(calls.gestures).toHaveLength(1);
+    if (calls.gestures[0]?.type === "swipe") {
+      expect(calls.gestures[0].x).toBe(10);
+      expect(calls.gestures[0].y).toBe(10);
+      expect(calls.gestures[0].x2).toBe(500);
+      expect(calls.gestures[0].y2).toBe(800);
+    }
+  });
+
+  it("drag rejects single-point paths", async () => {
+    const { iface } = makeIface();
+    await expect(
+      iface.drag({ displayId: 0, path: [{ x: 1, y: 1 }] }),
+    ).rejects.toThrow(/at least two points/);
+  });
+
+  it("scroll inverts sign — dy>0 (scroll down) = swipe UP", async () => {
+    const { iface, calls } = makeIface();
+    await iface.scroll({ displayId: 0, x: 540, y: 960, dx: 0, dy: 2 });
+    expect(calls.gestures).toHaveLength(1);
+    if (calls.gestures[0]?.type === "swipe") {
+      expect(calls.gestures[0].x).toBe(540);
+      expect(calls.gestures[0].y).toBe(960);
+      // dy=2 -> endY = 960 - 2*200 = 560 (swipe upward)
+      expect(calls.gestures[0].y2).toBe(560);
+      expect(calls.gestures[0].x2).toBe(540);
+    }
+  });
+
+  it("scroll dy<0 swipes DOWN (content scrolls up)", async () => {
+    const { iface, calls } = makeIface();
+    await iface.scroll({ displayId: 0, x: 540, y: 960, dx: 0, dy: -1 });
+    expect(calls.gestures).toHaveLength(1);
+    if (calls.gestures[0]?.type === "swipe") {
+      // dy=-1 -> endY = 960 - (-1)*200 = 1160
+      expect(calls.gestures[0].y2).toBe(1160);
+    }
+  });
+
+  it("scroll clamps to display bounds", async () => {
+    const { iface, calls } = makeIface();
+    // Large dy that would land off-screen.
+    await iface.scroll({ displayId: 0, x: 540, y: 1900, dx: 0, dy: 20 });
+    if (calls.gestures[0]?.type === "swipe") {
+      // 1900 - 20*200 = -2100 → clamped to 0
+      expect(calls.gestures[0].y2).toBe(0);
+    }
+  });
+
+  it("scroll dx=dy=0 is a no-op", async () => {
+    const { iface, calls } = makeIface();
+    await iface.scroll({ displayId: 0, x: 540, y: 960, dx: 0, dy: 0 });
+    expect(calls.gestures).toHaveLength(0);
+  });
+});
+
+describe("MobileComputerInterface — keyboard", () => {
+  it("pressKey 'back' → performGlobalAction('back')", async () => {
+    const { iface, calls } = makeIface();
+    await iface.pressKey({ key: "back" });
+    expect(calls.globalActions).toEqual(["back"]);
+  });
+
+  it("pressKey 'home' → performGlobalAction('home')", async () => {
+    const { iface, calls } = makeIface();
+    await iface.pressKey({ key: "home" });
+    expect(calls.globalActions).toEqual(["home"]);
+  });
+
+  it("pressKey 'escape' aliases to 'back'", async () => {
+    const { iface, calls } = makeIface();
+    await iface.pressKey({ key: "escape" });
+    expect(calls.globalActions).toEqual(["back"]);
+  });
+
+  it("pressKey on unsupported key throws", async () => {
+    const { iface } = makeIface();
+    await expect(iface.pressKey({ key: "Enter" })).rejects.toThrow(
+      /no Android equivalent/,
+    );
+  });
+
+  it("hotkey throws on Android", async () => {
+    const { iface } = makeIface();
+    await expect(iface.hotkey({ keys: ["ctrl", "s"] })).rejects.toThrow(
+      /not supported/,
+    );
+  });
+
+  it("typeText routes to the focused editable AX node", async () => {
+    const { iface, calls } = makeIface();
+    await iface.typeText({ text: "hi" });
+    expect(calls.setTexts).toEqual(["hi"]);
+  });
+});
+
+describe("MobileComputerInterface — screenshot", () => {
+  it("decodes captureFrame JPEG and returns a ScreenshotResult", async () => {
+    const { iface, calls } = makeIface();
+    const shot = await iface.screenshot({});
+    expect(calls.capturedFrames).toBe(1);
+    expect(shot.displayId).toBe(0);
+    expect(shot.bounds).toEqual([0, 0, 1080, 1920]);
+    expect(shot.frame.subarray(0, 4)).toEqual(
+      Buffer.from([0xff, 0xd8, 0xff, 0xe0]),
+    );
+  });
+
+  it("propagates capture failure as a thrown Error", async () => {
+    const { iface } = makeIface({
+      captureError: { code: "capture_unavailable", message: "no frame" },
+    });
+    await expect(iface.screenshot({})).rejects.toThrow(/capture_unavailable/);
+  });
+});
+
+describe("MobileComputerInterface — error propagation", () => {
+  it("rejects unknown displayId on tap", async () => {
+    const { iface } = makeIface();
+    await expect(
+      iface.leftClick({ displayId: 99, x: 10, y: 10 }),
+    ).rejects.toThrow(/unknown Android displayId 99/);
+  });
+
+  it("rejects non-finite coords on tap", async () => {
+    const { iface } = makeIface();
+    await expect(
+      iface.leftClick({ displayId: 0, x: Number.NaN, y: 0 }),
+    ).rejects.toThrow(/non-finite coords/);
+  });
+
+  it("throws when bridge returns ok:false on dispatchGesture", async () => {
+    const { iface } = makeIface({
+      gestureError: {
+        code: "accessibility_unavailable",
+        message: "service off",
+      },
+    });
+    await expect(
+      iface.leftClick({ displayId: 0, x: 10, y: 10 }),
+    ).rejects.toThrow(/accessibility_unavailable/);
+  });
+
+  it("throws when no bridge is registered", async () => {
+    const iface = new MobileComputerInterface({ getBridge: () => null });
+    await expect(
+      iface.leftClick({ displayId: 0, x: 10, y: 10 }),
+    ).rejects.toThrow(/bridge is not registered/);
+  });
+});
+
+describe("MobileComputerInterface — coord helpers + metadata", () => {
+  it("getScreenSize returns the display bounds w/h", () => {
+    const { iface } = makeIface();
+    expect(iface.getScreenSize({ displayId: 0 })).toEqual({ w: 1080, h: 1920 });
+  });
+
+  it("toScreenCoordinates round-trips with toScreenshotCoordinates", () => {
+    const { iface } = makeIface();
+    const s = iface.toScreenCoordinates({
+      displayId: 0,
+      imgX: 540,
+      imgY: 960,
+      imgW: 1080,
+      imgH: 1920,
+    });
+    expect(s).toEqual({ x: 540, y: 960 });
+    const back = iface.toScreenshotCoordinates({
+      displayId: 0,
+      x: s.x,
+      y: s.y,
+      imgW: 1080,
+      imgH: 1920,
+    });
+    expect(back).toEqual({ imgX: 540, imgY: 960 });
+  });
+
+  it("getAccessibilityTree forwards from the scene accessor", () => {
+    const scene = {
+      timestamp: 1,
+      displays: [fakeDisplay()],
+      focused_window: null,
+      apps: [],
+      ocr: [],
+      ax: [
+        {
+          id: "a0-1",
+          role: "Button",
+          bbox: [0, 0, 10, 10] as [number, number, number, number],
+          actions: [],
+          displayId: 0,
+        },
+      ],
+      vlm_scene: null,
+      vlm_elements: null,
+    };
+    const { bridge } = makeFakeBridge({});
+    const iface = new MobileComputerInterface({
+      getBridge: () => bridge,
+      getDisplay: () => fakeDisplay(),
+      getScene: () => scene,
+    });
+    expect(iface.getAccessibilityTree({})).toHaveLength(1);
+    expect(iface.getAccessibilityTree({ displayId: 0 })).toHaveLength(1);
+    expect(iface.getAccessibilityTree({ displayId: 1 })).toHaveLength(0);
+  });
+});
+
+describe("MobileComputerInterface — global action override map", () => {
+  it("custom globalActionMap takes precedence", async () => {
+    const { bridge, calls } = makeFakeBridge({});
+    const iface = new MobileComputerInterface({
+      getBridge: () => bridge,
+      getDisplay: () => fakeDisplay(),
+      globalActionMap: new Map([["bksp", "back"]]),
+    });
+    await iface.pressKey({ key: "bksp" });
+    expect(calls.globalActions).toEqual(["back"]);
+    // The default `back` alias is replaced because the override map is the sole resolver
+    await expect(iface.pressKey({ key: "back" })).rejects.toThrow(
+      /no Android equivalent/,
+    );
+  });
+});
+
+describe("MobileComputerInterface — integration with WS7 dispatch", () => {
+  it("WS7 dispatch routes a click ProposedAction through the mobile bridge", async () => {
+    const { iface, calls } = makeIface();
+    const res = await dispatch(
+      { kind: "click", displayId: 0, x: 540, y: 960, rationale: "tap save" },
+      {
+        interface: iface,
+        listDisplays: () => [fakeDisplay()],
+      },
+    );
+    expect(res.success).toBe(true);
+    expect(calls.gestures).toEqual([{ type: "tap", x: 540, y: 960 }]);
+  });
+
+  it("WS7 dispatch wraps a mobile-bridge ok:false as driver_error", async () => {
+    const { iface } = makeIface({
+      gestureError: {
+        code: "accessibility_unavailable",
+        message: "service off",
+      },
+    });
+    const res = await dispatch(
+      { kind: "click", displayId: 0, x: 10, y: 10, rationale: "" },
+      { interface: iface, listDisplays: () => [fakeDisplay()] },
+    );
+    expect(res.success).toBe(false);
+    expect(res.error?.code).toBe("driver_error");
+    expect(res.error?.message).toContain("accessibility_unavailable");
+  });
+
+  it("WS7 dispatch on out-of-bounds tap still rejects before calling the bridge", async () => {
+    const { iface, calls } = makeIface();
+    const res = await dispatch(
+      { kind: "click", displayId: 0, x: 9_000, y: 9_000, rationale: "" },
+      { interface: iface, listDisplays: () => [fakeDisplay()] },
+    );
+    expect(res.error?.code).toBe("out_of_bounds");
+    expect(calls.gestures).toHaveLength(0);
+  });
+});
+
+describe("makeMobileComputerInterface factory", () => {
+  it("returns a MobileComputerInterface instance", () => {
+    const { bridge } = makeFakeBridge({});
+    const iface = makeMobileComputerInterface({ getBridge: () => bridge });
+    expect(iface).toBeInstanceOf(MobileComputerInterface);
+  });
+});

plugins/plugin-computeruse/src/__tests__/mobile-screen-capture.test.ts

@@ -0,0 +1,147 @@
+/**
+ * WS8 — MobileScreenCaptureSource tests.
+ *
+ * Exercises the adapter that drains `bridge.captureFrame()` (JPEG base64
+ * envelope) into the WS5-shape `DisplayCapture` the WS7 cascade expects.
+ *
+ * Pure-JS — the Kotlin `ScreenCaptureService` is not exercised here.
+ */
+
+import { describe, expect, it } from "vitest";
+import type {
+  AndroidComputerUseBridge,
+  CapturedScreenFrame,
+} from "../mobile/android-bridge.js";
+import {
+  ANDROID_LOGICAL_DISPLAY_ID,
+  MobileScreenCaptureSource,
+} from "../mobile/mobile-screen-capture.js";
+
+function fakeFrame(
+  overrides: Partial<CapturedScreenFrame> = {},
+): CapturedScreenFrame {
+  return {
+    jpegBase64: Buffer.from([0xff, 0xd8, 0xff, 0xe0, 0xaa, 0xbb]).toString(
+      "base64",
+    ),
+    width: 1080,
+    height: 1920,
+    timestampMs: 1700000000000,
+    ...overrides,
+  };
+}
+
+function fakeBridge(
+  frame:
+    | CapturedScreenFrame
+    | { ok: false; code: "capture_unavailable"; message: string },
+): AndroidComputerUseBridge {
+  const bridgeFailure = <T>(): Promise<
+    { ok: false; code: "internal_error"; message: string } & { data?: T }
+  > =>
+    Promise.resolve({
+      ok: false as const,
+      code: "internal_error" as const,
+      message: "fake bridge failure",
+    });
+  return {
+    startMediaProjection: () => bridgeFailure(),
+    stopMediaProjection: () => bridgeFailure(),
+    captureFrame: async () => {
+      if ("ok" in frame) return frame;
+      return { ok: true, data: frame };
+    },
+    getAccessibilityTree: () => bridgeFailure(),
+    dispatchGesture: () => bridgeFailure(),
+    performGlobalAction: () => bridgeFailure(),
+    setText: () => bridgeFailure(),
+    enumerateApps: () => bridgeFailure(),
+    getMemoryPressureSnapshot: () => bridgeFailure(),
+    dispatchMemoryPressure: () => bridgeFailure(),
+    startCamera: () => bridgeFailure(),
+    stopCamera: () => Promise.resolve({ ok: true, data: { ok: true } }),
+    captureFrameCamera: () => bridgeFailure(),
+  } as AndroidComputerUseBridge;
+}
+
+describe("MobileScreenCaptureSource — happy path", () => {
+  it("captureDisplay returns DisplayCapture with decoded JPEG bytes", async () => {
+    const frame = fakeFrame();
+    const src = new MobileScreenCaptureSource({
+      getBridge: () => fakeBridge(frame),
+    });
+    const cap = await src.captureDisplay();
+    expect(cap.display.id).toBe(ANDROID_LOGICAL_DISPLAY_ID);
+    expect(cap.display.bounds).toEqual([0, 0, 1080, 1920]);
+    expect(cap.display.primary).toBe(true);
+    // Frame is base64-decoded JPEG bytes — first 4 bytes are JPEG SOI + APP0.
+    expect(cap.frame.subarray(0, 4)).toEqual(
+      Buffer.from([0xff, 0xd8, 0xff, 0xe0]),
+    );
+  });
+
+  it("captureAllDisplays returns a single-element array", async () => {
+    const src = new MobileScreenCaptureSource({
+      getBridge: () => fakeBridge(fakeFrame()),
+    });
+    const caps = await src.captureAllDisplays();
+    expect(caps).toHaveLength(1);
+    expect(caps[0]?.display.id).toBe(ANDROID_LOGICAL_DISPLAY_ID);
+  });
+
+  it("getDisplay override is honored", async () => {
+    const customDisplay = {
+      id: 0,
+      bounds: [0, 0, 2000, 3000] as [number, number, number, number],
+      scaleFactor: 2,
+      primary: true,
+      name: "custom",
+    };
+    const src = new MobileScreenCaptureSource({
+      getBridge: () => fakeBridge(fakeFrame()),
+      getDisplay: () => customDisplay,
+    });
+    const cap = await src.captureDisplay();
+    expect(cap.display).toBe(customDisplay);
+  });
+
+  it("decodeJpeg override is honored", async () => {
+    const tag = Buffer.from("decoded-by-test-decoder");
+    const src = new MobileScreenCaptureSource({
+      getBridge: () => fakeBridge(fakeFrame()),
+      decodeJpeg: () => tag,
+    });
+    const cap = await src.captureDisplay();
+    expect(cap.frame).toBe(tag);
+  });
+});
+
+describe("MobileScreenCaptureSource — error paths", () => {
+  it("rejects when no bridge is registered (off-platform)", async () => {
+    const src = new MobileScreenCaptureSource({ getBridge: () => null });
+    await expect(src.captureDisplay()).rejects.toThrow(
+      /bridge is not registered/,
+    );
+  });
+
+  it("rejects on unknown displayId", async () => {
+    const src = new MobileScreenCaptureSource({
+      getBridge: () => fakeBridge(fakeFrame()),
+    });
+    await expect(src.captureDisplay(99)).rejects.toThrow(
+      /unknown Android displayId/,
+    );
+  });
+
+  it("propagates bridge ok:false as a thrown Error", async () => {
+    const src = new MobileScreenCaptureSource({
+      getBridge: () =>
+        fakeBridge({
+          ok: false,
+          code: "capture_unavailable",
+          message: "no frame yet",
+        }),
+    });
+    await expect(src.captureDisplay()).rejects.toThrow(/capture_unavailable/);
+  });
+});

plugins/plugin-computeruse/src/__tests__/normalized-coords.test.ts

@@ -0,0 +1,104 @@
+/**
+ * Unit coverage for the 0–1000 normalized coordinate space (#9105 M2).
+ *
+ * This is the DPI-stable canonical coordinate space the GET_SCREEN envelope and
+ * grounders use so a model's chosen target maps back to a click identically
+ * regardless of capture resolution. Pure + deterministic; was untested.
+ */
+
+import { describe, expect, it } from "vitest";
+import {
+  boxFromNormalized,
+  boxToNormalized,
+  clampNormalized,
+  fromNormalized,
+  NORMALIZED_COORD_MAX,
+  normalizedBoxCenter,
+  toNormalized,
+} from "../platform/normalized-coords.js";
+
+const HD = { width: 1280, height: 720 };
+
+describe("clampNormalized", () => {
+  it("clamps both axes into [0, 1000] and maps NaN to 0", () => {
+    expect(clampNormalized({ nx: 500, ny: 500 })).toEqual({ nx: 500, ny: 500 });
+    expect(clampNormalized({ nx: -10, ny: 1200 })).toEqual({
+      nx: 0,
+      ny: NORMALIZED_COORD_MAX,
+    });
+    expect(clampNormalized({ nx: Number.NaN, ny: Number.NaN })).toEqual({
+      nx: 0,
+      ny: 0,
+    });
+  });
+});
+
+describe("toNormalized / fromNormalized", () => {
+  it("maps pixels to the 0–1000 grid (DPI-independent fraction)", () => {
+    expect(toNormalized({ x: 640, y: 360 }, HD)).toEqual({ nx: 500, ny: 500 });
+    expect(toNormalized({ x: 0, y: 0 }, HD)).toEqual({ nx: 0, ny: 0 });
+    expect(toNormalized({ x: 1280, y: 720 }, HD)).toEqual({
+      nx: 1000,
+      ny: 1000,
+    });
+  });
+
+  it("clamps an off-screen pixel to the nearest edge, not out of range", () => {
+    expect(toNormalized({ x: -100, y: 2000 }, HD)).toEqual({ nx: 0, ny: 1000 });
+  });
+
+  it("returns 0 for a zero-extent axis (no divide-by-zero)", () => {
+    expect(toNormalized({ x: 5, y: 5 }, { width: 0, height: 0 })).toEqual({
+      nx: 0,
+      ny: 0,
+    });
+  });
+
+  it("maps normalized back to pixels", () => {
+    expect(fromNormalized({ nx: 500, ny: 500 }, HD)).toEqual({
+      x: 640,
+      y: 360,
+    });
+    expect(fromNormalized({ nx: 1000, ny: 1000 }, HD)).toEqual({
+      x: 1280,
+      y: 720,
+    });
+  });
+
+  it("round-trips a pixel to within one bucket (1px) of itself", () => {
+    for (const p of [
+      { x: 17, y: 503 },
+      { x: 999, y: 5 },
+      { x: 1279, y: 719 },
+    ]) {
+      const back = fromNormalized(toNormalized(p, HD), HD);
+      expect(Math.abs(back.x - p.x)).toBeLessThanOrEqual(1);
+      expect(Math.abs(back.y - p.y)).toBeLessThanOrEqual(1);
+    }
+  });
+});
+
+describe("box conversions", () => {
+  it("round-trips a screen region through the normalized box", () => {
+    const region = { x: 0, y: 0, width: 640, height: 360 };
+    const box = boxToNormalized(region, HD);
+    expect(box).toEqual({ nx0: 0, ny0: 0, nx1: 500, ny1: 500 });
+    expect(boxFromNormalized(box, HD)).toEqual(region);
+  });
+
+  it("normalizes inverted corners to a non-negative region", () => {
+    const region = boxFromNormalized(
+      { nx0: 500, ny0: 500, nx1: 0, ny1: 0 },
+      HD,
+    );
+    expect(region.width).toBeGreaterThanOrEqual(0);
+    expect(region.height).toBeGreaterThanOrEqual(0);
+    expect(region).toEqual({ x: 0, y: 0, width: 640, height: 360 });
+  });
+
+  it("normalizedBoxCenter returns the pixel center of the box", () => {
+    expect(
+      normalizedBoxCenter({ nx0: 0, ny0: 0, nx1: 1000, ny1: 1000 }, HD),
+    ).toEqual({ x: 640, y: 360 });
+  });
+});

plugins/plugin-computeruse/src/__tests__/nut-driver-input.test.ts

@@ -0,0 +1,122 @@
+/**
+ * Unit tests for the nutjs driver input-correctness helpers (M3.5, #9105).
+ *
+ * Per-notch scroll clamping and manual-drag interpolation are pure and run in
+ * the default lane. The real-driver behavior (coordinate consistency + that
+ * scroll/drag actually fire) is exercised by the gated real-driver lane.
+ *
+ * NOTE: an empirical probe on the Windows backend showed nutjs is
+ * logical-coordinate-consistent here (setPosition(x,y) → cursor exactly (x,y)),
+ * so NO DPI scale-factor multiply is applied to input dispatch — adding one
+ * would mis-place clicks. See #9105.
+ */
+
+import { readFileSync } from "node:fs";
+import { dirname, join } from "node:path";
+import { fileURLToPath } from "node:url";
+import { describe, expect, it } from "vitest";
+import {
+  clampScrollNotches,
+  densifyDragPath,
+  interpolateDragSteps,
+} from "../platform/nut-driver.js";
+
+const nutDriverSrc = readFileSync(
+  join(
+    dirname(fileURLToPath(import.meta.url)),
+    "..",
+    "platform",
+    "nut-driver.ts",
+  ),
+  "utf8",
+);
+
+describe("clampScrollNotches", () => {
+  it("clamps to the 1..20 notch range and rounds", () => {
+    expect(clampScrollNotches(3)).toBe(3);
+    expect(clampScrollNotches(0)).toBe(1);
+    expect(clampScrollNotches(-5)).toBe(1);
+    expect(clampScrollNotches(999)).toBe(20);
+    expect(clampScrollNotches(2.7)).toBe(3);
+  });
+});
+
+describe("interpolateDragSteps", () => {
+  it("produces integer waypoints ending exactly at the target", () => {
+    const pts = interpolateDragSteps(0, 0, 100, 50, 5);
+    expect(pts).toHaveLength(5);
+    // Excludes the start, includes the end.
+    expect(pts[pts.length - 1]).toEqual({ x: 100, y: 50 });
+    expect(pts[0]).toEqual({ x: 20, y: 10 });
+    expect(
+      pts.every((p) => Number.isInteger(p.x) && Number.isInteger(p.y)),
+    ).toBe(true);
+  });
+
+  it("always lands on the target even with a single step", () => {
+    expect(interpolateDragSteps(10, 10, 33, 44, 1)).toEqual([{ x: 33, y: 44 }]);
+  });
+
+  it("is monotonic toward the target", () => {
+    const pts = interpolateDragSteps(0, 0, 200, 0, 10);
+    for (let i = 1; i < pts.length; i += 1) {
+      expect(pts[i].x).toBeGreaterThanOrEqual(pts[i - 1].x);
+    }
+    expect(pts[pts.length - 1].x).toBe(200);
+  });
+});
+
+describe("densifyDragPath (M8 multi-point drag)", () => {
+  it("returns [] for an empty path", () => {
+    expect(densifyDragPath([])).toEqual([]);
+  });
+
+  it("keeps the first vertex and lands on every segment endpoint", () => {
+    const dense = densifyDragPath(
+      [
+        { x: 0, y: 0 },
+        { x: 100, y: 0 },
+        { x: 100, y: 100 },
+      ],
+      4,
+    );
+    // First point preserved exactly.
+    expect(dense[0]).toEqual({ x: 0, y: 0 });
+    // Every vertex appears in order.
+    expect(dense).toContainEqual({ x: 100, y: 0 });
+    expect(dense[dense.length - 1]).toEqual({ x: 100, y: 100 });
+    // 2 segments × 4 steps + the start vertex.
+    expect(dense).toHaveLength(1 + 2 * 4);
+    expect(
+      dense.every((p) => Number.isInteger(p.x) && Number.isInteger(p.y)),
+    ).toBe(true);
+  });
+
+  it("handles a degenerate single-point path", () => {
+    expect(densifyDragPath([{ x: 7, y: 9 }])).toEqual([{ x: 7, y: 9 }]);
+  });
+});
+
+describe("resolveKeyCode modifier support (M8 key_down/key_up regression)", () => {
+  // key_down("shift") / key_up("shift") are the primary use of the press-hold
+  // key primitives, but `resolveKeyCode` originally only mapped function keys,
+  // named keys, and single chars — so a bare modifier threw "Unsupported key".
+  // It now must consult MODIFIER_KEYS so holding shift/ctrl/alt/cmd works.
+  // (Verified live on Windows: driverKeyDown("shift") no longer throws.) The
+  // resolver loads the native nut module, so this is a static source guard that
+  // runs on every OS in the default lane — same convention as
+  // windows-powershell-safety.test.ts.
+  it("resolveKeyCode falls back to MODIFIER_KEYS for bare modifiers", () => {
+    const start = nutDriverSrc.indexOf("function resolveKeyCode(");
+    expect(start, "resolveKeyCode not found").toBeGreaterThan(-1);
+    const body = nutDriverSrc.slice(start, start + 1200);
+    expect(body).toContain("MODIFIER_KEYS[");
+  });
+
+  it("MODIFIER_KEYS maps shift/ctrl/alt/cmd to nutjs Key names", () => {
+    expect(nutDriverSrc).toContain('shift: ["LeftShift"]');
+    expect(nutDriverSrc).toContain('ctrl: ["LeftControl"]');
+    expect(nutDriverSrc).toContain('alt: ["LeftAlt"]');
+    expect(nutDriverSrc).toContain('cmd: ["LeftSuper"]');
+  });
+});

plugins/plugin-computeruse/src/__tests__/ocr-adapter-coord-seam.test.ts

@@ -0,0 +1,150 @@
+/**
+ * scene-builder OCR adapter — coord seam preference + line-only fallback.
+ *
+ * Regression coverage for the unified OCR seam (issue #9105 / M1): the
+ * scene-builder must consume the coord-aware `CoordOcrProvider` when one is
+ * registered (e.g. the plugin-vision bridge), map its blocks to SceneOcrBoxes
+ * in display-absolute coords, and fall back to the line-only `OcrProvider`
+ * registry only when no coord provider is present.
+ */
+
+import { afterEach, beforeEach, describe, expect, it } from "vitest";
+import {
+  _resetOcrProvidersForTests,
+  type CoordOcrProvider,
+  type OcrProvider,
+  registerCoordOcrProvider,
+  registerOcrProvider,
+} from "../mobile/ocr-provider.js";
+import {
+  makeOcrIdState,
+  runOcrOnPng,
+  runOcrOnRegions,
+} from "../scene/ocr-adapter.js";
+
+function fakeCoordProvider(name = "fake-coord"): CoordOcrProvider {
+  return {
+    name,
+    // Echoes sourceX/sourceY into the returned block bbox so callers can prove
+    // the offset is threaded through (matches the real adapter's shift).
+    async describe(input) {
+      return {
+        blocks: [
+          {
+            text: "Save",
+            bbox: {
+              x: input.sourceX + 10,
+              y: input.sourceY + 20,
+              width: 40,
+              height: 16,
+            },
+            words: [],
+            semantic_position: "upper-left",
+          },
+          {
+            text: "Cancel",
+            bbox: {
+              x: input.sourceX + 60,
+              y: input.sourceY + 20,
+              width: 50,
+              height: 16,
+            },
+            words: [],
+            semantic_position: "upper-center",
+          },
+        ],
+      };
+    },
+  };
+}
+
+function fakeLineProvider(name = "fake-line"): OcrProvider {
+  return {
+    name,
+    priority: 1,
+    available: () => true,
+    async recognize() {
+      return {
+        lines: [
+          {
+            text: "LineOnly",
+            boundingBox: { x: 1, y: 2, width: 3, height: 4 },
+            confidence: 0.5,
+          },
+        ],
+        fullText: "LineOnly",
+        elapsedMs: 0,
+        providerName: name,
+        languagesUsed: [],
+      };
+    },
+  };
+}
+
+const PNG = Buffer.from([0x89, 0x50, 0x4e, 0x47]);
+
+describe("ocr-adapter coord seam", () => {
+  beforeEach(() => {
+    _resetOcrProvidersForTests();
+    registerCoordOcrProvider(null);
+  });
+  afterEach(() => {
+    _resetOcrProvidersForTests();
+    registerCoordOcrProvider(null);
+  });
+
+  it("maps CoordOcrProvider blocks to SceneOcrBoxes (full frame)", async () => {
+    registerCoordOcrProvider(fakeCoordProvider());
+    const boxes = await runOcrOnPng(PNG, 0, makeOcrIdState());
+    expect(boxes).toHaveLength(2);
+    expect(boxes[0]).toMatchObject({
+      text: "Save",
+      bbox: [10, 20, 40, 16],
+      conf: 1,
+      displayId: 0,
+    });
+    expect(boxes[0].id).toBe("t0-1");
+    expect(boxes[1]).toMatchObject({ text: "Cancel", bbox: [60, 20, 50, 16] });
+    expect(boxes[1].id).toBe("t0-2");
+  });
+
+  it("prefers the coord seam over a registered line-only provider", async () => {
+    registerOcrProvider(fakeLineProvider());
+    registerCoordOcrProvider(fakeCoordProvider());
+    const boxes = await runOcrOnPng(PNG, 1, makeOcrIdState());
+    // Coord provider wins → "Save"/"Cancel", not "LineOnly".
+    expect(boxes.map((b) => b.text)).toEqual(["Save", "Cancel"]);
+    expect(boxes.every((b) => b.displayId === 1)).toBe(true);
+  });
+
+  it("falls back to the line-only registry when no coord provider", async () => {
+    registerOcrProvider(fakeLineProvider());
+    const boxes = await runOcrOnPng(PNG, 2, makeOcrIdState());
+    expect(boxes).toHaveLength(1);
+    expect(boxes[0]).toMatchObject({
+      text: "LineOnly",
+      bbox: [1, 2, 3, 4],
+      conf: 0.5,
+      displayId: 2,
+    });
+  });
+
+  it("returns empty when nothing is registered", async () => {
+    const boxes = await runOcrOnPng(PNG, 0, makeOcrIdState());
+    expect(boxes).toEqual([]);
+  });
+
+  it("threads the crop offset through the coord seam for dirty regions", async () => {
+    registerCoordOcrProvider(fakeCoordProvider());
+    const boxes = await runOcrOnRegions(
+      [{ png: PNG, bbox: [100, 200, 80, 40] }],
+      0,
+      makeOcrIdState(),
+    );
+    // The provider echoes sourceX/sourceY (100/200) into the block bbox; the
+    // returned coords must already be display-absolute (no double-shift).
+    expect(boxes).toHaveLength(2);
+    expect(boxes[0].bbox).toEqual([110, 220, 40, 16]);
+    expect(boxes[1].bbox).toEqual([160, 220, 50, 16]);
+  });
+});

plugins/plugin-computeruse/src/__tests__/ocr-grounding-actor.test.ts

@@ -0,0 +1,187 @@
+/**
+ * WS7 — OCR/AX deterministic grounding tests.
+ *
+ * The `OcrCoordinateGroundingActor` resolves `ref: "t<displayId>-<seq>"` or
+ * `ref: "a<displayId>-<seq>"` strings into concrete coordinates from the
+ * latest Scene — fully reproducible, no model call.
+ *
+ * Asserted:
+ *   - ref="t0-3" → coordinates that are exactly the bbox center of `t0-3`.
+ *   - ref="a0-1" → matches the AX node, not the OCR list.
+ *   - missing ref + hint label match → falls back to label search.
+ *   - cross-display preference: when a label appears on two displays, prefer
+ *     the requested `preferredDisplay`.
+ *   - `resolveReference` returns null on absolute miss.
+ */
+
+import { describe, expect, it } from "vitest";
+import {
+  OcrCoordinateGroundingActor,
+  resolveReference,
+} from "../actor/actor.js";
+import type { Scene } from "../scene/scene-types.js";
+
+function multiDisplayScene(): Scene {
+  return {
+    timestamp: 1,
+    displays: [
+      {
+        id: 0,
+        bounds: [0, 0, 1920, 1080],
+        scaleFactor: 1,
+        primary: true,
+        name: "p",
+      },
+      {
+        id: 1,
+        bounds: [1920, 0, 2560, 1440],
+        scaleFactor: 1,
+        primary: false,
+        name: "s",
+      },
+    ],
+    focused_window: null,
+    apps: [],
+    ocr: [
+      {
+        id: "t0-1",
+        text: "Save",
+        bbox: [100, 100, 60, 24],
+        conf: 0.95,
+        displayId: 0,
+      },
+      {
+        id: "t0-2",
+        text: "Cancel",
+        bbox: [200, 100, 80, 24],
+        conf: 0.95,
+        displayId: 0,
+      },
+      {
+        id: "t0-3",
+        text: "Open File",
+        bbox: [300, 200, 100, 30],
+        conf: 0.99,
+        displayId: 0,
+      },
+      {
+        id: "t1-1",
+        text: "Save",
+        bbox: [50, 50, 60, 24],
+        conf: 0.85,
+        displayId: 1,
+      },
+    ],
+    ax: [
+      {
+        id: "a0-1",
+        role: "button",
+        label: "Submit",
+        bbox: [500, 400, 80, 40],
+        actions: ["press"],
+        displayId: 0,
+      },
+    ],
+    vlm_scene: null,
+    vlm_elements: null,
+  };
+}
+
+describe("OcrCoordinateGroundingActor", () => {
+  it("resolves OCR ref `t0-3` to bbox center", async () => {
+    const scene = multiDisplayScene();
+    const actor = new OcrCoordinateGroundingActor(() => scene);
+    const result = await actor.ground({
+      displayId: 0,
+      croppedImage: Buffer.alloc(0),
+      hint: "the open-file button",
+      ref: "t0-3",
+    });
+    // bbox [300, 200, 100, 30] → center (350, 215)
+    expect(result.x).toBe(350);
+    expect(result.y).toBe(215);
+    expect(result.displayId).toBe(0);
+    expect(result.confidence).toBe(1);
+    expect(result.reason).toContain("ocr");
+    expect(result.reason).toContain("t0-3");
+  });
+
+  it("resolves AX ref `a0-1` to AX bbox center", async () => {
+    const scene = multiDisplayScene();
+    const actor = new OcrCoordinateGroundingActor(() => scene);
+    const result = await actor.ground({
+      displayId: 0,
+      croppedImage: Buffer.alloc(0),
+      hint: "submit",
+      ref: "a0-1",
+    });
+    // bbox [500, 400, 80, 40] → center (540, 420)
+    expect(result.x).toBe(540);
+    expect(result.y).toBe(420);
+    expect(result.reason).toContain("ax");
+  });
+
+  it("falls back to hint-based label search when no ref is given", async () => {
+    const scene = multiDisplayScene();
+    const actor = new OcrCoordinateGroundingActor(() => scene);
+    const result = await actor.ground({
+      displayId: 0,
+      croppedImage: Buffer.alloc(0),
+      hint: "Open File",
+    });
+    expect(result.x).toBe(350);
+    expect(result.y).toBe(215);
+  });
+
+  it("throws when the ref and hint match nothing", async () => {
+    const scene = multiDisplayScene();
+    const actor = new OcrCoordinateGroundingActor(() => scene);
+    await expect(
+      actor.ground({
+        displayId: 0,
+        croppedImage: Buffer.alloc(0),
+        hint: "definitely-not-on-screen",
+        ref: "t0-99",
+      }),
+    ).rejects.toThrow(/no OCR\/AX target/);
+  });
+
+  it("throws when there's no scene at all", async () => {
+    const actor = new OcrCoordinateGroundingActor(() => null);
+    await expect(
+      actor.ground({
+        displayId: 0,
+        croppedImage: Buffer.alloc(0),
+        hint: "save",
+      }),
+    ).rejects.toThrow(/cannot ground without a current scene/);
+  });
+});
+
+describe("resolveReference — preferred display tie-breaker", () => {
+  it("prefers an OCR box on the requested display when both displays have 'Save'", () => {
+    const scene = multiDisplayScene();
+    // Preferring display 1 should give us t1-1, not t0-1.
+    const target = resolveReference(scene, undefined, "Save", 1);
+    expect(target?.displayId).toBe(1);
+    expect(target?.kind).toBe("ocr");
+  });
+
+  it("prefers display 0 when requested", () => {
+    const scene = multiDisplayScene();
+    const target = resolveReference(scene, undefined, "Save", 0);
+    expect(target?.displayId).toBe(0);
+  });
+
+  it("returns null when nothing matches at all", () => {
+    const scene = multiDisplayScene();
+    expect(resolveReference(scene, undefined, "Klingon", 0)).toBeNull();
+  });
+
+  it("ref takes precedence over hint", () => {
+    const scene = multiDisplayScene();
+    // ref `t0-1` is "Save" — even though we say hint "Cancel", the ref wins.
+    const target = resolveReference(scene, "t0-1", "Cancel", 0);
+    expect(target?.label).toBe("Save");
+  });
+});

plugins/plugin-computeruse/src/__tests__/ocr-provider-registry.test.ts

@@ -0,0 +1,107 @@
+/**
+ * Unit coverage for the OCR provider registries (#9105 M1 / #9170).
+ *
+ * `mobile/ocr-provider.ts` owns the two seams the scene/`detect_elements`/`ocr`
+ * path resolves through: the line-only `OcrProvider` registry (priority-ordered,
+ * highest-available wins) and the single-slot `CoordOcrProvider` (last-call-wins,
+ * the seam Windows.Media.Ocr / Apple Vision / docTR register into). The existing
+ * coord-seam test registers ONE provider and checks the adapter consumes it;
+ * these cases pin the registry's own multi-provider precedence, dedup, and
+ * selection/throw rules.
+ */
+
+import { beforeEach, describe, expect, it } from "vitest";
+import {
+  _resetOcrProvidersForTests,
+  type CoordOcrProvider,
+  getCoordOcrProvider,
+  listOcrProviders,
+  type OcrProvider,
+  registerCoordOcrProvider,
+  registerOcrProvider,
+  selectOcrProvider,
+  unregisterOcrProvider,
+} from "../mobile/ocr-provider.js";
+
+const lineProvider = (
+  name: string,
+  priority: number,
+  available = true,
+): OcrProvider => ({
+  name,
+  priority,
+  available: () => available,
+  recognize: async () => ({
+    lines: [],
+    fullText: "",
+    elapsedMs: 0,
+    providerName: name,
+    languagesUsed: [],
+  }),
+});
+
+const coordProvider = (name: string): CoordOcrProvider => ({
+  name,
+  describe: async () => ({ blocks: [] }),
+});
+
+beforeEach(() => {
+  _resetOcrProvidersForTests();
+  registerCoordOcrProvider(null);
+});
+
+describe("OcrProvider line registry", () => {
+  it("lists providers sorted by priority descending", () => {
+    registerOcrProvider(lineProvider("low", 1));
+    registerOcrProvider(lineProvider("high", 100));
+    registerOcrProvider(lineProvider("mid", 50));
+    expect(listOcrProviders().map((p) => p.name)).toEqual([
+      "high",
+      "mid",
+      "low",
+    ]);
+  });
+
+  it("dedups by name — re-registering a name replaces rather than appends", () => {
+    registerOcrProvider(lineProvider("docTR", 10));
+    registerOcrProvider(lineProvider("docTR", 99));
+    const list = listOcrProviders();
+    expect(list).toHaveLength(1);
+    expect(list[0].priority).toBe(99);
+  });
+
+  it("unregisters by name", () => {
+    registerOcrProvider(lineProvider("a", 1));
+    registerOcrProvider(lineProvider("b", 2));
+    unregisterOcrProvider("b");
+    expect(listOcrProviders().map((p) => p.name)).toEqual(["a"]);
+  });
+
+  it("selects the highest-priority provider that reports available()", () => {
+    registerOcrProvider(lineProvider("hi-unavailable", 100, false));
+    registerOcrProvider(lineProvider("mid-available", 50, true));
+    registerOcrProvider(lineProvider("lo-available", 10, true));
+    expect(selectOcrProvider().name).toBe("mid-available");
+  });
+
+  it("throws when the registry is empty or nothing is available", () => {
+    expect(() => selectOcrProvider()).toThrow(/No OCR provider available/);
+    registerOcrProvider(lineProvider("only-unavailable", 1, false));
+    expect(() => selectOcrProvider()).toThrow(/No OCR provider available/);
+  });
+});
+
+describe("CoordOcrProvider single slot", () => {
+  it("is empty until a provider is registered", () => {
+    expect(getCoordOcrProvider()).toBeNull();
+  });
+
+  it("returns the registered provider; last call wins; null unregisters", () => {
+    registerCoordOcrProvider(coordProvider("first"));
+    expect(getCoordOcrProvider()?.name).toBe("first");
+    registerCoordOcrProvider(coordProvider("second"));
+    expect(getCoordOcrProvider()?.name).toBe("second");
+    registerCoordOcrProvider(null);
+    expect(getCoordOcrProvider()).toBeNull();
+  });
+});

plugins/plugin-computeruse/src/__tests__/osworld-action-converter.test.ts

@@ -0,0 +1,109 @@
+/**
+ * OSWorld action conversion + adapter, with the screenshot platform mocked.
+ * Deterministic unit test of the OSWorld to ComputerInterface translation.
+ */
+import { describe, expect, it, vi } from "vitest";
+import {
+  fromOSWorldAction,
+  type OSWorldPointerState,
+} from "../osworld/action-converter.js";
+import { OSWorldAdapter } from "../osworld/adapter.js";
+
+vi.mock("../platform/screenshot.js", () => ({
+  captureScreenshot: vi.fn(() => Buffer.from("screen")),
+}));
+
+vi.mock("../platform/a11y.js", () => ({
+  extractA11yTree: vi.fn(() => "window tree"),
+  isA11yAvailable: vi.fn(() => true),
+}));
+
+describe("OSWorld action conversion", () => {
+  it("keeps stateless mouse up/down conversion compatible", () => {
+    expect(
+      fromOSWorldAction({ action_type: "MOUSE_DOWN", x: 10, y: 20 }),
+    ).toEqual({
+      action: "mouse_move",
+      coordinate: [10, 20],
+    });
+    expect(
+      fromOSWorldAction({ action_type: "MOUSE_UP", x: 30, y: 40 }),
+    ).toEqual({
+      action: "click",
+      coordinate: [30, 40],
+    });
+  });
+
+  it("combines stateful mouse down/up into a drag", () => {
+    const pointerState: OSWorldPointerState = {};
+
+    expect(
+      fromOSWorldAction(
+        { action_type: "MOUSE_DOWN", x: 10, y: 20 },
+        pointerState,
+      ),
+    ).toEqual({
+      action: "mouse_move",
+      coordinate: [10, 20],
+    });
+    expect(pointerState.mouseDownAt).toEqual([10, 20]);
+
+    expect(
+      fromOSWorldAction(
+        { action_type: "MOUSE_UP", x: 30, y: 40 },
+        pointerState,
+      ),
+    ).toEqual({
+      action: "drag",
+      startCoordinate: [10, 20],
+      coordinate: [30, 40],
+    });
+    expect(pointerState.mouseDownAt).toBeUndefined();
+  });
+
+  it("executes OSWorld mouse down/up through the adapter as a drag", async () => {
+    const service = {
+      executeDesktopAction: vi.fn(async () => ({
+        success: true,
+        message: "done",
+      })),
+    };
+    const adapter = new OSWorldAdapter(service as never, {
+      screenshotDelayMs: 0,
+    });
+
+    await adapter.executeAction({ action_type: "MOUSE_DOWN", x: 10, y: 20 });
+    await adapter.executeAction({ action_type: "MOUSE_UP", x: 30, y: 40 });
+
+    expect(service.executeDesktopAction).toHaveBeenNthCalledWith(1, {
+      action: "mouse_move",
+      coordinate: [10, 20],
+    });
+    expect(service.executeDesktopAction).toHaveBeenNthCalledWith(2, {
+      action: "drag",
+      startCoordinate: [10, 20],
+      coordinate: [30, 40],
+    });
+  });
+
+  it("clears pending mouse-down state on adapter reset", async () => {
+    const service = {
+      executeDesktopAction: vi.fn(async () => ({
+        success: true,
+        message: "done",
+      })),
+    };
+    const adapter = new OSWorldAdapter(service as never, {
+      screenshotDelayMs: 0,
+    });
+
+    await adapter.executeAction({ action_type: "MOUSE_DOWN", x: 10, y: 20 });
+    adapter.reset();
+    await adapter.executeAction({ action_type: "MOUSE_UP", x: 30, y: 40 });
+
+    expect(service.executeDesktopAction).toHaveBeenLastCalledWith({
+      action: "click",
+      coordinate: [30, 40],
+    });
+  });
+});

plugins/plugin-computeruse/src/__tests__/parity-matrix.test.ts

@@ -0,0 +1,92 @@
+/**
+ * trycua/cua parity matrix (#9170 M14).
+ *
+ * The matrix is only trustworthy if it can't drift from the code, so the key
+ * test validates every `have` verb against the LIVE registered action surface.
+ * Add a verb to the matrix without registering it (or rename a promoted action)
+ * and this fails.
+ */
+
+import { describe, expect, it } from "vitest";
+import { computerUsePlugin } from "../index.js";
+import {
+  PARITY_MATRIX,
+  parityCoverageByOs,
+  parityMatrixSummary,
+  validateParityCoverage,
+  validateParityMatrix,
+} from "../parity/parity-matrix.js";
+
+const actionNames = (computerUsePlugin.actions ?? []).map((a) => a.name);
+
+describe("validateParityMatrix", () => {
+  it("every `have` verb in the matrix is a registered action", () => {
+    const result = validateParityMatrix(actionNames);
+    expect(
+      result.ok,
+      `parity drift:\n${result.problems
+        .map((p) => `  - ${p.capability}: ${p.problem}`)
+        .join("\n")}`,
+    ).toBe(true);
+    expect(result.confirmed).toBeGreaterThan(0);
+  });
+
+  it("flags a `have` verb that is not registered", () => {
+    const result = validateParityMatrix(
+      actionNames.filter((n) => n !== "COMPUTER_USE_OPEN"),
+    );
+    expect(result.ok).toBe(false);
+    expect(result.problems.some((p) => p.capability === "open")).toBe(true);
+  });
+
+  it("na capabilities never declare an elizaVerb", () => {
+    for (const cap of PARITY_MATRIX) {
+      if (cap.status === "na") {
+        expect(cap.elizaVerb, `${cap.id}`).toBeUndefined();
+      }
+    }
+  });
+});
+
+describe("parityMatrixSummary", () => {
+  it("counts have / partial / na to the matrix length", () => {
+    const s = parityMatrixSummary();
+    expect(s.have + s.partial + s.na).toBe(s.total);
+    expect(s.total).toBe(PARITY_MATRIX.length);
+    expect(s.have).toBeGreaterThan(10);
+    expect(s.na).toBeGreaterThanOrEqual(3);
+  });
+});
+
+describe("validateParityCoverage", () => {
+  it("every per-OS coverage record in the matrix is well-formed", () => {
+    const result = validateParityCoverage();
+    expect(
+      result.ok,
+      `coverage drift:\n${result.problems
+        .map((p) => `  - ${p.capability}: ${p.problem}`)
+        .join("\n")}`,
+    ).toBe(true);
+    expect(result.confirmed).toBeGreaterThan(0);
+  });
+});
+
+describe("parityCoverageByOs", () => {
+  it("rolls up covered/planned/na for all four OSes", () => {
+    const rollup = parityCoverageByOs();
+    expect(rollup.map((r) => r.os).sort()).toEqual([
+      "aosp",
+      "linux",
+      "macos",
+      "windows",
+    ]);
+    const nonNa = PARITY_MATRIX.filter((c) => c.status !== "na").length;
+    for (const r of rollup) {
+      // every non-na capability is accounted for exactly once per OS
+      expect(r.covered + r.planned + r.na).toBe(nonNa);
+    }
+    // windows is the primary dev box → at least one verb is covered there
+    const windows = rollup.find((r) => r.os === "windows");
+    expect(windows?.covered).toBeGreaterThan(0);
+  });
+});

plugins/plugin-computeruse/src/__tests__/permissions-classify.test.ts

@@ -0,0 +1,100 @@
+/**
+ * Unit coverage for classifyPermissionDeniedError (#9170 / #9058).
+ *
+ * The computer-use service's dispatch catch routes raw OS errors through this
+ * classifier to surface a typed, actionable permission-denied result instead of
+ * a bare stack trace. The macOS Accessibility / Screen-Recording paths run on
+ * every OS; the Windows privacy/Group-Policy/UAC paths are gated to win32 (and
+ * therefore actually execute on this Windows box). Untested until now.
+ */
+
+import { platform } from "node:os";
+import { describe, expect, it } from "vitest";
+import {
+  classifyPermissionDeniedError,
+  createPermissionDeniedError,
+} from "../platform/permissions.js";
+
+const IS_WIN = platform() === "win32";
+
+describe("classifyPermissionDeniedError", () => {
+  it("returns an already-classified PermissionDeniedError unchanged", () => {
+    const original = createPermissionDeniedError({
+      permissionType: "accessibility",
+      operation: "click",
+      message: "already denied",
+    });
+    expect(
+      classifyPermissionDeniedError(original, {
+        permissionType: "screen_recording",
+        operation: "screenshot",
+      }),
+    ).toBe(original);
+  });
+
+  it("classifies an accessibility permission error (any OS)", () => {
+    const out = classifyPermissionDeniedError(
+      new Error(
+        "System Events got an error: not authorized to send Apple events",
+      ),
+      { permissionType: "accessibility", operation: "type" },
+    );
+    expect(out?.permissionDenied).toBe(true);
+    expect(out?.permissionType).toBe("accessibility");
+    expect(out?.message).toMatch(/Accessibility/);
+  });
+
+  it("classifies a screen-recording permission error (any OS)", () => {
+    const out = classifyPermissionDeniedError(
+      new Error("could not create image from display 0"),
+      { permissionType: "screen_recording", operation: "screenshot" },
+    );
+    expect(out?.permissionType).toBe("screen_recording");
+    expect(out?.message).toMatch(/Screen Recording/);
+  });
+
+  it("returns null for an unrelated error", () => {
+    expect(
+      classifyPermissionDeniedError(new Error("disk full"), {
+        permissionType: "screen_recording",
+        operation: "screenshot",
+      }),
+    ).toBeNull();
+  });
+
+  it.skipIf(!IS_WIN)(
+    "classifies a Windows screen-capture block (win32)",
+    () => {
+      const out = classifyPermissionDeniedError(
+        new Error("GraphicsCaptureSession could not start"),
+        { permissionType: "screen_recording", operation: "screenshot" },
+      );
+      expect(out?.permissionType).toBe("screen_recording");
+      expect(out?.message).toMatch(/Windows privacy|Group Policy/);
+    },
+  );
+
+  it.skipIf(!IS_WIN)(
+    "classifies a Windows camera privacy denial (win32)",
+    () => {
+      const out = classifyPermissionDeniedError(
+        new Error("Access is denied (0x80070005)"),
+        { permissionType: "camera", operation: "capture" },
+      );
+      expect(out?.permissionType).toBe("camera");
+      expect(out?.message).toMatch(/Camera access is denied/);
+    },
+  );
+
+  it.skipIf(!IS_WIN)(
+    "classifies a Windows input refusal as accessibility / UAC (win32)",
+    () => {
+      const out = classifyPermissionDeniedError(new Error("Access is denied"), {
+        permissionType: "accessibility",
+        operation: "click",
+      });
+      expect(out?.permissionType).toBe("accessibility");
+      expect(out?.message).toMatch(/elevated \(UAC\)|protected process/);
+    },
+  );
+});

plugins/plugin-computeruse/src/__tests__/platform-capabilities.test.ts

@@ -0,0 +1,180 @@
+/**
+ * Desktop platform-capability detection and the parity matrix it exposes.
+ * Deterministic unit test across simulated host OSes.
+ */
+import fs from "node:fs";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+import { describe, expect, it } from "vitest";
+import {
+  DESKTOP_PARITY,
+  detectPlatformCapabilities,
+  parityFor,
+} from "../platform/capabilities.js";
+import type { PlatformOS } from "../platform/helpers.js";
+
+const repoRoot = path.resolve(
+  path.dirname(fileURLToPath(import.meta.url)),
+  "../../../..",
+);
+
+function readRepoFile(relativePath: string): string {
+  return fs.readFileSync(path.join(repoRoot, relativePath), "utf8");
+}
+
+function detectFor(
+  osName: PlatformOS,
+  availableCommands: string[],
+  browserAvailable = true,
+) {
+  const commands = new Set(availableCommands);
+  return detectPlatformCapabilities({
+    osName,
+    commandExists: (command) => commands.has(command),
+    isBrowserAvailable: () => browserAvailable,
+    shell: "/bin/zsh",
+  });
+}
+
+describe("cross-platform computer-use capabilities", () => {
+  it("reports macOS desktop control through built-ins plus cliclick when installed", () => {
+    const caps = detectFor("darwin", ["cliclick", "/bin/zsh"]);
+
+    expect(caps.screenshot).toMatchObject({
+      available: true,
+      tool: "screencapture (built-in)",
+    });
+    expect(caps.computerUse).toMatchObject({
+      available: true,
+      tool: "cliclick",
+    });
+    expect(caps.windowList.available).toBe(true);
+    expect(caps.fileSystem.available).toBe(true);
+    expect(caps.browser.available).toBe(true);
+  });
+
+  it("reports Linux desktop control through xdotool and screenshot tools", () => {
+    const caps = detectFor("linux", ["xdotool", "scrot", "wmctrl", "/bin/zsh"]);
+
+    expect(caps.screenshot).toMatchObject({
+      available: true,
+      tool: "scrot",
+    });
+    expect(caps.computerUse).toMatchObject({
+      available: true,
+      tool: "xdotool",
+    });
+    expect(caps.windowList).toMatchObject({
+      available: true,
+      tool: "wmctrl",
+    });
+    expect(caps.terminal).toMatchObject({
+      available: true,
+      tool: "/bin/zsh",
+    });
+  });
+
+  it("falls back to ffmpeg x11grab for screenshots when no dedicated tool is present (#9105)", () => {
+    const caps = detectFor("linux", ["ffmpeg"]);
+    expect(caps.screenshot).toMatchObject({
+      available: true,
+      tool: "ffmpeg x11grab",
+    });
+  });
+
+  it("reports no screenshot tool when none of import/scrot/gnome/ffmpeg exist", () => {
+    const caps = detectFor("linux", []);
+    expect(caps.screenshot.available).toBe(false);
+  });
+
+  it("reports Windows desktop control through built-in PowerShell capabilities", () => {
+    const caps = detectFor("win32", [], false);
+
+    expect(caps.screenshot).toMatchObject({
+      available: true,
+      tool: "PowerShell System.Drawing",
+    });
+    expect(caps.computerUse).toMatchObject({
+      available: true,
+      tool: "PowerShell user32.dll",
+    });
+    expect(caps.windowList).toMatchObject({
+      available: true,
+      tool: "PowerShell Get-Process",
+    });
+    expect(caps.terminal).toMatchObject({
+      available: true,
+      tool: "powershell.exe",
+    });
+    expect(caps.browser.available).toBe(false);
+    expect(caps.fileSystem.available).toBe(true);
+  });
+
+  it("keeps Linux explicit about missing desktop dependencies", () => {
+    const caps = detectFor("linux", ["/bin/zsh"], false);
+
+    expect(caps.screenshot).toMatchObject({
+      available: false,
+      tool: "none (install ImageMagick, scrot, gnome-screenshot, or ffmpeg)",
+    });
+    expect(caps.computerUse).toMatchObject({
+      available: false,
+      tool: "none (install xdotool)",
+    });
+    expect(caps.windowList.available).toBe(false);
+    expect(caps.browser.available).toBe(false);
+  });
+});
+
+describe("desktop parity matrix", () => {
+  it("Linux is the verified reference for every capability", () => {
+    for (const cap of Object.keys(DESKTOP_PARITY.linux) as Array<
+      keyof typeof DESKTOP_PARITY.linux
+    >) {
+      expect(parityFor("linux", cap).status).toBe("verified");
+    }
+  });
+
+  it("macOS and Windows declare code-parity for the desktop-control surface", () => {
+    for (const cap of [
+      "screenshot",
+      "computerUse",
+      "windowList",
+      "browser",
+      "terminal",
+    ] as const) {
+      expect(parityFor("darwin", cap).status).toBe("code-parity");
+      expect(parityFor("win32", cap).status).toBe("code-parity");
+    }
+  });
+
+  it("fileSystem is verified everywhere — pure node:fs", () => {
+    expect(parityFor("linux", "fileSystem").status).toBe("verified");
+    expect(parityFor("darwin", "fileSystem").status).toBe("verified");
+    expect(parityFor("win32", "fileSystem").status).toBe("verified");
+  });
+
+  it("documents mobile and desktop platform constraints without task-owner placeholders", () => {
+    const matrix = readRepoFile(
+      "plugins/plugin-computeruse/src/mobile/parity-status.md",
+    );
+
+    expect(matrix).not.toContain("OWNED BY TASK");
+    expect(matrix).not.toContain("TBD");
+    expect(matrix).toContain(
+      "| computerUse — mouse / keyboard | verified (`xdotool`)",
+    );
+    expect(matrix).toContain("blocked: stock iOS forbids cross-app input");
+    expect(matrix).toContain(
+      "| browser (Puppeteer-core driving Chromium) | verified",
+    );
+    expect(matrix).toContain("| clipboard | verified");
+    expect(matrix).toContain(
+      "unavailable (no mobile clipboard bridge method yet)",
+    );
+    expect(matrix).toContain(
+      "code-parity (MediaProjection, requires user consent)",
+    );
+    expect(matrix).toContain("blocked: no Chromium on iOS");
+  });
+});

plugins/plugin-computeruse/src/__tests__/platform-evidence-validator.test.ts

@@ -0,0 +1,270 @@
+/**
+ * Exercises the validate-platform-evidence CLI over temp manifests via spawnSync,
+ * asserting accept/reject on the platform-contract rules.
+ */
+import { spawnSync } from "node:child_process";
+import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+import { describe, expect, it } from "vitest";
+
+const packageRoot = path.resolve(
+  path.dirname(fileURLToPath(import.meta.url)),
+  "../..",
+);
+const validator = path.join(
+  packageRoot,
+  "scripts/validate-platform-evidence.mjs",
+);
+
+function runValidator(args: string[] = []) {
+  return spawnSync(process.execPath, [validator, ...args], {
+    cwd: packageRoot,
+    encoding: "utf8",
+  });
+}
+
+function withTempManifest(
+  manifest: unknown,
+  run: (manifestPath: string) => void,
+): void {
+  const dir = mkdtempSync(
+    path.join(tmpdir(), "computeruse-platform-evidence-"),
+  );
+  try {
+    const manifestPath = path.join(dir, "manifest.json");
+    writeFileSync(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`);
+    run(manifestPath);
+  } finally {
+    rmSync(dir, { force: true, recursive: true });
+  }
+}
+
+function completeMacosManifest() {
+  const checks = [
+    "capabilityProbe",
+    "screenRecordingPermission",
+    "screenshotCapture",
+    "accessibilityPermission",
+    "mouseKeyboardInput",
+    "windowListFocus",
+    "browserAutomation",
+    "clipboardRoundTrip",
+    "approvalMode",
+  ].map((id) => ({
+    id,
+    method: id,
+    status: "passed",
+    requiredEvidence: [`${id} evidence recorded`],
+  }));
+
+  return {
+    schemaVersion: 1,
+    platform: "macos-desktop",
+    status: "passed",
+    target: {
+      minimumMacos: "macOS 13 or newer",
+      requiredPermissions: ["Screen Recording", "Accessibility"],
+      driver: "nutjs",
+    },
+    evidence: {
+      machineModel: "MacBookPro-test",
+      macosVersion: "15.5",
+      buildId: "test-build",
+      validatedAt: "2026-06-23T00:00:00.000Z",
+      validator: "vitest",
+      artifacts: ["evidence/macos-smoke.png"],
+    },
+    checks,
+  };
+}
+
+function completeLinuxManifest() {
+  const checks = [
+    "capabilityProbe",
+    "dependencyProbe",
+    "screenshotCapture",
+    "mouseKeyboardInput",
+    "windowListFocus",
+    "browserAutomation",
+    "clipboardRoundTrip",
+    "terminalSafety",
+    "approvalMode",
+  ].map((id) => ({
+    id,
+    method: id,
+    status: "passed",
+    requiredEvidence: [`${id} evidence recorded`],
+  }));
+
+  return {
+    schemaVersion: 1,
+    platform: "linux-desktop",
+    status: "passed",
+    target: {
+      minimumDistribution: "Ubuntu 22.04",
+      displayServer: "X11",
+      driver: "nutjs",
+    },
+    evidence: {
+      machineId: "linux-test-machine",
+      distribution: "Ubuntu 24.04",
+      kernelVersion: "6.8.0-test",
+      displayServer: "X11",
+      buildId: "test-build",
+      validatedAt: "2026-06-23T00:00:00.000Z",
+      validator: "vitest",
+      artifacts: ["evidence/linux-smoke.png"],
+    },
+    checks,
+  };
+}
+
+function completeWindowsManifest() {
+  const checks = [
+    "capabilityProbe",
+    "screenshotCapture",
+    "mouseKeyboardInput",
+    "windowListFocus",
+    "browserAutomation",
+    "clipboardRoundTrip",
+    "terminalSafety",
+    "approvalMode",
+    "windowsHardeningRegression",
+  ].map((id) => ({
+    id,
+    method: id,
+    status: "passed",
+    requiredEvidence: [`${id} evidence recorded`],
+  }));
+
+  return {
+    schemaVersion: 1,
+    platform: "windows-desktop",
+    status: "passed",
+    target: {
+      minimumWindows: "Windows 10 or newer",
+      driver: "nutjs",
+      shell: "PowerShell -NoProfile",
+    },
+    evidence: {
+      machineModel: "Windows-test-machine",
+      windowsVersion: "Windows 11 Pro",
+      buildId: "test-build",
+      validatedAt: "2026-06-23T00:00:00.000Z",
+      validator: "vitest",
+      artifacts: ["evidence/windows-smoke.png"],
+    },
+    checks,
+  };
+}
+
+describe("platform evidence validator", () => {
+  it("validates all tracked manifests in non-complete mode", () => {
+    const result = runValidator();
+
+    expect(result.status).toBe(0);
+    expect(result.stdout).toContain(
+      "[ios-device-evidence] 10 checks validated (requires_device_evidence)",
+    );
+    expect(result.stdout).toContain(
+      "[android-device-evidence] 10 checks validated (requires_device_evidence)",
+    );
+    expect(result.stdout).toContain(
+      "[android-aosp-evidence] 8 checks validated (requires_device_evidence)",
+    );
+    expect(result.stdout).toContain(
+      "[macos-desktop-evidence] 9 checks validated (passed)",
+    );
+    expect(result.stdout).toContain(
+      "[linux-desktop-evidence] 9 checks validated (requires_device_evidence)",
+    );
+    // Windows desktop CUA is fully device-verified (#9581 — real Windows 11 host,
+    // 9/9 passed; see test-results/evidence/9581-windows-cua/). Its release
+    // manifest is promoted to `passed`, like macOS evidence fields were promoted
+    // when that on-device evidence landed.
+    expect(result.stdout).toContain(
+      "[windows-desktop-evidence] 9 checks validated (passed)",
+    );
+  });
+
+  it("fails the complete gate while real platform evidence is still missing", () => {
+    const result = runValidator(["--require-complete"]);
+
+    expect(result.status).toBe(1);
+    expect(result.stderr).toContain(
+      "ios-device-validation.json: --require-complete needs evidence.deviceModel",
+    );
+    expect(result.stderr).toContain(
+      "android-device-validation.json: --require-complete needs evidence.deviceModel",
+    );
+    expect(result.stderr).toContain(
+      "android-aosp-validation.json: --require-complete needs evidence.imageName",
+    );
+    expect(result.stderr).not.toContain("macos-desktop-validation.json");
+    expect(result.stderr).toContain(
+      "linux-desktop-validation.json: --require-complete needs evidence.machineId",
+    );
+    // windows-desktop is intentionally NOT asserted here: it is fully
+    // device-verified (#9581, 9/9 passed) and its release manifest is promoted to
+    // `passed`, so it does not trip the complete gate. The gate still fails
+    // overall on iOS/Android/macOS/Linux below.
+    expect(result.stderr).toContain(
+      "--require-complete: check mediaProjectionCapture is requires_device_evidence",
+    );
+    expect(result.stderr).toContain(
+      "--require-complete: check dependencyProbe is requires_device_evidence",
+    );
+  });
+
+  it("rejects malformed manifests instead of silently narrowing the gate", () => {
+    withTempManifest(
+      {
+        schemaVersion: 1,
+        platform: "android-consumer",
+        status: "requires_device_evidence",
+        target: {},
+        evidence: { artifacts: [] },
+        checks: [],
+      },
+      (manifestPath) => {
+        const result = runValidator([manifestPath]);
+
+        expect(result.status).toBe(1);
+        expect(result.stderr).toContain("target.minimumApi");
+        expect(result.stderr).toContain("missing check id: permissionsSetup");
+        expect(result.stderr).toContain(
+          "missing check id: lifeOpsScheduledTaskHandoff",
+        );
+      },
+    );
+  });
+
+  it("allows complete desktop manifests only when evidence and statuses are present", () => {
+    withTempManifest(completeMacosManifest(), (manifestPath) => {
+      const result = runValidator(["--require-complete", manifestPath]);
+
+      expect(result.status).toBe(0);
+      expect(result.stdout).toContain(
+        "[macos-desktop-evidence] 9 checks validated (passed)",
+      );
+    });
+    withTempManifest(completeLinuxManifest(), (manifestPath) => {
+      const result = runValidator(["--require-complete", manifestPath]);
+
+      expect(result.status).toBe(0);
+      expect(result.stdout).toContain(
+        "[linux-desktop-evidence] 9 checks validated (passed)",
+      );
+    });
+    withTempManifest(completeWindowsManifest(), (manifestPath) => {
+      const result = runValidator(["--require-complete", manifestPath]);
+
+      expect(result.status).toBe(0);
+      expect(result.stdout).toContain(
+        "[windows-desktop-evidence] 9 checks validated (passed)",
+      );
+    });
+  });
+});

plugins/plugin-computeruse/src/__tests__/process-list.test.ts

@@ -0,0 +1,89 @@
+/**
+ * Real-host process-list tests. Linux path runs against /proc on this host;
+ * macOS / Windows paths are parser-only fixtures.
+ */
+
+import { readdirSync } from "node:fs";
+import { describe, expect, it } from "vitest";
+import { currentPlatform } from "../platform/helpers.js";
+import {
+  listProcesses,
+  parsePsOutput,
+  parseWindowsProcessJson,
+} from "../platform/process-list.js";
+
+describe("process-list — Linux /proc (real host)", () => {
+  if (currentPlatform() !== "linux") {
+    it.skip("not on linux", () => {});
+    return;
+  }
+  it("enumerates running processes", () => {
+    const procs = listProcesses();
+    expect(procs.length).toBeGreaterThan(5);
+    for (const p of procs) {
+      expect(p.pid).toBeGreaterThan(0);
+      expect(p.name.length).toBeGreaterThan(0);
+    }
+  });
+
+  it("includes our own process (vitest)", () => {
+    const procs = listProcesses();
+    const ownPid = process.pid;
+    const me = procs.find((p) => p.pid === ownPid);
+    expect(me).toBeDefined();
+    expect(me?.name.length).toBeGreaterThan(0);
+  });
+
+  it("matches /proc entry count within reason", () => {
+    const procs = listProcesses();
+    const procEntries = readdirSync("/proc").filter((e) => /^\d+$/.test(e));
+    // Some pids may have vanished between the two reads, but the count
+    // should be in the same order of magnitude.
+    expect(procs.length).toBeGreaterThanOrEqual(
+      Math.floor(procEntries.length * 0.5),
+    );
+    expect(procs.length).toBeLessThanOrEqual(procEntries.length + 10);
+  });
+});
+
+describe("process-list — parsePsOutput (darwin parser)", () => {
+  it("parses canonical BSD ps output", () => {
+    const fixture = `   1 launchd
+ 100 /System/Library/CoreServices/loginwindow.app/Contents/MacOS/loginwindow
+ 200 mds_stores
+`;
+    const parsed = parsePsOutput(fixture);
+    expect(parsed).toHaveLength(3);
+    expect(parsed[0]).toEqual({ pid: 1, name: "launchd" });
+    // basename normalization strips the full path.
+    expect(parsed[1]).toEqual({ pid: 100, name: "loginwindow" });
+    expect(parsed[2]).toEqual({ pid: 200, name: "mds_stores" });
+  });
+
+  it("ignores blank lines and non-numeric pids", () => {
+    expect(parsePsOutput("")).toEqual([]);
+    expect(parsePsOutput("   \n   foo\n").length).toBe(0);
+  });
+});
+
+describe("process-list — parseWindowsProcessJson", () => {
+  it("parses an array of Get-Process rows", () => {
+    const json = JSON.stringify([
+      { Id: 1234, ProcessName: "explorer" },
+      { Id: 5678, ProcessName: "chrome" },
+    ]);
+    expect(parseWindowsProcessJson(json)).toEqual([
+      { pid: 1234, name: "explorer" },
+      { pid: 5678, name: "chrome" },
+    ]);
+  });
+
+  it("tolerates a single-object form", () => {
+    const json = JSON.stringify({ Id: 1, ProcessName: "System" });
+    expect(parseWindowsProcessJson(json)).toEqual([{ pid: 1, name: "System" }]);
+  });
+
+  it("returns [] for malformed input", () => {
+    expect(parseWindowsProcessJson("not json")).toEqual([]);
+  });
+});

plugins/plugin-computeruse/src/__tests__/progress.test.ts

@@ -0,0 +1,164 @@
+/**
+ * Unit coverage for the action-progress + approval-prompt builders
+ * (#9170 / #8912 per-step progress streaming).
+ *
+ * These pure builders shape the transient progress lines and the approval
+ * CHOICE prompt that the COMPUTER_USE / COMPUTER_USE_AGENT loop streams back to
+ * the user — untested until now.
+ */
+
+import type { Content, HandlerCallback } from "@elizaos/core";
+import { describe, expect, it } from "vitest";
+import {
+  ACTION_PROGRESS_SOURCE,
+  buildApprovalPromptContent,
+  buildStepProgressContent,
+  COMPUTER_USE_APPROVAL_SOURCE,
+  formatStepProgressText,
+  isStreamProgressEnabled,
+  withApprovalRelay,
+} from "../actions/progress.js";
+import type { ApprovalSnapshot, PendingApproval } from "../types.js";
+
+describe("isStreamProgressEnabled", () => {
+  it("is a strict === true guard", () => {
+    expect(isStreamProgressEnabled(true)).toBe(true);
+    expect(isStreamProgressEnabled(false)).toBe(false);
+    expect(isStreamProgressEnabled("true")).toBe(false);
+    expect(isStreamProgressEnabled(1)).toBe(false);
+    expect(isStreamProgressEnabled(undefined)).toBe(false);
+  });
+});
+
+describe("formatStepProgressText", () => {
+  it("renders step / kind / rationale", () => {
+    expect(formatStepProgressText(3, "click", "press the button")).toBe(
+      "Step 3: click — press the button",
+    );
+  });
+
+  it("falls back to 'dispatched' for a missing/blank rationale", () => {
+    expect(formatStepProgressText(1, "type")).toBe("Step 1: type — dispatched");
+    expect(formatStepProgressText(2, "scroll", "   ")).toBe(
+      "Step 2: scroll — dispatched",
+    );
+  });
+});
+
+describe("buildStepProgressContent", () => {
+  it("builds a transient compact-progress Content with the full payload", () => {
+    const c = buildStepProgressContent({
+      actionName: "COMPUTER_USE",
+      step: 2,
+      kind: "click",
+      rationale: "  hit OK  ",
+      success: true,
+    });
+    expect(c.text).toBe("Step 2: click — hit OK");
+    expect(c.source).toBe(ACTION_PROGRESS_SOURCE);
+    expect(c.merge).toBe("replace");
+    expect(c.metadata).toMatchObject({
+      transient: true,
+      compactProgress: true,
+      progress: {
+        source: "computeruse",
+        actionName: "COMPUTER_USE",
+        step: 2,
+        kind: "click",
+        rationale: "hit OK",
+        success: true,
+      },
+    });
+  });
+
+  it("defaults the rationale and honors a source override", () => {
+    const c = buildStepProgressContent({
+      actionName: "WINDOW",
+      step: 1,
+      kind: "focus",
+      source: "custom",
+    });
+    expect(c.text).toContain("dispatched");
+    expect(c.metadata).toMatchObject({
+      progress: { source: "custom", rationale: "dispatched" },
+    });
+  });
+});
+
+describe("buildApprovalPromptContent", () => {
+  const approval = {
+    id: "abc",
+    command: "terminal_execute",
+    requestedAt: 1000,
+  } as PendingApproval;
+
+  it("emits a CHOICE block with approve/deny callbacks", () => {
+    const c = buildApprovalPromptContent(approval);
+    expect(c.source).toBe(COMPUTER_USE_APPROVAL_SOURCE);
+    expect(c.text).toContain("[CHOICE:computeruse-approval id=abc]");
+    expect(c.text).toContain("cua:abc:approve=Approve");
+    expect(c.text).toContain("cua:abc:deny=Deny");
+    expect(c.text).toContain("`terminal_execute`");
+  });
+
+  it("scopes the callback to an owner when ownerId is given", () => {
+    const c = buildApprovalPromptContent(approval, { ownerId: "7" });
+    expect(c.text).toContain("cua:abc:approve:u7=Approve");
+    expect(c.text).toContain("cua:abc:deny:u7=Deny");
+    expect(c.metadata).toMatchObject({
+      computeruse: {
+        ownerId: "7",
+        approvalId: "abc",
+        command: "terminal_execute",
+      },
+    });
+  });
+});
+
+describe("withApprovalRelay", () => {
+  const emptySnapshot = { pendingApprovals: [] } as unknown as ApprovalSnapshot;
+
+  it("runs and returns the result when there is no callback", async () => {
+    const service = {
+      getApprovalSnapshot: () => emptySnapshot,
+      subscribeApprovals: () => () => {},
+    };
+    await expect(
+      withApprovalRelay(service, undefined, async () => "done"),
+    ).resolves.toBe("done");
+  });
+
+  it("relays a newly-pending approval to the callback as a prompt", async () => {
+    let emit: ((s: ApprovalSnapshot) => void) | null = null;
+    const service = {
+      getApprovalSnapshot: () => emptySnapshot,
+      subscribeApprovals: (listener: (s: ApprovalSnapshot) => void) => {
+        emit = listener;
+        return () => {};
+      },
+    };
+    const seen: Content[] = [];
+    const callback: HandlerCallback = async (content) => {
+      seen.push(content);
+      return [];
+    };
+
+    const result = await withApprovalRelay(service, callback, async () => {
+      emit?.({
+        pendingApprovals: [
+          {
+            id: "x1",
+            command: "file_write",
+            requestedAt: 5,
+          } as PendingApproval,
+        ],
+      } as unknown as ApprovalSnapshot);
+      return 42;
+    });
+
+    expect(result).toBe(42);
+    expect(seen).toHaveLength(1);
+    expect(seen[0].text).toContain("cua:x1:approve=Approve");
+    expect(seen[0].source).toBe(COMPUTER_USE_APPROVAL_SOURCE);
+  });
+});

plugins/plugin-computeruse/src/__tests__/ps-host.real.test.ts

@@ -0,0 +1,105 @@
+/**
+ * Real verification of the warm PowerShell host (#9581 follow-up).
+ * Gated: runs only on Windows (the host is win32-only; everywhere else
+ * `psHostAvailable()` is false and callers use their one-shot path).
+ *
+ * Proves the host:
+ *   - starts and runs commands (text round-trips through stdin/stdout framing),
+ *   - is fast once warm (the whole point — no per-call cold `powershell.exe`),
+ *   - surfaces script errors as rejections (so callers fall back), and
+ *   - stays usable after an error.
+ *
+ * See `src/platform/ps-host.ts`. Evidence: the cold-vs-warm latency table is in
+ * the PR; here we assert correctness + a generous warm-latency ceiling.
+ */
+
+import { platform } from "node:os";
+import { afterAll, describe, expect, it } from "vitest";
+import {
+  disposePsHost,
+  psHostAvailable,
+  runPsHost,
+  shutdownPsHost,
+  warmPsHost,
+} from "../platform/ps-host.js";
+
+const RUN = platform() === "win32";
+
+describe("ps-host (warm PowerShell host, Windows)", () => {
+  afterAll(() => {
+    shutdownPsHost();
+  });
+
+  it.skipIf(!RUN)("is reported available on win32", () => {
+    expect(psHostAvailable()).toBe(true);
+  });
+
+  it.skipIf(!RUN)(
+    "runs a command and round-trips its stdout",
+    async () => {
+      await warmPsHost();
+      const out = await runPsHost("Write-Output 'pshost-ok'", 30_000);
+      expect(out).toContain("pshost-ok");
+    },
+    60_000,
+  );
+
+  it.skipIf(!RUN)(
+    "round-trips UTF-8 via base64-in / text-out",
+    async () => {
+      const text = "héllo 世界 — ps-host";
+      const b64 = Buffer.from(text, "utf-8").toString("base64");
+      const out = await runPsHost(
+        `[Console]::Out.Write([System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String('${b64}')))`,
+        30_000,
+      );
+      expect(out).toBe(text);
+    },
+    60_000,
+  );
+
+  it.skipIf(!RUN)(
+    "is fast once warm (no per-call cold spawn)",
+    async () => {
+      await warmPsHost();
+      // 5 sequential calls; once warm each should be well under a second even on
+      // a Defender-heavy host. A cold spawn alone is ~10-16s, so this ceiling
+      // can only pass through the persistent host.
+      const start = Date.now();
+      for (let i = 0; i < 5; i++) {
+        await runPsHost("Write-Output (2 + 2)", 30_000);
+      }
+      const perCall = (Date.now() - start) / 5;
+      expect(perCall).toBeLessThan(2_000);
+    },
+    60_000,
+  );
+
+  it.skipIf(!RUN)(
+    "rejects on script error and stays usable afterward",
+    async () => {
+      await warmPsHost();
+      await expect(runPsHost("throw 'boom'", 30_000)).rejects.toThrow();
+      const out = await runPsHost("Write-Output 'still-alive'", 30_000);
+      expect(out).toContain("still-alive");
+    },
+    60_000,
+  );
+
+  it.skipIf(!RUN)(
+    "disposePsHost latches spawning off until re-warmed",
+    async () => {
+      await warmPsHost();
+      // After an owner dispose, a fresh call must NOT resurrect the host (so a
+      // late fire-and-forget continuation can't leak a process post-stop). The
+      // caller sees a rejection and falls back to a one-shot spawn.
+      disposePsHost();
+      await expect(runPsHost("Write-Output 'nope'", 30_000)).rejects.toThrow();
+      // An explicit warm re-enables a fresh session.
+      await warmPsHost();
+      const out = await runPsHost("Write-Output 're-enabled'", 30_000);
+      expect(out).toContain("re-enabled");
+    },
+    60_000,
+  );
+});

plugins/plugin-computeruse/src/__tests__/remote-guest.test.ts

@@ -0,0 +1,213 @@
+/**
+ * Remote-guest RPC seam + VM provider backends (#9170 M13).
+ *
+ * The op→RPC translation, URL resolution, and the success/failure unwrap run in
+ * the default lane with a fake transport (no VM, no network). WSB/QEMU boot is
+ * driven through injected launchers; real-hypervisor behavior is out of scope
+ * for unit tests.
+ */
+
+import { describe, expect, it, vi } from "vitest";
+import { QEMUBackend } from "../sandbox/qemu-backend.js";
+import {
+  type GuestRpcRequest,
+  type GuestRpcResponse,
+  HttpGuestTransport,
+  RemoteGuestBackend,
+  type RemoteGuestTransport,
+  resolveGuestRpcUrl,
+  sandboxOpToRpc,
+} from "../sandbox/remote-guest.js";
+import { SandboxBackendUnavailableError } from "../sandbox/types.js";
+import { WSBBackend } from "../sandbox/wsb-backend.js";
+
+function recordingTransport(
+  result: unknown = { ok: true },
+  success = true,
+): RemoteGuestTransport & { calls: GuestRpcRequest[] } {
+  const calls: GuestRpcRequest[] = [];
+  return {
+    name: "fake",
+    calls,
+    async dispatch(req): Promise<GuestRpcResponse> {
+      calls.push(req);
+      return success
+        ? { success: true, result }
+        : { success: false, error: "boom" };
+    },
+  };
+}
+
+describe("sandboxOpToRpc", () => {
+  it("maps mouse + keyboard + screenshot ops to cua command names", () => {
+    expect(sandboxOpToRpc({ kind: "mouse_move", x: 1, y: 2 })).toEqual({
+      command: "move_cursor",
+      params: { x: 1, y: 2 },
+    });
+    expect(sandboxOpToRpc({ kind: "mouse_click", x: 3, y: 4 })).toEqual({
+      command: "left_click",
+      params: { x: 3, y: 4 },
+    });
+    expect(sandboxOpToRpc({ kind: "keyboard_type", text: "hi" })).toEqual({
+      command: "type_text",
+      params: { text: "hi" },
+    });
+    expect(
+      sandboxOpToRpc({
+        kind: "screenshot",
+        region: { x: 0, y: 0, width: 10, height: 10 },
+      }),
+    ).toEqual({
+      command: "screenshot",
+      params: { region: { x: 0, y: 0, width: 10, height: 10 } },
+    });
+  });
+
+  it("maps run_command with optional cwd/timeout", () => {
+    expect(
+      sandboxOpToRpc({
+        kind: "run_command",
+        command: "ls",
+        cwd: "/tmp",
+        timeout_seconds: 5,
+      }),
+    ).toEqual({
+      command: "run_command",
+      params: { command: "ls", cwd: "/tmp", timeout_seconds: 5 },
+    });
+    expect(sandboxOpToRpc({ kind: "run_command", command: "ls" })).toEqual({
+      command: "run_command",
+      params: { command: "ls" },
+    });
+  });
+});
+
+describe("resolveGuestRpcUrl", () => {
+  it("prefers explicit url, then port, then default", () => {
+    expect(resolveGuestRpcUrl({ rpcUrl: "http://x/y" })).toBe("http://x/y");
+    expect(resolveGuestRpcUrl({ rpcPort: 9001 })).toBe(
+      "http://127.0.0.1:9001/cua",
+    );
+    expect(resolveGuestRpcUrl({})).toBe("http://127.0.0.1:8000/cua");
+  });
+});
+
+class FakeBackend extends RemoteGuestBackend {
+  readonly name = "fake";
+  constructor(private readonly t: RemoteGuestTransport) {
+    super();
+  }
+  protected transport(): RemoteGuestTransport {
+    return this.t;
+  }
+  async start(): Promise<void> {}
+  async stop(): Promise<void> {}
+}
+
+describe("RemoteGuestBackend.invoke", () => {
+  it("returns the result on success", async () => {
+    const t = recordingTransport({ base64Png: "AAAA" });
+    const backend = new FakeBackend(t);
+    const out = await backend.invoke<{ base64Png: string }>({
+      kind: "screenshot",
+    });
+    expect(out).toEqual({ base64Png: "AAAA" });
+    expect(t.calls[0]?.command).toBe("screenshot");
+  });
+
+  it("throws SandboxInvocationError on failure", async () => {
+    const t = recordingTransport(undefined, false);
+    const backend = new FakeBackend(t);
+    await expect(
+      backend.invoke({ kind: "mouse_click", x: 1, y: 1 }),
+    ).rejects.toThrow(/left_click.*failed/);
+  });
+});
+
+describe("HttpGuestTransport", () => {
+  it("POSTs the RPC and returns the parsed body", async () => {
+    const fetchImpl = vi.fn(async () => ({
+      ok: true,
+      json: async () => ({ success: true, result: { windows: [] } }),
+    })) as unknown as typeof fetch;
+    const t = new HttpGuestTransport({ url: "http://guest/cua", fetchImpl });
+    const res = await t.dispatch({ command: "get_windows", params: {} });
+    expect(res).toEqual({ success: true, result: { windows: [] } });
+    expect(fetchImpl).toHaveBeenCalledOnce();
+  });
+
+  it("returns a failure response on a non-2xx status", async () => {
+    const fetchImpl = vi.fn(async () => ({
+      ok: false,
+      status: 503,
+      json: async () => ({}),
+    })) as unknown as typeof fetch;
+    const t = new HttpGuestTransport({ url: "http://guest/cua", fetchImpl });
+    const res = await t.dispatch({ command: "screenshot", params: {} });
+    expect(res.success).toBe(false);
+    expect(res.error).toContain("503");
+  });
+});
+
+describe("WSBBackend", () => {
+  it("throws when Windows Sandbox is unavailable and nothing is injected", () => {
+    expect(() => new WSBBackend({ available: false })).toThrow(
+      SandboxBackendUnavailableError,
+    );
+  });
+
+  it("boots via the injected launcher and drives ops over the transport", async () => {
+    const t = recordingTransport({ ok: true });
+    const launch = vi.fn(async () => {});
+    const shutdown = vi.fn(async () => {});
+    const backend = new WSBBackend({
+      available: false,
+      transport: t,
+      launcher: { launch, shutdown },
+      rpcPort: 8123,
+    });
+    await backend.start();
+    await backend.invoke({ kind: "mouse_move", x: 5, y: 6 });
+    await backend.stop();
+    expect(launch).toHaveBeenCalledWith({ rpcPort: 8123 });
+    expect(shutdown).toHaveBeenCalledOnce();
+    expect(t.calls[0]?.command).toBe("move_cursor");
+  });
+});
+
+describe("QEMUBackend", () => {
+  it("throws when QEMU is unavailable and no transport is injected", () => {
+    expect(
+      () =>
+        new QEMUBackend({
+          image: "guest.qcow2",
+          launcher: {
+            isAvailable: () => false,
+            launch: async () => {},
+            shutdown: async () => {},
+          },
+        }),
+    ).toThrow(SandboxBackendUnavailableError);
+  });
+
+  it("boots via the injected launcher and drives ops over the transport", async () => {
+    const t = recordingTransport({ ok: true });
+    const launch = vi.fn(async () => {});
+    const backend = new QEMUBackend({
+      image: "guest.qcow2",
+      transport: t,
+      launcher: { isAvailable: () => true, launch, shutdown: async () => {} },
+      rpcPort: 9000,
+    });
+    await backend.start();
+    await backend.invoke({ kind: "keyboard_key_press", key: "Enter" });
+    expect(launch).toHaveBeenCalledWith({
+      image: "guest.qcow2",
+      rpcPort: 9000,
+    });
+    expect(t.calls[0]).toEqual({
+      command: "press_key",
+      params: { key: "Enter" },
+    });
+  });
+});

plugins/plugin-computeruse/src/__tests__/routes-e2e.test.ts

@@ -0,0 +1,322 @@
+/**
+ * Route-level e2e for plugin-computeruse (issue #8802).
+ *
+ * Boots the plugin's declared `Route[]` (`computerUseRoutes`) through the real
+ * production dispatcher (`tryHandleRuntimePluginRoute`) over a loopback
+ * `http.createServer` — exercising the real auth gate, JSON body parsing, query
+ * parsing, and handler dispatch — with a faked `ComputerUseService` standing in
+ * for the only external dependency. No mocked `json`/`error` functions: every
+ * assertion is on a real HTTP response over `fetch`.
+ *
+ * The plugin's compat handler (`computer-use-compat-routes.ts`) trusts loopback
+ * requests internally (`isTrustedLocalRequest`), so the handler always runs once
+ * the dispatcher lets the request through. The dispatcher-level auth gate
+ * (`route.public !== true && !isAuthorized()`) runs *before* the handler, so it
+ * is fully exercisable here without any real desktop control.
+ */
+
+import http from "node:http";
+import type { AddressInfo } from "node:net";
+import type { AgentRuntime } from "@elizaos/core";
+import { afterEach, describe, expect, it } from "vitest";
+
+import { tryHandleRuntimePluginRoute } from "../../../../packages/agent/src/api/runtime-plugin-routes.ts";
+import { computerUsePlugin } from "../index.ts";
+
+const servers: http.Server[] = [];
+
+afterEach(async () => {
+  await Promise.all(
+    servers.map(
+      (server) =>
+        new Promise<void>((resolve) => {
+          server.closeAllConnections?.();
+          server.close(() => resolve());
+        }),
+    ),
+  );
+  servers.length = 0;
+});
+
+type ApprovalMode = "full_control" | "smart_approve" | "approve_all" | "off";
+
+interface ApprovalSnapshot {
+  mode: ApprovalMode;
+  pendingCount: number;
+  pendingApprovals: Array<{
+    id: string;
+    command: string;
+    parameters: Record<string, unknown>;
+    requestedAt: string;
+  }>;
+}
+
+interface ApprovalResolution {
+  id: string;
+  command: string;
+  approved: boolean;
+  cancelled: boolean;
+  mode: ApprovalMode;
+  requestedAt: string;
+  resolvedAt: string;
+  reason?: string;
+}
+
+interface FakeServiceState {
+  mode: ApprovalMode;
+  calls: string[];
+  resolveResult: ApprovalResolution | null;
+}
+
+const SNAPSHOT: ApprovalSnapshot = {
+  mode: "smart_approve",
+  pendingCount: 1,
+  pendingApprovals: [
+    {
+      id: "approval-1",
+      command: "COMPUTER_USE_CLICK",
+      parameters: { x: 10, y: 20 },
+      requestedAt: "2026-01-01T00:00:00.000Z",
+    },
+  ],
+};
+
+function fakeService(state: FakeServiceState) {
+  return {
+    getApprovalSnapshot(): ApprovalSnapshot {
+      state.calls.push("getApprovalSnapshot");
+      return { ...SNAPSHOT, mode: state.mode };
+    },
+    setApprovalMode(mode: ApprovalMode): ApprovalMode {
+      state.calls.push(`setApprovalMode:${mode}`);
+      state.mode = mode;
+      return mode;
+    },
+    resolveApproval(
+      id: string,
+      approved: boolean,
+      reason?: string,
+    ): ApprovalResolution | null {
+      state.calls.push(`resolveApproval:${id}:${approved}:${reason ?? ""}`);
+      return state.resolveResult;
+    },
+    subscribeApprovals(
+      _listener: (snapshot: ApprovalSnapshot) => void,
+    ): () => void {
+      return () => {};
+    },
+  };
+}
+
+function makeRuntime(
+  options: { withService?: boolean; state?: FakeServiceState } = {},
+): AgentRuntime {
+  const { withService = true, state } = options;
+  const service = state ? fakeService(state) : null;
+  return {
+    routes: computerUsePlugin.routes,
+    getService: (key: string) =>
+      withService && key === "computeruse" ? service : null,
+  } as unknown as AgentRuntime;
+}
+
+async function startServer(
+  runtime: AgentRuntime,
+  isAuthorized: () => boolean = () => true,
+): Promise<string> {
+  const server = http.createServer(async (req, res) => {
+    const url = new URL(req.url ?? "/", "http://127.0.0.1");
+    const handled = await tryHandleRuntimePluginRoute({
+      req,
+      res,
+      method: req.method ?? "GET",
+      pathname: url.pathname,
+      url,
+      runtime,
+      isAuthorized,
+    });
+    if (!handled && !res.headersSent) {
+      res.statusCode = 404;
+      res.end("not found");
+    }
+  });
+  servers.push(server);
+  await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
+  const { port } = server.address() as AddressInfo;
+  return `http://127.0.0.1:${port}`;
+}
+
+async function postJson(base: string, path: string, body: unknown) {
+  return fetch(`${base}${path}`, {
+    method: "POST",
+    headers: { "content-type": "application/json" },
+    body: JSON.stringify(body),
+  });
+}
+
+function freshState(
+  overrides: Partial<FakeServiceState> = {},
+): FakeServiceState {
+  return {
+    mode: "smart_approve",
+    calls: [],
+    resolveResult: null,
+    ...overrides,
+  };
+}
+
+describe("plugin-computeruse routes (real dispatch)", () => {
+  it("returns the approval snapshot on GET /approvals when authorized", async () => {
+    const state = freshState();
+    const base = await startServer(makeRuntime({ state }));
+    const res = await fetch(`${base}/api/computer-use/approvals`);
+    expect(res.status).toBe(200);
+    const body = (await res.json()) as ApprovalSnapshot;
+    expect(body.mode).toBe("smart_approve");
+    expect(body.pendingCount).toBe(1);
+    expect(body.pendingApprovals[0]?.id).toBe("approval-1");
+    expect(state.calls).toContain("getApprovalSnapshot");
+  });
+
+  it("enforces the dispatcher auth gate on the non-public GET /approvals route", async () => {
+    const state = freshState();
+    const base = await startServer(makeRuntime({ state }), () => false);
+    const res = await fetch(`${base}/api/computer-use/approvals`);
+    expect(res.status).toBe(401);
+    expect(((await res.json()) as { error: string }).error).toBe(
+      "Unauthorized",
+    );
+    // Handler never ran: the gate short-circuits before service access.
+    expect(state.calls).toHaveLength(0);
+  });
+
+  it("serves the public approvals/stream route even when auth is denied", async () => {
+    // Auth denied, but the stream route declares `public: true`. With no
+    // service the handler writes a single snapshot frame and closes the stream
+    // (a live service would keep the SSE connection open via heartbeat), so the
+    // response body resolves deterministically here.
+    const base = await startServer(
+      makeRuntime({ withService: false }),
+      () => false,
+    );
+    const res = await fetch(`${base}/api/computer-use/approvals/stream`);
+    expect(res.status).toBe(200);
+    expect(res.headers.get("content-type")).toContain("text/event-stream");
+    const text = await res.text();
+    expect(text).toContain('"type":"snapshot"');
+    // EMPTY_APPROVAL_SNAPSHOT is served when no service is registered.
+    expect(text).toContain('"mode":"full_control"');
+  });
+
+  it("changes the approval mode on POST /approval-mode with valid input", async () => {
+    const state = freshState();
+    const base = await startServer(makeRuntime({ state }));
+    const res = await postJson(base, "/api/computer-use/approval-mode", {
+      mode: "approve_all",
+    });
+    expect(res.status).toBe(200);
+    expect(((await res.json()) as { mode: ApprovalMode }).mode).toBe(
+      "approve_all",
+    );
+    expect(state.calls).toContain("setApprovalMode:approve_all");
+  });
+
+  it("rejects an invalid approval mode with 400 from the real handler", async () => {
+    const state = freshState();
+    const base = await startServer(makeRuntime({ state }));
+    const res = await postJson(base, "/api/computer-use/approval-mode", {
+      mode: "not-a-mode",
+    });
+    expect(res.status).toBe(400);
+    expect(((await res.json()) as { error: string }).error).toContain(
+      "full_control",
+    );
+    expect(state.calls).toHaveLength(0);
+  });
+
+  it("returns 404 from POST /approval-mode when the service is unavailable", async () => {
+    const base = await startServer(makeRuntime({ withService: false }));
+    const res = await postJson(base, "/api/computer-use/approval-mode", {
+      mode: "off",
+    });
+    expect(res.status).toBe(404);
+    expect(((await res.json()) as { error: string }).error).toContain(
+      "service not available",
+    );
+  });
+
+  it("enforces the dispatcher auth gate on POST /approval-mode", async () => {
+    const state = freshState();
+    const base = await startServer(makeRuntime({ state }), () => false);
+    const res = await postJson(base, "/api/computer-use/approval-mode", {
+      mode: "off",
+    });
+    expect(res.status).toBe(401);
+    expect(state.calls).toHaveLength(0);
+  });
+
+  it("resolves an approval on POST /approvals/:id and decodes the id", async () => {
+    const resolution: ApprovalResolution = {
+      id: "approval 1",
+      command: "COMPUTER_USE_CLICK",
+      approved: true,
+      cancelled: false,
+      mode: "smart_approve",
+      requestedAt: "2026-01-01T00:00:00.000Z",
+      resolvedAt: "2026-01-01T00:00:01.000Z",
+      reason: "looks fine",
+    };
+    const state = freshState({ resolveResult: resolution });
+    const base = await startServer(makeRuntime({ state }));
+    // %20 in the id segment must be URL-decoded by the handler.
+    const res = await postJson(
+      base,
+      "/api/computer-use/approvals/approval%201",
+      {
+        approved: true,
+        reason: "looks fine",
+      },
+    );
+    expect(res.status).toBe(200);
+    const body = (await res.json()) as ApprovalResolution;
+    expect(body.approved).toBe(true);
+    expect(body.id).toBe("approval 1");
+    expect(state.calls).toContain("resolveApproval:approval 1:true:looks fine");
+  });
+
+  it("returns 400 from POST /approvals/:id when `approved` is missing", async () => {
+    const state = freshState();
+    const base = await startServer(makeRuntime({ state }));
+    const res = await postJson(base, "/api/computer-use/approvals/approval-1", {
+      reason: "no decision",
+    });
+    expect(res.status).toBe(400);
+    expect(((await res.json()) as { error: string }).error).toContain(
+      "approved must be a boolean",
+    );
+    expect(state.calls).toHaveLength(0);
+  });
+
+  it("returns 404 from POST /approvals/:id for an unknown approval id", async () => {
+    const state = freshState({ resolveResult: null });
+    const base = await startServer(makeRuntime({ state }));
+    const res = await postJson(base, "/api/computer-use/approvals/missing", {
+      approved: false,
+    });
+    expect(res.status).toBe(404);
+    expect(((await res.json()) as { error: string }).error).toContain(
+      "Approval not found",
+    );
+    expect(state.calls).toContain("resolveApproval:missing:false:");
+  });
+
+  it("enforces the dispatcher auth gate on POST /approvals/:id", async () => {
+    const state = freshState();
+    const base = await startServer(makeRuntime({ state }), () => false);
+    const res = await postJson(base, "/api/computer-use/approvals/approval-1", {
+      approved: true,
+    });
+    expect(res.status).toBe(401);
+    expect(state.calls).toHaveLength(0);
+  });
+});

plugins/plugin-computeruse/src/__tests__/runtime.live.e2e.test.ts

@@ -0,0 +1,212 @@
+/**
+ * Live end-to-end: spawns a real runtime child process with the plugin loaded and
+ * drives it over HTTP. Real process + real routes, no mocks.
+ */
+import { spawn } from "node:child_process";
+import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
+import net from "node:net";
+import os from "node:os";
+import path from "node:path";
+import { setTimeout as sleep } from "node:timers/promises";
+import { afterAll, beforeAll, describe, expect, it } from "vitest";
+import { createLiveRuntimeChildEnv } from "../../../../packages/app-core/test/helpers/live-child-env.ts";
+
+type RuntimeHarness = {
+  port: number;
+  close: () => Promise<void>;
+  logs: () => string;
+};
+
+type RuntimeOrderItem = {
+  name: string;
+  id: string | null;
+};
+
+type RuntimeServiceOrderItem = {
+  serviceType: string;
+};
+
+type RuntimeDebugSnapshot = {
+  runtimeAvailable: boolean;
+  order: {
+    plugins: RuntimeOrderItem[];
+    actions: RuntimeOrderItem[];
+    providers: RuntimeOrderItem[];
+    services: RuntimeServiceOrderItem[];
+  };
+};
+
+const REPO_ROOT = path.resolve(import.meta.dirname, "..", "..", "..", "..");
+
+async function getFreePort(): Promise<number> {
+  return new Promise((resolve, reject) => {
+    const server = net.createServer();
+    server.once("error", reject);
+    server.listen(0, "127.0.0.1", () => {
+      const addr = server.address();
+      if (!addr || typeof addr === "string") {
+        server.close();
+        reject(new Error("Failed to allocate port."));
+        return;
+      }
+      server.close((error) => {
+        if (error) {
+          reject(error);
+          return;
+        }
+        resolve(addr.port);
+      });
+    });
+  });
+}
+
+async function startRuntime(): Promise<RuntimeHarness> {
+  const tmp = await mkdtemp(path.join(os.tmpdir(), "computeruse-runtime-"));
+  const stateDir = path.join(tmp, "state");
+  const configPath = path.join(tmp, "eliza.json");
+  const port = await getFreePort();
+  const logBuf: string[] = [];
+
+  await mkdir(stateDir, { recursive: true });
+  await mkdir(path.join(stateDir, "cache"), { recursive: true });
+  await writeFile(
+    configPath,
+    JSON.stringify({
+      logging: { level: "info" },
+      plugins: { allow: ["computeruse"] },
+    }),
+    "utf8",
+  );
+
+  const child = spawn("bun", ["run", "start"], {
+    cwd: REPO_ROOT,
+    env: createLiveRuntimeChildEnv({
+      ELIZA_CONFIG_PATH: configPath,
+      ELIZA_STATE_DIR: stateDir,
+      ELIZA_PORT: String(port),
+      ELIZA_API_PORT: String(port),
+      CACHE_DIR: path.join(stateDir, "cache"),
+      ELIZA_DISABLE_LOCAL_EMBEDDINGS: "1",
+      COMPUTER_USE_ENABLED: "1",
+      ALLOW_NO_DATABASE: "",
+    }),
+    stdio: ["pipe", "pipe", "pipe"],
+  });
+
+  child.stdout.setEncoding("utf8");
+  child.stderr.setEncoding("utf8");
+  child.stdout.on("data", (chunk: string) => logBuf.push(chunk));
+  child.stderr.on("data", (chunk: string) => logBuf.push(chunk));
+
+  const deadline = Date.now() + 150_000;
+  let ready = false;
+  while (Date.now() < deadline) {
+    try {
+      const response = await fetch(`http://127.0.0.1:${port}/api/health`);
+      if (response.ok) {
+        const data = (await response.json()) as {
+          ready?: boolean;
+          runtime?: string;
+        };
+        if (data.ready === true && data.runtime === "ok") {
+          ready = true;
+          break;
+        }
+      }
+    } catch {
+      // Runtime is still booting.
+    }
+    await sleep(1000);
+  }
+
+  if (!ready) {
+    if (child.exitCode == null) {
+      child.kill("SIGTERM");
+      await new Promise<void>((resolve) => {
+        child.once("exit", () => resolve());
+        setTimeout(() => resolve(), 10_000);
+      });
+    }
+    await rm(tmp, { recursive: true, force: true });
+    throw new Error(
+      `Computer-use runtime failed to become ready.\n${logBuf.join("").slice(-8000)}`,
+    );
+  }
+
+  return {
+    port,
+    logs: () => logBuf.join("").slice(-8000),
+    close: async () => {
+      if (child.exitCode == null) {
+        child.kill("SIGTERM");
+        await new Promise<void>((resolve) => {
+          child.once("exit", () => resolve());
+          setTimeout(() => resolve(), 10_000);
+        });
+        if (child.exitCode == null) {
+          child.kill("SIGKILL");
+        }
+      }
+      await rm(tmp, { recursive: true, force: true });
+    },
+  };
+}
+
+const runE2E =
+  process.env.COMPUTER_USE_E2E_RUNTIME === "1" ||
+  process.env.COMPUTER_USE_LIVE_E2E === "1";
+
+describe.runIf(runE2E)("computer-use runtime registration (live e2e)", () => {
+  let runtime: RuntimeHarness | undefined;
+
+  beforeAll(async () => {
+    runtime = await startRuntime();
+  }, 180_000);
+
+  afterAll(async () => {
+    if (runtime) {
+      await runtime.close();
+    }
+  });
+
+  it("registers the plugin actions, provider, and service in a booted runtime", async () => {
+    const response = await fetch(
+      `http://127.0.0.1:${runtime.port}/api/runtime?maxDepth=1&maxArrayLength=200&maxObjectEntries=200&maxStringLength=256`,
+    );
+
+    expect(response.ok, runtime.logs()).toBe(true);
+    const snapshot = (await response.json()) as RuntimeDebugSnapshot;
+    expect(snapshot.runtimeAvailable).toBe(true);
+
+    const pluginNames = snapshot.order.plugins.map((item) => item.name);
+    const actionNames = snapshot.order.actions.map((item) => item.name);
+    const providerNames = snapshot.order.providers.map((item) => item.name);
+    const serviceTypes = snapshot.order.services.map(
+      (item) => item.serviceType,
+    );
+
+    expect(pluginNames).toContain("@elizaos/plugin-computeruse");
+    expect(actionNames).toEqual(
+      expect.arrayContaining(["COMPUTER_USE", "WINDOW"]),
+    );
+    expect(providerNames).toContain("computerState");
+    expect(serviceTypes).toContain("computeruse");
+  });
+
+  it("exposes the computer-use approval API against the live runtime", async () => {
+    const response = await fetch(
+      `http://127.0.0.1:${runtime.port}/api/computer-use/approvals`,
+    );
+
+    expect(response.ok, runtime.logs()).toBe(true);
+    const snapshot = (await response.json()) as {
+      mode: string;
+      pendingCount: number;
+      pendingApprovals: unknown[];
+    };
+
+    expect(snapshot.mode).toBeTruthy();
+    expect(typeof snapshot.pendingCount).toBe("number");
+    expect(Array.isArray(snapshot.pendingApprovals)).toBe(true);
+  });
+});

plugins/plugin-computeruse/src/__tests__/scene-builder.test.ts

@@ -0,0 +1,434 @@
+/**
+ * Scene-builder pipeline tests with fully synthetic deps.
+ *
+ * Asserts:
+ *   - JSON shape (typed Scene matches contract for WS7)
+ *   - Stable OCR id tagging across ticks
+ *   - dHash short-circuit: unchanged frames reuse cached OCR
+ *   - onAgentTurn always runs full pipeline (OCR+AX+apps refresh)
+ *   - Subscribers are notified once per tick
+ */
+
+import { deflateSync } from "node:zlib";
+import { describe, expect, it } from "vitest";
+import type { DisplayCapture } from "../platform/capture.js";
+import type { AccessibilityProvider } from "../scene/a11y-provider.js";
+import { SceneBuilder, type SceneUpdateEvent } from "../scene/scene-builder.js";
+import type {
+  SceneApp,
+  SceneAxNode,
+  SceneOcrBox,
+} from "../scene/scene-types.js";
+import type { DisplayDescriptor } from "../types.js";
+
+// ── tiny PNG builder, shared with dhash test ────────────────────────────────
+
+function crc32(bytes: Buffer): number {
+  let crc = 0xffffffff >>> 0;
+  for (let i = 0; i < bytes.length; i += 1) {
+    crc = (crc ^ bytes[i]!) >>> 0;
+    for (let j = 0; j < 8; j += 1) {
+      crc = (crc >>> 1) ^ (crc & 1 ? 0xedb88320 : 0);
+    }
+  }
+  return (crc ^ 0xffffffff) >>> 0;
+}
+
+function pngChunk(type: string, data: Buffer): Buffer {
+  const len = Buffer.alloc(4);
+  len.writeUInt32BE(data.length);
+  const t = Buffer.from(type, "ascii");
+  const crc = Buffer.alloc(4);
+  crc.writeUInt32BE(crc32(Buffer.concat([t, data])));
+  return Buffer.concat([len, t, data, crc]);
+}
+
+function makePng(seed: number): Buffer {
+  const w = 16;
+  const h = 16;
+  const rows: number[] = [];
+  for (let y = 0; y < h; y += 1) {
+    rows.push(0);
+    for (let x = 0; x < w; x += 1) {
+      const v = ((x + seed * 4) * 16) % 255;
+      rows.push(v, v, v);
+    }
+  }
+  const idat = deflateSync(Buffer.from(rows));
+  const ihdr = Buffer.alloc(13);
+  ihdr.writeUInt32BE(w, 0);
+  ihdr.writeUInt32BE(h, 4);
+  ihdr[8] = 8;
+  ihdr[9] = 2;
+  return Buffer.concat([
+    Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
+    pngChunk("IHDR", ihdr),
+    pngChunk("IDAT", idat),
+    pngChunk("IEND", Buffer.alloc(0)),
+  ]);
+}
+
+// ── fakes ───────────────────────────────────────────────────────────────────
+
+const fakeDisplays: DisplayDescriptor[] = [
+  {
+    id: 0,
+    bounds: [0, 0, 1920, 1080],
+    scaleFactor: 1,
+    primary: true,
+    name: "fake-1",
+  },
+];
+
+function makeFakeAxProvider(nodes: SceneAxNode[]): AccessibilityProvider {
+  return {
+    name: "fake",
+    available: () => true,
+    snapshot: async () => nodes,
+  };
+}
+
+function makeFakeOcr(perFrame: SceneOcrBox[][]): {
+  fn: (
+    png: Buffer,
+    displayId: number,
+    idState: { perDisplay: Map<number, number> },
+  ) => Promise<SceneOcrBox[]>;
+  callCount: () => number;
+} {
+  let i = 0;
+  let calls = 0;
+  return {
+    fn: async (_png, displayId, idState) => {
+      calls += 1;
+      const slice = perFrame[Math.min(i, perFrame.length - 1)] ?? [];
+      i += 1;
+      // Stamp fresh ids via the provided id state so we exercise the
+      // builder's per-display sequence assignment.
+      return slice.map((box) => {
+        const seq = (idState.perDisplay.get(displayId) ?? 0) + 1;
+        idState.perDisplay.set(displayId, seq);
+        return { ...box, id: `t${displayId}-${seq}`, displayId };
+      });
+    },
+    callCount: () => calls,
+  };
+}
+
+function makeBuilder(
+  options: {
+    captures?: DisplayCapture[][];
+    apps?: SceneApp[];
+    ax?: SceneAxNode[];
+    ocrBoxesPerCall?: SceneOcrBox[][];
+  } = {},
+): { builder: SceneBuilder; ocrCalls: () => number } {
+  const captures = options.captures ?? [
+    [
+      {
+        display: {
+          id: 0,
+          bounds: [0, 0, 1920, 1080],
+          scaleFactor: 1,
+          primary: true,
+          name: "fake-1",
+        },
+        frame: makePng(1),
+      },
+    ],
+  ];
+  let i = 0;
+  const ocr = makeFakeOcr(options.ocrBoxesPerCall ?? [[]]);
+  const builder = new SceneBuilder({
+    captureAll: async () =>
+      captures[Math.min(i++, captures.length - 1)] ??
+      captures[captures.length - 1]!,
+    captureOne: async () => {
+      const first = captures[0]?.[0];
+      if (!first) throw new Error("capture fixture list is empty");
+      return first;
+    },
+    listDisplays: () => fakeDisplays,
+    enumerateApps: () => options.apps ?? [],
+    accessibilityProvider: makeFakeAxProvider(options.ax ?? []),
+    runOcrOnFrame: ocr.fn,
+    log: () => {},
+  });
+  return { builder, ocrCalls: ocr.callCount };
+}
+
+// ── tests ───────────────────────────────────────────────────────────────────
+
+describe("SceneBuilder — JSON shape", () => {
+  it("produces a fully-typed Scene", async () => {
+    const { builder } = makeBuilder({
+      apps: [
+        {
+          name: "Firefox",
+          pid: 1234,
+          windows: [
+            {
+              id: "w1",
+              title: "GitHub",
+              bounds: [0, 0, 800, 600],
+              displayId: 0,
+            },
+          ],
+        },
+      ],
+      ax: [
+        {
+          id: "a0-1",
+          role: "window",
+          label: "GitHub",
+          bbox: [0, 0, 800, 600],
+          actions: ["focus"],
+          displayId: 0,
+        },
+      ],
+      ocrBoxesPerCall: [
+        [
+          {
+            id: "sample",
+            text: "Hello world",
+            bbox: [10, 20, 100, 24],
+            conf: 0.92,
+            displayId: 0,
+          },
+        ],
+      ],
+    });
+    const scene = await builder.tick("active");
+    expect(scene.timestamp).toBeGreaterThan(0);
+    expect(scene.displays).toHaveLength(1);
+    expect(scene.displays[0]?.id).toBe(0);
+    expect(scene.apps).toHaveLength(1);
+    expect(scene.apps[0]?.windows[0]?.title).toBe("GitHub");
+    expect(scene.ax).toHaveLength(1);
+    expect(scene.ax[0]?.id).toBe("a0-1");
+    expect(scene.ocr).toHaveLength(1);
+    // id was rewritten by the fake OCR adapter to t<display>-<seq>.
+    expect(scene.ocr[0]?.id).toBe("t0-1");
+    expect(scene.focused_window?.app).toBe("Firefox");
+    expect(scene.vlm_scene).toBeNull();
+    expect(scene.vlm_elements).toBeNull();
+  });
+});
+
+describe("SceneBuilder — dHash short-circuit", () => {
+  it("reuses cached OCR when the frame dHash is unchanged", async () => {
+    // Both captures return the same PNG bytes -> identical dHash.
+    const samePng = makePng(7);
+    const cap: DisplayCapture[] = [
+      {
+        display: {
+          id: 0,
+          bounds: [0, 0, 1920, 1080],
+          scaleFactor: 1,
+          primary: true,
+          name: "fake-1",
+        },
+        frame: samePng,
+      },
+    ];
+    const { builder, ocrCalls } = makeBuilder({
+      captures: [cap, cap, cap],
+      ocrBoxesPerCall: [
+        [
+          {
+            id: "x",
+            text: "frame1",
+            bbox: [0, 0, 10, 10],
+            conf: 0.5,
+            displayId: 0,
+          },
+        ],
+      ],
+    });
+    await builder.tick("active");
+    // Wait long enough that the idle-gate (2s) would skip OCR on the
+    // next tick, but instead we rely on the dHash short-circuit
+    // because the frame is identical.
+    const second = await builder.tick("active");
+    expect(ocrCalls()).toBe(1);
+    // Cached OCR is replayed.
+    expect(second.ocr).toHaveLength(1);
+    expect(second.ocr[0]?.text).toBe("frame1");
+  });
+
+  it("re-runs OCR when the frame dHash changes", async () => {
+    const cap1: DisplayCapture[] = [
+      {
+        display: {
+          id: 0,
+          bounds: [0, 0, 1920, 1080],
+          scaleFactor: 1,
+          primary: true,
+          name: "fake-1",
+        },
+        frame: makePng(1),
+      },
+    ];
+    const cap2: DisplayCapture[] = [
+      {
+        display: {
+          id: 0,
+          bounds: [0, 0, 1920, 1080],
+          scaleFactor: 1,
+          primary: true,
+          name: "fake-1",
+        },
+        frame: makePng(200),
+      },
+    ];
+    const { builder, ocrCalls } = makeBuilder({
+      captures: [cap1, cap2],
+      ocrBoxesPerCall: [
+        [
+          {
+            id: "a",
+            text: "first",
+            bbox: [0, 0, 10, 10],
+            conf: 0.5,
+            displayId: 0,
+          },
+        ],
+        [
+          {
+            id: "b",
+            text: "second",
+            bbox: [0, 0, 10, 10],
+            conf: 0.5,
+            displayId: 0,
+          },
+        ],
+      ],
+    });
+    await builder.tick("active");
+    const second = await builder.tick("active");
+    expect(ocrCalls()).toBe(2);
+    expect(second.ocr[0]?.text).toBe("second");
+  });
+});
+
+describe("SceneBuilder — subscribers + onAgentTurn", () => {
+  it("notifies subscribers once per tick with the produced Scene", async () => {
+    const { builder } = makeBuilder({});
+    const events: SceneUpdateEvent[] = [];
+    const unsub = builder.subscribe((e) => events.push(e));
+    await builder.tick("active");
+    await builder.onAgentTurn();
+    unsub();
+    expect(events).toHaveLength(2);
+    expect(events[0]?.reason).toBe("active");
+    expect(events[1]?.reason).toBe("agent-turn");
+  });
+
+  it("onAgentTurn forces a full-pipeline build even if dHash is unchanged", async () => {
+    const samePng = makePng(11);
+    const cap: DisplayCapture[] = [
+      {
+        display: {
+          id: 0,
+          bounds: [0, 0, 1920, 1080],
+          scaleFactor: 1,
+          primary: true,
+          name: "fake-1",
+        },
+        frame: samePng,
+      },
+    ];
+    const { builder, ocrCalls } = makeBuilder({
+      captures: [cap, cap],
+    });
+    await builder.tick("active");
+    await builder.onAgentTurn();
+    // Agent-turn forces OCR even if frame is unchanged.
+    expect(ocrCalls()).toBe(2);
+  });
+});
+
+describe("SceneBuilder — VLM annotations (M3)", () => {
+  it("populates vlm_scene / vlm_elements on the current scene", async () => {
+    const { builder } = makeBuilder({});
+    const scene = await builder.tick("active");
+    // Default: nothing wrote them yet.
+    expect(scene.vlm_scene).toBeNull();
+    expect(scene.vlm_elements).toBeNull();
+
+    builder.setVlmAnnotations("a save dialog is open", [
+      {
+        id: "tile-0-0",
+        kind: "tile",
+        desc: "Save button bottom-right",
+        bbox: [0, 0, 100, 50],
+        displayId: 0,
+      },
+    ]);
+    const after = builder.getCurrentScene();
+    expect(after?.vlm_scene).toBe("a save dialog is open");
+    expect(after?.vlm_elements).toHaveLength(1);
+    expect(after?.vlm_elements?.[0]?.desc).toBe("Save button bottom-right");
+  });
+
+  it("carries the VLM annotations forward to the next tick", async () => {
+    const samePng = makePng(11);
+    const cap: DisplayCapture[] = [
+      {
+        display: {
+          id: 0,
+          bounds: [0, 0, 1920, 1080],
+          scaleFactor: 1,
+          primary: true,
+          name: "fake-1",
+        },
+        frame: samePng,
+      },
+    ];
+    const { builder } = makeBuilder({ captures: [cap, cap] });
+    await builder.tick("active");
+    builder.setVlmAnnotations("scene paragraph", null);
+    // The next tick re-uses latestScene?.vlm_scene pass-through.
+    const next = await builder.tick("active");
+    expect(next.vlm_scene).toBe("scene paragraph");
+  });
+});
+
+describe("SceneBuilder — display-local coords", () => {
+  it("OCR bboxes are display-local (not OS-global)", async () => {
+    const { builder } = makeBuilder({
+      ocrBoxesPerCall: [
+        [
+          {
+            id: "x",
+            text: "Hello",
+            bbox: [10, 20, 50, 24],
+            conf: 0.9,
+            displayId: 0,
+          },
+        ],
+      ],
+    });
+    const scene = await builder.tick("active");
+    expect(scene.ocr[0]?.bbox).toEqual([10, 20, 50, 24]);
+    expect(scene.ocr[0]?.displayId).toBe(0);
+  });
+
+  it("AX bboxes carry displayId so coords.ts translation works", async () => {
+    const { builder } = makeBuilder({
+      ax: [
+        {
+          id: "a0-1",
+          role: "button",
+          label: "Submit",
+          bbox: [100, 50, 80, 30],
+          actions: ["press"],
+          displayId: 0,
+        },
+      ],
+    });
+    const scene = await builder.tick("active");
+    expect(scene.ax[0]?.displayId).toBe(0);
+    expect(scene.ax[0]?.bbox).toEqual([100, 50, 80, 30]);
+  });
+});

plugins/plugin-computeruse/src/__tests__/scene-multimon-coords.test.ts

@@ -0,0 +1,743 @@
+/**
+ * Cross-WS5/WS6 integration tests.
+ *
+ * Covers the seven audit dimensions:
+ *   1. Display coord round-trip: OCR bbox on a secondary display at origin
+ *      (2560,0) translates to a global click at (2660, 200).
+ *   2. dHash short-circuit holds over 10 identical frames in active mode.
+ *   3. Block-grid dirty re-OCR: a single-block change triggers exactly one
+ *      region capture + one OCR call on the cropped region.
+ *   5. Multi-display scene-builder: two displays produce non-overlapping
+ *      OCR ids `t0-X` and `t1-X` with proper displayId tags.
+ *   6. Wayland compositor dispatcher behavior with various XDG_CURRENT_DESKTOP
+ *      values.
+ *   7. captureDisplayRegion is wired into the dirty-block path (verified by
+ *      asserting captureRegion is invoked and runOcrOnFrame is not).
+ *   8. macOS retina scaleFactor pass-through for backing-store coords.
+ *
+ * Notes:
+ *   - These tests use synthetic PNGs and fully dependency-injected
+ *     SceneBuilder deps. No live host required.
+ *   - The Wayland dispatcher tests replace `commandExists` indirectly via env
+ *     manipulation and observe the snapshot() result.
+ */
+
+import { deflateSync } from "node:zlib";
+import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
+import type { DisplayCapture } from "../platform/capture.js";
+import { localToGlobal } from "../platform/coords.js";
+import type { DisplayInfo } from "../platform/displays.js";
+import * as displaysModule from "../platform/displays.js";
+import {
+  findDisplay,
+  listDisplays,
+  refreshDisplays,
+} from "../platform/displays.js";
+import { LinuxAccessibilityProvider } from "../scene/a11y-provider.js";
+import {
+  blockGrid,
+  coalesceDirtyBlocks,
+  diffBlocks,
+  pngDimensions,
+} from "../scene/dhash.js";
+import { SceneBuilder } from "../scene/scene-builder.js";
+import type { SceneOcrBox } from "../scene/scene-types.js";
+import type { DisplayDescriptor } from "../types.js";
+
+// ── tiny PNG builder (shared shape with scene-builder.test.ts) ──────────────
+
+function crc32(bytes: Buffer): number {
+  let crc = 0xffffffff >>> 0;
+  for (let i = 0; i < bytes.length; i += 1) {
+    crc = (crc ^ bytes[i]!) >>> 0;
+    for (let j = 0; j < 8; j += 1) {
+      crc = (crc >>> 1) ^ (crc & 1 ? 0xedb88320 : 0);
+    }
+  }
+  return (crc ^ 0xffffffff) >>> 0;
+}
+
+function pngChunk(type: string, data: Buffer): Buffer {
+  const len = Buffer.alloc(4);
+  len.writeUInt32BE(data.length);
+  const t = Buffer.from(type, "ascii");
+  const crc = Buffer.alloc(4);
+  crc.writeUInt32BE(crc32(Buffer.concat([t, data])));
+  return Buffer.concat([len, t, data, crc]);
+}
+
+/**
+ * Build a `wxh` RGB PNG. If `dirtyCell` is set, paint a contrasting square
+ * in the given block coordinates of a 16x16 block grid — used to trigger
+ * a single-block dirty-grid diff.
+ */
+function makePng(
+  seed: number,
+  w = 256,
+  h = 256,
+  dirtyCell?: {
+    col: number;
+    row: number;
+    cols?: number;
+    rows?: number;
+    value?: number;
+  },
+): Buffer {
+  const stride = w * 3;
+  const rows = Buffer.alloc((stride + 1) * h);
+  for (let y = 0; y < h; y += 1) {
+    const off = y * (stride + 1);
+    rows[off] = 0; // filter None
+    for (let x = 0; x < w; x += 1) {
+      const v = ((x + y + seed) * 7) % 200; // background gradient
+      const p = off + 1 + x * 3;
+      rows[p] = v;
+      rows[p + 1] = v;
+      rows[p + 2] = v;
+    }
+  }
+  if (dirtyCell) {
+    const cols = dirtyCell.cols ?? 16;
+    const grows = dirtyCell.rows ?? 16;
+    const x0 = Math.floor((dirtyCell.col * w) / cols);
+    const x1 = Math.floor(((dirtyCell.col + 1) * w) / cols);
+    const y0 = Math.floor((dirtyCell.row * h) / grows);
+    const y1 = Math.floor(((dirtyCell.row + 1) * h) / grows);
+    const value = dirtyCell.value ?? 250;
+    for (let y = y0; y < y1; y += 1) {
+      const off = y * (stride + 1);
+      for (let x = x0; x < x1; x += 1) {
+        const p = off + 1 + x * 3;
+        rows[p] = value;
+        rows[p + 1] = value;
+        rows[p + 2] = value;
+      }
+    }
+  }
+  const idat = deflateSync(rows);
+  const ihdr = Buffer.alloc(13);
+  ihdr.writeUInt32BE(w, 0);
+  ihdr.writeUInt32BE(h, 4);
+  ihdr[8] = 8;
+  ihdr[9] = 2;
+  return Buffer.concat([
+    Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
+    pngChunk("IHDR", ihdr),
+    pngChunk("IDAT", idat),
+    pngChunk("IEND", Buffer.alloc(0)),
+  ]);
+}
+
+// ── shared fake fixtures ────────────────────────────────────────────────────
+
+function fakeDisplay(
+  id: number,
+  x: number,
+  y: number,
+  w = 1920,
+  h = 1080,
+  scale = 1,
+): DisplayDescriptor {
+  return {
+    id,
+    bounds: [x, y, w, h],
+    scaleFactor: scale,
+    primary: id === 0,
+    name: `fake-${id}`,
+  };
+}
+
+function fakeCapture(id: number, png: Buffer): DisplayCapture {
+  return {
+    display: {
+      id,
+      bounds: [id === 0 ? 0 : 2560, 0, 1920, 1080],
+      scaleFactor: 1,
+      primary: id === 0,
+      name: `fake-${id}`,
+    } as DisplayInfo,
+    frame: png,
+  };
+}
+
+// ── 1. Display coord round-trip on a real secondary display ─────────────────
+
+describe("coord round-trip — secondary display", () => {
+  // Replace the displays singleton so coords.ts → localToGlobal sees a
+  // 2-display registry independent of the live host.
+  const twoDisplays: DisplayInfo[] = [
+    {
+      id: 0,
+      bounds: [0, 0, 2560, 1600],
+      scaleFactor: 1,
+      primary: true,
+      name: "eDP-1",
+    },
+    {
+      id: 1,
+      bounds: [2560, 0, 3840, 2160],
+      scaleFactor: 1,
+      primary: false,
+      name: "HDMI-0",
+    },
+  ];
+
+  let listSpy: ReturnType<typeof vi.spyOn>;
+  let findSpy: ReturnType<typeof vi.spyOn>;
+  beforeEach(() => {
+    listSpy = vi
+      .spyOn(displaysModule, "listDisplays")
+      .mockReturnValue(twoDisplays);
+    findSpy = vi
+      .spyOn(displaysModule, "findDisplay")
+      .mockImplementation(
+        (id: number) => twoDisplays.find((d) => d.id === id) ?? null,
+      );
+  });
+  afterEach(() => {
+    listSpy.mockRestore();
+    findSpy.mockRestore();
+  });
+
+  it("OCR bbox (100, 200) on displayId=1 origin (2560,0) translates to global (2660, 200)", () => {
+    // This is the canonical end-to-end check for the audit dimension 1:
+    // a Scene with an OCR box at display-local (100, 200) on displayId=1
+    // becomes a global click at (2660, 200).
+    const ocrBoxBboxLocal: [number, number, number, number] = [
+      100, 200, 50, 24,
+    ];
+    const displayId = 1;
+    const result = localToGlobal({
+      displayId,
+      x: ocrBoxBboxLocal[0],
+      y: ocrBoxBboxLocal[1],
+    });
+    expect(result).toEqual({ x: 2660, y: 200 });
+  });
+
+  it("primary display origin (0,0) is also honoured", () => {
+    const result = localToGlobal({ displayId: 0, x: 100, y: 200 });
+    expect(result).toEqual({ x: 100, y: 200 });
+  });
+
+  it("clicking with displayId=1 at corner (0,0) lands at the secondary origin", () => {
+    expect(localToGlobal({ displayId: 1, x: 0, y: 0 })).toEqual({
+      x: 2560,
+      y: 0,
+    });
+  });
+
+  it("parser fixture for parseXrandrMonitors matches the synthetic registry", () => {
+    const fixture = [
+      "Monitors: 2",
+      " 0: +*eDP-1 2560/390x1600/240+0+0  eDP-1",
+      " 1: +HDMI-0 3840/600x2160/340+2560+0  HDMI-0",
+      "",
+    ].join("\n");
+    const parsed = displaysModule.parseXrandrMonitors(fixture);
+    expect(parsed[1]?.bounds).toEqual([2560, 0, 3840, 2160]);
+  });
+});
+
+// ── 2. dHash short-circuit holds across 10 identical frames ─────────────────
+
+describe("dHash short-circuit — 10 identical frames", () => {
+  it("OCR runs exactly once across 10 identical active-mode ticks", async () => {
+    const png = makePng(42);
+    const cap: DisplayCapture[] = [fakeCapture(0, png)];
+    let ocrCalls = 0;
+    const builder = new SceneBuilder({
+      captureAll: async () => cap,
+      captureOne: async () => cap[0]!,
+      captureRegion: async () => {
+        throw new Error(
+          "captureRegion should not be invoked for identical frames",
+        );
+      },
+      listDisplays: () => [fakeDisplay(0, 0, 0)],
+      enumerateApps: () => [],
+      accessibilityProvider: {
+        name: "f",
+        available: () => true,
+        snapshot: async () => [],
+      },
+      runOcrOnFrame: async (_p, displayId, idState) => {
+        ocrCalls += 1;
+        const seq = (idState.perDisplay.get(displayId) ?? 0) + 1;
+        idState.perDisplay.set(displayId, seq);
+        return [
+          {
+            id: `t${displayId}-${seq}`,
+            text: "static-content",
+            bbox: [10, 20, 100, 24],
+            conf: 0.9,
+            displayId,
+          },
+        ];
+      },
+      runOcrOnCrops: async () => {
+        throw new Error(
+          "runOcrOnCrops should not be invoked for identical frames",
+        );
+      },
+      log: () => {},
+    });
+
+    // First tick = miss; subsequent 9 = cache hits.
+    const scenes = [];
+    for (let i = 0; i < 10; i += 1) {
+      scenes.push(await builder.tick("active"));
+    }
+    expect(ocrCalls).toBe(1);
+    // All 10 scenes carry the same OCR id (the original).
+    for (const s of scenes) {
+      expect(s.ocr).toHaveLength(1);
+      expect(s.ocr[0]?.text).toBe("static-content");
+      expect(s.ocr[0]?.id).toBe("t0-1");
+    }
+  });
+});
+
+// ── 3 + 7. Block-grid dirty re-OCR uses captureRegion ───────────────────────
+
+describe("dirty-block re-OCR — wired to captureRegion", () => {
+  it("single dirty block triggers region capture + cropped OCR (not full-frame)", async () => {
+    // Frame 1: clean background.
+    const frame1 = makePng(0, 256, 256);
+    // Frame 2: one block (col 4, row 4) painted bright — dirty fraction = 1/256 ~ 0.4%.
+    const frame2 = makePng(0, 256, 256, { col: 4, row: 4 });
+
+    const captures: DisplayCapture[][] = [
+      [fakeCapture(0, frame1)],
+      [fakeCapture(0, frame2)],
+    ];
+    const captureRegionCalls: Array<{
+      displayId: number;
+      region: { x: number; y: number; width: number; height: number };
+    }> = [];
+    let runOcrOnFrameCalls = 0;
+    let runOcrOnCropsCalls = 0;
+    let i = 0;
+    const builder = new SceneBuilder({
+      captureAll: async () =>
+        captures[Math.min(i++, captures.length - 1)] ??
+        captures[captures.length - 1]!,
+      captureOne: async () => {
+        const first = captures[0]?.[0];
+        if (!first) throw new Error("capture fixture list is empty");
+        return first;
+      },
+      captureRegion: async (displayId, region) => {
+        captureRegionCalls.push({ displayId, region });
+        // Return a tiny PNG of arbitrary content; OCR is fully mocked so
+        // contents don't matter, only the bbox bookkeeping does.
+        return {
+          display: {
+            id: displayId,
+            bounds: [0, 0, 1920, 1080],
+            scaleFactor: 1,
+            primary: true,
+            name: "fake-0",
+          },
+          frame: makePng(
+            displayId * 100 + region.x,
+            Math.max(8, region.width),
+            Math.max(8, region.height),
+          ),
+        };
+      },
+      listDisplays: () => [fakeDisplay(0, 0, 0)],
+      enumerateApps: () => [],
+      accessibilityProvider: {
+        name: "f",
+        available: () => true,
+        snapshot: async () => [],
+      },
+      runOcrOnFrame: async (_p, displayId, idState) => {
+        runOcrOnFrameCalls += 1;
+        const seq = (idState.perDisplay.get(displayId) ?? 0) + 1;
+        idState.perDisplay.set(displayId, seq);
+        return [
+          {
+            id: `t${displayId}-${seq}`,
+            text: "frame-text",
+            // A box well outside the dirty region (col=4 row=4 maps roughly to
+            // x∈[64,80], y∈[64,80] for a 256×256 / 16×16 grid). Use bbox at
+            // (200, 200) so dirty rect won't intersect it on Frame 2.
+            bbox: [200, 200, 40, 14],
+            conf: 0.9,
+            displayId,
+          },
+        ];
+      },
+      runOcrOnCrops: async (crops, displayId, idState) => {
+        runOcrOnCropsCalls += 1;
+        const out: SceneOcrBox[] = [];
+        for (const crop of crops) {
+          const seq = (idState.perDisplay.get(displayId) ?? 0) + 1;
+          idState.perDisplay.set(displayId, seq);
+          out.push({
+            id: `t${displayId}-${seq}`,
+            text: "dirty-cell-text",
+            // Bbox is in display-local source coords (offset by crop origin).
+            bbox: [crop.bbox[0] + 2, crop.bbox[1] + 2, 10, 10],
+            conf: 0.95,
+            displayId,
+          });
+        }
+        return out;
+      },
+      log: () => {},
+    });
+
+    // First tick — establishes lastBlockGrid + lastScene cache.
+    await builder.tick("active");
+    expect(runOcrOnFrameCalls).toBe(1);
+    expect(captureRegionCalls).toHaveLength(0);
+
+    // Second tick — only one block changed. Expect dirty path.
+    const scene2 = await builder.tick("active");
+    expect(runOcrOnCropsCalls).toBe(1);
+    // captureRegion should fire exactly once (one coalesced rect).
+    expect(captureRegionCalls).toHaveLength(1);
+    // The dirty rect should be inside the source frame and roughly near
+    // (col=4, row=4) for a 16×16 grid on a 256×256 frame → ~(64, 64, 16, 16).
+    const reg = captureRegionCalls[0]?.region;
+    if (!reg) throw new Error("expected dirty-region capture call");
+    expect(reg.x).toBeGreaterThanOrEqual(60);
+    expect(reg.x).toBeLessThanOrEqual(80);
+    expect(reg.y).toBeGreaterThanOrEqual(60);
+    expect(reg.y).toBeLessThanOrEqual(80);
+    expect(reg.width).toBeGreaterThanOrEqual(8);
+    expect(reg.height).toBeGreaterThanOrEqual(8);
+
+    // Scene OCR carries the retained previous-frame box (frame-text @ 200,200)
+    // plus the new dirty-cell-text box from the cropped OCR.
+    expect(scene2.ocr.length).toBe(2);
+    const dirtyBox = scene2.ocr.find((b) => b.text === "dirty-cell-text");
+    expect(dirtyBox).toBeDefined();
+    expect(dirtyBox?.bbox[0]).toBeGreaterThanOrEqual(60);
+    const retained = scene2.ocr.find((b) => b.text === "frame-text");
+    expect(retained).toBeDefined();
+    expect(retained?.bbox).toEqual([200, 200, 40, 14]);
+  });
+});
+
+// ── 5. Multi-display capture → non-overlapping ids ──────────────────────────
+
+describe("multi-display scene-builder", () => {
+  it("emits non-overlapping `t0-X` and `t1-X` ids tagged by displayId", async () => {
+    const cap = [fakeCapture(0, makePng(1)), fakeCapture(1, makePng(2))];
+    const builder = new SceneBuilder({
+      captureAll: async () => cap,
+      captureOne: async (displayId) =>
+        cap.find((c) => c.display.id === displayId)!,
+      listDisplays: () => [
+        fakeDisplay(0, 0, 0, 1920, 1080),
+        fakeDisplay(1, 2560, 0, 3840, 2160),
+      ],
+      enumerateApps: () => [],
+      accessibilityProvider: {
+        name: "f",
+        available: () => true,
+        snapshot: async () => [],
+      },
+      runOcrOnFrame: async (_p, displayId, idState) => {
+        const seq = (idState.perDisplay.get(displayId) ?? 0) + 1;
+        idState.perDisplay.set(displayId, seq);
+        return [
+          {
+            id: `t${displayId}-${seq}`,
+            text: `box-on-${displayId}`,
+            bbox: [10, 20, 30, 14],
+            conf: 0.9,
+            displayId,
+          },
+        ];
+      },
+      log: () => {},
+    });
+    const scene = await builder.tick("active");
+    expect(scene.displays).toHaveLength(2);
+    expect(scene.ocr.length).toBe(2);
+    const ids = scene.ocr.map((b) => b.id).sort();
+    expect(ids).toEqual(["t0-1", "t1-1"]);
+    const byDisplay = new Map(scene.ocr.map((b) => [b.displayId, b]));
+    expect(byDisplay.get(0)?.text).toBe("box-on-0");
+    expect(byDisplay.get(1)?.text).toBe("box-on-1");
+    // Bboxes are display-local — both at (10, 20).
+    expect(byDisplay.get(0)?.bbox).toEqual([10, 20, 30, 14]);
+    expect(byDisplay.get(1)?.bbox).toEqual([10, 20, 30, 14]);
+  });
+});
+
+// ── 6. Wayland compositor dispatcher behavior ───────────────────────────────
+
+describe("LinuxAccessibilityProvider — Wayland dispatcher", () => {
+  let originalXdg: string | undefined;
+  beforeEach(() => {
+    originalXdg = process.env.XDG_CURRENT_DESKTOP;
+  });
+  afterEach(() => {
+    if (originalXdg === undefined) delete process.env.XDG_CURRENT_DESKTOP;
+    else process.env.XDG_CURRENT_DESKTOP = originalXdg;
+  });
+
+  it("returns empty array when AT-SPI is unreachable and no Wayland compositor matches", async () => {
+    // Simulate a GNOME Wayland host where hyprctl/swaymsg aren't installed.
+    // We can't mock commandExists from here without re-loading the module,
+    // so we instead drive the provider on the real host: if neither hyprctl
+    // nor swaymsg is installed, snapshot() must return [] (not throw) even
+    // when XDG is set to something unknown.
+    process.env.XDG_CURRENT_DESKTOP = "GNOME";
+    const provider = new LinuxAccessibilityProvider();
+    const nodes = await provider.snapshot();
+    // We only assert no throw + valid shape. The real host might have AT-SPI
+    // and produce nodes — that's fine; the contract is "returns Array".
+    expect(Array.isArray(nodes)).toBe(true);
+  });
+
+  it("handles undefined XDG_CURRENT_DESKTOP gracefully (defaults to AT-SPI then empty)", async () => {
+    delete process.env.XDG_CURRENT_DESKTOP;
+    const provider = new LinuxAccessibilityProvider();
+    const nodes = await provider.snapshot();
+    expect(Array.isArray(nodes)).toBe(true);
+  });
+
+  it("handles unknown XDG_CURRENT_DESKTOP value cleanly", async () => {
+    process.env.XDG_CURRENT_DESKTOP = "Plasma:KDE:wayland:something-weird";
+    const provider = new LinuxAccessibilityProvider();
+    const nodes = await provider.snapshot();
+    expect(Array.isArray(nodes)).toBe(true);
+  });
+});
+
+// ── 8. macOS retina scaleFactor pass-through ────────────────────────────────
+
+describe("coords — backing-store retina translation", () => {
+  it("backing-store coords on a retina display divide by scaleFactor", async () => {
+    // Inject a synthetic retina display into the cache via the parser path.
+    // The displays singleton on Linux will be xrandr; we exercise the
+    // translate() math directly via localToGlobal against a fixture display
+    // by forcing findDisplay through globalToLocal. The cleanest test
+    // is to assert the arithmetic against parseDarwinDisplays output.
+    const displaysModule = await import("../platform/displays.js");
+    const fixture = JSON.stringify([
+      {
+        id: 1,
+        bounds: { x: 0, y: 0, width: 2560, height: 1440 },
+        pixelWidth: 5120,
+        pixelHeight: 2880,
+        primary: true,
+        name: "main",
+      },
+    ]);
+    const parsed = displaysModule.parseDarwinDisplays(fixture);
+    expect(parsed[0]?.scaleFactor).toBe(2);
+    // Manual mirror of coords.ts translate() with coordSource="backing" on
+    // darwin: divide by scaleFactor THEN add display origin.
+    const d = parsed[0]!;
+    const localBacking = { x: 100, y: 200 };
+    const lx = localBacking.x / d.scaleFactor;
+    const ly = localBacking.y / d.scaleFactor;
+    const result = {
+      x: Math.round(d.bounds[0] + lx),
+      y: Math.round(d.bounds[1] + ly),
+    };
+    expect(result).toEqual({ x: 50, y: 100 });
+  });
+
+  it("Scene.displays[0].bounds carries the right LOGICAL size for retina captures", async () => {
+    // The retina display's `bounds[2..3]` are the LOGICAL points (2560×1440),
+    // even though the captured PNG is backing-store (5120×2880). That's the
+    // contract for WS6: bboxes inside OCR/AX are display-local in the model's
+    // coord space; backing-store conversion is the click-dispatch boundary.
+    const displaysModule = await import("../platform/displays.js");
+    const fixture = JSON.stringify([
+      {
+        id: 1,
+        bounds: { x: 0, y: 0, width: 2560, height: 1440 },
+        pixelWidth: 5120,
+        pixelHeight: 2880,
+        primary: true,
+        name: "main",
+      },
+    ]);
+    const parsed = displaysModule.parseDarwinDisplays(fixture);
+    expect(parsed[0]?.bounds).toEqual([0, 0, 2560, 1440]);
+    expect(parsed[0]?.scaleFactor).toBe(2);
+  });
+});
+
+// ── coalesceDirtyBlocks unit tests (supports dim 3 + dim 7 wiring) ─────────
+
+describe("coalesceDirtyBlocks", () => {
+  it("merges a horizontal strip into one rect", () => {
+    const grid = blockGrid(makePng(0, 256, 256), 16, 16)!;
+    const dirty = [
+      {
+        col: 3,
+        row: 5,
+        bbox: [0, 0, 1, 1] as [number, number, number, number],
+      },
+      {
+        col: 4,
+        row: 5,
+        bbox: [0, 0, 1, 1] as [number, number, number, number],
+      },
+      {
+        col: 5,
+        row: 5,
+        bbox: [0, 0, 1, 1] as [number, number, number, number],
+      },
+    ];
+    const rects = coalesceDirtyBlocks(dirty, grid, 256, 256);
+    expect(rects).toHaveLength(1);
+    expect(rects[0]?.bbox[0]).toBe(Math.floor((3 * 256) / 16));
+    expect(rects[0]?.bbox[2]).toBeGreaterThan(0);
+  });
+
+  it("merges a vertical strip into one rect", () => {
+    const grid = blockGrid(makePng(0, 256, 256), 16, 16)!;
+    const dirty = [
+      {
+        col: 2,
+        row: 4,
+        bbox: [0, 0, 1, 1] as [number, number, number, number],
+      },
+      {
+        col: 2,
+        row: 5,
+        bbox: [0, 0, 1, 1] as [number, number, number, number],
+      },
+      {
+        col: 2,
+        row: 6,
+        bbox: [0, 0, 1, 1] as [number, number, number, number],
+      },
+    ];
+    const rects = coalesceDirtyBlocks(dirty, grid, 256, 256);
+    expect(rects).toHaveLength(1);
+    expect(rects[0]?.bbox[3]).toBeGreaterThan(0);
+  });
+
+  it("keeps disjoint regions separate", () => {
+    const grid = blockGrid(makePng(0, 256, 256), 16, 16)!;
+    const dirty = [
+      {
+        col: 1,
+        row: 1,
+        bbox: [0, 0, 1, 1] as [number, number, number, number],
+      },
+      {
+        col: 10,
+        row: 10,
+        bbox: [0, 0, 1, 1] as [number, number, number, number],
+      },
+    ];
+    const rects = coalesceDirtyBlocks(dirty, grid, 256, 256);
+    expect(rects).toHaveLength(2);
+  });
+
+  it("handles empty dirty list", () => {
+    const grid = blockGrid(makePng(0, 256, 256), 16, 16)!;
+    expect(coalesceDirtyBlocks([], grid, 256, 256)).toEqual([]);
+  });
+});
+
+// ── pngDimensions sanity ────────────────────────────────────────────────────
+
+describe("pngDimensions", () => {
+  it("reads IHDR dimensions without inflating IDAT", () => {
+    expect(pngDimensions(makePng(0, 256, 192))).toEqual({
+      width: 256,
+      height: 192,
+    });
+    expect(pngDimensions(makePng(7, 96, 32))).toEqual({
+      width: 96,
+      height: 32,
+    });
+  });
+  it("returns null for non-PNG input", () => {
+    expect(pngDimensions(Buffer.from("garbage"))).toBeNull();
+  });
+});
+
+// ── live secondary-display round-trip (only when host has 2+ displays) ─────
+
+describe("live secondary-display round-trip", () => {
+  it("if the host has a secondary display, localToGlobal honours its origin", () => {
+    refreshDisplays();
+    const all = listDisplays();
+    if (all.length < 2) return; // Linux CI host has 1 display — covered by direct-arithmetic test above.
+    const secondary = all.find((d) => !d.primary);
+    if (!secondary) return;
+    const display = findDisplay(secondary.id);
+    expect(display).not.toBeNull();
+    const result = localToGlobal({ displayId: secondary.id, x: 100, y: 200 });
+    expect(result.x).toBe(secondary.bounds[0] + 100);
+    expect(result.y).toBe(secondary.bounds[1] + 200);
+  });
+});
+
+// ── live single-display smoke test (this Linux host) ────────────────────────
+
+describe("live smoke — getCurrentScene on this host", () => {
+  it("listDisplays returns at least one display with positive bounds", () => {
+    const all = listDisplays();
+    expect(all.length).toBeGreaterThan(0);
+    const firstDisplay = all[0];
+    if (!firstDisplay) throw new Error("expected at least one display");
+    expect(firstDisplay.bounds[2]).toBeGreaterThan(0);
+    expect(firstDisplay.bounds[3]).toBeGreaterThan(0);
+  });
+});
+
+// ── diffBlocks ↔ coalesceDirtyBlocks sanity for a real PNG round-trip ──────
+
+describe("diffBlocks + coalesce round-trip on a synthetic frame", () => {
+  it("a single-block change produces one dirty block and one rect", () => {
+    const frame1 = makePng(0, 256, 256);
+    const frame2 = makePng(0, 256, 256, { col: 7, row: 8 });
+    const g1 = blockGrid(frame1, 16, 16)!;
+    const g2 = blockGrid(frame2, 16, 16)!;
+    const dirty = diffBlocks(g1, g2, 256, 256);
+    expect(dirty.length).toBeGreaterThanOrEqual(1);
+    expect(dirty.length).toBeLessThanOrEqual(4);
+    const rects = coalesceDirtyBlocks(dirty, g2, 256, 256);
+    expect(rects.length).toBeGreaterThanOrEqual(1);
+    expect(rects.length).toBeLessThanOrEqual(dirty.length);
+  });
+});
+
+// ── dim 4: agent-turn cache miss confirmation (defense-in-depth) ───────────
+
+describe("agent-turn cache miss — confirmation", () => {
+  it("agent-turn re-runs OCR even when 5 consecutive frames are pixel-identical", async () => {
+    const png = makePng(99);
+    const cap: DisplayCapture[] = [fakeCapture(0, png)];
+    let ocrCalls = 0;
+    const builder = new SceneBuilder({
+      captureAll: async () => cap,
+      captureOne: async () => cap[0]!,
+      listDisplays: () => [fakeDisplay(0, 0, 0)],
+      enumerateApps: () => [],
+      accessibilityProvider: {
+        name: "f",
+        available: () => true,
+        snapshot: async () => [],
+      },
+      runOcrOnFrame: async (_p, displayId, idState) => {
+        ocrCalls += 1;
+        const seq = (idState.perDisplay.get(displayId) ?? 0) + 1;
+        idState.perDisplay.set(displayId, seq);
+        return [];
+      },
+      log: () => {},
+    });
+    for (let i = 0; i < 5; i += 1) {
+      await builder.onAgentTurn();
+    }
+    expect(ocrCalls).toBe(5);
+  });
+});

plugins/plugin-computeruse/src/__tests__/scene-provider.test.ts

@@ -0,0 +1,168 @@
+/**
+ * Provider integration test for the `scene` provider.
+ *
+ * The provider is supposed to:
+ *   1. Read `service.getCurrentScene()` first.
+ *   2. If no scene yet, force a `refreshScene("agent-turn")`.
+ *   3. Serialize the result via `serializeSceneForPrompt` and emit both
+ *      `text` (fenced JSON for the prompt) and `data.scene` (the full
+ *      Scene for downstream pieces like WS7).
+ */
+
+import type { IAgentRuntime, Memory, State } from "@elizaos/core";
+import { describe, expect, it } from "vitest";
+import { sceneProvider } from "../providers/scene.js";
+import type {
+  Scene,
+  SceneApp,
+  SceneAxNode,
+  SceneOcrBox,
+} from "../scene/scene-types.js";
+
+function makeScene(): Scene {
+  const ocr: SceneOcrBox[] = [
+    {
+      id: "t0-1",
+      text: "Hello",
+      bbox: [10, 20, 100, 24],
+      conf: 0.95,
+      displayId: 0,
+    },
+    {
+      id: "t0-2",
+      text: "World",
+      bbox: [10, 60, 100, 24],
+      conf: 0.83,
+      displayId: 0,
+    },
+  ];
+  const ax: SceneAxNode[] = [
+    {
+      id: "a0-1",
+      role: "button",
+      label: "Submit",
+      bbox: [10, 100, 80, 30],
+      actions: ["press"],
+      displayId: 0,
+    },
+  ];
+  const apps: SceneApp[] = [
+    {
+      name: "Firefox",
+      pid: 1234,
+      windows: [
+        {
+          id: "w1",
+          title: "Eliza Dashboard",
+          bounds: [0, 0, 1920, 1080],
+          displayId: 0,
+        },
+      ],
+    },
+  ];
+  return {
+    timestamp: 1_700_000_000_000,
+    displays: [
+      {
+        id: 0,
+        bounds: [0, 0, 1920, 1080],
+        scaleFactor: 1,
+        primary: true,
+        name: "fake-1",
+      },
+    ],
+    focused_window: {
+      app: "Firefox",
+      pid: 1234,
+      bounds: [0, 0, 1920, 1080],
+      title: "Eliza Dashboard",
+      displayId: 0,
+    },
+    apps,
+    ocr,
+    ax,
+    vlm_scene: null,
+    vlm_elements: null,
+  };
+}
+
+function makeRuntime(
+  scene: Scene | null,
+  refresh?: () => Promise<Scene>,
+): IAgentRuntime & { reportedErrors: Array<{ scope: string }> } {
+  const service = {
+    getCurrentScene: () => scene,
+    refreshScene:
+      refresh ??
+      (async () => {
+        throw new Error("no scene available");
+      }),
+  };
+  const reportedErrors: Array<{ scope: string }> = [];
+  return {
+    getService: (name: string) =>
+      name === "computeruse" ? (service as unknown) : undefined,
+    reportError: (scope: string) => {
+      reportedErrors.push({ scope });
+    },
+    reportedErrors,
+  } as unknown as IAgentRuntime & { reportedErrors: Array<{ scope: string }> };
+}
+
+const dummyMessage: Memory = {} as Memory;
+const dummyState: State = {} as State;
+
+describe("sceneProvider", () => {
+  it("returns parsed text + data for an existing scene", async () => {
+    const scene = makeScene();
+    const result = await sceneProvider.get(
+      makeRuntime(scene),
+      dummyMessage,
+      dummyState,
+    );
+    expect(result.text).toContain("```json");
+    expect(result.text).toContain('"id": "t0-1"');
+    expect(result.text).toContain("Firefox");
+    expect(result.data?.scene?.ocr).toHaveLength(2);
+    expect(result.values?.sceneDisplayCount).toBe(1);
+    expect(result.values?.sceneOcrCount).toBe(2);
+    expect(result.values?.sceneAxCount).toBe(1);
+  });
+
+  it("forces refresh when no scene is cached", async () => {
+    const scene = makeScene();
+    let calls = 0;
+    const refresh = async (): Promise<Scene> => {
+      calls += 1;
+      return scene;
+    };
+    const result = await sceneProvider.get(
+      makeRuntime(null, refresh),
+      dummyMessage,
+      dummyState,
+    );
+    expect(calls).toBe(1);
+    expect(result.text).toContain('"id": "t0-1"');
+  });
+
+  it("returns empty when the service is missing", async () => {
+    const runtime = {
+      getService: () => undefined,
+    } as unknown as IAgentRuntime;
+    const result = await sceneProvider.get(runtime, dummyMessage, dummyState);
+    expect(result.text).toBe("");
+  });
+
+  it("returns empty AND reports the failure when refresh fails and no scene is cached", async () => {
+    const runtime = makeRuntime(null, async () => {
+      throw new Error("boom");
+    });
+    const result = await sceneProvider.get(runtime, dummyMessage, dummyState);
+    expect(result.text).toBe("");
+    // A broken scene pipeline must be agent-visible (#12273), not a
+    // silently-empty provider result.
+    expect(runtime.reportedErrors).toEqual([
+      { scope: "Computeruse.sceneProvider" },
+    ]);
+  });
+});

plugins/plugin-computeruse/src/__tests__/screen-state.test.ts

@@ -0,0 +1,155 @@
+/**
+ * Unified ScreenState store (#9105 M3).
+ *
+ * One capture per turn, reused inside the freshness window; change events only
+ * fire when the frame dHash actually moved. Uses real synthesized PNGs so
+ * `frameDhash` / `blockGrid` are meaningful.
+ */
+
+import { deflateSync } from "node:zlib";
+import { describe, expect, it } from "vitest";
+import type { DisplayCapture } from "../platform/capture.js";
+import { ScreenStateStore } from "../scene/screen-state.js";
+
+// ── minimal PNG synthesizer (32x32 RGB, gradient seeded so dHash differs) ────
+function crc32(buf: Buffer): number {
+  let crc = 0xffffffff;
+  for (let i = 0; i < buf.length; i += 1) {
+    crc ^= buf[i] ?? 0;
+    for (let k = 0; k < 8; k += 1) {
+      crc = crc & 1 ? (crc >>> 1) ^ 0xedb88320 : crc >>> 1;
+    }
+  }
+  return (crc ^ 0xffffffff) >>> 0;
+}
+function pngChunk(type: string, data: Buffer): Buffer {
+  const len = Buffer.alloc(4);
+  len.writeUInt32BE(data.length);
+  const t = Buffer.from(type, "ascii");
+  const crc = Buffer.alloc(4);
+  crc.writeUInt32BE(crc32(Buffer.concat([t, data])));
+  return Buffer.concat([len, t, data, crc]);
+}
+function makePng(seed = 0): Buffer {
+  const w = 32;
+  const h = 32;
+  const rows: number[] = [];
+  for (let y = 0; y < h; y += 1) {
+    rows.push(0);
+    for (let x = 0; x < w; x += 1) {
+      const v = ((x + seed) * 8) % 255;
+      rows.push(v, v, v);
+    }
+  }
+  const idat = deflateSync(Buffer.from(rows));
+  const ihdr = Buffer.alloc(13);
+  ihdr.writeUInt32BE(w, 0);
+  ihdr.writeUInt32BE(h, 4);
+  ihdr[8] = 8;
+  ihdr[9] = 2;
+  const sig = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
+  return Buffer.concat([
+    sig,
+    pngChunk("IHDR", ihdr),
+    pngChunk("IDAT", idat),
+    pngChunk("IEND", Buffer.alloc(0)),
+  ]);
+}
+
+function capture(frame: Buffer): DisplayCapture {
+  return {
+    display: {
+      id: 0,
+      bounds: [0, 0, 32, 32],
+      scaleFactor: 1,
+      primary: true,
+      name: "f",
+    },
+    frame,
+  };
+}
+
+describe("ScreenStateStore (M3)", () => {
+  it("computes dHash + blockGrid + dimensions on first capture", async () => {
+    const store = new ScreenStateStore({
+      capture: async () => capture(makePng(3)),
+    });
+    const state = await store.get(0, true);
+    expect(state.displayId).toBe(0);
+    expect(state.width).toBe(32);
+    expect(state.height).toBe(32);
+    expect(state.dhash).not.toBeNull();
+    expect(state.blockGrid).not.toBeNull();
+    // First frame has no prior to diff against.
+    expect(state.dirtyBlocks).toBeNull();
+    // First capture counts as a change (distance Infinity ≥ threshold).
+    expect(store.getStats()).toEqual({
+      captures: 1,
+      cacheHits: 0,
+      changes: 1,
+    });
+  });
+
+  it("serves a fresh capture from cache (one OS capture per turn)", async () => {
+    let calls = 0;
+    let clock = 1000;
+    const store = new ScreenStateStore({
+      capture: async () => {
+        calls += 1;
+        return capture(makePng(5));
+      },
+      freshnessMs: 400,
+      now: () => clock,
+    });
+    await store.get(0); // fresh capture
+    clock += 100; // still within freshness window
+    await store.get(0); // served from cache
+    clock += 100;
+    await store.get(0); // served from cache
+    expect(calls).toBe(1);
+    expect(store.getStats().captures).toBe(1);
+    expect(store.getStats().cacheHits).toBe(2);
+  });
+
+  it("re-captures once the freshness window elapses", async () => {
+    let calls = 0;
+    let clock = 0;
+    const store = new ScreenStateStore({
+      capture: async () => {
+        calls += 1;
+        return capture(makePng(5));
+      },
+      freshnessMs: 400,
+      now: () => clock,
+    });
+    await store.get(0);
+    clock += 500; // window elapsed
+    await store.get(0);
+    expect(calls).toBe(2);
+  });
+
+  it("fires a change event only when the frame dHash moves ≥ threshold", async () => {
+    const frames = [makePng(0), makePng(0), makePng(80)];
+    let i = 0;
+    let clock = 0;
+    const store = new ScreenStateStore({
+      capture: async () => capture(frames[i++] ?? frames[frames.length - 1]!),
+      now: () => clock,
+    });
+    const changes: number[] = [];
+    store.onChange((c) => changes.push(c.distance));
+
+    await store.refresh(0); // first frame: distance Infinity → change
+    clock += 1;
+    await store.refresh(0); // identical pixels → no change
+    clock += 1;
+    const third = await store.refresh(0); // visually different → change
+
+    // First frame and the visually-different third frame fire; the identical
+    // middle frame does not.
+    expect(changes).toHaveLength(2);
+    expect(changes[0]).toBe(Number.POSITIVE_INFINITY);
+    expect(third.dirtyBlocks).not.toBeNull();
+    expect(store.getStats().changes).toBe(2);
+  });
+});

plugins/plugin-computeruse/src/__tests__/screenshot-errors.test.ts

@@ -0,0 +1,166 @@
+/**
+ * Unit tests for the structured screenshot error contract (#9105 M3.5).
+ *
+ * Verifies the code-classification mapping has a single source of truth and
+ * that `tagScreenshotError` is purely additive — a `PermissionDeniedError`
+ * keeps its identity while gaining a `screenshotErrorCode`.
+ */
+
+import { describe, expect, it } from "vitest";
+import {
+  createPermissionDeniedError,
+  isPermissionDeniedError,
+} from "../platform/permissions.js";
+import {
+  classifyScreenshotErrorCode,
+  createScreenshotError,
+  isScreenshotError,
+  type ScreenshotErrorCode,
+  tagScreenshotError,
+} from "../platform/screenshot-errors.js";
+
+describe("classifyScreenshotErrorCode", () => {
+  it("maps a permission-denied error to permission_denied", () => {
+    const err = createPermissionDeniedError({
+      permissionType: "screen_recording",
+      operation: "screenshot_capture",
+      message: "Screen Recording permission required.",
+    });
+    expect(classifyScreenshotErrorCode(err)).toBe("permission_denied");
+  });
+
+  it("maps the Linux no-tool message to tool_missing", () => {
+    const err = new Error(
+      "No screenshot tool available. Install ImageMagick (import), scrot, or gnome-screenshot.",
+    );
+    expect(classifyScreenshotErrorCode(err)).toBe("tool_missing");
+  });
+
+  it("maps ENOENT (spawned tool not found) to tool_missing", () => {
+    const err = new Error("spawn scrot ENOENT");
+    expect(classifyScreenshotErrorCode(err)).toBe("tool_missing");
+  });
+
+  it("maps an empty-output message to empty_output", () => {
+    expect(
+      classifyScreenshotErrorCode(
+        new Error("screencapture returned an empty file."),
+      ),
+    ).toBe("empty_output");
+    expect(
+      classifyScreenshotErrorCode(
+        new Error("capture produced a zero-byte image"),
+      ),
+    ).toBe("empty_output");
+  });
+
+  it("maps timeout messages to timeout", () => {
+    expect(classifyScreenshotErrorCode(new Error("Command timed out"))).toBe(
+      "timeout",
+    );
+    expect(
+      classifyScreenshotErrorCode(
+        new Error("spawnSync screencapture ETIMEDOUT"),
+      ),
+    ).toBe("timeout");
+  });
+
+  it("falls back to capture_failed for unrecognized errors", () => {
+    expect(classifyScreenshotErrorCode(new Error("something odd"))).toBe(
+      "capture_failed",
+    );
+  });
+
+  it("classifies a non-Error value as capture_failed", () => {
+    expect(classifyScreenshotErrorCode("weird string failure")).toBe(
+      "capture_failed",
+    );
+  });
+});
+
+describe("tagScreenshotError", () => {
+  it("annotates an Error in place and preserves its identity", () => {
+    const original = new Error("Command timed out");
+    const tagged = tagScreenshotError(original, "screenshot_capture");
+    expect(tagged).toBe(original); // same object — additive, not a wrapper
+    expect(tagged.screenshotErrorCode).toBe("timeout");
+    expect(tagged.operation).toBe("screenshot_capture");
+    expect(isScreenshotError(tagged)).toBe(true);
+  });
+
+  it("keeps a PermissionDeniedError recognizable while adding the code", () => {
+    const perm = createPermissionDeniedError({
+      permissionType: "screen_recording",
+      operation: "screenshot_capture",
+      message: "Screen Recording permission required.",
+    });
+    const tagged = tagScreenshotError(perm, "screenshot_region");
+    // Still a permission-denied error for existing callers...
+    expect(isPermissionDeniedError(tagged)).toBe(true);
+    // ...AND now also a screenshot error with the right code.
+    expect(isScreenshotError(tagged)).toBe(true);
+    expect(tagged.screenshotErrorCode).toBe("permission_denied");
+    // The error carried its own operation, so it is not overwritten.
+    expect(tagged.operation).toBe("screenshot_capture");
+  });
+
+  it("is idempotent", () => {
+    const err = new Error("No screenshot tool available.");
+    const once = tagScreenshotError(err, "screenshot_capture");
+    const twice = tagScreenshotError(once, "screenshot_region");
+    expect(twice).toBe(once);
+    expect(twice.screenshotErrorCode).toBe("tool_missing");
+    expect(twice.operation).toBe("screenshot_capture");
+  });
+
+  it("wraps a non-Error value in a fresh ScreenshotError", () => {
+    const tagged = tagScreenshotError("kernel said no", "screenshot_capture");
+    expect(isScreenshotError(tagged)).toBe(true);
+    expect(tagged.screenshotErrorCode).toBe("capture_failed");
+    expect(tagged.operation).toBe("screenshot_capture");
+    expect(tagged.message).toBe("kernel said no");
+  });
+
+  it("defaults operation only when the error lacks one", () => {
+    const noOp = new Error("boom");
+    const tagged = tagScreenshotError(noOp, "screenshot_region");
+    expect(tagged.operation).toBe("screenshot_region");
+  });
+});
+
+describe("createScreenshotError / isScreenshotError", () => {
+  it("builds a fully-formed ScreenshotError", () => {
+    const err = createScreenshotError(
+      "empty_output",
+      "screenshot_capture",
+      "empty image",
+      "underlying detail",
+    );
+    expect(err).toBeInstanceOf(Error);
+    expect(err.name).toBe("ScreenshotError");
+    expect(err.screenshotErrorCode).toBe("empty_output");
+    expect(err.operation).toBe("screenshot_capture");
+    expect(err.details).toBe("underlying detail");
+    expect(isScreenshotError(err)).toBe(true);
+  });
+
+  it("rejects plain errors and non-errors from the type guard", () => {
+    expect(isScreenshotError(new Error("plain"))).toBe(false);
+    expect(isScreenshotError({ screenshotErrorCode: "timeout" })).toBe(false);
+    expect(isScreenshotError(null)).toBe(false);
+  });
+
+  it("covers every declared error code", () => {
+    const codes: ScreenshotErrorCode[] = [
+      "permission_denied",
+      "tool_missing",
+      "empty_output",
+      "timeout",
+      "capture_failed",
+    ];
+    for (const code of codes) {
+      const err = createScreenshotError(code, "op", `msg-${code}`);
+      expect(err.screenshotErrorCode).toBe(code);
+    }
+  });
+});

plugins/plugin-computeruse/src/__tests__/screenspot.test.ts

@@ -0,0 +1,77 @@
+/**
+ * ScreenSpot grounding harness (#9170 M14).
+ *
+ * Pure point-in-bbox scoring + the harness fold, exercised with synthetic
+ * samples and fake grounders (no dataset, no model).
+ */
+
+import { describe, expect, it } from "vitest";
+import {
+  pointInBbox,
+  type ScreenSpotSample,
+  scoreScreenSpot,
+} from "../parity/screenspot.js";
+
+describe("pointInBbox", () => {
+  it("is true inside and on the edges, false outside", () => {
+    expect(pointInBbox({ x: 50, y: 50 }, [0, 0, 100, 100])).toBe(true);
+    expect(pointInBbox({ x: 0, y: 0 }, [0, 0, 100, 100])).toBe(true);
+    expect(pointInBbox({ x: 100, y: 100 }, [0, 0, 100, 100])).toBe(true);
+    expect(pointInBbox({ x: 101, y: 50 }, [0, 0, 100, 100])).toBe(false);
+    expect(pointInBbox({ x: -1, y: 50 }, [0, 0, 100, 100])).toBe(false);
+  });
+});
+
+const SAMPLES: ScreenSpotSample[] = [
+  {
+    id: "icon-1",
+    instruction: "click settings",
+    bbox: [10, 10, 40, 40],
+    imageWidth: 800,
+    imageHeight: 600,
+    group: "icon",
+  },
+  {
+    id: "text-1",
+    instruction: "click Save",
+    bbox: [200, 100, 80, 24],
+    imageWidth: 800,
+    imageHeight: 600,
+    group: "text",
+  },
+];
+
+describe("scoreScreenSpot", () => {
+  it("scores 100% for a grounder that hits every bbox center", async () => {
+    const score = await scoreScreenSpot(SAMPLES, (s) => ({
+      x: s.bbox[0] + s.bbox[2] / 2,
+      y: s.bbox[1] + s.bbox[3] / 2,
+    }));
+    expect(score.accuracy).toBe(1);
+    expect(score.correct).toBe(2);
+    expect(score.byGroup.icon?.accuracy).toBe(1);
+    expect(score.byGroup.text?.accuracy).toBe(1);
+  });
+
+  it("scores 0% for a grounder that always misses", async () => {
+    const score = await scoreScreenSpot(SAMPLES, () => ({ x: 9999, y: 9999 }));
+    expect(score.accuracy).toBe(0);
+    expect(score.correct).toBe(0);
+  });
+
+  it("counts a null (abstain) prediction as wrong", async () => {
+    const score = await scoreScreenSpot(SAMPLES, (s) =>
+      s.id === "icon-1" ? { x: s.bbox[0] + 1, y: s.bbox[1] + 1 } : null,
+    );
+    expect(score.correct).toBe(1);
+    expect(score.total).toBe(2);
+    expect(score.accuracy).toBe(0.5);
+    expect(score.results.find((r) => r.id === "text-1")?.predicted).toBeNull();
+  });
+
+  it("handles an empty sample set", async () => {
+    const score = await scoreScreenSpot([], () => ({ x: 0, y: 0 }));
+    expect(score.total).toBe(0);
+    expect(score.accuracy).toBe(0);
+  });
+});

plugins/plugin-computeruse/src/__tests__/security-file-target.test.ts

@@ -0,0 +1,47 @@
+/**
+ * resolveSafeFileTarget path-security checks against real temp directories and
+ * symlinks. Deterministic; exercises the real fs.
+ */
+import fs from "node:fs/promises";
+import os from "node:os";
+import path from "node:path";
+import { afterEach, describe, expect, it } from "vitest";
+import { resolveSafeFileTarget } from "../platform/security.js";
+
+describe("resolveSafeFileTarget", () => {
+  const dirs: string[] = [];
+
+  afterEach(async () => {
+    await Promise.all(
+      dirs.splice(0).map((dir) => fs.rm(dir, { recursive: true, force: true })),
+    );
+  });
+
+  async function tempDir(): Promise<string> {
+    const dir = await fs.mkdtemp(path.join(os.tmpdir(), "eliza-cu-"));
+    dirs.push(dir);
+    return dir;
+  }
+
+  it("rejects symlink targets for writes", async () => {
+    const root = await tempDir();
+    const secret = path.join(root, "secret.txt");
+    const link = path.join(root, "link.txt");
+    await fs.writeFile(secret, "secret", "utf8");
+    await fs.symlink(secret, link);
+
+    const result = await resolveSafeFileTarget(link, "write");
+    expect(result.allowed).toBe(false);
+    expect(result.reason).toMatch(/symbolic link/i);
+  });
+
+  it("resolves regular files through realpath", async () => {
+    const root = await tempDir();
+    const file = path.join(root, "notes.txt");
+    await fs.writeFile(file, "hello", "utf8");
+
+    const result = await resolveSafeFileTarget(file, "read");
+    expect(result.allowed).toBe(true);
+    expect(result.resolvedPath).toBe(await fs.realpath(file));
+  });
+});

plugins/plugin-computeruse/src/__tests__/security-hardening.test.ts

@@ -0,0 +1,226 @@
+/**
+ * Security hardening tests cover file-path credential blocklists and child
+ * process environment sanitization across POSIX and Windows path semantics.
+ */
+import os from "node:os";
+import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
+import { ComputerUseApprovalManager } from "../approval-manager.js";
+import { sanitizeChildEnv, validateFilePath } from "../platform/security.js";
+
+// Captured at module load before any test fakes `process.platform`. POSIX
+// absolute paths (`/etc/shadow`, `/home/<user>/…`) are only meaningful when
+// `node:path` resolves with POSIX semantics; on a Windows host `path.win32`
+// rewrites `/etc/shadow` → `C:\etc\shadow`, so the POSIX assertions cannot be
+// exercised there. Those describe blocks skip on Windows and a dedicated
+// "(windows)" block asserts the real Win32 blocklist instead.
+const IS_WINDOWS_HOST = process.platform === "win32";
+
+describe("validateFilePath credential blocklist (cross-home)", () => {
+  it.skipIf(IS_WINDOWS_HOST)(
+    "blocks SSH/cred files under any POSIX user's home",
+    () => {
+      expect(validateFilePath("/root/.ssh/id_rsa", "read").allowed).toBe(false);
+      expect(
+        validateFilePath("/home/otheruser/.ssh/id_rsa", "read").allowed,
+      ).toBe(false);
+      expect(
+        validateFilePath("/home/otheruser/.aws/credentials", "read").allowed,
+      ).toBe(false);
+    },
+  );
+
+  it("blocks SSH/cred files under any Windows user's home", () => {
+    // Drive-letter home roots are normalized cross-platform, so these block on
+    // every host.
+    expect(
+      validateFilePath("C:\\Users\\Other\\.ssh\\id_rsa", "read").allowed,
+    ).toBe(false);
+    expect(
+      validateFilePath("C:\\Users\\Other\\.aws\\credentials", "read").allowed,
+    ).toBe(false);
+  });
+
+  it("blocks credential files under the current home for reads", () => {
+    const home = os.homedir().replace(/\\/g, "/");
+    expect(validateFilePath(`${home}/.aws/credentials`, "read").allowed).toBe(
+      false,
+    );
+    expect(validateFilePath(`${home}/.netrc`, "read").allowed).toBe(false);
+  });
+
+  it("allows benign files in a home directory", () => {
+    const benign = IS_WINDOWS_HOST
+      ? "C:\\Users\\Other\\notes.txt"
+      : "/home/otheruser/notes.txt";
+    expect(validateFilePath(benign, "read").allowed).toBe(true);
+  });
+});
+
+describe.skipIf(IS_WINDOWS_HOST)(
+  "validateFilePath system-dir read protection (unix)",
+  () => {
+    const original = process.platform;
+
+    beforeEach(() => {
+      Object.defineProperty(process, "platform", { value: "linux" });
+    });
+
+    afterEach(() => {
+      Object.defineProperty(process, "platform", { value: original });
+    });
+
+    it("blocks reads of /etc/shadow and other auth config", () => {
+      const result = validateFilePath("/etc/shadow", "read");
+      expect(result.allowed).toBe(false);
+      expect(result.reason).toMatch(/system auth config/i);
+    });
+
+    it("blocks reads of /proc, /sys, and /dev", () => {
+      expect(validateFilePath("/proc/1/environ", "read").allowed).toBe(false);
+      expect(validateFilePath("/sys/kernel/notes", "read").allowed).toBe(false);
+      expect(validateFilePath("/dev/sda", "read").allowed).toBe(false);
+    });
+
+    it("still allows reads outside system dirs", () => {
+      expect(validateFilePath("/tmp/output.txt", "read").allowed).toBe(true);
+    });
+  },
+);
+
+describe.skipIf(!IS_WINDOWS_HOST)(
+  "validateFilePath system-dir protection (windows)",
+  () => {
+    it("blocks writes/deletes into Windows system directories", () => {
+      const sysWrite = validateFilePath(
+        "C:\\Windows\\System32\\drivers\\etc\\hosts",
+        "write",
+      );
+      expect(sysWrite.allowed).toBe(false);
+      expect(sysWrite.reason).toMatch(/Windows system directory/i);
+
+      expect(
+        validateFilePath("C:\\Program Files\\app\\bin.exe", "delete").allowed,
+      ).toBe(false);
+      expect(
+        validateFilePath("C:\\ProgramData\\secret\\x.dat", "write").allowed,
+      ).toBe(false);
+    });
+
+    it("allows READS of files under Windows system dirs (logs/config)", () => {
+      // Reading config/log files under system dirs is a legitimate use case,
+      // so the Win32 system-dir patterns are write/delete-only.
+      expect(
+        validateFilePath("C:\\Windows\\System32\\drivers\\etc\\hosts", "read")
+          .allowed,
+      ).toBe(true);
+      expect(
+        validateFilePath("C:\\Program Files\\app\\readme.txt", "read").allowed,
+      ).toBe(true);
+    });
+
+    it("blocks Windows reserved device names for writes", () => {
+      expect(
+        validateFilePath("C:\\Users\\Administrator\\NUL", "write").allowed,
+      ).toBe(false);
+      expect(
+        validateFilePath("C:\\Users\\Administrator\\CON.txt", "write").allowed,
+      ).toBe(false);
+    });
+
+    it("blocks writing the drive root and UNC network paths", () => {
+      expect(validateFilePath("C:\\", "write").allowed).toBe(false);
+      expect(
+        validateFilePath("\\\\server\\share\\file.txt", "read").allowed,
+      ).toBe(false);
+    });
+
+    it("allows benign writes under the user profile", () => {
+      const home = os.homedir().replace(/\\/g, "/");
+      expect(
+        validateFilePath(`${home}/Documents/notes.txt`, "write").allowed,
+      ).toBe(true);
+    });
+  },
+);
+
+describe("sanitizeChildEnv provider-key stripping", () => {
+  const saved: Record<string, string | undefined> = {};
+  const keys = [
+    "ANTHROPIC_API_KEY",
+    "OPENAI_API_KEY",
+    "OPENROUTER_API_KEY",
+    "GROQ_API_KEY",
+    "XAI_API_KEY",
+    "DISCORD_API_TOKEN",
+    "TELEGRAM_BOT_TOKEN",
+    "AWS_SECRET_ACCESS_KEY",
+    "ELIZA_SECRET_KEY",
+    "SOME_HARMLESS_VALUE",
+  ];
+
+  beforeEach(() => {
+    for (const k of keys) {
+      saved[k] = process.env[k];
+    }
+    process.env.ANTHROPIC_API_KEY = "sk-ant-test";
+    process.env.OPENAI_API_KEY = "sk-test";
+    process.env.OPENROUTER_API_KEY = "or-test";
+    process.env.GROQ_API_KEY = "gsk-test";
+    process.env.XAI_API_KEY = "xai-test";
+    process.env.DISCORD_API_TOKEN = "discord-test";
+    process.env.TELEGRAM_BOT_TOKEN = "tg-test";
+    process.env.AWS_SECRET_ACCESS_KEY = "aws-test";
+    process.env.ELIZA_SECRET_KEY = "eliza-test";
+    process.env.SOME_HARMLESS_VALUE = "keep-me";
+  });
+
+  afterEach(() => {
+    for (const k of keys) {
+      if (saved[k] === undefined) {
+        delete process.env[k];
+      } else {
+        process.env[k] = saved[k];
+      }
+    }
+  });
+
+  it("strips provider/connector API keys from the child env", () => {
+    const env = sanitizeChildEnv();
+    expect(env.ANTHROPIC_API_KEY).toBeUndefined();
+    expect(env.OPENAI_API_KEY).toBeUndefined();
+    expect(env.OPENROUTER_API_KEY).toBeUndefined();
+    expect(env.GROQ_API_KEY).toBeUndefined();
+    expect(env.XAI_API_KEY).toBeUndefined();
+    expect(env.DISCORD_API_TOKEN).toBeUndefined();
+    expect(env.TELEGRAM_BOT_TOKEN).toBeUndefined();
+    expect(env.AWS_SECRET_ACCESS_KEY).toBeUndefined();
+    expect(env.ELIZA_SECRET_KEY).toBeUndefined();
+  });
+
+  it("keeps harmless env values", () => {
+    const env = sanitizeChildEnv();
+    expect(env.SOME_HARMLESS_VALUE).toBe("keep-me");
+  });
+});
+
+describe("ComputerUseApprovalManager default mode", () => {
+  it("defaults to smart_approve and requires approval for destructive verbs", () => {
+    // Force loadConfig to keep the default by failing the file read.
+    vi.spyOn(
+      ComputerUseApprovalManager.prototype as unknown as {
+        loadConfig: () => void;
+      },
+      "loadConfig",
+    ).mockImplementation(() => {});
+    const manager = new ComputerUseApprovalManager();
+    expect(manager.getMode()).toBe("smart_approve");
+    // Read-only safe command auto-approves.
+    expect(manager.shouldAutoApprove("screenshot")).toBe(true);
+    expect(manager.shouldAutoApprove("file_read")).toBe(true);
+    // Destructive verbs are NOT auto-approved by default.
+    expect(manager.shouldAutoApprove("execute_command")).toBe(false);
+    expect(manager.shouldAutoApprove("file_delete")).toBe(false);
+    expect(manager.shouldAutoApprove("file_write")).toBe(false);
+    vi.restoreAllMocks();
+  });
+});

plugins/plugin-computeruse/src/__tests__/service.real.test.ts

@@ -0,0 +1,392 @@
+/**
+ * Service lifecycle and integration tests for ComputerUseService.
+ *
+ * Tests service start/stop, capability detection, action dispatch,
+ * and the action history buffer.
+ */
+
+import type { IAgentRuntime } from "@elizaos/core";
+import { afterEach, beforeEach, describe, expect, it } from "vitest";
+import { assertScreenshotBase64NotBlank } from "../../test/helpers/screenshot-quality.ts";
+import { desktopMouseMove } from "../platform/desktop.js";
+import { currentPlatform } from "../platform/helpers.js";
+import { captureScreenshot } from "../platform/screenshot.js";
+import { ComputerUseService } from "../services/computer-use-service.js";
+
+type RawActionParams = { action: string };
+
+function executeRawDesktopAction(
+  service: ComputerUseService,
+  params: RawActionParams,
+): ReturnType<ComputerUseService["executeDesktopAction"]> {
+  const execute = service.executeDesktopAction as (
+    rawParams: RawActionParams,
+  ) => ReturnType<ComputerUseService["executeDesktopAction"]>;
+  return execute.call(service, params);
+}
+
+function executeRawWindowAction(
+  service: ComputerUseService,
+  params: RawActionParams,
+): ReturnType<ComputerUseService["executeWindowAction"]> {
+  const execute = service.executeWindowAction as (
+    rawParams: RawActionParams,
+  ) => ReturnType<ComputerUseService["executeWindowAction"]>;
+  return execute.call(service, params);
+}
+
+const os = currentPlatform();
+
+// Check if screenshot/desktop tools actually work (permissions may be missing)
+let hasScreenCapture = false;
+try {
+  await captureScreenshot();
+  hasScreenCapture = true;
+} catch {
+  // permissions not granted
+}
+
+let hasDesktopControl = false;
+try {
+  desktopMouseMove(0, 0);
+  hasDesktopControl = true;
+} catch {
+  // permissions not granted or tools missing
+}
+
+function createMockRuntime(
+  settings: Record<string, string> = {},
+): IAgentRuntime {
+  const merged: Record<string, string> = {
+    COMPUTER_USE_APPROVAL_MODE: "full_control",
+    ...settings,
+  };
+  return {
+    character: {},
+    getSetting(key: string) {
+      return merged[key] ?? undefined;
+    },
+    getService() {
+      return null;
+    },
+  } as IAgentRuntime;
+}
+
+function skipIfAccessibilityPermissionMissing(
+  skip: (message?: string) => void,
+  result: {
+    permissionDenied?: boolean;
+    permissionType?: string;
+    message?: string;
+    error?: string;
+  },
+): void {
+  if (result.permissionDenied && result.permissionType === "accessibility") {
+    skip(
+      result.message ?? result.error ?? "Accessibility permission is missing",
+    );
+  }
+}
+
+describe("ComputerUseService lifecycle", () => {
+  let service: ComputerUseService;
+
+  beforeEach(async () => {
+    const runtime = createMockRuntime();
+    service = (await ComputerUseService.start(runtime)) as ComputerUseService;
+  });
+
+  afterEach(async () => {
+    if (service) {
+      await service.stop();
+    }
+  });
+
+  it("starts and exposes serviceType", () => {
+    expect(ComputerUseService.serviceType).toBe("computeruse");
+  });
+
+  it("has a capabilityDescription", () => {
+    expect(service.capabilityDescription).toBeDefined();
+    expect(typeof service.capabilityDescription).toBe("string");
+    expect(service.capabilityDescription.length).toBeGreaterThan(0);
+  });
+
+  it("detects platform capabilities", () => {
+    const caps = service.getCapabilities();
+
+    expect(caps).toHaveProperty("screenshot");
+    expect(caps).toHaveProperty("computerUse");
+    expect(caps).toHaveProperty("windowList");
+    expect(caps).toHaveProperty("browser");
+
+    // Each capability has available and tool
+    for (const key of [
+      "screenshot",
+      "computerUse",
+      "windowList",
+      "browser",
+    ] as const) {
+      expect(typeof caps[key].available).toBe("boolean");
+      expect(typeof caps[key].tool).toBe("string");
+    }
+
+    // On macOS, screenshot should always be available (screencapture is built-in)
+    if (os === "darwin") {
+      expect(caps.screenshot.available).toBe(true);
+      expect(caps.screenshot.tool).toContain("screencapture");
+    }
+
+    // On Windows, screenshot and computer use should be available (PowerShell)
+    if (os === "win32") {
+      expect(caps.screenshot.available).toBe(true);
+      expect(caps.computerUse.available).toBe(true);
+    }
+  });
+
+  it("returns valid screen dimensions", () => {
+    const size = service.getScreenDimensions();
+
+    expect(size).toHaveProperty("width");
+    expect(size).toHaveProperty("height");
+    expect(size.width).toBeGreaterThanOrEqual(640);
+    expect(size.height).toBeGreaterThanOrEqual(480);
+  });
+
+  it("starts with empty action history", () => {
+    const history = service.getRecentActions();
+    expect(Array.isArray(history)).toBe(true);
+    expect(history.length).toBe(0);
+  });
+
+  it("stops cleanly", async () => {
+    await expect(service.stop()).resolves.toBeUndefined();
+  });
+});
+
+describe("ComputerUseService config", () => {
+  it("uses default config when no settings provided", async () => {
+    const runtime = createMockRuntime();
+    const svc = (await ComputerUseService.start(runtime)) as ComputerUseService;
+
+    // Defaults: screenshotAfterAction=true, actionTimeoutMs=10000
+    // We can verify by executing a screenshot action and checking it works
+    const result = await svc.executeDesktopAction({ action: "screenshot" });
+    // Screenshot should be attempted (may fail on CI but shouldn't error on config)
+    expect(result).toHaveProperty("success");
+
+    await svc.stop();
+  });
+
+  it("respects COMPUTER_USE_SCREENSHOT_AFTER_ACTION=false", async () => {
+    const runtime = createMockRuntime({
+      COMPUTER_USE_SCREENSHOT_AFTER_ACTION: "false",
+    });
+    const svc = (await ComputerUseService.start(runtime)) as ComputerUseService;
+
+    // When disabled, non-screenshot desktop actions should not return a screenshot
+    // We can only truly test this with a real desktop action, but we can verify the service starts
+    expect(svc.getCapabilities()).toBeDefined();
+
+    await svc.stop();
+  });
+
+  it("respects COMPUTER_USE_ACTION_TIMEOUT_MS setting", async () => {
+    const runtime = createMockRuntime({
+      COMPUTER_USE_ACTION_TIMEOUT_MS: "5000",
+    });
+    const svc = (await ComputerUseService.start(runtime)) as ComputerUseService;
+
+    // Service should start without error with custom timeout
+    expect(svc.getCapabilities()).toBeDefined();
+
+    await svc.stop();
+  });
+});
+
+// Tests that require working desktop control
+const describeIfDesktop = hasDesktopControl ? describe : describe.skip;
+
+describeIfDesktop("ComputerUseService desktop actions (real)", () => {
+  let service: ComputerUseService;
+
+  beforeEach(async () => {
+    const runtime = createMockRuntime({
+      COMPUTER_USE_SCREENSHOT_AFTER_ACTION: "false", // don't capture after every action in tests
+    });
+    service = (await ComputerUseService.start(runtime)) as ComputerUseService;
+  });
+
+  afterEach(async () => {
+    if (service) await service.stop();
+  });
+
+  it("executes screenshot action", async () => {
+    const result = await service.executeDesktopAction({ action: "screenshot" });
+    // Even if screenshot fails (permissions), the action should not crash
+    expect(result).toHaveProperty("success");
+    if (hasScreenCapture) {
+      expect(result.success).toBe(true);
+      assertScreenshotBase64NotBlank(
+        result.screenshot,
+        "ComputerUseService screenshot action",
+      );
+    }
+  });
+
+  it("executes mouse_move action", async ({ skip }) => {
+    const result = await service.executeDesktopAction({
+      action: "mouse_move",
+      coordinate: [200, 200],
+    });
+
+    skipIfAccessibilityPermissionMissing(skip, result);
+    expect(result.success).toBe(true);
+  });
+
+  it("executes click action", async ({ skip }) => {
+    const result = await service.executeDesktopAction({
+      action: "click",
+      coordinate: [200, 200],
+    });
+
+    skipIfAccessibilityPermissionMissing(skip, result);
+    expect(result.success).toBe(true);
+  });
+
+  it("executes key action", async ({ skip }) => {
+    const result = await service.executeDesktopAction({
+      action: "key",
+      key: "Escape",
+    });
+
+    skipIfAccessibilityPermissionMissing(skip, result);
+    expect(result.success).toBe(true);
+  });
+
+  it("executes key_combo action", async ({ skip }) => {
+    const result = await service.executeDesktopAction({
+      action: "key_combo",
+      key: "shift+Escape",
+    });
+
+    skipIfAccessibilityPermissionMissing(skip, result);
+    expect(result.success).toBe(true);
+  });
+
+  it("executes scroll action", async ({ skip }) => {
+    const result = await service.executeDesktopAction({
+      action: "scroll",
+      coordinate: [400, 400],
+      scrollDirection: "down",
+      scrollAmount: 2,
+    });
+
+    skipIfAccessibilityPermissionMissing(skip, result);
+    expect(result.success).toBe(true);
+  });
+
+  it("records actions in history regardless of success", async () => {
+    await service.executeDesktopAction({ action: "screenshot" });
+    await service.executeDesktopAction({
+      action: "mouse_move",
+      coordinate: [100, 100],
+    });
+
+    const history = service.getRecentActions();
+    expect(history.length).toBe(2);
+    expect(history[0].action).toBe("screenshot");
+    expect(history[1].action).toBe("mouse_move");
+    // Both should have recorded (success or failure depends on platform permissions)
+    expect(typeof history[0].success).toBe("boolean");
+    expect(typeof history[1].success).toBe("boolean");
+  });
+
+  it("caps history at max (10)", async () => {
+    for (let i = 0; i < 15; i++) {
+      await service.executeDesktopAction({
+        action: "click",
+      });
+    }
+
+    const history = service.getRecentActions();
+    expect(history.length).toBe(10);
+  }, 15000);
+
+  it("returns error for missing coordinate", async () => {
+    const result = await service.executeDesktopAction({ action: "click" });
+
+    expect(result.success).toBe(false);
+    expect(result.error).toContain("coordinate");
+  });
+
+  it("returns error for missing text on type action", async () => {
+    const result = await service.executeDesktopAction({ action: "type" });
+
+    expect(result.success).toBe(false);
+    expect(result.error).toContain("text is required");
+  });
+
+  it("returns error for missing key on key action", async () => {
+    const result = await service.executeDesktopAction({ action: "key" });
+
+    expect(result.success).toBe(false);
+    expect(result.error).toContain("key is required");
+  });
+
+  it("returns error for unknown action", async () => {
+    const result = await executeRawDesktopAction(service, {
+      action: "nonexistent",
+    });
+
+    expect(result.success).toBe(false);
+    expect(result.error).toContain("Unknown desktop action");
+  });
+
+  it("returns error for drag without startCoordinate", async () => {
+    const result = await service.executeDesktopAction({
+      action: "drag",
+      coordinate: [100, 100],
+    });
+
+    expect(result.success).toBe(false);
+    expect(result.error).toContain("startCoordinate");
+  });
+});
+
+describe("ComputerUseService window actions (real)", () => {
+  let service: ComputerUseService;
+
+  beforeEach(async () => {
+    service = (await ComputerUseService.start(
+      createMockRuntime(),
+    )) as ComputerUseService;
+  });
+
+  afterEach(async () => {
+    if (service) await service.stop();
+  });
+
+  it("lists windows", async () => {
+    const result = await service.executeWindowAction({ action: "list" });
+
+    expect(result.success).toBe(true);
+    expect(Array.isArray(result.windows)).toBe(true);
+  }, 15000);
+
+  it("returns error for focus without windowId", async () => {
+    const result = await service.executeWindowAction({ action: "focus" });
+
+    expect(result.success).toBe(false);
+    expect(result.error).toContain("windowId");
+  });
+
+  it("returns error for unknown window action", async () => {
+    const result = await executeRawWindowAction(service, {
+      action: "nonexistent",
+    });
+
+    expect(result.success).toBe(false);
+    expect(result.error).toContain("Unknown window action");
+  });
+});

plugins/plugin-computeruse/src/__tests__/set-of-marks-seam.test.ts

@@ -0,0 +1,73 @@
+/**
+ * Set-of-Marks provider seam (#9170 M9).
+ *
+ * detect_elements consumes whatever Set-of-Marks provider plugin-vision
+ * registers here. The seam itself is a last-call-wins registration slot — same
+ * contract as the CoordOcrProvider seam — so this guards register/get/unregister
+ * and the shape the consumer relies on.
+ */
+
+import { afterEach, describe, expect, it } from "vitest";
+import {
+  getSetOfMarksProvider,
+  registerSetOfMarksProvider,
+  type SetOfMarksProvider,
+} from "../mobile/ocr-provider.js";
+
+function fakeProvider(name = "fake-som"): SetOfMarksProvider {
+  return {
+    name,
+    async describe(input) {
+      return {
+        marks: [
+          {
+            index: 1,
+            bbox: [input.sourceX + 10, input.sourceY + 10, 40, 40],
+            center: [input.sourceX + 30, input.sourceY + 30],
+            source: "icon",
+            label: "button",
+            score: 0.9,
+          },
+        ],
+        overlayPngBase64: input.renderOverlay ? "ZmFrZQ==" : undefined,
+      };
+    },
+  };
+}
+
+describe("Set-of-Marks provider seam", () => {
+  afterEach(() => registerSetOfMarksProvider(null));
+
+  it("is empty until a provider registers", () => {
+    registerSetOfMarksProvider(null);
+    expect(getSetOfMarksProvider()).toBeNull();
+  });
+
+  it("returns the registered provider (last-call-wins)", () => {
+    registerSetOfMarksProvider(fakeProvider("first"));
+    registerSetOfMarksProvider(fakeProvider("second"));
+    expect(getSetOfMarksProvider()?.name).toBe("second");
+  });
+
+  it("unregisters with null", () => {
+    registerSetOfMarksProvider(fakeProvider());
+    expect(getSetOfMarksProvider()).not.toBeNull();
+    registerSetOfMarksProvider(null);
+    expect(getSetOfMarksProvider()).toBeNull();
+  });
+
+  it("describe yields numbered marks + a center click target", async () => {
+    registerSetOfMarksProvider(fakeProvider());
+    const provider = getSetOfMarksProvider();
+    const result = await provider?.describe({
+      displayId: "0",
+      sourceX: 100,
+      sourceY: 200,
+      pngBytes: new Uint8Array([1]),
+      renderOverlay: true,
+    });
+    expect(result.marks[0]?.index).toBe(1);
+    expect(result.marks[0]?.center).toEqual([130, 230]);
+    expect(result.overlayPngBase64).toBe("ZmFrZQ==");
+  });
+});

plugins/plugin-computeruse/src/__tests__/set-value.test.ts

@@ -0,0 +1,87 @@
+/**
+ * set_value (a11y element value write) parity (#9170 — trycua/cua `set_value`).
+ *
+ * Surface + driver-seam + per-OS shape assertions in the DEFAULT lane (runs on
+ * Windows/Linux/macOS/AOSP-Node). The real end-to-end actuation (UIAutomation
+ * ValuePattern on a live control) runs in the interactive real-driver lane —
+ * the win32 ValuePattern path needs a UIA-capable desktop, and the universal
+ * fallback is composed of the already-real-tested click/key-combo/type verbs.
+ */
+
+import { readFileSync } from "node:fs";
+import { dirname, join } from "node:path";
+import { fileURLToPath } from "node:url";
+import { describe, expect, it } from "vitest";
+import { computerUsePlugin } from "../index.js";
+import * as desktop from "../platform/desktop.js";
+import * as driver from "../platform/driver.js";
+
+const actionNames = (computerUsePlugin.actions ?? []).map((a) => a.name);
+const driverSrc = readFileSync(
+  join(dirname(fileURLToPath(import.meta.url)), "..", "platform", "driver.ts"),
+  "utf8",
+);
+const desktopSrc = readFileSync(
+  join(dirname(fileURLToPath(import.meta.url)), "..", "platform", "desktop.ts"),
+  "utf8",
+);
+
+describe("set_value surface", () => {
+  it("promotes set_value under COMPUTER_USE", () => {
+    expect(actionNames, `actions: ${actionNames.join(", ")}`).toContain(
+      "COMPUTER_USE_SET_VALUE",
+    );
+  });
+
+  it("set_value is in the COMPUTER_USE action enum", () => {
+    const cu = (computerUsePlugin.actions ?? []).find(
+      (a) => a.name === "COMPUTER_USE",
+    ) as
+      | { parameters?: Array<{ name: string; schema?: { enum?: string[] } }> }
+      | undefined;
+    const en =
+      cu?.parameters?.find((p) => p.name === "action")?.schema?.enum ?? [];
+    expect(en).toContain("set_value");
+  });
+});
+
+describe("set_value driver seam", () => {
+  it("exports driverSetValue + the win32 ValuePattern helper", () => {
+    expect(typeof driver.driverSetValue).toBe("function");
+    expect(typeof desktop.win32TrySetValueByPattern).toBe("function");
+  });
+
+  it("win32TrySetValueByPattern no-ops to false off win32 (pure guard)", () => {
+    // On a non-win32 test runner this returns false without spawning anything;
+    // on win32 it would attempt UIAutomation. Either way it must be boolean.
+    expect(typeof desktop.win32TrySetValueByPattern(10, 10, "x")).toBe(
+      "boolean",
+    );
+  });
+});
+
+describe("set_value implementation shape", () => {
+  it("driverSetValue tries the win32 ValuePattern fast-path then falls back to click→select-all→type", () => {
+    const start = driverSrc.indexOf("export async function driverSetValue");
+    expect(start).toBeGreaterThan(-1);
+    const body = driverSrc.slice(start, start + 700);
+    expect(body).toContain("win32TrySetValueByPattern");
+    expect(body).toContain("driverClick");
+    expect(body).toContain("driverKeyCombo");
+    expect(body).toContain("driverType");
+    // macOS uses cmd+a, others ctrl+a.
+    expect(body).toContain('"cmd+a"');
+    expect(body).toContain('"ctrl+a"');
+  });
+
+  it("the win32 fast-path uses UIAutomation ValuePattern (not synthesized keys)", () => {
+    const start = desktopSrc.indexOf(
+      "export function win32TrySetValueByPattern",
+    );
+    expect(start).toBeGreaterThan(-1);
+    const body = desktopSrc.slice(start, start + 1200);
+    expect(body).toContain("UIAutomationClient");
+    expect(body).toContain("ValuePattern");
+    expect(body).toContain("FromPoint");
+  });
+});

plugins/plugin-computeruse/src/__tests__/terminal.real.test.ts

@@ -0,0 +1,154 @@
+/**
+ * Real integration tests for terminal command execution.
+ */
+import { mkdtempSync, realpathSync, rmSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { basename, join } from "node:path";
+import { describe, expect, it } from "vitest";
+import {
+  checkDangerousCommand,
+  sanitizeChildEnv,
+} from "../platform/security.js";
+import {
+  closeTerminal,
+  connectTerminal,
+  executeTerminal,
+} from "../platform/terminal.js";
+
+/**
+ * A real, guaranteed-existing temp directory whose canonical path we can match
+ * against shell output. `/tmp` is unsafe as a cwd on Windows — it resolves to
+ * `C:\tmp`, which need not exist on a CI runner, so `execFile` would ENOENT
+ * before the command ever runs. macOS resolves the temp dir through `/private`,
+ * so we canonicalize with `realpathSync` and assert on the unique leaf name.
+ */
+function makeRealCwd(): string {
+  return realpathSync(mkdtempSync(join(tmpdir(), "cua-cwd-")));
+}
+
+describe("terminal execution (real)", () => {
+  it("executes a simple command", async () => {
+    const result = await executeTerminal({ command: "echo hello" });
+    expect(result.success).toBe(true);
+    expect(result.output).toContain("hello");
+    expect(result.exitCode).toBe(0);
+  });
+
+  it("captures stderr on failure", async () => {
+    const result = await executeTerminal({
+      command: "ls /nonexistent_path_xyz",
+    });
+    expect(result.success).toBe(false);
+    expect(result.exitCode).not.toBe(0);
+  });
+
+  it("respects cwd", async () => {
+    const dir = makeRealCwd();
+    try {
+      const result = await executeTerminal({ command: "pwd", cwd: dir });
+      expect(result.success).toBe(true);
+      // `pwd` prints the path with backslashes on Windows (PowerShell
+      // Get-Location); normalize separators and match on the unique leaf so the
+      // assertion holds on Windows/macOS/Linux alike.
+      const leaf = basename(dir);
+      expect(result.output.replace(/\\/g, "/")).toContain(leaf);
+    } finally {
+      rmSync(dir, { recursive: true, force: true });
+    }
+  });
+
+  it("creates and uses terminal sessions", async () => {
+    const dir = makeRealCwd();
+    try {
+      const session = await connectTerminal(dir);
+      expect(session.success).toBe(true);
+      expect(session.sessionId).toBeDefined();
+
+      const exec = await executeTerminal({
+        command: "echo session-test",
+        sessionId: session.sessionId,
+      });
+      expect(exec.success).toBe(true);
+      expect(exec.output).toContain("session-test");
+
+      const close = await closeTerminal(session.sessionId);
+      expect(close.success).toBe(true);
+    } finally {
+      rmSync(dir, { recursive: true, force: true });
+    }
+  });
+
+  it("handles command timeout", async () => {
+    const result = await executeTerminal({
+      command: "sleep 10",
+      timeoutSeconds: 1,
+    });
+    expect(result.success).toBe(false);
+  }, 15000);
+
+  it("truncates long output", async () => {
+    // Generate a lot of output
+    const result = await executeTerminal({
+      command:
+        'for i in $(seq 1 10000); do echo "line $i padding text to make it longer"; done',
+    });
+    // Output should be present but may be truncated
+    expect(result.output).toBeDefined();
+    if (result.output && result.output.length > 5000) {
+      // The terminal module truncates to 5000
+      expect(result.output.length).toBeLessThanOrEqual(5100); // some slack for truncation message
+    }
+  }, 15000);
+});
+
+describe("dangerous command detection", () => {
+  it("blocks rm -rf /", () => {
+    const result = checkDangerousCommand("rm -rf /");
+    expect(result.blocked).toBe(true);
+  });
+
+  it("blocks fork bombs", () => {
+    const result = checkDangerousCommand(":(){ :|:& };:");
+    expect(result.blocked).toBe(true);
+  });
+
+  it("blocks mkfs", () => {
+    const result = checkDangerousCommand("mkfs.ext4 /dev/sda1");
+    expect(result.blocked).toBe(true);
+  });
+
+  it("blocks dd to disk", () => {
+    const result = checkDangerousCommand("dd if=/dev/zero of=/dev/sda");
+    expect(result.blocked).toBe(true);
+  });
+
+  it("allows safe commands", () => {
+    expect(checkDangerousCommand("ls -la").blocked).toBe(false);
+    expect(checkDangerousCommand("echo hello").blocked).toBe(false);
+    expect(checkDangerousCommand("git status").blocked).toBe(false);
+    expect(checkDangerousCommand("cat /etc/hostname").blocked).toBe(false);
+  });
+});
+
+describe("env sanitization", () => {
+  it("strips known sensitive variables", () => {
+    const original = process.env.INTERNAL_API_KEY;
+    process.env.INTERNAL_API_KEY = "secret123";
+
+    const sanitized = sanitizeChildEnv();
+    expect(sanitized.INTERNAL_API_KEY).toBeUndefined();
+
+    // Restore
+    if (original === undefined) {
+      delete process.env.INTERNAL_API_KEY;
+    } else {
+      process.env.INTERNAL_API_KEY = original;
+    }
+  });
+
+  it("preserves safe variables", () => {
+    const sanitized = sanitizeChildEnv();
+    expect(sanitized.PATH).toBeDefined();
+    expect(sanitized.HOME).toBeDefined();
+  });
+});

plugins/plugin-computeruse/src/__tests__/trajectory.test.ts

@@ -0,0 +1,62 @@
+/**
+ * Trajectory recording for the computer-use agent loop, with the screenshot and
+ * a11y platform modules mocked. Deterministic unit test.
+ */
+import { beforeEach, describe, expect, it, vi } from "vitest";
+
+vi.mock("../platform/screenshot.js", () => ({
+  captureScreenshot: vi.fn(() => Buffer.from("screen")),
+}));
+
+vi.mock("../platform/a11y.js", () => ({
+  extractA11yTree: vi.fn(() => "window tree"),
+  isA11yAvailable: vi.fn(() => true),
+}));
+
+const { OSWorldAdapter } = await import("../osworld/adapter.js");
+
+describe("OSWorld trajectory capture", () => {
+  beforeEach(() => {
+    vi.clearAllMocks();
+  });
+
+  it("records each step with action, observation, and timestamp", async () => {
+    const service = {
+      executeDesktopAction: vi.fn(async () => ({
+        success: true,
+        message: "done",
+      })),
+    };
+    const adapter = new OSWorldAdapter(service as never, {
+      actionSpace: "computer_13",
+      observationType: "screenshot_a11y_tree",
+      includeA11yTree: true,
+      screenshotDelayMs: 0,
+      maxTrajectoryLength: 5,
+    });
+
+    await adapter.step({ action_type: "MOVE_TO", x: 12, y: 34 }, "move");
+    await adapter.step("WAIT", "wait");
+
+    expect(service.executeDesktopAction).toHaveBeenCalledWith({
+      action: "mouse_move",
+      coordinate: [12, 34],
+    });
+
+    const trajectory = adapter.getTrajectory();
+    expect(trajectory).toHaveLength(2);
+    expect(trajectory[0]).toMatchObject({
+      action: { action_type: "MOVE_TO", x: 12, y: 34 },
+      observation: {
+        screenshot: Buffer.from("screen").toString("base64"),
+        accessibility_tree: "window tree",
+        instruction: "move",
+      },
+    });
+    expect(typeof trajectory[0]?.timestamp).toBe("number");
+    expect(trajectory[1]).toMatchObject({
+      action: "WAIT",
+      observation: { instruction: "wait" },
+    });
+  });
+});

plugins/plugin-computeruse/src/__tests__/use-computer-action.test.ts

@@ -0,0 +1,184 @@
+/**
+ * COMPUTER_USE action handler over a mocked ComputerUseService: param resolution,
+ * approval snapshots, and result shaping. Deterministic.
+ */
+import type { HandlerCallback, IAgentRuntime, Memory } from "@elizaos/core";
+import { describe, expect, it, vi } from "vitest";
+import { useComputerAction } from "../actions/use-computer.js";
+import type { ComputerUseService } from "../services/computer-use-service.js";
+import type { ApprovalSnapshot, PendingApproval } from "../types.js";
+
+const message = (content: Memory["content"]): Memory =>
+  ({
+    id: "00000000-0000-0000-0000-000000000001",
+    entityId: "00000000-0000-0000-0000-000000000002",
+    agentId: "00000000-0000-0000-0000-000000000003",
+    roomId: "00000000-0000-0000-0000-000000000004",
+    content,
+  }) as Memory;
+
+function runtimeWithService(
+  service: Partial<ComputerUseService>,
+): IAgentRuntime {
+  return {
+    getService: (name: string) => (name === "computeruse" ? service : null),
+  } as unknown as IAgentRuntime;
+}
+
+describe("COMPUTER_USE action approvals", () => {
+  it("relays pending approval requests as chat choice buttons", async () => {
+    const pending: PendingApproval = {
+      id: "approval_123_abc",
+      command: "computer_use_click",
+      parameters: { action: "click" },
+      requestedAt: "2026-06-22T14:00:00.000Z",
+    };
+    let approvalListener: ((snapshot: ApprovalSnapshot) => void) | null = null;
+    const unsubscribe = vi.fn();
+    const service = {
+      getApprovalSnapshot: () => ({
+        mode: "approve_all",
+        pendingCount: 0,
+        pendingApprovals: [],
+      }),
+      subscribeApprovals: (listener: (snapshot: ApprovalSnapshot) => void) => {
+        approvalListener = listener;
+        return unsubscribe;
+      },
+      executeDesktopAction: vi.fn(async () => {
+        approvalListener?.({
+          mode: "approve_all",
+          pendingCount: 1,
+          pendingApprovals: [pending],
+        });
+        return { success: true, message: "clicked" };
+      }),
+    } as Partial<ComputerUseService>;
+    const callback = vi.fn(async () => []) satisfies HandlerCallback;
+    const handler = useComputerAction.handler;
+    if (!handler) {
+      throw new Error("COMPUTER_USE handler missing");
+    }
+
+    const result = await handler(
+      runtimeWithService(service),
+      message({ action: "click" }),
+      undefined,
+      undefined,
+      callback,
+    );
+
+    expect(result?.success).toBe(true);
+    expect(unsubscribe).toHaveBeenCalledTimes(1);
+    const approvalCall = callback.mock.calls.find(
+      ([, actionName]) => actionName === "COMPUTER_USE_APPROVAL",
+    );
+    expect(approvalCall?.[0]).toMatchObject({
+      source: "computeruse_approval",
+      text: expect.stringContaining("[CHOICE:computeruse-approval"),
+    });
+    expect(approvalCall?.[0].text).toContain(
+      "cua:approval_123_abc:approve=Approve",
+    );
+    expect(approvalCall?.[0].text).toContain("cua:approval_123_abc:deny=Deny");
+  });
+
+  it("supports typed legacy approve/deny text for manual fallback", async () => {
+    const resolveApproval = vi.fn(() => ({
+      id: "approval_123_abc",
+      command: "computer_use_click",
+      approved: true,
+      cancelled: false,
+      mode: "approve_all",
+      requestedAt: "2026-06-22T14:00:00.000Z",
+      resolvedAt: "2026-06-22T14:00:01.000Z",
+    }));
+    const service = { resolveApproval } as Partial<ComputerUseService>;
+    const callback = vi.fn(async () => []) satisfies HandlerCallback;
+    const handler = useComputerAction.handler;
+    if (!handler) {
+      throw new Error("COMPUTER_USE handler missing");
+    }
+
+    const result = await handler(
+      runtimeWithService(service),
+      message({ text: "approve:approval_123_abc" }),
+      undefined,
+      undefined,
+      callback,
+    );
+
+    expect(resolveApproval).toHaveBeenCalledWith(
+      "approval_123_abc",
+      true,
+      "Resolved from chat button (approve)",
+    );
+    expect(result?.success).toBe(true);
+  });
+
+  it("resolves approve/deny callbacks returned from chat buttons", async () => {
+    const resolveApproval = vi.fn(() => ({
+      id: "approval_123_abc",
+      command: "computer_use_click",
+      approved: true,
+      cancelled: false,
+      mode: "approve_all",
+      requestedAt: "2026-06-22T14:00:00.000Z",
+      resolvedAt: "2026-06-22T14:00:01.000Z",
+    }));
+    const service = { resolveApproval } as Partial<ComputerUseService>;
+    const callback = vi.fn(async () => []) satisfies HandlerCallback;
+    const handler = useComputerAction.handler;
+    if (!handler) {
+      throw new Error("COMPUTER_USE handler missing");
+    }
+
+    const result = await handler(
+      runtimeWithService(service),
+      message({ text: "cua:approval_123_abc:approve" }),
+      undefined,
+      undefined,
+      callback,
+    );
+
+    expect(resolveApproval).toHaveBeenCalledWith(
+      "approval_123_abc",
+      true,
+      "Resolved from chat button (approve)",
+    );
+    expect(result?.success).toBe(true);
+    expect(result?.text).toBe(
+      "Computer-use approval approval_123_abc approved.",
+    );
+    expect(callback).toHaveBeenCalledWith(
+      { text: "Computer-use approval approval_123_abc approved." },
+      "COMPUTER_USE",
+    );
+  });
+
+  it("rejects owner-scoped callbacks from another Telegram user", async () => {
+    const service = {
+      resolveApproval: vi.fn(),
+    } as Partial<ComputerUseService>;
+    const callback = vi.fn(async () => []) satisfies HandlerCallback;
+    const handler = useComputerAction.handler;
+    if (!handler) {
+      throw new Error("COMPUTER_USE handler missing");
+    }
+
+    const result = await handler(
+      runtimeWithService(service),
+      {
+        ...message({ text: "cua:approval_123_abc:approve:u42" }),
+        metadata: { telegramUserId: "7" },
+      } as Memory,
+      undefined,
+      undefined,
+      callback,
+    );
+
+    expect(service.resolveApproval).not.toHaveBeenCalled();
+    expect(result?.success).toBe(false);
+    expect(result?.error).toContain("does not belong to this user");
+  });
+});