chore: import upstream snapshot with attribution
FreeBSD Smoke / FreeBSD Smoke (x86_64) (push) Has been cancelled
CI / Quality Guardrails (push) Has been cancelled
CI / Build & Test (macos-latest) (push) Has been cancelled
CI / Build & Test (ubuntu-latest) (push) Has been cancelled
CI / Build & Test (windows-latest) (push) Has been cancelled
CI / Format (push) Has been cancelled
CI / PowerShell Syntax (push) Has been cancelled
CI / Windows Cross-Target Check (Linux) (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 13:10:34 +08:00
commit a789495a98
1551 changed files with 718128 additions and 0 deletions
@@ -0,0 +1,174 @@
# Cross-Device Worktree and Build Sync (Proposed)
Status: Proposed (design only, not implemented)
## Problem
A user develops jcode (and other repos) on multiple machines, e.g. a MacBook
(aarch64-darwin) and a Linux laptop. On a single machine, `selfdev build` +
reload means every local jcode instance runs the new version, and multiple
agents can share one worktree because the server mediates edits and tracks
conflicts (`FileTouchService`, `file_activity.rs`).
Split across machines, this breaks:
- A `selfdev build` on the Linux laptop does not update the MacBook's jcode,
and vice versa.
- There is no shared worktree, so changes made on one machine are invisible
to agents/sessions on the other until manually pushed/pulled.
Goal: make multiple machines behave like one logical worktree + one logical
build channel, the same way multiple agents already share one worktree on a
single machine.
## Existing building blocks (verified in code)
| Building block | Where | Why it matters |
| --- | --- | --- |
| Arch-independent version identity | `jcode-build-support/src/source_state.rs` (`SourceState::version_label`, `fingerprint`) | Fingerprint hashes full commit hash + status + `diff --binary HEAD` + untracked contents. Identical source trees on two machines produce the same label, even though binaries differ per-arch. |
| Auto-reload on newer binary | `server/util.rs::server_has_newer_binary`, `reload_exec_target` | Mtime-based channel scan. A peer-triggered local build that publishes to `builds/current` triggers the existing reload flow with zero changes. |
| Pull → build → install → exec | `session_rebuild.rs` | Already implements the receiving side's pipeline shape. |
| Network door, same protocol | `jcode-base/src/gateway.rs` (WS + plain HTTP on :7643) | Remote clients speak the identical newline-JSON protocol as Unix-socket clients. Plain HTTP handler (`/pair`, `/health`) is a natural place for `/peer/*` endpoints. |
| NAT-friendly device event bus | `server/jade_relay.rs` | Device IDs, heartbeats, long-polled command events. Works when machines cannot reach each other directly. |
| Server-side tool execution | server architecture | Tools (bash, edit) run in the server process; a remote client attaching to another machine's server gets the full multi-agent-one-worktree behavior, including conflict warnings. |
| Remote build precedent | `scripts/remote_build.sh` | rsync + ssh + sync-back pattern. |
### Known gap: repo identity across machines
`repo_scope_key` / `worktree_scope_key` hash the *local canonical path* of the
git common dir / worktree. These will never match across machines
(`/Users/jeremy/...` vs `/home/jeremy/...`). Cross-device features must key
repos by something portable: normalized origin URL, or an explicit repo name
in config (`[sync] repo_id = "jcode"`), falling back to origin URL hash.
## Design tensions
1. **Binaries cannot be shared** across darwin-aarch64 and linux-x86_64.
"Same version everywhere" must mean *same source state, built per machine*,
with `version_label` as the cross-machine equality check.
2. **Not all writes are server-mediated.** Tool edits flow through the server,
but `bash`, editors, and `cargo` mutate the worktree invisibly. Cross-device
sync therefore needs a byte-level capture mechanism (git plumbing snapshot
and/or fs watcher), not just tool-event forwarding.
3. **Laptops go offline.** Pure live-sync (mutagen/syncthing style) has poor
conflict semantics. Git-based convergence (per-device sync refs, merges)
handles offline divergence honestly.
## Mechanism: shipping a worktree state without touching HEAD
To capture a possibly-dirty worktree atomically without moving the user's
HEAD or index:
```
GIT_INDEX_FILE=$tmp git add -A # tracked + untracked into temp index
tree=$(GIT_INDEX_FILE=$tmp git write-tree)
commit=$(git commit-tree $tree -p HEAD -m "jcode sync: <device> <fingerprint>")
git push origin $commit:refs/jcode/sync/<device>
```
- Atomic, content-addressed, includes untracked files, excludes gitignored.
- Receiver fetches the ref, applies it (checkout into worktree or materialize
diff), then verifies `current_source_state().fingerprint` matches the
beacon's fingerprint, guaranteeing byte-exact reproduction.
- The sender's worktree/index/HEAD never move.
## Phased plan
### Phase B - selfdev build parity (do first; kills the stated pain)
After a successful `selfdev build` + publish on machine X:
1. Compute `SourceState` (already done by the build pipeline).
2. Snapshot the worktree to `refs/jcode/sync/<device>` (mechanism above) and
push to the shared git remote. Clean trees can skip the snapshot and use
the existing commit.
3. Announce a **version beacon** `{repo_id, version_label, fingerprint,
full_hash, sync_ref, device, timestamp}`:
- Fast path: HTTP POST to peer gateways (`/peer/version-beacon`) over
Tailscale.
- Fallback: jade relay device event (works through NAT).
- Slow path: peer polls sync refs on the git remote.
Machine Y's server runs a small peer-sync task (same shape as
`jade_relay::spawn_if_configured`):
1. Receives beacon; ignores if `version_label` matches what it already runs
or recently applied (**echo suppression**, prevents rebuild ping-pong).
2. Policy gate, default conservative:
- Worktree clean AND local HEAD is an ancestor of the beacon commit
→ auto-apply: fetch, advance, build via the selfdev build queue
(native arch), publish. The existing `server_has_newer_binary()` poll
then auto-reloads.
- Otherwise → do not clobber. Surface in TUI/status:
`peer build available: <label> from <device> (blocked: local changes)`
with a one-keystroke accept.
3. After publish, Y's `version_label` equals X's. Cross-device parity is
verifiable by comparing labels (e.g. in `selfdev status` and the beacon
acks).
Config sketch:
```toml
[sync]
enabled = true
repo_id = "jcode" # portable repo identity
peers = ["macbook.tail-net.ts.net:7643"]
auto_apply = "clean-ff-only" # off | clean-ff-only | always-notify
```
### Phase A - hub attach (one authoritative worktree when both online)
`jcode attach <host>`: TUI connects to the peer machine's server through the
existing gateway WS. Because tools execute server-side, the attached client
participates fully in that machine's worktree, conflict tracking included.
Work items:
- Client transport: WS stream in place of Unix socket (bridge already exists
server-side; needs a client-side counterpart).
- Pairing/auth UX for a trusted personal device (DeviceRegistry exists).
- Audit client-side local-disk reads that assume the session's filesystem,
e.g. `jcode-tui/src/tui/ui_file_diff.rs:270` (`std::fs::read_to_string` of
the diffed file). These need server RPCs (a `read_file` control request) or
graceful degradation.
### Phase C - true worktree federation (long-term)
Generalize Phase B's snapshot + Phase A's peering into continuous two-way
sync:
- **Data plane:** throttled auto-snapshots to per-device sync refs, triggered
by (a) server-mediated tool edits, (b) an fs watcher for bash/editor/cargo
writes, (c) timers. Peers fetch refs directly (ssh/Tailscale) or via the
shared remote.
- **Convergence:** if local HEAD/state is an ancestor → fast-forward apply.
If diverged → keep both snapshots, mark the repo "split", and let the
harness spawn an agent to perform the merge (the cross-device analog of
the server managing same-machine conflicts).
- **Coordination plane:** federate `FileTouchService` events over the peer
link so agents on both machines see "another agent edited lines 40-60"
warnings across devices. Optionally add advisory write leases for hot
files.
## Alternatives considered
- **Syncthing/mutagen for the worktree:** simple, but byte-level conflicts
(`.sync-conflict` files), no atomicity across multi-file edits, and target
dirs / build artifacts need careful exclusion. Git-plumbing snapshots give
atomic, content-addressed, mergeable states using semantics git users
already understand.
- **Always build on one machine + copy binaries:** broken by arch mismatch;
cross-compiling darwin from linux (and vice versa) is not worth the
toolchain cost given both machines have working local toolchains.
- **NFS/SSHFS shared worktree:** punishes offline use and IDE/file-watcher
performance; a non-starter for laptops.
## Suggested order of implementation
1. Phase B beacon + receiver with `auto_apply = "clean-ff-only"`, git-remote
polling only (no new ports), `selfdev status` showing peer parity.
2. Add gateway `/peer/version-beacon` fast path + TUI notification for the
blocked case.
3. Phase A `jcode attach` (WS client transport + pairing + file-read RPC).
4. Phase C federation, reusing the beacon snapshot machinery and the peer
link from A.
@@ -0,0 +1,229 @@
# Roadmap: maximal macOS control for the `computer` tool
Goal: give the agent as much reliable control over macOS as the platform allows,
including **background control that does not disturb what the user is looking at**.
This builds on the v1 `computer` tool (PR #345): screenshot, coordinate
mouse/keyboard, scroll, AX-tree read, cursor, permission check.
Everything below is implementable in Rust with crates that are already in the
lockfile or available on crates.io (`accessibility-sys`, `screencapturekit`,
`objc2-app-kit`, `core-graphics`). No Swift/ObjC build step.
---
## The one hard constraint
macOS has **one HID cursor and one keyboard-focus** shared by the whole login
session. Synthetic *coordinate* input (CGEvent) is therefore always visible: it
moves the real cursor and types into the focused app.
**Background / not-in-view control must avoid CGEvent** and instead go through:
1. **Accessibility (AX) actions** - act on a specific element by reference.
2. **Apple Events / scripting** - drive scriptable apps with no UI.
3. **Per-window capture** - "see" a window without raising it.
True simultaneous "you work + I work independently" needs a **separate display
or login session** (see Tier 4).
---
## Tier 0 - done (v1, PR #345)
- `screenshot` (main display, point/pixel scale aware)
- `move` / `click` / `double_click` / `right_click` / `drag` / `scroll`
- `type` / `key` (chords)
- `ui` (AX tree read via osascript)
- `cursor`, `check_permissions`
## Tier 1 - AX semantic actions ← highest leverage for background control
Read + act on elements by reference, no cursor movement, target app need not be
frontmost. Uses `accessibility-sys` (`AXUIElementPerformAction`,
`AXUIElementSetAttributeValue`, `AXUIElementCopyElementAtPosition`,
`AXUIElementCopyAttributeValue`).
- `find_element { role?, title?, value?, pid?, app? }` -> stable element handles
- `element_at { x, y }` -> element under a point (AXUIElementCopyElementAtPosition)
- `press { element }` -> `AXPress` (click a button in a background window)
- `set_value { element, value }` -> type into a field without focus
- `get_value { element }`
- `perform_action { element, ax_action }` -> any advertised AX action
- `select_menu { app, path: ["File","Export…"] }` -> drive the menu bar of any app
Handle format: `pid` + AX path (index chain) or a session-scoped element id cache,
so the model can act structurally instead of by pixels.
Why it matters: this is the actual "click things you're not looking at" capability.
## Tier 2 - app / window / system management
Mostly `objc2-app-kit` (`NSWorkspace`, `NSRunningApplication`) + AX window
attributes + CoreGraphics window list.
- `list_apps` / `activate_app { app }` / `hide_app` / `quit_app`
- `list_windows { pid? }` (CGWindowList) with ids, titles, bounds, on/off-screen
- `focus_window` / `move_window` / `resize_window` / `minimize_window` / `close_window`
(AX window actions - can target background windows)
- `window_screenshot { window_id }` -> capture a specific window even if occluded
(`CGWindowListCreateImage` now, ScreenCaptureKit later)
- `spaces` awareness (which Space an app is on; activating may switch Spaces - visible)
## Tier 3 - clipboard, input fidelity, observation
- `get_clipboard` / `set_clipboard { text }` (`NSPasteboard` via objc2-app-kit)
- `key_down` / `key_up` (hold modifiers, game-style input)
- `type_into { element, text }` (AX set value + confirm) for reliability over blind typing
- `wait_for { element|condition, timeout }` using `AXObserver*` notifications
(e.g. wait for a sheet to appear) instead of sleep-and-poll
- `paste_type { text }` - set clipboard + Cmd-V for fast/large text entry
## Tier 4 - true background / parallel operation (advanced)
These give genuinely off-screen, non-interfering control. Higher setup cost.
- **Apple Events scripting bridge**: `run_applescript { script }` / `run_jxa`.
Fully headless for scriptable apps (Mail, Notes, Safari, Finder, Music, System
Settings panes, Terminal, many pro apps). No cursor, no focus. Per-app
Automation permission (prompts on first use).
- **Virtual / headless display**: route the agent's cursor+windows to a second
(virtual) display the user isn't looking at. Options: a virtual display driver
(e.g. BetterDisplay/`CGVirtualDisplay` private API) or a real unused monitor.
Lets the agent move windows there and use coordinate input without touching the
user's screen.
- **Separate login / Screen Sharing session**: a second macOS session has its own
cursor and focus; the agent drives that one. Strongest isolation, most setup.
- **Shortcuts integration**: invoke the user's `Shortcuts` automations
(`shortcuts run …`) as high-level, sanctioned actions.
## Tier 5 - sensors / extras (optional, opt-in)
- `ocr { region|window }` via Vision framework (read text in images / non-AX apps).
- `screen_record { seconds }` short clips via ScreenCaptureKit.
- Audio in/out control, notifications, `do_not_disturb` toggling via scripting.
- Camera/mic are separate TCC permissions; keep strictly opt-in.
---
## Permissions (TCC) - the gatekeeping reality
| Permission | Unlocks | Auto-grantable? |
|---|---|---|
| **Accessibility** | CGEvent input, all AX read/act, window control | No - user toggles once (we can prompt + deep-link) |
| **Screen Recording** | screenshots, window/ocr capture | Request API exists (`CGRequestScreenCaptureAccess`) |
| **Automation (Apple Events)** | scripting each app | Prompts per target app on first send |
| **Input Monitoring** | reading global input stream (only if we add capture) | Request API exists |
Plan: a `request_permissions` action that calls
`AXIsProcessTrustedWithOptions(prompt=true)` (adds jcode to the list + shows the
dialog) and deep-links to the exact System Settings pane, then polls
`AXIsProcessTrusted()`. One prompt + one toggle; never zero-touch for Accessibility
(Apple's anti-malware boundary).
Important: the permission attaches to the **host binary/terminal** running jcode.
For a stable experience we likely want a signed jcode.app with a fixed bundle id so
the grant persists across updates (otherwise each new binary path re-prompts).
## Safety model (high blast radius)
- Gated like `bash`: refuses early if required permission missing.
- `dry_run` on mutating actions: resolve + report target without acting.
- Prefer AX semantic actions over blind coordinate clicks (auditable, robust).
- Screenshot/element echo on destructive coordinate clicks.
- No global input *capture* unless explicitly enabled (keeps us out of Input
Monitoring by default).
- Per-action audit log; optional allowlist/denylist of target apps.
## Suggested build order
1. **Tier 1 (AX actions)** - biggest capability jump, enables background control.
2. **Tier 2 window mgmt + per-window screenshot** - "see and act on hidden windows".
3. **Tier 3 clipboard + AXObserver waits** - reliability.
4. **`run_applescript`/JXA bridge (Tier 4)** - headless scripting for many apps.
5. **Virtual-display / second-session (Tier 4)** - true parallel, non-interfering.
6. Signed jcode.app bundle for durable permissions.
7. Vision OCR (Tier 5) as needed.
## Crates
- `accessibility-sys` 0.2 (AX read/act/observe) - on crates.io
- `screencapturekit` 7 (modern capture) - on crates.io; `core-graphics` window list as fallback
- `objc2-app-kit` / `objc2-foundation` 0.3 - already in lockfile (NSWorkspace, NSPasteboard)
- `core-graphics` 0.23 - already a direct dep (CGEvent, CGWindowList, CGDisplay)
---
## Tool interface design (decided)
### Single tool, progressive disclosure
One `computer` tool, `action`-dispatched (like `browser`). To keep always-on
context flat regardless of how many tiers exist, the schema uses **progressive
disclosure**:
- **Always-on core (~370 tokens, measured with tiktoken cl100k_base):**
`screenshot, ui, ocr, click, type, key, press, set_value, run_applescript,
setup, discover`.
- **`discover { category }`** returns full specs for advanced actions on demand
(`mouse|keyboard|ax|windows|apps|clipboard|scripting|displays|system|all`),
~130 tokens per category, paid only when used.
- **Shared handle types** (`element`, `window_id`, `region`) defined once and
reused, so params do not multiply with actions.
Measured always-on cost:
| Design | Actions visible | Always-on tokens |
|---|---|---|
| Current v1 tool | 12 | ~720 |
| Flat, all tiers (~46 actions) | 46 | ~1,020 |
| **Progressive core** | 11 | **~370** |
Background control is a property of the *mechanism*, not the tier: CGEvent =
visible; **AX actions (press/set_value/select_menu) + Apple Events = background**.
### `setup` / `check_permissions` action
A first-class `setup` action that:
1. **Reports** status of every requirement: Accessibility (`AXIsProcessTrusted`),
Screen Recording (`CGPreflightScreenCaptureAccess`), Automation (per-app, via
first Apple Event), plus install/bundle health.
2. **Requests** what it can programmatically:
- `AXIsProcessTrustedWithOptions(prompt=true)` — shows the Accessibility dialog
and pre-adds jcode to the list (toggled off).
- `CGRequestScreenCaptureAccess()` — prompts for Screen Recording.
- First Apple Event to a target app — triggers its Automation prompt.
3. **Deep-links** to the exact System Settings pane for anything still missing:
- `x-apple.systempreferences:com.apple.preference.security?Privacy_Accessibility`
- `…?Privacy_ScreenCapture`
- `…?Privacy_Automation`
4. **Polls** `AXIsProcessTrusted()` until granted, then reports "ready".
**Hard limit:** the Accessibility *toggle itself cannot be flipped by any API*
(Apple anti-malware boundary). `tccutil` can only reset, not grant. So the best
achievable UX is **"one or two prompts + one toggle,"** never zero-touch.
### Durable permissions: signed app bundle
TCC permissions attach to the **running binary's identity**. A bare dev/cli binary
changes path/signature across updates, so macOS re-prompts every time. To make a
grant stick:
- Ship/install jcode as a **signed `.app` bundle with a stable bundle id**
(e.g. `com.jcode.app`) and a Designated Requirement, so the Accessibility /
Screen Recording grant persists across updates.
- `setup` should detect "running from an unstable/unsigned path" and offer to
install the proper bundle, so the user grants **once**.
### Build order (updated)
1. Progressive-disclosure refactor of the v1 tool (core + `discover`).
2. `setup` action (check + request + deep-link + poll).
3. Tier 1 AX actions (background control).
4. Tier 2 window/app management + per-window screenshot.
5. Tier 3 clipboard + AXObserver waits.
6. `run_applescript`/JXA bridge (Tier 4 headless scripting).
7. Signed app bundle for durable permissions.
8. Tier 5 OCR (Vision). (Camera/audio intentionally excluded.)
9. Virtual-display / second-session for true parallel work (advanced).
+149
View File
@@ -0,0 +1,149 @@
# Proposal: native `computer` tool for macOS computer use
## Summary
Add a single native tool, **`computer`**, that lets the agent observe and control
the macOS GUI — screenshots, the accessibility (AX) tree, mouse/keyboard input,
window/app management, and clipboard — through one `action`-dispatched interface.
This mirrors the existing **`browser`** tool (`crates/jcode-app-core/src/tool/browser.rs`):
one registered tool, an `action: String` that selects a sub-operation, with optional
typed params. It gives jcode a closed control loop (*see screen → decide → act*)
without depending on a browser or external automation tooling.
## Motivation
- The agent can already drive a browser; it cannot drive native macOS apps, system
UI, or anything outside the browser sandbox.
- "Computer use" agents need exactly three primitives: **read the screen**, **read
UI structure**, and **synthesize input**. macOS exposes all three through the
Accessibility / Quartz Event Services / ScreenCaptureKit stack.
- A single, well-scoped tool keeps the tool surface small and the permission story
in one place.
## Architecture
```
crates/jcode-macos-control/ (new) cfg(target_os = "macos") platform crate
└─ AX (accessibility-sys), CGEvent (core-graphics),
CoreFoundation (core-foundation), screenshots (ScreenCaptureKit / CGDisplay),
app/window control (objc2 + objc2-app-kit), clipboard (objc2 NSPasteboard)
crates/jcode-app-core/src/tool/computer.rs (new) ComputerTool
└─ thin dispatch layer: parse input -> call jcode-macos-control -> ToolOutput
└─ registered in crates/jcode-app-core/src/tool/mod.rs base_tools()
```
- All native APIs are reached through existing Rust bindings (`objc2`,
`accessibility-sys`, `core-graphics`, `core-foundation`) — **no Swift/ObjC build
step**.
- On non-macOS targets the tool still registers but every action returns a clean
`unsupported on this platform` error, so the tool list stays stable across OSes.
- `screenshot` returns its image via `ToolOutput::with_image` (base64), matching how
`browser` returns screenshots today.
## Permissions (the important part)
macOS splits this across **four** TCC permissions. Programmatic *request* support
differs per permission:
| Permission | Used for | Programmatic request |
|---|---|---|
| **Accessibility** | drive other apps' UI, inject `CGEvent` input | ⚠️ prompt + deep-link only; user must toggle |
| **Screen Recording** | screenshots / `get_ui_tree` of some apps | ✅ `CGRequestScreenCaptureAccess()` |
| **Input Monitoring** | reading the global input stream | ✅ `IOHIDRequestAccess(...)` |
| **Automation** (Apple Events) | scripting cooperating apps | ✅ prompts on first send, per target app |
**Accessibility is the one that cannot be auto-granted** (Apple's anti-malware
boundary), and it is required for input injection. Best achievable flow, exposed via
the `request_permissions` action:
1. `AXIsProcessTrustedWithOptions([kAXTrustedCheckOptionPrompt: true])` — shows the
system dialog *and auto-adds jcode to the Accessibility list* (toggled off).
2. Deep-link to the exact pane:
`open "x-apple.systempreferences:com.apple.preference.security?Privacy_Accessibility"`.
3. Poll `AXIsProcessTrusted()` until granted, then report ready.
So the experience becomes **one prompt + one toggle**, not "go hunt in Settings" —
but never zero-touch for Accessibility.
`check_permissions` reports the current state of all four so the agent can tell the
user precisely what is missing before attempting control.
## Actions
`action` (required) selects the operation. Params below are optional and validated
per action.
**Permissions**
- `check_permissions` → status of accessibility / screen-recording / input-monitoring
- `request_permissions` → prompt + deep-link flow above
**Observe**
- `screenshot``{ display?, window_id?, region? }` → image
- `get_ui_tree``{ pid? | frontmost, max_depth? }` → serialized AX tree (role, title, value, position, size, actions)
- `find_element``{ role?, title?, value?, pid? }` → matching elements + their identifiers
- `element_at``{ x, y }` → element under the point
**Mouse**
- `move``{ x, y }`
- `click` / `double_click` / `right_click``{ x?, y? }` (current position if omitted)
- `drag``{ from: [x,y], to: [x,y] }`
- `scroll``{ x?, y?, dx, dy }`
**Keyboard**
- `type``{ text }`
- `key``{ keys: "cmd+shift+4" }` (chord)
- `key_down` / `key_up``{ key }`
**Semantic AX (preferred over raw input when available)**
- `press``{ element }` (AXPress)
- `set_value``{ element, value }`
- `get_value``{ element }`
- `perform_action``{ element, ax_action }`
- `select_menu``{ app, path: ["File", "Export…"] }`
**Window / app**
- `list_apps`, `activate_app` `{ app }`
- `list_windows` `{ pid? }`, `focus_window` `{ window }`
- `move_window` `{ window, x, y }`, `resize_window` `{ window, w, h }`
- `minimize_window` / `close_window` `{ window }`
**Clipboard**
- `get_clipboard`, `set_clipboard` `{ text }`
> Element identifiers: `find_element` / `get_ui_tree` return stable-enough handles
> (e.g. `pid` + AX path or a session-scoped element id) that semantic actions accept,
> so the agent can act structurally instead of by pixel coordinates when possible.
## Safety
Computer use is high-blast-radius, so:
- **Permission-gated** like other powerful tools; refuses early with a clear message
if Accessibility/Screen Recording is missing.
- **`dry_run` param** on mutating actions — resolves and reports the target without
acting.
- **Screenshot-assisted confirmation** for destructive coordinate clicks (return the
region/element being targeted).
- **No global input *capture*** in v1 (we synthesize input but do not log the user's
keystrokes), keeping us out of Input Monitoring unless a future feature needs it.
- Prefer **semantic AX actions** over blind coordinate input wherever the element is
resolvable — more robust and more auditable.
## Implementation plan
1. `jcode-macos-control` crate: permissions, screenshot, AX read, AX action,
CGEvent input, window/app control, clipboard. Unit-test the pure parts
(input parsing, chord parsing, tree serialization).
2. `ComputerTool` in `tool/computer.rs`: input struct + `action` dispatch +
schema + description; register `"computer"` in `tool/mod.rs` `base_tools()`.
3. Default-off / gated rollout + docs in `docs/`.
4. Follow-up: Windows/Linux backends behind the same tool surface.
## Open questions
- Element handle format — `pid`+AX-path vs an opaque session-scoped id cache?
- Should `request_permissions` block-and-poll, or return immediately with status and
let the agent re-check?
- Default enablement: opt-in flag vs always-registered-but-gated?