chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,398 @@
|
||||
# MacroCLI Demo — gedit 完整测试方案
|
||||
|
||||
本文档记录 MacroCLI 在 **Linux 无头服务器 + Xvfb + noVNC** 环境下的完整测试过程，
|
||||
包括虚拟桌面搭建、VNC 远程查看、宏执行和录制回放的全链路验证。
|
||||
|
||||
---
|
||||
|
||||
## 环境架构
|
||||
|
||||
```
|
||||
本地浏览器
|
||||
│ SSH 隧道 (16080 → 16080)
|
||||
▼
|
||||
远程 Linux 服务器
|
||||
│
|
||||
├─ Xvfb :99 虚拟显示器 1920x1080
|
||||
├─ openbox 窗口管理器
|
||||
├─ x11vnc VNC 服务 (只监听 localhost:5900)
|
||||
├─ websockify WebSocket → VNC 代理 (16080)
|
||||
└─ gedit 被操控的 GUI 应用
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 第一步：安装依赖
|
||||
|
||||
```bash
|
||||
# 虚拟显示器和 VNC
|
||||
sudo apt install xvfb x11vnc openbox xdotool wmctrl
|
||||
|
||||
# GUI 应用（demo 目标）
|
||||
sudo apt install gedit
|
||||
|
||||
# noVNC（浏览器查看桌面）
|
||||
sudo apt install novnc websockify
|
||||
|
||||
# AT-SPI 语义控制（可选，GTK 应用）
|
||||
sudo apt install python3-pyatspi
|
||||
|
||||
# MacroCLI Python 依赖
|
||||
conda activate macrocli # 或你的环境名
|
||||
pip install -e "path/to/macrocli/agent-harness[visual]"
|
||||
# 等价于: pip install mss Pillow numpy pynput
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 第二步：启动虚拟桌面
|
||||
|
||||
```bash
|
||||
# 1. 启动 Xvfb 虚拟显示器（:99 号，1920x1080 24位色）
|
||||
Xvfb :99 -screen 0 1920x1080x24 -ac &
|
||||
|
||||
# 2. 设置 DISPLAY（后续所有命令都需要）
|
||||
export DISPLAY=:99
|
||||
|
||||
# 3. 启动窗口管理器（让窗口能正常显示和拖动）
|
||||
openbox &
|
||||
|
||||
# 4. 设置桌面背景色（避免纯黑）
|
||||
xsetroot -solid gray
|
||||
|
||||
# 5. 启动 VNC 服务，只监听本地，允许多连接
|
||||
x11vnc -display :99 -nopw -listen localhost -rfbport 5900 \
|
||||
-forever -shared -bg -o /tmp/x11vnc.log
|
||||
|
||||
# 6. 启动 noVNC websocket 代理
|
||||
websockify --web /usr/share/novnc 16080 localhost:5900 &
|
||||
|
||||
# 验证服务启动
|
||||
ss -tlnp | grep -E "5900|16080"
|
||||
```
|
||||
|
||||
期望输出：
|
||||
```
|
||||
LISTEN 127.0.0.1:5900 ← x11vnc
|
||||
LISTEN 0.0.0.0:16080 ← websockify
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 第三步：连接 VNC 查看桌面
|
||||
|
||||
在**本地机器**开 SSH 隧道：
|
||||
|
||||
```bash
|
||||
ssh -L 16080:localhost:16080 user@your-server -N
|
||||
```
|
||||
|
||||
然后浏览器访问：
|
||||
|
||||
```
|
||||
http://localhost:16080/vnc.html
|
||||
```
|
||||
|
||||
点击 **Connect**，无需密码，即可看到虚拟桌面。
|
||||
|
||||
> **注意：** 如果 Connect 后显示"连接已中断"，通常是 x11vnc 的客户端连接数超限。
|
||||
> 重启 x11vnc 并加 `-shared` 参数即可：
|
||||
> ```bash
|
||||
> pkill x11vnc
|
||||
> x11vnc -display :99 -nopw -listen localhost -rfbport 5900 -forever -shared -bg
|
||||
> ```
|
||||
|
||||
---
|
||||
|
||||
## 第四步：检查后端可用性
|
||||
|
||||
```bash
|
||||
export DISPLAY=:99
|
||||
conda activate macrocli
|
||||
cd path/to/macrocli/agent-harness
|
||||
|
||||
cli-anything-macrocli --json backends
|
||||
```
|
||||
|
||||
期望输出（所有后端 available: true）：
|
||||
|
||||
```json
|
||||
{
|
||||
"native_api": { "available": true, "priority": 100 },
|
||||
"gui_macro": { "available": true, "priority": 80 },
|
||||
"visual_anchor": { "available": true, "priority": 75 },
|
||||
"file_transform":{ "available": true, "priority": 70 },
|
||||
"semantic_ui": { "available": true, "priority": 50 },
|
||||
"recovery": { "available": true, "priority": 10 }
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Demo A：手写宏执行
|
||||
|
||||
### A1. 打开 gedit
|
||||
|
||||
```bash
|
||||
cli-anything-macrocli --json macro run gedit_new_window
|
||||
```
|
||||
|
||||
VNC 里能看到 gedit 窗口弹出。
|
||||
|
||||
输出示例：
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"telemetry": { "duration_ms": 39, "backends_used": ["native_api", "semantic_ui"] }
|
||||
}
|
||||
```
|
||||
|
||||
> **实现细节：** `gedit_new_window` 用 `native_api` 的 `start_process` action 后台
|
||||
> 启动 gedit（不等待进程退出），然后 `semantic_ui` 的 `wait_for_window` 等待窗口出现。
|
||||
|
||||
### A2. 输入文字并保存
|
||||
|
||||
```bash
|
||||
cli-anything-macrocli --json macro run gedit_type_and_save \
|
||||
--param "text=Hello from MacroCLI!"
|
||||
```
|
||||
|
||||
VNC 里能看到文字逐字输入到 gedit，然后触发 Ctrl+S。
|
||||
|
||||
输出示例：
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"steps": [
|
||||
{ "backend_used": "semantic_ui", "output": {"focused": "gedit", "method": "wmctrl"} },
|
||||
{ "backend_used": "visual_anchor", "output": {"typed": 20} },
|
||||
{ "backend_used": "visual_anchor", "output": {"hotkey": "ctrl+s"} }
|
||||
],
|
||||
"telemetry": { "duration_ms": 774 }
|
||||
}
|
||||
```
|
||||
|
||||
> **实现细节：** 三个后端串联——`semantic_ui` 用 wmctrl 聚焦窗口，`visual_anchor`
|
||||
> 用 pynput 注入键盘事件，`visual_anchor` 发送 Ctrl+S 快捷键。
|
||||
|
||||
### A3. 另存为指定路径
|
||||
|
||||
```bash
|
||||
cli-anything-macrocli --json macro run gedit_save_as \
|
||||
--param output_path=/tmp/macrocli_demo.txt
|
||||
```
|
||||
|
||||
执行过程（可在 VNC 观察）：
|
||||
1. Ctrl+Shift+S 弹出 Save As 对话框
|
||||
2. Ctrl+L 打开路径输入框
|
||||
3. Ctrl+A 清空已有内容
|
||||
4. 输入 `/tmp/macrocli_demo.txt`
|
||||
5. Enter 确认
|
||||
|
||||
```bash
|
||||
# 验证文件已创建
|
||||
cat /tmp/macrocli_demo.txt
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Demo B：录制宏（核心功能）
|
||||
|
||||
录制功能把一次手工操作自动转成可重复调用的宏。
|
||||
|
||||
### 录制原理
|
||||
|
||||
```
|
||||
用户操作 GUI
|
||||
│
|
||||
pynput 监听鼠标键盘事件
|
||||
│
|
||||
每次点击：
|
||||
xdotool getwindowfocus → 获取焦点窗口名称和位置
|
||||
计算点击在窗口内的百分比坐标 (x_pct, y_pct)
|
||||
→ 生成 click_relative 步骤（窗口锚点，不依赖绝对坐标）
|
||||
│
|
||||
键盘输入：
|
||||
累积字符（含空格）→ 生成 type_text 步骤
|
||||
快捷键组合 → 生成 hotkey 步骤
|
||||
│
|
||||
Ctrl+Alt+S 停止录制
|
||||
│
|
||||
自动写出 YAML 宏文件
|
||||
```
|
||||
|
||||
**为什么用窗口锚点而不是绝对坐标：**
|
||||
点击空白文本区会截到全白图片，模板匹配无法识别。改用窗口相对坐标
|
||||
（`click_relative`），回放时先找到窗口位置，再按百分比算出实际坐标，
|
||||
窗口移动到任何地方都能正确点击。
|
||||
|
||||
### 录制操作
|
||||
|
||||
```bash
|
||||
# 开始录制，命名为 my_workflow
|
||||
cli-anything-macrocli macro record my_workflow \
|
||||
--output-dir /tmp/my_recording \
|
||||
--timeout 30 # 30秒后自动停止，或手动 Ctrl+Alt+S
|
||||
```
|
||||
|
||||
终端显示：
|
||||
```
|
||||
Recording 'my_workflow'. Press Ctrl+Alt+S to stop.
|
||||
[recorder] Recording 'my_workflow'. Press Ctrl+Alt+S to stop.
|
||||
```
|
||||
|
||||
**现在操作 gedit（录制器会捕获这些动作）：**
|
||||
1. 在 VNC 里点击 gedit 文本区
|
||||
2. 输入一段文字
|
||||
3. 按 Ctrl+S 保存
|
||||
|
||||
按 Ctrl+Alt+S 停止（或等待超时），自动生成：
|
||||
|
||||
```
|
||||
/tmp/my_recording/
|
||||
├── my_workflow.yaml ← 宏定义文件
|
||||
└── my_workflow_templates/ ← 有特征区域的截图模板
|
||||
└── step_001_click.png ← 若点击区域有特征才会保存
|
||||
```
|
||||
|
||||
生成的 YAML 示例：
|
||||
|
||||
```yaml
|
||||
name: my_workflow
|
||||
steps:
|
||||
- id: step_001_click
|
||||
backend: visual_anchor
|
||||
action: click_relative # 窗口锚点，不是绝对坐标
|
||||
params:
|
||||
window_title: gedit # 从 xdotool getwindowfocus 获取
|
||||
x_pct: 0.35 # 点击位置在窗口宽度的 35%
|
||||
y_pct: 0.33 # 点击位置在窗口高度的 33%
|
||||
|
||||
- id: step_002_type
|
||||
backend: visual_anchor
|
||||
action: type_text
|
||||
params:
|
||||
text: hello world # 空格正确合并（不再被切成 hotkey）
|
||||
|
||||
- id: step_003_hotkey
|
||||
backend: visual_anchor
|
||||
action: hotkey
|
||||
params:
|
||||
keys: ctrl+s
|
||||
```
|
||||
|
||||
### 回放录制的宏
|
||||
|
||||
```bash
|
||||
# 先清空 gedit 内容
|
||||
# 然后回放
|
||||
cli-anything-macrocli --json macro run my_workflow \
|
||||
--macro-file /tmp/my_recording/my_workflow.yaml
|
||||
```
|
||||
|
||||
实际测试输出：
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"telemetry": {
|
||||
"duration_ms": 486,
|
||||
"steps_run": 3,
|
||||
"backends_used": ["visual_anchor"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
VNC 截图验证（`hello world` 出现在 gedit）：
|
||||
|
||||
```
|
||||
gedit 窗口标题: macrocli_test_final.txt (/tmp) - gedit
|
||||
内容: hello world
|
||||
状态: 已保存（标题栏无 * 号）
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Demo C：transform_json（file_transform 后端）
|
||||
|
||||
> ⚠️ **未在本次测试中验证，仅供参考。**
|
||||
|
||||
不需要 GUI，直接操作 JSON 文件：
|
||||
|
||||
```bash
|
||||
# 创建测试文件
|
||||
echo '{"app": "draw.io", "version": 1}' > /tmp/config.json
|
||||
|
||||
# 用宏修改嵌套 key
|
||||
cli-anything-macrocli --json macro run transform_json \
|
||||
--param file=/tmp/config.json \
|
||||
--param key=settings.theme \
|
||||
--param value=dark
|
||||
|
||||
# 验证
|
||||
cat /tmp/config.json
|
||||
# {"app": "draw.io", "version": 1, "settings": {"theme": "dark"}}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 常见问题
|
||||
|
||||
**Q: `macro run gedit_new_window` 超时 30 秒**
|
||||
|
||||
原因：`run_command` 会等待进程退出，GUI 应用永远不会退出。
|
||||
解决：改用 `start_process` action（本项目已修复）。
|
||||
|
||||
**Q: `wait_for_window` 找不到窗口**
|
||||
|
||||
原因：`conda run` 会清除 `DISPLAY` 环境变量。
|
||||
解决：激活 conda 环境后直接运行，不要用 `conda run`：
|
||||
```bash
|
||||
conda activate macrocli
|
||||
export DISPLAY=:99
|
||||
cli-anything-macrocli ...
|
||||
```
|
||||
|
||||
**Q: VNC Connect 后显示"连接已中断"**
|
||||
|
||||
原因：x11vnc 默认不允许多个客户端同时连接。
|
||||
解决：重启 x11vnc 并加 `-shared`：
|
||||
```bash
|
||||
pkill x11vnc
|
||||
x11vnc -display :99 -nopw -listen localhost -rfbport 5900 -forever -shared -bg
|
||||
```
|
||||
|
||||
**Q: 录制回放时点击位置不准**
|
||||
|
||||
原因：窗口大小与录制时不同，百分比坐标偏移。
|
||||
解决：确保回放时窗口大小与录制时一致，或调整宏里的 `x_pct`/`y_pct`。
|
||||
|
||||
**Q: 模板图全白，click_image 永远超时**
|
||||
|
||||
原因：点击了空白区域（如文本编辑区），截图无特征。
|
||||
解决：录制器已自动检测低方差图片，改用 `click_relative` 替代。
|
||||
|
||||
---
|
||||
|
||||
## 验证结果汇总
|
||||
|
||||
| 宏 | 后端 | 结果 | 耗时 |
|
||||
|----|------|------|------|
|
||||
| `gedit_new_window` | native_api + semantic_ui | ✓ | ~40ms |
|
||||
| `gedit_type_and_save` | semantic_ui + visual_anchor | ✓ | ~774ms |
|
||||
| `gedit_save_as` | semantic_ui + visual_anchor | ✓ | ~1847ms |
|
||||
| `transform_json` | file_transform | ⚠️ 未验证 | — |
|
||||
| `macro record` + 回放 | visual_anchor | ✓ | ~486ms |
|
||||
|
||||
---
|
||||
|
||||
## 对接其他应用
|
||||
|
||||
同样的流程适用于任何 Linux GUI 应用：
|
||||
|
||||
| 应用 | 推荐后端 | 说明 |
|
||||
|------|----------|------|
|
||||
| Inkscape | `native_api` (`--actions`) | 有完整命令行接口 |
|
||||
| GIMP | `native_api` (Script-Fu) | 脚本接口强大 |
|
||||
| LibreOffice | `native_api` (`--headless`) | UNO API |
|
||||
| draw.io | `file_transform` + `visual_anchor` | XML 格式可直接编辑 |
|
||||
| 任意 GUI 应用 | `macro record` + `visual_anchor` | 录制一次，窗口锚点回放 |
|
||||
@@ -0,0 +1,210 @@
|
||||
# MacroCLI — Agent Harness SOP
|
||||
|
||||
## What Is This?
|
||||
|
||||
**MacroCLI** is a layered CLI that turns valuable GUI workflows into
|
||||
parameterized, agent-callable macros. The agent sends one command:
|
||||
|
||||
```bash
|
||||
cli-anything-macrocli macro run export_png --param output=/tmp/out.png --json
|
||||
```
|
||||
|
||||
The system handles everything else: parameter validation, precondition checks,
|
||||
backend selection, step execution, postcondition verification, and structured
|
||||
result output. The agent never touches the GUI directly.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Agent
|
||||
└─▶ cli-anything-macrocli macro run <name> --param k=v --json (L6: CLI)
|
||||
│
|
||||
MacroRuntime (L5)
|
||||
│ 1. Validate params against MacroDefinition schema
|
||||
│ 2. Check preconditions (file_exists, process_running, …)
|
||||
│ 3. For each step:
|
||||
│ RoutingEngine → select backend by priority (L3)
|
||||
│ Backend.execute(step, resolved_params) (L2)
|
||||
│ 4. Check postconditions
|
||||
│ 5. Collect declared outputs
|
||||
│ 6. Record telemetry in ExecutionSession
|
||||
└─▶ { success, output, error, telemetry }
|
||||
```
|
||||
|
||||
## Layer Mapping
|
||||
|
||||
| Layer | Name | Implementation |
|
||||
|-------|------|---------------|
|
||||
| L7 | Agent Task Interface | Caller (any AI agent) |
|
||||
| L6 | Unified CLI Entry | `macrocli_cli.py` — Click CLI |
|
||||
| L5 | Macro Execution Runtime | `core/runtime.py` |
|
||||
| L4 | Parameterized Macro Model | `core/macro_model.py` + `macro_definitions/*.yaml` |
|
||||
| L3 | Backend Routing Engine | `core/routing.py` |
|
||||
| L2 | Execution Backends | `backends/` (7 backends) |
|
||||
| L1 | Target Application | Any GUI-first or closed-source app |
|
||||
|
||||
## Execution Backends
|
||||
|
||||
| Backend | Priority | Trigger | Use case |
|
||||
|---------|----------|---------|----------|
|
||||
| `native_api` | 100 | `backend: native_api` | subprocess / shell commands |
|
||||
| `gui_macro` | 80 | `backend: gui_macro` | precompiled coordinate replay (pyautogui) |
|
||||
| `visual_anchor` | 75 | `backend: visual_anchor` | template-matching click/type (requires `[visual]`) |
|
||||
| `file_transform` | 70 | `backend: file_transform` | XML, JSON, text file editing |
|
||||
| `gui_agent` | 60 | `backend: gui_agent` | vision-model-driven automation (requires `[gui_agent]`) |
|
||||
| `semantic_ui` | 50 | `backend: semantic_ui` | accessibility API + keyboard (xdotool) |
|
||||
| `recovery` | 10 | `backend: recovery` | retry + fallback orchestration |
|
||||
|
||||
The RoutingEngine respects the step's explicit `backend:` field; if that backend
|
||||
is unavailable it walks down the priority list.
|
||||
|
||||
## Macro Definition Format
|
||||
|
||||
Macros live in `cli_anything/macrocli/macro_definitions/` as YAML files:
|
||||
|
||||
```yaml
|
||||
name: export_png
|
||||
version: "1.0"
|
||||
description: Export the active diagram to PNG.
|
||||
|
||||
parameters:
|
||||
output:
|
||||
type: string
|
||||
required: true
|
||||
example: /tmp/diagram.png
|
||||
|
||||
preconditions:
|
||||
- process_running: draw.io
|
||||
- file_exists: /path/to/input.drawio
|
||||
|
||||
steps:
|
||||
- id: export
|
||||
backend: native_api
|
||||
action: run_command
|
||||
params:
|
||||
command: [draw.io, --export, --output, "${output}", input.drawio]
|
||||
timeout_ms: 30000
|
||||
on_failure: fail # or: skip | continue
|
||||
|
||||
postconditions:
|
||||
- file_exists: ${output}
|
||||
- file_size_gt:
|
||||
- ${output}
|
||||
- 100
|
||||
|
||||
outputs:
|
||||
- name: exported_file
|
||||
path: ${output}
|
||||
|
||||
agent_hints:
|
||||
danger_level: safe
|
||||
side_effects: [creates_file]
|
||||
reversible: true
|
||||
```
|
||||
|
||||
### Supported Condition Types
|
||||
|
||||
| Type | Args | Checks |
|
||||
|------|------|--------|
|
||||
| `file_exists` | path | `os.path.exists(path)` |
|
||||
| `file_size_gt` | [path, min_bytes] | `os.stat(path).st_size > min_bytes` |
|
||||
| `process_running` | name | `pgrep -x name` or psutil |
|
||||
| `env_var` | name | `name in os.environ` |
|
||||
| `always` | true/false | constant pass/fail |
|
||||
|
||||
## Package Layout
|
||||
|
||||
```
|
||||
macrocli/
|
||||
└── agent-harness/
|
||||
├── setup.py entry_point: cli-anything-macrocli
|
||||
└── cli_anything/macrocli/
|
||||
├── macrocli_cli.py Main Click CLI
|
||||
├── macro_definitions/ YAML macro registry
|
||||
│ ├── manifest.yaml
|
||||
│ └── examples/
|
||||
│ ├── export_file.yaml
|
||||
│ ├── transform_json.yaml
|
||||
│ └── undo_last.yaml
|
||||
├── core/
|
||||
│ ├── macro_model.py MacroDefinition + YAML loader
|
||||
│ ├── registry.py MacroRegistry
|
||||
│ ├── routing.py RoutingEngine
|
||||
│ ├── runtime.py MacroRuntime (full lifecycle)
|
||||
│ └── session.py ExecutionSession + telemetry
|
||||
├── backends/
|
||||
│ ├── base.py Backend ABC + StepResult
|
||||
│ ├── native_api.py subprocess backend
|
||||
│ ├── file_transform.py XML/JSON/text backend
|
||||
│ ├── semantic_ui.py accessibility backend
|
||||
│ ├── visual_anchor.py template-matching backend
|
||||
│ ├── gui_agent.py vision-model automation backend
|
||||
│ ├── gui_macro.py compiled replay backend
|
||||
│ └── recovery.py retry/fallback backend
|
||||
├── skills/SKILL.md Agent-readable skill definition
|
||||
├── utils/repl_skin.py Unified REPL skin (cli-anything standard)
|
||||
└── tests/
|
||||
├── test_core.py Unit tests (49 tests, no external deps)
|
||||
└── test_full_e2e.py E2E + CLI subprocess tests (15 tests)
|
||||
```
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
cd macrocli/agent-harness
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
**Runtime dependencies:** Python 3.10+, PyYAML, click, prompt-toolkit.
|
||||
|
||||
**Optional extras:**
|
||||
|
||||
```bash
|
||||
pip install -e ".[visual]" # visual_anchor backend (mss, Pillow, numpy, pynput)
|
||||
pip install -e ".[gui_agent]" # gui_agent backend (openai, mss, Pillow)
|
||||
pip install -e ".[all]" # everything
|
||||
```
|
||||
|
||||
**gui_agent backend configuration:**
|
||||
|
||||
The `gui_agent` backend uses the OpenAI SDK and is compatible with any
|
||||
OpenAI-compatible API. Configure via environment variables:
|
||||
|
||||
| Variable | Description |
|
||||
|--------------------|---------------------------------------------|
|
||||
| `MACROCLI_MODEL` | Model name (required, e.g. `gpt-4o`) |
|
||||
| `MACROCLI_API_KEY` | API key for the provider |
|
||||
| `MACROCLI_BASE_URL`| Base URL (only needed for non-OpenAI hosts) |
|
||||
|
||||
**Other optional dependencies:**
|
||||
- `xdotool` — semantic_ui backend on Linux
|
||||
- `pyautogui` — gui_macro backend
|
||||
- `psutil` — richer process_running checks
|
||||
|
||||
## Running Tests
|
||||
|
||||
```bash
|
||||
cd macrocli/agent-harness
|
||||
python3 -m pytest cli_anything/macrocli/tests/ -v -s
|
||||
# 64 passed
|
||||
```
|
||||
|
||||
## Key Design Decisions
|
||||
|
||||
**Why YAML macros, not Python?** YAML macros are readable by agents without
|
||||
running code, inspectable via `macro info`, and editable without touching the
|
||||
harness source.
|
||||
|
||||
**Why 7 backends?** Real GUI applications expose many different control
|
||||
surfaces. The routing engine picks the most reliable one available — the agent
|
||||
doesn't need to know which one ran. The `visual_anchor` backend uses template
|
||||
matching for robust UI element detection, while `gui_agent` uses vision models
|
||||
for dynamic decision-making when the UI state is unpredictable.
|
||||
|
||||
**Why preconditions and postconditions?** Agents operate in environments where
|
||||
state is uncertain. Failing loudly before execution (preconditions) and
|
||||
verifying after (postconditions) catches problems the agent can act on.
|
||||
|
||||
**Why `on_failure: skip | continue`?** Some macro steps are best-effort (e.g.,
|
||||
confirming a dialog that may or may not appear). Skipping lets the macro
|
||||
continue to the real work.
|
||||
@@ -0,0 +1,94 @@
|
||||
# MacroCLI
|
||||
|
||||
**MacroCLI** is a layered CLI that converts GUI workflows into
|
||||
parameterized, agent-callable macros. Agents call `macro run <name>` through
|
||||
a stable CLI; the system routes execution to the right backend (native plugin,
|
||||
file transform, semantic UI, or compiled GUI replay) — invisible to the agent.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
**Dependencies:** Python 3.10+, PyYAML, click, prompt-toolkit
|
||||
|
||||
**Optional extras:**
|
||||
|
||||
```bash
|
||||
pip install -e ".[visual]" # visual_anchor backend (mss, Pillow, numpy, pynput)
|
||||
pip install -e ".[gui_agent]" # gui_agent backend (openai, mss, Pillow)
|
||||
pip install -e ".[all]" # everything
|
||||
```
|
||||
|
||||
The `gui_agent` backend uses the OpenAI SDK and is compatible with any
|
||||
OpenAI-compatible API. Configure via environment variables:
|
||||
|
||||
| Variable | Description |
|
||||
|--------------------|---------------------------------------------|
|
||||
| `MACROCLI_MODEL` | Model name (required, e.g. `gpt-4o`) |
|
||||
| `MACROCLI_API_KEY` | API key for the provider |
|
||||
| `MACROCLI_BASE_URL`| Base URL (only needed for non-OpenAI hosts) |
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
# List available macros
|
||||
cli-anything-macrocli macro list --json
|
||||
|
||||
# Inspect a macro
|
||||
cli-anything-macrocli macro info export_file --json
|
||||
|
||||
# Execute a macro
|
||||
cli-anything-macrocli macro run transform_json \
|
||||
--param file=/tmp/config.json \
|
||||
--param key=theme --param value=dark --json
|
||||
|
||||
# Dry run
|
||||
cli-anything-macrocli --dry-run macro run export_file \
|
||||
--param output=/tmp/out.txt --json
|
||||
|
||||
# Interactive REPL
|
||||
cli-anything-macrocli
|
||||
```
|
||||
|
||||
## Run Tests
|
||||
|
||||
```bash
|
||||
cd macrocli/agent-harness
|
||||
pip install -e ".[dev]"
|
||||
python -m pytest cli_anything/macrocli/tests/ -v -s
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
cli-anything-macrocli (CLI)
|
||||
└─▶ macro run <name> --param key=value
|
||||
│
|
||||
MacroRuntime
|
||||
│ validate params
|
||||
│ check preconditions
|
||||
│ for each step:
|
||||
│ RoutingEngine → select backend
|
||||
│ Backend.execute(step, params)
|
||||
│ check postconditions
|
||||
└─▶ ExecutionResult { success, output, telemetry }
|
||||
```
|
||||
|
||||
**Backends:**
|
||||
- `native_api` — subprocess / shell commands
|
||||
- `file_transform` — XML, JSON, text file editing
|
||||
- `semantic_ui` — accessibility controls + keyboard shortcuts
|
||||
- `visual_anchor` — template-matching click/type (requires `[visual]`)
|
||||
- `gui_agent` — vision-model-driven automation (requires `[gui_agent]`)
|
||||
- `gui_macro` — precompiled coordinate-based replay
|
||||
- `recovery` — retry + fallback orchestration
|
||||
|
||||
## Adding a Macro
|
||||
|
||||
1. Create `cli_anything/macrocli/macro_definitions/my_macro.yaml`
|
||||
2. Add it to `macro_definitions/manifest.yaml`
|
||||
3. Verify: `cli-anything-macrocli macro validate my_macro --json`
|
||||
|
||||
See `skills/SKILL.md` (installed with the package) for full macro YAML schema.
|
||||
@@ -0,0 +1 @@
|
||||
# cli_anything/macrocli package
|
||||
@@ -0,0 +1,5 @@
|
||||
"""Enable: python3 -m cli_anything.macrocli"""
|
||||
from cli_anything.macrocli.macrocli_cli import cli
|
||||
|
||||
if __name__ == "__main__":
|
||||
cli()
|
||||
@@ -0,0 +1,89 @@
|
||||
"""Backend base classes and result types.
|
||||
|
||||
All execution backends inherit from Backend and return StepResult.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
from abc import ABC, abstractmethod
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Optional
|
||||
|
||||
|
||||
@dataclass
|
||||
class StepResult:
|
||||
"""Result of a single macro step execution."""
|
||||
success: bool
|
||||
output: dict = field(default_factory=dict)
|
||||
error: str = ""
|
||||
duration_ms: float = 0.0
|
||||
backend_used: str = ""
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"success": self.success,
|
||||
"output": self.output,
|
||||
"error": self.error,
|
||||
"duration_ms": self.duration_ms,
|
||||
"backend_used": self.backend_used,
|
||||
}
|
||||
|
||||
|
||||
class BackendContext:
|
||||
"""Runtime context passed to each backend during step execution."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
params: dict,
|
||||
previous_results: Optional[list[StepResult]] = None,
|
||||
dry_run: bool = False,
|
||||
timeout_ms: int = 30_000,
|
||||
):
|
||||
self.params = params
|
||||
self.previous_results: list[StepResult] = previous_results or []
|
||||
self.dry_run = dry_run
|
||||
self.timeout_ms = timeout_ms
|
||||
self._start = time.time()
|
||||
|
||||
def elapsed_ms(self) -> float:
|
||||
return (time.time() - self._start) * 1000
|
||||
|
||||
|
||||
class Backend(ABC):
|
||||
"""Abstract base class for all execution backends.
|
||||
|
||||
Concrete backends implement execute() and return a StepResult.
|
||||
"""
|
||||
|
||||
name: str = "base"
|
||||
priority: int = 0
|
||||
|
||||
@abstractmethod
|
||||
def execute(
|
||||
self,
|
||||
step: "MacroStep", # type: ignore[name-defined]
|
||||
params: dict,
|
||||
context: BackendContext,
|
||||
) -> StepResult:
|
||||
"""Execute a macro step.
|
||||
|
||||
Args:
|
||||
step: The MacroStep definition being executed.
|
||||
params: Fully resolved (substituted) parameters.
|
||||
context: Runtime context with previous results and flags.
|
||||
|
||||
Returns:
|
||||
StepResult describing success/failure and captured output.
|
||||
"""
|
||||
|
||||
def is_available(self) -> bool:
|
||||
"""Return True if this backend can be used in the current environment."""
|
||||
return True
|
||||
|
||||
def describe(self) -> dict:
|
||||
return {
|
||||
"name": self.name,
|
||||
"priority": self.priority,
|
||||
"available": self.is_available(),
|
||||
}
|
||||
@@ -0,0 +1,205 @@
|
||||
"""FileTransformBackend — read, transform, and write project files.
|
||||
|
||||
Supports XML (ElementTree), JSON, and plain text transformations.
|
||||
|
||||
Example macro step:
|
||||
|
||||
- backend: file_transform
|
||||
action: json_set
|
||||
params:
|
||||
input_file: ${project_file}
|
||||
output_file: ${project_file}
|
||||
path: settings.grid_size
|
||||
value: 20
|
||||
|
||||
- backend: file_transform
|
||||
action: xml_set_attr
|
||||
params:
|
||||
input_file: diagram.drawio
|
||||
output_file: diagram.drawio
|
||||
xpath: .//mxCell[@id='1']
|
||||
attr: style
|
||||
value: rounded=1;
|
||||
|
||||
- backend: file_transform
|
||||
action: text_replace
|
||||
params:
|
||||
input_file: config.ini
|
||||
output_file: config.ini
|
||||
find: "theme=default"
|
||||
replace: "theme=dark"
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
from cli_anything.macrocli.backends.base import Backend, BackendContext, StepResult
|
||||
from cli_anything.macrocli.core.macro_model import MacroStep, substitute
|
||||
|
||||
|
||||
class FileTransformBackend(Backend):
|
||||
"""Transform project files without invoking the target application."""
|
||||
|
||||
name = "file_transform"
|
||||
priority = 70
|
||||
|
||||
def execute(self, step: MacroStep, params: dict, context: BackendContext) -> StepResult:
|
||||
t0 = time.time()
|
||||
action = step.action
|
||||
step_params = substitute(step.params, params)
|
||||
|
||||
if context.dry_run:
|
||||
return StepResult(
|
||||
success=True,
|
||||
output={"dry_run": True, "action": action},
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
dispatch = {
|
||||
"json_get": self._json_get,
|
||||
"json_set": self._json_set,
|
||||
"json_delete": self._json_delete,
|
||||
"xml_set_attr": self._xml_set_attr,
|
||||
"xml_get_attr": self._xml_get_attr,
|
||||
"text_replace": self._text_replace,
|
||||
"copy_file": self._copy_file,
|
||||
}
|
||||
|
||||
handler = dispatch.get(action)
|
||||
if handler is None:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"FileTransformBackend: unknown action '{action}'.",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
try:
|
||||
output = handler(step_params)
|
||||
return StepResult(
|
||||
success=True,
|
||||
output=output or {},
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
except Exception as exc:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"FileTransformBackend.{action}: {exc}",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
# ── JSON actions ─────────────────────────────────────────────────────
|
||||
|
||||
def _json_get(self, p: dict) -> dict:
|
||||
"""Read a value from a JSON file by dot-path."""
|
||||
data = self._load_json(p["input_file"])
|
||||
val = self._dotpath_get(data, p["path"])
|
||||
return {"value": val, "path": p["path"]}
|
||||
|
||||
def _json_set(self, p: dict) -> dict:
|
||||
"""Set a value in a JSON file by dot-path and write it back."""
|
||||
path = p.get("path", "")
|
||||
value = p["value"]
|
||||
data = self._load_json(p["input_file"]) if Path(p["input_file"]).is_file() else {}
|
||||
self._dotpath_set(data, path, value)
|
||||
self._save_json(p.get("output_file", p["input_file"]), data)
|
||||
return {"path": path, "value": value}
|
||||
|
||||
def _json_delete(self, p: dict) -> dict:
|
||||
"""Delete a key from a JSON file by dot-path."""
|
||||
data = self._load_json(p["input_file"])
|
||||
self._dotpath_delete(data, p["path"])
|
||||
self._save_json(p.get("output_file", p["input_file"]), data)
|
||||
return {"deleted": p["path"]}
|
||||
|
||||
# ── XML actions ──────────────────────────────────────────────────────
|
||||
|
||||
def _xml_set_attr(self, p: dict) -> dict:
|
||||
"""Set an XML element attribute matched by XPath."""
|
||||
from xml.etree import ElementTree as ET
|
||||
from defusedxml.ElementTree import parse as _defused_parse
|
||||
tree = _defused_parse(p["input_file"])
|
||||
root = tree.getroot()
|
||||
elements = root.findall(p["xpath"])
|
||||
if not elements:
|
||||
raise ValueError(f"XPath matched nothing: {p['xpath']}")
|
||||
for el in elements:
|
||||
el.set(p["attr"], str(p["value"]))
|
||||
tree.write(p.get("output_file", p["input_file"]), encoding="unicode", xml_declaration=True)
|
||||
return {"matched": len(elements), "attr": p["attr"]}
|
||||
|
||||
def _xml_get_attr(self, p: dict) -> dict:
|
||||
"""Get an XML element attribute matched by XPath."""
|
||||
from xml.etree import ElementTree as ET
|
||||
from defusedxml.ElementTree import parse as _defused_parse
|
||||
tree = _defused_parse(p["input_file"])
|
||||
root = tree.getroot()
|
||||
elements = root.findall(p["xpath"])
|
||||
values = [el.get(p["attr"]) for el in elements]
|
||||
return {"values": values, "attr": p["attr"]}
|
||||
|
||||
# ── Text actions ─────────────────────────────────────────────────────
|
||||
|
||||
def _text_replace(self, p: dict) -> dict:
|
||||
"""Simple find-and-replace in a text file."""
|
||||
content = Path(p["input_file"]).read_text(encoding="utf-8")
|
||||
count = content.count(p["find"])
|
||||
content = content.replace(p["find"], p["replace"])
|
||||
out = p.get("output_file", p["input_file"])
|
||||
Path(out).write_text(content, encoding="utf-8")
|
||||
return {"replacements": count}
|
||||
|
||||
def _copy_file(self, p: dict) -> dict:
|
||||
"""Copy a file from src to dst."""
|
||||
import shutil
|
||||
shutil.copy2(p["src"], p["dst"])
|
||||
size = os.path.getsize(p["dst"])
|
||||
return {"src": p["src"], "dst": p["dst"], "size": size}
|
||||
|
||||
# ── Helpers ──────────────────────────────────────────────────────────
|
||||
|
||||
def _load_json(self, path: str) -> dict:
|
||||
with open(path, encoding="utf-8") as f:
|
||||
return json.load(f)
|
||||
|
||||
def _save_json(self, path: str, data: dict) -> None:
|
||||
Path(path).parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(path, "w", encoding="utf-8") as f:
|
||||
json.dump(data, f, indent=2, ensure_ascii=False)
|
||||
|
||||
def _dotpath_get(self, data: dict, path: str):
|
||||
keys = path.split(".")
|
||||
cur = data
|
||||
for k in keys:
|
||||
if isinstance(cur, dict) and k in cur:
|
||||
cur = cur[k]
|
||||
else:
|
||||
return None
|
||||
return cur
|
||||
|
||||
def _dotpath_set(self, data: dict, path: str, value) -> None:
|
||||
keys = path.split(".")
|
||||
cur = data
|
||||
for k in keys[:-1]:
|
||||
if k not in cur or not isinstance(cur[k], dict):
|
||||
cur[k] = {}
|
||||
cur = cur[k]
|
||||
cur[keys[-1]] = value
|
||||
|
||||
def _dotpath_delete(self, data: dict, path: str) -> None:
|
||||
keys = path.split(".")
|
||||
cur = data
|
||||
for k in keys[:-1]:
|
||||
if isinstance(cur, dict) and k in cur:
|
||||
cur = cur[k]
|
||||
else:
|
||||
return
|
||||
if isinstance(cur, dict) and keys[-1] in cur:
|
||||
del cur[keys[-1]]
|
||||
@@ -0,0 +1,561 @@
|
||||
"""GUIAgentBackend — execute a macro step by letting a vision model
|
||||
look at the screen and decide what to do.
|
||||
|
||||
This backend is used for steps that cannot be expressed as fixed
|
||||
coordinates or hotkeys because the interface state is unpredictable.
|
||||
The macro author provides:
|
||||
- description: what needs to be accomplished in this step
|
||||
- end_state_description: text description of the desired end state
|
||||
- end_state_snapshot: screenshot of the desired end state (taken
|
||||
by the macro author at recording time)
|
||||
|
||||
At runtime the backend:
|
||||
1. Takes a screenshot of the current screen
|
||||
2. Sends current screenshot + end_state_snapshot + description to the model
|
||||
3. Model returns the next action (click x,y / type text / hotkey)
|
||||
4. Executes the action
|
||||
5. Takes another screenshot
|
||||
6. Asks model: "have we reached the end state?"
|
||||
7. Loops until end state reached or max_steps exceeded
|
||||
|
||||
The backend uses the OpenAI SDK, which is compatible with any
|
||||
OpenAI-compatible API provider (OpenAI, Azure, local vLLM, Ollama,
|
||||
LiteLLM, etc.). Configure model and endpoint via environment
|
||||
variables or per-step YAML params:
|
||||
|
||||
Environment variables:
|
||||
MACROCLI_MODEL — model name (required, no default)
|
||||
MACROCLI_API_KEY — API key
|
||||
MACROCLI_BASE_URL — base URL for non-OpenAI providers
|
||||
|
||||
Example YAML step:
|
||||
|
||||
- id: select_png_format
|
||||
backend: gui_agent
|
||||
action: instruct
|
||||
params:
|
||||
description: >
|
||||
The export dialog is open. Find the Format dropdown and
|
||||
select PNG. Then ensure Resolution shows 300.
|
||||
end_state_description: >
|
||||
Format dropdown shows PNG, Resolution input shows 300.
|
||||
end_state_snapshot: snapshots/step_003_end_state.png
|
||||
max_steps: 8
|
||||
model: ${MACROCLI_MODEL}
|
||||
api_key: ${MACROCLI_API_KEY}
|
||||
base_url: ${MACROCLI_BASE_URL}
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from cli_anything.macrocli.backends.base import Backend, BackendContext, StepResult
|
||||
from cli_anything.macrocli.core.macro_model import MacroStep, substitute
|
||||
|
||||
# ── Strict action space prompt ────────────────────────────────────────────────
|
||||
|
||||
_SYSTEM_PROMPT = """\
|
||||
You are a GUI automation agent. You will be shown:
|
||||
1. A screenshot of the CURRENT screen state
|
||||
2. A screenshot of the TARGET end state (optional)
|
||||
3. A description of what needs to be accomplished
|
||||
|
||||
Your job is to figure out what single action to take next.
|
||||
|
||||
OUTPUT FORMAT: Respond with ONLY a JSON object, one of:
|
||||
|
||||
{"action": "click", "x": <int>, "y": <int>, "button": "left"}
|
||||
{"action": "double_click", "x": <int>, "y": <int>}
|
||||
{"action": "right_click", "x": <int>, "y": <int>}
|
||||
{"action": "drag", "from_x": <int>, "from_y": <int>, "to_x": <int>, "to_y": <int>, "duration_ms": 300}
|
||||
{"action": "type", "text": "<string>"}
|
||||
{"action": "hotkey", "keys": "<key1+key2+...>"}
|
||||
{"action": "scroll", "x": <int>, "y": <int>, "dy": <int>}
|
||||
{"action": "done"}
|
||||
|
||||
Use {"action": "done"} ONLY when the current state matches the target state.
|
||||
|
||||
RULES:
|
||||
- Output RAW JSON ONLY. No markdown, no explanation.
|
||||
- Use pixel coordinates from the CURRENT screenshot.
|
||||
- For drag: from_x/from_y is where you start pressing, to_x/to_y is where you release.
|
||||
- Prefer clicking on visible labeled controls over guessing coordinates.
|
||||
- If the target state is already achieved, output {"action": "done"}.
|
||||
- Never output any action not listed above.
|
||||
"""
|
||||
|
||||
_CHECK_PROMPT = """\
|
||||
Compare these two screenshots:
|
||||
1. CURRENT state
|
||||
2. TARGET end state
|
||||
|
||||
Has the current state reached the target end state?
|
||||
Answer with ONLY: {"reached": true} or {"reached": false, "reason": "<brief reason>"}
|
||||
"""
|
||||
|
||||
|
||||
# ── Image helpers ─────────────────────────────────────────────────────────────
|
||||
|
||||
def _screenshot_b64() -> str:
|
||||
"""Take a screenshot and return as base64 PNG string."""
|
||||
try:
|
||||
import mss
|
||||
from PIL import Image
|
||||
import io
|
||||
with mss.mss() as sct:
|
||||
monitor = sct.monitors[1]
|
||||
raw = sct.grab(monitor)
|
||||
img = Image.frombytes("RGB", raw.size, raw.bgra, "raw", "BGRX")
|
||||
buf = io.BytesIO()
|
||||
img.save(buf, format="PNG")
|
||||
return base64.b64encode(buf.getvalue()).decode("utf-8")
|
||||
except ImportError:
|
||||
raise ImportError("mss and Pillow required: pip install mss Pillow")
|
||||
|
||||
|
||||
def _file_to_b64(path: str) -> str:
|
||||
with open(path, "rb") as f:
|
||||
return base64.b64encode(f.read()).decode("utf-8")
|
||||
|
||||
|
||||
# ── Action executor ───────────────────────────────────────────────────────────
|
||||
|
||||
def _execute_action(action_dict: dict, context: BackendContext) -> None:
|
||||
"""Execute a single action returned by the model."""
|
||||
from cli_anything.macrocli.backends.visual_anchor import (
|
||||
_mouse_click, _mouse_drag, _require_pynput
|
||||
)
|
||||
|
||||
action = action_dict.get("action", "")
|
||||
|
||||
if action == "click":
|
||||
x, y = int(action_dict["x"]), int(action_dict["y"])
|
||||
_mouse_click(x, y, button=action_dict.get("button", "left"))
|
||||
|
||||
elif action == "double_click":
|
||||
x, y = int(action_dict["x"]), int(action_dict["y"])
|
||||
_mouse_click(x, y, double=True)
|
||||
|
||||
elif action == "right_click":
|
||||
x, y = int(action_dict["x"]), int(action_dict["y"])
|
||||
_mouse_click(x, y, button="right")
|
||||
|
||||
elif action == "type":
|
||||
text = action_dict.get("text", "")
|
||||
_, keyboard_mod = _require_pynput()
|
||||
ctrl = keyboard_mod.Controller()
|
||||
for char in text:
|
||||
ctrl.press(char)
|
||||
ctrl.release(char)
|
||||
time.sleep(0.03)
|
||||
|
||||
elif action == "hotkey":
|
||||
keys_str = action_dict.get("keys", "")
|
||||
_, keyboard_mod = _require_pynput()
|
||||
Key = keyboard_mod.Key
|
||||
ctrl = keyboard_mod.Controller()
|
||||
_KEY_MAP = {
|
||||
"ctrl": Key.ctrl, "shift": Key.shift, "alt": Key.alt,
|
||||
"enter": Key.enter, "tab": Key.tab, "esc": Key.esc,
|
||||
"escape": Key.esc, "space": Key.space, "backspace": Key.backspace,
|
||||
}
|
||||
keys = [_KEY_MAP.get(k.lower(), k) for k in keys_str.split("+")]
|
||||
for k in keys:
|
||||
ctrl.press(k)
|
||||
for k in reversed(keys):
|
||||
ctrl.release(k)
|
||||
|
||||
elif action == "scroll":
|
||||
x, y = int(action_dict["x"]), int(action_dict["y"])
|
||||
dy = int(action_dict.get("dy", -3))
|
||||
mouse_mod, _ = _require_pynput()
|
||||
ctrl = mouse_mod.Controller()
|
||||
ctrl.position = (x, y)
|
||||
ctrl.scroll(0, dy)
|
||||
|
||||
elif action == "drag":
|
||||
fx, fy = int(action_dict["from_x"]), int(action_dict["from_y"])
|
||||
tx, ty = int(action_dict["to_x"]), int(action_dict["to_y"])
|
||||
duration_ms = int(action_dict.get("duration_ms", 300))
|
||||
from cli_anything.macrocli.backends.visual_anchor import _mouse_drag
|
||||
_mouse_drag(fx, fy, tx, ty, duration_ms=duration_ms)
|
||||
|
||||
elif action == "done":
|
||||
pass # caller checks for done
|
||||
|
||||
else:
|
||||
raise ValueError(f"GUIAgentBackend: unknown action '{action}'")
|
||||
|
||||
|
||||
# ── Backend ───────────────────────────────────────────────────────────────────
|
||||
|
||||
class GUIAgentBackend(Backend):
|
||||
"""Execute GUI steps using a vision model (OpenAI-compatible API) to decide actions."""
|
||||
|
||||
name = "gui_agent"
|
||||
priority = 60 # between semantic_ui(50) and file_transform(70)
|
||||
|
||||
def execute(
|
||||
self, step: MacroStep, params: dict, context: BackendContext
|
||||
) -> StepResult:
|
||||
t0 = time.time()
|
||||
p = substitute(step.params, params)
|
||||
|
||||
if context.dry_run:
|
||||
return StepResult(
|
||||
success=True,
|
||||
output={"dry_run": True, "action": step.action},
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
if step.action == "instruct":
|
||||
return self._instruct(p, context, t0)
|
||||
elif step.action == "instruct_with_refine":
|
||||
return self._instruct_with_refine(p, context, t0)
|
||||
else:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"GUIAgentBackend: unknown action '{step.action}'. "
|
||||
"Supported: 'instruct', 'instruct_with_refine'.",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
def is_available(self) -> bool:
|
||||
try:
|
||||
import openai # noqa: F401
|
||||
import mss # noqa: F401
|
||||
return True
|
||||
except ImportError:
|
||||
return False
|
||||
|
||||
def _instruct(
|
||||
self, p: dict, context: BackendContext, t0: float
|
||||
) -> StepResult:
|
||||
"""Execute exactly ONE action decided by the vision model.
|
||||
|
||||
The macro author is responsible for:
|
||||
- focusing the target window before calling gui_agent
|
||||
- writing multiple gui_agent steps if multiple actions are needed
|
||||
- verifying the outcome via postconditions or subsequent steps
|
||||
|
||||
This step:
|
||||
1. Takes a screenshot
|
||||
2. Sends it + description + end_state_snapshot to the model
|
||||
3. Model returns one action (click/type/hotkey/scroll/done)
|
||||
4. Executes that action
|
||||
5. Returns success with the action taken
|
||||
"""
|
||||
description: str = p.get("description", "")
|
||||
end_state_desc: str = p.get("end_state_description", "")
|
||||
snapshot_path: str = p.get("end_state_snapshot", "")
|
||||
window_title: str = p.get("window_title", "") # focus this window first
|
||||
model_name: str = p.get("model", os.environ.get("MACROCLI_MODEL", ""))
|
||||
api_key: str = p.get("api_key", os.environ.get("MACROCLI_API_KEY", ""))
|
||||
base_url: str = p.get("base_url", os.environ.get("MACROCLI_BASE_URL", ""))
|
||||
|
||||
if not model_name:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error="GUIAgentBackend: model required. Set MACROCLI_MODEL env var or pass model in step params.",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
if not api_key:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error="GUIAgentBackend: api_key required. Set MACROCLI_API_KEY env var.",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
try:
|
||||
from openai import OpenAI
|
||||
except ImportError:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error="openai required: pip install openai",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
client_kwargs = {"api_key": api_key}
|
||||
if base_url:
|
||||
client_kwargs["base_url"] = base_url
|
||||
client = OpenAI(**client_kwargs)
|
||||
|
||||
def _call_model(messages: list, max_tokens: int = 1024) -> str:
|
||||
resp = client.chat.completions.create(
|
||||
model=model_name,
|
||||
messages=messages,
|
||||
max_tokens=max_tokens,
|
||||
)
|
||||
return resp.choices[0].message.content.strip()
|
||||
|
||||
def _extract_json(raw: str) -> dict:
|
||||
"""Extract JSON from model output robustly."""
|
||||
if raw.startswith("```"):
|
||||
raw = "\n".join(
|
||||
l for l in raw.split("\n") if not l.startswith("```")
|
||||
).strip()
|
||||
start = raw.find('{')
|
||||
end = raw.rfind('}')
|
||||
if start != -1 and end != -1:
|
||||
raw = raw[start:end+1]
|
||||
return json.loads(raw)
|
||||
|
||||
# Step 1: Focus the target window if specified
|
||||
if window_title and not context.dry_run:
|
||||
import shutil, subprocess
|
||||
env = os.environ.copy()
|
||||
if "DISPLAY" not in env:
|
||||
env["DISPLAY"] = ":0"
|
||||
if shutil.which("wmctrl"):
|
||||
subprocess.run(["wmctrl", "-a", window_title],
|
||||
capture_output=True, env=env)
|
||||
elif shutil.which("xdotool"):
|
||||
subprocess.run(
|
||||
["xdotool", "search", "--name", window_title,
|
||||
"windowfocus", "--sync"],
|
||||
capture_output=True, env=env
|
||||
)
|
||||
time.sleep(0.3)
|
||||
|
||||
if context.dry_run:
|
||||
return StepResult(
|
||||
success=True,
|
||||
output={"dry_run": True, "description": description},
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
# Step 2: Take screenshot
|
||||
current_b64 = _screenshot_b64()
|
||||
|
||||
# Step 3: Load end state snapshot if provided
|
||||
end_state_b64: Optional[str] = None
|
||||
if snapshot_path and Path(snapshot_path).is_file():
|
||||
end_state_b64 = _file_to_b64(snapshot_path)
|
||||
|
||||
# Step 4: Build prompt
|
||||
content = []
|
||||
content.append({"type": "text", "text": "CURRENT SCREEN STATE:"})
|
||||
content.append({
|
||||
"type": "image_url",
|
||||
"image_url": {"url": f"data:image/png;base64,{current_b64}"}
|
||||
})
|
||||
if end_state_b64:
|
||||
content.append({"type": "text", "text": "TARGET END STATE:"})
|
||||
content.append({
|
||||
"type": "image_url",
|
||||
"image_url": {"url": f"data:image/png;base64,{end_state_b64}"}
|
||||
})
|
||||
task_text = f"TASK: {description}"
|
||||
if end_state_desc:
|
||||
task_text += f"\nTARGET: {end_state_desc}"
|
||||
task_text += "\nOutput ONE action as JSON only."
|
||||
content.append({"type": "text", "text": task_text})
|
||||
|
||||
messages = [
|
||||
{"role": "system", "content": _SYSTEM_PROMPT},
|
||||
{"role": "user", "content": content},
|
||||
]
|
||||
|
||||
# Step 5: Ask model for one action
|
||||
try:
|
||||
raw = _call_model(messages)
|
||||
except Exception as exc:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"GUIAgentBackend: model error: {exc}",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
try:
|
||||
action_dict = _extract_json(raw)
|
||||
except json.JSONDecodeError:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"GUIAgentBackend: invalid JSON from model: {raw[:200]}",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
action_name = action_dict.get("action", "")
|
||||
print(f"[gui_agent] action: {action_dict}", flush=True)
|
||||
|
||||
# Step 6: Execute the action (unless model says done)
|
||||
if action_name != "done":
|
||||
try:
|
||||
_execute_action(action_dict, context)
|
||||
except Exception as exc:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"GUIAgentBackend: action execution failed: {exc}",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
time.sleep(0.5)
|
||||
|
||||
return StepResult(
|
||||
success=True,
|
||||
output={
|
||||
"action": action_dict,
|
||||
"done": action_name == "done",
|
||||
},
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
def _instruct_with_refine(
|
||||
self, p: dict, context: BackendContext, t0: float
|
||||
) -> StepResult:
|
||||
"""Execute one action, then compare result vs end_state_snapshot,
|
||||
and if needed undo and re-execute with a refined action.
|
||||
|
||||
Sends to model on refine:
|
||||
- Screenshot BEFORE first action (original state)
|
||||
- The first action that was taken
|
||||
- Screenshot AFTER first action (current result)
|
||||
- end_state_snapshot (target state)
|
||||
- Request for corrected action
|
||||
|
||||
This allows the model to see exactly what went wrong and correct it.
|
||||
"""
|
||||
snapshot_path: str = p.get("end_state_snapshot", "")
|
||||
|
||||
if not snapshot_path or not Path(snapshot_path).is_file():
|
||||
# No end state snapshot → fall back to single instruct
|
||||
return self._instruct(p, context, t0)
|
||||
|
||||
# ── Round 1: initial action ───────────────────────────────────────────
|
||||
before_b64 = _screenshot_b64()
|
||||
result1 = self._instruct(p, context, t0)
|
||||
|
||||
if not result1.success:
|
||||
return result1
|
||||
|
||||
first_action = result1.output.get("action", {})
|
||||
if first_action.get("action") == "done":
|
||||
return result1
|
||||
|
||||
time.sleep(0.5)
|
||||
after_b64 = _screenshot_b64()
|
||||
end_state_b64 = _file_to_b64(snapshot_path)
|
||||
|
||||
# ── Round 2: compare and refine ───────────────────────────────────────
|
||||
description: str = p.get("description", "")
|
||||
end_state_desc: str = p.get("end_state_description", "")
|
||||
model_name: str = p.get("model", os.environ.get("MACROCLI_MODEL", ""))
|
||||
api_key: str = p.get("api_key", os.environ.get("MACROCLI_API_KEY", ""))
|
||||
base_url: str = p.get("base_url", os.environ.get("MACROCLI_BASE_URL", ""))
|
||||
|
||||
from openai import OpenAI
|
||||
client_kwargs = {"api_key": api_key}
|
||||
if base_url:
|
||||
client_kwargs["base_url"] = base_url
|
||||
client = OpenAI(**client_kwargs)
|
||||
|
||||
def _call(messages):
|
||||
resp = client.chat.completions.create(
|
||||
model=model_name, messages=messages, max_tokens=1024,
|
||||
)
|
||||
return resp.choices[0].message.content.strip()
|
||||
|
||||
def _extract_json(raw):
|
||||
if raw.startswith("```"):
|
||||
raw = "\n".join(l for l in raw.split("\n") if not l.startswith("```")).strip()
|
||||
s, e = raw.find('{'), raw.rfind('}')
|
||||
if s != -1 and e != -1:
|
||||
raw = raw[s:e+1]
|
||||
return json.loads(raw)
|
||||
|
||||
refine_prompt = f"""You are refining a GUI automation action.
|
||||
|
||||
ORIGINAL TASK: {description}
|
||||
TARGET STATE: {end_state_desc}
|
||||
|
||||
WHAT HAPPENED:
|
||||
- First action taken: {json.dumps(first_action)}
|
||||
|
||||
Now compare these three screenshots:
|
||||
|
||||
1. BEFORE (original state before any action):
|
||||
2. AFTER FIRST ACTION (current result):
|
||||
3. TARGET END STATE (what it should look like):
|
||||
|
||||
The first action was not quite right. Looking at:
|
||||
- Where the rectangle was drawn vs where it should be
|
||||
- The difference between AFTER and TARGET
|
||||
|
||||
Provide a corrected drag action with better coordinates.
|
||||
Output ONE JSON action only."""
|
||||
|
||||
content = [
|
||||
{"type": "text", "text": refine_prompt},
|
||||
{"type": "text", "text": "BEFORE:"},
|
||||
{"type": "image_url", "image_url": {"url": f"data:image/png;base64,{before_b64}"}},
|
||||
{"type": "text", "text": "AFTER FIRST ACTION:"},
|
||||
{"type": "image_url", "image_url": {"url": f"data:image/png;base64,{after_b64}"}},
|
||||
{"type": "text", "text": "TARGET END STATE:"},
|
||||
{"type": "image_url", "image_url": {"url": f"data:image/png;base64,{end_state_b64}"}},
|
||||
]
|
||||
|
||||
messages = [
|
||||
{"role": "system", "content": _SYSTEM_PROMPT},
|
||||
{"role": "user", "content": content},
|
||||
]
|
||||
|
||||
try:
|
||||
raw = _call(messages)
|
||||
refined_action = _extract_json(raw)
|
||||
print(f"[gui_agent] refined action: {refined_action}", flush=True)
|
||||
except Exception as exc:
|
||||
# Refine failed, return original result
|
||||
print(f"[gui_agent] refine failed: {exc}, keeping original", flush=True)
|
||||
return result1
|
||||
|
||||
if refined_action.get("action") == "done":
|
||||
return result1
|
||||
|
||||
# Undo the first action, then execute the refined one
|
||||
import shutil, subprocess as sp
|
||||
env = os.environ.copy()
|
||||
if "DISPLAY" not in env:
|
||||
env["DISPLAY"] = ":0"
|
||||
if shutil.which("xdotool"):
|
||||
sp.run(["xdotool", "key", "ctrl+z"], env=env)
|
||||
time.sleep(0.3)
|
||||
|
||||
try:
|
||||
_execute_action(refined_action, context)
|
||||
except Exception as exc:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"GUIAgentBackend: refine action failed: {exc}",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
time.sleep(0.5)
|
||||
|
||||
return StepResult(
|
||||
success=True,
|
||||
output={
|
||||
"action": refined_action,
|
||||
"first_action": first_action,
|
||||
"refined": True,
|
||||
"done": False,
|
||||
},
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
@@ -0,0 +1,205 @@
|
||||
"""GUIMacroBackend — replay precompiled coordinate-based macro sequences.
|
||||
|
||||
A compiled macro is a JSON blob describing an exact sequence of mouse clicks,
|
||||
key presses, and wait conditions. These are fast to execute but fragile to
|
||||
layout changes.
|
||||
|
||||
Compiled macro format (stored separately, referenced by step params):
|
||||
|
||||
{
|
||||
"version": 1,
|
||||
"screen_resolution": "1920x1080",
|
||||
"layout_hash": "abc123",
|
||||
"steps": [
|
||||
{"type": "click", "x": 100, "y": 200, "button": "left", "delay_ms": 200},
|
||||
{"type": "key", "keys": "ctrl+s", "delay_ms": 100},
|
||||
{"type": "type", "text": "output.png", "delay_ms": 50},
|
||||
{"type": "wait_file", "path": "/tmp/out.png", "timeout_ms": 5000},
|
||||
{"type": "sleep", "ms": 500}
|
||||
]
|
||||
}
|
||||
|
||||
Example macro step:
|
||||
|
||||
- backend: gui_macro
|
||||
action: replay
|
||||
params:
|
||||
macro_file: macros/compiled/export_png.json
|
||||
layout_strict: false # if true, fail when screen res changes
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
from cli_anything.macrocli.backends.base import Backend, BackendContext, StepResult
|
||||
from cli_anything.macrocli.core.macro_model import MacroStep, substitute
|
||||
|
||||
|
||||
class GUIMacroBackend(Backend):
|
||||
"""Replay precompiled GUI automation sequences."""
|
||||
|
||||
name = "gui_macro"
|
||||
priority = 80
|
||||
|
||||
def execute(self, step: MacroStep, params: dict, context: BackendContext) -> StepResult:
|
||||
t0 = time.time()
|
||||
action = step.action
|
||||
step_params = substitute(step.params, params)
|
||||
|
||||
if context.dry_run:
|
||||
return StepResult(
|
||||
success=True,
|
||||
output={"dry_run": True, "action": action},
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
if action != "replay":
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"GUIMacroBackend: unknown action '{action}'. Expected 'replay'.",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
return self._replay(step_params, context, t0)
|
||||
|
||||
def is_available(self) -> bool:
|
||||
"""Available when at least one automation library is present."""
|
||||
for lib in ("pyautogui", "pynput"):
|
||||
try:
|
||||
__import__(lib)
|
||||
return True
|
||||
except ImportError:
|
||||
pass
|
||||
return False
|
||||
|
||||
def _replay(self, p: dict, context: BackendContext, t0: float) -> StepResult:
|
||||
"""Load and replay a compiled macro file."""
|
||||
macro_file = p.get("macro_file", "")
|
||||
if not macro_file:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error="GUIMacroBackend.replay: 'macro_file' param is required.",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
macro_path = Path(macro_file)
|
||||
if not macro_path.is_file():
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"GUIMacroBackend: compiled macro not found: {macro_file}",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
try:
|
||||
with open(macro_path, encoding="utf-8") as f:
|
||||
macro_blob = json.load(f)
|
||||
except Exception as exc:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"GUIMacroBackend: failed to load macro file: {exc}",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
layout_strict: bool = p.get("layout_strict", False)
|
||||
if layout_strict:
|
||||
check = self._check_layout(macro_blob)
|
||||
if check:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"GUIMacroBackend: layout mismatch — {check}",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
try:
|
||||
steps_run = self._execute_steps(macro_blob.get("steps", []), context)
|
||||
return StepResult(
|
||||
success=True,
|
||||
output={"steps_executed": steps_run, "macro_file": macro_file},
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
except Exception as exc:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"GUIMacroBackend.replay: {exc}",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
def _check_layout(self, macro_blob: dict) -> str:
|
||||
"""Return error string if current screen doesn't match expected."""
|
||||
expected_res = macro_blob.get("screen_resolution", "")
|
||||
if not expected_res:
|
||||
return ""
|
||||
try:
|
||||
import pyautogui
|
||||
w, h = pyautogui.size()
|
||||
current_res = f"{w}x{h}"
|
||||
if current_res != expected_res:
|
||||
return f"screen is {current_res}, macro expects {expected_res}"
|
||||
except ImportError:
|
||||
pass # Can't verify — allow through
|
||||
return ""
|
||||
|
||||
def _execute_steps(self, steps: list, context: BackendContext) -> int:
|
||||
"""Execute each step in the compiled macro."""
|
||||
try:
|
||||
import pyautogui
|
||||
has_pyautogui = True
|
||||
except ImportError:
|
||||
has_pyautogui = False
|
||||
|
||||
count = 0
|
||||
for s in steps:
|
||||
stype = s.get("type", "")
|
||||
delay = s.get("delay_ms", 100) / 1000.0
|
||||
|
||||
if stype == "click":
|
||||
if not has_pyautogui:
|
||||
raise ImportError("pyautogui required for click steps. pip install pyautogui")
|
||||
button = s.get("button", "left")
|
||||
pyautogui.click(s["x"], s["y"], button=button)
|
||||
|
||||
elif stype == "key":
|
||||
if not has_pyautogui:
|
||||
raise ImportError("pyautogui required for key steps. pip install pyautogui")
|
||||
keys = s.get("keys", "").split("+")
|
||||
if len(keys) == 1:
|
||||
pyautogui.press(keys[0])
|
||||
else:
|
||||
pyautogui.hotkey(*keys)
|
||||
|
||||
elif stype == "type":
|
||||
if not has_pyautogui:
|
||||
raise ImportError("pyautogui required for type steps. pip install pyautogui")
|
||||
pyautogui.typewrite(s.get("text", ""), interval=0.03)
|
||||
|
||||
elif stype == "wait_file":
|
||||
deadline = time.time() + s.get("timeout_ms", 5000) / 1000.0
|
||||
path = s.get("path", "")
|
||||
while time.time() < deadline:
|
||||
if Path(path).exists():
|
||||
break
|
||||
time.sleep(0.1)
|
||||
else:
|
||||
raise TimeoutError(f"wait_file timed out: {path}")
|
||||
delay = 0 # no additional delay after file wait
|
||||
|
||||
elif stype == "sleep":
|
||||
time.sleep(s.get("ms", 500) / 1000.0)
|
||||
delay = 0
|
||||
|
||||
if delay > 0:
|
||||
time.sleep(delay)
|
||||
count += 1
|
||||
|
||||
return count
|
||||
@@ -0,0 +1,246 @@
|
||||
"""NativeAPIBackend — executes macro steps via subprocess.
|
||||
|
||||
Supports these action types (configured in macro step params):
|
||||
|
||||
action: run_command
|
||||
params:
|
||||
command: [inkscape, --export-filename, /tmp/out.png, input.svg]
|
||||
cwd: /optional/working/dir # optional
|
||||
env: {KEY: value} # optional extra env vars
|
||||
capture_stdout: true # store stdout in output.stdout
|
||||
|
||||
action: find_executable
|
||||
params:
|
||||
name: inkscape
|
||||
candidates: [inkscape, inkscape-1.0, /usr/bin/inkscape]
|
||||
install_hint: "apt install inkscape"
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
from cli_anything.macrocli.backends.base import Backend, BackendContext, StepResult
|
||||
from cli_anything.macrocli.core.macro_model import MacroStep, substitute
|
||||
|
||||
|
||||
class NativeAPIBackend(Backend):
|
||||
"""Execute a macro step by running an external command."""
|
||||
|
||||
name = "native_api"
|
||||
priority = 100
|
||||
|
||||
def execute(self, step: MacroStep, params: dict, context: BackendContext) -> StepResult:
|
||||
t0 = time.time()
|
||||
action = step.action
|
||||
|
||||
if action == "find_executable":
|
||||
return self._find_executable(step, params, context, t0)
|
||||
elif action == "run_command":
|
||||
return self._run_command(step, params, context, t0)
|
||||
elif action == "start_process":
|
||||
return self._start_process(step, params, context, t0)
|
||||
else:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"NativeAPIBackend: unknown action '{action}'.",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
# ── Actions ──────────────────────────────────────────────────────────
|
||||
|
||||
def _find_executable(
|
||||
self, step: MacroStep, params: dict, context: BackendContext, t0: float
|
||||
) -> StepResult:
|
||||
"""Check that an executable exists; return its path."""
|
||||
step_params = substitute(step.params, params)
|
||||
exe_name = step_params.get("name", "")
|
||||
candidates: list[str] = step_params.get("candidates", [exe_name] if exe_name else [])
|
||||
install_hint: str = step_params.get("install_hint", f"Install {exe_name}")
|
||||
|
||||
for candidate in candidates:
|
||||
found = shutil.which(candidate)
|
||||
if found:
|
||||
return StepResult(
|
||||
success=True,
|
||||
output={"executable": found, "name": candidate},
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=(
|
||||
f"Executable not found: {exe_name}. "
|
||||
f"Tried: {candidates}. "
|
||||
f"Install with: {install_hint}"
|
||||
),
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
def _run_command(
|
||||
self, step: MacroStep, params: dict, context: BackendContext, t0: float
|
||||
) -> StepResult:
|
||||
"""Run an external command."""
|
||||
step_params = substitute(step.params, params)
|
||||
command: list[str] = step_params.get("command", [])
|
||||
if not command:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error="NativeAPIBackend.run_command: 'command' param is required.",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
if isinstance(command, str):
|
||||
import shlex
|
||||
command = shlex.split(command)
|
||||
command = [str(c) for c in command]
|
||||
|
||||
cwd: str = step_params.get("cwd", "")
|
||||
extra_env: dict = step_params.get("env", {})
|
||||
capture_stdout: bool = step_params.get("capture_stdout", False)
|
||||
|
||||
env = os.environ.copy()
|
||||
if extra_env:
|
||||
env.update({k: str(v) for k, v in extra_env.items()})
|
||||
|
||||
timeout_s = context.timeout_ms / 1000.0
|
||||
|
||||
if context.dry_run:
|
||||
return StepResult(
|
||||
success=True,
|
||||
output={"dry_run": True, "command": command},
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
command,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=timeout_s,
|
||||
cwd=cwd or None,
|
||||
env=env,
|
||||
)
|
||||
except FileNotFoundError as exc:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"Command not found: {command[0]}. {exc}",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
except subprocess.TimeoutExpired:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"Command timed out after {timeout_s:.0f}s: {command}",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
duration = (time.time() - t0) * 1000
|
||||
if result.returncode != 0:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=(
|
||||
f"Command failed (exit {result.returncode}): {command}\n"
|
||||
f"stderr: {result.stderr.strip()}"
|
||||
),
|
||||
output={"returncode": result.returncode, "stderr": result.stderr},
|
||||
backend_used=self.name,
|
||||
duration_ms=duration,
|
||||
)
|
||||
|
||||
output: dict[str, Any] = {"returncode": 0}
|
||||
if capture_stdout:
|
||||
output["stdout"] = result.stdout
|
||||
return StepResult(
|
||||
success=True,
|
||||
output=output,
|
||||
backend_used=self.name,
|
||||
duration_ms=duration,
|
||||
)
|
||||
|
||||
def _start_process(
|
||||
self, step: MacroStep, params: dict, context: BackendContext, t0: float
|
||||
) -> StepResult:
|
||||
"""Launch a GUI application in the background without waiting for it to exit.
|
||||
|
||||
Use this instead of run_command for GUI apps like gedit, inkscape, etc.
|
||||
The process is detached immediately after launch.
|
||||
|
||||
Params:
|
||||
command: list[str] — the command to run
|
||||
cwd: str — working directory (optional)
|
||||
env: dict — extra environment variables (optional)
|
||||
log_file: str — redirect stdout+stderr here (default /dev/null)
|
||||
"""
|
||||
import subprocess
|
||||
step_params = substitute(step.params, params)
|
||||
command: list[str] = step_params.get("command", [])
|
||||
if not command:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error="NativeAPIBackend.start_process: 'command' param is required.",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
if isinstance(command, str):
|
||||
import shlex
|
||||
command = shlex.split(command)
|
||||
command = [str(c) for c in command]
|
||||
|
||||
cwd: str = step_params.get("cwd", "")
|
||||
extra_env: dict = step_params.get("env", {})
|
||||
log_file: str = step_params.get("log_file", "/dev/null")
|
||||
|
||||
env = os.environ.copy()
|
||||
if extra_env:
|
||||
env.update({k: str(v) for k, v in extra_env.items()})
|
||||
|
||||
if context.dry_run:
|
||||
return StepResult(
|
||||
success=True,
|
||||
output={"dry_run": True, "command": command},
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
try:
|
||||
with open(log_file, "a") as log:
|
||||
proc = subprocess.Popen(
|
||||
command,
|
||||
stdout=log,
|
||||
stderr=log,
|
||||
cwd=cwd or None,
|
||||
env=env,
|
||||
start_new_session=True, # detach from current process group
|
||||
)
|
||||
return StepResult(
|
||||
success=True,
|
||||
output={"pid": proc.pid, "command": command},
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
except FileNotFoundError as exc:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"Command not found: {command[0]}. {exc}",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
except Exception as exc:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"start_process failed: {exc}",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
@@ -0,0 +1,128 @@
|
||||
"""RecoveryBackend — retry and fallback orchestration.
|
||||
|
||||
This backend wraps another backend and retries failed steps with exponential
|
||||
backoff. It can also fall back to an alternative backend on exhausted retries.
|
||||
|
||||
Example macro step using recovery explicitly:
|
||||
|
||||
- backend: recovery
|
||||
action: retry_with_fallback
|
||||
params:
|
||||
primary_backend: native_api
|
||||
fallback_backend: file_transform
|
||||
max_retries: 3
|
||||
backoff_ms: [1000, 2000, 5000]
|
||||
step:
|
||||
action: run_command
|
||||
params:
|
||||
command: [inkscape, --export-filename, ${output}, input.svg]
|
||||
|
||||
The MacroRuntime also uses the RecoveryBackend automatically when a step
|
||||
specifies retry_max > 0 in the macro definition.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
|
||||
from cli_anything.macrocli.backends.base import Backend, BackendContext, StepResult
|
||||
from cli_anything.macrocli.core.macro_model import MacroStep, substitute
|
||||
|
||||
|
||||
class RecoveryBackend(Backend):
|
||||
"""Retry and fallback orchestration backend."""
|
||||
|
||||
name = "recovery"
|
||||
priority = 10 # lowest — last resort
|
||||
|
||||
def __init__(self, backends: dict[str, "Backend"] | None = None):
|
||||
"""
|
||||
Args:
|
||||
backends: Dict of backend_name -> Backend instance.
|
||||
Injected by the RoutingEngine at runtime.
|
||||
"""
|
||||
self._backends = backends or {}
|
||||
|
||||
def register_backend(self, backend: "Backend") -> None:
|
||||
self._backends[backend.name] = backend
|
||||
|
||||
def execute(self, step: MacroStep, params: dict, context: BackendContext) -> StepResult:
|
||||
t0 = time.time()
|
||||
action = step.action
|
||||
|
||||
if action not in ("retry", "retry_with_fallback"):
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"RecoveryBackend: unknown action '{action}'. "
|
||||
"Use 'retry' or 'retry_with_fallback'.",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
step_params = substitute(step.params, params)
|
||||
inner_step_raw = step_params.get("step", {})
|
||||
if not inner_step_raw:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error="RecoveryBackend: 'step' param is required.",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
# Build an inner MacroStep from the nested definition
|
||||
inner_step = MacroStep(
|
||||
id=inner_step_raw.get("id", "recovery_inner"),
|
||||
backend=step_params.get("primary_backend", inner_step_raw.get("backend", "native_api")),
|
||||
action=inner_step_raw.get("action", ""),
|
||||
params=inner_step_raw.get("params", {}),
|
||||
timeout_ms=context.timeout_ms,
|
||||
)
|
||||
|
||||
max_retries: int = int(step_params.get("max_retries", step.retry_max or 2))
|
||||
backoff_ms: list[int] = step_params.get("backoff_ms", [1000, 2000, 5000])
|
||||
fallback_name: str = step_params.get("fallback_backend", "")
|
||||
|
||||
last_result = None
|
||||
for attempt in range(max_retries + 1):
|
||||
backend = self._backends.get(inner_step.backend)
|
||||
if backend is None:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"RecoveryBackend: backend '{inner_step.backend}' not registered.",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
last_result = backend.execute(inner_step, params, context)
|
||||
if last_result.success:
|
||||
last_result.backend_used = f"{self.name}({inner_step.backend}, attempt={attempt + 1})"
|
||||
return last_result
|
||||
|
||||
# Failed — decide whether to retry or fall back
|
||||
if attempt < max_retries:
|
||||
wait = backoff_ms[min(attempt, len(backoff_ms) - 1)] / 1000.0
|
||||
time.sleep(wait)
|
||||
elif fallback_name and fallback_name in self._backends:
|
||||
# Switch to fallback backend for one final attempt
|
||||
inner_step = MacroStep(
|
||||
id=inner_step.id,
|
||||
backend=fallback_name,
|
||||
action=inner_step.action,
|
||||
params=inner_step.params,
|
||||
timeout_ms=inner_step.timeout_ms,
|
||||
)
|
||||
fallback_result = self._backends[fallback_name].execute(inner_step, params, context)
|
||||
if fallback_result.success:
|
||||
fallback_result.backend_used = f"{self.name}(fallback={fallback_name})"
|
||||
return fallback_result
|
||||
last_result = fallback_result
|
||||
|
||||
if last_result is None:
|
||||
last_result = StepResult(
|
||||
success=False,
|
||||
error="RecoveryBackend: no attempts made.",
|
||||
backend_used=self.name,
|
||||
)
|
||||
last_result.duration_ms = (time.time() - t0) * 1000
|
||||
last_result.backend_used = self.name
|
||||
return last_result
|
||||
@@ -0,0 +1,587 @@
|
||||
"""SemanticUIBackend — drive applications via accessibility APIs and keyboard shortcuts.
|
||||
|
||||
Backends by platform:
|
||||
Linux: AT-SPI via python3-pyatspi (apt install python3-pyatspi)
|
||||
Fallback: xdotool for keyboard/shortcuts
|
||||
macOS: ApplicationServices / Quartz via pyobjc
|
||||
Fallback: osascript (AppleScript)
|
||||
Windows: UI Automation via pywinauto (pip install pywinauto)
|
||||
|
||||
Action space:
|
||||
|
||||
shortcut — send keyboard shortcut to focused window
|
||||
type_text — type text into focused control
|
||||
menu_click — activate a menu item by path
|
||||
button_click — click a button by label/role
|
||||
wait_for_window — wait for a window with given title to appear
|
||||
focus_window — bring a window to foreground
|
||||
get_controls — list interactive controls (for discovery)
|
||||
|
||||
Example YAML steps:
|
||||
|
||||
- backend: semantic_ui
|
||||
action: menu_click
|
||||
params:
|
||||
menu_path: [File, Export As, PNG Image]
|
||||
|
||||
- backend: semantic_ui
|
||||
action: shortcut
|
||||
params:
|
||||
keys: ctrl+shift+e
|
||||
|
||||
- backend: semantic_ui
|
||||
action: wait_for_window
|
||||
params:
|
||||
title_contains: Export
|
||||
timeout_ms: 5000
|
||||
|
||||
- backend: semantic_ui
|
||||
action: button_click
|
||||
params:
|
||||
label: OK
|
||||
|
||||
- backend: semantic_ui
|
||||
action: focus_window
|
||||
params:
|
||||
title_contains: Inkscape
|
||||
|
||||
- backend: semantic_ui
|
||||
action: get_controls
|
||||
params:
|
||||
window_title: Inkscape
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import platform
|
||||
import shutil
|
||||
import subprocess
|
||||
import time
|
||||
from typing import Optional
|
||||
|
||||
from cli_anything.macrocli.backends.base import Backend, BackendContext, StepResult
|
||||
from cli_anything.macrocli.core.macro_model import MacroStep, substitute
|
||||
|
||||
_SYSTEM = platform.system()
|
||||
|
||||
|
||||
def _x_env() -> dict:
|
||||
"""Return env dict with DISPLAY set, for subprocess calls to X tools."""
|
||||
env = os.environ.copy()
|
||||
if "DISPLAY" not in env:
|
||||
env["DISPLAY"] = ":0"
|
||||
return env
|
||||
|
||||
|
||||
# ── AT-SPI helpers (Linux) ────────────────────────────────────────────────────
|
||||
|
||||
def _atspi_available() -> bool:
|
||||
try:
|
||||
import pyatspi # noqa: F401
|
||||
return True
|
||||
except ImportError:
|
||||
return False
|
||||
|
||||
|
||||
def _atspi_find_app(name_fragment: str):
|
||||
"""Return the first AT-SPI application matching name_fragment."""
|
||||
import pyatspi
|
||||
desktop = pyatspi.Registry.getDesktop(0)
|
||||
for app in desktop:
|
||||
if app and name_fragment.lower() in (app.name or "").lower():
|
||||
return app
|
||||
return None
|
||||
|
||||
|
||||
def _atspi_find_control(root, role_name: str, label_fragment: str, max_depth: int = 20):
|
||||
"""BFS search for a control by role and label."""
|
||||
import pyatspi
|
||||
role_map = {
|
||||
"button": pyatspi.ROLE_PUSH_BUTTON,
|
||||
"menu": pyatspi.ROLE_MENU,
|
||||
"menu_item": pyatspi.ROLE_MENU_ITEM,
|
||||
"menu_bar": pyatspi.ROLE_MENU_BAR,
|
||||
"text": pyatspi.ROLE_TEXT,
|
||||
"combo_box": pyatspi.ROLE_COMBO_BOX,
|
||||
"check_box": pyatspi.ROLE_CHECK_BOX,
|
||||
"radio": pyatspi.ROLE_RADIO_BUTTON,
|
||||
"list_item": pyatspi.ROLE_LIST_ITEM,
|
||||
"dialog": pyatspi.ROLE_DIALOG,
|
||||
"window": pyatspi.ROLE_FRAME,
|
||||
}
|
||||
target_role = role_map.get(role_name.lower())
|
||||
|
||||
from collections import deque
|
||||
queue = deque([(root, 0)])
|
||||
while queue:
|
||||
node, depth = queue.popleft()
|
||||
if depth > max_depth:
|
||||
continue
|
||||
try:
|
||||
node_role = node.getRole()
|
||||
node_name = node.name or ""
|
||||
if (target_role is None or node_role == target_role):
|
||||
if label_fragment.lower() in node_name.lower():
|
||||
return node
|
||||
for i in range(node.childCount):
|
||||
child = node.getChildAtIndex(i)
|
||||
if child:
|
||||
queue.append((child, depth + 1))
|
||||
except Exception:
|
||||
continue
|
||||
return None
|
||||
|
||||
|
||||
def _atspi_menu_path(app, menu_path: list[str]):
|
||||
"""Navigate a menu path and activate the final item."""
|
||||
import pyatspi
|
||||
|
||||
# Find the menu bar
|
||||
menu_bar = _atspi_find_control(app, "menu_bar", "", max_depth=3)
|
||||
if menu_bar is None:
|
||||
raise RuntimeError("AT-SPI: menu bar not found in application.")
|
||||
|
||||
current = menu_bar
|
||||
for label in menu_path:
|
||||
item = _atspi_find_control(current, "menu", label)
|
||||
if item is None:
|
||||
item = _atspi_find_control(current, "menu_item", label)
|
||||
if item is None:
|
||||
raise RuntimeError(f"AT-SPI: menu item '{label}' not found.")
|
||||
# Activate / click
|
||||
try:
|
||||
action = item.queryAction()
|
||||
for i in range(action.nActions):
|
||||
if action.getName(i).lower() in ("click", "activate", "open"):
|
||||
action.doAction(i)
|
||||
break
|
||||
except Exception:
|
||||
pass
|
||||
current = item
|
||||
time.sleep(0.15)
|
||||
|
||||
return True
|
||||
|
||||
|
||||
# ── xdotool helpers (Linux fallback) ─────────────────────────────────────────
|
||||
|
||||
def _xdotool_key(keys: str) -> None:
|
||||
if not shutil.which("xdotool"):
|
||||
raise RuntimeError("xdotool not found. Install with: apt install xdotool")
|
||||
# ctrl+shift+e → ctrl+shift+e (xdotool accepts this format directly)
|
||||
subprocess.run(["xdotool", "key", "--clearmodifiers", keys], check=True, env=_x_env())
|
||||
|
||||
|
||||
def _xdotool_type(text: str) -> None:
|
||||
if not shutil.which("xdotool"):
|
||||
raise RuntimeError("xdotool not found. Install with: apt install xdotool")
|
||||
subprocess.run(["xdotool", "type", "--clearmodifiers", "--delay", "30", text], check=True, env=_x_env())
|
||||
|
||||
|
||||
def _xdotool_focus(title: str) -> None:
|
||||
if not shutil.which("xdotool"):
|
||||
raise RuntimeError("xdotool not found. Install with: apt install xdotool")
|
||||
subprocess.run(
|
||||
["xdotool", "search", "--name", title, "windowfocus", "--sync"],
|
||||
check=True, env=_x_env(),
|
||||
)
|
||||
|
||||
|
||||
# ── osascript helpers (macOS) ─────────────────────────────────────────────────
|
||||
|
||||
def _osascript(script: str) -> str:
|
||||
r = subprocess.run(
|
||||
["osascript", "-e", script], capture_output=True, text=True
|
||||
)
|
||||
if r.returncode != 0:
|
||||
raise RuntimeError(f"osascript failed: {r.stderr.strip()}")
|
||||
return r.stdout.strip()
|
||||
|
||||
|
||||
def _macos_menu_click(app_name: str, menu_path: list[str]) -> None:
|
||||
if len(menu_path) < 2:
|
||||
raise ValueError("menu_path needs at least 2 elements (menu name + item).")
|
||||
menu_name = menu_path[0]
|
||||
items = menu_path[1:]
|
||||
# Build nested AppleScript path
|
||||
item_script = " of menu ".join(
|
||||
[f'menu item "{i}"' for i in reversed(items)]
|
||||
)
|
||||
script = f"""
|
||||
tell application "{app_name}"
|
||||
activate
|
||||
end tell
|
||||
tell application "System Events"
|
||||
tell process "{app_name}"
|
||||
click {item_script} of menu "{menu_name}" of menu bar 1
|
||||
end tell
|
||||
end tell
|
||||
"""
|
||||
_osascript(script)
|
||||
|
||||
|
||||
# ── pywinauto helpers (Windows) ───────────────────────────────────────────────
|
||||
|
||||
def _win_find_app(title_fragment: str):
|
||||
from pywinauto import Application, findwindows
|
||||
handles = findwindows.find_windows(title_re=f".*{title_fragment}.*")
|
||||
if not handles:
|
||||
raise RuntimeError(f"Window not found: '{title_fragment}'")
|
||||
app = Application().connect(handle=handles[0])
|
||||
return app.window(handle=handles[0])
|
||||
|
||||
|
||||
# ── Backend ───────────────────────────────────────────────────────────────────
|
||||
|
||||
class SemanticUIBackend(Backend):
|
||||
"""Drive applications through semantic (accessibility) controls."""
|
||||
|
||||
name = "semantic_ui"
|
||||
priority = 50
|
||||
|
||||
def execute(self, step: MacroStep, params: dict, context: BackendContext) -> StepResult:
|
||||
t0 = time.time()
|
||||
action = step.action
|
||||
p = substitute(step.params, params)
|
||||
|
||||
if context.dry_run:
|
||||
return StepResult(
|
||||
success=True,
|
||||
output={"dry_run": True, "action": action},
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
dispatch = {
|
||||
"shortcut": self._shortcut,
|
||||
"type_text": self._type_text,
|
||||
"menu_click": self._menu_click,
|
||||
"button_click": self._button_click,
|
||||
"wait_for_window": self._wait_for_window,
|
||||
"focus_window": self._focus_window,
|
||||
"get_controls": self._get_controls,
|
||||
}
|
||||
|
||||
handler = dispatch.get(action)
|
||||
if handler is None:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"SemanticUIBackend: unknown action '{action}'. "
|
||||
f"Available: {sorted(dispatch)}",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
try:
|
||||
output = handler(p, context)
|
||||
return StepResult(
|
||||
success=True,
|
||||
output=output or {},
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
except Exception as exc:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"SemanticUIBackend.{action}: {exc}",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
def is_available(self) -> bool:
|
||||
if _SYSTEM == "Linux":
|
||||
return _atspi_available() or bool(shutil.which("xdotool"))
|
||||
elif _SYSTEM == "Darwin":
|
||||
return bool(shutil.which("osascript"))
|
||||
elif _SYSTEM == "Windows":
|
||||
try:
|
||||
import pywinauto # noqa: F401
|
||||
return True
|
||||
except ImportError:
|
||||
return False
|
||||
return False
|
||||
|
||||
# ── shortcut ─────────────────────────────────────────────────────────────
|
||||
|
||||
def _shortcut(self, p: dict, context: BackendContext) -> dict:
|
||||
keys: str = p.get("keys", "")
|
||||
if not keys:
|
||||
raise ValueError("shortcut requires 'keys' param.")
|
||||
|
||||
if _SYSTEM == "Linux":
|
||||
_xdotool_key(keys)
|
||||
return {"keys": keys, "method": "xdotool"}
|
||||
|
||||
elif _SYSTEM == "Darwin":
|
||||
# Use pynput (cross-platform) or AppleScript key code
|
||||
from cli_anything.macrocli.backends.visual_anchor import VisualAnchorBackend
|
||||
from cli_anything.macrocli.core.macro_model import MacroStep as MS
|
||||
va = VisualAnchorBackend()
|
||||
step = MS(id="x", backend="visual_anchor", action="hotkey", params={"keys": keys})
|
||||
result = va._hotkey({"keys": keys}, context)
|
||||
return result
|
||||
|
||||
elif _SYSTEM == "Windows":
|
||||
import pywinauto.keyboard as kb
|
||||
# Convert ctrl+s → {VK_CONTROL}s
|
||||
kb.send_keys(keys.replace("+", ""))
|
||||
return {"keys": keys, "method": "pywinauto"}
|
||||
|
||||
raise NotImplementedError(f"shortcut not implemented for {_SYSTEM}")
|
||||
|
||||
# ── type_text ─────────────────────────────────────────────────────────────
|
||||
|
||||
def _type_text(self, p: dict, context: BackendContext) -> dict:
|
||||
text: str = p.get("text", "")
|
||||
if not text:
|
||||
raise ValueError("type_text requires 'text' param.")
|
||||
|
||||
if _SYSTEM == "Linux":
|
||||
_xdotool_type(text)
|
||||
return {"typed": len(text), "method": "xdotool"}
|
||||
|
||||
# macOS / Windows: fall through to visual_anchor type_text
|
||||
from cli_anything.macrocli.backends.visual_anchor import VisualAnchorBackend
|
||||
va = VisualAnchorBackend()
|
||||
return va._type_text(p, context)
|
||||
|
||||
# ── menu_click ────────────────────────────────────────────────────────────
|
||||
|
||||
def _menu_click(self, p: dict, context: BackendContext) -> dict:
|
||||
menu_path: list = p.get("menu_path", [])
|
||||
app_name: str = p.get("app_name", "")
|
||||
|
||||
if not menu_path:
|
||||
raise ValueError("menu_click requires 'menu_path' param (list of strings).")
|
||||
|
||||
if _SYSTEM == "Linux":
|
||||
if _atspi_available():
|
||||
if not app_name:
|
||||
raise ValueError(
|
||||
"menu_click on Linux AT-SPI requires 'app_name' param."
|
||||
)
|
||||
app = _atspi_find_app(app_name)
|
||||
if app is None:
|
||||
raise RuntimeError(f"AT-SPI: application '{app_name}' not found.")
|
||||
_atspi_menu_path(app, menu_path)
|
||||
return {"menu_path": menu_path, "method": "at-spi"}
|
||||
else:
|
||||
raise RuntimeError(
|
||||
"menu_click on Linux requires AT-SPI.\n"
|
||||
" apt install python3-pyatspi\n"
|
||||
" Or use visual_anchor backend instead."
|
||||
)
|
||||
|
||||
elif _SYSTEM == "Darwin":
|
||||
if not app_name:
|
||||
raise ValueError("menu_click on macOS requires 'app_name' param.")
|
||||
_macos_menu_click(app_name, menu_path)
|
||||
return {"menu_path": menu_path, "method": "osascript"}
|
||||
|
||||
elif _SYSTEM == "Windows":
|
||||
if not app_name:
|
||||
raise ValueError("menu_click on Windows requires 'app_name' param.")
|
||||
win = _win_find_app(app_name)
|
||||
# pywinauto menu navigation
|
||||
menu = win.menu()
|
||||
for item in menu_path:
|
||||
menu = menu.item_by_path(item)
|
||||
menu.click_input()
|
||||
return {"menu_path": menu_path, "method": "pywinauto"}
|
||||
|
||||
raise NotImplementedError(f"menu_click not implemented for {_SYSTEM}")
|
||||
|
||||
# ── button_click ──────────────────────────────────────────────────────────
|
||||
|
||||
def _button_click(self, p: dict, context: BackendContext) -> dict:
|
||||
label: str = p.get("label", "")
|
||||
app_name: str = p.get("app_name", "")
|
||||
|
||||
if not label:
|
||||
raise ValueError("button_click requires 'label' param.")
|
||||
|
||||
if _SYSTEM == "Linux" and _atspi_available():
|
||||
if not app_name:
|
||||
raise ValueError("button_click on Linux AT-SPI requires 'app_name'.")
|
||||
app = _atspi_find_app(app_name)
|
||||
if app is None:
|
||||
raise RuntimeError(f"AT-SPI: application '{app_name}' not found.")
|
||||
btn = _atspi_find_control(app, "button", label)
|
||||
if btn is None:
|
||||
raise RuntimeError(f"AT-SPI: button '{label}' not found in '{app_name}'.")
|
||||
action = btn.queryAction()
|
||||
for i in range(action.nActions):
|
||||
if action.getName(i).lower() == "click":
|
||||
action.doAction(i)
|
||||
return {"clicked": label, "method": "at-spi"}
|
||||
raise RuntimeError(f"AT-SPI: no click action on button '{label}'.")
|
||||
|
||||
elif _SYSTEM == "Darwin":
|
||||
script = f"""
|
||||
tell application "System Events"
|
||||
click button "{label}" of front window of (first process whose frontmost is true)
|
||||
end tell
|
||||
"""
|
||||
_osascript(script)
|
||||
return {"clicked": label, "method": "osascript"}
|
||||
|
||||
elif _SYSTEM == "Windows":
|
||||
win = _win_find_app(app_name or "")
|
||||
win.child_window(title=label, control_type="Button").click_input()
|
||||
return {"clicked": label, "method": "pywinauto"}
|
||||
|
||||
raise NotImplementedError(
|
||||
f"button_click not fully implemented for {_SYSTEM} without AT-SPI.\n"
|
||||
"Use visual_anchor backend as fallback."
|
||||
)
|
||||
|
||||
# ── wait_for_window ───────────────────────────────────────────────────────
|
||||
|
||||
def _wait_for_window(self, p: dict, context: BackendContext) -> dict:
|
||||
title: str = p.get("title_contains", "")
|
||||
timeout_ms: int = int(p.get("timeout_ms", 5000))
|
||||
poll_ms: int = int(p.get("poll_ms", 300))
|
||||
|
||||
if not title:
|
||||
raise ValueError("wait_for_window requires 'title_contains' param.")
|
||||
|
||||
deadline = time.time() + timeout_ms / 1000.0
|
||||
|
||||
if _SYSTEM == "Linux":
|
||||
while time.time() < deadline:
|
||||
if shutil.which("wmctrl"):
|
||||
r = subprocess.run(
|
||||
["wmctrl", "-l"], capture_output=True, text=True, env=_x_env()
|
||||
)
|
||||
if title.lower() in r.stdout.lower():
|
||||
return {"found": title, "method": "wmctrl"}
|
||||
elif shutil.which("xdotool"):
|
||||
r = subprocess.run(
|
||||
["xdotool", "search", "--name", title],
|
||||
capture_output=True, text=True, env=_x_env(),
|
||||
)
|
||||
if r.returncode == 0 and r.stdout.strip():
|
||||
return {"found": title, "method": "xdotool"}
|
||||
time.sleep(poll_ms / 1000.0)
|
||||
|
||||
elif _SYSTEM == "Darwin":
|
||||
while time.time() < deadline:
|
||||
script = f"""
|
||||
tell application "System Events"
|
||||
set ws to name of every window of every process
|
||||
set found to false
|
||||
repeat with wlist in ws
|
||||
repeat with wname in wlist
|
||||
if "{title}" is in (wname as text) then
|
||||
set found to true
|
||||
end if
|
||||
end repeat
|
||||
end repeat
|
||||
return found
|
||||
end tell
|
||||
"""
|
||||
result = _osascript(script)
|
||||
if result.lower() == "true":
|
||||
return {"found": title, "method": "osascript"}
|
||||
time.sleep(poll_ms / 1000.0)
|
||||
|
||||
elif _SYSTEM == "Windows":
|
||||
import pywinauto.findwindows as fw
|
||||
while time.time() < deadline:
|
||||
try:
|
||||
handles = fw.find_windows(title_re=f".*{title}.*")
|
||||
if handles:
|
||||
return {"found": title, "method": "pywinauto"}
|
||||
except Exception:
|
||||
pass
|
||||
time.sleep(poll_ms / 1000.0)
|
||||
|
||||
raise TimeoutError(
|
||||
f"wait_for_window: window containing '{title}' did not appear "
|
||||
f"within {timeout_ms}ms."
|
||||
)
|
||||
|
||||
# ── focus_window ──────────────────────────────────────────────────────────
|
||||
|
||||
def _focus_window(self, p: dict, context: BackendContext) -> dict:
|
||||
title: str = p.get("title_contains", "")
|
||||
if not title:
|
||||
raise ValueError("focus_window requires 'title_contains' param.")
|
||||
|
||||
if _SYSTEM == "Linux":
|
||||
if shutil.which("wmctrl"):
|
||||
subprocess.run(["wmctrl", "-a", title], check=True, env=_x_env())
|
||||
return {"focused": title, "method": "wmctrl"}
|
||||
_xdotool_focus(title)
|
||||
return {"focused": title, "method": "xdotool"}
|
||||
|
||||
elif _SYSTEM == "Darwin":
|
||||
_osascript(f'tell application "{title}" to activate')
|
||||
return {"focused": title, "method": "osascript"}
|
||||
|
||||
elif _SYSTEM == "Windows":
|
||||
win = _win_find_app(title)
|
||||
win.set_focus()
|
||||
return {"focused": title, "method": "pywinauto"}
|
||||
|
||||
raise NotImplementedError(f"focus_window not implemented for {_SYSTEM}")
|
||||
|
||||
# ── get_controls ──────────────────────────────────────────────────────────
|
||||
|
||||
def _get_controls(self, p: dict, context: BackendContext) -> dict:
|
||||
"""List interactive controls in a window (for macro authoring / discovery)."""
|
||||
window_title: str = p.get("window_title", "")
|
||||
max_depth: int = int(p.get("max_depth", 5))
|
||||
|
||||
if _SYSTEM == "Linux" and _atspi_available():
|
||||
import pyatspi
|
||||
app = _atspi_find_app(window_title) if window_title else None
|
||||
root = app or pyatspi.Registry.getDesktop(0)
|
||||
controls = []
|
||||
interactive_roles = {
|
||||
pyatspi.ROLE_PUSH_BUTTON,
|
||||
pyatspi.ROLE_MENU,
|
||||
pyatspi.ROLE_MENU_ITEM,
|
||||
pyatspi.ROLE_TEXT,
|
||||
pyatspi.ROLE_COMBO_BOX,
|
||||
pyatspi.ROLE_CHECK_BOX,
|
||||
pyatspi.ROLE_RADIO_BUTTON,
|
||||
pyatspi.ROLE_TOGGLE_BUTTON,
|
||||
}
|
||||
from collections import deque
|
||||
queue = deque([(root, 0)])
|
||||
while queue:
|
||||
node, depth = queue.popleft()
|
||||
if depth > max_depth:
|
||||
continue
|
||||
try:
|
||||
if node.getRole() in interactive_roles:
|
||||
controls.append({
|
||||
"role": node.getRoleName(),
|
||||
"name": node.name,
|
||||
})
|
||||
for i in range(node.childCount):
|
||||
child = node.getChildAtIndex(i)
|
||||
if child:
|
||||
queue.append((child, depth + 1))
|
||||
except Exception:
|
||||
continue
|
||||
return {"controls": controls, "count": len(controls)}
|
||||
|
||||
elif _SYSTEM == "Windows":
|
||||
win = _win_find_app(window_title)
|
||||
controls = []
|
||||
for ctrl in win.descendants():
|
||||
try:
|
||||
controls.append({
|
||||
"role": ctrl.element_info.control_type,
|
||||
"name": ctrl.element_info.name,
|
||||
})
|
||||
except Exception:
|
||||
pass
|
||||
return {"controls": controls, "count": len(controls)}
|
||||
|
||||
raise NotImplementedError(
|
||||
f"get_controls not implemented for {_SYSTEM} without AT-SPI / pywinauto."
|
||||
)
|
||||
@@ -0,0 +1,730 @@
|
||||
"""VisualAnchorBackend — find UI elements by image template and interact.
|
||||
|
||||
Approach:
|
||||
1. Capture full screen with mss (pure Python, cross-platform)
|
||||
2. Find the template image inside the screenshot using numpy correlation
|
||||
3. Use pynput to click / type / scroll at the discovered coordinates
|
||||
|
||||
This backend never uses hardcoded absolute coordinates in macro definitions.
|
||||
Instead, macros store small PNG templates of the UI elements they want to
|
||||
interact with, and coordinates are computed at runtime.
|
||||
|
||||
Supported actions:
|
||||
|
||||
click_image — find template on screen and click its center
|
||||
click_relative — click at (x_pct, y_pct) relative to a named window bounds
|
||||
wait_image — wait until template appears on screen
|
||||
type_text — type a string (keyboard injection, no coordinates needed)
|
||||
hotkey — send a keyboard shortcut
|
||||
scroll — scroll at the position of a template image
|
||||
capture_region — screenshot a region and save it (for template creation)
|
||||
|
||||
Example YAML steps:
|
||||
|
||||
- backend: visual_anchor
|
||||
action: click_image
|
||||
params:
|
||||
template: templates/export_button.png
|
||||
confidence: 0.85 # 0..1, lower = more tolerant
|
||||
timeout_ms: 5000 # wait this long for the image to appear
|
||||
|
||||
- backend: visual_anchor
|
||||
action: click_relative
|
||||
params:
|
||||
window_title: "Draw.io" # partial window title match
|
||||
x_pct: 0.5 # 50% across the window
|
||||
y_pct: 0.1 # 10% down the window
|
||||
|
||||
- backend: visual_anchor
|
||||
action: type_text
|
||||
params:
|
||||
text: "output.png"
|
||||
interval_ms: 30 # delay between key presses
|
||||
|
||||
- backend: visual_anchor
|
||||
action: hotkey
|
||||
params:
|
||||
keys: ctrl+shift+e # + separated
|
||||
|
||||
- backend: visual_anchor
|
||||
action: wait_image
|
||||
params:
|
||||
template: templates/dialog_ok.png
|
||||
timeout_ms: 10000
|
||||
|
||||
- backend: visual_anchor
|
||||
action: capture_region
|
||||
params:
|
||||
output: templates/my_button.png
|
||||
x: 100
|
||||
y: 200
|
||||
width: 80
|
||||
height: 30
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from cli_anything.macrocli.backends.base import Backend, BackendContext, StepResult
|
||||
from cli_anything.macrocli.core.macro_model import MacroStep, substitute
|
||||
|
||||
|
||||
def _x_env() -> dict:
|
||||
"""Return env dict with DISPLAY set, for subprocess calls to X tools."""
|
||||
env = os.environ.copy()
|
||||
if "DISPLAY" not in env:
|
||||
env["DISPLAY"] = ":0"
|
||||
return env
|
||||
|
||||
|
||||
# ── lazy imports (only needed when backend is actually used) ──────────────────
|
||||
|
||||
def _require_numpy():
|
||||
try:
|
||||
import numpy as np
|
||||
return np
|
||||
except ImportError:
|
||||
raise ImportError(
|
||||
"numpy is required for the visual_anchor backend.\n"
|
||||
" pip install numpy"
|
||||
)
|
||||
|
||||
|
||||
def _require_pil():
|
||||
try:
|
||||
from PIL import Image
|
||||
return Image
|
||||
except ImportError:
|
||||
raise ImportError(
|
||||
"Pillow is required for the visual_anchor backend.\n"
|
||||
" pip install Pillow"
|
||||
)
|
||||
|
||||
|
||||
def _require_mss():
|
||||
try:
|
||||
import mss
|
||||
return mss
|
||||
except ImportError:
|
||||
raise ImportError(
|
||||
"mss is required for screen capture.\n"
|
||||
" pip install mss"
|
||||
)
|
||||
|
||||
|
||||
def _require_pynput():
|
||||
try:
|
||||
from pynput import mouse as _m, keyboard as _k
|
||||
return _m, _k
|
||||
except ImportError:
|
||||
raise ImportError(
|
||||
"pynput is required for mouse/keyboard control.\n"
|
||||
" pip install pynput"
|
||||
)
|
||||
|
||||
|
||||
# ── template matching ─────────────────────────────────────────────────────────
|
||||
|
||||
def _load_image_as_array(path: str):
|
||||
"""Load an image file as a numpy uint8 RGB array."""
|
||||
np = _require_numpy()
|
||||
Image = _require_pil()
|
||||
img = Image.open(path).convert("RGB")
|
||||
return np.array(img, dtype=np.uint8)
|
||||
|
||||
|
||||
def _screenshot_as_array():
|
||||
"""Capture the full screen and return as numpy RGB array."""
|
||||
np = _require_numpy()
|
||||
mss = _require_mss()
|
||||
Image = _require_pil()
|
||||
with mss.mss() as sct:
|
||||
# Monitor 1 = first physical monitor (index 0 = all monitors combined)
|
||||
monitor = sct.monitors[1]
|
||||
raw = sct.grab(monitor)
|
||||
img = Image.frombytes("RGB", raw.size, raw.bgra, "raw", "BGRX")
|
||||
return np.array(img, dtype=np.uint8), monitor
|
||||
|
||||
|
||||
def _find_template(
|
||||
screen: "np.ndarray",
|
||||
template: "np.ndarray",
|
||||
confidence: float = 0.85,
|
||||
step: int = 1,
|
||||
) -> Optional[tuple[int, int, float]]:
|
||||
"""Find template in screen. Returns (center_x, center_y, score) or None.
|
||||
|
||||
score is 0..1 where 1 = perfect match.
|
||||
Confidence threshold: only return match if score >= confidence.
|
||||
"""
|
||||
np = _require_numpy()
|
||||
|
||||
sh, sw = screen.shape[:2]
|
||||
th, tw = template.shape[:2]
|
||||
|
||||
if th > sh or tw > sw:
|
||||
return None
|
||||
|
||||
screen_f = screen.astype(np.float32)
|
||||
tmpl_f = template.astype(np.float32)
|
||||
tmpl_norm = tmpl_f - tmpl_f.mean()
|
||||
tmpl_std = tmpl_f.std()
|
||||
if tmpl_std < 1e-6:
|
||||
return None # blank template
|
||||
|
||||
best_score = -1.0
|
||||
best_pos: Optional[tuple[int, int]] = None
|
||||
|
||||
for y in range(0, sh - th + 1, step):
|
||||
for x in range(0, sw - tw + 1, step):
|
||||
region = screen_f[y:y + th, x:x + tw]
|
||||
region_norm = region - region.mean()
|
||||
region_std = region.std()
|
||||
if region_std < 1e-6:
|
||||
continue
|
||||
score = float(
|
||||
(region_norm * tmpl_norm).sum()
|
||||
/ (th * tw * region_std * tmpl_std)
|
||||
)
|
||||
if score > best_score:
|
||||
best_score = score
|
||||
best_pos = (x + tw // 2, y + th // 2)
|
||||
|
||||
if best_pos is None or best_score < confidence:
|
||||
return None
|
||||
|
||||
return (best_pos[0], best_pos[1], best_score)
|
||||
|
||||
|
||||
def _wait_for_template(
|
||||
template_array: "np.ndarray",
|
||||
confidence: float,
|
||||
timeout_ms: int,
|
||||
poll_ms: int = 300,
|
||||
) -> Optional[tuple[int, int, float]]:
|
||||
"""Poll until template found on screen or timeout. Returns match or None."""
|
||||
deadline = time.time() + timeout_ms / 1000.0
|
||||
while time.time() < deadline:
|
||||
screen, _ = _screenshot_as_array()
|
||||
result = _find_template(screen, template_array, confidence)
|
||||
if result is not None:
|
||||
return result
|
||||
time.sleep(poll_ms / 1000.0)
|
||||
return None
|
||||
|
||||
|
||||
# ── window bounds helper ──────────────────────────────────────────────────────
|
||||
|
||||
def _get_window_bounds(title_fragment: str) -> Optional[dict]:
|
||||
"""Return {x, y, width, height} of the first window whose title contains
|
||||
title_fragment. Works on Linux (xwininfo + wmctrl) and macOS (AppleScript).
|
||||
Returns None if not found or not available.
|
||||
"""
|
||||
import subprocess
|
||||
import shutil
|
||||
import platform
|
||||
|
||||
system = platform.system()
|
||||
|
||||
if system == "Linux":
|
||||
# Try wmctrl first (most reliable)
|
||||
if shutil.which("wmctrl"):
|
||||
r = subprocess.run(
|
||||
["wmctrl", "-lG"], capture_output=True, text=True, env=_x_env()
|
||||
)
|
||||
for line in r.stdout.splitlines():
|
||||
parts = line.split(None, 9)
|
||||
if len(parts) >= 9 and title_fragment.lower() in parts[-1].lower():
|
||||
try:
|
||||
# wmctrl -lG: wid desktop x y w h host title
|
||||
x, y, w, h = int(parts[2]), int(parts[3]), int(parts[4]), int(parts[5])
|
||||
return {"x": x, "y": y, "width": w, "height": h}
|
||||
except ValueError:
|
||||
pass
|
||||
# Fallback: xwininfo
|
||||
if shutil.which("xwininfo"):
|
||||
r = subprocess.run(
|
||||
["xwininfo", "-name", title_fragment],
|
||||
capture_output=True, text=True, env=_x_env()
|
||||
)
|
||||
bounds = {}
|
||||
for line in r.stdout.splitlines():
|
||||
line = line.strip()
|
||||
if "Absolute upper-left X:" in line:
|
||||
bounds["x"] = int(line.split()[-1])
|
||||
elif "Absolute upper-left Y:" in line:
|
||||
bounds["y"] = int(line.split()[-1])
|
||||
elif "Width:" in line:
|
||||
bounds["width"] = int(line.split()[-1])
|
||||
elif "Height:" in line:
|
||||
bounds["height"] = int(line.split()[-1])
|
||||
if len(bounds) == 4:
|
||||
return bounds
|
||||
|
||||
elif system == "Darwin":
|
||||
# macOS: use AppleScript to get window position
|
||||
script = f"""
|
||||
tell application "System Events"
|
||||
set ws to every window of every process whose name contains "{title_fragment}"
|
||||
if ws is not {{}} then
|
||||
set w to item 1 of item 1 of ws
|
||||
set p to position of w
|
||||
set s to size of w
|
||||
return (item 1 of p as text) & "," & (item 2 of p as text) & "," & (item 1 of s as text) & "," & (item 2 of s as text)
|
||||
end if
|
||||
end tell
|
||||
"""
|
||||
r = subprocess.run(["osascript", "-e", script], capture_output=True, text=True)
|
||||
if r.returncode == 0 and r.stdout.strip():
|
||||
parts = r.stdout.strip().split(",")
|
||||
if len(parts) == 4:
|
||||
try:
|
||||
return {
|
||||
"x": int(parts[0]), "y": int(parts[1]),
|
||||
"width": int(parts[2]), "height": int(parts[3])
|
||||
}
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
return None
|
||||
|
||||
|
||||
# ── Backend ───────────────────────────────────────────────────────────────────
|
||||
|
||||
class VisualAnchorBackend(Backend):
|
||||
"""Find UI elements by image template and interact with them."""
|
||||
|
||||
name = "visual_anchor"
|
||||
priority = 75 # between file_transform(70) and gui_macro(80)
|
||||
|
||||
def execute(self, step: MacroStep, params: dict, context: BackendContext) -> StepResult:
|
||||
t0 = time.time()
|
||||
action = step.action
|
||||
p = substitute(step.params, params)
|
||||
|
||||
if context.dry_run:
|
||||
return StepResult(
|
||||
success=True,
|
||||
output={"dry_run": True, "action": action},
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
dispatch = {
|
||||
"click_image": self._click_image,
|
||||
"click_relative": self._click_relative,
|
||||
"wait_image": self._wait_image,
|
||||
"type_text": self._type_text,
|
||||
"hotkey": self._hotkey,
|
||||
"scroll": self._scroll,
|
||||
"drag": self._drag,
|
||||
"drag_relative": self._drag_relative,
|
||||
"capture_region": self._capture_region,
|
||||
}
|
||||
|
||||
handler = dispatch.get(action)
|
||||
if handler is None:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"VisualAnchorBackend: unknown action '{action}'. "
|
||||
f"Available: {sorted(dispatch)}",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
try:
|
||||
output = handler(p, context)
|
||||
return StepResult(
|
||||
success=True,
|
||||
output=output or {},
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
except Exception as exc:
|
||||
return StepResult(
|
||||
success=False,
|
||||
error=f"VisualAnchorBackend.{action}: {exc}",
|
||||
backend_used=self.name,
|
||||
duration_ms=(time.time() - t0) * 1000,
|
||||
)
|
||||
|
||||
def is_available(self) -> bool:
|
||||
for pkg in ("mss", "numpy", "PIL", "pynput"):
|
||||
try:
|
||||
__import__(pkg if pkg != "PIL" else "PIL.Image")
|
||||
except ImportError:
|
||||
return False
|
||||
return True
|
||||
|
||||
# ── Actions ──────────────────────────────────────────────────────────────
|
||||
|
||||
def _click_image(self, p: dict, context: BackendContext) -> dict:
|
||||
"""Find template on screen and click its center."""
|
||||
template_path = p.get("template", "")
|
||||
if not template_path or not Path(template_path).is_file():
|
||||
raise FileNotFoundError(
|
||||
f"Template image not found: '{template_path}'. "
|
||||
"Use 'macro record' or 'capture_region' to create one."
|
||||
)
|
||||
|
||||
confidence = float(p.get("confidence", 0.85))
|
||||
timeout_ms = int(p.get("timeout_ms", 5000))
|
||||
button = p.get("button", "left") # left | right | middle
|
||||
double = bool(p.get("double", False))
|
||||
|
||||
template_arr = _load_image_as_array(template_path)
|
||||
match = _wait_for_template(template_arr, confidence, timeout_ms)
|
||||
|
||||
if match is None:
|
||||
raise RuntimeError(
|
||||
f"Template not found on screen after {timeout_ms}ms: {template_path} "
|
||||
f"(confidence={confidence})"
|
||||
)
|
||||
|
||||
cx, cy, score = match
|
||||
_mouse_click(cx, cy, button=button, double=double)
|
||||
|
||||
return {
|
||||
"clicked_at": [cx, cy],
|
||||
"match_score": round(score, 4),
|
||||
"template": template_path,
|
||||
}
|
||||
|
||||
def _click_relative(self, p: dict, context: BackendContext) -> dict:
|
||||
"""Click at a fractional position within a named window."""
|
||||
title = p.get("window_title", "")
|
||||
x_pct = float(p.get("x_pct", 0.5))
|
||||
y_pct = float(p.get("y_pct", 0.5))
|
||||
button = p.get("button", "left")
|
||||
double = bool(p.get("double", False))
|
||||
|
||||
if title:
|
||||
bounds = _get_window_bounds(title)
|
||||
if bounds is None:
|
||||
raise RuntimeError(
|
||||
f"Window not found: '{title}'. "
|
||||
"Make sure the application is open and the title matches."
|
||||
)
|
||||
cx = int(bounds["x"] + bounds["width"] * x_pct)
|
||||
cy = int(bounds["y"] + bounds["height"] * y_pct)
|
||||
else:
|
||||
# Relative to full screen
|
||||
_, monitor = _screenshot_as_array()
|
||||
cx = int(monitor["width"] * x_pct)
|
||||
cy = int(monitor["height"] * y_pct)
|
||||
|
||||
_mouse_click(cx, cy, button=button, double=double)
|
||||
return {"clicked_at": [cx, cy], "x_pct": x_pct, "y_pct": y_pct}
|
||||
|
||||
def _wait_image(self, p: dict, context: BackendContext) -> dict:
|
||||
"""Wait until a template image appears on screen."""
|
||||
template_path = p.get("template", "")
|
||||
if not template_path or not Path(template_path).is_file():
|
||||
raise FileNotFoundError(f"Template image not found: '{template_path}'")
|
||||
|
||||
confidence = float(p.get("confidence", 0.85))
|
||||
timeout_ms = int(p.get("timeout_ms", 10000))
|
||||
|
||||
template_arr = _load_image_as_array(template_path)
|
||||
match = _wait_for_template(template_arr, confidence, timeout_ms)
|
||||
|
||||
if match is None:
|
||||
raise RuntimeError(
|
||||
f"Template never appeared within {timeout_ms}ms: {template_path}"
|
||||
)
|
||||
|
||||
cx, cy, score = match
|
||||
return {"found_at": [cx, cy], "match_score": round(score, 4)}
|
||||
|
||||
def _type_text(self, p: dict, context: BackendContext) -> dict:
|
||||
"""Type a string using keyboard injection."""
|
||||
text = p.get("text", "")
|
||||
interval_ms = int(p.get("interval_ms", 30))
|
||||
if not text:
|
||||
raise ValueError("type_text requires 'text' param.")
|
||||
|
||||
_, keyboard_mod = _require_pynput()
|
||||
ctrl = keyboard_mod.Controller()
|
||||
|
||||
import time as _time
|
||||
for char in text:
|
||||
ctrl.press(char)
|
||||
ctrl.release(char)
|
||||
if interval_ms > 0:
|
||||
_time.sleep(interval_ms / 1000.0)
|
||||
|
||||
return {"typed": len(text), "text_preview": text[:40]}
|
||||
|
||||
def _hotkey(self, p: dict, context: BackendContext) -> dict:
|
||||
"""Send a keyboard shortcut (e.g. ctrl+shift+e)."""
|
||||
keys_str = p.get("keys", "")
|
||||
if not keys_str:
|
||||
raise ValueError("hotkey requires 'keys' param (e.g. 'ctrl+s').")
|
||||
|
||||
_, keyboard_mod = _require_pynput()
|
||||
Key = keyboard_mod.Key
|
||||
ctrl = keyboard_mod.Controller()
|
||||
|
||||
# Parse keys: ctrl+shift+e → [Key.ctrl, Key.shift, 'e']
|
||||
key_objects = []
|
||||
for k in keys_str.split("+"):
|
||||
k = k.strip().lower()
|
||||
# Map common names to pynput Key enum
|
||||
mapping = {
|
||||
"ctrl": Key.ctrl, "control": Key.ctrl,
|
||||
"shift": Key.shift,
|
||||
"alt": Key.alt,
|
||||
"cmd": Key.cmd, "super": Key.cmd, "win": Key.cmd,
|
||||
"enter": Key.enter, "return": Key.enter,
|
||||
"tab": Key.tab,
|
||||
"esc": Key.esc, "escape": Key.esc,
|
||||
"space": Key.space,
|
||||
"backspace": Key.backspace,
|
||||
"delete": Key.delete,
|
||||
"up": Key.up, "down": Key.down,
|
||||
"left": Key.left, "right": Key.right,
|
||||
"home": Key.home, "end": Key.end,
|
||||
"f1": Key.f1, "f2": Key.f2, "f3": Key.f3, "f4": Key.f4,
|
||||
"f5": Key.f5, "f6": Key.f6, "f7": Key.f7, "f8": Key.f8,
|
||||
"f9": Key.f9, "f10": Key.f10, "f11": Key.f11, "f12": Key.f12,
|
||||
}
|
||||
if k in mapping:
|
||||
key_objects.append(mapping[k])
|
||||
elif len(k) == 1:
|
||||
key_objects.append(k)
|
||||
else:
|
||||
raise ValueError(f"Unknown key name: '{k}'")
|
||||
|
||||
# Press all, then release all in reverse
|
||||
for k in key_objects:
|
||||
ctrl.press(k)
|
||||
for k in reversed(key_objects):
|
||||
ctrl.release(k)
|
||||
|
||||
return {"hotkey": keys_str}
|
||||
|
||||
def _scroll(self, p: dict, context: BackendContext) -> dict:
|
||||
"""Scroll at the position of a template image."""
|
||||
template_path = p.get("template", "")
|
||||
dx = int(p.get("dx", 0))
|
||||
dy = int(p.get("dy", -3)) # negative = scroll down
|
||||
timeout_ms = int(p.get("timeout_ms", 5000))
|
||||
confidence = float(p.get("confidence", 0.85))
|
||||
|
||||
if template_path and Path(template_path).is_file():
|
||||
template_arr = _load_image_as_array(template_path)
|
||||
match = _wait_for_template(template_arr, confidence, timeout_ms)
|
||||
if match is None:
|
||||
raise RuntimeError(f"Template not found: {template_path}")
|
||||
cx, cy, _ = match
|
||||
else:
|
||||
# Scroll at current mouse position
|
||||
mouse_mod, _ = _require_pynput()
|
||||
pos = mouse_mod.Controller().position
|
||||
cx, cy = int(pos[0]), int(pos[1])
|
||||
|
||||
mouse_mod, _ = _require_pynput()
|
||||
mouse_ctrl = mouse_mod.Controller()
|
||||
mouse_ctrl.position = (cx, cy)
|
||||
mouse_ctrl.scroll(dx, dy)
|
||||
|
||||
return {"scrolled_at": [cx, cy], "dx": dx, "dy": dy}
|
||||
|
||||
def _drag(self, p: dict, context: BackendContext) -> dict:
|
||||
"""Drag from one template image to another (or to absolute coords).
|
||||
|
||||
Params:
|
||||
from_template: path to template image for drag start (optional)
|
||||
to_template: path to template image for drag end (optional)
|
||||
from_x / from_y: fallback absolute coords if no from_template
|
||||
to_x / to_y: fallback absolute coords if no to_template
|
||||
button: left | right | middle (default left)
|
||||
duration_ms: how long to hold during drag (default 200)
|
||||
confidence: template match threshold (default 0.85)
|
||||
timeout_ms: how long to wait for templates (default 5000)
|
||||
"""
|
||||
button = p.get("button", "left")
|
||||
duration_ms = int(p.get("duration_ms", 200))
|
||||
confidence = float(p.get("confidence", 0.85))
|
||||
timeout_ms = int(p.get("timeout_ms", 5000))
|
||||
|
||||
# Resolve start position
|
||||
from_tmpl = p.get("from_template", "")
|
||||
if from_tmpl and Path(from_tmpl).is_file():
|
||||
tmpl = _load_image_as_array(from_tmpl)
|
||||
match = _wait_for_template(tmpl, confidence, timeout_ms)
|
||||
if match is None:
|
||||
raise RuntimeError(f"drag: from_template not found: {from_tmpl}")
|
||||
fx, fy = match[0], match[1]
|
||||
else:
|
||||
fx = int(p.get("from_x", 0))
|
||||
fy = int(p.get("from_y", 0))
|
||||
|
||||
# Resolve end position
|
||||
to_tmpl = p.get("to_template", "")
|
||||
if to_tmpl and Path(to_tmpl).is_file():
|
||||
tmpl = _load_image_as_array(to_tmpl)
|
||||
match = _wait_for_template(tmpl, confidence, timeout_ms)
|
||||
if match is None:
|
||||
raise RuntimeError(f"drag: to_template not found: {to_tmpl}")
|
||||
tx, ty = match[0], match[1]
|
||||
else:
|
||||
tx = int(p.get("to_x", fx))
|
||||
ty = int(p.get("to_y", fy))
|
||||
|
||||
_mouse_drag(fx, fy, tx, ty, button=button, duration_ms=duration_ms)
|
||||
return {"dragged_from": [fx, fy], "dragged_to": [tx, ty]}
|
||||
|
||||
def _drag_relative(self, p: dict, context: BackendContext) -> dict:
|
||||
"""Drag within a window using fractional coordinates.
|
||||
|
||||
Params:
|
||||
window_title: partial window title (uses focused window if empty)
|
||||
from_x_pct: drag start x as fraction of window width
|
||||
from_y_pct: drag start y as fraction of window height
|
||||
to_x_pct: drag end x as fraction of window width
|
||||
to_y_pct: drag end y as fraction of window height
|
||||
button: left | right | middle (default left)
|
||||
duration_ms: hold duration in ms (default 200)
|
||||
"""
|
||||
title = p.get("window_title", "")
|
||||
button = p.get("button", "left")
|
||||
duration_ms = int(p.get("duration_ms", 200))
|
||||
|
||||
if title:
|
||||
bounds = _get_window_bounds(title)
|
||||
if bounds is None:
|
||||
raise RuntimeError(f"drag_relative: window not found: '{title}'")
|
||||
wx, wy = bounds["x"], bounds["y"]
|
||||
ww, wh = bounds["width"], bounds["height"]
|
||||
else:
|
||||
_, monitor = _screenshot_as_array()
|
||||
wx, wy = 0, 0
|
||||
ww, wh = monitor["width"], monitor["height"]
|
||||
|
||||
fx = int(wx + ww * float(p.get("from_x_pct", 0.0)))
|
||||
fy = int(wy + wh * float(p.get("from_y_pct", 0.0)))
|
||||
tx = int(wx + ww * float(p.get("to_x_pct", 1.0)))
|
||||
ty = int(wy + wh * float(p.get("to_y_pct", 1.0)))
|
||||
|
||||
_mouse_drag(fx, fy, tx, ty, button=button, duration_ms=duration_ms)
|
||||
return {
|
||||
"dragged_from": [fx, fy],
|
||||
"dragged_to": [tx, ty],
|
||||
"from_pct": [p.get("from_x_pct"), p.get("from_y_pct")],
|
||||
"to_pct": [p.get("to_x_pct"), p.get("to_y_pct")],
|
||||
}
|
||||
|
||||
def _capture_region(self, p: dict, context: BackendContext) -> dict:
|
||||
"""Screenshot a region of the screen and save as a template."""
|
||||
output_path = p.get("output", "")
|
||||
if not output_path:
|
||||
raise ValueError("capture_region requires 'output' param.")
|
||||
|
||||
x = int(p.get("x", 0))
|
||||
y = int(p.get("y", 0))
|
||||
width = int(p.get("width", 100))
|
||||
height = int(p.get("height", 50))
|
||||
|
||||
mss = _require_mss()
|
||||
Image = _require_pil()
|
||||
|
||||
Path(output_path).parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
with mss.mss() as sct:
|
||||
region = {"left": x, "top": y, "width": width, "height": height}
|
||||
raw = sct.grab(region)
|
||||
img = Image.frombytes("RGB", raw.size, raw.bgra, "raw", "BGRX")
|
||||
img.save(output_path)
|
||||
|
||||
size = Path(output_path).stat().st_size
|
||||
return {
|
||||
"saved": output_path,
|
||||
"region": [x, y, width, height],
|
||||
"file_size": size,
|
||||
}
|
||||
|
||||
|
||||
# ── pynput mouse helpers ──────────────────────────────────────────────────────
|
||||
|
||||
def _mouse_click(x: int, y: int, button: str = "left", double: bool = False):
|
||||
"""Move mouse to (x, y) and click."""
|
||||
mouse_mod, _ = _require_pynput()
|
||||
Button = mouse_mod.Button
|
||||
ctrl = mouse_mod.Controller()
|
||||
|
||||
btn_map = {
|
||||
"left": Button.left,
|
||||
"right": Button.right,
|
||||
"middle": Button.middle,
|
||||
}
|
||||
btn = btn_map.get(button.lower(), Button.left)
|
||||
|
||||
ctrl.position = (x, y)
|
||||
time.sleep(0.05)
|
||||
ctrl.press(btn)
|
||||
ctrl.release(btn)
|
||||
if double:
|
||||
time.sleep(0.08)
|
||||
ctrl.press(btn)
|
||||
ctrl.release(btn)
|
||||
|
||||
|
||||
def _mouse_drag(
|
||||
fx: int, fy: int, tx: int, ty: int,
|
||||
button: str = "left", duration_ms: int = 200
|
||||
):
|
||||
"""Press at (fx, fy), move to (tx, ty) over duration_ms, release.
|
||||
|
||||
Tries xdotool first (works with Qt5/KDE apps), falls back to pynput.
|
||||
"""
|
||||
import shutil, subprocess, os
|
||||
|
||||
env = os.environ.copy()
|
||||
if "DISPLAY" not in env:
|
||||
env["DISPLAY"] = ":0"
|
||||
|
||||
if shutil.which("xdotool"):
|
||||
# xdotool is more reliable with Qt5 apps
|
||||
steps = max(5, duration_ms // 30)
|
||||
subprocess.run(["xdotool", "mousemove", str(fx), str(fy)], env=env)
|
||||
time.sleep(0.05)
|
||||
subprocess.run(["xdotool", "mousedown", "1"], env=env)
|
||||
time.sleep(0.05)
|
||||
for i in range(1, steps + 1):
|
||||
ix = int(fx + (tx - fx) * i / steps)
|
||||
iy = int(fy + (ty - fy) * i / steps)
|
||||
subprocess.run(["xdotool", "mousemove", str(ix), str(iy)], env=env)
|
||||
time.sleep(duration_ms / 1000.0 / steps)
|
||||
subprocess.run(["xdotool", "mousemove", str(tx), str(ty)], env=env)
|
||||
time.sleep(0.05)
|
||||
subprocess.run(["xdotool", "mouseup", "1"], env=env)
|
||||
return
|
||||
|
||||
# Fallback: pynput
|
||||
mouse_mod, _ = _require_pynput()
|
||||
Button = mouse_mod.Button
|
||||
ctrl = mouse_mod.Controller()
|
||||
btn_map = {"left": Button.left, "right": Button.right, "middle": Button.middle}
|
||||
btn = btn_map.get(button.lower(), Button.left)
|
||||
|
||||
ctrl.position = (fx, fy)
|
||||
time.sleep(0.05)
|
||||
ctrl.press(btn)
|
||||
time.sleep(0.05)
|
||||
|
||||
steps = max(10, duration_ms // 20)
|
||||
step_sleep = duration_ms / 1000.0 / steps
|
||||
for i in range(1, steps + 1):
|
||||
ix = int(fx + (tx - fx) * i / steps)
|
||||
iy = int(fy + (ty - fy) * i / steps)
|
||||
ctrl.position = (ix, iy)
|
||||
time.sleep(step_sleep)
|
||||
|
||||
ctrl.position = (tx, ty)
|
||||
time.sleep(0.05)
|
||||
ctrl.release(btn)
|
||||
@@ -0,0 +1,460 @@
|
||||
"""LLMAssist — use a vision model to generate macro steps from screenshots.
|
||||
|
||||
This module is OPTIONAL. It requires:
|
||||
pip install openai mss Pillow
|
||||
|
||||
Uses the OpenAI SDK, which is compatible with any OpenAI-compatible API
|
||||
provider (OpenAI, Azure, local vLLM, Ollama, LiteLLM, etc.).
|
||||
|
||||
Configure via environment variables:
|
||||
MACROCLI_MODEL — model name (required)
|
||||
MACROCLI_API_KEY — API key
|
||||
MACROCLI_BASE_URL — base URL (only needed for non-OpenAI hosts)
|
||||
|
||||
How it works:
|
||||
1. Capture a screenshot of the current screen (or use a provided image)
|
||||
2. Send the image + user goal to the vision model with a strict system prompt
|
||||
3. The model returns a JSON array of steps (constrained action space)
|
||||
4. Steps are validated and written as a macro YAML file
|
||||
|
||||
The action space the model is allowed to produce:
|
||||
|
||||
{"type": "click_image", "description": "...", "confidence": 0.85}
|
||||
{"type": "click_relative", "window_title": "...", "x_pct": 0.5, "y_pct": 0.1}
|
||||
{"type": "type_text", "text": "..."}
|
||||
{"type": "hotkey", "keys": "ctrl+s"}
|
||||
{"type": "wait_image", "description": "...", "timeout_ms": 5000}
|
||||
{"type": "wait_for_window","title_contains": "...", "timeout_ms": 5000}
|
||||
{"type": "menu_click", "app_name": "...", "menu_path": ["File", "Export"]}
|
||||
{"type": "scroll", "description": "...", "dy": -3}
|
||||
|
||||
The model is NOT allowed to:
|
||||
- Produce shell commands, Python code, or arbitrary actions
|
||||
- Use absolute pixel coordinates
|
||||
- Output anything other than the JSON array
|
||||
|
||||
The "description" field in click_image / wait_image / scroll tells the user
|
||||
what template image to capture with 'macro record' or 'capture_region'.
|
||||
|
||||
Usage:
|
||||
cli-anything-macrocli macro define my_export --assist \\
|
||||
--goal "Export the current diagram as PNG to /tmp/out.png" \\
|
||||
--screenshot current # takes a fresh screenshot
|
||||
--screenshot /path/to/img.png # use existing image
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
try:
|
||||
import yaml
|
||||
except ImportError:
|
||||
raise ImportError("PyYAML required: pip install PyYAML")
|
||||
|
||||
|
||||
# ── Strict system prompt ──────────────────────────────────────────────────────
|
||||
|
||||
_SYSTEM_PROMPT = """\
|
||||
You are a GUI macro step generator. Given a screenshot and a user goal, \
|
||||
output ONLY a valid JSON array of macro steps.
|
||||
|
||||
ALLOWED step types (use EXACTLY these schemas):
|
||||
|
||||
1. Click a UI element by visual description (template matching will be used):
|
||||
{"type": "click_image", "description": "<what the element looks like>", \
|
||||
"confidence": 0.85, "timeout_ms": 5000}
|
||||
|
||||
2. Click at a fractional position within a named window:
|
||||
{"type": "click_relative", "window_title": "<partial title>", \
|
||||
"x_pct": 0.0-1.0, "y_pct": 0.0-1.0}
|
||||
|
||||
3. Type text into the focused field:
|
||||
{"type": "type_text", "text": "<text to type>"}
|
||||
|
||||
4. Send a keyboard shortcut:
|
||||
{"type": "hotkey", "keys": "<key1+key2+...>"}
|
||||
|
||||
5. Wait for a visual element to appear:
|
||||
{"type": "wait_image", "description": "<what to wait for>", \
|
||||
"timeout_ms": 5000}
|
||||
|
||||
6. Wait for a window with a certain title:
|
||||
{"type": "wait_for_window", "title_contains": "<partial title>", \
|
||||
"timeout_ms": 5000}
|
||||
|
||||
7. Click a menu item by path:
|
||||
{"type": "menu_click", "app_name": "<app name>", \
|
||||
"menu_path": ["Menu", "Submenu", "Item"]}
|
||||
|
||||
8. Scroll near a visual element:
|
||||
{"type": "scroll", "description": "<near what element>", "dy": -3}
|
||||
|
||||
STRICT RULES:
|
||||
- Output RAW JSON ONLY. No markdown, no explanation, no code blocks.
|
||||
- The output must be a JSON array: [step1, step2, ...]
|
||||
- NEVER use absolute pixel coordinates (x, y numbers).
|
||||
- NEVER output shell commands, Python, or any non-JSON content.
|
||||
- NEVER invent step types not listed above.
|
||||
- Prefer menu_click and hotkey over click_image when possible.
|
||||
- For click_image: describe the element clearly so a human can find and \
|
||||
photograph it.
|
||||
- Keep the plan minimal: use the fewest steps that achieve the goal.
|
||||
"""
|
||||
|
||||
|
||||
# ── Screenshot helpers ────────────────────────────────────────────────────────
|
||||
|
||||
def _take_screenshot() -> bytes:
|
||||
"""Capture the current screen and return as PNG bytes."""
|
||||
try:
|
||||
import mss
|
||||
from PIL import Image
|
||||
import io
|
||||
with mss.mss() as sct:
|
||||
monitor = sct.monitors[1]
|
||||
raw = sct.grab(monitor)
|
||||
img = Image.frombytes("RGB", raw.size, raw.bgra, "raw", "BGRX")
|
||||
buf = io.BytesIO()
|
||||
img.save(buf, format="PNG")
|
||||
return buf.getvalue()
|
||||
except ImportError:
|
||||
raise ImportError("mss and Pillow required: pip install mss Pillow")
|
||||
|
||||
|
||||
def _load_image_bytes(path: str) -> bytes:
|
||||
with open(path, "rb") as f:
|
||||
return f.read()
|
||||
|
||||
|
||||
# ── Step validation ───────────────────────────────────────────────────────────
|
||||
|
||||
_ALLOWED_TYPES = {
|
||||
"click_image", "click_relative", "type_text", "hotkey",
|
||||
"wait_image", "wait_for_window", "menu_click", "scroll",
|
||||
}
|
||||
|
||||
_REQUIRED_FIELDS = {
|
||||
"click_image": {"type", "description"},
|
||||
"click_relative": {"type", "window_title", "x_pct", "y_pct"},
|
||||
"type_text": {"type", "text"},
|
||||
"hotkey": {"type", "keys"},
|
||||
"wait_image": {"type", "description"},
|
||||
"wait_for_window": {"type", "title_contains"},
|
||||
"menu_click": {"type", "app_name", "menu_path"},
|
||||
"scroll": {"type"},
|
||||
}
|
||||
|
||||
|
||||
def _validate_steps(raw_steps: list) -> tuple[list[dict], list[str]]:
|
||||
"""Validate and sanitize steps from model output.
|
||||
|
||||
Returns (valid_steps, error_messages).
|
||||
"""
|
||||
valid = []
|
||||
errors = []
|
||||
|
||||
for i, step in enumerate(raw_steps):
|
||||
if not isinstance(step, dict):
|
||||
errors.append(f"Step {i}: not a dict, skipped.")
|
||||
continue
|
||||
|
||||
stype = step.get("type", "")
|
||||
if stype not in _ALLOWED_TYPES:
|
||||
errors.append(f"Step {i}: unknown type '{stype}', skipped.")
|
||||
continue
|
||||
|
||||
required = _REQUIRED_FIELDS.get(stype, {"type"})
|
||||
missing = required - set(step.keys())
|
||||
if missing:
|
||||
errors.append(f"Step {i} ({stype}): missing fields {missing}, skipped.")
|
||||
continue
|
||||
|
||||
# Reject any absolute coordinate fields
|
||||
for bad_field in ("x", "y", "px", "pixels"):
|
||||
if bad_field in step:
|
||||
errors.append(
|
||||
f"Step {i} ({stype}): absolute coordinate field '{bad_field}' rejected."
|
||||
)
|
||||
step.pop(bad_field)
|
||||
|
||||
valid.append(step)
|
||||
|
||||
return valid, errors
|
||||
|
||||
|
||||
# ── Step → YAML step dict conversion ─────────────────────────────────────────
|
||||
|
||||
def _step_to_yaml_step(step: dict, index: int) -> dict:
|
||||
"""Convert a validated model step to a macro YAML step dict."""
|
||||
stype = step["type"]
|
||||
sid = f"step_{index:03d}_{stype}"
|
||||
|
||||
if stype == "click_image":
|
||||
return {
|
||||
"id": sid,
|
||||
"backend": "visual_anchor",
|
||||
"action": "click_image",
|
||||
"params": {
|
||||
"template": f"templates/{index:03d}_{stype}.png",
|
||||
"confidence": step.get("confidence", 0.85),
|
||||
"timeout_ms": step.get("timeout_ms", 5000),
|
||||
"_template_description": step.get("description", ""),
|
||||
},
|
||||
"on_failure": "fail",
|
||||
"_model_description": step.get("description", ""),
|
||||
}
|
||||
elif stype == "click_relative":
|
||||
return {
|
||||
"id": sid,
|
||||
"backend": "visual_anchor",
|
||||
"action": "click_relative",
|
||||
"params": {
|
||||
"window_title": step["window_title"],
|
||||
"x_pct": step["x_pct"],
|
||||
"y_pct": step["y_pct"],
|
||||
},
|
||||
"on_failure": "fail",
|
||||
}
|
||||
elif stype == "type_text":
|
||||
return {
|
||||
"id": sid,
|
||||
"backend": "visual_anchor",
|
||||
"action": "type_text",
|
||||
"params": {"text": step["text"]},
|
||||
"on_failure": "fail",
|
||||
}
|
||||
elif stype == "hotkey":
|
||||
return {
|
||||
"id": sid,
|
||||
"backend": "visual_anchor",
|
||||
"action": "hotkey",
|
||||
"params": {"keys": step["keys"]},
|
||||
"on_failure": "fail",
|
||||
}
|
||||
elif stype == "wait_image":
|
||||
return {
|
||||
"id": sid,
|
||||
"backend": "visual_anchor",
|
||||
"action": "wait_image",
|
||||
"params": {
|
||||
"template": f"templates/{index:03d}_{stype}.png",
|
||||
"confidence": step.get("confidence", 0.85),
|
||||
"timeout_ms": step.get("timeout_ms", 10000),
|
||||
"_template_description": step.get("description", ""),
|
||||
},
|
||||
"on_failure": "fail",
|
||||
"_model_description": step.get("description", ""),
|
||||
}
|
||||
elif stype == "wait_for_window":
|
||||
return {
|
||||
"id": sid,
|
||||
"backend": "semantic_ui",
|
||||
"action": "wait_for_window",
|
||||
"params": {
|
||||
"title_contains": step["title_contains"],
|
||||
"timeout_ms": step.get("timeout_ms", 5000),
|
||||
},
|
||||
"on_failure": "fail",
|
||||
}
|
||||
elif stype == "menu_click":
|
||||
return {
|
||||
"id": sid,
|
||||
"backend": "semantic_ui",
|
||||
"action": "menu_click",
|
||||
"params": {
|
||||
"app_name": step["app_name"],
|
||||
"menu_path": step["menu_path"],
|
||||
},
|
||||
"on_failure": "fail",
|
||||
}
|
||||
elif stype == "scroll":
|
||||
return {
|
||||
"id": sid,
|
||||
"backend": "visual_anchor",
|
||||
"action": "scroll",
|
||||
"params": {
|
||||
"template": f"templates/{index:03d}_{stype}.png"
|
||||
if step.get("description") else "",
|
||||
"dy": step.get("dy", -3),
|
||||
"dx": step.get("dx", 0),
|
||||
"_template_description": step.get("description", ""),
|
||||
},
|
||||
"on_failure": "fail",
|
||||
}
|
||||
return {}
|
||||
|
||||
|
||||
# ── Main API ──────────────────────────────────────────────────────────────────
|
||||
|
||||
def generate_macro(
|
||||
goal: str,
|
||||
macro_name: str,
|
||||
screenshot_source: str = "current", # "current" | path to image file
|
||||
api_key: Optional[str] = None,
|
||||
model: Optional[str] = None,
|
||||
base_url: Optional[str] = None,
|
||||
output_path: Optional[str] = None,
|
||||
) -> dict:
|
||||
"""Generate a macro YAML from a user goal and screenshot using a vision model.
|
||||
|
||||
Args:
|
||||
goal: Natural language description of what the macro should do.
|
||||
macro_name: Name for the generated macro.
|
||||
screenshot_source: "current" to take a fresh screenshot, or a
|
||||
file path to use an existing image.
|
||||
api_key: API key. Falls back to MACROCLI_API_KEY env var.
|
||||
model: Model name. Falls back to MACROCLI_MODEL env var.
|
||||
base_url: Base URL for non-OpenAI providers. Falls back to
|
||||
MACROCLI_BASE_URL env var.
|
||||
output_path: Where to write the YAML file. Defaults to
|
||||
<macro_name>.yaml in the current directory.
|
||||
|
||||
Returns:
|
||||
dict with keys: yaml_path, steps_count, warnings, raw_steps
|
||||
"""
|
||||
import base64
|
||||
|
||||
try:
|
||||
from openai import OpenAI
|
||||
except ImportError:
|
||||
raise ImportError(
|
||||
"openai is required for LLM assist.\n"
|
||||
" pip install openai"
|
||||
)
|
||||
|
||||
# Resolve config
|
||||
resolved_model = model or os.environ.get("MACROCLI_MODEL", "")
|
||||
key = api_key or os.environ.get("MACROCLI_API_KEY", "")
|
||||
resolved_base_url = base_url or os.environ.get("MACROCLI_BASE_URL", "")
|
||||
|
||||
if not resolved_model:
|
||||
raise ValueError(
|
||||
"Model required. Pass --model or set MACROCLI_MODEL env var."
|
||||
)
|
||||
if not key:
|
||||
raise ValueError(
|
||||
"API key required. Pass --api-key or set MACROCLI_API_KEY env var."
|
||||
)
|
||||
|
||||
client_kwargs = {"api_key": key}
|
||||
if resolved_base_url:
|
||||
client_kwargs["base_url"] = resolved_base_url
|
||||
client = OpenAI(**client_kwargs)
|
||||
|
||||
# Get screenshot
|
||||
if screenshot_source == "current":
|
||||
image_bytes = _take_screenshot()
|
||||
else:
|
||||
if not Path(screenshot_source).is_file():
|
||||
raise FileNotFoundError(f"Screenshot not found: {screenshot_source}")
|
||||
image_bytes = _load_image_bytes(screenshot_source)
|
||||
|
||||
image_b64 = base64.b64encode(image_bytes).decode("utf-8")
|
||||
|
||||
# Build prompt
|
||||
user_content = [
|
||||
{"type": "text", "text": (
|
||||
f"Goal: {goal}\n\n"
|
||||
"Generate the minimal sequence of steps to achieve this goal. "
|
||||
"Output ONLY the JSON array, nothing else."
|
||||
)},
|
||||
{"type": "image_url", "image_url": {"url": f"data:image/png;base64,{image_b64}"}},
|
||||
]
|
||||
|
||||
response = client.chat.completions.create(
|
||||
model=resolved_model,
|
||||
messages=[
|
||||
{"role": "system", "content": _SYSTEM_PROMPT},
|
||||
{"role": "user", "content": user_content},
|
||||
],
|
||||
max_tokens=2048,
|
||||
)
|
||||
raw_text = response.choices[0].message.content.strip()
|
||||
|
||||
# Strip markdown code fences if model added them despite instructions
|
||||
if raw_text.startswith("```"):
|
||||
lines = raw_text.split("\n")
|
||||
raw_text = "\n".join(
|
||||
line for line in lines
|
||||
if not line.startswith("```")
|
||||
).strip()
|
||||
|
||||
# Parse JSON
|
||||
try:
|
||||
raw_steps = json.loads(raw_text)
|
||||
except json.JSONDecodeError as e:
|
||||
raise ValueError(
|
||||
f"Model returned invalid JSON: {e}\n"
|
||||
f"Raw response (first 500 chars):\n{raw_text[:500]}"
|
||||
)
|
||||
|
||||
if not isinstance(raw_steps, list):
|
||||
raise ValueError(
|
||||
f"Model returned non-array JSON (expected list): {type(raw_steps)}"
|
||||
)
|
||||
|
||||
# Validate
|
||||
valid_steps, warnings = _validate_steps(raw_steps)
|
||||
|
||||
# Convert to YAML step dicts
|
||||
yaml_steps = [
|
||||
_step_to_yaml_step(s, i + 1)
|
||||
for i, s in enumerate(valid_steps)
|
||||
]
|
||||
|
||||
# Build macro dict
|
||||
macro = {
|
||||
"name": macro_name,
|
||||
"version": "1.0",
|
||||
"description": goal,
|
||||
"tags": ["generated", "llm-assist"],
|
||||
"parameters": {},
|
||||
"preconditions": [],
|
||||
"steps": yaml_steps,
|
||||
"postconditions": [],
|
||||
"outputs": [],
|
||||
"agent_hints": {
|
||||
"danger_level": "moderate",
|
||||
"side_effects": ["gui_interaction"],
|
||||
"reversible": False,
|
||||
"generated_by": "llm-assist",
|
||||
"model": resolved_model,
|
||||
},
|
||||
}
|
||||
|
||||
# Add note about templates that need to be captured
|
||||
templates_needed = [
|
||||
{
|
||||
"step_id": s["id"],
|
||||
"template_path": s["params"].get("template", ""),
|
||||
"description": s.get("_model_description", ""),
|
||||
}
|
||||
for s in yaml_steps
|
||||
if s.get("params", {}).get("template") and s.get("_model_description")
|
||||
]
|
||||
|
||||
if templates_needed:
|
||||
macro["_templates_to_capture"] = templates_needed
|
||||
|
||||
# Write YAML
|
||||
if output_path is None:
|
||||
output_path = f"{macro_name}.yaml"
|
||||
|
||||
Path(output_path).parent.mkdir(parents=True, exist_ok=True)
|
||||
Path(output_path).write_text(
|
||||
yaml.dump(macro, allow_unicode=True, sort_keys=False, default_flow_style=False),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
return {
|
||||
"yaml_path": str(Path(output_path).resolve()),
|
||||
"steps_count": len(yaml_steps),
|
||||
"warnings": warnings,
|
||||
"raw_steps": raw_steps,
|
||||
"templates_to_capture": templates_needed,
|
||||
}
|
||||
@@ -0,0 +1,342 @@
|
||||
"""Macro data model — parse and validate YAML macro definitions.
|
||||
|
||||
A macro definition file (YAML) describes a reusable, parameterized workflow
|
||||
that the MacroRuntime can execute against any backend.
|
||||
|
||||
Example (minimal):
|
||||
|
||||
name: export_file
|
||||
version: "1.0"
|
||||
description: Export a file using the target app's CLI.
|
||||
|
||||
parameters:
|
||||
output:
|
||||
type: string
|
||||
required: true
|
||||
example: /tmp/out.png
|
||||
|
||||
steps:
|
||||
- backend: native_api
|
||||
action: run_command
|
||||
params:
|
||||
command: [echo, "exported", "${output}"]
|
||||
|
||||
postconditions:
|
||||
- file_exists: ${output}
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any, Optional
|
||||
|
||||
try:
|
||||
import yaml
|
||||
except ImportError as e:
|
||||
raise ImportError("PyYAML is required: pip install PyYAML") from e
|
||||
|
||||
|
||||
# ── Dataclasses ──────────────────────────────────────────────────────────────
|
||||
|
||||
@dataclass
|
||||
class MacroParameter:
|
||||
name: str
|
||||
type: str = "string" # string | integer | boolean | list | dict
|
||||
required: bool = False
|
||||
default: Any = None
|
||||
description: str = ""
|
||||
example: Any = None
|
||||
enum: Optional[list] = None
|
||||
min: Optional[float] = None
|
||||
max: Optional[float] = None
|
||||
|
||||
def validate_value(self, value: Any) -> list[str]:
|
||||
"""Return list of validation error strings (empty if valid)."""
|
||||
errors: list[str] = []
|
||||
if value is None:
|
||||
if self.required:
|
||||
errors.append(f"Parameter '{self.name}' is required.")
|
||||
return errors
|
||||
|
||||
if self.type == "integer":
|
||||
if not isinstance(value, int):
|
||||
try:
|
||||
value = int(value)
|
||||
except (ValueError, TypeError):
|
||||
errors.append(f"Parameter '{self.name}' must be an integer.")
|
||||
return errors
|
||||
if self.min is not None and value < self.min:
|
||||
errors.append(f"Parameter '{self.name}' must be >= {self.min}.")
|
||||
if self.max is not None and value > self.max:
|
||||
errors.append(f"Parameter '{self.name}' must be <= {self.max}.")
|
||||
|
||||
if self.enum and value not in self.enum:
|
||||
errors.append(
|
||||
f"Parameter '{self.name}' must be one of {self.enum}, got {value!r}."
|
||||
)
|
||||
return errors
|
||||
|
||||
|
||||
@dataclass
|
||||
class MacroStep:
|
||||
backend: str # native_api | file_transform | semantic_ui | gui_macro | recovery
|
||||
action: str # backend-specific action name
|
||||
id: str = ""
|
||||
params: dict = field(default_factory=dict)
|
||||
timeout_ms: int = 30_000
|
||||
on_failure: str = "fail" # fail | skip | continue
|
||||
retry_max: int = 0
|
||||
retry_backoff_ms: list[int] = field(default_factory=lambda: [1000])
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"id": self.id,
|
||||
"backend": self.backend,
|
||||
"action": self.action,
|
||||
"params": self.params,
|
||||
"timeout_ms": self.timeout_ms,
|
||||
"on_failure": self.on_failure,
|
||||
"retry_max": self.retry_max,
|
||||
"retry_backoff_ms": self.retry_backoff_ms,
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class MacroCondition:
|
||||
"""A single pre- or post-condition check.
|
||||
|
||||
Supported types (derived from the YAML key):
|
||||
file_exists: <path>
|
||||
file_size_gt: [<path>, <bytes>]
|
||||
process_running: <name>
|
||||
env_var: <name>
|
||||
always: true | false
|
||||
"""
|
||||
type: str
|
||||
args: Any # depends on type
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {"type": self.type, "args": self.args}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, d: dict) -> "MacroCondition":
|
||||
"""Parse a condition dict like {file_exists: /tmp/out.png}."""
|
||||
if not isinstance(d, dict) or len(d) != 1:
|
||||
raise ValueError(f"Condition must be a single-key dict, got: {d!r}")
|
||||
ctype, args = next(iter(d.items()))
|
||||
return cls(type=ctype, args=args)
|
||||
|
||||
|
||||
@dataclass
|
||||
class MacroOutput:
|
||||
name: str
|
||||
description: str = ""
|
||||
path: Optional[str] = None # raw template string (may contain ${...})
|
||||
value: Optional[Any] = None
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"name": self.name,
|
||||
"description": self.description,
|
||||
"path": self.path,
|
||||
"value": self.value,
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class MacroDefinition:
|
||||
name: str
|
||||
version: str = "1.0"
|
||||
description: str = ""
|
||||
parameters: dict[str, MacroParameter] = field(default_factory=dict)
|
||||
preconditions: list[MacroCondition] = field(default_factory=list)
|
||||
steps: list[MacroStep] = field(default_factory=list)
|
||||
postconditions: list[MacroCondition] = field(default_factory=list)
|
||||
outputs: list[MacroOutput] = field(default_factory=list)
|
||||
tags: list[str] = field(default_factory=list)
|
||||
composable: bool = False
|
||||
agent_hints: dict = field(default_factory=dict)
|
||||
source_path: str = "" # absolute path to the .yaml file
|
||||
|
||||
# ── Validation ────────────────────────────────────────────────────
|
||||
|
||||
def validate(self) -> list[str]:
|
||||
"""Structural validation — returns list of error strings."""
|
||||
errors: list[str] = []
|
||||
if not self.name:
|
||||
errors.append("Macro name is required.")
|
||||
if not self.steps:
|
||||
errors.append(f"Macro '{self.name}' has no steps.")
|
||||
valid_backends = {"native_api", "file_transform", "semantic_ui", "gui_macro", "recovery", "visual_anchor", "gui_agent"}
|
||||
for i, step in enumerate(self.steps):
|
||||
if step.backend not in valid_backends:
|
||||
errors.append(
|
||||
f"Step {i} has unknown backend '{step.backend}'. "
|
||||
f"Valid: {sorted(valid_backends)}"
|
||||
)
|
||||
if not step.action:
|
||||
errors.append(f"Step {i} (backend={step.backend}) is missing 'action'.")
|
||||
for pname, pspec in self.parameters.items():
|
||||
if pspec.type not in ("string", "integer", "boolean", "list", "dict", "float"):
|
||||
errors.append(f"Parameter '{pname}' has unknown type '{pspec.type}'.")
|
||||
return errors
|
||||
|
||||
def validate_params(self, params: dict) -> list[str]:
|
||||
"""Validate runtime parameter values against schema."""
|
||||
errors: list[str] = []
|
||||
for pname, pspec in self.parameters.items():
|
||||
value = params.get(pname, pspec.default)
|
||||
errors.extend(pspec.validate_value(value))
|
||||
return errors
|
||||
|
||||
def resolve_params(self, params: dict) -> dict:
|
||||
"""Return params with defaults filled in."""
|
||||
resolved = {}
|
||||
for pname, pspec in self.parameters.items():
|
||||
resolved[pname] = params.get(pname, pspec.default)
|
||||
# Pass through any extra params not in schema
|
||||
for k, v in params.items():
|
||||
if k not in resolved:
|
||||
resolved[k] = v
|
||||
return resolved
|
||||
|
||||
# ── Serialisation ─────────────────────────────────────────────────
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"name": self.name,
|
||||
"version": self.version,
|
||||
"description": self.description,
|
||||
"parameters": {
|
||||
n: {
|
||||
"type": p.type,
|
||||
"required": p.required,
|
||||
"default": p.default,
|
||||
"description": p.description,
|
||||
"example": p.example,
|
||||
"enum": p.enum,
|
||||
}
|
||||
for n, p in self.parameters.items()
|
||||
},
|
||||
"preconditions": [c.to_dict() for c in self.preconditions],
|
||||
"steps": [s.to_dict() for s in self.steps],
|
||||
"postconditions": [c.to_dict() for c in self.postconditions],
|
||||
"outputs": [o.to_dict() for o in self.outputs],
|
||||
"tags": self.tags,
|
||||
"composable": self.composable,
|
||||
"agent_hints": self.agent_hints,
|
||||
"source_path": self.source_path,
|
||||
}
|
||||
|
||||
|
||||
# ── Substitution ─────────────────────────────────────────────────────────────
|
||||
|
||||
_SUBST_RE = re.compile(r"\$\{([^}]+)\}")
|
||||
|
||||
|
||||
def substitute(template: Any, params: dict) -> Any:
|
||||
"""Replace ${key} placeholders in strings (and nested structures).
|
||||
|
||||
Works recursively on strings, lists, and dicts.
|
||||
Leaves non-string types (int, bool, None) untouched.
|
||||
"""
|
||||
if isinstance(template, str):
|
||||
def _replace(m: re.Match) -> str:
|
||||
key = m.group(1).strip()
|
||||
val = params.get(key)
|
||||
return str(val) if val is not None else m.group(0)
|
||||
return _SUBST_RE.sub(_replace, template)
|
||||
if isinstance(template, list):
|
||||
return [substitute(item, params) for item in template]
|
||||
if isinstance(template, dict):
|
||||
return {k: substitute(v, params) for k, v in template.items()}
|
||||
return template
|
||||
|
||||
|
||||
# ── YAML loader ───────────────────────────────────────────────────────────────
|
||||
|
||||
def _parse_parameter(name: str, raw: dict) -> MacroParameter:
|
||||
return MacroParameter(
|
||||
name=name,
|
||||
type=raw.get("type", "string"),
|
||||
required=raw.get("required", False),
|
||||
default=raw.get("default"),
|
||||
description=raw.get("description", ""),
|
||||
example=raw.get("example"),
|
||||
enum=raw.get("enum"),
|
||||
min=raw.get("min"),
|
||||
max=raw.get("max"),
|
||||
)
|
||||
|
||||
|
||||
def _parse_step(i: int, raw: dict) -> MacroStep:
|
||||
retry = raw.get("retry_policy", {}) or {}
|
||||
return MacroStep(
|
||||
id=raw.get("id", f"step_{i}"),
|
||||
backend=raw.get("backend", "native_api"),
|
||||
action=raw.get("action", ""),
|
||||
params=raw.get("params", {}),
|
||||
timeout_ms=int(raw.get("timeout_ms", raw.get("timeout", "30s")
|
||||
.replace("s", "000") if isinstance(raw.get("timeout"), str)
|
||||
else raw.get("timeout_ms", 30_000))),
|
||||
on_failure=raw.get("on_failure", "fail"),
|
||||
retry_max=retry.get("max_retries", raw.get("retry_max", 0)),
|
||||
retry_backoff_ms=retry.get("backoff_ms", [1000]),
|
||||
)
|
||||
|
||||
|
||||
def _parse_condition(raw: Any) -> MacroCondition:
|
||||
if isinstance(raw, dict):
|
||||
return MacroCondition.from_dict(raw)
|
||||
raise ValueError(f"Cannot parse condition from {raw!r}")
|
||||
|
||||
|
||||
def _parse_output(raw: dict) -> MacroOutput:
|
||||
return MacroOutput(
|
||||
name=raw.get("name", ""),
|
||||
description=raw.get("description", ""),
|
||||
path=raw.get("path"),
|
||||
value=raw.get("value"),
|
||||
)
|
||||
|
||||
|
||||
def load_from_yaml(path: str) -> MacroDefinition:
|
||||
"""Load and parse a macro definition from a YAML file."""
|
||||
p = Path(path)
|
||||
if not p.is_file():
|
||||
raise FileNotFoundError(f"Macro file not found: {path}")
|
||||
with open(p, encoding="utf-8") as f:
|
||||
raw = yaml.safe_load(f)
|
||||
if not isinstance(raw, dict):
|
||||
raise ValueError(f"Macro YAML must be a mapping, got {type(raw).__name__}: {path}")
|
||||
|
||||
parameters: dict[str, MacroParameter] = {}
|
||||
for pname, praw in (raw.get("parameters") or {}).items():
|
||||
if isinstance(praw, dict):
|
||||
parameters[pname] = _parse_parameter(pname, praw)
|
||||
else:
|
||||
# shorthand: parameter_name: string
|
||||
parameters[pname] = MacroParameter(name=pname, type=str(praw))
|
||||
|
||||
steps = [_parse_step(i, s) for i, s in enumerate(raw.get("steps") or [])]
|
||||
preconditions = [_parse_condition(c) for c in (raw.get("preconditions") or [])]
|
||||
postconditions = [_parse_condition(c) for c in (raw.get("postconditions") or [])]
|
||||
outputs = [_parse_output(o) for o in (raw.get("outputs") or [])]
|
||||
|
||||
macro = MacroDefinition(
|
||||
name=raw.get("name", p.stem),
|
||||
version=str(raw.get("version", "1.0")),
|
||||
description=raw.get("description", ""),
|
||||
parameters=parameters,
|
||||
preconditions=preconditions,
|
||||
steps=steps,
|
||||
postconditions=postconditions,
|
||||
outputs=outputs,
|
||||
tags=raw.get("tags", []),
|
||||
composable=raw.get("composable", False),
|
||||
agent_hints=raw.get("agent_hints", {}),
|
||||
source_path=str(p.resolve()),
|
||||
)
|
||||
return macro
|
||||
@@ -0,0 +1,322 @@
|
||||
"""Parameterization helpers — interactive and LLM-assisted.
|
||||
|
||||
Interactive flow (no external deps):
|
||||
assignments = interactive_parameterize(type_steps)
|
||||
parameters = recorder.apply_parameterization(assignments)
|
||||
recorder.save(parameters=parameters)
|
||||
|
||||
Post-hoc flow on an existing YAML file:
|
||||
parameterize_yaml_file(yaml_path) # modifies in-place
|
||||
|
||||
LLM-assisted flow (optional, requires openai):
|
||||
assignments = llm_suggest_parameters(type_steps, api_key=...)
|
||||
# returns same shape as interactive_parameterize, can be passed directly
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
try:
|
||||
import yaml
|
||||
except ImportError:
|
||||
raise ImportError("PyYAML required: pip install PyYAML")
|
||||
|
||||
|
||||
# ── Parameter name validation ─────────────────────────────────────────────────
|
||||
|
||||
_PARAM_NAME_RE = re.compile(r"^[a-z][a-z0-9_]*$")
|
||||
|
||||
|
||||
def _valid_param_name(name: str) -> bool:
|
||||
return bool(_PARAM_NAME_RE.match(name))
|
||||
|
||||
|
||||
def _prompt_param_name(prompt: str) -> Optional[str]:
|
||||
"""Prompt user for a parameter name. Return None if skipped (empty input)."""
|
||||
while True:
|
||||
try:
|
||||
raw = input(prompt).strip()
|
||||
except (EOFError, KeyboardInterrupt):
|
||||
print()
|
||||
return None
|
||||
|
||||
if not raw:
|
||||
return None # skip
|
||||
|
||||
if _valid_param_name(raw):
|
||||
return raw
|
||||
|
||||
print(f" ✗ '{raw}' is not valid. Use lowercase letters, digits, "
|
||||
f"underscores only (must start with a letter). "
|
||||
f"Press Enter to skip.")
|
||||
|
||||
|
||||
# ── Interactive parameterization ──────────────────────────────────────────────
|
||||
|
||||
def interactive_parameterize(
|
||||
type_steps: list[tuple[int, object]],
|
||||
existing_params: Optional[set[str]] = None,
|
||||
) -> dict[int, str]:
|
||||
"""Interactively ask the user which type_text steps to parameterize.
|
||||
|
||||
Args:
|
||||
type_steps: List of (list_index, RecordedStep) from recorder.get_type_steps().
|
||||
existing_params: Already-used parameter names (to avoid duplicates).
|
||||
|
||||
Returns:
|
||||
{list_index: param_name} — only for steps the user chose to parameterize.
|
||||
"""
|
||||
if not type_steps:
|
||||
print(" No type_text steps found to parameterize.")
|
||||
return {}
|
||||
|
||||
used_names: set[str] = set(existing_params or [])
|
||||
assignments: dict[int, str] = {}
|
||||
|
||||
print()
|
||||
print("─" * 60)
|
||||
print(" Parameterization — press Enter to keep a value hardcoded,")
|
||||
print(" or type a parameter name (e.g. output_path) to make it dynamic.")
|
||||
print("─" * 60)
|
||||
|
||||
for n, (idx, step) in enumerate(type_steps, 1):
|
||||
value = step.text # type: ignore[attr-defined]
|
||||
# Truncate long values for display
|
||||
display = value if len(value) <= 50 else value[:47] + "..."
|
||||
|
||||
print(f"\n [{n}/{len(type_steps)}] step typed: {display!r}")
|
||||
|
||||
while True:
|
||||
name = _prompt_param_name(" → Parameter name (Enter to skip): ")
|
||||
if name is None:
|
||||
break # skip
|
||||
if name in used_names:
|
||||
print(f" ✗ '{name}' already used. Choose a different name.")
|
||||
continue
|
||||
used_names.add(name)
|
||||
assignments[idx] = name
|
||||
print(f" ✓ Will become: ${{{{name}}}}")
|
||||
break
|
||||
|
||||
print()
|
||||
if assignments:
|
||||
print(f" Parameterized {len(assignments)} step(s): "
|
||||
f"{', '.join(assignments.values())}")
|
||||
else:
|
||||
print(" No steps parameterized — macro will use hardcoded values.")
|
||||
print("─" * 60)
|
||||
|
||||
return assignments
|
||||
|
||||
|
||||
# ── Post-hoc YAML parameterization ───────────────────────────────────────────
|
||||
|
||||
class _YamlTypeStep:
|
||||
"""Lightweight wrapper for a type_text step found inside a YAML dict."""
|
||||
def __init__(self, step_idx: int, step_dict: dict):
|
||||
self.list_index = step_idx
|
||||
self._step = step_dict
|
||||
self.text: str = step_dict["params"]["text"]
|
||||
|
||||
def apply(self, param_name: str) -> None:
|
||||
self._step["params"]["text"] = f"${{{param_name}}}"
|
||||
|
||||
|
||||
def parameterize_yaml_file(yaml_path: str) -> bool:
|
||||
"""Interactively parameterize an existing macro YAML file in-place.
|
||||
|
||||
Finds all steps with action=type_text, runs the interactive flow,
|
||||
updates the file, and returns True if any changes were made.
|
||||
"""
|
||||
p = Path(yaml_path)
|
||||
if not p.is_file():
|
||||
raise FileNotFoundError(f"Macro file not found: {yaml_path}")
|
||||
|
||||
with open(p, encoding="utf-8") as f:
|
||||
macro = yaml.safe_load(f)
|
||||
|
||||
if not isinstance(macro, dict):
|
||||
raise ValueError("Invalid macro YAML: expected a mapping at top level.")
|
||||
|
||||
steps: list[dict] = macro.get("steps") or []
|
||||
type_steps_raw = [
|
||||
(i, s) for i, s in enumerate(steps)
|
||||
if isinstance(s, dict)
|
||||
and s.get("action") == "type_text"
|
||||
and s.get("params", {}).get("text", "").strip()
|
||||
# Skip already-parameterized steps
|
||||
and not s["params"]["text"].startswith("${")
|
||||
]
|
||||
|
||||
if not type_steps_raw:
|
||||
print(" No hardcoded type_text steps found to parameterize.")
|
||||
return False
|
||||
|
||||
# Wrap in lightweight objects for interactive_parameterize
|
||||
wrapped = [
|
||||
(i, _YamlTypeStep(i, s))
|
||||
for i, s in type_steps_raw
|
||||
]
|
||||
|
||||
existing_params = set((macro.get("parameters") or {}).keys())
|
||||
assignments = interactive_parameterize(wrapped, existing_params)
|
||||
|
||||
if not assignments:
|
||||
return False
|
||||
|
||||
# Apply in-place and collect parameter specs
|
||||
parameters: dict = dict(macro.get("parameters") or {})
|
||||
for idx, param_name in assignments.items():
|
||||
yw = next(w for i, w in wrapped if i == idx)
|
||||
original = yw.text
|
||||
yw.apply(param_name)
|
||||
|
||||
# Infer type
|
||||
ptype = "string"
|
||||
try:
|
||||
int(original); ptype = "integer"
|
||||
except ValueError:
|
||||
try:
|
||||
float(original); ptype = "float"
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
parameters[param_name] = {
|
||||
"type": ptype,
|
||||
"required": True,
|
||||
"description": f"Value typed at step {idx + 1}",
|
||||
"example": original,
|
||||
}
|
||||
|
||||
macro["parameters"] = parameters
|
||||
|
||||
with open(p, "w", encoding="utf-8") as f:
|
||||
yaml.dump(macro, f, allow_unicode=True, sort_keys=False,
|
||||
default_flow_style=False)
|
||||
|
||||
print(f"\n ✓ Updated: {p.resolve()}")
|
||||
return True
|
||||
|
||||
|
||||
# ── LLM-assisted parameterization ─────────────────────────────────────────
|
||||
|
||||
def llm_suggest_parameters(
|
||||
type_steps: list[tuple[int, object]],
|
||||
api_key: Optional[str] = None,
|
||||
model: Optional[str] = None,
|
||||
base_url: Optional[str] = None,
|
||||
) -> dict[int, str]:
|
||||
"""Use a vision model to suggest which type_text steps should be parameterized
|
||||
and what to name the parameters.
|
||||
|
||||
Args:
|
||||
type_steps: Same format as interactive_parameterize input.
|
||||
api_key: API key. Falls back to MACROCLI_API_KEY env var.
|
||||
model: Model name. Falls back to MACROCLI_MODEL env var.
|
||||
base_url: Base URL for non-OpenAI providers. Falls back to
|
||||
MACROCLI_BASE_URL env var.
|
||||
|
||||
Returns:
|
||||
{list_index: suggested_param_name} — same shape as interactive output.
|
||||
The caller can pass this directly to recorder.apply_parameterization()
|
||||
or show it to the user for confirmation first.
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
|
||||
try:
|
||||
from openai import OpenAI
|
||||
except ImportError:
|
||||
raise ImportError(
|
||||
"openai required for auto-parameterization.\n"
|
||||
" pip install openai"
|
||||
)
|
||||
|
||||
resolved_model = model or os.environ.get("MACROCLI_MODEL", "")
|
||||
key = api_key or os.environ.get("MACROCLI_API_KEY", "")
|
||||
resolved_base_url = base_url or os.environ.get("MACROCLI_BASE_URL", "")
|
||||
|
||||
if not resolved_model:
|
||||
raise ValueError(
|
||||
"Model required. Set MACROCLI_MODEL env var or pass --model."
|
||||
)
|
||||
if not key:
|
||||
raise ValueError(
|
||||
"API key required. Pass --api-key or set MACROCLI_API_KEY."
|
||||
)
|
||||
|
||||
client_kwargs = {"api_key": key}
|
||||
if resolved_base_url:
|
||||
client_kwargs["base_url"] = resolved_base_url
|
||||
client = OpenAI(**client_kwargs)
|
||||
|
||||
_SYSTEM = """\
|
||||
You are a macro parameterization assistant. Given a list of text values
|
||||
that a user typed during a GUI recording session, decide which ones should
|
||||
become CLI parameters (so the macro can be reused with different values)
|
||||
and suggest a snake_case parameter name for each.
|
||||
|
||||
Rules:
|
||||
- File paths, URLs, usernames, numeric sizes/counts → ALWAYS parameterize
|
||||
- Generic user content (e.g. document body text) → parameterize if variable
|
||||
- Fixed UI inputs that never change (e.g. "OK", "yes", "1") → do NOT parameterize
|
||||
- Parameter names: lowercase, snake_case, descriptive (e.g. output_path, width)
|
||||
|
||||
Output ONLY a JSON object mapping the step index (as a string) to the
|
||||
parameter name, for steps that SHOULD be parameterized.
|
||||
Steps that should NOT be parameterized must be omitted entirely.
|
||||
Example: {"0": "output_path", "2": "export_width"}
|
||||
"""
|
||||
|
||||
items = "\n".join(
|
||||
f' index {idx}: {step.text!r}' # type: ignore[attr-defined]
|
||||
for idx, step in type_steps
|
||||
)
|
||||
prompt = f"Typed values from the recording:\n{items}\n\nOutput JSON only."
|
||||
|
||||
response = client.chat.completions.create(
|
||||
model=resolved_model,
|
||||
messages=[
|
||||
{"role": "system", "content": _SYSTEM},
|
||||
{"role": "user", "content": prompt},
|
||||
],
|
||||
max_tokens=1024,
|
||||
)
|
||||
raw = response.choices[0].message.content.strip()
|
||||
|
||||
# Strip markdown fences if present
|
||||
if raw.startswith("```"):
|
||||
raw = "\n".join(
|
||||
line for line in raw.split("\n")
|
||||
if not line.startswith("```")
|
||||
).strip()
|
||||
|
||||
try:
|
||||
raw_dict: dict = json.loads(raw)
|
||||
except json.JSONDecodeError as e:
|
||||
raise ValueError(
|
||||
f"Model returned invalid JSON: {e}\nRaw: {raw[:300]}"
|
||||
)
|
||||
|
||||
# Convert string keys to int, validate names
|
||||
result: dict[int, str] = {}
|
||||
for k, v in raw_dict.items():
|
||||
try:
|
||||
idx = int(k)
|
||||
except ValueError:
|
||||
continue
|
||||
if not isinstance(v, str) or not _valid_param_name(v):
|
||||
continue
|
||||
# Check the index is actually in the provided steps
|
||||
if any(i == idx for i, _ in type_steps):
|
||||
result[idx] = v
|
||||
|
||||
return result
|
||||
|
||||
|
||||
# Keep old name as alias for backwards compatibility
|
||||
gemini_suggest_parameters = llm_suggest_parameters
|
||||
@@ -0,0 +1,764 @@
|
||||
"""Macro recorder — record GUI interactions and generate macro YAML.
|
||||
|
||||
Usage:
|
||||
cli-anything-macrocli macro record my_workflow
|
||||
|
||||
What it does:
|
||||
1. Starts listening for mouse clicks and keyboard events (pynput)
|
||||
2. On each click: captures a small screenshot region around the click
|
||||
point and saves it as a template image
|
||||
3. On each hotkey / type event: records the keystroke
|
||||
4. When the user presses Ctrl+Alt+S (or sends SIGINT): stops recording
|
||||
and writes a macro YAML file
|
||||
|
||||
The generated macro uses the `visual_anchor` backend for click steps
|
||||
(template images, not hardcoded coordinates) so it is robust to window
|
||||
movement and minor layout changes.
|
||||
|
||||
Output layout:
|
||||
<macro_name>.yaml
|
||||
<macro_name>_templates/
|
||||
step_001_click.png
|
||||
step_002_click.png
|
||||
...
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import threading
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
try:
|
||||
import yaml
|
||||
except ImportError:
|
||||
raise ImportError("PyYAML required: pip install PyYAML")
|
||||
|
||||
|
||||
# ── Step data classes ─────────────────────────────────────────────────────────
|
||||
|
||||
@dataclass
|
||||
class RecordedStep:
|
||||
index: int
|
||||
kind: str # click | type | hotkey | scroll
|
||||
# click fields
|
||||
x: int = 0
|
||||
y: int = 0
|
||||
button: str = "left"
|
||||
double: bool = False
|
||||
template_path: str = "" # relative path to saved template png (may be empty)
|
||||
window_title: str = "" # title of window under click
|
||||
x_pct: float = 0.5 # click x as fraction of window width
|
||||
y_pct: float = 0.5 # click y as fraction of window height
|
||||
# type fields
|
||||
text: str = ""
|
||||
# hotkey fields
|
||||
keys: str = ""
|
||||
# scroll fields
|
||||
dx: int = 0
|
||||
dy: int = 0
|
||||
# timing
|
||||
timestamp: float = field(default_factory=time.time)
|
||||
# agent step fields (set during post-recording review)
|
||||
is_agent_step: bool = False
|
||||
agent_description: str = ""
|
||||
agent_end_state_description: str = ""
|
||||
agent_end_state_snapshot: str = "" # relative path to snapshot png
|
||||
|
||||
def to_step_dict(self) -> dict:
|
||||
"""Convert to a macro YAML step dict.
|
||||
|
||||
If marked as agent step, emits a gui_agent/instruct step.
|
||||
Otherwise uses visual_anchor with window-relative coords or template.
|
||||
"""
|
||||
# Agent step overrides everything
|
||||
if self.is_agent_step:
|
||||
params: dict = {
|
||||
"description": self.agent_description,
|
||||
"end_state_description": self.agent_end_state_description,
|
||||
"max_steps": 8,
|
||||
}
|
||||
if self.agent_end_state_snapshot:
|
||||
params["end_state_snapshot"] = self.agent_end_state_snapshot
|
||||
return {
|
||||
"id": f"step_{self.index:03d}_agent",
|
||||
"backend": "gui_agent",
|
||||
"action": "instruct",
|
||||
"params": params,
|
||||
"on_failure": "fail",
|
||||
}
|
||||
if self.kind == "click":
|
||||
if self.window_title:
|
||||
# Best case: window-relative fractional coordinates
|
||||
params: dict = {
|
||||
"window_title": self.window_title,
|
||||
"x_pct": self.x_pct,
|
||||
"y_pct": self.y_pct,
|
||||
}
|
||||
if self.button != "left":
|
||||
params["button"] = self.button
|
||||
if self.double:
|
||||
params["double"] = True
|
||||
step = {
|
||||
"id": f"step_{self.index:03d}_click",
|
||||
"backend": "visual_anchor",
|
||||
"action": "click_relative",
|
||||
"params": params,
|
||||
"on_failure": "fail",
|
||||
}
|
||||
# Attach template as a comment if available (for debugging)
|
||||
if self.template_path:
|
||||
step["_template"] = self.template_path
|
||||
return step
|
||||
|
||||
elif self.template_path:
|
||||
# Has a usable template image
|
||||
params = {
|
||||
"template": self.template_path,
|
||||
"confidence": 0.85,
|
||||
"timeout_ms": 5000,
|
||||
}
|
||||
if self.button != "left":
|
||||
params["button"] = self.button
|
||||
if self.double:
|
||||
params["double"] = True
|
||||
return {
|
||||
"id": f"step_{self.index:03d}_click",
|
||||
"backend": "visual_anchor",
|
||||
"action": "click_image",
|
||||
"params": params,
|
||||
"on_failure": "fail",
|
||||
}
|
||||
|
||||
else:
|
||||
# Fallback: screen-relative fractional coordinates
|
||||
return {
|
||||
"id": f"step_{self.index:03d}_click",
|
||||
"backend": "visual_anchor",
|
||||
"action": "click_relative",
|
||||
"params": {
|
||||
"x_pct": self.x_pct,
|
||||
"y_pct": self.y_pct,
|
||||
},
|
||||
"on_failure": "fail",
|
||||
}
|
||||
elif self.kind == "type":
|
||||
return {
|
||||
"id": f"step_{self.index:03d}_type",
|
||||
"backend": "visual_anchor",
|
||||
"action": "type_text",
|
||||
"params": {"text": self.text},
|
||||
"on_failure": "fail",
|
||||
}
|
||||
elif self.kind == "hotkey":
|
||||
return {
|
||||
"id": f"step_{self.index:03d}_hotkey",
|
||||
"backend": "visual_anchor",
|
||||
"action": "hotkey",
|
||||
"params": {"keys": self.keys},
|
||||
"on_failure": "fail",
|
||||
}
|
||||
elif self.kind == "scroll":
|
||||
return {
|
||||
"id": f"step_{self.index:03d}_scroll",
|
||||
"backend": "visual_anchor",
|
||||
"action": "scroll",
|
||||
"params": {
|
||||
"template": self.template_path or "",
|
||||
"dx": self.dx,
|
||||
"dy": self.dy,
|
||||
},
|
||||
"on_failure": "fail",
|
||||
}
|
||||
return {}
|
||||
|
||||
|
||||
# ── Window detection ──────────────────────────────────────────────────────────
|
||||
|
||||
def _get_active_window_at(x: int, y: int) -> tuple[str, Optional[dict]]:
|
||||
"""Return (app_name, bounds) of the currently focused window.
|
||||
|
||||
Uses xdotool getwindowfocus — reliable, no text parsing of hostnames,
|
||||
works regardless of hostname format.
|
||||
"""
|
||||
import shutil
|
||||
import subprocess
|
||||
import os
|
||||
|
||||
env = os.environ.copy()
|
||||
if "DISPLAY" not in env:
|
||||
env["DISPLAY"] = ":0"
|
||||
|
||||
if not shutil.which("xdotool"):
|
||||
return "", None
|
||||
|
||||
try:
|
||||
# Get geometry of focused window
|
||||
gr = subprocess.run(
|
||||
["xdotool", "getwindowfocus", "getwindowgeometry", "--shell"],
|
||||
capture_output=True, text=True, env=env, timeout=2
|
||||
)
|
||||
geo = {}
|
||||
for line in gr.stdout.splitlines():
|
||||
if "=" in line:
|
||||
k, v = line.split("=", 1)
|
||||
geo[k.strip()] = v.strip()
|
||||
|
||||
wx = int(geo.get("X", 0))
|
||||
wy = int(geo.get("Y", 0))
|
||||
ww = int(geo.get("WIDTH", 0))
|
||||
wh = int(geo.get("HEIGHT", 0))
|
||||
wid = geo.get("WINDOW", "")
|
||||
|
||||
if not wid or ww == 0 or wh == 0:
|
||||
return "", None
|
||||
|
||||
# Get window title
|
||||
nr = subprocess.run(
|
||||
["xdotool", "getwindowname", wid],
|
||||
capture_output=True, text=True, env=env, timeout=2
|
||||
)
|
||||
full_title = nr.stdout.strip()
|
||||
|
||||
# Extract app name: last segment after " - "
|
||||
# e.g. "macrocli.txt (/tmp) - gedit" → "gedit"
|
||||
# Plain titles (e.g. "Terminal") are returned as-is
|
||||
if " - " in full_title:
|
||||
app_name = full_title.split(" - ")[-1].strip()
|
||||
else:
|
||||
app_name = full_title.strip()
|
||||
|
||||
bounds = {"x": wx, "y": wy, "width": ww, "height": wh}
|
||||
return app_name, bounds
|
||||
|
||||
except Exception:
|
||||
return "", None
|
||||
|
||||
|
||||
# ── Template capture ──────────────────────────────────────────────────────────
|
||||
|
||||
_TEMPLATE_PADDING = 60 # pixels around click point to capture
|
||||
|
||||
|
||||
def _capture_template(x: int, y: int, output_path: str) -> bool:
|
||||
"""Capture a small region around (x, y) and save as PNG template.
|
||||
|
||||
Returns False if the region has too little variance (blank/featureless)
|
||||
to be useful as a template.
|
||||
"""
|
||||
try:
|
||||
import mss
|
||||
from PIL import Image
|
||||
import numpy as np
|
||||
|
||||
pad = _TEMPLATE_PADDING
|
||||
region = {
|
||||
"left": max(0, x - pad),
|
||||
"top": max(0, y - pad),
|
||||
"width": pad * 2,
|
||||
"height": pad * 2,
|
||||
}
|
||||
Path(output_path).parent.mkdir(parents=True, exist_ok=True)
|
||||
with mss.mss() as sct:
|
||||
raw = sct.grab(region)
|
||||
img = Image.frombytes("RGB", raw.size, raw.bgra, "raw", "BGRX")
|
||||
|
||||
# Check variance — featureless templates are useless
|
||||
arr = np.array(img, dtype=np.float32)
|
||||
if arr.std() < 8.0:
|
||||
return False # blank region, skip
|
||||
|
||||
img.save(output_path)
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f"[recorder] Warning: could not capture template: {e}", file=sys.stderr)
|
||||
return False
|
||||
|
||||
|
||||
# ── Key accumulation helper ───────────────────────────────────────────────────
|
||||
|
||||
_MODIFIER_KEYS = frozenset([
|
||||
"ctrl_l", "ctrl_r", "shift", "shift_r", "alt_l", "alt_r",
|
||||
"alt_gr", "cmd", "cmd_r", "super_l", "super_r",
|
||||
])
|
||||
|
||||
_KEY_NAME_MAP = {
|
||||
"ctrl_l": "ctrl", "ctrl_r": "ctrl",
|
||||
"shift_r": "shift",
|
||||
"alt_l": "alt", "alt_r": "alt", "alt_gr": "alt",
|
||||
"cmd_r": "cmd",
|
||||
"super_l": "super", "super_r": "super",
|
||||
}
|
||||
|
||||
|
||||
def _key_to_str(key) -> str:
|
||||
"""Convert a pynput Key or KeyCode to a string."""
|
||||
try:
|
||||
from pynput.keyboard import Key
|
||||
if isinstance(key, Key):
|
||||
name = key.name # e.g. "ctrl_l", "shift", "f5"
|
||||
return _KEY_NAME_MAP.get(name, name)
|
||||
# KeyCode
|
||||
if hasattr(key, "char") and key.char:
|
||||
return key.char
|
||||
if hasattr(key, "vk") and key.vk:
|
||||
return f"vk{key.vk}"
|
||||
except Exception:
|
||||
pass
|
||||
return str(key)
|
||||
|
||||
|
||||
# ── Recorder ─────────────────────────────────────────────────────────────────
|
||||
|
||||
class MacroRecorder:
|
||||
"""Records mouse and keyboard events and converts them to macro steps."""
|
||||
|
||||
STOP_HOTKEY = frozenset(["ctrl", "alt", "s"]) # Ctrl+Alt+S to stop
|
||||
|
||||
def __init__(self, macro_name: str, output_dir: str = "."):
|
||||
self.macro_name = macro_name
|
||||
self.output_dir = Path(output_dir)
|
||||
self.templates_dir = self.output_dir / f"{macro_name}_templates"
|
||||
|
||||
self._steps: list[RecordedStep] = []
|
||||
self._step_index = 0
|
||||
self._pressed_modifiers: set[str] = set()
|
||||
self._pending_chars: list[str] = []
|
||||
self._last_event_time = time.time()
|
||||
self._lock = threading.Lock()
|
||||
self._stop_event = threading.Event()
|
||||
|
||||
# Double-click detection
|
||||
self._last_click_pos: Optional[tuple[int, int]] = None
|
||||
self._last_click_time: float = 0.0
|
||||
self._DOUBLE_CLICK_MS = 400
|
||||
|
||||
def _next_index(self) -> int:
|
||||
self._step_index += 1
|
||||
return self._step_index
|
||||
|
||||
def _flush_pending_chars(self):
|
||||
"""Accumulate consecutive character presses into a single type step."""
|
||||
if self._pending_chars:
|
||||
text = "".join(self._pending_chars)
|
||||
step = RecordedStep(
|
||||
index=self._next_index(),
|
||||
kind="type",
|
||||
text=text,
|
||||
)
|
||||
self._steps.append(step)
|
||||
self._pending_chars.clear()
|
||||
|
||||
# ── Mouse callbacks ───────────────────────────────────────────────────────
|
||||
|
||||
def on_click(self, x: int, y: int, button, pressed: bool):
|
||||
if not pressed:
|
||||
return # only record press events
|
||||
|
||||
with self._lock:
|
||||
self._flush_pending_chars()
|
||||
|
||||
btn_str = button.name if hasattr(button, "name") else str(button)
|
||||
|
||||
# Detect double click
|
||||
now = time.time()
|
||||
is_double = (
|
||||
self._last_click_pos == (x, y)
|
||||
and (now - self._last_click_time) * 1000 < self._DOUBLE_CLICK_MS
|
||||
)
|
||||
if is_double:
|
||||
# Upgrade last click step to double=True
|
||||
if self._steps and self._steps[-1].kind == "click":
|
||||
self._steps[-1].double = True
|
||||
self._last_click_pos = None
|
||||
return
|
||||
|
||||
self._last_click_pos = (x, y)
|
||||
self._last_click_time = now
|
||||
|
||||
# Try to find the window under the click point
|
||||
window_title, window_bounds = _get_active_window_at(x, y)
|
||||
|
||||
# Compute relative coords within window (fallback to screen pct)
|
||||
if window_bounds:
|
||||
wx, wy = window_bounds["x"], window_bounds["y"]
|
||||
ww, wh = window_bounds["width"], window_bounds["height"]
|
||||
x_pct = round((x - wx) / ww, 4) if ww > 0 else 0.5
|
||||
y_pct = round((y - wy) / wh, 4) if wh > 0 else 0.5
|
||||
else:
|
||||
window_title = ""
|
||||
x_pct = round(x / 1920, 4)
|
||||
y_pct = round(y / 1080, 4)
|
||||
|
||||
# Also try to capture a template (useful when region has features)
|
||||
idx = self._next_index()
|
||||
template_file = str(
|
||||
self.templates_dir / f"step_{idx:03d}_click.png"
|
||||
)
|
||||
captured = _capture_template(x, y, template_file)
|
||||
|
||||
step = RecordedStep(
|
||||
index=idx,
|
||||
kind="click",
|
||||
x=x,
|
||||
y=y,
|
||||
button=btn_str,
|
||||
template_path=template_file if captured else "",
|
||||
window_title=window_title,
|
||||
x_pct=x_pct,
|
||||
y_pct=y_pct,
|
||||
)
|
||||
self._steps.append(step)
|
||||
print(
|
||||
f"[recorder] click #{idx} at ({x},{y}) "
|
||||
f"window='{window_title}' rel=({x_pct},{y_pct})",
|
||||
flush=True
|
||||
)
|
||||
|
||||
def on_scroll(self, x: int, y: int, dx: int, dy: int):
|
||||
with self._lock:
|
||||
self._flush_pending_chars()
|
||||
idx = self._next_index()
|
||||
# Capture template near scroll position
|
||||
template_file = str(
|
||||
self.templates_dir / f"step_{idx:03d}_scroll.png"
|
||||
)
|
||||
captured = _capture_template(x, y, template_file)
|
||||
|
||||
step = RecordedStep(
|
||||
index=idx,
|
||||
kind="scroll",
|
||||
x=x, y=y,
|
||||
dx=dx, dy=dy,
|
||||
template_path=template_file if captured else "",
|
||||
)
|
||||
self._steps.append(step)
|
||||
|
||||
# ── Keyboard callbacks ────────────────────────────────────────────────────
|
||||
|
||||
def on_key_press(self, key):
|
||||
key_str = _key_to_str(key)
|
||||
|
||||
# Check stop hotkey
|
||||
if key_str.lower() in ("ctrl", "alt"):
|
||||
self._pressed_modifiers.add(key_str.lower())
|
||||
elif key_str.lower() == "s" and self._pressed_modifiers >= {"ctrl", "alt"}:
|
||||
print("\n[recorder] Stop hotkey detected (Ctrl+Alt+S). Stopping...", flush=True)
|
||||
self._stop_event.set()
|
||||
return False # stop listener
|
||||
|
||||
if key_str in _MODIFIER_KEYS or _KEY_NAME_MAP.get(key_str, key_str) in _MODIFIER_KEYS:
|
||||
self._pressed_modifiers.add(_KEY_NAME_MAP.get(key_str, key_str))
|
||||
return
|
||||
|
||||
# If modifiers are pressed, it's a hotkey combination
|
||||
# But only if the key itself is NOT a modifier
|
||||
normalized_key = _KEY_NAME_MAP.get(key_str, key_str)
|
||||
is_modifier = normalized_key in {"ctrl", "shift", "alt", "cmd", "super"}
|
||||
active_mods = {_KEY_NAME_MAP.get(m, m) for m in self._pressed_modifiers}
|
||||
|
||||
if active_mods and not is_modifier:
|
||||
with self._lock:
|
||||
self._flush_pending_chars()
|
||||
combo = "+".join(sorted(active_mods) + [key_str])
|
||||
idx = self._next_index()
|
||||
step = RecordedStep(index=idx, kind="hotkey", keys=combo)
|
||||
self._steps.append(step)
|
||||
print(f"[recorder] hotkey #{idx}: {combo}", flush=True)
|
||||
elif not is_modifier:
|
||||
# Regular character or special key
|
||||
# space key comes as Key.space (len > 1), treat as printable
|
||||
is_space = (key_str == "space")
|
||||
if len(key_str) == 1 or is_space:
|
||||
char = " " if is_space else key_str
|
||||
with self._lock:
|
||||
self._pending_chars.append(char)
|
||||
else:
|
||||
# Special key alone (enter, tab, backspace, etc.)
|
||||
with self._lock:
|
||||
self._flush_pending_chars()
|
||||
idx = self._next_index()
|
||||
step = RecordedStep(index=idx, kind="hotkey", keys=key_str)
|
||||
self._steps.append(step)
|
||||
|
||||
def on_key_release(self, key):
|
||||
key_str = _key_to_str(key)
|
||||
normalized = _KEY_NAME_MAP.get(key_str, key_str)
|
||||
self._pressed_modifiers.discard(normalized)
|
||||
|
||||
# ── Main record loop ──────────────────────────────────────────────────────
|
||||
|
||||
def record(self, timeout_s: Optional[float] = None) -> list[RecordedStep]:
|
||||
"""Start recording. Blocks until Ctrl+Alt+S or timeout_s seconds."""
|
||||
try:
|
||||
from pynput import mouse as mouse_mod, keyboard as kb_mod
|
||||
except ImportError:
|
||||
raise ImportError(
|
||||
"pynput is required for recording.\n"
|
||||
" pip install pynput"
|
||||
)
|
||||
|
||||
self.templates_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
print(
|
||||
f"[recorder] Recording '{self.macro_name}'. "
|
||||
"Press Ctrl+Alt+S to stop.",
|
||||
flush=True,
|
||||
)
|
||||
|
||||
mouse_listener = mouse_mod.Listener(
|
||||
on_click=self.on_click,
|
||||
on_scroll=self.on_scroll,
|
||||
)
|
||||
kb_listener = kb_mod.Listener(
|
||||
on_press=self.on_key_press,
|
||||
on_release=self.on_key_release,
|
||||
)
|
||||
|
||||
mouse_listener.start()
|
||||
kb_listener.start()
|
||||
|
||||
try:
|
||||
self._stop_event.wait(timeout=timeout_s)
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
finally:
|
||||
mouse_listener.stop()
|
||||
kb_listener.stop()
|
||||
|
||||
with self._lock:
|
||||
self._flush_pending_chars()
|
||||
|
||||
print(f"[recorder] Recorded {len(self._steps)} steps.", flush=True)
|
||||
return self._steps
|
||||
|
||||
# ── YAML output ───────────────────────────────────────────────────────────
|
||||
|
||||
def to_yaml(self, parameters: Optional[dict] = None) -> str:
|
||||
"""Generate macro YAML from recorded steps.
|
||||
|
||||
Args:
|
||||
parameters: Dict of {param_name: spec} built by
|
||||
apply_parameterization(). If None, all type_text
|
||||
values remain hardcoded.
|
||||
"""
|
||||
steps = [s.to_step_dict() for s in self._steps if s.to_step_dict()]
|
||||
|
||||
macro = {
|
||||
"name": self.macro_name,
|
||||
"version": "1.0",
|
||||
"description": f"Recorded macro: {self.macro_name}",
|
||||
"tags": ["recorded", "visual_anchor"],
|
||||
"parameters": parameters or {},
|
||||
"preconditions": [],
|
||||
"steps": steps,
|
||||
"postconditions": [],
|
||||
"outputs": [],
|
||||
"agent_hints": {
|
||||
"danger_level": "moderate",
|
||||
"side_effects": ["gui_interaction"],
|
||||
"reversible": False,
|
||||
"recorded": True,
|
||||
},
|
||||
}
|
||||
return yaml.dump(macro, allow_unicode=True, sort_keys=False,
|
||||
default_flow_style=False)
|
||||
|
||||
def save(self, output_path: Optional[str] = None,
|
||||
parameters: Optional[dict] = None) -> str:
|
||||
"""Write the generated YAML to a file. Returns the path."""
|
||||
if output_path is None:
|
||||
output_path = str(self.output_dir / f"{self.macro_name}.yaml")
|
||||
Path(output_path).parent.mkdir(parents=True, exist_ok=True)
|
||||
Path(output_path).write_text(
|
||||
self.to_yaml(parameters=parameters), encoding="utf-8"
|
||||
)
|
||||
print(f"[recorder] Saved macro to: {output_path}", flush=True)
|
||||
return output_path
|
||||
|
||||
def save_as_package(
|
||||
self,
|
||||
output_dir: Optional[str] = None,
|
||||
parameters: Optional[dict] = None,
|
||||
) -> str:
|
||||
"""Save as a macro package folder:
|
||||
<macro_name>/
|
||||
macro.yaml
|
||||
snapshots/
|
||||
step_XXX_end_state.png (copied from recorded locations)
|
||||
|
||||
Returns path to macro.yaml.
|
||||
"""
|
||||
pkg_dir = Path(output_dir or self.output_dir) / self.macro_name
|
||||
pkg_dir.mkdir(parents=True, exist_ok=True)
|
||||
snapshots_dir = pkg_dir / "snapshots"
|
||||
snapshots_dir.mkdir(exist_ok=True)
|
||||
|
||||
# Copy any end_state snapshots into the package and update paths
|
||||
for step in self._steps:
|
||||
if step.is_agent_step and step.agent_end_state_snapshot:
|
||||
src = Path(step.agent_end_state_snapshot)
|
||||
if src.is_file():
|
||||
dst = snapshots_dir / src.name
|
||||
import shutil
|
||||
if src.resolve() != dst.resolve():
|
||||
shutil.copy2(src, dst)
|
||||
# Update path to be relative to macro.yaml
|
||||
step.agent_end_state_snapshot = f"snapshots/{src.name}"
|
||||
|
||||
yaml_path = str(pkg_dir / "macro.yaml")
|
||||
Path(yaml_path).write_text(
|
||||
self.to_yaml(parameters=parameters), encoding="utf-8"
|
||||
)
|
||||
print(f"[recorder] Saved macro package to: {pkg_dir}/", flush=True)
|
||||
return yaml_path
|
||||
|
||||
# ── Agent step review ─────────────────────────────────────────────────────
|
||||
|
||||
def interactive_agent_review(self, snapshots_dir: Optional[str] = None) -> None:
|
||||
"""Interactively review all steps and mark some as agent steps.
|
||||
|
||||
For each step the user can:
|
||||
- Press Enter → keep as fixed step
|
||||
- Type 'a' → mark as agent step, then provide description,
|
||||
end_state_description, and take a snapshot
|
||||
|
||||
snapshots_dir: where to save end_state snapshots (default: output_dir/snapshots)
|
||||
"""
|
||||
snap_dir = Path(snapshots_dir or self.output_dir / "snapshots")
|
||||
snap_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
print()
|
||||
print("─" * 60)
|
||||
print(" Step Review — mark steps as 'fixed' or 'agent'")
|
||||
print(" Enter = fixed (fast, deterministic)")
|
||||
print(" a = agent step (vision model decides at runtime)")
|
||||
print("─" * 60)
|
||||
|
||||
for i, step in enumerate(self._steps):
|
||||
# Build a human-readable description of the step
|
||||
if step.kind == "click":
|
||||
step_desc = f"click {step.window_title or 'screen'} ({step.x_pct:.2f}, {step.y_pct:.2f})"
|
||||
elif step.kind == "type":
|
||||
preview = step.text[:40] + "..." if len(step.text) > 40 else step.text
|
||||
step_desc = f"type_text {preview!r}"
|
||||
elif step.kind == "hotkey":
|
||||
step_desc = f"hotkey {step.keys}"
|
||||
elif step.kind == "scroll":
|
||||
step_desc = f"scroll dy={step.dy}"
|
||||
else:
|
||||
step_desc = step.kind
|
||||
|
||||
print(f"\n [{i+1}/{len(self._steps)}] {step_desc}")
|
||||
|
||||
try:
|
||||
choice = input(" → fixed or agent? [Enter=fixed / a=agent]: ").strip().lower()
|
||||
except (EOFError, KeyboardInterrupt):
|
||||
print()
|
||||
break
|
||||
|
||||
if choice != "a":
|
||||
continue
|
||||
|
||||
# Mark as agent step
|
||||
step.is_agent_step = True
|
||||
|
||||
try:
|
||||
step.agent_description = input(
|
||||
" → Describe what this step needs to do:\n "
|
||||
).strip()
|
||||
step.agent_end_state_description = input(
|
||||
" → Describe the target end state (what should the screen show):\n "
|
||||
).strip()
|
||||
except (EOFError, KeyboardInterrupt):
|
||||
print()
|
||||
break
|
||||
|
||||
# Capture end-state snapshot
|
||||
print(" → Now manually operate the UI to reach the end state.")
|
||||
try:
|
||||
input(" Press Enter when ready to take snapshot...")
|
||||
except (EOFError, KeyboardInterrupt):
|
||||
print()
|
||||
continue
|
||||
|
||||
snapshot_path = str(snap_dir / f"step_{step.index:03d}_end_state.png")
|
||||
if self._capture_end_state_snapshot(snapshot_path):
|
||||
step.agent_end_state_snapshot = snapshot_path
|
||||
print(f" ✓ Snapshot saved: {snapshot_path}")
|
||||
else:
|
||||
print(" ⚠ Snapshot failed — no snapshot will be used for this step.")
|
||||
|
||||
print()
|
||||
print("─" * 60)
|
||||
agent_count = sum(1 for s in self._steps if s.is_agent_step)
|
||||
fixed_count = len(self._steps) - agent_count
|
||||
print(f" Review complete: {fixed_count} fixed, {agent_count} agent steps")
|
||||
print("─" * 60)
|
||||
|
||||
def _capture_end_state_snapshot(self, output_path: str) -> bool:
|
||||
"""Capture the current screen and save as end-state snapshot."""
|
||||
try:
|
||||
import mss
|
||||
from PIL import Image
|
||||
Path(output_path).parent.mkdir(parents=True, exist_ok=True)
|
||||
with mss.mss() as sct:
|
||||
monitor = sct.monitors[1]
|
||||
raw = sct.grab(monitor)
|
||||
img = Image.frombytes("RGB", raw.size, raw.bgra, "raw", "BGRX")
|
||||
img.save(output_path)
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f"[recorder] Snapshot failed: {e}", file=sys.stderr)
|
||||
return False
|
||||
|
||||
# ── Parameterization ──────────────────────────────────────────────────────
|
||||
|
||||
def get_type_steps(self) -> list[tuple[int, "RecordedStep"]]:
|
||||
"""Return (list_index, step) for every non-empty type_text step."""
|
||||
return [
|
||||
(i, s) for i, s in enumerate(self._steps)
|
||||
if s.kind == "type" and s.text.strip()
|
||||
]
|
||||
|
||||
def apply_parameterization(
|
||||
self, assignments: dict[int, str]
|
||||
) -> dict[str, dict]:
|
||||
"""Replace type_text values with ${param} placeholders in-place.
|
||||
|
||||
Args:
|
||||
assignments: {list_index: param_name} for steps to parameterize.
|
||||
Steps not present keep their hardcoded value.
|
||||
|
||||
Returns:
|
||||
parameters block ready to pass into to_yaml().
|
||||
"""
|
||||
parameters: dict[str, dict] = {}
|
||||
for idx, param_name in assignments.items():
|
||||
step = self._steps[idx]
|
||||
original = step.text
|
||||
step.text = f"${{{param_name}}}"
|
||||
|
||||
# Infer type from the original value
|
||||
ptype = "string"
|
||||
try:
|
||||
int(original)
|
||||
ptype = "integer"
|
||||
except ValueError:
|
||||
try:
|
||||
float(original)
|
||||
ptype = "float"
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
parameters[param_name] = {
|
||||
"type": ptype,
|
||||
"required": True,
|
||||
"description": f"Value typed at step {idx + 1}",
|
||||
"example": original,
|
||||
}
|
||||
return parameters
|
||||
@@ -0,0 +1,164 @@
|
||||
"""MacroRegistry — discovers and loads macro definitions from a directory.
|
||||
|
||||
The registry scans a macros/ directory (and subdirectories) for *.yaml files,
|
||||
optionally guided by a manifest.yaml index.
|
||||
|
||||
Usage:
|
||||
from cli_anything.macrocli.core.registry import MacroRegistry
|
||||
registry = MacroRegistry("/path/to/macros")
|
||||
macro = registry.load("export_file")
|
||||
all_macros = registry.list_all()
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
try:
|
||||
import yaml
|
||||
except ImportError as e:
|
||||
raise ImportError("PyYAML is required: pip install PyYAML") from e
|
||||
|
||||
from cli_anything.macrocli.core.macro_model import MacroDefinition, load_from_yaml
|
||||
|
||||
|
||||
class MacroRegistry:
|
||||
"""Discovers and caches macro definitions from a macros/ directory."""
|
||||
|
||||
def __init__(self, macros_dir: Optional[str] = None):
|
||||
"""
|
||||
Args:
|
||||
macros_dir: Path to the directory containing macro YAML files.
|
||||
Defaults to the macros/ directory bundled with the package.
|
||||
"""
|
||||
if macros_dir is None:
|
||||
macros_dir = str(Path(__file__).resolve().parent.parent / "macro_definitions")
|
||||
self.macros_dir = Path(macros_dir)
|
||||
self._cache: dict[str, MacroDefinition] = {}
|
||||
self._scanned = False
|
||||
|
||||
# ── Internal scan ────────────────────────────────────────────────────
|
||||
|
||||
def _scan(self) -> None:
|
||||
"""Scan macros_dir and populate the cache."""
|
||||
if self._scanned:
|
||||
return
|
||||
|
||||
if not self.macros_dir.is_dir():
|
||||
self._scanned = True
|
||||
return
|
||||
|
||||
# Try manifest.yaml first (explicit ordered index)
|
||||
manifest_path = self.macros_dir / "manifest.yaml"
|
||||
if manifest_path.is_file():
|
||||
self._load_from_manifest(manifest_path)
|
||||
else:
|
||||
# Fallback: scan all *.yaml files recursively (except manifest.yaml)
|
||||
for yaml_path in sorted(self.macros_dir.rglob("*.yaml")):
|
||||
if yaml_path.name == "manifest.yaml":
|
||||
continue
|
||||
self._load_file(yaml_path)
|
||||
|
||||
self._scanned = True
|
||||
|
||||
def _load_from_manifest(self, manifest_path: Path) -> None:
|
||||
"""Load macros listed in manifest.yaml."""
|
||||
with open(manifest_path, encoding="utf-8") as f:
|
||||
manifest = yaml.safe_load(f) or {}
|
||||
|
||||
macros_list = manifest.get("macros", [])
|
||||
for entry in macros_list:
|
||||
if isinstance(entry, dict):
|
||||
rel_path = entry.get("path")
|
||||
else:
|
||||
rel_path = str(entry)
|
||||
if not rel_path:
|
||||
continue
|
||||
yaml_path = self.macros_dir / rel_path
|
||||
if yaml_path.is_file():
|
||||
self._load_file(yaml_path)
|
||||
|
||||
# Also scan for any yaml files NOT in the manifest (permissive)
|
||||
listed_names = {m.name for m in self._cache.values()}
|
||||
for yaml_path in sorted(self.macros_dir.rglob("*.yaml")):
|
||||
if yaml_path.name == "manifest.yaml":
|
||||
continue
|
||||
try:
|
||||
# Quick peek to get the name without full parse
|
||||
with open(yaml_path, encoding="utf-8") as f:
|
||||
raw = yaml.safe_load(f) or {}
|
||||
name = raw.get("name", yaml_path.stem)
|
||||
if name not in listed_names:
|
||||
self._load_file(yaml_path)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def _load_file(self, yaml_path: Path) -> Optional[MacroDefinition]:
|
||||
"""Parse one yaml file and cache the result."""
|
||||
try:
|
||||
macro = load_from_yaml(str(yaml_path))
|
||||
self._cache[macro.name] = macro
|
||||
return macro
|
||||
except Exception as exc:
|
||||
# Log but don't crash — bad macros should not block the registry
|
||||
import sys
|
||||
print(f"[registry] Warning: failed to load {yaml_path}: {exc}", file=sys.stderr)
|
||||
return None
|
||||
|
||||
# ── Public API ───────────────────────────────────────────────────────
|
||||
|
||||
def load(self, name: str) -> MacroDefinition:
|
||||
"""Load a macro by name.
|
||||
|
||||
Raises:
|
||||
KeyError: if the macro is not found.
|
||||
"""
|
||||
self._scan()
|
||||
if name not in self._cache:
|
||||
available = sorted(self._cache.keys())
|
||||
raise KeyError(
|
||||
f"Macro '{name}' not found. Available: {available}"
|
||||
)
|
||||
return self._cache[name]
|
||||
|
||||
def list_all(self) -> list[MacroDefinition]:
|
||||
"""Return all loaded macro definitions, sorted by name."""
|
||||
self._scan()
|
||||
return sorted(self._cache.values(), key=lambda m: m.name)
|
||||
|
||||
def list_names(self) -> list[str]:
|
||||
"""Return all macro names, sorted."""
|
||||
self._scan()
|
||||
return sorted(self._cache.keys())
|
||||
|
||||
def reload(self, name: Optional[str] = None) -> None:
|
||||
"""Force reload from disk.
|
||||
|
||||
Args:
|
||||
name: If given, reload just that macro file.
|
||||
If None, rescan the entire directory.
|
||||
"""
|
||||
if name is None:
|
||||
self._cache.clear()
|
||||
self._scanned = False
|
||||
self._scan()
|
||||
elif name in self._cache:
|
||||
path = self._cache[name].source_path
|
||||
if path and Path(path).is_file():
|
||||
self._load_file(Path(path))
|
||||
|
||||
def register(self, macro: MacroDefinition) -> None:
|
||||
"""Programmatically register a macro (e.g. from tests)."""
|
||||
self._cache[macro.name] = macro
|
||||
self._scanned = True # Don't re-scan over in-memory registrations
|
||||
|
||||
def info(self) -> dict:
|
||||
"""Return registry metadata."""
|
||||
self._scan()
|
||||
return {
|
||||
"macros_dir": str(self.macros_dir),
|
||||
"total": len(self._cache),
|
||||
"names": self.list_names(),
|
||||
}
|
||||
@@ -0,0 +1,121 @@
|
||||
"""RoutingEngine — select the best backend for each macro step.
|
||||
|
||||
Priority order (higher = preferred):
|
||||
native_api 100
|
||||
gui_macro 80
|
||||
visual_anchor 75
|
||||
file_transform 70
|
||||
gui_agent 60
|
||||
semantic_ui 50
|
||||
recovery 10 (only via explicit backend: recovery, or auto-retry)
|
||||
|
||||
The router respects the step's explicit `backend:` field.
|
||||
It then checks availability; if the primary is unavailable it walks down
|
||||
the priority list.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from cli_anything.macrocli.backends.base import Backend, BackendContext, StepResult
|
||||
from cli_anything.macrocli.backends.native_api import NativeAPIBackend
|
||||
from cli_anything.macrocli.backends.file_transform import FileTransformBackend
|
||||
from cli_anything.macrocli.backends.semantic_ui import SemanticUIBackend
|
||||
from cli_anything.macrocli.backends.gui_macro import GUIMacroBackend
|
||||
from cli_anything.macrocli.backends.visual_anchor import VisualAnchorBackend
|
||||
from cli_anything.macrocli.backends.recovery import RecoveryBackend
|
||||
from cli_anything.macrocli.backends.gui_agent import GUIAgentBackend
|
||||
from cli_anything.macrocli.core.macro_model import MacroStep
|
||||
|
||||
|
||||
_BACKEND_PRIORITY: dict[str, int] = {
|
||||
"native_api": 100,
|
||||
"gui_macro": 80,
|
||||
"visual_anchor": 75,
|
||||
"file_transform": 70,
|
||||
"gui_agent": 60,
|
||||
"semantic_ui": 50,
|
||||
"recovery": 10,
|
||||
}
|
||||
|
||||
|
||||
class RoutingEngine:
|
||||
"""Selects and manages execution backends for macro steps."""
|
||||
|
||||
def __init__(self):
|
||||
self._recovery = RecoveryBackend()
|
||||
self._backends: dict[str, Backend] = {
|
||||
"native_api": NativeAPIBackend(),
|
||||
"file_transform": FileTransformBackend(),
|
||||
"semantic_ui": SemanticUIBackend(),
|
||||
"gui_macro": GUIMacroBackend(),
|
||||
"visual_anchor": VisualAnchorBackend(),
|
||||
"gui_agent": GUIAgentBackend(),
|
||||
"recovery": self._recovery,
|
||||
}
|
||||
# Wire recovery with all other backends so it can delegate
|
||||
for b in self._backends.values():
|
||||
self._recovery.register_backend(b)
|
||||
|
||||
def select(self, step: MacroStep) -> Backend:
|
||||
"""Return the best available backend for the given step.
|
||||
|
||||
Respects step.backend if set; falls back down the priority list
|
||||
if that backend is unavailable.
|
||||
|
||||
Raises:
|
||||
RuntimeError: if no backend is available.
|
||||
"""
|
||||
requested = step.backend
|
||||
# Try the requested backend first
|
||||
if requested in self._backends:
|
||||
b = self._backends[requested]
|
||||
if b.is_available():
|
||||
return b
|
||||
|
||||
# Walk by priority (descending) for a fallback
|
||||
for name in sorted(_BACKEND_PRIORITY, key=lambda k: -_BACKEND_PRIORITY[k]):
|
||||
if name == "recovery":
|
||||
continue # Never auto-fall-through to recovery
|
||||
b = self._backends.get(name)
|
||||
if b and b.is_available():
|
||||
return b
|
||||
|
||||
raise RuntimeError(
|
||||
f"No backend available for step '{step.id}' "
|
||||
f"(requested: '{requested}'). "
|
||||
"Check that required tools are installed."
|
||||
)
|
||||
|
||||
def execute_step(
|
||||
self,
|
||||
step: MacroStep,
|
||||
params: dict,
|
||||
context: BackendContext,
|
||||
) -> StepResult:
|
||||
"""Route and execute a step, applying retry logic if configured."""
|
||||
backend = self.select(step)
|
||||
|
||||
if step.retry_max <= 0:
|
||||
return backend.execute(step, params, context)
|
||||
|
||||
# Retry with backoff
|
||||
last_result: StepResult | None = None
|
||||
backoff = step.retry_backoff_ms or [1000]
|
||||
for attempt in range(step.retry_max + 1):
|
||||
last_result = backend.execute(step, params, context)
|
||||
if last_result.success:
|
||||
return last_result
|
||||
if attempt < step.retry_max:
|
||||
import time
|
||||
wait = backoff[min(attempt, len(backoff) - 1)] / 1000.0
|
||||
time.sleep(wait)
|
||||
|
||||
assert last_result is not None
|
||||
return last_result
|
||||
|
||||
def describe(self) -> dict:
|
||||
"""Return a description of all registered backends and their status."""
|
||||
return {
|
||||
name: b.describe()
|
||||
for name, b in self._backends.items()
|
||||
}
|
||||
@@ -0,0 +1,306 @@
|
||||
"""MacroRuntime — orchestrates the full macro execution lifecycle.
|
||||
|
||||
Lifecycle for execute(macro_name, params):
|
||||
|
||||
1. Load macro definition from registry
|
||||
2. Resolve + validate parameters (fill defaults, type-check)
|
||||
3. Check preconditions
|
||||
4. For each step:
|
||||
a. substitute ${params} into step.params
|
||||
b. route to backend
|
||||
c. execute (with retry if configured)
|
||||
d. handle on_failure = fail | skip | continue
|
||||
5. Check postconditions
|
||||
6. Collect declared outputs
|
||||
7. Record telemetry in session
|
||||
8. Return ExecutionResult
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Optional
|
||||
|
||||
from cli_anything.macrocli.core.macro_model import (
|
||||
MacroCondition,
|
||||
MacroDefinition,
|
||||
MacroStep,
|
||||
substitute,
|
||||
)
|
||||
from cli_anything.macrocli.core.registry import MacroRegistry
|
||||
from cli_anything.macrocli.core.routing import RoutingEngine
|
||||
from cli_anything.macrocli.core.session import ExecutionSession, RunRecord
|
||||
from cli_anything.macrocli.backends.base import BackendContext, StepResult
|
||||
|
||||
|
||||
# ── Result types ─────────────────────────────────────────────────────────────
|
||||
|
||||
@dataclass
|
||||
class ExecutionResult:
|
||||
success: bool
|
||||
macro_name: str
|
||||
output: dict = field(default_factory=dict)
|
||||
error: str = ""
|
||||
step_results: list[StepResult] = field(default_factory=list)
|
||||
telemetry: dict = field(default_factory=dict)
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"success": self.success,
|
||||
"macro_name": self.macro_name,
|
||||
"output": self.output,
|
||||
"error": self.error,
|
||||
"telemetry": self.telemetry,
|
||||
"steps": [s.to_dict() for s in self.step_results],
|
||||
}
|
||||
|
||||
|
||||
# ── Condition checker ────────────────────────────────────────────────────────
|
||||
|
||||
def _check_condition(cond: MacroCondition, resolved_params: dict) -> Optional[str]:
|
||||
"""Evaluate one condition.
|
||||
|
||||
Returns None if the condition passes, or an error string if it fails.
|
||||
"""
|
||||
ctype = cond.type
|
||||
args = substitute(cond.args, resolved_params)
|
||||
|
||||
if ctype == "file_exists":
|
||||
path = str(args)
|
||||
if not os.path.exists(path):
|
||||
return f"file_exists: '{path}' not found."
|
||||
return None
|
||||
|
||||
elif ctype == "file_size_gt":
|
||||
if not isinstance(args, (list, tuple)) or len(args) < 2:
|
||||
return f"file_size_gt: expected [path, min_bytes], got {args!r}"
|
||||
path, min_bytes = str(args[0]), int(args[1])
|
||||
if not os.path.exists(path):
|
||||
return f"file_size_gt: '{path}' not found."
|
||||
size = os.path.getsize(path)
|
||||
if size <= min_bytes:
|
||||
return f"file_size_gt: '{path}' is {size} bytes, expected > {min_bytes}."
|
||||
return None
|
||||
|
||||
elif ctype == "process_running":
|
||||
name = str(args)
|
||||
# Try pgrep first, then psutil
|
||||
import shutil
|
||||
import subprocess
|
||||
if shutil.which("pgrep"):
|
||||
r = subprocess.run(["pgrep", "-x", name], capture_output=True)
|
||||
if r.returncode == 0:
|
||||
return None
|
||||
return f"process_running: '{name}' not found (pgrep)."
|
||||
try:
|
||||
import psutil
|
||||
for proc in psutil.process_iter(["name"]):
|
||||
if proc.info["name"] == name:
|
||||
return None
|
||||
return f"process_running: '{name}' not found."
|
||||
except ImportError:
|
||||
# Can't verify — let it pass with a warning
|
||||
return None
|
||||
|
||||
elif ctype == "env_var":
|
||||
name = str(args)
|
||||
if name not in os.environ:
|
||||
return f"env_var: '{name}' is not set in the environment."
|
||||
return None
|
||||
|
||||
elif ctype == "always":
|
||||
if str(args).lower() in ("false", "0", "no"):
|
||||
return "always: false condition."
|
||||
return None
|
||||
|
||||
else:
|
||||
# Unknown condition type — warn but don't block
|
||||
return None
|
||||
|
||||
|
||||
# ── Runtime ──────────────────────────────────────────────────────────────────
|
||||
|
||||
class MacroRuntime:
|
||||
"""Executes macros end-to-end."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
registry: Optional[MacroRegistry] = None,
|
||||
routing_engine: Optional[RoutingEngine] = None,
|
||||
session: Optional[ExecutionSession] = None,
|
||||
):
|
||||
self.registry = registry or MacroRegistry()
|
||||
self.routing = routing_engine or RoutingEngine()
|
||||
self.session = session or ExecutionSession()
|
||||
|
||||
# ── Public API ───────────────────────────────────────────────────────
|
||||
|
||||
def execute(
|
||||
self,
|
||||
macro_name: str,
|
||||
params: dict,
|
||||
dry_run: bool = False,
|
||||
) -> ExecutionResult:
|
||||
"""Execute a macro by name with the given parameters.
|
||||
|
||||
Args:
|
||||
macro_name: Name of the macro to execute.
|
||||
params: Input parameters (raw, may be strings from CLI).
|
||||
dry_run: If True, skip all side effects and return simulated success.
|
||||
|
||||
Returns:
|
||||
ExecutionResult with success status, outputs, and telemetry.
|
||||
"""
|
||||
t0 = time.time()
|
||||
|
||||
# 1. Load macro
|
||||
try:
|
||||
macro = self.registry.load(macro_name)
|
||||
except KeyError as exc:
|
||||
return ExecutionResult(
|
||||
success=False, macro_name=macro_name, error=str(exc)
|
||||
)
|
||||
|
||||
# 2. Resolve + validate params
|
||||
resolved = macro.resolve_params(params)
|
||||
param_errors = macro.validate_params(resolved)
|
||||
if param_errors:
|
||||
return ExecutionResult(
|
||||
success=False,
|
||||
macro_name=macro_name,
|
||||
error="Parameter validation failed:\n" + "\n".join(f" - {e}" for e in param_errors),
|
||||
)
|
||||
|
||||
# 3. Check preconditions
|
||||
precond_errors = self.check_conditions(macro.preconditions, resolved)
|
||||
if precond_errors:
|
||||
return ExecutionResult(
|
||||
success=False,
|
||||
macro_name=macro_name,
|
||||
error="Preconditions not met:\n" + "\n".join(f" - {e}" for e in precond_errors),
|
||||
)
|
||||
|
||||
# 4. Execute steps
|
||||
step_results: list[StepResult] = []
|
||||
context = BackendContext(
|
||||
params=resolved,
|
||||
previous_results=step_results,
|
||||
dry_run=dry_run,
|
||||
)
|
||||
|
||||
aborted = False
|
||||
abort_error = ""
|
||||
for step in macro.steps:
|
||||
context.timeout_ms = step.timeout_ms
|
||||
try:
|
||||
result = self.routing.execute_step(step, resolved, context)
|
||||
except Exception as exc:
|
||||
result = StepResult(
|
||||
success=False,
|
||||
error=f"Unhandled exception in step '{step.id}': {exc}",
|
||||
backend_used=step.backend,
|
||||
)
|
||||
|
||||
step_results.append(result)
|
||||
|
||||
if not result.success:
|
||||
if step.on_failure == "fail":
|
||||
aborted = True
|
||||
abort_error = f"Step '{step.id}' failed: {result.error}"
|
||||
break
|
||||
elif step.on_failure == "skip":
|
||||
continue
|
||||
# on_failure == "continue" — keep going regardless
|
||||
|
||||
# 5. Check postconditions (skip if already failed)
|
||||
postcond_errors: list[str] = []
|
||||
if not aborted:
|
||||
postcond_errors = self.check_conditions(macro.postconditions, resolved)
|
||||
|
||||
success = not aborted and not postcond_errors
|
||||
|
||||
# 6. Collect outputs
|
||||
output = self._collect_outputs(macro, resolved, step_results) if success else {}
|
||||
|
||||
# 7. Build error string
|
||||
error = ""
|
||||
if aborted:
|
||||
error = abort_error
|
||||
elif postcond_errors:
|
||||
error = "Postconditions failed:\n" + "\n".join(f" - {e}" for e in postcond_errors)
|
||||
|
||||
# 8. Telemetry
|
||||
duration_ms = (time.time() - t0) * 1000
|
||||
backends_used = list({r.backend_used for r in step_results if r.backend_used})
|
||||
telemetry = {
|
||||
"duration_ms": duration_ms,
|
||||
"steps_total": len(macro.steps),
|
||||
"steps_run": len(step_results),
|
||||
"backends_used": backends_used,
|
||||
"dry_run": dry_run,
|
||||
}
|
||||
|
||||
# 9. Record in session
|
||||
record = RunRecord(
|
||||
macro_name=macro_name,
|
||||
params=params,
|
||||
success=success,
|
||||
output=output,
|
||||
error=error,
|
||||
duration_ms=duration_ms,
|
||||
backends_used=backends_used,
|
||||
steps_run=len(step_results),
|
||||
)
|
||||
self.session.record(record)
|
||||
|
||||
return ExecutionResult(
|
||||
success=success,
|
||||
macro_name=macro_name,
|
||||
output=output,
|
||||
error=error,
|
||||
step_results=step_results,
|
||||
telemetry=telemetry,
|
||||
)
|
||||
|
||||
def check_conditions(
|
||||
self,
|
||||
conditions: list[MacroCondition],
|
||||
params: dict,
|
||||
) -> list[str]:
|
||||
"""Evaluate a list of conditions; return list of failure messages."""
|
||||
errors: list[str] = []
|
||||
for cond in conditions:
|
||||
err = _check_condition(cond, params)
|
||||
if err:
|
||||
errors.append(err)
|
||||
return errors
|
||||
|
||||
def validate_macro(self, macro_name: str) -> list[str]:
|
||||
"""Load and structurally validate a macro; return error list."""
|
||||
try:
|
||||
macro = self.registry.load(macro_name)
|
||||
except KeyError as exc:
|
||||
return [str(exc)]
|
||||
return macro.validate()
|
||||
|
||||
# ── Helpers ──────────────────────────────────────────────────────────
|
||||
|
||||
def _collect_outputs(
|
||||
self,
|
||||
macro: MacroDefinition,
|
||||
params: dict,
|
||||
step_results: list[StepResult],
|
||||
) -> dict:
|
||||
"""Resolve declared macro outputs into a concrete dict."""
|
||||
out: dict[str, Any] = {}
|
||||
for output_spec in macro.outputs:
|
||||
name = output_spec.name
|
||||
if output_spec.path:
|
||||
out[name] = substitute(output_spec.path, params)
|
||||
elif output_spec.value is not None:
|
||||
out[name] = substitute(output_spec.value, params)
|
||||
# Always include combined step outputs under 'steps'
|
||||
out["_steps"] = [r.output for r in step_results]
|
||||
return out
|
||||
@@ -0,0 +1,186 @@
|
||||
"""ExecutionSession — tracks macro run history and telemetry.
|
||||
|
||||
Persists run records to ~/.macrocli/sessions/ so agents can inspect
|
||||
what was run, what succeeded, and what the outputs were.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
SESSION_DIR = Path.home() / ".macrocli" / "sessions"
|
||||
MAX_HISTORY = 200
|
||||
|
||||
|
||||
def _locked_save_json(path: str, data, **dump_kwargs) -> None:
|
||||
"""Atomically write JSON with exclusive file locking."""
|
||||
try:
|
||||
f = open(path, "r+")
|
||||
except FileNotFoundError:
|
||||
os.makedirs(os.path.dirname(os.path.abspath(path)), exist_ok=True)
|
||||
f = open(path, "w")
|
||||
with f:
|
||||
_locked = False
|
||||
try:
|
||||
import fcntl
|
||||
fcntl.flock(f.fileno(), fcntl.LOCK_EX)
|
||||
_locked = True
|
||||
except (ImportError, OSError):
|
||||
pass
|
||||
try:
|
||||
f.seek(0)
|
||||
f.truncate()
|
||||
json.dump(data, f, **dump_kwargs)
|
||||
f.flush()
|
||||
finally:
|
||||
if _locked:
|
||||
import fcntl
|
||||
fcntl.flock(f.fileno(), fcntl.LOCK_UN)
|
||||
|
||||
|
||||
class RunRecord:
|
||||
"""A single macro execution record."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
macro_name: str,
|
||||
params: dict,
|
||||
success: bool,
|
||||
output: dict,
|
||||
error: str,
|
||||
duration_ms: float,
|
||||
backends_used: list[str],
|
||||
steps_run: int,
|
||||
timestamp: Optional[float] = None,
|
||||
):
|
||||
self.macro_name = macro_name
|
||||
self.params = params
|
||||
self.success = success
|
||||
self.output = output
|
||||
self.error = error
|
||||
self.duration_ms = duration_ms
|
||||
self.backends_used = backends_used
|
||||
self.steps_run = steps_run
|
||||
self.timestamp = timestamp or time.time()
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"macro_name": self.macro_name,
|
||||
"params": self.params,
|
||||
"success": self.success,
|
||||
"output": self.output,
|
||||
"error": self.error,
|
||||
"duration_ms": self.duration_ms,
|
||||
"backends_used": self.backends_used,
|
||||
"steps_run": self.steps_run,
|
||||
"timestamp": self.timestamp,
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, d: dict) -> "RunRecord":
|
||||
return cls(
|
||||
macro_name=d.get("macro_name", ""),
|
||||
params=d.get("params", {}),
|
||||
success=d.get("success", False),
|
||||
output=d.get("output", {}),
|
||||
error=d.get("error", ""),
|
||||
duration_ms=d.get("duration_ms", 0),
|
||||
backends_used=d.get("backends_used", []),
|
||||
steps_run=d.get("steps_run", 0),
|
||||
timestamp=d.get("timestamp"),
|
||||
)
|
||||
|
||||
|
||||
class ExecutionSession:
|
||||
"""Tracks macro run history for the current session."""
|
||||
|
||||
def __init__(self, session_id: Optional[str] = None):
|
||||
self.session_id = session_id or f"session_{int(time.time())}"
|
||||
self._history: list[RunRecord] = []
|
||||
|
||||
# ── Record management ─────────────────────────────────────────────
|
||||
|
||||
def record(self, run: RunRecord) -> None:
|
||||
"""Add a run record to history."""
|
||||
self._history.append(run)
|
||||
if len(self._history) > MAX_HISTORY:
|
||||
self._history = self._history[-MAX_HISTORY:]
|
||||
|
||||
def last(self) -> Optional[RunRecord]:
|
||||
"""Return the most recent run record."""
|
||||
return self._history[-1] if self._history else None
|
||||
|
||||
def history(self, limit: int = 20) -> list[RunRecord]:
|
||||
"""Return recent run records, newest first."""
|
||||
return list(reversed(self._history[-limit:]))
|
||||
|
||||
def stats(self) -> dict:
|
||||
"""Return aggregate statistics for this session."""
|
||||
total = len(self._history)
|
||||
if total == 0:
|
||||
return {"total": 0, "success_rate": 0.0, "avg_duration_ms": 0.0}
|
||||
successes = sum(1 for r in self._history if r.success)
|
||||
avg_dur = sum(r.duration_ms for r in self._history) / total
|
||||
return {
|
||||
"total": total,
|
||||
"success": successes,
|
||||
"failure": total - successes,
|
||||
"success_rate": successes / total,
|
||||
"avg_duration_ms": avg_dur,
|
||||
}
|
||||
|
||||
def status(self) -> dict:
|
||||
return {
|
||||
"session_id": self.session_id,
|
||||
"runs": len(self._history),
|
||||
**self.stats(),
|
||||
}
|
||||
|
||||
# ── Persistence ───────────────────────────────────────────────────
|
||||
|
||||
def save(self) -> str:
|
||||
"""Persist session to disk. Returns the file path."""
|
||||
SESSION_DIR.mkdir(parents=True, exist_ok=True)
|
||||
path = str(SESSION_DIR / f"{self.session_id}.json")
|
||||
data = {
|
||||
"session_id": self.session_id,
|
||||
"timestamp": time.time(),
|
||||
"history": [r.to_dict() for r in self._history],
|
||||
}
|
||||
_locked_save_json(path, data, indent=2, sort_keys=True)
|
||||
return path
|
||||
|
||||
@classmethod
|
||||
def load(cls, session_id: str) -> Optional["ExecutionSession"]:
|
||||
"""Load a session from disk."""
|
||||
path = SESSION_DIR / f"{session_id}.json"
|
||||
if not path.is_file():
|
||||
return None
|
||||
with open(path, encoding="utf-8") as f:
|
||||
data = json.load(f)
|
||||
session = cls(session_id=data.get("session_id", session_id))
|
||||
session._history = [RunRecord.from_dict(r) for r in data.get("history", [])]
|
||||
return session
|
||||
|
||||
@classmethod
|
||||
def list_sessions(cls) -> list[dict]:
|
||||
"""List all saved sessions (metadata only)."""
|
||||
SESSION_DIR.mkdir(parents=True, exist_ok=True)
|
||||
sessions = []
|
||||
for p in SESSION_DIR.glob("*.json"):
|
||||
try:
|
||||
with open(p, encoding="utf-8") as f:
|
||||
d = json.load(f)
|
||||
sessions.append({
|
||||
"session_id": d.get("session_id"),
|
||||
"timestamp": d.get("timestamp", 0),
|
||||
"runs": len(d.get("history", [])),
|
||||
})
|
||||
except Exception:
|
||||
continue
|
||||
sessions.sort(key=lambda s: s.get("timestamp", 0), reverse=True)
|
||||
return sessions
|
||||
+90
@@ -0,0 +1,90 @@
|
||||
name: gedit_find_and_replace
|
||||
version: "1.0"
|
||||
description: >
|
||||
Open Find & Replace in gedit (Ctrl+H), replace a word, then close the dialog.
|
||||
Uses visual_anchor hotkey + type_text steps.
|
||||
tags: [demo, gedit, visual_anchor]
|
||||
|
||||
parameters:
|
||||
find_text:
|
||||
type: string
|
||||
required: true
|
||||
description: Text to search for.
|
||||
example: foo
|
||||
replace_text:
|
||||
type: string
|
||||
required: true
|
||||
description: Text to replace with.
|
||||
example: bar
|
||||
|
||||
preconditions:
|
||||
- process_running: gedit
|
||||
|
||||
steps:
|
||||
- id: focus_gedit
|
||||
backend: semantic_ui
|
||||
action: focus_window
|
||||
params:
|
||||
title_contains: gedit
|
||||
on_failure: fail
|
||||
|
||||
- id: open_find_replace
|
||||
backend: visual_anchor
|
||||
action: hotkey
|
||||
params:
|
||||
keys: ctrl+h
|
||||
on_failure: fail
|
||||
|
||||
- id: wait_dialog
|
||||
backend: semantic_ui
|
||||
action: wait_for_window
|
||||
params:
|
||||
title_contains: Replace
|
||||
timeout_ms: 4000
|
||||
on_failure: fail
|
||||
|
||||
- id: type_find
|
||||
backend: visual_anchor
|
||||
action: type_text
|
||||
params:
|
||||
text: "${find_text}"
|
||||
interval_ms: 30
|
||||
on_failure: fail
|
||||
|
||||
- id: tab_to_replace_field
|
||||
backend: visual_anchor
|
||||
action: hotkey
|
||||
params:
|
||||
keys: tab
|
||||
on_failure: fail
|
||||
|
||||
- id: type_replace
|
||||
backend: visual_anchor
|
||||
action: type_text
|
||||
params:
|
||||
text: "${replace_text}"
|
||||
interval_ms: 30
|
||||
on_failure: fail
|
||||
|
||||
- id: replace_all
|
||||
backend: visual_anchor
|
||||
action: hotkey
|
||||
params:
|
||||
keys: alt+a
|
||||
on_failure: fail
|
||||
|
||||
- id: close_dialog
|
||||
backend: visual_anchor
|
||||
action: hotkey
|
||||
params:
|
||||
keys: escape
|
||||
on_failure: skip
|
||||
|
||||
postconditions: []
|
||||
outputs: []
|
||||
|
||||
agent_hints:
|
||||
danger_level: moderate
|
||||
side_effects: [modifies_file, gui_interaction]
|
||||
reversible: true
|
||||
estimated_duration_ms: 3000
|
||||
+44
@@ -0,0 +1,44 @@
|
||||
name: gedit_new_window
|
||||
version: "1.0"
|
||||
description: >
|
||||
Open a new gedit window using native_api backend (subprocess).
|
||||
This macro does not require an existing gedit window.
|
||||
tags: [demo, gedit, native_api]
|
||||
|
||||
parameters:
|
||||
file_path:
|
||||
type: string
|
||||
required: false
|
||||
default: ""
|
||||
description: Optional file to open in the new window.
|
||||
|
||||
preconditions:
|
||||
- file_exists: /usr/bin/gedit
|
||||
|
||||
steps:
|
||||
- id: launch_gedit
|
||||
backend: native_api
|
||||
action: start_process
|
||||
params:
|
||||
command: [gedit]
|
||||
log_file: /tmp/gedit.log
|
||||
env:
|
||||
DISPLAY: ":99"
|
||||
on_failure: fail
|
||||
|
||||
- id: wait_window
|
||||
backend: semantic_ui
|
||||
action: wait_for_window
|
||||
params:
|
||||
title_contains: gedit
|
||||
timeout_ms: 8000
|
||||
on_failure: fail
|
||||
|
||||
postconditions: []
|
||||
outputs: []
|
||||
|
||||
agent_hints:
|
||||
danger_level: safe
|
||||
side_effects: [opens_application]
|
||||
reversible: true
|
||||
estimated_duration_ms: 3000
|
||||
+95
@@ -0,0 +1,95 @@
|
||||
name: gedit_save_as
|
||||
version: "1.0"
|
||||
description: >
|
||||
Save the current gedit document to a specific file path.
|
||||
Uses Ctrl+Shift+S to open Save As, then Ctrl+L to type the full path.
|
||||
tags: [demo, gedit, visual_anchor]
|
||||
|
||||
parameters:
|
||||
output_path:
|
||||
type: string
|
||||
required: true
|
||||
description: Full path to save the file to.
|
||||
example: /tmp/demo_output.txt
|
||||
|
||||
preconditions:
|
||||
- process_running: gedit
|
||||
|
||||
steps:
|
||||
- id: focus_gedit
|
||||
backend: semantic_ui
|
||||
action: focus_window
|
||||
params:
|
||||
title_contains: gedit
|
||||
on_failure: fail
|
||||
|
||||
- id: open_save_as
|
||||
backend: visual_anchor
|
||||
action: hotkey
|
||||
params:
|
||||
keys: ctrl+shift+s
|
||||
on_failure: fail
|
||||
|
||||
- id: wait_dialog
|
||||
backend: semantic_ui
|
||||
action: wait_for_window
|
||||
params:
|
||||
title_contains: Save As
|
||||
timeout_ms: 5000
|
||||
on_failure: fail
|
||||
|
||||
- id: open_path_entry
|
||||
backend: visual_anchor
|
||||
action: hotkey
|
||||
params:
|
||||
keys: ctrl+l
|
||||
on_failure: fail
|
||||
|
||||
- id: clear_path_entry
|
||||
backend: visual_anchor
|
||||
action: hotkey
|
||||
params:
|
||||
keys: ctrl+a
|
||||
on_failure: fail
|
||||
|
||||
- id: type_path
|
||||
backend: visual_anchor
|
||||
action: type_text
|
||||
params:
|
||||
text: "${output_path}"
|
||||
interval_ms: 30
|
||||
on_failure: fail
|
||||
|
||||
- id: confirm_save
|
||||
backend: visual_anchor
|
||||
action: hotkey
|
||||
params:
|
||||
keys: enter
|
||||
on_failure: fail
|
||||
|
||||
- id: wait_dialog_close
|
||||
backend: semantic_ui
|
||||
action: wait_for_window
|
||||
params:
|
||||
title_contains: gedit
|
||||
timeout_ms: 3000
|
||||
on_failure: skip
|
||||
|
||||
- id: wait_file_flush
|
||||
backend: native_api
|
||||
action: run_command
|
||||
params:
|
||||
command: [sleep, "0.5"]
|
||||
on_failure: skip
|
||||
|
||||
postconditions: []
|
||||
|
||||
outputs:
|
||||
- name: saved_file
|
||||
path: ${output_path}
|
||||
|
||||
agent_hints:
|
||||
danger_level: safe
|
||||
side_effects: [creates_file, gui_interaction]
|
||||
reversible: true
|
||||
estimated_duration_ms: 4000
|
||||
+50
@@ -0,0 +1,50 @@
|
||||
name: gedit_type_and_save
|
||||
version: "1.0"
|
||||
description: >
|
||||
Type a line of text into gedit and save the file with Ctrl+S.
|
||||
Uses semantic_ui (AT-SPI) to focus the window, then visual_anchor
|
||||
to type and send the shortcut.
|
||||
tags: [demo, gedit, semantic_ui, visual_anchor]
|
||||
|
||||
parameters:
|
||||
text:
|
||||
type: string
|
||||
required: false
|
||||
default: "Hello from MacroCLI!"
|
||||
description: Text to type into gedit.
|
||||
|
||||
preconditions:
|
||||
- process_running: gedit
|
||||
|
||||
steps:
|
||||
- id: focus_gedit
|
||||
backend: semantic_ui
|
||||
action: focus_window
|
||||
params:
|
||||
title_contains: gedit
|
||||
on_failure: fail
|
||||
|
||||
- id: type_text
|
||||
backend: visual_anchor
|
||||
action: type_text
|
||||
params:
|
||||
text: "${text}"
|
||||
interval_ms: 30
|
||||
on_failure: fail
|
||||
|
||||
- id: save_file
|
||||
backend: visual_anchor
|
||||
action: hotkey
|
||||
params:
|
||||
keys: ctrl+s
|
||||
on_failure: fail
|
||||
|
||||
postconditions: []
|
||||
|
||||
outputs: []
|
||||
|
||||
agent_hints:
|
||||
danger_level: safe
|
||||
side_effects: [modifies_file, gui_interaction]
|
||||
reversible: true
|
||||
estimated_duration_ms: 2000
|
||||
+63
@@ -0,0 +1,63 @@
|
||||
name: export_file
|
||||
version: "1.0"
|
||||
description: >
|
||||
Export a file from the target application using its native CLI.
|
||||
Demonstrates the native_api backend with a run_command action.
|
||||
Replace the command with your actual application's export command.
|
||||
tags: [export, native_api, example]
|
||||
|
||||
parameters:
|
||||
output:
|
||||
type: string
|
||||
required: true
|
||||
description: Destination file path for the export.
|
||||
example: /tmp/exported.txt
|
||||
input:
|
||||
type: string
|
||||
required: false
|
||||
default: ""
|
||||
description: Source file to export from (optional, app-dependent).
|
||||
format:
|
||||
type: string
|
||||
required: false
|
||||
default: plain
|
||||
description: Output format.
|
||||
enum: [plain, json, csv]
|
||||
|
||||
preconditions:
|
||||
- file_exists: /bin/echo # sanity check: a real macro would check the app binary
|
||||
|
||||
steps:
|
||||
- id: step_export
|
||||
backend: native_api
|
||||
action: run_command
|
||||
params:
|
||||
# Replace this with your real export command, e.g.:
|
||||
# command: [inkscape, --export-filename, "${output}", "${input}"]
|
||||
command: [echo, "Exported to ${output} (format=${format})"]
|
||||
capture_stdout: true
|
||||
timeout_ms: 30000
|
||||
on_failure: fail
|
||||
|
||||
- id: step_write_output
|
||||
backend: file_transform
|
||||
action: text_replace
|
||||
params:
|
||||
input_file: /dev/null
|
||||
output_file: ${output}
|
||||
find: ""
|
||||
replace: ""
|
||||
on_failure: skip # Writing the marker file is best-effort
|
||||
|
||||
postconditions: []
|
||||
|
||||
outputs:
|
||||
- name: exported_file
|
||||
path: ${output}
|
||||
description: Path where the export was written.
|
||||
|
||||
agent_hints:
|
||||
danger_level: safe
|
||||
side_effects: [creates_file]
|
||||
reversible: true
|
||||
estimated_duration_ms: 2000
|
||||
+52
@@ -0,0 +1,52 @@
|
||||
name: transform_json
|
||||
version: "1.0"
|
||||
description: >
|
||||
Read a JSON file, set a nested key to a new value, and write it back.
|
||||
Demonstrates the file_transform backend with json_set action.
|
||||
tags: [json, file_transform, example]
|
||||
|
||||
parameters:
|
||||
file:
|
||||
type: string
|
||||
required: true
|
||||
description: Path to the JSON file to transform.
|
||||
example: /tmp/config.json
|
||||
key:
|
||||
type: string
|
||||
required: true
|
||||
description: Dot-separated key path to set (e.g. settings.theme).
|
||||
example: settings.theme
|
||||
value:
|
||||
type: string
|
||||
required: true
|
||||
description: Value to write at the key path.
|
||||
example: dark
|
||||
|
||||
preconditions:
|
||||
- file_exists: ${file}
|
||||
|
||||
steps:
|
||||
- id: step_set_key
|
||||
backend: file_transform
|
||||
action: json_set
|
||||
params:
|
||||
input_file: ${file}
|
||||
output_file: ${file}
|
||||
path: ${key}
|
||||
value: ${value}
|
||||
timeout_ms: 10000
|
||||
on_failure: fail
|
||||
|
||||
postconditions:
|
||||
- file_exists: ${file}
|
||||
|
||||
outputs:
|
||||
- name: modified_file
|
||||
path: ${file}
|
||||
description: Path to the modified JSON file.
|
||||
|
||||
agent_hints:
|
||||
danger_level: moderate
|
||||
side_effects: [modifies_file]
|
||||
reversible: false
|
||||
estimated_duration_ms: 100
|
||||
+37
@@ -0,0 +1,37 @@
|
||||
name: undo_last
|
||||
version: "1.0"
|
||||
description: >
|
||||
Trigger an undo operation in the target application using a keyboard shortcut
|
||||
(Ctrl+Z). This macro demonstrates the semantic_ui backend.
|
||||
On Linux, requires xdotool. On macOS/Windows, adapt to your platform.
|
||||
tags: [undo, semantic_ui, example]
|
||||
|
||||
parameters:
|
||||
app_name:
|
||||
type: string
|
||||
required: false
|
||||
default: ""
|
||||
description: Name of the target application process (used to verify it is running).
|
||||
example: inkscape
|
||||
|
||||
preconditions:
|
||||
- always: true # process check is optional in this example
|
||||
|
||||
steps:
|
||||
- id: step_undo
|
||||
backend: semantic_ui
|
||||
action: shortcut
|
||||
params:
|
||||
keys: ctrl+z
|
||||
timeout_ms: 5000
|
||||
on_failure: fail
|
||||
|
||||
postconditions: []
|
||||
|
||||
outputs: []
|
||||
|
||||
agent_hints:
|
||||
danger_level: safe
|
||||
side_effects: [modifies_app_state]
|
||||
reversible: true # can be redone with ctrl+y
|
||||
estimated_duration_ms: 500
|
||||
@@ -0,0 +1,23 @@
|
||||
macros:
|
||||
- name: export_file
|
||||
path: examples/export_file.yaml
|
||||
version: "1.0"
|
||||
- name: undo_last
|
||||
path: examples/undo_last.yaml
|
||||
version: "1.0"
|
||||
- name: transform_json
|
||||
path: examples/transform_json.yaml
|
||||
version: "1.0"
|
||||
# ── Demo macros (gedit) ──────────────────────────────────────
|
||||
- name: gedit_new_window
|
||||
path: demo/gedit_new_window.yaml
|
||||
version: "1.0"
|
||||
- name: gedit_type_and_save
|
||||
path: demo/gedit_type_and_save.yaml
|
||||
version: "1.0"
|
||||
- name: gedit_save_as
|
||||
path: demo/gedit_save_as.yaml
|
||||
version: "1.0"
|
||||
- name: gedit_find_and_replace
|
||||
path: demo/gedit_find_and_replace.yaml
|
||||
version: "1.0"
|
||||
@@ -0,0 +1,977 @@
|
||||
#!/usr/bin/env python3
|
||||
"""MacroCLI — agent-callable interface for the Macro System.
|
||||
|
||||
This CLI is the L6 "Unified CLI Entry" in the MacroCLI.
|
||||
It provides a stable, machine-readable interface for AI agents and
|
||||
power users to invoke macros without touching the GUI.
|
||||
|
||||
Usage (one-shot):
|
||||
cli-anything-macrocli macro run export_file --param output=/tmp/out.png --json
|
||||
cli-anything-macrocli macro list --json
|
||||
cli-anything-macrocli macro info export_file --json
|
||||
|
||||
Usage (REPL):
|
||||
cli-anything-macrocli # enters interactive REPL
|
||||
cli-anything-macrocli repl
|
||||
"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
import json
|
||||
from pathlib import Path
|
||||
import click
|
||||
from typing import Optional
|
||||
|
||||
from cli_anything.macrocli.core.registry import MacroRegistry
|
||||
from cli_anything.macrocli.core.runtime import MacroRuntime
|
||||
from cli_anything.macrocli.core.session import ExecutionSession
|
||||
|
||||
# ── Global state ─────────────────────────────────────────────────────────────
|
||||
|
||||
_json_output = False
|
||||
_repl_mode = False
|
||||
_dry_run = False
|
||||
|
||||
_session: Optional[ExecutionSession] = None
|
||||
_runtime: Optional[MacroRuntime] = None
|
||||
|
||||
|
||||
def get_runtime() -> MacroRuntime:
|
||||
global _runtime, _session
|
||||
if _runtime is None:
|
||||
_session = _session or ExecutionSession()
|
||||
_runtime = MacroRuntime(session=_session)
|
||||
return _runtime
|
||||
|
||||
|
||||
def get_session() -> ExecutionSession:
|
||||
global _session
|
||||
if _session is None:
|
||||
_session = ExecutionSession()
|
||||
return _session
|
||||
|
||||
|
||||
# ── Output helpers ────────────────────────────────────────────────────────────
|
||||
|
||||
def output(data, message: str = ""):
|
||||
"""Print result: JSON in --json mode, human-readable otherwise."""
|
||||
if _json_output:
|
||||
click.echo(json.dumps(data, indent=2, default=str))
|
||||
else:
|
||||
if message:
|
||||
click.echo(message)
|
||||
_print_value(data)
|
||||
|
||||
|
||||
def _print_value(val, indent: int = 0):
|
||||
prefix = " " * indent
|
||||
if isinstance(val, dict):
|
||||
for k, v in val.items():
|
||||
if isinstance(v, (dict, list)):
|
||||
click.echo(f"{prefix}{k}:")
|
||||
_print_value(v, indent + 1)
|
||||
else:
|
||||
click.echo(f"{prefix}{k}: {v}")
|
||||
elif isinstance(val, list):
|
||||
for i, item in enumerate(val):
|
||||
if isinstance(item, dict):
|
||||
click.echo(f"{prefix}[{i}]")
|
||||
_print_value(item, indent + 1)
|
||||
else:
|
||||
click.echo(f"{prefix}- {item}")
|
||||
else:
|
||||
click.echo(f"{prefix}{val}")
|
||||
|
||||
|
||||
def handle_error(func):
|
||||
"""Decorator: consistent error handling across commands."""
|
||||
import functools
|
||||
|
||||
@functools.wraps(func)
|
||||
def wrapper(*args, **kwargs):
|
||||
try:
|
||||
return func(*args, **kwargs)
|
||||
except KeyError as e:
|
||||
msg = str(e).strip("'\"")
|
||||
if _json_output:
|
||||
click.echo(json.dumps({"error": msg, "type": "not_found"}))
|
||||
else:
|
||||
click.echo(f"Error: {msg}", err=True)
|
||||
if not _repl_mode:
|
||||
sys.exit(1)
|
||||
except FileNotFoundError as e:
|
||||
if _json_output:
|
||||
click.echo(json.dumps({"error": str(e), "type": "file_not_found"}))
|
||||
else:
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
if not _repl_mode:
|
||||
sys.exit(1)
|
||||
except Exception as e:
|
||||
if _json_output:
|
||||
click.echo(json.dumps({"error": str(e), "type": type(e).__name__}))
|
||||
else:
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
if not _repl_mode:
|
||||
sys.exit(1)
|
||||
|
||||
return wrapper
|
||||
|
||||
|
||||
# ── Parameter parsing ─────────────────────────────────────────────────────────
|
||||
|
||||
def _parse_params(param_tuples: tuple) -> dict:
|
||||
"""Convert --param key=value tuples to a dict."""
|
||||
result = {}
|
||||
for pair in param_tuples:
|
||||
if "=" in pair:
|
||||
k, v = pair.split("=", 1)
|
||||
result[k.strip()] = v.strip()
|
||||
else:
|
||||
click.echo(f"Warning: --param '{pair}' ignored (expected key=value format).", err=True)
|
||||
return result
|
||||
|
||||
|
||||
# ── Main CLI group ────────────────────────────────────────────────────────────
|
||||
|
||||
@click.group(invoke_without_command=True)
|
||||
@click.option("--json", "json_flag", is_flag=True, help="Machine-readable JSON output.")
|
||||
@click.option("--dry-run", "dry_run_flag", is_flag=True,
|
||||
help="Simulate execution without side effects.")
|
||||
@click.option("--session-id", default=None, help="Resume or create a named session.")
|
||||
@click.pass_context
|
||||
def cli(ctx, json_flag, dry_run_flag, session_id):
|
||||
"""MacroCLI — run GUI workflows as CLI commands.
|
||||
|
||||
\b
|
||||
Quick start:
|
||||
cli-anything-macrocli macro list
|
||||
cli-anything-macrocli macro info <name>
|
||||
cli-anything-macrocli macro run <name> --param key=value
|
||||
|
||||
Enter interactive REPL by running without arguments.
|
||||
"""
|
||||
global _json_output, _dry_run, _session
|
||||
|
||||
_json_output = json_flag
|
||||
_dry_run = dry_run_flag
|
||||
|
||||
if session_id:
|
||||
loaded = ExecutionSession.load(session_id)
|
||||
_session = loaded or ExecutionSession(session_id=session_id)
|
||||
|
||||
ctx.ensure_object(dict)
|
||||
|
||||
if ctx.invoked_subcommand is None:
|
||||
ctx.invoke(repl)
|
||||
|
||||
|
||||
# ── macro group ──────────────────────────────────────────────────────────────
|
||||
|
||||
@cli.group()
|
||||
def macro():
|
||||
"""Macro management and execution."""
|
||||
|
||||
|
||||
@macro.command("run")
|
||||
@click.argument("name")
|
||||
@click.option("--param", "-p", multiple=True,
|
||||
help="Macro parameter in key=value format. Repeat for multiple.")
|
||||
@click.option("--macro-file", default=None,
|
||||
help="Run a macro directly from a YAML file path (bypasses registry).")
|
||||
@handle_error
|
||||
def macro_run(name, param, macro_file):
|
||||
"""Execute a macro by name, or from a YAML file with --macro-file.
|
||||
|
||||
\b
|
||||
Example:
|
||||
macro run export_file --param output=/tmp/out.txt
|
||||
macro run my_macro --macro-file /tmp/recording/my_macro.yaml --param key=val
|
||||
"""
|
||||
params = _parse_params(param)
|
||||
|
||||
if macro_file:
|
||||
# Load macro directly from file, bypassing the registry
|
||||
from cli_anything.macrocli.core.macro_model import load_from_yaml
|
||||
from cli_anything.macrocli.core.routing import RoutingEngine
|
||||
from cli_anything.macrocli.core.runtime import MacroRuntime, ExecutionResult
|
||||
from cli_anything.macrocli.core.registry import MacroRegistry
|
||||
try:
|
||||
macro_def = load_from_yaml(macro_file)
|
||||
except Exception as e:
|
||||
if _json_output:
|
||||
click.echo(json.dumps({"success": False, "error": str(e)}))
|
||||
else:
|
||||
click.echo(f"Error loading macro file: {e}", err=True)
|
||||
if not _repl_mode:
|
||||
sys.exit(1)
|
||||
return
|
||||
reg = MacroRegistry.__new__(MacroRegistry)
|
||||
reg._cache = {macro_def.name: macro_def}
|
||||
reg._scanned = True
|
||||
reg.macros_dir = None
|
||||
runtime = MacroRuntime(registry=reg, session=get_session())
|
||||
result = runtime.execute(macro_def.name, params, dry_run=_dry_run)
|
||||
else:
|
||||
runtime = get_runtime()
|
||||
result = runtime.execute(name, params, dry_run=_dry_run)
|
||||
|
||||
if _json_output:
|
||||
output(result.to_dict())
|
||||
else:
|
||||
if result.success:
|
||||
click.echo(f"✓ Macro '{name}' completed successfully.")
|
||||
if result.output:
|
||||
for k, v in result.output.items():
|
||||
if not k.startswith("_"):
|
||||
click.echo(f" {k}: {v}")
|
||||
else:
|
||||
click.echo(f"✗ Macro '{name}' failed.", err=True)
|
||||
click.echo(f" {result.error}", err=True)
|
||||
if result.telemetry:
|
||||
click.echo(
|
||||
f" [{result.telemetry.get('duration_ms', 0):.0f}ms, "
|
||||
f"backends: {', '.join(result.telemetry.get('backends_used', []))}]"
|
||||
)
|
||||
if not result.success and not _repl_mode:
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
@macro.command("list")
|
||||
@handle_error
|
||||
def macro_list():
|
||||
"""List all available macros."""
|
||||
runtime = get_runtime()
|
||||
macros = runtime.registry.list_all()
|
||||
|
||||
if _json_output:
|
||||
output([{
|
||||
"name": m.name,
|
||||
"version": m.version,
|
||||
"description": m.description,
|
||||
"tags": m.tags,
|
||||
"parameters": list(m.parameters.keys()),
|
||||
} for m in macros])
|
||||
else:
|
||||
if not macros:
|
||||
click.echo("No macros found.")
|
||||
return
|
||||
click.echo(f"Available macros ({len(macros)}):\n")
|
||||
for m in macros:
|
||||
tags = f" [{', '.join(m.tags)}]" if m.tags else ""
|
||||
click.echo(f" {m.name:<30} {m.description}{tags}")
|
||||
|
||||
|
||||
@macro.command("info")
|
||||
@click.argument("name")
|
||||
@handle_error
|
||||
def macro_info(name):
|
||||
"""Show full details for a macro (schema, parameters, steps)."""
|
||||
runtime = get_runtime()
|
||||
m = runtime.registry.load(name)
|
||||
|
||||
if _json_output:
|
||||
output(m.to_dict())
|
||||
else:
|
||||
click.echo(f"\nMacro: {m.name} (v{m.version})")
|
||||
click.echo(f" {m.description}\n")
|
||||
|
||||
if m.parameters:
|
||||
click.echo("Parameters:")
|
||||
for pname, pspec in m.parameters.items():
|
||||
req = "(required)" if pspec.required else f"(default: {pspec.default!r})"
|
||||
click.echo(f" --param {pname}=<{pspec.type}> {req}")
|
||||
if pspec.description:
|
||||
click.echo(f" {pspec.description}")
|
||||
|
||||
if m.preconditions:
|
||||
click.echo(f"\nPreconditions ({len(m.preconditions)}):")
|
||||
for c in m.preconditions:
|
||||
click.echo(f" {c.type}: {c.args}")
|
||||
|
||||
if m.steps:
|
||||
click.echo(f"\nSteps ({len(m.steps)}):")
|
||||
for s in m.steps:
|
||||
click.echo(f" [{s.id}] backend={s.backend} action={s.action}")
|
||||
|
||||
if m.postconditions:
|
||||
click.echo(f"\nPostconditions ({len(m.postconditions)}):")
|
||||
for c in m.postconditions:
|
||||
click.echo(f" {c.type}: {c.args}")
|
||||
|
||||
if m.outputs:
|
||||
click.echo(f"\nOutputs:")
|
||||
for o in m.outputs:
|
||||
click.echo(f" {o.name}: {o.path or o.value}")
|
||||
|
||||
if m.agent_hints:
|
||||
click.echo(f"\nAgent hints: {m.agent_hints}")
|
||||
click.echo()
|
||||
|
||||
|
||||
@macro.command("validate")
|
||||
@click.argument("name", required=False)
|
||||
@handle_error
|
||||
def macro_validate(name):
|
||||
"""Validate macro definition(s). Pass a name or omit to validate all."""
|
||||
runtime = get_runtime()
|
||||
|
||||
if name:
|
||||
names = [name]
|
||||
else:
|
||||
names = runtime.registry.list_names()
|
||||
|
||||
results = {}
|
||||
for n in names:
|
||||
errors = runtime.validate_macro(n)
|
||||
results[n] = errors
|
||||
|
||||
if _json_output:
|
||||
output({n: {"valid": len(e) == 0, "errors": e} for n, e in results.items()})
|
||||
else:
|
||||
all_ok = True
|
||||
for n, errors in results.items():
|
||||
if errors:
|
||||
all_ok = False
|
||||
click.echo(f"✗ {n}:")
|
||||
for err in errors:
|
||||
click.echo(f" - {err}", err=True)
|
||||
else:
|
||||
click.echo(f"✓ {n}")
|
||||
if all_ok:
|
||||
click.echo("\nAll macros valid.")
|
||||
else:
|
||||
if not _repl_mode:
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
@macro.command("dry-run")
|
||||
@click.argument("name")
|
||||
@click.option("--param", "-p", multiple=True, help="Parameter in key=value format.")
|
||||
@handle_error
|
||||
def macro_dry_run(name, param):
|
||||
"""Simulate macro execution without any side effects."""
|
||||
params = _parse_params(param)
|
||||
runtime = get_runtime()
|
||||
result = runtime.execute(name, params, dry_run=True)
|
||||
|
||||
if _json_output:
|
||||
output(result.to_dict())
|
||||
else:
|
||||
click.echo(f"[dry-run] Macro '{name}'")
|
||||
if result.success:
|
||||
click.echo(" Would execute successfully.")
|
||||
click.echo(f" Steps: {len(result.step_results)}")
|
||||
else:
|
||||
click.echo(f" Would fail: {result.error}", err=True)
|
||||
|
||||
|
||||
@macro.command("define")
|
||||
@click.argument("name")
|
||||
@click.option("--output", "-o", default=None, help="Write YAML to this file path.")
|
||||
@handle_error
|
||||
def macro_define(name, output):
|
||||
"""Scaffold a new macro YAML definition."""
|
||||
import textwrap
|
||||
template = textwrap.dedent(f"""\
|
||||
name: {name}
|
||||
version: "1.0"
|
||||
description: "Describe what this macro does."
|
||||
tags: []
|
||||
|
||||
parameters:
|
||||
# Add your parameters here
|
||||
# output:
|
||||
# type: string
|
||||
# required: true
|
||||
# description: Output file path
|
||||
# example: /tmp/result.txt
|
||||
|
||||
preconditions:
|
||||
# Conditions that must be true before execution
|
||||
# - file_exists: /path/to/input
|
||||
# - process_running: my-app
|
||||
|
||||
steps:
|
||||
- id: step_1
|
||||
backend: native_api # or: file_transform, semantic_ui, gui_macro
|
||||
action: run_command
|
||||
params:
|
||||
command: [echo, "Hello from {name}"]
|
||||
timeout_ms: 30000
|
||||
on_failure: fail # or: skip, continue
|
||||
|
||||
postconditions:
|
||||
# Conditions verified after execution
|
||||
# - file_exists: ${{output}}
|
||||
|
||||
outputs:
|
||||
# Named outputs the agent can use
|
||||
# - name: result_file
|
||||
# path: ${{output}}
|
||||
|
||||
agent_hints:
|
||||
danger_level: safe # safe | moderate | dangerous
|
||||
side_effects: []
|
||||
reversible: true
|
||||
""")
|
||||
if output:
|
||||
from pathlib import Path
|
||||
p = Path(output)
|
||||
p.parent.mkdir(parents=True, exist_ok=True)
|
||||
p.write_text(template, encoding="utf-8")
|
||||
if _json_output:
|
||||
click.echo(json.dumps({"created": str(p.resolve())}))
|
||||
else:
|
||||
click.echo(f"✓ Macro scaffold written to: {p.resolve()}")
|
||||
else:
|
||||
click.echo(template)
|
||||
|
||||
|
||||
@macro.command("record")
|
||||
@click.argument("name")
|
||||
@click.option("--output-dir", "-d", default=".",
|
||||
help="Directory to write the macro package folder.")
|
||||
@click.option("--timeout", default=0, type=float,
|
||||
help="Auto-stop after N seconds (0 = wait for Ctrl+Alt+S).")
|
||||
@click.option("--agent-review", "do_agent_review", is_flag=True,
|
||||
help="After recording, review each step and mark some as "
|
||||
"agent steps with descriptions and end-state snapshots.")
|
||||
@click.option("--parameterize", "do_parameterize", is_flag=True,
|
||||
help="After recording, interactively choose which typed values "
|
||||
"become CLI parameters.")
|
||||
@click.option("--auto-parameterize", "do_auto_param", is_flag=True,
|
||||
help="After recording, use an LLM to automatically suggest "
|
||||
"parameter names (requires --api-key or MACROCLI_API_KEY).")
|
||||
@click.option("--api-key", default=None, envvar="MACROCLI_API_KEY",
|
||||
help="API key for --auto-parameterize.")
|
||||
@handle_error
|
||||
def macro_record(name, output_dir, timeout, do_agent_review,
|
||||
do_parameterize, do_auto_param, api_key):
|
||||
"""Record GUI interactions and generate a macro YAML package.
|
||||
|
||||
\b
|
||||
The macro is saved as a folder:
|
||||
<name>/
|
||||
macro.yaml
|
||||
snapshots/ (end-state screenshots for agent steps)
|
||||
|
||||
\b
|
||||
Examples:
|
||||
# Basic recording
|
||||
macro record my_export
|
||||
|
||||
# Record + mark agent steps interactively
|
||||
macro record my_export --agent-review
|
||||
|
||||
# Record + agent review + parameterize typed values
|
||||
macro record my_export --agent-review --parameterize
|
||||
|
||||
Requires: pip install mss Pillow pynput
|
||||
"""
|
||||
try:
|
||||
from cli_anything.macrocli.core.recorder import MacroRecorder
|
||||
except ImportError as e:
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
sys.exit(1)
|
||||
|
||||
if do_parameterize and do_auto_param:
|
||||
click.echo(
|
||||
"Error: --parameterize and --auto-parameterize are mutually exclusive.",
|
||||
err=True,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
recorder = MacroRecorder(macro_name=name, output_dir=output_dir)
|
||||
|
||||
if not _json_output:
|
||||
click.echo(f"Recording '{name}'. Press Ctrl+Alt+S to stop...")
|
||||
|
||||
try:
|
||||
recorder.record(timeout_s=timeout if timeout > 0 else None)
|
||||
except Exception as e:
|
||||
if _json_output:
|
||||
output({"error": str(e), "success": False})
|
||||
else:
|
||||
click.echo(f"Error during recording: {e}", err=True)
|
||||
if not _repl_mode:
|
||||
sys.exit(1)
|
||||
return
|
||||
|
||||
# ── Agent step review ─────────────────────────────────────────────────────
|
||||
if do_agent_review and recorder._steps:
|
||||
snap_dir = Path(output_dir) / name / "snapshots"
|
||||
recorder.interactive_agent_review(snapshots_dir=str(snap_dir))
|
||||
|
||||
# ── Parameterization phase ────────────────────────────────────────────────
|
||||
parameters = None
|
||||
type_steps = recorder.get_type_steps()
|
||||
|
||||
if do_auto_param and type_steps:
|
||||
try:
|
||||
from cli_anything.macrocli.core.parameterize import (
|
||||
llm_suggest_parameters,
|
||||
interactive_parameterize,
|
||||
)
|
||||
if not _json_output:
|
||||
click.echo(f"\nAsking LLM to suggest parameters...")
|
||||
suggestions = llm_suggest_parameters(type_steps, api_key=api_key)
|
||||
if suggestions and not _json_output:
|
||||
click.echo(" LLM suggestions:")
|
||||
for idx, pname in suggestions.items():
|
||||
step = recorder._steps[idx]
|
||||
click.echo(f" step {idx+1} {step.text!r} → ${{{pname}}}")
|
||||
click.echo()
|
||||
confirmed = interactive_parameterize(
|
||||
[(i, s) for i, s in type_steps if i in suggestions],
|
||||
)
|
||||
final = {**suggestions, **confirmed}
|
||||
parameters = recorder.apply_parameterization(final)
|
||||
elif not suggestions and not _json_output:
|
||||
click.echo(" LLM found no values to parameterize.")
|
||||
except Exception as e:
|
||||
click.echo(f" Warning: LLM parameterization failed: {e}", err=True)
|
||||
do_parameterize = True
|
||||
|
||||
if do_parameterize and type_steps:
|
||||
from cli_anything.macrocli.core.parameterize import interactive_parameterize
|
||||
assignments = interactive_parameterize(type_steps)
|
||||
if assignments:
|
||||
parameters = recorder.apply_parameterization(assignments)
|
||||
|
||||
# ── Save as package ───────────────────────────────────────────────────────
|
||||
try:
|
||||
yaml_path = recorder.save_as_package(
|
||||
output_dir=output_dir,
|
||||
parameters=parameters,
|
||||
)
|
||||
except Exception as e:
|
||||
if _json_output:
|
||||
output({"error": str(e), "success": False})
|
||||
else:
|
||||
click.echo(f"Error saving macro: {e}", err=True)
|
||||
if not _repl_mode:
|
||||
sys.exit(1)
|
||||
return
|
||||
|
||||
pkg_dir = str(Path(output_dir) / name)
|
||||
agent_count = sum(1 for s in recorder._steps if s.is_agent_step)
|
||||
|
||||
if _json_output:
|
||||
output({
|
||||
"success": True,
|
||||
"yaml_path": yaml_path,
|
||||
"package_dir": pkg_dir,
|
||||
"steps": len(recorder._steps),
|
||||
"agent_steps": agent_count,
|
||||
"parameters": list((parameters or {}).keys()),
|
||||
})
|
||||
else:
|
||||
click.echo(f"✓ Saved {len(recorder._steps)} steps to: {pkg_dir}/")
|
||||
if agent_count:
|
||||
click.echo(f" Agent steps: {agent_count} (will use vision model at runtime)")
|
||||
if parameters:
|
||||
click.echo(f" Parameters: {', '.join(parameters.keys())}")
|
||||
click.echo(
|
||||
f"\n Run with:\n"
|
||||
f" macro run {name} --macro-file {yaml_path}"
|
||||
+ (
|
||||
"".join(f" --param {k}=<value>" for k in (parameters or {}))
|
||||
if parameters else ""
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@macro.command("parameterize")
|
||||
@click.argument("yaml_file")
|
||||
@click.option("--auto", "do_auto", is_flag=True,
|
||||
help="Use an LLM to suggest parameter names automatically.")
|
||||
@click.option("--api-key", default=None, envvar="MACROCLI_API_KEY",
|
||||
help="API key for --auto.")
|
||||
@handle_error
|
||||
def macro_parameterize(yaml_file, do_auto, api_key):
|
||||
"""Interactively parameterize typed values in an existing macro YAML.
|
||||
|
||||
\b
|
||||
Finds all hardcoded type_text steps in the file and lets you choose
|
||||
which values become CLI parameters (e.g. ${output_path}).
|
||||
|
||||
\b
|
||||
Examples:
|
||||
macro parameterize /tmp/recording/my_export.yaml
|
||||
macro parameterize my_export.yaml --auto --api-key $MACROCLI_API_KEY
|
||||
"""
|
||||
from cli_anything.macrocli.core.parameterize import (
|
||||
parameterize_yaml_file,
|
||||
llm_suggest_parameters,
|
||||
interactive_parameterize,
|
||||
_YamlTypeStep,
|
||||
)
|
||||
|
||||
p = Path(yaml_file)
|
||||
if not p.is_file():
|
||||
click.echo(f"Error: file not found: {yaml_file}", err=True)
|
||||
if not _repl_mode:
|
||||
sys.exit(1)
|
||||
return
|
||||
|
||||
if do_auto:
|
||||
# Load the file, extract type_text steps, ask LLM, then apply
|
||||
import yaml as _yaml
|
||||
with open(p, encoding="utf-8") as f:
|
||||
macro_dict = _yaml.safe_load(f)
|
||||
|
||||
steps = macro_dict.get("steps") or []
|
||||
type_steps_raw = [
|
||||
(i, s) for i, s in enumerate(steps)
|
||||
if isinstance(s, dict)
|
||||
and s.get("action") == "type_text"
|
||||
and not s.get("params", {}).get("text", "").startswith("${")
|
||||
]
|
||||
if not type_steps_raw:
|
||||
click.echo("No hardcoded type_text steps found.")
|
||||
return
|
||||
|
||||
wrapped = [(i, _YamlTypeStep(i, s)) for i, s in type_steps_raw]
|
||||
|
||||
try:
|
||||
click.echo(f"Asking LLM to suggest parameters for "
|
||||
f"{len(wrapped)} type_text step(s)...")
|
||||
suggestions = llm_suggest_parameters(wrapped, api_key=api_key)
|
||||
except Exception as e:
|
||||
click.echo(f"LLM failed: {e}\nFalling back to interactive.", err=True)
|
||||
suggestions = {}
|
||||
do_auto = False
|
||||
|
||||
if suggestions:
|
||||
click.echo(" Suggestions:")
|
||||
for idx, pname in suggestions.items():
|
||||
w = next(w for i, w in wrapped if i == idx)
|
||||
click.echo(f" step {idx+1} {w.text!r} → ${{{pname}}}")
|
||||
click.echo()
|
||||
|
||||
# Let user confirm (pre-fill LLM suggestions as defaults)
|
||||
existing = set((macro_dict.get("parameters") or {}).keys())
|
||||
confirmed = interactive_parameterize(
|
||||
[(i, w) for i, w in wrapped if i in suggestions],
|
||||
existing_params=existing,
|
||||
)
|
||||
final = {**suggestions, **confirmed}
|
||||
|
||||
if not final:
|
||||
click.echo("No parameters applied.")
|
||||
return
|
||||
|
||||
# Apply and save
|
||||
import yaml as _yaml
|
||||
from cli_anything.macrocli.core.recorder import MacroRecorder
|
||||
parameters: dict = dict(macro_dict.get("parameters") or {})
|
||||
for idx, param_name in final.items():
|
||||
w = next(w for i, w in wrapped if i == idx)
|
||||
original = w.text
|
||||
w.apply(param_name)
|
||||
ptype = "string"
|
||||
try:
|
||||
int(original); ptype = "integer"
|
||||
except ValueError:
|
||||
try:
|
||||
float(original); ptype = "float"
|
||||
except ValueError:
|
||||
pass
|
||||
parameters[param_name] = {
|
||||
"type": ptype, "required": True,
|
||||
"description": f"Value typed at step {idx+1}",
|
||||
"example": original,
|
||||
}
|
||||
macro_dict["parameters"] = parameters
|
||||
with open(p, "w", encoding="utf-8") as f:
|
||||
_yaml.dump(macro_dict, f, allow_unicode=True,
|
||||
sort_keys=False, default_flow_style=False)
|
||||
if _json_output:
|
||||
output({"success": True, "file": str(p.resolve()),
|
||||
"parameters": list(final.values())})
|
||||
else:
|
||||
click.echo(f"✓ Updated: {p.resolve()}")
|
||||
click.echo(f" Parameters added: {', '.join(final.values())}")
|
||||
else:
|
||||
changed = parameterize_yaml_file(yaml_file)
|
||||
if _json_output:
|
||||
output({"success": True, "changed": changed, "file": str(p.resolve())})
|
||||
|
||||
|
||||
@macro.command("assist")
|
||||
@click.argument("name")
|
||||
@click.option("--goal", "-g", required=True,
|
||||
help="Natural language goal (what the macro should do).")
|
||||
@click.option("--screenshot", default="current",
|
||||
help="'current' to take a screenshot now, or path to an image file.")
|
||||
@click.option("--output", "-o", default=None,
|
||||
help="Output YAML file path (default: <name>.yaml).")
|
||||
@click.option("--api-key", default=None, envvar="MACROCLI_API_KEY",
|
||||
help="API key (or set MACROCLI_API_KEY env var).")
|
||||
@click.option("--model", default=None,
|
||||
help="Model name (or set MACROCLI_MODEL env var).")
|
||||
@handle_error
|
||||
def macro_assist(name, goal, screenshot, output, api_key, model):
|
||||
"""Generate a macro YAML from a screenshot using a vision model (optional).
|
||||
|
||||
\b
|
||||
Takes a screenshot, sends it to the configured model with your goal, and
|
||||
generates a macro YAML. Steps that require visual templates will include
|
||||
instructions for which template images to capture.
|
||||
|
||||
Requires: pip install openai mss Pillow
|
||||
|
||||
\b
|
||||
Example:
|
||||
macro assist export_png \\
|
||||
--goal "Export the current diagram as PNG to /tmp/out.png" \\
|
||||
--api-key $MACROCLI_API_KEY
|
||||
"""
|
||||
try:
|
||||
from cli_anything.macrocli.core.llm_assist import generate_macro
|
||||
except ImportError as e:
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
sys.exit(1)
|
||||
|
||||
if not _json_output:
|
||||
click.echo(f"Sending screenshot to model ({model or os.environ.get('MACROCLI_MODEL', 'unset')})...")
|
||||
|
||||
result = generate_macro(
|
||||
goal=goal,
|
||||
macro_name=name,
|
||||
screenshot_source=screenshot,
|
||||
api_key=api_key,
|
||||
model=model,
|
||||
output_path=output,
|
||||
)
|
||||
|
||||
if _json_output:
|
||||
output(result)
|
||||
else:
|
||||
click.echo(f"✓ Generated {result['steps_count']} steps → {result['yaml_path']}")
|
||||
if result["warnings"]:
|
||||
for w in result["warnings"]:
|
||||
click.echo(f" ⚠ {w}")
|
||||
if result.get("templates_to_capture"):
|
||||
click.echo("\n Templates to capture (use 'macro capture-template'):")
|
||||
for t in result["templates_to_capture"]:
|
||||
click.echo(f" {t['template_path']}: {t['description']}")
|
||||
|
||||
|
||||
@macro.command("capture-template")
|
||||
@click.argument("output_path")
|
||||
@click.option("--x", type=int, required=True, help="Left edge of region.")
|
||||
@click.option("--y", type=int, required=True, help="Top edge of region.")
|
||||
@click.option("--width", type=int, required=True, help="Region width in pixels.")
|
||||
@click.option("--height", type=int, required=True, help="Region height in pixels.")
|
||||
@handle_error
|
||||
def macro_capture_template(output_path, x, y, width, height):
|
||||
"""Capture a screen region and save it as a template image.
|
||||
|
||||
\b
|
||||
Use this to create the template PNG files that visual_anchor macros need.
|
||||
|
||||
\b
|
||||
Example:
|
||||
macro capture-template templates/export_button.png \\
|
||||
--x 245 --y 110 --width 80 --height 30
|
||||
|
||||
Requires: pip install mss Pillow
|
||||
"""
|
||||
try:
|
||||
from cli_anything.macrocli.backends.visual_anchor import VisualAnchorBackend
|
||||
from cli_anything.macrocli.backends.base import BackendContext
|
||||
from cli_anything.macrocli.core.macro_model import MacroStep
|
||||
except ImportError as e:
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
sys.exit(1)
|
||||
|
||||
va = VisualAnchorBackend()
|
||||
step = MacroStep(id="capture", backend="visual_anchor", action="capture_region",
|
||||
params={"output": output_path, "x": x, "y": y,
|
||||
"width": width, "height": height})
|
||||
ctx = BackendContext(params={})
|
||||
result = va.execute(step, {}, ctx)
|
||||
|
||||
if _json_output:
|
||||
output(result.to_dict())
|
||||
else:
|
||||
if result.success:
|
||||
click.echo(f"✓ Template saved: {result.output.get('saved')}")
|
||||
click.echo(f" Size: {result.output.get('file_size', 0)} bytes")
|
||||
else:
|
||||
click.echo(f"✗ {result.error}", err=True)
|
||||
if not _repl_mode:
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
# ── session group ─────────────────────────────────────────────────────────────
|
||||
|
||||
@cli.group()
|
||||
def session():
|
||||
"""Session management and run history."""
|
||||
|
||||
|
||||
@session.command("status")
|
||||
@handle_error
|
||||
def session_status():
|
||||
"""Show current session status and statistics."""
|
||||
sess = get_session()
|
||||
data = sess.status()
|
||||
output(data, "Session status:")
|
||||
|
||||
|
||||
@session.command("history")
|
||||
@click.option("--limit", default=10, show_default=True, help="Number of records to show.")
|
||||
@handle_error
|
||||
def session_history(limit):
|
||||
"""Show recent macro execution history."""
|
||||
sess = get_session()
|
||||
records = sess.history(limit=limit)
|
||||
|
||||
if _json_output:
|
||||
output([r.to_dict() for r in records])
|
||||
else:
|
||||
if not records:
|
||||
click.echo("No runs recorded in this session.")
|
||||
return
|
||||
click.echo(f"Recent runs ({len(records)}):\n")
|
||||
for r in records:
|
||||
status = "✓" if r.success else "✗"
|
||||
import datetime
|
||||
ts = datetime.datetime.fromtimestamp(r.timestamp).strftime("%H:%M:%S")
|
||||
click.echo(f" {status} [{ts}] {r.macro_name} ({r.duration_ms:.0f}ms)")
|
||||
if not r.success:
|
||||
click.echo(f" Error: {r.error}", err=True)
|
||||
|
||||
|
||||
@session.command("save")
|
||||
@handle_error
|
||||
def session_save():
|
||||
"""Persist current session to disk."""
|
||||
sess = get_session()
|
||||
path = sess.save()
|
||||
output({"saved": path, "session_id": sess.session_id},
|
||||
f"Session saved: {path}")
|
||||
|
||||
|
||||
@session.command("list")
|
||||
@handle_error
|
||||
def session_list():
|
||||
"""List all saved sessions."""
|
||||
sessions = ExecutionSession.list_sessions()
|
||||
if _json_output:
|
||||
output(sessions)
|
||||
else:
|
||||
if not sessions:
|
||||
click.echo("No saved sessions.")
|
||||
return
|
||||
click.echo("Saved sessions:\n")
|
||||
for s in sessions:
|
||||
import datetime
|
||||
ts = datetime.datetime.fromtimestamp(s.get("timestamp", 0)).strftime("%Y-%m-%d %H:%M:%S")
|
||||
click.echo(f" {s['session_id']} ({s['runs']} runs) {ts}")
|
||||
|
||||
|
||||
# ── backends command ──────────────────────────────────────────────────────────
|
||||
|
||||
@cli.command()
|
||||
@handle_error
|
||||
def backends():
|
||||
"""Show available execution backends and their status."""
|
||||
runtime = get_runtime()
|
||||
data = runtime.routing.describe()
|
||||
if _json_output:
|
||||
output(data)
|
||||
else:
|
||||
click.echo("Execution backends:\n")
|
||||
for name, info in sorted(data.items(), key=lambda x: -x[1].get("priority", 0)):
|
||||
status = "✓" if info.get("available") else "✗"
|
||||
click.echo(
|
||||
f" {status} {name:<20} priority={info.get('priority', '?'):<5}"
|
||||
f" available={info.get('available')}"
|
||||
)
|
||||
|
||||
|
||||
# ── repl command ──────────────────────────────────────────────────────────────
|
||||
|
||||
@cli.command()
|
||||
@click.pass_context
|
||||
def repl(ctx):
|
||||
"""Enter the interactive REPL (default when no command given)."""
|
||||
global _repl_mode
|
||||
_repl_mode = True
|
||||
|
||||
from cli_anything.macrocli.utils.repl_skin import ReplSkin
|
||||
skin = ReplSkin("macrocli", version="1.0.0")
|
||||
skin.print_banner()
|
||||
|
||||
runtime = get_runtime()
|
||||
|
||||
# Show quick summary on startup
|
||||
macros = runtime.registry.list_all()
|
||||
skin.info(f"{len(macros)} macros loaded. Type 'macro list' to see them.")
|
||||
skin.info("Type 'help' for commands, 'quit' to exit.\n")
|
||||
|
||||
pt_session = skin.create_prompt_session()
|
||||
session_obj = get_session()
|
||||
|
||||
while True:
|
||||
try:
|
||||
line = skin.get_input(
|
||||
pt_session,
|
||||
context=f"{session_obj.session_id[:12]}",
|
||||
)
|
||||
except (EOFError, KeyboardInterrupt):
|
||||
skin.print_goodbye()
|
||||
break
|
||||
|
||||
if not line:
|
||||
continue
|
||||
if line.lower() in ("quit", "exit", "q"):
|
||||
skin.print_goodbye()
|
||||
break
|
||||
if line.lower() in ("help", "?"):
|
||||
skin.help({
|
||||
"macro list": "List all available macros",
|
||||
"macro info <name>": "Show macro schema",
|
||||
"macro run <name> [--param k=v ...]": "Execute a macro",
|
||||
"macro dry-run <name>": "Simulate without side effects",
|
||||
"macro validate [name]": "Validate macro definitions",
|
||||
"macro define <name>": "Scaffold a new macro YAML",
|
||||
"session status": "Show session statistics",
|
||||
"session history": "Show recent runs",
|
||||
"backends": "Show backend availability",
|
||||
"quit": "Exit the REPL",
|
||||
})
|
||||
continue
|
||||
|
||||
# Parse and dispatch via Click's standalone_mode=False
|
||||
import shlex
|
||||
try:
|
||||
args = shlex.split(line)
|
||||
except ValueError as e:
|
||||
skin.error(f"Parse error: {e}")
|
||||
continue
|
||||
|
||||
try:
|
||||
ctx_obj = cli.make_context(
|
||||
"cli-anything-macrocli",
|
||||
args,
|
||||
standalone_mode=False,
|
||||
parent=ctx,
|
||||
)
|
||||
with ctx_obj:
|
||||
cli.invoke(ctx_obj)
|
||||
except SystemExit:
|
||||
pass
|
||||
except click.ClickException as e:
|
||||
skin.error(str(e))
|
||||
except Exception as e:
|
||||
skin.error(str(e))
|
||||
|
||||
|
||||
# ── Entry point ───────────────────────────────────────────────────────────────
|
||||
|
||||
if __name__ == "__main__":
|
||||
cli()
|
||||
@@ -0,0 +1,220 @@
|
||||
---
|
||||
name: cli-anything-macrocli
|
||||
description: >
|
||||
Use when the agent wants to define, list, inspect, or execute GUI macros
|
||||
via the MacroCLI CLI. Macros are parameterized, CLI-callable
|
||||
workflows — the agent invokes `macro run <name>` and the system handles
|
||||
backend routing (plugin, file transform, accessibility, compiled GUI replay).
|
||||
---
|
||||
|
||||
# MacroCLI CLI
|
||||
|
||||
## What It Is
|
||||
|
||||
The MacroCLI converts valuable GUI workflows into parameterized,
|
||||
CLI-callable macros. Agents **never touch the GUI directly** — they call macros
|
||||
through this stable CLI, and the runtime routes execution to the best available
|
||||
backend (native plugin/API, file transformation, semantic UI control, or
|
||||
precompiled GUI macro replay).
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
cd macrocli/agent-harness
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
**Requirements:** Python 3.10+, PyYAML, click, prompt-toolkit.
|
||||
|
||||
## Quick Start (for agents)
|
||||
|
||||
```bash
|
||||
# 1. See what macros are available
|
||||
cli-anything-macrocli macro list --json
|
||||
|
||||
# 2. Inspect a macro's parameters
|
||||
cli-anything-macrocli macro info export_file --json
|
||||
|
||||
# 3. Dry-run to check params without side effects
|
||||
cli-anything-macrocli --dry-run macro run export_file \
|
||||
--param output=/tmp/test.txt --json
|
||||
|
||||
# 4. Execute a macro
|
||||
cli-anything-macrocli macro run export_file \
|
||||
--param output=/tmp/result.txt --json
|
||||
|
||||
# 5. See what backends are available
|
||||
cli-anything-macrocli backends --json
|
||||
```
|
||||
|
||||
## Command Reference
|
||||
|
||||
### Global Flags
|
||||
|
||||
| Flag | Description |
|
||||
|------|-------------|
|
||||
| `--json` | Machine-readable JSON output on stdout |
|
||||
| `--dry-run` | Simulate all steps, skip side effects |
|
||||
| `--session-id <id>` | Resume or create a named session |
|
||||
|
||||
### `macro` group
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `macro list` | List all available macros |
|
||||
| `macro info <name>` | Show macro schema (parameters, steps, conditions) |
|
||||
| `macro run <name> --param k=v` | Execute a macro |
|
||||
| `macro dry-run <name> --param k=v` | Simulate without side effects |
|
||||
| `macro validate [name]` | Structural validation |
|
||||
| `macro define <name>` | Scaffold a new macro YAML |
|
||||
|
||||
### `session` group
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `session status` | Show session statistics |
|
||||
| `session history` | Show recent run history |
|
||||
| `session save` | Persist session to disk |
|
||||
| `session list` | List all saved sessions |
|
||||
|
||||
### `backends`
|
||||
|
||||
```bash
|
||||
cli-anything-macrocli backends --json
|
||||
# Shows: native_api, file_transform, semantic_ui, gui_macro, recovery
|
||||
# and whether each is available in the current environment.
|
||||
```
|
||||
|
||||
## Macro Parameters
|
||||
|
||||
Pass parameters with `--param key=value`. Repeat for multiple:
|
||||
|
||||
```bash
|
||||
cli-anything-macrocli macro run transform_json \
|
||||
--param file=/path/to/data.json \
|
||||
--param key=settings.theme \
|
||||
--param value=dark \
|
||||
--json
|
||||
```
|
||||
|
||||
## Output Format (--json)
|
||||
|
||||
All commands output JSON when `--json` is set:
|
||||
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"macro_name": "export_file",
|
||||
"output": {
|
||||
"exported_file": "/tmp/result.txt"
|
||||
},
|
||||
"error": "",
|
||||
"telemetry": {
|
||||
"duration_ms": 312,
|
||||
"steps_total": 2,
|
||||
"steps_run": 2,
|
||||
"backends_used": ["native_api"],
|
||||
"dry_run": false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
On failure (`"success": false`), read the `"error"` field for the reason.
|
||||
Exit code is 1 on failure.
|
||||
|
||||
## Execution Backends
|
||||
|
||||
Backends are selected automatically based on the macro step definition:
|
||||
|
||||
| Backend | Triggered by | Use case |
|
||||
|---------|-------------|----------|
|
||||
| `native_api` | `backend: native_api` | Subprocess / shell command |
|
||||
| `file_transform` | `backend: file_transform` | XML, JSON, text file editing |
|
||||
| `semantic_ui` | `backend: semantic_ui` | Accessibility / keyboard shortcuts |
|
||||
| `gui_macro` | `backend: gui_macro` | Precompiled coordinate replay |
|
||||
| `recovery` | `backend: recovery` | Retry / fallback orchestration |
|
||||
|
||||
## Writing Macros
|
||||
|
||||
Macros are YAML files in `cli_anything/macrocli/macro_definitions/`.
|
||||
Scaffold one with:
|
||||
|
||||
```bash
|
||||
cli-anything-macrocli macro define my_macro --output \
|
||||
cli_anything/macrocli/macro_definitions/examples/my_macro.yaml
|
||||
```
|
||||
|
||||
Minimal schema:
|
||||
|
||||
```yaml
|
||||
name: my_macro
|
||||
version: "1.0"
|
||||
description: What this macro does.
|
||||
|
||||
parameters:
|
||||
output:
|
||||
type: string
|
||||
required: true
|
||||
description: Where to write results.
|
||||
example: /tmp/result.txt
|
||||
|
||||
preconditions:
|
||||
- file_exists: /path/to/input
|
||||
|
||||
steps:
|
||||
- id: step1
|
||||
backend: native_api
|
||||
action: run_command
|
||||
params:
|
||||
command: [my-app, --export, "${output}"]
|
||||
timeout_ms: 30000
|
||||
on_failure: fail # or: skip, continue
|
||||
|
||||
postconditions:
|
||||
- file_exists: ${output}
|
||||
- file_size_gt: [${output}, 100]
|
||||
|
||||
outputs:
|
||||
- name: result_file
|
||||
path: ${output}
|
||||
|
||||
agent_hints:
|
||||
danger_level: safe # safe | moderate | dangerous
|
||||
side_effects: [creates_file]
|
||||
reversible: true
|
||||
```
|
||||
|
||||
## Agent Usage Rules
|
||||
|
||||
1. **Always use `--json`** for programmatic output.
|
||||
2. **Use `--dry-run` to validate params** before executing side-effectful macros.
|
||||
3. **Check `success` field** — do not assume success from exit code alone.
|
||||
4. **Read `error` field** when `success` is false — it explains what failed.
|
||||
5. **Use `macro info <name>` to discover params** before calling `macro run`.
|
||||
6. **Use absolute paths** for all file parameters.
|
||||
|
||||
## Example Workflow
|
||||
|
||||
```bash
|
||||
# Step 1: What's available?
|
||||
cli-anything-macrocli macro list --json
|
||||
|
||||
# Step 2: What params does transform_json need?
|
||||
cli-anything-macrocli macro info transform_json --json
|
||||
|
||||
# Step 3: Test safely
|
||||
cli-anything-macrocli --dry-run macro run transform_json \
|
||||
--param file=/tmp/config.json \
|
||||
--param key=theme \
|
||||
--param value=dark --json
|
||||
|
||||
# Step 4: Execute for real
|
||||
cli-anything-macrocli macro run transform_json \
|
||||
--param file=/tmp/config.json \
|
||||
--param key=theme \
|
||||
--param value=dark --json
|
||||
```
|
||||
|
||||
## Version
|
||||
|
||||
1.0.0
|
||||
@@ -0,0 +1,595 @@
|
||||
"""Unit tests for MacroCLI core modules.
|
||||
|
||||
Covers: MacroDefinition, MacroRegistry, MacroRuntime, backends, routing.
|
||||
All tests use synthetic data and do not require external software.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import textwrap
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
# ── Helpers ───────────────────────────────────────────────────────────────────
|
||||
|
||||
SIMPLE_MACRO_YAML = textwrap.dedent("""\
|
||||
name: test_macro
|
||||
version: "1.0"
|
||||
description: A test macro.
|
||||
parameters:
|
||||
output:
|
||||
type: string
|
||||
required: true
|
||||
description: Output path
|
||||
count:
|
||||
type: integer
|
||||
required: false
|
||||
default: 1
|
||||
min: 1
|
||||
max: 100
|
||||
steps:
|
||||
- id: step1
|
||||
backend: native_api
|
||||
action: run_command
|
||||
params:
|
||||
command: [echo, hello]
|
||||
postconditions: []
|
||||
""")
|
||||
|
||||
|
||||
def write_macro(tmp_path: Path, name: str, content: str) -> Path:
|
||||
p = tmp_path / f"{name}.yaml"
|
||||
p.write_text(content, encoding="utf-8")
|
||||
return p
|
||||
|
||||
|
||||
# ── macro_model tests ─────────────────────────────────────────────────────────
|
||||
|
||||
class TestMacroModel:
|
||||
def test_load_from_yaml(self, tmp_path):
|
||||
from cli_anything.macrocli.core.macro_model import load_from_yaml
|
||||
p = write_macro(tmp_path, "test_macro", SIMPLE_MACRO_YAML)
|
||||
m = load_from_yaml(str(p))
|
||||
assert m.name == "test_macro"
|
||||
assert m.version == "1.0"
|
||||
assert "output" in m.parameters
|
||||
assert len(m.steps) == 1
|
||||
assert m.steps[0].backend == "native_api"
|
||||
|
||||
def test_load_missing_file(self):
|
||||
from cli_anything.macrocli.core.macro_model import load_from_yaml
|
||||
with pytest.raises(FileNotFoundError):
|
||||
load_from_yaml("/nonexistent/path.yaml")
|
||||
|
||||
def test_validate_params_required(self, tmp_path):
|
||||
from cli_anything.macrocli.core.macro_model import load_from_yaml
|
||||
p = write_macro(tmp_path, "test_macro", SIMPLE_MACRO_YAML)
|
||||
m = load_from_yaml(str(p))
|
||||
errors = m.validate_params({})
|
||||
assert any("output" in e for e in errors)
|
||||
|
||||
def test_validate_params_type_error(self, tmp_path):
|
||||
from cli_anything.macrocli.core.macro_model import load_from_yaml
|
||||
p = write_macro(tmp_path, "test_macro", SIMPLE_MACRO_YAML)
|
||||
m = load_from_yaml(str(p))
|
||||
errors = m.validate_params({"output": "/tmp/x", "count": "not_an_int"})
|
||||
assert any("count" in e for e in errors)
|
||||
|
||||
def test_validate_params_range(self, tmp_path):
|
||||
from cli_anything.macrocli.core.macro_model import load_from_yaml
|
||||
p = write_macro(tmp_path, "test_macro", SIMPLE_MACRO_YAML)
|
||||
m = load_from_yaml(str(p))
|
||||
errors = m.validate_params({"output": "/tmp/x", "count": 200})
|
||||
assert any("count" in e for e in errors)
|
||||
|
||||
def test_validate_params_ok(self, tmp_path):
|
||||
from cli_anything.macrocli.core.macro_model import load_from_yaml
|
||||
p = write_macro(tmp_path, "test_macro", SIMPLE_MACRO_YAML)
|
||||
m = load_from_yaml(str(p))
|
||||
errors = m.validate_params({"output": "/tmp/x"})
|
||||
assert errors == []
|
||||
|
||||
def test_resolve_params_defaults(self, tmp_path):
|
||||
from cli_anything.macrocli.core.macro_model import load_from_yaml
|
||||
p = write_macro(tmp_path, "test_macro", SIMPLE_MACRO_YAML)
|
||||
m = load_from_yaml(str(p))
|
||||
resolved = m.resolve_params({"output": "/tmp/x"})
|
||||
assert resolved["count"] == 1
|
||||
|
||||
def test_structural_validation_no_steps(self, tmp_path):
|
||||
from cli_anything.macrocli.core.macro_model import load_from_yaml
|
||||
yaml_content = "name: bad\nsteps: []\n"
|
||||
p = write_macro(tmp_path, "bad", yaml_content)
|
||||
m = load_from_yaml(str(p))
|
||||
errors = m.validate()
|
||||
assert any("steps" in e for e in errors)
|
||||
|
||||
def test_structural_validation_bad_backend(self, tmp_path):
|
||||
from cli_anything.macrocli.core.macro_model import load_from_yaml
|
||||
yaml_content = textwrap.dedent("""\
|
||||
name: bad
|
||||
steps:
|
||||
- id: x
|
||||
backend: fake_backend
|
||||
action: do_thing
|
||||
""")
|
||||
p = write_macro(tmp_path, "bad_backend", yaml_content)
|
||||
m = load_from_yaml(str(p))
|
||||
errors = m.validate()
|
||||
assert any("fake_backend" in e for e in errors)
|
||||
|
||||
def test_to_dict(self, tmp_path):
|
||||
from cli_anything.macrocli.core.macro_model import load_from_yaml
|
||||
p = write_macro(tmp_path, "test_macro", SIMPLE_MACRO_YAML)
|
||||
m = load_from_yaml(str(p))
|
||||
d = m.to_dict()
|
||||
assert d["name"] == "test_macro"
|
||||
assert "parameters" in d
|
||||
assert "steps" in d
|
||||
|
||||
|
||||
class TestSubstitute:
|
||||
def test_string_substitution(self):
|
||||
from cli_anything.macrocli.core.macro_model import substitute
|
||||
result = substitute("hello ${name}", {"name": "world"})
|
||||
assert result == "hello world"
|
||||
|
||||
def test_nested_list(self):
|
||||
from cli_anything.macrocli.core.macro_model import substitute
|
||||
result = substitute(["echo", "${output}"], {"output": "/tmp/x"})
|
||||
assert result == ["echo", "/tmp/x"]
|
||||
|
||||
def test_nested_dict(self):
|
||||
from cli_anything.macrocli.core.macro_model import substitute
|
||||
result = substitute({"path": "${output}", "other": 42}, {"output": "/out"})
|
||||
assert result["path"] == "/out"
|
||||
assert result["other"] == 42
|
||||
|
||||
def test_missing_key_left_as_is(self):
|
||||
from cli_anything.macrocli.core.macro_model import substitute
|
||||
result = substitute("${missing}", {})
|
||||
assert result == "${missing}"
|
||||
|
||||
|
||||
# ── MacroRegistry tests ───────────────────────────────────────────────────────
|
||||
|
||||
class TestMacroRegistry:
|
||||
def test_load_macro(self, tmp_path):
|
||||
from cli_anything.macrocli.core.registry import MacroRegistry
|
||||
write_macro(tmp_path, "test_macro", SIMPLE_MACRO_YAML)
|
||||
reg = MacroRegistry(str(tmp_path))
|
||||
m = reg.load("test_macro")
|
||||
assert m.name == "test_macro"
|
||||
|
||||
def test_load_missing_raises(self, tmp_path):
|
||||
from cli_anything.macrocli.core.registry import MacroRegistry
|
||||
reg = MacroRegistry(str(tmp_path))
|
||||
with pytest.raises(KeyError):
|
||||
reg.load("nonexistent_macro")
|
||||
|
||||
def test_list_all(self, tmp_path):
|
||||
from cli_anything.macrocli.core.registry import MacroRegistry
|
||||
write_macro(tmp_path, "test_macro", SIMPLE_MACRO_YAML)
|
||||
write_macro(tmp_path, "another", SIMPLE_MACRO_YAML.replace("test_macro", "another"))
|
||||
reg = MacroRegistry(str(tmp_path))
|
||||
names = reg.list_names()
|
||||
assert "test_macro" in names
|
||||
assert "another" in names
|
||||
|
||||
def test_manifest_index(self, tmp_path):
|
||||
from cli_anything.macrocli.core.registry import MacroRegistry
|
||||
sub = tmp_path / "sub"
|
||||
sub.mkdir()
|
||||
write_macro(sub, "alpha", SIMPLE_MACRO_YAML.replace("test_macro", "alpha"))
|
||||
manifest = tmp_path / "manifest.yaml"
|
||||
manifest.write_text("macros:\n - name: alpha\n path: sub/alpha.yaml\n")
|
||||
reg = MacroRegistry(str(tmp_path))
|
||||
m = reg.load("alpha")
|
||||
assert m.name == "alpha"
|
||||
|
||||
def test_register_programmatic(self):
|
||||
from cli_anything.macrocli.core.registry import MacroRegistry
|
||||
from cli_anything.macrocli.core.macro_model import MacroDefinition, MacroStep
|
||||
reg = MacroRegistry("/nonexistent")
|
||||
macro = MacroDefinition(
|
||||
name="inline_macro",
|
||||
steps=[MacroStep(backend="native_api", action="run_command")],
|
||||
)
|
||||
reg.register(macro)
|
||||
assert reg.load("inline_macro").name == "inline_macro"
|
||||
|
||||
def test_info(self, tmp_path):
|
||||
from cli_anything.macrocli.core.registry import MacroRegistry
|
||||
write_macro(tmp_path, "test_macro", SIMPLE_MACRO_YAML)
|
||||
reg = MacroRegistry(str(tmp_path))
|
||||
info = reg.info()
|
||||
assert info["total"] >= 1
|
||||
assert "macros_dir" in info
|
||||
|
||||
|
||||
# ── Backend tests ─────────────────────────────────────────────────────────────
|
||||
|
||||
class TestNativeAPIBackend:
|
||||
def _make_context(self, params=None):
|
||||
from cli_anything.macrocli.backends.base import BackendContext
|
||||
return BackendContext(params=params or {})
|
||||
|
||||
def _make_step(self, action, step_params):
|
||||
from cli_anything.macrocli.core.macro_model import MacroStep
|
||||
return MacroStep(id="test", backend="native_api", action=action, params=step_params)
|
||||
|
||||
def test_run_command_success(self):
|
||||
from cli_anything.macrocli.backends.native_api import NativeAPIBackend
|
||||
b = NativeAPIBackend()
|
||||
step = self._make_step("run_command", {"command": ["echo", "hello"]})
|
||||
result = b.execute(step, {}, self._make_context())
|
||||
assert result.success
|
||||
|
||||
def test_run_command_fails_bad_exit(self):
|
||||
from cli_anything.macrocli.backends.native_api import NativeAPIBackend
|
||||
b = NativeAPIBackend()
|
||||
step = self._make_step("run_command", {"command": ["false"]})
|
||||
result = b.execute(step, {}, self._make_context())
|
||||
assert not result.success
|
||||
assert "exit" in result.error.lower() or "failed" in result.error.lower()
|
||||
|
||||
def test_run_command_not_found(self):
|
||||
from cli_anything.macrocli.backends.native_api import NativeAPIBackend
|
||||
b = NativeAPIBackend()
|
||||
step = self._make_step("run_command", {"command": ["__nonexistent_cmd__"]})
|
||||
result = b.execute(step, {}, self._make_context())
|
||||
assert not result.success
|
||||
|
||||
def test_find_executable_found(self):
|
||||
from cli_anything.macrocli.backends.native_api import NativeAPIBackend
|
||||
b = NativeAPIBackend()
|
||||
step = self._make_step("find_executable", {"name": "echo"})
|
||||
result = b.execute(step, {}, self._make_context())
|
||||
assert result.success
|
||||
assert "executable" in result.output
|
||||
|
||||
def test_find_executable_missing(self):
|
||||
from cli_anything.macrocli.backends.native_api import NativeAPIBackend
|
||||
b = NativeAPIBackend()
|
||||
step = self._make_step("find_executable", {
|
||||
"name": "__nonexistent__",
|
||||
"install_hint": "brew install nonexistent"
|
||||
})
|
||||
result = b.execute(step, {}, self._make_context())
|
||||
assert not result.success
|
||||
assert "brew install" in result.error
|
||||
|
||||
def test_dry_run_does_not_execute(self):
|
||||
from cli_anything.macrocli.backends.native_api import NativeAPIBackend
|
||||
from cli_anything.macrocli.backends.base import BackendContext
|
||||
b = NativeAPIBackend()
|
||||
step = self._make_step("run_command", {"command": ["false"]})
|
||||
ctx = BackendContext(params={}, dry_run=True)
|
||||
result = b.execute(step, {}, ctx)
|
||||
assert result.success
|
||||
assert result.output.get("dry_run")
|
||||
|
||||
def test_param_substitution_in_command(self):
|
||||
from cli_anything.macrocli.backends.native_api import NativeAPIBackend
|
||||
b = NativeAPIBackend()
|
||||
step = self._make_step("run_command", {"command": ["echo", "${msg}"]})
|
||||
result = b.execute(step, {"msg": "substituted"}, self._make_context({"msg": "substituted"}))
|
||||
assert result.success
|
||||
|
||||
def test_unknown_action(self):
|
||||
from cli_anything.macrocli.backends.native_api import NativeAPIBackend
|
||||
b = NativeAPIBackend()
|
||||
step = self._make_step("unknown_action", {})
|
||||
result = b.execute(step, {}, self._make_context())
|
||||
assert not result.success
|
||||
assert "unknown action" in result.error.lower()
|
||||
|
||||
|
||||
class TestFileTransformBackend:
|
||||
def _make_context(self):
|
||||
from cli_anything.macrocli.backends.base import BackendContext
|
||||
return BackendContext(params={})
|
||||
|
||||
def test_json_set_and_get(self, tmp_path):
|
||||
from cli_anything.macrocli.backends.file_transform import FileTransformBackend
|
||||
from cli_anything.macrocli.core.macro_model import MacroStep
|
||||
b = FileTransformBackend()
|
||||
ctx = self._make_context()
|
||||
|
||||
json_file = tmp_path / "data.json"
|
||||
json_file.write_text('{"a": 1}', encoding="utf-8")
|
||||
|
||||
step = MacroStep(id="set", backend="file_transform", action="json_set", params={
|
||||
"input_file": str(json_file),
|
||||
"output_file": str(json_file),
|
||||
"path": "settings.theme",
|
||||
"value": "dark",
|
||||
})
|
||||
result = b.execute(step, {}, ctx)
|
||||
assert result.success
|
||||
|
||||
import json
|
||||
data = json.loads(json_file.read_text())
|
||||
assert data["settings"]["theme"] == "dark"
|
||||
|
||||
def test_text_replace(self, tmp_path):
|
||||
from cli_anything.macrocli.backends.file_transform import FileTransformBackend
|
||||
from cli_anything.macrocli.core.macro_model import MacroStep
|
||||
b = FileTransformBackend()
|
||||
ctx = self._make_context()
|
||||
|
||||
txt_file = tmp_path / "config.ini"
|
||||
txt_file.write_text("theme=default\nsize=10\n", encoding="utf-8")
|
||||
|
||||
step = MacroStep(id="replace", backend="file_transform", action="text_replace", params={
|
||||
"input_file": str(txt_file),
|
||||
"output_file": str(txt_file),
|
||||
"find": "theme=default",
|
||||
"replace": "theme=dark",
|
||||
})
|
||||
result = b.execute(step, {}, ctx)
|
||||
assert result.success
|
||||
assert "theme=dark" in txt_file.read_text()
|
||||
assert result.output["replacements"] == 1
|
||||
|
||||
def test_copy_file(self, tmp_path):
|
||||
from cli_anything.macrocli.backends.file_transform import FileTransformBackend
|
||||
from cli_anything.macrocli.core.macro_model import MacroStep
|
||||
b = FileTransformBackend()
|
||||
ctx = self._make_context()
|
||||
|
||||
src = tmp_path / "src.txt"
|
||||
dst = tmp_path / "dst.txt"
|
||||
src.write_text("content", encoding="utf-8")
|
||||
|
||||
step = MacroStep(id="copy", backend="file_transform", action="copy_file", params={
|
||||
"src": str(src),
|
||||
"dst": str(dst),
|
||||
})
|
||||
result = b.execute(step, {}, ctx)
|
||||
assert result.success
|
||||
assert dst.read_text() == "content"
|
||||
|
||||
def test_unknown_action(self):
|
||||
from cli_anything.macrocli.backends.file_transform import FileTransformBackend
|
||||
from cli_anything.macrocli.core.macro_model import MacroStep
|
||||
b = FileTransformBackend()
|
||||
step = MacroStep(id="x", backend="file_transform", action="unknown_op", params={})
|
||||
result = b.execute(step, {}, self._make_context())
|
||||
assert not result.success
|
||||
|
||||
|
||||
class TestStepResult:
|
||||
def test_to_dict(self):
|
||||
from cli_anything.macrocli.backends.base import StepResult
|
||||
r = StepResult(success=True, output={"key": "val"}, backend_used="native_api")
|
||||
d = r.to_dict()
|
||||
assert d["success"] is True
|
||||
assert d["output"]["key"] == "val"
|
||||
assert d["backend_used"] == "native_api"
|
||||
|
||||
|
||||
# ── Routing tests ─────────────────────────────────────────────────────────────
|
||||
|
||||
class TestRoutingEngine:
|
||||
def test_select_native_api(self):
|
||||
from cli_anything.macrocli.core.routing import RoutingEngine
|
||||
from cli_anything.macrocli.core.macro_model import MacroStep
|
||||
engine = RoutingEngine()
|
||||
step = MacroStep(id="x", backend="native_api", action="run_command")
|
||||
backend = engine.select(step)
|
||||
assert backend.name == "native_api"
|
||||
|
||||
def test_select_file_transform(self):
|
||||
from cli_anything.macrocli.core.routing import RoutingEngine
|
||||
from cli_anything.macrocli.core.macro_model import MacroStep
|
||||
engine = RoutingEngine()
|
||||
step = MacroStep(id="x", backend="file_transform", action="json_set")
|
||||
backend = engine.select(step)
|
||||
assert backend.name == "file_transform"
|
||||
|
||||
def test_describe(self):
|
||||
from cli_anything.macrocli.core.routing import RoutingEngine
|
||||
engine = RoutingEngine()
|
||||
desc = engine.describe()
|
||||
assert "native_api" in desc
|
||||
assert "file_transform" in desc
|
||||
assert "recovery" in desc
|
||||
|
||||
def test_execute_step_native_api(self):
|
||||
from cli_anything.macrocli.core.routing import RoutingEngine
|
||||
from cli_anything.macrocli.core.macro_model import MacroStep
|
||||
from cli_anything.macrocli.backends.base import BackendContext
|
||||
engine = RoutingEngine()
|
||||
step = MacroStep(id="x", backend="native_api", action="run_command",
|
||||
params={"command": ["echo", "hello"]})
|
||||
ctx = BackendContext(params={})
|
||||
result = engine.execute_step(step, {}, ctx)
|
||||
assert result.success
|
||||
|
||||
|
||||
# ── Runtime tests ─────────────────────────────────────────────────────────────
|
||||
|
||||
class TestMacroRuntime:
|
||||
def _make_runtime(self, tmp_path):
|
||||
from cli_anything.macrocli.core.registry import MacroRegistry
|
||||
from cli_anything.macrocli.core.runtime import MacroRuntime
|
||||
# Register a real macro that just echoes
|
||||
yaml_content = textwrap.dedent("""\
|
||||
name: echo_macro
|
||||
parameters:
|
||||
msg:
|
||||
type: string
|
||||
required: false
|
||||
default: hello
|
||||
steps:
|
||||
- id: step1
|
||||
backend: native_api
|
||||
action: run_command
|
||||
params:
|
||||
command: [echo, "${msg}"]
|
||||
""")
|
||||
write_macro(tmp_path, "echo_macro", yaml_content)
|
||||
reg = MacroRegistry(str(tmp_path))
|
||||
return MacroRuntime(registry=reg)
|
||||
|
||||
def test_execute_success(self, tmp_path):
|
||||
rt = self._make_runtime(tmp_path)
|
||||
result = rt.execute("echo_macro", {"msg": "test"})
|
||||
assert result.success
|
||||
assert result.telemetry["steps_run"] == 1
|
||||
|
||||
def test_execute_unknown_macro(self, tmp_path):
|
||||
rt = self._make_runtime(tmp_path)
|
||||
result = rt.execute("nonexistent_macro", {})
|
||||
assert not result.success
|
||||
assert "not found" in result.error.lower()
|
||||
|
||||
def test_execute_param_validation_failure(self, tmp_path):
|
||||
from cli_anything.macrocli.core.registry import MacroRegistry
|
||||
from cli_anything.macrocli.core.runtime import MacroRuntime
|
||||
yaml_content = textwrap.dedent("""\
|
||||
name: required_param_macro
|
||||
parameters:
|
||||
output:
|
||||
type: string
|
||||
required: true
|
||||
steps:
|
||||
- id: s1
|
||||
backend: native_api
|
||||
action: run_command
|
||||
params:
|
||||
command: [echo, "${output}"]
|
||||
""")
|
||||
write_macro(tmp_path, "required_param_macro", yaml_content)
|
||||
reg = MacroRegistry(str(tmp_path))
|
||||
rt = MacroRuntime(registry=reg)
|
||||
result = rt.execute("required_param_macro", {})
|
||||
assert not result.success
|
||||
assert "output" in result.error
|
||||
|
||||
def test_precondition_failure(self, tmp_path):
|
||||
from cli_anything.macrocli.core.registry import MacroRegistry
|
||||
from cli_anything.macrocli.core.runtime import MacroRuntime
|
||||
yaml_content = textwrap.dedent("""\
|
||||
name: precond_macro
|
||||
preconditions:
|
||||
- file_exists: /nonexistent_file_xyz_abc
|
||||
steps:
|
||||
- id: s1
|
||||
backend: native_api
|
||||
action: run_command
|
||||
params:
|
||||
command: [echo, ok]
|
||||
""")
|
||||
write_macro(tmp_path, "precond_macro", yaml_content)
|
||||
reg = MacroRegistry(str(tmp_path))
|
||||
rt = MacroRuntime(registry=reg)
|
||||
result = rt.execute("precond_macro", {})
|
||||
assert not result.success
|
||||
assert "precondition" in result.error.lower()
|
||||
|
||||
def test_postcondition_failure(self, tmp_path):
|
||||
from cli_anything.macrocli.core.registry import MacroRegistry
|
||||
from cli_anything.macrocli.core.runtime import MacroRuntime
|
||||
yaml_content = textwrap.dedent("""\
|
||||
name: postcond_macro
|
||||
steps:
|
||||
- id: s1
|
||||
backend: native_api
|
||||
action: run_command
|
||||
params:
|
||||
command: [echo, ok]
|
||||
postconditions:
|
||||
- file_exists: /nonexistent_output_xyz
|
||||
""")
|
||||
write_macro(tmp_path, "postcond_macro", yaml_content)
|
||||
reg = MacroRegistry(str(tmp_path))
|
||||
rt = MacroRuntime(registry=reg)
|
||||
result = rt.execute("postcond_macro", {})
|
||||
assert not result.success
|
||||
assert "postcondition" in result.error.lower()
|
||||
|
||||
def test_dry_run(self, tmp_path):
|
||||
rt = self._make_runtime(tmp_path)
|
||||
result = rt.execute("echo_macro", {}, dry_run=True)
|
||||
assert result.success
|
||||
assert result.telemetry["dry_run"] is True
|
||||
|
||||
def test_session_records_run(self, tmp_path):
|
||||
rt = self._make_runtime(tmp_path)
|
||||
rt.execute("echo_macro", {})
|
||||
last = rt.session.last()
|
||||
assert last is not None
|
||||
assert last.macro_name == "echo_macro"
|
||||
|
||||
def test_validate_macro(self, tmp_path):
|
||||
rt = self._make_runtime(tmp_path)
|
||||
errors = rt.validate_macro("echo_macro")
|
||||
assert errors == []
|
||||
|
||||
def test_on_failure_skip(self, tmp_path):
|
||||
from cli_anything.macrocli.core.registry import MacroRegistry
|
||||
from cli_anything.macrocli.core.runtime import MacroRuntime
|
||||
yaml_content = textwrap.dedent("""\
|
||||
name: skip_macro
|
||||
steps:
|
||||
- id: bad_step
|
||||
backend: native_api
|
||||
action: run_command
|
||||
params:
|
||||
command: [false]
|
||||
on_failure: skip
|
||||
- id: good_step
|
||||
backend: native_api
|
||||
action: run_command
|
||||
params:
|
||||
command: [echo, reached]
|
||||
""")
|
||||
write_macro(tmp_path, "skip_macro", yaml_content)
|
||||
reg = MacroRegistry(str(tmp_path))
|
||||
rt = MacroRuntime(registry=reg)
|
||||
result = rt.execute("skip_macro", {})
|
||||
# Should succeed because bad step was skipped
|
||||
assert result.success
|
||||
assert result.telemetry["steps_run"] == 2
|
||||
|
||||
|
||||
# ── Session tests ─────────────────────────────────────────────────────────────
|
||||
|
||||
class TestExecutionSession:
|
||||
def test_record_and_retrieve(self):
|
||||
from cli_anything.macrocli.core.session import ExecutionSession, RunRecord
|
||||
sess = ExecutionSession(session_id="test_sess")
|
||||
rec = RunRecord("m1", {}, True, {}, "", 100.0, ["native_api"], 1)
|
||||
sess.record(rec)
|
||||
assert sess.last().macro_name == "m1"
|
||||
assert len(sess.history()) == 1
|
||||
|
||||
def test_stats(self):
|
||||
from cli_anything.macrocli.core.session import ExecutionSession, RunRecord
|
||||
sess = ExecutionSession()
|
||||
sess.record(RunRecord("m1", {}, True, {}, "", 100.0, [], 1))
|
||||
sess.record(RunRecord("m2", {}, False, {}, "err", 50.0, [], 0))
|
||||
stats = sess.stats()
|
||||
assert stats["total"] == 2
|
||||
assert stats["success"] == 1
|
||||
assert stats["success_rate"] == 0.5
|
||||
|
||||
def test_save_and_load(self, tmp_path, monkeypatch):
|
||||
from cli_anything.macrocli.core import session as sess_mod
|
||||
import cli_anything.macrocli.core.session as sess_module
|
||||
# Redirect SESSION_DIR to tmp_path
|
||||
monkeypatch.setattr(sess_module, "SESSION_DIR", tmp_path)
|
||||
from cli_anything.macrocli.core.session import ExecutionSession, RunRecord
|
||||
sess = ExecutionSession(session_id="save_test")
|
||||
sess.record(RunRecord("m1", {"k": "v"}, True, {}, "", 200.0, ["native_api"], 1))
|
||||
sess.save()
|
||||
|
||||
loaded = ExecutionSession.load("save_test")
|
||||
assert loaded is not None
|
||||
assert loaded.session_id == "save_test"
|
||||
assert loaded.last().macro_name == "m1"
|
||||
@@ -0,0 +1,369 @@
|
||||
"""End-to-end integration tests for the MacroCLI.
|
||||
|
||||
Tests the full lifecycle: macro discovery → runtime execution → CLI subprocess.
|
||||
Uses real file I/O and subprocess calls (echo, cat, etc.) as the "target apps".
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import textwrap
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
# ── CLI resolver (same pattern as in HARNESS.md) ─────────────────────────────
|
||||
|
||||
def _resolve_cli(name: str) -> list[str]:
|
||||
"""Resolve installed CLI command; falls back to python -m for dev."""
|
||||
import shutil
|
||||
force = os.environ.get("CLI_ANYTHING_FORCE_INSTALLED", "").strip() == "1"
|
||||
path = shutil.which(name)
|
||||
if path:
|
||||
print(f"[_resolve_cli] Using installed command: {path}")
|
||||
return [path]
|
||||
if force:
|
||||
raise RuntimeError(f"{name} not found in PATH. Install with: pip install -e .")
|
||||
module = "cli_anything.macrocli.macrocli_cli"
|
||||
print(f"[_resolve_cli] Falling back to: {sys.executable} -m {module}")
|
||||
return [sys.executable, "-m", "cli_anything.macrocli"]
|
||||
|
||||
|
||||
# ── Helpers ───────────────────────────────────────────────────────────────────
|
||||
|
||||
def write_macro(tmp_path: Path, name: str, content: str) -> Path:
|
||||
p = tmp_path / f"{name}.yaml"
|
||||
p.write_text(content, encoding="utf-8")
|
||||
return p
|
||||
|
||||
|
||||
# ── E2E: File transform workflow ──────────────────────────────────────────────
|
||||
|
||||
class TestFileTransformE2E:
|
||||
def test_json_set_and_verify(self, tmp_path):
|
||||
"""Write a JSON file, transform it, verify the result."""
|
||||
from cli_anything.macrocli.core.macro_model import MacroDefinition, MacroStep, MacroCondition, MacroOutput
|
||||
from cli_anything.macrocli.core.registry import MacroRegistry
|
||||
from cli_anything.macrocli.core.runtime import MacroRuntime
|
||||
|
||||
json_file = tmp_path / "settings.json"
|
||||
json_file.write_text('{"version": 1}', encoding="utf-8")
|
||||
|
||||
yaml_content = textwrap.dedent(f"""\
|
||||
name: set_json_key
|
||||
parameters:
|
||||
file:
|
||||
type: string
|
||||
required: true
|
||||
key:
|
||||
type: string
|
||||
required: true
|
||||
value:
|
||||
type: string
|
||||
required: true
|
||||
preconditions:
|
||||
- file_exists: ${{file}}
|
||||
steps:
|
||||
- id: transform
|
||||
backend: file_transform
|
||||
action: json_set
|
||||
params:
|
||||
input_file: ${{file}}
|
||||
output_file: ${{file}}
|
||||
path: ${{key}}
|
||||
value: ${{value}}
|
||||
postconditions:
|
||||
- file_exists: ${{file}}
|
||||
outputs:
|
||||
- name: modified_file
|
||||
path: ${{file}}
|
||||
""")
|
||||
write_macro(tmp_path, "set_json_key", yaml_content)
|
||||
|
||||
reg = MacroRegistry(str(tmp_path))
|
||||
rt = MacroRuntime(registry=reg)
|
||||
result = rt.execute("set_json_key", {
|
||||
"file": str(json_file),
|
||||
"key": "config.theme",
|
||||
"value": "dark",
|
||||
})
|
||||
|
||||
assert result.success, f"Failed: {result.error}"
|
||||
data = json.loads(json_file.read_text())
|
||||
assert data["config"]["theme"] == "dark"
|
||||
print(f"\n Modified JSON: {json_file} → {json.dumps(data)}")
|
||||
|
||||
|
||||
class TestNativeAPIE2E:
|
||||
def test_run_real_command_and_capture(self, tmp_path):
|
||||
"""Run a real shell command and capture its stdout."""
|
||||
from cli_anything.macrocli.core.macro_model import MacroDefinition
|
||||
from cli_anything.macrocli.core.registry import MacroRegistry
|
||||
from cli_anything.macrocli.core.runtime import MacroRuntime
|
||||
|
||||
output_file = tmp_path / "result.txt"
|
||||
yaml_content = textwrap.dedent(f"""\
|
||||
name: capture_date
|
||||
parameters:
|
||||
output:
|
||||
type: string
|
||||
required: true
|
||||
steps:
|
||||
- id: run_date
|
||||
backend: native_api
|
||||
action: run_command
|
||||
params:
|
||||
command: [date, "+%Y-%m-%d"]
|
||||
capture_stdout: true
|
||||
postconditions: []
|
||||
outputs:
|
||||
- name: output_file
|
||||
path: ${{output}}
|
||||
""")
|
||||
write_macro(tmp_path, "capture_date", yaml_content)
|
||||
|
||||
reg = MacroRegistry(str(tmp_path))
|
||||
rt = MacroRuntime(registry=reg)
|
||||
result = rt.execute("capture_date", {"output": str(output_file)})
|
||||
|
||||
assert result.success, f"Failed: {result.error}"
|
||||
# Step output contains stdout
|
||||
steps_output = result.output.get("_steps", [])
|
||||
assert steps_output, "Expected _steps in output"
|
||||
stdout = steps_output[0].get("stdout", "")
|
||||
# Date format YYYY-MM-DD
|
||||
import re
|
||||
assert re.match(r"\d{4}-\d{2}-\d{2}", stdout.strip()), f"Unexpected stdout: {stdout!r}"
|
||||
print(f"\n Captured date: {stdout.strip()}")
|
||||
|
||||
def test_step_failure_aborts_macro(self, tmp_path):
|
||||
"""A failing step with on_failure=fail should abort the macro."""
|
||||
from cli_anything.macrocli.core.registry import MacroRegistry
|
||||
from cli_anything.macrocli.core.runtime import MacroRuntime
|
||||
|
||||
yaml_content = textwrap.dedent("""\
|
||||
name: fail_abort
|
||||
steps:
|
||||
- id: bad
|
||||
backend: native_api
|
||||
action: run_command
|
||||
params:
|
||||
command: [false]
|
||||
on_failure: fail
|
||||
- id: good
|
||||
backend: native_api
|
||||
action: run_command
|
||||
params:
|
||||
command: [echo, should_not_run]
|
||||
""")
|
||||
write_macro(tmp_path, "fail_abort", yaml_content)
|
||||
reg = MacroRegistry(str(tmp_path))
|
||||
rt = MacroRuntime(registry=reg)
|
||||
result = rt.execute("fail_abort", {})
|
||||
|
||||
assert not result.success
|
||||
# Only the first step should have run
|
||||
assert result.telemetry["steps_run"] == 1
|
||||
print(f"\n Correctly aborted after first step: {result.error}")
|
||||
|
||||
def test_step_failure_skip_continues(self, tmp_path):
|
||||
"""A failing step with on_failure=skip should allow the macro to continue."""
|
||||
from cli_anything.macrocli.core.registry import MacroRegistry
|
||||
from cli_anything.macrocli.core.runtime import MacroRuntime
|
||||
|
||||
yaml_content = textwrap.dedent("""\
|
||||
name: fail_skip
|
||||
steps:
|
||||
- id: bad
|
||||
backend: native_api
|
||||
action: run_command
|
||||
params:
|
||||
command: [false]
|
||||
on_failure: skip
|
||||
- id: good
|
||||
backend: native_api
|
||||
action: run_command
|
||||
params:
|
||||
command: [echo, still_ran]
|
||||
""")
|
||||
write_macro(tmp_path, "fail_skip", yaml_content)
|
||||
reg = MacroRegistry(str(tmp_path))
|
||||
rt = MacroRuntime(registry=reg)
|
||||
result = rt.execute("fail_skip", {})
|
||||
|
||||
assert result.success
|
||||
assert result.telemetry["steps_run"] == 2
|
||||
print(f"\n Macro succeeded despite skipped step")
|
||||
|
||||
|
||||
class TestPostconditionE2E:
|
||||
def test_postcondition_file_exists_passes(self, tmp_path):
|
||||
from cli_anything.macrocli.core.registry import MacroRegistry
|
||||
from cli_anything.macrocli.core.runtime import MacroRuntime
|
||||
|
||||
output_file = tmp_path / "output.txt"
|
||||
yaml_content = textwrap.dedent(f"""\
|
||||
name: write_and_verify
|
||||
parameters:
|
||||
output:
|
||||
type: string
|
||||
required: true
|
||||
steps:
|
||||
- id: write
|
||||
backend: file_transform
|
||||
action: text_replace
|
||||
params:
|
||||
input_file: /dev/null
|
||||
output_file: ${{output}}
|
||||
find: ""
|
||||
replace: ""
|
||||
postconditions:
|
||||
- file_exists: ${{output}}
|
||||
""")
|
||||
write_macro(tmp_path, "write_and_verify", yaml_content)
|
||||
reg = MacroRegistry(str(tmp_path))
|
||||
rt = MacroRuntime(registry=reg)
|
||||
result = rt.execute("write_and_verify", {"output": str(output_file)})
|
||||
assert result.success, f"Failed: {result.error}"
|
||||
print(f"\n Output file: {output_file} ({output_file.stat().st_size} bytes)")
|
||||
|
||||
def test_postcondition_file_size_gt(self, tmp_path):
|
||||
from cli_anything.macrocli.core.registry import MacroRegistry
|
||||
from cli_anything.macrocli.core.runtime import MacroRuntime
|
||||
|
||||
output_file = tmp_path / "output.txt"
|
||||
output_file.write_text("x" * 200, encoding="utf-8")
|
||||
|
||||
yaml_content = textwrap.dedent(f"""\
|
||||
name: size_check
|
||||
parameters:
|
||||
output:
|
||||
type: string
|
||||
required: true
|
||||
steps:
|
||||
- id: noop
|
||||
backend: native_api
|
||||
action: run_command
|
||||
params:
|
||||
command: [echo, noop]
|
||||
postconditions:
|
||||
- file_size_gt:
|
||||
- ${{output}}
|
||||
- 100
|
||||
""")
|
||||
write_macro(tmp_path, "size_check", yaml_content)
|
||||
reg = MacroRegistry(str(tmp_path))
|
||||
rt = MacroRuntime(registry=reg)
|
||||
result = rt.execute("size_check", {"output": str(output_file)})
|
||||
assert result.success, f"Failed: {result.error}"
|
||||
print(f"\n File size: {output_file.stat().st_size} bytes")
|
||||
|
||||
|
||||
# ── CLI subprocess tests ──────────────────────────────────────────────────────
|
||||
|
||||
class TestCLISubprocess:
|
||||
CLI_BASE = _resolve_cli("cli-anything-macrocli")
|
||||
|
||||
def _run(self, args: list[str], check: bool = True) -> subprocess.CompletedProcess:
|
||||
return subprocess.run(
|
||||
self.CLI_BASE + args,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=check,
|
||||
)
|
||||
|
||||
def test_help(self):
|
||||
result = self._run(["--help"])
|
||||
assert result.returncode == 0
|
||||
assert "macro" in result.stdout.lower()
|
||||
|
||||
def test_macro_list_json(self):
|
||||
result = self._run(["--json", "macro", "list"])
|
||||
assert result.returncode == 0
|
||||
data = json.loads(result.stdout)
|
||||
assert isinstance(data, list)
|
||||
print(f"\n macros: {[m['name'] for m in data]}")
|
||||
|
||||
def test_macro_info_json(self):
|
||||
# Info on a bundled example macro
|
||||
result = self._run(["--json", "macro", "info", "export_file"])
|
||||
assert result.returncode == 0, f"stderr: {result.stderr}"
|
||||
data = json.loads(result.stdout)
|
||||
assert data["name"] == "export_file"
|
||||
assert "parameters" in data
|
||||
print(f"\n Macro info: {data['name']} v{data['version']}")
|
||||
|
||||
def test_macro_validate_all(self):
|
||||
result = self._run(["--json", "macro", "validate"])
|
||||
assert result.returncode == 0
|
||||
data = json.loads(result.stdout)
|
||||
assert isinstance(data, dict)
|
||||
for name, info in data.items():
|
||||
assert info["valid"], f"Macro {name} failed: {info['errors']}"
|
||||
print(f"\n Validated {len(data)} macros, all valid")
|
||||
|
||||
def test_macro_dry_run_json(self):
|
||||
result = self._run([
|
||||
"--json", "--dry-run",
|
||||
"macro", "run", "export_file",
|
||||
"--param", "output=/tmp/test_macrocli_e2e.txt",
|
||||
])
|
||||
assert result.returncode == 0, f"stderr: {result.stderr}"
|
||||
data = json.loads(result.stdout)
|
||||
assert data["success"] is True
|
||||
assert data["telemetry"]["dry_run"] is True
|
||||
print(f"\n Dry run result: {data['success']}")
|
||||
|
||||
def test_backends_json(self):
|
||||
result = self._run(["--json", "backends"])
|
||||
assert result.returncode == 0
|
||||
data = json.loads(result.stdout)
|
||||
assert "native_api" in data
|
||||
assert "file_transform" in data
|
||||
print(f"\n Backends: {list(data.keys())}")
|
||||
|
||||
def test_session_status_json(self):
|
||||
result = self._run(["--json", "session", "status"])
|
||||
assert result.returncode == 0
|
||||
data = json.loads(result.stdout)
|
||||
assert "session_id" in data
|
||||
print(f"\n Session: {data['session_id']}")
|
||||
|
||||
def test_macro_run_json_transform_workflow(self, tmp_path):
|
||||
"""Full E2E: create a JSON file, run transform_json macro, verify output."""
|
||||
json_file = tmp_path / "data.json"
|
||||
json_file.write_text('{"name": "test"}', encoding="utf-8")
|
||||
|
||||
result = self._run([
|
||||
"--json",
|
||||
"macro", "run", "transform_json",
|
||||
"--param", f"file={json_file}",
|
||||
"--param", "key=config.mode",
|
||||
"--param", "value=production",
|
||||
])
|
||||
print(f"\n CLI stdout: {result.stdout[:200]}")
|
||||
print(f" CLI stderr: {result.stderr[:200]}")
|
||||
|
||||
assert result.returncode == 0, f"CLI failed:\n{result.stderr}"
|
||||
data = json.loads(result.stdout)
|
||||
assert data["success"] is True, f"Macro failed: {data.get('error')}"
|
||||
|
||||
# Verify file was actually modified
|
||||
modified = json.loads(json_file.read_text())
|
||||
assert modified["config"]["mode"] == "production"
|
||||
print(f"\n Modified JSON: {modified}")
|
||||
print(f" File: {json_file} ({json_file.stat().st_size} bytes)")
|
||||
|
||||
def test_unknown_macro_returns_error_json(self):
|
||||
result = self._run(
|
||||
["--json", "macro", "run", "nonexistent_macro_xyz"],
|
||||
check=False,
|
||||
)
|
||||
assert result.returncode != 0
|
||||
data = json.loads(result.stdout)
|
||||
assert data["success"] is False
|
||||
assert "error" in data
|
||||
print(f"\n Error: {data['error']}")
|
||||
@@ -0,0 +1,521 @@
|
||||
"""cli-anything REPL Skin — Unified terminal interface for all CLI harnesses.
|
||||
|
||||
Copy this file into your CLI package at:
|
||||
cli_anything/<software>/utils/repl_skin.py
|
||||
|
||||
Usage:
|
||||
from cli_anything.<software>.utils.repl_skin import ReplSkin
|
||||
|
||||
skin = ReplSkin("shotcut", version="1.0.0")
|
||||
skin.print_banner() # auto-detects skills/SKILL.md inside the package
|
||||
prompt_text = skin.prompt(project_name="my_video.mlt", modified=True)
|
||||
skin.success("Project saved")
|
||||
skin.error("File not found")
|
||||
skin.warning("Unsaved changes")
|
||||
skin.info("Processing 24 clips...")
|
||||
skin.status("Track 1", "3 clips, 00:02:30")
|
||||
skin.table(headers, rows)
|
||||
skin.print_goodbye()
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
# ── ANSI color codes (no external deps for core styling) ──────────────
|
||||
|
||||
_RESET = "\033[0m"
|
||||
_BOLD = "\033[1m"
|
||||
_DIM = "\033[2m"
|
||||
_ITALIC = "\033[3m"
|
||||
_UNDERLINE = "\033[4m"
|
||||
|
||||
# Brand colors
|
||||
_CYAN = "\033[38;5;80m" # cli-anything brand cyan
|
||||
_CYAN_BG = "\033[48;5;80m"
|
||||
_WHITE = "\033[97m"
|
||||
_GRAY = "\033[38;5;245m"
|
||||
_DARK_GRAY = "\033[38;5;240m"
|
||||
_LIGHT_GRAY = "\033[38;5;250m"
|
||||
|
||||
# Software accent colors — each software gets a unique accent
|
||||
_ACCENT_COLORS = {
|
||||
"gimp": "\033[38;5;214m", # warm orange
|
||||
"blender": "\033[38;5;208m", # deep orange
|
||||
"inkscape": "\033[38;5;39m", # bright blue
|
||||
"audacity": "\033[38;5;33m", # navy blue
|
||||
"libreoffice": "\033[38;5;40m", # green
|
||||
"obs_studio": "\033[38;5;55m", # purple
|
||||
"kdenlive": "\033[38;5;69m", # slate blue
|
||||
"shotcut": "\033[38;5;35m", # teal green
|
||||
}
|
||||
_DEFAULT_ACCENT = "\033[38;5;75m" # default sky blue
|
||||
|
||||
# Status colors
|
||||
_GREEN = "\033[38;5;78m"
|
||||
_YELLOW = "\033[38;5;220m"
|
||||
_RED = "\033[38;5;196m"
|
||||
_BLUE = "\033[38;5;75m"
|
||||
_MAGENTA = "\033[38;5;176m"
|
||||
|
||||
# ── Brand icon ────────────────────────────────────────────────────────
|
||||
|
||||
# The cli-anything icon: a small colored diamond/chevron mark
|
||||
_ICON = f"{_CYAN}{_BOLD}◆{_RESET}"
|
||||
_ICON_SMALL = f"{_CYAN}▸{_RESET}"
|
||||
|
||||
# ── Box drawing characters ────────────────────────────────────────────
|
||||
|
||||
_H_LINE = "─"
|
||||
_V_LINE = "│"
|
||||
_TL = "╭"
|
||||
_TR = "╮"
|
||||
_BL = "╰"
|
||||
_BR = "╯"
|
||||
_T_DOWN = "┬"
|
||||
_T_UP = "┴"
|
||||
_T_RIGHT = "├"
|
||||
_T_LEFT = "┤"
|
||||
_CROSS = "┼"
|
||||
|
||||
|
||||
def _strip_ansi(text: str) -> str:
|
||||
"""Remove ANSI escape codes for length calculation."""
|
||||
import re
|
||||
return re.sub(r"\033\[[^m]*m", "", text)
|
||||
|
||||
|
||||
def _visible_len(text: str) -> int:
|
||||
"""Get visible length of text (excluding ANSI codes)."""
|
||||
return len(_strip_ansi(text))
|
||||
|
||||
|
||||
class ReplSkin:
|
||||
"""Unified REPL skin for cli-anything CLIs.
|
||||
|
||||
Provides consistent branding, prompts, and message formatting
|
||||
across all CLI harnesses built with the cli-anything methodology.
|
||||
"""
|
||||
|
||||
def __init__(self, software: str, version: str = "1.0.0",
|
||||
history_file: str | None = None, skill_path: str | None = None):
|
||||
"""Initialize the REPL skin.
|
||||
|
||||
Args:
|
||||
software: Software name (e.g., "gimp", "shotcut", "blender").
|
||||
version: CLI version string.
|
||||
history_file: Path for persistent command history.
|
||||
Defaults to ~/.cli-anything-<software>/history
|
||||
skill_path: Path to the SKILL.md file for agent discovery.
|
||||
Auto-detected from the package's skills/ directory if not provided.
|
||||
Displayed in banner for AI agents to know where to read skill info.
|
||||
"""
|
||||
self.software = software.lower().replace("-", "_")
|
||||
self.display_name = software.replace("_", " ").title()
|
||||
self.version = version
|
||||
|
||||
# Auto-detect skill path from package layout:
|
||||
# cli_anything/<software>/utils/repl_skin.py (this file)
|
||||
# cli_anything/<software>/skills/SKILL.md (target)
|
||||
if skill_path is None:
|
||||
from pathlib import Path
|
||||
_auto = Path(__file__).resolve().parent.parent / "skills" / "SKILL.md"
|
||||
if _auto.is_file():
|
||||
skill_path = str(_auto)
|
||||
self.skill_path = skill_path
|
||||
self.accent = _ACCENT_COLORS.get(self.software, _DEFAULT_ACCENT)
|
||||
|
||||
# History file
|
||||
if history_file is None:
|
||||
from pathlib import Path
|
||||
hist_dir = Path.home() / f".cli-anything-{self.software}"
|
||||
hist_dir.mkdir(parents=True, exist_ok=True)
|
||||
self.history_file = str(hist_dir / "history")
|
||||
else:
|
||||
self.history_file = history_file
|
||||
|
||||
# Detect terminal capabilities
|
||||
self._color = self._detect_color_support()
|
||||
|
||||
def _detect_color_support(self) -> bool:
|
||||
"""Check if terminal supports color."""
|
||||
if os.environ.get("NO_COLOR"):
|
||||
return False
|
||||
if os.environ.get("CLI_ANYTHING_NO_COLOR"):
|
||||
return False
|
||||
if not hasattr(sys.stdout, "isatty"):
|
||||
return False
|
||||
return sys.stdout.isatty()
|
||||
|
||||
def _c(self, code: str, text: str) -> str:
|
||||
"""Apply color code if colors are supported."""
|
||||
if not self._color:
|
||||
return text
|
||||
return f"{code}{text}{_RESET}"
|
||||
|
||||
# ── Banner ────────────────────────────────────────────────────────
|
||||
|
||||
def print_banner(self):
|
||||
"""Print the startup banner with branding."""
|
||||
inner = 54
|
||||
|
||||
def _box_line(content: str) -> str:
|
||||
"""Wrap content in box drawing, padding to inner width."""
|
||||
pad = inner - _visible_len(content)
|
||||
vl = self._c(_DARK_GRAY, _V_LINE)
|
||||
return f"{vl}{content}{' ' * max(0, pad)}{vl}"
|
||||
|
||||
top = self._c(_DARK_GRAY, f"{_TL}{_H_LINE * inner}{_TR}")
|
||||
bot = self._c(_DARK_GRAY, f"{_BL}{_H_LINE * inner}{_BR}")
|
||||
|
||||
# Title: ◆ cli-anything · Shotcut
|
||||
icon = self._c(_CYAN + _BOLD, "◆")
|
||||
brand = self._c(_CYAN + _BOLD, "cli-anything")
|
||||
dot = self._c(_DARK_GRAY, "·")
|
||||
name = self._c(self.accent + _BOLD, self.display_name)
|
||||
title = f" {icon} {brand} {dot} {name}"
|
||||
|
||||
ver = f" {self._c(_DARK_GRAY, f' v{self.version}')}"
|
||||
tip = f" {self._c(_DARK_GRAY, ' Type help for commands, quit to exit')}"
|
||||
empty = ""
|
||||
|
||||
# Skill path for agent discovery
|
||||
skill_line = None
|
||||
if self.skill_path:
|
||||
skill_icon = self._c(_MAGENTA, "◇")
|
||||
skill_label = self._c(_DARK_GRAY, " Skill:")
|
||||
skill_path_display = self._c(_LIGHT_GRAY, self.skill_path)
|
||||
skill_line = f" {skill_icon} {skill_label} {skill_path_display}"
|
||||
|
||||
print(top)
|
||||
print(_box_line(title))
|
||||
print(_box_line(ver))
|
||||
if skill_line:
|
||||
print(_box_line(skill_line))
|
||||
print(_box_line(empty))
|
||||
print(_box_line(tip))
|
||||
print(bot)
|
||||
print()
|
||||
|
||||
# ── Prompt ────────────────────────────────────────────────────────
|
||||
|
||||
def prompt(self, project_name: str = "", modified: bool = False,
|
||||
context: str = "") -> str:
|
||||
"""Build a styled prompt string for prompt_toolkit or input().
|
||||
|
||||
Args:
|
||||
project_name: Current project name (empty if none open).
|
||||
modified: Whether the project has unsaved changes.
|
||||
context: Optional extra context to show in prompt.
|
||||
|
||||
Returns:
|
||||
Formatted prompt string.
|
||||
"""
|
||||
parts = []
|
||||
|
||||
# Icon
|
||||
if self._color:
|
||||
parts.append(f"{_CYAN}◆{_RESET} ")
|
||||
else:
|
||||
parts.append("> ")
|
||||
|
||||
# Software name
|
||||
parts.append(self._c(self.accent + _BOLD, self.software))
|
||||
|
||||
# Project context
|
||||
if project_name or context:
|
||||
ctx = context or project_name
|
||||
mod = "*" if modified else ""
|
||||
parts.append(f" {self._c(_DARK_GRAY, '[')}")
|
||||
parts.append(self._c(_LIGHT_GRAY, f"{ctx}{mod}"))
|
||||
parts.append(self._c(_DARK_GRAY, ']'))
|
||||
|
||||
parts.append(self._c(_GRAY, " ❯ "))
|
||||
|
||||
return "".join(parts)
|
||||
|
||||
def prompt_tokens(self, project_name: str = "", modified: bool = False,
|
||||
context: str = ""):
|
||||
"""Build prompt_toolkit formatted text tokens for the prompt.
|
||||
|
||||
Use with prompt_toolkit's FormattedText for proper ANSI handling.
|
||||
|
||||
Returns:
|
||||
list of (style, text) tuples for prompt_toolkit.
|
||||
"""
|
||||
accent_hex = _ANSI_256_TO_HEX.get(self.accent, "#5fafff")
|
||||
tokens = []
|
||||
|
||||
tokens.append(("class:icon", "◆ "))
|
||||
tokens.append(("class:software", self.software))
|
||||
|
||||
if project_name or context:
|
||||
ctx = context or project_name
|
||||
mod = "*" if modified else ""
|
||||
tokens.append(("class:bracket", " ["))
|
||||
tokens.append(("class:context", f"{ctx}{mod}"))
|
||||
tokens.append(("class:bracket", "]"))
|
||||
|
||||
tokens.append(("class:arrow", " ❯ "))
|
||||
|
||||
return tokens
|
||||
|
||||
def get_prompt_style(self):
|
||||
"""Get a prompt_toolkit Style object matching the skin.
|
||||
|
||||
Returns:
|
||||
prompt_toolkit.styles.Style
|
||||
"""
|
||||
try:
|
||||
from prompt_toolkit.styles import Style
|
||||
except ImportError:
|
||||
return None
|
||||
|
||||
accent_hex = _ANSI_256_TO_HEX.get(self.accent, "#5fafff")
|
||||
|
||||
return Style.from_dict({
|
||||
"icon": "#5fdfdf bold", # cyan brand color
|
||||
"software": f"{accent_hex} bold",
|
||||
"bracket": "#585858",
|
||||
"context": "#bcbcbc",
|
||||
"arrow": "#808080",
|
||||
# Completion menu
|
||||
"completion-menu.completion": "bg:#303030 #bcbcbc",
|
||||
"completion-menu.completion.current": f"bg:{accent_hex} #000000",
|
||||
"completion-menu.meta.completion": "bg:#303030 #808080",
|
||||
"completion-menu.meta.completion.current": f"bg:{accent_hex} #000000",
|
||||
# Auto-suggest
|
||||
"auto-suggest": "#585858",
|
||||
# Bottom toolbar
|
||||
"bottom-toolbar": "bg:#1c1c1c #808080",
|
||||
"bottom-toolbar.text": "#808080",
|
||||
})
|
||||
|
||||
# ── Messages ──────────────────────────────────────────────────────
|
||||
|
||||
def success(self, message: str):
|
||||
"""Print a success message with green checkmark."""
|
||||
icon = self._c(_GREEN + _BOLD, "✓")
|
||||
print(f" {icon} {self._c(_GREEN, message)}")
|
||||
|
||||
def error(self, message: str):
|
||||
"""Print an error message with red cross."""
|
||||
icon = self._c(_RED + _BOLD, "✗")
|
||||
print(f" {icon} {self._c(_RED, message)}", file=sys.stderr)
|
||||
|
||||
def warning(self, message: str):
|
||||
"""Print a warning message with yellow triangle."""
|
||||
icon = self._c(_YELLOW + _BOLD, "⚠")
|
||||
print(f" {icon} {self._c(_YELLOW, message)}")
|
||||
|
||||
def info(self, message: str):
|
||||
"""Print an info message with blue dot."""
|
||||
icon = self._c(_BLUE, "●")
|
||||
print(f" {icon} {self._c(_LIGHT_GRAY, message)}")
|
||||
|
||||
def hint(self, message: str):
|
||||
"""Print a subtle hint message."""
|
||||
print(f" {self._c(_DARK_GRAY, message)}")
|
||||
|
||||
def section(self, title: str):
|
||||
"""Print a section header."""
|
||||
print()
|
||||
print(f" {self._c(self.accent + _BOLD, title)}")
|
||||
print(f" {self._c(_DARK_GRAY, _H_LINE * len(title))}")
|
||||
|
||||
# ── Status display ────────────────────────────────────────────────
|
||||
|
||||
def status(self, label: str, value: str):
|
||||
"""Print a key-value status line."""
|
||||
lbl = self._c(_GRAY, f" {label}:")
|
||||
val = self._c(_WHITE, f" {value}")
|
||||
print(f"{lbl}{val}")
|
||||
|
||||
def status_block(self, items: dict[str, str], title: str = ""):
|
||||
"""Print a block of status key-value pairs.
|
||||
|
||||
Args:
|
||||
items: Dict of label -> value pairs.
|
||||
title: Optional title for the block.
|
||||
"""
|
||||
if title:
|
||||
self.section(title)
|
||||
|
||||
max_key = max(len(k) for k in items) if items else 0
|
||||
for label, value in items.items():
|
||||
lbl = self._c(_GRAY, f" {label:<{max_key}}")
|
||||
val = self._c(_WHITE, f" {value}")
|
||||
print(f"{lbl}{val}")
|
||||
|
||||
def progress(self, current: int, total: int, label: str = ""):
|
||||
"""Print a simple progress indicator.
|
||||
|
||||
Args:
|
||||
current: Current step number.
|
||||
total: Total number of steps.
|
||||
label: Optional label for the progress.
|
||||
"""
|
||||
pct = int(current / total * 100) if total > 0 else 0
|
||||
bar_width = 20
|
||||
filled = int(bar_width * current / total) if total > 0 else 0
|
||||
bar = "█" * filled + "░" * (bar_width - filled)
|
||||
text = f" {self._c(_CYAN, bar)} {self._c(_GRAY, f'{pct:3d}%')}"
|
||||
if label:
|
||||
text += f" {self._c(_LIGHT_GRAY, label)}"
|
||||
print(text)
|
||||
|
||||
# ── Table display ─────────────────────────────────────────────────
|
||||
|
||||
def table(self, headers: list[str], rows: list[list[str]],
|
||||
max_col_width: int = 40):
|
||||
"""Print a formatted table with box-drawing characters.
|
||||
|
||||
Args:
|
||||
headers: Column header strings.
|
||||
rows: List of rows, each a list of cell strings.
|
||||
max_col_width: Maximum column width before truncation.
|
||||
"""
|
||||
if not headers:
|
||||
return
|
||||
|
||||
# Calculate column widths
|
||||
col_widths = [min(len(h), max_col_width) for h in headers]
|
||||
for row in rows:
|
||||
for i, cell in enumerate(row):
|
||||
if i < len(col_widths):
|
||||
col_widths[i] = min(
|
||||
max(col_widths[i], len(str(cell))), max_col_width
|
||||
)
|
||||
|
||||
def pad(text: str, width: int) -> str:
|
||||
t = str(text)[:width]
|
||||
return t + " " * (width - len(t))
|
||||
|
||||
# Header
|
||||
header_cells = [
|
||||
self._c(_CYAN + _BOLD, pad(h, col_widths[i]))
|
||||
for i, h in enumerate(headers)
|
||||
]
|
||||
sep = self._c(_DARK_GRAY, f" {_V_LINE} ")
|
||||
header_line = f" {sep.join(header_cells)}"
|
||||
print(header_line)
|
||||
|
||||
# Separator
|
||||
sep_parts = [self._c(_DARK_GRAY, _H_LINE * w) for w in col_widths]
|
||||
sep_line = self._c(_DARK_GRAY, f" {'───'.join([_H_LINE * w for w in col_widths])}")
|
||||
print(sep_line)
|
||||
|
||||
# Rows
|
||||
for row in rows:
|
||||
cells = []
|
||||
for i, cell in enumerate(row):
|
||||
if i < len(col_widths):
|
||||
cells.append(self._c(_LIGHT_GRAY, pad(str(cell), col_widths[i])))
|
||||
row_sep = self._c(_DARK_GRAY, f" {_V_LINE} ")
|
||||
print(f" {row_sep.join(cells)}")
|
||||
|
||||
# ── Help display ──────────────────────────────────────────────────
|
||||
|
||||
def help(self, commands: dict[str, str]):
|
||||
"""Print a formatted help listing.
|
||||
|
||||
Args:
|
||||
commands: Dict of command -> description pairs.
|
||||
"""
|
||||
self.section("Commands")
|
||||
max_cmd = max(len(c) for c in commands) if commands else 0
|
||||
for cmd, desc in commands.items():
|
||||
cmd_styled = self._c(self.accent, f" {cmd:<{max_cmd}}")
|
||||
desc_styled = self._c(_GRAY, f" {desc}")
|
||||
print(f"{cmd_styled}{desc_styled}")
|
||||
print()
|
||||
|
||||
# ── Goodbye ───────────────────────────────────────────────────────
|
||||
|
||||
def print_goodbye(self):
|
||||
"""Print a styled goodbye message."""
|
||||
print(f"\n {_ICON_SMALL} {self._c(_GRAY, 'Goodbye!')}\n")
|
||||
|
||||
# ── Prompt toolkit session factory ────────────────────────────────
|
||||
|
||||
def create_prompt_session(self):
|
||||
"""Create a prompt_toolkit PromptSession with skin styling.
|
||||
|
||||
Returns:
|
||||
A configured PromptSession, or None if prompt_toolkit unavailable.
|
||||
"""
|
||||
try:
|
||||
from prompt_toolkit import PromptSession
|
||||
from prompt_toolkit.history import FileHistory
|
||||
from prompt_toolkit.auto_suggest import AutoSuggestFromHistory
|
||||
from prompt_toolkit.formatted_text import FormattedText
|
||||
|
||||
style = self.get_prompt_style()
|
||||
|
||||
session = PromptSession(
|
||||
history=FileHistory(self.history_file),
|
||||
auto_suggest=AutoSuggestFromHistory(),
|
||||
style=style,
|
||||
enable_history_search=True,
|
||||
)
|
||||
return session
|
||||
except ImportError:
|
||||
return None
|
||||
|
||||
def get_input(self, pt_session, project_name: str = "",
|
||||
modified: bool = False, context: str = "") -> str:
|
||||
"""Get input from user using prompt_toolkit or fallback.
|
||||
|
||||
Args:
|
||||
pt_session: A prompt_toolkit PromptSession (or None).
|
||||
project_name: Current project name.
|
||||
modified: Whether project has unsaved changes.
|
||||
context: Optional context string.
|
||||
|
||||
Returns:
|
||||
User input string (stripped).
|
||||
"""
|
||||
if pt_session is not None:
|
||||
from prompt_toolkit.formatted_text import FormattedText
|
||||
tokens = self.prompt_tokens(project_name, modified, context)
|
||||
return pt_session.prompt(FormattedText(tokens)).strip()
|
||||
else:
|
||||
raw_prompt = self.prompt(project_name, modified, context)
|
||||
return input(raw_prompt).strip()
|
||||
|
||||
# ── Toolbar builder ───────────────────────────────────────────────
|
||||
|
||||
def bottom_toolbar(self, items: dict[str, str]):
|
||||
"""Create a bottom toolbar callback for prompt_toolkit.
|
||||
|
||||
Args:
|
||||
items: Dict of label -> value pairs to show in toolbar.
|
||||
|
||||
Returns:
|
||||
A callable that returns FormattedText for the toolbar.
|
||||
"""
|
||||
def toolbar():
|
||||
from prompt_toolkit.formatted_text import FormattedText
|
||||
parts = []
|
||||
for i, (k, v) in enumerate(items.items()):
|
||||
if i > 0:
|
||||
parts.append(("class:bottom-toolbar.text", " │ "))
|
||||
parts.append(("class:bottom-toolbar.text", f" {k}: "))
|
||||
parts.append(("class:bottom-toolbar", v))
|
||||
return FormattedText(parts)
|
||||
return toolbar
|
||||
|
||||
|
||||
# ── ANSI 256-color to hex mapping (for prompt_toolkit styles) ─────────
|
||||
|
||||
_ANSI_256_TO_HEX = {
|
||||
"\033[38;5;33m": "#0087ff", # audacity navy blue
|
||||
"\033[38;5;35m": "#00af5f", # shotcut teal
|
||||
"\033[38;5;39m": "#00afff", # inkscape bright blue
|
||||
"\033[38;5;40m": "#00d700", # libreoffice green
|
||||
"\033[38;5;55m": "#5f00af", # obs purple
|
||||
"\033[38;5;69m": "#5f87ff", # kdenlive slate blue
|
||||
"\033[38;5;75m": "#5fafff", # default sky blue
|
||||
"\033[38;5;80m": "#5fd7d7", # brand cyan
|
||||
"\033[38;5;208m": "#ff8700", # blender deep orange
|
||||
"\033[38;5;214m": "#ffaf00", # gimp warm orange
|
||||
}
|
||||
@@ -0,0 +1,82 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
setup.py for cli-anything-macrocli
|
||||
|
||||
Install with: pip install -e .
|
||||
"""
|
||||
|
||||
from setuptools import setup, find_namespace_packages
|
||||
|
||||
with open("cli_anything/macrocli/README.md", "r", encoding="utf-8") as fh:
|
||||
long_description = fh.read()
|
||||
|
||||
setup(
|
||||
name="cli-anything-macrocli",
|
||||
version="1.0.0",
|
||||
author="cli-anything contributors",
|
||||
author_email="",
|
||||
description=(
|
||||
"MacroCLI — A layered CLI that converts GUI workflows into "
|
||||
"parameterized, agent-callable macros. Requires: PyYAML, click, prompt-toolkit."
|
||||
),
|
||||
long_description=long_description,
|
||||
long_description_content_type="text/markdown",
|
||||
url="https://github.com/HKUDS/CLI-Anything",
|
||||
packages=find_namespace_packages(include=["cli_anything.*"]),
|
||||
classifiers=[
|
||||
"Development Status :: 4 - Beta",
|
||||
"Intended Audience :: Developers",
|
||||
"Topic :: Software Development :: Libraries :: Python Modules",
|
||||
"License :: OSI Approved :: MIT License",
|
||||
"Programming Language :: Python :: 3",
|
||||
"Programming Language :: Python :: 3.10",
|
||||
"Programming Language :: Python :: 3.11",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
],
|
||||
python_requires=">=3.10",
|
||||
install_requires=[
|
||||
"click>=8.0.0",
|
||||
"prompt-toolkit>=3.0.0",
|
||||
"PyYAML>=6.0",
|
||||
"defusedxml>=0.7.1",
|
||||
],
|
||||
extras_require={
|
||||
"dev": [
|
||||
"pytest>=7.0.0",
|
||||
"pytest-cov>=4.0.0",
|
||||
],
|
||||
"visual": [
|
||||
"mss>=9.0.0",
|
||||
"Pillow>=10.0.0",
|
||||
"numpy>=1.24.0",
|
||||
"pynput>=1.7.0",
|
||||
],
|
||||
"gui_agent": [
|
||||
"openai>=1.0.0",
|
||||
"mss>=9.0.0",
|
||||
"Pillow>=10.0.0",
|
||||
],
|
||||
"all": [
|
||||
"mss>=9.0.0",
|
||||
"Pillow>=10.0.0",
|
||||
"numpy>=1.24.0",
|
||||
"pynput>=1.7.0",
|
||||
"openai>=1.0.0",
|
||||
],
|
||||
},
|
||||
entry_points={
|
||||
"console_scripts": [
|
||||
"cli-anything-macrocli=cli_anything.macrocli.macrocli_cli:cli",
|
||||
],
|
||||
},
|
||||
package_data={
|
||||
"cli_anything.macrocli": [
|
||||
"skills/*.md",
|
||||
"macro_definitions/*.yaml",
|
||||
"macro_definitions/examples/*.yaml",
|
||||
"macro_definitions/demo/*.yaml",
|
||||
],
|
||||
},
|
||||
include_package_data=True,
|
||||
zip_safe=False,
|
||||
)
|
||||
Reference in New Issue
Block a user