Compare commits
131 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 9b2efd795c | |||
| 257916db67 | |||
| 1763a2f708 | |||
| 256f7fc2dd | |||
| a41011de19 | |||
| bf5931e7c1 | |||
| a68d0f984d | |||
| 033dce67b1 | |||
| 4c3bf5bbc9 | |||
| 63ac368975 | |||
| fea0b75bc4 | |||
| c281d34c4a | |||
| f5608951e1 | |||
| 92e298852c | |||
| 27bb93b810 | |||
| ee0af9a24c | |||
| 9a24b8dfe2 | |||
| 4321bd824d | |||
| 175fe501a6 | |||
| 254ed90aed | |||
| 09bc641aad | |||
| 6243793ec0 | |||
| 330ece7a62 | |||
| 6747d12385 | |||
| 7a0bfe9122 | |||
| 595a9ab3a4 | |||
| 4ba71f2d4a | |||
| 889d9dcbda | |||
| a2888506ba | |||
| 0531963435 | |||
| c6b7552008 | |||
| 635d77fad4 | |||
| 7b44b7dab1 | |||
| 484502168d | |||
| 8b2ec8300d | |||
| a1fef67cdc | |||
| 9cff3a05b0 | |||
| c38e2fc066 | |||
| 653369d983 | |||
| f61f70d7b1 | |||
| b93b2cfb30 | |||
| efb4b71eb3 | |||
| c70811cc40 | |||
| 73fb0fa0bf | |||
| a75053a740 | |||
| 5b46a2f7b3 | |||
| 1929ad8051 | |||
| f61901e842 | |||
| 0c6b81d61f | |||
| 225e87f7d7 | |||
| 18fc6dea13 | |||
| a0f8552c69 | |||
| b22d5e9368 | |||
| 0e557a65d3 | |||
| ffc50221df | |||
| a07837c657 | |||
| 807e3ecd47 | |||
| d2489758f0 | |||
| 373147907a | |||
| e0860bd3bd | |||
| 2ab4fdb2f7 | |||
| 0f7e9a20f4 | |||
| 57a9134997 | |||
| d5b1977fab | |||
| 30d4fb3ebf | |||
| 626ad2daf4 | |||
| ffa7d9c06a | |||
| 0528a7a2c2 | |||
| ebb402cfd3 | |||
| 8f30deffe9 | |||
| 52a0551569 | |||
| a98292b814 | |||
| 03f4c47ebb | |||
| 4e33118c48 | |||
| 5f97b2416e | |||
| a857455fd0 | |||
| 7873f0d109 | |||
| f10d8875ee | |||
| 56a954a0a9 | |||
| d611d6ddf6 | |||
| e536394812 | |||
| 47e0e93387 | |||
| c835d7cf2e | |||
| d9002c3cb2 | |||
| 11faa13d63 | |||
| 1bcdc62a50 | |||
| cc825efb83 | |||
| df348b6335 | |||
| 1498a87806 | |||
| 726b3ee028 | |||
| 438e373097 | |||
| 380bab4d7b | |||
| 6bfc5c50d0 | |||
| 9a8a14331d | |||
| 25f2043f53 | |||
| 1fb617d4f4 | |||
| b618f639d7 | |||
| a24a28b38a | |||
| bed1e4777e | |||
| be66e015bf | |||
| 90ec299221 | |||
| 593dcbcaeb | |||
| 0305dfb3ec | |||
| b731c3ede1 | |||
| 3e09dde114 | |||
| 801f6b321c | |||
| 1325770b28 | |||
| 6c83a4c978 | |||
| e7eab63852 | |||
| f683b4aed2 | |||
| 835588daa0 | |||
| 0bba07fe19 | |||
| 75c0481dba | |||
| 25d0bc7ab1 | |||
| 9078071803 | |||
| c4075ffea3 | |||
| 699bab3d3a | |||
| 4dd7b76114 | |||
| 559ba76f23 | |||
| 8d48829c20 | |||
| 6ef371f62f | |||
| ee97f8e78d | |||
| 1158027082 | |||
| a2bb51483d | |||
| 55dc030208 | |||
| df7fde80e8 | |||
| badfb6b592 | |||
| cd68ff1e25 | |||
| b11a83df66 | |||
| 1c00020d36 | |||
| d4eae153d3 |
@@ -8,6 +8,34 @@ The format is based on Keep a Changelog, and this project currently tracks chang
|
||||
|
||||
### Added
|
||||
|
||||
- Hooks now support a `priority` field (default `0`). Within an event, hooks run highest-priority first, and hooks sharing a priority keep their registration order. This lets users order, for example, a security-check hook ahead of a logging hook regardless of where each is declared in settings or contributed by plugins.
|
||||
- `edit_file` and `write_file` in the React TUI now preview a unified diff before applying file changes, let users approve once or for the rest of the session, and skip the extra prompt automatically in `full_auto` mode.
|
||||
|
||||
### Fixed
|
||||
|
||||
- Codex subscription requests now pass reasoning effort separately, enabling `gpt-5.5` with `xhigh` effort instead of treating `gpt-5.5 xhigh` as an unsupported model name.
|
||||
- Telegram channel now delivers replies again under `ohmo init --no-interactive` and other configs that do not write a `reply_to_message` field. `TelegramConfig` declares `reply_to_message: bool = True` so the attribute access in `TelegramChannel.send` no longer raises `AttributeError` and outbound progress/tool-hint/final messages are sent as expected. See issue #243.
|
||||
|
||||
## [0.1.9] - 2026-05-07
|
||||
|
||||
### Added
|
||||
|
||||
- Added a bundled `skill-creator` skill for creating, improving, and verifying OpenHarness/ohmo skills.
|
||||
- User-invocable skills can now be triggered directly as slash commands, with support for skill-specific arguments and model override metadata.
|
||||
|
||||
### Fixed
|
||||
|
||||
- `oh setup` can now update the API key for an already-configured API-key provider profile instead of only changing the model.
|
||||
- `oh provider edit <profile> --api-key <key>` can now replace a saved profile API key, and `oh provider add ... --api-key <key>` can store one during profile creation.
|
||||
|
||||
## [0.1.8] - 2026-05-06
|
||||
|
||||
### Added
|
||||
|
||||
- Built-in `nvidia` provider profile so `oh setup` offers NVIDIA NIM as a first-class OpenAI-compatible provider choice, with `NVIDIA_API_KEY` auth source, `openai/gpt-oss-120b` as the default model, and the NVIDIA NIM endpoint.
|
||||
- Built-in `qwen` provider profile so `oh setup` offers Qwen (DashScope) as a first-class provider choice, with `dashscope_api_key` auth source, `qwen-plus` as the default model, and the DashScope OpenAI-compatible endpoint.
|
||||
- Plugin tool discovery: plugins can now provide `BaseTool` subclasses in a `<plugin>/tools/` directory and they are auto-discovered, instantiated, and registered in the tool registry at runtime. Add `tools_dir` to `plugin.json` (defaults to `"tools"`).
|
||||
- `oh --dry-run` safe preview mode for inspecting resolved runtime settings, auth state, prompt assembly, commands, skills, tools, and configured MCP servers without executing the model or tools.
|
||||
- Built-in `minimax` provider profile so `oh setup` offers MiniMax as a first-class provider choice, with `MINIMAX_API_KEY` auth source, `MiniMax-M2.7` as the default model, and `MiniMax-M2.7-highspeed` in the model picker.
|
||||
- Docker as an alternative sandbox backend (`sandbox.backend = "docker"`) for stronger execution isolation with configurable resource limits, network isolation, and automatic image management.
|
||||
- Built-in `gemini` provider profile so `oh setup` offers Google Gemini as a first-class provider choice, with `gemini_api_key` auth source and `gemini-2.5-flash` as the default model.
|
||||
@@ -24,6 +52,15 @@ The format is based on Keep a Changelog, and this project currently tracks chang
|
||||
|
||||
### Fixed
|
||||
|
||||
- Subprocess teammate spawn (`agent` tool, `task_create`) now works on Windows under Git Bash. `subprocess_backend.spawn` builds a direct-exec `argv` list and passes it through new `argv=` and `env=` kwargs on `BackgroundTaskManager.create_agent_task` / `create_shell_task`; `_start_process` then runs the executable via `asyncio.create_subprocess_exec(*argv)` with no shell in between. Previously the spawn command was a single string interpreted by `bash -lc`, which on Windows could not reliably exec a Windows-pathed Python interpreter (e.g. `C:\Users\...\python.exe`) — Git Bash's escape parser consumed the backslashes from the embedded env-prefix and, even with proper quoting, bash launched via `asyncio.create_subprocess_exec` returned `command not found` for Windows-pathed binaries that worked perfectly when invoked interactively. Bypassing the shell sidesteps the entire class of cross-platform quoting and path-translation hazard. The legacy shell-evaluated `command=` path is preserved for callers (e.g. `BashTool`) that legitimately want shell semantics. See issue #230.
|
||||
- Bundled skill loader now uses `yaml.safe_load` for SKILL.md frontmatter, matching the user-skill loader. The shared parser is extracted to `openharness.skills._frontmatter` so bundled and user skills handle YAML block scalars (`>`, `|`), quoted values, and other standard YAML constructs the same way.
|
||||
- Compaction now detects llama.cpp/OpenAI-compatible context overflow errors, accounts for image blocks in auto-compact token estimates, and strips image payloads from summarizer-only compaction requests.
|
||||
- Large tool results are now bounded in conversation history: oversized outputs are saved under `tool_artifacts`, old MCP results become microcompactable, and context collapse trims stale tool-result payloads.
|
||||
- ohmo now keeps personal memory isolated from OpenHarness project memory: `/memory` in ohmo sessions targets the ohmo workspace memory store, and ohmo runtime prompt refreshes no longer inject project memory unless explicitly requested.
|
||||
- Fixed `glob` and `grep` tools hanging indefinitely when the `rg` subprocess produced enough stderr output to fill the OS pipe buffer. `stderr` is now redirected to `DEVNULL` so it is discarded rather than blocking the child process.
|
||||
- Fixed `bash_tool` hanging after a timed-out command when the subprocess stdout stream stayed open. `_read_remaining_output` now applies a 2-second `asyncio.wait_for` timeout so the tool always returns promptly.
|
||||
- Fixed `session_runner` background task deadlock caused by an unread `stderr=PIPE` stream. The subprocess now uses `stderr=STDOUT` so all output merges into the single readable stdout pipe.
|
||||
- React TUI prompt input now treats the raw DEL byte (`0x7f`) as backward delete while preserving true forward-delete escape sequences, fixing backspace failures seen in some macOS terminal environments.
|
||||
- `todo_write` tool now updates an existing unchecked item in-place when `checked=True` instead of appending a duplicate `[x]` line.
|
||||
|
||||
- Built-in `Explore` and `claude-code-guide` agents no longer hard-code `model="haiku"`, which caused them to fail for users on non-Anthropic providers (OpenAI, Bedrock, custom base URLs, etc.). Both agents now use `model="inherit"` so they run with whatever model the parent session is using. `build_inherited_cli_flags` is also fixed to skip the `--model` flag entirely when the value is `"inherit"`, letting the subprocess correctly inherit the parent model via the `OPENHARNESS_MODEL` environment variable instead of receiving the literal string `"inherit"` as a model name.
|
||||
@@ -45,6 +82,8 @@ The format is based on Keep a Changelog, and this project currently tracks chang
|
||||
|
||||
### Changed
|
||||
|
||||
- ohmo Feishu group routing now supports managed group creation, gateway-scoped provider/model commands, and stricter group mention handling so group conversations only wake ohmo when explicitly addressed.
|
||||
- Dry-run output now reports a `ready` / `warning` / `blocked` readiness verdict, concrete `next_actions`, likely matching skills/tools for normal prompts, and richer slash-command previews for read-only vs stateful command paths.
|
||||
- React TUI now groups consecutive `tool` + `tool_result` transcript rows into a single compound row: success shows the result line count inline (e.g. `→ 24L`), errors show a red icon and up to 5 lines of error detail beneath the tool row. Standalone successful tool results are suppressed to reduce transcript noise; standalone errors are still surfaced.
|
||||
- README now links to contribution docs, changelog, showcase material, and provider compatibility guidance.
|
||||
- README quick start now includes a one-command demo and clearer provider compatibility notes.
|
||||
|
||||
@@ -153,6 +153,10 @@ OpenHarness is an open-source Python implementation designed for **researchers,
|
||||
|
||||
## 📰 What's New
|
||||
|
||||
- **Unreleased** 🔍 **Dry-run safe preview**:
|
||||
- `oh --dry-run` previews resolved runtime settings, auth state, skills, commands, tools, and configured MCP servers without executing the model, tools, or subagents.
|
||||
- Dry-run now reports a `ready` / `warning` / `blocked` readiness verdict with concrete next-step suggestions such as fixing auth, fixing MCP config, or running the prompt directly.
|
||||
- Prompt previews include likely matching skills and tools, while slash-command previews show whether the command is mostly read-only or stateful.
|
||||
- **2026-04-18** ⚙️ **v0.1.7** — Packaging & TUI polish:
|
||||
- Install script now links `oh`, `ohmo`, and `openharness` into `~/.local/bin` instead of prepending the virtualenv `bin` directory to `PATH`, which avoids clobbering Conda-managed shells.
|
||||
- React TUI now supports `Shift+Enter` to insert a newline while keeping plain `Enter` as submit.
|
||||
@@ -223,7 +227,7 @@ oh setup # interactive wizard — pick a provider, authenticate, done
|
||||
# On Windows PowerShell, use: openh setup
|
||||
```
|
||||
|
||||
Supports **Claude / OpenAI / Copilot / Codex / Moonshot(Kimi) / GLM / MiniMax** and any compatible endpoint.
|
||||
Supports **Claude / OpenAI / Copilot / Codex / Moonshot(Kimi) / GLM / MiniMax / NVIDIA NIM** and any compatible endpoint.
|
||||
|
||||
### 3. Run
|
||||
|
||||
@@ -261,6 +265,43 @@ oh -p "List all functions in main.py" --output-format json
|
||||
oh -p "Fix the bug" --output-format stream-json
|
||||
```
|
||||
|
||||
### Dry Run (Safe Preview)
|
||||
|
||||
Use `--dry-run` when you want to inspect what OpenHarness would use before any live execution starts.
|
||||
|
||||
```bash
|
||||
# Preview an interactive session setup
|
||||
oh --dry-run
|
||||
|
||||
# Preview one prompt without executing the model or tools
|
||||
oh --dry-run -p "Review this bug fix and grep for failing tests"
|
||||
|
||||
# Preview a slash command path
|
||||
oh --dry-run -p "/plugin list"
|
||||
|
||||
# Get structured output for scripts or channels
|
||||
oh --dry-run -p "Explain this repository" --output-format json
|
||||
```
|
||||
|
||||
Dry-run is intentionally static:
|
||||
|
||||
- It does **not** call the model
|
||||
- It does **not** execute tools or spawn subagents
|
||||
- It does **not** connect to MCP servers
|
||||
- It **does** resolve settings, auth status, prompt assembly, skills, commands, tools, and obvious MCP config problems
|
||||
|
||||
Readiness levels:
|
||||
|
||||
- `ready`: configuration looks usable; the next suggested action is usually to run the prompt directly
|
||||
- `warning`: OpenHarness can resolve the session, but something important still looks wrong, such as broken MCP config or missing auth for later model work
|
||||
- `blocked`: the requested path will not run successfully as-is, for example an unknown slash command or a prompt that cannot resolve a runtime client
|
||||
|
||||
`next actions` in the dry-run output tell you the shortest fix or follow-up step, such as:
|
||||
|
||||
- run `oh auth login`
|
||||
- fix or disable broken MCP configuration
|
||||
- run the prompt directly with `oh -p "..."` or open the interactive UI with `oh`
|
||||
|
||||
## 🔌 Provider Compatibility
|
||||
|
||||
OpenHarness treats providers as **workflows** backed by named profiles. In day-to-day use, prefer:
|
||||
@@ -306,6 +347,7 @@ Any provider implementing the OpenAI `/v1/chat/completions` style API works:
|
||||
| **DeepSeek** | `https://api.deepseek.com` | `deepseek-chat`, `deepseek-reasoner` |
|
||||
| **GitHub Models** | `https://models.inference.ai.azure.com` | `gpt-4o`, `Meta-Llama-3.1-405B-Instruct` |
|
||||
| **SiliconFlow** | `https://api.siliconflow.cn/v1` | `deepseek-ai/DeepSeek-V3` |
|
||||
| **NVIDIA NIM** | `https://integrate.api.nvidia.com/v1` | `openai/gpt-oss-120b`, `nvidia/llama-3.3-nemotron-super-49b-v1` |
|
||||
| **Google Gemini** | `https://generativelanguage.googleapis.com/v1beta/openai` | `gemini-2.5-flash`, `gemini-2.5-pro` |
|
||||
| **Groq** | `https://api.groq.com/openai/v1` | `llama-3.3-70b-versatile` |
|
||||
| **Ollama (local)** | `http://localhost:11434/v1` | any local model |
|
||||
@@ -499,7 +541,47 @@ Available Skills:
|
||||
- ... 40+ more
|
||||
```
|
||||
|
||||
**Compatible with [anthropics/skills](https://github.com/anthropics/skills)** — just copy `.md` files to `~/.openharness/skills/`.
|
||||
Skills can live in bundled, user, ohmo, project, or plugin locations. User-level skills are loaded from:
|
||||
|
||||
```text
|
||||
~/.openharness/skills/<skill>/SKILL.md
|
||||
~/.claude/skills/<skill>/SKILL.md
|
||||
~/.agents/skills/<skill>/SKILL.md
|
||||
```
|
||||
|
||||
Project-level skills are enabled by default and are discovered from the current working directory up to the git root:
|
||||
|
||||
```text
|
||||
<project>/.openharness/skills/<skill>/SKILL.md
|
||||
<project>/.agents/skills/<skill>/SKILL.md
|
||||
<project>/.claude/skills/<skill>/SKILL.md
|
||||
```
|
||||
|
||||
Disable project skills for untrusted repositories with:
|
||||
|
||||
```bash
|
||||
oh config set allow_project_skills false
|
||||
```
|
||||
|
||||
Use `/skills` to list loaded skills with their source and path. User-invocable skills can be run directly as slash commands, for example `/deploy staging`.
|
||||
|
||||
**Compatible with [anthropics/skills](https://github.com/anthropics/skills)** — use the `SKILL.md` directory layout above.
|
||||
|
||||
### 🌐 Web search and proxy settings
|
||||
|
||||
Built-in `web_search` uses DuckDuckGo HTML search by default. In regions where that endpoint is unreachable, point OpenHarness at a trusted public HTML search endpoint or your own SearXNG instance:
|
||||
|
||||
```bash
|
||||
export OPENHARNESS_WEB_SEARCH_URL="https://your-searxng.example/search"
|
||||
```
|
||||
|
||||
`web_search` and `web_fetch` keep `trust_env=False` for SSRF safety, so they do not automatically inherit `HTTP_PROXY` / `HTTPS_PROXY`. If you need a proxy, opt in with an OpenHarness-specific variable:
|
||||
|
||||
```bash
|
||||
export OPENHARNESS_WEB_PROXY="http://127.0.0.1:7890"
|
||||
```
|
||||
|
||||
The proxy URL must be HTTP/HTTPS and cannot contain embedded credentials.
|
||||
|
||||
### 🔌 Plugin System
|
||||
|
||||
@@ -757,6 +839,16 @@ Useful contributor entry points:
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Troubleshooting
|
||||
|
||||
### Backspace key in macOS Terminal.app
|
||||
|
||||
OpenHarness handles both common terminal delete sequences, including the raw `DEL` byte (`0x7f`) that macOS Terminal.app sends for Backspace. If Backspace inserts spaces or visible control characters instead of deleting text, upgrade OpenHarness first.
|
||||
|
||||
For older versions that do not include this fix, use a terminal that sends a standard Backspace sequence or adjust your terminal keyboard profile as a temporary workaround.
|
||||
|
||||
---
|
||||
|
||||
## 📄 License
|
||||
|
||||
MIT — see [LICENSE](LICENSE).
|
||||
|
||||
@@ -20,6 +20,12 @@
|
||||
|
||||
## 最新更新
|
||||
|
||||
### Unreleased · Dry-run 安全预览
|
||||
|
||||
- 新增 `oh --dry-run`,可以在**不执行模型、不执行工具、不 spawn subagent** 的前提下,预览当前会话会使用的配置、skills、commands、tools 和 MCP 配置。
|
||||
- Dry-run 会给出 `ready / warning / blocked` 结论,并直接告诉你下一步该做什么,例如先修认证、先修 MCP 配置,或者可以直接运行。
|
||||
- 对普通 prompt,会给出可能命中的 skills / tools;对 slash command,会展示它更偏只读还是会改本地状态。
|
||||
|
||||
### 2026-04-06 · v0.1.2
|
||||
|
||||
- 新增统一配置入口 `oh setup`
|
||||
@@ -169,6 +175,44 @@ oh -p "List all functions in main.py" --output-format json
|
||||
oh -p "Fix the bug" --output-format stream-json
|
||||
```
|
||||
|
||||
### Dry-run 安全预览
|
||||
|
||||
如果你想先看 OpenHarness **会怎么跑**,但又不想真的执行模型或工具,可以用:
|
||||
|
||||
```bash
|
||||
# 预览交互会话本身
|
||||
oh --dry-run
|
||||
|
||||
# 预览一个普通 prompt
|
||||
oh --dry-run -p "Review this bug fix and grep for failing tests"
|
||||
|
||||
# 预览 slash command
|
||||
oh --dry-run -p "/plugin list"
|
||||
|
||||
# 输出结构化 JSON,方便脚本或 channel 使用
|
||||
oh --dry-run -p "Explain this repository" --output-format json
|
||||
```
|
||||
|
||||
Dry-run 的边界是明确的:
|
||||
|
||||
- **不会**调用模型
|
||||
- **不会**执行 tools
|
||||
- **不会**启动 subagent
|
||||
- **不会**连接 MCP server
|
||||
- **会**解析 settings、auth 状态、system prompt、skills、commands、tools,以及明显错误的 MCP 配置
|
||||
|
||||
Readiness 结论说明:
|
||||
|
||||
- `ready`:当前配置基本可直接运行
|
||||
- `warning`:能解析会话,但仍有重要问题需要先处理,比如 MCP 配置错误或后续模型调用缺认证
|
||||
- `blocked`:按当前状态直接运行会失败,比如 slash command 不存在,或者普通 prompt 无法解析 runtime client
|
||||
|
||||
Dry-run 输出里的 `next actions` 会直接给出下一步建议,例如:
|
||||
|
||||
- 先执行 `oh auth login`
|
||||
- 先修或禁用坏掉的 MCP 配置
|
||||
- 直接运行 `oh -p "..."` 或进入 `oh`
|
||||
|
||||
---
|
||||
|
||||
## Provider 兼容性概览
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
# v0.1.8 — Providers, ohmo Feishu Groups, and Safer Remotes
|
||||
|
||||
OpenHarness v0.1.8 is a stabilization and provider-expansion release. It adds more first-class provider workflows, improves ohmo's Feishu group experience, hardens remote-channel security, and fixes several Windows/MCP reliability issues.
|
||||
|
||||
## Highlights
|
||||
|
||||
- **New provider workflows**
|
||||
- Added NVIDIA NIM as a built-in OpenAI-compatible provider using `NVIDIA_API_KEY`.
|
||||
- Added ModelScope Inference API support.
|
||||
- Added Qwen (DashScope), MiniMax, and Gemini provider profiles.
|
||||
- Improved OpenAI-compatible API behavior, including explicit bearer authorization headers and `<think>` block filtering for compatible streaming responses.
|
||||
|
||||
- **ohmo Feishu group support**
|
||||
- Added Feishu managed group creation flow for ohmo.
|
||||
- Added gateway-scoped provider/model commands for chat-based operation.
|
||||
- Improved group routing and mention policy so ohmo responds in shared Feishu groups only when explicitly addressed.
|
||||
- Hardened Feishu attachment filename handling.
|
||||
|
||||
- **Security and remote-channel hardening**
|
||||
- Kept sensitive config/auth/provider/model/ship commands local-only by default in remote channels.
|
||||
- Kept bridge commands local-only by default.
|
||||
- Added coverage for bridge spawn blocking and remote gateway security behavior.
|
||||
- Rejected path traversal names during plugin uninstall.
|
||||
|
||||
- **Windows and shell reliability**
|
||||
- Fixed Windows agent/subagent spawning by direct-executing teammate argv instead of shell-wrapping Python paths.
|
||||
- Windows shell resolution now skips discovered `bash.exe` binaries that cannot actually run commands, falling back to PowerShell/cmd.
|
||||
- Improved Windows gateway process lifecycle handling.
|
||||
- Avoided shell execution when opening browsers on Windows.
|
||||
|
||||
- **MCP, tools, and stability**
|
||||
- MCP startup now isolates failed servers instead of aborting the whole OpenHarness startup.
|
||||
- Fixed subprocess stderr pipe deadlocks in grep/glob/bash/session runner paths.
|
||||
- Bounded large tool results in conversation history and improved compaction under large/vision contexts.
|
||||
- Improved skill frontmatter parsing with YAML `safe_load` for bundled and user skills.
|
||||
|
||||
- **TUI and UX fixes**
|
||||
- Restored raw `DEL` backspace handling for macOS Terminal-style environments.
|
||||
- Added better slash-command completion, markdown table rendering, spinner behavior, and escape interruption.
|
||||
- Added image-to-text fallback support for text-only models and `--vision-model` override.
|
||||
|
||||
## External contributors
|
||||
|
||||
Thanks to the external contributors whose PRs are included in this release:
|
||||
|
||||
- @Litianhui888 — MCP startup isolation (#237)
|
||||
- @Hinotoi-agent — remote command security hardening (#232, #209, #208, #198, #197)
|
||||
- @nsxdavid — Windows agent spawn fix (#231)
|
||||
- @voidborne-d — bundled skill frontmatter parsing (#229)
|
||||
- @Mcy0618 — image-to-text fallback and ModelScope support (#227, #224)
|
||||
- @WANG-Guangxin — invalid regex fallback fix (#219)
|
||||
- @glitch-ux — Windows browser auth safety, async coordinator draining, autopilot shell safety, hook events (#217, #200, #188, #170)
|
||||
- @Escapingbug — Windows gateway lifecycle and Telegram proxy config fixes (#193, #192)
|
||||
- @ZevGit — Qwen provider profile (#207)
|
||||
- @yl-jiang — subprocess stderr deadlock fixes and OpenAI-compatible think-block filtering (#205, #174)
|
||||
- @he-yufeng — TUI slash-command completion polish (#185)
|
||||
- @powAu3 — raw DEL backspace fix (#182)
|
||||
- @flobo3 — shell subprocess stdin default fix (#179)
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
pip install --upgrade openharness-ai==0.1.8
|
||||
```
|
||||
|
||||
Or run the installer from the repository docs.
|
||||
@@ -0,0 +1,32 @@
|
||||
# v0.1.9 — Skill Workflows and Provider Key Updates
|
||||
|
||||
OpenHarness v0.1.9 is a small follow-up release after v0.1.8 focused on making skills easier to create and invoke, plus fixing provider API key updates.
|
||||
|
||||
## Highlights
|
||||
|
||||
- **Bundled skill creator**
|
||||
- Added a bundled `skill-creator` skill for creating, improving, and verifying OpenHarness/ohmo skills.
|
||||
- This makes repeatable workflows easier to capture as first-class skills.
|
||||
|
||||
- **User-invocable skill slash commands**
|
||||
- Skills marked as user-invocable can now be triggered directly with slash commands.
|
||||
- Slash-invoked skills support user arguments and can request a model override through skill metadata.
|
||||
|
||||
- **Provider API key update fix**
|
||||
- `oh setup` now lets users update the API key for an already-configured API-key provider profile.
|
||||
- `oh provider edit <profile> --api-key <key>` can replace a saved profile key directly.
|
||||
- `oh provider add ... --api-key <key>` can store a key while creating a provider profile.
|
||||
|
||||
## Fixes
|
||||
|
||||
- Fixed issue #238, where users could change a configured provider model but had no supported path to update the saved API key.
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
pip install --upgrade openharness-ai==0.1.9
|
||||
```
|
||||
|
||||
## Contributors
|
||||
|
||||
This release is primarily a maintainer follow-up release. Thanks to the users and contributors who reported and validated the provider key update workflow, especially the reporter of #238.
|
||||
+134
-13
@@ -1,6 +1,7 @@
|
||||
import React, {useDeferredValue, useEffect, useMemo, useState} from 'react';
|
||||
import React, {useDeferredValue, useEffect, useMemo, useRef, useState} from 'react';
|
||||
import {Box, Text, useApp, useInput} from 'ink';
|
||||
|
||||
import {readClipboardImage, type ImageAttachment} from './clipboardImage.js';
|
||||
import {CommandPicker} from './components/CommandPicker.js';
|
||||
import {ConversationView} from './components/ConversationView.js';
|
||||
import {ModalHost} from './components/ModalHost.js';
|
||||
@@ -11,7 +12,7 @@ import {SwarmPanel} from './components/SwarmPanel.js';
|
||||
import {TodoPanel} from './components/TodoPanel.js';
|
||||
import {useBackendSession} from './hooks/useBackendSession.js';
|
||||
import {ThemeProvider, useTheme} from './theme/ThemeContext.js';
|
||||
import type {FrontendConfig} from './types.js';
|
||||
import type {FrontendConfig, ImageAttachmentPayload} from './types.js';
|
||||
|
||||
const rawReturnSubmit = process.env.OPENHARNESS_FRONTEND_RAW_RETURN === '1';
|
||||
const scriptedSteps = (() => {
|
||||
@@ -64,6 +65,8 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
|
||||
const [modalInput, setModalInput] = useState('');
|
||||
const [history, setHistory] = useState<string[]>([]);
|
||||
const [historyIndex, setHistoryIndex] = useState(-1);
|
||||
const [imageAttachments, setImageAttachments] = useState<ImageAttachment[]>([]);
|
||||
const [clipboardStatus, setClipboardStatus] = useState<string | null>(null);
|
||||
const [lastEscapeAt, setLastEscapeAt] = useState(0);
|
||||
const [scriptIndex, setScriptIndex] = useState(0);
|
||||
const [pickerIndex, setPickerIndex] = useState(0);
|
||||
@@ -77,6 +80,7 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
|
||||
const deferredTodoMarkdown = useDeferredValue(session.todoMarkdown);
|
||||
const deferredSwarmTeammates = useDeferredValue(session.swarmTeammates);
|
||||
const deferredSwarmNotifications = useDeferredValue(session.swarmNotifications);
|
||||
const clipboardStatusTimerRef = useRef<NodeJS.Timeout | null>(null);
|
||||
|
||||
useEffect(() => {
|
||||
const nextTheme = session.status.theme;
|
||||
@@ -85,6 +89,47 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
|
||||
}
|
||||
}, [session.status.theme, setThemeName]);
|
||||
|
||||
useEffect(() => {
|
||||
return () => {
|
||||
if (clipboardStatusTimerRef.current) {
|
||||
clearTimeout(clipboardStatusTimerRef.current);
|
||||
}
|
||||
};
|
||||
}, []);
|
||||
|
||||
const setTemporaryClipboardStatus = (message: string): void => {
|
||||
setClipboardStatus(message);
|
||||
if (clipboardStatusTimerRef.current) {
|
||||
clearTimeout(clipboardStatusTimerRef.current);
|
||||
}
|
||||
clipboardStatusTimerRef.current = setTimeout(() => {
|
||||
setClipboardStatus(null);
|
||||
clipboardStatusTimerRef.current = null;
|
||||
}, 2500);
|
||||
};
|
||||
|
||||
const attachClipboardImage = (): void => {
|
||||
void (async () => {
|
||||
const image = await readClipboardImage();
|
||||
if (!image) {
|
||||
setTemporaryClipboardStatus('No image found in clipboard');
|
||||
return;
|
||||
}
|
||||
setImageAttachments((items) => [...items, image]);
|
||||
setTemporaryClipboardStatus(`Attached ${image.label}`);
|
||||
})().catch((error: unknown) => {
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
setTemporaryClipboardStatus(`Clipboard image unavailable: ${message}`);
|
||||
});
|
||||
};
|
||||
|
||||
const imagePayloads = (): ImageAttachmentPayload[] =>
|
||||
imageAttachments.map((image) => ({
|
||||
media_type: image.media_type,
|
||||
data: image.data,
|
||||
source_path: image.source_path,
|
||||
}));
|
||||
|
||||
// Current tool name for spinner
|
||||
const currentToolName = useMemo(() => {
|
||||
for (let i = deferredTranscript.length - 1; i >= 0; i--) {
|
||||
@@ -177,14 +222,25 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
|
||||
|
||||
useInput((chunk, key) => {
|
||||
const isPaste = chunk.length > 1 && !key.ctrl && !key.meta;
|
||||
const isEscape = key.escape || chunk === '\u001B';
|
||||
|
||||
// Ctrl+C → exit
|
||||
// Ctrl+C interrupts a running turn; when idle it exits the TUI.
|
||||
if (key.ctrl && chunk === 'c') {
|
||||
if (session.busy) {
|
||||
session.sendRequest({type: 'interrupt'});
|
||||
session.setBusyLabel('Stopping current operation...');
|
||||
return;
|
||||
}
|
||||
session.sendRequest({type: 'shutdown'});
|
||||
exit();
|
||||
return;
|
||||
}
|
||||
|
||||
if (!session.busy && key.ctrl && chunk === 'v') {
|
||||
attachClipboardImage();
|
||||
return;
|
||||
}
|
||||
|
||||
// Let ink-text-input handle pasted text directly.
|
||||
if (isPaste) {
|
||||
return;
|
||||
@@ -252,7 +308,7 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
|
||||
session.setModal(null);
|
||||
return;
|
||||
}
|
||||
if (chunk.toLowerCase() === 'n' || key.escape) {
|
||||
if (chunk.toLowerCase() === 'n' || isEscape) {
|
||||
session.sendRequest({
|
||||
type: 'permission_response',
|
||||
request_id: session.modal.request_id,
|
||||
@@ -264,16 +320,64 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
|
||||
return;
|
||||
}
|
||||
|
||||
// --- Edit diff modal (also appears while busy) ---
|
||||
if (session.modal?.kind === 'edit_diff') {
|
||||
if (chunk.toLowerCase() === 'y') {
|
||||
session.sendRequest({
|
||||
type: 'permission_response',
|
||||
request_id: session.modal.request_id,
|
||||
allowed: true,
|
||||
permission_reply: 'once',
|
||||
});
|
||||
session.setModal(null);
|
||||
return;
|
||||
}
|
||||
if (chunk.toLowerCase() === 'a') {
|
||||
session.sendRequest({
|
||||
type: 'permission_response',
|
||||
request_id: session.modal.request_id,
|
||||
allowed: true,
|
||||
permission_reply: 'always',
|
||||
});
|
||||
session.setModal(null);
|
||||
return;
|
||||
}
|
||||
if (chunk.toLowerCase() === 'n' || isEscape) {
|
||||
session.sendRequest({
|
||||
type: 'permission_response',
|
||||
request_id: session.modal.request_id,
|
||||
allowed: false,
|
||||
permission_reply: 'reject',
|
||||
});
|
||||
session.setModal(null);
|
||||
return;
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
// --- Question modal (also appears while busy) ---
|
||||
if (session.modal?.kind === 'question') {
|
||||
return; // Let TextInput in ModalHost handle input
|
||||
}
|
||||
|
||||
if (session.busy && isEscape) {
|
||||
session.sendRequest({type: 'interrupt'});
|
||||
session.setBusyLabel('Stopping current operation...');
|
||||
return;
|
||||
}
|
||||
|
||||
// --- Ignore input while busy ---
|
||||
if (session.busy) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Empty-input Tab opens the permission mode picker. This makes leaving
|
||||
// plan mode explicit without requiring users to remember /permissions.
|
||||
if (!showPicker && key.tab && input.trim() === '') {
|
||||
session.sendRequest({type: 'select_command', command: 'permissions'});
|
||||
return;
|
||||
}
|
||||
|
||||
// --- Command picker ---
|
||||
if (showPicker) {
|
||||
if (key.upArrow) {
|
||||
@@ -297,20 +401,25 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
|
||||
if (key.tab) {
|
||||
const selected = commandHints[pickerIndex];
|
||||
if (selected) {
|
||||
setInput(selected + ' ');
|
||||
// Complete to the selected command with no trailing space —
|
||||
// the user can hit Enter immediately to run it, or keep
|
||||
// typing to add args. The trailing space made it look like
|
||||
// Tab was "committing" with a token, which broke the flow.
|
||||
setInput(selected);
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (key.escape) {
|
||||
if (isEscape) {
|
||||
setInput('');
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
if (key.escape) {
|
||||
if (isEscape) {
|
||||
const now = Date.now();
|
||||
if (input && now - lastEscapeAt < 500) {
|
||||
if ((input || imageAttachments.length > 0) && now - lastEscapeAt < 500) {
|
||||
setInput('');
|
||||
setImageAttachments([]);
|
||||
setHistoryIndex(-1);
|
||||
setLastEscapeAt(0);
|
||||
return;
|
||||
@@ -350,20 +459,28 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
|
||||
setModalInput('');
|
||||
return;
|
||||
}
|
||||
if (!value.trim() || session.busy || !session.ready) {
|
||||
if ((!value.trim() && imageAttachments.length === 0) || session.busy || !session.ready) {
|
||||
if (session.busy && value.trim() === '/stop') {
|
||||
session.sendRequest({type: 'interrupt'});
|
||||
session.setBusyLabel('Stopping current operation...');
|
||||
setInput('');
|
||||
}
|
||||
return;
|
||||
}
|
||||
// Check if it's an interactive command
|
||||
if (handleCommand(value)) {
|
||||
if (imageAttachments.length === 0 && handleCommand(value)) {
|
||||
setHistory((items) => [...items, value]);
|
||||
setHistoryIndex(-1);
|
||||
setInput('');
|
||||
return;
|
||||
}
|
||||
session.sendRequest({type: 'submit_line', line: value});
|
||||
setHistory((items) => [...items, value]);
|
||||
session.sendRequest({type: 'submit_line', line: value, images: imagePayloads()});
|
||||
if (value.trim()) {
|
||||
setHistory((items) => [...items, value]);
|
||||
}
|
||||
setHistoryIndex(-1);
|
||||
setInput('');
|
||||
setImageAttachments([]);
|
||||
session.setBusy(true);
|
||||
};
|
||||
|
||||
@@ -448,6 +565,8 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
|
||||
toolName={session.busy ? currentToolName : undefined}
|
||||
statusLabel={session.busy ? (session.busyLabel ?? (currentToolName ? `Running ${currentToolName}...` : 'Running agent loop...')) : undefined}
|
||||
suppressSubmit={showPicker}
|
||||
imageAttachmentLabels={imageAttachments.map((image) => image.label)}
|
||||
clipboardStatus={clipboardStatus}
|
||||
/>
|
||||
)}
|
||||
|
||||
@@ -458,8 +577,10 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
|
||||
<Text color={theme.colors.primary}>shift+enter</Text> newline{' '}
|
||||
<Text color={theme.colors.primary}>enter</Text> send{' '}
|
||||
<Text color={theme.colors.primary}>/</Text> commands{' '}
|
||||
<Text color={theme.colors.primary}>tab</Text> mode{' '}
|
||||
<Text color={theme.colors.primary}>{'\u2191\u2193'}</Text> history{' '}
|
||||
<Text color={theme.colors.primary}>ctrl+c</Text> exit
|
||||
<Text color={theme.colors.primary}>{session.busy ? '/stop' : 'esc'}</Text> stop{' '}
|
||||
<Text color={theme.colors.primary}>ctrl+c</Text> {session.busy ? 'stop' : 'exit'}
|
||||
</Text>
|
||||
</Box>
|
||||
) : null}
|
||||
|
||||
@@ -0,0 +1,173 @@
|
||||
import {execFile} from 'node:child_process';
|
||||
import {mkdtemp, readFile, rm, stat} from 'node:fs/promises';
|
||||
import {tmpdir} from 'node:os';
|
||||
import {join} from 'node:path';
|
||||
|
||||
export type ImageAttachment = {
|
||||
id: string;
|
||||
label: string;
|
||||
media_type: string;
|
||||
data: string;
|
||||
source_path?: string;
|
||||
size_bytes?: number;
|
||||
};
|
||||
|
||||
const MAX_CLIPBOARD_IMAGE_BYTES = 15 * 1024 * 1024;
|
||||
const EXEC_TIMEOUT_MS = 2500;
|
||||
|
||||
type ClipboardImageRead = {
|
||||
data: Buffer;
|
||||
mediaType: string;
|
||||
label: string;
|
||||
};
|
||||
|
||||
export async function readClipboardImage(): Promise<ImageAttachment | null> {
|
||||
const image = await readClipboardImageData();
|
||||
if (!image) {
|
||||
return null;
|
||||
}
|
||||
return {
|
||||
id: `clipboard-${Date.now()}-${Math.random().toString(16).slice(2)}`,
|
||||
label: image.label,
|
||||
media_type: image.mediaType,
|
||||
data: image.data.toString('base64'),
|
||||
source_path: `clipboard:${image.label}`,
|
||||
size_bytes: image.data.length,
|
||||
};
|
||||
}
|
||||
|
||||
async function readClipboardImageData(): Promise<ClipboardImageRead | null> {
|
||||
if (process.platform === 'darwin') {
|
||||
return readMacClipboardImage();
|
||||
}
|
||||
if (process.platform === 'win32') {
|
||||
return readWindowsClipboardImage();
|
||||
}
|
||||
return readLinuxClipboardImage();
|
||||
}
|
||||
|
||||
async function readMacClipboardImage(): Promise<ClipboardImageRead | null> {
|
||||
const tempDir = await mkdtemp(join(tmpdir(), 'openharness-clipboard-'));
|
||||
try {
|
||||
const pngPath = join(tempDir, 'clipboard.png');
|
||||
if (await runFileCommand('pngpaste', [pngPath])) {
|
||||
return await readImageFile(pngPath, 'image/png', 'clipboard.png');
|
||||
}
|
||||
if (await writeMacClipboardClass('PNGf', pngPath)) {
|
||||
return await readImageFile(pngPath, 'image/png', 'clipboard.png');
|
||||
}
|
||||
|
||||
const tiffPath = join(tempDir, 'clipboard.tiff');
|
||||
if (await writeMacClipboardClass('TIFF', tiffPath)) {
|
||||
if (await runFileCommand('sips', ['-s', 'format', 'png', tiffPath, '--out', pngPath])) {
|
||||
return await readImageFile(pngPath, 'image/png', 'clipboard.png');
|
||||
}
|
||||
return await readImageFile(tiffPath, 'image/tiff', 'clipboard.tiff');
|
||||
}
|
||||
return null;
|
||||
} finally {
|
||||
await rm(tempDir, {recursive: true, force: true});
|
||||
}
|
||||
}
|
||||
|
||||
async function writeMacClipboardClass(classCode: 'PNGf' | 'TIFF', outputPath: string): Promise<boolean> {
|
||||
const appleClass = `${String.fromCharCode(0xab)}class ${classCode}${String.fromCharCode(0xbb)}`;
|
||||
const escapedPath = outputPath.replace(/\\/g, '\\\\').replace(/"/g, '\\"');
|
||||
return runFileCommand('osascript', [
|
||||
'-e',
|
||||
`set clipboardData to the clipboard as ${appleClass}`,
|
||||
'-e',
|
||||
`set outFile to open for access POSIX file "${escapedPath}" with write permission`,
|
||||
'-e',
|
||||
'set eof outFile to 0',
|
||||
'-e',
|
||||
'write clipboardData to outFile',
|
||||
'-e',
|
||||
'close access outFile',
|
||||
]);
|
||||
}
|
||||
|
||||
async function readWindowsClipboardImage(): Promise<ClipboardImageRead | null> {
|
||||
const tempDir = await mkdtemp(join(tmpdir(), 'openharness-clipboard-'));
|
||||
try {
|
||||
const pngPath = join(tempDir, 'clipboard.png');
|
||||
const escapedPath = pngPath.replace(/'/g, "''");
|
||||
const powershell = join(
|
||||
process.env.SystemRoot ?? 'C:\\Windows',
|
||||
'System32',
|
||||
'WindowsPowerShell',
|
||||
'v1.0',
|
||||
'powershell.exe',
|
||||
);
|
||||
const script = [
|
||||
'Add-Type -AssemblyName System.Windows.Forms',
|
||||
'Add-Type -AssemblyName System.Drawing',
|
||||
'if (-not [Windows.Forms.Clipboard]::ContainsImage()) { exit 2 }',
|
||||
'$image = [Windows.Forms.Clipboard]::GetImage()',
|
||||
`$image.Save('${escapedPath}', [Drawing.Imaging.ImageFormat]::Png)`,
|
||||
].join('; ');
|
||||
if (!(await runFileCommand(powershell, ['-NoProfile', '-STA', '-Command', script]))) {
|
||||
return null;
|
||||
}
|
||||
return await readImageFile(pngPath, 'image/png', 'clipboard.png');
|
||||
} finally {
|
||||
await rm(tempDir, {recursive: true, force: true});
|
||||
}
|
||||
}
|
||||
|
||||
async function readLinuxClipboardImage(): Promise<ClipboardImageRead | null> {
|
||||
const attempts: Array<[string, string[], string, string]> = [
|
||||
['wl-paste', ['--no-newline', '--type', 'image/png'], 'image/png', 'clipboard.png'],
|
||||
['wl-paste', ['--no-newline', '--type', 'image/jpeg'], 'image/jpeg', 'clipboard.jpg'],
|
||||
['xclip', ['-selection', 'clipboard', '-target', 'image/png', '-out'], 'image/png', 'clipboard.png'],
|
||||
['xclip', ['-selection', 'clipboard', '-target', 'image/jpeg', '-out'], 'image/jpeg', 'clipboard.jpg'],
|
||||
['xsel', ['--clipboard', '--output', '--mime-type', 'image/png'], 'image/png', 'clipboard.png'],
|
||||
['xsel', ['--clipboard', '--output', '--mime-type', 'image/jpeg'], 'image/jpeg', 'clipboard.jpg'],
|
||||
];
|
||||
for (const [command, args, mediaType, label] of attempts) {
|
||||
const data = await runBufferCommand(command, args);
|
||||
if (data && data.length > 0) {
|
||||
return {data, mediaType, label};
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
async function readImageFile(path: string, mediaType: string, label: string): Promise<ClipboardImageRead | null> {
|
||||
const fileStat = await stat(path).catch(() => null);
|
||||
if (!fileStat || fileStat.size <= 0 || fileStat.size > MAX_CLIPBOARD_IMAGE_BYTES) {
|
||||
return null;
|
||||
}
|
||||
const data = await readFile(path);
|
||||
return {data, mediaType, label};
|
||||
}
|
||||
|
||||
async function runFileCommand(command: string, args: string[]): Promise<boolean> {
|
||||
return new Promise((resolve) => {
|
||||
execFile(command, args, {timeout: EXEC_TIMEOUT_MS, windowsHide: true}, (error) => {
|
||||
resolve(!error);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
async function runBufferCommand(command: string, args: string[]): Promise<Buffer | null> {
|
||||
return new Promise((resolve) => {
|
||||
execFile(
|
||||
command,
|
||||
args,
|
||||
{
|
||||
encoding: 'buffer',
|
||||
maxBuffer: MAX_CLIPBOARD_IMAGE_BYTES + 1024,
|
||||
timeout: EXEC_TIMEOUT_MS,
|
||||
windowsHide: true,
|
||||
},
|
||||
(error, stdout) => {
|
||||
if (error || !Buffer.isBuffer(stdout) || stdout.length > MAX_CLIPBOARD_IMAGE_BYTES) {
|
||||
resolve(null);
|
||||
return;
|
||||
}
|
||||
resolve(stdout);
|
||||
},
|
||||
);
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import {PassThrough} from 'node:stream';
|
||||
import React from 'react';
|
||||
import {render} from 'ink';
|
||||
|
||||
import {ModalHost} from './ModalHost.js';
|
||||
|
||||
const stripAnsi = (value: string): string => value.replace(/\u001B\[[0-9;?]*[ -/]*[@-~]/g, '');
|
||||
const nextLoopTurn = (): Promise<void> => new Promise((resolve) => setImmediate(resolve));
|
||||
|
||||
type InkTestStdout = PassThrough & {
|
||||
isTTY: boolean;
|
||||
columns: number;
|
||||
rows: number;
|
||||
cursorTo: () => boolean;
|
||||
clearLine: () => boolean;
|
||||
moveCursor: () => boolean;
|
||||
};
|
||||
|
||||
function createTestStdout(): InkTestStdout {
|
||||
return Object.assign(new PassThrough(), {
|
||||
isTTY: true,
|
||||
columns: 120,
|
||||
rows: 40,
|
||||
cursorTo: () => true,
|
||||
clearLine: () => true,
|
||||
moveCursor: () => true,
|
||||
});
|
||||
}
|
||||
|
||||
async function waitForOutputToStabilize(getOutput: () => string): Promise<string> {
|
||||
let previous = '';
|
||||
let sawOutput = false;
|
||||
|
||||
for (let i = 0; i < 50; i += 1) {
|
||||
await nextLoopTurn();
|
||||
const current = getOutput();
|
||||
sawOutput ||= current.length > 0;
|
||||
if (sawOutput && current === previous) {
|
||||
return current;
|
||||
}
|
||||
previous = current;
|
||||
}
|
||||
|
||||
throw new Error(`Ink output did not stabilize: ${JSON.stringify(previous)}`);
|
||||
}
|
||||
|
||||
test('renders edit diff preview with stats and always shortcut', async () => {
|
||||
const stdout = createTestStdout();
|
||||
let output = '';
|
||||
|
||||
stdout.on('data', (chunk) => {
|
||||
output += chunk.toString();
|
||||
});
|
||||
|
||||
const instance = render(
|
||||
<ModalHost
|
||||
modal={{
|
||||
kind: 'edit_diff',
|
||||
path: 'src/demo.txt',
|
||||
diff: '@@ -1 +1 @@\n-old line\n+new line',
|
||||
added: 1,
|
||||
removed: 1,
|
||||
}}
|
||||
modalInput=""
|
||||
setModalInput={() => undefined}
|
||||
onSubmit={() => undefined}
|
||||
/>,
|
||||
{stdout: stdout as unknown as NodeJS.WriteStream, debug: true, patchConsole: false},
|
||||
);
|
||||
|
||||
const exitPromise = instance.waitUntilExit();
|
||||
const stableOutput = await waitForOutputToStabilize(() => output);
|
||||
instance.unmount();
|
||||
await exitPromise;
|
||||
instance.cleanup();
|
||||
stdout.destroy();
|
||||
|
||||
const rendered = stripAnsi(stableOutput);
|
||||
assert.match(rendered, /Edit src\/demo\.txt/);
|
||||
assert.match(rendered, /\+1/);
|
||||
assert.match(rendered, /-1/);
|
||||
assert.match(rendered, /\+new line/);
|
||||
assert.match(rendered, /-old line/);
|
||||
assert.match(rendered, /\[a\] Always/);
|
||||
});
|
||||
@@ -8,6 +8,7 @@ const WAIT_FRAMES = [
|
||||
'Agent is waiting for your input.. ',
|
||||
'Agent is waiting for your input...',
|
||||
];
|
||||
const MAX_DIFF_LINES = 40;
|
||||
|
||||
function WaitingAnimation(): React.JSX.Element {
|
||||
const [frame, setFrame] = useState(0);
|
||||
@@ -85,6 +86,104 @@ function QuestionModal({
|
||||
);
|
||||
}
|
||||
|
||||
type DiffLineKind = 'add' | 'del' | 'hunk' | 'context';
|
||||
|
||||
type ParsedDiffLine = {
|
||||
kind: DiffLineKind;
|
||||
content: string;
|
||||
};
|
||||
|
||||
function parseDiffLines(diffText: string): ParsedDiffLine[] {
|
||||
return diffText
|
||||
.split('\n')
|
||||
.flatMap((raw): ParsedDiffLine[] => {
|
||||
if (!raw || raw.startsWith('+++') || raw.startsWith('---')) {
|
||||
return [];
|
||||
}
|
||||
if (raw.startsWith('@@')) {
|
||||
return [{kind: 'hunk', content: raw}];
|
||||
}
|
||||
if (raw.startsWith('+')) {
|
||||
return [{kind: 'add', content: raw.slice(1)}];
|
||||
}
|
||||
if (raw.startsWith('-')) {
|
||||
return [{kind: 'del', content: raw.slice(1)}];
|
||||
}
|
||||
return [{kind: 'context', content: raw.startsWith(' ') ? raw.slice(1) : raw}];
|
||||
});
|
||||
}
|
||||
|
||||
function EditDiffModal({modal}: {modal: Record<string, unknown>}): React.JSX.Element {
|
||||
const path = String(modal.path ?? '');
|
||||
const added = Number(modal.added ?? 0);
|
||||
const removed = Number(modal.removed ?? 0);
|
||||
const lines = parseDiffLines(String(modal.diff ?? ''));
|
||||
const visibleLines = lines.slice(0, MAX_DIFF_LINES);
|
||||
const hiddenCount = lines.length - visibleLines.length;
|
||||
|
||||
return (
|
||||
<Box flexDirection="column" marginTop={1}>
|
||||
<Text>
|
||||
<Text color="yellow" bold>{'\u250C '}</Text>
|
||||
<Text bold>Edit </Text>
|
||||
<Text color="cyan" bold>{path}</Text>
|
||||
<Text>{' '}</Text>
|
||||
<Text color="green">{`+${added}`}</Text>
|
||||
<Text>{' '}</Text>
|
||||
<Text color="red">{`-${removed}`}</Text>
|
||||
</Text>
|
||||
{visibleLines.map((line, index) => {
|
||||
if (line.kind === 'hunk') {
|
||||
return (
|
||||
<Text key={index}>
|
||||
<Text color="yellow">{'\u2502 '}</Text>
|
||||
<Text color="cyan" dimColor>{line.content}</Text>
|
||||
</Text>
|
||||
);
|
||||
}
|
||||
if (line.kind === 'add') {
|
||||
return (
|
||||
<Text key={index}>
|
||||
<Text color="yellow">{'\u2502 '}</Text>
|
||||
<Text color="green">+</Text>
|
||||
<Text color="green">{line.content}</Text>
|
||||
</Text>
|
||||
);
|
||||
}
|
||||
if (line.kind === 'del') {
|
||||
return (
|
||||
<Text key={index}>
|
||||
<Text color="yellow">{'\u2502 '}</Text>
|
||||
<Text color="red">-</Text>
|
||||
<Text color="red">{line.content}</Text>
|
||||
</Text>
|
||||
);
|
||||
}
|
||||
return (
|
||||
<Text key={index}>
|
||||
<Text color="yellow">{'\u2502 '}</Text>
|
||||
<Text dimColor>{' '}{line.content}</Text>
|
||||
</Text>
|
||||
);
|
||||
})}
|
||||
{hiddenCount > 0 ? (
|
||||
<Text>
|
||||
<Text color="yellow">{'\u2502 '}</Text>
|
||||
<Text dimColor>... {hiddenCount} more lines hidden</Text>
|
||||
</Text>
|
||||
) : null}
|
||||
<Text>
|
||||
<Text color="yellow">{'\u2514 '}</Text>
|
||||
<Text color="green">[y] Once</Text>
|
||||
<Text>{' '}</Text>
|
||||
<Text color="green">[a] Always</Text>
|
||||
<Text>{' '}</Text>
|
||||
<Text color="red">[n] Deny</Text>
|
||||
</Text>
|
||||
</Box>
|
||||
);
|
||||
}
|
||||
|
||||
function ModalHostInner({
|
||||
modal,
|
||||
modalInput,
|
||||
@@ -120,6 +219,9 @@ function ModalHostInner({
|
||||
</Box>
|
||||
);
|
||||
}
|
||||
if (modal?.kind === 'edit_diff') {
|
||||
return <EditDiffModal modal={modal} />;
|
||||
}
|
||||
if (modal?.kind === 'question') {
|
||||
return (
|
||||
<QuestionModal
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
|
||||
import {getBackspaceDeleteCount} from './PromptInput.js';
|
||||
|
||||
test('counts repeated backspace control characters from a single input chunk', () => {
|
||||
assert.equal(getBackspaceDeleteCount('\b\b\b'), 3);
|
||||
assert.equal(getBackspaceDeleteCount('\u007f\u007f'), 2);
|
||||
assert.equal(getBackspaceDeleteCount('\x7f\x7f\x7f'), 3);
|
||||
});
|
||||
|
||||
test('falls back to a single delete for empty or unexpected input', () => {
|
||||
assert.equal(getBackspaceDeleteCount(''), 1);
|
||||
assert.equal(getBackspaceDeleteCount('abc'), 1);
|
||||
assert.equal(getBackspaceDeleteCount('\bA'), 1);
|
||||
});
|
||||
@@ -0,0 +1,244 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import {PassThrough} from 'node:stream';
|
||||
import React, {useState} from 'react';
|
||||
import {render} from 'ink';
|
||||
|
||||
import {ThemeProvider} from '../theme/ThemeContext.js';
|
||||
import {PromptInput} from './PromptInput.js';
|
||||
|
||||
const nextLoopTurn = (): Promise<void> => new Promise((resolve) => setImmediate(resolve));
|
||||
|
||||
type InkTestStdout = PassThrough & {
|
||||
isTTY: boolean;
|
||||
columns: number;
|
||||
rows: number;
|
||||
cursorTo: () => boolean;
|
||||
clearLine: () => boolean;
|
||||
moveCursor: () => boolean;
|
||||
};
|
||||
|
||||
type InkTestStdin = PassThrough & {
|
||||
isTTY: boolean;
|
||||
setRawMode: (_mode: boolean) => void;
|
||||
resume: () => InkTestStdin;
|
||||
pause: () => InkTestStdin;
|
||||
ref: () => InkTestStdin;
|
||||
unref: () => InkTestStdin;
|
||||
};
|
||||
|
||||
function createTestStdout(): InkTestStdout {
|
||||
return Object.assign(new PassThrough(), {
|
||||
isTTY: true,
|
||||
columns: 120,
|
||||
rows: 40,
|
||||
cursorTo: () => true,
|
||||
clearLine: () => true,
|
||||
moveCursor: () => true,
|
||||
});
|
||||
}
|
||||
|
||||
function createTestStdin(): InkTestStdin {
|
||||
return Object.assign(new PassThrough(), {
|
||||
isTTY: true,
|
||||
setRawMode: () => undefined,
|
||||
resume() {
|
||||
return this;
|
||||
},
|
||||
pause() {
|
||||
return this;
|
||||
},
|
||||
ref() {
|
||||
return this;
|
||||
},
|
||||
unref() {
|
||||
return this;
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
async function sendKey(stdin: InkTestStdin, chunk: string | Buffer): Promise<void> {
|
||||
stdin.write(chunk);
|
||||
await nextLoopTurn();
|
||||
await nextLoopTurn();
|
||||
}
|
||||
|
||||
async function waitForValue(getValue: () => string, expected: string): Promise<void> {
|
||||
for (let i = 0; i < 50; i += 1) {
|
||||
await nextLoopTurn();
|
||||
if (getValue() === expected) {
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
assert.equal(getValue(), expected);
|
||||
}
|
||||
|
||||
function PromptHarness({
|
||||
busy = false,
|
||||
onInputChange,
|
||||
onSubmit = () => undefined,
|
||||
}: {
|
||||
busy?: boolean;
|
||||
onInputChange: (value: string) => void;
|
||||
onSubmit?: (value: string) => void;
|
||||
}): React.JSX.Element {
|
||||
const [input, setInput] = useState('');
|
||||
|
||||
return (
|
||||
<ThemeProvider initialTheme="default">
|
||||
<PromptInput
|
||||
busy={busy}
|
||||
input={input}
|
||||
setInput={(value) => {
|
||||
onInputChange(value);
|
||||
setInput(value);
|
||||
}}
|
||||
onSubmit={onSubmit}
|
||||
/>
|
||||
</ThemeProvider>
|
||||
);
|
||||
}
|
||||
|
||||
test('treats terminal DEL at end-of-line as backward delete', async () => {
|
||||
const stdin = createTestStdin();
|
||||
const stdout = createTestStdout();
|
||||
let currentValue = '';
|
||||
|
||||
const instance = render(<PromptHarness onInputChange={(value) => {
|
||||
currentValue = value;
|
||||
}} />, {
|
||||
stdin: stdin as unknown as NodeJS.ReadStream & {fd: 0},
|
||||
stdout: stdout as unknown as NodeJS.WriteStream,
|
||||
debug: true,
|
||||
patchConsole: false,
|
||||
});
|
||||
const exitPromise = instance.waitUntilExit();
|
||||
|
||||
try {
|
||||
await nextLoopTurn();
|
||||
|
||||
await sendKey(stdin, 'a');
|
||||
await waitForValue(() => currentValue, 'a');
|
||||
|
||||
await sendKey(stdin, 'b');
|
||||
await waitForValue(() => currentValue, 'ab');
|
||||
|
||||
await sendKey(stdin, Buffer.from([0x7f]));
|
||||
await waitForValue(() => currentValue, 'a');
|
||||
} finally {
|
||||
instance.unmount();
|
||||
await exitPromise;
|
||||
instance.cleanup();
|
||||
stdin.destroy();
|
||||
stdout.destroy();
|
||||
}
|
||||
});
|
||||
|
||||
test('keeps forward delete behavior when cursor is inside the line', async () => {
|
||||
const stdin = createTestStdin();
|
||||
const stdout = createTestStdout();
|
||||
let currentValue = '';
|
||||
|
||||
const instance = render(<PromptHarness onInputChange={(value) => {
|
||||
currentValue = value;
|
||||
}} />, {
|
||||
stdin: stdin as unknown as NodeJS.ReadStream & {fd: 0},
|
||||
stdout: stdout as unknown as NodeJS.WriteStream,
|
||||
debug: true,
|
||||
patchConsole: false,
|
||||
});
|
||||
const exitPromise = instance.waitUntilExit();
|
||||
|
||||
try {
|
||||
await nextLoopTurn();
|
||||
|
||||
await sendKey(stdin, 'a');
|
||||
await waitForValue(() => currentValue, 'a');
|
||||
|
||||
await sendKey(stdin, 'b');
|
||||
await waitForValue(() => currentValue, 'ab');
|
||||
|
||||
await sendKey(stdin, '\u001B[D');
|
||||
await nextLoopTurn();
|
||||
|
||||
await sendKey(stdin, '\u001B[3~');
|
||||
await waitForValue(() => currentValue, 'a');
|
||||
} finally {
|
||||
instance.unmount();
|
||||
await exitPromise;
|
||||
instance.cleanup();
|
||||
stdin.destroy();
|
||||
stdout.destroy();
|
||||
}
|
||||
});
|
||||
|
||||
test('ignores ctrl+v so the app can attach clipboard images', async () => {
|
||||
const stdin = createTestStdin();
|
||||
const stdout = createTestStdout();
|
||||
let currentValue = '';
|
||||
|
||||
const instance = render(<PromptHarness onInputChange={(value) => {
|
||||
currentValue = value;
|
||||
}} />, {
|
||||
stdin: stdin as unknown as NodeJS.ReadStream & {fd: 0},
|
||||
stdout: stdout as unknown as NodeJS.WriteStream,
|
||||
debug: true,
|
||||
patchConsole: false,
|
||||
});
|
||||
const exitPromise = instance.waitUntilExit();
|
||||
|
||||
try {
|
||||
await nextLoopTurn();
|
||||
|
||||
await sendKey(stdin, Buffer.from([0x16]));
|
||||
await nextLoopTurn();
|
||||
assert.equal(currentValue, '');
|
||||
|
||||
await sendKey(stdin, 'v');
|
||||
await waitForValue(() => currentValue, 'v');
|
||||
} finally {
|
||||
instance.unmount();
|
||||
await exitPromise;
|
||||
instance.cleanup();
|
||||
stdin.destroy();
|
||||
stdout.destroy();
|
||||
}
|
||||
});
|
||||
|
||||
test('accepts explicit /stop submission while busy', async () => {
|
||||
const stdin = createTestStdin();
|
||||
const stdout = createTestStdout();
|
||||
let currentValue = '';
|
||||
let submitted = '';
|
||||
|
||||
const instance = render(<PromptHarness busy onInputChange={(value) => {
|
||||
currentValue = value;
|
||||
}} onSubmit={(value) => {
|
||||
submitted = value;
|
||||
}} />, {
|
||||
stdin: stdin as unknown as NodeJS.ReadStream & {fd: 0},
|
||||
stdout: stdout as unknown as NodeJS.WriteStream,
|
||||
debug: true,
|
||||
patchConsole: false,
|
||||
});
|
||||
const exitPromise = instance.waitUntilExit();
|
||||
|
||||
try {
|
||||
await nextLoopTurn();
|
||||
|
||||
for (const char of '/stop') {
|
||||
await sendKey(stdin, char);
|
||||
}
|
||||
await waitForValue(() => currentValue, '/stop');
|
||||
|
||||
await sendKey(stdin, '\r');
|
||||
await waitForValue(() => submitted, '/stop');
|
||||
} finally {
|
||||
instance.unmount();
|
||||
await exitPromise;
|
||||
instance.cleanup();
|
||||
stdin.destroy();
|
||||
stdout.destroy();
|
||||
}
|
||||
});
|
||||
@@ -1,11 +1,20 @@
|
||||
import React, {useEffect, useState} from 'react';
|
||||
import {Box, Text, useInput} from 'ink';
|
||||
import React, {useEffect, useRef, useState} from 'react';
|
||||
import {Box, Text, useInput, useStdin} from 'ink';
|
||||
import chalk from 'chalk';
|
||||
|
||||
import {useTheme} from '../theme/ThemeContext.js';
|
||||
import {Spinner} from './Spinner.js';
|
||||
|
||||
const noop = (): void => {};
|
||||
const BACKSPACE_CONTROL_PATTERN = /^[\b\u007f]+$/;
|
||||
|
||||
export function getBackspaceDeleteCount(sequence: string): number {
|
||||
if (!sequence || !BACKSPACE_CONTROL_PATTERN.test(sequence)) {
|
||||
return 1;
|
||||
}
|
||||
|
||||
return [...sequence].length;
|
||||
}
|
||||
|
||||
function MultilineTextInput({
|
||||
value,
|
||||
@@ -23,18 +32,60 @@ function MultilineTextInput({
|
||||
promptColor: string;
|
||||
}): React.JSX.Element {
|
||||
const [cursorOffset, setCursorOffset] = useState(value.length);
|
||||
const {internal_eventEmitter} = useStdin();
|
||||
const lastSequenceRef = useRef('');
|
||||
// Tracks the last value this component produced via onChange. If the
|
||||
// incoming `value` prop diverges from this, the change came from outside
|
||||
// (tab completion, history recall, programmatic clear) and we should
|
||||
// move the cursor to the end — otherwise the cursor stays wherever the
|
||||
// user had it, which puts subsequent keystrokes in the middle of the
|
||||
// newly-completed text. See HKUDS/OpenHarness#183.
|
||||
const lastInternalValueRef = useRef<string>(value);
|
||||
|
||||
useEffect(() => {
|
||||
setCursorOffset((previous) => Math.min(previous, value.length));
|
||||
if (value === lastInternalValueRef.current) {
|
||||
// Self-authored update; cursor was already positioned by the
|
||||
// handler that called onChange.
|
||||
return;
|
||||
}
|
||||
lastInternalValueRef.current = value;
|
||||
setCursorOffset(value.length);
|
||||
}, [value]);
|
||||
|
||||
const commitValue = (nextValue: string): void => {
|
||||
lastInternalValueRef.current = nextValue;
|
||||
onChange(nextValue);
|
||||
};
|
||||
|
||||
useEffect(() => {
|
||||
if (!focus) {
|
||||
return;
|
||||
}
|
||||
|
||||
const handleRawInput = (chunk: string | Buffer): void => {
|
||||
lastSequenceRef.current = Buffer.isBuffer(chunk) ? chunk.toString('utf8') : String(chunk);
|
||||
};
|
||||
|
||||
internal_eventEmitter.on('input', handleRawInput);
|
||||
return () => {
|
||||
internal_eventEmitter.removeListener('input', handleRawInput);
|
||||
};
|
||||
}, [focus, internal_eventEmitter]);
|
||||
|
||||
useInput(
|
||||
(input, key) => {
|
||||
if (!focus) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (key.upArrow || key.downArrow || key.tab || (key.shift && key.tab) || key.escape || (key.ctrl && input === 'c')) {
|
||||
if (
|
||||
key.upArrow ||
|
||||
key.downArrow ||
|
||||
key.tab ||
|
||||
(key.shift && key.tab) ||
|
||||
key.escape ||
|
||||
(key.ctrl && (input === 'c' || input === 'v'))
|
||||
) {
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -42,7 +93,7 @@ function MultilineTextInput({
|
||||
if (key.shift) {
|
||||
const nextValue = value.slice(0, cursorOffset) + '\n' + value.slice(cursorOffset);
|
||||
setCursorOffset(cursorOffset + 1);
|
||||
onChange(nextValue);
|
||||
commitValue(nextValue);
|
||||
return;
|
||||
}
|
||||
onSubmit?.(value);
|
||||
@@ -59,22 +110,41 @@ function MultilineTextInput({
|
||||
return;
|
||||
}
|
||||
|
||||
if (key.backspace || key.delete) {
|
||||
if (key.delete) {
|
||||
if (cursorOffset >= value.length) {
|
||||
return;
|
||||
}
|
||||
const nextValue = value.slice(0, cursorOffset) + value.slice(cursorOffset + 1);
|
||||
onChange(nextValue);
|
||||
return;
|
||||
}
|
||||
|
||||
if (key.backspace) {
|
||||
if (cursorOffset === 0) {
|
||||
return;
|
||||
}
|
||||
const nextValue = value.slice(0, cursorOffset - 1) + value.slice(cursorOffset);
|
||||
setCursorOffset(cursorOffset - 1);
|
||||
onChange(nextValue);
|
||||
const deleteCount = Math.min(cursorOffset, getBackspaceDeleteCount(lastSequenceRef.current || input));
|
||||
const nextValue = value.slice(0, cursorOffset - deleteCount) + value.slice(cursorOffset);
|
||||
setCursorOffset(cursorOffset - deleteCount);
|
||||
commitValue(nextValue);
|
||||
return;
|
||||
}
|
||||
|
||||
if (key.delete) {
|
||||
// Ink reports the common DEL byte (`0x7f`) as `delete`, even though
|
||||
// many terminals emit it for the Backspace key. Use the raw sequence
|
||||
// to distinguish that case from a true forward-delete escape sequence.
|
||||
if (
|
||||
lastSequenceRef.current === '\x7f' ||
|
||||
lastSequenceRef.current === '\x1b\x7f' ||
|
||||
BACKSPACE_CONTROL_PATTERN.test(lastSequenceRef.current)
|
||||
) {
|
||||
if (cursorOffset === 0) {
|
||||
return;
|
||||
}
|
||||
const deleteCount = Math.min(cursorOffset, getBackspaceDeleteCount(lastSequenceRef.current));
|
||||
const nextValue = value.slice(0, cursorOffset - deleteCount) + value.slice(cursorOffset);
|
||||
setCursorOffset(cursorOffset - deleteCount);
|
||||
commitValue(nextValue);
|
||||
return;
|
||||
}
|
||||
|
||||
if (cursorOffset >= value.length) {
|
||||
return;
|
||||
}
|
||||
const nextValue = value.slice(0, cursorOffset) + value.slice(cursorOffset + 1);
|
||||
commitValue(nextValue);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -84,7 +154,7 @@ function MultilineTextInput({
|
||||
|
||||
const nextValue = value.slice(0, cursorOffset) + input + value.slice(cursorOffset);
|
||||
setCursorOffset(cursorOffset + input.length);
|
||||
onChange(nextValue);
|
||||
commitValue(nextValue);
|
||||
},
|
||||
{isActive: focus},
|
||||
);
|
||||
@@ -134,6 +204,8 @@ export function PromptInput({
|
||||
toolName,
|
||||
suppressSubmit,
|
||||
statusLabel,
|
||||
imageAttachmentLabels = [],
|
||||
clipboardStatus,
|
||||
}: {
|
||||
busy: boolean;
|
||||
input: string;
|
||||
@@ -142,6 +214,8 @@ export function PromptInput({
|
||||
toolName?: string;
|
||||
suppressSubmit?: boolean;
|
||||
statusLabel?: string;
|
||||
imageAttachmentLabels?: string[];
|
||||
clipboardStatus?: string | null;
|
||||
}): React.JSX.Element {
|
||||
const {theme} = useTheme();
|
||||
const promptPrefix = busy ? '… ' : '> ';
|
||||
@@ -155,11 +229,23 @@ export function PromptInput({
|
||||
</Box>
|
||||
</Box>
|
||||
) : null}
|
||||
{imageAttachmentLabels.length > 0 ? (
|
||||
<Box>
|
||||
<Text color={theme.colors.accent}>
|
||||
{imageAttachmentLabels.map((label, index) => `[image ${index + 1}: ${label}]`).join(' ')}
|
||||
</Text>
|
||||
</Box>
|
||||
) : null}
|
||||
{clipboardStatus ? (
|
||||
<Box>
|
||||
<Text color={theme.colors.muted}>{clipboardStatus}</Text>
|
||||
</Box>
|
||||
) : null}
|
||||
<MultilineTextInput
|
||||
value={input}
|
||||
onChange={setInput}
|
||||
onSubmit={suppressSubmit || busy ? noop : onSubmit}
|
||||
focus={!busy}
|
||||
onSubmit={suppressSubmit ? noop : onSubmit}
|
||||
focus
|
||||
promptPrefix={promptPrefix}
|
||||
promptColor={theme.colors.primary}
|
||||
/>
|
||||
|
||||
@@ -454,6 +454,7 @@ export function useBackendSession(config: FrontendConfig, onExit: (code?: number
|
||||
setModal,
|
||||
setSelectRequest,
|
||||
setBusy,
|
||||
setBusyLabel,
|
||||
sendRequest,
|
||||
}),
|
||||
[assistantBuffer, bridgeSessions, busy, busyLabel, commands, mcpServers, modal, ready, selectRequest, status, swarmNotifications, swarmTeammates, tasks, todoMarkdown, transcript]
|
||||
|
||||
@@ -11,6 +11,12 @@ export type TranscriptItem = {
|
||||
is_error?: boolean;
|
||||
};
|
||||
|
||||
export type ImageAttachmentPayload = {
|
||||
media_type: string;
|
||||
data: string;
|
||||
source_path?: string;
|
||||
};
|
||||
|
||||
export type TaskSnapshot = {
|
||||
id: string;
|
||||
type: string;
|
||||
|
||||
+1
-1
@@ -2,4 +2,4 @@
|
||||
|
||||
__all__ = ["__version__"]
|
||||
|
||||
__version__ = "0.1.7"
|
||||
__version__ = "0.1.9"
|
||||
|
||||
+60
-2
@@ -25,6 +25,7 @@ from ohmo.runtime import launch_ohmo_react_tui, run_ohmo_backend, run_ohmo_print
|
||||
from ohmo.session_storage import OhmoSessionBackend
|
||||
from ohmo.workspace import (
|
||||
get_gateway_config_path,
|
||||
get_logs_dir,
|
||||
get_workspace_root,
|
||||
get_soul_path,
|
||||
get_state_path,
|
||||
@@ -269,6 +270,14 @@ def _prompt_channels(existing: GatewayConfig) -> tuple[list[str], dict[str, dict
|
||||
default_value=str(prior.get("group_policy", "mention")),
|
||||
)
|
||||
elif channel == "feishu":
|
||||
config["domain"] = _select_from_menu(
|
||||
"Feishu domain:",
|
||||
[
|
||||
("https://open.feishu.cn", "Feishu (China)"),
|
||||
("https://open.larksuite.com", "Lark (International)"),
|
||||
],
|
||||
default_value=str(prior.get("domain", "https://open.feishu.cn")),
|
||||
)
|
||||
config["app_id"] = _text_prompt(
|
||||
"Feishu app id",
|
||||
default=str(prior.get("app_id", "")),
|
||||
@@ -289,6 +298,29 @@ def _prompt_channels(existing: GatewayConfig) -> tuple[list[str], dict[str, dict
|
||||
"Feishu reaction emoji",
|
||||
default=str(prior.get("react_emoji", "OK")),
|
||||
)
|
||||
config["group_policy"] = _select_from_menu(
|
||||
"Feishu group policy:",
|
||||
[
|
||||
("managed_or_mention", "Managed groups open; other groups require @mention"),
|
||||
("mention", "Always require @mention in groups"),
|
||||
("open", "Always reply to group messages"),
|
||||
],
|
||||
default_value=str(prior.get("group_policy", "managed_or_mention")),
|
||||
)
|
||||
prior_bot_names = prior.get("bot_names", ["ohmo", "openclaw", "openharness"])
|
||||
if isinstance(prior_bot_names, str):
|
||||
prior_bot_names_default = prior_bot_names
|
||||
else:
|
||||
prior_bot_names_default = ",".join(str(item) for item in prior_bot_names)
|
||||
bot_names_raw = _text_prompt(
|
||||
"Feishu bot mention names (comma separated)",
|
||||
default=prior_bot_names_default,
|
||||
)
|
||||
config["bot_names"] = [item.strip() for item in bot_names_raw.split(",") if item.strip()]
|
||||
config["bot_open_id"] = _text_prompt(
|
||||
"Feishu bot open_id for exact mention detection (optional)",
|
||||
default=str(prior.get("bot_open_id", "")),
|
||||
)
|
||||
configs[channel] = config
|
||||
return enabled, configs
|
||||
|
||||
@@ -376,18 +408,42 @@ def _maybe_restart_gateway(*, cwd: str | Path, workspace: str | Path) -> None:
|
||||
print(f"ohmo gateway restarted (pid={pid})")
|
||||
|
||||
|
||||
def _configure_gateway_logging(workspace: str | Path | None = None) -> None:
|
||||
def _configure_gateway_logging(
|
||||
workspace: str | Path | None = None,
|
||||
*,
|
||||
console: bool = True,
|
||||
log_file: bool = True,
|
||||
) -> None:
|
||||
"""Configure foreground gateway logging."""
|
||||
config = load_gateway_config(workspace)
|
||||
level_name = str(config.log_level or "INFO").upper()
|
||||
level = getattr(logging, level_name, logging.INFO)
|
||||
handlers = _build_gateway_logging_handlers(workspace, console=console, log_file=log_file)
|
||||
logging.basicConfig(
|
||||
level=level,
|
||||
format="%(asctime)s [%(name)s] %(levelname)s %(message)s",
|
||||
handlers=handlers,
|
||||
force=True,
|
||||
)
|
||||
|
||||
|
||||
def _build_gateway_logging_handlers(
|
||||
workspace: str | Path | None = None,
|
||||
*,
|
||||
console: bool,
|
||||
log_file: bool,
|
||||
) -> list[logging.Handler]:
|
||||
"""Build gateway log handlers for foreground and daemon modes."""
|
||||
handlers: list[logging.Handler] = []
|
||||
if console:
|
||||
handlers.append(logging.StreamHandler())
|
||||
if log_file:
|
||||
log_path = get_logs_dir(workspace) / "gateway.log"
|
||||
log_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
handlers.append(logging.FileHandler(log_path, encoding="utf-8", delay=True))
|
||||
return handlers
|
||||
|
||||
|
||||
@app.callback(invoke_without_command=True)
|
||||
def main(
|
||||
ctx: typer.Context,
|
||||
@@ -606,9 +662,11 @@ def user_edit_cmd(
|
||||
def gateway_run_cmd(
|
||||
cwd: str = typer.Option(str(Path.cwd()), "--cwd", help="Project working directory"),
|
||||
workspace: str | None = typer.Option(None, "--workspace", help=_WORKSPACE_HELP),
|
||||
console_log: bool = typer.Option(True, "--console-log/--no-console-log", hidden=True),
|
||||
log_file: bool = typer.Option(True, "--log-file/--no-log-file", hidden=True),
|
||||
) -> None:
|
||||
"""Run the ohmo gateway in the foreground."""
|
||||
_configure_gateway_logging(workspace)
|
||||
_configure_gateway_logging(workspace, console=console_log, log_file=log_file)
|
||||
service = OhmoGatewayService(cwd, workspace)
|
||||
raise SystemExit(asyncio.run(service.run_foreground()))
|
||||
|
||||
|
||||
+170
-3
@@ -5,10 +5,13 @@ from __future__ import annotations
|
||||
import asyncio
|
||||
import logging
|
||||
from collections.abc import Awaitable, Callable
|
||||
from pathlib import Path
|
||||
|
||||
from openharness.channels.bus.events import InboundMessage
|
||||
from openharness.channels.bus.events import OutboundMessage
|
||||
from openharness.channels.bus.queue import MessageBus
|
||||
|
||||
from ohmo.group_registry import load_managed_group_record
|
||||
from ohmo.gateway.router import session_key_for_message
|
||||
from ohmo.gateway.runtime import OhmoSessionRuntimePool
|
||||
|
||||
@@ -59,10 +62,14 @@ class OhmoGatewayBridge:
|
||||
bus: MessageBus,
|
||||
runtime_pool: OhmoSessionRuntimePool,
|
||||
restart_gateway: Callable[[object, str], Awaitable[None] | None] | None = None,
|
||||
workspace: str | Path | None = None,
|
||||
feishu_group_policy: str = "open",
|
||||
) -> None:
|
||||
self._bus = bus
|
||||
self._runtime_pool = runtime_pool
|
||||
self._restart_gateway = restart_gateway
|
||||
self._workspace = workspace
|
||||
self._feishu_group_policy = _normalize_feishu_group_policy(feishu_group_policy)
|
||||
self._running = False
|
||||
self._session_tasks: dict[str, asyncio.Task[None]] = {}
|
||||
self._session_cancel_reasons: dict[str, str] = {}
|
||||
@@ -77,6 +84,17 @@ class OhmoGatewayBridge:
|
||||
except asyncio.CancelledError:
|
||||
break
|
||||
|
||||
if not self._should_process_message(message):
|
||||
logger.info(
|
||||
"ohmo inbound ignored channel=%s chat_id=%s sender_id=%s reason=feishu_group_policy policy=%s content=%r",
|
||||
message.channel,
|
||||
message.chat_id,
|
||||
message.sender_id,
|
||||
self._feishu_group_policy,
|
||||
_content_snippet(message.content),
|
||||
)
|
||||
continue
|
||||
|
||||
session_key = session_key_for_message(message)
|
||||
logger.info(
|
||||
"ohmo inbound received channel=%s chat_id=%s sender_id=%s session_key=%s content=%r",
|
||||
@@ -92,6 +110,13 @@ class OhmoGatewayBridge:
|
||||
if message.content.strip() == "/restart":
|
||||
await self._handle_restart(message, session_key)
|
||||
continue
|
||||
group_args = _parse_group_command(message.content)
|
||||
if group_args is not None:
|
||||
prepared = await self._prepare_group_prompt_message(message, session_key, group_args)
|
||||
if prepared is None:
|
||||
continue
|
||||
message = prepared
|
||||
session_key = session_key_for_message(message)
|
||||
await self._interrupt_session(
|
||||
session_key,
|
||||
reason="replaced by a newer user message",
|
||||
@@ -148,6 +173,58 @@ class OhmoGatewayBridge:
|
||||
if asyncio.iscoroutine(result):
|
||||
await result
|
||||
|
||||
async def _prepare_group_prompt_message(
|
||||
self,
|
||||
message,
|
||||
session_key: str,
|
||||
args: str,
|
||||
) -> InboundMessage | None:
|
||||
"""Convert a private /group command into an agent task."""
|
||||
if message.channel != "feishu":
|
||||
await self._publish_command_reply(
|
||||
message,
|
||||
session_key,
|
||||
"/group 当前只支持飞书。\n/group is currently only available for Feishu.",
|
||||
)
|
||||
return None
|
||||
|
||||
chat_type = str(message.metadata.get("chat_type") or "").strip().lower()
|
||||
is_private = chat_type in {"p2p", "private", "im", "direct"} or (
|
||||
not chat_type and str(message.chat_id) == str(message.sender_id)
|
||||
)
|
||||
if not is_private:
|
||||
await self._publish_command_reply(
|
||||
message,
|
||||
session_key,
|
||||
"请在和 ohmo 的私聊里使用 /group 创建新群。\nUse /group in a private chat with ohmo to create a new group.",
|
||||
)
|
||||
return None
|
||||
|
||||
metadata = dict(message.metadata)
|
||||
metadata["_ohmo_group_command"] = True
|
||||
metadata["_ohmo_group_raw_request"] = args
|
||||
prompt = _build_group_agent_prompt(args)
|
||||
return InboundMessage(
|
||||
channel=message.channel,
|
||||
sender_id=message.sender_id,
|
||||
chat_id=message.chat_id,
|
||||
content=prompt,
|
||||
timestamp=message.timestamp,
|
||||
media=list(message.media),
|
||||
metadata=metadata,
|
||||
session_key_override=message.session_key_override,
|
||||
)
|
||||
|
||||
async def _publish_command_reply(self, message, session_key: str, content: str) -> None:
|
||||
await self._bus.publish_outbound(
|
||||
OutboundMessage(
|
||||
channel=message.channel,
|
||||
chat_id=message.chat_id,
|
||||
content=content,
|
||||
metadata={"_session_key": session_key},
|
||||
)
|
||||
)
|
||||
|
||||
async def _interrupt_session(
|
||||
self,
|
||||
session_key: str,
|
||||
@@ -169,15 +246,24 @@ class OhmoGatewayBridge:
|
||||
return True
|
||||
|
||||
async def _process_message(self, message, session_key: str) -> None:
|
||||
# Preserve inbound message_id so channels can reply in-thread
|
||||
# Preserve thread metadata only for shared chats. Feishu p2p replies
|
||||
# should stay as normal private messages, not topic replies.
|
||||
inbound_meta = {
|
||||
k: message.metadata[k] for k in ("message_id", "thread_id") if k in message.metadata
|
||||
k: message.metadata[k] for k in ("thread_id",) if k in message.metadata
|
||||
}
|
||||
chat_type = str(message.metadata.get("chat_type") or "").lower()
|
||||
if chat_type == "group" or inbound_meta.get("thread_id"):
|
||||
if "message_id" in message.metadata:
|
||||
inbound_meta["message_id"] = message.metadata["message_id"]
|
||||
try:
|
||||
reply = ""
|
||||
final_media: list[str] = []
|
||||
final_metadata: dict[str, object] = {}
|
||||
async for update in self._runtime_pool.stream_message(message, session_key):
|
||||
if update.kind == "final":
|
||||
reply = update.text
|
||||
final_media = list(getattr(update, "media", None) or (update.metadata or {}).get("_media") or [])
|
||||
final_metadata = dict(update.metadata or {})
|
||||
continue
|
||||
if not update.text:
|
||||
continue
|
||||
@@ -194,6 +280,7 @@ class OhmoGatewayBridge:
|
||||
channel=message.channel,
|
||||
chat_id=message.chat_id,
|
||||
content=update.text,
|
||||
media=list(getattr(update, "media", None) or (update.metadata or {}).get("_media") or []),
|
||||
metadata={**inbound_meta, **(update.metadata or {})},
|
||||
)
|
||||
)
|
||||
@@ -236,7 +323,8 @@ class OhmoGatewayBridge:
|
||||
channel=message.channel,
|
||||
chat_id=message.chat_id,
|
||||
content=reply,
|
||||
metadata={**inbound_meta, "_session_key": session_key},
|
||||
media=final_media,
|
||||
metadata={**inbound_meta, **final_metadata, "_session_key": session_key},
|
||||
)
|
||||
)
|
||||
|
||||
@@ -245,3 +333,82 @@ class OhmoGatewayBridge:
|
||||
if current is task:
|
||||
self._session_tasks.pop(session_key, None)
|
||||
self._session_cancel_reasons.pop(session_key, None)
|
||||
|
||||
def _should_process_message(self, message: InboundMessage) -> bool:
|
||||
if message.channel != "feishu":
|
||||
return True
|
||||
chat_type = str(message.metadata.get("chat_type") or "").strip().lower()
|
||||
if chat_type != "group":
|
||||
return True
|
||||
policy = self._feishu_group_policy
|
||||
if policy == "open":
|
||||
return True
|
||||
mentioned = _message_mentions_bot(message)
|
||||
if policy == "mention":
|
||||
return mentioned
|
||||
if policy == "managed":
|
||||
return self._is_managed_feishu_group(message.chat_id)
|
||||
if policy == "managed_or_mention":
|
||||
return mentioned or self._is_managed_feishu_group(message.chat_id)
|
||||
return mentioned
|
||||
|
||||
def _is_managed_feishu_group(self, chat_id: str) -> bool:
|
||||
try:
|
||||
return load_managed_group_record(
|
||||
workspace=self._workspace,
|
||||
channel="feishu",
|
||||
chat_id=chat_id,
|
||||
) is not None
|
||||
except Exception:
|
||||
logger.exception("failed to load ohmo managed group metadata chat_id=%s", chat_id)
|
||||
return False
|
||||
|
||||
|
||||
def _parse_group_command(content: str) -> str | None:
|
||||
stripped = content.strip()
|
||||
parts = stripped.split(maxsplit=1)
|
||||
if not parts or parts[0] != "/group":
|
||||
return None
|
||||
if len(parts) == 1:
|
||||
return ""
|
||||
return parts[1].strip()
|
||||
|
||||
|
||||
def _build_group_agent_prompt(raw_request: str) -> str:
|
||||
request = raw_request.strip() or "(user did not provide details)"
|
||||
return (
|
||||
"The user invoked `/group` from a Feishu private chat.\n"
|
||||
"Your task is to create a dedicated Feishu group for this request.\n\n"
|
||||
"Use the `ohmo_create_feishu_group` tool exactly once if you can infer a safe group name. "
|
||||
"You, the model, must decide the final `name`, optional `repo`, and optional `cwd` from the user's "
|
||||
"natural-language request and available local context. If the cwd is not obvious, inspect the filesystem "
|
||||
"before calling the tool. If there is not enough information to choose safely, ask one concise clarification "
|
||||
"instead of calling the tool. Do not create the group via bash or direct API calls.\n\n"
|
||||
f"User /group request:\n{request}"
|
||||
)
|
||||
|
||||
|
||||
def _normalize_feishu_group_policy(value: str | None) -> str:
|
||||
normalized = str(value or "").strip().lower().replace("-", "_")
|
||||
aliases = {
|
||||
"all": "open",
|
||||
"always": "open",
|
||||
"always_reply": "open",
|
||||
"managed_mention": "managed_or_mention",
|
||||
"managed_or_at": "managed_or_mention",
|
||||
"at": "mention",
|
||||
"mentions": "mention",
|
||||
}
|
||||
normalized = aliases.get(normalized, normalized)
|
||||
if normalized in {"open", "mention", "managed", "managed_or_mention"}:
|
||||
return normalized
|
||||
return "managed_or_mention"
|
||||
|
||||
|
||||
def _message_mentions_bot(message: InboundMessage) -> bool:
|
||||
value = message.metadata.get("mentions_bot")
|
||||
if isinstance(value, bool):
|
||||
return value
|
||||
if isinstance(value, str):
|
||||
return value.strip().lower() in {"1", "true", "yes", "y"}
|
||||
return False
|
||||
|
||||
@@ -0,0 +1,158 @@
|
||||
"""ohmo-only Feishu group management tool."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from collections.abc import Awaitable, Callable
|
||||
from pathlib import Path
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from openharness.tools.base import BaseTool, ToolExecutionContext, ToolResult
|
||||
|
||||
from ohmo.group_registry import normalize_group_name, save_managed_group_record
|
||||
|
||||
|
||||
CreateFeishuGroup = Callable[[str, str], Awaitable[str] | str]
|
||||
PublishGroupWelcome = Callable[[str, str, str], Awaitable[None] | None]
|
||||
|
||||
|
||||
class OhmoCreateFeishuGroupInput(BaseModel):
|
||||
"""Arguments selected by the model for creating a Feishu group."""
|
||||
|
||||
name: str = Field(description="Final Feishu group name to create.")
|
||||
cwd: str | None = Field(
|
||||
default=None,
|
||||
description="Workspace directory to bind to the group. Use an absolute path, ~ path, or path relative to cwd.",
|
||||
)
|
||||
repo: str | None = Field(
|
||||
default=None,
|
||||
description="Repository identifier associated with the group, for example HKUDS/OpenHarness.",
|
||||
)
|
||||
reason: str | None = Field(
|
||||
default=None,
|
||||
description="Short reason explaining why these name/cwd/repo values were chosen.",
|
||||
)
|
||||
|
||||
|
||||
class OhmoCreateFeishuGroupTool(BaseTool):
|
||||
"""Create a Feishu group for the current ohmo private-chat requester."""
|
||||
|
||||
name = "ohmo_create_feishu_group"
|
||||
description = (
|
||||
"Create a Feishu group for the current private-chat /group request and bind optional cwd/repo metadata. "
|
||||
"Only use this after the user explicitly invokes /group. Infer name, cwd, and repo from the user's "
|
||||
"natural-language request and the local workspace context; inspect files first if needed."
|
||||
)
|
||||
input_model = OhmoCreateFeishuGroupInput
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
workspace: str | Path | None,
|
||||
create_group: CreateFeishuGroup,
|
||||
publish_group_welcome: PublishGroupWelcome | None = None,
|
||||
) -> None:
|
||||
self._workspace = workspace
|
||||
self._create_group = create_group
|
||||
self._publish_group_welcome = publish_group_welcome
|
||||
|
||||
def is_read_only(self, arguments: OhmoCreateFeishuGroupInput) -> bool:
|
||||
# Permission is enforced by the slash-command context guard below. This
|
||||
# tool is only registered inside ohmo gateway sessions and cannot run
|
||||
# unless the current inbound message was a private Feishu /group request.
|
||||
del arguments
|
||||
return True
|
||||
|
||||
async def execute(self, arguments: OhmoCreateFeishuGroupInput, context: ToolExecutionContext) -> ToolResult:
|
||||
request = context.metadata.get("ohmo_group_request")
|
||||
if not isinstance(request, dict):
|
||||
return ToolResult(
|
||||
output="ohmo_create_feishu_group can only run immediately after a Feishu private /group request.",
|
||||
is_error=True,
|
||||
)
|
||||
if request.get("used"):
|
||||
return ToolResult(output="This /group request has already created a group.", is_error=True)
|
||||
if request.get("channel") != "feishu" or request.get("chat_type") not in {"p2p", "private", "im", "direct", ""}:
|
||||
return ToolResult(output="/group group creation is only allowed from a Feishu private chat.", is_error=True)
|
||||
|
||||
owner_open_id = str(request.get("sender_id") or "").strip()
|
||||
if not owner_open_id:
|
||||
return ToolResult(output="Cannot create group: missing Feishu requester open_id.", is_error=True)
|
||||
|
||||
try:
|
||||
name = normalize_group_name(arguments.name)
|
||||
except ValueError as exc:
|
||||
return ToolResult(output=f"Cannot create group: {exc}", is_error=True)
|
||||
|
||||
cwd = _resolve_cwd(arguments.cwd, context.cwd)
|
||||
if cwd is not None and not Path(cwd).is_dir():
|
||||
return ToolResult(output=f"Cannot bind cwd because the directory does not exist: {cwd}", is_error=True)
|
||||
|
||||
try:
|
||||
result = self._create_group(owner_open_id, name)
|
||||
chat_id = await result if asyncio.iscoroutine(result) else result
|
||||
except Exception as exc:
|
||||
return ToolResult(output=f"Cannot create Feishu group: {exc}", is_error=True)
|
||||
chat_id = str(chat_id).strip()
|
||||
if not chat_id:
|
||||
return ToolResult(output="Cannot create group: Feishu returned an empty chat_id.", is_error=True)
|
||||
|
||||
request["used"] = True
|
||||
try:
|
||||
record_path = save_managed_group_record(
|
||||
workspace=self._workspace,
|
||||
channel="feishu",
|
||||
chat_id=chat_id,
|
||||
owner_open_id=owner_open_id,
|
||||
name=name,
|
||||
cwd=cwd,
|
||||
repo=arguments.repo,
|
||||
binding_status="bound" if cwd else "pending_agent",
|
||||
metadata={
|
||||
"source": "slash_group_tool",
|
||||
"raw_group_request": request.get("raw_request"),
|
||||
"source_chat_id": request.get("source_chat_id"),
|
||||
"source_session_key": request.get("source_session_key"),
|
||||
"sender_display_name": request.get("sender_display_name"),
|
||||
"tool_reason": arguments.reason,
|
||||
},
|
||||
)
|
||||
except Exception as exc:
|
||||
return ToolResult(
|
||||
output=f"Created Feishu group {chat_id}, but failed to save metadata: {exc}",
|
||||
is_error=True,
|
||||
metadata={"chat_id": chat_id},
|
||||
)
|
||||
|
||||
welcome = (
|
||||
"这个群已经创建好。"
|
||||
+ (f"\n已绑定工作目录:{cwd}" if cwd else "")
|
||||
+ (f"\n关联仓库:{arguments.repo}" if arguments.repo else "")
|
||||
+ "\n这个 ohmo 管理的群默认可以不用 @ 直接和我说话;普通群仍建议 @ohmo 触发。"
|
||||
+ "\n如果我没有响应,请确认飞书应用已开通接收群聊所有消息的权限。"
|
||||
)
|
||||
if self._publish_group_welcome is not None:
|
||||
published = self._publish_group_welcome(chat_id, welcome, owner_open_id)
|
||||
if asyncio.iscoroutine(published):
|
||||
await published
|
||||
|
||||
lines = [
|
||||
f"Created Feishu group: {name}",
|
||||
f"chat_id: {chat_id}",
|
||||
f"metadata: {record_path}",
|
||||
]
|
||||
if cwd:
|
||||
lines.append(f"cwd: {cwd}")
|
||||
if arguments.repo:
|
||||
lines.append(f"repo: {arguments.repo}")
|
||||
return ToolResult(output="\n".join(lines), metadata={"chat_id": chat_id, "cwd": cwd, "repo": arguments.repo})
|
||||
|
||||
|
||||
def _resolve_cwd(raw: str | None, base_cwd: Path) -> str | None:
|
||||
if raw is None or not str(raw).strip():
|
||||
return None
|
||||
path = Path(str(raw).strip()).expanduser()
|
||||
if not path.is_absolute():
|
||||
path = base_cwd / path
|
||||
return str(path.resolve())
|
||||
@@ -0,0 +1,78 @@
|
||||
"""Proactive notification helpers for ohmo gateway channels."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from ohmo.gateway.config import load_gateway_config
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class OhmoNotificationError(RuntimeError):
|
||||
"""Raised when a proactive notification cannot be delivered."""
|
||||
|
||||
|
||||
def _chunk_text(text: str, *, max_chars: int = 1800) -> list[str]:
|
||||
"""Split text into message-sized chunks without losing content."""
|
||||
stripped = text.strip()
|
||||
if not stripped:
|
||||
return []
|
||||
chunks: list[str] = []
|
||||
remaining = stripped
|
||||
while len(remaining) > max_chars:
|
||||
split_at = remaining.rfind("\n", 0, max_chars)
|
||||
if split_at < max_chars // 2:
|
||||
split_at = max_chars
|
||||
chunks.append(remaining[:split_at].strip())
|
||||
remaining = remaining[split_at:].strip()
|
||||
if remaining:
|
||||
chunks.append(remaining)
|
||||
return chunks
|
||||
|
||||
|
||||
def _send_feishu_text_sync(*, user_open_id: str, content: str, workspace: str | Path | None = None) -> None:
|
||||
"""Send a Feishu direct message using ohmo gateway Feishu credentials."""
|
||||
try:
|
||||
import lark_oapi as lark
|
||||
from lark_oapi.api.im.v1 import CreateMessageRequest, CreateMessageRequestBody
|
||||
except ImportError as exc: # pragma: no cover - depends on optional extra
|
||||
raise OhmoNotificationError("Feishu SDK is not installed. Run: pip install lark-oapi") from exc
|
||||
|
||||
config = load_gateway_config(workspace)
|
||||
feishu_config: dict[str, Any] = config.channel_configs.get("feishu", {})
|
||||
app_id = str(feishu_config.get("app_id") or "").strip()
|
||||
app_secret = str(feishu_config.get("app_secret") or "").strip()
|
||||
if not app_id or not app_secret:
|
||||
raise OhmoNotificationError("Feishu app_id/app_secret are not configured in ohmo gateway config.")
|
||||
|
||||
client = lark.Client.builder().app_id(app_id).app_secret(app_secret).log_level(lark.LogLevel.INFO).build()
|
||||
for chunk in _chunk_text(content):
|
||||
request = (
|
||||
CreateMessageRequest.builder()
|
||||
.receive_id_type("open_id")
|
||||
.request_body(
|
||||
CreateMessageRequestBody.builder()
|
||||
.receive_id(user_open_id)
|
||||
.msg_type("text")
|
||||
.content(json.dumps({"text": chunk}, ensure_ascii=False))
|
||||
.build()
|
||||
)
|
||||
.build()
|
||||
)
|
||||
response = client.im.v1.message.create(request)
|
||||
if not response.success():
|
||||
log_id = response.get_log_id() if hasattr(response, "get_log_id") else ""
|
||||
raise OhmoNotificationError(
|
||||
f"send Feishu DM failed: code={response.code}, msg={response.msg}, log_id={log_id}"
|
||||
)
|
||||
|
||||
|
||||
async def send_feishu_dm(*, user_open_id: str, content: str, workspace: str | Path | None = None) -> None:
|
||||
"""Send a proactive Feishu direct message to a user open_id."""
|
||||
await asyncio.to_thread(_send_feishu_text_sync, user_open_id=user_open_id, content=content, workspace=workspace)
|
||||
logger.info("Sent proactive Feishu DM to open_id=%s", user_open_id)
|
||||
@@ -0,0 +1,139 @@
|
||||
"""ohmo gateway-scoped provider and model commands."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from openharness.auth.manager import AuthManager
|
||||
from openharness.config import load_settings
|
||||
|
||||
from ohmo.gateway.config import load_gateway_config, save_gateway_config
|
||||
|
||||
|
||||
def handle_gateway_provider_command(args: str, *, workspace: str | Path | None) -> tuple[str, bool]:
|
||||
"""Handle ``/provider`` against the ohmo gateway config."""
|
||||
tokens = args.split()
|
||||
statuses = AuthManager(load_settings()).get_profile_statuses()
|
||||
config = load_gateway_config(workspace)
|
||||
active = config.provider_profile
|
||||
if not tokens or tokens[0] == "show":
|
||||
info = statuses.get(active)
|
||||
if info is None:
|
||||
return f"ohmo gateway provider_profile: {active}\nStatus: unknown profile", False
|
||||
return (
|
||||
f"ohmo gateway provider_profile: {active}\n"
|
||||
f"Label: {info['label']}\n"
|
||||
f"Configured: {'yes' if info['configured'] else 'no'}\n"
|
||||
f"Base URL: {info['base_url'] or '(default)'}\n"
|
||||
f"Model: {info['model']}",
|
||||
False,
|
||||
)
|
||||
if tokens[0] == "list":
|
||||
lines = ["ohmo gateway provider profiles:"]
|
||||
for name, info in statuses.items():
|
||||
marker = "*" if name == active else " "
|
||||
configured = "ready" if info["configured"] else "missing auth"
|
||||
lines.append(f"{marker} {name} [{configured}] {info['label']} -> {info['model']}")
|
||||
return "\n".join(lines), False
|
||||
target = tokens[1] if tokens[0] == "use" and len(tokens) == 2 else (tokens[0] if len(tokens) == 1 else None)
|
||||
if target is None:
|
||||
return "Usage: /provider [show|list|PROFILE]", False
|
||||
if target not in statuses:
|
||||
return f"Unknown provider profile: {target}", False
|
||||
if target == active:
|
||||
return f"ohmo gateway already uses provider_profile={target}.", False
|
||||
save_gateway_config(config.model_copy(update={"provider_profile": target}), workspace)
|
||||
info = statuses[target]
|
||||
configured = "ready" if info["configured"] else "missing auth"
|
||||
return (
|
||||
f"ohmo gateway provider_profile set to {target} ({info['label']}, {configured}).\n"
|
||||
"Refreshing the current ohmo runtime to apply it.",
|
||||
True,
|
||||
)
|
||||
|
||||
|
||||
def handle_gateway_model_command(args: str, *, workspace: str | Path | None) -> tuple[str, bool]:
|
||||
"""Handle ``/model`` against the profile selected by ohmo gateway."""
|
||||
settings = load_settings()
|
||||
manager = AuthManager(settings)
|
||||
config = load_gateway_config(workspace)
|
||||
profile_name = config.provider_profile
|
||||
profiles = manager.list_profiles()
|
||||
profile = profiles.get(profile_name)
|
||||
if profile is None:
|
||||
return f"ohmo gateway provider_profile is unknown: {profile_name}", False
|
||||
|
||||
tokens = args.split(maxsplit=1)
|
||||
if not tokens or tokens[0] == "show":
|
||||
return _format_model_status(profile_name, profile), False
|
||||
if tokens[0] == "list":
|
||||
if profile.allowed_models:
|
||||
return (
|
||||
f"Switchable models for ohmo gateway profile '{profile_name}':\n"
|
||||
+ "\n".join(f"- {model}" for model in profile.allowed_models)
|
||||
), False
|
||||
return (
|
||||
f"Profile '{profile_name}' has no pinned model list. "
|
||||
"Any model value is accepted. Use /model add MODEL to pin one."
|
||||
), False
|
||||
if tokens[0] == "add" and len(tokens) == 2:
|
||||
model_name = tokens[1].strip()
|
||||
if not model_name:
|
||||
return "Usage: /model add MODEL", False
|
||||
manager.update_profile(profile_name, allowed_models=_dedupe([*_seed_models(profile), model_name]))
|
||||
return f"Added model '{model_name}' to ohmo gateway profile '{profile_name}'.", False
|
||||
if tokens[0] == "add":
|
||||
return "Usage: /model add MODEL", False
|
||||
if tokens[0] in {"remove", "rm"} and len(tokens) == 2:
|
||||
model_name = tokens[1].strip()
|
||||
models = [model for model in _dedupe(profile.allowed_models) if model != model_name]
|
||||
if len(models) == len(_dedupe(profile.allowed_models)):
|
||||
return f"Model '{model_name}' is not pinned for ohmo gateway profile '{profile_name}'.", False
|
||||
reset_current = (profile.last_model or "").strip() == model_name
|
||||
manager.update_profile(profile_name, allowed_models=models, last_model="" if reset_current else None)
|
||||
return f"Removed model '{model_name}' from ohmo gateway profile '{profile_name}'.", True
|
||||
if tokens[0] in {"remove", "rm"}:
|
||||
return "Usage: /model remove MODEL", False
|
||||
if tokens[0] == "clear":
|
||||
manager.update_profile(profile_name, allowed_models=[])
|
||||
return f"Cleared pinned models for ohmo gateway profile '{profile_name}'.", False
|
||||
model_name = tokens[1].strip() if tokens[0] == "set" and len(tokens) == 2 else args.strip()
|
||||
if not model_name:
|
||||
return "Usage: /model [show|list|add MODEL|remove MODEL|clear|MODEL]", False
|
||||
if profile.allowed_models and model_name.lower() != "default" and model_name not in profile.allowed_models:
|
||||
allowed = ", ".join(profile.allowed_models)
|
||||
return f"Model '{model_name}' is not allowed for ohmo gateway profile '{profile_name}'. Allowed models: {allowed}", False
|
||||
if model_name.lower() == "default":
|
||||
manager.update_profile(profile_name, last_model="")
|
||||
return f"ohmo gateway model reset to default for profile '{profile_name}'. Refreshing runtime to apply it.", True
|
||||
manager.update_profile(profile_name, last_model=model_name)
|
||||
return f"ohmo gateway model set to {model_name} for profile '{profile_name}'. Refreshing runtime to apply it.", True
|
||||
|
||||
|
||||
def _format_model_status(profile_name, profile) -> str:
|
||||
lines = [
|
||||
f"ohmo gateway model: {profile.resolved_model}",
|
||||
f"Profile: {profile_name}",
|
||||
]
|
||||
if profile.allowed_models:
|
||||
lines.append("Available models:")
|
||||
lines.extend(f"- {model}" for model in profile.allowed_models)
|
||||
else:
|
||||
lines.append("Available models: unrestricted for this profile")
|
||||
lines.append("Use /model add MODEL to pin switchable models.")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _dedupe(values) -> list[str]:
|
||||
seen: set[str] = set()
|
||||
result: list[str] = []
|
||||
for value in values:
|
||||
model = str(value).strip()
|
||||
if model and model not in seen:
|
||||
result.append(model)
|
||||
seen.add(model)
|
||||
return result
|
||||
|
||||
|
||||
def _seed_models(profile) -> list[str]:
|
||||
return _dedupe([*profile.allowed_models, profile.resolved_model])
|
||||
+14
-4
@@ -6,16 +6,26 @@ from openharness.channels.bus.events import InboundMessage
|
||||
|
||||
|
||||
def session_key_for_message(message: InboundMessage) -> str:
|
||||
"""Route sessions by sender plus chat/thread when available."""
|
||||
"""Route sessions by chat, isolating shared chats by thread/sender.
|
||||
|
||||
Private chats keep the original ``channel:chat_id`` key so existing long
|
||||
ohmo sessions remain resumable. Group/shared chats include sender identity
|
||||
to avoid multiple people sharing one agent memory.
|
||||
"""
|
||||
if message.session_key_override:
|
||||
return message.session_key_override
|
||||
sender_id = str(message.sender_id).strip() or "anonymous"
|
||||
chat_type = str(message.metadata.get("chat_type") or "").strip().lower()
|
||||
is_shared_chat = chat_type in {"group", "chat", "supergroup", "channel", "room"}
|
||||
thread_id = (
|
||||
message.metadata.get("thread_id")
|
||||
or message.metadata.get("thread_ts")
|
||||
or message.metadata.get("message_thread_id")
|
||||
)
|
||||
if thread_id:
|
||||
return f"{message.channel}:{message.chat_id}:{thread_id}:{sender_id}"
|
||||
return f"{message.channel}:{message.chat_id}:{sender_id}"
|
||||
|
||||
if is_shared_chat:
|
||||
return f"{message.channel}:{message.chat_id}:{thread_id}:{sender_id}"
|
||||
return f"{message.channel}:{message.chat_id}:{thread_id}"
|
||||
if is_shared_chat:
|
||||
return f"{message.channel}:{message.chat_id}:{sender_id}"
|
||||
return f"{message.channel}:{message.chat_id}"
|
||||
|
||||
+499
-44
@@ -9,11 +9,17 @@ import mimetypes
|
||||
from pathlib import Path
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import string
|
||||
|
||||
from openharness.channels.bus.events import InboundMessage
|
||||
from openharness.commands import CommandContext, CommandResult
|
||||
from openharness.engine.messages import ConversationMessage, ImageBlock, TextBlock
|
||||
from openharness.commands import CommandContext, CommandResult, lookup_skill_slash_command
|
||||
from openharness.engine.messages import (
|
||||
ConversationMessage,
|
||||
ImageBlock,
|
||||
TextBlock,
|
||||
sanitize_conversation_messages,
|
||||
)
|
||||
from openharness.engine.query import MaxTurnsExceeded
|
||||
from openharness.engine.stream_events import (
|
||||
AssistantTextDelta,
|
||||
@@ -28,9 +34,13 @@ from openharness.prompts import build_runtime_system_prompt
|
||||
from openharness.ui.runtime import RuntimeBundle, _last_user_text, build_runtime, close_runtime, start_runtime
|
||||
|
||||
from ohmo.gateway.config import load_gateway_config
|
||||
from ohmo.gateway.group_tool import CreateFeishuGroup, OhmoCreateFeishuGroupTool, PublishGroupWelcome
|
||||
from ohmo.gateway.provider_commands import handle_gateway_model_command, handle_gateway_provider_command
|
||||
from ohmo.group_registry import load_managed_group_record, normalize_cwd
|
||||
from ohmo.memory import create_memory_command_backend
|
||||
from ohmo.prompts import build_ohmo_system_prompt
|
||||
from ohmo.session_storage import OhmoSessionBackend
|
||||
from ohmo.workspace import get_plugins_dir, get_skills_dir, initialize_workspace
|
||||
from ohmo.workspace import get_memory_dir, get_plugins_dir, get_sessions_dir, get_skills_dir, initialize_workspace
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -53,6 +63,25 @@ _CHANNEL_THINKING_PHRASES_EN = (
|
||||
_TEXT_PREVIEW_BYTES = 4096
|
||||
_TEXT_PREVIEW_CHARS = 900
|
||||
_BINARY_HEAD_BYTES = 32
|
||||
_FINAL_REPLY_IMAGE_PATH_RE = re.compile(
|
||||
r"(?P<path>(?:[A-Za-z]:[\\/]|/)[^\r\n`\"'<>|?*\x00]+?\.(?:png|jpe?g|webp|gif|bmp))",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
_IMAGE_FALLBACK_NOTE = (
|
||||
"[Image attachment omitted because the active model does not support image input. "
|
||||
"Use the attachment paths and summaries above if needed.]"
|
||||
)
|
||||
_NO_GROUP_REQUEST = object()
|
||||
_GROUP_TOOL_NAME = "ohmo_create_feishu_group"
|
||||
_GROUP_AGENT_PROMPT_PREFIX = "The user invoked `/group` from a Feishu private chat."
|
||||
_GROUP_AGENT_PROMPT_REQUEST_MARKER = "User /group request:"
|
||||
_GROUP_METADATA_KEYS = (
|
||||
"task_focus_state",
|
||||
"recent_work_log",
|
||||
"recent_verified_work",
|
||||
"compact_checkpoints",
|
||||
"compact_last",
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -62,6 +91,7 @@ class GatewayStreamUpdate:
|
||||
kind: str
|
||||
text: str
|
||||
metadata: dict[str, object]
|
||||
media: list[str] | None = None
|
||||
|
||||
|
||||
class OhmoSessionRuntimePool:
|
||||
@@ -75,12 +105,16 @@ class OhmoSessionRuntimePool:
|
||||
provider_profile: str,
|
||||
model: str | None = None,
|
||||
max_turns: int | None = None,
|
||||
create_feishu_group: CreateFeishuGroup | None = None,
|
||||
publish_group_welcome: PublishGroupWelcome | None = None,
|
||||
) -> None:
|
||||
self._cwd = str(Path(cwd).resolve())
|
||||
self._workspace = workspace
|
||||
self._provider_profile = provider_profile
|
||||
self._model = model
|
||||
self._max_turns = max_turns
|
||||
self._create_feishu_group = create_feishu_group
|
||||
self._publish_group_welcome = publish_group_welcome
|
||||
self._workspace = initialize_workspace(workspace)
|
||||
self._gateway_config = load_gateway_config(self._workspace)
|
||||
self._session_backend = OhmoSessionBackend(self._workspace)
|
||||
@@ -102,18 +136,48 @@ class OhmoSessionRuntimePool:
|
||||
}
|
||||
return command.name.lower() in allowed
|
||||
|
||||
async def get_bundle(self, session_key: str, latest_user_prompt: str | None = None) -> RuntimeBundle:
|
||||
def _handle_gateway_scoped_command(self, command_name: str, args: str) -> tuple[str, bool] | None:
|
||||
lowered = command_name.lower()
|
||||
if lowered == "provider":
|
||||
result = handle_gateway_provider_command(args, workspace=self._workspace)
|
||||
elif lowered == "model":
|
||||
result = handle_gateway_model_command(args, workspace=self._workspace)
|
||||
else:
|
||||
return None
|
||||
if result[1]:
|
||||
self._gateway_config = load_gateway_config(self._workspace)
|
||||
self._provider_profile = self._gateway_config.provider_profile
|
||||
return result
|
||||
|
||||
async def get_bundle(
|
||||
self,
|
||||
session_key: str,
|
||||
latest_user_prompt: str | None = None,
|
||||
cwd: str | Path | None = None,
|
||||
) -> RuntimeBundle:
|
||||
"""Return an existing bundle or create a new one."""
|
||||
session_cwd = str(Path(cwd or self._cwd).expanduser().resolve())
|
||||
bundle = self._bundles.get(session_key)
|
||||
if bundle is not None:
|
||||
logger.info(
|
||||
"ohmo runtime reusing session session_key=%s session_id=%s prompt=%r",
|
||||
session_key,
|
||||
bundle.session_id,
|
||||
_content_snippet(latest_user_prompt or ""),
|
||||
)
|
||||
bundle.engine.set_system_prompt(self._runtime_system_prompt(bundle, latest_user_prompt))
|
||||
return bundle
|
||||
bundle_cwd = str(Path(getattr(bundle, "cwd", self._cwd)).resolve())
|
||||
if bundle_cwd != session_cwd:
|
||||
logger.info(
|
||||
"ohmo runtime recreating session for cwd change session_key=%s old_cwd=%s new_cwd=%s",
|
||||
session_key,
|
||||
bundle_cwd,
|
||||
session_cwd,
|
||||
)
|
||||
await close_runtime(bundle)
|
||||
self._bundles.pop(session_key, None)
|
||||
else:
|
||||
logger.info(
|
||||
"ohmo runtime reusing session session_key=%s session_id=%s prompt=%r",
|
||||
session_key,
|
||||
bundle.session_id,
|
||||
_content_snippet(latest_user_prompt or ""),
|
||||
)
|
||||
bundle.engine.set_system_prompt(self._runtime_system_prompt(bundle, latest_user_prompt))
|
||||
return bundle
|
||||
|
||||
snapshot = self._session_backend.load_latest_for_session_key(session_key)
|
||||
logger.info(
|
||||
@@ -123,19 +187,29 @@ class OhmoSessionRuntimePool:
|
||||
_content_snippet(latest_user_prompt or ""),
|
||||
)
|
||||
bundle = await build_runtime(
|
||||
cwd=session_cwd,
|
||||
model=self._model,
|
||||
max_turns=self._max_turns,
|
||||
system_prompt=build_ohmo_system_prompt(self._cwd, workspace=self._workspace, extra_prompt=None),
|
||||
system_prompt=build_ohmo_system_prompt(session_cwd, workspace=self._workspace, extra_prompt=None),
|
||||
active_profile=self._provider_profile,
|
||||
session_backend=self._session_backend,
|
||||
enforce_max_turns=self._max_turns is not None,
|
||||
restore_messages=snapshot.get("messages") if snapshot else None,
|
||||
restore_tool_metadata=snapshot.get("tool_metadata") if snapshot else None,
|
||||
restore_messages=_sanitize_snapshot_messages(snapshot.get("messages") if snapshot else None),
|
||||
restore_tool_metadata=_sanitize_group_command_metadata(snapshot.get("tool_metadata") if snapshot else None),
|
||||
extra_skill_dirs=(str(get_skills_dir(self._workspace)),),
|
||||
extra_plugin_roots=(str(get_plugins_dir(self._workspace)),),
|
||||
memory_backend=create_memory_command_backend(self._workspace),
|
||||
include_project_memory=False,
|
||||
autodream_context={
|
||||
"memory_dir": str(get_memory_dir(self._workspace)),
|
||||
"session_dir": str(get_sessions_dir(self._workspace)),
|
||||
"app_label": "ohmo personal memory",
|
||||
"runner_module": "ohmo",
|
||||
},
|
||||
)
|
||||
if snapshot and snapshot.get("session_id"):
|
||||
bundle.session_id = str(snapshot["session_id"])
|
||||
self._register_gateway_tools(bundle)
|
||||
await start_runtime(bundle)
|
||||
bundle.engine.set_system_prompt(self._runtime_system_prompt(bundle, latest_user_prompt))
|
||||
logger.info(
|
||||
@@ -151,7 +225,9 @@ class OhmoSessionRuntimePool:
|
||||
"""Submit an inbound channel message and yield progress + final reply updates."""
|
||||
user_message = _build_inbound_user_message(message)
|
||||
user_prompt = user_message.text
|
||||
bundle = await self.get_bundle(session_key, latest_user_prompt=user_prompt)
|
||||
command_prompt = (message.content or "").strip()
|
||||
session_cwd = self._cwd_for_message(message)
|
||||
bundle = await self.get_bundle(session_key, latest_user_prompt=user_prompt, cwd=session_cwd)
|
||||
logger.info(
|
||||
"ohmo runtime processing start channel=%s chat_id=%s session_key=%s session_id=%s content=%r",
|
||||
message.channel,
|
||||
@@ -161,9 +237,34 @@ class OhmoSessionRuntimePool:
|
||||
_content_snippet(user_prompt),
|
||||
)
|
||||
|
||||
parsed = bundle.commands.lookup(user_prompt)
|
||||
command_context: CommandContext | None = None
|
||||
|
||||
def get_command_context() -> CommandContext:
|
||||
nonlocal command_context
|
||||
if command_context is None:
|
||||
command_context = CommandContext(
|
||||
engine=bundle.engine,
|
||||
hooks_summary=getattr(bundle, "hook_summary", lambda: "")(),
|
||||
mcp_summary=getattr(bundle, "mcp_summary", lambda: "")(),
|
||||
plugin_summary=getattr(bundle, "plugin_summary", lambda: "")(),
|
||||
cwd=getattr(bundle, "cwd", str(self._cwd)),
|
||||
tool_registry=getattr(bundle, "tool_registry", None),
|
||||
app_state=getattr(bundle, "app_state", None),
|
||||
session_backend=getattr(bundle, "session_backend", self._session_backend),
|
||||
session_id=getattr(bundle, "session_id", None),
|
||||
extra_skill_dirs=getattr(bundle, "extra_skill_dirs", ()),
|
||||
extra_plugin_roots=getattr(bundle, "extra_plugin_roots", ()),
|
||||
memory_backend=create_memory_command_backend(self._workspace),
|
||||
include_project_memory=False,
|
||||
)
|
||||
return command_context
|
||||
|
||||
parsed = bundle.commands.lookup(command_prompt)
|
||||
if parsed is None and not message.media:
|
||||
parsed = lookup_skill_slash_command(command_prompt, get_command_context())
|
||||
if parsed is not None and not message.media:
|
||||
command, args = parsed
|
||||
command_name = str(getattr(command, "name", "") or "")
|
||||
remote_allowed = getattr(command, "remote_invocable", True)
|
||||
if not remote_allowed and self._remote_admin_allowed(command):
|
||||
remote_allowed = True
|
||||
@@ -172,11 +273,11 @@ class OhmoSessionRuntimePool:
|
||||
message.channel,
|
||||
message.chat_id,
|
||||
message.sender_id,
|
||||
command.name,
|
||||
command_name,
|
||||
)
|
||||
if not remote_allowed:
|
||||
result = CommandResult(
|
||||
message=f"/{command.name} is only available in the local OpenHarness UI."
|
||||
message=f"/{command_name} is only available in the local OpenHarness UI."
|
||||
)
|
||||
async for update in self._stream_command_result(
|
||||
bundle=bundle,
|
||||
@@ -187,21 +288,22 @@ class OhmoSessionRuntimePool:
|
||||
):
|
||||
yield update
|
||||
return
|
||||
gateway_result = self._handle_gateway_scoped_command(command_name, args)
|
||||
if gateway_result is not None:
|
||||
message_text, refresh_runtime = gateway_result
|
||||
result = CommandResult(message=message_text, refresh_runtime=refresh_runtime)
|
||||
async for update in self._stream_command_result(
|
||||
bundle=bundle,
|
||||
message=message,
|
||||
session_key=session_key,
|
||||
user_prompt=user_prompt,
|
||||
result=result,
|
||||
):
|
||||
yield update
|
||||
return
|
||||
result = await command.handler(
|
||||
args,
|
||||
CommandContext(
|
||||
engine=bundle.engine,
|
||||
hooks_summary=bundle.hook_summary(),
|
||||
mcp_summary=bundle.mcp_summary(),
|
||||
plugin_summary=bundle.plugin_summary(),
|
||||
cwd=bundle.cwd,
|
||||
tool_registry=bundle.tool_registry,
|
||||
app_state=bundle.app_state,
|
||||
session_backend=bundle.session_backend,
|
||||
session_id=bundle.session_id,
|
||||
extra_skill_dirs=bundle.extra_skill_dirs,
|
||||
extra_plugin_roots=bundle.extra_plugin_roots,
|
||||
),
|
||||
get_command_context(),
|
||||
)
|
||||
async for update in self._stream_command_result(
|
||||
bundle=bundle,
|
||||
@@ -308,6 +410,7 @@ class OhmoSessionRuntimePool:
|
||||
):
|
||||
bundle.engine.set_system_prompt(self._runtime_system_prompt(bundle, user_prompt))
|
||||
reply_parts: list[str] = []
|
||||
emitted_media: set[str] = set()
|
||||
yield GatewayStreamUpdate(
|
||||
kind="progress",
|
||||
text=_format_channel_progress(
|
||||
@@ -319,8 +422,43 @@ class OhmoSessionRuntimePool:
|
||||
),
|
||||
metadata={"_progress": True, "_session_key": session_key},
|
||||
)
|
||||
previous_group_request = self._set_group_request_context(bundle, message, session_key)
|
||||
try:
|
||||
async for event in bundle.engine.submit_message(user_message):
|
||||
if isinstance(event, ErrorEvent) and _should_retry_without_image_input(
|
||||
event.message,
|
||||
bundle.engine.messages,
|
||||
):
|
||||
logger.warning(
|
||||
"ohmo runtime image input rejected; retrying without image blocks session_key=%s session_id=%s message=%r",
|
||||
session_key,
|
||||
bundle.session_id,
|
||||
_content_snippet(event.message),
|
||||
)
|
||||
_strip_image_blocks_from_engine_history(bundle.engine)
|
||||
yield GatewayStreamUpdate(
|
||||
kind="progress",
|
||||
text=_format_channel_progress(
|
||||
channel=message.channel,
|
||||
kind="image_fallback",
|
||||
text=event.message,
|
||||
session_key=session_key,
|
||||
content=user_prompt,
|
||||
),
|
||||
metadata={"_progress": True, "_session_key": session_key, "_image_fallback": True},
|
||||
)
|
||||
async for retry_event in bundle.engine.continue_pending(max_turns=bundle.engine.max_turns):
|
||||
async for update in self._convert_stream_event(
|
||||
event=retry_event,
|
||||
bundle=bundle,
|
||||
message=message,
|
||||
session_key=session_key,
|
||||
content=user_prompt,
|
||||
reply_parts=reply_parts,
|
||||
):
|
||||
_remember_update_media(emitted_media, update)
|
||||
yield update
|
||||
break
|
||||
async for update in self._convert_stream_event(
|
||||
event=event,
|
||||
bundle=bundle,
|
||||
@@ -329,6 +467,7 @@ class OhmoSessionRuntimePool:
|
||||
content=user_prompt,
|
||||
reply_parts=reply_parts,
|
||||
):
|
||||
_remember_update_media(emitted_media, update)
|
||||
yield update
|
||||
except MaxTurnsExceeded as exc:
|
||||
yield GatewayStreamUpdate(
|
||||
@@ -336,8 +475,13 @@ class OhmoSessionRuntimePool:
|
||||
text=f"Stopped after {exc.max_turns} turns (max_turns).",
|
||||
metadata={"_session_key": session_key},
|
||||
)
|
||||
self._restore_group_request_context(bundle, previous_group_request)
|
||||
await self._save_snapshot(bundle, session_key, user_prompt)
|
||||
return
|
||||
except Exception:
|
||||
self._restore_group_request_context(bundle, previous_group_request)
|
||||
raise
|
||||
self._restore_group_request_context(bundle, previous_group_request)
|
||||
await self._save_snapshot(bundle, session_key, user_prompt)
|
||||
reply = "".join(reply_parts).strip()
|
||||
if reply:
|
||||
@@ -347,10 +491,15 @@ class OhmoSessionRuntimePool:
|
||||
bundle.session_id,
|
||||
_content_snippet(reply),
|
||||
)
|
||||
final_media = _extract_final_reply_media(reply, emitted_media)
|
||||
metadata: dict[str, object] = {"_session_key": session_key}
|
||||
if final_media:
|
||||
metadata.update({"_media": final_media, "_final_media_fallback": True})
|
||||
yield GatewayStreamUpdate(
|
||||
kind="final",
|
||||
text=reply,
|
||||
metadata={"_session_key": session_key},
|
||||
metadata=metadata,
|
||||
media=final_media or None,
|
||||
)
|
||||
|
||||
async def _convert_stream_event(
|
||||
@@ -446,6 +595,14 @@ class OhmoSessionRuntimePool:
|
||||
bundle.session_id,
|
||||
event.tool_name,
|
||||
)
|
||||
media = _extract_tool_media(event)
|
||||
if media:
|
||||
yield GatewayStreamUpdate(
|
||||
kind="media",
|
||||
text=_format_tool_media_caption(event, media),
|
||||
metadata={"_session_key": session_key, "_media": media, "_tool_media": True},
|
||||
media=media,
|
||||
)
|
||||
return
|
||||
if isinstance(event, ErrorEvent):
|
||||
logger.error(
|
||||
@@ -464,12 +621,20 @@ class OhmoSessionRuntimePool:
|
||||
reply_parts.append(event.message.text.strip())
|
||||
|
||||
async def _save_snapshot(self, bundle: RuntimeBundle, session_key: str, user_prompt: str) -> None:
|
||||
tool_metadata = getattr(bundle.engine, "tool_metadata", {}) or {}
|
||||
tool_metadata = _sanitize_group_command_metadata(getattr(bundle.engine, "tool_metadata", {}) or {})
|
||||
if isinstance(getattr(bundle.engine, "tool_metadata", None), dict) and isinstance(tool_metadata, dict):
|
||||
bundle.engine.tool_metadata.update(tool_metadata)
|
||||
messages = _sanitize_group_command_prompts(list(bundle.engine.messages))
|
||||
if messages != list(bundle.engine.messages):
|
||||
if hasattr(bundle.engine, "load_messages"):
|
||||
bundle.engine.load_messages(messages)
|
||||
else:
|
||||
bundle.engine.messages = messages
|
||||
self._session_backend.save_snapshot(
|
||||
cwd=self._cwd,
|
||||
cwd=getattr(bundle, "cwd", self._cwd),
|
||||
model=bundle.current_settings().model,
|
||||
system_prompt=self._runtime_system_prompt(bundle, user_prompt),
|
||||
messages=bundle.engine.messages,
|
||||
messages=messages,
|
||||
usage=bundle.engine.total_usage,
|
||||
session_id=bundle.session_id,
|
||||
session_key=session_key,
|
||||
@@ -488,23 +653,33 @@ class OhmoSessionRuntimePool:
|
||||
bundle: RuntimeBundle,
|
||||
latest_user_prompt: str | None,
|
||||
) -> RuntimeBundle:
|
||||
snapshot = list(bundle.engine.messages)
|
||||
snapshot = sanitize_conversation_messages(list(bundle.engine.messages))
|
||||
prior_session_id = bundle.session_id
|
||||
bundle_cwd = str(Path(getattr(bundle, "cwd", self._cwd)).resolve())
|
||||
await close_runtime(bundle)
|
||||
refreshed = await build_runtime(
|
||||
cwd=self._cwd,
|
||||
cwd=bundle_cwd,
|
||||
model=self._model,
|
||||
max_turns=self._max_turns,
|
||||
system_prompt=build_ohmo_system_prompt(self._cwd, workspace=self._workspace, extra_prompt=None),
|
||||
system_prompt=build_ohmo_system_prompt(bundle_cwd, workspace=self._workspace, extra_prompt=None),
|
||||
active_profile=self._provider_profile,
|
||||
session_backend=self._session_backend,
|
||||
enforce_max_turns=self._max_turns is not None,
|
||||
restore_messages=[message.model_dump(mode="json") for message in snapshot],
|
||||
restore_tool_metadata=getattr(bundle.engine, "tool_metadata", {}) or {},
|
||||
restore_messages=[message.model_dump(mode="json") for message in _sanitize_group_command_prompts(snapshot)],
|
||||
restore_tool_metadata=_sanitize_group_command_metadata(getattr(bundle.engine, "tool_metadata", {}) or {}),
|
||||
extra_skill_dirs=(str(get_skills_dir(self._workspace)),),
|
||||
extra_plugin_roots=(str(get_plugins_dir(self._workspace)),),
|
||||
memory_backend=create_memory_command_backend(self._workspace),
|
||||
include_project_memory=False,
|
||||
autodream_context={
|
||||
"memory_dir": str(get_memory_dir(self._workspace)),
|
||||
"session_dir": str(get_sessions_dir(self._workspace)),
|
||||
"app_label": "ohmo personal memory",
|
||||
"runner_module": "ohmo",
|
||||
},
|
||||
)
|
||||
refreshed.session_id = prior_session_id
|
||||
self._register_gateway_tools(refreshed)
|
||||
await start_runtime(refreshed)
|
||||
refreshed.engine.set_system_prompt(self._runtime_system_prompt(refreshed, latest_user_prompt))
|
||||
self._bundles[session_key] = refreshed
|
||||
@@ -517,19 +692,99 @@ class OhmoSessionRuntimePool:
|
||||
return refreshed
|
||||
|
||||
def _runtime_system_prompt(self, bundle: RuntimeBundle, latest_user_prompt: str | None) -> str:
|
||||
bundle_cwd = str(Path(getattr(bundle, "cwd", self._cwd)).resolve())
|
||||
if not hasattr(bundle, "current_settings"):
|
||||
return build_ohmo_system_prompt(self._cwd, workspace=self._workspace, extra_prompt=None)
|
||||
return build_ohmo_system_prompt(bundle_cwd, workspace=self._workspace, extra_prompt=None)
|
||||
settings = bundle.current_settings()
|
||||
if not hasattr(settings, "system_prompt"):
|
||||
return build_ohmo_system_prompt(self._cwd, workspace=self._workspace, extra_prompt=None)
|
||||
return build_ohmo_system_prompt(bundle_cwd, workspace=self._workspace, extra_prompt=None)
|
||||
return build_runtime_system_prompt(
|
||||
settings,
|
||||
cwd=self._cwd,
|
||||
cwd=bundle_cwd,
|
||||
latest_user_prompt=latest_user_prompt,
|
||||
extra_skill_dirs=getattr(bundle, "extra_skill_dirs", ()),
|
||||
extra_plugin_roots=getattr(bundle, "extra_plugin_roots", ()),
|
||||
include_project_memory=False,
|
||||
)
|
||||
|
||||
def _cwd_for_message(self, message: InboundMessage) -> str:
|
||||
record = load_managed_group_record(
|
||||
workspace=self._workspace,
|
||||
channel=message.channel,
|
||||
chat_id=message.chat_id,
|
||||
)
|
||||
cwd = record.get("cwd") if record else None
|
||||
if not cwd:
|
||||
return self._cwd
|
||||
normalized = normalize_cwd(str(cwd))
|
||||
if not Path(normalized).is_dir():
|
||||
logger.warning(
|
||||
"ohmo managed group cwd does not exist channel=%s chat_id=%s cwd=%s",
|
||||
message.channel,
|
||||
message.chat_id,
|
||||
normalized,
|
||||
)
|
||||
return self._cwd
|
||||
return normalized
|
||||
|
||||
def _register_gateway_tools(self, bundle: RuntimeBundle) -> None:
|
||||
self._unregister_group_tool(bundle)
|
||||
|
||||
def _register_group_tool(self, bundle: RuntimeBundle) -> None:
|
||||
if self._create_feishu_group is None or not hasattr(bundle, "tool_registry"):
|
||||
return
|
||||
if bundle.tool_registry is None or bundle.tool_registry.get(_GROUP_TOOL_NAME) is not None:
|
||||
return
|
||||
bundle.tool_registry.register(
|
||||
OhmoCreateFeishuGroupTool(
|
||||
workspace=self._workspace,
|
||||
create_group=self._create_feishu_group,
|
||||
publish_group_welcome=self._publish_group_welcome,
|
||||
)
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _unregister_group_tool(bundle: RuntimeBundle) -> None:
|
||||
registry = getattr(bundle, "tool_registry", None)
|
||||
tools = getattr(registry, "_tools", None)
|
||||
if isinstance(tools, dict):
|
||||
tools.pop(_GROUP_TOOL_NAME, None)
|
||||
|
||||
def _set_group_request_context(
|
||||
self,
|
||||
bundle: RuntimeBundle,
|
||||
message: InboundMessage,
|
||||
session_key: str,
|
||||
) -> object:
|
||||
metadata = getattr(bundle.engine, "tool_metadata", {})
|
||||
previous = metadata.get("ohmo_group_request", _NO_GROUP_REQUEST)
|
||||
if not message.metadata.get("_ohmo_group_command"):
|
||||
metadata.pop("ohmo_group_request", None)
|
||||
metadata.pop("_suppress_next_user_goal", None)
|
||||
self._unregister_group_tool(bundle)
|
||||
return _NO_GROUP_REQUEST
|
||||
self._register_group_tool(bundle)
|
||||
metadata["_suppress_next_user_goal"] = True
|
||||
metadata["ohmo_group_request"] = {
|
||||
"channel": message.channel,
|
||||
"chat_type": str(message.metadata.get("chat_type") or "").strip().lower(),
|
||||
"sender_id": str(message.sender_id),
|
||||
"source_chat_id": str(message.chat_id),
|
||||
"source_session_key": session_key,
|
||||
"sender_display_name": message.metadata.get("sender_display_name"),
|
||||
"raw_request": message.metadata.get("_ohmo_group_raw_request") or "",
|
||||
"used": False,
|
||||
}
|
||||
return previous
|
||||
|
||||
@staticmethod
|
||||
def _restore_group_request_context(bundle: RuntimeBundle, previous: object) -> None:
|
||||
metadata = getattr(bundle.engine, "tool_metadata", {})
|
||||
del previous
|
||||
metadata.pop("ohmo_group_request", None)
|
||||
metadata.pop("_suppress_next_user_goal", None)
|
||||
OhmoSessionRuntimePool._unregister_group_tool(bundle)
|
||||
|
||||
|
||||
def _content_snippet(text: str, *, limit: int = 160) -> str:
|
||||
"""Return a compact single-line preview for logs."""
|
||||
@@ -539,6 +794,148 @@ def _content_snippet(text: str, *, limit: int = 160) -> str:
|
||||
return normalized[: limit - 3] + "..."
|
||||
|
||||
|
||||
def _sanitize_snapshot_messages(raw_messages: object) -> list[dict[str, object]] | None:
|
||||
"""Validate and sanitize restored messages from persisted ohmo snapshots."""
|
||||
if not raw_messages or not isinstance(raw_messages, list):
|
||||
return None
|
||||
messages: list[ConversationMessage] = []
|
||||
for raw in raw_messages:
|
||||
try:
|
||||
messages.append(ConversationMessage.model_validate(raw))
|
||||
except Exception:
|
||||
logger.warning("ohmo runtime skipped invalid restored message while sanitizing snapshot")
|
||||
return [message.model_dump(mode="json") for message in _sanitize_group_command_prompts(messages)]
|
||||
|
||||
|
||||
def _extract_tool_media(event: ToolExecutionCompleted) -> list[str]:
|
||||
"""Return local media paths produced by a tool completion event."""
|
||||
if event.is_error or not isinstance(event.metadata, dict):
|
||||
return []
|
||||
raw_paths = event.metadata.get("paths") or event.metadata.get("media")
|
||||
if isinstance(raw_paths, str):
|
||||
candidates = [raw_paths]
|
||||
elif isinstance(raw_paths, list):
|
||||
candidates = [str(item) for item in raw_paths if isinstance(item, str) and item.strip()]
|
||||
else:
|
||||
candidates = []
|
||||
media: list[str] = []
|
||||
seen: set[str] = set()
|
||||
for raw in candidates:
|
||||
path = Path(raw).expanduser()
|
||||
if not path.is_absolute():
|
||||
path = path.resolve()
|
||||
if not path.is_file():
|
||||
continue
|
||||
resolved = str(path)
|
||||
if resolved not in seen:
|
||||
seen.add(resolved)
|
||||
media.append(resolved)
|
||||
return media
|
||||
|
||||
|
||||
def _remember_update_media(seen: set[str], update: GatewayStreamUpdate) -> None:
|
||||
"""Track media already emitted during this gateway turn."""
|
||||
raw_media = update.media or (update.metadata or {}).get("_media") or []
|
||||
if isinstance(raw_media, str):
|
||||
candidates = [raw_media]
|
||||
elif isinstance(raw_media, list):
|
||||
candidates = [str(item) for item in raw_media if isinstance(item, str) and item.strip()]
|
||||
else:
|
||||
candidates = []
|
||||
for raw in candidates:
|
||||
try:
|
||||
path = Path(raw).expanduser()
|
||||
if not path.is_absolute():
|
||||
path = path.resolve()
|
||||
seen.add(str(path))
|
||||
except Exception:
|
||||
continue
|
||||
|
||||
|
||||
def _extract_final_reply_media(reply: str, emitted_media: set[str]) -> list[str]:
|
||||
"""Return local image paths mentioned in final text that were not already emitted."""
|
||||
media: list[str] = []
|
||||
seen = set(emitted_media)
|
||||
for match in _FINAL_REPLY_IMAGE_PATH_RE.finditer(reply or ""):
|
||||
raw = match.group("path").strip(" \t\r\n\"'.,;:,。;:、)]}")
|
||||
if not raw:
|
||||
continue
|
||||
path = Path(raw).expanduser()
|
||||
if not path.is_absolute():
|
||||
continue
|
||||
if not path.is_file():
|
||||
continue
|
||||
resolved = str(path)
|
||||
if resolved in seen:
|
||||
continue
|
||||
seen.add(resolved)
|
||||
media.append(resolved)
|
||||
return media
|
||||
|
||||
|
||||
def _format_tool_media_caption(event: ToolExecutionCompleted, media: list[str]) -> str:
|
||||
"""Return a short caption for media generated by tools."""
|
||||
if event.tool_name == "image_generation":
|
||||
provider = ""
|
||||
if isinstance(event.metadata, dict):
|
||||
provider = str(event.metadata.get("provider") or "").strip()
|
||||
suffix = f" via {provider}" if provider else ""
|
||||
names = ", ".join(Path(path).name for path in media)
|
||||
return f"已生成图片{suffix}:{names}"
|
||||
names = ", ".join(Path(path).name for path in media)
|
||||
return f"已生成文件:{names}"
|
||||
|
||||
|
||||
def _sanitize_group_command_prompts(messages: list[ConversationMessage]) -> list[ConversationMessage]:
|
||||
"""Replace internal /group tool-driving prompts with durable user-facing history."""
|
||||
return [_sanitize_group_command_prompt(message) for message in messages]
|
||||
|
||||
|
||||
def _sanitize_group_command_prompt(message: ConversationMessage) -> ConversationMessage:
|
||||
changed = False
|
||||
content: list[TextBlock | ImageBlock] = []
|
||||
for block in message.content:
|
||||
if isinstance(block, TextBlock) and _GROUP_AGENT_PROMPT_PREFIX in block.text:
|
||||
content.append(TextBlock(text=_format_group_command_history_note(block.text)))
|
||||
changed = True
|
||||
else:
|
||||
content.append(block)
|
||||
if not changed:
|
||||
return message
|
||||
return message.model_copy(update={"content": content})
|
||||
|
||||
|
||||
def _format_group_command_history_note(prompt: str) -> str:
|
||||
raw_request = prompt
|
||||
if _GROUP_AGENT_PROMPT_REQUEST_MARKER in prompt:
|
||||
raw_request = prompt.split(_GROUP_AGENT_PROMPT_REQUEST_MARKER, 1)[1].strip()
|
||||
raw_request = raw_request.strip() or "(empty request)"
|
||||
return f"[Handled /group request]\nThe user asked ohmo to create a Feishu group:\n{raw_request}"
|
||||
|
||||
|
||||
def _sanitize_group_command_metadata(raw_metadata: object) -> object:
|
||||
"""Remove internal /group tool-driving text from compact carry-over metadata."""
|
||||
if not isinstance(raw_metadata, dict):
|
||||
return raw_metadata
|
||||
sanitized = dict(raw_metadata)
|
||||
for key in _GROUP_METADATA_KEYS:
|
||||
if key in sanitized:
|
||||
sanitized[key] = _sanitize_group_command_metadata_value(sanitized[key])
|
||||
return sanitized
|
||||
|
||||
|
||||
def _sanitize_group_command_metadata_value(value: object) -> object:
|
||||
if isinstance(value, str):
|
||||
if _GROUP_AGENT_PROMPT_PREFIX in value:
|
||||
return _format_group_command_history_note(value)
|
||||
return value
|
||||
if isinstance(value, dict):
|
||||
return {key: _sanitize_group_command_metadata_value(item) for key, item in value.items()}
|
||||
if isinstance(value, (list, tuple, set)):
|
||||
return [_sanitize_group_command_metadata_value(item) for item in value]
|
||||
return value
|
||||
|
||||
|
||||
def _summarize_tool_input(tool_name: str, tool_input: dict[str, object]) -> str:
|
||||
if not tool_input:
|
||||
return ""
|
||||
@@ -590,6 +987,10 @@ def _format_channel_progress(
|
||||
return "🛠️ " + text.replace("Using ", "正在使用 ", 1)
|
||||
return f"🛠️ {text}"
|
||||
return text if text.startswith("🛠️ ") else f"🛠️ {text}"
|
||||
if kind == "image_fallback":
|
||||
if prefers_chinese:
|
||||
return "🖼️ 当前模型不支持图片输入,我先改用附件路径和摘要继续。"
|
||||
return "🖼️ The active model does not support image input. I’ll retry with attachment paths and summaries."
|
||||
if kind == "status":
|
||||
normalized = text.strip()
|
||||
if normalized == "Auto-compacting conversation memory to keep things fast and focused.":
|
||||
@@ -667,6 +1068,60 @@ def _build_inbound_user_message(message: InboundMessage) -> ConversationMessage:
|
||||
return ConversationMessage.from_user_content(content)
|
||||
|
||||
|
||||
def _should_retry_without_image_input(error_message: str, messages: list[ConversationMessage]) -> bool:
|
||||
"""Return True when a provider rejects image input and history contains images."""
|
||||
if not _history_has_image_blocks(messages):
|
||||
return False
|
||||
normalized = error_message.lower()
|
||||
image_signal = any(
|
||||
phrase in normalized
|
||||
for phrase in (
|
||||
"image input",
|
||||
"image_url",
|
||||
"multimodal",
|
||||
"vision",
|
||||
"image content",
|
||||
)
|
||||
)
|
||||
rejection_signal = any(
|
||||
phrase in normalized
|
||||
for phrase in (
|
||||
"no endpoints found",
|
||||
"not support",
|
||||
"does not support",
|
||||
"unsupported",
|
||||
"cannot support",
|
||||
"can't support",
|
||||
)
|
||||
)
|
||||
return image_signal and rejection_signal
|
||||
|
||||
|
||||
def _history_has_image_blocks(messages: list[ConversationMessage]) -> bool:
|
||||
return any(any(isinstance(block, ImageBlock) for block in message.content) for message in messages)
|
||||
|
||||
|
||||
def _strip_image_blocks_from_engine_history(engine) -> None:
|
||||
messages = _strip_image_blocks_from_messages(list(engine.messages))
|
||||
if hasattr(engine, "load_messages"):
|
||||
engine.load_messages(messages)
|
||||
else:
|
||||
engine.messages = messages
|
||||
|
||||
|
||||
def _strip_image_blocks_from_messages(messages: list[ConversationMessage]) -> list[ConversationMessage]:
|
||||
return [_strip_image_blocks_from_message(message) for message in messages]
|
||||
|
||||
|
||||
def _strip_image_blocks_from_message(message: ConversationMessage) -> ConversationMessage:
|
||||
if not any(isinstance(block, ImageBlock) for block in message.content):
|
||||
return message
|
||||
content = [block for block in message.content if not isinstance(block, ImageBlock)]
|
||||
if not any(isinstance(block, TextBlock) for block in content):
|
||||
content.append(TextBlock(text=_IMAGE_FALLBACK_NOTE))
|
||||
return message.model_copy(update={"content": content})
|
||||
|
||||
|
||||
def _build_speaker_context(message: InboundMessage) -> str:
|
||||
"""Return a lightweight speaker header for group-chat messages."""
|
||||
metadata = message.metadata or {}
|
||||
|
||||
+163
-44
@@ -13,6 +13,9 @@ import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
if sys.platform == "win32":
|
||||
import ctypes
|
||||
|
||||
from openharness.channels.bus.events import OutboundMessage
|
||||
from openharness.channels.bus.queue import MessageBus
|
||||
from openharness.channels.impl.manager import ChannelManager
|
||||
@@ -49,10 +52,13 @@ class OhmoGatewayService:
|
||||
",".join(self._config.allowed_remote_admin_commands),
|
||||
)
|
||||
self._bus = MessageBus()
|
||||
self._manager = ChannelManager(build_channel_manager_config(self._config), self._bus)
|
||||
self._runtime_pool = OhmoSessionRuntimePool(
|
||||
cwd=self._cwd,
|
||||
workspace=self._workspace,
|
||||
provider_profile=self._config.provider_profile,
|
||||
create_feishu_group=self.create_group_for_user,
|
||||
publish_group_welcome=self.publish_group_welcome,
|
||||
)
|
||||
self._stop_event: asyncio.Event | None = None
|
||||
self._restart_requested = False
|
||||
@@ -60,8 +66,11 @@ class OhmoGatewayService:
|
||||
bus=self._bus,
|
||||
runtime_pool=self._runtime_pool,
|
||||
restart_gateway=self.request_restart,
|
||||
workspace=root,
|
||||
feishu_group_policy=str(
|
||||
self._config.channel_configs.get("feishu", {}).get("group_policy", "managed_or_mention")
|
||||
),
|
||||
)
|
||||
self._manager = ChannelManager(build_channel_manager_config(self._config), self._bus)
|
||||
|
||||
@property
|
||||
def pid_file(self) -> Path:
|
||||
@@ -75,6 +84,13 @@ class OhmoGatewayService:
|
||||
def state_file(self) -> Path:
|
||||
return get_state_path(self._workspace)
|
||||
|
||||
def _channel_last_error(self) -> str | None:
|
||||
for name, channel in self._manager.channels.items():
|
||||
error = getattr(channel, "last_error", None)
|
||||
if error:
|
||||
return f"{name}: {error}"
|
||||
return None
|
||||
|
||||
def write_state(self, *, running: bool, last_error: str | None = None) -> None:
|
||||
state = GatewayState(
|
||||
running=running,
|
||||
@@ -82,7 +98,7 @@ class OhmoGatewayService:
|
||||
active_sessions=self._runtime_pool.active_sessions,
|
||||
provider_profile=self._config.provider_profile,
|
||||
enabled_channels=self._config.enabled_channels,
|
||||
last_error=last_error,
|
||||
last_error=last_error or self._channel_last_error(),
|
||||
)
|
||||
self.state_file.write_text(state.model_dump_json(indent=2) + "\n", encoding="utf-8")
|
||||
|
||||
@@ -105,6 +121,34 @@ class OhmoGatewayService:
|
||||
if self._stop_event is not None:
|
||||
self._stop_event.set()
|
||||
|
||||
async def create_group(self, message, name: str) -> str:
|
||||
"""Create a managed group through the active channel implementation."""
|
||||
if message.channel != "feishu":
|
||||
raise RuntimeError(f"{message.channel} does not support managed group creation.")
|
||||
return await self.create_group_for_user(str(message.sender_id), name)
|
||||
|
||||
async def create_group_for_user(self, user_open_id: str, name: str) -> str:
|
||||
"""Create a managed Feishu group for a user open_id."""
|
||||
channel = self._manager.get_channel("feishu")
|
||||
if channel is None:
|
||||
raise RuntimeError("Feishu channel is not enabled.")
|
||||
creator = getattr(channel, "create_managed_group", None)
|
||||
if creator is None:
|
||||
raise RuntimeError("Feishu channel does not support managed group creation.")
|
||||
result = creator(user_open_id=str(user_open_id), name=name)
|
||||
return str(await result if asyncio.iscoroutine(result) else result)
|
||||
|
||||
async def publish_group_welcome(self, chat_id: str, content: str, owner_open_id: str) -> None:
|
||||
"""Send a welcome message to a newly created managed group."""
|
||||
await self._bus.publish_outbound(
|
||||
OutboundMessage(
|
||||
channel="feishu",
|
||||
chat_id=chat_id,
|
||||
content=content,
|
||||
metadata={"chat_type": "group", "_session_key": f"feishu:{chat_id}:{owner_open_id}"},
|
||||
)
|
||||
)
|
||||
|
||||
def _exec_restart(self) -> None:
|
||||
root = str(get_workspace_root(self._workspace))
|
||||
argv = [
|
||||
@@ -172,8 +216,18 @@ class OhmoGatewayService:
|
||||
with contextlib.suppress(NotImplementedError):
|
||||
loop.add_signal_handler(sig, _stop)
|
||||
|
||||
async def _state_heartbeat() -> None:
|
||||
while not stop_event.is_set():
|
||||
self.write_state(running=True)
|
||||
await asyncio.sleep(5.0)
|
||||
|
||||
state_task = asyncio.create_task(_state_heartbeat(), name="ohmo-gateway-state")
|
||||
|
||||
try:
|
||||
await stop_event.wait()
|
||||
except Exception as exc:
|
||||
self.write_state(running=False, last_error=str(exc))
|
||||
raise
|
||||
finally:
|
||||
self._bridge.stop()
|
||||
bridge_task.cancel()
|
||||
@@ -182,6 +236,10 @@ class OhmoGatewayService:
|
||||
await bridge_task
|
||||
with contextlib.suppress(asyncio.CancelledError):
|
||||
await manager_task
|
||||
if not state_task.done():
|
||||
state_task.cancel()
|
||||
with contextlib.suppress(asyncio.CancelledError):
|
||||
await state_task
|
||||
if not restart_notice_task.done():
|
||||
restart_notice_task.cancel()
|
||||
with contextlib.suppress(asyncio.CancelledError):
|
||||
@@ -205,7 +263,22 @@ def start_gateway_process(cwd: str | Path | None = None, workspace: str | Path |
|
||||
if existing_pythonpath:
|
||||
pythonpath_entries.append(existing_pythonpath)
|
||||
env["PYTHONPATH"] = os.pathsep.join(pythonpath_entries)
|
||||
|
||||
popen_kwargs: dict = {
|
||||
"cwd": service._cwd,
|
||||
"stdout": None,
|
||||
"stderr": None,
|
||||
"env": env,
|
||||
}
|
||||
if sys.platform == "win32":
|
||||
popen_kwargs["creationflags"] = subprocess.CREATE_NEW_PROCESS_GROUP | subprocess.DETACHED_PROCESS # type: ignore[attr-defined]
|
||||
popen_kwargs["stdin"] = subprocess.DEVNULL
|
||||
else:
|
||||
popen_kwargs["start_new_session"] = True
|
||||
|
||||
with service.log_file.open("a", encoding="utf-8") as log_file:
|
||||
popen_kwargs["stdout"] = log_file
|
||||
popen_kwargs["stderr"] = log_file
|
||||
process = subprocess.Popen(
|
||||
[
|
||||
sys.executable,
|
||||
@@ -217,56 +290,93 @@ def start_gateway_process(cwd: str | Path | None = None, workspace: str | Path |
|
||||
service._cwd,
|
||||
"--workspace",
|
||||
str(get_workspace_root(workspace)),
|
||||
"--no-console-log",
|
||||
],
|
||||
cwd=service._cwd,
|
||||
stdout=log_file,
|
||||
stderr=log_file,
|
||||
start_new_session=True,
|
||||
env=env,
|
||||
**popen_kwargs,
|
||||
)
|
||||
return process.pid
|
||||
|
||||
|
||||
def _pid_is_running(pid: int) -> bool:
|
||||
try:
|
||||
os.kill(pid, 0)
|
||||
except OSError:
|
||||
return False
|
||||
return True
|
||||
if sys.platform == "win32":
|
||||
kernel32 = ctypes.windll.kernel32 # type: ignore[attr-defined]
|
||||
PROCESS_QUERY_LIMITED_INFORMATION = 0x1000
|
||||
handle = kernel32.OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, False, pid)
|
||||
if not handle:
|
||||
return False
|
||||
try:
|
||||
exit_code = ctypes.c_ulong()
|
||||
if kernel32.GetExitCodeProcess(handle, ctypes.byref(exit_code)):
|
||||
return exit_code.value == 259 # STILL_ACTIVE
|
||||
return False
|
||||
finally:
|
||||
kernel32.CloseHandle(handle)
|
||||
else:
|
||||
try:
|
||||
os.kill(pid, 0)
|
||||
except OSError:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _iter_workspace_gateway_pids(workspace: str | Path | None = None) -> list[int]:
|
||||
root = str(get_workspace_root(workspace))
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["ps", "-eo", "pid=,args="],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
except Exception:
|
||||
return []
|
||||
|
||||
current_pid = os.getpid()
|
||||
pids: list[int] = []
|
||||
for line in result.stdout.splitlines():
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
if sys.platform == "win32":
|
||||
try:
|
||||
pid_text, args = line.split(None, 1)
|
||||
pid = int(pid_text)
|
||||
except ValueError:
|
||||
continue
|
||||
if pid == current_pid:
|
||||
continue
|
||||
if "-m ohmo gateway run" not in args:
|
||||
continue
|
||||
if f"--workspace {root}" not in args:
|
||||
continue
|
||||
if _pid_is_running(pid):
|
||||
pids.append(pid)
|
||||
return pids
|
||||
result = subprocess.run(
|
||||
["wmic", "process", "where",
|
||||
f"commandline like '%-m ohmo gateway run%' and commandline like '%--workspace {root}%'",
|
||||
"get", "processid"],
|
||||
capture_output=True, text=True, check=True,
|
||||
)
|
||||
except Exception:
|
||||
return []
|
||||
current_pid = os.getpid()
|
||||
pids: list[int] = []
|
||||
for line in result.stdout.splitlines():
|
||||
line = line.strip()
|
||||
if not line or line.lower() == "processid":
|
||||
continue
|
||||
try:
|
||||
pid = int(line)
|
||||
except ValueError:
|
||||
continue
|
||||
if pid == current_pid:
|
||||
continue
|
||||
if _pid_is_running(pid):
|
||||
pids.append(pid)
|
||||
return pids
|
||||
else:
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["ps", "-eo", "pid=,args="],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
except Exception:
|
||||
return []
|
||||
|
||||
current_pid = os.getpid()
|
||||
pids = []
|
||||
for line in result.stdout.splitlines():
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
try:
|
||||
pid_text, args = line.split(None, 1)
|
||||
pid = int(pid_text)
|
||||
except ValueError:
|
||||
continue
|
||||
if pid == current_pid:
|
||||
continue
|
||||
if "-m ohmo gateway run" not in args:
|
||||
continue
|
||||
if f"--workspace {root}" not in args:
|
||||
continue
|
||||
if _pid_is_running(pid):
|
||||
pids.append(pid)
|
||||
return pids
|
||||
|
||||
|
||||
def stop_gateway_process(cwd: str | Path | None = None, workspace: str | Path | None = None) -> bool:
|
||||
@@ -286,9 +396,18 @@ def stop_gateway_process(cwd: str | Path | None = None, workspace: str | Path |
|
||||
if not unique_pids:
|
||||
service.pid_file.unlink(missing_ok=True)
|
||||
return False
|
||||
for pid in unique_pids:
|
||||
with contextlib.suppress(ProcessLookupError):
|
||||
os.kill(pid, signal.SIGTERM)
|
||||
if sys.platform == "win32":
|
||||
for pid in unique_pids:
|
||||
with contextlib.suppress(Exception):
|
||||
subprocess.run(
|
||||
["taskkill", "/F", "/T", "/PID", str(pid)],
|
||||
capture_output=True,
|
||||
check=False,
|
||||
)
|
||||
else:
|
||||
for pid in unique_pids:
|
||||
with contextlib.suppress(ProcessLookupError):
|
||||
os.kill(pid, signal.SIGTERM)
|
||||
service.pid_file.unlink(missing_ok=True)
|
||||
service.write_state(running=False)
|
||||
return True
|
||||
|
||||
@@ -0,0 +1,95 @@
|
||||
"""Persistent metadata for ohmo-managed chat groups."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import asdict, dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
import json
|
||||
import re
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from ohmo.workspace import get_groups_dir
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ManagedGroupRecord:
|
||||
"""Metadata for a chat group created and managed by ohmo."""
|
||||
|
||||
channel: str
|
||||
chat_id: str
|
||||
owner_open_id: str
|
||||
name: str
|
||||
created_at: str
|
||||
cwd: str | None = None
|
||||
repo: str | None = None
|
||||
binding_status: str = "pending_agent"
|
||||
metadata: dict[str, Any] = field(default_factory=dict)
|
||||
|
||||
|
||||
def normalize_group_name(raw: str) -> str:
|
||||
"""Return a safe Feishu group name from command text."""
|
||||
name = " ".join(str(raw).strip().split())
|
||||
if not name:
|
||||
raise ValueError("Group name is required.")
|
||||
if len(name) > 100:
|
||||
raise ValueError("Group name is too long; keep it within 100 characters.")
|
||||
return name
|
||||
|
||||
|
||||
def group_record_path(
|
||||
*,
|
||||
workspace: str | Path | None,
|
||||
channel: str,
|
||||
chat_id: str,
|
||||
) -> Path:
|
||||
safe_chat_id = re.sub(r"[^A-Za-z0-9_.-]+", "_", str(chat_id)).strip("._") or "unknown"
|
||||
return get_groups_dir(workspace) / channel / f"{safe_chat_id}.json"
|
||||
|
||||
|
||||
def save_managed_group_record(
|
||||
*,
|
||||
workspace: str | Path | None,
|
||||
channel: str,
|
||||
chat_id: str,
|
||||
owner_open_id: str,
|
||||
name: str,
|
||||
cwd: str | None = None,
|
||||
repo: str | None = None,
|
||||
binding_status: str = "pending_agent",
|
||||
metadata: dict[str, Any] | None = None,
|
||||
) -> Path:
|
||||
"""Persist metadata for an ohmo-managed group and return the path."""
|
||||
record = ManagedGroupRecord(
|
||||
channel=channel,
|
||||
chat_id=chat_id,
|
||||
owner_open_id=owner_open_id,
|
||||
name=name,
|
||||
created_at=datetime.now(timezone.utc).isoformat(),
|
||||
cwd=normalize_cwd(cwd) if cwd else None,
|
||||
repo=repo,
|
||||
binding_status=binding_status,
|
||||
metadata=metadata or {},
|
||||
)
|
||||
path = group_record_path(workspace=workspace, channel=channel, chat_id=chat_id)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps(asdict(record), ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
return path
|
||||
|
||||
|
||||
def load_managed_group_record(
|
||||
*,
|
||||
workspace: str | Path | None,
|
||||
channel: str,
|
||||
chat_id: str,
|
||||
) -> dict[str, Any] | None:
|
||||
"""Load metadata for an ohmo-managed group if present."""
|
||||
path = group_record_path(workspace=workspace, channel=channel, chat_id=chat_id)
|
||||
if not path.exists():
|
||||
return None
|
||||
return json.loads(path.read_text(encoding="utf-8"))
|
||||
|
||||
|
||||
def normalize_cwd(cwd: str | Path) -> str:
|
||||
"""Normalize a cwd binding."""
|
||||
return str(Path(cwd).expanduser().resolve())
|
||||
+166
-16
@@ -5,13 +5,37 @@ from __future__ import annotations
|
||||
from pathlib import Path
|
||||
from re import sub
|
||||
|
||||
from openharness.commands import MemoryCommandBackend
|
||||
from openharness.memory.scan import scan_memory_files
|
||||
from openharness.memory.schema import (
|
||||
SCHEMA_VERSION,
|
||||
coerce_int,
|
||||
compute_memory_signature,
|
||||
first_content_line,
|
||||
format_datetime,
|
||||
generate_memory_id,
|
||||
memory_metadata_from_path,
|
||||
render_memory_file,
|
||||
split_memory_file,
|
||||
utc_now,
|
||||
)
|
||||
from openharness.utils.file_lock import exclusive_file_lock
|
||||
from openharness.utils.fs import atomic_write_text
|
||||
|
||||
from ohmo.workspace import get_memory_dir, get_memory_index_path
|
||||
|
||||
|
||||
def list_memory_files(workspace: str | Path | None = None) -> list[Path]:
|
||||
"""List ``.ohmo`` memory markdown files."""
|
||||
memory_dir = get_memory_dir(workspace)
|
||||
return sorted(path for path in memory_dir.glob("*.md") if path.name != "MEMORY.md")
|
||||
return sorted(
|
||||
header.path
|
||||
for header in scan_memory_files(
|
||||
_scan_cwd(workspace, memory_dir),
|
||||
max_files=None,
|
||||
memory_dir=memory_dir,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def add_memory_entry(workspace: str | Path | None, title: str, content: str) -> Path:
|
||||
@@ -19,30 +43,113 @@ def add_memory_entry(workspace: str | Path | None, title: str, content: str) ->
|
||||
memory_dir = get_memory_dir(workspace)
|
||||
memory_dir.mkdir(parents=True, exist_ok=True)
|
||||
slug = sub(r"[^a-zA-Z0-9]+", "_", title.strip().lower()).strip("_") or "memory"
|
||||
path = memory_dir / f"{slug}.md"
|
||||
path.write_text(content.strip() + "\n", encoding="utf-8")
|
||||
with exclusive_file_lock(memory_dir / ".memory.lock"):
|
||||
memory_type = "personal"
|
||||
category = "preference"
|
||||
body = content.strip() + "\n"
|
||||
signature = compute_memory_signature(body, memory_type, category)
|
||||
existing = scan_memory_files(
|
||||
_scan_cwd(workspace, memory_dir),
|
||||
max_files=None,
|
||||
include_disabled=True,
|
||||
include_expired=True,
|
||||
memory_dir=memory_dir,
|
||||
)
|
||||
duplicate = next(
|
||||
(header for header in existing if _effective_signature(header.path, header.signature) == signature),
|
||||
None,
|
||||
)
|
||||
path = duplicate.path if duplicate is not None else _next_memory_path(memory_dir, slug)
|
||||
now = utc_now()
|
||||
now_text = format_datetime(now)
|
||||
if path.exists():
|
||||
metadata, old_body, _, _ = split_memory_file(path.read_text(encoding="utf-8"))
|
||||
metadata = memory_metadata_from_path(
|
||||
path,
|
||||
metadata,
|
||||
old_body,
|
||||
now=now,
|
||||
source=str(metadata.get("source") or "manual"),
|
||||
default_type=memory_type,
|
||||
default_category=category,
|
||||
)
|
||||
created_at = str(metadata.get("created_at") or now_text)
|
||||
memory_id = str(metadata.get("id") or generate_memory_id(now))
|
||||
else:
|
||||
metadata = {}
|
||||
created_at = now_text
|
||||
memory_id = generate_memory_id(now)
|
||||
metadata.update(
|
||||
{
|
||||
"schema_version": SCHEMA_VERSION,
|
||||
"id": memory_id,
|
||||
"name": title.strip(),
|
||||
"description": first_content_line(body) or title.strip(),
|
||||
"type": str(metadata.get("type") or memory_type),
|
||||
"category": str(metadata.get("category") or category),
|
||||
"importance": max(coerce_int(metadata.get("importance"), default=0), 1),
|
||||
"source": "manual",
|
||||
"signature": signature,
|
||||
"created_at": created_at,
|
||||
"updated_at": now_text,
|
||||
"ttl_days": metadata.get("ttl_days"),
|
||||
"disabled": False,
|
||||
"supersedes": metadata.get("supersedes") or [],
|
||||
}
|
||||
)
|
||||
atomic_write_text(path, render_memory_file(metadata, body))
|
||||
|
||||
index_path = get_memory_index_path(workspace)
|
||||
existing = index_path.read_text(encoding="utf-8") if index_path.exists() else "# Memory Index\n"
|
||||
if path.name not in existing:
|
||||
existing = existing.rstrip() + f"\n- [{title}]({path.name})\n"
|
||||
index_path.write_text(existing, encoding="utf-8")
|
||||
index_path = get_memory_index_path(workspace)
|
||||
existing_index = index_path.read_text(encoding="utf-8") if index_path.exists() else "# Memory Index\n"
|
||||
if path.name not in existing_index:
|
||||
existing_index = existing_index.rstrip() + f"\n- [{title}]({path.name})\n"
|
||||
atomic_write_text(index_path, existing_index)
|
||||
return path
|
||||
|
||||
|
||||
def remove_memory_entry(workspace: str | Path | None, name: str) -> bool:
|
||||
"""Delete a memory file and remove its index entry."""
|
||||
"""Soft-delete a memory file and remove its index entry."""
|
||||
memory_dir = get_memory_dir(workspace)
|
||||
matches = [path for path in memory_dir.glob("*.md") if path.stem == name or path.name == name]
|
||||
matches = [
|
||||
header
|
||||
for header in scan_memory_files(
|
||||
_scan_cwd(workspace, memory_dir),
|
||||
max_files=None,
|
||||
include_disabled=True,
|
||||
include_expired=True,
|
||||
memory_dir=memory_dir,
|
||||
)
|
||||
if name in {header.path.stem, header.path.name, header.title, header.id}
|
||||
]
|
||||
if not matches:
|
||||
return False
|
||||
path = matches[0]
|
||||
path.unlink(missing_ok=True)
|
||||
header = matches[0]
|
||||
if header.disabled:
|
||||
return False
|
||||
path = header.path
|
||||
with exclusive_file_lock(memory_dir / ".memory.lock"):
|
||||
content = path.read_text(encoding="utf-8")
|
||||
metadata, body, _, _ = split_memory_file(content)
|
||||
metadata = memory_metadata_from_path(
|
||||
path,
|
||||
metadata,
|
||||
body,
|
||||
source="manual",
|
||||
default_type="personal",
|
||||
default_category="preference",
|
||||
)
|
||||
metadata["disabled"] = True
|
||||
metadata["updated_at"] = format_datetime(utc_now())
|
||||
atomic_write_text(path, render_memory_file(metadata, body))
|
||||
|
||||
index_path = get_memory_index_path(workspace)
|
||||
if index_path.exists():
|
||||
lines = [line for line in index_path.read_text(encoding="utf-8").splitlines() if path.name not in line]
|
||||
index_path.write_text("\n".join(lines).rstrip() + "\n", encoding="utf-8")
|
||||
index_path = get_memory_index_path(workspace)
|
||||
if index_path.exists():
|
||||
lines = [
|
||||
line
|
||||
for line in index_path.read_text(encoding="utf-8").splitlines()
|
||||
if path.name not in line
|
||||
]
|
||||
atomic_write_text(index_path, "\n".join(lines).rstrip() + "\n")
|
||||
return True
|
||||
|
||||
|
||||
@@ -67,3 +174,46 @@ def load_memory_prompt(workspace: str | Path | None = None, *, max_files: int =
|
||||
lines.extend(["", f"## {path.name}", "```md", content[:4000], "```"])
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def create_memory_command_backend(workspace: str | Path | None = None) -> MemoryCommandBackend:
|
||||
"""Return a ``/memory`` backend bound to ohmo's personal memory store."""
|
||||
|
||||
return MemoryCommandBackend(
|
||||
label="ohmo personal memory",
|
||||
default_type="personal",
|
||||
default_category="preference",
|
||||
get_memory_dir=lambda: get_memory_dir(workspace),
|
||||
get_entrypoint=lambda: get_memory_index_path(workspace),
|
||||
list_files=lambda: list_memory_files(workspace),
|
||||
add_entry=lambda title, content: add_memory_entry(workspace, title, content),
|
||||
remove_entry=lambda name: remove_memory_entry(workspace, name),
|
||||
)
|
||||
|
||||
|
||||
def _scan_cwd(workspace: str | Path | None, memory_dir: Path) -> Path:
|
||||
return Path(workspace) if workspace is not None else memory_dir.parent
|
||||
|
||||
|
||||
def _next_memory_path(memory_dir: Path, slug: str) -> Path:
|
||||
path = memory_dir / f"{slug}.md"
|
||||
if not path.exists():
|
||||
return path
|
||||
index = 2
|
||||
while True:
|
||||
candidate = memory_dir / f"{slug}_{index}.md"
|
||||
if not candidate.exists():
|
||||
return candidate
|
||||
index += 1
|
||||
|
||||
|
||||
def _effective_signature(path: Path, existing_signature: str) -> str:
|
||||
if existing_signature:
|
||||
return existing_signature
|
||||
try:
|
||||
metadata, body, _, _ = split_memory_file(path.read_text(encoding="utf-8"))
|
||||
except OSError:
|
||||
return ""
|
||||
memory_type = str(metadata.get("type") or "personal")
|
||||
category = str(metadata.get("category") or "preference")
|
||||
return compute_memory_signature(body, memory_type, category)
|
||||
|
||||
+23
-2
@@ -14,9 +14,10 @@ from openharness.ui.backend_host import run_backend_host
|
||||
from openharness.ui.runtime import build_runtime, close_runtime, handle_line, start_runtime
|
||||
from openharness.ui.react_launcher import _resolve_npm, _resolve_tsx, get_frontend_dir
|
||||
|
||||
from ohmo.memory import create_memory_command_backend
|
||||
from ohmo.prompts import build_ohmo_system_prompt
|
||||
from ohmo.session_storage import OhmoSessionBackend
|
||||
from ohmo.workspace import get_plugins_dir, get_skills_dir, initialize_workspace
|
||||
from ohmo.workspace import get_memory_dir, get_plugins_dir, get_sessions_dir, get_skills_dir, initialize_workspace
|
||||
|
||||
|
||||
def _ohmo_extra_roots(workspace: str | Path | None) -> tuple[tuple[str, ...], tuple[str, ...]]:
|
||||
@@ -54,6 +55,14 @@ async def run_ohmo_backend(
|
||||
session_backend=OhmoSessionBackend(workspace_root),
|
||||
extra_skill_dirs=extra_skill_dirs,
|
||||
extra_plugin_roots=extra_plugin_roots,
|
||||
memory_backend=create_memory_command_backend(workspace_root),
|
||||
include_project_memory=False,
|
||||
autodream_context={
|
||||
"memory_dir": str(get_memory_dir(workspace_root)),
|
||||
"session_dir": str(get_sessions_dir(workspace_root)),
|
||||
"app_label": "ohmo personal memory",
|
||||
"runner_module": "ohmo",
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
@@ -160,13 +169,24 @@ async def run_ohmo_print_mode(
|
||||
enforce_max_turns=max_turns is not None,
|
||||
extra_skill_dirs=extra_skill_dirs,
|
||||
extra_plugin_roots=extra_plugin_roots,
|
||||
memory_backend=create_memory_command_backend(workspace_root),
|
||||
include_project_memory=False,
|
||||
autodream_context={
|
||||
"memory_dir": str(get_memory_dir(workspace_root)),
|
||||
"session_dir": str(get_sessions_dir(workspace_root)),
|
||||
"app_label": "ohmo personal memory",
|
||||
"runner_module": "ohmo",
|
||||
},
|
||||
)
|
||||
await start_runtime(bundle)
|
||||
|
||||
async def _print_system(message: str) -> None:
|
||||
print(message, file=sys.stderr)
|
||||
|
||||
saw_error = False
|
||||
|
||||
async def _render_event(event) -> None:
|
||||
nonlocal saw_error
|
||||
if isinstance(event, AssistantTextDelta):
|
||||
sys.stdout.write(event.text)
|
||||
sys.stdout.flush()
|
||||
@@ -174,6 +194,7 @@ async def run_ohmo_print_mode(
|
||||
sys.stdout.write("\n")
|
||||
sys.stdout.flush()
|
||||
elif isinstance(event, ErrorEvent):
|
||||
saw_error = True
|
||||
print(event.message, file=sys.stderr)
|
||||
elif isinstance(event, CompactProgressEvent):
|
||||
if event.message:
|
||||
@@ -192,6 +213,6 @@ async def run_ohmo_print_mode(
|
||||
clear_output=_clear_output,
|
||||
)
|
||||
await close_runtime(bundle)
|
||||
return 0
|
||||
return 1 if saw_error else 0
|
||||
finally:
|
||||
os.chdir(previous_cwd)
|
||||
|
||||
@@ -203,6 +203,10 @@ def get_plugins_dir(workspace: str | Path | None = None) -> Path:
|
||||
return get_workspace_root(workspace) / "plugins"
|
||||
|
||||
|
||||
def get_groups_dir(workspace: str | Path | None = None) -> Path:
|
||||
return get_workspace_root(workspace) / "groups"
|
||||
|
||||
|
||||
def get_memory_index_path(workspace: str | Path | None = None) -> Path:
|
||||
return get_memory_dir(workspace) / "MEMORY.md"
|
||||
|
||||
@@ -238,6 +242,7 @@ def ensure_workspace(workspace: str | Path | None = None) -> Path:
|
||||
get_memory_dir(root).mkdir(parents=True, exist_ok=True)
|
||||
get_skills_dir(root).mkdir(parents=True, exist_ok=True)
|
||||
get_plugins_dir(root).mkdir(parents=True, exist_ok=True)
|
||||
get_groups_dir(root).mkdir(parents=True, exist_ok=True)
|
||||
get_sessions_dir(root).mkdir(parents=True, exist_ok=True)
|
||||
get_logs_dir(root).mkdir(parents=True, exist_ok=True)
|
||||
get_attachments_dir(root).mkdir(parents=True, exist_ok=True)
|
||||
@@ -307,6 +312,7 @@ def workspace_health(workspace: str | Path | None = None) -> dict[str, bool]:
|
||||
"memory_dir": get_memory_dir(root).exists(),
|
||||
"skills_dir": get_skills_dir(root).exists(),
|
||||
"plugins_dir": get_plugins_dir(root).exists(),
|
||||
"groups_dir": get_groups_dir(root).exists(),
|
||||
"memory_index": get_memory_index_path(root).exists(),
|
||||
"sessions_dir": get_sessions_dir(root).exists(),
|
||||
"gateway_config": get_gateway_config_path(root).exists(),
|
||||
|
||||
+1
-1
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
||||
|
||||
[project]
|
||||
name = "openharness-ai"
|
||||
version = "0.1.7"
|
||||
version = "0.1.9"
|
||||
description = "Open-source Python port of Claude Code - an AI-powered CLI coding assistant"
|
||||
readme = "README.md"
|
||||
license = "MIT"
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
"""Run the OpenHarness memory schema migration from a source checkout."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
|
||||
|
||||
from openharness.memory.migrate import main
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -45,6 +45,7 @@ class ApiMessageRequest:
|
||||
system_prompt: str | None = None
|
||||
max_tokens: int = 4096
|
||||
tools: list[dict[str, Any]] = field(default_factory=list)
|
||||
effort: str | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -149,6 +150,10 @@ class AnthropicApiClient:
|
||||
kwargs["base_url"] = self._base_url
|
||||
return AsyncAnthropic(**kwargs)
|
||||
|
||||
async def close(self) -> None:
|
||||
"""Close the underlying HTTP client."""
|
||||
await self._client.close()
|
||||
|
||||
def _refresh_client_auth(self) -> None:
|
||||
if not self._claude_oauth or self._auth_token_resolver is None:
|
||||
return
|
||||
|
||||
@@ -78,6 +78,17 @@ def _convert_messages_to_codex(messages: list[ConversationMessage]) -> list[dict
|
||||
result: list[dict[str, Any]] = []
|
||||
for msg in messages:
|
||||
if msg.role == "user":
|
||||
# Responses API requires function_call_output items to appear before
|
||||
# any following user input. A ConversationMessage can contain both
|
||||
# tool results and user text, so emit the tool outputs first to keep
|
||||
# every prior function_call immediately satisfied.
|
||||
for block in msg.content:
|
||||
if isinstance(block, ToolResultBlock):
|
||||
result.append({
|
||||
"type": "function_call_output",
|
||||
"call_id": block.tool_use_id,
|
||||
"output": block.content,
|
||||
})
|
||||
user_content: list[dict[str, Any]] = []
|
||||
for block in msg.content:
|
||||
if isinstance(block, TextBlock) and block.text.strip():
|
||||
@@ -89,13 +100,6 @@ def _convert_messages_to_codex(messages: list[ConversationMessage]) -> list[dict
|
||||
})
|
||||
if user_content:
|
||||
result.append({"role": "user", "content": user_content})
|
||||
for block in msg.content:
|
||||
if isinstance(block, ToolResultBlock):
|
||||
result.append({
|
||||
"type": "function_call_output",
|
||||
"call_id": block.tool_use_id,
|
||||
"output": block.content,
|
||||
})
|
||||
continue
|
||||
|
||||
assistant_text = "".join(block.text for block in msg.content if isinstance(block, TextBlock))
|
||||
@@ -129,6 +133,15 @@ def _convert_tools_to_codex(tools: list[dict[str, Any]]) -> list[dict[str, Any]]
|
||||
]
|
||||
|
||||
|
||||
def _normalize_reasoning_effort(effort: str | None) -> str | None:
|
||||
normalized = (effort or "").strip().lower()
|
||||
if normalized == "max":
|
||||
return "xhigh"
|
||||
if normalized in {"low", "medium", "high", "xhigh"}:
|
||||
return normalized
|
||||
return None
|
||||
|
||||
|
||||
def _usage_from_response(response: dict[str, Any]) -> UsageSnapshot:
|
||||
usage = response.get("usage")
|
||||
if not isinstance(usage, dict):
|
||||
@@ -251,6 +264,9 @@ class CodexApiClient:
|
||||
}
|
||||
if request.tools:
|
||||
body["tools"] = _convert_tools_to_codex(request.tools)
|
||||
effort = _normalize_reasoning_effort(request.effort)
|
||||
if effort:
|
||||
body["reasoning"] = {"effort": effort}
|
||||
|
||||
content: list[TextBlock | ToolUseBlock] = []
|
||||
current_text_parts: list[str] = []
|
||||
|
||||
@@ -128,3 +128,7 @@ class CopilotClient:
|
||||
)
|
||||
async for event in self._inner.stream_message(patched):
|
||||
yield event
|
||||
|
||||
async def close(self) -> None:
|
||||
"""Close the underlying OpenAI-compatible client."""
|
||||
await self._inner.close()
|
||||
|
||||
@@ -5,6 +5,8 @@ from __future__ import annotations
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
from typing import Any, AsyncIterator
|
||||
from urllib.parse import urlsplit, urlunsplit
|
||||
|
||||
@@ -144,13 +146,43 @@ def _convert_user_content_to_openai(blocks: list[ContentBlock]) -> str | list[di
|
||||
return content
|
||||
|
||||
|
||||
_EMPTY_REASONING_ENV = "OPENHARNESS_REQUIRE_EMPTY_REASONING_CONTENT"
|
||||
|
||||
|
||||
def _empty_reasoning_required() -> bool:
|
||||
"""True when the operator's provider requires an empty
|
||||
``reasoning_content`` field on tool-using assistant messages
|
||||
(Kimi-on-Anthropic style). Default off — strict-OpenAI providers
|
||||
reject the field outright.
|
||||
"""
|
||||
raw = os.environ.get(_EMPTY_REASONING_ENV, "").strip().lower()
|
||||
return raw in {"1", "true", "yes", "on"}
|
||||
|
||||
|
||||
def _convert_assistant_message(msg: ConversationMessage) -> dict[str, Any]:
|
||||
"""Convert an assistant ConversationMessage to OpenAI format.
|
||||
|
||||
Providers with thinking models (e.g. Kimi k2.5) require a
|
||||
``reasoning_content`` field on every assistant message that contains
|
||||
tool calls. We stash the raw reasoning text on ``msg._reasoning``
|
||||
during parsing and replay it here.
|
||||
``reasoning_content`` is a non-standard field used by thinking models
|
||||
(e.g. Kimi k2.5) to carry the model's internal chain-of-thought across
|
||||
turns. Some thinking-model providers require it on every assistant
|
||||
message with tool calls — even when empty — or they reject the request.
|
||||
Other OpenAI-compatible providers (Cerebras, OpenAI's own
|
||||
endpoint, etc.) reject the field outright with a 400
|
||||
``wrong_api_format`` error.
|
||||
|
||||
Behaviour:
|
||||
|
||||
- When the streaming parser captured non-empty reasoning on
|
||||
``msg._reasoning``, we always replay it. Models that emit reasoning
|
||||
tokens are by definition thinking models that round-trip them.
|
||||
- When there is no captured reasoning but the message has tool calls,
|
||||
we emit ``reasoning_content: ""`` only if the operator opts in via
|
||||
``OPENHARNESS_REQUIRE_EMPTY_REASONING_CONTENT=1``. The default is
|
||||
omit, which matches strict-OpenAI providers.
|
||||
|
||||
The opt-in default keeps strict-OpenAI providers (Cerebras, NVIDIA NIM,
|
||||
OpenAI direct, etc.) working out-of-the-box; Kimi-on-Anthropic users
|
||||
set the env var in their dotfiles or settings.
|
||||
"""
|
||||
text_parts = [b.text for b in msg.content if isinstance(b, TextBlock)]
|
||||
tool_uses = [b for b in msg.content if isinstance(b, ToolUseBlock)]
|
||||
@@ -164,8 +196,9 @@ def _convert_assistant_message(msg: ConversationMessage) -> dict[str, Any]:
|
||||
reasoning = getattr(msg, "_reasoning", None)
|
||||
if reasoning:
|
||||
openai_msg["reasoning_content"] = reasoning
|
||||
elif tool_uses:
|
||||
# Thinking models require this field even if empty
|
||||
elif tool_uses and _empty_reasoning_required():
|
||||
# Kimi-style providers reject tool_use messages without this field
|
||||
# even when there's nothing to put in it. Opt-in via env var.
|
||||
openai_msg["reasoning_content"] = ""
|
||||
|
||||
if tool_uses:
|
||||
@@ -232,7 +265,10 @@ class OpenAICompatibleClient:
|
||||
"""
|
||||
|
||||
def __init__(self, api_key: str, *, base_url: str | None = None, timeout: float | None = None) -> None:
|
||||
kwargs: dict[str, Any] = {"api_key": api_key}
|
||||
kwargs: dict[str, Any] = {
|
||||
"api_key": api_key,
|
||||
"default_headers": {"Authorization": f"Bearer {api_key}"},
|
||||
}
|
||||
normalized_base_url = _normalize_openai_base_url(base_url)
|
||||
if normalized_base_url:
|
||||
kwargs["base_url"] = normalized_base_url
|
||||
@@ -240,6 +276,10 @@ class OpenAICompatibleClient:
|
||||
kwargs["timeout"] = timeout
|
||||
self._client = AsyncOpenAI(**kwargs)
|
||||
|
||||
async def close(self) -> None:
|
||||
"""Close the underlying HTTP client."""
|
||||
await self._client.close()
|
||||
|
||||
async def stream_message(self, request: ApiMessageRequest) -> AsyncIterator[ApiStreamEvent]:
|
||||
"""Yield text deltas and the final message, matching the Anthropic client interface."""
|
||||
last_error: Exception | None = None
|
||||
@@ -298,6 +338,8 @@ class OpenAICompatibleClient:
|
||||
collected_tool_calls: dict[int, dict[str, Any]] = {}
|
||||
finish_reason: str | None = None
|
||||
usage_data: dict[str, int] = {}
|
||||
# Buffer to strip inline <think>…</think> blocks across streaming chunks.
|
||||
_think_buf = ""
|
||||
|
||||
stream = await self._client.chat.completions.create(**params)
|
||||
async for chunk in stream:
|
||||
@@ -321,10 +363,13 @@ class OpenAICompatibleClient:
|
||||
if reasoning_piece:
|
||||
collected_reasoning += reasoning_piece
|
||||
|
||||
# Stream text content to user
|
||||
# Stream text content to user, stripping inline <think> blocks
|
||||
if delta.content:
|
||||
collected_content += delta.content
|
||||
yield ApiTextDeltaEvent(text=delta.content)
|
||||
_think_buf += delta.content
|
||||
visible, _think_buf = _strip_think_blocks(_think_buf)
|
||||
if visible:
|
||||
collected_content += visible
|
||||
yield ApiTextDeltaEvent(text=visible)
|
||||
|
||||
# Accumulate tool calls
|
||||
if delta.tool_calls:
|
||||
@@ -406,3 +451,34 @@ class OpenAICompatibleClient:
|
||||
if status == 429:
|
||||
return RateLimitFailure(msg)
|
||||
return RequestFailure(msg)
|
||||
|
||||
|
||||
# Matches complete <think>…</think> blocks (DOTALL so newlines are included).
|
||||
_THINK_RE = re.compile(r"<think>.*?</think>", re.DOTALL)
|
||||
_THINK_OPEN_TAG = "<think>"
|
||||
|
||||
|
||||
def _strip_think_blocks(buf: str) -> tuple[str, str]:
|
||||
"""Strip complete ``<think>…</think>`` blocks and return ``(visible_text, leftover)``.
|
||||
|
||||
Complete pairs are removed via regex. An unclosed ``<think>`` is held in
|
||||
*leftover* so it can be re-evaluated once the closing tag arrives in the
|
||||
next streaming chunk.
|
||||
"""
|
||||
# Remove fully-closed blocks.
|
||||
cleaned = _THINK_RE.sub("", buf)
|
||||
|
||||
# Hold back any unclosed <think> for the next chunk.
|
||||
open_idx = cleaned.find(_THINK_OPEN_TAG)
|
||||
if open_idx != -1:
|
||||
return cleaned[:open_idx], cleaned[open_idx:]
|
||||
|
||||
# Streaming providers may split the opening tag itself across chunk
|
||||
# boundaries (e.g. ``"<thi"`` then ``"nk>..."``). Hold back the longest
|
||||
# suffix that could still become ``<think>`` on the next chunk.
|
||||
max_prefix = min(len(cleaned), len(_THINK_OPEN_TAG) - 1)
|
||||
for prefix_len in range(max_prefix, 0, -1):
|
||||
if _THINK_OPEN_TAG.startswith(cleaned[-prefix_len:]):
|
||||
return cleaned[:-prefix_len], cleaned[-prefix_len:]
|
||||
|
||||
return cleaned, ""
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
|
||||
from openharness.auth.external import describe_external_binding
|
||||
@@ -123,3 +124,63 @@ def auth_status(settings: Settings) -> str:
|
||||
if resolved.source.startswith("external:"):
|
||||
return f"configured ({resolved.source.removeprefix('external:')})"
|
||||
return "configured"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Multimodal (vision) capability detection
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Known multimodal model patterns (lowercase, regex).
|
||||
# These models can accept image input natively.
|
||||
_MULTIMODAL_MODEL_PATTERNS: list[re.Pattern[str]] = [
|
||||
# Anthropic Claude 3+ (all Claude 3 and later support images)
|
||||
re.compile(r"^claude-3(?:\.\d+)?(?:-sonnet|-opus|-haiku)?"),
|
||||
re.compile(r"^claude-(?:sonnet|opus|haiku)-\d"),
|
||||
# OpenAI GPT-4o / o-series
|
||||
re.compile(r"^gpt-4o"),
|
||||
re.compile(r"^o[1349]-"),
|
||||
# Google Gemini
|
||||
re.compile(r"^gemini-(?:pro-)?vision"),
|
||||
re.compile(r"^gemini-2\.\d+"),
|
||||
# Qwen / DashScope VL series
|
||||
re.compile(r"^qwen-vl"),
|
||||
re.compile(r"^qwen2\.5?-vl"),
|
||||
re.compile(r"^qvq-"),
|
||||
# DeepSeek VL
|
||||
re.compile(r"^deepseek-vl"),
|
||||
re.compile(r"^deepseek-vision"),
|
||||
# Open-source multimodal
|
||||
re.compile(r"^llava"),
|
||||
re.compile(r"^cogvlm"),
|
||||
re.compile(r"^internvl"),
|
||||
re.compile(r"^glm-4v"),
|
||||
# Moonshot / Kimi (k2.5 supports images)
|
||||
re.compile(r"^kimi-k2\.5"),
|
||||
# StepFun (阶跃星辰) — Step-2 and Step-1v support images
|
||||
re.compile(r"^step-2"),
|
||||
re.compile(r"^step-1v"),
|
||||
# MiniMax VL
|
||||
re.compile(r"^minimax-vl"),
|
||||
# Zhipu GLM-4V
|
||||
re.compile(r"^glm-4v"),
|
||||
# Mistral Pixtral
|
||||
re.compile(r"^pixtral"),
|
||||
# Groq vision models (llama-3.2-vision, etc.)
|
||||
re.compile(r"vision"),
|
||||
# Generic: model names containing "vl" or "vision" as a word boundary
|
||||
re.compile(r"(?:^|[-\s/])vl(?:$|[-\s])"),
|
||||
]
|
||||
|
||||
|
||||
def is_model_multimodal(model: str) -> bool:
|
||||
"""Return True when the model name indicates multimodal (vision) capability.
|
||||
|
||||
This is a heuristic based on known model naming conventions. It errs on
|
||||
the side of returning False for unknown models so that the image-to-text
|
||||
fallback tool is used rather than silently failing.
|
||||
"""
|
||||
normalized = model.strip().lower()
|
||||
# Strip provider prefix like "anthropic/" or "openai/"
|
||||
if "/" in normalized:
|
||||
normalized = normalized.split("/", 1)[-1]
|
||||
return any(pattern.search(normalized) is not None for pattern in _MULTIMODAL_MODEL_PATTERNS)
|
||||
|
||||
@@ -124,6 +124,20 @@ PROVIDERS: tuple[ProviderSpec, ...] = (
|
||||
is_local=False,
|
||||
is_oauth=False,
|
||||
),
|
||||
# ModelScope: OpenAI-compatible inference API
|
||||
ProviderSpec(
|
||||
name="modelscope",
|
||||
keywords=("modelscope",),
|
||||
env_key="MODELSCOPE_API_KEY",
|
||||
display_name="ModelScope",
|
||||
backend_type="openai_compat",
|
||||
default_base_url="https://api-inference.modelscope.cn/v1",
|
||||
detect_by_key_prefix="",
|
||||
detect_by_base_keyword="modelscope",
|
||||
is_gateway=False,
|
||||
is_local=False,
|
||||
is_oauth=False,
|
||||
),
|
||||
# === Standard cloud providers (matched by model-name keyword) ============
|
||||
# Anthropic: native SDK for claude-* models
|
||||
ProviderSpec(
|
||||
|
||||
@@ -7,11 +7,13 @@ performs the interactive authentication and returns the obtained credential.
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import platform
|
||||
import subprocess
|
||||
import sys
|
||||
from abc import ABC, abstractmethod
|
||||
from typing import Any
|
||||
from urllib.parse import urlparse
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
@@ -76,18 +78,26 @@ class DeviceCodeFlow(AuthFlow):
|
||||
@staticmethod
|
||||
def _try_open_browser(url: str) -> bool:
|
||||
"""Attempt to open *url* in the default browser; return True if likely succeeded."""
|
||||
# Only http(s) URLs are valid here. ShellExecute / xdg-open / `open`
|
||||
# all resolve unrecognised tokens (e.g. ``file:``, ``javascript:``, a
|
||||
# bare executable name) against the registry or PATH, so refusing
|
||||
# everything else removes a class of unintended-action footguns.
|
||||
if urlparse(url).scheme not in {"http", "https"}:
|
||||
return False
|
||||
try:
|
||||
plat = platform.system()
|
||||
if plat == "Darwin":
|
||||
subprocess.Popen(["open", url], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||
return True
|
||||
if plat == "Windows":
|
||||
subprocess.Popen(
|
||||
["start", "", url],
|
||||
shell=True,
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
)
|
||||
# ShellExecute via os.startfile — does NOT route through
|
||||
# cmd.exe, so ``&`` / ``|`` / ``^`` in the URL cannot be
|
||||
# interpreted as command separators. Replaces the prior
|
||||
# ``subprocess.Popen([...], shell=True)`` form, which would
|
||||
# execute appended commands when a hostile or compromised
|
||||
# device-flow endpoint returned a URL like
|
||||
# ``https://x.com&calc.exe``.
|
||||
os.startfile(url) # type: ignore[attr-defined]
|
||||
return True
|
||||
# Linux / WSL
|
||||
proc = subprocess.Popen(
|
||||
@@ -102,6 +112,7 @@ class DeviceCodeFlow(AuthFlow):
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
return False
|
||||
|
||||
def run(self) -> str:
|
||||
from openharness.api.copilot_auth import poll_for_access_token, request_device_code
|
||||
|
||||
@@ -37,6 +37,7 @@ _KNOWN_PROVIDERS = [
|
||||
"moonshot",
|
||||
"gemini",
|
||||
"minimax",
|
||||
"modelscope",
|
||||
]
|
||||
|
||||
_AUTH_SOURCES = [
|
||||
@@ -51,6 +52,7 @@ _AUTH_SOURCES = [
|
||||
"moonshot_api_key",
|
||||
"gemini_api_key",
|
||||
"minimax_api_key",
|
||||
"modelscope_api_key",
|
||||
]
|
||||
|
||||
_PROFILE_BY_PROVIDER = {
|
||||
@@ -62,6 +64,7 @@ _PROFILE_BY_PROVIDER = {
|
||||
"moonshot": "moonshot",
|
||||
"gemini": "gemini",
|
||||
"minimax": "minimax",
|
||||
"modelscope": "modelscope",
|
||||
}
|
||||
|
||||
|
||||
@@ -157,6 +160,15 @@ class AuthManager:
|
||||
configured = True
|
||||
origin = "file"
|
||||
state = "configured"
|
||||
elif source == "modelscope_api_key":
|
||||
if os.environ.get("MODELSCOPE_API_KEY"):
|
||||
configured = True
|
||||
origin = "env"
|
||||
state = "configured"
|
||||
elif load_credential(storage_provider, "api_key"):
|
||||
configured = True
|
||||
origin = "file"
|
||||
state = "configured"
|
||||
elif load_credential(storage_provider, "api_key"):
|
||||
configured = True
|
||||
origin = "file"
|
||||
@@ -253,6 +265,14 @@ class AuthManager:
|
||||
configured = True
|
||||
source = "file"
|
||||
|
||||
elif provider == "modelscope":
|
||||
if os.environ.get("MODELSCOPE_API_KEY"):
|
||||
configured = True
|
||||
source = "env"
|
||||
elif load_credential("modelscope", "api_key"):
|
||||
configured = True
|
||||
source = "file"
|
||||
|
||||
elif provider in ("bedrock", "vertex"):
|
||||
# These typically use environment-level credentials (AWS/GCP).
|
||||
cred = load_credential(provider, "api_key")
|
||||
|
||||
@@ -5,9 +5,11 @@ from __future__ import annotations
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import shlex
|
||||
import subprocess
|
||||
import tempfile
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from hashlib import sha1
|
||||
from html import escape
|
||||
from pathlib import Path
|
||||
@@ -94,11 +96,14 @@ _DEFAULT_VERIFICATION_POLICY = {
|
||||
"commands": [
|
||||
"uv run pytest -q",
|
||||
"uv run ruff check src tests scripts",
|
||||
(
|
||||
"cd frontend/terminal && "
|
||||
"([ -x ./node_modules/.bin/tsc ] || npm ci --no-audit --no-fund) && "
|
||||
"./node_modules/.bin/tsc --noEmit"
|
||||
),
|
||||
{
|
||||
"command": (
|
||||
"cd frontend/terminal && "
|
||||
"([ -x ./node_modules/.bin/tsc ] || npm ci --no-audit --no-fund) && "
|
||||
"./node_modules/.bin/tsc --noEmit"
|
||||
),
|
||||
"shell": True,
|
||||
},
|
||||
],
|
||||
"require_tests_before_merge": True,
|
||||
}
|
||||
@@ -128,6 +133,69 @@ def _json_default(value: object) -> object:
|
||||
return str(value)
|
||||
|
||||
|
||||
_SHELL_METACHARS = frozenset(";&|`$<>\n\r")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _VerificationCommand:
|
||||
"""Parsed verification-policy entry.
|
||||
|
||||
When ``shell`` is false, ``argv`` is executed with ``shell=False``.
|
||||
When ``shell`` is true, ``raw`` is handed to the shell (explicit opt-in).
|
||||
``error`` signals a policy entry that must not be executed; callers emit
|
||||
an error step so the verification gate fails loudly.
|
||||
"""
|
||||
|
||||
raw: str
|
||||
argv: tuple[str, ...]
|
||||
shell: bool
|
||||
error: str | None = None
|
||||
|
||||
|
||||
def _parse_verification_entry(entry: object) -> _VerificationCommand:
|
||||
if isinstance(entry, dict):
|
||||
raw = str(entry.get("command", "")).strip()
|
||||
if not raw:
|
||||
return _VerificationCommand(raw=str(entry), argv=(), shell=False, error="empty command")
|
||||
if bool(entry.get("shell", False)):
|
||||
return _VerificationCommand(raw=raw, argv=(), shell=True)
|
||||
# fall through and validate as an argv-form command
|
||||
elif isinstance(entry, str):
|
||||
raw = entry.strip()
|
||||
if not raw:
|
||||
return _VerificationCommand(raw=entry, argv=(), shell=False, error="empty command")
|
||||
else:
|
||||
return _VerificationCommand(
|
||||
raw=str(entry),
|
||||
argv=(),
|
||||
shell=False,
|
||||
error="entry must be a string or a mapping with a 'command' key",
|
||||
)
|
||||
|
||||
if any(ch in _SHELL_METACHARS for ch in raw):
|
||||
return _VerificationCommand(
|
||||
raw=raw,
|
||||
argv=(),
|
||||
shell=False,
|
||||
error=(
|
||||
"command contains shell metacharacters; use the mapping form "
|
||||
"{command: '...', shell: true} in verification_policy.yaml to opt in"
|
||||
),
|
||||
)
|
||||
try:
|
||||
argv = shlex.split(raw)
|
||||
except ValueError as exc:
|
||||
return _VerificationCommand(
|
||||
raw=raw,
|
||||
argv=(),
|
||||
shell=False,
|
||||
error=f"could not tokenize command: {exc}",
|
||||
)
|
||||
if not argv:
|
||||
return _VerificationCommand(raw=raw, argv=(), shell=False, error="empty command")
|
||||
return _VerificationCommand(raw=raw, argv=tuple(argv), shell=False)
|
||||
|
||||
|
||||
def _looks_available(command: str, cwd: Path) -> bool:
|
||||
lowered = command.lower()
|
||||
if lowered.startswith("uv "):
|
||||
@@ -2011,19 +2079,37 @@ class RepoAutopilotStore:
|
||||
await close_runtime(bundle)
|
||||
return "".join(collected).strip()
|
||||
|
||||
def _verification_commands(self, policies: dict[str, Any]) -> list[str]:
|
||||
def _verification_commands(self, policies: dict[str, Any]) -> list[_VerificationCommand]:
|
||||
configured = policies.get("verification", {}).get("commands", [])
|
||||
commands = [str(item).strip() for item in configured if str(item).strip()]
|
||||
return [command for command in commands if _looks_available(command, self._cwd)]
|
||||
parsed = [_parse_verification_entry(entry) for entry in configured]
|
||||
selected: list[_VerificationCommand] = []
|
||||
for cmd in parsed:
|
||||
if cmd.error is not None:
|
||||
selected.append(cmd)
|
||||
continue
|
||||
if _looks_available(cmd.raw, self._cwd):
|
||||
selected.append(cmd)
|
||||
return selected
|
||||
|
||||
def _run_verification_steps(self, policies: dict[str, Any], *, cwd: Path | None = None) -> list[RepoVerificationStep]:
|
||||
steps: list[RepoVerificationStep] = []
|
||||
for command in self._verification_commands(policies):
|
||||
for cmd in self._verification_commands(policies):
|
||||
if cmd.error is not None:
|
||||
steps.append(
|
||||
RepoVerificationStep(
|
||||
command=cmd.raw,
|
||||
returncode=-1,
|
||||
status="error",
|
||||
stderr=f"verification policy error: {cmd.error}",
|
||||
)
|
||||
)
|
||||
continue
|
||||
target: str | list[str] = cmd.raw if cmd.shell else list(cmd.argv)
|
||||
try:
|
||||
completed = subprocess.run(
|
||||
command,
|
||||
target,
|
||||
cwd=cwd or self._cwd,
|
||||
shell=True,
|
||||
shell=cmd.shell,
|
||||
text=True,
|
||||
capture_output=True,
|
||||
check=False,
|
||||
@@ -2031,17 +2117,26 @@ class RepoAutopilotStore:
|
||||
)
|
||||
steps.append(
|
||||
RepoVerificationStep(
|
||||
command=command,
|
||||
command=cmd.raw,
|
||||
returncode=completed.returncode,
|
||||
status="success" if completed.returncode == 0 else "failed",
|
||||
stdout=(completed.stdout or "")[-4000:],
|
||||
stderr=(completed.stderr or "")[-4000:],
|
||||
)
|
||||
)
|
||||
except FileNotFoundError as exc:
|
||||
steps.append(
|
||||
RepoVerificationStep(
|
||||
command=cmd.raw,
|
||||
returncode=-1,
|
||||
status="error",
|
||||
stderr=f"executable not found: {exc}",
|
||||
)
|
||||
)
|
||||
except subprocess.TimeoutExpired as exc:
|
||||
steps.append(
|
||||
RepoVerificationStep(
|
||||
command=command,
|
||||
command=cmd.raw,
|
||||
returncode=-1,
|
||||
status="error",
|
||||
stdout=_safe_text(getattr(exc, "stdout", ""))[-4000:],
|
||||
@@ -2051,7 +2146,7 @@ class RepoAutopilotStore:
|
||||
except Exception as exc: # pragma: no cover - defensive
|
||||
steps.append(
|
||||
RepoVerificationStep(
|
||||
command=command,
|
||||
command=cmd.raw,
|
||||
returncode=-1,
|
||||
status="error",
|
||||
stderr=str(exc),
|
||||
|
||||
@@ -41,6 +41,6 @@ async def spawn_session(
|
||||
command,
|
||||
cwd=resolved_cwd,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.STDOUT,
|
||||
)
|
||||
return SessionHandle(session_id=session_id, process=process, cwd=resolved_cwd)
|
||||
|
||||
@@ -15,6 +15,7 @@ from openharness.channels.bus.queue import MessageBus
|
||||
from openharness.channels.impl.base import BaseChannel
|
||||
from openharness.channels.impl.base import resolve_channel_media_dir
|
||||
from openharness.config.schema import FeishuConfig
|
||||
from openharness.utils.helpers import safe_filename
|
||||
|
||||
import importlib.util
|
||||
import logging
|
||||
@@ -38,6 +39,174 @@ class _FeishuSenderInfo:
|
||||
display_name: str
|
||||
|
||||
|
||||
def _clean_mention_value(value: Any) -> str:
|
||||
if value is None:
|
||||
return ""
|
||||
return str(value).strip()
|
||||
|
||||
|
||||
def _normalize_mention_name(value: str) -> str:
|
||||
return value.strip().lstrip("@").strip().lower()
|
||||
|
||||
|
||||
def _configured_bot_names(config: FeishuConfig) -> set[str]:
|
||||
raw_names = getattr(config, "bot_names", []) or []
|
||||
if isinstance(raw_names, str):
|
||||
items = [item.strip() for item in raw_names.split(",")]
|
||||
else:
|
||||
items = [str(item).strip() for item in raw_names]
|
||||
names = {_normalize_mention_name(item) for item in items if item.strip()}
|
||||
return names or {"ohmo"}
|
||||
|
||||
|
||||
def _mention_value(raw: Any, key: str) -> Any:
|
||||
if isinstance(raw, dict):
|
||||
return raw.get(key)
|
||||
return getattr(raw, key, None)
|
||||
|
||||
|
||||
def _mention_id_value(raw_id: Any, key: str) -> Any:
|
||||
if isinstance(raw_id, dict):
|
||||
return raw_id.get(key)
|
||||
return getattr(raw_id, key, None)
|
||||
|
||||
|
||||
def _extract_feishu_mentions(
|
||||
content_json: dict,
|
||||
raw_mentions: list[Any] | tuple[Any, ...] | None = None,
|
||||
) -> list[dict[str, str]]:
|
||||
"""Return normalized mention metadata from Feishu text/post payloads."""
|
||||
mentions: list[dict[str, str]] = []
|
||||
seen: set[tuple[str, str, str, str, str]] = set()
|
||||
|
||||
def add(raw: Any) -> None:
|
||||
mention_id = _mention_value(raw, "id") or {}
|
||||
record = {
|
||||
"key": _clean_mention_value(_mention_value(raw, "key")),
|
||||
"name": _clean_mention_value(_mention_value(raw, "name") or _mention_value(raw, "user_name")),
|
||||
"open_id": _clean_mention_value(
|
||||
_mention_value(raw, "open_id") or _mention_id_value(mention_id, "open_id")
|
||||
),
|
||||
"user_id": _clean_mention_value(
|
||||
_mention_value(raw, "user_id") or _mention_id_value(mention_id, "user_id")
|
||||
),
|
||||
"union_id": _clean_mention_value(
|
||||
_mention_value(raw, "union_id") or _mention_id_value(mention_id, "union_id")
|
||||
),
|
||||
}
|
||||
key = (record["key"], record["name"], record["open_id"], record["user_id"], record["union_id"])
|
||||
if any(record.values()) and key not in seen:
|
||||
seen.add(key)
|
||||
mentions.append(record)
|
||||
|
||||
if raw_mentions:
|
||||
for item in raw_mentions:
|
||||
add(item)
|
||||
|
||||
raw_mentions = content_json.get("mentions")
|
||||
if isinstance(raw_mentions, list):
|
||||
for item in raw_mentions:
|
||||
if isinstance(item, dict):
|
||||
add(item)
|
||||
|
||||
def walk(value: Any) -> None:
|
||||
if isinstance(value, dict):
|
||||
if value.get("tag") == "at":
|
||||
add(value)
|
||||
for child in value.values():
|
||||
walk(child)
|
||||
elif isinstance(value, list):
|
||||
for child in value:
|
||||
walk(child)
|
||||
|
||||
walk(content_json)
|
||||
return mentions
|
||||
|
||||
|
||||
def _feishu_mentions_bot(
|
||||
content_json: dict,
|
||||
text: str,
|
||||
config: FeishuConfig,
|
||||
*,
|
||||
mentions: list[dict[str, str]] | None = None,
|
||||
) -> bool:
|
||||
"""Best-effort bot mention detection for Feishu group messages."""
|
||||
bot_open_id = str(getattr(config, "bot_open_id", "") or "").strip()
|
||||
bot_names = _configured_bot_names(config)
|
||||
for mention in mentions if mentions is not None else _extract_feishu_mentions(content_json):
|
||||
ids = {mention.get("open_id", ""), mention.get("user_id", ""), mention.get("union_id", "")}
|
||||
if bot_open_id and bot_open_id in ids:
|
||||
return True
|
||||
name = _normalize_mention_name(mention.get("name", ""))
|
||||
if name and any(name == candidate or candidate in name for candidate in bot_names):
|
||||
return True
|
||||
|
||||
normalized_text = text.strip().lower()
|
||||
for name in bot_names:
|
||||
if re.search(rf"(^|\s)@{re.escape(name)}(?=$|\s|[::,,])", normalized_text):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _normalize_feishu_group_policy(value: str | None) -> str:
|
||||
normalized = str(value or "").strip().lower().replace("-", "_")
|
||||
aliases = {
|
||||
"all": "open",
|
||||
"always": "open",
|
||||
"always_reply": "open",
|
||||
"managed_mention": "managed_or_mention",
|
||||
"managed_or_at": "managed_or_mention",
|
||||
"at": "mention",
|
||||
"mentions": "mention",
|
||||
}
|
||||
normalized = aliases.get(normalized, normalized)
|
||||
if normalized in {"open", "mention", "managed", "managed_or_mention"}:
|
||||
return normalized
|
||||
return "managed_or_mention"
|
||||
|
||||
|
||||
def _is_ohmo_managed_feishu_group(chat_id: str) -> bool:
|
||||
workspace = os.environ.get("OHMO_WORKSPACE")
|
||||
if not workspace:
|
||||
return False
|
||||
try:
|
||||
from ohmo.group_registry import load_managed_group_record
|
||||
|
||||
return load_managed_group_record(
|
||||
workspace=workspace,
|
||||
channel="feishu",
|
||||
chat_id=chat_id,
|
||||
) is not None
|
||||
except Exception:
|
||||
logger.exception("Failed to load ohmo managed Feishu group metadata chat_id=%s", chat_id)
|
||||
return False
|
||||
|
||||
|
||||
def _should_process_feishu_group_message(
|
||||
*,
|
||||
chat_type: str,
|
||||
chat_id: str,
|
||||
mentions_bot: bool,
|
||||
config: FeishuConfig,
|
||||
) -> bool:
|
||||
if str(chat_type or "").strip().lower() != "group":
|
||||
return True
|
||||
policy = _normalize_feishu_group_policy(getattr(config, "group_policy", "managed_or_mention"))
|
||||
if policy == "open":
|
||||
return True
|
||||
if policy == "mention":
|
||||
return mentions_bot
|
||||
if policy == "managed":
|
||||
return _is_ohmo_managed_feishu_group(chat_id)
|
||||
if policy == "managed_or_mention":
|
||||
return mentions_bot or _is_ohmo_managed_feishu_group(chat_id)
|
||||
return mentions_bot
|
||||
|
||||
|
||||
class FeishuApiError(RuntimeError):
|
||||
"""Raised when Feishu returns an unsuccessful API response."""
|
||||
|
||||
|
||||
def _extract_share_card_content(content_json: dict, msg_type: str) -> str:
|
||||
"""Extract text representation from share cards and interactive messages."""
|
||||
parts = []
|
||||
@@ -263,6 +432,26 @@ class FeishuChannel(BaseChannel):
|
||||
self._sender_cache: OrderedDict[str, _FeishuSenderInfo] = OrderedDict()
|
||||
self._loop: asyncio.AbstractEventLoop | None = None
|
||||
|
||||
def _ensure_rest_client(self) -> bool:
|
||||
"""Initialize the Feishu REST client without starting the WebSocket receiver."""
|
||||
if self._client is not None:
|
||||
return True
|
||||
if not FEISHU_AVAILABLE:
|
||||
logger.error("Feishu SDK not installed. Run: pip install lark-oapi")
|
||||
return False
|
||||
if not self.config.app_id or not self.config.app_secret:
|
||||
logger.error("Feishu app_id and app_secret not configured")
|
||||
return False
|
||||
import lark_oapi as lark
|
||||
|
||||
self._client = lark.Client.builder() \
|
||||
.app_id(self.config.app_id) \
|
||||
.app_secret(self.config.app_secret) \
|
||||
.domain(self.config.domain) \
|
||||
.log_level(lark.LogLevel.INFO) \
|
||||
.build()
|
||||
return True
|
||||
|
||||
async def start(self) -> None:
|
||||
"""Start the Feishu bot with WebSocket long connection."""
|
||||
if not FEISHU_AVAILABLE:
|
||||
@@ -277,12 +466,8 @@ class FeishuChannel(BaseChannel):
|
||||
self._running = True
|
||||
self._loop = asyncio.get_running_loop()
|
||||
|
||||
# Create Lark client for sending messages
|
||||
self._client = lark.Client.builder() \
|
||||
.app_id(self.config.app_id) \
|
||||
.app_secret(self.config.app_secret) \
|
||||
.log_level(lark.LogLevel.INFO) \
|
||||
.build()
|
||||
if not self._ensure_rest_client():
|
||||
return
|
||||
|
||||
# Create event handler (only register message receive, ignore other events)
|
||||
event_handler = lark.EventDispatcherHandler.builder(
|
||||
@@ -297,6 +482,7 @@ class FeishuChannel(BaseChannel):
|
||||
self.config.app_id,
|
||||
self.config.app_secret,
|
||||
event_handler=event_handler,
|
||||
domain=self.config.domain,
|
||||
log_level=lark.LogLevel.INFO
|
||||
)
|
||||
|
||||
@@ -749,10 +935,20 @@ class FeishuChannel(BaseChannel):
|
||||
filename = f"{file_key[:16]}{ext}"
|
||||
|
||||
if data and filename:
|
||||
file_path = media_dir / filename
|
||||
safe_name = safe_filename(filename)
|
||||
if not safe_name:
|
||||
logger.warning("Rejected %s download with unsafe filename %r", msg_type, filename)
|
||||
return None, f"[{msg_type}: download failed]"
|
||||
|
||||
media_root = media_dir.resolve()
|
||||
file_path = (media_root / safe_name).resolve()
|
||||
if not file_path.is_relative_to(media_root):
|
||||
logger.warning("Rejected %s download outside media directory: %r", msg_type, filename)
|
||||
return None, f"[{msg_type}: download failed]"
|
||||
|
||||
file_path.write_bytes(data)
|
||||
logger.debug("Downloaded %s to %s", msg_type, file_path)
|
||||
return str(file_path), f"[{msg_type}: {filename}]"
|
||||
return str(file_path), f"[{msg_type}: {safe_name}]"
|
||||
|
||||
return None, f"[{msg_type}: download failed]"
|
||||
|
||||
@@ -800,30 +996,109 @@ class FeishuChannel(BaseChannel):
|
||||
logger.error("Error sending Feishu %s message: %s", msg_type, e)
|
||||
return False
|
||||
|
||||
@staticmethod
|
||||
def _format_response_error(action: str, response: Any) -> str:
|
||||
log_id = response.get_log_id() if hasattr(response, "get_log_id") else ""
|
||||
return (
|
||||
f"{action} failed: code={getattr(response, 'code', '')}, "
|
||||
f"msg={getattr(response, 'msg', '')}, log_id={log_id}"
|
||||
)
|
||||
|
||||
def _create_managed_group_sync(self, user_open_id: str, name: str) -> str:
|
||||
"""Create a Feishu group containing the user and the app bot."""
|
||||
from lark_oapi.api.im.v1 import CreateChatRequest, CreateChatRequestBody
|
||||
|
||||
if not self._client:
|
||||
raise RuntimeError("Feishu client not initialized")
|
||||
request = (
|
||||
CreateChatRequest.builder()
|
||||
.user_id_type("open_id")
|
||||
.set_bot_manager(True)
|
||||
.request_body(
|
||||
CreateChatRequestBody.builder()
|
||||
.name(name)
|
||||
.user_id_list([user_open_id])
|
||||
.chat_mode("group")
|
||||
.chat_type("private")
|
||||
.build()
|
||||
)
|
||||
.build()
|
||||
)
|
||||
response = self._client.im.v1.chat.create(request)
|
||||
if not response.success():
|
||||
raise FeishuApiError(self._format_response_error("create Feishu group", response))
|
||||
chat_id = getattr(getattr(response, "data", None), "chat_id", None)
|
||||
if not chat_id:
|
||||
raise FeishuApiError("create Feishu group failed: response missing chat_id")
|
||||
logger.info("Created Feishu managed group name=%r chat_id=%s", name, chat_id)
|
||||
return str(chat_id)
|
||||
|
||||
async def create_managed_group(self, *, user_open_id: str, name: str) -> str:
|
||||
"""Create a Feishu group for a single user and the ohmo bot."""
|
||||
loop = asyncio.get_running_loop()
|
||||
return await loop.run_in_executor(None, self._create_managed_group_sync, user_open_id, name)
|
||||
|
||||
def _rename_group_sync(self, chat_id: str, name: str) -> None:
|
||||
"""Rename a Feishu group."""
|
||||
from lark_oapi.api.im.v1 import UpdateChatRequest, UpdateChatRequestBody
|
||||
|
||||
if not self._client:
|
||||
raise RuntimeError("Feishu client not initialized")
|
||||
request = (
|
||||
UpdateChatRequest.builder()
|
||||
.user_id_type("open_id")
|
||||
.chat_id(chat_id)
|
||||
.request_body(UpdateChatRequestBody.builder().name(name).build())
|
||||
.build()
|
||||
)
|
||||
response = self._client.im.v1.chat.update(request)
|
||||
if not response.success():
|
||||
raise FeishuApiError(self._format_response_error("rename Feishu group", response))
|
||||
logger.info("Renamed Feishu group chat_id=%s name=%r", chat_id, name)
|
||||
|
||||
async def rename_group(self, *, chat_id: str, name: str) -> None:
|
||||
"""Rename a Feishu group."""
|
||||
loop = asyncio.get_running_loop()
|
||||
await loop.run_in_executor(None, self._rename_group_sync, chat_id, name)
|
||||
|
||||
async def send(self, msg: OutboundMessage) -> None:
|
||||
"""Send a message through Feishu, including media (images/files) if present."""
|
||||
if not self._client:
|
||||
if not self._ensure_rest_client():
|
||||
logger.warning("Feishu client not initialized")
|
||||
return
|
||||
|
||||
try:
|
||||
receive_id_type = "chat_id" if msg.chat_id.startswith("oc_") else "open_id"
|
||||
loop = asyncio.get_running_loop()
|
||||
reply_mid = msg.metadata.get("message_id")
|
||||
chat_type = str(msg.metadata.get("chat_type") or "").lower()
|
||||
reply_mid = (
|
||||
msg.metadata.get("message_id")
|
||||
if chat_type == "group" or msg.metadata.get("thread_id") or msg.metadata.get("root_id")
|
||||
else None
|
||||
)
|
||||
|
||||
failed_media: list[str] = []
|
||||
sent_media: list[str] = []
|
||||
for file_path in msg.media:
|
||||
if not os.path.isfile(file_path):
|
||||
logger.warning("Media file not found: %s", file_path)
|
||||
failed_media.append(file_path)
|
||||
continue
|
||||
ext = os.path.splitext(file_path)[1].lower()
|
||||
if ext in self._IMAGE_EXTS:
|
||||
key = await loop.run_in_executor(None, self._upload_image_sync, file_path)
|
||||
if key:
|
||||
await loop.run_in_executor(
|
||||
ok = await loop.run_in_executor(
|
||||
None, self._send_message_sync,
|
||||
receive_id_type, msg.chat_id, "image", json.dumps({"image_key": key}, ensure_ascii=False),
|
||||
reply_mid,
|
||||
)
|
||||
if ok:
|
||||
sent_media.append(file_path)
|
||||
else:
|
||||
failed_media.append(file_path)
|
||||
else:
|
||||
failed_media.append(file_path)
|
||||
else:
|
||||
key = await loop.run_in_executor(None, self._upload_file_sync, file_path)
|
||||
if key:
|
||||
@@ -833,11 +1108,26 @@ class FeishuChannel(BaseChannel):
|
||||
media_type = "media"
|
||||
else:
|
||||
media_type = "file"
|
||||
await loop.run_in_executor(
|
||||
ok = await loop.run_in_executor(
|
||||
None, self._send_message_sync,
|
||||
receive_id_type, msg.chat_id, media_type, json.dumps({"file_key": key}, ensure_ascii=False),
|
||||
reply_mid,
|
||||
)
|
||||
if ok:
|
||||
sent_media.append(file_path)
|
||||
else:
|
||||
failed_media.append(file_path)
|
||||
else:
|
||||
failed_media.append(file_path)
|
||||
|
||||
if failed_media:
|
||||
names = ", ".join(os.path.basename(path) for path in failed_media)
|
||||
failure_body = json.dumps({"text": f"文件发送失败:{names}"}, ensure_ascii=False)
|
||||
await loop.run_in_executor(
|
||||
None, self._send_message_sync,
|
||||
receive_id_type, msg.chat_id, "text", failure_body,
|
||||
reply_mid,
|
||||
)
|
||||
|
||||
if msg.content and msg.content.strip():
|
||||
fmt = self._detect_msg_format(msg.content)
|
||||
@@ -952,9 +1242,6 @@ class FeishuChannel(BaseChannel):
|
||||
chat_type = message.chat_type
|
||||
msg_type = message.message_type
|
||||
|
||||
# Add reaction
|
||||
await self._add_reaction(message_id, self.config.react_emoji)
|
||||
|
||||
# Parse content
|
||||
content_parts = []
|
||||
media_paths = []
|
||||
@@ -1001,6 +1288,31 @@ class FeishuChannel(BaseChannel):
|
||||
|
||||
if not content and not media_paths:
|
||||
return
|
||||
mentions = _extract_feishu_mentions(content_json, getattr(message, "mentions", None))
|
||||
mentions_bot = chat_type == "group" and _feishu_mentions_bot(
|
||||
content_json,
|
||||
content,
|
||||
self.config,
|
||||
mentions=mentions,
|
||||
)
|
||||
if not _should_process_feishu_group_message(
|
||||
chat_type=chat_type,
|
||||
chat_id=chat_id,
|
||||
mentions_bot=mentions_bot,
|
||||
config=self.config,
|
||||
):
|
||||
logger.info(
|
||||
"Feishu group message ignored by policy chat_id=%s policy=%s mentions_bot=%s mention_names=%s mention_open_ids=%s",
|
||||
chat_id,
|
||||
getattr(self.config, "group_policy", "managed_or_mention"),
|
||||
mentions_bot,
|
||||
[item.get("name", "") for item in mentions],
|
||||
[item.get("open_id", "") for item in mentions],
|
||||
)
|
||||
return
|
||||
|
||||
# Add reaction only after policy says this message will be handled.
|
||||
await self._add_reaction(message_id, self.config.react_emoji)
|
||||
|
||||
# Forward to message bus
|
||||
reply_to = chat_id if chat_type == "group" else sender_id
|
||||
@@ -1019,6 +1331,8 @@ class FeishuChannel(BaseChannel):
|
||||
"root_id": getattr(message, "root_id", None),
|
||||
"chat_type": chat_type,
|
||||
"msg_type": msg_type,
|
||||
"mentions": mentions,
|
||||
"mentions_bot": mentions_bot,
|
||||
"sender_display_name": sender_display_name,
|
||||
"sender_label": sender_display_name or sender_id,
|
||||
}
|
||||
|
||||
@@ -165,7 +165,8 @@ class ChannelManager:
|
||||
try:
|
||||
await channel.start()
|
||||
except Exception as e:
|
||||
logger.error("Failed to start channel %s: %s", name, e)
|
||||
setattr(channel, "last_error", str(e))
|
||||
logger.exception("Failed to start channel %s", name)
|
||||
|
||||
async def start_all(self) -> None:
|
||||
"""Start all channels and the outbound dispatcher."""
|
||||
|
||||
@@ -179,8 +179,11 @@ class SlackChannel(BaseChannel):
|
||||
except Exception as e:
|
||||
logger.debug("Slack reactions_add failed: %s", e)
|
||||
|
||||
# Thread-scoped session key for channel/group messages
|
||||
session_key = f"slack:{chat_id}:{thread_ts}" if thread_ts and channel_type != "im" else None
|
||||
# Preserve Slack thread metadata for reply routing, but let the ohmo
|
||||
# gateway router derive the session key. The router includes sender
|
||||
# identity for shared chats; passing a senderless Slack override here
|
||||
# would make different people in the same thread share one session.
|
||||
chat_type = "p2p" if channel_type == "im" else "group"
|
||||
|
||||
try:
|
||||
await self._handle_message(
|
||||
@@ -188,13 +191,14 @@ class SlackChannel(BaseChannel):
|
||||
chat_id=chat_id,
|
||||
content=text,
|
||||
metadata={
|
||||
"thread_ts": thread_ts,
|
||||
"chat_type": chat_type,
|
||||
"slack": {
|
||||
"event": event,
|
||||
"thread_ts": thread_ts,
|
||||
"channel_type": channel_type,
|
||||
},
|
||||
},
|
||||
session_key=session_key,
|
||||
)
|
||||
except Exception:
|
||||
logger.exception("Error handling Slack message from %s", sender_id)
|
||||
|
||||
@@ -19,6 +19,14 @@ from openharness.utils.helpers import split_message
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
TELEGRAM_MAX_MESSAGE_LEN = 4000 # Telegram message character limit
|
||||
_TELEGRAM_URL_LOGGERS = ("httpx", "httpcore", "telegram.ext")
|
||||
|
||||
|
||||
def silence_telegram_token_url_loggers() -> None:
|
||||
"""Prevent Telegram bot tokens from appearing in dependency INFO logs."""
|
||||
for name in _TELEGRAM_URL_LOGGERS:
|
||||
logging.getLogger(name).setLevel(logging.WARNING)
|
||||
|
||||
|
||||
|
||||
def _markdown_to_telegram_html(text: str) -> str:
|
||||
@@ -111,6 +119,8 @@ class TelegramChannel(BaseChannel):
|
||||
self.config: TelegramConfig = config
|
||||
self.groq_api_key = groq_api_key
|
||||
self._app: Application | None = None
|
||||
self.last_error: str | None = None
|
||||
self.polling_started = False
|
||||
self._chat_ids: dict[str, int] = {} # Map sender_id to chat_id for replies
|
||||
self._typing_tasks: dict[str, asyncio.Task] = {} # chat_id -> typing loop task
|
||||
self._media_group_buffers: dict[str, dict] = {}
|
||||
@@ -123,6 +133,9 @@ class TelegramChannel(BaseChannel):
|
||||
return
|
||||
|
||||
self._running = True
|
||||
self.last_error = None
|
||||
self.polling_started = False
|
||||
silence_telegram_token_url_loggers()
|
||||
|
||||
# Build the application with larger connection pool to avoid pool-timeout on long runs
|
||||
req = HTTPXRequest(connection_pool_size=16, pool_timeout=5.0, connect_timeout=30.0, read_timeout=30.0)
|
||||
@@ -165,8 +178,10 @@ class TelegramChannel(BaseChannel):
|
||||
# Start polling (this runs until stopped)
|
||||
await self._app.updater.start_polling(
|
||||
allowed_updates=["message"],
|
||||
drop_pending_updates=True # Ignore old messages on startup
|
||||
drop_pending_updates=False,
|
||||
)
|
||||
self.polling_started = True
|
||||
logger.info("Telegram polling started")
|
||||
|
||||
# Keep running until stopped
|
||||
while self._running:
|
||||
@@ -175,6 +190,7 @@ class TelegramChannel(BaseChannel):
|
||||
async def stop(self) -> None:
|
||||
"""Stop the Telegram bot."""
|
||||
self._running = False
|
||||
self.polling_started = False
|
||||
|
||||
# Cancel all typing indicators
|
||||
for chat_id in list(self._typing_tasks):
|
||||
@@ -301,7 +317,7 @@ class TelegramChannel(BaseChannel):
|
||||
|
||||
user = update.effective_user
|
||||
await update.message.reply_text(
|
||||
f"👋 Hi {user.first_name}! I'm nanobot.\n\n"
|
||||
f"👋 Hi {user.first_name}! I'm {self.config.bot_name}.\n\n"
|
||||
"Send me a message and I'll respond!\n"
|
||||
"Type /help to see available commands."
|
||||
)
|
||||
@@ -311,7 +327,7 @@ class TelegramChannel(BaseChannel):
|
||||
if not update.message:
|
||||
return
|
||||
await update.message.reply_text(
|
||||
"🐈 nanobot commands:\n"
|
||||
f"🐈 {self.config.bot_name} commands:\n"
|
||||
"/new — Start a new conversation\n"
|
||||
"/stop — Stop the current task\n"
|
||||
"/help — Show available commands"
|
||||
@@ -492,6 +508,7 @@ class TelegramChannel(BaseChannel):
|
||||
|
||||
async def _on_error(self, update: object, context: ContextTypes.DEFAULT_TYPE) -> None:
|
||||
"""Log polling / handler errors instead of silently swallowing them."""
|
||||
self.last_error = str(context.error)
|
||||
logger.error("Telegram error: %s", context.error)
|
||||
|
||||
def _get_extension(self, media_type: str, mime_type: str | None) -> str:
|
||||
|
||||
+904
-9
File diff suppressed because it is too large
Load Diff
@@ -4,14 +4,18 @@ from openharness.commands.registry import (
|
||||
CommandContext,
|
||||
CommandRegistry,
|
||||
CommandResult,
|
||||
MemoryCommandBackend,
|
||||
SlashCommand,
|
||||
create_default_command_registry,
|
||||
lookup_skill_slash_command,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"CommandContext",
|
||||
"CommandRegistry",
|
||||
"CommandResult",
|
||||
"MemoryCommandBackend",
|
||||
"SlashCommand",
|
||||
"create_default_command_registry",
|
||||
"lookup_skill_slash_command",
|
||||
]
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -34,6 +34,9 @@ class BaseChannelConfig(_CompatModel):
|
||||
class TelegramConfig(BaseChannelConfig):
|
||||
token: str = ""
|
||||
chat_id: str | None = None
|
||||
proxy: str | None = None
|
||||
reply_to_message: bool = True
|
||||
bot_name: str = "ohmo"
|
||||
|
||||
|
||||
class SlackConfig(BaseChannelConfig):
|
||||
@@ -51,6 +54,12 @@ class FeishuConfig(BaseChannelConfig):
|
||||
app_secret: str = ""
|
||||
encrypt_key: str = ""
|
||||
verification_token: str = ""
|
||||
# Group reply policy is enforced by ohmo gateway because managed-group
|
||||
# metadata lives outside the generic Feishu channel adapter.
|
||||
group_policy: str = "managed_or_mention"
|
||||
bot_open_id: str = ""
|
||||
bot_names: list[str] = Field(default_factory=lambda: ["ohmo", "openclaw", "openharness"])
|
||||
domain: str = "https://open.feishu.cn" # use https://open.larksuite.com for Lark international
|
||||
|
||||
|
||||
class DingTalkConfig(BaseChannelConfig):
|
||||
@@ -108,4 +117,3 @@ class ChannelConfigs(_CompatModel):
|
||||
class Config(_CompatModel):
|
||||
channels: ChannelConfigs = Field(default_factory=ChannelConfigs)
|
||||
providers: ProviderConfigs = Field(default_factory=ProviderConfigs)
|
||||
|
||||
|
||||
@@ -63,8 +63,15 @@ class MemorySettings(BaseModel):
|
||||
enabled: bool = True
|
||||
max_files: int = 5
|
||||
max_entrypoint_lines: int = 200
|
||||
max_entrypoint_bytes: int = 25_000
|
||||
context_window_tokens: int | None = None
|
||||
auto_compact_threshold_tokens: int | None = None
|
||||
auto_extract_enabled: bool = False
|
||||
auto_extract_max_records: int = 3
|
||||
session_memory_enabled: bool = True
|
||||
auto_dream_enabled: bool = False
|
||||
auto_dream_min_hours: float = 24.0
|
||||
auto_dream_min_sessions: int = 5
|
||||
|
||||
|
||||
class SandboxNetworkSettings(BaseModel):
|
||||
@@ -106,6 +113,14 @@ class SandboxSettings(BaseModel):
|
||||
docker: DockerSandboxSettings = Field(default_factory=DockerSandboxSettings)
|
||||
|
||||
|
||||
class WebSettings(BaseModel):
|
||||
"""Outbound web tool configuration."""
|
||||
|
||||
proxy: str | None = None
|
||||
resolution_mode: str = "auto"
|
||||
synthetic_dns_cidrs: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class ProviderProfile(BaseModel):
|
||||
"""Named provider workflow configuration."""
|
||||
|
||||
@@ -240,6 +255,30 @@ def default_provider_profiles() -> dict[str, ProviderProfile]:
|
||||
default_model="MiniMax-M2.7",
|
||||
base_url="https://api.minimax.io/v1",
|
||||
),
|
||||
"nvidia": ProviderProfile(
|
||||
label="NVIDIA NIM",
|
||||
provider="nvidia",
|
||||
api_format="openai",
|
||||
auth_source="nvidia_api_key",
|
||||
default_model="openai/gpt-oss-120b",
|
||||
base_url="https://integrate.api.nvidia.com/v1",
|
||||
),
|
||||
"qwen": ProviderProfile(
|
||||
label="Qwen (DashScope)",
|
||||
provider="dashscope",
|
||||
api_format="openai",
|
||||
auth_source="dashscope_api_key",
|
||||
default_model="qwen-plus",
|
||||
base_url="https://dashscope.aliyuncs.com/compatible-mode/v1",
|
||||
),
|
||||
"modelscope": ProviderProfile(
|
||||
label="ModelScope",
|
||||
provider="modelscope",
|
||||
api_format="openai",
|
||||
auth_source="modelscope_api_key",
|
||||
default_model="deepseek-ai/DeepSeek-V4-Flash",
|
||||
base_url="https://api-inference.modelscope.cn/v1",
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
@@ -328,6 +367,8 @@ def auth_source_provider_name(auth_source: str) -> str:
|
||||
"moonshot_api_key": "moonshot",
|
||||
"gemini_api_key": "gemini",
|
||||
"minimax_api_key": "minimax",
|
||||
"nvidia_api_key": "nvidia",
|
||||
"modelscope_api_key": "modelscope",
|
||||
}
|
||||
return mapping.get(auth_source, auth_source)
|
||||
|
||||
@@ -337,6 +378,30 @@ def auth_source_uses_api_key(auth_source: str) -> bool:
|
||||
return auth_source.endswith("_api_key")
|
||||
|
||||
|
||||
def auth_source_env_var_candidates(auth_source: str) -> tuple[str, ...]:
|
||||
"""Return env vars to probe for an auth source in precedence order."""
|
||||
mapping = {
|
||||
"anthropic_api_key": ("OPENHARNESS_ANTHROPIC_API_KEY", "ANTHROPIC_API_KEY"),
|
||||
"openai_api_key": ("OPENHARNESS_OPENAI_API_KEY", "OPENAI_API_KEY"),
|
||||
"dashscope_api_key": ("OPENHARNESS_DASHSCOPE_API_KEY", "DASHSCOPE_API_KEY"),
|
||||
"moonshot_api_key": ("OPENHARNESS_MOONSHOT_API_KEY", "MOONSHOT_API_KEY"),
|
||||
"gemini_api_key": ("OPENHARNESS_GEMINI_API_KEY", "GEMINI_API_KEY"),
|
||||
"minimax_api_key": ("OPENHARNESS_MINIMAX_API_KEY", "MINIMAX_API_KEY"),
|
||||
"nvidia_api_key": ("OPENHARNESS_NVIDIA_API_KEY", "NVIDIA_API_KEY"),
|
||||
"modelscope_api_key": ("OPENHARNESS_MODELSCOPE_API_KEY", "MODELSCOPE_API_KEY"),
|
||||
}
|
||||
return mapping.get(auth_source, ())
|
||||
|
||||
|
||||
def resolve_auth_env_value(auth_source: str) -> tuple[str, str] | None:
|
||||
"""Return the first configured env var/value pair for an auth source."""
|
||||
for env_var in auth_source_env_var_candidates(auth_source):
|
||||
env_value = os.environ.get(env_var, "")
|
||||
if env_value:
|
||||
return env_var, env_value
|
||||
return None
|
||||
|
||||
|
||||
def credential_storage_provider_name(profile_name: str, profile: ProviderProfile) -> str:
|
||||
"""Return the storage namespace used for this profile's credential.
|
||||
|
||||
@@ -369,6 +434,10 @@ def default_auth_source_for_provider(provider: str, api_format: str | None = Non
|
||||
return "gemini_api_key"
|
||||
if provider == "minimax":
|
||||
return "minimax_api_key"
|
||||
if provider == "nvidia":
|
||||
return "nvidia_api_key"
|
||||
if provider == "modelscope":
|
||||
return "modelscope_api_key"
|
||||
if provider == "openai" or api_format == "openai":
|
||||
return "openai_api_key"
|
||||
return "anthropic_api_key"
|
||||
@@ -437,6 +506,63 @@ def _profile_from_flat_settings(settings: "Settings") -> tuple[str, ProviderProf
|
||||
return name, profile
|
||||
|
||||
|
||||
class ImageGenerationConfig(BaseModel):
|
||||
"""Configuration for the image_generation tool."""
|
||||
|
||||
provider: str = "auto"
|
||||
model: str = "gpt-image-2"
|
||||
api_key: str = ""
|
||||
base_url: str = ""
|
||||
codex_model: str = "gpt-5.4"
|
||||
codex_base_url: str = ""
|
||||
|
||||
@classmethod
|
||||
def from_env(cls) -> "ImageGenerationConfig":
|
||||
"""Load image generation config from environment variables."""
|
||||
return cls(
|
||||
provider=os.environ.get("OPENHARNESS_IMAGE_GENERATION_PROVIDER", "auto").strip()
|
||||
or "auto",
|
||||
model=os.environ.get("OPENHARNESS_IMAGE_GENERATION_MODEL", "gpt-image-2").strip()
|
||||
or "gpt-image-2",
|
||||
api_key=os.environ.get("OPENHARNESS_IMAGE_GENERATION_API_KEY", "").strip(),
|
||||
base_url=os.environ.get("OPENHARNESS_IMAGE_GENERATION_BASE_URL", "").strip(),
|
||||
codex_model=os.environ.get("OPENHARNESS_IMAGE_GENERATION_CODEX_MODEL", "gpt-5.4").strip()
|
||||
or "gpt-5.4",
|
||||
codex_base_url=os.environ.get("OPENHARNESS_IMAGE_GENERATION_CODEX_BASE_URL", "").strip(),
|
||||
)
|
||||
|
||||
@property
|
||||
def is_configured(self) -> bool:
|
||||
"""Return True when either a key provider or Codex provider is selected."""
|
||||
return bool(self.api_key or self.provider in {"auto", "codex"})
|
||||
|
||||
|
||||
class VisionModelConfig(BaseModel):
|
||||
"""Configuration for the vision model used by the image_to_text tool.
|
||||
|
||||
When the active model does not support multimodal input, the agent loop
|
||||
automatically falls back to this vision model to describe images.
|
||||
"""
|
||||
|
||||
model: str = ""
|
||||
api_key: str = ""
|
||||
base_url: str = ""
|
||||
|
||||
@classmethod
|
||||
def from_env(cls) -> "VisionModelConfig":
|
||||
"""Load vision model config from environment variables."""
|
||||
return cls(
|
||||
model=os.environ.get("OPENHARNESS_VISION_MODEL", "").strip(),
|
||||
api_key=os.environ.get("OPENHARNESS_VISION_API_KEY", "").strip(),
|
||||
base_url=os.environ.get("OPENHARNESS_VISION_BASE_URL", "").strip(),
|
||||
)
|
||||
|
||||
@property
|
||||
def is_configured(self) -> bool:
|
||||
"""Return True when both model and api_key are set."""
|
||||
return bool(self.model and self.api_key)
|
||||
|
||||
|
||||
class Settings(BaseModel):
|
||||
"""Main settings model for OpenHarness."""
|
||||
|
||||
@@ -460,8 +586,13 @@ class Settings(BaseModel):
|
||||
hooks: dict[str, list[HookDefinition]] = Field(default_factory=dict)
|
||||
memory: MemorySettings = Field(default_factory=MemorySettings)
|
||||
sandbox: SandboxSettings = Field(default_factory=SandboxSettings)
|
||||
web: WebSettings = Field(default_factory=WebSettings)
|
||||
enabled_plugins: dict[str, bool] = Field(default_factory=dict)
|
||||
allow_project_plugins: bool = False
|
||||
allow_project_skills: bool = True
|
||||
project_skill_dirs: list[str] = Field(
|
||||
default_factory=lambda: [".openharness/skills", ".agents/skills", ".claude/skills"]
|
||||
)
|
||||
mcp_servers: dict[str, McpServerConfig] = Field(default_factory=dict)
|
||||
|
||||
# UI
|
||||
@@ -474,6 +605,12 @@ class Settings(BaseModel):
|
||||
passes: int = 1
|
||||
verbose: bool = False
|
||||
|
||||
# Vision model (image-to-text fallback)
|
||||
vision: VisionModelConfig = Field(default_factory=VisionModelConfig)
|
||||
|
||||
# Image generation model
|
||||
image_generation: ImageGenerationConfig = Field(default_factory=ImageGenerationConfig)
|
||||
|
||||
def merged_profiles(self) -> dict[str, ProviderProfile]:
|
||||
"""Return the saved profiles merged over the built-in catalog."""
|
||||
merged = default_provider_profiles()
|
||||
@@ -492,7 +629,7 @@ class Settings(BaseModel):
|
||||
def resolve_profile(self, name: str | None = None) -> tuple[str, ProviderProfile]:
|
||||
"""Return the active provider profile."""
|
||||
profiles = self.merged_profiles()
|
||||
profile_name = (name or self.active_profile or "").strip() or "claude-api"
|
||||
profile_name = (name or self.active_profile or os.environ.get("OPENHARNESS_PROFILE") or "").strip() or "claude-api"
|
||||
if profile_name not in profiles:
|
||||
fallback_name, fallback = _profile_from_flat_settings(self)
|
||||
profiles[fallback_name] = fallback
|
||||
@@ -529,9 +666,15 @@ class Settings(BaseModel):
|
||||
directly before the profile layer is used everywhere.
|
||||
"""
|
||||
profile_name, profile = self.resolve_profile()
|
||||
next_provider = (self.provider or "").strip() or profile.provider
|
||||
next_api_format = (self.api_format or "").strip() or profile.api_format
|
||||
next_base_url = self.base_url if self.base_url is not None else profile.base_url
|
||||
profile_from_env = bool(os.environ.get("OPENHARNESS_PROFILE"))
|
||||
flat_profile_fields_match_profile = profile_from_env or (
|
||||
(self.provider or "").strip() == profile.provider
|
||||
and (self.api_format or "").strip() == profile.api_format
|
||||
and self.base_url == profile.base_url
|
||||
)
|
||||
next_provider = profile.provider if flat_profile_fields_match_profile else (self.provider or "").strip() or profile.provider
|
||||
next_api_format = profile.api_format if flat_profile_fields_match_profile else (self.api_format or "").strip() or profile.api_format
|
||||
next_base_url = profile.base_url if flat_profile_fields_match_profile else (self.base_url if self.base_url is not None else profile.base_url)
|
||||
next_context_window_tokens = (
|
||||
self.context_window_tokens
|
||||
if self.context_window_tokens is not None
|
||||
@@ -602,19 +745,15 @@ class Settings(BaseModel):
|
||||
if self.api_key:
|
||||
return self.api_key
|
||||
|
||||
env_key = os.environ.get("ANTHROPIC_API_KEY", "")
|
||||
if env_key:
|
||||
return env_key
|
||||
|
||||
# Also check OPENAI_API_KEY for openai-format providers
|
||||
openai_key = os.environ.get("OPENAI_API_KEY", "")
|
||||
if openai_key:
|
||||
return openai_key
|
||||
env_resolved = resolve_auth_env_value(profile.auth_source)
|
||||
if env_resolved:
|
||||
_, env_value = env_resolved
|
||||
return env_value
|
||||
|
||||
raise ValueError(
|
||||
"No API key found. Set ANTHROPIC_API_KEY (or OPENAI_API_KEY for openai-format "
|
||||
"providers) environment variable, or configure api_key in "
|
||||
"~/.openharness/settings.json"
|
||||
"No API key found. Set an OPENHARNESS_* provider API key "
|
||||
"(preferred) or the matching native provider environment variable, "
|
||||
"or configure api_key in ~/.openharness/settings.json"
|
||||
)
|
||||
|
||||
def resolve_auth(self) -> ResolvedAuth:
|
||||
@@ -623,6 +762,15 @@ class Settings(BaseModel):
|
||||
provider = profile.provider.strip()
|
||||
auth_source = profile.auth_source.strip() or default_auth_source_for_provider(provider, profile.api_format)
|
||||
if auth_source in {"codex_subscription", "claude_subscription"}:
|
||||
env_auth_token = os.environ.get("ANTHROPIC_AUTH_TOKEN", "").strip()
|
||||
if auth_source == "claude_subscription" and env_auth_token:
|
||||
return ResolvedAuth(
|
||||
provider=provider,
|
||||
auth_kind="oauth",
|
||||
value=env_auth_token,
|
||||
source="env:ANTHROPIC_AUTH_TOKEN",
|
||||
state="configured",
|
||||
)
|
||||
from openharness.auth.external import (
|
||||
is_third_party_anthropic_endpoint,
|
||||
load_external_credential,
|
||||
@@ -681,23 +829,16 @@ class Settings(BaseModel):
|
||||
|
||||
storage_provider = credential_storage_provider_name(profile_name, profile)
|
||||
|
||||
env_var = {
|
||||
"anthropic_api_key": "ANTHROPIC_API_KEY",
|
||||
"openai_api_key": "OPENAI_API_KEY",
|
||||
"dashscope_api_key": "DASHSCOPE_API_KEY",
|
||||
"moonshot_api_key": "MOONSHOT_API_KEY",
|
||||
"minimax_api_key": "MINIMAX_API_KEY",
|
||||
}.get(auth_source)
|
||||
if env_var:
|
||||
env_value = os.environ.get(env_var, "")
|
||||
if env_value:
|
||||
return ResolvedAuth(
|
||||
provider=provider or storage_provider,
|
||||
auth_kind="api_key",
|
||||
value=env_value,
|
||||
source=f"env:{env_var}",
|
||||
state="configured",
|
||||
)
|
||||
env_resolved = resolve_auth_env_value(auth_source)
|
||||
if env_resolved:
|
||||
env_var, env_value = env_resolved
|
||||
return ResolvedAuth(
|
||||
provider=provider or storage_provider,
|
||||
auth_kind="api_key",
|
||||
value=env_value,
|
||||
source=f"env:{env_var}",
|
||||
state="configured",
|
||||
)
|
||||
|
||||
explicit_key = "" if profile.credential_slot else self.api_key
|
||||
if explicit_key:
|
||||
@@ -727,10 +868,25 @@ class Settings(BaseModel):
|
||||
def merge_cli_overrides(self, **overrides: Any) -> Settings:
|
||||
"""Return a new Settings with CLI overrides applied (non-None values only)."""
|
||||
updates = {k: v for k, v in overrides.items() if v is not None}
|
||||
permission_mode = updates.pop("permission_mode", None)
|
||||
|
||||
def apply_permission_mode(settings: Settings) -> Settings:
|
||||
if permission_mode is None:
|
||||
return settings
|
||||
return settings.model_copy(
|
||||
update={
|
||||
"permission": settings.permission.model_copy(
|
||||
update={"mode": PermissionMode(str(permission_mode))}
|
||||
)
|
||||
}
|
||||
)
|
||||
|
||||
# Strip ANSI escape sequences from model name if present
|
||||
if "model" in updates and isinstance(updates["model"], str):
|
||||
updates["model"] = strip_ansi_escape_sequences(updates["model"])
|
||||
merged = self.model_copy(update=updates)
|
||||
if "effort" in updates and isinstance(updates["effort"], str):
|
||||
updates["effort"] = "xhigh" if updates["effort"].strip().lower() == "max" else updates["effort"].strip().lower()
|
||||
merged = apply_permission_mode(self.model_copy(update=updates))
|
||||
if not updates:
|
||||
return merged
|
||||
profile_keys = {
|
||||
@@ -747,8 +903,25 @@ class Settings(BaseModel):
|
||||
profile_updates = profile_keys.intersection(updates)
|
||||
if not profile_updates:
|
||||
return merged
|
||||
if profile_updates.issubset({"active_profile"}):
|
||||
return merged.materialize_active_profile()
|
||||
if "active_profile" in profile_updates:
|
||||
switch_updates = {
|
||||
key: value
|
||||
for key, value in updates.items()
|
||||
if key not in profile_keys or key in {"active_profile", "profiles"}
|
||||
}
|
||||
switched = apply_permission_mode(self.model_copy(update=switch_updates)).materialize_active_profile()
|
||||
remaining_profile_updates = {
|
||||
key: value
|
||||
for key, value in updates.items()
|
||||
if key in profile_keys and key not in {"active_profile", "profiles"}
|
||||
}
|
||||
if not remaining_profile_updates:
|
||||
return switched
|
||||
return (
|
||||
switched.model_copy(update=remaining_profile_updates)
|
||||
.sync_active_profile_from_flat_fields()
|
||||
.materialize_active_profile()
|
||||
)
|
||||
return merged.sync_active_profile_from_flat_fields().materialize_active_profile()
|
||||
|
||||
|
||||
@@ -806,15 +979,23 @@ def _apply_env_overrides(settings: Settings) -> Settings:
|
||||
if auto_compact_threshold_tokens:
|
||||
updates["auto_compact_threshold_tokens"] = int(auto_compact_threshold_tokens)
|
||||
|
||||
api_key = os.environ.get("ANTHROPIC_API_KEY") or os.environ.get("OPENAI_API_KEY")
|
||||
if api_key:
|
||||
provider = os.environ.get("OPENHARNESS_PROVIDER")
|
||||
api_format = os.environ.get("OPENHARNESS_API_FORMAT")
|
||||
env_auth_source = active_profile.auth_source
|
||||
if provider or api_format:
|
||||
env_auth_source = default_auth_source_for_provider(
|
||||
provider or active_profile.provider,
|
||||
api_format or active_profile.api_format,
|
||||
)
|
||||
|
||||
env_resolved = resolve_auth_env_value(env_auth_source)
|
||||
if env_resolved:
|
||||
_, api_key = env_resolved
|
||||
updates["api_key"] = api_key
|
||||
|
||||
api_format = os.environ.get("OPENHARNESS_API_FORMAT")
|
||||
if api_format:
|
||||
updates["api_format"] = api_format
|
||||
|
||||
provider = os.environ.get("OPENHARNESS_PROVIDER")
|
||||
if provider:
|
||||
updates["provider"] = provider
|
||||
|
||||
@@ -836,6 +1017,23 @@ def _apply_env_overrides(settings: Settings) -> Settings:
|
||||
if sandbox_updates:
|
||||
updates["sandbox"] = settings.sandbox.model_copy(update=sandbox_updates)
|
||||
|
||||
web_updates: dict[str, Any] = {}
|
||||
web_proxy = os.environ.get("OPENHARNESS_WEB_PROXY")
|
||||
if web_proxy:
|
||||
web_updates["proxy"] = web_proxy
|
||||
web_resolution_mode = os.environ.get("OPENHARNESS_WEB_RESOLUTION_MODE")
|
||||
if web_resolution_mode:
|
||||
web_updates["resolution_mode"] = web_resolution_mode
|
||||
web_synthetic_dns_cidrs = os.environ.get("OPENHARNESS_WEB_SYNTHETIC_DNS_CIDRS")
|
||||
if web_synthetic_dns_cidrs:
|
||||
web_updates["synthetic_dns_cidrs"] = [
|
||||
entry.strip()
|
||||
for entry in web_synthetic_dns_cidrs.split(",")
|
||||
if entry.strip()
|
||||
]
|
||||
if web_updates:
|
||||
updates["web"] = settings.web.model_copy(update=web_updates)
|
||||
|
||||
if not updates:
|
||||
return settings
|
||||
return settings.model_copy(update=updates)
|
||||
@@ -863,6 +1061,9 @@ def load_settings(config_path: Path | None = None) -> Settings:
|
||||
if config_path.exists():
|
||||
raw = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
settings = Settings.model_validate(raw)
|
||||
env_profile = os.environ.get("OPENHARNESS_PROFILE")
|
||||
if env_profile:
|
||||
settings = settings.model_copy(update={"active_profile": env_profile.strip()})
|
||||
if "profiles" not in raw or "active_profile" not in raw:
|
||||
profile_name, profile = _profile_from_flat_settings(settings)
|
||||
merged_profiles = settings.merged_profiles()
|
||||
@@ -875,7 +1076,11 @@ def load_settings(config_path: Path | None = None) -> Settings:
|
||||
)
|
||||
return _apply_env_overrides(settings.materialize_active_profile())
|
||||
|
||||
return _apply_env_overrides(Settings().materialize_active_profile())
|
||||
settings = Settings()
|
||||
env_profile = os.environ.get("OPENHARNESS_PROFILE")
|
||||
if env_profile:
|
||||
settings = settings.model_copy(update={"active_profile": env_profile.strip()})
|
||||
return _apply_env_overrides(settings.materialize_active_profile())
|
||||
|
||||
|
||||
def save_settings(settings: Settings, config_path: Path | None = None) -> None:
|
||||
|
||||
@@ -6,6 +6,7 @@ import os
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Optional
|
||||
from xml.sax.saxutils import escape, unescape
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -109,12 +110,12 @@ def format_task_notification(n: TaskNotification) -> str:
|
||||
"""Serialize a TaskNotification to the canonical XML envelope."""
|
||||
parts = [
|
||||
"<task-notification>",
|
||||
f"<task-id>{n.task_id}</task-id>",
|
||||
f"<status>{n.status}</status>",
|
||||
f"<summary>{n.summary}</summary>",
|
||||
f"<task-id>{escape(n.task_id)}</task-id>",
|
||||
f"<status>{escape(n.status)}</status>",
|
||||
f"<summary>{escape(n.summary)}</summary>",
|
||||
]
|
||||
if n.result is not None:
|
||||
parts.append(f"<result>{n.result}</result>")
|
||||
parts.append(f"<result>{escape(n.result)}</result>")
|
||||
if n.usage:
|
||||
parts.append("<usage>")
|
||||
for key in _USAGE_FIELDS:
|
||||
@@ -130,7 +131,7 @@ def parse_task_notification(xml: str) -> TaskNotification:
|
||||
|
||||
def _extract(tag: str) -> Optional[str]:
|
||||
m = re.search(rf"<{tag}>(.*?)</{tag}>", xml, re.DOTALL)
|
||||
return m.group(1).strip() if m else None
|
||||
return unescape(m.group(1).strip()) if m else None
|
||||
|
||||
task_id = _extract("task-id") or ""
|
||||
status = _extract("status") or ""
|
||||
|
||||
@@ -53,6 +53,7 @@ class ToolResultBlock(BaseModel):
|
||||
tool_use_id: str
|
||||
content: str
|
||||
is_error: bool = False
|
||||
result_metadata: dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
|
||||
ContentBlock = Annotated[
|
||||
@@ -116,12 +117,57 @@ class ConversationMessage(BaseModel):
|
||||
|
||||
|
||||
def sanitize_conversation_messages(messages: list[ConversationMessage]) -> list[ConversationMessage]:
|
||||
"""Drop legacy empty assistant messages while preserving other content."""
|
||||
"""Normalize restored conversation history into a provider-safe sequence.
|
||||
|
||||
This drops legacy empty assistant messages and trims malformed trailing tool
|
||||
turns, such as an assistant ``tool_use`` message that never received a
|
||||
matching user ``tool_result`` response. Those broken tails can happen when a
|
||||
session is interrupted mid-turn and would later cause OpenAI-compatible
|
||||
providers to reject the resumed conversation.
|
||||
"""
|
||||
sanitized: list[ConversationMessage] = []
|
||||
pending_tool_use_ids: set[str] = set()
|
||||
pending_tool_use_index: int | None = None
|
||||
|
||||
for message in messages:
|
||||
if message.role == "assistant" and message.is_effectively_empty():
|
||||
continue
|
||||
|
||||
tool_uses = message.tool_uses if message.role == "assistant" else []
|
||||
tool_results = [
|
||||
block for block in message.content if isinstance(block, ToolResultBlock)
|
||||
] if message.role == "user" else []
|
||||
|
||||
matched_pending_tool_results = False
|
||||
if pending_tool_use_ids:
|
||||
result_ids = {block.tool_use_id for block in tool_results}
|
||||
if message.role != "user" or not pending_tool_use_ids.issubset(result_ids):
|
||||
if pending_tool_use_index is not None and pending_tool_use_index < len(sanitized):
|
||||
sanitized.pop(pending_tool_use_index)
|
||||
pending_tool_use_ids = set()
|
||||
pending_tool_use_index = None
|
||||
else:
|
||||
matched_pending_tool_results = True
|
||||
pending_tool_use_ids = set()
|
||||
pending_tool_use_index = None
|
||||
|
||||
if message.role == "user" and tool_results and not matched_pending_tool_results:
|
||||
content = [
|
||||
block for block in message.content if not isinstance(block, ToolResultBlock)
|
||||
]
|
||||
if not content:
|
||||
continue
|
||||
message = ConversationMessage(role="user", content=content)
|
||||
|
||||
sanitized.append(message)
|
||||
|
||||
if tool_uses:
|
||||
pending_tool_use_ids = {block.id for block in tool_uses}
|
||||
pending_tool_use_index = len(sanitized) - 1
|
||||
|
||||
if pending_tool_use_ids and pending_tool_use_index is not None and pending_tool_use_index < len(sanitized):
|
||||
sanitized.pop(pending_tool_use_index)
|
||||
|
||||
return sanitized
|
||||
|
||||
|
||||
|
||||
@@ -4,10 +4,12 @@ from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import re
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any, AsyncIterator, Awaitable, Callable
|
||||
from uuid import uuid4
|
||||
|
||||
from openharness.api.client import (
|
||||
ApiMessageCompleteEvent,
|
||||
@@ -16,8 +18,15 @@ from openharness.api.client import (
|
||||
ApiTextDeltaEvent,
|
||||
SupportsStreamingMessages,
|
||||
)
|
||||
from openharness.api.provider import is_model_multimodal
|
||||
from openharness.api.usage import UsageSnapshot
|
||||
from openharness.engine.messages import ConversationMessage, ToolResultBlock
|
||||
from openharness.config.paths import get_data_dir
|
||||
from openharness.engine.messages import (
|
||||
ConversationMessage,
|
||||
ImageBlock,
|
||||
TextBlock,
|
||||
ToolResultBlock,
|
||||
)
|
||||
from openharness.engine.stream_events import (
|
||||
AssistantTextDelta,
|
||||
AssistantTurnComplete,
|
||||
@@ -30,11 +39,13 @@ from openharness.engine.stream_events import (
|
||||
)
|
||||
from openharness.hooks import HookEvent, HookExecutor
|
||||
from openharness.permissions.checker import PermissionChecker
|
||||
from openharness.services.tool_outputs import tool_output_inline_chars, tool_output_preview_chars
|
||||
from openharness.tools.base import ToolExecutionContext
|
||||
from openharness.tools.base import ToolRegistry
|
||||
|
||||
AUTO_COMPACT_STATUS_MESSAGE = "Auto-compacting conversation memory to keep things fast and focused."
|
||||
REACTIVE_COMPACT_STATUS_MESSAGE = "Prompt too long; compacting conversation memory and retrying."
|
||||
MAX_SAFE_COMPLETION_TOKENS = 128_000
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
@@ -45,6 +56,7 @@ AskUserPrompt = Callable[[str], Awaitable[str]]
|
||||
MAX_TRACKED_READ_FILES = 6
|
||||
MAX_TRACKED_SKILLS = 8
|
||||
MAX_TRACKED_ASYNC_AGENT_EVENTS = 8
|
||||
MAX_TRACKED_ASYNC_AGENT_TASKS = 12
|
||||
MAX_TRACKED_WORK_LOG = 10
|
||||
MAX_TRACKED_USER_GOALS = 5
|
||||
MAX_TRACKED_ACTIVE_ARTIFACTS = 8
|
||||
@@ -57,16 +69,63 @@ def _is_prompt_too_long_error(exc: Exception) -> bool:
|
||||
needle in text
|
||||
for needle in (
|
||||
"prompt too long",
|
||||
"context_length_exceeded",
|
||||
"context length",
|
||||
"maximum context",
|
||||
"context window",
|
||||
"input tokens exceed",
|
||||
"messages resulted in",
|
||||
"reduce the length of the messages",
|
||||
"configured limit",
|
||||
"too many tokens",
|
||||
"too large for the model",
|
||||
"maximum context length",
|
||||
"exceed_context",
|
||||
"exceeds the available context size",
|
||||
"available context size",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _bounded_completion_tokens(max_tokens: int, context_window_tokens: int | None = None) -> int:
|
||||
"""Return a conservative per-request output token cap.
|
||||
|
||||
Some OpenAI-compatible providers reject very large ``max_tokens`` before
|
||||
the request reaches model-side context management. Keep oversized user
|
||||
config from making every turn fail while preserving normal defaults.
|
||||
"""
|
||||
limit = MAX_SAFE_COMPLETION_TOKENS
|
||||
if context_window_tokens is not None and context_window_tokens > 0:
|
||||
limit = min(limit, int(context_window_tokens))
|
||||
return max(1, min(int(max_tokens), limit))
|
||||
|
||||
|
||||
def _extract_completion_token_limit(exc: Exception) -> int | None:
|
||||
"""Parse provider errors such as "supports at most 128000 completion tokens"."""
|
||||
text = str(exc).lower().replace(",", "")
|
||||
patterns = (
|
||||
r"supports at most\s+(\d+)\s+completion tokens",
|
||||
r"at most\s+(\d+)\s+completion tokens",
|
||||
r"max(?:imum)?(?:_completion)?[_\s-]tokens.*?(?:<=|less than or equal to|at most)\s+(\d+)",
|
||||
)
|
||||
for pattern in patterns:
|
||||
match = re.search(pattern, text)
|
||||
if match:
|
||||
try:
|
||||
return max(1, int(match.group(1)))
|
||||
except ValueError:
|
||||
return None
|
||||
return None
|
||||
|
||||
|
||||
def _is_completion_token_limit_error(exc: Exception) -> bool:
|
||||
text = str(exc).lower()
|
||||
return (
|
||||
("max_tokens" in text or "max_completion_tokens" in text)
|
||||
and ("too large" in text or "at most" in text or "completion tokens" in text)
|
||||
)
|
||||
|
||||
|
||||
class MaxTurnsExceeded(RuntimeError):
|
||||
"""Raised when the agent exceeds the configured max_turns for one user prompt."""
|
||||
|
||||
@@ -86,6 +145,7 @@ class QueryContext:
|
||||
model: str
|
||||
system_prompt: str
|
||||
max_tokens: int
|
||||
effort: str | None = None
|
||||
context_window_tokens: int | None = None
|
||||
auto_compact_threshold_tokens: int | None = None
|
||||
permission_prompt: PermissionPrompt | None = None
|
||||
@@ -263,6 +323,55 @@ def _remember_async_agent_activity(
|
||||
del bucket[:-MAX_TRACKED_ASYNC_AGENT_EVENTS]
|
||||
|
||||
|
||||
def _parse_spawned_agent_identity(
|
||||
output: str,
|
||||
metadata: dict[str, object] | None = None,
|
||||
) -> tuple[str, str] | None:
|
||||
if isinstance(metadata, dict):
|
||||
agent_id = str(metadata.get("agent_id") or "").strip()
|
||||
task_id = str(metadata.get("task_id") or "").strip()
|
||||
if agent_id and task_id:
|
||||
return agent_id, task_id
|
||||
match = re.search(r"Spawned agent (.+?) \(task_id=(\S+?)(?:[,)]|$)", output.strip())
|
||||
if match is None:
|
||||
return None
|
||||
return match.group(1).strip(), match.group(2).strip()
|
||||
|
||||
|
||||
def _remember_async_agent_task(
|
||||
tool_metadata: dict[str, object] | None,
|
||||
*,
|
||||
tool_name: str,
|
||||
tool_input: dict[str, object],
|
||||
output: str,
|
||||
result_metadata: dict[str, object] | None = None,
|
||||
) -> None:
|
||||
if tool_name != "agent":
|
||||
return
|
||||
identity = _parse_spawned_agent_identity(output, result_metadata)
|
||||
if identity is None:
|
||||
return
|
||||
agent_id, task_id = identity
|
||||
bucket = _tool_metadata_bucket(tool_metadata, "async_agent_tasks")
|
||||
description = str(tool_input.get("description") or tool_input.get("prompt") or "").strip()
|
||||
entry = {
|
||||
"agent_id": agent_id,
|
||||
"task_id": task_id,
|
||||
"description": description[:240],
|
||||
"status": "spawned",
|
||||
"notification_sent": False,
|
||||
"spawned_at": time.time(),
|
||||
}
|
||||
bucket[:] = [
|
||||
existing
|
||||
for existing in bucket
|
||||
if not isinstance(existing, dict) or str(existing.get("task_id") or "") != task_id
|
||||
]
|
||||
bucket.append(entry)
|
||||
if len(bucket) > MAX_TRACKED_ASYNC_AGENT_TASKS:
|
||||
del bucket[:-MAX_TRACKED_ASYNC_AGENT_TASKS]
|
||||
|
||||
|
||||
def _remember_work_log(
|
||||
tool_metadata: dict[str, object] | None,
|
||||
*,
|
||||
@@ -289,6 +398,7 @@ def _record_tool_carryover(
|
||||
tool_name: str,
|
||||
tool_input: dict[str, object],
|
||||
tool_output: str,
|
||||
tool_result_metadata: dict[str, object] | None,
|
||||
is_error: bool,
|
||||
resolved_file_path: str | None,
|
||||
) -> None:
|
||||
@@ -326,6 +436,13 @@ def _record_tool_carryover(
|
||||
tool_input=tool_input,
|
||||
output=tool_output,
|
||||
)
|
||||
_remember_async_agent_task(
|
||||
context.tool_metadata,
|
||||
tool_name=tool_name,
|
||||
tool_input=tool_input,
|
||||
output=tool_output,
|
||||
result_metadata=tool_result_metadata,
|
||||
)
|
||||
description = str(tool_input.get("description") or tool_input.get("prompt") or tool_name).strip()
|
||||
_remember_verified_work(
|
||||
context.tool_metadata,
|
||||
@@ -393,6 +510,126 @@ def _record_tool_carryover(
|
||||
_remember_work_log(context.tool_metadata, entry="Exited plan mode")
|
||||
|
||||
|
||||
def _tool_artifact_dir() -> Path:
|
||||
artifact_dir = get_data_dir() / "tool_artifacts"
|
||||
artifact_dir.mkdir(parents=True, exist_ok=True)
|
||||
return artifact_dir
|
||||
|
||||
|
||||
def _safe_tool_artifact_name(tool_name: str) -> str:
|
||||
normalized = re.sub(r"[^A-Za-z0-9_.-]+", "_", tool_name.strip())
|
||||
return (normalized or "tool")[:80]
|
||||
|
||||
|
||||
def _offload_tool_output_if_needed(
|
||||
*,
|
||||
tool_name: str,
|
||||
tool_use_id: str,
|
||||
output: str,
|
||||
) -> tuple[str, Path | None]:
|
||||
inline_limit = tool_output_inline_chars()
|
||||
if len(output) <= inline_limit:
|
||||
return output, None
|
||||
|
||||
artifact_path = (
|
||||
_tool_artifact_dir()
|
||||
/ f"{time.strftime('%Y%m%d-%H%M%S')}-{_safe_tool_artifact_name(tool_name)}-{uuid4().hex[:12]}.txt"
|
||||
)
|
||||
artifact_path.write_text(output, encoding="utf-8", errors="replace")
|
||||
preview = output[:tool_output_preview_chars()]
|
||||
omitted = max(0, len(output) - len(preview))
|
||||
inline = (
|
||||
"[Tool output truncated]\n"
|
||||
f"Tool: {tool_name}\n"
|
||||
f"Tool use id: {tool_use_id}\n"
|
||||
f"Original size: {len(output)} chars\n"
|
||||
f"Full output saved to: {artifact_path}\n"
|
||||
f"Inline preview: first {len(preview)} chars"
|
||||
)
|
||||
if omitted:
|
||||
inline += f" ({omitted} chars omitted)"
|
||||
if preview:
|
||||
inline += f"\n\nPreview:\n{preview}"
|
||||
return inline, artifact_path
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Image preprocessing — convert ImageBlocks to text for non-multimodal models
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_IMAGE_PREPROCESS_STATUS = "Converting image to text description via vision model…"
|
||||
|
||||
|
||||
async def _preprocess_images_in_messages(
|
||||
messages: list[ConversationMessage],
|
||||
context: QueryContext,
|
||||
) -> AsyncIterator[StreamEvent]:
|
||||
"""Scan messages for ImageBlocks and convert them to text if the active
|
||||
model does not support multimodal input.
|
||||
|
||||
Yields status events during conversion so the UI stays responsive.
|
||||
"""
|
||||
if is_model_multimodal(context.model):
|
||||
return
|
||||
|
||||
vision_config = context.tool_metadata.get("vision_model_config")
|
||||
if not vision_config:
|
||||
# No vision model configured — skip preprocessing.
|
||||
return
|
||||
|
||||
# Collect all ImageBlocks with their parent message index and block index
|
||||
pending: list[tuple[int, int, ImageBlock]] = []
|
||||
for msg_idx, msg in enumerate(messages):
|
||||
if msg.role != "user":
|
||||
continue
|
||||
for blk_idx, block in enumerate(msg.content):
|
||||
if isinstance(block, ImageBlock):
|
||||
pending.append((msg_idx, blk_idx, block))
|
||||
|
||||
if not pending:
|
||||
return
|
||||
|
||||
yield StatusEvent(message=_IMAGE_PREPROCESS_STATUS)
|
||||
|
||||
# Process images in parallel
|
||||
async def _describe(msg_idx: int, blk_idx: int, block: ImageBlock) -> tuple[int, int, str]:
|
||||
tool = context.tool_registry.get("image_to_text")
|
||||
if tool is None:
|
||||
return msg_idx, blk_idx, "[Image: could not describe — image_to_text tool not available]"
|
||||
|
||||
# Build tool input
|
||||
tool_input_data: dict[str, object] = {
|
||||
"image_data": block.data,
|
||||
"media_type": block.media_type,
|
||||
"prompt": "Describe this image in detail, including any text, "
|
||||
"UI elements, code, diagrams, or visual information present.",
|
||||
}
|
||||
|
||||
try:
|
||||
parsed = tool.input_model.model_validate(tool_input_data)
|
||||
except Exception:
|
||||
return msg_idx, blk_idx, "[Image: could not parse image data]"
|
||||
|
||||
exec_context = ToolExecutionContext(
|
||||
cwd=context.cwd,
|
||||
metadata={
|
||||
"vision_model_config": vision_config,
|
||||
**(context.tool_metadata or {}),
|
||||
},
|
||||
)
|
||||
result = await tool.execute(parsed, exec_context)
|
||||
if result.is_error:
|
||||
return msg_idx, blk_idx, f"[Image description failed: {result.output}]"
|
||||
return msg_idx, blk_idx, result.output
|
||||
|
||||
results = await asyncio.gather(*[_describe(mi, bi, blk) for mi, bi, blk in pending])
|
||||
|
||||
# Replace ImageBlocks with TextBlocks in-place
|
||||
for msg_idx, blk_idx, description in results:
|
||||
msg = messages[msg_idx]
|
||||
msg.content[blk_idx] = TextBlock(text=description)
|
||||
|
||||
|
||||
async def run_query(
|
||||
context: QueryContext,
|
||||
messages: list[ConversationMessage],
|
||||
@@ -413,6 +650,11 @@ async def run_query(
|
||||
compact_state = AutoCompactState()
|
||||
reactive_compact_attempted = False
|
||||
last_compaction_result: tuple[list[ConversationMessage], bool] = (messages, False)
|
||||
effective_max_tokens = _bounded_completion_tokens(
|
||||
context.max_tokens,
|
||||
context.context_window_tokens,
|
||||
)
|
||||
reported_token_clamp = False
|
||||
|
||||
async def _stream_compaction(
|
||||
*,
|
||||
@@ -457,12 +699,28 @@ async def run_query(
|
||||
turn_count = 0
|
||||
while context.max_turns is None or turn_count < context.max_turns:
|
||||
turn_count += 1
|
||||
if effective_max_tokens != context.max_tokens and not reported_token_clamp:
|
||||
reported_token_clamp = True
|
||||
yield StatusEvent(
|
||||
message=(
|
||||
"Requested max_tokens="
|
||||
f"{context.max_tokens} exceeds the safe per-request output cap; "
|
||||
f"using {effective_max_tokens}."
|
||||
)
|
||||
), None
|
||||
# --- auto-compact check before calling the model ---------------
|
||||
async for event, usage in _stream_compaction(trigger="auto"):
|
||||
yield event, usage
|
||||
messages, was_compacted = last_compaction_result
|
||||
compacted_messages, was_compacted = last_compaction_result
|
||||
if compacted_messages is not messages:
|
||||
messages[:] = compacted_messages
|
||||
# ---------------------------------------------------------------
|
||||
|
||||
# --- image preprocessing: convert ImageBlocks to text for non-vision models ---
|
||||
async for event in _preprocess_images_in_messages(messages, context):
|
||||
yield event, None
|
||||
# -----------------------------------------------------------------------------
|
||||
|
||||
final_message: ConversationMessage | None = None
|
||||
usage = UsageSnapshot()
|
||||
|
||||
@@ -472,8 +730,9 @@ async def run_query(
|
||||
model=context.model,
|
||||
messages=messages,
|
||||
system_prompt=context.system_prompt,
|
||||
max_tokens=context.max_tokens,
|
||||
max_tokens=effective_max_tokens,
|
||||
tools=context.tool_registry.to_api_schema(),
|
||||
effort=context.effort,
|
||||
)
|
||||
):
|
||||
if isinstance(event, ApiTextDeltaEvent):
|
||||
@@ -493,12 +752,27 @@ async def run_query(
|
||||
usage = event.usage
|
||||
except Exception as exc:
|
||||
error_msg = str(exc)
|
||||
if _is_completion_token_limit_error(exc):
|
||||
supported_limit = _extract_completion_token_limit(exc)
|
||||
if supported_limit is not None and effective_max_tokens > supported_limit:
|
||||
previous_max_tokens = effective_max_tokens
|
||||
effective_max_tokens = supported_limit
|
||||
yield StatusEvent(
|
||||
message=(
|
||||
f"Model rejected max_tokens={previous_max_tokens}; "
|
||||
f"retrying with provider limit {effective_max_tokens}."
|
||||
)
|
||||
), None
|
||||
turn_count = max(0, turn_count - 1)
|
||||
continue
|
||||
if not reactive_compact_attempted and _is_prompt_too_long_error(exc):
|
||||
reactive_compact_attempted = True
|
||||
yield StatusEvent(message=REACTIVE_COMPACT_STATUS_MESSAGE), None
|
||||
async for event, usage in _stream_compaction(trigger="reactive", force=True):
|
||||
yield event, usage
|
||||
messages, was_compacted = last_compaction_result
|
||||
compacted_messages, was_compacted = last_compaction_result
|
||||
if compacted_messages is not messages:
|
||||
messages[:] = compacted_messages
|
||||
if was_compacted:
|
||||
continue
|
||||
if "connect" in error_msg.lower() or "timeout" in error_msg.lower() or "network" in error_msg.lower():
|
||||
@@ -532,6 +806,14 @@ async def run_query(
|
||||
messages.append(coordinator_context_message)
|
||||
|
||||
if not final_message.tool_uses:
|
||||
if context.hook_executor is not None:
|
||||
await context.hook_executor.execute(
|
||||
HookEvent.STOP,
|
||||
{
|
||||
"event": HookEvent.STOP.value,
|
||||
"stop_reason": "tool_uses_empty",
|
||||
},
|
||||
)
|
||||
return
|
||||
|
||||
tool_calls = final_message.tool_uses
|
||||
@@ -540,11 +822,20 @@ async def run_query(
|
||||
# Single tool: sequential (stream events immediately)
|
||||
tc = tool_calls[0]
|
||||
yield ToolExecutionStarted(tool_name=tc.name, tool_input=tc.input), None
|
||||
result = await _execute_tool_call(context, tc.name, tc.id, tc.input)
|
||||
try:
|
||||
result = await _execute_tool_call(context, tc.name, tc.id, tc.input)
|
||||
except Exception as exc:
|
||||
log.exception("tool execution raised: name=%s id=%s", tc.name, tc.id)
|
||||
result = ToolResultBlock(
|
||||
tool_use_id=tc.id,
|
||||
content=f"Tool {tc.name} failed: {type(exc).__name__}: {exc}",
|
||||
is_error=True,
|
||||
)
|
||||
yield ToolExecutionCompleted(
|
||||
tool_name=tc.name,
|
||||
output=result.content,
|
||||
is_error=result.is_error,
|
||||
metadata=result.result_metadata,
|
||||
), None
|
||||
tool_results = [result]
|
||||
else:
|
||||
@@ -583,6 +874,7 @@ async def run_query(
|
||||
tool_name=tc.name,
|
||||
output=result.content,
|
||||
is_error=result.is_error,
|
||||
metadata=result.result_metadata,
|
||||
), None
|
||||
|
||||
messages.append(ConversationMessage(role="user", content=tool_results))
|
||||
@@ -647,6 +939,16 @@ async def _execute_tool_call(
|
||||
if not decision.allowed:
|
||||
if decision.requires_confirmation and context.permission_prompt is not None:
|
||||
log.debug("permission prompt for %s: %s", tool_name, decision.reason)
|
||||
if context.hook_executor is not None:
|
||||
await context.hook_executor.execute(
|
||||
HookEvent.NOTIFICATION,
|
||||
{
|
||||
"event": HookEvent.NOTIFICATION.value,
|
||||
"notification_type": "permission_prompt",
|
||||
"tool_name": tool_name,
|
||||
"reason": decision.reason,
|
||||
},
|
||||
)
|
||||
confirmed = await context.permission_prompt(tool_name, decision.reason)
|
||||
if not confirmed:
|
||||
log.debug("permission denied by user for %s", tool_name)
|
||||
@@ -674,21 +976,31 @@ async def _execute_tool_call(
|
||||
"ask_user_prompt": context.ask_user_prompt,
|
||||
**(context.tool_metadata or {}),
|
||||
},
|
||||
hook_executor=context.hook_executor,
|
||||
),
|
||||
)
|
||||
elapsed = time.monotonic() - t0
|
||||
log.debug("executed %s in %.2fs err=%s output_len=%d",
|
||||
tool_name, elapsed, result.is_error, len(result.output or ""))
|
||||
inline_output, artifact_path = _offload_tool_output_if_needed(
|
||||
tool_name=tool_name,
|
||||
tool_use_id=tool_use_id,
|
||||
output=result.output,
|
||||
)
|
||||
if artifact_path is not None:
|
||||
_remember_active_artifact(context.tool_metadata, str(artifact_path))
|
||||
tool_result = ToolResultBlock(
|
||||
tool_use_id=tool_use_id,
|
||||
content=result.output,
|
||||
content=inline_output,
|
||||
is_error=result.is_error,
|
||||
result_metadata=dict(result.metadata or {}),
|
||||
)
|
||||
_record_tool_carryover(
|
||||
context,
|
||||
tool_name=tool_name,
|
||||
tool_input=tool_input,
|
||||
tool_output=tool_result.content,
|
||||
tool_result_metadata=result.metadata,
|
||||
is_error=tool_result.is_error,
|
||||
resolved_file_path=_file_path,
|
||||
)
|
||||
|
||||
@@ -8,11 +8,13 @@ from typing import AsyncIterator
|
||||
from openharness.api.client import SupportsStreamingMessages
|
||||
from openharness.engine.cost_tracker import CostTracker
|
||||
from openharness.coordinator.coordinator_mode import get_coordinator_user_context
|
||||
from openharness.engine.messages import ConversationMessage, TextBlock, ToolResultBlock
|
||||
from openharness.engine.messages import ConversationMessage, TextBlock, ToolResultBlock, sanitize_conversation_messages
|
||||
from openharness.engine.query import AskUserPrompt, PermissionPrompt, QueryContext, remember_user_goal, run_query
|
||||
from openharness.engine.stream_events import AssistantTurnComplete, StreamEvent
|
||||
from openharness.hooks import HookExecutor
|
||||
from openharness.config.settings import Settings
|
||||
from openharness.hooks import HookEvent, HookExecutor
|
||||
from openharness.permissions.checker import PermissionChecker
|
||||
from openharness.services.autodream.service import schedule_auto_dream
|
||||
from openharness.tools.base import ToolRegistry
|
||||
|
||||
|
||||
@@ -36,6 +38,7 @@ class QueryEngine:
|
||||
ask_user_prompt: AskUserPrompt | None = None,
|
||||
hook_executor: HookExecutor | None = None,
|
||||
tool_metadata: dict[str, object] | None = None,
|
||||
settings: Settings | None = None,
|
||||
) -> None:
|
||||
self._api_client = api_client
|
||||
self._tool_registry = tool_registry
|
||||
@@ -44,6 +47,7 @@ class QueryEngine:
|
||||
self._model = model
|
||||
self._system_prompt = system_prompt
|
||||
self._max_tokens = max_tokens
|
||||
self._effort = settings.effort if settings is not None else None
|
||||
self._context_window_tokens = context_window_tokens
|
||||
self._auto_compact_threshold_tokens = auto_compact_threshold_tokens
|
||||
self._max_turns = max_turns
|
||||
@@ -51,6 +55,7 @@ class QueryEngine:
|
||||
self._ask_user_prompt = ask_user_prompt
|
||||
self._hook_executor = hook_executor
|
||||
self._tool_metadata = tool_metadata or {}
|
||||
self._settings = settings
|
||||
self._messages: list[ConversationMessage] = []
|
||||
self._cost_tracker = CostTracker()
|
||||
|
||||
@@ -102,6 +107,10 @@ class QueryEngine:
|
||||
"""Update the active model for future turns."""
|
||||
self._model = model
|
||||
|
||||
def set_effort(self, effort: str | None) -> None:
|
||||
"""Update the active reasoning effort for future turns."""
|
||||
self._effort = effort
|
||||
|
||||
def set_api_client(self, api_client: SupportsStreamingMessages) -> None:
|
||||
"""Update the active API client for future turns."""
|
||||
self._api_client = api_client
|
||||
@@ -129,6 +138,77 @@ class QueryEngine:
|
||||
"""Replace the in-memory conversation history."""
|
||||
self._messages = list(messages)
|
||||
|
||||
def _schedule_auto_dream(self) -> None:
|
||||
"""Fire-and-forget background memory consolidation after a user turn."""
|
||||
if self._settings is None:
|
||||
return
|
||||
context = self._tool_metadata.get("autodream_context")
|
||||
kwargs = dict(context) if isinstance(context, dict) else {}
|
||||
schedule_auto_dream(
|
||||
cwd=self._cwd,
|
||||
settings=self._settings,
|
||||
model=self._model,
|
||||
current_session_id=str(self._tool_metadata.get("session_id") or ""),
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
def _prepare_session_memory(self) -> None:
|
||||
"""Expose file-backed session memory to compaction when enabled."""
|
||||
|
||||
if self._settings is None or not self._settings.memory.session_memory_enabled:
|
||||
return
|
||||
if not self._settings.memory.enabled:
|
||||
return
|
||||
from openharness.services.session_memory import prepare_session_memory_metadata
|
||||
|
||||
prepare_session_memory_metadata(
|
||||
self._cwd,
|
||||
self._tool_metadata,
|
||||
session_id=str(self._tool_metadata.get("session_id") or "default"),
|
||||
)
|
||||
|
||||
async def _update_session_memory(self) -> None:
|
||||
"""Persist a session checkpoint after a user turn."""
|
||||
|
||||
if self._settings is None or not self._settings.memory.session_memory_enabled:
|
||||
return
|
||||
if not self._settings.memory.enabled:
|
||||
return
|
||||
from openharness.services.session_memory import update_session_memory_file
|
||||
|
||||
update_session_memory_file(
|
||||
self._cwd,
|
||||
list(self._messages),
|
||||
tool_metadata=self._tool_metadata,
|
||||
session_id=str(self._tool_metadata.get("session_id") or "default"),
|
||||
)
|
||||
|
||||
async def _extract_durable_memories(self) -> None:
|
||||
"""Run the optional durable memory extraction pass."""
|
||||
|
||||
if self._settings is None or not self._settings.memory.auto_extract_enabled:
|
||||
return
|
||||
if not self._settings.memory.enabled:
|
||||
return
|
||||
from openharness.services.memory_extract import extract_memories_from_turn
|
||||
|
||||
try:
|
||||
result = await extract_memories_from_turn(
|
||||
cwd=self._cwd,
|
||||
api_client=self._api_client,
|
||||
model=self._model,
|
||||
messages=list(self._messages),
|
||||
max_records=self._settings.memory.auto_extract_max_records,
|
||||
)
|
||||
except Exception as exc:
|
||||
self._tool_metadata["memory_extract_last_error"] = str(exc)
|
||||
return
|
||||
self._tool_metadata["memory_extract_last"] = {
|
||||
"skipped": result.skipped,
|
||||
"reason": result.reason,
|
||||
"written_paths": [str(path) for path in result.written_paths],
|
||||
}
|
||||
|
||||
def has_pending_continuation(self) -> bool:
|
||||
"""Return True when the conversation ends with tool results awaiting a follow-up model turn."""
|
||||
if not self._messages:
|
||||
@@ -151,9 +231,19 @@ class QueryEngine:
|
||||
if isinstance(prompt, ConversationMessage)
|
||||
else ConversationMessage.from_user_text(prompt)
|
||||
)
|
||||
if user_message.text.strip():
|
||||
if user_message.text.strip() and not self._tool_metadata.pop("_suppress_next_user_goal", False):
|
||||
remember_user_goal(self._tool_metadata, user_message.text)
|
||||
self._prepare_session_memory()
|
||||
self._messages = sanitize_conversation_messages(self._messages)
|
||||
self._messages.append(user_message)
|
||||
if self._hook_executor is not None:
|
||||
await self._hook_executor.execute(
|
||||
HookEvent.USER_PROMPT_SUBMIT,
|
||||
{
|
||||
"event": HookEvent.USER_PROMPT_SUBMIT.value,
|
||||
"prompt": user_message.text,
|
||||
},
|
||||
)
|
||||
context = QueryContext(
|
||||
api_client=self._api_client,
|
||||
tool_registry=self._tool_registry,
|
||||
@@ -162,6 +252,7 @@ class QueryEngine:
|
||||
model=self._model,
|
||||
system_prompt=self._system_prompt,
|
||||
max_tokens=self._max_tokens,
|
||||
effort=self._effort,
|
||||
context_window_tokens=self._context_window_tokens,
|
||||
auto_compact_threshold_tokens=self._auto_compact_threshold_tokens,
|
||||
max_turns=self._max_turns,
|
||||
@@ -174,15 +265,22 @@ class QueryEngine:
|
||||
coordinator_context = self._build_coordinator_context_message()
|
||||
if coordinator_context is not None:
|
||||
query_messages.append(coordinator_context)
|
||||
async for event, usage in run_query(context, query_messages):
|
||||
if isinstance(event, AssistantTurnComplete):
|
||||
self._messages = list(query_messages)
|
||||
if usage is not None:
|
||||
self._cost_tracker.add(usage)
|
||||
yield event
|
||||
try:
|
||||
async for event, usage in run_query(context, query_messages):
|
||||
if isinstance(event, AssistantTurnComplete):
|
||||
self._messages = list(query_messages)
|
||||
if usage is not None:
|
||||
self._cost_tracker.add(usage)
|
||||
yield event
|
||||
finally:
|
||||
await self._update_session_memory()
|
||||
await self._extract_durable_memories()
|
||||
self._schedule_auto_dream()
|
||||
|
||||
async def continue_pending(self, *, max_turns: int | None = None) -> AsyncIterator[StreamEvent]:
|
||||
"""Continue an interrupted tool loop without appending a new user message."""
|
||||
self._prepare_session_memory()
|
||||
self._messages = sanitize_conversation_messages(self._messages)
|
||||
context = QueryContext(
|
||||
api_client=self._api_client,
|
||||
tool_registry=self._tool_registry,
|
||||
@@ -191,6 +289,7 @@ class QueryEngine:
|
||||
model=self._model,
|
||||
system_prompt=self._system_prompt,
|
||||
max_tokens=self._max_tokens,
|
||||
effort=self._effort,
|
||||
context_window_tokens=self._context_window_tokens,
|
||||
auto_compact_threshold_tokens=self._auto_compact_threshold_tokens,
|
||||
max_turns=max_turns if max_turns is not None else self._max_turns,
|
||||
@@ -203,3 +302,5 @@ class QueryEngine:
|
||||
if usage is not None:
|
||||
self._cost_tracker.add(usage)
|
||||
yield event
|
||||
await self._update_session_memory()
|
||||
await self._extract_durable_memories()
|
||||
|
||||
@@ -39,6 +39,7 @@ class ToolExecutionCompleted:
|
||||
tool_name: str
|
||||
output: str
|
||||
is_error: bool = False
|
||||
metadata: dict[str, Any] | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
|
||||
@@ -14,3 +14,7 @@ class HookEvent(str, Enum):
|
||||
POST_COMPACT = "post_compact"
|
||||
PRE_TOOL_USE = "pre_tool_use"
|
||||
POST_TOOL_USE = "post_tool_use"
|
||||
USER_PROMPT_SUBMIT = "user_prompt_submit"
|
||||
NOTIFICATION = "notification"
|
||||
STOP = "stop"
|
||||
SUBAGENT_STOP = "subagent_stop"
|
||||
|
||||
@@ -18,8 +18,13 @@ class HookRegistry:
|
||||
self._hooks[event].append(hook)
|
||||
|
||||
def get(self, event: HookEvent) -> list[HookDefinition]:
|
||||
"""Return hooks registered for an event."""
|
||||
return list(self._hooks.get(event, []))
|
||||
"""Return hooks registered for an event, ordered by priority.
|
||||
|
||||
Hooks with a higher ``priority`` run first. ``sorted`` is stable, so
|
||||
hooks sharing the same priority keep their registration order.
|
||||
"""
|
||||
hooks = self._hooks.get(event, [])
|
||||
return sorted(hooks, key=lambda hook: -getattr(hook, "priority", 0))
|
||||
|
||||
def summary(self) -> str:
|
||||
"""Return a human-readable hook summary."""
|
||||
@@ -33,6 +38,9 @@ class HookRegistry:
|
||||
matcher = getattr(hook, "matcher", None)
|
||||
detail = getattr(hook, "command", None) or getattr(hook, "prompt", None) or getattr(hook, "url", None) or ""
|
||||
suffix = f" matcher={matcher}" if matcher else ""
|
||||
priority = getattr(hook, "priority", 0)
|
||||
if priority:
|
||||
suffix += f" priority={priority}"
|
||||
lines.append(f" - {hook.type}{suffix}: {detail}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
@@ -15,6 +15,8 @@ class CommandHookDefinition(BaseModel):
|
||||
timeout_seconds: int = Field(default=30, ge=1, le=600)
|
||||
matcher: str | None = None
|
||||
block_on_failure: bool = False
|
||||
priority: int = Field(default=0)
|
||||
"""Higher priority runs first within an event; ties keep registration order."""
|
||||
|
||||
|
||||
class PromptHookDefinition(BaseModel):
|
||||
@@ -26,6 +28,8 @@ class PromptHookDefinition(BaseModel):
|
||||
timeout_seconds: int = Field(default=30, ge=1, le=600)
|
||||
matcher: str | None = None
|
||||
block_on_failure: bool = True
|
||||
priority: int = Field(default=0)
|
||||
"""Higher priority runs first within an event; ties keep registration order."""
|
||||
|
||||
|
||||
class HttpHookDefinition(BaseModel):
|
||||
@@ -37,6 +41,8 @@ class HttpHookDefinition(BaseModel):
|
||||
timeout_seconds: int = Field(default=30, ge=1, le=600)
|
||||
matcher: str | None = None
|
||||
block_on_failure: bool = False
|
||||
priority: int = Field(default=0)
|
||||
"""Higher priority runs first within an event; ties keep registration order."""
|
||||
|
||||
|
||||
class AgentHookDefinition(BaseModel):
|
||||
@@ -48,6 +54,8 @@ class AgentHookDefinition(BaseModel):
|
||||
timeout_seconds: int = Field(default=60, ge=1, le=1200)
|
||||
matcher: str | None = None
|
||||
block_on_failure: bool = True
|
||||
priority: int = Field(default=0)
|
||||
"""Higher priority runs first within an event; ties keep registration order."""
|
||||
|
||||
|
||||
HookDefinition = (
|
||||
|
||||
@@ -75,6 +75,31 @@ class McpClientManager:
|
||||
"""Return one configured server object if present."""
|
||||
return self._server_configs.get(name)
|
||||
|
||||
async def _close_failed_stack(self, stack: AsyncExitStack) -> None:
|
||||
"""Best-effort cleanup for a connection attempt that never finished."""
|
||||
try:
|
||||
await stack.aclose()
|
||||
except BaseException as exc:
|
||||
if isinstance(exc, (KeyboardInterrupt, SystemExit)):
|
||||
raise
|
||||
|
||||
def _mark_connection_failed(
|
||||
self,
|
||||
name: str,
|
||||
config: object,
|
||||
*,
|
||||
auth_configured: bool,
|
||||
exc: BaseException,
|
||||
) -> None:
|
||||
"""Record one MCP connection failure without aborting startup."""
|
||||
self._statuses[name] = McpConnectionStatus(
|
||||
name=name,
|
||||
state="failed",
|
||||
transport=getattr(config, "type", "unknown"),
|
||||
auth_configured=auth_configured,
|
||||
detail=str(exc) or exc.__class__.__name__,
|
||||
)
|
||||
|
||||
async def close(self) -> None:
|
||||
"""Close all active MCP sessions."""
|
||||
for stack in list(self._stacks.values()):
|
||||
@@ -173,14 +198,21 @@ class McpClientManager:
|
||||
write_stream=write_stream,
|
||||
auth_configured=bool(config.env),
|
||||
)
|
||||
except Exception as exc:
|
||||
await stack.aclose()
|
||||
self._statuses[name] = McpConnectionStatus(
|
||||
name=name,
|
||||
state="failed",
|
||||
transport=config.type,
|
||||
except asyncio.CancelledError as exc:
|
||||
await self._close_failed_stack(stack)
|
||||
self._mark_connection_failed(
|
||||
name,
|
||||
config,
|
||||
auth_configured=bool(config.env),
|
||||
detail=str(exc),
|
||||
exc=exc,
|
||||
)
|
||||
except Exception as exc:
|
||||
await self._close_failed_stack(stack)
|
||||
self._mark_connection_failed(
|
||||
name,
|
||||
config,
|
||||
auth_configured=bool(config.env),
|
||||
exc=exc,
|
||||
)
|
||||
|
||||
async def _connect_http(self, name: str, config: McpHttpServerConfig) -> None:
|
||||
@@ -200,14 +232,21 @@ class McpClientManager:
|
||||
write_stream=write_stream,
|
||||
auth_configured=bool(config.headers),
|
||||
)
|
||||
except Exception as exc:
|
||||
await stack.aclose()
|
||||
self._statuses[name] = McpConnectionStatus(
|
||||
name=name,
|
||||
state="failed",
|
||||
transport=config.type,
|
||||
except asyncio.CancelledError as exc:
|
||||
await self._close_failed_stack(stack)
|
||||
self._mark_connection_failed(
|
||||
name,
|
||||
config,
|
||||
auth_configured=bool(config.headers),
|
||||
detail=str(exc),
|
||||
exc=exc,
|
||||
)
|
||||
except Exception as exc:
|
||||
await self._close_failed_stack(stack)
|
||||
self._mark_connection_failed(
|
||||
name,
|
||||
config,
|
||||
auth_configured=bool(config.headers),
|
||||
exc=exc,
|
||||
)
|
||||
|
||||
async def _register_connected_session(
|
||||
|
||||
@@ -2,17 +2,24 @@
|
||||
|
||||
from openharness.memory.memdir import load_memory_prompt
|
||||
from openharness.memory.manager import add_memory_entry, list_memory_files, remove_memory_entry
|
||||
from openharness.memory.migrate import migrate_memory
|
||||
from openharness.memory.paths import get_memory_entrypoint, get_project_memory_dir
|
||||
from openharness.memory.scan import scan_memory_files
|
||||
from openharness.memory.search import find_relevant_memories
|
||||
from openharness.memory.usage import mark_memory_used
|
||||
from openharness.memory.relevance import format_relevant_memories, select_relevant_memories
|
||||
|
||||
__all__ = [
|
||||
"add_memory_entry",
|
||||
"find_relevant_memories",
|
||||
"format_relevant_memories",
|
||||
"get_memory_entrypoint",
|
||||
"get_project_memory_dir",
|
||||
"list_memory_files",
|
||||
"load_memory_prompt",
|
||||
"mark_memory_used",
|
||||
"migrate_memory",
|
||||
"remove_memory_entry",
|
||||
"scan_memory_files",
|
||||
"select_relevant_memories",
|
||||
]
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
"""Agent-scoped memory paths and snapshots."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
|
||||
from openharness.config.paths import get_data_dir
|
||||
from openharness.memory.paths import get_project_memory_dir
|
||||
|
||||
AgentMemoryScope = Literal["user", "project", "local"]
|
||||
|
||||
MEMORY_INDEX = "MEMORY.md"
|
||||
SNAPSHOT_DIR_NAME = "agent-memory-snapshots"
|
||||
|
||||
|
||||
def sanitize_agent_type(agent_type: str) -> str:
|
||||
"""Return a path-safe agent type."""
|
||||
|
||||
return re.sub(r"[^a-zA-Z0-9_.-]+", "_", agent_type.strip()).strip("._") or "default"
|
||||
|
||||
|
||||
def get_agent_memory_dir(cwd: str | Path, agent_type: str, scope: AgentMemoryScope) -> Path:
|
||||
"""Return an agent memory vault for the requested scope."""
|
||||
|
||||
safe = sanitize_agent_type(agent_type)
|
||||
if scope == "project":
|
||||
return get_project_memory_dir(cwd) / "agent" / safe
|
||||
if scope == "local":
|
||||
return Path(cwd).resolve() / ".openharness" / "agent-memory-local" / safe
|
||||
return get_data_dir() / "agent-memory" / safe
|
||||
|
||||
|
||||
def ensure_agent_memory_vault(cwd: str | Path, agent_type: str, scope: AgentMemoryScope) -> Path:
|
||||
"""Create and return an agent-scoped memory vault."""
|
||||
|
||||
memory_dir = get_agent_memory_dir(cwd, agent_type, scope)
|
||||
memory_dir.mkdir(parents=True, exist_ok=True)
|
||||
entrypoint = memory_dir / MEMORY_INDEX
|
||||
if not entrypoint.exists():
|
||||
entrypoint.write_text("# Memory Index\n", encoding="utf-8")
|
||||
return memory_dir
|
||||
|
||||
|
||||
def get_agent_memory_entrypoint(cwd: str | Path, agent_type: str, scope: AgentMemoryScope) -> Path:
|
||||
"""Return an agent memory ``MEMORY.md`` path."""
|
||||
|
||||
return ensure_agent_memory_vault(cwd, agent_type, scope) / MEMORY_INDEX
|
||||
|
||||
|
||||
def get_agent_snapshot_dir(cwd: str | Path, agent_type: str) -> Path:
|
||||
"""Return the project snapshot directory for an agent type."""
|
||||
|
||||
return Path(cwd).resolve() / ".openharness" / SNAPSHOT_DIR_NAME / sanitize_agent_type(agent_type)
|
||||
|
||||
|
||||
def initialize_agent_memory_from_snapshot(
|
||||
cwd: str | Path,
|
||||
agent_type: str,
|
||||
scope: AgentMemoryScope,
|
||||
*,
|
||||
replace: bool = False,
|
||||
) -> Path | None:
|
||||
"""Initialize local agent memory from a project snapshot if present."""
|
||||
|
||||
snapshot_dir = get_agent_snapshot_dir(cwd, agent_type)
|
||||
if not snapshot_dir.exists():
|
||||
return None
|
||||
target = ensure_agent_memory_vault(cwd, agent_type, scope)
|
||||
if replace and target.exists():
|
||||
shutil.rmtree(target)
|
||||
target.mkdir(parents=True, exist_ok=True)
|
||||
for src in snapshot_dir.rglob("*.md"):
|
||||
rel = src.relative_to(snapshot_dir)
|
||||
dest = target / rel
|
||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||
if replace or not dest.exists() or _is_default_agent_index(dest):
|
||||
shutil.copy2(src, dest)
|
||||
return target
|
||||
|
||||
|
||||
def _is_default_agent_index(path: Path) -> bool:
|
||||
if path.name != MEMORY_INDEX or not path.exists():
|
||||
return False
|
||||
try:
|
||||
text = path.read_text(encoding="utf-8", errors="replace")
|
||||
except OSError:
|
||||
return False
|
||||
return text.startswith("# Memory Index\n")
|
||||
@@ -6,6 +6,23 @@ from pathlib import Path
|
||||
from re import sub
|
||||
|
||||
from openharness.memory.paths import get_memory_entrypoint, get_project_memory_dir
|
||||
from openharness.memory.scan import scan_memory_files
|
||||
from openharness.memory.schema import (
|
||||
DEFAULT_MEMORY_SCOPE,
|
||||
DEFAULT_MEMORY_TYPE,
|
||||
MemoryScope,
|
||||
MemoryType,
|
||||
SCHEMA_VERSION,
|
||||
compute_memory_signature,
|
||||
first_content_line,
|
||||
format_datetime,
|
||||
generate_memory_id,
|
||||
memory_metadata_from_path,
|
||||
render_memory_file,
|
||||
split_memory_file,
|
||||
coerce_int,
|
||||
utc_now,
|
||||
)
|
||||
from openharness.utils.file_lock import exclusive_file_lock
|
||||
from openharness.utils.fs import atomic_write_text
|
||||
|
||||
@@ -16,36 +33,120 @@ def _memory_lock_path(cwd: str | Path) -> Path:
|
||||
|
||||
def list_memory_files(cwd: str | Path) -> list[Path]:
|
||||
"""List memory markdown files for the project."""
|
||||
memory_dir = get_project_memory_dir(cwd)
|
||||
return sorted(path for path in memory_dir.glob("*.md"))
|
||||
return sorted(header.path for header in scan_memory_files(cwd, max_files=None))
|
||||
|
||||
|
||||
def add_memory_entry(cwd: str | Path, title: str, content: str) -> Path:
|
||||
"""Create a memory file and append it to MEMORY.md."""
|
||||
memory_dir = get_project_memory_dir(cwd)
|
||||
def add_memory_entry(
|
||||
cwd: str | Path,
|
||||
title: str,
|
||||
content: str,
|
||||
*,
|
||||
memory_type: MemoryType = DEFAULT_MEMORY_TYPE,
|
||||
scope: MemoryScope = DEFAULT_MEMORY_SCOPE,
|
||||
description: str = "",
|
||||
tags: tuple[str, ...] = (),
|
||||
) -> Path:
|
||||
"""Create or refresh a memory file and append it to MEMORY.md."""
|
||||
if scope == "team":
|
||||
from openharness.memory.team import check_team_memory_secrets, ensure_team_memory_vault
|
||||
|
||||
secret_error = check_team_memory_secrets(content)
|
||||
if secret_error:
|
||||
raise ValueError(secret_error)
|
||||
memory_dir = ensure_team_memory_vault(cwd)
|
||||
else:
|
||||
memory_dir = get_project_memory_dir(cwd)
|
||||
slug = sub(r"[^a-zA-Z0-9]+", "_", title.strip().lower()).strip("_") or "memory"
|
||||
path = memory_dir / f"{slug}.md"
|
||||
with exclusive_file_lock(_memory_lock_path(cwd)):
|
||||
atomic_write_text(path, content.strip() + "\n")
|
||||
with exclusive_file_lock(memory_dir / ".memory.lock"):
|
||||
category = "knowledge"
|
||||
body = content.strip() + "\n"
|
||||
signature = compute_memory_signature(body, memory_type, category)
|
||||
existing = scan_memory_files(
|
||||
cwd,
|
||||
max_files=None,
|
||||
include_disabled=True,
|
||||
include_expired=True,
|
||||
memory_dir=memory_dir,
|
||||
)
|
||||
duplicate = next(
|
||||
(header for header in existing if _effective_signature(header.path, header.signature) == signature),
|
||||
None,
|
||||
)
|
||||
path = duplicate.path if duplicate is not None else _next_memory_path(memory_dir, slug)
|
||||
now = utc_now()
|
||||
now_text = format_datetime(now)
|
||||
if path.exists():
|
||||
metadata, old_body, _, _ = split_memory_file(path.read_text(encoding="utf-8"))
|
||||
metadata = memory_metadata_from_path(
|
||||
path,
|
||||
metadata,
|
||||
old_body,
|
||||
now=now,
|
||||
source=str(metadata.get("source") or "manual"),
|
||||
)
|
||||
created_at = str(metadata.get("created_at") or now_text)
|
||||
memory_id = str(metadata.get("id") or generate_memory_id(now))
|
||||
else:
|
||||
metadata = {}
|
||||
created_at = now_text
|
||||
memory_id = generate_memory_id(now)
|
||||
|
||||
entrypoint = get_memory_entrypoint(cwd)
|
||||
existing = entrypoint.read_text(encoding="utf-8") if entrypoint.exists() else "# Memory Index\n"
|
||||
if path.name not in existing:
|
||||
existing = existing.rstrip() + f"\n- [{title}]({path.name})\n"
|
||||
atomic_write_text(entrypoint, existing)
|
||||
metadata.update(
|
||||
{
|
||||
"schema_version": SCHEMA_VERSION,
|
||||
"id": memory_id,
|
||||
"name": title.strip(),
|
||||
"description": description.strip() or first_content_line(body) or title.strip(),
|
||||
"type": str(metadata.get("type") or memory_type),
|
||||
"scope": str(metadata.get("scope") or scope),
|
||||
"category": str(metadata.get("category") or category),
|
||||
"importance": max(coerce_int(metadata.get("importance"), default=0), 1),
|
||||
"source": "manual",
|
||||
"signature": signature,
|
||||
"created_at": created_at,
|
||||
"updated_at": now_text,
|
||||
"ttl_days": metadata.get("ttl_days"),
|
||||
"disabled": False,
|
||||
"supersedes": metadata.get("supersedes") or [],
|
||||
"tags": list(dict.fromkeys(str(tag).strip() for tag in tags if str(tag).strip())),
|
||||
}
|
||||
)
|
||||
atomic_write_text(path, render_memory_file(metadata, body))
|
||||
|
||||
entrypoint = memory_dir / "MEMORY.md"
|
||||
index_text = entrypoint.read_text(encoding="utf-8") if entrypoint.exists() else "# Memory Index\n"
|
||||
if path.name not in index_text:
|
||||
index_text = index_text.rstrip() + f"\n- [{title}]({path.name})\n"
|
||||
atomic_write_text(entrypoint, index_text)
|
||||
return path
|
||||
|
||||
|
||||
def remove_memory_entry(cwd: str | Path, name: str) -> bool:
|
||||
"""Delete a memory file and remove its index entry."""
|
||||
memory_dir = get_project_memory_dir(cwd)
|
||||
matches = [path for path in memory_dir.glob("*.md") if path.stem == name or path.name == name]
|
||||
"""Soft-delete a memory file and remove its index entry."""
|
||||
matches = [
|
||||
header
|
||||
for header in scan_memory_files(
|
||||
cwd,
|
||||
max_files=None,
|
||||
include_disabled=True,
|
||||
include_expired=True,
|
||||
)
|
||||
if name in {header.path.stem, header.path.name, header.title, header.id}
|
||||
]
|
||||
if not matches:
|
||||
return False
|
||||
path = matches[0]
|
||||
header = matches[0]
|
||||
if header.disabled:
|
||||
return False
|
||||
path = header.path
|
||||
with exclusive_file_lock(_memory_lock_path(cwd)):
|
||||
if path.exists():
|
||||
path.unlink()
|
||||
content = path.read_text(encoding="utf-8")
|
||||
metadata, body, _, _ = split_memory_file(content)
|
||||
metadata = memory_metadata_from_path(path, metadata, body, source="manual")
|
||||
metadata["disabled"] = True
|
||||
metadata["updated_at"] = format_datetime(utc_now())
|
||||
atomic_write_text(path, render_memory_file(metadata, body))
|
||||
|
||||
entrypoint = get_memory_entrypoint(cwd)
|
||||
if entrypoint.exists():
|
||||
@@ -56,3 +157,27 @@ def remove_memory_entry(cwd: str | Path, name: str) -> bool:
|
||||
]
|
||||
atomic_write_text(entrypoint, "\n".join(lines).rstrip() + "\n")
|
||||
return True
|
||||
|
||||
|
||||
def _next_memory_path(memory_dir: Path, slug: str) -> Path:
|
||||
path = memory_dir / f"{slug}.md"
|
||||
if not path.exists():
|
||||
return path
|
||||
index = 2
|
||||
while True:
|
||||
candidate = memory_dir / f"{slug}_{index}.md"
|
||||
if not candidate.exists():
|
||||
return candidate
|
||||
index += 1
|
||||
|
||||
|
||||
def _effective_signature(path: Path, existing_signature: str) -> str:
|
||||
if existing_signature:
|
||||
return existing_signature
|
||||
try:
|
||||
metadata, body, _, _ = split_memory_file(path.read_text(encoding="utf-8"))
|
||||
except OSError:
|
||||
return ""
|
||||
memory_type = str(metadata.get("type") or "project")
|
||||
category = str(metadata.get("category") or "knowledge")
|
||||
return compute_memory_signature(body, memory_type, category)
|
||||
|
||||
@@ -5,23 +5,41 @@ from __future__ import annotations
|
||||
from pathlib import Path
|
||||
|
||||
from openharness.memory.paths import get_memory_entrypoint, get_project_memory_dir
|
||||
from openharness.memory.schema import (
|
||||
MAX_ENTRYPOINT_BYTES,
|
||||
MEMORY_POLICY_LINES,
|
||||
truncate_entrypoint_content,
|
||||
)
|
||||
|
||||
|
||||
def load_memory_prompt(cwd: str | Path, *, max_entrypoint_lines: int = 200) -> str | None:
|
||||
def load_memory_prompt(
|
||||
cwd: str | Path,
|
||||
*,
|
||||
max_entrypoint_lines: int = 200,
|
||||
max_entrypoint_bytes: int = MAX_ENTRYPOINT_BYTES,
|
||||
) -> str | None:
|
||||
"""Return the memory prompt section for the current project."""
|
||||
memory_dir = get_project_memory_dir(cwd)
|
||||
entrypoint = get_memory_entrypoint(cwd)
|
||||
lines = [
|
||||
"# Memory",
|
||||
f"- Persistent memory directory: {memory_dir}",
|
||||
"- Use this directory to store durable user or project context that should survive future sessions.",
|
||||
"- Use this directory to store durable project and repository context that should survive future sessions.",
|
||||
"- Prefer concise topic files plus an index entry in MEMORY.md.",
|
||||
"",
|
||||
*MEMORY_POLICY_LINES,
|
||||
]
|
||||
|
||||
if entrypoint.exists():
|
||||
content_lines = entrypoint.read_text(encoding="utf-8").splitlines()[:max_entrypoint_lines]
|
||||
if content_lines:
|
||||
lines.extend(["", "## MEMORY.md", "```md", *content_lines, "```"])
|
||||
raw = entrypoint.read_text(encoding="utf-8", errors="replace")
|
||||
view = truncate_entrypoint_content(
|
||||
raw,
|
||||
max_lines=max_entrypoint_lines,
|
||||
max_bytes=max_entrypoint_bytes,
|
||||
)
|
||||
content = view.content.strip()
|
||||
if content:
|
||||
lines.extend(["", "## MEMORY.md", "```md", content, "```"])
|
||||
else:
|
||||
lines.extend(
|
||||
[
|
||||
|
||||
@@ -0,0 +1,137 @@
|
||||
"""Migration utilities for schema-v1 memory frontmatter."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import shutil
|
||||
from dataclasses import asdict, dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from openharness.memory.paths import get_project_memory_dir
|
||||
from openharness.memory.schema import (
|
||||
memory_metadata_from_path,
|
||||
render_memory_file,
|
||||
split_memory_file,
|
||||
utc_now,
|
||||
)
|
||||
from openharness.utils.fs import atomic_write_text
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class MigrationSummary:
|
||||
"""Summary returned by a memory schema migration run."""
|
||||
|
||||
scanned: int
|
||||
changed: int
|
||||
unchanged: int
|
||||
failed: int
|
||||
dry_run: bool
|
||||
backup_dir: str
|
||||
changed_files: tuple[str, ...]
|
||||
failed_files: tuple[str, ...]
|
||||
|
||||
def as_dict(self) -> dict[str, object]:
|
||||
return asdict(self)
|
||||
|
||||
|
||||
def migrate_memory(
|
||||
cwd: str | Path,
|
||||
*,
|
||||
memory_dir: str | Path | None = None,
|
||||
default_type: str = "project",
|
||||
default_category: str = "knowledge",
|
||||
apply: bool = False,
|
||||
) -> MigrationSummary:
|
||||
"""Backfill schema-v1 frontmatter for top-level memory markdown files."""
|
||||
|
||||
root = Path(memory_dir).expanduser().resolve() if memory_dir is not None else get_project_memory_dir(cwd)
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
files = sorted(path for path in root.glob("*.md") if path.name != "MEMORY.md")
|
||||
changed_payloads: list[tuple[Path, str]] = []
|
||||
failed_files: list[str] = []
|
||||
seen_ids: set[str] = set()
|
||||
now = utc_now()
|
||||
|
||||
for path in files:
|
||||
try:
|
||||
content = path.read_text(encoding="utf-8")
|
||||
metadata, body, _, _ = split_memory_file(content)
|
||||
migrated = memory_metadata_from_path(
|
||||
path,
|
||||
metadata,
|
||||
body,
|
||||
now=now,
|
||||
source="migration",
|
||||
default_type=default_type,
|
||||
default_category=default_category,
|
||||
seen_ids=seen_ids,
|
||||
)
|
||||
rendered = render_memory_file(migrated, body)
|
||||
if rendered != content:
|
||||
changed_payloads.append((path, rendered))
|
||||
except OSError:
|
||||
failed_files.append(path.name)
|
||||
|
||||
backup_dir = ""
|
||||
if apply and changed_payloads:
|
||||
backup_path = _create_migration_backup(root)
|
||||
backup_dir = str(backup_path)
|
||||
for path, rendered in changed_payloads:
|
||||
atomic_write_text(path, rendered)
|
||||
|
||||
changed_files = tuple(path.name for path, _ in changed_payloads)
|
||||
return MigrationSummary(
|
||||
scanned=len(files),
|
||||
changed=len(changed_payloads),
|
||||
unchanged=len(files) - len(changed_payloads) - len(failed_files),
|
||||
failed=len(failed_files),
|
||||
dry_run=not apply,
|
||||
backup_dir=backup_dir,
|
||||
changed_files=changed_files,
|
||||
failed_files=tuple(failed_files),
|
||||
)
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
"""Command-line entrypoint for one-off memory migrations."""
|
||||
|
||||
parser = argparse.ArgumentParser(description="Backfill OpenHarness memory schema metadata.")
|
||||
parser.add_argument("--cwd", default=".", help="Project cwd whose memory store should be migrated.")
|
||||
parser.add_argument("--memory-dir", default=None, help="Explicit memory directory to migrate.")
|
||||
parser.add_argument("--default-type", default="project", help="Type for legacy files without one.")
|
||||
parser.add_argument("--default-category", default="knowledge", help="Category for legacy files without one.")
|
||||
parser.add_argument("--dry-run", action="store_true", help="Report files that would change.")
|
||||
parser.add_argument("--apply", action="store_true", help="Write migrated files and create a backup.")
|
||||
args = parser.parse_args(argv)
|
||||
if args.dry_run == args.apply:
|
||||
parser.error("pass exactly one of --dry-run or --apply")
|
||||
summary = migrate_memory(
|
||||
args.cwd,
|
||||
memory_dir=args.memory_dir,
|
||||
default_type=args.default_type,
|
||||
default_category=args.default_category,
|
||||
apply=args.apply,
|
||||
)
|
||||
print(json.dumps(summary.as_dict(), indent=2, ensure_ascii=False))
|
||||
return 0
|
||||
|
||||
|
||||
def _create_migration_backup(memory_dir: Path) -> Path:
|
||||
timestamp = utc_now().strftime("%Y%m%d-%H%M%S")
|
||||
backup_dir = memory_dir / "backups" / f"migration-{timestamp}"
|
||||
suffix = 2
|
||||
while backup_dir.exists():
|
||||
backup_dir = memory_dir / "backups" / f"migration-{timestamp}-{suffix}"
|
||||
suffix += 1
|
||||
backup_dir.mkdir(parents=True, exist_ok=False)
|
||||
for path in sorted(memory_dir.glob("*.md")):
|
||||
shutil.copy2(path, backup_dir / path.name)
|
||||
usage_index = memory_dir / "usage_index.json"
|
||||
if usage_index.exists():
|
||||
shutil.copy2(usage_index, backup_dir / usage_index.name)
|
||||
return backup_dir
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,145 @@
|
||||
"""Relevant memory selection and formatting."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Callable, Iterable
|
||||
|
||||
from openharness.memory.scan import scan_memory_files
|
||||
from openharness.memory.schema import memory_age_label, memory_freshness_text
|
||||
from openharness.memory.search import find_relevant_memories
|
||||
from openharness.memory.types import MemoryHeader
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RelevantMemory:
|
||||
"""A memory selected for prompt injection."""
|
||||
|
||||
header: MemoryHeader
|
||||
freshness: str = ""
|
||||
|
||||
|
||||
MemorySelector = Callable[[str, list[MemoryHeader]], list[str]]
|
||||
|
||||
|
||||
def build_memory_manifest(headers: Iterable[MemoryHeader]) -> str:
|
||||
"""Render a compact manifest for selector prompts and diagnostics."""
|
||||
|
||||
lines: list[str] = []
|
||||
for header in headers:
|
||||
prefix = f"[{header.memory_type or 'memory'}]"
|
||||
bits = [
|
||||
prefix,
|
||||
header.relative_path or header.path.name,
|
||||
f"({memory_age_label(header.modified_at)})",
|
||||
]
|
||||
if header.description:
|
||||
bits.append(f"- {header.description}")
|
||||
lines.append(" ".join(bits))
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def select_relevant_memories(
|
||||
query: str,
|
||||
cwd: str | Path,
|
||||
*,
|
||||
max_results: int = 5,
|
||||
already_surfaced: set[str] | None = None,
|
||||
selector: MemorySelector | None = None,
|
||||
) -> list[RelevantMemory]:
|
||||
"""Return relevant memories with duplicate and freshness handling.
|
||||
|
||||
``selector`` is an optional side-query style reranker. It receives the query
|
||||
and a heuristic shortlist, and returns relative paths in desired order.
|
||||
"""
|
||||
|
||||
surfaced = already_surfaced or set()
|
||||
heuristic = [
|
||||
header
|
||||
for header in find_relevant_memories(query, cwd, max_results=max(10, max_results * 3))
|
||||
if (header.relative_path or str(header.path)) not in surfaced
|
||||
]
|
||||
selected = _apply_selector(query, heuristic, selector=selector, max_results=max_results)
|
||||
result: list[RelevantMemory] = []
|
||||
for header in selected[:max_results]:
|
||||
result.append(RelevantMemory(header=header, freshness=memory_freshness_text(header.modified_at)))
|
||||
return result
|
||||
|
||||
|
||||
def select_manifest_memories(
|
||||
query: str,
|
||||
cwd: str | Path,
|
||||
*,
|
||||
max_results: int = 5,
|
||||
selector: MemorySelector | None = None,
|
||||
) -> list[RelevantMemory]:
|
||||
"""Select from the full manifest instead of heuristic matches only."""
|
||||
|
||||
headers = scan_memory_files(cwd, max_files=200)
|
||||
selected = _apply_selector(query, headers, selector=selector, max_results=max_results)
|
||||
return [
|
||||
RelevantMemory(header=header, freshness=memory_freshness_text(header.modified_at))
|
||||
for header in selected[:max_results]
|
||||
]
|
||||
|
||||
|
||||
def format_relevant_memories(memories: Iterable[RelevantMemory], *, max_chars: int = 8000) -> str:
|
||||
"""Render selected memories for prompt context."""
|
||||
|
||||
lines = ["# Relevant Memories"]
|
||||
for item in memories:
|
||||
header = item.header
|
||||
content = header.path.read_text(encoding="utf-8", errors="replace").strip()
|
||||
if item.freshness:
|
||||
lines.extend(["", f"## {header.relative_path or header.path.name}", f"> {item.freshness}"])
|
||||
else:
|
||||
lines.extend(["", f"## {header.relative_path or header.path.name}"])
|
||||
lines.extend(["```md", content[:max_chars], "```"])
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def json_selector_from_text(text: str) -> list[str]:
|
||||
"""Parse a selector response as either JSON list or newline paths."""
|
||||
|
||||
stripped = text.strip()
|
||||
if not stripped:
|
||||
return []
|
||||
try:
|
||||
payload = json.loads(stripped)
|
||||
except json.JSONDecodeError:
|
||||
return [line.strip("- ").strip() for line in stripped.splitlines() if line.strip()]
|
||||
if isinstance(payload, list):
|
||||
return [str(item).strip() for item in payload if str(item).strip()]
|
||||
if isinstance(payload, dict) and isinstance(payload.get("paths"), list):
|
||||
return [str(item).strip() for item in payload["paths"] if str(item).strip()]
|
||||
return []
|
||||
|
||||
|
||||
def _apply_selector(
|
||||
query: str,
|
||||
headers: list[MemoryHeader],
|
||||
*,
|
||||
selector: MemorySelector | None,
|
||||
max_results: int,
|
||||
) -> list[MemoryHeader]:
|
||||
if not headers or selector is None:
|
||||
return headers[:max_results]
|
||||
requested = selector(query, headers)
|
||||
by_path = {header.relative_path or header.path.name: header for header in headers}
|
||||
selected: list[MemoryHeader] = []
|
||||
seen: set[str] = set()
|
||||
for path in requested:
|
||||
header = by_path.get(path)
|
||||
if header is None or path in seen:
|
||||
continue
|
||||
selected.append(header)
|
||||
seen.add(path)
|
||||
if len(selected) < max_results:
|
||||
selected.extend(
|
||||
header
|
||||
for header in headers
|
||||
if (header.relative_path or header.path.name) not in seen
|
||||
)
|
||||
return selected[:max_results]
|
||||
@@ -5,12 +5,28 @@ from __future__ import annotations
|
||||
from pathlib import Path
|
||||
|
||||
from openharness.memory.paths import get_project_memory_dir
|
||||
from openharness.memory.schema import (
|
||||
coerce_bool,
|
||||
coerce_int,
|
||||
coerce_optional_int,
|
||||
coerce_str_list,
|
||||
first_content_line,
|
||||
is_memory_expired,
|
||||
split_memory_file,
|
||||
)
|
||||
from openharness.memory.types import MemoryHeader
|
||||
|
||||
|
||||
def scan_memory_files(cwd: str | Path, *, max_files: int = 50) -> list[MemoryHeader]:
|
||||
def scan_memory_files(
|
||||
cwd: str | Path,
|
||||
*,
|
||||
max_files: int | None = 50,
|
||||
include_disabled: bool = False,
|
||||
include_expired: bool = False,
|
||||
memory_dir: str | Path | None = None,
|
||||
) -> list[MemoryHeader]:
|
||||
"""Return memory headers sorted by newest first."""
|
||||
memory_dir = get_project_memory_dir(cwd)
|
||||
memory_dir = Path(memory_dir) if memory_dir is not None else get_project_memory_dir(cwd)
|
||||
headers: list[MemoryHeader] = []
|
||||
for path in memory_dir.glob("*.md"):
|
||||
if path.name == "MEMORY.md":
|
||||
@@ -20,45 +36,40 @@ def scan_memory_files(cwd: str | Path, *, max_files: int = 50) -> list[MemoryHea
|
||||
except OSError:
|
||||
continue
|
||||
header = _parse_memory_file(path, text)
|
||||
if header.disabled and not include_disabled:
|
||||
continue
|
||||
if is_memory_expired(_metadata_from_header(header)) and not include_expired:
|
||||
continue
|
||||
headers.append(header)
|
||||
headers.sort(key=lambda item: item.modified_at, reverse=True)
|
||||
if max_files is None:
|
||||
return headers
|
||||
return headers[:max_files]
|
||||
|
||||
|
||||
def _parse_memory_file(path: Path, content: str) -> MemoryHeader:
|
||||
"""Parse a memory file, extracting YAML frontmatter when present."""
|
||||
lines = content.splitlines()
|
||||
metadata, body, _, _ = split_memory_file(content)
|
||||
lines = body.splitlines()
|
||||
title = path.stem
|
||||
description = ""
|
||||
memory_type = ""
|
||||
body_start = 0
|
||||
|
||||
# Parse YAML frontmatter (--- ... ---)
|
||||
if lines and lines[0].strip() == "---":
|
||||
for i, line in enumerate(lines[1:], 1):
|
||||
if line.strip() == "---":
|
||||
for fm_line in lines[1:i]:
|
||||
key, _, value = fm_line.partition(":")
|
||||
key = key.strip()
|
||||
value = value.strip().strip("'\"")
|
||||
if not value:
|
||||
continue
|
||||
if key == "name":
|
||||
title = value
|
||||
elif key == "description":
|
||||
description = value
|
||||
elif key == "type":
|
||||
memory_type = value
|
||||
body_start = i + 1
|
||||
break
|
||||
if metadata.get("name"):
|
||||
title = str(metadata["name"])
|
||||
if metadata.get("description"):
|
||||
description = str(metadata["description"])
|
||||
if metadata.get("type"):
|
||||
memory_type = str(metadata["type"])
|
||||
|
||||
# Fallback: first non-empty, non-frontmatter line as description
|
||||
desc_line_idx: int | None = None
|
||||
if not description:
|
||||
for idx, line in enumerate(lines[body_start:body_start + 10], body_start):
|
||||
fallback = first_content_line("\n".join(lines[:10]))
|
||||
if fallback:
|
||||
description = fallback
|
||||
for idx, line in enumerate(lines[:10]):
|
||||
stripped = line.strip()
|
||||
if stripped and stripped != "---" and not stripped.startswith("#"):
|
||||
description = stripped[:200]
|
||||
if stripped[:200] == description:
|
||||
desc_line_idx = idx
|
||||
break
|
||||
|
||||
@@ -66,7 +77,7 @@ def _parse_memory_file(path: Path, content: str) -> MemoryHeader:
|
||||
# line already used as description so search scoring stays consistent.
|
||||
body_lines = [
|
||||
line.strip()
|
||||
for idx, line in enumerate(lines[body_start:], body_start)
|
||||
for idx, line in enumerate(lines)
|
||||
if line.strip()
|
||||
and not line.strip().startswith("#")
|
||||
and idx != desc_line_idx
|
||||
@@ -80,4 +91,26 @@ def _parse_memory_file(path: Path, content: str) -> MemoryHeader:
|
||||
modified_at=path.stat().st_mtime,
|
||||
memory_type=memory_type,
|
||||
body_preview=body_preview,
|
||||
id=str(metadata.get("id") or ""),
|
||||
schema_version=coerce_int(metadata.get("schema_version"), default=0),
|
||||
category=str(metadata.get("category") or ""),
|
||||
importance=coerce_int(metadata.get("importance"), default=0),
|
||||
source=str(metadata.get("source") or ""),
|
||||
signature=str(metadata.get("signature") or ""),
|
||||
created_at=str(metadata.get("created_at") or ""),
|
||||
updated_at=str(metadata.get("updated_at") or ""),
|
||||
ttl_days=coerce_optional_int(metadata.get("ttl_days")),
|
||||
disabled=coerce_bool(metadata.get("disabled"), default=False),
|
||||
supersedes=coerce_str_list(metadata.get("supersedes")),
|
||||
relative_path=path.name,
|
||||
tags=coerce_str_list(metadata.get("tags")),
|
||||
frontmatter=metadata,
|
||||
)
|
||||
|
||||
|
||||
def _metadata_from_header(header: MemoryHeader) -> dict[str, object]:
|
||||
return {
|
||||
"created_at": header.created_at,
|
||||
"updated_at": header.updated_at,
|
||||
"ttl_days": header.ttl_days,
|
||||
}
|
||||
|
||||
@@ -0,0 +1,443 @@
|
||||
"""Structured memory metadata helpers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import re
|
||||
import secrets
|
||||
import string
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any, Literal
|
||||
|
||||
import yaml
|
||||
|
||||
SCHEMA_VERSION = 1
|
||||
|
||||
MemoryType = Literal["user", "feedback", "project", "reference"]
|
||||
MemoryScope = Literal["private", "project", "team"]
|
||||
|
||||
MEMORY_TYPES: tuple[MemoryType, ...] = ("user", "feedback", "project", "reference")
|
||||
MEMORY_SCOPES: tuple[MemoryScope, ...] = ("private", "project", "team")
|
||||
|
||||
DEFAULT_MEMORY_TYPE: MemoryType = "project"
|
||||
DEFAULT_MEMORY_SCOPE: MemoryScope = "project"
|
||||
|
||||
MAX_ENTRYPOINT_LINES = 200
|
||||
MAX_ENTRYPOINT_BYTES = 25_000
|
||||
MAX_MANIFEST_FILES = 200
|
||||
|
||||
FRONTMATTER_FIELDS = (
|
||||
"schema_version",
|
||||
"id",
|
||||
"name",
|
||||
"description",
|
||||
"type",
|
||||
"scope",
|
||||
"category",
|
||||
"importance",
|
||||
"source",
|
||||
"signature",
|
||||
"created_at",
|
||||
"updated_at",
|
||||
"ttl_days",
|
||||
"disabled",
|
||||
"supersedes",
|
||||
"tags",
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class EntrypointView:
|
||||
"""A bounded view of ``MEMORY.md`` plus truncation diagnostics."""
|
||||
|
||||
content: str
|
||||
was_truncated: bool
|
||||
reason: str = ""
|
||||
|
||||
|
||||
def utc_now() -> datetime:
|
||||
"""Return the current UTC time without sub-second noise."""
|
||||
|
||||
return datetime.now(timezone.utc).replace(microsecond=0)
|
||||
|
||||
|
||||
def format_datetime(value: datetime) -> str:
|
||||
"""Format a datetime as an ISO-8601 UTC string."""
|
||||
|
||||
return value.astimezone(timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z")
|
||||
|
||||
|
||||
def parse_datetime(value: object) -> datetime | None:
|
||||
"""Parse an ISO datetime value used in memory frontmatter."""
|
||||
|
||||
if isinstance(value, datetime):
|
||||
if value.tzinfo is None:
|
||||
return value.replace(tzinfo=timezone.utc)
|
||||
return value.astimezone(timezone.utc)
|
||||
if not isinstance(value, str) or not value.strip():
|
||||
return None
|
||||
raw = value.strip()
|
||||
if raw.endswith("Z"):
|
||||
raw = raw[:-1] + "+00:00"
|
||||
try:
|
||||
parsed = datetime.fromisoformat(raw)
|
||||
except ValueError:
|
||||
return None
|
||||
if parsed.tzinfo is None:
|
||||
parsed = parsed.replace(tzinfo=timezone.utc)
|
||||
return parsed.astimezone(timezone.utc)
|
||||
|
||||
|
||||
def normalize_memory_content(text: str) -> str:
|
||||
"""Normalize memory content for deterministic signatures."""
|
||||
|
||||
lowered = text.lower()
|
||||
collapsed = re.sub(r"\s+", " ", lowered)
|
||||
punctuation_table = str.maketrans("", "", string.punctuation)
|
||||
return collapsed.translate(punctuation_table).strip()
|
||||
|
||||
|
||||
def compute_memory_signature(content: str, memory_type: str, category: str) -> str:
|
||||
"""Compute a deterministic content signature for duplicate detection."""
|
||||
|
||||
normalized = normalize_memory_content(content)
|
||||
payload = f"{normalized}|{memory_type.strip().lower()}|{category.strip().lower()}"
|
||||
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def parse_memory_type(raw: Any, *, default: MemoryType | None = None) -> MemoryType | None:
|
||||
"""Parse a frontmatter ``type`` value into the canonical runtime taxonomy."""
|
||||
|
||||
if isinstance(raw, str):
|
||||
value = raw.strip().lower()
|
||||
if value in MEMORY_TYPES:
|
||||
return value # type: ignore[return-value]
|
||||
if value in {"note", "memory", "core", "knowledge"}:
|
||||
return default
|
||||
return default
|
||||
|
||||
|
||||
def parse_memory_scope(raw: Any, *, default: MemoryScope | None = None) -> MemoryScope | None:
|
||||
"""Parse a frontmatter ``scope`` value into the canonical scope taxonomy."""
|
||||
|
||||
if isinstance(raw, str):
|
||||
value = raw.strip().lower()
|
||||
if value in MEMORY_SCOPES:
|
||||
return value # type: ignore[return-value]
|
||||
if value in {"personal", "user"}:
|
||||
return "private"
|
||||
if value in {"shared"}:
|
||||
return "team"
|
||||
return default
|
||||
|
||||
|
||||
def truncate_entrypoint_content(
|
||||
raw: str,
|
||||
*,
|
||||
max_lines: int = MAX_ENTRYPOINT_LINES,
|
||||
max_bytes: int = MAX_ENTRYPOINT_BYTES,
|
||||
) -> EntrypointView:
|
||||
"""Bound ``MEMORY.md`` by line count and UTF-8 byte count."""
|
||||
|
||||
lines = raw.splitlines()
|
||||
was_line_truncated = len(lines) > max_lines
|
||||
text = "\n".join(lines[:max_lines])
|
||||
encoded = text.encode("utf-8")
|
||||
was_byte_truncated = len(encoded) > max_bytes
|
||||
if was_byte_truncated:
|
||||
encoded = encoded[:max_bytes]
|
||||
text = encoded.decode("utf-8", errors="ignore")
|
||||
cut_at = text.rfind("\n")
|
||||
if cut_at > 0:
|
||||
text = text[:cut_at]
|
||||
if raw.endswith("\n") and not text.endswith("\n"):
|
||||
text += "\n"
|
||||
if not was_line_truncated and not was_byte_truncated:
|
||||
return EntrypointView(content=text, was_truncated=False)
|
||||
reason = (
|
||||
f"{len(raw.encode('utf-8'))} bytes (limit: {max_bytes})"
|
||||
if was_byte_truncated
|
||||
else f"{len(lines)} lines (limit: {max_lines})"
|
||||
)
|
||||
warning = (
|
||||
f"\n\n> WARNING: MEMORY.md is {reason}. Only part of it was loaded. "
|
||||
"Keep index entries one line and move detail into topic notes.\n"
|
||||
)
|
||||
return EntrypointView(content=text.rstrip() + warning, was_truncated=True, reason=reason)
|
||||
|
||||
|
||||
def memory_age_days(mtime: float, *, now: float | None = None) -> int:
|
||||
"""Return floor-rounded days elapsed since a file modification time."""
|
||||
|
||||
import time
|
||||
|
||||
current = time.time() if now is None else now
|
||||
return max(0, int((current - mtime) // 86_400))
|
||||
|
||||
|
||||
def memory_age_label(mtime: float, *, now: float | None = None) -> str:
|
||||
"""Return a model-friendly age label."""
|
||||
|
||||
days = memory_age_days(mtime, now=now)
|
||||
if days == 0:
|
||||
return "today"
|
||||
if days == 1:
|
||||
return "yesterday"
|
||||
return f"{days} days ago"
|
||||
|
||||
|
||||
def memory_freshness_text(mtime: float, *, now: float | None = None) -> str:
|
||||
"""Return a staleness warning for older memories."""
|
||||
|
||||
days = memory_age_days(mtime, now=now)
|
||||
if days <= 1:
|
||||
return ""
|
||||
return (
|
||||
f"This memory is {days} days old. Memories are point-in-time observations; "
|
||||
"verify claims against the current project state before treating them as facts."
|
||||
)
|
||||
|
||||
|
||||
def path_is_relative_to(path: str | Path, root: str | Path) -> bool:
|
||||
"""Compatibility helper for containment checks."""
|
||||
|
||||
try:
|
||||
Path(path).resolve().relative_to(Path(root).resolve())
|
||||
except ValueError:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
MEMORY_POLICY_LINES: tuple[str, ...] = (
|
||||
"## Durable memory policy",
|
||||
"- Store durable memory only when the information is not cheaply derivable from current files, docs, git history, or tool output.",
|
||||
"- Use `type: user|feedback|project|reference` and optional `scope: private|project|team` frontmatter.",
|
||||
"- `MEMORY.md` is an index, not a memory body. Keep each pointer one line.",
|
||||
"- Update or remove stale contradictions instead of duplicating notes.",
|
||||
"- If the user says to ignore memory, proceed as if no memory was loaded and do not cite, apply, or mention memory contents.",
|
||||
"- Memory can be stale. Verify remembered project/code state against current files before acting on it.",
|
||||
"- Do not save secrets, credentials, private personal context in team memory, or temporary task chatter.",
|
||||
)
|
||||
|
||||
|
||||
def generate_memory_id(now: datetime | None = None) -> str:
|
||||
"""Generate a stable-looking memory id for a new memory file."""
|
||||
|
||||
timestamp = format_datetime(now or utc_now()).replace("-", "").replace(":", "")
|
||||
timestamp = timestamp.replace("T", "-").replace("Z", "")
|
||||
return f"mem-{timestamp}-{secrets.token_hex(4)}"
|
||||
|
||||
|
||||
def split_memory_file(content: str) -> tuple[dict[str, Any], str, int, bool]:
|
||||
"""Split a memory file into frontmatter metadata and body text.
|
||||
|
||||
Returns ``(metadata, body, body_start_line, has_closed_frontmatter)``.
|
||||
Unclosed frontmatter is treated as body content after the opening delimiter.
|
||||
"""
|
||||
|
||||
lines = content.splitlines(keepends=True)
|
||||
if not lines or lines[0].strip() != "---":
|
||||
return {}, content, 0, False
|
||||
|
||||
for idx, line in enumerate(lines[1:], 1):
|
||||
if line.strip() == "---":
|
||||
raw_frontmatter = "".join(lines[1:idx])
|
||||
metadata = _load_frontmatter(raw_frontmatter)
|
||||
return metadata, "".join(lines[idx + 1 :]), idx + 1, True
|
||||
|
||||
return {}, "".join(lines[1:]), 1, False
|
||||
|
||||
|
||||
def render_memory_file(metadata: dict[str, Any], body: str) -> str:
|
||||
"""Render metadata and body as a memory markdown file."""
|
||||
|
||||
frontmatter = render_frontmatter(metadata)
|
||||
normalized_body = body.lstrip("\n")
|
||||
if normalized_body and not normalized_body.endswith("\n"):
|
||||
normalized_body += "\n"
|
||||
return f"---\n{frontmatter}---\n\n{normalized_body}"
|
||||
|
||||
|
||||
def render_frontmatter(metadata: dict[str, Any]) -> str:
|
||||
"""Render memory frontmatter in a stable field order."""
|
||||
|
||||
ordered: list[tuple[str, Any]] = []
|
||||
for field in FRONTMATTER_FIELDS:
|
||||
if field in metadata:
|
||||
ordered.append((field, metadata[field]))
|
||||
for key, value in metadata.items():
|
||||
if key not in FRONTMATTER_FIELDS:
|
||||
ordered.append((key, value))
|
||||
return "".join(f"{key}: {_format_yaml_value(value)}\n" for key, value in ordered)
|
||||
|
||||
|
||||
def is_disabled_metadata(metadata: dict[str, Any]) -> bool:
|
||||
"""Return whether a memory metadata object marks the file disabled."""
|
||||
|
||||
return _as_bool(metadata.get("disabled"), default=False)
|
||||
|
||||
|
||||
def is_memory_expired(metadata: dict[str, Any], *, now: datetime | None = None) -> bool:
|
||||
"""Return whether a memory should be hidden because its TTL has elapsed."""
|
||||
|
||||
ttl_days = _as_optional_int(metadata.get("ttl_days"))
|
||||
if ttl_days is None or ttl_days <= 0:
|
||||
return False
|
||||
base_time = parse_datetime(metadata.get("updated_at")) or parse_datetime(metadata.get("created_at"))
|
||||
if base_time is None:
|
||||
return False
|
||||
return (now or utc_now()) >= base_time + timedelta(days=ttl_days)
|
||||
|
||||
|
||||
def coerce_int(value: object, *, default: int = 0) -> int:
|
||||
"""Coerce a metadata value to int."""
|
||||
|
||||
try:
|
||||
return int(value) # type: ignore[arg-type]
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
|
||||
|
||||
def coerce_optional_int(value: object) -> int | None:
|
||||
"""Coerce a metadata value to optional int."""
|
||||
|
||||
return _as_optional_int(value)
|
||||
|
||||
|
||||
def coerce_bool(value: object, *, default: bool = False) -> bool:
|
||||
"""Coerce a metadata value to bool."""
|
||||
|
||||
return _as_bool(value, default=default)
|
||||
|
||||
|
||||
def coerce_str_list(value: object) -> tuple[str, ...]:
|
||||
"""Coerce a metadata value to a tuple of strings."""
|
||||
|
||||
if isinstance(value, str):
|
||||
return (value,) if value else ()
|
||||
if isinstance(value, (list, tuple)):
|
||||
return tuple(str(item) for item in value if str(item))
|
||||
return ()
|
||||
|
||||
|
||||
def memory_metadata_from_path(
|
||||
path: Path,
|
||||
metadata: dict[str, Any],
|
||||
body: str,
|
||||
*,
|
||||
now: datetime | None = None,
|
||||
source: str = "migration",
|
||||
default_type: str = "project",
|
||||
default_category: str = "knowledge",
|
||||
seen_ids: set[str] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Return complete schema-v1 metadata while preserving existing values."""
|
||||
|
||||
updated = dict(metadata)
|
||||
timestamp = _mtime_timestamp(path)
|
||||
created_at = str(updated.get("created_at") or timestamp)
|
||||
updated_at = str(updated.get("updated_at") or timestamp)
|
||||
memory_type = str(updated.get("type") or default_type)
|
||||
category = str(updated.get("category") or default_category)
|
||||
memory_id = str(updated.get("id") or "")
|
||||
if not memory_id or (seen_ids is not None and memory_id in seen_ids):
|
||||
memory_id = _generate_unique_memory_id(now=now, seen_ids=seen_ids)
|
||||
if seen_ids is not None:
|
||||
seen_ids.add(memory_id)
|
||||
|
||||
updated["schema_version"] = coerce_int(updated.get("schema_version"), default=SCHEMA_VERSION)
|
||||
updated["id"] = memory_id
|
||||
updated["name"] = str(updated.get("name") or path.stem)
|
||||
updated["description"] = str(updated.get("description") or first_content_line(body) or path.stem)
|
||||
updated["type"] = memory_type
|
||||
updated["category"] = category
|
||||
updated["importance"] = coerce_int(updated.get("importance"), default=0)
|
||||
updated["source"] = str(updated.get("source") or source)
|
||||
updated["signature"] = str(
|
||||
updated.get("signature") or compute_memory_signature(body, memory_type, category)
|
||||
)
|
||||
updated["created_at"] = created_at
|
||||
updated["updated_at"] = updated_at
|
||||
updated["ttl_days"] = coerce_optional_int(updated.get("ttl_days"))
|
||||
updated["disabled"] = coerce_bool(updated.get("disabled"), default=False)
|
||||
updated["supersedes"] = list(coerce_str_list(updated.get("supersedes")))
|
||||
return updated
|
||||
|
||||
|
||||
def first_content_line(body: str, *, limit: int = 200) -> str:
|
||||
"""Return the first useful body line for descriptions."""
|
||||
|
||||
for line in body.splitlines():
|
||||
stripped = line.strip()
|
||||
if stripped and stripped != "---" and not stripped.startswith("#"):
|
||||
return stripped[:limit]
|
||||
return ""
|
||||
|
||||
|
||||
def _load_frontmatter(raw_frontmatter: str) -> dict[str, Any]:
|
||||
try:
|
||||
loaded = yaml.safe_load(raw_frontmatter) or {}
|
||||
except yaml.YAMLError:
|
||||
return {}
|
||||
if not isinstance(loaded, dict):
|
||||
return {}
|
||||
return {str(key): value for key, value in loaded.items()}
|
||||
|
||||
|
||||
def _format_yaml_value(value: Any) -> str:
|
||||
if value is None:
|
||||
return "null"
|
||||
if isinstance(value, bool):
|
||||
return "true" if value else "false"
|
||||
if isinstance(value, int):
|
||||
return str(value)
|
||||
if isinstance(value, (list, tuple)):
|
||||
return json.dumps(list(value), ensure_ascii=False)
|
||||
return json.dumps(str(value), ensure_ascii=False)
|
||||
|
||||
|
||||
def _mtime_timestamp(path: Path) -> str:
|
||||
try:
|
||||
modified = datetime.fromtimestamp(path.stat().st_mtime, tz=timezone.utc)
|
||||
except OSError:
|
||||
modified = utc_now()
|
||||
return format_datetime(modified)
|
||||
|
||||
|
||||
def _generate_unique_memory_id(
|
||||
*,
|
||||
now: datetime | None = None,
|
||||
seen_ids: set[str] | None = None,
|
||||
) -> str:
|
||||
while True:
|
||||
memory_id = generate_memory_id(now=now)
|
||||
if seen_ids is None or memory_id not in seen_ids:
|
||||
return memory_id
|
||||
|
||||
|
||||
def _as_bool(value: object, *, default: bool) -> bool:
|
||||
if isinstance(value, bool):
|
||||
return value
|
||||
if isinstance(value, str):
|
||||
lowered = value.strip().lower()
|
||||
if lowered in {"1", "true", "yes", "on"}:
|
||||
return True
|
||||
if lowered in {"0", "false", "no", "off"}:
|
||||
return False
|
||||
if value is None:
|
||||
return default
|
||||
return bool(value)
|
||||
|
||||
|
||||
def _as_optional_int(value: object) -> int | None:
|
||||
if value is None:
|
||||
return None
|
||||
if isinstance(value, str) and not value.strip():
|
||||
return None
|
||||
try:
|
||||
return int(value) # type: ignore[arg-type]
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
@@ -3,10 +3,13 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from datetime import timedelta
|
||||
from pathlib import Path
|
||||
|
||||
from openharness.memory.scan import scan_memory_files
|
||||
from openharness.memory.schema import parse_datetime, utc_now
|
||||
from openharness.memory.types import MemoryHeader
|
||||
from openharness.memory.usage import get_memory_usage
|
||||
|
||||
|
||||
def find_relevant_memories(
|
||||
@@ -32,8 +35,15 @@ def find_relevant_memories(
|
||||
# Metadata matches are weighted 2x; body matches 1x.
|
||||
meta_hits = sum(1 for t in tokens if t in meta)
|
||||
body_hits = sum(1 for t in tokens if t in body)
|
||||
score = meta_hits * 2.0 + body_hits
|
||||
if score > 0:
|
||||
usage = get_memory_usage(cwd, header.id, memory_dir=header.path.parent)
|
||||
score = (
|
||||
meta_hits * 2.0
|
||||
+ body_hits
|
||||
+ header.importance * 0.4
|
||||
+ min(int(usage["use_count"]), 5) * 0.1
|
||||
+ _recency_boost(header)
|
||||
)
|
||||
if meta_hits or body_hits:
|
||||
scored.append((score, header))
|
||||
|
||||
scored.sort(key=lambda item: (-item[0], -item[1].modified_at))
|
||||
@@ -47,3 +57,15 @@ def _tokenize(text: str) -> set[str]:
|
||||
# Han ideographs (each character carries independent meaning)
|
||||
han_chars = set(re.findall(r"[\u4e00-\u9fff\u3400-\u4dbf]", text))
|
||||
return ascii_tokens | han_chars
|
||||
|
||||
|
||||
def _recency_boost(header: MemoryHeader) -> float:
|
||||
timestamp = parse_datetime(header.updated_at) or parse_datetime(header.created_at)
|
||||
if timestamp is None:
|
||||
return 0.0
|
||||
age = utc_now() - timestamp
|
||||
if age <= timedelta(days=14):
|
||||
return 0.3
|
||||
if age <= timedelta(days=30):
|
||||
return 0.1
|
||||
return 0.0
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
"""Local team-memory vault helpers and safety guards."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from openharness.memory.paths import get_project_memory_dir
|
||||
from openharness.memory.schema import path_is_relative_to
|
||||
|
||||
TEAM_DIR_NAME = "team"
|
||||
MEMORY_INDEX = "MEMORY.md"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SecretMatch:
|
||||
"""A possible secret found in shared memory content."""
|
||||
|
||||
rule_id: str
|
||||
label: str
|
||||
|
||||
|
||||
SECRET_RULES: tuple[tuple[str, str, re.Pattern[str]], ...] = (
|
||||
("private-key", "private key", re.compile(r"-----BEGIN [A-Z ]*PRIVATE KEY-----")),
|
||||
("aws-access-key", "AWS access key", re.compile(r"\bAKIA[0-9A-Z]{16}\b")),
|
||||
("github-token", "GitHub token", re.compile(r"\bgh[pousr]_[A-Za-z0-9_]{20,}\b")),
|
||||
("openai-key", "OpenAI API key", re.compile(r"\bsk-[A-Za-z0-9_-]{20,}\b")),
|
||||
("anthropic-key", "Anthropic API key", re.compile(r"\bsk-ant-[A-Za-z0-9_-]{20,}\b")),
|
||||
("generic-secret", "secret assignment", re.compile(r"(?i)\b(secret|token|api[_-]?key|password)\s*[:=]\s*['\"]?[^'\"\s]{12,}")),
|
||||
)
|
||||
|
||||
|
||||
def get_team_memory_dir(cwd: str | Path) -> Path:
|
||||
"""Return the project-local shared team memory vault."""
|
||||
|
||||
return get_project_memory_dir(cwd) / TEAM_DIR_NAME
|
||||
|
||||
|
||||
def ensure_team_memory_vault(cwd: str | Path) -> Path:
|
||||
"""Create and return the team memory vault."""
|
||||
|
||||
team_dir = get_team_memory_dir(cwd)
|
||||
team_dir.mkdir(parents=True, exist_ok=True)
|
||||
entrypoint = team_dir / MEMORY_INDEX
|
||||
if not entrypoint.exists():
|
||||
entrypoint.write_text("# Memory Index\n", encoding="utf-8")
|
||||
return team_dir
|
||||
|
||||
|
||||
def validate_team_memory_write_path(cwd: str | Path, candidate: str | Path) -> tuple[Path | None, str | None]:
|
||||
"""Validate a write target against traversal and symlink escape."""
|
||||
|
||||
team_dir = ensure_team_memory_vault(cwd).resolve()
|
||||
path = Path(candidate).expanduser()
|
||||
if not path.is_absolute():
|
||||
path = team_dir / path
|
||||
resolved = path.resolve()
|
||||
if not path_is_relative_to(resolved, team_dir):
|
||||
return None, f"Path escapes team memory directory: {candidate}"
|
||||
parent = resolved.parent
|
||||
deepest = parent
|
||||
while not deepest.exists() and deepest != deepest.parent:
|
||||
deepest = deepest.parent
|
||||
if deepest.exists() and not path_is_relative_to(deepest.resolve(), team_dir):
|
||||
return None, f"Path escapes team memory directory via symlink: {candidate}"
|
||||
return resolved, None
|
||||
|
||||
|
||||
def scan_for_secrets(content: str) -> list[SecretMatch]:
|
||||
"""Return possible secrets in content without exposing matched values."""
|
||||
|
||||
matches: list[SecretMatch] = []
|
||||
for rule_id, label, pattern in SECRET_RULES:
|
||||
if pattern.search(content):
|
||||
matches.append(SecretMatch(rule_id=rule_id, label=label))
|
||||
return matches
|
||||
|
||||
|
||||
def check_team_memory_secrets(content: str) -> str | None:
|
||||
"""Return an error message when shared memory content appears sensitive."""
|
||||
|
||||
matches = scan_for_secrets(content)
|
||||
if not matches:
|
||||
return None
|
||||
labels = ", ".join(match.label for match in matches)
|
||||
return (
|
||||
f"Content contains potential secrets ({labels}) and cannot be written to team memory. "
|
||||
"Team memory is shared with project collaborators."
|
||||
)
|
||||
@@ -2,8 +2,9 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -16,3 +17,17 @@ class MemoryHeader:
|
||||
modified_at: float
|
||||
memory_type: str = ""
|
||||
body_preview: str = ""
|
||||
id: str = ""
|
||||
schema_version: int = 0
|
||||
category: str = ""
|
||||
importance: int = 0
|
||||
source: str = ""
|
||||
signature: str = ""
|
||||
created_at: str = ""
|
||||
updated_at: str = ""
|
||||
ttl_days: int | None = None
|
||||
disabled: bool = False
|
||||
supersedes: tuple[str, ...] = ()
|
||||
relative_path: str = ""
|
||||
tags: tuple[str, ...] = field(default_factory=tuple)
|
||||
frontmatter: dict[str, Any] = field(default_factory=dict)
|
||||
|
||||
@@ -0,0 +1,153 @@
|
||||
"""Usage index for recalled memory entries."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from datetime import timedelta
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from openharness.memory.paths import get_project_memory_dir
|
||||
from openharness.memory.scan import scan_memory_files
|
||||
from openharness.memory.schema import format_datetime, parse_datetime, utc_now
|
||||
from openharness.memory.types import MemoryHeader
|
||||
from openharness.utils.file_lock import exclusive_file_lock
|
||||
from openharness.utils.fs import atomic_write_text
|
||||
|
||||
USAGE_INDEX_NAME = "usage_index.json"
|
||||
STALE_UNUSED_DAYS = 60
|
||||
STALE_MAX_IMPORTANCE = 1
|
||||
|
||||
|
||||
def get_usage_index_path(cwd: str | Path, *, memory_dir: str | Path | None = None) -> Path:
|
||||
"""Return the usage index path for a memory store."""
|
||||
|
||||
root = Path(memory_dir) if memory_dir is not None else get_project_memory_dir(cwd)
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
return root / USAGE_INDEX_NAME
|
||||
|
||||
|
||||
def load_usage_index(cwd: str | Path, *, memory_dir: str | Path | None = None) -> dict[str, Any]:
|
||||
"""Load usage index data, returning an empty index for invalid files."""
|
||||
|
||||
path = get_usage_index_path(cwd, memory_dir=memory_dir)
|
||||
if not path.exists():
|
||||
return _empty_index()
|
||||
try:
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
except (OSError, json.JSONDecodeError):
|
||||
return _empty_index()
|
||||
if not isinstance(data, dict):
|
||||
return _empty_index()
|
||||
memories = data.get("memories")
|
||||
if not isinstance(memories, dict):
|
||||
memories = {}
|
||||
normalized = _empty_index()
|
||||
normalized["memories"] = {
|
||||
str(memory_id): _normalize_usage_record(record)
|
||||
for memory_id, record in memories.items()
|
||||
if isinstance(record, dict)
|
||||
}
|
||||
return normalized
|
||||
|
||||
|
||||
def save_usage_index(
|
||||
cwd: str | Path,
|
||||
index: dict[str, Any],
|
||||
*,
|
||||
memory_dir: str | Path | None = None,
|
||||
) -> None:
|
||||
"""Persist usage index data atomically."""
|
||||
|
||||
path = get_usage_index_path(cwd, memory_dir=memory_dir)
|
||||
payload = json.dumps(index, indent=2, ensure_ascii=False, sort_keys=True) + "\n"
|
||||
atomic_write_text(path, payload)
|
||||
|
||||
|
||||
def get_memory_usage(
|
||||
cwd: str | Path,
|
||||
memory_id: str,
|
||||
*,
|
||||
memory_dir: str | Path | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Return usage data for a memory id."""
|
||||
|
||||
if not memory_id:
|
||||
return _normalize_usage_record({})
|
||||
index = load_usage_index(cwd, memory_dir=memory_dir)
|
||||
record = index["memories"].get(memory_id, {})
|
||||
return _normalize_usage_record(record)
|
||||
|
||||
|
||||
def mark_memory_used(
|
||||
cwd: str | Path,
|
||||
memories: list[MemoryHeader],
|
||||
*,
|
||||
memory_dir: str | Path | None = None,
|
||||
) -> None:
|
||||
"""Record that memory entries were recalled into a runtime prompt."""
|
||||
|
||||
usable = [header for header in memories if header.id]
|
||||
if not usable:
|
||||
return
|
||||
resolved_memory_dir = Path(memory_dir) if memory_dir is not None else usable[0].path.parent
|
||||
lock_path = resolved_memory_dir / ".usage_index.lock"
|
||||
with exclusive_file_lock(lock_path):
|
||||
index = load_usage_index(cwd, memory_dir=resolved_memory_dir)
|
||||
now = format_datetime(utc_now())
|
||||
for header in usable:
|
||||
record = _normalize_usage_record(index["memories"].get(header.id, {}))
|
||||
record["use_count"] = int(record["use_count"]) + 1
|
||||
record["last_used_at"] = now
|
||||
record["path"] = header.path.name
|
||||
index["memories"][header.id] = record
|
||||
save_usage_index(cwd, index, memory_dir=resolved_memory_dir)
|
||||
|
||||
|
||||
def find_stale_memory_candidates(
|
||||
cwd: str | Path,
|
||||
*,
|
||||
memory_dir: str | Path | None = None,
|
||||
) -> list[MemoryHeader]:
|
||||
"""Return low-value unused memories that auto-dream should review for pruning."""
|
||||
|
||||
resolved_memory_dir = Path(memory_dir) if memory_dir is not None else None
|
||||
headers = scan_memory_files(
|
||||
cwd,
|
||||
max_files=None,
|
||||
include_disabled=False,
|
||||
include_expired=False,
|
||||
memory_dir=resolved_memory_dir,
|
||||
)
|
||||
now = utc_now()
|
||||
candidates: list[MemoryHeader] = []
|
||||
for header in headers:
|
||||
if header.importance > STALE_MAX_IMPORTANCE:
|
||||
continue
|
||||
usage = get_memory_usage(cwd, header.id, memory_dir=resolved_memory_dir or header.path.parent)
|
||||
if int(usage["use_count"]) > 0:
|
||||
continue
|
||||
updated_at = parse_datetime(header.updated_at) or parse_datetime(header.created_at)
|
||||
if updated_at is None:
|
||||
continue
|
||||
if now - updated_at >= timedelta(days=STALE_UNUSED_DAYS):
|
||||
candidates.append(header)
|
||||
candidates.sort(key=lambda item: (item.importance, item.updated_at or "", item.path.name))
|
||||
return candidates
|
||||
|
||||
|
||||
def _empty_index() -> dict[str, Any]:
|
||||
return {"version": 1, "memories": {}}
|
||||
|
||||
|
||||
def _normalize_usage_record(record: dict[str, Any]) -> dict[str, Any]:
|
||||
use_count = record.get("use_count", 0)
|
||||
try:
|
||||
use_count = max(0, int(use_count))
|
||||
except (TypeError, ValueError):
|
||||
use_count = 0
|
||||
return {
|
||||
"last_used_at": str(record.get("last_used_at") or ""),
|
||||
"use_count": use_count,
|
||||
"path": str(record.get("path") or ""),
|
||||
}
|
||||
@@ -37,7 +37,7 @@ def detect_platform(
|
||||
|
||||
if system == "darwin":
|
||||
return "macos"
|
||||
if system == "windows":
|
||||
if system in {"windows", "win32"}:
|
||||
return "windows"
|
||||
if system == "linux":
|
||||
if "microsoft" in kernel_release or env_map.get("WSL_DISTRO_NAME") or env_map.get("WSL_INTEROP"):
|
||||
@@ -84,4 +84,3 @@ def get_platform_capabilities(platform_name: PlatformName | None = None) -> Plat
|
||||
supports_sandbox_runtime=False,
|
||||
supports_docker_sandbox=False,
|
||||
)
|
||||
|
||||
|
||||
@@ -8,6 +8,18 @@ from pathlib import Path
|
||||
from openharness.plugins.loader import get_user_plugins_dir
|
||||
|
||||
|
||||
def _resolve_user_plugin_dir(name: str) -> Path:
|
||||
"""Resolve a user plugin name to a direct child of the plugin directory."""
|
||||
if not name or name != Path(name).name or "\\" in name:
|
||||
raise ValueError("invalid plugin name")
|
||||
|
||||
plugins_dir = get_user_plugins_dir().resolve()
|
||||
path = (plugins_dir / name).resolve()
|
||||
if path.parent != plugins_dir:
|
||||
raise ValueError("invalid plugin name")
|
||||
return path
|
||||
|
||||
|
||||
def install_plugin_from_path(source: str | Path) -> Path:
|
||||
"""Install a plugin directory into the user plugin directory."""
|
||||
src = Path(source).resolve()
|
||||
@@ -20,7 +32,7 @@ def install_plugin_from_path(source: str | Path) -> Path:
|
||||
|
||||
def uninstall_plugin(name: str) -> bool:
|
||||
"""Remove a user plugin by directory name."""
|
||||
path = get_user_plugins_dir() / name
|
||||
path = _resolve_user_plugin_dir(name)
|
||||
if not path.exists():
|
||||
return False
|
||||
shutil.rmtree(path)
|
||||
|
||||
@@ -2,9 +2,12 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import importlib.util
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any, Iterable
|
||||
|
||||
@@ -24,7 +27,7 @@ from openharness.coordinator.agent_definitions import (
|
||||
)
|
||||
from openharness.plugins.schemas import PluginManifest
|
||||
from openharness.plugins.types import LoadedPlugin, PluginCommandDefinition
|
||||
from openharness.skills.loader import _parse_skill_markdown
|
||||
from openharness.skills.loader import _parse_skill_metadata
|
||||
from openharness.skills.types import SkillDefinition
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -135,6 +138,7 @@ def load_plugin(path: Path, enabled_plugins: dict[str, bool]) -> LoadedPlugin |
|
||||
skills = _load_plugin_skills(path / manifest.skills_dir)
|
||||
commands = _load_plugin_commands(path, manifest)
|
||||
agents = _load_plugin_agents(path, manifest)
|
||||
tools = _load_plugin_tools(path, manifest) if enabled else []
|
||||
hooks = _load_plugin_hooks(path / manifest.hooks_file)
|
||||
hooks_dir_file = path / "hooks" / "hooks.json"
|
||||
if not hooks and hooks_dir_file.exists():
|
||||
@@ -154,6 +158,7 @@ def load_plugin(path: Path, enabled_plugins: dict[str, bool]) -> LoadedPlugin |
|
||||
agents=agents,
|
||||
hooks=hooks,
|
||||
mcp_servers=mcp,
|
||||
tools=tools,
|
||||
)
|
||||
|
||||
|
||||
@@ -251,7 +256,10 @@ def _load_plugin_skills(path: Path) -> list[SkillDefinition]:
|
||||
direct_skill = path / "SKILL.md"
|
||||
if direct_skill.exists():
|
||||
content = direct_skill.read_text(encoding="utf-8")
|
||||
name, description = _parse_skill_markdown(path.name, content)
|
||||
metadata = _parse_skill_metadata(path.name, content)
|
||||
name = metadata["name"]
|
||||
description = metadata["description"]
|
||||
display_name = name if name != path.name else None
|
||||
skills.append(
|
||||
SkillDefinition(
|
||||
name=name,
|
||||
@@ -259,6 +267,13 @@ def _load_plugin_skills(path: Path) -> list[SkillDefinition]:
|
||||
content=content,
|
||||
source="plugin",
|
||||
path=str(direct_skill),
|
||||
base_dir=str(path),
|
||||
command_name=path.name,
|
||||
display_name=display_name,
|
||||
user_invocable=metadata["user_invocable"],
|
||||
disable_model_invocation=metadata["disable_model_invocation"],
|
||||
model=metadata["model"],
|
||||
argument_hint=metadata["argument_hint"],
|
||||
)
|
||||
)
|
||||
return skills
|
||||
@@ -269,7 +284,10 @@ def _load_plugin_skills(path: Path) -> list[SkillDefinition]:
|
||||
if not skill_path.exists():
|
||||
continue
|
||||
content = skill_path.read_text(encoding="utf-8")
|
||||
name, description = _parse_skill_markdown(child.name, content)
|
||||
metadata = _parse_skill_metadata(child.name, content)
|
||||
name = metadata["name"]
|
||||
description = metadata["description"]
|
||||
display_name = name if name != child.name else None
|
||||
skills.append(
|
||||
SkillDefinition(
|
||||
name=name,
|
||||
@@ -277,6 +295,13 @@ def _load_plugin_skills(path: Path) -> list[SkillDefinition]:
|
||||
content=content,
|
||||
source="plugin",
|
||||
path=str(skill_path),
|
||||
base_dir=str(child),
|
||||
command_name=child.name,
|
||||
display_name=display_name,
|
||||
user_invocable=metadata["user_invocable"],
|
||||
disable_model_invocation=metadata["disable_model_invocation"],
|
||||
model=metadata["model"],
|
||||
argument_hint=metadata["argument_hint"],
|
||||
)
|
||||
)
|
||||
return skills
|
||||
@@ -661,3 +686,45 @@ def _load_plugin_mcp(path: Path) -> dict[str, object]:
|
||||
raw = json.loads(path.read_text(encoding="utf-8"))
|
||||
parsed = McpJsonConfig.model_validate(raw)
|
||||
return parsed.mcpServers
|
||||
|
||||
|
||||
def _load_plugin_tools(path: Path, manifest: PluginManifest) -> list:
|
||||
"""Discover and instantiate BaseTool subclasses from a plugin's tools/ directory."""
|
||||
from openharness.tools.base import BaseTool
|
||||
|
||||
tools_dir = path / manifest.tools_dir
|
||||
if not tools_dir.is_dir():
|
||||
return []
|
||||
|
||||
tools: list[BaseTool] = []
|
||||
for py_file in sorted(tools_dir.glob("*.py")):
|
||||
if py_file.name.startswith("_"):
|
||||
continue
|
||||
module_name = f"_plugin_tools_{manifest.name}_{py_file.stem}"
|
||||
try:
|
||||
spec = importlib.util.spec_from_file_location(module_name, py_file)
|
||||
if spec is None or spec.loader is None:
|
||||
continue
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[module_name] = module
|
||||
spec.loader.exec_module(module)
|
||||
except Exception:
|
||||
logger.debug("Failed to load plugin tool module %s", py_file, exc_info=True)
|
||||
continue
|
||||
|
||||
for attr_name in dir(module):
|
||||
attr = getattr(module, attr_name, None)
|
||||
if (
|
||||
isinstance(attr, type)
|
||||
and issubclass(attr, BaseTool)
|
||||
and attr is not BaseTool
|
||||
and hasattr(attr, "name")
|
||||
and hasattr(attr, "description")
|
||||
):
|
||||
try:
|
||||
instance = attr()
|
||||
tools.append(instance)
|
||||
logger.debug("Loaded plugin tool: %s from %s", instance.name, py_file)
|
||||
except Exception:
|
||||
logger.debug("Failed to instantiate tool %s from %s", attr_name, py_file, exc_info=True)
|
||||
return tools
|
||||
|
||||
@@ -13,6 +13,7 @@ class PluginManifest(BaseModel):
|
||||
description: str = ""
|
||||
enabled_by_default: bool = True
|
||||
skills_dir: str = "skills"
|
||||
tools_dir: str = "tools"
|
||||
hooks_file: str = "hooks.json"
|
||||
mcp_file: str = "mcp.json"
|
||||
# Extended fields: optional author, commands, agents, etc.
|
||||
|
||||
@@ -4,12 +4,16 @@ from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from openharness.coordinator.agent_definitions import AgentDefinition
|
||||
from openharness.mcp.types import McpServerConfig
|
||||
from openharness.plugins.schemas import PluginManifest
|
||||
from openharness.skills.types import SkillDefinition
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from openharness.tools.base import BaseTool
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PluginCommandDefinition:
|
||||
@@ -42,6 +46,7 @@ class LoadedPlugin:
|
||||
skills: list[SkillDefinition] = field(default_factory=list)
|
||||
commands: list[PluginCommandDefinition] = field(default_factory=list)
|
||||
agents: list[AgentDefinition] = field(default_factory=list)
|
||||
tools: list[BaseTool] = field(default_factory=list)
|
||||
hooks: dict[str, list] = field(default_factory=dict)
|
||||
mcp_servers: dict[str, McpServerConfig] = field(default_factory=dict)
|
||||
|
||||
|
||||
@@ -12,8 +12,11 @@ from openharness.config.paths import (
|
||||
)
|
||||
from openharness.config.settings import Settings
|
||||
from openharness.coordinator.coordinator_mode import get_coordinator_system_prompt, is_coordinator_mode
|
||||
from openharness.memory import find_relevant_memories, load_memory_prompt
|
||||
from openharness.memory import load_memory_prompt
|
||||
from openharness.memory.relevance import format_relevant_memories, select_relevant_memories
|
||||
from openharness.memory.usage import mark_memory_used
|
||||
from openharness.personalization.rules import load_local_rules
|
||||
from openharness.permissions.modes import PermissionMode
|
||||
from openharness.prompts.claudemd import load_claude_md_prompt
|
||||
from openharness.prompts.system_prompt import build_system_prompt
|
||||
from openharness.skills.loader import load_skill_registry
|
||||
@@ -33,7 +36,7 @@ def _build_skills_section(
|
||||
extra_plugin_roots=extra_plugin_roots,
|
||||
settings=settings,
|
||||
)
|
||||
skills = registry.list_skills()
|
||||
skills = [skill for skill in registry.list_skills() if not skill.disable_model_invocation]
|
||||
if not skills:
|
||||
return None
|
||||
lines = [
|
||||
@@ -41,11 +44,14 @@ def _build_skills_section(
|
||||
"",
|
||||
"The following skills are available via the `skill` tool. "
|
||||
"When a user's request matches a skill, invoke it with `skill(name=\"<skill_name>\")` "
|
||||
"to load detailed instructions before proceeding.",
|
||||
"to load detailed instructions before proceeding. "
|
||||
"User-invocable skills can also be run directly by the user as `/<skill-name>`.",
|
||||
"",
|
||||
]
|
||||
for skill in skills:
|
||||
lines.append(f"- **{skill.name}**: {skill.description}")
|
||||
command_name = skill.command_name or skill.name
|
||||
display = f" ({skill.display_name})" if skill.display_name else ""
|
||||
lines.append(f"- **{command_name}**{display}: {skill.description}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
@@ -71,6 +77,28 @@ def _build_delegation_section() -> str:
|
||||
)
|
||||
|
||||
|
||||
def _build_permission_mode_section(settings: Settings) -> str:
|
||||
"""Build current permission-mode guidance for the model."""
|
||||
mode = settings.permission.mode
|
||||
if mode == PermissionMode.PLAN:
|
||||
guidance = (
|
||||
"Plan mode is enabled. Treat this session as read-only planning and analysis. "
|
||||
"Do not call mutating tools such as file writes, edits, package installs, "
|
||||
"state-changing shell commands, or task-spawning actions unless the user exits plan mode."
|
||||
)
|
||||
elif mode == PermissionMode.FULL_AUTO:
|
||||
guidance = (
|
||||
"Full-auto permission mode is enabled. You may use mutating tools when they are necessary "
|
||||
"for the user's request, while still keeping changes scoped and intentional."
|
||||
)
|
||||
else:
|
||||
guidance = (
|
||||
"Default permission mode is enabled. Read-only tools can run directly; mutating tools "
|
||||
"may require explicit user approval."
|
||||
)
|
||||
return f"# Current Permission Mode\n{guidance}"
|
||||
|
||||
|
||||
def build_runtime_system_prompt(
|
||||
settings: Settings,
|
||||
*,
|
||||
@@ -78,6 +106,7 @@ def build_runtime_system_prompt(
|
||||
latest_user_prompt: str | None = None,
|
||||
extra_skill_dirs: Iterable[str | Path] | None = None,
|
||||
extra_plugin_roots: Iterable[str | Path] | None = None,
|
||||
include_project_memory: bool = True,
|
||||
) -> str:
|
||||
"""Build the runtime system prompt with project instructions and memory."""
|
||||
if is_coordinator_mode():
|
||||
@@ -88,6 +117,8 @@ def build_runtime_system_prompt(
|
||||
if not is_coordinator_mode() and settings.system_prompt is None:
|
||||
sections[0] = build_system_prompt(cwd=str(cwd))
|
||||
|
||||
sections.append(_build_permission_mode_section(settings))
|
||||
|
||||
if settings.fast_mode:
|
||||
sections.append(
|
||||
"# Session Mode\nFast mode is enabled. Prefer concise replies, minimal tool use, and quicker progress over exhaustive exploration."
|
||||
@@ -130,33 +161,27 @@ def build_runtime_system_prompt(
|
||||
if content:
|
||||
sections.append(f"# {title}\n\n```md\n{content[:12000]}\n```")
|
||||
|
||||
if settings.memory.enabled:
|
||||
if include_project_memory and settings.memory.enabled:
|
||||
memory_section = load_memory_prompt(
|
||||
cwd,
|
||||
max_entrypoint_lines=settings.memory.max_entrypoint_lines,
|
||||
max_entrypoint_bytes=settings.memory.max_entrypoint_bytes,
|
||||
)
|
||||
if memory_section:
|
||||
sections.append(memory_section)
|
||||
|
||||
if latest_user_prompt:
|
||||
relevant = find_relevant_memories(
|
||||
relevant = select_relevant_memories(
|
||||
latest_user_prompt,
|
||||
cwd,
|
||||
max_results=settings.memory.max_files,
|
||||
)
|
||||
if relevant:
|
||||
lines = ["# Relevant Memories"]
|
||||
for header in relevant:
|
||||
content = header.path.read_text(encoding="utf-8", errors="replace").strip()
|
||||
lines.extend(
|
||||
[
|
||||
"",
|
||||
f"## {header.path.name}",
|
||||
"```md",
|
||||
content[:8000],
|
||||
"```",
|
||||
]
|
||||
)
|
||||
sections.append("\n".join(lines))
|
||||
try:
|
||||
headers = [item.header for item in relevant]
|
||||
mark_memory_used(cwd, headers, memory_dir=headers[0].path.parent)
|
||||
except OSError:
|
||||
pass
|
||||
sections.append(format_relevant_memories(relevant))
|
||||
|
||||
return "\n\n".join(section for section in sections if section.strip())
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
"""Automatic memory consolidation (auto-dream)."""
|
||||
|
||||
from openharness.services.autodream.backup import (
|
||||
create_memory_backup,
|
||||
diff_memory_dirs,
|
||||
format_memory_diff,
|
||||
latest_memory_backup,
|
||||
restore_memory_backup,
|
||||
)
|
||||
from openharness.services.autodream.lock import (
|
||||
list_sessions_touched_since,
|
||||
read_last_consolidated_at,
|
||||
record_consolidation,
|
||||
rollback_consolidation_lock,
|
||||
try_acquire_consolidation_lock,
|
||||
)
|
||||
from openharness.services.autodream.prompt import build_consolidation_prompt
|
||||
from openharness.services.autodream.service import execute_auto_dream, start_dream_now
|
||||
|
||||
__all__ = [
|
||||
"build_consolidation_prompt",
|
||||
"create_memory_backup",
|
||||
"diff_memory_dirs",
|
||||
"execute_auto_dream",
|
||||
"format_memory_diff",
|
||||
"latest_memory_backup",
|
||||
"list_sessions_touched_since",
|
||||
"read_last_consolidated_at",
|
||||
"record_consolidation",
|
||||
"restore_memory_backup",
|
||||
"rollback_consolidation_lock",
|
||||
"start_dream_now",
|
||||
"try_acquire_consolidation_lock",
|
||||
]
|
||||
@@ -0,0 +1,104 @@
|
||||
"""Backup, diff, and rollback helpers for auto-dream memory directories."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import filecmp
|
||||
import shutil
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
from openharness.config.paths import get_data_dir
|
||||
|
||||
|
||||
def default_backup_root(memory_dir: str | Path, *, app_label: str = "openharness") -> Path:
|
||||
"""Return the backup root for a memory directory."""
|
||||
|
||||
memory_dir = Path(memory_dir).expanduser().resolve()
|
||||
if ".ohmo" in memory_dir.parts:
|
||||
try:
|
||||
idx = memory_dir.parts.index(".ohmo")
|
||||
return Path(*memory_dir.parts[: idx + 1]) / "backups"
|
||||
except ValueError:
|
||||
pass
|
||||
safe_label = "".join(ch if ch.isalnum() or ch in {"-", "_"} else "-" for ch in app_label).strip("-")
|
||||
return get_data_dir() / "memory-backups" / (safe_label or "openharness")
|
||||
|
||||
|
||||
def create_memory_backup(
|
||||
memory_dir: str | Path,
|
||||
*,
|
||||
backup_root: str | Path | None = None,
|
||||
app_label: str = "openharness",
|
||||
) -> Path:
|
||||
"""Create a timestamped copy of ``memory_dir`` and return the backup path."""
|
||||
|
||||
memory_dir = Path(memory_dir).expanduser().resolve()
|
||||
root = Path(backup_root).expanduser().resolve() if backup_root is not None else default_backup_root(memory_dir, app_label=app_label)
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
timestamp = time.strftime("memory-%Y%m%d-%H%M%S")
|
||||
backup = root / timestamp
|
||||
suffix = 1
|
||||
while backup.exists():
|
||||
suffix += 1
|
||||
backup = root / f"{timestamp}-{suffix}"
|
||||
if memory_dir.exists():
|
||||
shutil.copytree(memory_dir, backup, ignore=shutil.ignore_patterns(".consolidate-lock"))
|
||||
else:
|
||||
backup.mkdir(parents=True)
|
||||
return backup
|
||||
|
||||
|
||||
def diff_memory_dirs(before: str | Path, after: str | Path) -> dict[str, list[str]]:
|
||||
"""Return added/removed/changed file names between two memory dirs."""
|
||||
|
||||
before = Path(before).expanduser().resolve()
|
||||
after = Path(after).expanduser().resolve()
|
||||
before_files = {p.name: p for p in before.glob("*.md")} if before.exists() else {}
|
||||
after_files = {p.name: p for p in after.glob("*.md")} if after.exists() else {}
|
||||
added = sorted(set(after_files) - set(before_files))
|
||||
removed = sorted(set(before_files) - set(after_files))
|
||||
changed = sorted(
|
||||
name
|
||||
for name in set(before_files) & set(after_files)
|
||||
if not filecmp.cmp(before_files[name], after_files[name], shallow=False)
|
||||
)
|
||||
return {"added": added, "removed": removed, "changed": changed}
|
||||
|
||||
|
||||
def format_memory_diff(diff: dict[str, list[str]]) -> str:
|
||||
"""Format a compact memory diff summary."""
|
||||
|
||||
lines: list[str] = []
|
||||
for label in ("added", "changed", "removed"):
|
||||
values = diff.get(label, [])
|
||||
if values:
|
||||
lines.append(f"{label}: " + ", ".join(values))
|
||||
return "\n".join(lines) if lines else "no markdown file changes"
|
||||
|
||||
|
||||
def latest_memory_backup(memory_dir: str | Path, *, app_label: str = "openharness") -> Path | None:
|
||||
"""Return the latest backup for a memory directory, if any."""
|
||||
|
||||
root = default_backup_root(memory_dir, app_label=app_label)
|
||||
if not root.exists():
|
||||
return None
|
||||
backups = [path for path in root.iterdir() if path.is_dir() and path.name.startswith("memory-")]
|
||||
if not backups:
|
||||
return None
|
||||
return max(backups, key=lambda path: path.stat().st_mtime)
|
||||
|
||||
|
||||
def restore_memory_backup(backup_dir: str | Path, memory_dir: str | Path) -> None:
|
||||
"""Restore memory_dir from a backup directory."""
|
||||
|
||||
backup_dir = Path(backup_dir).expanduser().resolve()
|
||||
memory_dir = Path(memory_dir).expanduser().resolve()
|
||||
if not backup_dir.exists() or not backup_dir.is_dir():
|
||||
raise FileNotFoundError(f"Backup not found: {backup_dir}")
|
||||
tmp = memory_dir.with_name(f".{memory_dir.name}.restore-tmp")
|
||||
if tmp.exists():
|
||||
shutil.rmtree(tmp)
|
||||
shutil.copytree(backup_dir, tmp)
|
||||
if memory_dir.exists():
|
||||
shutil.rmtree(memory_dir)
|
||||
tmp.rename(memory_dir)
|
||||
@@ -0,0 +1,138 @@
|
||||
"""Locking and session scanning for auto-dream."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
from openharness.memory.paths import get_project_memory_dir
|
||||
from openharness.services.session_storage import get_project_session_dir
|
||||
from openharness.utils.fs import atomic_write_text
|
||||
|
||||
LOCK_FILE = ".consolidate-lock"
|
||||
HOLDER_STALE_SECONDS = 60 * 60
|
||||
|
||||
|
||||
def _lock_path(cwd: str | Path, memory_dir: str | Path | None = None) -> Path:
|
||||
return Path(memory_dir) / LOCK_FILE if memory_dir is not None else get_project_memory_dir(cwd) / LOCK_FILE
|
||||
|
||||
|
||||
def read_last_consolidated_at(cwd: str | Path, memory_dir: str | Path | None = None) -> float:
|
||||
"""Return lock mtime as the last successful consolidation timestamp."""
|
||||
|
||||
try:
|
||||
return _lock_path(cwd, memory_dir).stat().st_mtime
|
||||
except OSError:
|
||||
return 0.0
|
||||
|
||||
|
||||
def _holder_pid(path: Path) -> int | None:
|
||||
try:
|
||||
raw = path.read_text(encoding="utf-8").strip()
|
||||
pid = int(raw)
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
return pid if pid > 0 else None
|
||||
|
||||
|
||||
def _is_process_running(pid: int) -> bool:
|
||||
if pid == os.getpid():
|
||||
return True
|
||||
try:
|
||||
os.kill(pid, 0)
|
||||
except ProcessLookupError:
|
||||
return False
|
||||
except PermissionError:
|
||||
return True
|
||||
return True
|
||||
|
||||
|
||||
def try_acquire_consolidation_lock(cwd: str | Path, memory_dir: str | Path | None = None) -> float | None:
|
||||
"""Acquire the consolidation lock and return prior mtime, or None if held."""
|
||||
|
||||
path = _lock_path(cwd, memory_dir)
|
||||
prior_mtime: float | None = None
|
||||
try:
|
||||
stat = path.stat()
|
||||
prior_mtime = stat.st_mtime
|
||||
holder = _holder_pid(path)
|
||||
except OSError:
|
||||
holder = None
|
||||
|
||||
if prior_mtime is not None and time.time() - prior_mtime < HOLDER_STALE_SECONDS:
|
||||
if holder is not None and _is_process_running(holder):
|
||||
return None
|
||||
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
atomic_write_text(path, f"{os.getpid()}\n")
|
||||
try:
|
||||
if _holder_pid(path) != os.getpid():
|
||||
return None
|
||||
except OSError:
|
||||
return None
|
||||
return prior_mtime or 0.0
|
||||
|
||||
|
||||
def rollback_consolidation_lock(
|
||||
cwd: str | Path,
|
||||
prior_mtime: float,
|
||||
memory_dir: str | Path | None = None,
|
||||
) -> None:
|
||||
"""Restore lock mtime to its pre-acquire value after failed/killed dream."""
|
||||
|
||||
path = _lock_path(cwd, memory_dir)
|
||||
try:
|
||||
if prior_mtime <= 0:
|
||||
path.unlink(missing_ok=True)
|
||||
return
|
||||
atomic_write_text(path, "")
|
||||
os.utime(path, (prior_mtime, prior_mtime))
|
||||
except OSError:
|
||||
# Best effort: a failed rollback only delays the next auto trigger.
|
||||
return
|
||||
|
||||
|
||||
def record_consolidation(cwd: str | Path, memory_dir: str | Path | None = None) -> None:
|
||||
"""Stamp a manual consolidation time."""
|
||||
|
||||
path = _lock_path(cwd, memory_dir)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
atomic_write_text(path, f"{os.getpid()}\n")
|
||||
|
||||
|
||||
def list_sessions_touched_since(
|
||||
cwd: str | Path,
|
||||
since_ts: float,
|
||||
*,
|
||||
current_session_id: str | None = None,
|
||||
session_dir: str | Path | None = None,
|
||||
) -> list[str]:
|
||||
"""Return saved session IDs whose snapshot files were touched after ``since_ts``."""
|
||||
|
||||
resolved_session_dir = Path(session_dir) if session_dir is not None else get_project_session_dir(cwd)
|
||||
session_ids: list[str] = []
|
||||
seen: set[str] = set()
|
||||
for path in sorted(resolved_session_dir.glob("session-*.json"), key=lambda item: item.stat().st_mtime, reverse=True):
|
||||
try:
|
||||
mtime = path.stat().st_mtime
|
||||
except OSError:
|
||||
continue
|
||||
if mtime <= since_ts:
|
||||
continue
|
||||
session_id = path.stem.removeprefix("session-")
|
||||
try:
|
||||
payload = json.loads(path.read_text(encoding="utf-8"))
|
||||
raw_id = payload.get("session_id")
|
||||
if isinstance(raw_id, str) and raw_id.strip():
|
||||
session_id = raw_id.strip()
|
||||
except (OSError, json.JSONDecodeError):
|
||||
pass
|
||||
if current_session_id and session_id == current_session_id:
|
||||
continue
|
||||
if session_id in seen:
|
||||
continue
|
||||
seen.add(session_id)
|
||||
session_ids.append(session_id)
|
||||
return session_ids
|
||||
@@ -0,0 +1,128 @@
|
||||
"""Prompt builder for memory consolidation dreams."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date
|
||||
from pathlib import Path
|
||||
|
||||
MAX_ENTRYPOINT_LINES = 200
|
||||
ENTRYPOINT_NAME = "MEMORY.md"
|
||||
|
||||
|
||||
def build_consolidation_prompt(
|
||||
memory_root: str | Path,
|
||||
session_dir: str | Path,
|
||||
extra: str = "",
|
||||
*,
|
||||
preview: bool = False,
|
||||
) -> str:
|
||||
"""Build the dream prompt used by manual and automatic memory consolidation."""
|
||||
|
||||
memory_root = Path(memory_root)
|
||||
session_dir = Path(session_dir)
|
||||
extra_section = f"\n\n## Additional context\n\n{extra.strip()}" if extra.strip() else ""
|
||||
write_mode = "PREVIEW MODE: do not write files; propose a concise patch plan only." if preview else "APPLY MODE: update memory files directly when changes are clearly warranted."
|
||||
return f"""# Dream: Memory Consolidation
|
||||
|
||||
You are performing a dream — a reflective pass over OpenHarness/ohmo memory files. Synthesize recent signal into durable, well-organized memories so future sessions can orient quickly.
|
||||
|
||||
Current date: {date.today().isoformat()}
|
||||
Memory directory: `{memory_root}`
|
||||
Session snapshots: `{session_dir}` (JSON files can be large; inspect narrowly, do not dump everything)
|
||||
Mode: {write_mode}
|
||||
|
||||
---
|
||||
|
||||
## Non-negotiable memory policy
|
||||
|
||||
### Evidence discipline
|
||||
|
||||
- Do not infer user mistakes, motives, personality traits, or habits from incidental logs/config.
|
||||
- Only record facts directly supported by user statements, repeated behavior, or explicit artifacts.
|
||||
- Prefer neutral safety policies over accusations.
|
||||
- If a secret appears in context, do not copy it. Record only a generic safety reminder if useful.
|
||||
- Never preserve API keys, tokens, app secrets, verification tokens, credential-bearing URLs, or bearer strings.
|
||||
|
||||
### Classify every fact before writing
|
||||
|
||||
Use these categories:
|
||||
|
||||
1. **Stable Preference** — user-stated or repeatedly demonstrated durable preference.
|
||||
2. **Durable Project Context** — repo paths, canonical repos, project boundaries, validation commands.
|
||||
3. **Recent Snapshot** — active branches, current commits, temporary worktrees, recent test counts. Must include `Last observed: YYYY-MM-DD` and a reminder to verify current state.
|
||||
4. **Sensitive/Private Context** — revenue, personal identity, private repos, business metrics. Must include `Privacy: personal/private; do not share externally or in group chats unless explicitly asked.`
|
||||
5. **Operational Reminder** — short safety or workflow reminders.
|
||||
|
||||
### Staleness and scope
|
||||
|
||||
- Short-lived facts must be marked as snapshots, not permanent truths.
|
||||
- Prefer updating existing files over creating new ones.
|
||||
- Create at most 2 new markdown files in one dream.
|
||||
- If a topic is transient, prefer `recent_work.md` or an existing status file over a new topic file.
|
||||
- Do not move personal/business context into project memory; keep sensitive personal context in personal memory only.
|
||||
- Every top-level memory file must include schema-v1 frontmatter with:
|
||||
`schema_version`, `id`, `name`, `description`, `type`, `category`, `importance`, `source`,
|
||||
`signature`, `created_at`, `updated_at`, `ttl_days`, `disabled`, and `supersedes`.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — Orient
|
||||
|
||||
- List the memory directory to see what already exists.
|
||||
- Read `{ENTRYPOINT_NAME}` if present; it is the memory index.
|
||||
- Skim existing topic files so you update or merge instead of creating duplicates.
|
||||
|
||||
## Phase 2 — Gather recent signal
|
||||
|
||||
Look for information worth persisting. Sources in rough priority order:
|
||||
|
||||
1. Existing memory files that may need updates or contradiction fixes.
|
||||
2. Recent session snapshots (`session-*.json`) when you need concrete context.
|
||||
3. Focused grep/search terms based on recent work; avoid exhaustive transcript reading.
|
||||
|
||||
Skip idle chats, failed retry noise, implementation details that only matter for the current turn, and facts that cannot be supported.
|
||||
|
||||
## Phase 3 — Consolidate
|
||||
|
||||
For each durable thing worth remembering, write or update concise top-level markdown files in the memory directory.
|
||||
|
||||
Focus on:
|
||||
- Merging new signal into existing topic files rather than creating near-duplicates.
|
||||
- Converting relative dates ("yesterday", "last week") to absolute dates.
|
||||
- Correcting or deleting contradicted facts at the source.
|
||||
- Keeping memories useful for future sessions, not as raw transcripts.
|
||||
- Adding `Last observed` for snapshots and `Privacy` for sensitive/private context.
|
||||
|
||||
## Phase 4 — Prune and index
|
||||
|
||||
Update `{ENTRYPOINT_NAME}` so it stays under {MAX_ENTRYPOINT_LINES} lines and remains an index, not a content dump.
|
||||
|
||||
- Each entry should be one concise line: `- [Title](file.md): one-line hook`.
|
||||
- Remove pointers to memories that are stale, wrong, or superseded.
|
||||
- For stale, wrong, or superseded memory files, set `disabled: true`; do not delete files.
|
||||
- Treat usage-based stale candidates as review candidates, not automatic deletion instructions.
|
||||
- Add pointers to newly important memories.
|
||||
- Resolve contradictions if multiple files disagree.
|
||||
|
||||
## Required final response
|
||||
|
||||
Return a structured summary:
|
||||
|
||||
```md
|
||||
## Dream Summary
|
||||
Changed:
|
||||
- file.md: what changed
|
||||
|
||||
Confidence:
|
||||
- High: directly supported facts
|
||||
- Medium: recent snapshots that should be verified before use
|
||||
- Low: uncertain/stale candidates not written as facts
|
||||
|
||||
Privacy:
|
||||
- Any private/sensitive context touched and how it is marked
|
||||
|
||||
Stale candidates:
|
||||
- Items that may need review later
|
||||
```
|
||||
|
||||
If nothing changed, say so and explain why.{extra_section}"""
|
||||
@@ -0,0 +1,313 @@
|
||||
"""Auto-dream service."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
from openharness.config.settings import Settings
|
||||
from openharness.memory.paths import get_project_memory_dir
|
||||
from openharness.memory.usage import find_stale_memory_candidates
|
||||
from openharness.services.autodream.backup import create_memory_backup, diff_memory_dirs
|
||||
from openharness.services.autodream.lock import (
|
||||
list_sessions_touched_since,
|
||||
read_last_consolidated_at,
|
||||
rollback_consolidation_lock,
|
||||
try_acquire_consolidation_lock,
|
||||
)
|
||||
from openharness.services.autodream.prompt import build_consolidation_prompt
|
||||
from openharness.services.session_storage import get_project_session_dir
|
||||
from openharness.tasks.manager import get_task_manager
|
||||
from openharness.tasks.types import TaskRecord
|
||||
|
||||
SESSION_SCAN_INTERVAL_SECONDS = 10 * 60
|
||||
_CHILD_ENV = "OPENHARNESS_AUTODREAM_CHILD"
|
||||
_last_session_scan_at: dict[str, float] = {}
|
||||
_listener_registered = False
|
||||
|
||||
|
||||
def _enabled(settings: Settings) -> bool:
|
||||
return bool(settings.memory.enabled and settings.memory.auto_dream_enabled)
|
||||
|
||||
|
||||
def _has_dream_signal(session_ids: list[str], *, force: bool) -> bool:
|
||||
"""Return whether recent sessions are worth consolidating."""
|
||||
|
||||
if force:
|
||||
return True
|
||||
return bool(session_ids)
|
||||
|
||||
|
||||
def _memory_files_mtime_snapshot(memory_dir: Path) -> dict[str, float]:
|
||||
snapshot: dict[str, float] = {}
|
||||
for path in memory_dir.glob("*.md"):
|
||||
try:
|
||||
snapshot[path.name] = path.stat().st_mtime
|
||||
except OSError:
|
||||
continue
|
||||
return snapshot
|
||||
|
||||
|
||||
def _files_changed_since(memory_dir: Path, before: dict[str, float]) -> list[str]:
|
||||
changed: list[str] = []
|
||||
for path in sorted(memory_dir.glob("*.md")):
|
||||
try:
|
||||
mtime = path.stat().st_mtime
|
||||
except OSError:
|
||||
continue
|
||||
if before.get(path.name) != mtime:
|
||||
changed.append(path.name)
|
||||
return changed
|
||||
|
||||
|
||||
def _ensure_listener_registered() -> None:
|
||||
global _listener_registered
|
||||
if _listener_registered:
|
||||
return
|
||||
|
||||
async def _listener(task: TaskRecord) -> None:
|
||||
if task.type != "dream":
|
||||
return
|
||||
prior_raw = task.metadata.get("prior_mtime", "")
|
||||
memory_dir = task.metadata.get("memory_dir") or None
|
||||
if not prior_raw:
|
||||
return
|
||||
try:
|
||||
prior_mtime = float(prior_raw)
|
||||
except ValueError:
|
||||
return
|
||||
if task.status in {"failed", "killed"} or task.metadata.get("preview") == "true":
|
||||
rollback_consolidation_lock(task.cwd, prior_mtime, memory_dir=memory_dir)
|
||||
|
||||
get_task_manager().register_completion_listener(_listener)
|
||||
_listener_registered = True
|
||||
|
||||
|
||||
def _resolve_memory_dir(cwd: str | Path, memory_dir: str | Path | None) -> Path:
|
||||
return Path(memory_dir).expanduser().resolve() if memory_dir is not None else get_project_memory_dir(cwd)
|
||||
|
||||
|
||||
def _resolve_session_dir(cwd: str | Path, session_dir: str | Path | None) -> Path:
|
||||
return Path(session_dir).expanduser().resolve() if session_dir is not None else get_project_session_dir(cwd)
|
||||
|
||||
|
||||
async def start_dream_now(
|
||||
*,
|
||||
cwd: str | Path,
|
||||
settings: Settings,
|
||||
model: str | None = None,
|
||||
current_session_id: str | None = None,
|
||||
force: bool = False,
|
||||
memory_dir: str | Path | None = None,
|
||||
session_dir: str | Path | None = None,
|
||||
app_label: str = "openharness",
|
||||
runner_module: str = "openharness",
|
||||
preview: bool = False,
|
||||
) -> TaskRecord | None:
|
||||
"""Start a dream task immediately, optionally bypassing time/session gates."""
|
||||
|
||||
if os.environ.get(_CHILD_ENV):
|
||||
return None
|
||||
if not settings.memory.enabled:
|
||||
return None
|
||||
|
||||
cwd = Path(cwd).resolve()
|
||||
resolved_memory_dir = _resolve_memory_dir(cwd, memory_dir)
|
||||
resolved_session_dir = _resolve_session_dir(cwd, session_dir)
|
||||
last_at = read_last_consolidated_at(cwd, memory_dir=resolved_memory_dir)
|
||||
session_ids = list_sessions_touched_since(
|
||||
cwd,
|
||||
last_at,
|
||||
current_session_id=current_session_id,
|
||||
session_dir=resolved_session_dir,
|
||||
)
|
||||
if not force:
|
||||
hours_since = (time.time() - last_at) / 3600
|
||||
if hours_since < settings.memory.auto_dream_min_hours:
|
||||
return None
|
||||
if len(session_ids) < settings.memory.auto_dream_min_sessions:
|
||||
return None
|
||||
if not _has_dream_signal(session_ids, force=force):
|
||||
return None
|
||||
|
||||
prior_mtime = try_acquire_consolidation_lock(cwd, memory_dir=resolved_memory_dir)
|
||||
if prior_mtime is None:
|
||||
return None
|
||||
|
||||
_ensure_listener_registered()
|
||||
resolved_memory_dir.mkdir(parents=True, exist_ok=True)
|
||||
resolved_session_dir.mkdir(parents=True, exist_ok=True)
|
||||
before = _memory_files_mtime_snapshot(resolved_memory_dir)
|
||||
backup_dir = create_memory_backup(resolved_memory_dir, app_label=app_label) if not preview else None
|
||||
stale_candidates = find_stale_memory_candidates(cwd, memory_dir=resolved_memory_dir)
|
||||
stale_section = "\n".join(
|
||||
f"- {header.id or header.path.name}: {header.path.name} "
|
||||
f"(importance={header.importance}, updated_at={header.updated_at or 'unknown'})"
|
||||
for header in stale_candidates[:20]
|
||||
) or "- (none)"
|
||||
extra = (
|
||||
f"Application context: `{app_label}`.\n"
|
||||
"Tool constraints for this run: only modify files under the memory directory. "
|
||||
"Use shell commands only for read-only inspection.\n\n"
|
||||
f"Sessions since last consolidation ({len(session_ids)}):\n"
|
||||
+ "\n".join(f"- {session_id}" for session_id in session_ids)
|
||||
+ "\n\nUsage-based stale candidates:\n"
|
||||
+ stale_section
|
||||
)
|
||||
prompt = build_consolidation_prompt(resolved_memory_dir, resolved_session_dir, extra, preview=preview)
|
||||
src_root = Path(__file__).resolve().parents[3]
|
||||
existing_pythonpath = os.environ.get("PYTHONPATH", "")
|
||||
env = {
|
||||
_CHILD_ENV: "1",
|
||||
"OPENHARNESS_AUTODREAM_MEMORY_DIR": str(resolved_memory_dir),
|
||||
"OPENHARNESS_CONFIG_DIR": str(Path.home() / ".openharness"),
|
||||
"OPENHARNESS_PROFILE": settings.active_profile,
|
||||
"PYTHONPATH": str(src_root) + ((os.pathsep + existing_pythonpath) if existing_pythonpath else ""),
|
||||
}
|
||||
try:
|
||||
argv = [
|
||||
sys.executable,
|
||||
"-m",
|
||||
runner_module,
|
||||
]
|
||||
if runner_module == "openharness":
|
||||
argv.append("--dangerously-skip-permissions")
|
||||
if runner_module == "ohmo":
|
||||
workspace = resolved_memory_dir.parent
|
||||
argv.extend(["--workspace", str(workspace)])
|
||||
if settings.active_profile:
|
||||
argv.extend(["--profile", settings.active_profile])
|
||||
if model:
|
||||
argv.extend(["--model", model])
|
||||
if runner_module == "openharness" and settings.provider != "anthropic_claude":
|
||||
if settings.base_url:
|
||||
argv.extend(["--base-url", settings.base_url])
|
||||
if settings.api_format:
|
||||
argv.extend(["--api-format", settings.api_format])
|
||||
try:
|
||||
auth = settings.resolve_auth()
|
||||
if runner_module == "openharness" and auth.auth_kind == "api_key":
|
||||
argv.extend(["--api-key", auth.value])
|
||||
elif runner_module == "ohmo" and auth.auth_kind == "api_key":
|
||||
env["OPENHARNESS_API_KEY"] = auth.value
|
||||
elif auth.value:
|
||||
env["ANTHROPIC_AUTH_TOKEN"] = auth.value
|
||||
env.pop("ANTHROPIC_API_KEY", None)
|
||||
env.pop("OPENAI_API_KEY", None)
|
||||
env.pop("OPENHARNESS_API_KEY", None)
|
||||
except Exception:
|
||||
pass
|
||||
argv.extend(["--print", prompt])
|
||||
task = await get_task_manager().create_shell_task(
|
||||
description="dreaming",
|
||||
cwd=cwd,
|
||||
task_type="dream",
|
||||
env=env,
|
||||
argv=argv,
|
||||
)
|
||||
task.prompt = prompt
|
||||
except Exception:
|
||||
rollback_consolidation_lock(cwd, prior_mtime, memory_dir=resolved_memory_dir)
|
||||
raise
|
||||
|
||||
task.metadata.update(
|
||||
{
|
||||
"phase": "starting",
|
||||
"sessions_reviewing": str(len(session_ids)),
|
||||
"prior_mtime": str(prior_mtime),
|
||||
"memory_dir": str(resolved_memory_dir),
|
||||
"session_dir": str(resolved_session_dir),
|
||||
"force": str(force).lower(),
|
||||
"app_label": app_label,
|
||||
"runner_module": runner_module,
|
||||
"preview": str(preview).lower(),
|
||||
"backup_dir": str(backup_dir or ""),
|
||||
}
|
||||
)
|
||||
|
||||
async def _mark_changed_on_completion(done: TaskRecord) -> None:
|
||||
if done.id != task.id or done.status != "completed":
|
||||
return
|
||||
changed = _files_changed_since(resolved_memory_dir, before)
|
||||
if backup_dir is not None:
|
||||
diff = diff_memory_dirs(backup_dir, resolved_memory_dir)
|
||||
done.metadata["files_added"] = "\n".join(diff["added"])
|
||||
done.metadata["files_changed"] = "\n".join(diff["changed"])
|
||||
done.metadata["files_removed"] = "\n".join(diff["removed"])
|
||||
if changed:
|
||||
done.metadata["phase"] = "updating"
|
||||
done.metadata["files_touched"] = "\n".join(changed)
|
||||
|
||||
get_task_manager().register_completion_listener(_mark_changed_on_completion)
|
||||
return task
|
||||
|
||||
|
||||
async def execute_auto_dream(
|
||||
*,
|
||||
cwd: str | Path,
|
||||
settings: Settings,
|
||||
model: str | None = None,
|
||||
current_session_id: str | None = None,
|
||||
memory_dir: str | Path | None = None,
|
||||
session_dir: str | Path | None = None,
|
||||
app_label: str = "openharness",
|
||||
runner_module: str = "openharness",
|
||||
preview: bool = False,
|
||||
) -> TaskRecord | None:
|
||||
"""Run the cheap auto-dream gates and start a background dream when eligible."""
|
||||
|
||||
if os.environ.get(_CHILD_ENV):
|
||||
return None
|
||||
if not _enabled(settings):
|
||||
return None
|
||||
|
||||
cwd = Path(cwd).resolve()
|
||||
resolved_memory_dir = _resolve_memory_dir(cwd, memory_dir)
|
||||
resolved_session_dir = _resolve_session_dir(cwd, session_dir)
|
||||
last_at = read_last_consolidated_at(cwd, memory_dir=resolved_memory_dir)
|
||||
hours_since = (time.time() - last_at) / 3600
|
||||
if hours_since < settings.memory.auto_dream_min_hours:
|
||||
return None
|
||||
|
||||
key = str(resolved_memory_dir)
|
||||
now = time.time()
|
||||
if now - _last_session_scan_at.get(key, 0) < SESSION_SCAN_INTERVAL_SECONDS:
|
||||
return None
|
||||
_last_session_scan_at[key] = now
|
||||
|
||||
session_ids = list_sessions_touched_since(
|
||||
cwd,
|
||||
last_at,
|
||||
current_session_id=current_session_id,
|
||||
session_dir=resolved_session_dir,
|
||||
)
|
||||
if len(session_ids) < settings.memory.auto_dream_min_sessions:
|
||||
return None
|
||||
if not _has_dream_signal(session_ids, force=False):
|
||||
return None
|
||||
|
||||
return await start_dream_now(
|
||||
cwd=cwd,
|
||||
settings=settings,
|
||||
model=model,
|
||||
current_session_id=current_session_id,
|
||||
force=False,
|
||||
memory_dir=resolved_memory_dir,
|
||||
session_dir=resolved_session_dir,
|
||||
app_label=app_label,
|
||||
runner_module=runner_module,
|
||||
preview=preview,
|
||||
)
|
||||
|
||||
|
||||
def schedule_auto_dream(**kwargs: object) -> None:
|
||||
"""Fire-and-forget auto-dream scheduling."""
|
||||
|
||||
try:
|
||||
loop = asyncio.get_running_loop()
|
||||
except RuntimeError:
|
||||
return
|
||||
loop.create_task(execute_auto_dream(**kwargs)) # type: ignore[arg-type]
|
||||
@@ -11,6 +11,7 @@ from __future__ import annotations
|
||||
import asyncio
|
||||
import inspect
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
@@ -24,9 +25,11 @@ from openharness.engine.messages import (
|
||||
TextBlock,
|
||||
ToolResultBlock,
|
||||
ToolUseBlock,
|
||||
sanitize_conversation_messages,
|
||||
)
|
||||
from openharness.engine.stream_events import CompactProgressEvent
|
||||
from openharness.hooks import HookEvent, HookExecutor
|
||||
from openharness.services.tool_outputs import is_microcompactable_tool_result
|
||||
from openharness.services.token_estimation import estimate_tokens
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
@@ -70,6 +73,7 @@ DEFAULT_GAP_THRESHOLD_MINUTES = 60
|
||||
|
||||
# Token estimation padding (conservative)
|
||||
TOKEN_ESTIMATION_PADDING = 4 / 3
|
||||
_DEFAULT_VISION_IMAGE_TOKEN_ESTIMATE = 3_072
|
||||
|
||||
# Default context windows per model family
|
||||
_DEFAULT_CONTEXT_WINDOW = 200_000
|
||||
@@ -112,6 +116,7 @@ class CompactionResult:
|
||||
def estimate_message_tokens(messages: list[ConversationMessage]) -> int:
|
||||
"""Estimate total tokens for a conversation, including the 4/3 padding."""
|
||||
total = 0
|
||||
image_token_estimate = _vision_token_budget_per_image()
|
||||
for msg in messages:
|
||||
for block in msg.content:
|
||||
if isinstance(block, TextBlock):
|
||||
@@ -121,6 +126,8 @@ def estimate_message_tokens(messages: list[ConversationMessage]) -> int:
|
||||
elif isinstance(block, ToolUseBlock):
|
||||
total += estimate_tokens(block.name)
|
||||
total += estimate_tokens(str(block.input))
|
||||
elif isinstance(block, ImageBlock):
|
||||
total += image_token_estimate
|
||||
return int(total * TOKEN_ESTIMATION_PADDING)
|
||||
|
||||
|
||||
@@ -129,6 +136,42 @@ def estimate_conversation_tokens(messages: list[ConversationMessage]) -> int:
|
||||
return estimate_message_tokens(messages)
|
||||
|
||||
|
||||
def _vision_token_budget_per_image() -> int:
|
||||
raw = os.environ.get("OPENHARNESS_IMAGE_TOKEN_ESTIMATE", "").strip()
|
||||
if raw:
|
||||
try:
|
||||
return max(64, int(raw))
|
||||
except ValueError:
|
||||
log.warning("Ignoring invalid OPENHARNESS_IMAGE_TOKEN_ESTIMATE=%r", raw)
|
||||
return _DEFAULT_VISION_IMAGE_TOKEN_ESTIMATE
|
||||
|
||||
|
||||
def _replace_images_with_compaction_placeholders(
|
||||
messages: list[ConversationMessage],
|
||||
) -> list[ConversationMessage]:
|
||||
"""Strip image payloads from summarizer-only compact requests."""
|
||||
replaced: list[ConversationMessage] = []
|
||||
for message in messages:
|
||||
next_content: list[ContentBlock] = []
|
||||
changed = False
|
||||
for block in message.content:
|
||||
if isinstance(block, ImageBlock):
|
||||
changed = True
|
||||
label = block.source_path.strip() or "inline"
|
||||
next_content.append(
|
||||
TextBlock(
|
||||
text=f"[Image omitted from compaction summarization; source: {label}.]\n"
|
||||
)
|
||||
)
|
||||
else:
|
||||
next_content.append(block)
|
||||
if changed:
|
||||
replaced.append(message.model_copy(update={"content": next_content}))
|
||||
else:
|
||||
replaced.append(message)
|
||||
return replaced
|
||||
|
||||
|
||||
def _sanitize_metadata(value: Any) -> Any:
|
||||
if isinstance(value, (str, int, float, bool)) or value is None:
|
||||
return value
|
||||
@@ -209,11 +252,20 @@ def _is_prompt_too_long_error(exc: Exception) -> bool:
|
||||
needle in text
|
||||
for needle in (
|
||||
"prompt too long",
|
||||
"context_length_exceeded",
|
||||
"context length",
|
||||
"maximum context",
|
||||
"context window",
|
||||
"input tokens exceed",
|
||||
"messages resulted in",
|
||||
"reduce the length of the messages",
|
||||
"configured limit",
|
||||
"too many tokens",
|
||||
"too large for the model",
|
||||
"maximum context length",
|
||||
"exceed_context",
|
||||
"exceeds the available context size",
|
||||
"available context size",
|
||||
)
|
||||
)
|
||||
|
||||
@@ -256,8 +308,7 @@ def try_context_collapse(
|
||||
if len(messages) <= preserve_recent + 2:
|
||||
return None
|
||||
|
||||
older = messages[:-preserve_recent]
|
||||
newer = messages[-preserve_recent:]
|
||||
older, newer = _split_preserving_tool_pairs(messages, preserve_recent=preserve_recent)
|
||||
changed = False
|
||||
collapsed_older: list[ConversationMessage] = []
|
||||
for message in older:
|
||||
@@ -268,6 +319,17 @@ def try_context_collapse(
|
||||
if collapsed != block.text:
|
||||
changed = True
|
||||
new_blocks.append(TextBlock(text=collapsed))
|
||||
elif isinstance(block, ToolResultBlock):
|
||||
collapsed = _collapse_text(block.content)
|
||||
if collapsed != block.content:
|
||||
changed = True
|
||||
new_blocks.append(
|
||||
ToolResultBlock(
|
||||
tool_use_id=block.tool_use_id,
|
||||
content=collapsed,
|
||||
is_error=block.is_error,
|
||||
)
|
||||
)
|
||||
else:
|
||||
new_blocks.append(block)
|
||||
collapsed_older.append(ConversationMessage(role=message.role, content=new_blocks))
|
||||
@@ -406,6 +468,53 @@ def build_post_compact_messages(result: CompactionResult) -> list[ConversationMe
|
||||
]
|
||||
|
||||
|
||||
def _boundary_crosses_tool_pair(previous: ConversationMessage, current: ConversationMessage) -> bool:
|
||||
"""Return True when a preserve boundary would split a tool_use/result pair."""
|
||||
|
||||
if previous.role != "assistant" or current.role != "user":
|
||||
return False
|
||||
pending_tool_ids = {block.id for block in previous.content if isinstance(block, ToolUseBlock)}
|
||||
if not pending_tool_ids:
|
||||
return False
|
||||
result_ids = {block.tool_use_id for block in current.content if isinstance(block, ToolResultBlock)}
|
||||
return bool(pending_tool_ids & result_ids)
|
||||
|
||||
|
||||
def _split_preserving_tool_pairs(
|
||||
messages: list[ConversationMessage],
|
||||
*,
|
||||
preserve_recent: int,
|
||||
) -> tuple[list[ConversationMessage], list[ConversationMessage]]:
|
||||
"""Split older/newer segments without cutting through a tool_use/result pair.
|
||||
|
||||
The preserved segment is also sanitized so trailing orphan tool_use blocks
|
||||
never survive the compaction boundary.
|
||||
"""
|
||||
|
||||
if len(messages) <= preserve_recent:
|
||||
return [], sanitize_conversation_messages(list(messages))
|
||||
|
||||
split_index = max(0, len(messages) - preserve_recent)
|
||||
while split_index > 0 and _boundary_crosses_tool_pair(messages[split_index - 1], messages[split_index]):
|
||||
split_index -= 1
|
||||
|
||||
older = list(messages[:split_index])
|
||||
newer = sanitize_conversation_messages(list(messages[split_index:]))
|
||||
return older, newer
|
||||
|
||||
|
||||
def _sanitize_compaction_segments(result: CompactionResult) -> None:
|
||||
"""Normalize summary+preserved messages into a provider-safe sequence."""
|
||||
|
||||
if not result.summary_messages and not result.messages_to_keep:
|
||||
return
|
||||
combined = [*result.summary_messages, *result.messages_to_keep]
|
||||
sanitized = sanitize_conversation_messages(combined)
|
||||
summary_count = len(result.summary_messages)
|
||||
result.summary_messages = sanitized[:summary_count]
|
||||
result.messages_to_keep = sanitized[summary_count:]
|
||||
|
||||
|
||||
def _create_recent_attachments_attachment_if_needed(
|
||||
attachment_paths: list[str],
|
||||
) -> CompactAttachment | None:
|
||||
@@ -625,6 +734,7 @@ def _build_compact_attachments(
|
||||
|
||||
|
||||
def _finalize_compaction_result(result: CompactionResult) -> CompactionResult:
|
||||
_sanitize_compaction_segments(result)
|
||||
messages = build_post_compact_messages(result)
|
||||
result.compact_metadata.setdefault("post_compact_message_count", len(messages))
|
||||
result.compact_metadata.setdefault("post_compact_token_count", estimate_message_tokens(messages))
|
||||
@@ -674,14 +784,25 @@ def _build_passthrough_compaction_result(
|
||||
|
||||
def _collect_compactable_tool_ids(messages: list[ConversationMessage]) -> list[str]:
|
||||
"""Walk messages and collect tool_use IDs whose results are compactable."""
|
||||
ids: list[str] = []
|
||||
ordered_ids: list[str] = []
|
||||
tool_names: dict[str, str] = {}
|
||||
result_content: dict[str, str] = {}
|
||||
for msg in messages:
|
||||
if msg.role != "assistant":
|
||||
continue
|
||||
for block in msg.content:
|
||||
if isinstance(block, ToolUseBlock) and block.name in COMPACTABLE_TOOLS:
|
||||
ids.append(block.id)
|
||||
return ids
|
||||
if isinstance(block, ToolUseBlock):
|
||||
ordered_ids.append(block.id)
|
||||
tool_names[block.id] = block.name
|
||||
elif isinstance(block, ToolResultBlock):
|
||||
result_content[block.tool_use_id] = block.content
|
||||
return [
|
||||
tool_id
|
||||
for tool_id in ordered_ids
|
||||
if tool_names.get(tool_id, "") in COMPACTABLE_TOOLS
|
||||
or is_microcompactable_tool_result(
|
||||
tool_names.get(tool_id, ""),
|
||||
result_content.get(tool_id, ""),
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
def microcompact_messages(
|
||||
@@ -769,6 +890,28 @@ def _build_session_memory_message(messages: list[ConversationMessage]) -> Conver
|
||||
)
|
||||
|
||||
|
||||
def _build_file_session_memory_message(metadata: dict[str, Any] | None) -> ConversationMessage | None:
|
||||
"""Build a compaction message from the persisted session-memory file."""
|
||||
|
||||
if not metadata:
|
||||
return None
|
||||
path = metadata.get("session_memory_path")
|
||||
if not path:
|
||||
return None
|
||||
try:
|
||||
from openharness.services.session_memory import (
|
||||
get_session_memory_content,
|
||||
session_memory_to_compact_text,
|
||||
)
|
||||
|
||||
text = session_memory_to_compact_text(get_session_memory_content(str(path)))
|
||||
except Exception:
|
||||
return None
|
||||
if not text.strip():
|
||||
return None
|
||||
return ConversationMessage.from_user_text(text)
|
||||
|
||||
|
||||
def try_session_memory_compaction(
|
||||
messages: list[ConversationMessage],
|
||||
*,
|
||||
@@ -779,9 +922,9 @@ def try_session_memory_compaction(
|
||||
"""Cheap deterministic compaction for long chats before full LLM compaction."""
|
||||
if len(messages) <= preserve_recent + 4:
|
||||
return None
|
||||
older = messages[:-preserve_recent]
|
||||
newer = messages[-preserve_recent:]
|
||||
summary_message = _build_session_memory_message(older)
|
||||
older, newer = _split_preserving_tool_pairs(messages, preserve_recent=preserve_recent)
|
||||
file_summary_message = _build_file_session_memory_message(metadata)
|
||||
summary_message = file_summary_message or _build_session_memory_message(older)
|
||||
if summary_message is None:
|
||||
return None
|
||||
provisional = [summary_message, *newer]
|
||||
@@ -797,6 +940,7 @@ def try_session_memory_compaction(
|
||||
"pre_compact_token_count": estimate_message_tokens(messages),
|
||||
"preserve_recent": preserve_recent,
|
||||
"used_session_memory": True,
|
||||
"used_file_session_memory": file_summary_message is not None,
|
||||
"pre_compact_discovered_tools": _extract_discovered_tools(older),
|
||||
"attachments": _extract_attachment_paths(older),
|
||||
}
|
||||
@@ -1023,8 +1167,7 @@ async def compact_conversation(
|
||||
log.info("Compacting conversation: %d messages, ~%d tokens", len(messages), pre_compact_tokens)
|
||||
|
||||
# Step 2: split into older (summarize) and newer (preserve)
|
||||
older = messages[:-preserve_recent]
|
||||
newer = messages[-preserve_recent:]
|
||||
older, newer = _split_preserving_tool_pairs(messages, preserve_recent=preserve_recent)
|
||||
|
||||
# Step 3: build compact request — send older messages + compact prompt
|
||||
compact_prompt = get_compact_prompt(custom_instructions)
|
||||
@@ -1114,6 +1257,9 @@ async def compact_conversation(
|
||||
|
||||
async def _collect_summary(summary_request_messages: list[ConversationMessage]) -> str:
|
||||
collected = ""
|
||||
summary_request_messages = _replace_images_with_compaction_placeholders(
|
||||
summary_request_messages
|
||||
)
|
||||
stream = api_client.stream_message(
|
||||
ApiMessageRequest(
|
||||
model=model,
|
||||
@@ -1541,19 +1687,18 @@ def compact_messages(
|
||||
) -> list[ConversationMessage]:
|
||||
"""Replace older conversation history with a synthetic summary (legacy)."""
|
||||
if len(messages) <= preserve_recent:
|
||||
return list(messages)
|
||||
older = messages[:-preserve_recent]
|
||||
newer = messages[-preserve_recent:]
|
||||
return sanitize_conversation_messages(list(messages))
|
||||
older, newer = _split_preserving_tool_pairs(messages, preserve_recent=preserve_recent)
|
||||
summary = summarize_messages(older)
|
||||
if not summary:
|
||||
return list(newer)
|
||||
return [
|
||||
return sanitize_conversation_messages([
|
||||
ConversationMessage(
|
||||
role="user",
|
||||
content=[TextBlock(text=f"[conversation summary]\n{summary}")],
|
||||
),
|
||||
*newer,
|
||||
]
|
||||
])
|
||||
|
||||
|
||||
__all__ = [
|
||||
|
||||
@@ -4,6 +4,7 @@ from __future__ import annotations
|
||||
|
||||
import json
|
||||
from datetime import datetime, timezone
|
||||
from zoneinfo import ZoneInfo
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
@@ -44,9 +45,28 @@ def validate_cron_expression(expression: str) -> bool:
|
||||
return croniter.is_valid(expression)
|
||||
|
||||
|
||||
def next_run_time(expression: str, base: datetime | None = None) -> datetime:
|
||||
"""Return the next run time for a cron expression."""
|
||||
def validate_timezone(tz: str | None) -> bool:
|
||||
"""Return True if *tz* is a valid IANA timezone or empty."""
|
||||
if not tz:
|
||||
return True
|
||||
try:
|
||||
ZoneInfo(tz)
|
||||
except Exception:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def next_run_time(expression: str, base: datetime | None = None, tz: str | None = None) -> datetime:
|
||||
"""Return the next run time for a cron expression.
|
||||
|
||||
The returned datetime is always UTC. If *tz* is provided, the cron expression
|
||||
is interpreted in that IANA timezone.
|
||||
"""
|
||||
base = base or datetime.now(timezone.utc)
|
||||
if tz:
|
||||
local_base = base.astimezone(ZoneInfo(tz))
|
||||
local_next = croniter(expression, local_base).get_next(datetime)
|
||||
return local_next.astimezone(timezone.utc)
|
||||
return croniter(expression, base).get_next(datetime)
|
||||
|
||||
|
||||
@@ -61,7 +81,7 @@ def upsert_cron_job(job: dict[str, Any]) -> None:
|
||||
|
||||
schedule = job.get("schedule", "")
|
||||
if validate_cron_expression(schedule):
|
||||
job["next_run"] = next_run_time(schedule).isoformat()
|
||||
job["next_run"] = next_run_time(schedule, tz=job.get("timezone") or job.get("tz")).isoformat()
|
||||
|
||||
with exclusive_file_lock(_cron_lock_path()):
|
||||
jobs = [existing for existing in load_cron_jobs() if existing.get("name") != job.get("name")]
|
||||
@@ -112,6 +132,6 @@ def mark_job_run(name: str, *, success: bool) -> None:
|
||||
job["last_status"] = "success" if success else "failed"
|
||||
schedule = job.get("schedule", "")
|
||||
if validate_cron_expression(schedule):
|
||||
job["next_run"] = next_run_time(schedule, now).isoformat()
|
||||
job["next_run"] = next_run_time(schedule, now, tz=job.get("timezone") or job.get("tz")).isoformat()
|
||||
save_cron_jobs(jobs)
|
||||
return
|
||||
|
||||
@@ -9,17 +9,22 @@ log.
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import contextlib
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
import shlex
|
||||
import signal
|
||||
import sys
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from types import FrameType
|
||||
from typing import Any, Callable
|
||||
|
||||
from openharness.config.paths import get_data_dir, get_logs_dir
|
||||
from openharness.platforms import get_platform
|
||||
from openharness.services.cron import (
|
||||
load_cron_jobs,
|
||||
mark_job_run,
|
||||
@@ -28,6 +33,14 @@ from openharness.services.cron import (
|
||||
from openharness.sandbox import SandboxUnavailableError
|
||||
from openharness.utils.shell import create_shell_subprocess
|
||||
|
||||
try:
|
||||
from ohmo.gateway.config import load_gateway_config
|
||||
except Exception: # pragma: no cover - ohmo is optional for non-ohmo cron users
|
||||
load_gateway_config = None # type: ignore[assignment]
|
||||
|
||||
|
||||
NOTIFICATION_OUTPUT_LIMIT = 3500
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
TICK_INTERVAL_SECONDS = 30
|
||||
@@ -89,10 +102,7 @@ def read_pid() -> int | None:
|
||||
pid = int(path.read_text(encoding="utf-8").strip())
|
||||
except (ValueError, OSError):
|
||||
return None
|
||||
# Check if process is alive
|
||||
try:
|
||||
os.kill(pid, 0)
|
||||
except OSError:
|
||||
if not _pid_exists(pid):
|
||||
logger.debug("Removed stale scheduler PID file (pid=%d)", pid)
|
||||
path.unlink(missing_ok=True)
|
||||
return None
|
||||
@@ -122,37 +132,205 @@ def stop_scheduler() -> bool:
|
||||
if pid is None:
|
||||
return False
|
||||
try:
|
||||
os.kill(pid, signal.SIGTERM)
|
||||
_terminate_pid(pid)
|
||||
except OSError:
|
||||
remove_pid()
|
||||
return False
|
||||
# Wait briefly for process to exit
|
||||
for _ in range(10):
|
||||
try:
|
||||
os.kill(pid, 0)
|
||||
except OSError:
|
||||
if not _pid_exists(pid):
|
||||
remove_pid()
|
||||
return True
|
||||
time.sleep(0.2)
|
||||
# Force kill
|
||||
try:
|
||||
os.kill(pid, signal.SIGKILL)
|
||||
_kill_pid(pid)
|
||||
except OSError:
|
||||
pass
|
||||
remove_pid()
|
||||
return True
|
||||
|
||||
|
||||
def _pid_exists(pid: int) -> bool:
|
||||
"""Return True when *pid* currently refers to a live process."""
|
||||
if pid <= 0:
|
||||
return False
|
||||
if get_platform() == "windows":
|
||||
return _windows_pid_exists(pid)
|
||||
try:
|
||||
os.kill(pid, 0)
|
||||
except ProcessLookupError:
|
||||
return False
|
||||
except PermissionError:
|
||||
return True
|
||||
except OSError:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _windows_pid_exists(pid: int) -> bool:
|
||||
"""Windows implementation of ``kill(pid, 0)`` without requiring psutil."""
|
||||
try:
|
||||
import ctypes
|
||||
except Exception:
|
||||
return _pid_exists_with_kill_zero(pid)
|
||||
|
||||
try:
|
||||
kernel32 = ctypes.WinDLL("kernel32", use_last_error=True) # type: ignore[attr-defined]
|
||||
except AttributeError:
|
||||
return _pid_exists_with_kill_zero(pid)
|
||||
|
||||
synchronize = 0x00100000
|
||||
process_query_limited_information = 0x1000
|
||||
wait_timeout = 0x00000102
|
||||
|
||||
handle = kernel32.OpenProcess(
|
||||
synchronize | process_query_limited_information,
|
||||
False,
|
||||
pid,
|
||||
)
|
||||
if not handle:
|
||||
return ctypes.get_last_error() == 5 # ERROR_ACCESS_DENIED: process exists
|
||||
try:
|
||||
return kernel32.WaitForSingleObject(handle, 0) == wait_timeout
|
||||
finally:
|
||||
kernel32.CloseHandle(handle)
|
||||
|
||||
|
||||
def _pid_exists_with_kill_zero(pid: int) -> bool:
|
||||
try:
|
||||
os.kill(pid, 0)
|
||||
except ProcessLookupError:
|
||||
return False
|
||||
except PermissionError:
|
||||
return True
|
||||
except OSError:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _terminate_pid(pid: int) -> None:
|
||||
os.kill(pid, signal.SIGTERM)
|
||||
|
||||
|
||||
def _kill_pid(pid: int) -> None:
|
||||
if get_platform() == "windows":
|
||||
os.kill(pid, signal.SIGTERM)
|
||||
return
|
||||
os.kill(pid, signal.SIGKILL)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Job execution
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _format_notification(job: dict[str, Any], entry: dict[str, Any]) -> str:
|
||||
"""Build a concise notification body for a completed cron job."""
|
||||
status = entry.get("status", "?")
|
||||
rc = entry.get("returncode", "?")
|
||||
lines = [
|
||||
f"⏰ Cron job finished: {job.get('name', '?')}",
|
||||
f"Status: {status} (rc={rc})",
|
||||
f"Started: {entry.get('started_at', '?')}",
|
||||
f"Ended: {entry.get('ended_at', '?')}",
|
||||
]
|
||||
stdout = str(entry.get("stdout") or "").strip()
|
||||
stderr = str(entry.get("stderr") or "").strip()
|
||||
if stdout:
|
||||
lines.extend(["", "Output:", stdout[-NOTIFICATION_OUTPUT_LIMIT:]])
|
||||
if stderr:
|
||||
lines.extend(["", "Stderr:", stderr[-NOTIFICATION_OUTPUT_LIMIT:]])
|
||||
if not stdout and not stderr:
|
||||
lines.extend(["", "(no output)"])
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
async def _notify_job_result(job: dict[str, Any], entry: dict[str, Any]) -> None:
|
||||
"""Deliver an optional post-run notification for a cron job."""
|
||||
notify = job.get("notify")
|
||||
payload = job.get("payload")
|
||||
if not isinstance(notify, dict) and isinstance(payload, dict) and payload.get("deliver"):
|
||||
notify = {"type": payload.get("channel"), "to": payload.get("to")}
|
||||
if not isinstance(notify, dict):
|
||||
return
|
||||
notify_type = str(notify.get("type") or "").strip().lower()
|
||||
try:
|
||||
if notify_type in {"feishu_dm", "feishu"}:
|
||||
from ohmo.gateway.notify import send_feishu_dm
|
||||
|
||||
user_open_id = str(
|
||||
notify.get("user_open_id") or notify.get("open_id") or notify.get("to") or ""
|
||||
).strip()
|
||||
if not user_open_id:
|
||||
raise ValueError("missing notify.user_open_id")
|
||||
workspace = notify.get("workspace")
|
||||
await send_feishu_dm(
|
||||
user_open_id=user_open_id,
|
||||
content=_format_notification(job, entry),
|
||||
workspace=str(workspace) if workspace else None,
|
||||
)
|
||||
elif notify_type:
|
||||
raise ValueError(f"unsupported notify.type: {notify_type}")
|
||||
except Exception as exc:
|
||||
logger.error("Failed to notify cron job %r result: %s", job.get("name"), exc)
|
||||
entry["notification_status"] = "failed"
|
||||
entry["notification_error"] = str(exc)
|
||||
else:
|
||||
entry["notification_status"] = "sent"
|
||||
|
||||
|
||||
def _command_for_job(job: dict[str, Any]) -> str:
|
||||
"""Return the shell command used to execute a job."""
|
||||
command = job.get("command")
|
||||
if command:
|
||||
return str(command)
|
||||
payload = job.get("payload")
|
||||
if not isinstance(payload, dict) or payload.get("kind", "agent_turn") != "agent_turn":
|
||||
raise ValueError("cron job has no command or agent_turn payload")
|
||||
message = str(payload.get("message") or "").strip()
|
||||
if not message:
|
||||
raise ValueError("agent_turn cron job is missing payload.message")
|
||||
cwd = str(job.get("cwd") or ".")
|
||||
parts = ["ohmo"]
|
||||
profile = payload.get("profile") or job.get("provider_profile")
|
||||
if profile is None and load_gateway_config is not None:
|
||||
profile = load_gateway_config().provider_profile
|
||||
if profile:
|
||||
parts.extend(["--profile", str(profile)])
|
||||
parts.extend(
|
||||
[
|
||||
"--cwd",
|
||||
cwd,
|
||||
"--print",
|
||||
message,
|
||||
]
|
||||
)
|
||||
return " ".join(shlex.quote(part) for part in parts)
|
||||
|
||||
|
||||
async def execute_job(job: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Run a single cron job and return a history entry."""
|
||||
name = job["name"]
|
||||
command = job["command"]
|
||||
cwd = Path(job.get("cwd") or ".").expanduser()
|
||||
started_at = datetime.now(timezone.utc)
|
||||
try:
|
||||
command = _command_for_job(job)
|
||||
except Exception as exc:
|
||||
entry = {
|
||||
"name": name,
|
||||
"command": "",
|
||||
"started_at": started_at.isoformat(),
|
||||
"ended_at": datetime.now(timezone.utc).isoformat(),
|
||||
"returncode": -1,
|
||||
"status": "error",
|
||||
"stdout": "",
|
||||
"stderr": str(exc),
|
||||
}
|
||||
mark_job_run(name, success=False)
|
||||
await _notify_job_result(job, entry)
|
||||
append_history(entry)
|
||||
return entry
|
||||
|
||||
logger.info("Executing cron job %r: %s", name, command)
|
||||
try:
|
||||
@@ -183,6 +361,7 @@ async def execute_job(job: dict[str, Any]) -> dict[str, Any]:
|
||||
"stderr": "Job timed out after 300s",
|
||||
}
|
||||
mark_job_run(name, success=False)
|
||||
await _notify_job_result(job, entry)
|
||||
append_history(entry)
|
||||
return entry
|
||||
except SandboxUnavailableError as exc:
|
||||
@@ -197,6 +376,7 @@ async def execute_job(job: dict[str, Any]) -> dict[str, Any]:
|
||||
"stderr": str(exc),
|
||||
}
|
||||
mark_job_run(name, success=False)
|
||||
await _notify_job_result(job, entry)
|
||||
append_history(entry)
|
||||
return entry
|
||||
except Exception as exc:
|
||||
@@ -211,6 +391,7 @@ async def execute_job(job: dict[str, Any]) -> dict[str, Any]:
|
||||
"stderr": str(exc),
|
||||
}
|
||||
mark_job_run(name, success=False)
|
||||
await _notify_job_result(job, entry)
|
||||
append_history(entry)
|
||||
return entry
|
||||
|
||||
@@ -226,6 +407,7 @@ async def execute_job(job: dict[str, Any]) -> dict[str, Any]:
|
||||
"stderr": (stderr.decode("utf-8", errors="replace")[-2000:] if stderr else ""),
|
||||
}
|
||||
mark_job_run(name, success=success)
|
||||
await _notify_job_result(job, entry)
|
||||
append_history(entry)
|
||||
logger.info("Job %r finished: %s (rc=%s)", name, entry["status"], process.returncode)
|
||||
return entry
|
||||
@@ -262,13 +444,8 @@ async def run_scheduler_loop(*, once: bool = False) -> None:
|
||||
"""Main scheduler loop. Runs until SIGTERM or *once* is True (test mode)."""
|
||||
shutdown = asyncio.Event()
|
||||
|
||||
def _on_signal() -> None:
|
||||
logger.info("Received shutdown signal")
|
||||
shutdown.set()
|
||||
|
||||
loop = asyncio.get_running_loop()
|
||||
for sig in (signal.SIGTERM, signal.SIGINT):
|
||||
loop.add_signal_handler(sig, _on_signal)
|
||||
restore_signals = _install_shutdown_signal_handlers(loop, shutdown)
|
||||
|
||||
write_pid()
|
||||
logger.info("Cron scheduler started (pid=%d, tick=%ds)", os.getpid(), TICK_INTERVAL_SECONDS)
|
||||
@@ -297,10 +474,45 @@ async def run_scheduler_loop(*, once: bool = False) -> None:
|
||||
except asyncio.TimeoutError:
|
||||
pass
|
||||
finally:
|
||||
restore_signals()
|
||||
remove_pid()
|
||||
logger.info("Cron scheduler stopped")
|
||||
|
||||
|
||||
def _install_shutdown_signal_handlers(
|
||||
loop: asyncio.AbstractEventLoop,
|
||||
shutdown: asyncio.Event,
|
||||
) -> Callable[[], None]:
|
||||
"""Install portable signal handlers and return a restore callback."""
|
||||
previous_handlers: list[tuple[signal.Signals, Any]] = []
|
||||
|
||||
def _on_signal(signum: int, frame: FrameType | None) -> None:
|
||||
del frame
|
||||
logger.info("Received shutdown signal (%s)", signum)
|
||||
with contextlib.suppress(RuntimeError):
|
||||
loop.call_soon_threadsafe(shutdown.set)
|
||||
|
||||
signals: list[signal.Signals] = [signal.SIGTERM, signal.SIGINT]
|
||||
sigbreak = getattr(signal, "SIGBREAK", None)
|
||||
if sigbreak is not None:
|
||||
signals.append(sigbreak)
|
||||
|
||||
for sig in signals:
|
||||
try:
|
||||
previous = signal.getsignal(sig)
|
||||
signal.signal(sig, _on_signal)
|
||||
except (OSError, RuntimeError, ValueError):
|
||||
continue
|
||||
previous_handlers.append((sig, previous))
|
||||
|
||||
def _restore() -> None:
|
||||
for sig, previous in reversed(previous_handlers):
|
||||
with contextlib.suppress(OSError, RuntimeError, ValueError):
|
||||
signal.signal(sig, previous)
|
||||
|
||||
return _restore
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Daemon entry point (spawned by ``oh cron start``)
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -318,28 +530,49 @@ def _run_daemon() -> None:
|
||||
|
||||
|
||||
def start_daemon() -> int:
|
||||
"""Fork and start the scheduler daemon. Returns the child PID."""
|
||||
"""Start the scheduler daemon and return its PID."""
|
||||
existing = read_pid()
|
||||
if existing is not None:
|
||||
raise RuntimeError(f"Scheduler already running (pid={existing})")
|
||||
|
||||
pid = os.fork()
|
||||
if pid > 0:
|
||||
# Parent — wait a moment for the child to write its PID file
|
||||
time.sleep(0.3)
|
||||
return pid
|
||||
process = _spawn_scheduler_process()
|
||||
deadline = time.monotonic() + 2.0
|
||||
while time.monotonic() < deadline:
|
||||
pid = read_pid()
|
||||
if pid is not None:
|
||||
return pid
|
||||
if process.poll() is not None:
|
||||
break
|
||||
time.sleep(0.1)
|
||||
|
||||
# Child — detach
|
||||
os.setsid()
|
||||
# Redirect stdio
|
||||
devnull = os.open(os.devnull, os.O_RDWR)
|
||||
os.dup2(devnull, 0)
|
||||
os.dup2(devnull, 1)
|
||||
os.dup2(devnull, 2)
|
||||
os.close(devnull)
|
||||
if process.poll() is not None:
|
||||
log_file = get_logs_dir() / "cron_scheduler.log"
|
||||
raise RuntimeError(f"Cron scheduler failed to start; see {log_file}")
|
||||
return process.pid
|
||||
|
||||
_run_daemon()
|
||||
sys.exit(0)
|
||||
|
||||
def _spawn_scheduler_process() -> subprocess.Popen[bytes]:
|
||||
"""Spawn a detached scheduler subprocess on Unix and Windows."""
|
||||
popen_kwargs: dict[str, Any] = {
|
||||
"stdin": subprocess.DEVNULL,
|
||||
"stdout": subprocess.DEVNULL,
|
||||
"stderr": subprocess.DEVNULL,
|
||||
"close_fds": True,
|
||||
}
|
||||
if get_platform() == "windows":
|
||||
creationflags = getattr(subprocess, "DETACHED_PROCESS", 0) | getattr(
|
||||
subprocess,
|
||||
"CREATE_NEW_PROCESS_GROUP",
|
||||
0,
|
||||
)
|
||||
if creationflags:
|
||||
popen_kwargs["creationflags"] = creationflags
|
||||
else:
|
||||
popen_kwargs["start_new_session"] = True
|
||||
return subprocess.Popen(
|
||||
[sys.executable, "-m", "openharness.services.cron_scheduler"],
|
||||
**popen_kwargs,
|
||||
)
|
||||
|
||||
|
||||
def scheduler_status() -> dict[str, Any]:
|
||||
@@ -356,3 +589,7 @@ def scheduler_status() -> dict[str, Any]:
|
||||
"log_file": str(log_path),
|
||||
"history_file": str(get_history_path()),
|
||||
}
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
_run_daemon()
|
||||
|
||||
@@ -0,0 +1,261 @@
|
||||
"""Durable memory extraction from completed turns."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from openharness.api.client import ApiMessageCompleteEvent, ApiMessageRequest, SupportsStreamingMessages
|
||||
from openharness.engine.messages import ConversationMessage, ToolUseBlock
|
||||
from openharness.memory.manager import add_memory_entry
|
||||
from openharness.memory.paths import get_project_memory_dir
|
||||
from openharness.memory.relevance import build_memory_manifest
|
||||
from openharness.memory.scan import scan_memory_files
|
||||
from openharness.memory.schema import (
|
||||
DEFAULT_MEMORY_SCOPE,
|
||||
DEFAULT_MEMORY_TYPE,
|
||||
MemoryScope,
|
||||
MemoryType,
|
||||
parse_memory_scope,
|
||||
parse_memory_type,
|
||||
)
|
||||
from openharness.memory.team import check_team_memory_secrets, validate_team_memory_write_path
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
MEMORY_WRITE_TOOLS = {"write_file", "edit_file"}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ExtractionRecord:
|
||||
"""Structured memory record proposed by the extraction pass."""
|
||||
|
||||
title: str
|
||||
body: str
|
||||
memory_type: MemoryType = DEFAULT_MEMORY_TYPE
|
||||
scope: MemoryScope = DEFAULT_MEMORY_SCOPE
|
||||
description: str = ""
|
||||
tags: tuple[str, ...] = ()
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ExtractionResult:
|
||||
"""Outcome of a durable memory extraction run."""
|
||||
|
||||
skipped: bool
|
||||
reason: str = ""
|
||||
records: tuple[ExtractionRecord, ...] = ()
|
||||
written_paths: tuple[Path, ...] = ()
|
||||
|
||||
|
||||
def has_memory_writes_since(
|
||||
messages: list[ConversationMessage],
|
||||
memory_dir: str | Path,
|
||||
*,
|
||||
cwd: str | Path | None = None,
|
||||
) -> bool:
|
||||
"""Return whether the visible turn already wrote memory files."""
|
||||
|
||||
root = Path(memory_dir).expanduser().resolve()
|
||||
write_base = Path(cwd).expanduser().resolve() if cwd is not None else root
|
||||
for message in messages:
|
||||
for block in message.content:
|
||||
if not isinstance(block, ToolUseBlock):
|
||||
continue
|
||||
if block.name not in MEMORY_WRITE_TOOLS:
|
||||
continue
|
||||
raw_path = block.input.get("path") or block.input.get("file_path")
|
||||
if not raw_path:
|
||||
continue
|
||||
path = Path(str(raw_path)).expanduser()
|
||||
if not path.is_absolute():
|
||||
path = write_base / path
|
||||
try:
|
||||
path.resolve().relative_to(root)
|
||||
except ValueError:
|
||||
continue
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
async def extract_memories_from_turn(
|
||||
*,
|
||||
cwd: str | Path,
|
||||
api_client: SupportsStreamingMessages,
|
||||
model: str,
|
||||
messages: list[ConversationMessage],
|
||||
max_records: int = 3,
|
||||
) -> ExtractionResult:
|
||||
"""Ask the model for durable memory candidates and apply them."""
|
||||
|
||||
memory_dir = get_project_memory_dir(cwd)
|
||||
if len(messages) < 2:
|
||||
return ExtractionResult(skipped=True, reason="not enough messages")
|
||||
if has_memory_writes_since(messages, memory_dir, cwd=cwd):
|
||||
return ExtractionResult(skipped=True, reason="main conversation already wrote memory")
|
||||
|
||||
prompt = build_extraction_prompt(cwd, messages, max_records=max_records)
|
||||
final_text = ""
|
||||
async for event in api_client.stream_message(
|
||||
ApiMessageRequest(
|
||||
model=model,
|
||||
messages=[ConversationMessage.from_user_text(prompt)],
|
||||
system_prompt=EXTRACTION_SYSTEM_PROMPT,
|
||||
max_tokens=2048,
|
||||
tools=[],
|
||||
)
|
||||
):
|
||||
if isinstance(event, ApiMessageCompleteEvent):
|
||||
final_text = event.message.text
|
||||
break
|
||||
records = parse_extraction_records(final_text, max_records=max_records)
|
||||
if not records:
|
||||
return ExtractionResult(skipped=True, reason="no durable memories proposed")
|
||||
return apply_extraction_records(cwd, records)
|
||||
|
||||
|
||||
def build_extraction_prompt(cwd: str | Path, messages: list[ConversationMessage], *, max_records: int) -> str:
|
||||
"""Build the extraction request from recent messages and manifest."""
|
||||
|
||||
manifest = build_memory_manifest(scan_memory_files(cwd, max_files=80))
|
||||
transcript = "\n".join(_summarize_message(message) for message in messages[-12:])
|
||||
return (
|
||||
"Extract only durable memories from the recent conversation.\n"
|
||||
f"Return JSON with at most {max_records} records. Existing memory manifest:\n"
|
||||
f"{manifest or '(empty)'}\n\n"
|
||||
"Recent conversation:\n"
|
||||
f"{transcript}\n\n"
|
||||
"JSON schema: {\"memories\":[{\"title\":\"...\",\"type\":\"user|feedback|project|reference\","
|
||||
"\"scope\":\"private|project|team\",\"description\":\"...\",\"body\":\"...\",\"tags\":[\"...\"]}]}"
|
||||
)
|
||||
|
||||
|
||||
EXTRACTION_SYSTEM_PROMPT = """You maintain OpenHarness durable memory.
|
||||
Save only stable, future-useful facts that are not derivable from current files,
|
||||
git history, or documentation. Prefer updating existing memories conceptually
|
||||
over duplicating them. Do not save secrets. If nothing is worth saving, return
|
||||
{"memories": []}.
|
||||
"""
|
||||
|
||||
|
||||
def parse_extraction_records(text: str, *, max_records: int = 3) -> tuple[ExtractionRecord, ...]:
|
||||
"""Parse JSON memory extraction output."""
|
||||
|
||||
try:
|
||||
payload = json.loads(_extract_json_object(text))
|
||||
except json.JSONDecodeError:
|
||||
return ()
|
||||
raw_records = payload.get("memories") if isinstance(payload, dict) else None
|
||||
if not isinstance(raw_records, list):
|
||||
return ()
|
||||
records: list[ExtractionRecord] = []
|
||||
for item in raw_records[:max_records]:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
title = str(item.get("title") or "").strip()
|
||||
body = str(item.get("body") or "").strip()
|
||||
if not title or not body:
|
||||
continue
|
||||
memory_type = parse_memory_type(item.get("type"), default=DEFAULT_MEMORY_TYPE) or DEFAULT_MEMORY_TYPE
|
||||
scope = parse_memory_scope(item.get("scope"), default=DEFAULT_MEMORY_SCOPE) or DEFAULT_MEMORY_SCOPE
|
||||
tags_raw = item.get("tags") or ()
|
||||
tags = tuple(str(tag).strip() for tag in tags_raw if str(tag).strip()) if isinstance(tags_raw, list) else ()
|
||||
records.append(
|
||||
ExtractionRecord(
|
||||
title=title,
|
||||
body=body,
|
||||
memory_type=memory_type,
|
||||
scope=scope,
|
||||
description=str(item.get("description") or "").strip(),
|
||||
tags=tags,
|
||||
)
|
||||
)
|
||||
return tuple(records)
|
||||
|
||||
|
||||
def apply_extraction_records(cwd: str | Path, records: tuple[ExtractionRecord, ...]) -> ExtractionResult:
|
||||
"""Write accepted records to durable memory."""
|
||||
|
||||
written: list[Path] = []
|
||||
for record in records:
|
||||
if record.scope == "team":
|
||||
secret_error = check_team_memory_secrets(record.body)
|
||||
if secret_error:
|
||||
log.warning("memory extraction skipped team record %r: %s", record.title, secret_error)
|
||||
continue
|
||||
path, error = validate_team_memory_write_path(cwd, f"{record.title}.md")
|
||||
if error or path is None:
|
||||
log.warning("memory extraction skipped team record %r: %s", record.title, error)
|
||||
continue
|
||||
written.append(
|
||||
add_memory_entry(
|
||||
cwd,
|
||||
record.title,
|
||||
record.body,
|
||||
memory_type=record.memory_type,
|
||||
scope=record.scope,
|
||||
description=record.description,
|
||||
tags=record.tags,
|
||||
)
|
||||
)
|
||||
return ExtractionResult(skipped=not bool(written), reason="" if written else "all records rejected", records=records, written_paths=tuple(written))
|
||||
|
||||
|
||||
def validate_extraction_tool_request(tool_name: str, tool_input: dict[str, Any], memory_dir: str | Path) -> tuple[bool, str]:
|
||||
"""Permission guard for extraction-like agents."""
|
||||
|
||||
if tool_name in {"read_file", "grep", "glob"}:
|
||||
return True, ""
|
||||
if tool_name == "bash":
|
||||
command = str(tool_input.get("command") or "")
|
||||
if _is_read_only_shell(command):
|
||||
return True, ""
|
||||
return False, "memory extraction may only run read-only shell commands"
|
||||
if tool_name in {"write_file", "edit_file"}:
|
||||
raw_path = str(tool_input.get("path") or tool_input.get("file_path") or "")
|
||||
if not raw_path:
|
||||
return False, "memory extraction write requires a path"
|
||||
root = Path(memory_dir).expanduser().resolve()
|
||||
path = Path(raw_path).expanduser()
|
||||
if not path.is_absolute():
|
||||
path = root / path
|
||||
try:
|
||||
path.resolve().relative_to(root)
|
||||
except ValueError:
|
||||
return False, f"memory extraction writes must stay within {root}"
|
||||
return True, ""
|
||||
return False, f"memory extraction cannot use tool {tool_name}"
|
||||
|
||||
|
||||
def _extract_json_object(text: str) -> str:
|
||||
stripped = text.strip()
|
||||
if stripped.startswith("{") and stripped.endswith("}"):
|
||||
return stripped
|
||||
start = stripped.find("{")
|
||||
end = stripped.rfind("}")
|
||||
if start >= 0 and end > start:
|
||||
return stripped[start : end + 1]
|
||||
return stripped
|
||||
|
||||
|
||||
def _summarize_message(message: ConversationMessage) -> str:
|
||||
text = " ".join(message.text.split())
|
||||
if text:
|
||||
return f"{message.role}: {text[:1200]}"
|
||||
if message.tool_uses:
|
||||
return f"{message.role}: tool calls -> {', '.join(block.name for block in message.tool_uses)}"
|
||||
return f"{message.role}: [non-text content]"
|
||||
|
||||
|
||||
def _is_read_only_shell(command: str) -> bool:
|
||||
lowered = command.strip().lower()
|
||||
if not lowered:
|
||||
return False
|
||||
denied = (" > ", ">>", " rm ", " mv ", " cp ", " sed -i", " tee ", "python -c", "python3 -c")
|
||||
if any(marker in f" {lowered} " for marker in denied):
|
||||
return False
|
||||
first = lowered.split(maxsplit=1)[0]
|
||||
return first in {"ls", "pwd", "cat", "head", "tail", "rg", "grep", "find", "git", "wc", "sed", "awk", "stat"}
|
||||
@@ -0,0 +1,139 @@
|
||||
"""File-backed session memory for compact continuity."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from hashlib import sha1
|
||||
from pathlib import Path
|
||||
|
||||
from openharness.config.paths import get_data_dir
|
||||
from openharness.engine.messages import ConversationMessage, ToolResultBlock
|
||||
from openharness.services.token_estimation import estimate_tokens
|
||||
from openharness.utils.fs import atomic_write_text
|
||||
|
||||
MAX_SESSION_MEMORY_CHARS = 12_000
|
||||
MAX_RECENT_LINES = 80
|
||||
|
||||
|
||||
def get_session_memory_dir(cwd: str | Path) -> Path:
|
||||
"""Return the project session-memory directory."""
|
||||
|
||||
root = Path(cwd).resolve()
|
||||
digest = sha1(str(root).encode("utf-8")).hexdigest()[:12]
|
||||
path = get_data_dir() / "session-memory" / f"{root.name}-{digest}"
|
||||
path.mkdir(parents=True, exist_ok=True)
|
||||
return path
|
||||
|
||||
|
||||
def get_session_memory_path(cwd: str | Path, session_id: str | None = None) -> Path:
|
||||
"""Return the markdown session-memory path."""
|
||||
|
||||
safe_session = "".join(ch if ch.isalnum() or ch in "._-" else "_" for ch in (session_id or "default"))
|
||||
return get_session_memory_dir(cwd) / f"{safe_session or 'default'}.md"
|
||||
|
||||
|
||||
def prepare_session_memory_metadata(
|
||||
cwd: str | Path,
|
||||
tool_metadata: dict[str, object],
|
||||
*,
|
||||
session_id: str | None = None,
|
||||
) -> Path:
|
||||
"""Ensure metadata points compaction to the session-memory file."""
|
||||
|
||||
sid = session_id or str(tool_metadata.get("session_id") or "default")
|
||||
path = get_session_memory_path(cwd, sid)
|
||||
tool_metadata["session_memory_path"] = str(path)
|
||||
return path
|
||||
|
||||
|
||||
def get_session_memory_content(path: str | Path | None) -> str:
|
||||
"""Read session memory content if available."""
|
||||
|
||||
if not path:
|
||||
return ""
|
||||
candidate = Path(path).expanduser()
|
||||
if not candidate.exists():
|
||||
return ""
|
||||
try:
|
||||
return candidate.read_text(encoding="utf-8", errors="replace")
|
||||
except OSError:
|
||||
return ""
|
||||
|
||||
|
||||
def update_session_memory_file(
|
||||
cwd: str | Path,
|
||||
messages: list[ConversationMessage],
|
||||
*,
|
||||
tool_metadata: dict[str, object] | None = None,
|
||||
session_id: str | None = None,
|
||||
) -> Path:
|
||||
"""Update the deterministic session-memory checkpoint."""
|
||||
|
||||
path = prepare_session_memory_metadata(cwd, tool_metadata or {}, session_id=session_id)
|
||||
body = build_session_memory_document(messages, tool_metadata=tool_metadata)
|
||||
atomic_write_text(path, body)
|
||||
return path
|
||||
|
||||
|
||||
def build_session_memory_document(
|
||||
messages: list[ConversationMessage],
|
||||
*,
|
||||
tool_metadata: dict[str, object] | None = None,
|
||||
) -> str:
|
||||
"""Build a compact markdown checkpoint for the current session."""
|
||||
|
||||
state = tool_metadata.get("task_focus_state") if isinstance(tool_metadata, dict) else None
|
||||
goal = ""
|
||||
next_step = ""
|
||||
verified: list[str] = []
|
||||
artifacts: list[str] = []
|
||||
if isinstance(state, dict):
|
||||
goal = str(state.get("goal") or "").strip()
|
||||
next_step = str(state.get("next_step") or "").strip()
|
||||
verified = [str(item).strip() for item in state.get("verified_state", []) if str(item).strip()]
|
||||
artifacts = [str(item).strip() for item in state.get("active_artifacts", []) if str(item).strip()]
|
||||
|
||||
lines = ["# Session Memory", ""]
|
||||
lines.extend(["## Current State", goal or "(no current goal recorded)", ""])
|
||||
if next_step:
|
||||
lines.extend(["## Next Step", next_step, ""])
|
||||
if verified:
|
||||
lines.extend(["## Verified Work", *[f"- {item}" for item in verified[-10:]], ""])
|
||||
if artifacts:
|
||||
lines.extend(["## Active Artifacts", *[f"- {item}" for item in artifacts[-10:]], ""])
|
||||
lines.extend(["## Recent Conversation", *_recent_message_lines(messages), ""])
|
||||
text = "\n".join(lines).strip() + "\n"
|
||||
if len(text) > MAX_SESSION_MEMORY_CHARS:
|
||||
text = text[:MAX_SESSION_MEMORY_CHARS].rsplit("\n", 1)[0]
|
||||
text += "\n\n> Session memory was truncated to stay within budget.\n"
|
||||
return text
|
||||
|
||||
|
||||
def session_memory_to_compact_text(content: str) -> str:
|
||||
"""Prepare persisted session memory for insertion across compact boundaries."""
|
||||
|
||||
stripped = content.strip()
|
||||
if not stripped:
|
||||
return ""
|
||||
if estimate_tokens(stripped) > 4_000:
|
||||
stripped = stripped[:MAX_SESSION_MEMORY_CHARS].rsplit("\n", 1)[0]
|
||||
return "Session memory checkpoint from earlier in this conversation:\n" + stripped
|
||||
|
||||
|
||||
def _recent_message_lines(messages: list[ConversationMessage]) -> list[str]:
|
||||
lines: list[str] = []
|
||||
for message in messages[-MAX_RECENT_LINES:]:
|
||||
line = _summarize_message(message)
|
||||
if line:
|
||||
lines.append(f"- {line}")
|
||||
return lines or ["- (no recent messages)"]
|
||||
|
||||
|
||||
def _summarize_message(message: ConversationMessage) -> str:
|
||||
text = " ".join(message.text.split())
|
||||
if text:
|
||||
return f"{message.role}: {text[:220]}"
|
||||
if message.tool_uses:
|
||||
return f"{message.role}: tool calls -> {', '.join(block.name for block in message.tool_uses[:6])}"
|
||||
if any(isinstance(block, ToolResultBlock) for block in message.content):
|
||||
return f"{message.role}: tool results returned"
|
||||
return f"{message.role}: [non-text content]"
|
||||
@@ -20,6 +20,7 @@ _PERSISTED_TOOL_METADATA_KEYS = (
|
||||
"read_file_state",
|
||||
"invoked_skills",
|
||||
"async_agent_state",
|
||||
"async_agent_tasks",
|
||||
"recent_work_log",
|
||||
"recent_verified_work",
|
||||
"task_focus_state",
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
"""Tool-output context budget helpers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
DEFAULT_TOOL_OUTPUT_INLINE_CHARS = 16_000
|
||||
DEFAULT_TOOL_OUTPUT_PREVIEW_CHARS = 3_000
|
||||
DEFAULT_MICROCOMPACT_TOOL_RESULT_CHARS = 4_000
|
||||
|
||||
|
||||
def _read_positive_int_env(name: str, default: int, *, minimum: int = 1) -> int:
|
||||
raw = os.environ.get(name, "").strip()
|
||||
if not raw:
|
||||
return default
|
||||
try:
|
||||
return max(minimum, int(raw))
|
||||
except ValueError:
|
||||
log.warning("Ignoring invalid %s=%r", name, raw)
|
||||
return default
|
||||
|
||||
|
||||
def tool_output_inline_chars() -> int:
|
||||
return _read_positive_int_env(
|
||||
"OPENHARNESS_TOOL_OUTPUT_INLINE_CHARS",
|
||||
DEFAULT_TOOL_OUTPUT_INLINE_CHARS,
|
||||
minimum=256,
|
||||
)
|
||||
|
||||
|
||||
def tool_output_preview_chars() -> int:
|
||||
return _read_positive_int_env(
|
||||
"OPENHARNESS_TOOL_OUTPUT_PREVIEW_CHARS",
|
||||
DEFAULT_TOOL_OUTPUT_PREVIEW_CHARS,
|
||||
minimum=128,
|
||||
)
|
||||
|
||||
|
||||
def microcompact_tool_result_chars() -> int:
|
||||
return _read_positive_int_env(
|
||||
"OPENHARNESS_MICROCOMPACT_TOOL_RESULT_CHARS",
|
||||
DEFAULT_MICROCOMPACT_TOOL_RESULT_CHARS,
|
||||
minimum=256,
|
||||
)
|
||||
|
||||
|
||||
def is_microcompactable_tool_result(tool_name: str, content: str) -> bool:
|
||||
"""Return True when a tool result should be eligible for old-result clearing."""
|
||||
normalized = tool_name.strip()
|
||||
if normalized.startswith("mcp__"):
|
||||
return True
|
||||
return len(content) >= microcompact_tool_result_chars()
|
||||
@@ -8,14 +8,28 @@ if TYPE_CHECKING: # pragma: no cover
|
||||
from openharness.skills.registry import SkillRegistry
|
||||
from openharness.skills.types import SkillDefinition
|
||||
|
||||
__all__ = ["SkillDefinition", "SkillRegistry", "get_user_skills_dir", "load_skill_registry"]
|
||||
__all__ = [
|
||||
"SkillDefinition",
|
||||
"SkillRegistry",
|
||||
"discover_project_skill_dirs",
|
||||
"get_user_skill_dirs",
|
||||
"get_user_skills_dir",
|
||||
"load_skill_registry",
|
||||
]
|
||||
|
||||
|
||||
def __getattr__(name: str):
|
||||
if name in {"get_user_skills_dir", "load_skill_registry"}:
|
||||
from openharness.skills.loader import get_user_skills_dir, load_skill_registry
|
||||
if name in {"discover_project_skill_dirs", "get_user_skill_dirs", "get_user_skills_dir", "load_skill_registry"}:
|
||||
from openharness.skills.loader import (
|
||||
discover_project_skill_dirs,
|
||||
get_user_skill_dirs,
|
||||
get_user_skills_dir,
|
||||
load_skill_registry,
|
||||
)
|
||||
|
||||
return {
|
||||
"discover_project_skill_dirs": discover_project_skill_dirs,
|
||||
"get_user_skill_dirs": get_user_skill_dirs,
|
||||
"get_user_skills_dir": get_user_skills_dir,
|
||||
"load_skill_registry": load_skill_registry,
|
||||
}[name]
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
"""Shared YAML frontmatter parsing for SKILL.md files."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def parse_bool_frontmatter(value: Any, *, default: bool) -> bool:
|
||||
"""Parse permissive YAML frontmatter booleans."""
|
||||
if value is None:
|
||||
return default
|
||||
if isinstance(value, bool):
|
||||
return value
|
||||
normalized = str(value).strip().lower()
|
||||
if normalized in {"1", "true", "yes", "on"}:
|
||||
return True
|
||||
if normalized in {"0", "false", "no", "off"}:
|
||||
return False
|
||||
return default
|
||||
|
||||
|
||||
def optional_frontmatter_str(value: Any) -> str | None:
|
||||
"""Return a stripped string value, or ``None`` when absent/blank."""
|
||||
if isinstance(value, str) and value.strip():
|
||||
return value.strip()
|
||||
return None
|
||||
|
||||
|
||||
def parse_skill_metadata(
|
||||
default_name: str,
|
||||
content: str,
|
||||
*,
|
||||
fallback_template: str = "Skill: {name}",
|
||||
) -> dict[str, Any]:
|
||||
"""Extract metadata from a SKILL.md file.
|
||||
|
||||
Parses YAML frontmatter (``---`` delimited) via ``yaml.safe_load`` so that
|
||||
folded block scalars (``>``), literal block scalars (``|``), quoted values,
|
||||
and other standard YAML constructs are handled correctly. Falls back to
|
||||
``# heading`` + first body paragraph when no usable frontmatter is present,
|
||||
and finally to ``fallback_template`` when no description can be derived.
|
||||
"""
|
||||
name = default_name
|
||||
description = ""
|
||||
frontmatter: dict[str, Any] = {}
|
||||
lines = content.splitlines()
|
||||
|
||||
if content.startswith("---\n"):
|
||||
end_index = content.find("\n---\n", 4)
|
||||
if end_index != -1:
|
||||
try:
|
||||
metadata = yaml.safe_load(content[4:end_index])
|
||||
if isinstance(metadata, dict):
|
||||
frontmatter = metadata
|
||||
val = metadata.get("name")
|
||||
if isinstance(val, str) and val.strip():
|
||||
name = val.strip()
|
||||
val = metadata.get("description")
|
||||
if isinstance(val, str) and val.strip():
|
||||
description = val.strip()
|
||||
except yaml.YAMLError:
|
||||
logger.debug("Failed to parse YAML frontmatter for skill %s", default_name)
|
||||
|
||||
if not description:
|
||||
for line in lines:
|
||||
stripped = line.strip()
|
||||
if stripped.startswith("# "):
|
||||
if not name or name == default_name:
|
||||
name = stripped[2:].strip() or default_name
|
||||
continue
|
||||
if stripped and not stripped.startswith("---") and not stripped.startswith("#"):
|
||||
description = stripped[:200]
|
||||
break
|
||||
|
||||
if not description:
|
||||
description = fallback_template.format(name=name)
|
||||
return {"name": name, "description": description, "frontmatter": frontmatter}
|
||||
|
||||
|
||||
def parse_skill_frontmatter(
|
||||
default_name: str,
|
||||
content: str,
|
||||
*,
|
||||
fallback_template: str = "Skill: {name}",
|
||||
) -> tuple[str, str]:
|
||||
"""Extract ``name`` and ``description`` from a SKILL.md file."""
|
||||
metadata = parse_skill_metadata(
|
||||
default_name,
|
||||
content,
|
||||
fallback_template=fallback_template,
|
||||
)
|
||||
return str(metadata["name"]), str(metadata["description"])
|
||||
@@ -4,6 +4,12 @@ from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from openharness.skills._frontmatter import (
|
||||
optional_frontmatter_str,
|
||||
parse_bool_frontmatter,
|
||||
parse_skill_frontmatter,
|
||||
parse_skill_metadata,
|
||||
)
|
||||
from openharness.skills.types import SkillDefinition
|
||||
|
||||
_CONTENT_DIR = Path(__file__).parent / "content"
|
||||
@@ -16,53 +22,54 @@ def get_bundled_skills() -> list[SkillDefinition]:
|
||||
return skills
|
||||
for path in sorted(_CONTENT_DIR.glob("*.md")):
|
||||
content = path.read_text(encoding="utf-8")
|
||||
name, description = _parse_frontmatter(path.stem, content)
|
||||
metadata = _parse_metadata(path.stem, content)
|
||||
display_name = metadata["name"] if metadata["name"] != path.stem else None
|
||||
skills.append(
|
||||
SkillDefinition(
|
||||
name=name,
|
||||
description=description,
|
||||
name=metadata["name"],
|
||||
description=metadata["description"],
|
||||
content=content,
|
||||
source="bundled",
|
||||
path=str(path),
|
||||
base_dir=str(path.parent),
|
||||
command_name=path.stem,
|
||||
display_name=display_name,
|
||||
user_invocable=metadata["user_invocable"],
|
||||
disable_model_invocation=metadata["disable_model_invocation"],
|
||||
model=metadata["model"],
|
||||
argument_hint=metadata["argument_hint"],
|
||||
)
|
||||
)
|
||||
return skills
|
||||
|
||||
|
||||
def _parse_frontmatter(default_name: str, content: str) -> tuple[str, str]:
|
||||
"""Extract name and description from a skill markdown file.
|
||||
"""Extract name and description from a bundled skill markdown file.
|
||||
|
||||
Supports YAML frontmatter (``---`` delimited) and falls back to heading/paragraph parsing.
|
||||
Delegates to the shared parser so YAML block scalars (``>``, ``|``),
|
||||
quoted values, and other standard YAML constructs are handled the same
|
||||
way as user-installed skills.
|
||||
"""
|
||||
name = default_name
|
||||
description = ""
|
||||
lines = content.splitlines()
|
||||
return parse_skill_frontmatter(
|
||||
default_name,
|
||||
content,
|
||||
fallback_template="Bundled skill: {name}",
|
||||
)
|
||||
|
||||
# Try YAML frontmatter first
|
||||
if lines and lines[0].strip() == "---":
|
||||
for i, line in enumerate(lines[1:], 1):
|
||||
if line.strip() == "---":
|
||||
for fm_line in lines[1:i]:
|
||||
fm = fm_line.strip()
|
||||
if fm.startswith("name:"):
|
||||
val = fm[5:].strip().strip("'\"")
|
||||
if val:
|
||||
name = val
|
||||
elif fm.startswith("description:"):
|
||||
val = fm[12:].strip().strip("'\"")
|
||||
if val:
|
||||
description = val
|
||||
break
|
||||
if description:
|
||||
return name, description
|
||||
|
||||
# Fallback: heading + first paragraph
|
||||
for line in lines:
|
||||
stripped = line.strip()
|
||||
if stripped.startswith("# "):
|
||||
name = stripped[2:].strip() or default_name
|
||||
continue
|
||||
if stripped and not stripped.startswith("---") and not stripped.startswith("#"):
|
||||
description = stripped[:200]
|
||||
break
|
||||
return name, description or f"Bundled skill: {name}"
|
||||
def _parse_metadata(default_name: str, content: str) -> dict:
|
||||
parsed = parse_skill_metadata(default_name, content, fallback_template="Bundled skill: {name}")
|
||||
frontmatter = parsed.get("frontmatter")
|
||||
if not isinstance(frontmatter, dict):
|
||||
frontmatter = {}
|
||||
return {
|
||||
"name": str(parsed["name"]),
|
||||
"description": str(parsed["description"]),
|
||||
"user_invocable": parse_bool_frontmatter(frontmatter.get("user-invocable"), default=True),
|
||||
"disable_model_invocation": parse_bool_frontmatter(
|
||||
frontmatter.get("disable-model-invocation"),
|
||||
default=False,
|
||||
),
|
||||
"model": optional_frontmatter_str(frontmatter.get("model")),
|
||||
"argument_hint": optional_frontmatter_str(frontmatter.get("argument-hint")),
|
||||
}
|
||||
|
||||
@@ -0,0 +1,175 @@
|
||||
---
|
||||
name: skill-creator
|
||||
description: >
|
||||
Create, improve, and verify OpenHarness skills. Use this whenever the user
|
||||
asks to create a new skill, convert a workflow into a skill, update an
|
||||
existing SKILL.md, add skills for oh/ohmo, design skill trigger behavior,
|
||||
or test whether a skill loads and works correctly.
|
||||
---
|
||||
|
||||
# skill-creator
|
||||
|
||||
Create or improve OpenHarness skills in the directory-based `SKILL.md` format.
|
||||
|
||||
## When to use
|
||||
|
||||
Use this skill when the user wants to:
|
||||
|
||||
- create a new skill from a workflow, repeated task, domain guide, or tool process
|
||||
- convert existing notes, prompts, or instructions into a reusable skill
|
||||
- modify or harden an existing skill
|
||||
- make a skill available to `oh`, `ohmo`, or a plugin
|
||||
- debug why the `skill` tool cannot find or load a skill
|
||||
- test trigger wording, skill metadata, or skill behavior
|
||||
|
||||
## OpenHarness skill locations
|
||||
|
||||
Choose the target deliberately:
|
||||
|
||||
- Built-in OpenHarness skills live in `src/openharness/skills/bundled/content/*.md`.
|
||||
- User skills for `oh` live in `~/.openharness/skills/<skill-dir>/SKILL.md`.
|
||||
- Private `ohmo` skills live in `~/.ohmo/skills/<skill-dir>/SKILL.md`.
|
||||
- Plugin skills live in `<plugin-root>/skills/<skill-dir>/SKILL.md`.
|
||||
|
||||
OpenHarness user and plugin skills use a directory layout. Do not create flat
|
||||
`*.md` user skills under `skills/`; they will not be loaded by the normal loader.
|
||||
|
||||
## Skill anatomy
|
||||
|
||||
Use this structure for user, ohmo, and plugin skills:
|
||||
|
||||
```text
|
||||
my-skill/
|
||||
├── SKILL.md
|
||||
├── scripts/ # optional deterministic helpers
|
||||
├── references/ # optional long docs loaded only when relevant
|
||||
└── assets/ # optional templates or static files
|
||||
```
|
||||
|
||||
Use this structure inside `SKILL.md`:
|
||||
|
||||
```markdown
|
||||
---
|
||||
name: my-skill
|
||||
description: >
|
||||
What the skill does and exactly when to use it.
|
||||
---
|
||||
|
||||
# my-skill
|
||||
|
||||
Short purpose statement.
|
||||
|
||||
## When to use
|
||||
|
||||
Trigger contexts in plain language.
|
||||
|
||||
## Workflow
|
||||
|
||||
Concrete steps the agent should follow.
|
||||
|
||||
## Rules
|
||||
|
||||
Important boundaries, verification, and failure handling.
|
||||
```
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Capture intent before writing.
|
||||
Ask or infer what the skill should help with, when it should trigger, what
|
||||
output it should produce, and what success looks like. If the conversation
|
||||
already contains the workflow, extract the sequence from history before
|
||||
asking more questions.
|
||||
|
||||
2. Inspect existing examples.
|
||||
Read nearby skills and loader behavior before choosing a layout. For
|
||||
OpenHarness, confirm whether the target is bundled, user, ohmo-private, or
|
||||
plugin-provided.
|
||||
|
||||
3. Write the metadata first.
|
||||
The `description` is the primary trigger hint shown in the available skills
|
||||
list. Make it specific and slightly assertive: include both what the skill
|
||||
does and the phrases or situations that should trigger it. Avoid vague
|
||||
descriptions like "Helpful notes for X".
|
||||
|
||||
4. Keep the body procedural.
|
||||
Use concise imperative steps. Explain why non-obvious constraints matter.
|
||||
Prefer general principles over overfitted examples. If a skill is getting
|
||||
long, move details into `references/` and point to the relevant file.
|
||||
|
||||
5. Add deterministic resources only when they pay for themselves.
|
||||
If the skill repeatedly needs the same script, template, schema, or checklist,
|
||||
place it under `scripts/`, `assets/`, or `references/` rather than making the
|
||||
model regenerate it every time.
|
||||
|
||||
6. Verify loading.
|
||||
Run a local loader check for the chosen target. For bundled skills, use:
|
||||
|
||||
```bash
|
||||
PYTHONPATH=src python - <<'PY'
|
||||
from openharness.skills import load_skill_registry
|
||||
for skill in load_skill_registry().list_skills():
|
||||
print(skill.name, skill.source, skill.path)
|
||||
PY
|
||||
```
|
||||
|
||||
For ohmo-private skills, include the extra skill directory:
|
||||
|
||||
```bash
|
||||
PYTHONPATH=src python - <<'PY'
|
||||
from openharness.skills import load_skill_registry
|
||||
from ohmo.workspace import get_skills_dir
|
||||
for skill in load_skill_registry(extra_skill_dirs=[get_skills_dir()]).list_skills():
|
||||
print(skill.name, skill.source, skill.path)
|
||||
PY
|
||||
```
|
||||
|
||||
7. Test behavior.
|
||||
Create two or three realistic prompts that should use the skill. If the skill
|
||||
changes code or files, add regression tests around loading or command
|
||||
behavior. Run targeted tests first, then broader tests if the loader or
|
||||
runtime path changed.
|
||||
|
||||
8. Iterate from evidence.
|
||||
If a skill under-triggers, improve the frontmatter description. If it
|
||||
triggers but produces poor work, improve the body workflow. If agents repeat
|
||||
deterministic work, add a helper script or reference file.
|
||||
|
||||
## Writing rules
|
||||
|
||||
- Preserve the skill name when updating an existing skill unless the user
|
||||
explicitly asks to rename it.
|
||||
- Do not overwrite user-created skills without reading the existing `SKILL.md`.
|
||||
- Keep `SKILL.md` under roughly 500 lines; use `references/` for long material.
|
||||
- Keep trigger guidance in frontmatter `description`, not buried only in the
|
||||
body.
|
||||
- Make skills safe and unsurprising. Do not create skills that hide behavior,
|
||||
bypass permissions, exfiltrate data, or encourage unauthorized access.
|
||||
- Prefer directory names that are lowercase kebab-case, even if the displayed
|
||||
skill `name` is more human-readable.
|
||||
- Mention exact install or verification paths in the final response.
|
||||
|
||||
## Test prompts
|
||||
|
||||
After drafting a skill, propose a small eval set:
|
||||
|
||||
```json
|
||||
{
|
||||
"skill_name": "my-skill",
|
||||
"evals": [
|
||||
{
|
||||
"id": "happy-path",
|
||||
"prompt": "A realistic user request that should trigger the skill.",
|
||||
"expected_output": "What good behavior looks like."
|
||||
},
|
||||
{
|
||||
"id": "near-miss",
|
||||
"prompt": "A similar request that should not trigger this skill.",
|
||||
"expected_output": "The agent should choose a different path."
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
For objective skills, add assertions or tests. For subjective skills, collect
|
||||
human feedback and revise the workflow from concrete complaints rather than
|
||||
adding rigid rules.
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Skill loading from bundled and user directories."""
|
||||
"""Skill loading from bundled, user, compatibility, and project directories."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -6,24 +6,39 @@ import logging
|
||||
from pathlib import Path
|
||||
from typing import Iterable
|
||||
|
||||
import yaml
|
||||
|
||||
from openharness.config.paths import get_config_dir
|
||||
from openharness.config.settings import load_settings
|
||||
from openharness.skills._frontmatter import (
|
||||
optional_frontmatter_str,
|
||||
parse_bool_frontmatter,
|
||||
parse_skill_frontmatter,
|
||||
parse_skill_metadata,
|
||||
)
|
||||
from openharness.skills.bundled import get_bundled_skills
|
||||
from openharness.skills.registry import SkillRegistry
|
||||
from openharness.skills.types import SkillDefinition
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_USER_COMPAT_SKILL_DIRS = (
|
||||
(".claude", "skills"),
|
||||
(".agents", "skills"),
|
||||
)
|
||||
_DEFAULT_PROJECT_SKILL_DIRS = (".openharness/skills", ".agents/skills", ".claude/skills")
|
||||
|
||||
|
||||
def get_user_skills_dir() -> Path:
|
||||
"""Return the user skills directory."""
|
||||
"""Return the OpenHarness user skills directory."""
|
||||
path = get_config_dir() / "skills"
|
||||
path.mkdir(parents=True, exist_ok=True)
|
||||
return path
|
||||
|
||||
|
||||
def get_user_skill_dirs() -> list[Path]:
|
||||
"""Return user-level skill directories loaded by default."""
|
||||
return [get_user_skills_dir(), *(Path.home().joinpath(*parts) for parts in _USER_COMPAT_SKILL_DIRS)]
|
||||
|
||||
|
||||
def load_skill_registry(
|
||||
cwd: str | Path | None = None,
|
||||
*,
|
||||
@@ -31,18 +46,27 @@ def load_skill_registry(
|
||||
extra_plugin_roots: Iterable[str | Path] | None = None,
|
||||
settings=None,
|
||||
) -> SkillRegistry:
|
||||
"""Load bundled and user-defined skills."""
|
||||
"""Load bundled, user-defined, project, and plugin skills."""
|
||||
registry = SkillRegistry()
|
||||
for skill in get_bundled_skills():
|
||||
registry.register(skill)
|
||||
for skill in load_user_skills():
|
||||
registry.register(skill)
|
||||
for skill in load_skills_from_dirs(extra_skill_dirs):
|
||||
for skill in load_skills_from_dirs(extra_skill_dirs, source="user"):
|
||||
registry.register(skill)
|
||||
|
||||
resolved_settings = settings or load_settings()
|
||||
if cwd is not None and getattr(resolved_settings, "allow_project_skills", True):
|
||||
project_dirs = discover_project_skill_dirs(
|
||||
cwd,
|
||||
getattr(resolved_settings, "project_skill_dirs", list(_DEFAULT_PROJECT_SKILL_DIRS)),
|
||||
)
|
||||
for skill in load_skills_from_dirs(project_dirs, source="project", create_missing=False):
|
||||
registry.register(skill)
|
||||
|
||||
if cwd is not None:
|
||||
from openharness.plugins.loader import load_plugins
|
||||
|
||||
resolved_settings = settings or load_settings()
|
||||
for plugin in load_plugins(resolved_settings, cwd, extra_roots=extra_plugin_roots):
|
||||
if not plugin.enabled:
|
||||
continue
|
||||
@@ -52,14 +76,85 @@ def load_skill_registry(
|
||||
|
||||
|
||||
def load_user_skills() -> list[SkillDefinition]:
|
||||
"""Load markdown skills from the user config directory."""
|
||||
return load_skills_from_dirs([get_user_skills_dir()], source="user")
|
||||
"""Load markdown skills from user-level OpenHarness and compatibility directories."""
|
||||
return load_skills_from_dirs(get_user_skill_dirs(), source="user")
|
||||
|
||||
|
||||
def discover_project_skill_dirs(
|
||||
cwd: str | Path,
|
||||
project_skill_dirs: Iterable[str] | None = None,
|
||||
) -> list[Path]:
|
||||
"""Return existing project skill directories from cwd up to the git root.
|
||||
|
||||
Directories are ordered from least-specific to most-specific so later registry
|
||||
entries can override broader project or user skills deterministically.
|
||||
"""
|
||||
start = Path(cwd).expanduser().resolve()
|
||||
if not start.exists():
|
||||
start = start.parent
|
||||
if start.is_file():
|
||||
start = start.parent
|
||||
|
||||
relative_dirs = _valid_project_skill_dirs(project_skill_dirs or _DEFAULT_PROJECT_SKILL_DIRS)
|
||||
git_root = _find_git_root(start)
|
||||
home = Path.home().resolve()
|
||||
current = start
|
||||
levels: list[Path] = []
|
||||
while True:
|
||||
levels.append(current)
|
||||
if git_root is not None and current == git_root:
|
||||
break
|
||||
if git_root is None and current == home:
|
||||
break
|
||||
parent = current.parent
|
||||
if parent == current:
|
||||
break
|
||||
current = parent
|
||||
|
||||
roots: list[Path] = []
|
||||
seen: set[Path] = set()
|
||||
for base in reversed(levels):
|
||||
for rel in relative_dirs:
|
||||
candidate = (base / rel).resolve()
|
||||
if candidate in seen or not candidate.is_dir():
|
||||
continue
|
||||
seen.add(candidate)
|
||||
roots.append(candidate)
|
||||
return roots
|
||||
|
||||
|
||||
def _valid_project_skill_dirs(project_skill_dirs: Iterable[str]) -> list[Path]:
|
||||
"""Return safe relative project skill paths."""
|
||||
paths: list[Path] = []
|
||||
for raw in project_skill_dirs:
|
||||
value = str(raw).strip()
|
||||
if not value:
|
||||
continue
|
||||
rel = Path(value)
|
||||
if rel.is_absolute() or ".." in rel.parts:
|
||||
logger.warning("Ignoring unsafe project skill dir: %s", raw)
|
||||
continue
|
||||
paths.append(rel)
|
||||
return paths
|
||||
|
||||
|
||||
def _find_git_root(start: Path) -> Path | None:
|
||||
"""Find the nearest git root containing start, if any."""
|
||||
current = start
|
||||
while True:
|
||||
if (current / ".git").exists():
|
||||
return current
|
||||
parent = current.parent
|
||||
if parent == current:
|
||||
return None
|
||||
current = parent
|
||||
|
||||
|
||||
def load_skills_from_dirs(
|
||||
directories: Iterable[str | Path] | None,
|
||||
*,
|
||||
source: str = "user",
|
||||
create_missing: bool = True,
|
||||
) -> list[SkillDefinition]:
|
||||
"""Load markdown skills from one or more directories.
|
||||
|
||||
@@ -72,7 +167,10 @@ def load_skills_from_dirs(
|
||||
seen: set[Path] = set()
|
||||
for directory in directories:
|
||||
root = Path(directory).expanduser().resolve()
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
if create_missing:
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
elif not root.is_dir():
|
||||
continue
|
||||
candidates: list[Path] = []
|
||||
for child in sorted(root.iterdir()):
|
||||
if child.is_dir():
|
||||
@@ -85,7 +183,10 @@ def load_skills_from_dirs(
|
||||
seen.add(path)
|
||||
content = path.read_text(encoding="utf-8")
|
||||
default_name = path.parent.name
|
||||
name, description = _parse_skill_markdown(default_name, content)
|
||||
metadata = _parse_skill_metadata(default_name, content)
|
||||
name = metadata["name"]
|
||||
description = metadata["description"]
|
||||
display_name = name if name != default_name else None
|
||||
skills.append(
|
||||
SkillDefinition(
|
||||
name=name,
|
||||
@@ -93,6 +194,13 @@ def load_skills_from_dirs(
|
||||
content=content,
|
||||
source=source,
|
||||
path=str(path),
|
||||
base_dir=str(path.parent),
|
||||
command_name=default_name,
|
||||
display_name=display_name,
|
||||
user_invocable=metadata["user_invocable"],
|
||||
disable_model_invocation=metadata["disable_model_invocation"],
|
||||
model=metadata["model"],
|
||||
argument_hint=metadata["argument_hint"],
|
||||
)
|
||||
)
|
||||
return skills
|
||||
@@ -100,39 +208,22 @@ def load_skills_from_dirs(
|
||||
|
||||
def _parse_skill_markdown(default_name: str, content: str) -> tuple[str, str]:
|
||||
"""Parse name and description from a skill markdown file with YAML frontmatter support."""
|
||||
name = default_name
|
||||
description = ""
|
||||
return parse_skill_frontmatter(default_name, content, fallback_template="Skill: {name}")
|
||||
|
||||
lines = content.splitlines()
|
||||
|
||||
# Try YAML frontmatter first (--- ... ---)
|
||||
if content.startswith("---\n"):
|
||||
end_index = content.find("\n---\n", 4)
|
||||
if end_index != -1:
|
||||
try:
|
||||
metadata = yaml.safe_load(content[4:end_index])
|
||||
if isinstance(metadata, dict):
|
||||
val = metadata.get("name")
|
||||
if isinstance(val, str) and val.strip():
|
||||
name = val.strip()
|
||||
val = metadata.get("description")
|
||||
if isinstance(val, str) and val.strip():
|
||||
description = val.strip()
|
||||
except yaml.YAMLError:
|
||||
logger.debug("Failed to parse YAML frontmatter for skill %s", default_name)
|
||||
|
||||
# Fallback: extract from headings and first paragraph
|
||||
if not description:
|
||||
for line in lines:
|
||||
stripped = line.strip()
|
||||
if stripped.startswith("# "):
|
||||
if not name or name == default_name:
|
||||
name = stripped[2:].strip() or default_name
|
||||
continue
|
||||
if stripped and not stripped.startswith("---") and not stripped.startswith("#"):
|
||||
description = stripped[:200]
|
||||
break
|
||||
|
||||
if not description:
|
||||
description = f"Skill: {name}"
|
||||
return name, description
|
||||
def _parse_skill_metadata(default_name: str, content: str) -> dict:
|
||||
parsed = parse_skill_metadata(default_name, content, fallback_template="Skill: {name}")
|
||||
frontmatter = parsed.get("frontmatter")
|
||||
if not isinstance(frontmatter, dict):
|
||||
frontmatter = {}
|
||||
return {
|
||||
"name": str(parsed["name"]),
|
||||
"description": str(parsed["description"]),
|
||||
"user_invocable": parse_bool_frontmatter(frontmatter.get("user-invocable"), default=True),
|
||||
"disable_model_invocation": parse_bool_frontmatter(
|
||||
frontmatter.get("disable-model-invocation"),
|
||||
default=False,
|
||||
),
|
||||
"model": optional_frontmatter_str(frontmatter.get("model")),
|
||||
"argument_hint": optional_frontmatter_str(frontmatter.get("argument-hint")),
|
||||
}
|
||||
|
||||
@@ -13,7 +13,9 @@ class SkillRegistry:
|
||||
|
||||
def register(self, skill: SkillDefinition) -> None:
|
||||
"""Register one skill."""
|
||||
self._skills[skill.name] = skill
|
||||
for key in (skill.name, skill.command_name, skill.display_name, *skill.aliases):
|
||||
if key:
|
||||
self._skills[key] = skill
|
||||
|
||||
def get(self, name: str) -> SkillDefinition | None:
|
||||
"""Return a skill by name."""
|
||||
@@ -21,4 +23,7 @@ class SkillRegistry:
|
||||
|
||||
def list_skills(self) -> list[SkillDefinition]:
|
||||
"""Return all skills sorted by name."""
|
||||
return sorted(self._skills.values(), key=lambda skill: skill.name)
|
||||
unique: dict[tuple[str, str | None], SkillDefinition] = {}
|
||||
for skill in self._skills.values():
|
||||
unique[(skill.source, skill.path or skill.name)] = skill
|
||||
return sorted(unique.values(), key=lambda skill: skill.command_name or skill.name)
|
||||
|
||||
@@ -14,3 +14,11 @@ class SkillDefinition:
|
||||
content: str
|
||||
source: str
|
||||
path: str | None = None
|
||||
base_dir: str | None = None
|
||||
command_name: str | None = None
|
||||
display_name: str | None = None
|
||||
aliases: tuple[str, ...] = ()
|
||||
user_invocable: bool = True
|
||||
disable_model_invocation: bool = False
|
||||
model: str | None = None
|
||||
argument_hint: str | None = None
|
||||
|
||||
@@ -60,9 +60,22 @@ _TEAMMATE_ENV_VARS = [
|
||||
# --- OpenHarness-native provider settings --------------------------------
|
||||
# These are read by settings._apply_env_overrides() and must survive across
|
||||
# tmux boundaries so teammates use the same provider as the leader.
|
||||
"OPENHARNESS_CONFIG_DIR",
|
||||
"OPENHARNESS_DATA_DIR",
|
||||
"OPENHARNESS_LOGS_DIR",
|
||||
"OPENHARNESS_PROFILE",
|
||||
"OPENHARNESS_API_FORMAT",
|
||||
"OPENHARNESS_PROVIDER",
|
||||
"OPENHARNESS_BASE_URL",
|
||||
"OPENHARNESS_MODEL",
|
||||
"OPENHARNESS_ANTHROPIC_API_KEY",
|
||||
"OPENHARNESS_OPENAI_API_KEY",
|
||||
"OPENHARNESS_DASHSCOPE_API_KEY",
|
||||
"OPENHARNESS_MOONSHOT_API_KEY",
|
||||
"OPENHARNESS_GEMINI_API_KEY",
|
||||
"OPENHARNESS_MINIMAX_API_KEY",
|
||||
"OPENHARNESS_NVIDIA_API_KEY",
|
||||
"OPENHARNESS_MODELSCOPE_API_KEY",
|
||||
"OPENAI_API_KEY",
|
||||
]
|
||||
|
||||
@@ -96,6 +109,8 @@ def get_teammate_command() -> str:
|
||||
def build_inherited_cli_flags(
|
||||
*,
|
||||
model: str | None = None,
|
||||
system_prompt: str | None = None,
|
||||
system_prompt_mode: str | None = None,
|
||||
permission_mode: str | None = None,
|
||||
plan_mode_required: bool = False,
|
||||
settings_path: str | None = None,
|
||||
@@ -114,6 +129,10 @@ def build_inherited_cli_flags(
|
||||
|
||||
Args:
|
||||
model: Model override to forward (e.g. ``"claude-opus-4-6"``).
|
||||
system_prompt: System prompt override to forward to the teammate.
|
||||
system_prompt_mode: One of ``"replace"``/``"default"`` or ``"append"``.
|
||||
``append`` maps to ``--append-system-prompt``; anything else uses
|
||||
``--system-prompt``.
|
||||
permission_mode: One of ``"bypassPermissions"``, ``"acceptEdits"``, or None.
|
||||
plan_mode_required: When True, bypass-permissions flag is suppressed
|
||||
(plan mode takes precedence over bypass for safety).
|
||||
@@ -146,6 +165,13 @@ def build_inherited_cli_flags(
|
||||
if model and model != "inherit":
|
||||
flags.extend(["--model", shlex.quote(model)])
|
||||
|
||||
# --- System prompt override ------------------------------------------
|
||||
# Agent definitions can carry a dedicated worker system prompt. Forward it
|
||||
# explicitly so subprocess teammates preserve their role/personality.
|
||||
if system_prompt:
|
||||
prompt_flag = "--append-system-prompt" if system_prompt_mode == "append" else "--system-prompt"
|
||||
flags.extend([prompt_flag, shlex.quote(system_prompt)])
|
||||
|
||||
# --- Settings path propagation ----------------------------------------
|
||||
# Ensures teammates load the same settings JSON as the leader process.
|
||||
if settings_path:
|
||||
|
||||
@@ -54,25 +54,35 @@ class SubprocessBackend:
|
||||
|
||||
flags = build_inherited_cli_flags(
|
||||
model=config.model,
|
||||
system_prompt=config.system_prompt,
|
||||
system_prompt_mode=config.system_prompt_mode,
|
||||
plan_mode_required=config.plan_mode_required,
|
||||
)
|
||||
extra_env = build_inherited_env_vars()
|
||||
|
||||
command = config.command
|
||||
# Only inject the inherited teammate env vars when we are also
|
||||
# building the command ourselves. If the caller supplied a custom
|
||||
# ``config.command`` we honour their choice and don't second-guess
|
||||
# which env vars they want, matching pre-fix behaviour.
|
||||
extra_env: dict[str, str] | None = None
|
||||
argv: list[str] | None = None
|
||||
command: str | None = config.command
|
||||
if command is None:
|
||||
# Build environment export prefix for shell invocation
|
||||
env_prefix = " ".join(f"{k}={v!r}" for k, v in extra_env.items())
|
||||
|
||||
# Direct-exec argv list — no shell, no quoting required. This
|
||||
# avoids Git Bash on Windows being unable to exec a Windows-
|
||||
# pathed python interpreter when bash is itself launched via
|
||||
# ``asyncio.create_subprocess_exec`` (see issue #230).
|
||||
extra_env = build_inherited_env_vars()
|
||||
teammate_cmd = get_teammate_command()
|
||||
if (
|
||||
teammate_cmd.endswith("python")
|
||||
or teammate_cmd.endswith("python3")
|
||||
or teammate_cmd.endswith("python.exe")
|
||||
or teammate_cmd.endswith("python3.exe")
|
||||
or "/python" in teammate_cmd
|
||||
or "\\python" in teammate_cmd.lower()
|
||||
):
|
||||
cmd_parts = [teammate_cmd, "-m", "openharness", "--task-worker"] + flags
|
||||
argv = [teammate_cmd, "-m", "openharness", "--task-worker"] + flags
|
||||
else:
|
||||
cmd_parts = [teammate_cmd, "--task-worker"] + flags
|
||||
command = f"{env_prefix} {' '.join(cmd_parts)}" if env_prefix else " ".join(cmd_parts)
|
||||
argv = [teammate_cmd, "--task-worker"] + flags
|
||||
|
||||
manager = get_task_manager()
|
||||
try:
|
||||
@@ -83,6 +93,8 @@ class SubprocessBackend:
|
||||
task_type=config.task_type,
|
||||
model=config.model,
|
||||
command=command,
|
||||
argv=argv,
|
||||
env=extra_env,
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.error("Failed to spawn teammate %s: %s", agent_id, exc)
|
||||
|
||||
@@ -3,17 +3,48 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import shlex
|
||||
import time
|
||||
from dataclasses import replace
|
||||
from pathlib import Path
|
||||
from typing import Awaitable, Callable
|
||||
from uuid import uuid4
|
||||
|
||||
from openharness.config.paths import get_tasks_dir
|
||||
from openharness.tasks.types import TaskRecord, TaskStatus, TaskType
|
||||
from openharness.utils.shell import create_shell_subprocess
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
_TASK_RESTART_NOTICE = "[OpenHarness] Agent task restarted; prior interactive context was not preserved.\n"
|
||||
|
||||
|
||||
def _encode_task_worker_payload(data: str) -> bytes:
|
||||
"""Serialize one worker input as a single JSON line.
|
||||
|
||||
Plain-text prompts may contain embedded newlines, so they cannot be written
|
||||
directly to a readline()-based worker protocol. We wrap them in a JSON
|
||||
object with a ``text`` field, while preserving already-structured payloads
|
||||
emitted by teammate backends.
|
||||
"""
|
||||
|
||||
stripped = data.rstrip("\n")
|
||||
try:
|
||||
payload = json.loads(stripped)
|
||||
except json.JSONDecodeError:
|
||||
payload = None
|
||||
|
||||
if isinstance(payload, dict) and isinstance(payload.get("text"), str):
|
||||
framed = stripped
|
||||
elif "\n" not in stripped and "\r" not in stripped:
|
||||
framed = stripped
|
||||
else:
|
||||
framed = json.dumps({"text": stripped}, ensure_ascii=False)
|
||||
return (framed + "\n").encode("utf-8")
|
||||
|
||||
CompletionListener = Callable[[TaskRecord], Awaitable[None] | None]
|
||||
|
||||
|
||||
class BackgroundTaskManager:
|
||||
"""Manage shell and agent subprocess tasks."""
|
||||
@@ -25,16 +56,39 @@ class BackgroundTaskManager:
|
||||
self._output_locks: dict[str, asyncio.Lock] = {}
|
||||
self._input_locks: dict[str, asyncio.Lock] = {}
|
||||
self._generations: dict[str, int] = {}
|
||||
self._completion_listeners: dict[str, CompletionListener] = {}
|
||||
|
||||
async def create_shell_task(
|
||||
self,
|
||||
*,
|
||||
command: str,
|
||||
command: str | None = None,
|
||||
description: str,
|
||||
cwd: str | Path,
|
||||
task_type: TaskType = "local_bash",
|
||||
env: dict[str, str] | None = None,
|
||||
argv: list[str] | None = None,
|
||||
) -> TaskRecord:
|
||||
"""Start a background shell command."""
|
||||
"""Start a background command.
|
||||
|
||||
Either ``command`` (a shell-evaluated string) or ``argv`` (a direct
|
||||
argv list) must be supplied. The ``argv`` form bypasses shell
|
||||
invocation entirely — it spawns the executable directly via
|
||||
``asyncio.create_subprocess_exec(*argv)`` — which is the right choice
|
||||
for teammate spawning on Windows: Git Bash cannot reliably exec
|
||||
Windows-pathed binaries (e.g. ``C:\\Users\\...\\python.exe``) when it
|
||||
is itself launched via ``create_subprocess_exec`` with that path
|
||||
embedded in a ``-lc`` string, even though the same shell call works
|
||||
interactively. Bypassing the shell sidesteps that entire class of
|
||||
platform-quoting bug.
|
||||
|
||||
``env`` is merged with ``os.environ`` when the subprocess is launched,
|
||||
so callers should pass only the variables they want to add or
|
||||
override.
|
||||
"""
|
||||
if command is None and argv is None:
|
||||
raise ValueError("create_shell_task requires either command or argv")
|
||||
if command is not None and argv is not None:
|
||||
raise ValueError("create_shell_task accepts only one of command or argv")
|
||||
task_id = _task_id(task_type)
|
||||
output_path = get_tasks_dir() / f"{task_id}.log"
|
||||
record = TaskRecord(
|
||||
@@ -47,6 +101,8 @@ class BackgroundTaskManager:
|
||||
command=command,
|
||||
created_at=time.time(),
|
||||
started_at=time.time(),
|
||||
env=dict(env) if env is not None else None,
|
||||
argv=list(argv) if argv is not None else None,
|
||||
)
|
||||
output_path.write_text("", encoding="utf-8")
|
||||
self._tasks[task_id] = record
|
||||
@@ -65,24 +121,34 @@ class BackgroundTaskManager:
|
||||
model: str | None = None,
|
||||
api_key: str | None = None,
|
||||
command: str | None = None,
|
||||
env: dict[str, str] | None = None,
|
||||
argv: list[str] | None = None,
|
||||
) -> TaskRecord:
|
||||
"""Start a local agent task as a subprocess."""
|
||||
if command is None:
|
||||
"""Start a local agent task as a subprocess.
|
||||
|
||||
Prefer ``argv`` (direct exec, no shell) over ``command`` (shell-
|
||||
evaluated) for teammate spawn — see :meth:`create_shell_task` for
|
||||
the cross-platform reasoning. ``env`` is forwarded to
|
||||
:meth:`create_shell_task` and ultimately merged with ``os.environ``
|
||||
at process spawn time.
|
||||
"""
|
||||
if command is None and argv is None:
|
||||
effective_api_key = api_key or os.environ.get("ANTHROPIC_API_KEY")
|
||||
if not effective_api_key:
|
||||
raise ValueError(
|
||||
"Local agent tasks require ANTHROPIC_API_KEY or an explicit command override"
|
||||
"Local agent tasks require ANTHROPIC_API_KEY or an explicit command/argv override"
|
||||
)
|
||||
cmd = ["python", "-m", "openharness", "--api-key", effective_api_key]
|
||||
argv = ["python", "-m", "openharness", "--api-key", effective_api_key]
|
||||
if model:
|
||||
cmd.extend(["--model", model])
|
||||
command = " ".join(shlex.quote(part) for part in cmd)
|
||||
argv.extend(["--model", model])
|
||||
|
||||
record = await self.create_shell_task(
|
||||
command=command,
|
||||
description=description,
|
||||
cwd=cwd,
|
||||
task_type=task_type,
|
||||
env=env,
|
||||
argv=argv,
|
||||
)
|
||||
updated = replace(record, prompt=prompt)
|
||||
if task_type != "local_agent":
|
||||
@@ -143,21 +209,23 @@ class BackgroundTaskManager:
|
||||
|
||||
task.status = "killed"
|
||||
task.ended_at = time.time()
|
||||
await self._notify_completion_listeners(task)
|
||||
return task
|
||||
|
||||
async def write_to_task(self, task_id: str, data: str) -> None:
|
||||
"""Write one line to task stdin, auto-resuming local agents when needed."""
|
||||
task = self._require_task(task_id)
|
||||
payload = _encode_task_worker_payload(data)
|
||||
async with self._input_locks[task_id]:
|
||||
process = await self._ensure_writable_process(task)
|
||||
process.stdin.write((data.rstrip("\n") + "\n").encode("utf-8"))
|
||||
process.stdin.write(payload)
|
||||
try:
|
||||
await process.stdin.drain()
|
||||
except (BrokenPipeError, ConnectionResetError):
|
||||
if task.type not in {"local_agent", "remote_agent", "in_process_teammate"}:
|
||||
raise ValueError(f"Task {task_id} does not accept input") from None
|
||||
process = await self._restart_agent_task(task)
|
||||
process.stdin.write((data.rstrip("\n") + "\n").encode("utf-8"))
|
||||
process.stdin.write(payload)
|
||||
await process.stdin.drain()
|
||||
|
||||
def read_task_output(self, task_id: str, *, max_bytes: int = 12000) -> str:
|
||||
@@ -168,6 +236,16 @@ class BackgroundTaskManager:
|
||||
return content[-max_bytes:]
|
||||
return content
|
||||
|
||||
def register_completion_listener(self, listener: CompletionListener) -> Callable[[], None]:
|
||||
"""Register a callback fired whenever a task reaches a terminal state."""
|
||||
listener_id = uuid4().hex
|
||||
self._completion_listeners[listener_id] = listener
|
||||
|
||||
def _unregister() -> None:
|
||||
self._completion_listeners.pop(listener_id, None)
|
||||
|
||||
return _unregister
|
||||
|
||||
async def _watch_process(
|
||||
self,
|
||||
task_id: str,
|
||||
@@ -188,6 +266,7 @@ class BackgroundTaskManager:
|
||||
if task.status != "killed":
|
||||
task.status = "completed" if return_code == 0 else "failed"
|
||||
task.ended_at = time.time()
|
||||
await self._notify_completion_listeners(task)
|
||||
self._processes.pop(task_id, None)
|
||||
self._waiters.pop(task_id, None)
|
||||
|
||||
@@ -210,18 +289,43 @@ class BackgroundTaskManager:
|
||||
|
||||
async def _start_process(self, task_id: str) -> asyncio.subprocess.Process:
|
||||
task = self._require_task(task_id)
|
||||
if task.command is None:
|
||||
raise ValueError(f"Task {task_id} does not have a command to run")
|
||||
if task.command is None and task.argv is None:
|
||||
raise ValueError(f"Task {task_id} does not have a command or argv to run")
|
||||
|
||||
generation = self._generations.get(task_id, 0) + 1
|
||||
self._generations[task_id] = generation
|
||||
process = await create_shell_subprocess(
|
||||
task.command,
|
||||
cwd=task.cwd,
|
||||
stdin=asyncio.subprocess.PIPE,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.STDOUT,
|
||||
)
|
||||
# Merge task-specific env vars on top of the parent process environment
|
||||
# so the child sees both. Passing ``None`` lets the OS inherit env
|
||||
# directly, which is the legacy behaviour for plain shell tasks.
|
||||
merged_env: dict[str, str] | None
|
||||
if task.env:
|
||||
merged_env = {**os.environ, **task.env}
|
||||
else:
|
||||
merged_env = None
|
||||
|
||||
if task.argv is not None:
|
||||
# Direct-exec route. No shell. Used for teammate spawn so we
|
||||
# don't have to round-trip Windows paths through Git Bash, which
|
||||
# cannot reliably exec ``C:\\...\\python.exe`` when launched
|
||||
# itself via ``asyncio.create_subprocess_exec`` (see #230).
|
||||
process = await asyncio.create_subprocess_exec(
|
||||
*task.argv,
|
||||
cwd=str(Path(task.cwd).resolve()),
|
||||
stdin=asyncio.subprocess.PIPE,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.STDOUT,
|
||||
env=merged_env,
|
||||
)
|
||||
else:
|
||||
assert task.command is not None
|
||||
process = await create_shell_subprocess(
|
||||
task.command,
|
||||
cwd=task.cwd,
|
||||
stdin=asyncio.subprocess.PIPE,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.STDOUT,
|
||||
env=merged_env,
|
||||
)
|
||||
self._processes[task_id] = process
|
||||
self._waiters[task_id] = asyncio.create_task(
|
||||
self._watch_process(task_id, process, generation)
|
||||
@@ -240,8 +344,8 @@ class BackgroundTaskManager:
|
||||
return await self._restart_agent_task(task)
|
||||
|
||||
async def _restart_agent_task(self, task: TaskRecord) -> asyncio.subprocess.Process:
|
||||
if task.command is None:
|
||||
raise ValueError(f"Task {task.id} does not have a restart command")
|
||||
if task.command is None and task.argv is None:
|
||||
raise ValueError(f"Task {task.id} does not have a restart command or argv")
|
||||
|
||||
waiter = self._waiters.get(task.id)
|
||||
if waiter is not None and not waiter.done():
|
||||
@@ -249,12 +353,25 @@ class BackgroundTaskManager:
|
||||
|
||||
restart_count = int(task.metadata.get("restart_count", "0")) + 1
|
||||
task.metadata["restart_count"] = str(restart_count)
|
||||
task.metadata["status_note"] = "Task restarted; prior interactive context was not preserved."
|
||||
task.status = "running"
|
||||
task.started_at = time.time()
|
||||
task.ended_at = None
|
||||
task.return_code = None
|
||||
with task.output_file.open("ab") as handle:
|
||||
handle.write(_TASK_RESTART_NOTICE.encode("utf-8"))
|
||||
return await self._start_process(task.id)
|
||||
|
||||
async def _notify_completion_listeners(self, task: TaskRecord) -> None:
|
||||
snapshot = replace(task, metadata=dict(task.metadata))
|
||||
for listener_id, listener in list(self._completion_listeners.items()):
|
||||
try:
|
||||
maybe_awaitable = listener(snapshot)
|
||||
if maybe_awaitable is not None:
|
||||
await maybe_awaitable
|
||||
except Exception:
|
||||
log.exception("Task completion listener %s failed for task %s", listener_id, task.id)
|
||||
|
||||
def close(self) -> None:
|
||||
"""Best-effort cleanup for any tracked subprocesses and watcher tasks."""
|
||||
for waiter in list(self._waiters.values()):
|
||||
@@ -342,6 +459,7 @@ def _task_id(task_type: TaskType) -> str:
|
||||
"local_agent": "a",
|
||||
"remote_agent": "r",
|
||||
"in_process_teammate": "t",
|
||||
"dream": "d",
|
||||
}
|
||||
return f"{prefixes[task_type]}{uuid4().hex[:8]}"
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@ from pathlib import Path
|
||||
from typing import Literal
|
||||
|
||||
|
||||
TaskType = Literal["local_bash", "local_agent", "remote_agent", "in_process_teammate"]
|
||||
TaskType = Literal["local_bash", "local_agent", "remote_agent", "in_process_teammate", "dream"]
|
||||
TaskStatus = Literal["pending", "running", "completed", "failed", "killed"]
|
||||
|
||||
|
||||
@@ -28,3 +28,5 @@ class TaskRecord:
|
||||
ended_at: float | None = None
|
||||
return_code: int | None = None
|
||||
metadata: dict[str, str] = field(default_factory=dict)
|
||||
env: dict[str, str] | None = None
|
||||
argv: list[str] | None = None
|
||||
|
||||
@@ -19,6 +19,8 @@ from openharness.tools.file_read_tool import FileReadTool
|
||||
from openharness.tools.file_write_tool import FileWriteTool
|
||||
from openharness.tools.glob_tool import GlobTool
|
||||
from openharness.tools.grep_tool import GrepTool
|
||||
from openharness.tools.image_generation_tool import ImageGenerationTool
|
||||
from openharness.tools.image_to_text_tool import ImageToTextTool
|
||||
from openharness.tools.list_mcp_resources_tool import ListMcpResourcesTool
|
||||
from openharness.tools.lsp_tool import LspTool
|
||||
from openharness.tools.mcp_auth_tool import McpAuthTool
|
||||
@@ -57,6 +59,8 @@ def create_default_tool_registry(mcp_manager=None) -> ToolRegistry:
|
||||
McpAuthTool(),
|
||||
GlobTool(),
|
||||
GrepTool(),
|
||||
ImageToTextTool(),
|
||||
ImageGenerationTool(),
|
||||
SkillTool(),
|
||||
ToolSearchTool(),
|
||||
WebFetchTool(),
|
||||
|
||||
@@ -8,8 +8,10 @@ from pydantic import BaseModel, Field
|
||||
|
||||
from openharness.coordinator.agent_definitions import get_agent_definition
|
||||
from openharness.coordinator.coordinator_mode import get_team_registry
|
||||
from openharness.hooks import HookEvent
|
||||
from openharness.swarm.registry import get_backend_registry
|
||||
from openharness.swarm.types import TeammateSpawnConfig
|
||||
from openharness.tasks import get_task_manager
|
||||
from openharness.tools.base import BaseTool, ToolExecutionContext, ToolResult
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -93,9 +95,47 @@ class AgentTool(BaseTool):
|
||||
registry.create_team(arguments.team)
|
||||
registry.add_agent(arguments.team, result.task_id)
|
||||
|
||||
if context.hook_executor is not None:
|
||||
manager = get_task_manager()
|
||||
unregister = None
|
||||
|
||||
async def _emit_subagent_stop(task_record) -> None:
|
||||
nonlocal unregister
|
||||
if task_record.id != result.task_id:
|
||||
return
|
||||
if unregister is not None:
|
||||
unregister()
|
||||
unregister = None
|
||||
await context.hook_executor.execute(
|
||||
HookEvent.SUBAGENT_STOP,
|
||||
{
|
||||
"event": HookEvent.SUBAGENT_STOP.value,
|
||||
"agent_id": result.agent_id,
|
||||
"task_id": result.task_id,
|
||||
"backend_type": result.backend_type,
|
||||
"status": task_record.status,
|
||||
"return_code": task_record.return_code,
|
||||
"description": arguments.description,
|
||||
"subagent_type": arguments.subagent_type or "agent",
|
||||
"team": team,
|
||||
"mode": arguments.mode,
|
||||
},
|
||||
)
|
||||
|
||||
unregister = manager.register_completion_listener(_emit_subagent_stop)
|
||||
task_record = manager.get_task(result.task_id)
|
||||
if task_record is not None and task_record.status in {"completed", "failed", "killed"}:
|
||||
await _emit_subagent_stop(task_record)
|
||||
|
||||
return ToolResult(
|
||||
output=(
|
||||
f"Spawned agent {result.agent_id} "
|
||||
f"(task_id={result.task_id}, backend={result.backend_type})"
|
||||
)
|
||||
),
|
||||
metadata={
|
||||
"agent_id": result.agent_id,
|
||||
"task_id": result.task_id,
|
||||
"backend_type": result.backend_type,
|
||||
"description": arguments.description,
|
||||
},
|
||||
)
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user