Compare commits

...

45 Commits

Author SHA1 Message Date
tjb-tech 8e012b4c2c feat: add dynamic workflows 2026-06-04 00:53:00 +00:00
guyuming bf5931e7c1 fix(runtime): close API clients on shutdown
Close API clients when the runtime shuts down to release HTTP connection pool resources, and surface the active session ID in /session output.

Cherry-picked from PR #273. Excluded the embedded-wrapper session resolver and memory parser API changes because they need a narrower public API design.
2026-05-27 10:19:21 +00:00
baizenghu a68d0f984d feat(hooks): add priority ordering for lifecycle hooks (#279)
Hooks now accept an optional priority field and HookRegistry returns hooks in stable highest-priority-first order.\n\nVerified with:\n- uv run pytest -o addopts='' tests/test_hooks tests/test_plugins/test_lifecycle_flow.py tests/test_plugins/test_loader.py tests/test_ui/test_runtime_plugin_tools.py -q\n- uv run ruff check src/openharness/hooks tests/test_hooks tests/test_plugins/test_lifecycle_flow.py tests/test_plugins/test_loader.py tests/test_ui/test_runtime_plugin_tools.py
2026-05-27 18:17:16 +08:00
Mcy0618 033dce67b1 fix(runtime): refresh prompt for permission mode changes
Inject the active permission mode into the runtime system prompt so plan/default/full-auto state is visible to the model.

Refresh the engine model, effort, permission checker, and system prompt when runtime-affecting slash commands update settings.

Based-on: #268
2026-05-24 09:10:10 +00:00
SABUJ123472 4c3bf5bbc9 fix(platform): harden Windows lock detection
Recognize win32 as a Windows platform alias and wrap missing platform lock modules in SwarmLockUnavailableError instead of leaking raw ImportError.

Based-on: #34
2026-05-24 09:07:19 +00:00
Siaochuan 63ac368975 fix(auth): prefer scoped provider env keys
Prefer OPENHARNESS_<PROVIDER>_API_KEY variables over provider-native globals while resolving API-key auth. Keep provider-native variables as fallback and avoid applying unrelated native keys across active profiles.

Forward the OpenHarness-scoped auth/provider environment variables to spawned teammates.

Based-on: #93
2026-05-24 09:05:44 +00:00
tjb-tech fea0b75bc4 fix(auth): show subscription setup errors
Preserve subscription-auth failures instead of rewriting them as missing API keys, and point Claude/Codex subscription users at the correct login and provider commands.

Fixes #254
2026-05-24 09:01:35 +00:00
tjb-tech c281d34c4a fix(swarm): inherit OpenHarness permission config
Forward OpenHarness config/data env vars to spawned teammates and apply CLI permission_mode overrides to Settings.permission.mode so full_auto survives sub-agent startup paths.

Fixes #274
2026-05-24 08:18:20 +00:00
tjb-tech f5608951e1 test(ohmo): cover Feishu domain setup prompts 2026-05-24 08:14:31 +00:00
tjb-tech 92e298852c Merge branch 'pr-276' into codex/pr-merge-test-20260524 2026-05-24 08:13:17 +00:00
tjb-tech 27bb93b810 Merge branch 'pr-272' into codex/pr-merge-test-20260524 2026-05-24 08:13:15 +00:00
tjb-tech ee0af9a24c Merge branch 'pr-270' into codex/pr-merge-test-20260524 2026-05-24 08:13:13 +00:00
hinotoi-agent 9a24b8dfe2 fix(commands): keep session transcript commands local-only 2026-05-22 13:19:37 +08:00
hinotoi-agent 4321bd824d fix(commands): keep project context commands local-only 2026-05-22 08:15:41 +08:00
WcW 175fe501a6 fix(ohmo-init): add domain prompt to feishu setup flow
ohmo init rewrites gateway.json without a domain field, undoing the
larksuite.com fix on every reconfigure. Add a domain selector so the
choice is persisted through init.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-21 16:48:07 +08:00
WcW 254ed90aed fix(feishu): add domain config to support Lark international (larksuite.com)
REST client and WebSocket client both defaulted to open.feishu.cn; apps
created on open.larksuite.com were rejected with "Incorrect domain name".

Add `domain` field to FeishuConfig (default: https://open.feishu.cn for
backwards compatibility). Pass it to both lark.Client.builder().domain()
and lark.ws.Client(domain=).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-21 15:31:24 +08:00
Jiabin Tang 09bc641aad feat(memory): add Claude-style memory runtime (#267)
* Add Claude-style memory runtime

* fix(memory): trim relevance EOF whitespace
2026-05-17 15:31:35 +08:00
tjb-tech 6243793ec0 fix(tui): keep image paste CI stable 2026-05-17 06:18:59 +00:00
tjb-tech 330ece7a62 feat(tui): support clipboard image paste
Fixes #265
2026-05-17 06:13:19 +00:00
Yang XiuYuan 6747d12385 feat(memory): add structured schema and usage index (#266)
* feat(memory): add structured schema and usage index

Add schema-v1 frontmatter for memory files, including stable ids, deterministic signatures, soft delete metadata, TTL handling, and migration support for legacy stores.

Track recalled memories in usage_index.json so retrieval can prioritize useful memories and auto-dream can review stale unused entries before pruning.

Keep project and ohmo memory backends aligned under the same behavior while preserving runtime compatibility for unmigrated Markdown files.

* fix(memory): make backend migration defaults explicit
2026-05-17 14:01:24 +08:00
tjb-tech 7a0bfe9122 Merge remote-tracking branch 'origin/main' into review/pr-248 2026-05-16 13:11:47 +00:00
tjb-tech 595a9ab3a4 Merge remote-tracking branch 'origin/main' into review/pr-256
# Conflicts:
#	CHANGELOG.md
2026-05-16 13:04:45 +00:00
tjb-tech 4ba71f2d4a fix(telegram): use configured bot identity
Remove the hard-coded nanobot name from Telegram /start and /help responses and sanitize dangling tool-use turns before continue_pending resumes a session.
2026-05-16 12:45:08 +00:00
tjb-tech 889d9dcbda fix(grep): keep missing roots as tool errors 2026-05-16 12:24:21 +00:00
tjb-tech a2888506ba fix(web): support explicit search endpoint and proxy 2026-05-16 12:16:15 +00:00
tjb-tech 0531963435 Merge remote-tracking branch 'origin/main' into review/pr-260
# Conflicts:
#	CHANGELOG.md
2026-05-16 11:52:30 +00:00
tjb-tech c6b7552008 fix(telegram): hide bot tokens in dependency logs
Silence Telegram HTTP dependency loggers before polling starts, keep pending updates by default, and surface channel startup/handler failures through gateway state.
2026-05-16 11:50:59 +00:00
David Whatley 635d77fad4 fix(openai): omit empty reasoning_content by default (#263)
`reasoning_content` is a non-standard field that some thinking-model
providers (Kimi k2.5 on Anthropic-format) require on every assistant
message with tool calls — even when empty. Other OpenAI-compatible
providers (Cerebras, NVIDIA NIM, OpenAI direct, etc.) reject the field
outright with a 400 ``wrong_api_format`` validation error.

Until now, ``_convert_assistant_message`` unconditionally emitted
``reasoning_content: ''`` for tool-using assistant messages. That hard-
breaks every strict-OpenAI provider on the very first tool turn.

Behaviour change:

- Captured non-empty reasoning is still always replayed.
- The empty-string fallback is now opt-in via
  ``OPENHARNESS_REQUIRE_EMPTY_REASONING_CONTENT=1``.
- Default behaviour matches strict-OpenAI providers, which is the
  larger / standards-compliant population. Kimi-on-Anthropic users
  set the env var in their dotfiles or settings.

Verified end-to-end against Cerebras gpt-oss-120b (previously 100%
failure rate from this field) — zero ``wrong_api_format`` errors.

Tests added: 6 cases covering captured-replay, opt-in matrix
(truthy/falsy values), and the no-tool-calls invariant.
2026-05-16 19:46:00 +08:00
Hinotobi 7b44b7dab1 [security] fix(ohmo): scope Slack thread sessions by sender (#255) 2026-05-16 19:45:56 +08:00
tjb-tech 484502168d fix(commands): resolve remote admin command merges 2026-05-16 11:44:36 +00:00
tjb-tech 8b2ec8300d Merge branch 'review/pr-251-merge' 2026-05-16 11:40:52 +00:00
tjb-tech a1fef67cdc Merge branch 'review/pr-252-merge'
# Conflicts:
#	src/openharness/commands/registry.py
#	tests/test_commands/test_registry.py
#	tests/test_ohmo/test_gateway.py
2026-05-16 11:40:43 +00:00
tjb-tech 9cff3a05b0 Merge branch 'review/pr-253-merge'
# Conflicts:
#	tests/test_commands/test_registry.py
2026-05-16 11:40:01 +00:00
Hinotobi c38e2fc066 [security] fix(commands): keep autopilot local-only by default (#258) 2026-05-16 19:38:39 +08:00
Hinotobi 653369d983 [security] fix(commands): keep /commit local-only over remote channels (#261) 2026-05-16 19:38:35 +08:00
yl-jiang f61f70d7b1 feat(tui): preview diffs before edit and write approval
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-13 13:44:43 +08:00
Pat Yang b93b2cfb30 fix(codex): pass reasoning effort separately 2026-05-12 14:34:12 +08:00
hinotoi-agent efb4b71eb3 chore: retrigger CI for diff command fix 2026-05-12 10:21:54 +08:00
hinotoi-agent c70811cc40 fix(commands): keep diff local-only by default 2026-05-12 10:05:00 +08:00
hinotoi-agent 73fb0fa0bf fix(commands): keep tasks local-only remotely 2026-05-12 09:06:43 +08:00
d 🔹 a75053a740 fix(channels): declare reply_to_message field on TelegramConfig
`TelegramChannel.send` reads `self.config.reply_to_message` in the
outbound path, but the field was never declared on `TelegramConfig` —
only `ohmo init` (interactive mode) wrote it as a free-form key into
gateway.json. Configs that do not go through the interactive prompt —
`ohmo init --no-interactive`, pre-0.1.9 hand-written gateway.json, the
docs example — never have the key set, so every outbound message
crashes with `AttributeError: 'TelegramConfig' object has no attribute
'reply_to_message'` and the bot appears completely silent to the user.

Add `reply_to_message: bool = True` to `TelegramConfig` so the attribute
always exists. The default matches the interactive `ohmo init` default
(`default=bool(prior.get("reply_to_message", True))` in `ohmo/cli.py`),
so non-interactive and interactive configs that accept defaults behave
identically.

Same shape as #192 (proxy field), which added a missing `TelegramConfig`
attribute the implementation already assumed.

Closes #243

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 03:39:34 +08:00
Yudong An 5b46a2f7b3 Fix cron scheduler on Windows 2026-05-11 15:25:01 +08:00
Jiabin Tang 1929ad8051 feat(ohmo): add agent-turn cron delivery (#247) 2026-05-10 16:55:10 +08:00
Jiabin Tang f61901e842 feat(memory): add auto-dream consolidation (#246)
Add auto-dream memory consolidation with lock-based scheduling, manual /dream commands, backup/diff/rollback support, and ohmo integration for project and personal memory.
2026-05-09 14:09:55 +08:00
Jiabin Tang 0c6b81d61f feat: add configurable image generation tool (#244)
Add a configurable image_generation tool with OpenAI-compatible and Codex hosted providers, propagate generated media metadata through stream events, and let ohmo channels send generated image/file paths automatically.
2026-05-09 13:18:23 +08:00
119 changed files with 9984 additions and 299 deletions
+10
View File
@@ -6,6 +6,16 @@ The format is based on Keep a Changelog, and this project currently tracks chang
## [Unreleased]
### Added
- Hooks now support a `priority` field (default `0`). Within an event, hooks run highest-priority first, and hooks sharing a priority keep their registration order. This lets users order, for example, a security-check hook ahead of a logging hook regardless of where each is declared in settings or contributed by plugins.
- `edit_file` and `write_file` in the React TUI now preview a unified diff before applying file changes, let users approve once or for the rest of the session, and skip the extra prompt automatically in `full_auto` mode.
### Fixed
- Codex subscription requests now pass reasoning effort separately, enabling `gpt-5.5` with `xhigh` effort instead of treating `gpt-5.5 xhigh` as an unsupported model name.
- Telegram channel now delivers replies again under `ohmo init --no-interactive` and other configs that do not write a `reply_to_message` field. `TelegramConfig` declares `reply_to_message: bool = True` so the attribute access in `TelegramChannel.send` no longer raises `AttributeError` and outbound progress/tool-hint/final messages are sent as expected. See issue #243.
## [0.1.9] - 2026-05-07
### Added
+16
View File
@@ -567,6 +567,22 @@ Use `/skills` to list loaded skills with their source and path. User-invocable s
**Compatible with [anthropics/skills](https://github.com/anthropics/skills)** — use the `SKILL.md` directory layout above.
### 🌐 Web search and proxy settings
Built-in `web_search` uses DuckDuckGo HTML search by default. In regions where that endpoint is unreachable, point OpenHarness at a trusted public HTML search endpoint or your own SearXNG instance:
```bash
export OPENHARNESS_WEB_SEARCH_URL="https://your-searxng.example/search"
```
`web_search` and `web_fetch` keep `trust_env=False` for SSRF safety, so they do not automatically inherit `HTTP_PROXY` / `HTTPS_PROXY`. If you need a proxy, opt in with an OpenHarness-specific variable:
```bash
export OPENHARNESS_WEB_PROXY="http://127.0.0.1:7890"
```
The proxy URL must be HTTP/HTTPS and cannot contain embedded credentials.
### 🔌 Plugin System
**Compatible with [claude-code plugins](https://github.com/anthropics/claude-code/tree/main/plugins)**. Tested with 12 official plugins:
+103 -17
View File
@@ -1,6 +1,8 @@
import React, {useDeferredValue, useEffect, useMemo, useState} from 'react';
import React, {useDeferredValue, useEffect, useMemo, useRef, useState} from 'react';
import {Box, Text, useApp, useInput} from 'ink';
import {readClipboardImage, type ImageAttachment} from './clipboardImage.js';
import {filterCommandItems} from './commandPalette.js';
import {CommandPicker} from './components/CommandPicker.js';
import {ConversationView} from './components/ConversationView.js';
import {ModalHost} from './components/ModalHost.js';
@@ -11,7 +13,7 @@ import {SwarmPanel} from './components/SwarmPanel.js';
import {TodoPanel} from './components/TodoPanel.js';
import {useBackendSession} from './hooks/useBackendSession.js';
import {ThemeProvider, useTheme} from './theme/ThemeContext.js';
import type {FrontendConfig} from './types.js';
import type {FrontendConfig, ImageAttachmentPayload} from './types.js';
const rawReturnSubmit = process.env.OPENHARNESS_FRONTEND_RAW_RETURN === '1';
const scriptedSteps = (() => {
@@ -64,6 +66,8 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
const [modalInput, setModalInput] = useState('');
const [history, setHistory] = useState<string[]>([]);
const [historyIndex, setHistoryIndex] = useState(-1);
const [imageAttachments, setImageAttachments] = useState<ImageAttachment[]>([]);
const [clipboardStatus, setClipboardStatus] = useState<string | null>(null);
const [lastEscapeAt, setLastEscapeAt] = useState(0);
const [scriptIndex, setScriptIndex] = useState(0);
const [pickerIndex, setPickerIndex] = useState(0);
@@ -77,6 +81,7 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
const deferredTodoMarkdown = useDeferredValue(session.todoMarkdown);
const deferredSwarmTeammates = useDeferredValue(session.swarmTeammates);
const deferredSwarmNotifications = useDeferredValue(session.swarmNotifications);
const clipboardStatusTimerRef = useRef<NodeJS.Timeout | null>(null);
useEffect(() => {
const nextTheme = session.status.theme;
@@ -85,6 +90,47 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
}
}, [session.status.theme, setThemeName]);
useEffect(() => {
return () => {
if (clipboardStatusTimerRef.current) {
clearTimeout(clipboardStatusTimerRef.current);
}
};
}, []);
const setTemporaryClipboardStatus = (message: string): void => {
setClipboardStatus(message);
if (clipboardStatusTimerRef.current) {
clearTimeout(clipboardStatusTimerRef.current);
}
clipboardStatusTimerRef.current = setTimeout(() => {
setClipboardStatus(null);
clipboardStatusTimerRef.current = null;
}, 2500);
};
const attachClipboardImage = (): void => {
void (async () => {
const image = await readClipboardImage();
if (!image) {
setTemporaryClipboardStatus('No image found in clipboard');
return;
}
setImageAttachments((items) => [...items, image]);
setTemporaryClipboardStatus(`Attached ${image.label}`);
})().catch((error: unknown) => {
const message = error instanceof Error ? error.message : String(error);
setTemporaryClipboardStatus(`Clipboard image unavailable: ${message}`);
});
};
const imagePayloads = (): ImageAttachmentPayload[] =>
imageAttachments.map((image) => ({
media_type: image.media_type,
data: image.data,
source_path: image.source_path,
}));
// Current tool name for spinner
const currentToolName = useMemo(() => {
for (let i = deferredTranscript.length - 1; i >= 0; i--) {
@@ -100,13 +146,7 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
}, [deferredTranscript]);
// Command hints
const commandHints = useMemo(() => {
const value = input.trim();
if (!value.startsWith('/')) {
return [] as string[];
}
return session.commands.filter((cmd) => cmd.startsWith(value)).slice(0, 10);
}, [session.commands, input]);
const commandHints = useMemo(() => filterCommandItems(session.commandItems, input), [session.commandItems, input]);
const showPicker = commandHints.length > 0 && !session.busy && !session.modal && !selectModal;
const outputStyle = String(session.status.output_style ?? 'default');
@@ -191,6 +231,11 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
return;
}
if (!session.busy && key.ctrl && chunk === 'v') {
attachClipboardImage();
return;
}
// Let ink-text-input handle pasted text directly.
if (isPaste) {
return;
@@ -270,6 +315,41 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
return;
}
// --- Edit diff modal (also appears while busy) ---
if (session.modal?.kind === 'edit_diff') {
if (chunk.toLowerCase() === 'y') {
session.sendRequest({
type: 'permission_response',
request_id: session.modal.request_id,
allowed: true,
permission_reply: 'once',
});
session.setModal(null);
return;
}
if (chunk.toLowerCase() === 'a') {
session.sendRequest({
type: 'permission_response',
request_id: session.modal.request_id,
allowed: true,
permission_reply: 'always',
});
session.setModal(null);
return;
}
if (chunk.toLowerCase() === 'n' || isEscape) {
session.sendRequest({
type: 'permission_response',
request_id: session.modal.request_id,
allowed: false,
permission_reply: 'reject',
});
session.setModal(null);
return;
}
return;
}
// --- Question modal (also appears while busy) ---
if (session.modal?.kind === 'question') {
return; // Let TextInput in ModalHost handle input
@@ -307,8 +387,8 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
const selected = commandHints[pickerIndex];
if (selected) {
setInput('');
if (!handleCommand(selected)) {
onSubmit(selected);
if (!handleCommand(selected.name)) {
onSubmit(selected.name);
}
}
return;
@@ -320,7 +400,7 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
// the user can hit Enter immediately to run it, or keep
// typing to add args. The trailing space made it look like
// Tab was "committing" with a token, which broke the flow.
setInput(selected);
setInput(selected.name);
}
return;
}
@@ -332,8 +412,9 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
if (isEscape) {
const now = Date.now();
if (input && now - lastEscapeAt < 500) {
if ((input || imageAttachments.length > 0) && now - lastEscapeAt < 500) {
setInput('');
setImageAttachments([]);
setHistoryIndex(-1);
setLastEscapeAt(0);
return;
@@ -373,7 +454,7 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
setModalInput('');
return;
}
if (!value.trim() || session.busy || !session.ready) {
if ((!value.trim() && imageAttachments.length === 0) || session.busy || !session.ready) {
if (session.busy && value.trim() === '/stop') {
session.sendRequest({type: 'interrupt'});
session.setBusyLabel('Stopping current operation...');
@@ -382,16 +463,19 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
return;
}
// Check if it's an interactive command
if (handleCommand(value)) {
if (imageAttachments.length === 0 && handleCommand(value)) {
setHistory((items) => [...items, value]);
setHistoryIndex(-1);
setInput('');
return;
}
session.sendRequest({type: 'submit_line', line: value});
setHistory((items) => [...items, value]);
session.sendRequest({type: 'submit_line', line: value, images: imagePayloads()});
if (value.trim()) {
setHistory((items) => [...items, value]);
}
setHistoryIndex(-1);
setInput('');
setImageAttachments([]);
session.setBusy(true);
};
@@ -476,6 +560,8 @@ function AppInner({config}: {config: FrontendConfig}): React.JSX.Element {
toolName={session.busy ? currentToolName : undefined}
statusLabel={session.busy ? (session.busyLabel ?? (currentToolName ? `Running ${currentToolName}...` : 'Running agent loop...')) : undefined}
suppressSubmit={showPicker}
imageAttachmentLabels={imageAttachments.map((image) => image.label)}
clipboardStatus={clipboardStatus}
/>
)}
+173
View File
@@ -0,0 +1,173 @@
import {execFile} from 'node:child_process';
import {mkdtemp, readFile, rm, stat} from 'node:fs/promises';
import {tmpdir} from 'node:os';
import {join} from 'node:path';
export type ImageAttachment = {
id: string;
label: string;
media_type: string;
data: string;
source_path?: string;
size_bytes?: number;
};
const MAX_CLIPBOARD_IMAGE_BYTES = 15 * 1024 * 1024;
const EXEC_TIMEOUT_MS = 2500;
type ClipboardImageRead = {
data: Buffer;
mediaType: string;
label: string;
};
export async function readClipboardImage(): Promise<ImageAttachment | null> {
const image = await readClipboardImageData();
if (!image) {
return null;
}
return {
id: `clipboard-${Date.now()}-${Math.random().toString(16).slice(2)}`,
label: image.label,
media_type: image.mediaType,
data: image.data.toString('base64'),
source_path: `clipboard:${image.label}`,
size_bytes: image.data.length,
};
}
async function readClipboardImageData(): Promise<ClipboardImageRead | null> {
if (process.platform === 'darwin') {
return readMacClipboardImage();
}
if (process.platform === 'win32') {
return readWindowsClipboardImage();
}
return readLinuxClipboardImage();
}
async function readMacClipboardImage(): Promise<ClipboardImageRead | null> {
const tempDir = await mkdtemp(join(tmpdir(), 'openharness-clipboard-'));
try {
const pngPath = join(tempDir, 'clipboard.png');
if (await runFileCommand('pngpaste', [pngPath])) {
return await readImageFile(pngPath, 'image/png', 'clipboard.png');
}
if (await writeMacClipboardClass('PNGf', pngPath)) {
return await readImageFile(pngPath, 'image/png', 'clipboard.png');
}
const tiffPath = join(tempDir, 'clipboard.tiff');
if (await writeMacClipboardClass('TIFF', tiffPath)) {
if (await runFileCommand('sips', ['-s', 'format', 'png', tiffPath, '--out', pngPath])) {
return await readImageFile(pngPath, 'image/png', 'clipboard.png');
}
return await readImageFile(tiffPath, 'image/tiff', 'clipboard.tiff');
}
return null;
} finally {
await rm(tempDir, {recursive: true, force: true});
}
}
async function writeMacClipboardClass(classCode: 'PNGf' | 'TIFF', outputPath: string): Promise<boolean> {
const appleClass = `${String.fromCharCode(0xab)}class ${classCode}${String.fromCharCode(0xbb)}`;
const escapedPath = outputPath.replace(/\\/g, '\\\\').replace(/"/g, '\\"');
return runFileCommand('osascript', [
'-e',
`set clipboardData to the clipboard as ${appleClass}`,
'-e',
`set outFile to open for access POSIX file "${escapedPath}" with write permission`,
'-e',
'set eof outFile to 0',
'-e',
'write clipboardData to outFile',
'-e',
'close access outFile',
]);
}
async function readWindowsClipboardImage(): Promise<ClipboardImageRead | null> {
const tempDir = await mkdtemp(join(tmpdir(), 'openharness-clipboard-'));
try {
const pngPath = join(tempDir, 'clipboard.png');
const escapedPath = pngPath.replace(/'/g, "''");
const powershell = join(
process.env.SystemRoot ?? 'C:\\Windows',
'System32',
'WindowsPowerShell',
'v1.0',
'powershell.exe',
);
const script = [
'Add-Type -AssemblyName System.Windows.Forms',
'Add-Type -AssemblyName System.Drawing',
'if (-not [Windows.Forms.Clipboard]::ContainsImage()) { exit 2 }',
'$image = [Windows.Forms.Clipboard]::GetImage()',
`$image.Save('${escapedPath}', [Drawing.Imaging.ImageFormat]::Png)`,
].join('; ');
if (!(await runFileCommand(powershell, ['-NoProfile', '-STA', '-Command', script]))) {
return null;
}
return await readImageFile(pngPath, 'image/png', 'clipboard.png');
} finally {
await rm(tempDir, {recursive: true, force: true});
}
}
async function readLinuxClipboardImage(): Promise<ClipboardImageRead | null> {
const attempts: Array<[string, string[], string, string]> = [
['wl-paste', ['--no-newline', '--type', 'image/png'], 'image/png', 'clipboard.png'],
['wl-paste', ['--no-newline', '--type', 'image/jpeg'], 'image/jpeg', 'clipboard.jpg'],
['xclip', ['-selection', 'clipboard', '-target', 'image/png', '-out'], 'image/png', 'clipboard.png'],
['xclip', ['-selection', 'clipboard', '-target', 'image/jpeg', '-out'], 'image/jpeg', 'clipboard.jpg'],
['xsel', ['--clipboard', '--output', '--mime-type', 'image/png'], 'image/png', 'clipboard.png'],
['xsel', ['--clipboard', '--output', '--mime-type', 'image/jpeg'], 'image/jpeg', 'clipboard.jpg'],
];
for (const [command, args, mediaType, label] of attempts) {
const data = await runBufferCommand(command, args);
if (data && data.length > 0) {
return {data, mediaType, label};
}
}
return null;
}
async function readImageFile(path: string, mediaType: string, label: string): Promise<ClipboardImageRead | null> {
const fileStat = await stat(path).catch(() => null);
if (!fileStat || fileStat.size <= 0 || fileStat.size > MAX_CLIPBOARD_IMAGE_BYTES) {
return null;
}
const data = await readFile(path);
return {data, mediaType, label};
}
async function runFileCommand(command: string, args: string[]): Promise<boolean> {
return new Promise((resolve) => {
execFile(command, args, {timeout: EXEC_TIMEOUT_MS, windowsHide: true}, (error) => {
resolve(!error);
});
});
}
async function runBufferCommand(command: string, args: string[]): Promise<Buffer | null> {
return new Promise((resolve) => {
execFile(
command,
args,
{
encoding: 'buffer',
maxBuffer: MAX_CLIPBOARD_IMAGE_BYTES + 1024,
timeout: EXEC_TIMEOUT_MS,
windowsHide: true,
},
(error, stdout) => {
if (error || !Buffer.isBuffer(stdout) || stdout.length > MAX_CLIPBOARD_IMAGE_BYTES) {
resolve(null);
return;
}
resolve(stdout);
},
);
});
}
@@ -0,0 +1,58 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import {filterCommandItems, normalizeCommandItems} from './commandPalette.js';
import type {CommandItemPayload} from './types.js';
const commands: CommandItemPayload[] = [
{
name: '/help',
description: 'Show available commands',
aliases: [],
},
{
name: '/workflows',
description: 'List, run, pause, resume, and save dynamic workflows',
aliases: ['/wf'],
},
{
name: '/permissions',
description: 'Show or update permission mode',
aliases: [],
},
];
test('normalizes structured command metadata and legacy command names', () => {
const items = normalizeCommandItems(
['help', '/status'],
[{name: 'help', description: 'Show help', aliases: ['h']}],
);
assert.deepEqual(items, [
{name: '/help', description: 'Show help', aliases: ['/h']},
{name: '/status', description: '', aliases: []},
]);
});
test('shows the palette for bare slash input', () => {
const items = filterCommandItems(commands, '/');
assert.deepEqual(items.map((item) => item.name), ['/help', '/workflows', '/permissions']);
});
test('matches aliases but returns the canonical command name', () => {
const items = filterCommandItems(commands, '/wf');
assert.equal(items[0].name, '/workflows');
assert.equal(items[0].matchedAlias, '/wf');
});
test('supports fuzzy command search', () => {
const items = filterCommandItems(commands, '/wrkflw');
assert.equal(items[0].name, '/workflows');
});
test('hides command palette after the command token has arguments', () => {
assert.deepEqual(filterCommandItems(commands, '/workflows run'), []);
});
+154
View File
@@ -0,0 +1,154 @@
import type {CommandItemPayload} from './types.js';
export type CommandPaletteItem = {
name: string;
description: string;
aliases: string[];
matchedAlias?: string;
score: number;
};
function withSlash(value: string): string {
const trimmed = value.trim();
if (!trimmed) {
return '';
}
return trimmed.startsWith('/') ? trimmed : `/${trimmed}`;
}
function commandToken(value: string): string {
return withSlash(value).slice(1).toLowerCase();
}
export function normalizeCommandItems(
commands: string[],
commandItems: CommandItemPayload[] | null | undefined,
): CommandItemPayload[] {
const byName = new Map<string, CommandItemPayload>();
for (const item of commandItems ?? []) {
const name = withSlash(String(item.name ?? ''));
if (!name) {
continue;
}
byName.set(name, {
name,
description: item.description ?? '',
aliases: (item.aliases ?? []).map((alias) => withSlash(String(alias))).filter(Boolean),
});
}
for (const command of commands) {
const name = withSlash(command);
if (!name || byName.has(name)) {
continue;
}
byName.set(name, {name, description: '', aliases: []});
}
return [...byName.values()];
}
export function filterCommandItems(
commandItems: CommandItemPayload[],
input: string,
limit = 12,
): CommandPaletteItem[] {
const trimmed = input.trimStart();
if (!trimmed.startsWith('/')) {
return [];
}
const rawQuery = trimmed.slice(1);
if (/\s/.test(rawQuery)) {
return [];
}
const query = rawQuery.toLowerCase();
const scored = commandItems
.map((item, index) => scoreItem(item, query, index))
.filter((item): item is CommandPaletteItem => item !== null)
.sort((a, b) => a.score - b.score || a.name.localeCompare(b.name));
return scored.slice(0, limit);
}
function scoreItem(item: CommandItemPayload, query: string, index: number): CommandPaletteItem | null {
const name = withSlash(item.name);
if (!name) {
return null;
}
const aliases = (item.aliases ?? []).map((alias) => withSlash(alias)).filter(Boolean);
const description = item.description ?? '';
if (!query) {
return {name, description, aliases, score: index};
}
const candidates = [name, ...aliases];
let bestScore = Number.POSITIVE_INFINITY;
let matchedAlias: string | undefined;
for (const candidate of candidates) {
const token = commandToken(candidate);
const score = scoreToken(token, query);
if (score < bestScore) {
bestScore = score;
matchedAlias = candidate === name ? undefined : candidate;
}
}
const descriptionIndex = description.toLowerCase().indexOf(query);
if (descriptionIndex >= 0) {
bestScore = Math.min(bestScore, 80 + descriptionIndex);
}
if (!Number.isFinite(bestScore)) {
return null;
}
return {
name,
description,
aliases,
matchedAlias,
score: bestScore + index / 1000,
};
}
function scoreToken(token: string, query: string): number {
if (token === query) {
return 0;
}
if (token.startsWith(query)) {
return 10 + token.length - query.length;
}
const segmentIndex = token
.split(/[-_]/)
.findIndex((segment) => segment.startsWith(query));
if (segmentIndex >= 0) {
return 25 + segmentIndex;
}
const includesIndex = token.indexOf(query);
if (includesIndex >= 0) {
return 35 + includesIndex;
}
const fuzzyScore = fuzzyMatchScore(token, query);
if (fuzzyScore !== null) {
return 55 + fuzzyScore;
}
return Number.POSITIVE_INFINITY;
}
function fuzzyMatchScore(token: string, query: string): number | null {
let tokenIndex = 0;
let score = 0;
for (const char of query) {
const foundAt = token.indexOf(char, tokenIndex);
if (foundAt < 0) {
return null;
}
score += foundAt - tokenIndex;
tokenIndex = foundAt + 1;
}
return score;
}
@@ -1,11 +1,13 @@
import React from 'react';
import {Box, Text} from 'ink';
import type {CommandPaletteItem} from '../commandPalette.js';
function CommandPickerInner({
hints,
selectedIndex,
}: {
hints: string[];
hints: CommandPaletteItem[];
selectedIndex: number;
}): React.JSX.Element | null {
if (hints.length === 0) {
@@ -18,18 +20,33 @@ function CommandPickerInner({
{hints.map((hint, i) => {
const isSelected = i === selectedIndex;
return (
<Box key={hint}>
<Text color={isSelected ? 'cyan' : undefined} bold={isSelected}>
{isSelected ? '\u276F ' : ' '}
{hint}
</Text>
{isSelected ? <Text dimColor> [enter]</Text> : null}
<Box key={hint.name} flexDirection="column">
<Box>
<Text color={isSelected ? 'cyan' : undefined} bold={isSelected}>
{isSelected ? '\u276F ' : ' '}
{hint.name}
</Text>
{hint.matchedAlias ? <Text dimColor> alias {hint.matchedAlias}</Text> : null}
{isSelected ? <Text dimColor> [enter]</Text> : null}
</Box>
{isSelected && hint.description ? (
<Box paddingLeft={2}>
<Text dimColor>{truncate(hint.description, 86)}</Text>
</Box>
) : null}
</Box>
);
})}
<Text dimColor> {'\u2191\u2193'} navigate{' '}{'\u23CE'} select{' '}esc dismiss</Text>
<Text dimColor> {'\u2191\u2193'} navigate{' '}tab complete{' '}{'\u23CE'} run{' '}esc dismiss</Text>
</Box>
);
}
export const CommandPicker = React.memo(CommandPickerInner);
function truncate(value: string, maxLength: number): string {
if (value.length <= maxLength) {
return value;
}
return `${value.slice(0, Math.max(0, maxLength - 1))}`;
}
@@ -0,0 +1,87 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import {PassThrough} from 'node:stream';
import React from 'react';
import {render} from 'ink';
import {ModalHost} from './ModalHost.js';
const stripAnsi = (value: string): string => value.replace(/\u001B\[[0-9;?]*[ -/]*[@-~]/g, '');
const nextLoopTurn = (): Promise<void> => new Promise((resolve) => setImmediate(resolve));
type InkTestStdout = PassThrough & {
isTTY: boolean;
columns: number;
rows: number;
cursorTo: () => boolean;
clearLine: () => boolean;
moveCursor: () => boolean;
};
function createTestStdout(): InkTestStdout {
return Object.assign(new PassThrough(), {
isTTY: true,
columns: 120,
rows: 40,
cursorTo: () => true,
clearLine: () => true,
moveCursor: () => true,
});
}
async function waitForOutputToStabilize(getOutput: () => string): Promise<string> {
let previous = '';
let sawOutput = false;
for (let i = 0; i < 50; i += 1) {
await nextLoopTurn();
const current = getOutput();
sawOutput ||= current.length > 0;
if (sawOutput && current === previous) {
return current;
}
previous = current;
}
throw new Error(`Ink output did not stabilize: ${JSON.stringify(previous)}`);
}
test('renders edit diff preview with stats and always shortcut', async () => {
const stdout = createTestStdout();
let output = '';
stdout.on('data', (chunk) => {
output += chunk.toString();
});
const instance = render(
<ModalHost
modal={{
kind: 'edit_diff',
path: 'src/demo.txt',
diff: '@@ -1 +1 @@\n-old line\n+new line',
added: 1,
removed: 1,
}}
modalInput=""
setModalInput={() => undefined}
onSubmit={() => undefined}
/>,
{stdout: stdout as unknown as NodeJS.WriteStream, debug: true, patchConsole: false},
);
const exitPromise = instance.waitUntilExit();
const stableOutput = await waitForOutputToStabilize(() => output);
instance.unmount();
await exitPromise;
instance.cleanup();
stdout.destroy();
const rendered = stripAnsi(stableOutput);
assert.match(rendered, /Edit src\/demo\.txt/);
assert.match(rendered, /\+1/);
assert.match(rendered, /-1/);
assert.match(rendered, /\+new line/);
assert.match(rendered, /-old line/);
assert.match(rendered, /\[a\] Always/);
});
@@ -8,6 +8,7 @@ const WAIT_FRAMES = [
'Agent is waiting for your input.. ',
'Agent is waiting for your input...',
];
const MAX_DIFF_LINES = 40;
function WaitingAnimation(): React.JSX.Element {
const [frame, setFrame] = useState(0);
@@ -85,6 +86,104 @@ function QuestionModal({
);
}
type DiffLineKind = 'add' | 'del' | 'hunk' | 'context';
type ParsedDiffLine = {
kind: DiffLineKind;
content: string;
};
function parseDiffLines(diffText: string): ParsedDiffLine[] {
return diffText
.split('\n')
.flatMap((raw): ParsedDiffLine[] => {
if (!raw || raw.startsWith('+++') || raw.startsWith('---')) {
return [];
}
if (raw.startsWith('@@')) {
return [{kind: 'hunk', content: raw}];
}
if (raw.startsWith('+')) {
return [{kind: 'add', content: raw.slice(1)}];
}
if (raw.startsWith('-')) {
return [{kind: 'del', content: raw.slice(1)}];
}
return [{kind: 'context', content: raw.startsWith(' ') ? raw.slice(1) : raw}];
});
}
function EditDiffModal({modal}: {modal: Record<string, unknown>}): React.JSX.Element {
const path = String(modal.path ?? '');
const added = Number(modal.added ?? 0);
const removed = Number(modal.removed ?? 0);
const lines = parseDiffLines(String(modal.diff ?? ''));
const visibleLines = lines.slice(0, MAX_DIFF_LINES);
const hiddenCount = lines.length - visibleLines.length;
return (
<Box flexDirection="column" marginTop={1}>
<Text>
<Text color="yellow" bold>{'\u250C '}</Text>
<Text bold>Edit </Text>
<Text color="cyan" bold>{path}</Text>
<Text>{' '}</Text>
<Text color="green">{`+${added}`}</Text>
<Text>{' '}</Text>
<Text color="red">{`-${removed}`}</Text>
</Text>
{visibleLines.map((line, index) => {
if (line.kind === 'hunk') {
return (
<Text key={index}>
<Text color="yellow">{'\u2502 '}</Text>
<Text color="cyan" dimColor>{line.content}</Text>
</Text>
);
}
if (line.kind === 'add') {
return (
<Text key={index}>
<Text color="yellow">{'\u2502 '}</Text>
<Text color="green">+</Text>
<Text color="green">{line.content}</Text>
</Text>
);
}
if (line.kind === 'del') {
return (
<Text key={index}>
<Text color="yellow">{'\u2502 '}</Text>
<Text color="red">-</Text>
<Text color="red">{line.content}</Text>
</Text>
);
}
return (
<Text key={index}>
<Text color="yellow">{'\u2502 '}</Text>
<Text dimColor>{' '}{line.content}</Text>
</Text>
);
})}
{hiddenCount > 0 ? (
<Text>
<Text color="yellow">{'\u2502 '}</Text>
<Text dimColor>... {hiddenCount} more lines hidden</Text>
</Text>
) : null}
<Text>
<Text color="yellow">{'\u2514 '}</Text>
<Text color="green">[y] Once</Text>
<Text>{' '}</Text>
<Text color="green">[a] Always</Text>
<Text>{' '}</Text>
<Text color="red">[n] Deny</Text>
</Text>
</Box>
);
}
function ModalHostInner({
modal,
modalInput,
@@ -120,6 +219,9 @@ function ModalHostInner({
</Box>
);
}
if (modal?.kind === 'edit_diff') {
return <EditDiffModal modal={modal} />;
}
if (modal?.kind === 'question') {
return (
<QuestionModal
@@ -173,6 +173,39 @@ test('keeps forward delete behavior when cursor is inside the line', async () =>
}
});
test('ignores ctrl+v so the app can attach clipboard images', async () => {
const stdin = createTestStdin();
const stdout = createTestStdout();
let currentValue = '';
const instance = render(<PromptHarness onInputChange={(value) => {
currentValue = value;
}} />, {
stdin: stdin as unknown as NodeJS.ReadStream & {fd: 0},
stdout: stdout as unknown as NodeJS.WriteStream,
debug: true,
patchConsole: false,
});
const exitPromise = instance.waitUntilExit();
try {
await nextLoopTurn();
await sendKey(stdin, Buffer.from([0x16]));
await nextLoopTurn();
assert.equal(currentValue, '');
await sendKey(stdin, 'v');
await waitForValue(() => currentValue, 'v');
} finally {
instance.unmount();
await exitPromise;
instance.cleanup();
stdin.destroy();
stdout.destroy();
}
});
test('accepts explicit /stop submission while busy', async () => {
const stdin = createTestStdin();
const stdout = createTestStdout();
@@ -78,7 +78,14 @@ function MultilineTextInput({
return;
}
if (key.upArrow || key.downArrow || key.tab || (key.shift && key.tab) || key.escape || (key.ctrl && input === 'c')) {
if (
key.upArrow ||
key.downArrow ||
key.tab ||
(key.shift && key.tab) ||
key.escape ||
(key.ctrl && (input === 'c' || input === 'v'))
) {
return;
}
@@ -197,6 +204,8 @@ export function PromptInput({
toolName,
suppressSubmit,
statusLabel,
imageAttachmentLabels = [],
clipboardStatus,
}: {
busy: boolean;
input: string;
@@ -205,6 +214,8 @@ export function PromptInput({
toolName?: string;
suppressSubmit?: boolean;
statusLabel?: string;
imageAttachmentLabels?: string[];
clipboardStatus?: string | null;
}): React.JSX.Element {
const {theme} = useTheme();
const promptPrefix = busy ? '… ' : '> ';
@@ -218,6 +229,18 @@ export function PromptInput({
</Box>
</Box>
) : null}
{imageAttachmentLabels.length > 0 ? (
<Box>
<Text color={theme.colors.accent}>
{imageAttachmentLabels.map((label, index) => `[image ${index + 1}: ${label}]`).join(' ')}
</Text>
</Box>
) : null}
{clipboardStatus ? (
<Box>
<Text color={theme.colors.muted}>{clipboardStatus}</Text>
</Box>
) : null}
<MultilineTextInput
value={input}
onChange={setInput}
@@ -5,6 +5,7 @@ import readline from 'node:readline';
import type {
BackendEvent,
BridgeSessionSnapshot,
CommandItemPayload,
FrontendConfig,
McpServerSnapshot,
SelectOptionPayload,
@@ -13,6 +14,7 @@ import type {
TaskSnapshot,
TranscriptItem,
} from '../types.js';
import {normalizeCommandItems} from '../commandPalette.js';
const PROTOCOL_PREFIX = 'OHJSON:';
const ASSISTANT_DELTA_FLUSH_MS = 50;
@@ -27,6 +29,7 @@ export function useBackendSession(config: FrontendConfig, onExit: (code?: number
const [status, setStatus] = useState<Record<string, unknown>>({});
const [tasks, setTasks] = useState<TaskSnapshot[]>([]);
const [commands, setCommands] = useState<string[]>([]);
const [commandItems, setCommandItems] = useState<CommandItemPayload[]>([]);
const [mcpServers, setMcpServers] = useState<McpServerSnapshot[]>([]);
const [bridgeSessions, setBridgeSessions] = useState<BridgeSessionSnapshot[]>([]);
const [modal, setModal] = useState<Record<string, unknown> | null>(null);
@@ -191,7 +194,9 @@ export function useBackendSession(config: FrontendConfig, onExit: (code?: number
startTransition(() => {
setTasks(event.tasks ?? []);
});
setCommands(event.commands ?? []);
const nextCommands = event.commands ?? [];
setCommands(nextCommands);
setCommandItems(normalizeCommandItems(nextCommands, event.command_items));
const mcpSnapshot = stableStringify(event.mcp_servers ?? []);
lastMcpSnapshotRef.current = mcpSnapshot;
startTransition(() => {
@@ -441,6 +446,7 @@ export function useBackendSession(config: FrontendConfig, onExit: (code?: number
status,
tasks,
commands,
commandItems,
mcpServers,
bridgeSessions,
modal,
@@ -457,6 +463,6 @@ export function useBackendSession(config: FrontendConfig, onExit: (code?: number
setBusyLabel,
sendRequest,
}),
[assistantBuffer, bridgeSessions, busy, busyLabel, commands, mcpServers, modal, ready, selectRequest, status, swarmNotifications, swarmTeammates, tasks, todoMarkdown, transcript]
[assistantBuffer, bridgeSessions, busy, busyLabel, commandItems, commands, mcpServers, modal, ready, selectRequest, status, swarmNotifications, swarmTeammates, tasks, todoMarkdown, transcript]
);
}
+13
View File
@@ -11,6 +11,12 @@ export type TranscriptItem = {
is_error?: boolean;
};
export type ImageAttachmentPayload = {
media_type: string;
data: string;
source_path?: string;
};
export type TaskSnapshot = {
id: string;
type: string;
@@ -73,6 +79,7 @@ export type BackendEvent = {
mcp_servers?: McpServerSnapshot[] | null;
bridge_sessions?: BridgeSessionSnapshot[] | null;
commands?: string[] | null;
command_items?: CommandItemPayload[] | null;
modal?: Record<string, unknown> | null;
select_options?: SelectOptionPayload[] | null;
tool_name?: string | null;
@@ -90,3 +97,9 @@ export type BackendEvent = {
swarm_teammates?: SwarmTeammateSnapshot[] | null;
swarm_notifications?: SwarmNotificationSnapshot[] | null;
};
export type CommandItemPayload = {
name: string;
description?: string | null;
aliases?: string[] | null;
};
+8
View File
@@ -269,6 +269,14 @@ def _prompt_channels(existing: GatewayConfig) -> tuple[list[str], dict[str, dict
default_value=str(prior.get("group_policy", "mention")),
)
elif channel == "feishu":
config["domain"] = _select_from_menu(
"Feishu domain:",
[
("https://open.feishu.cn", "Feishu (China)"),
("https://open.larksuite.com", "Lark (International)"),
],
default_value=str(prior.get("domain", "https://open.feishu.cn")),
)
config["app_id"] = _text_prompt(
"Feishu app id",
default=str(prior.get("app_id", "")),
+1
View File
@@ -276,6 +276,7 @@ class OhmoGatewayBridge:
channel=message.channel,
chat_id=message.chat_id,
content=update.text,
media=list(getattr(update, "media", None) or (update.metadata or {}).get("_media") or []),
metadata={**inbound_meta, **(update.metadata or {})},
)
)
+78
View File
@@ -0,0 +1,78 @@
"""Proactive notification helpers for ohmo gateway channels."""
from __future__ import annotations
import asyncio
import json
import logging
from pathlib import Path
from typing import Any
from ohmo.gateway.config import load_gateway_config
logger = logging.getLogger(__name__)
class OhmoNotificationError(RuntimeError):
"""Raised when a proactive notification cannot be delivered."""
def _chunk_text(text: str, *, max_chars: int = 1800) -> list[str]:
"""Split text into message-sized chunks without losing content."""
stripped = text.strip()
if not stripped:
return []
chunks: list[str] = []
remaining = stripped
while len(remaining) > max_chars:
split_at = remaining.rfind("\n", 0, max_chars)
if split_at < max_chars // 2:
split_at = max_chars
chunks.append(remaining[:split_at].strip())
remaining = remaining[split_at:].strip()
if remaining:
chunks.append(remaining)
return chunks
def _send_feishu_text_sync(*, user_open_id: str, content: str, workspace: str | Path | None = None) -> None:
"""Send a Feishu direct message using ohmo gateway Feishu credentials."""
try:
import lark_oapi as lark
from lark_oapi.api.im.v1 import CreateMessageRequest, CreateMessageRequestBody
except ImportError as exc: # pragma: no cover - depends on optional extra
raise OhmoNotificationError("Feishu SDK is not installed. Run: pip install lark-oapi") from exc
config = load_gateway_config(workspace)
feishu_config: dict[str, Any] = config.channel_configs.get("feishu", {})
app_id = str(feishu_config.get("app_id") or "").strip()
app_secret = str(feishu_config.get("app_secret") or "").strip()
if not app_id or not app_secret:
raise OhmoNotificationError("Feishu app_id/app_secret are not configured in ohmo gateway config.")
client = lark.Client.builder().app_id(app_id).app_secret(app_secret).log_level(lark.LogLevel.INFO).build()
for chunk in _chunk_text(content):
request = (
CreateMessageRequest.builder()
.receive_id_type("open_id")
.request_body(
CreateMessageRequestBody.builder()
.receive_id(user_open_id)
.msg_type("text")
.content(json.dumps({"text": chunk}, ensure_ascii=False))
.build()
)
.build()
)
response = client.im.v1.message.create(request)
if not response.success():
log_id = response.get_log_id() if hasattr(response, "get_log_id") else ""
raise OhmoNotificationError(
f"send Feishu DM failed: code={response.code}, msg={response.msg}, log_id={log_id}"
)
async def send_feishu_dm(*, user_open_id: str, content: str, workspace: str | Path | None = None) -> None:
"""Send a proactive Feishu direct message to a user open_id."""
await asyncio.to_thread(_send_feishu_text_sync, user_open_id=user_open_id, content=content, workspace=workspace)
logger.info("Sent proactive Feishu DM to open_id=%s", user_open_id)
+61 -1
View File
@@ -39,7 +39,7 @@ from ohmo.group_registry import load_managed_group_record, normalize_cwd
from ohmo.memory import create_memory_command_backend
from ohmo.prompts import build_ohmo_system_prompt
from ohmo.session_storage import OhmoSessionBackend
from ohmo.workspace import get_plugins_dir, get_skills_dir, initialize_workspace
from ohmo.workspace import get_memory_dir, get_plugins_dir, get_sessions_dir, get_skills_dir, initialize_workspace
logger = logging.getLogger(__name__)
@@ -86,6 +86,7 @@ class GatewayStreamUpdate:
kind: str
text: str
metadata: dict[str, object]
media: list[str] | None = None
class OhmoSessionRuntimePool:
@@ -194,6 +195,12 @@ class OhmoSessionRuntimePool:
extra_plugin_roots=(str(get_plugins_dir(self._workspace)),),
memory_backend=create_memory_command_backend(self._workspace),
include_project_memory=False,
autodream_context={
"memory_dir": str(get_memory_dir(self._workspace)),
"session_dir": str(get_sessions_dir(self._workspace)),
"app_label": "ohmo personal memory",
"runner_module": "ohmo",
},
)
if snapshot and snapshot.get("session_id"):
bundle.session_id = str(snapshot["session_id"])
@@ -575,6 +582,14 @@ class OhmoSessionRuntimePool:
bundle.session_id,
event.tool_name,
)
media = _extract_tool_media(event)
if media:
yield GatewayStreamUpdate(
kind="media",
text=_format_tool_media_caption(event, media),
metadata={"_session_key": session_key, "_media": media, "_tool_media": True},
media=media,
)
return
if isinstance(event, ErrorEvent):
logger.error(
@@ -643,6 +658,12 @@ class OhmoSessionRuntimePool:
extra_plugin_roots=(str(get_plugins_dir(self._workspace)),),
memory_backend=create_memory_command_backend(self._workspace),
include_project_memory=False,
autodream_context={
"memory_dir": str(get_memory_dir(self._workspace)),
"session_dir": str(get_sessions_dir(self._workspace)),
"app_label": "ohmo personal memory",
"runner_module": "ohmo",
},
)
refreshed.session_id = prior_session_id
self._register_gateway_tools(refreshed)
@@ -773,6 +794,45 @@ def _sanitize_snapshot_messages(raw_messages: object) -> list[dict[str, object]]
return [message.model_dump(mode="json") for message in _sanitize_group_command_prompts(messages)]
def _extract_tool_media(event: ToolExecutionCompleted) -> list[str]:
"""Return local media paths produced by a tool completion event."""
if event.is_error or not isinstance(event.metadata, dict):
return []
raw_paths = event.metadata.get("paths") or event.metadata.get("media")
if isinstance(raw_paths, str):
candidates = [raw_paths]
elif isinstance(raw_paths, list):
candidates = [str(item) for item in raw_paths if isinstance(item, str) and item.strip()]
else:
candidates = []
media: list[str] = []
seen: set[str] = set()
for raw in candidates:
path = Path(raw).expanduser()
if not path.is_absolute():
path = path.resolve()
if not path.is_file():
continue
resolved = str(path)
if resolved not in seen:
seen.add(resolved)
media.append(resolved)
return media
def _format_tool_media_caption(event: ToolExecutionCompleted, media: list[str]) -> str:
"""Return a short caption for media generated by tools."""
if event.tool_name == "image_generation":
provider = ""
if isinstance(event.metadata, dict):
provider = str(event.metadata.get("provider") or "").strip()
suffix = f" via {provider}" if provider else ""
names = ", ".join(Path(path).name for path in media)
return f"已生成图片{suffix}{names}"
names = ", ".join(Path(path).name for path in media)
return f"已生成文件:{names}"
def _sanitize_group_command_prompts(messages: list[ConversationMessage]) -> list[ConversationMessage]:
"""Replace internal /group tool-driving prompts with durable user-facing history."""
return [_sanitize_group_command_prompt(message) for message in messages]
+22 -1
View File
@@ -84,6 +84,13 @@ class OhmoGatewayService:
def state_file(self) -> Path:
return get_state_path(self._workspace)
def _channel_last_error(self) -> str | None:
for name, channel in self._manager.channels.items():
error = getattr(channel, "last_error", None)
if error:
return f"{name}: {error}"
return None
def write_state(self, *, running: bool, last_error: str | None = None) -> None:
state = GatewayState(
running=running,
@@ -91,7 +98,7 @@ class OhmoGatewayService:
active_sessions=self._runtime_pool.active_sessions,
provider_profile=self._config.provider_profile,
enabled_channels=self._config.enabled_channels,
last_error=last_error,
last_error=last_error or self._channel_last_error(),
)
self.state_file.write_text(state.model_dump_json(indent=2) + "\n", encoding="utf-8")
@@ -209,8 +216,18 @@ class OhmoGatewayService:
with contextlib.suppress(NotImplementedError):
loop.add_signal_handler(sig, _stop)
async def _state_heartbeat() -> None:
while not stop_event.is_set():
self.write_state(running=True)
await asyncio.sleep(5.0)
state_task = asyncio.create_task(_state_heartbeat(), name="ohmo-gateway-state")
try:
await stop_event.wait()
except Exception as exc:
self.write_state(running=False, last_error=str(exc))
raise
finally:
self._bridge.stop()
bridge_task.cancel()
@@ -219,6 +236,10 @@ class OhmoGatewayService:
await bridge_task
with contextlib.suppress(asyncio.CancelledError):
await manager_task
if not state_task.done():
state_task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await state_task
if not restart_notice_task.done():
restart_notice_task.cancel()
with contextlib.suppress(asyncio.CancelledError):
+151 -16
View File
@@ -6,6 +6,21 @@ from pathlib import Path
from re import sub
from openharness.commands import MemoryCommandBackend
from openharness.memory.scan import scan_memory_files
from openharness.memory.schema import (
SCHEMA_VERSION,
coerce_int,
compute_memory_signature,
first_content_line,
format_datetime,
generate_memory_id,
memory_metadata_from_path,
render_memory_file,
split_memory_file,
utc_now,
)
from openharness.utils.file_lock import exclusive_file_lock
from openharness.utils.fs import atomic_write_text
from ohmo.workspace import get_memory_dir, get_memory_index_path
@@ -13,7 +28,14 @@ from ohmo.workspace import get_memory_dir, get_memory_index_path
def list_memory_files(workspace: str | Path | None = None) -> list[Path]:
"""List ``.ohmo`` memory markdown files."""
memory_dir = get_memory_dir(workspace)
return sorted(path for path in memory_dir.glob("*.md") if path.name != "MEMORY.md")
return sorted(
header.path
for header in scan_memory_files(
_scan_cwd(workspace, memory_dir),
max_files=None,
memory_dir=memory_dir,
)
)
def add_memory_entry(workspace: str | Path | None, title: str, content: str) -> Path:
@@ -21,30 +43,113 @@ def add_memory_entry(workspace: str | Path | None, title: str, content: str) ->
memory_dir = get_memory_dir(workspace)
memory_dir.mkdir(parents=True, exist_ok=True)
slug = sub(r"[^a-zA-Z0-9]+", "_", title.strip().lower()).strip("_") or "memory"
path = memory_dir / f"{slug}.md"
path.write_text(content.strip() + "\n", encoding="utf-8")
with exclusive_file_lock(memory_dir / ".memory.lock"):
memory_type = "personal"
category = "preference"
body = content.strip() + "\n"
signature = compute_memory_signature(body, memory_type, category)
existing = scan_memory_files(
_scan_cwd(workspace, memory_dir),
max_files=None,
include_disabled=True,
include_expired=True,
memory_dir=memory_dir,
)
duplicate = next(
(header for header in existing if _effective_signature(header.path, header.signature) == signature),
None,
)
path = duplicate.path if duplicate is not None else _next_memory_path(memory_dir, slug)
now = utc_now()
now_text = format_datetime(now)
if path.exists():
metadata, old_body, _, _ = split_memory_file(path.read_text(encoding="utf-8"))
metadata = memory_metadata_from_path(
path,
metadata,
old_body,
now=now,
source=str(metadata.get("source") or "manual"),
default_type=memory_type,
default_category=category,
)
created_at = str(metadata.get("created_at") or now_text)
memory_id = str(metadata.get("id") or generate_memory_id(now))
else:
metadata = {}
created_at = now_text
memory_id = generate_memory_id(now)
metadata.update(
{
"schema_version": SCHEMA_VERSION,
"id": memory_id,
"name": title.strip(),
"description": first_content_line(body) or title.strip(),
"type": str(metadata.get("type") or memory_type),
"category": str(metadata.get("category") or category),
"importance": max(coerce_int(metadata.get("importance"), default=0), 1),
"source": "manual",
"signature": signature,
"created_at": created_at,
"updated_at": now_text,
"ttl_days": metadata.get("ttl_days"),
"disabled": False,
"supersedes": metadata.get("supersedes") or [],
}
)
atomic_write_text(path, render_memory_file(metadata, body))
index_path = get_memory_index_path(workspace)
existing = index_path.read_text(encoding="utf-8") if index_path.exists() else "# Memory Index\n"
if path.name not in existing:
existing = existing.rstrip() + f"\n- [{title}]({path.name})\n"
index_path.write_text(existing, encoding="utf-8")
index_path = get_memory_index_path(workspace)
existing_index = index_path.read_text(encoding="utf-8") if index_path.exists() else "# Memory Index\n"
if path.name not in existing_index:
existing_index = existing_index.rstrip() + f"\n- [{title}]({path.name})\n"
atomic_write_text(index_path, existing_index)
return path
def remove_memory_entry(workspace: str | Path | None, name: str) -> bool:
"""Delete a memory file and remove its index entry."""
"""Soft-delete a memory file and remove its index entry."""
memory_dir = get_memory_dir(workspace)
matches = [path for path in memory_dir.glob("*.md") if path.stem == name or path.name == name]
matches = [
header
for header in scan_memory_files(
_scan_cwd(workspace, memory_dir),
max_files=None,
include_disabled=True,
include_expired=True,
memory_dir=memory_dir,
)
if name in {header.path.stem, header.path.name, header.title, header.id}
]
if not matches:
return False
path = matches[0]
path.unlink(missing_ok=True)
header = matches[0]
if header.disabled:
return False
path = header.path
with exclusive_file_lock(memory_dir / ".memory.lock"):
content = path.read_text(encoding="utf-8")
metadata, body, _, _ = split_memory_file(content)
metadata = memory_metadata_from_path(
path,
metadata,
body,
source="manual",
default_type="personal",
default_category="preference",
)
metadata["disabled"] = True
metadata["updated_at"] = format_datetime(utc_now())
atomic_write_text(path, render_memory_file(metadata, body))
index_path = get_memory_index_path(workspace)
if index_path.exists():
lines = [line for line in index_path.read_text(encoding="utf-8").splitlines() if path.name not in line]
index_path.write_text("\n".join(lines).rstrip() + "\n", encoding="utf-8")
index_path = get_memory_index_path(workspace)
if index_path.exists():
lines = [
line
for line in index_path.read_text(encoding="utf-8").splitlines()
if path.name not in line
]
atomic_write_text(index_path, "\n".join(lines).rstrip() + "\n")
return True
@@ -76,9 +181,39 @@ def create_memory_command_backend(workspace: str | Path | None = None) -> Memory
return MemoryCommandBackend(
label="ohmo personal memory",
default_type="personal",
default_category="preference",
get_memory_dir=lambda: get_memory_dir(workspace),
get_entrypoint=lambda: get_memory_index_path(workspace),
list_files=lambda: list_memory_files(workspace),
add_entry=lambda title, content: add_memory_entry(workspace, title, content),
remove_entry=lambda name: remove_memory_entry(workspace, name),
)
def _scan_cwd(workspace: str | Path | None, memory_dir: Path) -> Path:
return Path(workspace) if workspace is not None else memory_dir.parent
def _next_memory_path(memory_dir: Path, slug: str) -> Path:
path = memory_dir / f"{slug}.md"
if not path.exists():
return path
index = 2
while True:
candidate = memory_dir / f"{slug}_{index}.md"
if not candidate.exists():
return candidate
index += 1
def _effective_signature(path: Path, existing_signature: str) -> str:
if existing_signature:
return existing_signature
try:
metadata, body, _, _ = split_memory_file(path.read_text(encoding="utf-8"))
except OSError:
return ""
memory_type = str(metadata.get("type") or "personal")
category = str(metadata.get("category") or "preference")
return compute_memory_signature(body, memory_type, category)
+18 -2
View File
@@ -17,7 +17,7 @@ from openharness.ui.react_launcher import _resolve_npm, _resolve_tsx, get_fronte
from ohmo.memory import create_memory_command_backend
from ohmo.prompts import build_ohmo_system_prompt
from ohmo.session_storage import OhmoSessionBackend
from ohmo.workspace import get_plugins_dir, get_skills_dir, initialize_workspace
from ohmo.workspace import get_memory_dir, get_plugins_dir, get_sessions_dir, get_skills_dir, initialize_workspace
def _ohmo_extra_roots(workspace: str | Path | None) -> tuple[tuple[str, ...], tuple[str, ...]]:
@@ -57,6 +57,12 @@ async def run_ohmo_backend(
extra_plugin_roots=extra_plugin_roots,
memory_backend=create_memory_command_backend(workspace_root),
include_project_memory=False,
autodream_context={
"memory_dir": str(get_memory_dir(workspace_root)),
"session_dir": str(get_sessions_dir(workspace_root)),
"app_label": "ohmo personal memory",
"runner_module": "ohmo",
},
)
@@ -165,13 +171,22 @@ async def run_ohmo_print_mode(
extra_plugin_roots=extra_plugin_roots,
memory_backend=create_memory_command_backend(workspace_root),
include_project_memory=False,
autodream_context={
"memory_dir": str(get_memory_dir(workspace_root)),
"session_dir": str(get_sessions_dir(workspace_root)),
"app_label": "ohmo personal memory",
"runner_module": "ohmo",
},
)
await start_runtime(bundle)
async def _print_system(message: str) -> None:
print(message, file=sys.stderr)
saw_error = False
async def _render_event(event) -> None:
nonlocal saw_error
if isinstance(event, AssistantTextDelta):
sys.stdout.write(event.text)
sys.stdout.flush()
@@ -179,6 +194,7 @@ async def run_ohmo_print_mode(
sys.stdout.write("\n")
sys.stdout.flush()
elif isinstance(event, ErrorEvent):
saw_error = True
print(event.message, file=sys.stderr)
elif isinstance(event, CompactProgressEvent):
if event.message:
@@ -197,6 +213,6 @@ async def run_ohmo_print_mode(
clear_output=_clear_output,
)
await close_runtime(bundle)
return 0
return 1 if saw_error else 0
finally:
os.chdir(previous_cwd)
+14
View File
@@ -0,0 +1,14 @@
"""Run the OpenHarness memory schema migration from a source checkout."""
from __future__ import annotations
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
from openharness.memory.migrate import main
if __name__ == "__main__":
raise SystemExit(main())
+5
View File
@@ -45,6 +45,7 @@ class ApiMessageRequest:
system_prompt: str | None = None
max_tokens: int = 4096
tools: list[dict[str, Any]] = field(default_factory=list)
effort: str | None = None
@dataclass(frozen=True)
@@ -149,6 +150,10 @@ class AnthropicApiClient:
kwargs["base_url"] = self._base_url
return AsyncAnthropic(**kwargs)
async def close(self) -> None:
"""Close the underlying HTTP client."""
await self._client.close()
def _refresh_client_auth(self) -> None:
if not self._claude_oauth or self._auth_token_resolver is None:
return
+12
View File
@@ -133,6 +133,15 @@ def _convert_tools_to_codex(tools: list[dict[str, Any]]) -> list[dict[str, Any]]
]
def _normalize_reasoning_effort(effort: str | None) -> str | None:
normalized = (effort or "").strip().lower()
if normalized == "max":
return "xhigh"
if normalized in {"low", "medium", "high", "xhigh"}:
return normalized
return None
def _usage_from_response(response: dict[str, Any]) -> UsageSnapshot:
usage = response.get("usage")
if not isinstance(usage, dict):
@@ -255,6 +264,9 @@ class CodexApiClient:
}
if request.tools:
body["tools"] = _convert_tools_to_codex(request.tools)
effort = _normalize_reasoning_effort(request.effort)
if effort:
body["reasoning"] = {"effort": effort}
content: list[TextBlock | ToolUseBlock] = []
current_text_parts: list[str] = []
+4
View File
@@ -128,3 +128,7 @@ class CopilotClient:
)
async for event in self._inner.stream_message(patched):
yield event
async def close(self) -> None:
"""Close the underlying OpenAI-compatible client."""
await self._inner.close()
+42 -6
View File
@@ -5,6 +5,7 @@ from __future__ import annotations
import asyncio
import json
import logging
import os
import re
from typing import Any, AsyncIterator
from urllib.parse import urlsplit, urlunsplit
@@ -145,13 +146,43 @@ def _convert_user_content_to_openai(blocks: list[ContentBlock]) -> str | list[di
return content
_EMPTY_REASONING_ENV = "OPENHARNESS_REQUIRE_EMPTY_REASONING_CONTENT"
def _empty_reasoning_required() -> bool:
"""True when the operator's provider requires an empty
``reasoning_content`` field on tool-using assistant messages
(Kimi-on-Anthropic style). Default off — strict-OpenAI providers
reject the field outright.
"""
raw = os.environ.get(_EMPTY_REASONING_ENV, "").strip().lower()
return raw in {"1", "true", "yes", "on"}
def _convert_assistant_message(msg: ConversationMessage) -> dict[str, Any]:
"""Convert an assistant ConversationMessage to OpenAI format.
Providers with thinking models (e.g. Kimi k2.5) require a
``reasoning_content`` field on every assistant message that contains
tool calls. We stash the raw reasoning text on ``msg._reasoning``
during parsing and replay it here.
``reasoning_content`` is a non-standard field used by thinking models
(e.g. Kimi k2.5) to carry the model's internal chain-of-thought across
turns. Some thinking-model providers require it on every assistant
message with tool calls — even when empty — or they reject the request.
Other OpenAI-compatible providers (Cerebras, OpenAI's own
endpoint, etc.) reject the field outright with a 400
``wrong_api_format`` error.
Behaviour:
- When the streaming parser captured non-empty reasoning on
``msg._reasoning``, we always replay it. Models that emit reasoning
tokens are by definition thinking models that round-trip them.
- When there is no captured reasoning but the message has tool calls,
we emit ``reasoning_content: ""`` only if the operator opts in via
``OPENHARNESS_REQUIRE_EMPTY_REASONING_CONTENT=1``. The default is
omit, which matches strict-OpenAI providers.
The opt-in default keeps strict-OpenAI providers (Cerebras, NVIDIA NIM,
OpenAI direct, etc.) working out-of-the-box; Kimi-on-Anthropic users
set the env var in their dotfiles or settings.
"""
text_parts = [b.text for b in msg.content if isinstance(b, TextBlock)]
tool_uses = [b for b in msg.content if isinstance(b, ToolUseBlock)]
@@ -165,8 +196,9 @@ def _convert_assistant_message(msg: ConversationMessage) -> dict[str, Any]:
reasoning = getattr(msg, "_reasoning", None)
if reasoning:
openai_msg["reasoning_content"] = reasoning
elif tool_uses:
# Thinking models require this field even if empty
elif tool_uses and _empty_reasoning_required():
# Kimi-style providers reject tool_use messages without this field
# even when there's nothing to put in it. Opt-in via env var.
openai_msg["reasoning_content"] = ""
if tool_uses:
@@ -244,6 +276,10 @@ class OpenAICompatibleClient:
kwargs["timeout"] = timeout
self._client = AsyncOpenAI(**kwargs)
async def close(self) -> None:
"""Close the underlying HTTP client."""
await self._client.close()
async def stream_message(self, request: ApiMessageRequest) -> AsyncIterator[ApiStreamEvent]:
"""Yield text deltas and the final message, matching the Anthropic client interface."""
last_error: Exception | None = None
+50 -9
View File
@@ -432,6 +432,26 @@ class FeishuChannel(BaseChannel):
self._sender_cache: OrderedDict[str, _FeishuSenderInfo] = OrderedDict()
self._loop: asyncio.AbstractEventLoop | None = None
def _ensure_rest_client(self) -> bool:
"""Initialize the Feishu REST client without starting the WebSocket receiver."""
if self._client is not None:
return True
if not FEISHU_AVAILABLE:
logger.error("Feishu SDK not installed. Run: pip install lark-oapi")
return False
if not self.config.app_id or not self.config.app_secret:
logger.error("Feishu app_id and app_secret not configured")
return False
import lark_oapi as lark
self._client = lark.Client.builder() \
.app_id(self.config.app_id) \
.app_secret(self.config.app_secret) \
.domain(self.config.domain) \
.log_level(lark.LogLevel.INFO) \
.build()
return True
async def start(self) -> None:
"""Start the Feishu bot with WebSocket long connection."""
if not FEISHU_AVAILABLE:
@@ -446,12 +466,8 @@ class FeishuChannel(BaseChannel):
self._running = True
self._loop = asyncio.get_running_loop()
# Create Lark client for sending messages
self._client = lark.Client.builder() \
.app_id(self.config.app_id) \
.app_secret(self.config.app_secret) \
.log_level(lark.LogLevel.INFO) \
.build()
if not self._ensure_rest_client():
return
# Create event handler (only register message receive, ignore other events)
event_handler = lark.EventDispatcherHandler.builder(
@@ -466,6 +482,7 @@ class FeishuChannel(BaseChannel):
self.config.app_id,
self.config.app_secret,
event_handler=event_handler,
domain=self.config.domain,
log_level=lark.LogLevel.INFO
)
@@ -1046,7 +1063,7 @@ class FeishuChannel(BaseChannel):
async def send(self, msg: OutboundMessage) -> None:
"""Send a message through Feishu, including media (images/files) if present."""
if not self._client:
if not self._ensure_rest_client():
logger.warning("Feishu client not initialized")
return
@@ -1060,19 +1077,28 @@ class FeishuChannel(BaseChannel):
else None
)
failed_media: list[str] = []
sent_media: list[str] = []
for file_path in msg.media:
if not os.path.isfile(file_path):
logger.warning("Media file not found: %s", file_path)
failed_media.append(file_path)
continue
ext = os.path.splitext(file_path)[1].lower()
if ext in self._IMAGE_EXTS:
key = await loop.run_in_executor(None, self._upload_image_sync, file_path)
if key:
await loop.run_in_executor(
ok = await loop.run_in_executor(
None, self._send_message_sync,
receive_id_type, msg.chat_id, "image", json.dumps({"image_key": key}, ensure_ascii=False),
reply_mid,
)
if ok:
sent_media.append(file_path)
else:
failed_media.append(file_path)
else:
failed_media.append(file_path)
else:
key = await loop.run_in_executor(None, self._upload_file_sync, file_path)
if key:
@@ -1082,11 +1108,26 @@ class FeishuChannel(BaseChannel):
media_type = "media"
else:
media_type = "file"
await loop.run_in_executor(
ok = await loop.run_in_executor(
None, self._send_message_sync,
receive_id_type, msg.chat_id, media_type, json.dumps({"file_key": key}, ensure_ascii=False),
reply_mid,
)
if ok:
sent_media.append(file_path)
else:
failed_media.append(file_path)
else:
failed_media.append(file_path)
if failed_media:
names = ", ".join(os.path.basename(path) for path in failed_media)
failure_body = json.dumps({"text": f"文件发送失败:{names}"}, ensure_ascii=False)
await loop.run_in_executor(
None, self._send_message_sync,
receive_id_type, msg.chat_id, "text", failure_body,
reply_mid,
)
if msg.content and msg.content.strip():
fmt = self._detect_msg_format(msg.content)
+2 -1
View File
@@ -165,7 +165,8 @@ class ChannelManager:
try:
await channel.start()
except Exception as e:
logger.error("Failed to start channel %s: %s", name, e)
setattr(channel, "last_error", str(e))
logger.exception("Failed to start channel %s", name)
async def start_all(self) -> None:
"""Start all channels and the outbound dispatcher."""
+7 -3
View File
@@ -179,8 +179,11 @@ class SlackChannel(BaseChannel):
except Exception as e:
logger.debug("Slack reactions_add failed: %s", e)
# Thread-scoped session key for channel/group messages
session_key = f"slack:{chat_id}:{thread_ts}" if thread_ts and channel_type != "im" else None
# Preserve Slack thread metadata for reply routing, but let the ohmo
# gateway router derive the session key. The router includes sender
# identity for shared chats; passing a senderless Slack override here
# would make different people in the same thread share one session.
chat_type = "p2p" if channel_type == "im" else "group"
try:
await self._handle_message(
@@ -188,13 +191,14 @@ class SlackChannel(BaseChannel):
chat_id=chat_id,
content=text,
metadata={
"thread_ts": thread_ts,
"chat_type": chat_type,
"slack": {
"event": event,
"thread_ts": thread_ts,
"channel_type": channel_type,
},
},
session_key=session_key,
)
except Exception:
logger.exception("Error handling Slack message from %s", sender_id)
+20 -3
View File
@@ -19,6 +19,14 @@ from openharness.utils.helpers import split_message
logger = logging.getLogger(__name__)
TELEGRAM_MAX_MESSAGE_LEN = 4000 # Telegram message character limit
_TELEGRAM_URL_LOGGERS = ("httpx", "httpcore", "telegram.ext")
def silence_telegram_token_url_loggers() -> None:
"""Prevent Telegram bot tokens from appearing in dependency INFO logs."""
for name in _TELEGRAM_URL_LOGGERS:
logging.getLogger(name).setLevel(logging.WARNING)
def _markdown_to_telegram_html(text: str) -> str:
@@ -111,6 +119,8 @@ class TelegramChannel(BaseChannel):
self.config: TelegramConfig = config
self.groq_api_key = groq_api_key
self._app: Application | None = None
self.last_error: str | None = None
self.polling_started = False
self._chat_ids: dict[str, int] = {} # Map sender_id to chat_id for replies
self._typing_tasks: dict[str, asyncio.Task] = {} # chat_id -> typing loop task
self._media_group_buffers: dict[str, dict] = {}
@@ -123,6 +133,9 @@ class TelegramChannel(BaseChannel):
return
self._running = True
self.last_error = None
self.polling_started = False
silence_telegram_token_url_loggers()
# Build the application with larger connection pool to avoid pool-timeout on long runs
req = HTTPXRequest(connection_pool_size=16, pool_timeout=5.0, connect_timeout=30.0, read_timeout=30.0)
@@ -165,8 +178,10 @@ class TelegramChannel(BaseChannel):
# Start polling (this runs until stopped)
await self._app.updater.start_polling(
allowed_updates=["message"],
drop_pending_updates=True # Ignore old messages on startup
drop_pending_updates=False,
)
self.polling_started = True
logger.info("Telegram polling started")
# Keep running until stopped
while self._running:
@@ -175,6 +190,7 @@ class TelegramChannel(BaseChannel):
async def stop(self) -> None:
"""Stop the Telegram bot."""
self._running = False
self.polling_started = False
# Cancel all typing indicators
for chat_id in list(self._typing_tasks):
@@ -301,7 +317,7 @@ class TelegramChannel(BaseChannel):
user = update.effective_user
await update.message.reply_text(
f"👋 Hi {user.first_name}! I'm nanobot.\n\n"
f"👋 Hi {user.first_name}! I'm {self.config.bot_name}.\n\n"
"Send me a message and I'll respond!\n"
"Type /help to see available commands."
)
@@ -311,7 +327,7 @@ class TelegramChannel(BaseChannel):
if not update.message:
return
await update.message.reply_text(
"🐈 nanobot commands:\n"
f"🐈 {self.config.bot_name} commands:\n"
"/new — Start a new conversation\n"
"/stop — Stop the current task\n"
"/help — Show available commands"
@@ -492,6 +508,7 @@ class TelegramChannel(BaseChannel):
async def _on_error(self, update: object, context: ContextTypes.DEFAULT_TYPE) -> None:
"""Log polling / handler errors instead of silently swallowing them."""
self.last_error = str(context.error)
logger.error("Telegram error: %s", context.error)
def _get_extension(self, media_type: str, mime_type: str | None) -> str:
+22 -3
View File
@@ -405,6 +405,7 @@ def _build_dry_run_preview(
api_key: str | None,
api_format: str | None,
permission_mode: str | None,
effort: str | None = None,
) -> dict[str, object]:
from openharness.api.provider import auth_status, detect_provider
from openharness.commands import create_default_command_registry
@@ -425,6 +426,7 @@ def _build_dry_run_preview(
api_key=api_key,
api_format=api_format,
permission_mode=permission_mode,
effort=effort,
)
provider = detect_provider(settings)
auth = auth_status(settings)
@@ -930,8 +932,20 @@ def cron_list_cmd() -> None:
last = last[:19] # trim to readable datetime
last_status = job.get("last_status", "")
status_indicator = f" [{last_status}]" if last_status else ""
print(f" [{enabled}] {job['name']} {job.get('schedule', '?')}")
print(f" cmd: {job['command']}")
timezone = f" ({job['timezone']})" if job.get("timezone") else ""
print(f" [{enabled}] {job['name']} {job.get('schedule', '?')}{timezone}")
print(f" cmd: {job.get('command') or '(agent_turn)'}")
payload = job.get("payload")
if isinstance(payload, dict):
print(
f" payload: {payload.get('kind', 'agent_turn')} -> "
f"{payload.get('channel', '?')}:{payload.get('to', '?')}"
)
notify = job.get("notify")
if isinstance(notify, dict):
notify_type = notify.get("type", "?")
target = notify.get("user_open_id") or notify.get("open_id") or notify.get("chat_id") or "?"
print(f" notify: {notify_type} -> {target}")
print(f" last: {last}{status_indicator} next: {job.get('next_run', 'n/a')[:19]}")
@@ -2139,7 +2153,7 @@ def main(
effort: str | None = typer.Option(
None,
"--effort",
help="Effort level for the session (low, medium, high, max)",
help="Effort level for the session (low, medium, high, xhigh/max, ultracode)",
rich_help_panel="Model & Effort",
),
verbose: bool = typer.Option(
@@ -2334,6 +2348,7 @@ def main(
api_key=api_key,
api_format=api_format,
permission_mode=permission_mode,
effort=effort,
)
effective_output_format = output_format or "text"
if effective_output_format == "text":
@@ -2407,6 +2422,7 @@ def main(
restore_tool_metadata=session_data.get("tool_metadata"),
permission_mode=permission_mode,
api_format=api_format,
effort=effort,
)
)
return
@@ -2429,6 +2445,7 @@ def main(
api_format=api_format,
permission_mode=permission_mode,
max_turns=max_turns,
effort=effort,
)
)
return
@@ -2444,6 +2461,7 @@ def main(
api_key=api_key,
api_format=api_format,
permission_mode=permission_mode,
effort=effort,
)
)
return
@@ -2460,5 +2478,6 @@ def main(
api_key=api_key,
api_format=api_format,
permission_mode=permission_mode,
effort=effort,
)
)
+516 -16
View File
@@ -4,6 +4,7 @@ from __future__ import annotations
import importlib.metadata
import json
import os
import re
import shutil
import subprocess
@@ -36,7 +37,29 @@ from openharness.memory import (
get_memory_entrypoint,
get_project_memory_dir,
list_memory_files,
migrate_memory,
remove_memory_entry,
scan_memory_files,
)
from openharness.memory.agent import (
ensure_agent_memory_vault,
get_agent_memory_entrypoint,
initialize_agent_memory_from_snapshot,
)
from openharness.memory.schema import (
DEFAULT_MEMORY_SCOPE,
DEFAULT_MEMORY_TYPE,
MEMORY_TYPES,
is_disabled_metadata,
is_memory_expired,
parse_memory_scope,
parse_memory_type,
split_memory_file,
)
from openharness.memory.team import (
check_team_memory_secrets,
ensure_team_memory_vault,
get_team_memory_dir,
)
from openharness.output_styles import load_output_styles
from openharness.permissions import PermissionChecker, PermissionMode
@@ -50,11 +73,27 @@ from openharness.services import (
estimate_conversation_tokens,
summarize_messages,
)
from openharness.services.autodream import (
diff_memory_dirs,
format_memory_diff,
latest_memory_backup,
read_last_consolidated_at,
restore_memory_backup,
start_dream_now,
)
from openharness.services.memory_extract import extract_memories_from_turn
from openharness.services.session_memory import (
get_session_memory_content,
get_session_memory_path,
update_session_memory_file,
)
from openharness.services.session_backend import DEFAULT_SESSION_BACKEND, SessionBackend
from openharness.skills import load_skill_registry
from openharness.skills.types import SkillDefinition
from openharness.tasks import get_task_manager
from openharness.plugins.types import PluginCommandDefinition
from openharness.tools.workflow_tool import format_workflow_run, list_saved_workflows_text
from openharness.workflows.manager import get_workflow_manager
if TYPE_CHECKING:
from openharness.config.settings import ProviderProfile
@@ -82,6 +121,8 @@ class MemoryCommandBackend:
"""Storage backend used by the generic ``/memory`` slash command."""
label: str
default_type: str
default_category: str
get_memory_dir: Callable[[], Path]
get_entrypoint: Callable[[], Path]
list_files: Callable[[], list[Path]]
@@ -543,6 +584,97 @@ def create_default_command_registry(
)
)
async def _dream_handler(args: str, context: CommandContext) -> CommandResult:
settings = getattr(context.engine, "_settings", None) or load_settings().materialize_active_profile()
parts = args.split()
action = parts[0] if parts else "run"
backend = context.memory_backend if context.memory_backend is not None else None
memory_dir = backend.get_memory_dir() if backend is not None else get_project_memory_dir(context.cwd)
session_dir = context.session_backend.get_session_dir(context.cwd)
app_label = backend.label if backend is not None else "openharness project memory"
runner_module = "ohmo" if backend is not None and "ohmo" in backend.label.lower() else "openharness"
if action == "status":
last_at = read_last_consolidated_at(context.cwd, memory_dir=memory_dir)
last = "never" if last_at <= 0 else datetime.fromtimestamp(last_at).isoformat(timespec="seconds")
return CommandResult(
message=(
f"Auto-dream: {'on' if settings.memory.auto_dream_enabled else 'off'}\n"
f"Memory store: {app_label}\n"
f"Memory directory: {memory_dir}\n"
f"Session directory: {session_dir}\n"
f"Last consolidated: {last}"
)
)
if action == "diff":
target = parts[1] if len(parts) > 1 else "latest"
task_memory_dir = str(memory_dir)
backup_dir = ""
label = target
if target == "latest":
latest = latest_memory_backup(memory_dir, app_label=app_label)
backup_dir = str(latest) if latest is not None else ""
label = "latest backup"
else:
task = get_task_manager().get_task(target)
if task is not None and task.type == "dream":
backup_dir = task.metadata.get("backup_dir", "")
task_memory_dir = task.metadata.get("memory_dir", str(memory_dir))
label = task.id
if not backup_dir:
return CommandResult(message=f"No dream backup found for {target}.")
diff = diff_memory_dirs(backup_dir, task_memory_dir)
return CommandResult(
message=(
f"Dream diff for {label}:\n"
f"Backup: {backup_dir}\n"
f"Memory: {task_memory_dir}\n"
f"{format_memory_diff(diff)}"
)
)
if action == "rollback":
target = parts[1] if len(parts) > 1 else "latest"
task_memory_dir = str(memory_dir)
backup_dir = ""
label = target
if target == "latest":
latest = latest_memory_backup(memory_dir, app_label=app_label)
backup_dir = str(latest) if latest is not None else ""
label = "latest backup"
else:
task = get_task_manager().get_task(target)
if task is not None and task.type == "dream":
backup_dir = task.metadata.get("backup_dir", "")
task_memory_dir = task.metadata.get("memory_dir", str(memory_dir))
label = task.id
if not backup_dir:
return CommandResult(message=f"No dream backup found for {target}.")
restore_memory_backup(backup_dir, task_memory_dir)
return CommandResult(message=f"Rolled back memory from backup for {label}: {backup_dir}")
if action not in {"run", "now", "preview"}:
return CommandResult(message="Usage: /dream [run|now|preview|status|diff [task_id]|rollback [task_id]]")
task = await start_dream_now(
cwd=context.cwd,
settings=settings,
model=context.engine.model,
current_session_id=context.session_id,
force=True,
memory_dir=memory_dir,
session_dir=session_dir,
app_label=app_label,
runner_module=runner_module,
preview=action == "preview",
)
if task is None:
return CommandResult(message="Dream did not start. Memory may be disabled or another dream is running.")
return CommandResult(
message=(
f"Started {'preview ' if action == 'preview' else ''}memory consolidation task {task.id}.\n"
f"Memory directory: {task.metadata.get('memory_dir', get_project_memory_dir(context.cwd))}\n"
f"Backup directory: {task.metadata.get('backup_dir', '') or '(preview/no backup)'}\n"
"Use /tasks or task_output to inspect progress. After completion: /dream diff latest or /dream rollback latest."
)
)
async def _memory_handler(args: str, context: CommandContext) -> CommandResult:
backend = _memory_backend_for_context(context)
tokens = args.split(maxsplit=1)
@@ -551,7 +683,10 @@ def create_default_command_registry(
message=(
f"Memory store: {backend.label}\n"
f"Memory directory: {backend.get_memory_dir()}\n"
f"Entrypoint: {backend.get_entrypoint()}"
f"Entrypoint: {backend.get_entrypoint()}\n"
"Commands: list, show NAME, add TITLE :: CONTENT, remove NAME, "
"edit [NAME], validate, extract, session, team, agent, "
"migrate --dry-run, migrate --apply"
)
)
action = tokens[0]
@@ -561,6 +696,37 @@ def create_default_command_registry(
if not memory_files:
return CommandResult(message="No memory files.")
return CommandResult(message="\n".join(path.name for path in memory_files))
if action == "migrate":
if rest not in {"--dry-run", "--apply"}:
return CommandResult(
message=(
"Usage: /memory "
"[list|show NAME|add TITLE :: CONTENT|remove NAME|"
"migrate --dry-run|migrate --apply]"
)
)
summary = migrate_memory(
context.cwd,
memory_dir=backend.get_memory_dir(),
default_type=backend.default_type,
default_category=backend.default_category,
apply=rest == "--apply",
)
mode = "dry run" if summary.dry_run else "applied"
lines = [
f"Memory migration {mode}.",
f"Scanned: {summary.scanned}",
f"Changed: {summary.changed}",
f"Unchanged: {summary.unchanged}",
f"Failed: {summary.failed}",
]
if summary.backup_dir:
lines.append(f"Backup: {summary.backup_dir}")
if summary.changed_files:
lines.append("Changed files: " + ", ".join(summary.changed_files))
if summary.failed_files:
lines.append("Failed files: " + ", ".join(summary.failed_files))
return CommandResult(message="\n".join(lines))
if action == "show" and rest:
memory_dir = backend.get_memory_dir()
path, invalid = _resolve_memory_entry_path(memory_dir, rest)
@@ -570,18 +736,64 @@ def create_default_command_registry(
return CommandResult(message=f"Memory entry not found: {rest}")
if not path.exists():
return CommandResult(message=f"Memory entry not found: {rest}")
return CommandResult(message=path.read_text(encoding="utf-8"))
content = path.read_text(encoding="utf-8")
metadata, _, _, _ = split_memory_file(content)
if is_disabled_metadata(metadata) or is_memory_expired(metadata):
return CommandResult(message=f"Memory entry not found: {rest}")
return CommandResult(message=content)
if action == "add" and rest:
title, separator, content = rest.partition("::")
memory_type, scope, cleaned_rest = _parse_memory_add_flags(rest)
title, separator, content = cleaned_rest.partition("::")
if not separator or not title.strip() or not content.strip():
return CommandResult(message="Usage: /memory add TITLE :: CONTENT")
path = backend.add_entry(title.strip(), content.strip())
return CommandResult(message="Usage: /memory add [--type TYPE] [--scope SCOPE] TITLE :: CONTENT")
if context.memory_backend is None:
path = add_memory_entry(
context.cwd,
title.strip(),
content.strip(),
memory_type=memory_type,
scope=scope,
)
else:
path = backend.add_entry(title.strip(), content.strip())
return CommandResult(message=f"Added memory entry {path.name}")
if action == "remove" and rest:
if backend.remove_entry(rest.strip()):
return CommandResult(message=f"Removed memory entry {rest.strip()}")
return CommandResult(message=f"Memory entry not found: {rest.strip()}")
return CommandResult(message="Usage: /memory [list|show NAME|add TITLE :: CONTENT|remove NAME]")
if action == "edit":
return _handle_memory_edit_command(rest, context, backend)
if action == "validate":
return _handle_memory_validate_command(context)
if action == "extract":
if context.memory_backend is not None:
return CommandResult(message="Memory extraction is only supported for OpenHarness project memory.")
result = await extract_memories_from_turn(
cwd=context.cwd,
api_client=context.engine.api_client,
model=context.engine.model,
messages=context.engine.messages,
max_records=load_settings().memory.auto_extract_max_records,
)
if result.skipped:
return CommandResult(message=f"Memory extraction skipped: {result.reason}")
return CommandResult(
message="Memory extraction wrote:\n" + "\n".join(f"- {path}" for path in result.written_paths)
)
if action == "session":
return _handle_memory_session_command(rest, context)
if action == "team":
return _handle_memory_team_command(rest, context)
if action == "agent":
return _handle_memory_agent_command(rest, context)
return CommandResult(
message=(
"Usage: /memory "
"[list|show NAME|add TITLE :: CONTENT|remove NAME|edit [NAME]|"
"validate|extract|session|team|agent|"
"migrate --dry-run|migrate --apply]"
)
)
async def _hooks_handler(_: str, context: CommandContext) -> CommandResult:
return CommandResult(message=context.hooks_summary or "No hooks configured.")
@@ -657,6 +869,7 @@ def create_default_command_registry(
latest = session_dir / "latest.json"
transcript = session_dir / "transcript.md"
lines = [
f"Session ID: {context.session_id or '(none)'}",
f"Session directory: {session_dir}",
f"Latest snapshot: {'present' if latest.exists() else 'missing'}",
f"Transcript export: {'present' if transcript.exists() else 'missing'}",
@@ -1024,10 +1237,13 @@ def create_default_command_registry(
value = args.strip() or "show"
if value == "show":
return CommandResult(message=f"Reasoning effort: {current}")
if value not in {"low", "medium", "high"}:
return CommandResult(message="Usage: /effort [show|low|medium|high]")
if value == "max":
value = "xhigh"
if value not in {"low", "medium", "high", "xhigh", "ultracode"}:
return CommandResult(message="Usage: /effort [show|low|medium|high|xhigh|ultracode]")
settings.effort = value
save_settings(settings)
context.engine.set_effort(value)
context.engine.set_system_prompt(
build_runtime_system_prompt(
settings,
@@ -1834,6 +2050,70 @@ def create_default_command_registry(
)
)
async def _workflows_handler(args: str, context: CommandContext) -> CommandResult:
manager = get_workflow_manager()
tokens = args.split(maxsplit=2)
if not tokens or tokens[0] in {"list", "runs"}:
runs = manager.list_runs(limit=20)
if not runs:
return CommandResult(message="No workflow runs.")
return CommandResult(
message="\n".join(
f"{run.id} {run.status} {run.name} agents={run.agent_count} cached={run.cached_count}"
for run in runs
)
)
if tokens[0] in {"saved", "list-saved"}:
return CommandResult(message=list_saved_workflows_text(context.cwd))
if tokens[0] == "run" and len(tokens) >= 2:
name_or_path = args[len("run ") :].strip()
try:
run = await manager.start_saved(
cwd=context.cwd,
name_or_path=name_or_path,
model=context.engine.model,
)
except ValueError as exc:
return CommandResult(message=str(exc))
return CommandResult(message=f"Started workflow {run.id}: {run.name}")
if tokens[0] in {"show", "output"} and len(tokens) >= 2:
run = manager.get(tokens[1])
if run is None:
return CommandResult(message=f"Workflow run not found: {tokens[1]}")
return CommandResult(message=format_workflow_run(run, include_output=tokens[0] == "output"))
if tokens[0] == "pause" and len(tokens) >= 2:
try:
run = await manager.pause(tokens[1])
except ValueError as exc:
return CommandResult(message=str(exc))
return CommandResult(message=f"Paused workflow {run.id}: {run.status}")
if tokens[0] == "stop" and len(tokens) >= 2:
try:
run = await manager.stop(tokens[1])
except ValueError as exc:
return CommandResult(message=str(exc))
return CommandResult(message=f"Stopped workflow {run.id}: {run.status}")
if tokens[0] == "resume" and len(tokens) >= 2:
try:
run = await manager.resume(run_id=tokens[1], model=context.engine.model)
except ValueError as exc:
return CommandResult(message=str(exc))
return CommandResult(message=f"Resumed workflow {run.id}: {run.status}")
if tokens[0] == "save" and len(tokens) >= 3:
run_id = tokens[1]
name = tokens[2]
try:
path = manager.save_run_script(cwd=context.cwd, run_id=run_id, name=name)
except ValueError as exc:
return CommandResult(message=str(exc))
return CommandResult(message=f"Saved workflow {run_id} as {path}")
return CommandResult(
message=(
"Usage: /workflows "
"[list|saved|run NAME_OR_PATH|show ID|output ID|pause ID|resume ID|stop ID|save ID NAME]"
)
)
async def _autopilot_handler(args: str, context: CommandContext) -> CommandResult:
store = RepoAutopilotStore(context.cwd)
tokens = args.split()
@@ -2112,14 +2392,31 @@ def create_default_command_registry(
registry.register(SlashCommand("version", "Show the installed OpenHarness version", _version_handler))
registry.register(SlashCommand("status", "Show session status", _status_handler))
registry.register(SlashCommand("context", "Show the active runtime system prompt", _context_handler))
registry.register(SlashCommand("summary", "Summarize conversation history", _summary_handler))
registry.register(
SlashCommand(
"summary",
"Summarize conversation history",
_summary_handler,
remote_invocable=False,
remote_admin_opt_in=True,
)
)
registry.register(SlashCommand("compact", "Compact older conversation history", _compact_handler))
registry.register(SlashCommand("cost", "Show token usage and estimated cost", _cost_handler))
registry.register(SlashCommand("usage", "Show usage and token estimates", _usage_handler))
registry.register(SlashCommand("stats", "Show session statistics", _stats_handler))
registry.register(SlashCommand("dream", "Consolidate memory", _dream_handler))
registry.register(SlashCommand("memory", "Inspect and manage project memory", _memory_handler))
registry.register(SlashCommand("hooks", "Show configured hooks", _hooks_handler))
registry.register(SlashCommand("resume", "Restore the latest saved session", _resume_handler))
registry.register(
SlashCommand(
"resume",
"Restore the latest saved session",
_resume_handler,
remote_invocable=False,
remote_admin_opt_in=True,
)
)
registry.register(SlashCommand("session", "Inspect the current session storage", _session_handler))
registry.register(SlashCommand("export", "Export the current transcript", _export_handler))
registry.register(SlashCommand("share", "Create a shareable transcript snapshot", _share_handler))
@@ -2219,6 +2516,7 @@ def create_default_command_registry(
registry.register(SlashCommand("turns", "Show or update maximum agentic turn count", _turns_handler))
registry.register(SlashCommand("continue", "Continue the previous tool loop if it was interrupted", _continue_handler))
registry.register(SlashCommand("stop", "Interrupt the running turn from TUI/ohmo channels", _stop_handler))
registry.register(SlashCommand("workflows", "List, run, pause, resume, and save dynamic workflows", _workflows_handler))
registry.register(
SlashCommand(
"provider",
@@ -2243,19 +2541,67 @@ def create_default_command_registry(
registry.register(SlashCommand("vim", "Show or update Vim mode", _vim_handler))
registry.register(SlashCommand("voice", "Show or update voice mode", _voice_handler))
registry.register(SlashCommand("doctor", "Show environment diagnostics", _doctor_handler))
registry.register(SlashCommand("diff", "Show git diff output", _diff_handler))
registry.register(
SlashCommand(
"diff",
"Show git diff output",
_diff_handler,
remote_invocable=False,
remote_admin_opt_in=True,
)
)
registry.register(SlashCommand("branch", "Show git branch information", _branch_handler))
registry.register(SlashCommand("commit", "Show status or create a git commit", _commit_handler))
registry.register(SlashCommand("issue", "Show or update project issue context", _issue_handler))
registry.register(SlashCommand("pr_comments", "Show or update project PR comments context", _pr_comments_handler))
registry.register(
SlashCommand(
"commit",
"Show status or create a git commit",
_commit_handler,
remote_invocable=False,
remote_admin_opt_in=True,
)
)
registry.register(
SlashCommand(
"issue",
"Show or update project issue context",
_issue_handler,
remote_invocable=False,
remote_admin_opt_in=True,
)
)
registry.register(
SlashCommand(
"pr_comments",
"Show or update project PR comments context",
_pr_comments_handler,
remote_invocable=False,
remote_admin_opt_in=True,
)
)
registry.register(SlashCommand("privacy-settings", "Show local privacy and storage settings", _privacy_settings_handler))
registry.register(SlashCommand("rate-limit-options", "Show ways to reduce provider rate pressure", _rate_limit_options_handler))
registry.register(SlashCommand("release-notes", "Show recent OpenHarness release notes", _release_notes_handler))
registry.register(SlashCommand("upgrade", "Show upgrade instructions", _upgrade_handler))
registry.register(SlashCommand("agents", "List or inspect agent and teammate tasks", _agents_handler))
registry.register(SlashCommand("subagents", "Show subagent usage and inspect worker tasks", _agents_handler))
registry.register(SlashCommand("tasks", "Manage background tasks", _tasks_handler))
registry.register(SlashCommand("autopilot", "Manage repo autopilot intake and context", _autopilot_handler))
registry.register(
SlashCommand(
"tasks",
"Manage background tasks",
_tasks_handler,
remote_invocable=False,
remote_admin_opt_in=True,
)
)
registry.register(
SlashCommand(
"autopilot",
"Manage repo autopilot intake and context",
_autopilot_handler,
remote_invocable=False,
remote_admin_opt_in=True,
)
)
registry.register(
SlashCommand(
"ship",
@@ -2298,6 +2644,158 @@ def create_default_command_registry(
return registry
def _handle_memory_edit_command(
args: str,
context: CommandContext,
backend: MemoryCommandBackend,
) -> CommandResult:
memory_dir = backend.get_memory_dir()
target = backend.get_entrypoint()
if args.strip():
path, invalid = _resolve_memory_entry_path(memory_dir, args.strip())
if invalid:
return CommandResult(message="Memory entry path must stay within the configured memory directory.")
if path is None:
return CommandResult(message=f"Memory entry not found: {args.strip()}")
target = path
target.parent.mkdir(parents=True, exist_ok=True)
target.touch(exist_ok=True)
editor = os.environ.get("VISUAL") or os.environ.get("EDITOR")
if not editor:
return CommandResult(message=f"Memory file ready: {target}\nSet $VISUAL or $EDITOR to open it from /memory edit.")
result = subprocess.run([editor, str(target)], cwd=context.cwd, check=False)
if result.returncode != 0:
return CommandResult(message=f"Editor exited with status {result.returncode}: {editor}")
return CommandResult(message=f"Edited memory file: {target}")
def _parse_memory_add_flags(args: str):
"""Parse optional ``/memory add`` type/scope flags."""
memory_type = DEFAULT_MEMORY_TYPE
scope = DEFAULT_MEMORY_SCOPE
rest = args.strip()
changed = True
while changed:
changed = False
if rest.startswith("--type "):
_, _, tail = rest.partition(" ")
raw, _, rest = tail.partition(" ")
parsed = parse_memory_type(raw, default=DEFAULT_MEMORY_TYPE)
if parsed is not None:
memory_type = parsed
changed = True
if rest.startswith("--scope "):
_, _, tail = rest.partition(" ")
raw, _, rest = tail.partition(" ")
parsed_scope = parse_memory_scope(raw, default=DEFAULT_MEMORY_SCOPE)
if parsed_scope is not None:
scope = parsed_scope
changed = True
return memory_type, scope, rest
def _handle_memory_validate_command(context: CommandContext) -> CommandResult:
memory_dir = get_project_memory_dir(context.cwd)
headers = scan_memory_files(context.cwd, max_files=500)
issues: list[str] = []
for header in headers:
raw_type = header.frontmatter.get("type") or header.frontmatter.get("memory_type")
if parse_memory_type(raw_type) is None:
issues.append(
f"- {header.relative_path}: invalid or missing type {raw_type!r}; expected {', '.join(MEMORY_TYPES)}"
)
if "team" in Path(header.relative_path).parts:
try:
text = header.path.read_text(encoding="utf-8", errors="replace")
except OSError:
text = ""
secret_error = check_team_memory_secrets(text)
if secret_error:
issues.append(f"- {header.relative_path}: {secret_error}")
if not issues:
return CommandResult(
message=(
"Memory validation passed.\n"
f"- files: {len(headers)}\n"
f"- memory_dir: {memory_dir}"
)
)
return CommandResult(message="Memory validation issues:\n" + "\n".join(issues))
def _handle_memory_session_command(args: str, context: CommandContext) -> CommandResult:
action = args.split(maxsplit=1)[0] if args.strip() else "status"
path = get_session_memory_path(context.cwd, context.session_id or "default")
if action == "update":
path = update_session_memory_file(
context.cwd,
context.engine.messages,
tool_metadata=context.engine.tool_metadata,
session_id=context.session_id or "default",
)
return CommandResult(message=f"Updated session memory: {path}")
if action == "show":
content = get_session_memory_content(path)
return CommandResult(message=content or f"No session memory at {path}")
return CommandResult(
message=(
"Session memory:\n"
f"- path: {path}\n"
f"- exists: {path.exists()}\n"
"Commands: /memory session [status|show|update]"
)
)
def _handle_memory_team_command(args: str, context: CommandContext) -> CommandResult:
action = args.split(maxsplit=1)[0] if args.strip() else "status"
team_dir = ensure_team_memory_vault(context.cwd)
if action == "list":
files = sorted(path for path in team_dir.rglob("*.md") if path.name != "MEMORY.md")
return CommandResult(message="\n".join(str(path.relative_to(team_dir)) for path in files) or "No team memory files.")
if action == "validate":
issues: list[str] = []
for path in sorted(team_dir.rglob("*.md")):
if path.name == "MEMORY.md":
continue
text = path.read_text(encoding="utf-8", errors="replace")
secret_error = check_team_memory_secrets(text)
if secret_error:
issues.append(f"- {path.relative_to(team_dir)}: {secret_error}")
return CommandResult(message="Team memory validation passed." if not issues else "\n".join(issues))
return CommandResult(
message=(
"Team memory:\n"
f"- directory: {get_team_memory_dir(context.cwd)}\n"
f"- exists: {team_dir.exists()}\n"
"Commands: /memory team [status|list|validate]"
)
)
def _handle_memory_agent_command(args: str, context: CommandContext) -> CommandResult:
parts = args.split()
action = parts[0] if parts else "status"
agent_type = parts[1] if len(parts) > 1 else "default"
scope = parts[2] if len(parts) > 2 else "project"
if scope not in {"user", "project", "local"}:
return CommandResult(message="Agent memory scope must be one of: user, project, local")
if action == "snapshot":
target = initialize_agent_memory_from_snapshot(context.cwd, agent_type, scope) # type: ignore[arg-type]
return CommandResult(message=f"Initialized agent memory from snapshot: {target}" if target else "No snapshot found.")
vault = ensure_agent_memory_vault(context.cwd, agent_type, scope) # type: ignore[arg-type]
return CommandResult(
message=(
"Agent memory:\n"
f"- agent_type: {agent_type}\n"
f"- scope: {scope}\n"
f"- directory: {vault}\n"
f"- entrypoint: {get_agent_memory_entrypoint(context.cwd, agent_type, scope)}"
)
)
def _resolve_memory_entry_path(memory_dir: Path, candidate: str) -> tuple[Path | None, bool]:
"""Resolve a memory entry path while enforcing containment under ``memory_dir``."""
@@ -2330,6 +2828,8 @@ def _memory_backend_for_context(context: CommandContext) -> MemoryCommandBackend
cwd = context.cwd
return MemoryCommandBackend(
label="OpenHarness project memory",
default_type="project",
default_category="knowledge",
get_memory_dir=lambda: get_project_memory_dir(cwd),
get_entrypoint=lambda: get_memory_entrypoint(cwd),
list_files=lambda: list_memory_files(cwd),
+21
View File
@@ -82,6 +82,27 @@ def get_tasks_dir() -> Path:
return tasks_dir
def get_workflows_dir() -> Path:
"""Return the workflow runtime storage directory."""
workflows_dir = get_data_dir() / "workflows"
workflows_dir.mkdir(parents=True, exist_ok=True)
return workflows_dir
def get_user_workflows_dir() -> Path:
"""Return the user-level saved workflow directory."""
workflows_dir = get_config_dir() / "workflows"
workflows_dir.mkdir(parents=True, exist_ok=True)
return workflows_dir
def get_project_workflows_dir(cwd: str | Path) -> Path:
"""Return the project-level saved workflow directory."""
workflows_dir = get_project_config_dir(cwd) / "workflows"
workflows_dir.mkdir(parents=True, exist_ok=True)
return workflows_dir
def get_feedback_dir() -> Path:
"""Return the feedback storage directory."""
feedback_dir = get_data_dir() / "feedback"
+3
View File
@@ -35,6 +35,8 @@ class TelegramConfig(BaseChannelConfig):
token: str = ""
chat_id: str | None = None
proxy: str | None = None
reply_to_message: bool = True
bot_name: str = "ohmo"
class SlackConfig(BaseChannelConfig):
@@ -57,6 +59,7 @@ class FeishuConfig(BaseChannelConfig):
group_policy: str = "managed_or_mention"
bot_open_id: str = ""
bot_names: list[str] = Field(default_factory=lambda: ["ohmo", "openclaw", "openharness"])
domain: str = "https://open.feishu.cn" # use https://open.larksuite.com for Lark international
class DingTalkConfig(BaseChannelConfig):
+132 -39
View File
@@ -63,8 +63,15 @@ class MemorySettings(BaseModel):
enabled: bool = True
max_files: int = 5
max_entrypoint_lines: int = 200
max_entrypoint_bytes: int = 25_000
context_window_tokens: int | None = None
auto_compact_threshold_tokens: int | None = None
auto_extract_enabled: bool = False
auto_extract_max_records: int = 3
session_memory_enabled: bool = True
auto_dream_enabled: bool = False
auto_dream_min_hours: float = 24.0
auto_dream_min_sessions: int = 5
class SandboxNetworkSettings(BaseModel):
@@ -363,6 +370,30 @@ def auth_source_uses_api_key(auth_source: str) -> bool:
return auth_source.endswith("_api_key")
def auth_source_env_var_candidates(auth_source: str) -> tuple[str, ...]:
"""Return env vars to probe for an auth source in precedence order."""
mapping = {
"anthropic_api_key": ("OPENHARNESS_ANTHROPIC_API_KEY", "ANTHROPIC_API_KEY"),
"openai_api_key": ("OPENHARNESS_OPENAI_API_KEY", "OPENAI_API_KEY"),
"dashscope_api_key": ("OPENHARNESS_DASHSCOPE_API_KEY", "DASHSCOPE_API_KEY"),
"moonshot_api_key": ("OPENHARNESS_MOONSHOT_API_KEY", "MOONSHOT_API_KEY"),
"gemini_api_key": ("OPENHARNESS_GEMINI_API_KEY", "GEMINI_API_KEY"),
"minimax_api_key": ("OPENHARNESS_MINIMAX_API_KEY", "MINIMAX_API_KEY"),
"nvidia_api_key": ("OPENHARNESS_NVIDIA_API_KEY", "NVIDIA_API_KEY"),
"modelscope_api_key": ("OPENHARNESS_MODELSCOPE_API_KEY", "MODELSCOPE_API_KEY"),
}
return mapping.get(auth_source, ())
def resolve_auth_env_value(auth_source: str) -> tuple[str, str] | None:
"""Return the first configured env var/value pair for an auth source."""
for env_var in auth_source_env_var_candidates(auth_source):
env_value = os.environ.get(env_var, "")
if env_value:
return env_var, env_value
return None
def credential_storage_provider_name(profile_name: str, profile: ProviderProfile) -> str:
"""Return the storage namespace used for this profile's credential.
@@ -467,6 +498,37 @@ def _profile_from_flat_settings(settings: "Settings") -> tuple[str, ProviderProf
return name, profile
class ImageGenerationConfig(BaseModel):
"""Configuration for the image_generation tool."""
provider: str = "auto"
model: str = "gpt-image-2"
api_key: str = ""
base_url: str = ""
codex_model: str = "gpt-5.4"
codex_base_url: str = ""
@classmethod
def from_env(cls) -> "ImageGenerationConfig":
"""Load image generation config from environment variables."""
return cls(
provider=os.environ.get("OPENHARNESS_IMAGE_GENERATION_PROVIDER", "auto").strip()
or "auto",
model=os.environ.get("OPENHARNESS_IMAGE_GENERATION_MODEL", "gpt-image-2").strip()
or "gpt-image-2",
api_key=os.environ.get("OPENHARNESS_IMAGE_GENERATION_API_KEY", "").strip(),
base_url=os.environ.get("OPENHARNESS_IMAGE_GENERATION_BASE_URL", "").strip(),
codex_model=os.environ.get("OPENHARNESS_IMAGE_GENERATION_CODEX_MODEL", "gpt-5.4").strip()
or "gpt-5.4",
codex_base_url=os.environ.get("OPENHARNESS_IMAGE_GENERATION_CODEX_BASE_URL", "").strip(),
)
@property
def is_configured(self) -> bool:
"""Return True when either a key provider or Codex provider is selected."""
return bool(self.api_key or self.provider in {"auto", "codex"})
class VisionModelConfig(BaseModel):
"""Configuration for the vision model used by the image_to_text tool.
@@ -537,6 +599,9 @@ class Settings(BaseModel):
# Vision model (image-to-text fallback)
vision: VisionModelConfig = Field(default_factory=VisionModelConfig)
# Image generation model
image_generation: ImageGenerationConfig = Field(default_factory=ImageGenerationConfig)
def merged_profiles(self) -> dict[str, ProviderProfile]:
"""Return the saved profiles merged over the built-in catalog."""
merged = default_provider_profiles()
@@ -555,7 +620,7 @@ class Settings(BaseModel):
def resolve_profile(self, name: str | None = None) -> tuple[str, ProviderProfile]:
"""Return the active provider profile."""
profiles = self.merged_profiles()
profile_name = (name or self.active_profile or "").strip() or "claude-api"
profile_name = (name or self.active_profile or os.environ.get("OPENHARNESS_PROFILE") or "").strip() or "claude-api"
if profile_name not in profiles:
fallback_name, fallback = _profile_from_flat_settings(self)
profiles[fallback_name] = fallback
@@ -592,9 +657,15 @@ class Settings(BaseModel):
directly before the profile layer is used everywhere.
"""
profile_name, profile = self.resolve_profile()
next_provider = (self.provider or "").strip() or profile.provider
next_api_format = (self.api_format or "").strip() or profile.api_format
next_base_url = self.base_url if self.base_url is not None else profile.base_url
profile_from_env = bool(os.environ.get("OPENHARNESS_PROFILE"))
flat_profile_fields_match_profile = profile_from_env or (
(self.provider or "").strip() == profile.provider
and (self.api_format or "").strip() == profile.api_format
and self.base_url == profile.base_url
)
next_provider = profile.provider if flat_profile_fields_match_profile else (self.provider or "").strip() or profile.provider
next_api_format = profile.api_format if flat_profile_fields_match_profile else (self.api_format or "").strip() or profile.api_format
next_base_url = profile.base_url if flat_profile_fields_match_profile else (self.base_url if self.base_url is not None else profile.base_url)
next_context_window_tokens = (
self.context_window_tokens
if self.context_window_tokens is not None
@@ -665,19 +736,15 @@ class Settings(BaseModel):
if self.api_key:
return self.api_key
env_key = os.environ.get("ANTHROPIC_API_KEY", "")
if env_key:
return env_key
# Also check OPENAI_API_KEY for openai-format providers
openai_key = os.environ.get("OPENAI_API_KEY", "")
if openai_key:
return openai_key
env_resolved = resolve_auth_env_value(profile.auth_source)
if env_resolved:
_, env_value = env_resolved
return env_value
raise ValueError(
"No API key found. Set ANTHROPIC_API_KEY (or OPENAI_API_KEY for openai-format "
"providers) environment variable, or configure api_key in "
"~/.openharness/settings.json"
"No API key found. Set an OPENHARNESS_* provider API key "
"(preferred) or the matching native provider environment variable, "
"or configure api_key in ~/.openharness/settings.json"
)
def resolve_auth(self) -> ResolvedAuth:
@@ -686,6 +753,15 @@ class Settings(BaseModel):
provider = profile.provider.strip()
auth_source = profile.auth_source.strip() or default_auth_source_for_provider(provider, profile.api_format)
if auth_source in {"codex_subscription", "claude_subscription"}:
env_auth_token = os.environ.get("ANTHROPIC_AUTH_TOKEN", "").strip()
if auth_source == "claude_subscription" and env_auth_token:
return ResolvedAuth(
provider=provider,
auth_kind="oauth",
value=env_auth_token,
source="env:ANTHROPIC_AUTH_TOKEN",
state="configured",
)
from openharness.auth.external import (
is_third_party_anthropic_endpoint,
load_external_credential,
@@ -744,25 +820,16 @@ class Settings(BaseModel):
storage_provider = credential_storage_provider_name(profile_name, profile)
env_var = {
"anthropic_api_key": "ANTHROPIC_API_KEY",
"openai_api_key": "OPENAI_API_KEY",
"dashscope_api_key": "DASHSCOPE_API_KEY",
"moonshot_api_key": "MOONSHOT_API_KEY",
"minimax_api_key": "MINIMAX_API_KEY",
"nvidia_api_key": "NVIDIA_API_KEY",
"modelscope_api_key": "MODELSCOPE_API_KEY",
}.get(auth_source)
if env_var:
env_value = os.environ.get(env_var, "")
if env_value:
return ResolvedAuth(
provider=provider or storage_provider,
auth_kind="api_key",
value=env_value,
source=f"env:{env_var}",
state="configured",
)
env_resolved = resolve_auth_env_value(auth_source)
if env_resolved:
env_var, env_value = env_resolved
return ResolvedAuth(
provider=provider or storage_provider,
auth_kind="api_key",
value=env_value,
source=f"env:{env_var}",
state="configured",
)
explicit_key = "" if profile.credential_slot else self.api_key
if explicit_key:
@@ -792,10 +859,21 @@ class Settings(BaseModel):
def merge_cli_overrides(self, **overrides: Any) -> Settings:
"""Return a new Settings with CLI overrides applied (non-None values only)."""
updates = {k: v for k, v in overrides.items() if v is not None}
permission_mode = updates.pop("permission_mode", None)
# Strip ANSI escape sequences from model name if present
if "model" in updates and isinstance(updates["model"], str):
updates["model"] = strip_ansi_escape_sequences(updates["model"])
if "effort" in updates and isinstance(updates["effort"], str):
updates["effort"] = "xhigh" if updates["effort"].strip().lower() == "max" else updates["effort"].strip().lower()
merged = self.model_copy(update=updates)
if permission_mode is not None:
merged = merged.model_copy(
update={
"permission": merged.permission.model_copy(
update={"mode": PermissionMode(str(permission_mode))}
)
}
)
if not updates:
return merged
profile_keys = {
@@ -871,15 +949,23 @@ def _apply_env_overrides(settings: Settings) -> Settings:
if auto_compact_threshold_tokens:
updates["auto_compact_threshold_tokens"] = int(auto_compact_threshold_tokens)
api_key = os.environ.get("ANTHROPIC_API_KEY") or os.environ.get("OPENAI_API_KEY")
if api_key:
provider = os.environ.get("OPENHARNESS_PROVIDER")
api_format = os.environ.get("OPENHARNESS_API_FORMAT")
env_auth_source = active_profile.auth_source
if provider or api_format:
env_auth_source = default_auth_source_for_provider(
provider or active_profile.provider,
api_format or active_profile.api_format,
)
env_resolved = resolve_auth_env_value(env_auth_source)
if env_resolved:
_, api_key = env_resolved
updates["api_key"] = api_key
api_format = os.environ.get("OPENHARNESS_API_FORMAT")
if api_format:
updates["api_format"] = api_format
provider = os.environ.get("OPENHARNESS_PROVIDER")
if provider:
updates["provider"] = provider
@@ -928,6 +1014,9 @@ def load_settings(config_path: Path | None = None) -> Settings:
if config_path.exists():
raw = json.loads(config_path.read_text(encoding="utf-8"))
settings = Settings.model_validate(raw)
env_profile = os.environ.get("OPENHARNESS_PROFILE")
if env_profile:
settings = settings.model_copy(update={"active_profile": env_profile.strip()})
if "profiles" not in raw or "active_profile" not in raw:
profile_name, profile = _profile_from_flat_settings(settings)
merged_profiles = settings.merged_profiles()
@@ -940,7 +1029,11 @@ def load_settings(config_path: Path | None = None) -> Settings:
)
return _apply_env_overrides(settings.materialize_active_profile())
return _apply_env_overrides(Settings().materialize_active_profile())
settings = Settings()
env_profile = os.environ.get("OPENHARNESS_PROFILE")
if env_profile:
settings = settings.model_copy(update={"active_profile": env_profile.strip()})
return _apply_env_overrides(settings.materialize_active_profile())
def save_settings(settings: Settings, config_path: Path | None = None) -> None:
+1
View File
@@ -53,6 +53,7 @@ class ToolResultBlock(BaseModel):
tool_use_id: str
content: str
is_error: bool = False
result_metadata: dict[str, Any] = Field(default_factory=dict)
ContentBlock = Annotated[
+15 -1
View File
@@ -145,6 +145,7 @@ class QueryContext:
model: str
system_prompt: str
max_tokens: int
effort: str | None = None
context_window_tokens: int | None = None
auto_compact_threshold_tokens: int | None = None
permission_prompt: PermissionPrompt | None = None
@@ -731,6 +732,7 @@ async def run_query(
system_prompt=context.system_prompt,
max_tokens=effective_max_tokens,
tools=context.tool_registry.to_api_schema(),
effort=context.effort,
)
):
if isinstance(event, ApiTextDeltaEvent):
@@ -820,11 +822,20 @@ async def run_query(
# Single tool: sequential (stream events immediately)
tc = tool_calls[0]
yield ToolExecutionStarted(tool_name=tc.name, tool_input=tc.input), None
result = await _execute_tool_call(context, tc.name, tc.id, tc.input)
try:
result = await _execute_tool_call(context, tc.name, tc.id, tc.input)
except Exception as exc:
log.exception("tool execution raised: name=%s id=%s", tc.name, tc.id)
result = ToolResultBlock(
tool_use_id=tc.id,
content=f"Tool {tc.name} failed: {type(exc).__name__}: {exc}",
is_error=True,
)
yield ToolExecutionCompleted(
tool_name=tc.name,
output=result.content,
is_error=result.is_error,
metadata=result.result_metadata,
), None
tool_results = [result]
else:
@@ -863,6 +874,7 @@ async def run_query(
tool_name=tc.name,
output=result.content,
is_error=result.is_error,
metadata=result.result_metadata,
), None
messages.append(ConversationMessage(role="user", content=tool_results))
@@ -962,6 +974,7 @@ async def _execute_tool_call(
metadata={
"tool_registry": context.tool_registry,
"ask_user_prompt": context.ask_user_prompt,
"model": context.model,
**(context.tool_metadata or {}),
},
hook_executor=context.hook_executor,
@@ -981,6 +994,7 @@ async def _execute_tool_call(
tool_use_id=tool_use_id,
content=inline_output,
is_error=result.is_error,
result_metadata=dict(result.metadata or {}),
)
_record_tool_carryover(
context,
+98 -6
View File
@@ -11,8 +11,10 @@ from openharness.coordinator.coordinator_mode import get_coordinator_user_contex
from openharness.engine.messages import ConversationMessage, TextBlock, ToolResultBlock, sanitize_conversation_messages
from openharness.engine.query import AskUserPrompt, PermissionPrompt, QueryContext, remember_user_goal, run_query
from openharness.engine.stream_events import AssistantTurnComplete, StreamEvent
from openharness.config.settings import Settings
from openharness.hooks import HookEvent, HookExecutor
from openharness.permissions.checker import PermissionChecker
from openharness.services.autodream.service import schedule_auto_dream
from openharness.tools.base import ToolRegistry
@@ -36,6 +38,7 @@ class QueryEngine:
ask_user_prompt: AskUserPrompt | None = None,
hook_executor: HookExecutor | None = None,
tool_metadata: dict[str, object] | None = None,
settings: Settings | None = None,
) -> None:
self._api_client = api_client
self._tool_registry = tool_registry
@@ -44,6 +47,7 @@ class QueryEngine:
self._model = model
self._system_prompt = system_prompt
self._max_tokens = max_tokens
self._effort = settings.effort if settings is not None else None
self._context_window_tokens = context_window_tokens
self._auto_compact_threshold_tokens = auto_compact_threshold_tokens
self._max_turns = max_turns
@@ -51,6 +55,7 @@ class QueryEngine:
self._ask_user_prompt = ask_user_prompt
self._hook_executor = hook_executor
self._tool_metadata = tool_metadata or {}
self._settings = settings
self._messages: list[ConversationMessage] = []
self._cost_tracker = CostTracker()
@@ -102,6 +107,10 @@ class QueryEngine:
"""Update the active model for future turns."""
self._model = model
def set_effort(self, effort: str | None) -> None:
"""Update the active reasoning effort for future turns."""
self._effort = effort
def set_api_client(self, api_client: SupportsStreamingMessages) -> None:
"""Update the active API client for future turns."""
self._api_client = api_client
@@ -129,6 +138,77 @@ class QueryEngine:
"""Replace the in-memory conversation history."""
self._messages = list(messages)
def _schedule_auto_dream(self) -> None:
"""Fire-and-forget background memory consolidation after a user turn."""
if self._settings is None:
return
context = self._tool_metadata.get("autodream_context")
kwargs = dict(context) if isinstance(context, dict) else {}
schedule_auto_dream(
cwd=self._cwd,
settings=self._settings,
model=self._model,
current_session_id=str(self._tool_metadata.get("session_id") or ""),
**kwargs,
)
def _prepare_session_memory(self) -> None:
"""Expose file-backed session memory to compaction when enabled."""
if self._settings is None or not self._settings.memory.session_memory_enabled:
return
if not self._settings.memory.enabled:
return
from openharness.services.session_memory import prepare_session_memory_metadata
prepare_session_memory_metadata(
self._cwd,
self._tool_metadata,
session_id=str(self._tool_metadata.get("session_id") or "default"),
)
async def _update_session_memory(self) -> None:
"""Persist a session checkpoint after a user turn."""
if self._settings is None or not self._settings.memory.session_memory_enabled:
return
if not self._settings.memory.enabled:
return
from openharness.services.session_memory import update_session_memory_file
update_session_memory_file(
self._cwd,
list(self._messages),
tool_metadata=self._tool_metadata,
session_id=str(self._tool_metadata.get("session_id") or "default"),
)
async def _extract_durable_memories(self) -> None:
"""Run the optional durable memory extraction pass."""
if self._settings is None or not self._settings.memory.auto_extract_enabled:
return
if not self._settings.memory.enabled:
return
from openharness.services.memory_extract import extract_memories_from_turn
try:
result = await extract_memories_from_turn(
cwd=self._cwd,
api_client=self._api_client,
model=self._model,
messages=list(self._messages),
max_records=self._settings.memory.auto_extract_max_records,
)
except Exception as exc:
self._tool_metadata["memory_extract_last_error"] = str(exc)
return
self._tool_metadata["memory_extract_last"] = {
"skipped": result.skipped,
"reason": result.reason,
"written_paths": [str(path) for path in result.written_paths],
}
def has_pending_continuation(self) -> bool:
"""Return True when the conversation ends with tool results awaiting a follow-up model turn."""
if not self._messages:
@@ -153,6 +233,7 @@ class QueryEngine:
)
if user_message.text.strip() and not self._tool_metadata.pop("_suppress_next_user_goal", False):
remember_user_goal(self._tool_metadata, user_message.text)
self._prepare_session_memory()
self._messages = sanitize_conversation_messages(self._messages)
self._messages.append(user_message)
if self._hook_executor is not None:
@@ -171,6 +252,7 @@ class QueryEngine:
model=self._model,
system_prompt=self._system_prompt,
max_tokens=self._max_tokens,
effort=self._effort,
context_window_tokens=self._context_window_tokens,
auto_compact_threshold_tokens=self._auto_compact_threshold_tokens,
max_turns=self._max_turns,
@@ -183,15 +265,22 @@ class QueryEngine:
coordinator_context = self._build_coordinator_context_message()
if coordinator_context is not None:
query_messages.append(coordinator_context)
async for event, usage in run_query(context, query_messages):
if isinstance(event, AssistantTurnComplete):
self._messages = list(query_messages)
if usage is not None:
self._cost_tracker.add(usage)
yield event
try:
async for event, usage in run_query(context, query_messages):
if isinstance(event, AssistantTurnComplete):
self._messages = list(query_messages)
if usage is not None:
self._cost_tracker.add(usage)
yield event
finally:
await self._update_session_memory()
await self._extract_durable_memories()
self._schedule_auto_dream()
async def continue_pending(self, *, max_turns: int | None = None) -> AsyncIterator[StreamEvent]:
"""Continue an interrupted tool loop without appending a new user message."""
self._prepare_session_memory()
self._messages = sanitize_conversation_messages(self._messages)
context = QueryContext(
api_client=self._api_client,
tool_registry=self._tool_registry,
@@ -200,6 +289,7 @@ class QueryEngine:
model=self._model,
system_prompt=self._system_prompt,
max_tokens=self._max_tokens,
effort=self._effort,
context_window_tokens=self._context_window_tokens,
auto_compact_threshold_tokens=self._auto_compact_threshold_tokens,
max_turns=max_turns if max_turns is not None else self._max_turns,
@@ -212,3 +302,5 @@ class QueryEngine:
if usage is not None:
self._cost_tracker.add(usage)
yield event
await self._update_session_memory()
await self._extract_durable_memories()
+1
View File
@@ -39,6 +39,7 @@ class ToolExecutionCompleted:
tool_name: str
output: str
is_error: bool = False
metadata: dict[str, Any] | None = None
@dataclass(frozen=True)
+10 -2
View File
@@ -18,8 +18,13 @@ class HookRegistry:
self._hooks[event].append(hook)
def get(self, event: HookEvent) -> list[HookDefinition]:
"""Return hooks registered for an event."""
return list(self._hooks.get(event, []))
"""Return hooks registered for an event, ordered by priority.
Hooks with a higher ``priority`` run first. ``sorted`` is stable, so
hooks sharing the same priority keep their registration order.
"""
hooks = self._hooks.get(event, [])
return sorted(hooks, key=lambda hook: -getattr(hook, "priority", 0))
def summary(self) -> str:
"""Return a human-readable hook summary."""
@@ -33,6 +38,9 @@ class HookRegistry:
matcher = getattr(hook, "matcher", None)
detail = getattr(hook, "command", None) or getattr(hook, "prompt", None) or getattr(hook, "url", None) or ""
suffix = f" matcher={matcher}" if matcher else ""
priority = getattr(hook, "priority", 0)
if priority:
suffix += f" priority={priority}"
lines.append(f" - {hook.type}{suffix}: {detail}")
return "\n".join(lines)
+8
View File
@@ -15,6 +15,8 @@ class CommandHookDefinition(BaseModel):
timeout_seconds: int = Field(default=30, ge=1, le=600)
matcher: str | None = None
block_on_failure: bool = False
priority: int = Field(default=0)
"""Higher priority runs first within an event; ties keep registration order."""
class PromptHookDefinition(BaseModel):
@@ -26,6 +28,8 @@ class PromptHookDefinition(BaseModel):
timeout_seconds: int = Field(default=30, ge=1, le=600)
matcher: str | None = None
block_on_failure: bool = True
priority: int = Field(default=0)
"""Higher priority runs first within an event; ties keep registration order."""
class HttpHookDefinition(BaseModel):
@@ -37,6 +41,8 @@ class HttpHookDefinition(BaseModel):
timeout_seconds: int = Field(default=30, ge=1, le=600)
matcher: str | None = None
block_on_failure: bool = False
priority: int = Field(default=0)
"""Higher priority runs first within an event; ties keep registration order."""
class AgentHookDefinition(BaseModel):
@@ -48,6 +54,8 @@ class AgentHookDefinition(BaseModel):
timeout_seconds: int = Field(default=60, ge=1, le=1200)
matcher: str | None = None
block_on_failure: bool = True
priority: int = Field(default=0)
"""Higher priority runs first within an event; ties keep registration order."""
HookDefinition = (
+7
View File
@@ -2,17 +2,24 @@
from openharness.memory.memdir import load_memory_prompt
from openharness.memory.manager import add_memory_entry, list_memory_files, remove_memory_entry
from openharness.memory.migrate import migrate_memory
from openharness.memory.paths import get_memory_entrypoint, get_project_memory_dir
from openharness.memory.scan import scan_memory_files
from openharness.memory.search import find_relevant_memories
from openharness.memory.usage import mark_memory_used
from openharness.memory.relevance import format_relevant_memories, select_relevant_memories
__all__ = [
"add_memory_entry",
"find_relevant_memories",
"format_relevant_memories",
"get_memory_entrypoint",
"get_project_memory_dir",
"list_memory_files",
"load_memory_prompt",
"mark_memory_used",
"migrate_memory",
"remove_memory_entry",
"scan_memory_files",
"select_relevant_memories",
]
+91
View File
@@ -0,0 +1,91 @@
"""Agent-scoped memory paths and snapshots."""
from __future__ import annotations
import re
import shutil
from pathlib import Path
from typing import Literal
from openharness.config.paths import get_data_dir
from openharness.memory.paths import get_project_memory_dir
AgentMemoryScope = Literal["user", "project", "local"]
MEMORY_INDEX = "MEMORY.md"
SNAPSHOT_DIR_NAME = "agent-memory-snapshots"
def sanitize_agent_type(agent_type: str) -> str:
"""Return a path-safe agent type."""
return re.sub(r"[^a-zA-Z0-9_.-]+", "_", agent_type.strip()).strip("._") or "default"
def get_agent_memory_dir(cwd: str | Path, agent_type: str, scope: AgentMemoryScope) -> Path:
"""Return an agent memory vault for the requested scope."""
safe = sanitize_agent_type(agent_type)
if scope == "project":
return get_project_memory_dir(cwd) / "agent" / safe
if scope == "local":
return Path(cwd).resolve() / ".openharness" / "agent-memory-local" / safe
return get_data_dir() / "agent-memory" / safe
def ensure_agent_memory_vault(cwd: str | Path, agent_type: str, scope: AgentMemoryScope) -> Path:
"""Create and return an agent-scoped memory vault."""
memory_dir = get_agent_memory_dir(cwd, agent_type, scope)
memory_dir.mkdir(parents=True, exist_ok=True)
entrypoint = memory_dir / MEMORY_INDEX
if not entrypoint.exists():
entrypoint.write_text("# Memory Index\n", encoding="utf-8")
return memory_dir
def get_agent_memory_entrypoint(cwd: str | Path, agent_type: str, scope: AgentMemoryScope) -> Path:
"""Return an agent memory ``MEMORY.md`` path."""
return ensure_agent_memory_vault(cwd, agent_type, scope) / MEMORY_INDEX
def get_agent_snapshot_dir(cwd: str | Path, agent_type: str) -> Path:
"""Return the project snapshot directory for an agent type."""
return Path(cwd).resolve() / ".openharness" / SNAPSHOT_DIR_NAME / sanitize_agent_type(agent_type)
def initialize_agent_memory_from_snapshot(
cwd: str | Path,
agent_type: str,
scope: AgentMemoryScope,
*,
replace: bool = False,
) -> Path | None:
"""Initialize local agent memory from a project snapshot if present."""
snapshot_dir = get_agent_snapshot_dir(cwd, agent_type)
if not snapshot_dir.exists():
return None
target = ensure_agent_memory_vault(cwd, agent_type, scope)
if replace and target.exists():
shutil.rmtree(target)
target.mkdir(parents=True, exist_ok=True)
for src in snapshot_dir.rglob("*.md"):
rel = src.relative_to(snapshot_dir)
dest = target / rel
dest.parent.mkdir(parents=True, exist_ok=True)
if replace or not dest.exists() or _is_default_agent_index(dest):
shutil.copy2(src, dest)
return target
def _is_default_agent_index(path: Path) -> bool:
if path.name != MEMORY_INDEX or not path.exists():
return False
try:
text = path.read_text(encoding="utf-8", errors="replace")
except OSError:
return False
return text.startswith("# Memory Index\n")
+143 -18
View File
@@ -6,6 +6,23 @@ from pathlib import Path
from re import sub
from openharness.memory.paths import get_memory_entrypoint, get_project_memory_dir
from openharness.memory.scan import scan_memory_files
from openharness.memory.schema import (
DEFAULT_MEMORY_SCOPE,
DEFAULT_MEMORY_TYPE,
MemoryScope,
MemoryType,
SCHEMA_VERSION,
compute_memory_signature,
first_content_line,
format_datetime,
generate_memory_id,
memory_metadata_from_path,
render_memory_file,
split_memory_file,
coerce_int,
utc_now,
)
from openharness.utils.file_lock import exclusive_file_lock
from openharness.utils.fs import atomic_write_text
@@ -16,36 +33,120 @@ def _memory_lock_path(cwd: str | Path) -> Path:
def list_memory_files(cwd: str | Path) -> list[Path]:
"""List memory markdown files for the project."""
memory_dir = get_project_memory_dir(cwd)
return sorted(path for path in memory_dir.glob("*.md"))
return sorted(header.path for header in scan_memory_files(cwd, max_files=None))
def add_memory_entry(cwd: str | Path, title: str, content: str) -> Path:
"""Create a memory file and append it to MEMORY.md."""
memory_dir = get_project_memory_dir(cwd)
def add_memory_entry(
cwd: str | Path,
title: str,
content: str,
*,
memory_type: MemoryType = DEFAULT_MEMORY_TYPE,
scope: MemoryScope = DEFAULT_MEMORY_SCOPE,
description: str = "",
tags: tuple[str, ...] = (),
) -> Path:
"""Create or refresh a memory file and append it to MEMORY.md."""
if scope == "team":
from openharness.memory.team import check_team_memory_secrets, ensure_team_memory_vault
secret_error = check_team_memory_secrets(content)
if secret_error:
raise ValueError(secret_error)
memory_dir = ensure_team_memory_vault(cwd)
else:
memory_dir = get_project_memory_dir(cwd)
slug = sub(r"[^a-zA-Z0-9]+", "_", title.strip().lower()).strip("_") or "memory"
path = memory_dir / f"{slug}.md"
with exclusive_file_lock(_memory_lock_path(cwd)):
atomic_write_text(path, content.strip() + "\n")
with exclusive_file_lock(memory_dir / ".memory.lock"):
category = "knowledge"
body = content.strip() + "\n"
signature = compute_memory_signature(body, memory_type, category)
existing = scan_memory_files(
cwd,
max_files=None,
include_disabled=True,
include_expired=True,
memory_dir=memory_dir,
)
duplicate = next(
(header for header in existing if _effective_signature(header.path, header.signature) == signature),
None,
)
path = duplicate.path if duplicate is not None else _next_memory_path(memory_dir, slug)
now = utc_now()
now_text = format_datetime(now)
if path.exists():
metadata, old_body, _, _ = split_memory_file(path.read_text(encoding="utf-8"))
metadata = memory_metadata_from_path(
path,
metadata,
old_body,
now=now,
source=str(metadata.get("source") or "manual"),
)
created_at = str(metadata.get("created_at") or now_text)
memory_id = str(metadata.get("id") or generate_memory_id(now))
else:
metadata = {}
created_at = now_text
memory_id = generate_memory_id(now)
entrypoint = get_memory_entrypoint(cwd)
existing = entrypoint.read_text(encoding="utf-8") if entrypoint.exists() else "# Memory Index\n"
if path.name not in existing:
existing = existing.rstrip() + f"\n- [{title}]({path.name})\n"
atomic_write_text(entrypoint, existing)
metadata.update(
{
"schema_version": SCHEMA_VERSION,
"id": memory_id,
"name": title.strip(),
"description": description.strip() or first_content_line(body) or title.strip(),
"type": str(metadata.get("type") or memory_type),
"scope": str(metadata.get("scope") or scope),
"category": str(metadata.get("category") or category),
"importance": max(coerce_int(metadata.get("importance"), default=0), 1),
"source": "manual",
"signature": signature,
"created_at": created_at,
"updated_at": now_text,
"ttl_days": metadata.get("ttl_days"),
"disabled": False,
"supersedes": metadata.get("supersedes") or [],
"tags": list(dict.fromkeys(str(tag).strip() for tag in tags if str(tag).strip())),
}
)
atomic_write_text(path, render_memory_file(metadata, body))
entrypoint = memory_dir / "MEMORY.md"
index_text = entrypoint.read_text(encoding="utf-8") if entrypoint.exists() else "# Memory Index\n"
if path.name not in index_text:
index_text = index_text.rstrip() + f"\n- [{title}]({path.name})\n"
atomic_write_text(entrypoint, index_text)
return path
def remove_memory_entry(cwd: str | Path, name: str) -> bool:
"""Delete a memory file and remove its index entry."""
memory_dir = get_project_memory_dir(cwd)
matches = [path for path in memory_dir.glob("*.md") if path.stem == name or path.name == name]
"""Soft-delete a memory file and remove its index entry."""
matches = [
header
for header in scan_memory_files(
cwd,
max_files=None,
include_disabled=True,
include_expired=True,
)
if name in {header.path.stem, header.path.name, header.title, header.id}
]
if not matches:
return False
path = matches[0]
header = matches[0]
if header.disabled:
return False
path = header.path
with exclusive_file_lock(_memory_lock_path(cwd)):
if path.exists():
path.unlink()
content = path.read_text(encoding="utf-8")
metadata, body, _, _ = split_memory_file(content)
metadata = memory_metadata_from_path(path, metadata, body, source="manual")
metadata["disabled"] = True
metadata["updated_at"] = format_datetime(utc_now())
atomic_write_text(path, render_memory_file(metadata, body))
entrypoint = get_memory_entrypoint(cwd)
if entrypoint.exists():
@@ -56,3 +157,27 @@ def remove_memory_entry(cwd: str | Path, name: str) -> bool:
]
atomic_write_text(entrypoint, "\n".join(lines).rstrip() + "\n")
return True
def _next_memory_path(memory_dir: Path, slug: str) -> Path:
path = memory_dir / f"{slug}.md"
if not path.exists():
return path
index = 2
while True:
candidate = memory_dir / f"{slug}_{index}.md"
if not candidate.exists():
return candidate
index += 1
def _effective_signature(path: Path, existing_signature: str) -> str:
if existing_signature:
return existing_signature
try:
metadata, body, _, _ = split_memory_file(path.read_text(encoding="utf-8"))
except OSError:
return ""
memory_type = str(metadata.get("type") or "project")
category = str(metadata.get("category") or "knowledge")
return compute_memory_signature(body, memory_type, category)
+23 -5
View File
@@ -5,23 +5,41 @@ from __future__ import annotations
from pathlib import Path
from openharness.memory.paths import get_memory_entrypoint, get_project_memory_dir
from openharness.memory.schema import (
MAX_ENTRYPOINT_BYTES,
MEMORY_POLICY_LINES,
truncate_entrypoint_content,
)
def load_memory_prompt(cwd: str | Path, *, max_entrypoint_lines: int = 200) -> str | None:
def load_memory_prompt(
cwd: str | Path,
*,
max_entrypoint_lines: int = 200,
max_entrypoint_bytes: int = MAX_ENTRYPOINT_BYTES,
) -> str | None:
"""Return the memory prompt section for the current project."""
memory_dir = get_project_memory_dir(cwd)
entrypoint = get_memory_entrypoint(cwd)
lines = [
"# Memory",
f"- Persistent memory directory: {memory_dir}",
"- Use this directory to store durable user or project context that should survive future sessions.",
"- Use this directory to store durable project and repository context that should survive future sessions.",
"- Prefer concise topic files plus an index entry in MEMORY.md.",
"",
*MEMORY_POLICY_LINES,
]
if entrypoint.exists():
content_lines = entrypoint.read_text(encoding="utf-8").splitlines()[:max_entrypoint_lines]
if content_lines:
lines.extend(["", "## MEMORY.md", "```md", *content_lines, "```"])
raw = entrypoint.read_text(encoding="utf-8", errors="replace")
view = truncate_entrypoint_content(
raw,
max_lines=max_entrypoint_lines,
max_bytes=max_entrypoint_bytes,
)
content = view.content.strip()
if content:
lines.extend(["", "## MEMORY.md", "```md", content, "```"])
else:
lines.extend(
[
+137
View File
@@ -0,0 +1,137 @@
"""Migration utilities for schema-v1 memory frontmatter."""
from __future__ import annotations
import argparse
import json
import shutil
from dataclasses import asdict, dataclass
from pathlib import Path
from openharness.memory.paths import get_project_memory_dir
from openharness.memory.schema import (
memory_metadata_from_path,
render_memory_file,
split_memory_file,
utc_now,
)
from openharness.utils.fs import atomic_write_text
@dataclass(frozen=True)
class MigrationSummary:
"""Summary returned by a memory schema migration run."""
scanned: int
changed: int
unchanged: int
failed: int
dry_run: bool
backup_dir: str
changed_files: tuple[str, ...]
failed_files: tuple[str, ...]
def as_dict(self) -> dict[str, object]:
return asdict(self)
def migrate_memory(
cwd: str | Path,
*,
memory_dir: str | Path | None = None,
default_type: str = "project",
default_category: str = "knowledge",
apply: bool = False,
) -> MigrationSummary:
"""Backfill schema-v1 frontmatter for top-level memory markdown files."""
root = Path(memory_dir).expanduser().resolve() if memory_dir is not None else get_project_memory_dir(cwd)
root.mkdir(parents=True, exist_ok=True)
files = sorted(path for path in root.glob("*.md") if path.name != "MEMORY.md")
changed_payloads: list[tuple[Path, str]] = []
failed_files: list[str] = []
seen_ids: set[str] = set()
now = utc_now()
for path in files:
try:
content = path.read_text(encoding="utf-8")
metadata, body, _, _ = split_memory_file(content)
migrated = memory_metadata_from_path(
path,
metadata,
body,
now=now,
source="migration",
default_type=default_type,
default_category=default_category,
seen_ids=seen_ids,
)
rendered = render_memory_file(migrated, body)
if rendered != content:
changed_payloads.append((path, rendered))
except OSError:
failed_files.append(path.name)
backup_dir = ""
if apply and changed_payloads:
backup_path = _create_migration_backup(root)
backup_dir = str(backup_path)
for path, rendered in changed_payloads:
atomic_write_text(path, rendered)
changed_files = tuple(path.name for path, _ in changed_payloads)
return MigrationSummary(
scanned=len(files),
changed=len(changed_payloads),
unchanged=len(files) - len(changed_payloads) - len(failed_files),
failed=len(failed_files),
dry_run=not apply,
backup_dir=backup_dir,
changed_files=changed_files,
failed_files=tuple(failed_files),
)
def main(argv: list[str] | None = None) -> int:
"""Command-line entrypoint for one-off memory migrations."""
parser = argparse.ArgumentParser(description="Backfill OpenHarness memory schema metadata.")
parser.add_argument("--cwd", default=".", help="Project cwd whose memory store should be migrated.")
parser.add_argument("--memory-dir", default=None, help="Explicit memory directory to migrate.")
parser.add_argument("--default-type", default="project", help="Type for legacy files without one.")
parser.add_argument("--default-category", default="knowledge", help="Category for legacy files without one.")
parser.add_argument("--dry-run", action="store_true", help="Report files that would change.")
parser.add_argument("--apply", action="store_true", help="Write migrated files and create a backup.")
args = parser.parse_args(argv)
if args.dry_run == args.apply:
parser.error("pass exactly one of --dry-run or --apply")
summary = migrate_memory(
args.cwd,
memory_dir=args.memory_dir,
default_type=args.default_type,
default_category=args.default_category,
apply=args.apply,
)
print(json.dumps(summary.as_dict(), indent=2, ensure_ascii=False))
return 0
def _create_migration_backup(memory_dir: Path) -> Path:
timestamp = utc_now().strftime("%Y%m%d-%H%M%S")
backup_dir = memory_dir / "backups" / f"migration-{timestamp}"
suffix = 2
while backup_dir.exists():
backup_dir = memory_dir / "backups" / f"migration-{timestamp}-{suffix}"
suffix += 1
backup_dir.mkdir(parents=True, exist_ok=False)
for path in sorted(memory_dir.glob("*.md")):
shutil.copy2(path, backup_dir / path.name)
usage_index = memory_dir / "usage_index.json"
if usage_index.exists():
shutil.copy2(usage_index, backup_dir / usage_index.name)
return backup_dir
if __name__ == "__main__":
raise SystemExit(main())
+145
View File
@@ -0,0 +1,145 @@
"""Relevant memory selection and formatting."""
from __future__ import annotations
import json
from dataclasses import dataclass
from pathlib import Path
from typing import Callable, Iterable
from openharness.memory.scan import scan_memory_files
from openharness.memory.schema import memory_age_label, memory_freshness_text
from openharness.memory.search import find_relevant_memories
from openharness.memory.types import MemoryHeader
@dataclass(frozen=True)
class RelevantMemory:
"""A memory selected for prompt injection."""
header: MemoryHeader
freshness: str = ""
MemorySelector = Callable[[str, list[MemoryHeader]], list[str]]
def build_memory_manifest(headers: Iterable[MemoryHeader]) -> str:
"""Render a compact manifest for selector prompts and diagnostics."""
lines: list[str] = []
for header in headers:
prefix = f"[{header.memory_type or 'memory'}]"
bits = [
prefix,
header.relative_path or header.path.name,
f"({memory_age_label(header.modified_at)})",
]
if header.description:
bits.append(f"- {header.description}")
lines.append(" ".join(bits))
return "\n".join(lines)
def select_relevant_memories(
query: str,
cwd: str | Path,
*,
max_results: int = 5,
already_surfaced: set[str] | None = None,
selector: MemorySelector | None = None,
) -> list[RelevantMemory]:
"""Return relevant memories with duplicate and freshness handling.
``selector`` is an optional side-query style reranker. It receives the query
and a heuristic shortlist, and returns relative paths in desired order.
"""
surfaced = already_surfaced or set()
heuristic = [
header
for header in find_relevant_memories(query, cwd, max_results=max(10, max_results * 3))
if (header.relative_path or str(header.path)) not in surfaced
]
selected = _apply_selector(query, heuristic, selector=selector, max_results=max_results)
result: list[RelevantMemory] = []
for header in selected[:max_results]:
result.append(RelevantMemory(header=header, freshness=memory_freshness_text(header.modified_at)))
return result
def select_manifest_memories(
query: str,
cwd: str | Path,
*,
max_results: int = 5,
selector: MemorySelector | None = None,
) -> list[RelevantMemory]:
"""Select from the full manifest instead of heuristic matches only."""
headers = scan_memory_files(cwd, max_files=200)
selected = _apply_selector(query, headers, selector=selector, max_results=max_results)
return [
RelevantMemory(header=header, freshness=memory_freshness_text(header.modified_at))
for header in selected[:max_results]
]
def format_relevant_memories(memories: Iterable[RelevantMemory], *, max_chars: int = 8000) -> str:
"""Render selected memories for prompt context."""
lines = ["# Relevant Memories"]
for item in memories:
header = item.header
content = header.path.read_text(encoding="utf-8", errors="replace").strip()
if item.freshness:
lines.extend(["", f"## {header.relative_path or header.path.name}", f"> {item.freshness}"])
else:
lines.extend(["", f"## {header.relative_path or header.path.name}"])
lines.extend(["```md", content[:max_chars], "```"])
return "\n".join(lines)
def json_selector_from_text(text: str) -> list[str]:
"""Parse a selector response as either JSON list or newline paths."""
stripped = text.strip()
if not stripped:
return []
try:
payload = json.loads(stripped)
except json.JSONDecodeError:
return [line.strip("- ").strip() for line in stripped.splitlines() if line.strip()]
if isinstance(payload, list):
return [str(item).strip() for item in payload if str(item).strip()]
if isinstance(payload, dict) and isinstance(payload.get("paths"), list):
return [str(item).strip() for item in payload["paths"] if str(item).strip()]
return []
def _apply_selector(
query: str,
headers: list[MemoryHeader],
*,
selector: MemorySelector | None,
max_results: int,
) -> list[MemoryHeader]:
if not headers or selector is None:
return headers[:max_results]
requested = selector(query, headers)
by_path = {header.relative_path or header.path.name: header for header in headers}
selected: list[MemoryHeader] = []
seen: set[str] = set()
for path in requested:
header = by_path.get(path)
if header is None or path in seen:
continue
selected.append(header)
seen.add(path)
if len(selected) < max_results:
selected.extend(
header
for header in headers
if (header.relative_path or header.path.name) not in seen
)
return selected[:max_results]
+60 -27
View File
@@ -5,12 +5,28 @@ from __future__ import annotations
from pathlib import Path
from openharness.memory.paths import get_project_memory_dir
from openharness.memory.schema import (
coerce_bool,
coerce_int,
coerce_optional_int,
coerce_str_list,
first_content_line,
is_memory_expired,
split_memory_file,
)
from openharness.memory.types import MemoryHeader
def scan_memory_files(cwd: str | Path, *, max_files: int = 50) -> list[MemoryHeader]:
def scan_memory_files(
cwd: str | Path,
*,
max_files: int | None = 50,
include_disabled: bool = False,
include_expired: bool = False,
memory_dir: str | Path | None = None,
) -> list[MemoryHeader]:
"""Return memory headers sorted by newest first."""
memory_dir = get_project_memory_dir(cwd)
memory_dir = Path(memory_dir) if memory_dir is not None else get_project_memory_dir(cwd)
headers: list[MemoryHeader] = []
for path in memory_dir.glob("*.md"):
if path.name == "MEMORY.md":
@@ -20,45 +36,40 @@ def scan_memory_files(cwd: str | Path, *, max_files: int = 50) -> list[MemoryHea
except OSError:
continue
header = _parse_memory_file(path, text)
if header.disabled and not include_disabled:
continue
if is_memory_expired(_metadata_from_header(header)) and not include_expired:
continue
headers.append(header)
headers.sort(key=lambda item: item.modified_at, reverse=True)
if max_files is None:
return headers
return headers[:max_files]
def _parse_memory_file(path: Path, content: str) -> MemoryHeader:
"""Parse a memory file, extracting YAML frontmatter when present."""
lines = content.splitlines()
metadata, body, _, _ = split_memory_file(content)
lines = body.splitlines()
title = path.stem
description = ""
memory_type = ""
body_start = 0
# Parse YAML frontmatter (--- ... ---)
if lines and lines[0].strip() == "---":
for i, line in enumerate(lines[1:], 1):
if line.strip() == "---":
for fm_line in lines[1:i]:
key, _, value = fm_line.partition(":")
key = key.strip()
value = value.strip().strip("'\"")
if not value:
continue
if key == "name":
title = value
elif key == "description":
description = value
elif key == "type":
memory_type = value
body_start = i + 1
break
if metadata.get("name"):
title = str(metadata["name"])
if metadata.get("description"):
description = str(metadata["description"])
if metadata.get("type"):
memory_type = str(metadata["type"])
# Fallback: first non-empty, non-frontmatter line as description
desc_line_idx: int | None = None
if not description:
for idx, line in enumerate(lines[body_start:body_start + 10], body_start):
fallback = first_content_line("\n".join(lines[:10]))
if fallback:
description = fallback
for idx, line in enumerate(lines[:10]):
stripped = line.strip()
if stripped and stripped != "---" and not stripped.startswith("#"):
description = stripped[:200]
if stripped[:200] == description:
desc_line_idx = idx
break
@@ -66,7 +77,7 @@ def _parse_memory_file(path: Path, content: str) -> MemoryHeader:
# line already used as description so search scoring stays consistent.
body_lines = [
line.strip()
for idx, line in enumerate(lines[body_start:], body_start)
for idx, line in enumerate(lines)
if line.strip()
and not line.strip().startswith("#")
and idx != desc_line_idx
@@ -80,4 +91,26 @@ def _parse_memory_file(path: Path, content: str) -> MemoryHeader:
modified_at=path.stat().st_mtime,
memory_type=memory_type,
body_preview=body_preview,
id=str(metadata.get("id") or ""),
schema_version=coerce_int(metadata.get("schema_version"), default=0),
category=str(metadata.get("category") or ""),
importance=coerce_int(metadata.get("importance"), default=0),
source=str(metadata.get("source") or ""),
signature=str(metadata.get("signature") or ""),
created_at=str(metadata.get("created_at") or ""),
updated_at=str(metadata.get("updated_at") or ""),
ttl_days=coerce_optional_int(metadata.get("ttl_days")),
disabled=coerce_bool(metadata.get("disabled"), default=False),
supersedes=coerce_str_list(metadata.get("supersedes")),
relative_path=path.name,
tags=coerce_str_list(metadata.get("tags")),
frontmatter=metadata,
)
def _metadata_from_header(header: MemoryHeader) -> dict[str, object]:
return {
"created_at": header.created_at,
"updated_at": header.updated_at,
"ttl_days": header.ttl_days,
}
+443
View File
@@ -0,0 +1,443 @@
"""Structured memory metadata helpers."""
from __future__ import annotations
import hashlib
import json
import re
import secrets
import string
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Any, Literal
import yaml
SCHEMA_VERSION = 1
MemoryType = Literal["user", "feedback", "project", "reference"]
MemoryScope = Literal["private", "project", "team"]
MEMORY_TYPES: tuple[MemoryType, ...] = ("user", "feedback", "project", "reference")
MEMORY_SCOPES: tuple[MemoryScope, ...] = ("private", "project", "team")
DEFAULT_MEMORY_TYPE: MemoryType = "project"
DEFAULT_MEMORY_SCOPE: MemoryScope = "project"
MAX_ENTRYPOINT_LINES = 200
MAX_ENTRYPOINT_BYTES = 25_000
MAX_MANIFEST_FILES = 200
FRONTMATTER_FIELDS = (
"schema_version",
"id",
"name",
"description",
"type",
"scope",
"category",
"importance",
"source",
"signature",
"created_at",
"updated_at",
"ttl_days",
"disabled",
"supersedes",
"tags",
)
@dataclass(frozen=True)
class EntrypointView:
"""A bounded view of ``MEMORY.md`` plus truncation diagnostics."""
content: str
was_truncated: bool
reason: str = ""
def utc_now() -> datetime:
"""Return the current UTC time without sub-second noise."""
return datetime.now(timezone.utc).replace(microsecond=0)
def format_datetime(value: datetime) -> str:
"""Format a datetime as an ISO-8601 UTC string."""
return value.astimezone(timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z")
def parse_datetime(value: object) -> datetime | None:
"""Parse an ISO datetime value used in memory frontmatter."""
if isinstance(value, datetime):
if value.tzinfo is None:
return value.replace(tzinfo=timezone.utc)
return value.astimezone(timezone.utc)
if not isinstance(value, str) or not value.strip():
return None
raw = value.strip()
if raw.endswith("Z"):
raw = raw[:-1] + "+00:00"
try:
parsed = datetime.fromisoformat(raw)
except ValueError:
return None
if parsed.tzinfo is None:
parsed = parsed.replace(tzinfo=timezone.utc)
return parsed.astimezone(timezone.utc)
def normalize_memory_content(text: str) -> str:
"""Normalize memory content for deterministic signatures."""
lowered = text.lower()
collapsed = re.sub(r"\s+", " ", lowered)
punctuation_table = str.maketrans("", "", string.punctuation)
return collapsed.translate(punctuation_table).strip()
def compute_memory_signature(content: str, memory_type: str, category: str) -> str:
"""Compute a deterministic content signature for duplicate detection."""
normalized = normalize_memory_content(content)
payload = f"{normalized}|{memory_type.strip().lower()}|{category.strip().lower()}"
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
def parse_memory_type(raw: Any, *, default: MemoryType | None = None) -> MemoryType | None:
"""Parse a frontmatter ``type`` value into the canonical runtime taxonomy."""
if isinstance(raw, str):
value = raw.strip().lower()
if value in MEMORY_TYPES:
return value # type: ignore[return-value]
if value in {"note", "memory", "core", "knowledge"}:
return default
return default
def parse_memory_scope(raw: Any, *, default: MemoryScope | None = None) -> MemoryScope | None:
"""Parse a frontmatter ``scope`` value into the canonical scope taxonomy."""
if isinstance(raw, str):
value = raw.strip().lower()
if value in MEMORY_SCOPES:
return value # type: ignore[return-value]
if value in {"personal", "user"}:
return "private"
if value in {"shared"}:
return "team"
return default
def truncate_entrypoint_content(
raw: str,
*,
max_lines: int = MAX_ENTRYPOINT_LINES,
max_bytes: int = MAX_ENTRYPOINT_BYTES,
) -> EntrypointView:
"""Bound ``MEMORY.md`` by line count and UTF-8 byte count."""
lines = raw.splitlines()
was_line_truncated = len(lines) > max_lines
text = "\n".join(lines[:max_lines])
encoded = text.encode("utf-8")
was_byte_truncated = len(encoded) > max_bytes
if was_byte_truncated:
encoded = encoded[:max_bytes]
text = encoded.decode("utf-8", errors="ignore")
cut_at = text.rfind("\n")
if cut_at > 0:
text = text[:cut_at]
if raw.endswith("\n") and not text.endswith("\n"):
text += "\n"
if not was_line_truncated and not was_byte_truncated:
return EntrypointView(content=text, was_truncated=False)
reason = (
f"{len(raw.encode('utf-8'))} bytes (limit: {max_bytes})"
if was_byte_truncated
else f"{len(lines)} lines (limit: {max_lines})"
)
warning = (
f"\n\n> WARNING: MEMORY.md is {reason}. Only part of it was loaded. "
"Keep index entries one line and move detail into topic notes.\n"
)
return EntrypointView(content=text.rstrip() + warning, was_truncated=True, reason=reason)
def memory_age_days(mtime: float, *, now: float | None = None) -> int:
"""Return floor-rounded days elapsed since a file modification time."""
import time
current = time.time() if now is None else now
return max(0, int((current - mtime) // 86_400))
def memory_age_label(mtime: float, *, now: float | None = None) -> str:
"""Return a model-friendly age label."""
days = memory_age_days(mtime, now=now)
if days == 0:
return "today"
if days == 1:
return "yesterday"
return f"{days} days ago"
def memory_freshness_text(mtime: float, *, now: float | None = None) -> str:
"""Return a staleness warning for older memories."""
days = memory_age_days(mtime, now=now)
if days <= 1:
return ""
return (
f"This memory is {days} days old. Memories are point-in-time observations; "
"verify claims against the current project state before treating them as facts."
)
def path_is_relative_to(path: str | Path, root: str | Path) -> bool:
"""Compatibility helper for containment checks."""
try:
Path(path).resolve().relative_to(Path(root).resolve())
except ValueError:
return False
return True
MEMORY_POLICY_LINES: tuple[str, ...] = (
"## Durable memory policy",
"- Store durable memory only when the information is not cheaply derivable from current files, docs, git history, or tool output.",
"- Use `type: user|feedback|project|reference` and optional `scope: private|project|team` frontmatter.",
"- `MEMORY.md` is an index, not a memory body. Keep each pointer one line.",
"- Update or remove stale contradictions instead of duplicating notes.",
"- If the user says to ignore memory, proceed as if no memory was loaded and do not cite, apply, or mention memory contents.",
"- Memory can be stale. Verify remembered project/code state against current files before acting on it.",
"- Do not save secrets, credentials, private personal context in team memory, or temporary task chatter.",
)
def generate_memory_id(now: datetime | None = None) -> str:
"""Generate a stable-looking memory id for a new memory file."""
timestamp = format_datetime(now or utc_now()).replace("-", "").replace(":", "")
timestamp = timestamp.replace("T", "-").replace("Z", "")
return f"mem-{timestamp}-{secrets.token_hex(4)}"
def split_memory_file(content: str) -> tuple[dict[str, Any], str, int, bool]:
"""Split a memory file into frontmatter metadata and body text.
Returns ``(metadata, body, body_start_line, has_closed_frontmatter)``.
Unclosed frontmatter is treated as body content after the opening delimiter.
"""
lines = content.splitlines(keepends=True)
if not lines or lines[0].strip() != "---":
return {}, content, 0, False
for idx, line in enumerate(lines[1:], 1):
if line.strip() == "---":
raw_frontmatter = "".join(lines[1:idx])
metadata = _load_frontmatter(raw_frontmatter)
return metadata, "".join(lines[idx + 1 :]), idx + 1, True
return {}, "".join(lines[1:]), 1, False
def render_memory_file(metadata: dict[str, Any], body: str) -> str:
"""Render metadata and body as a memory markdown file."""
frontmatter = render_frontmatter(metadata)
normalized_body = body.lstrip("\n")
if normalized_body and not normalized_body.endswith("\n"):
normalized_body += "\n"
return f"---\n{frontmatter}---\n\n{normalized_body}"
def render_frontmatter(metadata: dict[str, Any]) -> str:
"""Render memory frontmatter in a stable field order."""
ordered: list[tuple[str, Any]] = []
for field in FRONTMATTER_FIELDS:
if field in metadata:
ordered.append((field, metadata[field]))
for key, value in metadata.items():
if key not in FRONTMATTER_FIELDS:
ordered.append((key, value))
return "".join(f"{key}: {_format_yaml_value(value)}\n" for key, value in ordered)
def is_disabled_metadata(metadata: dict[str, Any]) -> bool:
"""Return whether a memory metadata object marks the file disabled."""
return _as_bool(metadata.get("disabled"), default=False)
def is_memory_expired(metadata: dict[str, Any], *, now: datetime | None = None) -> bool:
"""Return whether a memory should be hidden because its TTL has elapsed."""
ttl_days = _as_optional_int(metadata.get("ttl_days"))
if ttl_days is None or ttl_days <= 0:
return False
base_time = parse_datetime(metadata.get("updated_at")) or parse_datetime(metadata.get("created_at"))
if base_time is None:
return False
return (now or utc_now()) >= base_time + timedelta(days=ttl_days)
def coerce_int(value: object, *, default: int = 0) -> int:
"""Coerce a metadata value to int."""
try:
return int(value) # type: ignore[arg-type]
except (TypeError, ValueError):
return default
def coerce_optional_int(value: object) -> int | None:
"""Coerce a metadata value to optional int."""
return _as_optional_int(value)
def coerce_bool(value: object, *, default: bool = False) -> bool:
"""Coerce a metadata value to bool."""
return _as_bool(value, default=default)
def coerce_str_list(value: object) -> tuple[str, ...]:
"""Coerce a metadata value to a tuple of strings."""
if isinstance(value, str):
return (value,) if value else ()
if isinstance(value, (list, tuple)):
return tuple(str(item) for item in value if str(item))
return ()
def memory_metadata_from_path(
path: Path,
metadata: dict[str, Any],
body: str,
*,
now: datetime | None = None,
source: str = "migration",
default_type: str = "project",
default_category: str = "knowledge",
seen_ids: set[str] | None = None,
) -> dict[str, Any]:
"""Return complete schema-v1 metadata while preserving existing values."""
updated = dict(metadata)
timestamp = _mtime_timestamp(path)
created_at = str(updated.get("created_at") or timestamp)
updated_at = str(updated.get("updated_at") or timestamp)
memory_type = str(updated.get("type") or default_type)
category = str(updated.get("category") or default_category)
memory_id = str(updated.get("id") or "")
if not memory_id or (seen_ids is not None and memory_id in seen_ids):
memory_id = _generate_unique_memory_id(now=now, seen_ids=seen_ids)
if seen_ids is not None:
seen_ids.add(memory_id)
updated["schema_version"] = coerce_int(updated.get("schema_version"), default=SCHEMA_VERSION)
updated["id"] = memory_id
updated["name"] = str(updated.get("name") or path.stem)
updated["description"] = str(updated.get("description") or first_content_line(body) or path.stem)
updated["type"] = memory_type
updated["category"] = category
updated["importance"] = coerce_int(updated.get("importance"), default=0)
updated["source"] = str(updated.get("source") or source)
updated["signature"] = str(
updated.get("signature") or compute_memory_signature(body, memory_type, category)
)
updated["created_at"] = created_at
updated["updated_at"] = updated_at
updated["ttl_days"] = coerce_optional_int(updated.get("ttl_days"))
updated["disabled"] = coerce_bool(updated.get("disabled"), default=False)
updated["supersedes"] = list(coerce_str_list(updated.get("supersedes")))
return updated
def first_content_line(body: str, *, limit: int = 200) -> str:
"""Return the first useful body line for descriptions."""
for line in body.splitlines():
stripped = line.strip()
if stripped and stripped != "---" and not stripped.startswith("#"):
return stripped[:limit]
return ""
def _load_frontmatter(raw_frontmatter: str) -> dict[str, Any]:
try:
loaded = yaml.safe_load(raw_frontmatter) or {}
except yaml.YAMLError:
return {}
if not isinstance(loaded, dict):
return {}
return {str(key): value for key, value in loaded.items()}
def _format_yaml_value(value: Any) -> str:
if value is None:
return "null"
if isinstance(value, bool):
return "true" if value else "false"
if isinstance(value, int):
return str(value)
if isinstance(value, (list, tuple)):
return json.dumps(list(value), ensure_ascii=False)
return json.dumps(str(value), ensure_ascii=False)
def _mtime_timestamp(path: Path) -> str:
try:
modified = datetime.fromtimestamp(path.stat().st_mtime, tz=timezone.utc)
except OSError:
modified = utc_now()
return format_datetime(modified)
def _generate_unique_memory_id(
*,
now: datetime | None = None,
seen_ids: set[str] | None = None,
) -> str:
while True:
memory_id = generate_memory_id(now=now)
if seen_ids is None or memory_id not in seen_ids:
return memory_id
def _as_bool(value: object, *, default: bool) -> bool:
if isinstance(value, bool):
return value
if isinstance(value, str):
lowered = value.strip().lower()
if lowered in {"1", "true", "yes", "on"}:
return True
if lowered in {"0", "false", "no", "off"}:
return False
if value is None:
return default
return bool(value)
def _as_optional_int(value: object) -> int | None:
if value is None:
return None
if isinstance(value, str) and not value.strip():
return None
try:
return int(value) # type: ignore[arg-type]
except (TypeError, ValueError):
return None
+24 -2
View File
@@ -3,10 +3,13 @@
from __future__ import annotations
import re
from datetime import timedelta
from pathlib import Path
from openharness.memory.scan import scan_memory_files
from openharness.memory.schema import parse_datetime, utc_now
from openharness.memory.types import MemoryHeader
from openharness.memory.usage import get_memory_usage
def find_relevant_memories(
@@ -32,8 +35,15 @@ def find_relevant_memories(
# Metadata matches are weighted 2x; body matches 1x.
meta_hits = sum(1 for t in tokens if t in meta)
body_hits = sum(1 for t in tokens if t in body)
score = meta_hits * 2.0 + body_hits
if score > 0:
usage = get_memory_usage(cwd, header.id, memory_dir=header.path.parent)
score = (
meta_hits * 2.0
+ body_hits
+ header.importance * 0.4
+ min(int(usage["use_count"]), 5) * 0.1
+ _recency_boost(header)
)
if meta_hits or body_hits:
scored.append((score, header))
scored.sort(key=lambda item: (-item[0], -item[1].modified_at))
@@ -47,3 +57,15 @@ def _tokenize(text: str) -> set[str]:
# Han ideographs (each character carries independent meaning)
han_chars = set(re.findall(r"[\u4e00-\u9fff\u3400-\u4dbf]", text))
return ascii_tokens | han_chars
def _recency_boost(header: MemoryHeader) -> float:
timestamp = parse_datetime(header.updated_at) or parse_datetime(header.created_at)
if timestamp is None:
return 0.0
age = utc_now() - timestamp
if age <= timedelta(days=14):
return 0.3
if age <= timedelta(days=30):
return 0.1
return 0.0
+90
View File
@@ -0,0 +1,90 @@
"""Local team-memory vault helpers and safety guards."""
from __future__ import annotations
import re
from dataclasses import dataclass
from pathlib import Path
from openharness.memory.paths import get_project_memory_dir
from openharness.memory.schema import path_is_relative_to
TEAM_DIR_NAME = "team"
MEMORY_INDEX = "MEMORY.md"
@dataclass(frozen=True)
class SecretMatch:
"""A possible secret found in shared memory content."""
rule_id: str
label: str
SECRET_RULES: tuple[tuple[str, str, re.Pattern[str]], ...] = (
("private-key", "private key", re.compile(r"-----BEGIN [A-Z ]*PRIVATE KEY-----")),
("aws-access-key", "AWS access key", re.compile(r"\bAKIA[0-9A-Z]{16}\b")),
("github-token", "GitHub token", re.compile(r"\bgh[pousr]_[A-Za-z0-9_]{20,}\b")),
("openai-key", "OpenAI API key", re.compile(r"\bsk-[A-Za-z0-9_-]{20,}\b")),
("anthropic-key", "Anthropic API key", re.compile(r"\bsk-ant-[A-Za-z0-9_-]{20,}\b")),
("generic-secret", "secret assignment", re.compile(r"(?i)\b(secret|token|api[_-]?key|password)\s*[:=]\s*['\"]?[^'\"\s]{12,}")),
)
def get_team_memory_dir(cwd: str | Path) -> Path:
"""Return the project-local shared team memory vault."""
return get_project_memory_dir(cwd) / TEAM_DIR_NAME
def ensure_team_memory_vault(cwd: str | Path) -> Path:
"""Create and return the team memory vault."""
team_dir = get_team_memory_dir(cwd)
team_dir.mkdir(parents=True, exist_ok=True)
entrypoint = team_dir / MEMORY_INDEX
if not entrypoint.exists():
entrypoint.write_text("# Memory Index\n", encoding="utf-8")
return team_dir
def validate_team_memory_write_path(cwd: str | Path, candidate: str | Path) -> tuple[Path | None, str | None]:
"""Validate a write target against traversal and symlink escape."""
team_dir = ensure_team_memory_vault(cwd).resolve()
path = Path(candidate).expanduser()
if not path.is_absolute():
path = team_dir / path
resolved = path.resolve()
if not path_is_relative_to(resolved, team_dir):
return None, f"Path escapes team memory directory: {candidate}"
parent = resolved.parent
deepest = parent
while not deepest.exists() and deepest != deepest.parent:
deepest = deepest.parent
if deepest.exists() and not path_is_relative_to(deepest.resolve(), team_dir):
return None, f"Path escapes team memory directory via symlink: {candidate}"
return resolved, None
def scan_for_secrets(content: str) -> list[SecretMatch]:
"""Return possible secrets in content without exposing matched values."""
matches: list[SecretMatch] = []
for rule_id, label, pattern in SECRET_RULES:
if pattern.search(content):
matches.append(SecretMatch(rule_id=rule_id, label=label))
return matches
def check_team_memory_secrets(content: str) -> str | None:
"""Return an error message when shared memory content appears sensitive."""
matches = scan_for_secrets(content)
if not matches:
return None
labels = ", ".join(match.label for match in matches)
return (
f"Content contains potential secrets ({labels}) and cannot be written to team memory. "
"Team memory is shared with project collaborators."
)
+16 -1
View File
@@ -2,8 +2,9 @@
from __future__ import annotations
from dataclasses import dataclass
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
@dataclass(frozen=True)
@@ -16,3 +17,17 @@ class MemoryHeader:
modified_at: float
memory_type: str = ""
body_preview: str = ""
id: str = ""
schema_version: int = 0
category: str = ""
importance: int = 0
source: str = ""
signature: str = ""
created_at: str = ""
updated_at: str = ""
ttl_days: int | None = None
disabled: bool = False
supersedes: tuple[str, ...] = ()
relative_path: str = ""
tags: tuple[str, ...] = field(default_factory=tuple)
frontmatter: dict[str, Any] = field(default_factory=dict)
+153
View File
@@ -0,0 +1,153 @@
"""Usage index for recalled memory entries."""
from __future__ import annotations
import json
from datetime import timedelta
from pathlib import Path
from typing import Any
from openharness.memory.paths import get_project_memory_dir
from openharness.memory.scan import scan_memory_files
from openharness.memory.schema import format_datetime, parse_datetime, utc_now
from openharness.memory.types import MemoryHeader
from openharness.utils.file_lock import exclusive_file_lock
from openharness.utils.fs import atomic_write_text
USAGE_INDEX_NAME = "usage_index.json"
STALE_UNUSED_DAYS = 60
STALE_MAX_IMPORTANCE = 1
def get_usage_index_path(cwd: str | Path, *, memory_dir: str | Path | None = None) -> Path:
"""Return the usage index path for a memory store."""
root = Path(memory_dir) if memory_dir is not None else get_project_memory_dir(cwd)
root.mkdir(parents=True, exist_ok=True)
return root / USAGE_INDEX_NAME
def load_usage_index(cwd: str | Path, *, memory_dir: str | Path | None = None) -> dict[str, Any]:
"""Load usage index data, returning an empty index for invalid files."""
path = get_usage_index_path(cwd, memory_dir=memory_dir)
if not path.exists():
return _empty_index()
try:
data = json.loads(path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return _empty_index()
if not isinstance(data, dict):
return _empty_index()
memories = data.get("memories")
if not isinstance(memories, dict):
memories = {}
normalized = _empty_index()
normalized["memories"] = {
str(memory_id): _normalize_usage_record(record)
for memory_id, record in memories.items()
if isinstance(record, dict)
}
return normalized
def save_usage_index(
cwd: str | Path,
index: dict[str, Any],
*,
memory_dir: str | Path | None = None,
) -> None:
"""Persist usage index data atomically."""
path = get_usage_index_path(cwd, memory_dir=memory_dir)
payload = json.dumps(index, indent=2, ensure_ascii=False, sort_keys=True) + "\n"
atomic_write_text(path, payload)
def get_memory_usage(
cwd: str | Path,
memory_id: str,
*,
memory_dir: str | Path | None = None,
) -> dict[str, Any]:
"""Return usage data for a memory id."""
if not memory_id:
return _normalize_usage_record({})
index = load_usage_index(cwd, memory_dir=memory_dir)
record = index["memories"].get(memory_id, {})
return _normalize_usage_record(record)
def mark_memory_used(
cwd: str | Path,
memories: list[MemoryHeader],
*,
memory_dir: str | Path | None = None,
) -> None:
"""Record that memory entries were recalled into a runtime prompt."""
usable = [header for header in memories if header.id]
if not usable:
return
resolved_memory_dir = Path(memory_dir) if memory_dir is not None else usable[0].path.parent
lock_path = resolved_memory_dir / ".usage_index.lock"
with exclusive_file_lock(lock_path):
index = load_usage_index(cwd, memory_dir=resolved_memory_dir)
now = format_datetime(utc_now())
for header in usable:
record = _normalize_usage_record(index["memories"].get(header.id, {}))
record["use_count"] = int(record["use_count"]) + 1
record["last_used_at"] = now
record["path"] = header.path.name
index["memories"][header.id] = record
save_usage_index(cwd, index, memory_dir=resolved_memory_dir)
def find_stale_memory_candidates(
cwd: str | Path,
*,
memory_dir: str | Path | None = None,
) -> list[MemoryHeader]:
"""Return low-value unused memories that auto-dream should review for pruning."""
resolved_memory_dir = Path(memory_dir) if memory_dir is not None else None
headers = scan_memory_files(
cwd,
max_files=None,
include_disabled=False,
include_expired=False,
memory_dir=resolved_memory_dir,
)
now = utc_now()
candidates: list[MemoryHeader] = []
for header in headers:
if header.importance > STALE_MAX_IMPORTANCE:
continue
usage = get_memory_usage(cwd, header.id, memory_dir=resolved_memory_dir or header.path.parent)
if int(usage["use_count"]) > 0:
continue
updated_at = parse_datetime(header.updated_at) or parse_datetime(header.created_at)
if updated_at is None:
continue
if now - updated_at >= timedelta(days=STALE_UNUSED_DAYS):
candidates.append(header)
candidates.sort(key=lambda item: (item.importance, item.updated_at or "", item.path.name))
return candidates
def _empty_index() -> dict[str, Any]:
return {"version": 1, "memories": {}}
def _normalize_usage_record(record: dict[str, Any]) -> dict[str, Any]:
use_count = record.get("use_count", 0)
try:
use_count = max(0, int(use_count))
except (TypeError, ValueError):
use_count = 0
return {
"last_used_at": str(record.get("last_used_at") or ""),
"use_count": use_count,
"path": str(record.get("path") or ""),
}
+1 -2
View File
@@ -37,7 +37,7 @@ def detect_platform(
if system == "darwin":
return "macos"
if system == "windows":
if system in {"windows", "win32"}:
return "windows"
if system == "linux":
if "microsoft" in kernel_release or env_map.get("WSL_DISTRO_NAME") or env_map.get("WSL_INTEROP"):
@@ -84,4 +84,3 @@ def get_platform_capabilities(platform_name: PlatformName | None = None) -> Plat
supports_sandbox_runtime=False,
supports_docker_sandbox=False,
)
+36 -15
View File
@@ -12,8 +12,11 @@ from openharness.config.paths import (
)
from openharness.config.settings import Settings
from openharness.coordinator.coordinator_mode import get_coordinator_system_prompt, is_coordinator_mode
from openharness.memory import find_relevant_memories, load_memory_prompt
from openharness.memory import load_memory_prompt
from openharness.memory.relevance import format_relevant_memories, select_relevant_memories
from openharness.memory.usage import mark_memory_used
from openharness.personalization.rules import load_local_rules
from openharness.permissions.modes import PermissionMode
from openharness.prompts.claudemd import load_claude_md_prompt
from openharness.prompts.system_prompt import build_system_prompt
from openharness.skills.loader import load_skill_registry
@@ -74,6 +77,28 @@ def _build_delegation_section() -> str:
)
def _build_permission_mode_section(settings: Settings) -> str:
"""Build current permission-mode guidance for the model."""
mode = settings.permission.mode
if mode == PermissionMode.PLAN:
guidance = (
"Plan mode is enabled. Treat this session as read-only planning and analysis. "
"Do not call mutating tools such as file writes, edits, package installs, "
"state-changing shell commands, or task-spawning actions unless the user exits plan mode."
)
elif mode == PermissionMode.FULL_AUTO:
guidance = (
"Full-auto permission mode is enabled. You may use mutating tools when they are necessary "
"for the user's request, while still keeping changes scoped and intentional."
)
else:
guidance = (
"Default permission mode is enabled. Read-only tools can run directly; mutating tools "
"may require explicit user approval."
)
return f"# Current Permission Mode\n{guidance}"
def build_runtime_system_prompt(
settings: Settings,
*,
@@ -92,6 +117,8 @@ def build_runtime_system_prompt(
if not is_coordinator_mode() and settings.system_prompt is None:
sections[0] = build_system_prompt(cwd=str(cwd))
sections.append(_build_permission_mode_section(settings))
if settings.fast_mode:
sections.append(
"# Session Mode\nFast mode is enabled. Prefer concise replies, minimal tool use, and quicker progress over exhaustive exploration."
@@ -138,29 +165,23 @@ def build_runtime_system_prompt(
memory_section = load_memory_prompt(
cwd,
max_entrypoint_lines=settings.memory.max_entrypoint_lines,
max_entrypoint_bytes=settings.memory.max_entrypoint_bytes,
)
if memory_section:
sections.append(memory_section)
if latest_user_prompt:
relevant = find_relevant_memories(
relevant = select_relevant_memories(
latest_user_prompt,
cwd,
max_results=settings.memory.max_files,
)
if relevant:
lines = ["# Relevant Memories"]
for header in relevant:
content = header.path.read_text(encoding="utf-8", errors="replace").strip()
lines.extend(
[
"",
f"## {header.path.name}",
"```md",
content[:8000],
"```",
]
)
sections.append("\n".join(lines))
try:
headers = [item.header for item in relevant]
mark_memory_used(cwd, headers, memory_dir=headers[0].path.parent)
except OSError:
pass
sections.append(format_relevant_memories(relevant))
return "\n\n".join(section for section in sections if section.strip())
@@ -0,0 +1,34 @@
"""Automatic memory consolidation (auto-dream)."""
from openharness.services.autodream.backup import (
create_memory_backup,
diff_memory_dirs,
format_memory_diff,
latest_memory_backup,
restore_memory_backup,
)
from openharness.services.autodream.lock import (
list_sessions_touched_since,
read_last_consolidated_at,
record_consolidation,
rollback_consolidation_lock,
try_acquire_consolidation_lock,
)
from openharness.services.autodream.prompt import build_consolidation_prompt
from openharness.services.autodream.service import execute_auto_dream, start_dream_now
__all__ = [
"build_consolidation_prompt",
"create_memory_backup",
"diff_memory_dirs",
"execute_auto_dream",
"format_memory_diff",
"latest_memory_backup",
"list_sessions_touched_since",
"read_last_consolidated_at",
"record_consolidation",
"restore_memory_backup",
"rollback_consolidation_lock",
"start_dream_now",
"try_acquire_consolidation_lock",
]
@@ -0,0 +1,104 @@
"""Backup, diff, and rollback helpers for auto-dream memory directories."""
from __future__ import annotations
import filecmp
import shutil
import time
from pathlib import Path
from openharness.config.paths import get_data_dir
def default_backup_root(memory_dir: str | Path, *, app_label: str = "openharness") -> Path:
"""Return the backup root for a memory directory."""
memory_dir = Path(memory_dir).expanduser().resolve()
if ".ohmo" in memory_dir.parts:
try:
idx = memory_dir.parts.index(".ohmo")
return Path(*memory_dir.parts[: idx + 1]) / "backups"
except ValueError:
pass
safe_label = "".join(ch if ch.isalnum() or ch in {"-", "_"} else "-" for ch in app_label).strip("-")
return get_data_dir() / "memory-backups" / (safe_label or "openharness")
def create_memory_backup(
memory_dir: str | Path,
*,
backup_root: str | Path | None = None,
app_label: str = "openharness",
) -> Path:
"""Create a timestamped copy of ``memory_dir`` and return the backup path."""
memory_dir = Path(memory_dir).expanduser().resolve()
root = Path(backup_root).expanduser().resolve() if backup_root is not None else default_backup_root(memory_dir, app_label=app_label)
root.mkdir(parents=True, exist_ok=True)
timestamp = time.strftime("memory-%Y%m%d-%H%M%S")
backup = root / timestamp
suffix = 1
while backup.exists():
suffix += 1
backup = root / f"{timestamp}-{suffix}"
if memory_dir.exists():
shutil.copytree(memory_dir, backup, ignore=shutil.ignore_patterns(".consolidate-lock"))
else:
backup.mkdir(parents=True)
return backup
def diff_memory_dirs(before: str | Path, after: str | Path) -> dict[str, list[str]]:
"""Return added/removed/changed file names between two memory dirs."""
before = Path(before).expanduser().resolve()
after = Path(after).expanduser().resolve()
before_files = {p.name: p for p in before.glob("*.md")} if before.exists() else {}
after_files = {p.name: p for p in after.glob("*.md")} if after.exists() else {}
added = sorted(set(after_files) - set(before_files))
removed = sorted(set(before_files) - set(after_files))
changed = sorted(
name
for name in set(before_files) & set(after_files)
if not filecmp.cmp(before_files[name], after_files[name], shallow=False)
)
return {"added": added, "removed": removed, "changed": changed}
def format_memory_diff(diff: dict[str, list[str]]) -> str:
"""Format a compact memory diff summary."""
lines: list[str] = []
for label in ("added", "changed", "removed"):
values = diff.get(label, [])
if values:
lines.append(f"{label}: " + ", ".join(values))
return "\n".join(lines) if lines else "no markdown file changes"
def latest_memory_backup(memory_dir: str | Path, *, app_label: str = "openharness") -> Path | None:
"""Return the latest backup for a memory directory, if any."""
root = default_backup_root(memory_dir, app_label=app_label)
if not root.exists():
return None
backups = [path for path in root.iterdir() if path.is_dir() and path.name.startswith("memory-")]
if not backups:
return None
return max(backups, key=lambda path: path.stat().st_mtime)
def restore_memory_backup(backup_dir: str | Path, memory_dir: str | Path) -> None:
"""Restore memory_dir from a backup directory."""
backup_dir = Path(backup_dir).expanduser().resolve()
memory_dir = Path(memory_dir).expanduser().resolve()
if not backup_dir.exists() or not backup_dir.is_dir():
raise FileNotFoundError(f"Backup not found: {backup_dir}")
tmp = memory_dir.with_name(f".{memory_dir.name}.restore-tmp")
if tmp.exists():
shutil.rmtree(tmp)
shutil.copytree(backup_dir, tmp)
if memory_dir.exists():
shutil.rmtree(memory_dir)
tmp.rename(memory_dir)
+138
View File
@@ -0,0 +1,138 @@
"""Locking and session scanning for auto-dream."""
from __future__ import annotations
import json
import os
import time
from pathlib import Path
from openharness.memory.paths import get_project_memory_dir
from openharness.services.session_storage import get_project_session_dir
from openharness.utils.fs import atomic_write_text
LOCK_FILE = ".consolidate-lock"
HOLDER_STALE_SECONDS = 60 * 60
def _lock_path(cwd: str | Path, memory_dir: str | Path | None = None) -> Path:
return Path(memory_dir) / LOCK_FILE if memory_dir is not None else get_project_memory_dir(cwd) / LOCK_FILE
def read_last_consolidated_at(cwd: str | Path, memory_dir: str | Path | None = None) -> float:
"""Return lock mtime as the last successful consolidation timestamp."""
try:
return _lock_path(cwd, memory_dir).stat().st_mtime
except OSError:
return 0.0
def _holder_pid(path: Path) -> int | None:
try:
raw = path.read_text(encoding="utf-8").strip()
pid = int(raw)
except (OSError, ValueError):
return None
return pid if pid > 0 else None
def _is_process_running(pid: int) -> bool:
if pid == os.getpid():
return True
try:
os.kill(pid, 0)
except ProcessLookupError:
return False
except PermissionError:
return True
return True
def try_acquire_consolidation_lock(cwd: str | Path, memory_dir: str | Path | None = None) -> float | None:
"""Acquire the consolidation lock and return prior mtime, or None if held."""
path = _lock_path(cwd, memory_dir)
prior_mtime: float | None = None
try:
stat = path.stat()
prior_mtime = stat.st_mtime
holder = _holder_pid(path)
except OSError:
holder = None
if prior_mtime is not None and time.time() - prior_mtime < HOLDER_STALE_SECONDS:
if holder is not None and _is_process_running(holder):
return None
path.parent.mkdir(parents=True, exist_ok=True)
atomic_write_text(path, f"{os.getpid()}\n")
try:
if _holder_pid(path) != os.getpid():
return None
except OSError:
return None
return prior_mtime or 0.0
def rollback_consolidation_lock(
cwd: str | Path,
prior_mtime: float,
memory_dir: str | Path | None = None,
) -> None:
"""Restore lock mtime to its pre-acquire value after failed/killed dream."""
path = _lock_path(cwd, memory_dir)
try:
if prior_mtime <= 0:
path.unlink(missing_ok=True)
return
atomic_write_text(path, "")
os.utime(path, (prior_mtime, prior_mtime))
except OSError:
# Best effort: a failed rollback only delays the next auto trigger.
return
def record_consolidation(cwd: str | Path, memory_dir: str | Path | None = None) -> None:
"""Stamp a manual consolidation time."""
path = _lock_path(cwd, memory_dir)
path.parent.mkdir(parents=True, exist_ok=True)
atomic_write_text(path, f"{os.getpid()}\n")
def list_sessions_touched_since(
cwd: str | Path,
since_ts: float,
*,
current_session_id: str | None = None,
session_dir: str | Path | None = None,
) -> list[str]:
"""Return saved session IDs whose snapshot files were touched after ``since_ts``."""
resolved_session_dir = Path(session_dir) if session_dir is not None else get_project_session_dir(cwd)
session_ids: list[str] = []
seen: set[str] = set()
for path in sorted(resolved_session_dir.glob("session-*.json"), key=lambda item: item.stat().st_mtime, reverse=True):
try:
mtime = path.stat().st_mtime
except OSError:
continue
if mtime <= since_ts:
continue
session_id = path.stem.removeprefix("session-")
try:
payload = json.loads(path.read_text(encoding="utf-8"))
raw_id = payload.get("session_id")
if isinstance(raw_id, str) and raw_id.strip():
session_id = raw_id.strip()
except (OSError, json.JSONDecodeError):
pass
if current_session_id and session_id == current_session_id:
continue
if session_id in seen:
continue
seen.add(session_id)
session_ids.append(session_id)
return session_ids
@@ -0,0 +1,128 @@
"""Prompt builder for memory consolidation dreams."""
from __future__ import annotations
from datetime import date
from pathlib import Path
MAX_ENTRYPOINT_LINES = 200
ENTRYPOINT_NAME = "MEMORY.md"
def build_consolidation_prompt(
memory_root: str | Path,
session_dir: str | Path,
extra: str = "",
*,
preview: bool = False,
) -> str:
"""Build the dream prompt used by manual and automatic memory consolidation."""
memory_root = Path(memory_root)
session_dir = Path(session_dir)
extra_section = f"\n\n## Additional context\n\n{extra.strip()}" if extra.strip() else ""
write_mode = "PREVIEW MODE: do not write files; propose a concise patch plan only." if preview else "APPLY MODE: update memory files directly when changes are clearly warranted."
return f"""# Dream: Memory Consolidation
You are performing a dream a reflective pass over OpenHarness/ohmo memory files. Synthesize recent signal into durable, well-organized memories so future sessions can orient quickly.
Current date: {date.today().isoformat()}
Memory directory: `{memory_root}`
Session snapshots: `{session_dir}` (JSON files can be large; inspect narrowly, do not dump everything)
Mode: {write_mode}
---
## Non-negotiable memory policy
### Evidence discipline
- Do not infer user mistakes, motives, personality traits, or habits from incidental logs/config.
- Only record facts directly supported by user statements, repeated behavior, or explicit artifacts.
- Prefer neutral safety policies over accusations.
- If a secret appears in context, do not copy it. Record only a generic safety reminder if useful.
- Never preserve API keys, tokens, app secrets, verification tokens, credential-bearing URLs, or bearer strings.
### Classify every fact before writing
Use these categories:
1. **Stable Preference** user-stated or repeatedly demonstrated durable preference.
2. **Durable Project Context** repo paths, canonical repos, project boundaries, validation commands.
3. **Recent Snapshot** active branches, current commits, temporary worktrees, recent test counts. Must include `Last observed: YYYY-MM-DD` and a reminder to verify current state.
4. **Sensitive/Private Context** revenue, personal identity, private repos, business metrics. Must include `Privacy: personal/private; do not share externally or in group chats unless explicitly asked.`
5. **Operational Reminder** short safety or workflow reminders.
### Staleness and scope
- Short-lived facts must be marked as snapshots, not permanent truths.
- Prefer updating existing files over creating new ones.
- Create at most 2 new markdown files in one dream.
- If a topic is transient, prefer `recent_work.md` or an existing status file over a new topic file.
- Do not move personal/business context into project memory; keep sensitive personal context in personal memory only.
- Every top-level memory file must include schema-v1 frontmatter with:
`schema_version`, `id`, `name`, `description`, `type`, `category`, `importance`, `source`,
`signature`, `created_at`, `updated_at`, `ttl_days`, `disabled`, and `supersedes`.
---
## Phase 1 — Orient
- List the memory directory to see what already exists.
- Read `{ENTRYPOINT_NAME}` if present; it is the memory index.
- Skim existing topic files so you update or merge instead of creating duplicates.
## Phase 2 — Gather recent signal
Look for information worth persisting. Sources in rough priority order:
1. Existing memory files that may need updates or contradiction fixes.
2. Recent session snapshots (`session-*.json`) when you need concrete context.
3. Focused grep/search terms based on recent work; avoid exhaustive transcript reading.
Skip idle chats, failed retry noise, implementation details that only matter for the current turn, and facts that cannot be supported.
## Phase 3 — Consolidate
For each durable thing worth remembering, write or update concise top-level markdown files in the memory directory.
Focus on:
- Merging new signal into existing topic files rather than creating near-duplicates.
- Converting relative dates ("yesterday", "last week") to absolute dates.
- Correcting or deleting contradicted facts at the source.
- Keeping memories useful for future sessions, not as raw transcripts.
- Adding `Last observed` for snapshots and `Privacy` for sensitive/private context.
## Phase 4 — Prune and index
Update `{ENTRYPOINT_NAME}` so it stays under {MAX_ENTRYPOINT_LINES} lines and remains an index, not a content dump.
- Each entry should be one concise line: `- [Title](file.md): one-line hook`.
- Remove pointers to memories that are stale, wrong, or superseded.
- For stale, wrong, or superseded memory files, set `disabled: true`; do not delete files.
- Treat usage-based stale candidates as review candidates, not automatic deletion instructions.
- Add pointers to newly important memories.
- Resolve contradictions if multiple files disagree.
## Required final response
Return a structured summary:
```md
## Dream Summary
Changed:
- file.md: what changed
Confidence:
- High: directly supported facts
- Medium: recent snapshots that should be verified before use
- Low: uncertain/stale candidates not written as facts
Privacy:
- Any private/sensitive context touched and how it is marked
Stale candidates:
- Items that may need review later
```
If nothing changed, say so and explain why.{extra_section}"""
@@ -0,0 +1,313 @@
"""Auto-dream service."""
from __future__ import annotations
import asyncio
import os
import sys
import time
from pathlib import Path
from openharness.config.settings import Settings
from openharness.memory.paths import get_project_memory_dir
from openharness.memory.usage import find_stale_memory_candidates
from openharness.services.autodream.backup import create_memory_backup, diff_memory_dirs
from openharness.services.autodream.lock import (
list_sessions_touched_since,
read_last_consolidated_at,
rollback_consolidation_lock,
try_acquire_consolidation_lock,
)
from openharness.services.autodream.prompt import build_consolidation_prompt
from openharness.services.session_storage import get_project_session_dir
from openharness.tasks.manager import get_task_manager
from openharness.tasks.types import TaskRecord
SESSION_SCAN_INTERVAL_SECONDS = 10 * 60
_CHILD_ENV = "OPENHARNESS_AUTODREAM_CHILD"
_last_session_scan_at: dict[str, float] = {}
_listener_registered = False
def _enabled(settings: Settings) -> bool:
return bool(settings.memory.enabled and settings.memory.auto_dream_enabled)
def _has_dream_signal(session_ids: list[str], *, force: bool) -> bool:
"""Return whether recent sessions are worth consolidating."""
if force:
return True
return bool(session_ids)
def _memory_files_mtime_snapshot(memory_dir: Path) -> dict[str, float]:
snapshot: dict[str, float] = {}
for path in memory_dir.glob("*.md"):
try:
snapshot[path.name] = path.stat().st_mtime
except OSError:
continue
return snapshot
def _files_changed_since(memory_dir: Path, before: dict[str, float]) -> list[str]:
changed: list[str] = []
for path in sorted(memory_dir.glob("*.md")):
try:
mtime = path.stat().st_mtime
except OSError:
continue
if before.get(path.name) != mtime:
changed.append(path.name)
return changed
def _ensure_listener_registered() -> None:
global _listener_registered
if _listener_registered:
return
async def _listener(task: TaskRecord) -> None:
if task.type != "dream":
return
prior_raw = task.metadata.get("prior_mtime", "")
memory_dir = task.metadata.get("memory_dir") or None
if not prior_raw:
return
try:
prior_mtime = float(prior_raw)
except ValueError:
return
if task.status in {"failed", "killed"} or task.metadata.get("preview") == "true":
rollback_consolidation_lock(task.cwd, prior_mtime, memory_dir=memory_dir)
get_task_manager().register_completion_listener(_listener)
_listener_registered = True
def _resolve_memory_dir(cwd: str | Path, memory_dir: str | Path | None) -> Path:
return Path(memory_dir).expanduser().resolve() if memory_dir is not None else get_project_memory_dir(cwd)
def _resolve_session_dir(cwd: str | Path, session_dir: str | Path | None) -> Path:
return Path(session_dir).expanduser().resolve() if session_dir is not None else get_project_session_dir(cwd)
async def start_dream_now(
*,
cwd: str | Path,
settings: Settings,
model: str | None = None,
current_session_id: str | None = None,
force: bool = False,
memory_dir: str | Path | None = None,
session_dir: str | Path | None = None,
app_label: str = "openharness",
runner_module: str = "openharness",
preview: bool = False,
) -> TaskRecord | None:
"""Start a dream task immediately, optionally bypassing time/session gates."""
if os.environ.get(_CHILD_ENV):
return None
if not settings.memory.enabled:
return None
cwd = Path(cwd).resolve()
resolved_memory_dir = _resolve_memory_dir(cwd, memory_dir)
resolved_session_dir = _resolve_session_dir(cwd, session_dir)
last_at = read_last_consolidated_at(cwd, memory_dir=resolved_memory_dir)
session_ids = list_sessions_touched_since(
cwd,
last_at,
current_session_id=current_session_id,
session_dir=resolved_session_dir,
)
if not force:
hours_since = (time.time() - last_at) / 3600
if hours_since < settings.memory.auto_dream_min_hours:
return None
if len(session_ids) < settings.memory.auto_dream_min_sessions:
return None
if not _has_dream_signal(session_ids, force=force):
return None
prior_mtime = try_acquire_consolidation_lock(cwd, memory_dir=resolved_memory_dir)
if prior_mtime is None:
return None
_ensure_listener_registered()
resolved_memory_dir.mkdir(parents=True, exist_ok=True)
resolved_session_dir.mkdir(parents=True, exist_ok=True)
before = _memory_files_mtime_snapshot(resolved_memory_dir)
backup_dir = create_memory_backup(resolved_memory_dir, app_label=app_label) if not preview else None
stale_candidates = find_stale_memory_candidates(cwd, memory_dir=resolved_memory_dir)
stale_section = "\n".join(
f"- {header.id or header.path.name}: {header.path.name} "
f"(importance={header.importance}, updated_at={header.updated_at or 'unknown'})"
for header in stale_candidates[:20]
) or "- (none)"
extra = (
f"Application context: `{app_label}`.\n"
"Tool constraints for this run: only modify files under the memory directory. "
"Use shell commands only for read-only inspection.\n\n"
f"Sessions since last consolidation ({len(session_ids)}):\n"
+ "\n".join(f"- {session_id}" for session_id in session_ids)
+ "\n\nUsage-based stale candidates:\n"
+ stale_section
)
prompt = build_consolidation_prompt(resolved_memory_dir, resolved_session_dir, extra, preview=preview)
src_root = Path(__file__).resolve().parents[3]
existing_pythonpath = os.environ.get("PYTHONPATH", "")
env = {
_CHILD_ENV: "1",
"OPENHARNESS_AUTODREAM_MEMORY_DIR": str(resolved_memory_dir),
"OPENHARNESS_CONFIG_DIR": str(Path.home() / ".openharness"),
"OPENHARNESS_PROFILE": settings.active_profile,
"PYTHONPATH": str(src_root) + ((os.pathsep + existing_pythonpath) if existing_pythonpath else ""),
}
try:
argv = [
sys.executable,
"-m",
runner_module,
]
if runner_module == "openharness":
argv.append("--dangerously-skip-permissions")
if runner_module == "ohmo":
workspace = resolved_memory_dir.parent
argv.extend(["--workspace", str(workspace)])
if settings.active_profile:
argv.extend(["--profile", settings.active_profile])
if model:
argv.extend(["--model", model])
if runner_module == "openharness" and settings.provider != "anthropic_claude":
if settings.base_url:
argv.extend(["--base-url", settings.base_url])
if settings.api_format:
argv.extend(["--api-format", settings.api_format])
try:
auth = settings.resolve_auth()
if runner_module == "openharness" and auth.auth_kind == "api_key":
argv.extend(["--api-key", auth.value])
elif runner_module == "ohmo" and auth.auth_kind == "api_key":
env["OPENHARNESS_API_KEY"] = auth.value
elif auth.value:
env["ANTHROPIC_AUTH_TOKEN"] = auth.value
env.pop("ANTHROPIC_API_KEY", None)
env.pop("OPENAI_API_KEY", None)
env.pop("OPENHARNESS_API_KEY", None)
except Exception:
pass
argv.extend(["--print", prompt])
task = await get_task_manager().create_shell_task(
description="dreaming",
cwd=cwd,
task_type="dream",
env=env,
argv=argv,
)
task.prompt = prompt
except Exception:
rollback_consolidation_lock(cwd, prior_mtime, memory_dir=resolved_memory_dir)
raise
task.metadata.update(
{
"phase": "starting",
"sessions_reviewing": str(len(session_ids)),
"prior_mtime": str(prior_mtime),
"memory_dir": str(resolved_memory_dir),
"session_dir": str(resolved_session_dir),
"force": str(force).lower(),
"app_label": app_label,
"runner_module": runner_module,
"preview": str(preview).lower(),
"backup_dir": str(backup_dir or ""),
}
)
async def _mark_changed_on_completion(done: TaskRecord) -> None:
if done.id != task.id or done.status != "completed":
return
changed = _files_changed_since(resolved_memory_dir, before)
if backup_dir is not None:
diff = diff_memory_dirs(backup_dir, resolved_memory_dir)
done.metadata["files_added"] = "\n".join(diff["added"])
done.metadata["files_changed"] = "\n".join(diff["changed"])
done.metadata["files_removed"] = "\n".join(diff["removed"])
if changed:
done.metadata["phase"] = "updating"
done.metadata["files_touched"] = "\n".join(changed)
get_task_manager().register_completion_listener(_mark_changed_on_completion)
return task
async def execute_auto_dream(
*,
cwd: str | Path,
settings: Settings,
model: str | None = None,
current_session_id: str | None = None,
memory_dir: str | Path | None = None,
session_dir: str | Path | None = None,
app_label: str = "openharness",
runner_module: str = "openharness",
preview: bool = False,
) -> TaskRecord | None:
"""Run the cheap auto-dream gates and start a background dream when eligible."""
if os.environ.get(_CHILD_ENV):
return None
if not _enabled(settings):
return None
cwd = Path(cwd).resolve()
resolved_memory_dir = _resolve_memory_dir(cwd, memory_dir)
resolved_session_dir = _resolve_session_dir(cwd, session_dir)
last_at = read_last_consolidated_at(cwd, memory_dir=resolved_memory_dir)
hours_since = (time.time() - last_at) / 3600
if hours_since < settings.memory.auto_dream_min_hours:
return None
key = str(resolved_memory_dir)
now = time.time()
if now - _last_session_scan_at.get(key, 0) < SESSION_SCAN_INTERVAL_SECONDS:
return None
_last_session_scan_at[key] = now
session_ids = list_sessions_touched_since(
cwd,
last_at,
current_session_id=current_session_id,
session_dir=resolved_session_dir,
)
if len(session_ids) < settings.memory.auto_dream_min_sessions:
return None
if not _has_dream_signal(session_ids, force=False):
return None
return await start_dream_now(
cwd=cwd,
settings=settings,
model=model,
current_session_id=current_session_id,
force=False,
memory_dir=resolved_memory_dir,
session_dir=resolved_session_dir,
app_label=app_label,
runner_module=runner_module,
preview=preview,
)
def schedule_auto_dream(**kwargs: object) -> None:
"""Fire-and-forget auto-dream scheduling."""
try:
loop = asyncio.get_running_loop()
except RuntimeError:
return
loop.create_task(execute_auto_dream(**kwargs)) # type: ignore[arg-type]
+25 -1
View File
@@ -890,6 +890,28 @@ def _build_session_memory_message(messages: list[ConversationMessage]) -> Conver
)
def _build_file_session_memory_message(metadata: dict[str, Any] | None) -> ConversationMessage | None:
"""Build a compaction message from the persisted session-memory file."""
if not metadata:
return None
path = metadata.get("session_memory_path")
if not path:
return None
try:
from openharness.services.session_memory import (
get_session_memory_content,
session_memory_to_compact_text,
)
text = session_memory_to_compact_text(get_session_memory_content(str(path)))
except Exception:
return None
if not text.strip():
return None
return ConversationMessage.from_user_text(text)
def try_session_memory_compaction(
messages: list[ConversationMessage],
*,
@@ -901,7 +923,8 @@ def try_session_memory_compaction(
if len(messages) <= preserve_recent + 4:
return None
older, newer = _split_preserving_tool_pairs(messages, preserve_recent=preserve_recent)
summary_message = _build_session_memory_message(older)
file_summary_message = _build_file_session_memory_message(metadata)
summary_message = file_summary_message or _build_session_memory_message(older)
if summary_message is None:
return None
provisional = [summary_message, *newer]
@@ -917,6 +940,7 @@ def try_session_memory_compaction(
"pre_compact_token_count": estimate_message_tokens(messages),
"preserve_recent": preserve_recent,
"used_session_memory": True,
"used_file_session_memory": file_summary_message is not None,
"pre_compact_discovered_tools": _extract_discovered_tools(older),
"attachments": _extract_attachment_paths(older),
}
+24 -4
View File
@@ -4,6 +4,7 @@ from __future__ import annotations
import json
from datetime import datetime, timezone
from zoneinfo import ZoneInfo
from pathlib import Path
from typing import Any
@@ -44,9 +45,28 @@ def validate_cron_expression(expression: str) -> bool:
return croniter.is_valid(expression)
def next_run_time(expression: str, base: datetime | None = None) -> datetime:
"""Return the next run time for a cron expression."""
def validate_timezone(tz: str | None) -> bool:
"""Return True if *tz* is a valid IANA timezone or empty."""
if not tz:
return True
try:
ZoneInfo(tz)
except Exception:
return False
return True
def next_run_time(expression: str, base: datetime | None = None, tz: str | None = None) -> datetime:
"""Return the next run time for a cron expression.
The returned datetime is always UTC. If *tz* is provided, the cron expression
is interpreted in that IANA timezone.
"""
base = base or datetime.now(timezone.utc)
if tz:
local_base = base.astimezone(ZoneInfo(tz))
local_next = croniter(expression, local_base).get_next(datetime)
return local_next.astimezone(timezone.utc)
return croniter(expression, base).get_next(datetime)
@@ -61,7 +81,7 @@ def upsert_cron_job(job: dict[str, Any]) -> None:
schedule = job.get("schedule", "")
if validate_cron_expression(schedule):
job["next_run"] = next_run_time(schedule).isoformat()
job["next_run"] = next_run_time(schedule, tz=job.get("timezone") or job.get("tz")).isoformat()
with exclusive_file_lock(_cron_lock_path()):
jobs = [existing for existing in load_cron_jobs() if existing.get("name") != job.get("name")]
@@ -112,6 +132,6 @@ def mark_job_run(name: str, *, success: bool) -> None:
job["last_status"] = "success" if success else "failed"
schedule = job.get("schedule", "")
if validate_cron_expression(schedule):
job["next_run"] = next_run_time(schedule, now).isoformat()
job["next_run"] = next_run_time(schedule, now, tz=job.get("timezone") or job.get("tz")).isoformat()
save_cron_jobs(jobs)
return
+270 -33
View File
@@ -9,17 +9,22 @@ log.
from __future__ import annotations
import asyncio
import contextlib
import json
import logging
import os
import subprocess
import shlex
import signal
import sys
import time
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
from types import FrameType
from typing import Any, Callable
from openharness.config.paths import get_data_dir, get_logs_dir
from openharness.platforms import get_platform
from openharness.services.cron import (
load_cron_jobs,
mark_job_run,
@@ -28,6 +33,14 @@ from openharness.services.cron import (
from openharness.sandbox import SandboxUnavailableError
from openharness.utils.shell import create_shell_subprocess
try:
from ohmo.gateway.config import load_gateway_config
except Exception: # pragma: no cover - ohmo is optional for non-ohmo cron users
load_gateway_config = None # type: ignore[assignment]
NOTIFICATION_OUTPUT_LIMIT = 3500
logger = logging.getLogger(__name__)
TICK_INTERVAL_SECONDS = 30
@@ -89,10 +102,7 @@ def read_pid() -> int | None:
pid = int(path.read_text(encoding="utf-8").strip())
except (ValueError, OSError):
return None
# Check if process is alive
try:
os.kill(pid, 0)
except OSError:
if not _pid_exists(pid):
logger.debug("Removed stale scheduler PID file (pid=%d)", pid)
path.unlink(missing_ok=True)
return None
@@ -122,37 +132,205 @@ def stop_scheduler() -> bool:
if pid is None:
return False
try:
os.kill(pid, signal.SIGTERM)
_terminate_pid(pid)
except OSError:
remove_pid()
return False
# Wait briefly for process to exit
for _ in range(10):
try:
os.kill(pid, 0)
except OSError:
if not _pid_exists(pid):
remove_pid()
return True
time.sleep(0.2)
# Force kill
try:
os.kill(pid, signal.SIGKILL)
_kill_pid(pid)
except OSError:
pass
remove_pid()
return True
def _pid_exists(pid: int) -> bool:
"""Return True when *pid* currently refers to a live process."""
if pid <= 0:
return False
if get_platform() == "windows":
return _windows_pid_exists(pid)
try:
os.kill(pid, 0)
except ProcessLookupError:
return False
except PermissionError:
return True
except OSError:
return False
return True
def _windows_pid_exists(pid: int) -> bool:
"""Windows implementation of ``kill(pid, 0)`` without requiring psutil."""
try:
import ctypes
except Exception:
return _pid_exists_with_kill_zero(pid)
try:
kernel32 = ctypes.WinDLL("kernel32", use_last_error=True) # type: ignore[attr-defined]
except AttributeError:
return _pid_exists_with_kill_zero(pid)
synchronize = 0x00100000
process_query_limited_information = 0x1000
wait_timeout = 0x00000102
handle = kernel32.OpenProcess(
synchronize | process_query_limited_information,
False,
pid,
)
if not handle:
return ctypes.get_last_error() == 5 # ERROR_ACCESS_DENIED: process exists
try:
return kernel32.WaitForSingleObject(handle, 0) == wait_timeout
finally:
kernel32.CloseHandle(handle)
def _pid_exists_with_kill_zero(pid: int) -> bool:
try:
os.kill(pid, 0)
except ProcessLookupError:
return False
except PermissionError:
return True
except OSError:
return False
return True
def _terminate_pid(pid: int) -> None:
os.kill(pid, signal.SIGTERM)
def _kill_pid(pid: int) -> None:
if get_platform() == "windows":
os.kill(pid, signal.SIGTERM)
return
os.kill(pid, signal.SIGKILL)
# ---------------------------------------------------------------------------
# Job execution
# ---------------------------------------------------------------------------
def _format_notification(job: dict[str, Any], entry: dict[str, Any]) -> str:
"""Build a concise notification body for a completed cron job."""
status = entry.get("status", "?")
rc = entry.get("returncode", "?")
lines = [
f"⏰ Cron job finished: {job.get('name', '?')}",
f"Status: {status} (rc={rc})",
f"Started: {entry.get('started_at', '?')}",
f"Ended: {entry.get('ended_at', '?')}",
]
stdout = str(entry.get("stdout") or "").strip()
stderr = str(entry.get("stderr") or "").strip()
if stdout:
lines.extend(["", "Output:", stdout[-NOTIFICATION_OUTPUT_LIMIT:]])
if stderr:
lines.extend(["", "Stderr:", stderr[-NOTIFICATION_OUTPUT_LIMIT:]])
if not stdout and not stderr:
lines.extend(["", "(no output)"])
return "\n".join(lines)
async def _notify_job_result(job: dict[str, Any], entry: dict[str, Any]) -> None:
"""Deliver an optional post-run notification for a cron job."""
notify = job.get("notify")
payload = job.get("payload")
if not isinstance(notify, dict) and isinstance(payload, dict) and payload.get("deliver"):
notify = {"type": payload.get("channel"), "to": payload.get("to")}
if not isinstance(notify, dict):
return
notify_type = str(notify.get("type") or "").strip().lower()
try:
if notify_type in {"feishu_dm", "feishu"}:
from ohmo.gateway.notify import send_feishu_dm
user_open_id = str(
notify.get("user_open_id") or notify.get("open_id") or notify.get("to") or ""
).strip()
if not user_open_id:
raise ValueError("missing notify.user_open_id")
workspace = notify.get("workspace")
await send_feishu_dm(
user_open_id=user_open_id,
content=_format_notification(job, entry),
workspace=str(workspace) if workspace else None,
)
elif notify_type:
raise ValueError(f"unsupported notify.type: {notify_type}")
except Exception as exc:
logger.error("Failed to notify cron job %r result: %s", job.get("name"), exc)
entry["notification_status"] = "failed"
entry["notification_error"] = str(exc)
else:
entry["notification_status"] = "sent"
def _command_for_job(job: dict[str, Any]) -> str:
"""Return the shell command used to execute a job."""
command = job.get("command")
if command:
return str(command)
payload = job.get("payload")
if not isinstance(payload, dict) or payload.get("kind", "agent_turn") != "agent_turn":
raise ValueError("cron job has no command or agent_turn payload")
message = str(payload.get("message") or "").strip()
if not message:
raise ValueError("agent_turn cron job is missing payload.message")
cwd = str(job.get("cwd") or ".")
parts = ["ohmo"]
profile = payload.get("profile") or job.get("provider_profile")
if profile is None and load_gateway_config is not None:
profile = load_gateway_config().provider_profile
if profile:
parts.extend(["--profile", str(profile)])
parts.extend(
[
"--cwd",
cwd,
"--print",
message,
]
)
return " ".join(shlex.quote(part) for part in parts)
async def execute_job(job: dict[str, Any]) -> dict[str, Any]:
"""Run a single cron job and return a history entry."""
name = job["name"]
command = job["command"]
cwd = Path(job.get("cwd") or ".").expanduser()
started_at = datetime.now(timezone.utc)
try:
command = _command_for_job(job)
except Exception as exc:
entry = {
"name": name,
"command": "",
"started_at": started_at.isoformat(),
"ended_at": datetime.now(timezone.utc).isoformat(),
"returncode": -1,
"status": "error",
"stdout": "",
"stderr": str(exc),
}
mark_job_run(name, success=False)
await _notify_job_result(job, entry)
append_history(entry)
return entry
logger.info("Executing cron job %r: %s", name, command)
try:
@@ -183,6 +361,7 @@ async def execute_job(job: dict[str, Any]) -> dict[str, Any]:
"stderr": "Job timed out after 300s",
}
mark_job_run(name, success=False)
await _notify_job_result(job, entry)
append_history(entry)
return entry
except SandboxUnavailableError as exc:
@@ -197,6 +376,7 @@ async def execute_job(job: dict[str, Any]) -> dict[str, Any]:
"stderr": str(exc),
}
mark_job_run(name, success=False)
await _notify_job_result(job, entry)
append_history(entry)
return entry
except Exception as exc:
@@ -211,6 +391,7 @@ async def execute_job(job: dict[str, Any]) -> dict[str, Any]:
"stderr": str(exc),
}
mark_job_run(name, success=False)
await _notify_job_result(job, entry)
append_history(entry)
return entry
@@ -226,6 +407,7 @@ async def execute_job(job: dict[str, Any]) -> dict[str, Any]:
"stderr": (stderr.decode("utf-8", errors="replace")[-2000:] if stderr else ""),
}
mark_job_run(name, success=success)
await _notify_job_result(job, entry)
append_history(entry)
logger.info("Job %r finished: %s (rc=%s)", name, entry["status"], process.returncode)
return entry
@@ -262,13 +444,8 @@ async def run_scheduler_loop(*, once: bool = False) -> None:
"""Main scheduler loop. Runs until SIGTERM or *once* is True (test mode)."""
shutdown = asyncio.Event()
def _on_signal() -> None:
logger.info("Received shutdown signal")
shutdown.set()
loop = asyncio.get_running_loop()
for sig in (signal.SIGTERM, signal.SIGINT):
loop.add_signal_handler(sig, _on_signal)
restore_signals = _install_shutdown_signal_handlers(loop, shutdown)
write_pid()
logger.info("Cron scheduler started (pid=%d, tick=%ds)", os.getpid(), TICK_INTERVAL_SECONDS)
@@ -297,10 +474,45 @@ async def run_scheduler_loop(*, once: bool = False) -> None:
except asyncio.TimeoutError:
pass
finally:
restore_signals()
remove_pid()
logger.info("Cron scheduler stopped")
def _install_shutdown_signal_handlers(
loop: asyncio.AbstractEventLoop,
shutdown: asyncio.Event,
) -> Callable[[], None]:
"""Install portable signal handlers and return a restore callback."""
previous_handlers: list[tuple[signal.Signals, Any]] = []
def _on_signal(signum: int, frame: FrameType | None) -> None:
del frame
logger.info("Received shutdown signal (%s)", signum)
with contextlib.suppress(RuntimeError):
loop.call_soon_threadsafe(shutdown.set)
signals: list[signal.Signals] = [signal.SIGTERM, signal.SIGINT]
sigbreak = getattr(signal, "SIGBREAK", None)
if sigbreak is not None:
signals.append(sigbreak)
for sig in signals:
try:
previous = signal.getsignal(sig)
signal.signal(sig, _on_signal)
except (OSError, RuntimeError, ValueError):
continue
previous_handlers.append((sig, previous))
def _restore() -> None:
for sig, previous in reversed(previous_handlers):
with contextlib.suppress(OSError, RuntimeError, ValueError):
signal.signal(sig, previous)
return _restore
# ---------------------------------------------------------------------------
# Daemon entry point (spawned by ``oh cron start``)
# ---------------------------------------------------------------------------
@@ -318,28 +530,49 @@ def _run_daemon() -> None:
def start_daemon() -> int:
"""Fork and start the scheduler daemon. Returns the child PID."""
"""Start the scheduler daemon and return its PID."""
existing = read_pid()
if existing is not None:
raise RuntimeError(f"Scheduler already running (pid={existing})")
pid = os.fork()
if pid > 0:
# Parent — wait a moment for the child to write its PID file
time.sleep(0.3)
return pid
process = _spawn_scheduler_process()
deadline = time.monotonic() + 2.0
while time.monotonic() < deadline:
pid = read_pid()
if pid is not None:
return pid
if process.poll() is not None:
break
time.sleep(0.1)
# Child — detach
os.setsid()
# Redirect stdio
devnull = os.open(os.devnull, os.O_RDWR)
os.dup2(devnull, 0)
os.dup2(devnull, 1)
os.dup2(devnull, 2)
os.close(devnull)
if process.poll() is not None:
log_file = get_logs_dir() / "cron_scheduler.log"
raise RuntimeError(f"Cron scheduler failed to start; see {log_file}")
return process.pid
_run_daemon()
sys.exit(0)
def _spawn_scheduler_process() -> subprocess.Popen[bytes]:
"""Spawn a detached scheduler subprocess on Unix and Windows."""
popen_kwargs: dict[str, Any] = {
"stdin": subprocess.DEVNULL,
"stdout": subprocess.DEVNULL,
"stderr": subprocess.DEVNULL,
"close_fds": True,
}
if get_platform() == "windows":
creationflags = getattr(subprocess, "DETACHED_PROCESS", 0) | getattr(
subprocess,
"CREATE_NEW_PROCESS_GROUP",
0,
)
if creationflags:
popen_kwargs["creationflags"] = creationflags
else:
popen_kwargs["start_new_session"] = True
return subprocess.Popen(
[sys.executable, "-m", "openharness.services.cron_scheduler"],
**popen_kwargs,
)
def scheduler_status() -> dict[str, Any]:
@@ -356,3 +589,7 @@ def scheduler_status() -> dict[str, Any]:
"log_file": str(log_path),
"history_file": str(get_history_path()),
}
if __name__ == "__main__":
_run_daemon()
@@ -0,0 +1,261 @@
"""Durable memory extraction from completed turns."""
from __future__ import annotations
import json
import logging
from dataclasses import dataclass
from pathlib import Path
from typing import Any
from openharness.api.client import ApiMessageCompleteEvent, ApiMessageRequest, SupportsStreamingMessages
from openharness.engine.messages import ConversationMessage, ToolUseBlock
from openharness.memory.manager import add_memory_entry
from openharness.memory.paths import get_project_memory_dir
from openharness.memory.relevance import build_memory_manifest
from openharness.memory.scan import scan_memory_files
from openharness.memory.schema import (
DEFAULT_MEMORY_SCOPE,
DEFAULT_MEMORY_TYPE,
MemoryScope,
MemoryType,
parse_memory_scope,
parse_memory_type,
)
from openharness.memory.team import check_team_memory_secrets, validate_team_memory_write_path
log = logging.getLogger(__name__)
MEMORY_WRITE_TOOLS = {"write_file", "edit_file"}
@dataclass(frozen=True)
class ExtractionRecord:
"""Structured memory record proposed by the extraction pass."""
title: str
body: str
memory_type: MemoryType = DEFAULT_MEMORY_TYPE
scope: MemoryScope = DEFAULT_MEMORY_SCOPE
description: str = ""
tags: tuple[str, ...] = ()
@dataclass(frozen=True)
class ExtractionResult:
"""Outcome of a durable memory extraction run."""
skipped: bool
reason: str = ""
records: tuple[ExtractionRecord, ...] = ()
written_paths: tuple[Path, ...] = ()
def has_memory_writes_since(
messages: list[ConversationMessage],
memory_dir: str | Path,
*,
cwd: str | Path | None = None,
) -> bool:
"""Return whether the visible turn already wrote memory files."""
root = Path(memory_dir).expanduser().resolve()
write_base = Path(cwd).expanduser().resolve() if cwd is not None else root
for message in messages:
for block in message.content:
if not isinstance(block, ToolUseBlock):
continue
if block.name not in MEMORY_WRITE_TOOLS:
continue
raw_path = block.input.get("path") or block.input.get("file_path")
if not raw_path:
continue
path = Path(str(raw_path)).expanduser()
if not path.is_absolute():
path = write_base / path
try:
path.resolve().relative_to(root)
except ValueError:
continue
return True
return False
async def extract_memories_from_turn(
*,
cwd: str | Path,
api_client: SupportsStreamingMessages,
model: str,
messages: list[ConversationMessage],
max_records: int = 3,
) -> ExtractionResult:
"""Ask the model for durable memory candidates and apply them."""
memory_dir = get_project_memory_dir(cwd)
if len(messages) < 2:
return ExtractionResult(skipped=True, reason="not enough messages")
if has_memory_writes_since(messages, memory_dir, cwd=cwd):
return ExtractionResult(skipped=True, reason="main conversation already wrote memory")
prompt = build_extraction_prompt(cwd, messages, max_records=max_records)
final_text = ""
async for event in api_client.stream_message(
ApiMessageRequest(
model=model,
messages=[ConversationMessage.from_user_text(prompt)],
system_prompt=EXTRACTION_SYSTEM_PROMPT,
max_tokens=2048,
tools=[],
)
):
if isinstance(event, ApiMessageCompleteEvent):
final_text = event.message.text
break
records = parse_extraction_records(final_text, max_records=max_records)
if not records:
return ExtractionResult(skipped=True, reason="no durable memories proposed")
return apply_extraction_records(cwd, records)
def build_extraction_prompt(cwd: str | Path, messages: list[ConversationMessage], *, max_records: int) -> str:
"""Build the extraction request from recent messages and manifest."""
manifest = build_memory_manifest(scan_memory_files(cwd, max_files=80))
transcript = "\n".join(_summarize_message(message) for message in messages[-12:])
return (
"Extract only durable memories from the recent conversation.\n"
f"Return JSON with at most {max_records} records. Existing memory manifest:\n"
f"{manifest or '(empty)'}\n\n"
"Recent conversation:\n"
f"{transcript}\n\n"
"JSON schema: {\"memories\":[{\"title\":\"...\",\"type\":\"user|feedback|project|reference\","
"\"scope\":\"private|project|team\",\"description\":\"...\",\"body\":\"...\",\"tags\":[\"...\"]}]}"
)
EXTRACTION_SYSTEM_PROMPT = """You maintain OpenHarness durable memory.
Save only stable, future-useful facts that are not derivable from current files,
git history, or documentation. Prefer updating existing memories conceptually
over duplicating them. Do not save secrets. If nothing is worth saving, return
{"memories": []}.
"""
def parse_extraction_records(text: str, *, max_records: int = 3) -> tuple[ExtractionRecord, ...]:
"""Parse JSON memory extraction output."""
try:
payload = json.loads(_extract_json_object(text))
except json.JSONDecodeError:
return ()
raw_records = payload.get("memories") if isinstance(payload, dict) else None
if not isinstance(raw_records, list):
return ()
records: list[ExtractionRecord] = []
for item in raw_records[:max_records]:
if not isinstance(item, dict):
continue
title = str(item.get("title") or "").strip()
body = str(item.get("body") or "").strip()
if not title or not body:
continue
memory_type = parse_memory_type(item.get("type"), default=DEFAULT_MEMORY_TYPE) or DEFAULT_MEMORY_TYPE
scope = parse_memory_scope(item.get("scope"), default=DEFAULT_MEMORY_SCOPE) or DEFAULT_MEMORY_SCOPE
tags_raw = item.get("tags") or ()
tags = tuple(str(tag).strip() for tag in tags_raw if str(tag).strip()) if isinstance(tags_raw, list) else ()
records.append(
ExtractionRecord(
title=title,
body=body,
memory_type=memory_type,
scope=scope,
description=str(item.get("description") or "").strip(),
tags=tags,
)
)
return tuple(records)
def apply_extraction_records(cwd: str | Path, records: tuple[ExtractionRecord, ...]) -> ExtractionResult:
"""Write accepted records to durable memory."""
written: list[Path] = []
for record in records:
if record.scope == "team":
secret_error = check_team_memory_secrets(record.body)
if secret_error:
log.warning("memory extraction skipped team record %r: %s", record.title, secret_error)
continue
path, error = validate_team_memory_write_path(cwd, f"{record.title}.md")
if error or path is None:
log.warning("memory extraction skipped team record %r: %s", record.title, error)
continue
written.append(
add_memory_entry(
cwd,
record.title,
record.body,
memory_type=record.memory_type,
scope=record.scope,
description=record.description,
tags=record.tags,
)
)
return ExtractionResult(skipped=not bool(written), reason="" if written else "all records rejected", records=records, written_paths=tuple(written))
def validate_extraction_tool_request(tool_name: str, tool_input: dict[str, Any], memory_dir: str | Path) -> tuple[bool, str]:
"""Permission guard for extraction-like agents."""
if tool_name in {"read_file", "grep", "glob"}:
return True, ""
if tool_name == "bash":
command = str(tool_input.get("command") or "")
if _is_read_only_shell(command):
return True, ""
return False, "memory extraction may only run read-only shell commands"
if tool_name in {"write_file", "edit_file"}:
raw_path = str(tool_input.get("path") or tool_input.get("file_path") or "")
if not raw_path:
return False, "memory extraction write requires a path"
root = Path(memory_dir).expanduser().resolve()
path = Path(raw_path).expanduser()
if not path.is_absolute():
path = root / path
try:
path.resolve().relative_to(root)
except ValueError:
return False, f"memory extraction writes must stay within {root}"
return True, ""
return False, f"memory extraction cannot use tool {tool_name}"
def _extract_json_object(text: str) -> str:
stripped = text.strip()
if stripped.startswith("{") and stripped.endswith("}"):
return stripped
start = stripped.find("{")
end = stripped.rfind("}")
if start >= 0 and end > start:
return stripped[start : end + 1]
return stripped
def _summarize_message(message: ConversationMessage) -> str:
text = " ".join(message.text.split())
if text:
return f"{message.role}: {text[:1200]}"
if message.tool_uses:
return f"{message.role}: tool calls -> {', '.join(block.name for block in message.tool_uses)}"
return f"{message.role}: [non-text content]"
def _is_read_only_shell(command: str) -> bool:
lowered = command.strip().lower()
if not lowered:
return False
denied = (" > ", ">>", " rm ", " mv ", " cp ", " sed -i", " tee ", "python -c", "python3 -c")
if any(marker in f" {lowered} " for marker in denied):
return False
first = lowered.split(maxsplit=1)[0]
return first in {"ls", "pwd", "cat", "head", "tail", "rg", "grep", "find", "git", "wc", "sed", "awk", "stat"}
@@ -0,0 +1,139 @@
"""File-backed session memory for compact continuity."""
from __future__ import annotations
from hashlib import sha1
from pathlib import Path
from openharness.config.paths import get_data_dir
from openharness.engine.messages import ConversationMessage, ToolResultBlock
from openharness.services.token_estimation import estimate_tokens
from openharness.utils.fs import atomic_write_text
MAX_SESSION_MEMORY_CHARS = 12_000
MAX_RECENT_LINES = 80
def get_session_memory_dir(cwd: str | Path) -> Path:
"""Return the project session-memory directory."""
root = Path(cwd).resolve()
digest = sha1(str(root).encode("utf-8")).hexdigest()[:12]
path = get_data_dir() / "session-memory" / f"{root.name}-{digest}"
path.mkdir(parents=True, exist_ok=True)
return path
def get_session_memory_path(cwd: str | Path, session_id: str | None = None) -> Path:
"""Return the markdown session-memory path."""
safe_session = "".join(ch if ch.isalnum() or ch in "._-" else "_" for ch in (session_id or "default"))
return get_session_memory_dir(cwd) / f"{safe_session or 'default'}.md"
def prepare_session_memory_metadata(
cwd: str | Path,
tool_metadata: dict[str, object],
*,
session_id: str | None = None,
) -> Path:
"""Ensure metadata points compaction to the session-memory file."""
sid = session_id or str(tool_metadata.get("session_id") or "default")
path = get_session_memory_path(cwd, sid)
tool_metadata["session_memory_path"] = str(path)
return path
def get_session_memory_content(path: str | Path | None) -> str:
"""Read session memory content if available."""
if not path:
return ""
candidate = Path(path).expanduser()
if not candidate.exists():
return ""
try:
return candidate.read_text(encoding="utf-8", errors="replace")
except OSError:
return ""
def update_session_memory_file(
cwd: str | Path,
messages: list[ConversationMessage],
*,
tool_metadata: dict[str, object] | None = None,
session_id: str | None = None,
) -> Path:
"""Update the deterministic session-memory checkpoint."""
path = prepare_session_memory_metadata(cwd, tool_metadata or {}, session_id=session_id)
body = build_session_memory_document(messages, tool_metadata=tool_metadata)
atomic_write_text(path, body)
return path
def build_session_memory_document(
messages: list[ConversationMessage],
*,
tool_metadata: dict[str, object] | None = None,
) -> str:
"""Build a compact markdown checkpoint for the current session."""
state = tool_metadata.get("task_focus_state") if isinstance(tool_metadata, dict) else None
goal = ""
next_step = ""
verified: list[str] = []
artifacts: list[str] = []
if isinstance(state, dict):
goal = str(state.get("goal") or "").strip()
next_step = str(state.get("next_step") or "").strip()
verified = [str(item).strip() for item in state.get("verified_state", []) if str(item).strip()]
artifacts = [str(item).strip() for item in state.get("active_artifacts", []) if str(item).strip()]
lines = ["# Session Memory", ""]
lines.extend(["## Current State", goal or "(no current goal recorded)", ""])
if next_step:
lines.extend(["## Next Step", next_step, ""])
if verified:
lines.extend(["## Verified Work", *[f"- {item}" for item in verified[-10:]], ""])
if artifacts:
lines.extend(["## Active Artifacts", *[f"- {item}" for item in artifacts[-10:]], ""])
lines.extend(["## Recent Conversation", *_recent_message_lines(messages), ""])
text = "\n".join(lines).strip() + "\n"
if len(text) > MAX_SESSION_MEMORY_CHARS:
text = text[:MAX_SESSION_MEMORY_CHARS].rsplit("\n", 1)[0]
text += "\n\n> Session memory was truncated to stay within budget.\n"
return text
def session_memory_to_compact_text(content: str) -> str:
"""Prepare persisted session memory for insertion across compact boundaries."""
stripped = content.strip()
if not stripped:
return ""
if estimate_tokens(stripped) > 4_000:
stripped = stripped[:MAX_SESSION_MEMORY_CHARS].rsplit("\n", 1)[0]
return "Session memory checkpoint from earlier in this conversation:\n" + stripped
def _recent_message_lines(messages: list[ConversationMessage]) -> list[str]:
lines: list[str] = []
for message in messages[-MAX_RECENT_LINES:]:
line = _summarize_message(message)
if line:
lines.append(f"- {line}")
return lines or ["- (no recent messages)"]
def _summarize_message(message: ConversationMessage) -> str:
text = " ".join(message.text.split())
if text:
return f"{message.role}: {text[:220]}"
if message.tool_uses:
return f"{message.role}: tool calls -> {', '.join(block.name for block in message.tool_uses[:6])}"
if any(isinstance(block, ToolResultBlock) for block in message.content):
return f"{message.role}: tool results returned"
return f"{message.role}: [non-text content]"
+13
View File
@@ -60,9 +60,22 @@ _TEAMMATE_ENV_VARS = [
# --- OpenHarness-native provider settings --------------------------------
# These are read by settings._apply_env_overrides() and must survive across
# tmux boundaries so teammates use the same provider as the leader.
"OPENHARNESS_CONFIG_DIR",
"OPENHARNESS_DATA_DIR",
"OPENHARNESS_LOGS_DIR",
"OPENHARNESS_PROFILE",
"OPENHARNESS_API_FORMAT",
"OPENHARNESS_PROVIDER",
"OPENHARNESS_BASE_URL",
"OPENHARNESS_MODEL",
"OPENHARNESS_ANTHROPIC_API_KEY",
"OPENHARNESS_OPENAI_API_KEY",
"OPENHARNESS_DASHSCOPE_API_KEY",
"OPENHARNESS_MOONSHOT_API_KEY",
"OPENHARNESS_GEMINI_API_KEY",
"OPENHARNESS_MINIMAX_API_KEY",
"OPENHARNESS_NVIDIA_API_KEY",
"OPENHARNESS_MODELSCOPE_API_KEY",
"OPENAI_API_KEY",
]
+2
View File
@@ -209,6 +209,7 @@ class BackgroundTaskManager:
task.status = "killed"
task.ended_at = time.time()
await self._notify_completion_listeners(task)
return task
async def write_to_task(self, task_id: str, data: str) -> None:
@@ -458,6 +459,7 @@ def _task_id(task_type: TaskType) -> str:
"local_agent": "a",
"remote_agent": "r",
"in_process_teammate": "t",
"dream": "d",
}
return f"{prefixes[task_type]}{uuid4().hex[:8]}"
+1 -1
View File
@@ -7,7 +7,7 @@ from pathlib import Path
from typing import Literal
TaskType = Literal["local_bash", "local_agent", "remote_agent", "in_process_teammate"]
TaskType = Literal["local_bash", "local_agent", "remote_agent", "in_process_teammate", "dream"]
TaskStatus = Literal["pending", "running", "completed", "failed", "killed"]
+6
View File
@@ -19,6 +19,7 @@ from openharness.tools.file_read_tool import FileReadTool
from openharness.tools.file_write_tool import FileWriteTool
from openharness.tools.glob_tool import GlobTool
from openharness.tools.grep_tool import GrepTool
from openharness.tools.image_generation_tool import ImageGenerationTool
from openharness.tools.image_to_text_tool import ImageToTextTool
from openharness.tools.list_mcp_resources_tool import ListMcpResourcesTool
from openharness.tools.lsp_tool import LspTool
@@ -42,6 +43,7 @@ from openharness.tools.todo_write_tool import TodoWriteTool
from openharness.tools.tool_search_tool import ToolSearchTool
from openharness.tools.web_fetch_tool import WebFetchTool
from openharness.tools.web_search_tool import WebSearchTool
from openharness.tools.workflow_tool import WorkflowCreateTool, WorkflowRunTool, WorkflowStatusTool
def create_default_tool_registry(mcp_manager=None) -> ToolRegistry:
@@ -59,6 +61,7 @@ def create_default_tool_registry(mcp_manager=None) -> ToolRegistry:
GlobTool(),
GrepTool(),
ImageToTextTool(),
ImageGenerationTool(),
SkillTool(),
ToolSearchTool(),
WebFetchTool(),
@@ -86,6 +89,9 @@ def create_default_tool_registry(mcp_manager=None) -> ToolRegistry:
SendMessageTool(),
TeamCreateTool(),
TeamDeleteTool(),
WorkflowCreateTool(),
WorkflowRunTool(),
WorkflowStatusTool(),
):
registry.register(tool)
if mcp_manager is not None:
+19 -2
View File
@@ -2,6 +2,8 @@
from __future__ import annotations
from typing import Any
from pydantic import BaseModel, Field
from openharness.config.settings import load_settings, save_settings
@@ -29,9 +31,24 @@ class ConfigTool(BaseTool):
if arguments.action == "show":
return ToolResult(output=settings.model_dump_json(indent=2))
if arguments.action == "set" and arguments.key and arguments.value is not None:
if not hasattr(settings, arguments.key):
target: Any = settings
parts = arguments.key.split(".")
for part in parts[:-1]:
if not hasattr(target, part):
return ToolResult(output=f"Unknown config key: {arguments.key}", is_error=True)
target = getattr(target, part)
leaf = parts[-1]
if not hasattr(target, leaf):
return ToolResult(output=f"Unknown config key: {arguments.key}", is_error=True)
setattr(settings, arguments.key, arguments.value)
current = getattr(target, leaf)
value: Any = arguments.value
if isinstance(current, bool):
value = arguments.value.strip().lower() in {"1", "true", "yes", "on"}
elif isinstance(current, int) and not isinstance(current, bool):
value = int(arguments.value)
elif isinstance(current, float):
value = float(arguments.value)
setattr(target, leaf, value)
save_settings(settings)
return ToolResult(output=f"Updated {arguments.key}")
return ToolResult(output="Usage: action=show or action=set with key/value", is_error=True)
+53 -11
View File
@@ -2,9 +2,11 @@
from __future__ import annotations
from typing import Any
from pydantic import BaseModel, Field
from openharness.services.cron import upsert_cron_job, validate_cron_expression
from openharness.services.cron import upsert_cron_job, validate_cron_expression, validate_timezone
from openharness.tools.base import BaseTool, ToolExecutionContext, ToolResult
@@ -18,9 +20,25 @@ class CronCreateToolInput(BaseModel):
"'0 9 * * 1-5' for weekdays at 9am)"
),
)
command: str = Field(description="Shell command to run when triggered")
command: str | None = Field(default=None, description="Shell command to run when triggered")
message: str | None = Field(default=None, description="Instruction for an agent_turn cron job")
timezone: str | None = Field(default=None, description="IANA timezone for interpreting cron schedule")
cwd: str | None = Field(default=None, description="Optional working directory override")
enabled: bool = Field(default=True, description="Whether the job is active")
payload: dict[str, Any] | None = Field(
default=None,
description=(
"Optional nanobot-style payload. Example: "
"{'kind': 'agent_turn', 'message': 'check GitHub', 'deliver': True, 'channel': 'feishu', 'to': 'ou_xxx'}."
),
)
notify: dict[str, Any] | None = Field(
default=None,
description=(
"Optional notification target. Example: "
"{'type': 'feishu_dm', 'user_open_id': 'ou_xxx'} to send job output to a Feishu private chat."
),
)
class CronCreateTool(BaseTool):
@@ -47,16 +65,40 @@ class CronCreateTool(BaseTool):
),
is_error=True,
)
if not validate_timezone(arguments.timezone):
return ToolResult(output=f"Invalid timezone: {arguments.timezone!r}", is_error=True)
upsert_cron_job(
{
"name": arguments.name,
"schedule": arguments.schedule,
"command": arguments.command,
"cwd": arguments.cwd or str(context.cwd),
"enabled": arguments.enabled,
}
)
payload = dict(arguments.payload or {})
if arguments.message:
payload.setdefault("kind", "agent_turn")
payload.setdefault("message", arguments.message)
if arguments.notify is not None:
payload.setdefault("deliver", True)
if str(arguments.notify.get("type") or "").strip().lower() == "feishu_dm":
payload.setdefault("channel", "feishu")
payload.setdefault("to", arguments.notify.get("user_open_id") or arguments.notify.get("open_id"))
if payload and not payload.get("message") and not arguments.command:
return ToolResult(output="Cron job requires payload.message, message, or command.", is_error=True)
if not payload and not arguments.command:
return ToolResult(output="Cron job requires command or message.", is_error=True)
job = {
"name": arguments.name,
"schedule": arguments.schedule,
"cwd": arguments.cwd or str(context.cwd),
"enabled": arguments.enabled,
}
if arguments.timezone:
job["timezone"] = arguments.timezone
if arguments.command is not None:
job["command"] = arguments.command
if payload:
payload.setdefault("kind", "agent_turn")
job["payload"] = payload
if arguments.notify is not None:
job["notify"] = arguments.notify
upsert_cron_job(job)
status = "enabled" if arguments.enabled else "disabled"
return ToolResult(
output=f"Created cron job '{arguments.name}' [{arguments.schedule}] ({status})"
+16 -2
View File
@@ -47,9 +47,23 @@ class CronListTool(BaseTool):
next_run = next_run[:19]
last_status = job.get("last_status", "")
status_str = f" ({last_status})" if last_status else ""
notify = job.get("notify")
notify_line = ""
if isinstance(notify, dict):
notify_type = notify.get("type", "?")
target = notify.get("user_open_id") or notify.get("open_id") or notify.get("chat_id") or "?"
notify_line = f"\n notify: {notify_type} -> {target}"
timezone = f" ({job['timezone']})" if job.get("timezone") else ""
payload = job.get("payload")
payload_line = ""
if isinstance(payload, dict):
payload_line = f"\n payload: {payload.get('kind', 'agent_turn')} -> {payload.get('channel', '?')}:{payload.get('to', '?')}"
command = job.get("command") or "(agent_turn)"
lines.append(
f"[{enabled}] {job['name']} {job.get('schedule', '?')}\n"
f" cmd: {job['command']}\n"
f"[{enabled}] {job['name']} {job.get('schedule', '?')}{timezone}\n"
f" cmd: {command}"
f"{payload_line}"
f"{notify_line}\n"
f" last: {last_run}{status_str} next: {next_run}"
)
return ToolResult(output="\n".join(lines))
+31
View File
@@ -2,6 +2,7 @@
from __future__ import annotations
import difflib
from pathlib import Path
from pydantic import BaseModel, Field
@@ -53,6 +54,16 @@ class FileEditTool(BaseTool):
else:
updated = original.replace(arguments.old_str, arguments.new_str, 1)
approval_prompt = context.metadata.get("edit_approval_prompt") if context.metadata else None
if approval_prompt is not None:
diff_text, added, removed = _compute_diff(str(path), original, updated)
reply = await approval_prompt(str(path), diff_text, added, removed)
if reply == "reject":
return ToolResult(output=f"Edit rejected by user: {path}", is_error=True)
path.write_text(updated, encoding="utf-8")
stats = f" ({_ANSI_GREEN}+{added}{_ANSI_RESET} {_ANSI_RED}-{removed}{_ANSI_RESET})"
return ToolResult(output=f"Updated {path}{stats}")
path.write_text(updated, encoding="utf-8")
return ToolResult(output=f"Updated {path}")
@@ -62,3 +73,23 @@ def _resolve_path(base: Path, candidate: str) -> Path:
if not path.is_absolute():
path = base / path
return path.resolve()
def _compute_diff(filename: str, original: str, updated: str) -> tuple[str, int, int]:
diff_lines = list(
difflib.unified_diff(
original.splitlines(keepends=True),
updated.splitlines(keepends=True),
fromfile=filename,
tofile=filename,
lineterm="",
)
)
added = sum(1 for line in diff_lines if line.startswith("+") and not line.startswith("+++"))
removed = sum(1 for line in diff_lines if line.startswith("-") and not line.startswith("---"))
return "".join(diff_lines), added, removed
_ANSI_GREEN = "\033[32m"
_ANSI_RED = "\033[31m"
_ANSI_RESET = "\033[0m"
+34
View File
@@ -2,6 +2,7 @@
from __future__ import annotations
import difflib
from pathlib import Path
from pydantic import BaseModel, Field
@@ -40,6 +41,19 @@ class FileWriteTool(BaseTool):
if not allowed:
return ToolResult(output=f"Sandbox: {reason}", is_error=True)
approval_prompt = context.metadata.get("edit_approval_prompt") if context.metadata else None
if approval_prompt is not None:
original = path.read_text(encoding="utf-8") if path.exists() else ""
diff_text, added, removed = _compute_diff(str(path), original, arguments.content)
reply = await approval_prompt(str(path), diff_text, added, removed)
if reply == "reject":
return ToolResult(output=f"Write rejected by user: {path}", is_error=True)
if arguments.create_directories:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(arguments.content, encoding="utf-8")
stats = f" ({_ANSI_GREEN}+{added}{_ANSI_RESET} {_ANSI_RED}-{removed}{_ANSI_RESET})"
return ToolResult(output=f"Wrote {path}{stats}")
if arguments.create_directories:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(arguments.content, encoding="utf-8")
@@ -51,3 +65,23 @@ def _resolve_path(base: Path, candidate: str) -> Path:
if not path.is_absolute():
path = base / path
return path.resolve()
def _compute_diff(filename: str, original: str, updated: str) -> tuple[str, int, int]:
diff_lines = list(
difflib.unified_diff(
original.splitlines(keepends=True),
updated.splitlines(keepends=True),
fromfile=filename,
tofile=filename,
lineterm="",
)
)
added = sum(1 for line in diff_lines if line.startswith("+") and not line.startswith("+++"))
removed = sum(1 for line in diff_lines if line.startswith("-") and not line.startswith("---"))
return "".join(diff_lines), added, removed
_ANSI_GREEN = "\033[32m"
_ANSI_RED = "\033[31m"
_ANSI_RESET = "\033[0m"
+12 -1
View File
@@ -16,7 +16,10 @@ class GrepToolInput(BaseModel):
"""Arguments for the grep tool."""
pattern: str = Field(description="Regular expression to search for")
root: str | None = Field(default=None, description="Search root directory")
root: str | None = Field(
default=None,
description="Search root directory or file. For multiple roots, call grep separately per root.",
)
file_glob: str = Field(default="**/*")
case_sensitive: bool = Field(default=True)
limit: int = Field(default=200, ge=1, le=2000)
@@ -36,6 +39,14 @@ class GrepTool(BaseTool):
async def execute(self, arguments: GrepToolInput, context: ToolExecutionContext) -> ToolResult:
root = _resolve_path(context.cwd, arguments.root) if arguments.root else context.cwd
if not root.exists():
return ToolResult(
output=(
f"Search root does not exist: {root}\n"
"If you intended multiple roots, call grep separately for each root."
),
is_error=True,
)
if root.is_file():
display_base = _display_base(root, context.cwd)
matches = await _rg_grep_file(
@@ -0,0 +1,356 @@
"""Generate or edit raster images with configurable image generation providers."""
from __future__ import annotations
import base64
import json
import logging
from pathlib import Path
from typing import Any, Literal
import httpx
from openai import AsyncOpenAI
from pydantic import BaseModel, Field
from openharness.api.codex_client import _build_codex_headers, _resolve_codex_url
from openharness.api.openai_client import _normalize_openai_base_url
from openharness.tools.base import BaseTool, ToolExecutionContext, ToolResult
log = logging.getLogger(__name__)
_DEFAULT_PROMPT = (
"Create a high-quality raster image that satisfies the user's request. "
"Avoid watermarks, unintended text, and unrelated logos."
)
_DEFAULT_MODEL = "gpt-image-2"
_DEFAULT_OUTPUT_DIR = "generated_images"
ImageGenerationProvider = Literal["auto", "openai", "codex"]
class ImageGenerationToolInput(BaseModel):
"""Arguments for image generation or editing."""
prompt: str = Field(default=_DEFAULT_PROMPT, description="Image generation or edit prompt.")
provider: ImageGenerationProvider = Field(
default="auto",
description="Image generation provider: auto, openai, or codex.",
)
image_paths: list[str] = Field(
default_factory=list,
description="Local image paths to edit or use as references. OpenAI provider uses image edit mode. Codex hosted generation currently treats these as visual context.",
)
mask_path: str | None = Field(default=None, description="Optional PNG mask path for OpenAI edit mode.")
output_path: str | None = Field(
default=None,
description="Optional output path. For multiple images, numeric suffixes are added.",
)
output_dir: str = Field(
default=_DEFAULT_OUTPUT_DIR,
description="Output directory used when output_path is not provided.",
)
model: str | None = Field(default=None, description="OpenAI image model override.")
n: int = Field(default=1, ge=1, le=10, description="Number of images to generate.")
size: str = Field(default="auto", description="OpenAI image size, e.g. auto, 1024x1024, 1536x1024.")
quality: str = Field(default="medium", description="OpenAI image quality, e.g. low, medium, high, auto.")
background: Literal["transparent", "opaque", "auto"] | None = Field(
default=None,
description="Optional OpenAI background mode when supported by the provider.",
)
output_format: Literal["png", "jpeg", "webp"] = Field(default="png", description="Output image format.")
output_compression: int | None = Field(
default=None,
ge=0,
le=100,
description="Optional OpenAI compression level for lossy output formats when supported.",
)
input_fidelity: Literal["low", "high"] | None = Field(
default=None,
description="Optional OpenAI edit input fidelity when supported by the provider.",
)
moderation: str | None = Field(default=None, description="Optional OpenAI moderation setting.")
overwrite: bool = Field(default=False, description="Whether to overwrite existing output files.")
class ImageGenerationTool(BaseTool):
"""Generate or edit raster images and save them to local files."""
name = "image_generation"
description = (
"Generate or edit raster images using a configurable image generation provider. "
"Use this for bitmap assets such as photos, illustrations, sprites, mockups, "
"transparent cutouts, or edited local images. Supports provider='codex' for "
"Codex hosted image_generation with Codex subscription auth, and provider='openai' "
"for OpenAI-compatible key/base_url image APIs."
)
input_model = ImageGenerationToolInput
async def execute(self, arguments: ImageGenerationToolInput, context: ToolExecutionContext) -> ToolResult:
config = context.metadata.get("image_generation_config", {})
if not isinstance(config, dict):
config = {}
provider = _resolve_provider(arguments.provider, config)
try:
output_paths = self._resolve_output_paths(arguments, context.cwd)
if provider == "codex":
image_b64, revised_prompt = await self._generate_with_codex(arguments, config)
written = self._write_images(image_b64, output_paths, overwrite=arguments.overwrite)
extra = f"\nRevised prompt: {revised_prompt}" if revised_prompt else ""
return ToolResult(
output=(
"[Image generation via Codex hosted image_generation]\n"
+ "\n".join(f"Wrote {path}" for path in written)
+ extra
),
metadata={
"paths": [str(path) for path in written],
"provider": "codex",
"revised_prompt": revised_prompt,
},
)
image_b64 = await self._generate_with_openai(arguments, config)
written = self._write_images(image_b64, output_paths, overwrite=arguments.overwrite)
except Exception as exc:
log.exception("image_generation failed")
return ToolResult(output=f"image_generation failed: {exc}", is_error=True)
mode = "edit" if arguments.image_paths else "generate"
model = (arguments.model or str(config.get("model") or _DEFAULT_MODEL)).strip()
return ToolResult(
output=(
f"[Image generation via {model} ({mode}, openai)]\n"
+ "\n".join(f"Wrote {path}" for path in written)
),
metadata={"paths": [str(path) for path in written], "model": model, "mode": mode, "provider": "openai"},
)
async def _generate_with_openai(self, arguments: ImageGenerationToolInput, config: dict[str, object]) -> list[str]:
model = (arguments.model or str(config.get("model") or _DEFAULT_MODEL)).strip()
api_key = str(config.get("api_key") or "").strip()
base_url = str(config.get("base_url") or "").strip()
if not api_key:
raise RuntimeError(
"OpenAI image generation API key is not configured. Set image_generation.api_key "
"or OPENHARNESS_IMAGE_GENERATION_API_KEY, or choose provider='codex'."
)
if arguments.image_paths:
return await self._edit_images(arguments, model, api_key, base_url)
return await self._generate_images(arguments, model, api_key, base_url)
async def _generate_with_codex(
self,
arguments: ImageGenerationToolInput,
config: dict[str, object],
) -> tuple[list[str], str | None]:
auth_token = str(config.get("codex_auth_token") or "").strip()
if not auth_token:
raise RuntimeError(
"Codex image generation auth is not configured. Run 'oh auth codex-login' "
"or use provider='openai' with OPENHARNESS_IMAGE_GENERATION_API_KEY."
)
model = str(config.get("codex_model") or "gpt-5.4").strip() or "gpt-5.4"
base_url = str(config.get("codex_base_url") or "").strip()
prompt = _codex_prompt(arguments)
body: dict[str, Any] = {
"model": model,
"store": False,
"stream": True,
"instructions": "Generate the requested image using the hosted image_generation tool.",
"input": [{"role": "user", "content": _codex_user_content(arguments, prompt)}],
"text": {"verbosity": "medium"},
"tools": [{"type": "image_generation", "output_format": arguments.output_format}],
"tool_choice": "auto",
"parallel_tool_calls": False,
}
headers = _build_codex_headers(auth_token)
url = _resolve_codex_url(base_url)
image_results: list[str] = []
revised_prompt: str | None = None
async with httpx.AsyncClient(timeout=180.0, follow_redirects=True) as client:
async with client.stream("POST", url, headers=headers, json=body) as response:
if response.status_code >= 400:
payload = await response.aread()
raise RuntimeError(payload.decode("utf-8", "replace") or f"Codex request failed: {response.status_code}")
async for event in _iter_sse_events(response):
if event.get("type") != "response.output_item.done":
if event.get("type") == "response.failed":
raise RuntimeError(json.dumps(event.get("response") or event, ensure_ascii=False))
if event.get("type") == "error":
raise RuntimeError(json.dumps(event, ensure_ascii=False))
continue
item = event.get("item")
if not isinstance(item, dict) or item.get("type") != "image_generation_call":
continue
result = item.get("result")
if isinstance(result, str) and result:
image_results.append(result)
candidate = item.get("revised_prompt")
if isinstance(candidate, str) and candidate:
revised_prompt = candidate
if not image_results:
raise RuntimeError("Codex hosted image_generation returned no image result")
return image_results, revised_prompt
@staticmethod
async def _generate_images(arguments: ImageGenerationToolInput, model: str, api_key: str, base_url: str) -> list[str]:
client = AsyncOpenAI(
api_key=api_key,
base_url=_normalize_openai_base_url(base_url),
default_headers={"Authorization": f"Bearer {api_key}"},
)
result = await client.images.generate(**_image_payload(arguments, model))
return _extract_b64_images(result)
@staticmethod
async def _edit_images(arguments: ImageGenerationToolInput, model: str, api_key: str, base_url: str) -> list[str]:
client = AsyncOpenAI(
api_key=api_key,
base_url=_normalize_openai_base_url(base_url),
default_headers={"Authorization": f"Bearer {api_key}"},
)
image_handles = [Path(path).expanduser().resolve().open("rb") for path in arguments.image_paths]
mask_handle = Path(arguments.mask_path).expanduser().resolve().open("rb") if arguments.mask_path else None
try:
payload = _image_payload(arguments, model)
payload["image"] = image_handles if len(image_handles) > 1 else image_handles[0]
if mask_handle is not None:
payload["mask"] = mask_handle
result = await client.images.edit(**payload)
finally:
for handle in image_handles:
handle.close()
if mask_handle is not None:
mask_handle.close()
return _extract_b64_images(result)
@staticmethod
def _resolve_output_paths(arguments: ImageGenerationToolInput, cwd: Path) -> list[Path]:
suffix = f".{arguments.output_format}"
if arguments.output_path:
base = Path(arguments.output_path)
if not base.is_absolute():
base = cwd / base
base = base.expanduser().resolve()
else:
out_dir = Path(arguments.output_dir)
if not out_dir.is_absolute():
out_dir = cwd / out_dir
out_dir = out_dir.expanduser().resolve()
base = out_dir / f"image{suffix}"
if base.suffix.lower() != suffix:
base = base.with_suffix(suffix)
if arguments.n == 1:
return [base]
return [base.with_name(f"{base.stem}-{idx}{base.suffix}") for idx in range(1, arguments.n + 1)]
@staticmethod
def _write_images(images: list[str], output_paths: list[Path], *, overwrite: bool) -> list[Path]:
written: list[Path] = []
for image_b64, output_path in zip(images, output_paths, strict=False):
if output_path.exists() and not overwrite:
raise FileExistsError(f"output already exists: {output_path} (set overwrite=true)")
output_path.parent.mkdir(parents=True, exist_ok=True)
output_path.write_bytes(base64.b64decode(image_b64))
written.append(output_path)
if not written:
raise RuntimeError("provider returned no image data")
return written
def _resolve_provider(requested: str, config: dict[str, object]) -> Literal["openai", "codex"]:
if requested in {"openai", "codex"}:
return requested # type: ignore[return-value]
configured = str(config.get("provider") or "auto").strip().lower()
if configured in {"openai", "codex"}:
return configured # type: ignore[return-value]
if str(config.get("codex_auth_token") or "").strip():
return "codex"
return "openai"
def _image_payload(arguments: ImageGenerationToolInput, model: str) -> dict[str, Any]:
payload: dict[str, Any] = {
"model": model,
"prompt": arguments.prompt,
"n": arguments.n,
"size": arguments.size,
"quality": arguments.quality,
"background": arguments.background,
"output_format": arguments.output_format,
"output_compression": arguments.output_compression,
"input_fidelity": arguments.input_fidelity,
"moderation": arguments.moderation,
}
return {key: value for key, value in payload.items() if value is not None}
def _codex_prompt(arguments: ImageGenerationToolInput) -> str:
lines = [arguments.prompt]
if arguments.n > 1:
lines.append(f"Generate {arguments.n} distinct variants.")
if arguments.image_paths:
lines.append("Use the attached image(s) as visual context/reference for the generation or edit.")
return "\n".join(line for line in lines if line.strip())
def _codex_user_content(arguments: ImageGenerationToolInput, prompt: str) -> list[dict[str, str]]:
content = [{"type": "input_text", "text": prompt}]
for path_str in arguments.image_paths:
path = Path(path_str).expanduser().resolve()
media_type = _media_type_for_path(path)
data = base64.b64encode(path.read_bytes()).decode("ascii")
content.append({"type": "input_image", "image_url": f"data:{media_type};base64,{data}"})
return content
def _media_type_for_path(path: Path) -> str:
return {
".png": "image/png",
".jpg": "image/jpeg",
".jpeg": "image/jpeg",
".webp": "image/webp",
".gif": "image/gif",
}.get(path.suffix.lower(), "image/png")
async def _iter_sse_events(response: httpx.Response):
data_lines: list[str] = []
async for line in response.aiter_lines():
if line == "":
if data_lines:
payload = "\n".join(data_lines).strip()
data_lines = []
if payload and payload != "[DONE]":
try:
event = json.loads(payload)
except json.JSONDecodeError:
continue
if isinstance(event, dict):
yield event
continue
if line.startswith("data:"):
data_lines.append(line[5:].strip())
if data_lines:
payload = "\n".join(data_lines).strip()
if payload and payload != "[DONE]":
try:
event = json.loads(payload)
except json.JSONDecodeError:
return
if isinstance(event, dict):
yield event
def _extract_b64_images(result: Any) -> list[str]:
images: list[str] = []
for item in getattr(result, "data", []) or []:
b64 = getattr(item, "b64_json", None)
if isinstance(b64, str) and b64:
images.append(b64)
continue
url = getattr(item, "url", None)
if isinstance(url, str) and url.startswith("data:image/") and ";base64," in url:
images.append(url.split(";base64,", 1)[1])
return images
+3 -1
View File
@@ -8,6 +8,7 @@ from pathlib import Path
from pydantic import BaseModel, Field
from openharness.services.cron import get_cron_job
from openharness.services.cron_scheduler import _command_for_job
from openharness.sandbox import SandboxUnavailableError
from openharness.tools.base import BaseTool, ToolExecutionContext, ToolResult
from openharness.utils.shell import create_shell_subprocess
@@ -38,8 +39,9 @@ class RemoteTriggerTool(BaseTool):
cwd = Path(job.get("cwd") or context.cwd).expanduser()
try:
command = _command_for_job(job)
process = await create_shell_subprocess(
str(job["command"]),
command,
cwd=cwd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
+2 -1
View File
@@ -3,6 +3,7 @@
from __future__ import annotations
import html
import os
import re
from urllib.parse import parse_qs, unquote, urlparse
@@ -41,7 +42,7 @@ class WebSearchTool(BaseTool):
context: ToolExecutionContext,
) -> ToolResult:
del context
endpoint = arguments.search_url or "https://html.duckduckgo.com/html/"
endpoint = arguments.search_url or os.environ.get("OPENHARNESS_WEB_SEARCH_URL") or "https://html.duckduckgo.com/html/"
try:
response = await fetch_public_http_response(
endpoint,
+174
View File
@@ -0,0 +1,174 @@
"""Tools for creating and running dynamic workflows."""
from __future__ import annotations
from pathlib import Path
from pydantic import BaseModel, Field
from openharness.tools.base import BaseTool, ToolExecutionContext, ToolResult
from openharness.workflows.manager import get_workflow_manager, list_saved_workflows
WORKFLOW_DSL_HELP = """\
Dynamic workflow scripts are deterministic JavaScript. The runtime injects:
- workflow(name, async () => result): name and run the workflow.
- agent(prompt, opts): run one OpenHarness subagent. opts may include phase, model, name, team, system_prompt, permissions.
- parallel(items, async (item, index) => result): barrier parallel map.
- pipeline(items, [stage1, stage2]): each item independently passes through all stages without a stage barrier.
- phase(name, async () => result): group progress and token accounting.
- log(message): append progress.
The script cannot directly use filesystem, shell, network, Date, Math.random, require, import, process, or fetch.
Agents perform all file/shell work through normal OpenHarness tools and permissions.
Return the final workflow result from workflow(...).
"""
class WorkflowCreateInput(BaseModel):
"""Arguments for creating a reusable workflow."""
name: str = Field(description="Short workflow name, used as the saved command name")
script: str = Field(description="Deterministic JavaScript workflow script using workflow/agent/parallel/pipeline/phase/log")
description: str = Field(default="", description="Short description of what the workflow does")
scope: str = Field(default="project", description="Save scope: project or user")
run_immediately: bool = Field(default=True, description="Start a run after saving the workflow")
wait_for_completion: bool = Field(default=False, description="Wait for the workflow run to finish before returning")
max_concurrency: int | None = Field(default=None, description="Optional concurrent agent cap, default min(16, cpu-2)")
max_agents: int = Field(default=1000, description="Hard cap for agent() calls in one run")
class WorkflowRunInput(BaseModel):
"""Arguments for starting a saved workflow."""
name_or_path: str = Field(description="Saved workflow name or path to a .js workflow script")
wait_for_completion: bool = Field(default=False, description="Wait for the workflow run to finish before returning")
max_concurrency: int | None = Field(default=None, description="Optional concurrent agent cap")
max_agents: int = Field(default=1000, description="Hard cap for agent() calls in one run")
class WorkflowStatusInput(BaseModel):
"""Arguments for inspecting a workflow run."""
run_id: str = Field(description="Workflow run ID returned by workflow_create or workflow_run")
class WorkflowCreateTool(BaseTool):
"""Create and optionally run a dynamic workflow."""
name = "workflow_create"
description = "Create a saved dynamic workflow and optionally run it.\n\n" + WORKFLOW_DSL_HELP
input_model = WorkflowCreateInput
async def execute(self, arguments: WorkflowCreateInput, context: ToolExecutionContext) -> ToolResult:
manager = get_workflow_manager()
scope = arguments.scope if arguments.scope in {"project", "user"} else "project"
saved_path = manager.save_script(
cwd=context.cwd,
name=arguments.name,
script=arguments.script,
scope=scope,
)
lines = [f"Saved workflow {arguments.name!r} to {saved_path}"]
metadata = {"workflow_path": str(saved_path)}
if arguments.run_immediately:
run = await manager.start_inline(
cwd=context.cwd,
name=arguments.name,
script=arguments.script,
description=arguments.description,
model=_active_model(context),
wait=arguments.wait_for_completion,
max_concurrency=arguments.max_concurrency,
max_agents=arguments.max_agents,
)
lines.append(_format_run_summary(run))
metadata["run_id"] = run.id
metadata["status"] = run.status
return ToolResult(output="\n".join(lines), metadata=metadata)
class WorkflowRunTool(BaseTool):
"""Run a saved dynamic workflow."""
name = "workflow_run"
description = "Run a saved dynamic workflow by name or .js path.\n\n" + WORKFLOW_DSL_HELP
input_model = WorkflowRunInput
async def execute(self, arguments: WorkflowRunInput, context: ToolExecutionContext) -> ToolResult:
try:
run = await get_workflow_manager().start_saved(
cwd=context.cwd,
name_or_path=arguments.name_or_path,
model=_active_model(context),
wait=arguments.wait_for_completion,
max_concurrency=arguments.max_concurrency,
max_agents=arguments.max_agents,
)
except ValueError as exc:
return ToolResult(output=str(exc), is_error=True)
return ToolResult(output=_format_run_summary(run), metadata={"run_id": run.id, "status": run.status})
class WorkflowStatusTool(BaseTool):
"""Inspect one dynamic workflow run."""
name = "workflow_status"
description = "Inspect the status, phases, agents, and final output of a dynamic workflow run."
input_model = WorkflowStatusInput
async def execute(self, arguments: WorkflowStatusInput, context: ToolExecutionContext) -> ToolResult:
del context
run = get_workflow_manager().get(arguments.run_id)
if run is None:
return ToolResult(output=f"Workflow run not found: {arguments.run_id}", is_error=True)
return ToolResult(output=format_workflow_run(run, include_output=True), metadata={"run_id": run.id, "status": run.status})
def format_workflow_run(run, *, include_output: bool = False) -> str:
"""Render a workflow run for tools and slash commands."""
lines = [
_format_run_summary(run),
f"script: {run.script_path}",
f"agents: {run.agent_count} live, {run.cached_count} cached",
]
if run.error:
lines.append(f"error: {run.error}")
if run.phases:
lines.append("phases:")
for phase in run.phases.values():
lines.append(
f"- {phase.name}: {phase.status}, agents={phase.agent_count}, cached={phase.cached_count}"
)
if run.agents:
lines.append("agent calls:")
for agent in run.agents.values():
suffix = f", task={agent.task_id}" if agent.task_id else ""
lines.append(f"- {agent.id}: {agent.status}, phase={agent.phase or '-'}{suffix}")
if run.logs:
lines.append("logs:")
for message in run.logs[-10:]:
lines.append(f"- {message}")
if include_output and run.result is not None:
lines.append("result:")
lines.append(str(run.result))
return "\n".join(lines)
def list_saved_workflows_text(cwd: str | Path) -> str:
"""Render saved workflows visible from a cwd."""
paths = list_saved_workflows(cwd)
if not paths:
return "No saved workflows."
return "\n".join(str(path) for path in paths)
def _format_run_summary(run) -> str:
return f"Workflow {run.id} {run.status}: {run.name}"
def _active_model(context: ToolExecutionContext) -> str | None:
model = context.metadata.get("model")
if isinstance(model, str) and model.strip():
return model.strip()
return None
+7
View File
@@ -43,6 +43,7 @@ async def run_repl(
cwd: str | None = None,
model: str | None = None,
max_turns: int | None = None,
effort: str | None = None,
base_url: str | None = None,
system_prompt: str | None = None,
api_key: str | None = None,
@@ -59,6 +60,7 @@ async def run_repl(
cwd=cwd,
model=model,
max_turns=max_turns,
effort=effort,
base_url=base_url,
system_prompt=system_prompt,
api_key=api_key,
@@ -76,6 +78,7 @@ async def run_repl(
cwd=cwd,
model=model,
max_turns=max_turns,
effort=effort,
base_url=base_url,
system_prompt=system_prompt,
api_key=api_key,
@@ -91,6 +94,7 @@ async def run_task_worker(
cwd: str | None = None,
model: str | None = None,
max_turns: int | None = None,
effort: str | None = None,
base_url: str | None = None,
system_prompt: str | None = None,
api_key: str | None = None,
@@ -135,6 +139,7 @@ async def run_task_worker(
cwd=cwd,
model=model,
max_turns=max_turns,
effort=effort,
base_url=base_url,
system_prompt=system_prompt,
api_key=api_key,
@@ -176,6 +181,7 @@ async def run_print_mode(
cwd: str | None = None,
model: str | None = None,
base_url: str | None = None,
effort: str | None = None,
system_prompt: str | None = None,
append_system_prompt: str | None = None,
api_key: str | None = None,
@@ -206,6 +212,7 @@ async def run_print_mode(
cwd=cwd,
model=model,
max_turns=max_turns,
effort=effort,
base_url=base_url,
system_prompt=system_prompt,
api_key=api_key,
+131 -13
View File
@@ -19,6 +19,7 @@ from openharness.commands import MemoryCommandBackend
from openharness.config.settings import CLAUDE_MODEL_ALIAS_OPTIONS, resolve_model_setting
from openharness.bridge import get_bridge_manager
from openharness.coordinator.coordinator_mode import is_coordinator_mode
from openharness.engine.messages import ConversationMessage, ImageBlock, TextBlock
from openharness.themes import list_themes
from openharness.engine.stream_events import (
AssistantTextDelta,
@@ -33,7 +34,13 @@ from openharness.engine.stream_events import (
from openharness.output_styles import load_output_styles
from openharness.tasks import get_task_manager
from openharness.ui.coordinator_drain import drain_coordinator_async_agents
from openharness.ui.protocol import BackendEvent, FrontendRequest, TranscriptItem
from openharness.ui.protocol import (
BackendEvent,
CommandSnapshot,
FrontendImageAttachment,
FrontendRequest,
TranscriptItem,
)
from openharness.ui.runtime import build_runtime, close_runtime, handle_line, start_runtime
from openharness.services.session_backend import SessionBackend
@@ -50,6 +57,7 @@ class BackendHostConfig:
model: str | None = None
max_turns: int | None = None
effort: str | None = None
base_url: str | None = None
system_prompt: str | None = None
api_key: str | None = None
@@ -77,6 +85,7 @@ class ReactBackendHost:
self._write_lock = asyncio.Lock()
self._request_queue: asyncio.Queue[FrontendRequest] = asyncio.Queue()
self._permission_requests: dict[str, asyncio.Future[bool]] = {}
self._edit_approval_requests: dict[str, asyncio.Future[str]] = {}
self._question_requests: dict[str, asyncio.Future[str]] = {}
self._permission_lock = asyncio.Lock()
self._busy = False
@@ -84,11 +93,13 @@ class ReactBackendHost:
self._active_request_task: asyncio.Task[bool] | None = None
# Track last tool input per name for rich event emission
self._last_tool_inputs: dict[str, dict] = {}
self._edit_always_approved = False
async def run(self) -> int:
self._bundle = await build_runtime(
model=self._config.model,
max_turns=self._config.max_turns,
effort=self._config.effort,
base_url=self._config.base_url,
system_prompt=self._config.system_prompt,
api_key=self._config.api_key,
@@ -100,6 +111,7 @@ class ReactBackendHost:
restore_tool_metadata=self._config.restore_tool_metadata,
permission_prompt=self._ask_permission,
ask_user_prompt=self._ask_question,
edit_approval_prompt=self._ask_edit_approval,
enforce_max_turns=self._config.enforce_max_turns,
permission_mode=self._config.permission_mode,
session_backend=self._config.session_backend,
@@ -109,11 +121,20 @@ class ReactBackendHost:
include_project_memory=self._config.include_project_memory,
)
await start_runtime(self._bundle)
command_items = [
CommandSnapshot(
name=f"/{command.name}",
description=command.description,
aliases=[f"/{alias}" for alias in command.aliases],
)
for command in self._bundle.commands.list_commands()
]
await self._emit(
BackendEvent.ready(
self._bundle.app_state.get(),
get_task_manager().list_tasks(),
[f"/{command.name}" for command in self._bundle.commands.list_commands()],
[item.name for item in command_items],
command_items,
)
)
await self._emit(self._status_snapshot())
@@ -161,11 +182,13 @@ class ReactBackendHost:
await self._emit(BackendEvent(type="error", message="Session is busy"))
continue
line = (request.line or "").strip()
if not line:
if not line and not request.images:
continue
self._busy = True
try:
should_continue = await self._run_active_request(self._process_line(line))
should_continue = await self._run_active_request(
self._process_line(line, images=request.images)
)
finally:
self._busy = False
if not should_continue:
@@ -193,6 +216,11 @@ class ReactBackendHost:
except Exception as exc: # pragma: no cover - defensive protocol handling
await self._emit(BackendEvent(type="error", message=f"Invalid request: {exc}"))
continue
if request.type == "permission_response" and request.request_id in self._edit_approval_requests:
future = self._edit_approval_requests[request.request_id]
if not future.done():
future.set_result(_edit_approval_reply_from_request(request))
continue
if request.type == "permission_response" and request.request_id in self._permission_requests:
future = self._permission_requests[request.request_id]
if not future.done():
@@ -234,10 +262,23 @@ class ReactBackendHost:
return
task.cancel()
async def _process_line(self, line: str, *, transcript_line: str | None = None) -> bool:
async def _process_line(
self,
line: str,
*,
transcript_line: str | None = None,
images: list[FrontendImageAttachment] | None = None,
) -> bool:
assert self._bundle is not None
user_message = _build_user_message_with_images(line, images or [])
await self._emit(
BackendEvent(type="transcript_item", item=TranscriptItem(role="user", text=transcript_line or line))
BackendEvent(
type="transcript_item",
item=TranscriptItem(
role="user",
text=transcript_line or _format_transcript_line(line, images or []),
),
)
)
async def _print_system(message: str) -> None:
@@ -342,13 +383,14 @@ class ReactBackendHost:
async def _clear_output() -> None:
await self._emit(BackendEvent(type="clear_transcript"))
should_continue = await handle_line(
self._bundle,
line,
print_system=_print_system,
render_event=_render_event,
clear_output=_clear_output,
)
handle_line_kwargs: dict[str, Any] = {
"print_system": _print_system,
"render_event": _render_event,
"clear_output": _clear_output,
}
if user_message is not None:
handle_line_kwargs["user_message"] = user_message
should_continue = await handle_line(self._bundle, line, **handle_line_kwargs)
if is_coordinator_mode():
await drain_coordinator_async_agents(
self._bundle,
@@ -549,6 +591,8 @@ class ReactBackendHost:
{"value": "low", "label": "Low", "description": "Fastest responses", "active": settings.effort == "low"},
{"value": "medium", "label": "Medium", "description": "Balanced reasoning", "active": settings.effort == "medium"},
{"value": "high", "label": "High", "description": "Deepest reasoning", "active": settings.effort == "high"},
{"value": "xhigh", "label": "XHigh", "description": "Extra high reasoning", "active": settings.effort == "xhigh"},
{"value": "ultracode", "label": "Ultracode", "description": "Prefer dynamic workflows for substantive tasks", "active": settings.effort == "ultracode"},
]
await self._emit(
BackendEvent(
@@ -755,6 +799,45 @@ class ReactBackendHost:
finally:
self._permission_requests.pop(request_id, None)
def _current_permission_mode(self) -> str:
if self._bundle is None:
return str(self._config.permission_mode or "")
return str(self._bundle.app_state.get().permission_mode or "")
async def _ask_edit_approval(self, path: str, diff: str, added: int, removed: int) -> str:
if self._edit_always_approved or self._current_permission_mode() == "full_auto":
return "always"
async with self._permission_lock:
request_id = uuid4().hex
future: asyncio.Future[str] = asyncio.get_running_loop().create_future()
self._edit_approval_requests[request_id] = future
await self._emit(
BackendEvent(
type="modal_request",
modal={
"kind": "edit_diff",
"request_id": request_id,
"path": path,
"diff": diff,
"added": added,
"removed": removed,
},
)
)
try:
reply = await asyncio.wait_for(future, timeout=300)
except asyncio.TimeoutError:
log.warning("Edit approval request %s timed out after 300s, denying", request_id)
reply = "reject"
finally:
self._edit_approval_requests.pop(request_id, None)
await self._emit(BackendEvent(type="modal_request", modal=None))
if reply == "always":
self._edit_always_approved = True
return reply
async def _ask_question(self, question: str) -> str:
request_id = uuid4().hex
future: asyncio.Future[str] = asyncio.get_running_loop().create_future()
@@ -787,10 +870,37 @@ class ReactBackendHost:
sys.stdout.flush()
def _build_user_message_with_images(
line: str,
images: list[FrontendImageAttachment],
) -> ConversationMessage | None:
if not images:
return None
content = [TextBlock(text=line or "Please analyze the attached image.")]
content.extend(
ImageBlock(
media_type=image.media_type,
data=image.data,
source_path=image.source_path or "",
)
for image in images
)
return ConversationMessage.from_user_content(content)
def _format_transcript_line(line: str, images: list[FrontendImageAttachment]) -> str:
if not images:
return line
noun = "image" if len(images) == 1 else "images"
attachment_line = f"[{len(images)} {noun} attached]"
return f"{line}\n{attachment_line}" if line else attachment_line
async def run_backend_host(
*,
model: str | None = None,
max_turns: int | None = None,
effort: str | None = None,
base_url: str | None = None,
system_prompt: str | None = None,
api_key: str | None = None,
@@ -815,6 +925,7 @@ async def run_backend_host(
BackendHostConfig(
model=model,
max_turns=max_turns,
effort=effort,
base_url=base_url,
system_prompt=system_prompt,
api_key=api_key,
@@ -837,3 +948,10 @@ async def run_backend_host(
__all__ = ["run_backend_host", "ReactBackendHost", "BackendHostConfig"]
def _edit_approval_reply_from_request(request: FrontendRequest) -> str:
reply = (request.permission_reply or "").strip().lower()
if reply in {"once", "always", "reject"}:
return reply
return "once" if bool(request.allowed) else "reject"
+37 -1
View File
@@ -4,7 +4,7 @@ from __future__ import annotations
from typing import Any, Literal
from pydantic import BaseModel, Field
from pydantic import BaseModel, Field, field_validator
from openharness.state.app_state import AppState
from openharness.bridge.manager import BridgeSessionRecord
@@ -12,6 +12,28 @@ from openharness.mcp.types import McpConnectionStatus
from openharness.tasks.types import TaskRecord
class FrontendImageAttachment(BaseModel):
"""Base64 image payload submitted from the React TUI."""
media_type: str
data: str
source_path: str | None = None
@field_validator("media_type")
@classmethod
def _validate_media_type(cls, value: str) -> str:
if not value.startswith("image/"):
raise ValueError("image attachment media_type must start with image/")
return value
@field_validator("data")
@classmethod
def _validate_data(cls, value: str) -> str:
if not value.strip():
raise ValueError("image attachment data is required")
return value
class FrontendRequest(BaseModel):
"""One request sent from the React frontend to the Python backend."""
@@ -30,7 +52,9 @@ class FrontendRequest(BaseModel):
value: str | None = None
request_id: str | None = None
allowed: bool | None = None
permission_reply: str | None = None
answer: str | None = None
images: list[FrontendImageAttachment] = Field(default_factory=list)
class TranscriptItem(BaseModel):
@@ -63,6 +87,14 @@ class TaskSnapshot(BaseModel):
)
class CommandSnapshot(BaseModel):
"""UI-safe slash command metadata."""
name: str
description: str = ""
aliases: list[str] = Field(default_factory=list)
class BackendEvent(BaseModel):
"""One event sent from the Python backend to the React frontend."""
@@ -94,6 +126,7 @@ class BackendEvent(BaseModel):
mcp_servers: list[dict[str, Any]] | None = None
bridge_sessions: list[dict[str, Any]] | None = None
commands: list[str] | None = None
command_items: list[CommandSnapshot] | None = None
modal: dict[str, Any] | None = None
tool_name: str | None = None
tool_input: dict[str, Any] | None = None
@@ -116,6 +149,7 @@ class BackendEvent(BaseModel):
state: AppState,
tasks: list[TaskRecord],
commands: list[str],
command_items: list[CommandSnapshot] | None = None,
) -> "BackendEvent":
return cls(
type="ready",
@@ -124,6 +158,7 @@ class BackendEvent(BaseModel):
mcp_servers=[],
bridge_sessions=[],
commands=commands,
command_items=command_items,
)
@classmethod
@@ -216,6 +251,7 @@ def _format_permission_mode(raw: str) -> str:
__all__ = [
"BackendEvent",
"FrontendImageAttachment",
"FrontendRequest",
"TaskSnapshot",
"TranscriptItem",
+5
View File
@@ -83,6 +83,7 @@ def build_backend_command(
cwd: str | None = None,
model: str | None = None,
max_turns: int | None = None,
effort: str | None = None,
base_url: str | None = None,
system_prompt: str | None = None,
api_key: str | None = None,
@@ -97,6 +98,8 @@ def build_backend_command(
command.extend(["--model", model])
if max_turns is not None:
command.extend(["--max-turns", str(max_turns)])
if effort:
command.extend(["--effort", effort])
if base_url:
command.extend(["--base-url", base_url])
if system_prompt:
@@ -116,6 +119,7 @@ async def launch_react_tui(
cwd: str | None = None,
model: str | None = None,
max_turns: int | None = None,
effort: str | None = None,
base_url: str | None = None,
system_prompt: str | None = None,
api_key: str | None = None,
@@ -148,6 +152,7 @@ async def launch_react_tui(
cwd=cwd or str(Path.cwd()),
model=model,
max_turns=max_turns,
effort=effort,
base_url=base_url,
system_prompt=system_prompt,
api_key=api_key,
+124 -10
View File
@@ -45,11 +45,51 @@ from openharness.keybindings import load_keybindings
PermissionPrompt = Callable[[str, str], Awaitable[bool]]
AskUserPrompt = Callable[[str], Awaitable[str]]
EditApprovalPrompt = Callable[[str, str, int, int], Awaitable[str]]
SystemPrinter = Callable[[str], Awaitable[None]]
StreamRenderer = Callable[[StreamEvent], Awaitable[None]]
ClearHandler = Callable[[], Awaitable[None]]
def _resolve_image_generation_config(settings) -> dict[str, str]:
"""Resolve image generation configuration from settings, environment, and Codex auth."""
from openharness.config.settings import ImageGenerationConfig, ProviderProfile
cfg = settings.image_generation
env_cfg = ImageGenerationConfig.from_env()
resolved = {
"provider": cfg.provider or env_cfg.provider,
"model": cfg.model or env_cfg.model,
"api_key": cfg.api_key or env_cfg.api_key,
"base_url": cfg.base_url or env_cfg.base_url,
"codex_model": cfg.codex_model or env_cfg.codex_model,
"codex_base_url": cfg.codex_base_url or env_cfg.codex_base_url,
}
try:
codex_profile = settings.merged_profiles().get("codex") or ProviderProfile(
label="Codex Subscription",
provider="openai_codex",
api_format="openai",
auth_source="codex_subscription",
default_model="gpt-5.4",
)
codex_settings = settings.model_copy(
update={
"active_profile": "codex",
"profiles": {**settings.profiles, "codex": codex_profile},
}
).materialize_active_profile()
codex_auth = codex_settings.resolve_auth()
resolved["codex_auth_token"] = codex_auth.value
resolved["codex_base_url"] = resolved["codex_base_url"] or (codex_settings.base_url or "")
resolved["codex_model"] = resolved["codex_model"] or codex_settings.model
except Exception:
pass
return resolved
def _resolve_vision_config(settings) -> dict[str, str]:
"""Resolve the vision model configuration from settings or environment.
@@ -98,6 +138,7 @@ class RuntimeBundle:
extra_plugin_roots: tuple[str, ...] = ()
memory_backend: MemoryCommandBackend | None = None
include_project_memory: bool = True
autodream_context: dict[str, object] | None = None
def current_settings(self):
"""Return the effective settings for this session.
@@ -157,13 +198,8 @@ def _resolve_api_client_from_settings(settings) -> SupportsStreamingMessages:
def _safe_resolve_auth():
try:
return settings.resolve_auth()
except (ValueError, Exception):
print(
"Error: No API key configured.\n"
" Run `oh auth login` to set up authentication, or set the\n"
" ANTHROPIC_API_KEY (or OPENAI_API_KEY) environment variable.",
file=sys.stderr,
)
except Exception as exc:
_print_auth_resolution_error(settings, exc)
raise SystemExit(1)
if settings.api_format == "copilot":
@@ -202,12 +238,46 @@ def _resolve_api_client_from_settings(settings) -> SupportsStreamingMessages:
)
def _print_auth_resolution_error(settings, exc: Exception) -> None:
"""Render auth failures without collapsing subscription errors into API-key advice."""
try:
profile_name, profile = settings.resolve_profile()
auth_source = (getattr(profile, "auth_source", "") or "").strip()
except Exception:
profile_name = ""
auth_source = ""
message = str(exc).strip() or exc.__class__.__name__
if auth_source in {"claude_subscription", "codex_subscription"}:
login_command = "claude-login" if auth_source == "claude_subscription" else "codex-login"
provider_name = profile_name or (
"claude-subscription" if auth_source == "claude_subscription" else "codex"
)
print(
f"Error: {message}\n"
f" This profile uses subscription auth, not an API key.\n"
f" Run `oh auth {login_command}` to bind the local CLI session, then\n"
f" run `oh provider use {provider_name}` to activate it.",
file=sys.stderr,
)
return
print(
"Error: No API key configured.\n"
f" {message}\n"
" Run `oh auth login` to set up authentication, or set the\n"
" ANTHROPIC_API_KEY (or OPENAI_API_KEY) environment variable.",
file=sys.stderr,
)
async def build_runtime(
*,
prompt: str | None = None,
cwd: str | None = None,
model: str | None = None,
max_turns: int | None = None,
effort: str | None = None,
base_url: str | None = None,
system_prompt: str | None = None,
api_key: str | None = None,
@@ -216,6 +286,7 @@ async def build_runtime(
api_client: SupportsStreamingMessages | None = None,
permission_prompt: PermissionPrompt | None = None,
ask_user_prompt: AskUserPrompt | None = None,
edit_approval_prompt: EditApprovalPrompt | None = None,
restore_messages: list[dict] | None = None,
restore_tool_metadata: dict[str, object] | None = None,
enforce_max_turns: bool = True,
@@ -225,11 +296,13 @@ async def build_runtime(
extra_plugin_roots: Iterable[str | Path] | None = None,
memory_backend: MemoryCommandBackend | None = None,
include_project_memory: bool = True,
autodream_context: dict[str, object] | None = None,
) -> RuntimeBundle:
"""Build the shared runtime for an OpenHarness session."""
settings_overrides: dict[str, Any] = {
"model": model,
"max_turns": max_turns,
"effort": effort,
"base_url": base_url,
"system_prompt": system_prompt,
"api_key": api_key,
@@ -341,16 +414,21 @@ async def build_runtime(
permission_prompt=permission_prompt,
ask_user_prompt=ask_user_prompt,
hook_executor=hook_executor,
settings=settings,
tool_metadata={
"mcp_manager": mcp_manager,
"bridge_manager": bridge_manager,
"extra_skill_dirs": normalized_skill_dirs,
"extra_plugin_roots": normalized_plugin_roots,
"session_id": session_id,
"edit_approval_prompt": edit_approval_prompt,
"vision_model_config": _resolve_vision_config(settings),
"image_generation_config": _resolve_image_generation_config(settings),
**restored_metadata,
},
)
if autodream_context is not None:
engine.tool_metadata["autodream_context"] = autodream_context
# Restore messages from a saved session if provided
if restore_messages:
restored = sanitize_conversation_messages(
@@ -389,6 +467,7 @@ async def build_runtime(
extra_plugin_roots=normalized_plugin_roots,
memory_backend=memory_backend,
include_project_memory=include_project_memory,
autodream_context=autodream_context,
)
@@ -417,6 +496,9 @@ async def close_runtime(bundle: RuntimeBundle) -> None:
HookEvent.SESSION_END,
{"cwd": bundle.cwd, "event": HookEvent.SESSION_END.value},
)
close_api_client = getattr(bundle.api_client, "close", None)
if close_api_client is not None:
await close_api_client()
def _last_user_text(messages: list[ConversationMessage]) -> str:
@@ -432,6 +514,21 @@ def _truncate(text: str, limit: int) -> str:
return text[:limit] + ""
def _maybe_ultracode_prompt(prompt: str, effort: str | None) -> str:
"""Ask the model to use dynamic workflows when ultracode is active."""
if (effort or "").strip().lower() != "ultracode":
return prompt
if "workflow_create" in prompt or "workflow_run" in prompt:
return prompt
return (
"Use dynamic workflows for this substantive task when it benefits from "
"parallel subagents or independent verification. Prefer the workflow_create "
"tool: write a deterministic JavaScript workflow using workflow(), agent(), "
"parallel(), pipeline(), phase(), and log(), run it, and summarize the run.\n\n"
f"Task:\n{prompt}"
)
def _format_pending_tool_results(messages: list[ConversationMessage]) -> str | None:
"""Render a compact summary when we stop after tool execution but before the follow-up model turn."""
if not messages:
@@ -522,6 +619,17 @@ def refresh_runtime_client(bundle: RuntimeBundle) -> None:
default_model=settings.model,
)
bundle.engine.set_model(settings.model)
bundle.engine.set_effort(settings.effort)
bundle.engine.set_permission_checker(PermissionChecker(settings.permission))
system_prompt = build_runtime_system_prompt(
settings,
cwd=bundle.cwd,
latest_user_prompt=_last_user_text(bundle.engine.messages),
extra_skill_dirs=bundle.extra_skill_dirs,
extra_plugin_roots=bundle.extra_plugin_roots,
include_project_memory=bundle.include_project_memory,
)
bundle.engine.set_system_prompt(system_prompt)
sync_app_state(bundle)
@@ -532,6 +640,7 @@ async def handle_line(
print_system: SystemPrinter,
render_event: StreamRenderer,
clear_output: ClearHandler,
user_message: ConversationMessage | None = None,
) -> bool:
"""Handle one submitted line for either headless or TUI rendering."""
if not bundle.external_api_client:
@@ -554,7 +663,9 @@ async def handle_line(
memory_backend=bundle.memory_backend,
include_project_memory=bundle.include_project_memory,
)
parsed = bundle.commands.lookup(line) or lookup_skill_slash_command(line, command_context)
parsed = None if user_message is not None else (
bundle.commands.lookup(line) or lookup_skill_slash_command(line, command_context)
)
if parsed is not None:
command, args = parsed
result = await command.handler(
@@ -636,17 +747,20 @@ async def handle_line(
settings = bundle.current_settings()
if bundle.enforce_max_turns:
bundle.engine.set_max_turns(settings.max_turns)
if user_message is None:
line = _maybe_ultracode_prompt(line, settings.effort)
latest_user_prompt = line or (user_message.text if user_message is not None else "")
system_prompt = build_runtime_system_prompt(
settings,
cwd=bundle.cwd,
latest_user_prompt=line,
latest_user_prompt=latest_user_prompt,
extra_skill_dirs=bundle.extra_skill_dirs,
extra_plugin_roots=bundle.extra_plugin_roots,
include_project_memory=bundle.include_project_memory,
)
bundle.engine.set_system_prompt(system_prompt)
try:
async for event in bundle.engine.submit_message(line):
async for event in bundle.engine.submit_message(user_message or line):
await render_event(event)
except MaxTurnsExceeded as exc:
await print_system(f"Stopped after {exc.max_turns} turns (max_turns).")
+8 -2
View File
@@ -46,7 +46,10 @@ def exclusive_file_lock(
@contextmanager
def _exclusive_posix_lock(lock_path: Path) -> Iterator[None]:
import fcntl
try:
import fcntl
except ImportError as exc:
raise SwarmLockUnavailableError(f"fcntl not available: {exc}") from exc
lock_path.parent.mkdir(parents=True, exist_ok=True)
lock_path.touch(exist_ok=True)
@@ -60,7 +63,10 @@ def _exclusive_posix_lock(lock_path: Path) -> Iterator[None]:
@contextmanager
def _exclusive_windows_lock(lock_path: Path) -> Iterator[None]:
import msvcrt
try:
import msvcrt
except ImportError as exc:
raise SwarmLockUnavailableError(f"msvcrt not available: {exc}") from exc
lock_path.parent.mkdir(parents=True, exist_ok=True)
with lock_path.open("a+b") as lock_file:
+7
View File
@@ -4,6 +4,7 @@ from __future__ import annotations
import asyncio
import ipaddress
import os
import socket
from urllib.parse import urljoin, urlparse
@@ -57,15 +58,21 @@ async def fetch_public_http_response(
params: dict[str, str] | None = None,
timeout: float = 15.0,
max_redirects: int = 5,
proxy: str | None = None,
) -> httpx.Response:
"""Fetch one HTTP resource while validating every redirect hop."""
current_url = url
current_params = params
resolved_proxy = proxy if proxy is not None else os.environ.get("OPENHARNESS_WEB_PROXY")
if resolved_proxy:
validate_http_url(resolved_proxy)
async with httpx.AsyncClient(
follow_redirects=False,
timeout=timeout,
trust_env=False,
proxy=resolved_proxy,
) as client:
for redirect_count in range(max_redirects + 1):
await ensure_public_http_url(current_url)
+20
View File
@@ -0,0 +1,20 @@
"""Dynamic workflow runtime."""
from openharness.workflows.manager import WorkflowManager, get_workflow_manager
from openharness.workflows.runtime import WorkflowRuntime
from openharness.workflows.store import WorkflowStore
from openharness.workflows.types import (
WorkflowAgentRecord,
WorkflowPhaseRecord,
WorkflowRunRecord,
)
__all__ = [
"WorkflowAgentRecord",
"WorkflowManager",
"WorkflowPhaseRecord",
"WorkflowRunRecord",
"WorkflowRuntime",
"WorkflowStore",
"get_workflow_manager",
]
+251
View File
@@ -0,0 +1,251 @@
"""In-process manager for dynamic workflow runs."""
from __future__ import annotations
import asyncio
from pathlib import Path
from typing import Any
from openharness.config.paths import get_project_workflows_dir, get_user_workflows_dir
from openharness.utils.fs import atomic_write_text
from openharness.workflows.runtime import WorkflowAgentRunner, WorkflowRuntime, default_max_concurrency
from openharness.workflows.store import WorkflowStore
from openharness.workflows.types import WorkflowRunRecord
class WorkflowManager:
"""Start, inspect, pause, and resume workflow runs."""
def __init__(self, store: WorkflowStore | None = None) -> None:
self.store = store or WorkflowStore()
self._tasks: dict[str, asyncio.Task[Any]] = {}
self._agent_runner: WorkflowAgentRunner | None = None
def set_agent_runner(self, runner: WorkflowAgentRunner | None) -> None:
"""Override the agent runner, primarily for tests."""
self._agent_runner = runner
async def start_inline(
self,
*,
cwd: str | Path,
name: str,
script: str,
description: str = "",
model: str | None = None,
wait: bool = False,
max_concurrency: int | None = None,
max_agents: int = 1000,
) -> WorkflowRunRecord:
"""Create and start an inline workflow script."""
run = self.store.create_run(
cwd=cwd,
name=name,
script=script,
description=description,
max_agents=max_agents,
max_concurrency=max_concurrency or default_max_concurrency(),
)
self._start_runtime(run, model=model, max_concurrency=max_concurrency, max_agents=max_agents)
if wait:
await self.wait(run.id)
latest = self.store.load_run(run.id)
return latest or run
async def start_saved(
self,
*,
cwd: str | Path,
name_or_path: str,
model: str | None = None,
wait: bool = False,
max_concurrency: int | None = None,
max_agents: int = 1000,
) -> WorkflowRunRecord:
"""Start a saved workflow by name or path."""
resolved = resolve_workflow_script(cwd, name_or_path)
if resolved is None:
raise ValueError(f"Workflow not found: {name_or_path}")
script = resolved.read_text(encoding="utf-8")
return await self.start_inline(
cwd=cwd,
name=resolved.stem,
script=script,
description=f"Saved workflow: {resolved}",
model=model,
wait=wait,
max_concurrency=max_concurrency,
max_agents=max_agents,
)
async def resume(
self,
*,
run_id: str,
model: str | None = None,
wait: bool = False,
max_concurrency: int | None = None,
) -> WorkflowRunRecord:
"""Resume a paused or failed workflow using its run cache."""
run = self.store.clone_run_for_resume(run_id)
if run is None:
raise ValueError(f"Workflow run not found: {run_id}")
self._start_runtime(run, model=model, max_concurrency=max_concurrency, max_agents=run.max_agents)
if wait:
await self.wait(run.id)
latest = self.store.load_run(run.id)
return latest or run
async def pause(self, run_id: str) -> WorkflowRunRecord:
"""Pause a running workflow by cancelling the runtime task."""
task = self._tasks.get(run_id)
if task is not None and not task.done():
task.cancel()
try:
await task
except asyncio.CancelledError:
pass
run = self.store.load_run(run_id)
if run is None:
raise ValueError(f"Workflow run not found: {run_id}")
if run.status == "running":
run.status = "paused"
self.store.write_snapshot(run)
self.store.append_event(run_id, "run_paused", {})
return run
async def stop(self, run_id: str) -> WorkflowRunRecord:
"""Stop a running workflow and mark it killed."""
await self.pause(run_id)
run = self.store.load_run(run_id)
if run is None:
raise ValueError(f"Workflow run not found: {run_id}")
run.status = "killed"
self.store.write_snapshot(run)
self.store.append_event(run_id, "run_killed", {})
return run
async def wait(self, run_id: str) -> WorkflowRunRecord:
"""Wait for a workflow task to finish."""
task = self._tasks.get(run_id)
if task is not None:
try:
await task
except Exception:
pass
run = self.store.load_run(run_id)
if run is None:
raise ValueError(f"Workflow run not found: {run_id}")
return run
def get(self, run_id: str) -> WorkflowRunRecord | None:
"""Return one workflow run."""
return self.store.load_run(run_id)
def list_runs(self, *, limit: int = 50) -> list[WorkflowRunRecord]:
"""List workflow runs."""
return self.store.list_runs(limit=limit)
def save_script(
self,
*,
cwd: str | Path,
name: str,
script: str,
scope: str = "project",
) -> Path:
"""Save a workflow script as a reusable command."""
safe_name = _safe_workflow_name(name)
target_dir = get_user_workflows_dir() if scope == "user" else get_project_workflows_dir(cwd)
path = target_dir / f"{safe_name}.js"
atomic_write_text(path, script.rstrip() + "\n")
return path
def save_run_script(
self,
*,
cwd: str | Path,
run_id: str,
name: str,
scope: str = "project",
) -> Path:
"""Save an existing run's script."""
run = self.store.load_run(run_id)
if run is None:
raise ValueError(f"Workflow run not found: {run_id}")
script = Path(run.script_path).read_text(encoding="utf-8")
return self.save_script(cwd=cwd, name=name, script=script, scope=scope)
def _start_runtime(
self,
run: WorkflowRunRecord,
*,
model: str | None,
max_concurrency: int | None,
max_agents: int,
) -> None:
runtime = WorkflowRuntime(
store=self.store,
run=run,
model=model,
agent_runner=self._agent_runner,
max_concurrency=max_concurrency,
max_agents=max_agents,
)
task = asyncio.create_task(runtime.run_script())
self._tasks[run.id] = task
def _cleanup(done: asyncio.Task[Any]) -> None:
if self._tasks.get(run.id) is done:
self._tasks.pop(run.id, None)
task.add_done_callback(_cleanup)
def list_saved_workflows(cwd: str | Path) -> list[Path]:
"""List saved workflow scripts visible to a project."""
project_dir = get_project_workflows_dir(cwd)
user_dir = get_user_workflows_dir()
paths = list(project_dir.glob("*.js")) + list(user_dir.glob("*.js"))
return sorted(paths, key=lambda path: (path.name, str(path)))
def resolve_workflow_script(cwd: str | Path, name_or_path: str) -> Path | None:
"""Resolve a saved workflow by explicit path or search path name."""
raw = name_or_path.strip()
if not raw:
return None
path = Path(raw).expanduser()
if path.exists() and path.is_file():
return path.resolve()
name = raw[:-3] if raw.endswith(".js") else raw
safe_name = _safe_workflow_name(name)
for base in (get_project_workflows_dir(cwd), get_user_workflows_dir()):
candidate = base / f"{safe_name}.js"
if candidate.exists():
return candidate
return None
def _safe_workflow_name(name: str) -> str:
cleaned = "".join(ch.lower() if ch.isalnum() else "-" for ch in name.strip())
while "--" in cleaned:
cleaned = cleaned.replace("--", "-")
return cleaned.strip("-") or "workflow"
_DEFAULT_MANAGER: WorkflowManager | None = None
def get_workflow_manager() -> WorkflowManager:
"""Return the process-local workflow manager."""
global _DEFAULT_MANAGER
if _DEFAULT_MANAGER is None:
_DEFAULT_MANAGER = WorkflowManager()
return _DEFAULT_MANAGER
def reset_workflow_manager() -> None:
"""Reset the process-local workflow manager."""
global _DEFAULT_MANAGER
_DEFAULT_MANAGER = None
+573
View File
@@ -0,0 +1,573 @@
"""JavaScript-backed dynamic workflow runtime."""
from __future__ import annotations
import asyncio
import json
import os
import textwrap
import time
from pathlib import Path
from typing import Any, Protocol
from openharness.swarm.registry import get_backend_registry
from openharness.swarm.types import TeammateSpawnConfig
from openharness.tasks import get_task_manager
from openharness.workflows.store import WorkflowStore, workflow_cache_key
from openharness.workflows.types import WorkflowAgentRecord, WorkflowPhaseRecord, WorkflowRunRecord
class WorkflowAgentRunner(Protocol):
"""Protocol for executing one workflow subagent call."""
async def run_agent(
self,
*,
prompt: str,
cwd: Path,
model: str | None,
phase: str | None,
metadata: dict[str, Any],
) -> tuple[str, str | None, str | None]:
"""Return ``(output, task_id, agent_id)``."""
class SubprocessWorkflowAgentRunner:
"""Run workflow subagents through OpenHarness' subprocess swarm backend."""
async def run_agent(
self,
*,
prompt: str,
cwd: Path,
model: str | None,
phase: str | None,
metadata: dict[str, Any],
) -> tuple[str, str | None, str | None]:
registry = get_backend_registry()
executor = registry.get_executor("subprocess")
name = str(metadata.get("name") or metadata.get("subagent_type") or "workflow-agent")
team = str(metadata.get("team") or "workflow")
result = await executor.spawn(
TeammateSpawnConfig(
name=name,
team=team,
prompt=prompt,
cwd=str(cwd),
parent_session_id=str(metadata.get("parent_session_id") or "workflow"),
model=model,
system_prompt=metadata.get("system_prompt") if isinstance(metadata.get("system_prompt"), str) else None,
system_prompt_mode=metadata.get("system_prompt_mode") if metadata.get("system_prompt_mode") in {"append", "replace", "default"} else None,
permissions=[str(item) for item in metadata.get("permissions", [])] if isinstance(metadata.get("permissions"), list) else [],
task_type="local_agent",
)
)
if not result.success:
raise RuntimeError(result.error or "failed to spawn workflow agent")
manager = get_task_manager()
while True:
task = manager.get_task(result.task_id)
if task is None:
raise RuntimeError(f"workflow agent task disappeared: {result.task_id}")
if task.status in {"completed", "failed", "killed"}:
output = manager.read_task_output(result.task_id, max_bytes=200_000)
if task.status != "completed":
raise RuntimeError(output.strip() or f"workflow agent {result.agent_id} {task.status}")
return output, result.task_id, result.agent_id
await asyncio.sleep(0.25)
class WorkflowRuntime:
"""Execute a dynamic workflow script and coordinate subagents."""
def __init__(
self,
*,
store: WorkflowStore,
run: WorkflowRunRecord,
model: str | None = None,
agent_runner: WorkflowAgentRunner | None = None,
max_concurrency: int | None = None,
max_agents: int | None = None,
) -> None:
self.store = store
self.run = run
self.model = model
self.agent_runner = agent_runner or SubprocessWorkflowAgentRunner()
self.max_concurrency = max_concurrency or run.max_concurrency or default_max_concurrency()
self.max_agents = max_agents or run.max_agents or 1000
self._semaphore = asyncio.Semaphore(max(1, self.max_concurrency))
self._write_lock = asyncio.Lock()
self._node_process: asyncio.subprocess.Process | None = None
self._pending_rpc: set[asyncio.Task[None]] = set()
async def run_script(self) -> Any:
"""Run the workflow script to completion."""
script_path = Path(self.run.script_path)
if not script_path.exists():
raise FileNotFoundError(f"workflow script not found: {script_path}")
self.run.status = "running"
self.run.started_at = time.time()
self.run.ended_at = None
self.run.error = None
self.run.max_concurrency = self.max_concurrency
self.run.max_agents = self.max_agents
self.store.write_snapshot(self.run)
self.store.append_event(self.run.id, "run_started", {"max_concurrency": self.max_concurrency})
driver_path = self.store.run_dir(self.run.id) / "driver.js"
driver_path.write_text(_NODE_DRIVER, encoding="utf-8")
try:
result = await self._run_node(driver_path, script_path)
except asyncio.CancelledError:
await self._kill_node()
self.run.status = "paused"
self.run.ended_at = time.time()
self.store.write_snapshot(self.run)
self.store.append_event(self.run.id, "run_paused", {})
raise
except Exception as exc:
await self._kill_node()
self._finalize_open_phases("failed")
self.run.status = "failed"
self.run.error = str(exc)
self.run.ended_at = time.time()
self.store.write_snapshot(self.run)
self.store.append_event(self.run.id, "run_failed", {"error": str(exc)})
raise
self._finalize_open_phases("completed")
self.run.status = "completed"
self.run.result = result
self.run.ended_at = time.time()
self.store.write_snapshot(self.run)
self.store.append_event(self.run.id, "run_completed", {"result": result})
return result
async def _run_node(self, driver_path: Path, script_path: Path) -> Any:
self._node_process = await asyncio.create_subprocess_exec(
"node",
str(driver_path),
str(script_path),
cwd=str(Path(self.run.cwd)),
stdin=asyncio.subprocess.PIPE,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
assert self._node_process.stdout is not None
assert self._node_process.stderr is not None
stderr_task = asyncio.create_task(self._drain_stderr(self._node_process.stderr))
final_seen = False
final_result: Any = None
try:
while True:
raw = await self._node_process.stdout.readline()
if not raw:
break
try:
message = json.loads(raw.decode("utf-8"))
except json.JSONDecodeError as exc:
raise RuntimeError(f"workflow runtime emitted invalid JSON: {raw!r}") from exc
op = message.get("op")
if op == "__done":
final_seen = True
if message.get("ok"):
final_result = message.get("result")
else:
raise RuntimeError(str(message.get("error") or "workflow failed"))
continue
task = asyncio.create_task(self._handle_rpc(message))
self._pending_rpc.add(task)
task.add_done_callback(self._pending_rpc.discard)
if self._pending_rpc:
await asyncio.gather(*list(self._pending_rpc))
return_code = await self._node_process.wait()
await stderr_task
if return_code != 0:
raise RuntimeError(f"workflow runtime exited with code {return_code}")
if not final_seen:
raise RuntimeError("workflow runtime exited without a result")
return final_result
finally:
stderr_task.cancel()
async def _handle_rpc(self, message: dict[str, Any]) -> None:
rpc_id = message.get("id")
op = message.get("op")
try:
if op == "agent":
result = await self._agent(message)
elif op == "phase_start":
result = self._phase_start(str(message.get("name") or "phase"))
elif op == "phase_end":
result = self._phase_end(str(message.get("name") or "phase"))
elif op == "log":
result = self._log(str(message.get("message") or ""))
elif op == "workflow_start":
result = self._workflow_start(str(message.get("name") or self.run.name))
elif op == "workflow_end":
result = self._workflow_end(str(message.get("name") or self.run.name), message.get("result"))
elif op == "workflow_error":
result = self._workflow_error(str(message.get("name") or self.run.name), str(message.get("error") or ""))
else:
raise RuntimeError(f"unknown workflow rpc op: {op}")
await self._respond(rpc_id, ok=True, result=result)
except Exception as exc:
await self._respond(rpc_id, ok=False, error=str(exc))
async def _respond(self, rpc_id: Any, *, ok: bool, result: Any = None, error: str | None = None) -> None:
process = self._node_process
if process is None or process.stdin is None or process.stdin.is_closing():
return
payload = {"id": rpc_id, "ok": ok, "result": result, "error": error}
data = json.dumps(payload, ensure_ascii=True, default=str) + "\n"
async with self._write_lock:
process.stdin.write(data.encode("utf-8"))
await process.stdin.drain()
async def _agent(self, message: dict[str, Any]) -> Any:
prompt = str(message.get("prompt") or "")
opts = message.get("opts")
if not isinstance(opts, dict):
opts = {}
phase = str(opts.get("phase") or self.run.current_phase or "") or None
cache_key = workflow_cache_key(script_hash=self.run.script_hash, prompt=prompt, opts=opts)
cached = self.store.read_cache(self.run.id, cache_key)
agent_id = f"agent_{cache_key[:12]}"
if cached is not None:
self.run.cached_count += 1
self._increment_phase(phase, cached=True)
record = WorkflowAgentRecord(
id=agent_id,
cache_key=cache_key,
prompt=prompt,
phase=phase,
status="cached",
output=str(cached),
model=str(opts.get("model") or self.model or ""),
started_at=time.time(),
ended_at=time.time(),
cached=True,
)
self.run.agents[agent_id] = record
self.store.write_snapshot(self.run)
self.store.append_event(self.run.id, "agent_cached", {"agent_id": agent_id, "phase": phase, "cache_key": cache_key})
return cached
if self.run.agent_count >= self.max_agents:
raise RuntimeError(f"workflow exceeded max agent calls ({self.max_agents})")
self.run.agent_count += 1
self._increment_phase(phase, cached=False)
record = WorkflowAgentRecord(
id=agent_id,
cache_key=cache_key,
prompt=prompt,
phase=phase,
status="pending",
model=str(opts.get("model") or self.model or ""),
metadata=dict(opts),
)
self.run.agents[agent_id] = record
self.store.write_snapshot(self.run)
async with self._semaphore:
record.status = "running"
record.started_at = time.time()
self.store.write_snapshot(self.run)
self.store.append_event(self.run.id, "agent_started", {"agent_id": agent_id, "phase": phase, "cache_key": cache_key})
try:
output, task_id, spawned_agent_id = await self.agent_runner.run_agent(
prompt=prompt,
cwd=Path(self.run.cwd),
model=str(opts.get("model") or self.model or "") or None,
phase=phase,
metadata=opts,
)
except Exception as exc:
record.status = "failed"
record.error = str(exc)
record.ended_at = time.time()
self.store.write_snapshot(self.run)
self.store.append_event(self.run.id, "agent_failed", {"agent_id": agent_id, "error": str(exc)})
raise
record.status = "completed"
record.task_id = task_id
if spawned_agent_id:
record.metadata["spawned_agent_id"] = spawned_agent_id
record.output = output
record.ended_at = time.time()
self.store.write_cache(self.run.id, cache_key, output)
self.store.write_snapshot(self.run)
self.store.append_event(self.run.id, "agent_completed", {"agent_id": agent_id, "phase": phase, "task_id": task_id})
return output
def _workflow_start(self, name: str) -> dict[str, str]:
self.run.name = name or self.run.name
self.store.write_snapshot(self.run)
self.store.append_event(self.run.id, "workflow_started", {"name": self.run.name})
return {"name": self.run.name}
def _workflow_end(self, name: str, result: Any) -> dict[str, str]:
self.run.name = name or self.run.name
self.run.result = result
self.store.write_snapshot(self.run)
self.store.append_event(self.run.id, "workflow_completed", {"name": self.run.name, "result": result})
return {"name": self.run.name}
def _workflow_error(self, name: str, error: str) -> dict[str, str]:
self.run.name = name or self.run.name
self.run.error = error
self.store.write_snapshot(self.run)
self.store.append_event(self.run.id, "workflow_error", {"name": self.run.name, "error": error})
return {"name": self.run.name}
def _phase_start(self, name: str) -> dict[str, str]:
phase = self.run.phases.get(name)
if phase is None:
phase = WorkflowPhaseRecord(name=name)
self.run.phases[name] = phase
phase.status = "running"
phase.started_at = phase.started_at or time.time()
self.run.current_phase = name
self.store.write_snapshot(self.run)
self.store.append_event(self.run.id, "phase_started", {"phase": name})
return {"phase": name}
def _phase_end(self, name: str) -> dict[str, str]:
phase = self.run.phases.get(name)
if phase is None:
phase = WorkflowPhaseRecord(name=name)
self.run.phases[name] = phase
phase.status = "completed"
phase.ended_at = time.time()
if self.run.current_phase == name:
self.run.current_phase = None
self.store.write_snapshot(self.run)
self.store.append_event(self.run.id, "phase_completed", {"phase": name})
return {"phase": name}
def _log(self, message: str) -> dict[str, int]:
if message:
self.run.logs.append(message)
self.run.logs = self.run.logs[-200:]
self.store.write_snapshot(self.run)
self.store.append_event(self.run.id, "log", {"message": message})
return {"log_count": len(self.run.logs)}
def _increment_phase(self, phase_name: str | None, *, cached: bool) -> None:
if not phase_name:
return
phase = self.run.phases.get(phase_name)
if phase is None:
phase = WorkflowPhaseRecord(name=phase_name, status="running", started_at=time.time())
self.run.phases[phase_name] = phase
phase.agent_count += 1
if cached:
phase.cached_count += 1
def _finalize_open_phases(self, status: str) -> None:
now = time.time()
for phase in self.run.phases.values():
if phase.status == "running":
phase.status = status
phase.ended_at = phase.ended_at or now
self.run.current_phase = None
async def _drain_stderr(self, stream: asyncio.StreamReader) -> None:
while True:
chunk = await stream.readline()
if not chunk:
return
text = chunk.decode("utf-8", errors="replace").strip()
if text:
self.store.append_event(self.run.id, "runtime_stderr", {"message": text})
async def _kill_node(self) -> None:
process = self._node_process
if process is None or process.returncode is not None:
return
process.kill()
await process.wait()
def default_max_concurrency() -> int:
cpu_count = os.cpu_count() or 2
return max(1, min(16, cpu_count - 2))
_NODE_DRIVER = textwrap.dedent(
r"""
const fs = require("fs");
const vm = require("vm");
const readline = require("readline");
const scriptPath = process.argv[2];
const source = fs.readFileSync(scriptPath, "utf8");
const forbiddenPatterns = [
/\bDate\s*\.\s*now\s*\(/,
/\bnew\s+Date\s*\(/,
/\bMath\s*\.\s*random\s*\(/,
/\brequire\s*\(/,
/\bimport\s*\(/,
/\bfetch\s*\(/,
];
for (const pattern of forbiddenPatterns) {
if (pattern.test(source)) {
throw new Error(`Forbidden workflow API matched ${pattern}`);
}
}
let nextId = 1;
const pending = new Map();
const rl = readline.createInterface({ input: process.stdin });
rl.on("line", line => {
let msg;
try {
msg = JSON.parse(line);
} catch (error) {
return;
}
const waiter = pending.get(msg.id);
if (!waiter) return;
pending.delete(msg.id);
if (msg.ok) {
waiter.resolve(msg.result);
} else {
waiter.reject(new Error(msg.error || "workflow rpc failed"));
}
});
const rootWorkflows = [];
function rpc(op, payload) {
const id = nextId++;
const msg = Object.assign({ id, op }, payload || {});
process.stdout.write(JSON.stringify(msg) + "\n");
return new Promise((resolve, reject) => pending.set(id, { resolve, reject }));
}
function workflow(name, fn) {
const run = (async () => {
await rpc("workflow_start", { name });
try {
const result = await fn();
await rpc("workflow_end", { name, result });
return result;
} catch (error) {
await rpc("workflow_error", { name, error: String(error && error.stack || error) });
throw error;
}
})();
rootWorkflows.push(run);
return run;
}
async function agent(prompt, opts) {
return await rpc("agent", { prompt: String(prompt), opts: opts || {} });
}
async function parallel(items, fn) {
if (!Array.isArray(items)) throw new Error("parallel(items, fn) requires an array");
if (typeof fn !== "function") throw new Error("parallel(items, fn) requires a function");
return await Promise.all(items.map((item, index) => fn(item, index)));
}
async function pipeline(items, stages) {
if (!Array.isArray(items)) throw new Error("pipeline(items, stages) requires an array of items");
if (!Array.isArray(stages)) throw new Error("pipeline(items, stages) requires an array of stage functions");
return await Promise.all(items.map(async (item, index) => {
let value = item;
for (const stage of stages) {
if (typeof stage !== "function") throw new Error("pipeline stages must be functions");
value = await stage(value, index);
}
return value;
}));
}
async function phase(name, fn) {
await rpc("phase_start", { name: String(name) });
try {
return await fn();
} finally {
await rpc("phase_end", { name: String(name) });
}
}
async function log(message) {
await rpc("log", { message: String(message) });
}
const safeMath = Object.create(Math);
safeMath.random = function () {
throw new Error("Math.random is disabled in deterministic workflows");
};
function DisabledDate() {
throw new Error("Date is disabled in deterministic workflows");
}
DisabledDate.now = function () {
throw new Error("Date.now is disabled in deterministic workflows");
};
const context = {
workflow,
agent,
parallel,
pipeline,
phase,
log,
console: { log: (...args) => log(args.join(" ")) },
JSON,
Array,
Object,
String,
Number,
Boolean,
Promise,
Error,
RegExp,
Map,
Set,
Math: safeMath,
Date: DisabledDate,
};
async function main() {
const wrapped = `(async () => {\n${source}\n})()`;
const vmContext = vm.createContext(context, {
codeGeneration: { strings: false, wasm: false },
});
const script = new vm.Script(wrapped, { filename: scriptPath });
const result = await script.runInContext(vmContext, { timeout: 1000 });
if (typeof result === "undefined" && rootWorkflows.length === 1) {
return await rootWorkflows[0];
}
if (typeof result === "undefined" && rootWorkflows.length > 1) {
return await Promise.all(rootWorkflows);
}
return result;
}
main()
.then(result => {
rl.close();
process.stdout.write(JSON.stringify({ op: "__done", ok: true, result }) + "\n", () => process.exit(0));
})
.catch(error => {
rl.close();
process.stdout.write(JSON.stringify({ op: "__done", ok: false, error: String(error && error.stack || error) }) + "\n", () => process.exit(1));
});
"""
).strip() + "\n"
+191
View File
@@ -0,0 +1,191 @@
"""Persistent storage for dynamic workflow runs."""
from __future__ import annotations
import json
import time
from dataclasses import asdict
from hashlib import sha1, sha256
from pathlib import Path
from typing import Any
from uuid import uuid4
from openharness.config.paths import get_workflows_dir
from openharness.utils.fs import atomic_write_text
from openharness.workflows.types import (
WorkflowAgentRecord,
WorkflowPhaseRecord,
WorkflowRunRecord,
)
def stable_json(value: Any) -> str:
"""Return deterministic JSON for cache keys."""
return json.dumps(value, ensure_ascii=True, sort_keys=True, separators=(",", ":"), default=str)
def workflow_script_hash(script: str) -> str:
"""Return the stable hash for a workflow script."""
return sha256(script.encode("utf-8")).hexdigest()
def workflow_cache_key(*, script_hash: str, prompt: str, opts: dict[str, Any] | None) -> str:
"""Return the deterministic cache key for one agent call."""
payload = stable_json(
{
"script_hash": script_hash,
"prompt": prompt,
"opts": opts or {},
}
)
return sha256(payload.encode("utf-8")).hexdigest()
class WorkflowStore:
"""JSONL journal and snapshot storage for workflow runs."""
def __init__(self, root: str | Path | None = None) -> None:
self.root = Path(root).expanduser().resolve() if root is not None else get_workflows_dir()
self.runs_dir = self.root / "runs"
self.runs_dir.mkdir(parents=True, exist_ok=True)
def create_run(
self,
*,
cwd: str | Path,
name: str,
script: str,
description: str = "",
max_agents: int = 1000,
max_concurrency: int = 1,
) -> WorkflowRunRecord:
"""Create a new run directory and initial snapshot."""
run_id = f"w{uuid4().hex[:10]}"
run_dir = self.run_dir(run_id)
run_dir.mkdir(parents=True, exist_ok=True)
(run_dir / "cache").mkdir(exist_ok=True)
script_path = run_dir / "script.js"
atomic_write_text(script_path, script.rstrip() + "\n")
now = time.time()
record = WorkflowRunRecord(
id=run_id,
name=name,
cwd=str(Path(cwd).resolve()),
script_path=str(script_path),
script_hash=workflow_script_hash(script),
status="pending",
description=description,
created_at=now,
max_agents=max_agents,
max_concurrency=max_concurrency,
)
self.write_snapshot(record)
self.append_event(run_id, "run_created", asdict(record))
return record
def run_dir(self, run_id: str) -> Path:
"""Return a run directory path."""
return self.runs_dir / run_id
def snapshot_path(self, run_id: str) -> Path:
return self.run_dir(run_id) / "snapshot.json"
def journal_path(self, run_id: str) -> Path:
return self.run_dir(run_id) / "journal.jsonl"
def cache_path(self, run_id: str, key: str) -> Path:
return self.run_dir(run_id) / "cache" / f"{key}.json"
def load_run(self, run_id: str) -> WorkflowRunRecord | None:
"""Load one run snapshot."""
path = self.snapshot_path(run_id)
if not path.exists():
return None
return self._record_from_payload(json.loads(path.read_text(encoding="utf-8")))
def list_runs(self, *, limit: int = 50) -> list[WorkflowRunRecord]:
"""List known workflow runs, newest first."""
records: list[WorkflowRunRecord] = []
for path in sorted(self.runs_dir.glob("*/snapshot.json"), key=lambda item: item.stat().st_mtime, reverse=True):
try:
records.append(self._record_from_payload(json.loads(path.read_text(encoding="utf-8"))))
except (OSError, json.JSONDecodeError, TypeError, ValueError):
continue
if len(records) >= limit:
break
return records
def write_snapshot(self, record: WorkflowRunRecord) -> None:
"""Persist a workflow run snapshot."""
run_dir = self.run_dir(record.id)
run_dir.mkdir(parents=True, exist_ok=True)
(run_dir / "cache").mkdir(exist_ok=True)
atomic_write_text(
self.snapshot_path(record.id),
json.dumps(asdict(record), indent=2, ensure_ascii=True, default=str) + "\n",
)
def append_event(self, run_id: str, event: str, payload: dict[str, Any] | None = None) -> None:
"""Append a journal event."""
entry = {
"ts": time.time(),
"event": event,
"payload": payload or {},
}
with self.journal_path(run_id).open("a", encoding="utf-8") as handle:
handle.write(json.dumps(entry, ensure_ascii=True, default=str) + "\n")
def read_cache(self, run_id: str, key: str) -> Any | None:
"""Read a cached agent result for a run."""
path = self.cache_path(run_id, key)
if not path.exists():
return None
try:
payload = json.loads(path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return None
return payload.get("result")
def write_cache(self, run_id: str, key: str, result: Any) -> None:
"""Write a cached agent result for a run."""
path = self.cache_path(run_id, key)
path.parent.mkdir(parents=True, exist_ok=True)
atomic_write_text(path, json.dumps({"result": result}, ensure_ascii=True, default=str) + "\n")
def clone_run_for_resume(self, run_id: str) -> WorkflowRunRecord | None:
"""Load a run for resume while preserving its cache directory."""
record = self.load_run(run_id)
if record is None:
return None
record.status = "pending"
record.error = None
record.result = None
record.started_at = None
record.ended_at = None
for agent in record.agents.values():
if agent.status not in {"completed", "cached"}:
agent.status = "pending"
agent.error = None
agent.ended_at = None
self.write_snapshot(record)
self.append_event(run_id, "run_resumed", {})
return record
def _record_from_payload(self, payload: dict[str, Any]) -> WorkflowRunRecord:
phases = {
name: WorkflowPhaseRecord(**value)
for name, value in dict(payload.get("phases") or {}).items()
}
agents = {
name: WorkflowAgentRecord(**value)
for name, value in dict(payload.get("agents") or {}).items()
}
payload = dict(payload)
payload["phases"] = phases
payload["agents"] = agents
return WorkflowRunRecord(**payload)
def project_digest(cwd: str | Path) -> str:
"""Return a short stable project digest for future storage partitioning."""
return sha1(str(Path(cwd).resolve()).encode("utf-8")).hexdigest()[:12]
+71
View File
@@ -0,0 +1,71 @@
"""Data models for dynamic workflow runs."""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any, Literal
WorkflowRunStatus = Literal["pending", "running", "paused", "completed", "failed", "killed"]
WorkflowAgentStatus = Literal["pending", "running", "cached", "completed", "failed", "killed"]
WorkflowPhaseStatus = Literal["pending", "running", "completed", "failed"]
@dataclass
class WorkflowAgentRecord:
"""One subagent invocation inside a workflow run."""
id: str
cache_key: str
prompt: str
phase: str | None = None
status: WorkflowAgentStatus = "pending"
task_id: str | None = None
output: str | None = None
error: str | None = None
model: str | None = None
started_at: float | None = None
ended_at: float | None = None
cached: bool = False
metadata: dict[str, Any] = field(default_factory=dict)
@dataclass
class WorkflowPhaseRecord:
"""A named workflow phase shown in progress views."""
name: str
status: WorkflowPhaseStatus = "pending"
started_at: float | None = None
ended_at: float | None = None
agent_count: int = 0
cached_count: int = 0
token_count: int = 0
metadata: dict[str, Any] = field(default_factory=dict)
@dataclass
class WorkflowRunRecord:
"""Runtime snapshot for a workflow run."""
id: str
name: str
cwd: str
script_path: str
script_hash: str
status: WorkflowRunStatus = "pending"
description: str = ""
result: Any = None
error: str | None = None
created_at: float = 0.0
started_at: float | None = None
ended_at: float | None = None
agent_count: int = 0
cached_count: int = 0
max_agents: int = 1000
max_concurrency: int = 1
current_phase: str | None = None
phases: dict[str, WorkflowPhaseRecord] = field(default_factory=dict)
agents: dict[str, WorkflowAgentRecord] = field(default_factory=dict)
logs: list[str] = field(default_factory=list)
metadata: dict[str, Any] = field(default_factory=dict)
+4 -1
View File
@@ -188,9 +188,10 @@ async def test_codex_client_streams_text(monkeypatch):
client = CodexApiClient(_fake_codex_token())
request = ApiMessageRequest(
model="gpt-5.4",
model="gpt-5.5",
messages=[ConversationMessage.from_user_text("hi")],
system_prompt="Be helpful.",
effort="xhigh",
)
events = [event async for event in client.stream_message(request)]
@@ -201,6 +202,8 @@ async def test_codex_client_streams_text(monkeypatch):
assert complete.usage.output_tokens == 3
assert sink["url"].endswith("/codex/responses")
assert sink["json"]["instructions"] == "Be helpful."
assert sink["json"]["model"] == "gpt-5.5"
assert sink["json"]["reasoning"] == {"effort": "xhigh"}
assert sink["headers"]["OpenAI-Beta"] == "responses=experimental"
+62
View File
@@ -11,6 +11,7 @@ import pytest
from openharness.api.client import ApiMessageRequest
from openharness.api.openai_client import (
OpenAICompatibleClient,
_convert_assistant_message,
_convert_messages_to_openai,
_convert_tools_to_openai,
_normalize_openai_base_url,
@@ -435,3 +436,64 @@ class TestStripThinkBlocks:
visible, buf = _strip_think_blocks(buf)
assert visible == "answer"
assert buf == ""
class TestReasoningContentEmission:
"""``reasoning_content`` is a non-standard field. It must round-trip
when the streaming parser captured non-empty reasoning, but the
legacy "emit empty string when there are tool calls" behaviour now
requires opt-in via ``OPENHARNESS_REQUIRE_EMPTY_REASONING_CONTENT=1``.
Strict-OpenAI providers (Cerebras, NVIDIA NIM, OpenAI direct) reject
requests carrying the field with a ``wrong_api_format`` 400, so the
default-off behaviour fixes them out-of-the-box; Kimi-on-Anthropic
users opt in via env var.
"""
def _msg_with_tool_use(self, *, reasoning: str | None = None) -> ConversationMessage:
msg = ConversationMessage(
role="assistant",
content=[
TextBlock(text="ok"),
ToolUseBlock(id="tool_1", name="read_file", input={"path": "x"}),
],
)
if reasoning is not None:
msg._reasoning = reasoning # type: ignore[attr-defined]
return msg
def test_omits_reasoning_when_no_captured_text(self, monkeypatch):
monkeypatch.delenv("OPENHARNESS_REQUIRE_EMPTY_REASONING_CONTENT", raising=False)
out = _convert_assistant_message(self._msg_with_tool_use())
assert "reasoning_content" not in out
def test_replays_captured_reasoning(self, monkeypatch):
monkeypatch.delenv("OPENHARNESS_REQUIRE_EMPTY_REASONING_CONTENT", raising=False)
out = _convert_assistant_message(self._msg_with_tool_use(reasoning="thinking…"))
assert out["reasoning_content"] == "thinking…"
def test_emits_empty_when_opted_in(self, monkeypatch):
monkeypatch.setenv("OPENHARNESS_REQUIRE_EMPTY_REASONING_CONTENT", "1")
out = _convert_assistant_message(self._msg_with_tool_use())
assert out["reasoning_content"] == ""
def test_opt_in_truthy_values(self, monkeypatch):
for v in ("1", "true", "TRUE", "yes", "on"):
monkeypatch.setenv("OPENHARNESS_REQUIRE_EMPTY_REASONING_CONTENT", v)
out = _convert_assistant_message(self._msg_with_tool_use())
assert out.get("reasoning_content") == "", f"value={v!r}"
def test_opt_in_falsy_values(self, monkeypatch):
for v in ("0", "false", "no", "off", ""):
monkeypatch.setenv("OPENHARNESS_REQUIRE_EMPTY_REASONING_CONTENT", v)
out = _convert_assistant_message(self._msg_with_tool_use())
assert "reasoning_content" not in out, f"value={v!r} should not opt in"
def test_no_tool_calls_never_emits_empty(self, monkeypatch):
# Pure-text assistant messages have always omitted the field; the
# opt-in is scoped to tool-use messages where Kimi specifically
# demands the placeholder.
monkeypatch.setenv("OPENHARNESS_REQUIRE_EMPTY_REASONING_CONTENT", "1")
msg = ConversationMessage(role="assistant", content=[TextBlock(text="hi")])
out = _convert_assistant_message(msg)
assert "reasoning_content" not in out
@@ -0,0 +1,64 @@
from __future__ import annotations
import logging
from types import SimpleNamespace
from unittest.mock import AsyncMock
import pytest
from openharness.channels.bus.queue import MessageBus
from openharness.channels.impl.manager import ChannelManager
from openharness.channels.impl.telegram import TelegramChannel, silence_telegram_token_url_loggers
from openharness.config.schema import Config, TelegramConfig
def test_silence_telegram_token_url_loggers_raises_dependency_log_levels():
for name in ("httpx", "httpcore", "telegram.ext"):
logging.getLogger(name).setLevel(logging.INFO)
silence_telegram_token_url_loggers()
for name in ("httpx", "httpcore", "telegram.ext"):
assert logging.getLogger(name).level == logging.WARNING
@pytest.mark.asyncio
async def test_telegram_start_and_help_use_configured_bot_name():
channel = TelegramChannel(TelegramConfig(token="token", bot_name="ohmo", allow_from=["*"]), MessageBus())
message = SimpleNamespace(chat_id=1, reply_text=AsyncMock())
user = SimpleNamespace(first_name="Jabin")
update = SimpleNamespace(message=message, effective_user=user)
await channel._on_start(update, SimpleNamespace())
await channel._on_help(update, SimpleNamespace())
start_text = message.reply_text.await_args_list[0].args[0]
help_text = message.reply_text.await_args_list[1].args[0]
assert "I'm ohmo" in start_text
assert "ohmo commands" in help_text
assert "nanobot" not in start_text
assert "nanobot" not in help_text
@pytest.mark.asyncio
async def test_telegram_error_handler_records_last_error():
channel = TelegramChannel(TelegramConfig(token="token", allow_from=["*"]), MessageBus())
await channel._on_error(None, SimpleNamespace(error=RuntimeError("poll failed")))
assert channel.last_error == "poll failed"
@pytest.mark.asyncio
async def test_channel_manager_records_start_failure_on_channel():
bus = MessageBus()
manager = ChannelManager(Config(), bus)
class BrokenChannel:
async def start(self):
raise RuntimeError("boom")
channel = BrokenChannel()
await manager._start_channel("telegram", channel) # type: ignore[arg-type]
assert getattr(channel, "last_error") == "boom"
+161 -1
View File
@@ -8,12 +8,18 @@ from pathlib import Path
import pytest
import openharness.commands.registry as registry_module
from openharness.commands.registry import CommandContext, create_default_command_registry, lookup_skill_slash_command
from openharness.commands.registry import (
CommandContext,
MemoryCommandBackend,
create_default_command_registry,
lookup_skill_slash_command,
)
from openharness.autopilot import RepoVerificationStep
from openharness.config.paths import get_feedback_log_path, get_project_issue_file, get_project_pr_comments_file
from openharness.config.settings import load_settings, save_settings, Settings
from openharness.engine.messages import ConversationMessage, TextBlock
from openharness.engine.query_engine import QueryEngine
from openharness.memory.paths import get_project_memory_dir
from openharness.mcp.types import McpHttpServerConfig, McpStdioServerConfig
from openharness.permissions import PermissionChecker
from openharness.plugins.types import PluginCommandDefinition
@@ -161,6 +167,87 @@ async def test_bridge_command_supports_explicit_remote_admin_opt_in(tmp_path: Pa
assert getattr(command, "remote_admin_opt_in", False) is True
@pytest.mark.asyncio
async def test_autopilot_command_is_marked_local_only(tmp_path: Path, monkeypatch):
monkeypatch.setenv("OPENHARNESS_CONFIG_DIR", str(tmp_path / "config"))
registry = create_default_command_registry()
command, _ = registry.lookup("/autopilot run-next")
assert command is not None
assert command.remote_invocable is False
@pytest.mark.asyncio
async def test_autopilot_command_supports_explicit_remote_admin_opt_in(tmp_path: Path, monkeypatch):
monkeypatch.setenv("OPENHARNESS_CONFIG_DIR", str(tmp_path / "config"))
registry = create_default_command_registry()
command, _ = registry.lookup("/autopilot run-next")
assert command is not None
assert getattr(command, "remote_admin_opt_in", False) is True
@pytest.mark.asyncio
async def test_diff_command_is_marked_local_only(tmp_path: Path, monkeypatch):
monkeypatch.setenv("OPENHARNESS_CONFIG_DIR", str(tmp_path / "config"))
registry = create_default_command_registry()
command, _ = registry.lookup("/diff full")
assert command is not None
assert command.remote_invocable is False
@pytest.mark.asyncio
async def test_diff_command_supports_explicit_remote_admin_opt_in(tmp_path: Path, monkeypatch):
monkeypatch.setenv("OPENHARNESS_CONFIG_DIR", str(tmp_path / "config"))
registry = create_default_command_registry()
command, _ = registry.lookup("/diff full")
assert command is not None
assert getattr(command, "remote_admin_opt_in", False) is True
@pytest.mark.asyncio
async def test_project_context_commands_are_marked_local_only(tmp_path: Path, monkeypatch):
monkeypatch.setenv("OPENHARNESS_CONFIG_DIR", str(tmp_path / "config"))
registry = create_default_command_registry()
for payload in (
"/issue set Remote supplied issue :: marker",
"/pr_comments add src/app.py:1 :: marker",
):
command, _ = registry.lookup(payload)
assert command is not None
assert command.remote_invocable is False, payload
@pytest.mark.asyncio
async def test_project_context_commands_support_explicit_remote_admin_opt_in(tmp_path: Path, monkeypatch):
monkeypatch.setenv("OPENHARNESS_CONFIG_DIR", str(tmp_path / "config"))
registry = create_default_command_registry()
for payload in (
"/issue set Remote supplied issue :: marker",
"/pr_comments add src/app.py:1 :: marker",
):
command, _ = registry.lookup(payload)
assert command is not None
assert getattr(command, "remote_admin_opt_in", False) is True, payload
@pytest.mark.asyncio
async def test_tasks_command_is_marked_local_only(tmp_path: Path, monkeypatch):
monkeypatch.setenv("OPENHARNESS_CONFIG_DIR", str(tmp_path / "config"))
registry = create_default_command_registry()
command, _ = registry.lookup("/tasks run id")
assert command is not None
assert command.remote_invocable is False
@pytest.mark.asyncio
async def test_tasks_command_supports_explicit_remote_admin_opt_in(tmp_path: Path, monkeypatch):
monkeypatch.setenv("OPENHARNESS_CONFIG_DIR", str(tmp_path / "config"))
registry = create_default_command_registry()
command, _ = registry.lookup("/tasks run id")
assert command is not None
assert getattr(command, "remote_admin_opt_in", False) is True
@pytest.mark.asyncio
async def test_sensitive_control_plane_commands_are_local_only(tmp_path: Path, monkeypatch):
@@ -174,7 +261,11 @@ async def test_sensitive_control_plane_commands_are_local_only(tmp_path: Path, m
"/mcp",
"/provider",
"/model show",
"/commit remote requested commit",
"/ship",
"/resume",
"/resume session-from-another-sender",
"/summary 10",
):
command, _ = registry.lookup(payload)
assert command is not None
@@ -845,6 +936,67 @@ async def test_memory_command_manages_entries(tmp_path: Path, monkeypatch):
remove_result = await remove_command.handler(remove_args, context)
assert "Removed memory entry" in remove_result.message
list_after_remove = await list_command.handler(list_args, context)
assert "pytest_tips.md" not in list_after_remove.message
show_after_remove = await show_command.handler(show_args, context)
assert show_after_remove.message == "Memory entry not found: pytest_tips"
@pytest.mark.asyncio
async def test_memory_command_migrates_entries(tmp_path: Path, monkeypatch):
monkeypatch.setenv("OPENHARNESS_CONFIG_DIR", str(tmp_path / "config"))
monkeypatch.setenv("OPENHARNESS_DATA_DIR", str(tmp_path / "data"))
memory_dir = get_project_memory_dir(tmp_path)
legacy = memory_dir / "legacy.md"
legacy.write_text("legacy command note\n", encoding="utf-8")
registry = create_default_command_registry()
context = _make_context(tmp_path)
dry_command, dry_args = registry.lookup("/memory migrate --dry-run")
dry_result = await dry_command.handler(dry_args, context)
assert "Memory migration dry run." in dry_result.message
assert "Changed: 1" in dry_result.message
assert "schema_version" not in legacy.read_text(encoding="utf-8")
apply_command, apply_args = registry.lookup("/memory migrate --apply")
apply_result = await apply_command.handler(apply_args, context)
assert "Memory migration applied." in apply_result.message
assert "Backup:" in apply_result.message
assert "schema_version: 1" in legacy.read_text(encoding="utf-8")
@pytest.mark.asyncio
async def test_memory_migrate_uses_backend_defaults_not_label(tmp_path: Path, monkeypatch):
monkeypatch.setenv("OPENHARNESS_CONFIG_DIR", str(tmp_path / "config"))
monkeypatch.setenv("OPENHARNESS_DATA_DIR", str(tmp_path / "data"))
memory_dir = tmp_path / "custom-memory"
memory_dir.mkdir()
legacy = memory_dir / "legacy.md"
legacy.write_text("personal preference note\n", encoding="utf-8")
registry = create_default_command_registry()
context = _make_context(tmp_path)
context.memory_backend = MemoryCommandBackend(
label="renamed user notes",
default_type="personal",
default_category="preference",
get_memory_dir=lambda: memory_dir,
get_entrypoint=lambda: memory_dir / "MEMORY.md",
list_files=lambda: [],
add_entry=lambda title, content: memory_dir / f"{title}.md",
remove_entry=lambda name: False,
)
apply_command, apply_args = registry.lookup("/memory migrate --apply")
result = await apply_command.handler(apply_args, context)
migrated = legacy.read_text(encoding="utf-8")
assert "Memory migration applied." in result.message
assert 'type: "personal"' in migrated
assert 'category: "preference"' in migrated
@pytest.mark.asyncio
async def test_compact_summary_and_usage_commands(tmp_path: Path, monkeypatch):
@@ -924,6 +1076,12 @@ async def test_ui_mode_commands_persist_and_update_state(tmp_path: Path, monkeyp
assert load_settings().effort == "high"
assert context.app_state.get().effort == "high"
effort_command, effort_args = registry.lookup("/effort xhigh")
effort_result = await effort_command.handler(effort_args, context)
assert "xhigh" in effort_result.message
assert load_settings().effort == "xhigh"
assert context.app_state.get().effort == "xhigh"
passes_command, passes_args = registry.lookup("/passes 3")
passes_result = await passes_command.handler(passes_args, context)
assert "3" in passes_result.message
@@ -997,7 +1155,9 @@ async def test_agents_session_files_and_reload_plugins_commands(tmp_path: Path,
(tmp_path / "src" / "app.py").write_text("print('hi')\n", encoding="utf-8")
session_command, session_args = registry.lookup("/session")
context.session_id = "session-smoke"
session_result = await session_command.handler(session_args, context)
assert "Session ID: session-smoke" in session_result.message
assert "Session directory:" in session_result.message
session_path_command, session_path_args = registry.lookup("/session path")
+42
View File
@@ -0,0 +1,42 @@
"""Schema regressions for openharness.config.schema channel configs."""
from __future__ import annotations
from openharness.config.schema import TelegramConfig
class TestTelegramConfig:
def test_reply_to_message_field_is_declared_with_true_default(self):
"""Regression for #243: every outbound send raises AttributeError when
``reply_to_message`` is not in the parsed config because the field
existed only as an interactive ``ohmo init`` prompt, not on the
pydantic model. The CLI default is ``True``; the schema default
mirrors that so non-interactive and hand-written configs behave the
same as interactive configs accepting the default.
"""
config = TelegramConfig()
assert config.reply_to_message is True
def test_reply_to_message_accessible_when_legacy_config_omits_field(self):
"""``ohmo init --no-interactive`` and pre-0.1.9 hand-written
``gateway.json`` files don't include ``reply_to_message``. Attribute
access on the parsed instance must not raise that AttributeError
was the root cause of #243 (every outbound Telegram send crashed).
"""
config = TelegramConfig.model_validate(
{"token": "test-token", "chat_id": "12345", "allow_from": ["12345"]}
)
assert config.reply_to_message is True
def test_reply_to_message_explicit_false_is_honored(self):
config = TelegramConfig.model_validate(
{"token": "t", "chat_id": "1", "reply_to_message": False}
)
assert config.reply_to_message is False
def test_bot_name_defaults_to_ohmo(self):
config = TelegramConfig()
assert config.bot_name == "ohmo"
+61
View File
@@ -38,16 +38,26 @@ class TestSettings:
assert s.resolve_api_key() == "sk-test-123"
def test_resolve_api_key_from_env(self, monkeypatch):
monkeypatch.delenv("OPENHARNESS_ANTHROPIC_API_KEY", raising=False)
monkeypatch.setenv("ANTHROPIC_API_KEY", "sk-env-456")
s = Settings()
assert s.resolve_api_key() == "sk-env-456"
def test_resolve_api_key_prefers_openharness_env(self, monkeypatch):
monkeypatch.setenv("OPENHARNESS_ANTHROPIC_API_KEY", "sk-oh-456")
monkeypatch.setenv("ANTHROPIC_API_KEY", "sk-env-456")
s = Settings()
assert s.resolve_api_key() == "sk-oh-456"
def test_resolve_api_key_instance_takes_precedence(self, monkeypatch):
monkeypatch.delenv("OPENHARNESS_ANTHROPIC_API_KEY", raising=False)
monkeypatch.setenv("ANTHROPIC_API_KEY", "sk-env-456")
s = Settings(api_key="sk-instance-789")
assert s.resolve_api_key() == "sk-instance-789"
def test_resolve_api_key_missing_raises(self, monkeypatch):
monkeypatch.delenv("OPENHARNESS_ANTHROPIC_API_KEY", raising=False)
monkeypatch.delenv("OPENHARNESS_OPENAI_API_KEY", raising=False)
monkeypatch.delenv("ANTHROPIC_API_KEY", raising=False)
monkeypatch.delenv("OPENAI_API_KEY", raising=False)
s = Settings()
@@ -62,6 +72,12 @@ class TestSettings:
# api_key=None should not override the default
assert updated.api_key == ""
def test_merge_cli_overrides_applies_permission_mode(self):
s = Settings()
updated = s.merge_cli_overrides(permission_mode="full_auto")
assert updated.permission.mode == "full_auto"
assert s.permission.mode == "default"
def test_merge_cli_overrides_returns_new_instance(self):
s = Settings()
updated = s.merge_cli_overrides(model="claude-opus-4-20250514")
@@ -72,6 +88,7 @@ class TestSettings:
"""When api_format=openai, resolve_auth() should use OPENAI_API_KEY
from the environment rather than the flat api_key field which may
contain an Anthropic key from settings.json."""
monkeypatch.delenv("OPENHARNESS_OPENAI_API_KEY", raising=False)
monkeypatch.setenv("OPENAI_API_KEY", "sk-openai-correct")
monkeypatch.delenv("ANTHROPIC_API_KEY", raising=False)
s = Settings(api_key="sk-ant-wrong-provider", api_format="openai")
@@ -80,9 +97,21 @@ class TestSettings:
assert auth.value == "sk-openai-correct"
assert "OPENAI" in auth.source
def test_resolve_auth_prefers_openharness_env_for_openai(self, monkeypatch):
monkeypatch.setenv("OPENHARNESS_OPENAI_API_KEY", "sk-oh-openai")
monkeypatch.setenv("OPENAI_API_KEY", "sk-openai-correct")
monkeypatch.delenv("ANTHROPIC_API_KEY", raising=False)
s = Settings(api_key="sk-ant-wrong-provider", api_format="openai")
s = s.sync_active_profile_from_flat_fields()
auth = s.resolve_auth()
assert auth.value == "sk-oh-openai"
assert auth.source == "env:OPENHARNESS_OPENAI_API_KEY"
def test_resolve_auth_falls_back_to_flat_api_key(self, monkeypatch):
"""When no provider-specific env var is set, resolve_auth() should
still fall back to the flat api_key field."""
monkeypatch.delenv("OPENHARNESS_ANTHROPIC_API_KEY", raising=False)
monkeypatch.delenv("OPENHARNESS_OPENAI_API_KEY", raising=False)
monkeypatch.delenv("ANTHROPIC_API_KEY", raising=False)
monkeypatch.delenv("OPENAI_API_KEY", raising=False)
s = Settings(api_key="sk-fallback-key")
@@ -103,6 +132,32 @@ class TestSettings:
s = load_settings(path)
assert s.base_url == "https://relay.example.com/v1"
def test_load_settings_uses_profile_specific_openharness_env_key(self, tmp_path: Path, monkeypatch):
monkeypatch.setenv("ANTHROPIC_API_KEY", "sk-ant-wrong")
monkeypatch.setenv("OPENHARNESS_OPENAI_API_KEY", "sk-oh-openai")
monkeypatch.delenv("OPENAI_API_KEY", raising=False)
path = tmp_path / "settings.json"
path.write_text(
Settings(active_profile="openai-compatible").model_dump_json(),
encoding="utf-8",
)
s = load_settings(path)
assert s.active_profile == "openai-compatible"
assert s.api_key == "sk-oh-openai"
def test_load_settings_ignores_wrong_provider_native_env_key(self, tmp_path: Path, monkeypatch):
monkeypatch.setenv("ANTHROPIC_API_KEY", "sk-ant-wrong")
monkeypatch.delenv("OPENHARNESS_OPENAI_API_KEY", raising=False)
monkeypatch.delenv("OPENAI_API_KEY", raising=False)
path = tmp_path / "settings.json"
path.write_text(
Settings(active_profile="openai-compatible").model_dump_json(),
encoding="utf-8",
)
s = load_settings(path)
assert s.active_profile == "openai-compatible"
assert s.api_key == ""
def test_env_overrides_pick_up_compact_threshold_settings(self, tmp_path: Path, monkeypatch):
monkeypatch.setenv("OPENHARNESS_CONTEXT_WINDOW_TOKENS", "123456")
monkeypatch.setenv("OPENHARNESS_AUTO_COMPACT_THRESHOLD_TOKENS", "120000")
@@ -125,6 +180,8 @@ class TestSettings:
class TestLoadSaveSettings:
def test_load_missing_file_returns_defaults(self, tmp_path: Path, monkeypatch):
monkeypatch.delenv("OPENHARNESS_ANTHROPIC_API_KEY", raising=False)
monkeypatch.delenv("OPENHARNESS_OPENAI_API_KEY", raising=False)
monkeypatch.delenv("ANTHROPIC_API_KEY", raising=False)
monkeypatch.delenv("OPENAI_API_KEY", raising=False)
monkeypatch.delenv("ANTHROPIC_BASE_URL", raising=False)
@@ -137,6 +194,8 @@ class TestLoadSaveSettings:
assert s == Settings().materialize_active_profile()
def test_load_existing_file(self, tmp_path: Path, monkeypatch):
monkeypatch.delenv("OPENHARNESS_ANTHROPIC_API_KEY", raising=False)
monkeypatch.delenv("OPENHARNESS_OPENAI_API_KEY", raising=False)
monkeypatch.delenv("ANTHROPIC_API_KEY", raising=False)
monkeypatch.delenv("OPENAI_API_KEY", raising=False)
monkeypatch.delenv("ANTHROPIC_BASE_URL", raising=False)
@@ -152,6 +211,8 @@ class TestLoadSaveSettings:
assert s.api_key == "" # default preserved
def test_save_and_load_roundtrip(self, tmp_path: Path, monkeypatch):
monkeypatch.delenv("OPENHARNESS_ANTHROPIC_API_KEY", raising=False)
monkeypatch.delenv("OPENHARNESS_OPENAI_API_KEY", raising=False)
monkeypatch.delenv("ANTHROPIC_API_KEY", raising=False)
monkeypatch.delenv("OPENAI_API_KEY", raising=False)
monkeypatch.delenv("ANTHROPIC_BASE_URL", raising=False)
+89 -1
View File
@@ -1182,7 +1182,7 @@ class _OkTool(BaseTool):
async def execute(self, arguments: BaseModel, context: ToolExecutionContext) -> ToolResult:
del arguments, context
return ToolResult(output="ok")
return ToolResult(output="ok", metadata={"sentinel": "metadata"})
class _BoomTool(BaseTool):
@@ -1198,6 +1198,64 @@ class _BoomTool(BaseTool):
raise RuntimeError("boom")
@pytest.mark.asyncio
async def test_query_engine_synthesizes_tool_result_when_single_tool_raises(tmp_path: Path):
registry = ToolRegistry()
registry.register(_BoomTool())
engine = QueryEngine(
api_client=FakeApiClient(
[
_FakeResponse(
message=ConversationMessage(
role="assistant",
content=[
TextBlock(text="Running one tool."),
ToolUseBlock(id="toolu_boom", name="boom_tool", input={}),
],
),
usage=UsageSnapshot(input_tokens=1, output_tokens=1),
),
_FakeResponse(
message=ConversationMessage(
role="assistant",
content=[TextBlock(text="Recovered from the failure.")],
),
usage=UsageSnapshot(input_tokens=1, output_tokens=1),
),
]
),
tool_registry=registry,
permission_checker=PermissionChecker(PermissionSettings(mode=PermissionMode.FULL_AUTO)),
cwd=tmp_path,
model="claude-test",
system_prompt="system",
)
events = [event async for event in engine.submit_message("run one tool")]
completed = [event for event in events if isinstance(event, ToolExecutionCompleted)]
assert len(completed) == 1
assert completed[0].tool_name == "boom_tool"
assert completed[0].is_error is True
assert "RuntimeError" in completed[0].output
assert "boom" in completed[0].output
user_tool_messages = [
msg
for msg in engine.messages
if msg.role == "user" and any(isinstance(block, ToolResultBlock) for block in msg.content)
]
assert len(user_tool_messages) == 1
result_blocks = [
block for block in user_tool_messages[0].content if isinstance(block, ToolResultBlock)
]
assert result_blocks[0].tool_use_id == "toolu_boom"
assert isinstance(events[-1], AssistantTurnComplete)
assert events[-1].message.text == "Recovered from the failure."
class _LargeOutputTool(BaseTool):
name = "mcp__playwright__browser_snapshot"
description = "Returns a large browser snapshot."
@@ -1340,6 +1398,7 @@ async def test_query_engine_synthesizes_tool_result_when_parallel_tool_raises(tm
assert set(completed_by_name) == {"ok_tool", "boom_tool"}
assert completed_by_name["ok_tool"].is_error is False
assert completed_by_name["ok_tool"].output == "ok"
assert completed_by_name["ok_tool"].metadata == {"sentinel": "metadata"}
assert completed_by_name["boom_tool"].is_error is True
assert "RuntimeError" in completed_by_name["boom_tool"].output
assert "boom" in completed_by_name["boom_tool"].output
@@ -1384,6 +1443,35 @@ async def test_query_engine_sanitizes_dangling_tool_use_before_new_prompt(tmp_pa
)
@pytest.mark.asyncio
async def test_query_engine_continue_pending_sanitizes_dangling_tool_use(tmp_path: Path):
engine = QueryEngine(
api_client=StaticApiClient("continued reply"),
tool_registry=ToolRegistry(),
permission_checker=PermissionChecker(PermissionSettings(mode=PermissionMode.FULL_AUTO)),
cwd=tmp_path,
model="claude-test",
system_prompt="system",
)
engine.load_messages([
ConversationMessage.from_user_text("previous request"),
ConversationMessage(
role="assistant",
content=[ToolUseBlock(id="call_missing_output", name="ok_tool", input={})],
),
])
events = [event async for event in engine.continue_pending()]
assert isinstance(events[-1], AssistantTurnComplete)
assert events[-1].message.text == "continued reply"
assert not any(
isinstance(block, ToolUseBlock) and block.id == "call_missing_output"
for message in engine.messages
for block in message.content
)
@pytest.mark.asyncio
async def test_query_engine_offloads_large_tool_result_outputs(tmp_path: Path, monkeypatch):
monkeypatch.setenv("OPENHARNESS_DATA_DIR", str(tmp_path / "data"))
+99
View File
@@ -0,0 +1,99 @@
"""Tests for hook priority ordering."""
from __future__ import annotations
from pathlib import Path
import pytest
from openharness.api.client import ApiMessageCompleteEvent
from openharness.api.usage import UsageSnapshot
from openharness.engine.messages import ConversationMessage, TextBlock
from openharness.hooks import HookEvent, HookExecutionContext, HookExecutor
from openharness.hooks.loader import HookRegistry
from openharness.hooks.schemas import CommandHookDefinition, HttpHookDefinition
class FakeApiClient:
"""Minimal fake streaming client."""
def __init__(self, text: str) -> None:
self._text = text
async def stream_message(self, request):
del request
yield ApiMessageCompleteEvent(
message=ConversationMessage(role="assistant", content=[TextBlock(text=self._text)]),
usage=UsageSnapshot(input_tokens=1, output_tokens=1),
stop_reason=None,
)
def test_priority_defaults_to_zero():
assert CommandHookDefinition(command="true").priority == 0
assert HttpHookDefinition(url="https://example.invalid").priority == 0
def test_registry_orders_by_priority_descending():
registry = HookRegistry()
registry.register(HookEvent.PRE_TOOL_USE, CommandHookDefinition(command="low", priority=1))
registry.register(HookEvent.PRE_TOOL_USE, CommandHookDefinition(command="high", priority=10))
registry.register(HookEvent.PRE_TOOL_USE, CommandHookDefinition(command="mid", priority=5))
commands = [hook.command for hook in registry.get(HookEvent.PRE_TOOL_USE)]
assert commands == ["high", "mid", "low"]
def test_registry_ties_keep_registration_order():
"""sorted() is stable, so equal priorities preserve insertion order."""
registry = HookRegistry()
registry.register(HookEvent.PRE_TOOL_USE, CommandHookDefinition(command="first", priority=5))
registry.register(HookEvent.PRE_TOOL_USE, CommandHookDefinition(command="second", priority=5))
registry.register(HookEvent.PRE_TOOL_USE, CommandHookDefinition(command="third", priority=5))
commands = [hook.command for hook in registry.get(HookEvent.PRE_TOOL_USE)]
assert commands == ["first", "second", "third"]
def test_negative_priority_runs_last():
registry = HookRegistry()
registry.register(HookEvent.SESSION_START, CommandHookDefinition(command="cleanup", priority=-10))
registry.register(HookEvent.SESSION_START, CommandHookDefinition(command="default"))
commands = [hook.command for hook in registry.get(HookEvent.SESSION_START)]
assert commands == ["default", "cleanup"]
def test_summary_includes_non_default_priority():
registry = HookRegistry()
registry.register(HookEvent.PRE_TOOL_USE, CommandHookDefinition(command="guard", priority=10))
registry.register(HookEvent.PRE_TOOL_USE, CommandHookDefinition(command="logger"))
summary = registry.summary()
assert "priority=10" in summary
# A default (zero) priority is not noisily printed.
assert "priority=0" not in summary
@pytest.mark.asyncio
async def test_executor_runs_hooks_in_priority_order(tmp_path: Path):
"""End-to-end: execute() honours the priority-sorted registry order."""
registry = HookRegistry()
registry.register(HookEvent.PRE_TOOL_USE, CommandHookDefinition(command="printf 'low'", priority=1))
registry.register(HookEvent.PRE_TOOL_USE, CommandHookDefinition(command="printf 'high'", priority=10))
executor = HookExecutor(
registry,
HookExecutionContext(
cwd=tmp_path,
api_client=FakeApiClient('{"ok": true}'),
default_model="claude-test",
),
)
result = await executor.execute(HookEvent.PRE_TOOL_USE, {"tool_name": "bash"})
assert [hook.output for hook in result.results] == ["high", "low"]
@@ -0,0 +1,223 @@
"""Tests for Claude Code-inspired memory runtime behavior."""
from __future__ import annotations
import os
import time
from pathlib import Path
import pytest
from openharness.api.client import ApiMessageCompleteEvent
from openharness.api.usage import UsageSnapshot
from openharness.commands.registry import CommandContext, create_default_command_registry
from openharness.config.settings import Settings
from openharness.engine.messages import ConversationMessage, ToolUseBlock
from openharness.engine.query_engine import QueryEngine
from openharness.memory import add_memory_entry, get_project_memory_dir
from openharness.memory.agent import (
ensure_agent_memory_vault,
get_agent_memory_entrypoint,
initialize_agent_memory_from_snapshot,
)
from openharness.memory.relevance import format_relevant_memories, select_relevant_memories
from openharness.memory.schema import (
memory_freshness_text,
parse_memory_scope,
parse_memory_type,
truncate_entrypoint_content,
)
from openharness.memory.team import (
check_team_memory_secrets,
ensure_team_memory_vault,
validate_team_memory_write_path,
)
from openharness.permissions import PermissionChecker
from openharness.services.memory_extract import (
extract_memories_from_turn,
has_memory_writes_since,
parse_extraction_records,
validate_extraction_tool_request,
)
from openharness.services.session_memory import (
get_session_memory_content,
get_session_memory_path,
update_session_memory_file,
)
from openharness.tools import create_default_tool_registry
class _FakeApiClient:
def __init__(self, text: str = "done") -> None:
self.text = text
self.requests = []
async def stream_message(self, request):
self.requests.append(request)
yield ApiMessageCompleteEvent(
message=ConversationMessage.from_user_text(self.text).model_copy(update={"role": "assistant"}),
usage=UsageSnapshot(input_tokens=1, output_tokens=1),
stop_reason="end_turn",
)
def test_schema_truncation_and_freshness() -> None:
raw = "\n".join(f"- item {idx}" for idx in range(240))
view = truncate_entrypoint_content(raw, max_lines=10, max_bytes=1_000)
assert view.was_truncated is True
assert "WARNING" in view.content
assert parse_memory_type("note", default="project") == "project"
assert parse_memory_scope("personal") == "private"
assert "2 days old" in memory_freshness_text(time.time() - 2 * 86_400)
def test_relevance_formats_staleness(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("OPENHARNESS_DATA_DIR", str(tmp_path / "data"))
project = tmp_path / "repo"
project.mkdir()
path = add_memory_entry(project, "Redis Rule", "Redis cache decisions matter.", memory_type="project")
old = time.time() - 3 * 86_400
os.utime(path, (old, old))
selected = select_relevant_memories("redis cache", project)
rendered = format_relevant_memories(selected)
assert selected
assert "3 days old" in rendered
assert "Redis cache decisions" in rendered
@pytest.mark.asyncio
async def test_memory_extraction_writes_typed_note(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("OPENHARNESS_DATA_DIR", str(tmp_path / "data"))
project = tmp_path / "repo"
project.mkdir()
api = _FakeApiClient(
'{"memories":[{"title":"Testing Policy","type":"feedback","scope":"project",'
'"description":"real database tests","body":"Use real database tests for migrations.",'
'"tags":["testing"]}]}'
)
messages = [
ConversationMessage.from_user_text("do not mock database migrations"),
ConversationMessage.from_user_text("noted").model_copy(update={"role": "assistant"}),
]
result = await extract_memories_from_turn(cwd=project, api_client=api, model="claude-test", messages=messages)
assert result.skipped is False
assert len(result.written_paths) == 1
text = result.written_paths[0].read_text(encoding="utf-8")
assert 'type: "feedback"' in text
assert 'scope: "project"' in text
assert "Use real database tests" in text
def test_memory_extraction_parser_and_tool_guard(tmp_path: Path) -> None:
records = parse_extraction_records(
'{"memories":[{"title":"User Role","type":"user","scope":"private","body":"User knows Go."}]}'
)
assert records[0].memory_type == "user"
assert records[0].scope == "private"
ok, _ = validate_extraction_tool_request("bash", {"command": "rg memory src"}, tmp_path)
denied, reason = validate_extraction_tool_request("write_file", {"path": "../x", "content": "x"}, tmp_path)
assert ok is True
assert denied is False
assert "within" in reason
def test_memory_write_detection_resolves_relative_paths_from_cwd(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setenv("OPENHARNESS_DATA_DIR", str(tmp_path / "data"))
project = tmp_path / "repo"
project.mkdir()
memory_dir = get_project_memory_dir(project)
normal_relative_write = ConversationMessage(
role="assistant",
content=[ToolUseBlock(name="write_file", input={"path": "REPORT.md", "content": "ok"})],
)
memory_absolute_write = ConversationMessage(
role="assistant",
content=[ToolUseBlock(name="write_file", input={"path": str(memory_dir / "report.md")})],
)
assert has_memory_writes_since([normal_relative_write], memory_dir, cwd=project) is False
assert has_memory_writes_since([memory_absolute_write], memory_dir, cwd=project) is True
def test_session_memory_file_round_trip(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("OPENHARNESS_DATA_DIR", str(tmp_path / "data"))
project = tmp_path / "repo"
project.mkdir()
messages = [ConversationMessage.from_user_text("current task: finish memory runtime")]
path = update_session_memory_file(project, messages, session_id="abc")
assert path == get_session_memory_path(project, "abc")
assert "finish memory runtime" in get_session_memory_content(path)
def test_team_memory_guards_and_agent_memory(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("OPENHARNESS_DATA_DIR", str(tmp_path / "data"))
project = tmp_path / "repo"
project.mkdir()
team_dir = ensure_team_memory_vault(project)
valid_path, error = validate_team_memory_write_path(project, "notes/shared.md")
escaped_path, escaped_error = validate_team_memory_write_path(project, "../outside.md")
agent_dir = ensure_agent_memory_vault(project, "reviewer/bot", "project")
assert team_dir.exists()
assert valid_path is not None and error is None
assert escaped_path is None and escaped_error
assert check_team_memory_secrets("OPENAI_API_KEY=sk-12345678901234567890")
assert agent_dir.exists()
assert get_agent_memory_entrypoint(project, "reviewer/bot", "project").exists()
def test_agent_memory_snapshot_initializes(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("OPENHARNESS_DATA_DIR", str(tmp_path / "data"))
project = tmp_path / "repo"
project.mkdir()
snapshot = project / ".openharness" / "agent-memory-snapshots" / "reviewer"
snapshot.mkdir(parents=True)
(snapshot / "MEMORY.md").write_text("# Snapshot\n", encoding="utf-8")
target = initialize_agent_memory_from_snapshot(project, "reviewer", "project")
assert target is not None
assert (target / "MEMORY.md").read_text(encoding="utf-8") == "# Snapshot\n"
@pytest.mark.asyncio
async def test_memory_commands_expose_session_team_and_agent(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("OPENHARNESS_DATA_DIR", str(tmp_path / "data"))
project = tmp_path / "repo"
project.mkdir()
engine = QueryEngine(
api_client=_FakeApiClient(),
tool_registry=create_default_tool_registry(),
permission_checker=PermissionChecker(Settings().permission),
cwd=project,
model="claude-test",
system_prompt="system",
)
context = CommandContext(engine=engine, cwd=str(project), session_id="s1")
registry = create_default_command_registry()
for raw in ("/memory session", "/memory team", "/memory agent status reviewer project"):
command, args = registry.lookup(raw)
assert command is not None
result = await command.handler(args, context)
assert result.message
command, args = registry.lookup("/memory add --type feedback --scope private Style :: Keep answers concise.")
assert command is not None
result = await command.handler(args, context)
assert "Added memory entry" in (result.message or "")
assert 'type: "feedback"' in (get_project_memory_dir(project) / "style.md").read_text(encoding="utf-8")
+101
View File
@@ -5,12 +5,16 @@ from __future__ import annotations
from pathlib import Path
from openharness.memory import (
add_memory_entry,
find_relevant_memories,
get_memory_entrypoint,
get_project_memory_dir,
load_memory_prompt,
migrate_memory,
remove_memory_entry,
)
from openharness.memory.scan import _parse_memory_file, scan_memory_files
from openharness.memory.usage import find_stale_memory_candidates, load_usage_index, mark_memory_used
def test_memory_paths_are_stable(tmp_path: Path, monkeypatch):
@@ -53,6 +57,103 @@ def test_find_relevant_memories(tmp_path: Path, monkeypatch):
assert matches[0].path.name == "pytest_tips.md"
def test_add_memory_entry_writes_schema_and_dedupes(tmp_path: Path, monkeypatch):
monkeypatch.setenv("OPENHARNESS_DATA_DIR", str(tmp_path / "data"))
project_dir = tmp_path / "repo"
project_dir.mkdir()
first = add_memory_entry(project_dir, "Pytest Tips", "Use fixtures for setup.")
second = add_memory_entry(project_dir, "Pytest Tips Copy", "Use fixtures for setup.")
assert second == first
headers = scan_memory_files(project_dir)
assert len(headers) == 1
assert headers[0].id.startswith("mem-")
assert headers[0].schema_version == 1
assert headers[0].source == "manual"
assert headers[0].importance == 1
assert headers[0].signature
def test_remove_memory_entry_soft_deletes_and_hides_entry(tmp_path: Path, monkeypatch):
monkeypatch.setenv("OPENHARNESS_DATA_DIR", str(tmp_path / "data"))
project_dir = tmp_path / "repo"
project_dir.mkdir()
path = add_memory_entry(project_dir, "Remove Me", "temporary note")
assert remove_memory_entry(project_dir, "remove_me") is True
assert path.exists()
assert scan_memory_files(project_dir) == []
headers = scan_memory_files(project_dir, include_disabled=True)
assert len(headers) == 1
assert headers[0].disabled is True
def test_migrate_memory_backfills_schema(tmp_path: Path, monkeypatch):
monkeypatch.setenv("OPENHARNESS_DATA_DIR", str(tmp_path / "data"))
project_dir = tmp_path / "repo"
project_dir.mkdir()
memory_dir = get_project_memory_dir(project_dir)
legacy = memory_dir / "legacy.md"
legacy.write_text("Legacy deployment note.\n", encoding="utf-8")
dry_run = migrate_memory(project_dir, apply=False)
assert dry_run.dry_run is True
assert dry_run.changed == 1
assert "schema_version" not in legacy.read_text(encoding="utf-8")
applied = migrate_memory(project_dir, apply=True)
assert applied.dry_run is False
assert applied.changed == 1
assert applied.backup_dir
migrated = legacy.read_text(encoding="utf-8")
assert "schema_version: 1" in migrated
assert "source: \"migration\"" in migrated
assert "Legacy deployment note." in migrated
rerun = migrate_memory(project_dir, apply=False)
assert rerun.changed == 0
def test_usage_index_marks_recalled_memories_and_stale_candidates(tmp_path: Path, monkeypatch):
monkeypatch.setenv("OPENHARNESS_DATA_DIR", str(tmp_path / "data"))
project_dir = tmp_path / "repo"
project_dir.mkdir()
memory_dir = get_project_memory_dir(project_dir)
(memory_dir / "old.md").write_text(
"---\n"
"schema_version: 1\n"
"id: \"mem-old\"\n"
"name: \"old\"\n"
"description: \"old note\"\n"
"type: \"project\"\n"
"category: \"knowledge\"\n"
"importance: 1\n"
"source: \"test\"\n"
"signature: \"sig-old\"\n"
"created_at: \"2020-01-01T00:00:00Z\"\n"
"updated_at: \"2020-01-01T00:00:00Z\"\n"
"ttl_days: null\n"
"disabled: false\n"
"supersedes: []\n"
"---\n\n"
"Old content.\n",
encoding="utf-8",
)
header = scan_memory_files(project_dir)[0]
assert [item.id for item in find_stale_memory_candidates(project_dir)] == ["mem-old"]
mark_memory_used(project_dir, [header])
index = load_usage_index(project_dir)
assert index["memories"]["mem-old"]["use_count"] == 1
assert find_stale_memory_candidates(project_dir) == []
# --- Frontmatter parsing tests ---

Some files were not shown because too many files have changed in this diff Show More