chore: import upstream snapshot with attribution
E2E Headed Chrome / e2e-headed (macos-15) (push) Has been cancelled
E2E Headed Chrome / e2e-headed (ubuntu-latest) (push) Has been cancelled
E2E Headed Chrome / e2e-headed (windows-latest) (push) Has been cancelled
CI / build (macos-latest) (push) Has been cancelled
CI / build (ubuntu-latest) (push) Has been cancelled
CI / build (windows-latest) (push) Has been cancelled
CI / unit-test (push) Has been cancelled
CI / bun-test (push) Has been cancelled
CI / adapter-test (push) Has been cancelled
CI / smoke-test (macos-latest) (push) Has been cancelled
CI / smoke-test (ubuntu-latest) (push) Has been cancelled
Security Audit / audit (push) Has been cancelled
Build Chrome Extension / build (push) Has been cancelled
Trigger Website Rebuild (Docs Updated) / dispatch (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 12:39:48 +08:00
commit 9b395f5cc3
2400 changed files with 376116 additions and 0 deletions
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,857 @@
# Daemon Lifecycle Redesign Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace the daemon's aggressive 5-minute idle timeout with a long-lived model (4h default) that requires both CLI inactivity AND Extension disconnection before exiting, plus add `daemon status/stop/restart` CLI commands.
**Architecture:** The daemon keeps its existing HTTP + WebSocket bridge architecture. We change the idle timeout logic to track two independent activity signals (CLI requests and Extension connection), add `/status` and `/shutdown` HTTP endpoints, reduce the Extension reconnect backoff cap, and register new CLI commands via Commander.js.
**Tech Stack:** Node.js, TypeScript, Commander.js, ws, Vitest
---
## File Structure
| File | Action | Responsibility |
|------|--------|----------------|
| `src/constants.ts` | Modify | Add `DEFAULT_DAEMON_IDLE_TIMEOUT` constant |
| `src/daemon.ts` | Modify | Dual-condition idle timer, `/status` endpoint, `/shutdown` endpoint |
| `src/daemon.test.ts` | Create | Unit tests for idle timer logic, `/status`, `/shutdown` |
| `extension/src/protocol.ts` | Modify | Change `WS_RECONNECT_MAX_DELAY` from 60000 to 5000 |
| `src/cli.ts` | Modify | Register `daemon` subcommand group |
| `src/commands/daemon.ts` | Create | `status`, `stop`, `restart` subcommand implementations |
| `src/commands/daemon.test.ts` | Create | Unit tests for daemon commands |
| `src/browser/mcp.ts` | Modify | Better connection-waiting UX messages, 200ms poll interval |
---
### Task 1: Add `DEFAULT_DAEMON_IDLE_TIMEOUT` constant
**Files:**
- Modify: `src/constants.ts`
- [ ] **Step 1: Add the constant**
In `src/constants.ts`, add after the `DEFAULT_DAEMON_PORT` line:
```typescript
/** Default idle timeout before daemon auto-exits (ms). */
export const DEFAULT_DAEMON_IDLE_TIMEOUT = 4 * 60 * 60 * 1000; // 4 hours
```
- [ ] **Step 2: Commit**
```bash
git add src/constants.ts
git commit -m "feat(daemon): add DEFAULT_DAEMON_IDLE_TIMEOUT constant (4 hours)"
```
---
### Task 2: Implement dual-condition idle timer in daemon
**Files:**
- Modify: `src/daemon.ts:27,29-57,116-123,196-198,245-262,265-269`
- Test: `src/daemon.test.ts` (create)
- [ ] **Step 1: Write failing tests for the new idle timer logic**
Create `src/daemon.test.ts`:
```typescript
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
// We test the idle timer logic by extracting it into testable functions.
// The daemon module has side effects (starts server), so we test the logic unit directly.
describe('IdleManager', () => {
beforeEach(() => {
vi.useFakeTimers();
});
afterEach(() => {
vi.useRealTimers();
});
it('does not start timer when extension is connected', async () => {
const { IdleManager } = await import('./daemon.js');
const exit = vi.fn();
const mgr = new IdleManager(300_000, exit); // 5 min for fast test
mgr.setExtensionConnected(true);
mgr.onCliRequest();
vi.advanceTimersByTime(300_000 + 1000);
expect(exit).not.toHaveBeenCalled();
});
it('starts timer when extension disconnects and CLI is idle', async () => {
const { IdleManager } = await import('./daemon.js');
const exit = vi.fn();
const mgr = new IdleManager(300_000, exit);
mgr.onCliRequest(); // CLI was active
mgr.setExtensionConnected(true);
mgr.setExtensionConnected(false); // Extension disconnects
// Should not exit immediately — CLI was just active
expect(exit).not.toHaveBeenCalled();
// Advance past timeout
vi.advanceTimersByTime(300_000 + 1000);
expect(exit).toHaveBeenCalledTimes(1);
});
it('exits immediately on extension disconnect if CLI has been idle past timeout', async () => {
const { IdleManager } = await import('./daemon.js');
const exit = vi.fn();
const mgr = new IdleManager(300_000, exit);
mgr.onCliRequest(); // Last CLI activity
vi.advanceTimersByTime(400_000); // 400s elapsed — past 300s timeout
mgr.setExtensionConnected(true);
mgr.setExtensionConnected(false);
expect(exit).toHaveBeenCalledTimes(1);
});
it('resets timer on new CLI request', async () => {
const { IdleManager } = await import('./daemon.js');
const exit = vi.fn();
const mgr = new IdleManager(300_000, exit);
mgr.onCliRequest();
vi.advanceTimersByTime(200_000); // 200s elapsed
mgr.onCliRequest(); // Reset
vi.advanceTimersByTime(200_000); // 200s more — only 200s since last request
expect(exit).not.toHaveBeenCalled();
vi.advanceTimersByTime(100_001); // Now 300s+ since last request
expect(exit).toHaveBeenCalledTimes(1);
});
it('does not exit when timeout is 0 (disabled)', async () => {
const { IdleManager } = await import('./daemon.js');
const exit = vi.fn();
const mgr = new IdleManager(0, exit);
mgr.onCliRequest();
vi.advanceTimersByTime(24 * 60 * 60 * 1000); // 24 hours
expect(exit).not.toHaveBeenCalled();
});
it('clears timer when extension connects', async () => {
const { IdleManager } = await import('./daemon.js');
const exit = vi.fn();
const mgr = new IdleManager(300_000, exit);
mgr.onCliRequest();
vi.advanceTimersByTime(200_000); // Timer running
mgr.setExtensionConnected(true); // Should clear timer
vi.advanceTimersByTime(200_000); // Would have fired
expect(exit).not.toHaveBeenCalled();
});
});
```
- [ ] **Step 2: Run tests to verify they fail**
```bash
npx vitest run src/daemon.test.ts
```
Expected: FAIL — `IdleManager` is not exported from `./daemon.js`
- [ ] **Step 3: Extract IdleManager class and refactor daemon.ts**
In `src/daemon.ts`, replace the idle timeout section (lines 27, 29-57) with:
Replace the `IDLE_TIMEOUT` constant (line 27):
```typescript
import { DEFAULT_DAEMON_PORT, DEFAULT_DAEMON_IDLE_TIMEOUT } from './constants.js';
const PORT = DEFAULT_DAEMON_PORT;
const IDLE_TIMEOUT = DEFAULT_DAEMON_IDLE_TIMEOUT;
```
Replace the idle timer state and `resetIdleTimer` function (lines 37, 49-57) with the `IdleManager` class:
```typescript
/**
* Manages daemon idle timeout with dual-condition logic:
* exits only when BOTH CLI is idle AND Extension is disconnected.
*/
export class IdleManager {
private _timer: ReturnType<typeof setTimeout> | null = null;
private _lastCliRequestTime = Date.now();
private _extensionConnected = false;
private _timeoutMs: number;
private _onExit: () => void;
constructor(timeoutMs: number, onExit: () => void) {
this._timeoutMs = timeoutMs;
this._onExit = onExit;
}
/** Call when an HTTP request arrives from CLI */
onCliRequest(): void {
this._lastCliRequestTime = Date.now();
this._resetTimer();
}
/** Call when Extension WebSocket connects or disconnects */
setExtensionConnected(connected: boolean): void {
this._extensionConnected = connected;
if (connected) {
// Extension is alive — clear any pending exit timer
this._clearTimer();
} else {
// Extension gone — check if CLI has also been idle long enough
this._resetTimer();
}
}
private _clearTimer(): void {
if (this._timer) {
clearTimeout(this._timer);
this._timer = null;
}
}
private _resetTimer(): void {
this._clearTimer();
// Timeout disabled
if (this._timeoutMs <= 0) return;
// Extension connected — don't start timer
if (this._extensionConnected) return;
const elapsed = Date.now() - this._lastCliRequestTime;
if (elapsed >= this._timeoutMs) {
// CLI has been idle past the timeout already
this._onExit();
return;
}
// Start timer for remaining duration
this._timer = setTimeout(() => {
this._onExit();
}, this._timeoutMs - elapsed);
}
}
```
Then create the global `idleManager` instance after the class definition:
```typescript
const idleManager = new IdleManager(IDLE_TIMEOUT, () => {
console.error('[daemon] Idle timeout (no CLI requests + no Extension), shutting down');
process.exit(0);
});
```
- [ ] **Step 4: Wire IdleManager into existing daemon code**
In the `handleRequest` function, replace `resetIdleTimer()` (line 142) with:
```typescript
idleManager.onCliRequest();
```
In the `wss.on('connection')` handler (around line 196-198), add after `extensionWs = ws;`:
```typescript
idleManager.setExtensionConnected(true);
```
In the `ws.on('close')` handler (around line 245-249), add after `extensionWs = null;`:
```typescript
idleManager.setExtensionConnected(false);
```
In the `ws.on('error')` handler (around line 259-261), add after `extensionWs = null;`:
```typescript
idleManager.setExtensionConnected(false);
```
In the `httpServer.listen` callback (line 268-269), replace `resetIdleTimer()` with:
```typescript
idleManager.onCliRequest(); // Start initial idle countdown
```
Remove the old `resetIdleTimer` function and `idleTimer` variable entirely.
- [ ] **Step 5: Run tests to verify they pass**
```bash
npx vitest run src/daemon.test.ts
```
Expected: All 6 tests PASS
- [ ] **Step 6: Commit**
```bash
git add src/daemon.ts src/daemon.test.ts
git commit -m "feat(daemon): replace fixed 5min timeout with dual-condition idle manager (4h default)"
```
---
### Task 3: Add `/status` and `/shutdown` endpoints to daemon
**Files:**
- Modify: `src/daemon.ts:116-123`
- [ ] **Step 1: Add tests for /status and /shutdown endpoints**
Append to `src/daemon.test.ts`:
```typescript
describe('/status endpoint', () => {
it('returns daemon status with correct fields', async () => {
// This is an integration test — tested via the daemon command tests.
// Here we just verify the shape of the status response type.
expect(true).toBe(true); // Placeholder — real coverage in Task 6
});
});
```
Note: The `/status` and `/shutdown` endpoints run inside the daemon process, which makes them hard to unit test in isolation. They are integration-tested via the `opencli daemon status/stop` commands in Task 6.
- [ ] **Step 2: Enhance the existing `/status` endpoint**
In `src/daemon.ts`, replace the existing `/status` handler (lines 116-123) with:
```typescript
if (req.method === 'GET' && pathname === '/status') {
const uptime = process.uptime();
const mem = process.memoryUsage();
jsonResponse(res, 200, {
ok: true,
pid: process.pid,
uptime,
extensionConnected: extensionWs?.readyState === WebSocket.OPEN,
pending: pending.size,
lastCliRequestTime: idleManager.lastCliRequestTime,
memoryMB: Math.round(mem.rss / 1024 / 1024 * 10) / 10,
port: PORT,
});
return;
}
```
Also add a public getter to `IdleManager`:
```typescript
get lastCliRequestTime(): number {
return this._lastCliRequestTime;
}
```
- [ ] **Step 3: Add the `/shutdown` endpoint**
In `src/daemon.ts`, add before the `POST /command` handler:
```typescript
if (req.method === 'POST' && pathname === '/shutdown') {
jsonResponse(res, 200, { ok: true, message: 'Shutting down' });
// Graceful shutdown after response is sent
setTimeout(() => shutdown(), 100);
return;
}
```
- [ ] **Step 4: Run all tests**
```bash
npx vitest run src/daemon.test.ts
```
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add src/daemon.ts src/daemon.test.ts
git commit -m "feat(daemon): enhance /status endpoint, add /shutdown endpoint"
```
---
### Task 4: Reduce Extension WebSocket reconnect backoff cap
**Files:**
- Modify: `extension/src/protocol.ts:57`
- [ ] **Step 1: Change the constant**
In `extension/src/protocol.ts`, change line 57:
```typescript
/** Max reconnect delay (ms) — kept short since daemon is long-lived */
export const WS_RECONNECT_MAX_DELAY = 5000;
```
- [ ] **Step 2: Commit**
```bash
git add extension/src/protocol.ts
git commit -m "feat(extension): reduce WS reconnect backoff cap from 60s to 5s"
```
---
### Task 5: Implement `daemon status/stop/restart` CLI commands
**Files:**
- Create: `src/commands/daemon.ts`
- Modify: `src/cli.ts`
- [ ] **Step 1: Create daemon command module**
Create `src/commands/daemon.ts`:
```typescript
/**
* CLI commands for daemon lifecycle management:
* opencli daemon status — show daemon state
* opencli daemon stop — graceful shutdown
* opencli daemon restart — stop + respawn
*/
import chalk from 'chalk';
import { DEFAULT_DAEMON_PORT } from '../constants.js';
const DAEMON_PORT = DEFAULT_DAEMON_PORT;
const DAEMON_URL = `http://127.0.0.1:${DAEMON_PORT}`;
interface DaemonStatus {
ok: boolean;
pid: number;
uptime: number;
extensionConnected: boolean;
pending: number;
lastCliRequestTime: number;
memoryMB: number;
port: number;
}
async function fetchStatus(): Promise<DaemonStatus | null> {
try {
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), 2000);
const res = await fetch(`${DAEMON_URL}/status`, {
headers: { 'X-OpenCLI': '1' },
signal: controller.signal,
});
clearTimeout(timer);
if (!res.ok) return null;
return await res.json() as DaemonStatus;
} catch {
return null;
}
}
async function requestShutdown(): Promise<boolean> {
try {
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), 5000);
const res = await fetch(`${DAEMON_URL}/shutdown`, {
method: 'POST',
headers: { 'X-OpenCLI': '1' },
signal: controller.signal,
});
clearTimeout(timer);
return res.ok;
} catch {
return false;
}
}
function formatUptime(seconds: number): string {
const h = Math.floor(seconds / 3600);
const m = Math.floor((seconds % 3600) / 60);
if (h > 0) return `${h}h ${m}m`;
if (m > 0) return `${m}m`;
return `${Math.floor(seconds)}s`;
}
function formatTimeSince(timestampMs: number): string {
const seconds = (Date.now() - timestampMs) / 1000;
if (seconds < 60) return `${Math.floor(seconds)}s ago`;
const m = Math.floor(seconds / 60);
if (m < 60) return `${m} min ago`;
const h = Math.floor(m / 60);
return `${h}h ${m % 60}m ago`;
}
export async function daemonStatus(): Promise<void> {
const status = await fetchStatus();
if (!status) {
console.log(`Daemon: ${chalk.dim('not running')}`);
return;
}
console.log(`Daemon: ${chalk.green('running')} (PID ${status.pid})`);
console.log(`Uptime: ${formatUptime(status.uptime)}`);
console.log(`Extension: ${status.extensionConnected ? chalk.green('connected') : chalk.yellow('disconnected')}`);
console.log(`Last CLI request: ${formatTimeSince(status.lastCliRequestTime)}`);
console.log(`Memory: ${status.memoryMB} MB`);
console.log(`Port: ${status.port}`);
}
export async function daemonStop(): Promise<void> {
const status = await fetchStatus();
if (!status) {
console.log(chalk.dim('Daemon is not running.'));
return;
}
const ok = await requestShutdown();
if (ok) {
console.log(chalk.green('Daemon stopped.'));
} else {
console.error(chalk.red('Failed to stop daemon.'));
process.exitCode = 1;
}
}
export async function daemonRestart(): Promise<void> {
const status = await fetchStatus();
if (status) {
const ok = await requestShutdown();
if (!ok) {
console.error(chalk.red('Failed to stop daemon.'));
process.exitCode = 1;
return;
}
// Wait for daemon to exit
await new Promise(r => setTimeout(r, 500));
}
// Import BrowserBridge to spawn a new daemon
const { BrowserBridge } = await import('../browser/mcp.js');
const bridge = new BrowserBridge();
try {
console.log('Starting daemon...');
await bridge.connect({ timeout: 10 });
console.log(chalk.green('Daemon restarted.'));
} catch (err) {
console.error(chalk.red(`Failed to restart daemon: ${err instanceof Error ? err.message : err}`));
process.exitCode = 1;
}
}
```
- [ ] **Step 2: Register daemon commands in cli.ts**
In `src/cli.ts`, add the import at the top:
```typescript
import { daemonStatus, daemonStop, daemonRestart } from './commands/daemon.js';
```
Add the daemon subcommand group before the `// ── External CLIs` section (around line 380):
```typescript
// ── Built-in: daemon ──────────────────────────────────────────────────────
const daemonCmd = program.command('daemon').description('Manage the opencli daemon');
daemonCmd
.command('status')
.description('Show daemon status')
.action(async () => { await daemonStatus(); });
daemonCmd
.command('stop')
.description('Stop the daemon')
.action(async () => { await daemonStop(); });
daemonCmd
.command('restart')
.description('Restart the daemon')
.action(async () => { await daemonRestart(); });
```
- [ ] **Step 3: Run linter/type check**
```bash
npx tsc --noEmit
```
Expected: No errors
- [ ] **Step 4: Commit**
```bash
git add src/commands/daemon.ts src/cli.ts
git commit -m "feat(daemon): add opencli daemon status/stop/restart commands"
```
---
### Task 6: Write tests for daemon commands
**Files:**
- Create: `src/commands/daemon.test.ts`
- [ ] **Step 1: Write tests**
Create `src/commands/daemon.test.ts`:
```typescript
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
// Mock fetch globally for all tests
const mockFetch = vi.fn();
vi.stubGlobal('fetch', mockFetch);
// Mock chalk to avoid ANSI in assertions
vi.mock('chalk', () => ({
default: {
green: (s: string) => s,
yellow: (s: string) => s,
red: (s: string) => s,
dim: (s: string) => s,
},
}));
describe('daemonStatus', () => {
let consoleSpy: ReturnType<typeof vi.spyOn>;
beforeEach(() => {
consoleSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
});
afterEach(() => {
consoleSpy.mockRestore();
mockFetch.mockReset();
});
it('shows "not running" when daemon is unreachable', async () => {
mockFetch.mockRejectedValue(new TypeError('fetch failed'));
const { daemonStatus } = await import('./daemon.js');
await daemonStatus();
expect(consoleSpy).toHaveBeenCalledWith(expect.stringContaining('not running'));
});
it('shows daemon info when running', async () => {
mockFetch.mockResolvedValue({
ok: true,
json: async () => ({
ok: true,
pid: 12345,
uptime: 7200,
extensionConnected: true,
pending: 0,
lastCliRequestTime: Date.now() - 60_000,
memoryMB: 12.3,
port: 19825,
}),
});
const { daemonStatus } = await import('./daemon.js');
await daemonStatus();
expect(consoleSpy).toHaveBeenCalledWith(expect.stringContaining('running'));
expect(consoleSpy).toHaveBeenCalledWith(expect.stringContaining('12345'));
});
});
describe('daemonStop', () => {
let consoleSpy: ReturnType<typeof vi.spyOn>;
let consoleErrSpy: ReturnType<typeof vi.spyOn>;
beforeEach(() => {
consoleSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
consoleErrSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
});
afterEach(() => {
consoleSpy.mockRestore();
consoleErrSpy.mockRestore();
mockFetch.mockReset();
});
it('reports when daemon is not running', async () => {
mockFetch.mockRejectedValue(new TypeError('fetch failed'));
const { daemonStop } = await import('./daemon.js');
await daemonStop();
expect(consoleSpy).toHaveBeenCalledWith(expect.stringContaining('not running'));
});
it('sends shutdown and reports success', async () => {
// First call: fetchStatus
// Second call: requestShutdown
mockFetch
.mockResolvedValueOnce({
ok: true,
json: async () => ({ ok: true, pid: 123, uptime: 100, extensionConnected: false, pending: 0, lastCliRequestTime: Date.now(), memoryMB: 10, port: 19825 }),
})
.mockResolvedValueOnce({ ok: true });
const { daemonStop } = await import('./daemon.js');
await daemonStop();
expect(consoleSpy).toHaveBeenCalledWith(expect.stringContaining('stopped'));
});
});
```
- [ ] **Step 2: Run tests**
```bash
npx vitest run src/commands/daemon.test.ts
```
Expected: PASS
- [ ] **Step 3: Commit**
```bash
git add src/commands/daemon.test.ts
git commit -m "test(daemon): add tests for daemon status/stop commands"
```
---
### Task 7: Improve CLI connection-waiting UX
**Files:**
- Modify: `src/browser/mcp.ts:58-118`
- [ ] **Step 1: Improve error messages and poll interval**
In `src/browser/mcp.ts`, replace the `_ensureDaemon` method (lines 58-118) with:
```typescript
private async _ensureDaemon(timeoutSeconds?: number): Promise<void> {
const effectiveSeconds = (timeoutSeconds && timeoutSeconds > 0) ? timeoutSeconds : Math.ceil(DAEMON_SPAWN_TIMEOUT / 1000);
const timeoutMs = effectiveSeconds * 1000;
// Fast path: extension already connected
if (await isExtensionConnected()) return;
// Daemon running but no extension — wait for extension with progress
if (await isDaemonRunning()) {
if (process.env.OPENCLI_VERBOSE || process.stderr.isTTY) {
process.stderr.write('⏳ Waiting for Chrome extension to connect...\n');
process.stderr.write(' Make sure Chrome is open and the OpenCLI extension is enabled.\n');
}
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
await new Promise(resolve => setTimeout(resolve, 200));
if (await isExtensionConnected()) return;
}
throw new Error(
'Daemon is running but the Browser Extension is not connected.\n' +
'Please install and enable the opencli Browser Bridge extension in Chrome.',
);
}
// No daemon — spawn one
const __dirname = path.dirname(fileURLToPath(import.meta.url));
const parentDir = path.resolve(__dirname, '..');
const daemonTs = path.join(parentDir, 'daemon.ts');
const daemonJs = path.join(parentDir, 'daemon.js');
const isTs = fs.existsSync(daemonTs);
const daemonPath = isTs ? daemonTs : daemonJs;
if (process.env.OPENCLI_VERBOSE || process.stderr.isTTY) {
process.stderr.write('⏳ Starting daemon...\n');
}
const spawnArgs = isTs
? [process.execPath, '--import', 'tsx/esm', daemonPath]
: [process.execPath, daemonPath];
this._daemonProc = spawn(spawnArgs[0], spawnArgs.slice(1), {
detached: true,
stdio: 'ignore',
env: { ...process.env },
});
this._daemonProc.unref();
// Wait for daemon + extension with faster polling
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
await new Promise(resolve => setTimeout(resolve, 200));
if (await isExtensionConnected()) return;
}
if (await isDaemonRunning()) {
throw new Error(
'Daemon is running but the Browser Extension is not connected.\n' +
'Please install and enable the opencli Browser Bridge extension in Chrome.',
);
}
throw new Error(
'Failed to start opencli daemon. Try running manually:\n' +
` node ${daemonPath}\n` +
`Make sure port ${DEFAULT_DAEMON_PORT} is available.`,
);
}
```
- [ ] **Step 2: Run existing browser tests to check for regressions**
```bash
npx vitest run src/browser.test.ts
```
Expected: PASS
- [ ] **Step 3: Commit**
```bash
git add src/browser/mcp.ts
git commit -m "feat(daemon): improve CLI connection-waiting UX with progress messages and 200ms polling"
```
---
### Task 8: Run full test suite and verify
- [ ] **Step 1: Run type check**
```bash
npx tsc --noEmit
```
Expected: No errors
- [ ] **Step 2: Run all tests**
```bash
npx vitest run
```
Expected: All tests pass, no regressions
- [ ] **Step 3: Manual smoke test**
```bash
# Check daemon status (should be "not running" if daemon isn't started)
npx tsx src/main.ts daemon status
# Start daemon by running any browser command, then check status
npx tsx src/main.ts daemon status
# Stop daemon
npx tsx src/main.ts daemon stop
# Verify stopped
npx tsx src/main.ts daemon status
```
- [ ] **Step 4: Final commit if any fixes needed**
```bash
git add -A
git commit -m "fix: address issues found during smoke testing"
```
@@ -0,0 +1,170 @@
# Performance: Smart Wait & INTERCEPT Fix
**Date**: 2026-03-28
**Status**: Approved
## Problem
Three distinct performance/correctness issues:
1. **INTERCEPT strategy semantic bug**: After `installInterceptor()` + `goto()`, adapters call `wait(N)` — which now uses `waitForDomStableJs` and returns early when the DOM settles. But DOM-settle != network capture. The API response may arrive *after* DOM is stable, causing `getInterceptedRequests()` to return an empty array.
2. **Blind `wait(N)` in adapters**: ~30 high-traffic adapters (Twitter family, Medium, Substack, etc.) call `wait(5)` waiting for React/Vue to hydrate. These should wait for a specific DOM element to appear, not a fixed cap.
3. **Daemon cold-start polling**: Fixed 300ms poll loop means ~600ms before first successful `isExtensionConnected()` check, even though the daemon is typically ready in 500800ms.
## Design
### Layer 1 — `waitForCapture()` (correctness fix + perf)
Add `waitForCapture(timeout?: number): Promise<void>` to `IPage`.
Polls `window.__opencli_xhr.length > 0` every 100ms inside the browser tab. Resolves as soon as ≥1 capture arrives; rejects after `timeout` seconds.
```typescript
// dom-helpers.ts
export function waitForCaptureJs(maxMs: number): string {
return `
new Promise((resolve, reject) => {
const deadline = Date.now() + ${maxMs};
const check = () => {
if ((window.__opencli_xhr || []).length > 0) return resolve('captured');
if (Date.now() > deadline) return reject(new Error('No capture within ${maxMs / 1000}s'));
setTimeout(check, 100);
};
check();
})
`;
}
```
`page.ts` and `cdp.ts` implement `waitForCapture()` by calling `waitForCaptureJs`.
**All INTERCEPT adapters** replace `wait(N)``waitForCapture(N+2)` (slightly longer timeout as safety margin).
`stepIntercept` in `pipeline/steps/intercept.ts` replaces its internal `wait(timeout)` with `waitForCapture(timeout)`.
**Expected gain**: 36kr hot/search: 6s → ~12s. Twitter search/followers: 58s → ~13s.
### Layer 2 — `wait({ selector })` (semantic precision)
Extend `WaitOptions` with `selector?: string`.
Add `waitForSelectorJs(selector, timeoutMs)` to `dom-helpers.ts` — polls `document.querySelector(selector)` every 100ms, resolves on first match, rejects on timeout.
```typescript
// types.ts
export interface WaitOptions {
text?: string;
selector?: string; // NEW
time?: number;
timeout?: number;
}
```
```typescript
// dom-helpers.ts
export function waitForSelectorJs(selector: string, timeoutMs: number): string {
return `
new Promise((resolve, reject) => {
const deadline = Date.now() + ${timeoutMs};
const check = () => {
if (document.querySelector(${JSON.stringify(selector)})) return resolve('found');
if (Date.now() > deadline) return reject(new Error('Selector not found: ' + ${JSON.stringify(selector)}));
setTimeout(check, 100);
};
check();
})
`;
}
```
`page.ts` and `cdp.ts` handle `selector` branch in `wait()`.
**High-impact adapter changes**:
| Adapter | Old | New |
|---------|-----|-----|
| `twitter/*` (15 adapters) | `wait(5)` | `wait({ selector: '[data-testid="primaryColumn"]', timeout: 6 })` |
| `twitter/reply.ts` | `wait(5)` | `wait({ selector: '[data-testid="tweetTextarea_0"]', timeout: 8 })` |
| `medium/utils.ts` | `wait(5)` + inline 3s setTimeout | `wait({ selector: 'article', timeout: 8 })` + remove inline sleep |
| `substack/utils.ts` | `wait(5)` × 2 | `wait({ selector: 'article', timeout: 8 })` |
| `bloomberg/news.ts` | `wait(5)` | `wait({ selector: 'article', timeout: 6 })` |
| `sinablog/utils.ts` | `wait(5)` | `wait({ selector: 'article, .article', timeout: 6 })` |
| `producthunt` (already covered by layer 1) | — | — |
**Expected gain**: Twitter commands: 5s → ~0.52s. Medium: 8s → ~13s.
### Layer 3 — Daemon exponential backoff (cold-start)
Replace fixed 300ms poll in `_ensureDaemon()` (`browser/mcp.ts`) with exponential backoff:
```typescript
// before
while (Date.now() < deadline) {
await new Promise(resolve => setTimeout(resolve, 300));
if (await isExtensionConnected()) return;
}
// after
const backoffs = [50, 100, 200, 400, 800, 1500, 3000];
let i = 0;
while (Date.now() < deadline) {
await new Promise(resolve => setTimeout(resolve, backoffs[Math.min(i++, backoffs.length - 1)]));
if (await isExtensionConnected()) return;
}
```
**Expected gain**: First cold-start check succeeds at ~150ms instead of ~600ms.
## Files Changed
### New / Modified (framework)
- `src/types.ts``WaitOptions.selector`, `IPage.waitForCapture()`
- `src/browser/dom-helpers.ts``waitForCaptureJs()`, `waitForSelectorJs()`
- `src/browser/page.ts``waitForCapture()`, `wait()` selector branch
- `src/browser/cdp.ts``waitForCapture()`, `wait()` selector branch
- `src/browser/mcp.ts` — exponential backoff in `_ensureDaemon()`
- `src/pipeline/steps/intercept.ts` — use `waitForCapture()`
### Modified (adapters — Layer 1, INTERCEPT)
- `clis/36kr/hot.ts`
- `clis/36kr/search.ts`
- `clis/twitter/search.ts`
- `clis/twitter/followers.ts`
- `clis/twitter/following.ts`
- `clis/producthunt/hot.ts`
- `clis/producthunt/browse.ts`
### Modified (adapters — Layer 2, selector)
- `clis/twitter/reply.ts`
- `clis/twitter/follow.ts`
- `clis/twitter/unfollow.ts`
- `clis/twitter/like.ts`
- `clis/twitter/bookmark.ts`
- `clis/twitter/unbookmark.ts`
- `clis/twitter/block.ts`
- `clis/twitter/unblock.ts`
- `clis/twitter/hide-reply.ts`
- `clis/twitter/notifications.ts`
- `clis/twitter/profile.ts`
- `clis/twitter/thread.ts`
- `clis/twitter/timeline.ts`
- `clis/twitter/delete.ts`
- `clis/twitter/reply-dm.ts`
- `clis/medium/utils.ts`
- `clis/substack/utils.ts`
- `clis/bloomberg/news.ts`
- `clis/sinablog/utils.ts`
## Delivery Order
1. Layer 1 (`waitForCapture`) — correctness fix, highest ROI
2. Layer 3 (backoff) — 3-line change, zero risk
3. Layer 2 (`wait({ selector })`) — largest adapter surface, can be done per-site
## Testing
- Unit tests: `waitForCaptureJs`, `waitForSelectorJs` exported and tested in `dom-helpers.test.ts` (if exists) or new test file
- Adapter tests: existing tests must continue to pass (mock `page.wait` / `page.waitForCapture`)
- Run: `npx vitest run --project unit --project adapter`
@@ -0,0 +1,207 @@
# Daemon Lifecycle Redesign
## Problem
OpenCLI's daemon auto-exits after 5 minutes of idle time. During typical development
cycles (write code → test → modify → test again), coding intervals frequently exceed
5 minutes. Each restart incurs 2-4 seconds of overhead (process spawn + Extension
WebSocket reconnection), creating a noticeable and frustrating delay.
The current design treats the daemon as a disposable process, but the actual cost
profile doesn't justify this:
| Cost of staying alive | Cost of restarting |
|-----------------------|--------------------|
| ~12 MB memory, 0% CPU | 2-4 seconds delay per restart |
The restart cost far outweighs the idle cost.
## Solution
Replace the aggressive 5-minute fixed timeout with a long-lived daemon model. The
daemon stays running for hours, exits only when truly abandoned, and reconnects to
the Chrome Extension faster when needed.
Four changes:
1. Extend idle timeout from 5 minutes to 4 hours (configurable)
2. Require dual idle condition: both no CLI requests AND no Extension connection
3. Reduce Extension WebSocket reconnect backoff cap from 60s to 5s
4. Add `opencli daemon status/stop/restart` commands
## Design
### Timeout Strategy
**Current behavior:** A single idle timer resets on each HTTP request. After 5
minutes without a request, the daemon calls `process.exit(0)`.
**New behavior:** The daemon tracks two activity signals independently:
- **CLI activity:** timestamp of the last HTTP request from any CLI invocation
- **Extension activity:** whether a WebSocket connection from the Chrome Extension
is currently open
The exit countdown starts only when BOTH conditions are met simultaneously:
- No CLI request for `IDLE_TIMEOUT` duration
- No Extension WebSocket connection
If either signal is active, the daemon stays alive. This means:
- A connected Extension keeps the daemon alive indefinitely (user has Chrome open,
likely still working)
- Recent CLI activity keeps the daemon alive even if Extension temporarily
disconnects (Chrome restarting, Extension updating)
**Timeout value:** 4 hours by default.
```typescript
const DEFAULT_IDLE_TIMEOUT = 4 * 60 * 60 * 1000; // 4 hours
const IDLE_TIMEOUT = DEFAULT_IDLE_TIMEOUT;
```
**Timer implementation:**
```
resetIdleTimer():
clear existing timer
if Extension is connected:
do not start timer (Extension connection keeps daemon alive)
return
start timer with IDLE_TIMEOUT duration
on timeout: process.exit(0)
On CLI HTTP request:
update lastRequestTime
resetIdleTimer()
On Extension WebSocket connect:
clear timer (Extension keeps daemon alive)
On Extension WebSocket disconnect:
elapsed = now - lastRequestTime
if elapsed >= IDLE_TIMEOUT:
process.exit(0) // CLI has been idle long enough already
else:
start timer with (IDLE_TIMEOUT - elapsed) // count remaining time
```
### Extension Fast Reconnect
**Current behavior:** When the Extension loses its WebSocket connection to the
daemon, it reconnects with exponential backoff: 2s → 4s → 8s → 16s → 32s → 60s
(capped). In the worst case, the Extension waits up to 60 seconds before attempting
reconnection.
**New behavior:** Cap the backoff at 5 seconds instead of 60 seconds.
```typescript
// extension/src/background.ts
const WS_RECONNECT_MAX_DELAY = 5000; // was 60000
```
Rationale: with a 4-hour daemon timeout, the daemon is almost always running. Long
backoff intervals are unnecessary and only increase reconnection latency. A 5-second
cap means the Extension reconnects within 5 seconds of the daemon becoming available.
### Daemon Management Commands
Add three new CLI commands for daemon lifecycle management:
**`opencli daemon status`**
Queries the daemon's `/status` endpoint (new) and displays:
```
Daemon: running (PID 12345)
Uptime: 2h 15m
Extension: connected
Last CLI request: 8 min ago
Memory: 12.3 MB
Port: 19825
```
If daemon is not running:
```
Daemon: not running
```
**`opencli daemon stop`**
Sends a `POST /shutdown` request to the daemon, which triggers a graceful shutdown:
reject pending requests with a shutdown message, close WebSocket connections, close
HTTP server, then exit.
**`opencli daemon restart`**
Equivalent to `stop` followed by spawning a new daemon. Useful when the daemon gets
into a bad state.
**Daemon-side endpoints:**
- `GET /status` — returns JSON with PID, uptime, extension connection state, last
request time, memory usage
- `POST /shutdown` — initiates graceful shutdown
Both endpoints require the same `X-OpenCLI` header as existing endpoints for CSRF
protection.
### CLI Connection Experience
**Current behavior:** When daemon is running but Extension is not connected, the CLI
silently polls every 300ms and eventually times out with a generic error.
**New behavior:** Show a progress indicator and actionable message:
```
⏳ Waiting for Chrome extension to connect...
Make sure Chrome is open and the OpenCLI extension is enabled.
```
Poll interval reduced from 300ms to 200ms for slightly faster detection.
If the daemon is not running at all (connection refused), the CLI spawns it as before
and shows:
```
⏳ Starting daemon...
```
## Files Changed
| File | Change | Estimated LOC |
|------|--------|---------------|
| `src/daemon.ts` | Dual-condition idle timeout, `/status` endpoint, `/shutdown` endpoint | ~40 |
| `extension/src/background.ts` | `WS_RECONNECT_MAX_DELAY` 60000 → 5000 | 1 |
| `src/browser/daemon-client.ts` | Better connection-waiting UX, 200ms poll interval | ~20 |
| `src/commands/daemon.ts` (new) | `status`, `stop`, `restart` subcommands | ~80 |
| `src/constants.ts` | `DEFAULT_IDLE_TIMEOUT` constant | 2 |
**Total: ~143 lines of new/changed code.**
## Backward Compatibility
- No breaking changes to CLI commands or Extension protocol
- The daemon and extension use the fixed Browser Bridge port.
- The only observable behavior change: daemon stays alive longer
- New `daemon` subcommands are additive
## Testing
- Unit test: idle timer starts only when both CLI and Extension are idle
- Unit test: idle timer is cleared when Extension connects
- Unit test: `/status` returns correct state
- Unit test: `/shutdown` triggers graceful exit
- Integration test: daemon survives 10+ minutes without CLI requests while Extension
is connected
- Integration test: daemon exits after configured timeout when fully idle
- Integration test: `opencli daemon status/stop/restart` work correctly
## Out of Scope
- OS-level daemon management (launchd/systemd) — can be added later if needed
- Daemon auto-update mechanism
- Multi-daemon coordination
- Persistent daemon state across restarts
@@ -0,0 +1,144 @@
# Browse Skill Testing Design
Two-layer testing framework for `opencli browse` commands and the
Claude Code skill integration.
## Goal
Verify that `opencli browse` works reliably on real websites and that
Claude Code can use the skill to complete browser tasks end-to-end.
## Architecture
```
autoresearch/
├── browse-tasks.json ← 59 task definitions with browse command sequences
├── eval-browse.ts ← Layer 1: deterministic browse command testing
├── eval-skill.ts ← Layer 2: Claude Code skill E2E testing
├── run-browse.sh ← Launch Layer 1
├── run-skill.sh ← Launch Layer 2
├── baseline-browse.txt ← Layer 1 best score
├── baseline-skill.txt ← Layer 2 best score
└── results/ ← Per-run results (gitignored)
```
## Layer 1: Deterministic Browse Command Testing
Tests `opencli browse` commands directly on real websites. No LLM
involved — pure command reliability testing.
### How It Works
Each task defines a sequence of browse commands and a judge for the
last command's output:
```json
{
"name": "hn-top-stories",
"steps": [
"opencli browse open https://news.ycombinator.com",
"opencli browse eval \"JSON.stringify([...document.querySelectorAll('.titleline a')].slice(0,5).map(a=>({title:a.textContent,url:a.href})))\""
],
"judge": { "type": "arrayMinLength", "minLength": 5 }
}
```
### Execution
```bash
./autoresearch/run-browse.sh
```
- Runs all 59 tasks serially
- Each task: execute steps → judge last step output → pass/fail
- `opencli browse close` between tasks for clean state
- Expected: ~2 minutes, $0 cost
### Task Categories
| Category | Count | Example |
|----------|-------|---------|
| extract | 9 | Open page, eval JS to extract data |
| list | 10 | Open page, eval JS to extract array |
| search | 6 | Open, type query, keys Enter, eval results |
| nav | 7 | Open, click link, eval new page title |
| scroll | 5 | Open, scroll, eval footer/hidden content |
| form | 6 | Open, type into fields, eval field values |
| complex | 6 | Multi-step: open → click → navigate → extract |
| bench | 10 | Test set (various) |
## Layer 2: Claude Code Skill E2E Testing
Spawns Claude Code with the opencli-browser skill to complete tasks
autonomously using browse commands.
### How It Works
```bash
claude -p \
--system-prompt "$(cat skills/opencli-browser/SKILL.md)" \
--dangerously-skip-permissions \
--allowedTools "Bash(opencli:*)" \
--output-format json \
"用 opencli browse 完成任务:Extract the top 5 stories from Hacker News with title and score. Start URL: https://news.ycombinator.com"
```
### Execution
```bash
./autoresearch/run-skill.sh
```
- Runs all 59 tasks serially
- Each task: spawn Claude Code → it uses browse commands autonomously → judge output
- Expected: ~20 minutes, ~$5-10
### Judge
Both layers use the same judge types:
| Type | Description |
|------|-------------|
| `contains` | Output contains a substring |
| `arrayMinLength` | Output is an array with ≥ N items |
| `arrayFieldsPresent` | Array items have required fields |
| `nonEmpty` | Output is non-empty |
| `matchesPattern` | Output matches a regex |
## Output Format
```
🔬 Layer 1: Browse Commands — 59 tasks
[1/59] extract-title-example... ✓ (0.5s)
[2/59] hn-top-stories... ✓ (1.2s)
...
Score: 55/59 (93%)
Time: 2min
Cost: $0
🔬 Layer 2: Skill E2E — 59 tasks
[1/59] extract-title-example... ✓ (8s, $0.01)
[2/59] hn-top-stories... ✓ (15s, $0.08)
...
Score: 52/59 (88%)
Time: 20min
Cost: $6.50
```
## Constraints
- All 59 tasks run on real websites (no mocks)
- Layer 1: zero LLM cost, ~2 min
- Layer 2: ~$5-10 LLM cost, ~20 min
- Results saved to `autoresearch/results/` (gitignored)
- Baselines tracked in `baseline-browse.txt` and `baseline-skill.txt`
## Success Criteria
- Layer 1 ≥ 90% (browse commands work on real sites)
- Layer 2 ≥ 85% (Claude Code can use skill effectively)
- Both layers cover all 8 task categories
@@ -0,0 +1,41 @@
# V2EX AutoResearch Test Suite Design
## Goal
Build a comprehensive test suite using V2EX (https://v2ex.com/) as the single target website to iteratively improve OpenCLI Browser's reliability and Claude Code skill effectiveness. Run 10 rounds of AutoResearch iteration (5 code-level + 5 SKILL.md-level).
## Test Suite Structure — 5 Layers, 40 Tasks
### Layer 1: Atomic (10 tasks)
Single browser commands testing command-level reliability.
### Layer 2: Single Page (10 tasks)
Meaningful extraction/interaction within one page.
### Layer 3: Multi-Step (10 tasks)
Cross-page navigation + extraction combos.
### Layer 4: Write Operations (5 tasks)
Login-required write operations (reply, favorite, thank).
### Layer 5: Complex Chain (5 tasks)
Long chains: cross-post reference, multi-node comparison, full workflows.
## Test Infrastructure
- **v2ex-tasks.json** — Layer 1 deterministic tasks (browse commands + judge criteria)
- **eval-v2ex.ts** — Runner for V2EX tasks (reuses eval-browse.ts pattern)
- **v2ex-skill-tasks** — Layer 2 LLM E2E tasks embedded in eval runner
- **presets/v2ex-reliability.ts** — AutoResearch preset for code optimization
- **presets/v2ex-skill.ts** — AutoResearch preset for SKILL.md optimization
## AutoResearch Iteration Plan
- Rounds 1-5: Layer 1 preset → optimize src/browser/*.ts code
- Rounds 6-10: Layer 2 preset → optimize skills/opencli-browser/SKILL.md
- Alternating: fix code issues first, then improve LLM guidance
## Success Criteria
- Layer 1 baseline → target 100% pass rate after 5 code iterations
- Layer 2 baseline → target 100% pass rate after 5 SKILL.md iterations