chore: import upstream snapshot with attribution
E2E Headed Chrome / e2e-headed (macos-15) (push) Has been cancelled
E2E Headed Chrome / e2e-headed (ubuntu-latest) (push) Has been cancelled
E2E Headed Chrome / e2e-headed (windows-latest) (push) Has been cancelled
CI / build (macos-latest) (push) Has been cancelled
CI / build (ubuntu-latest) (push) Has been cancelled
CI / build (windows-latest) (push) Has been cancelled
CI / unit-test (push) Has been cancelled
CI / bun-test (push) Has been cancelled
CI / adapter-test (push) Has been cancelled
CI / smoke-test (macos-latest) (push) Has been cancelled
CI / smoke-test (ubuntu-latest) (push) Has been cancelled
Security Audit / audit (push) Has been cancelled
Build Chrome Extension / build (push) Has been cancelled
Trigger Website Rebuild (Docs Updated) / dispatch (push) Has been cancelled
E2E Headed Chrome / e2e-headed (macos-15) (push) Has been cancelled
E2E Headed Chrome / e2e-headed (ubuntu-latest) (push) Has been cancelled
E2E Headed Chrome / e2e-headed (windows-latest) (push) Has been cancelled
CI / build (macos-latest) (push) Has been cancelled
CI / build (ubuntu-latest) (push) Has been cancelled
CI / build (windows-latest) (push) Has been cancelled
CI / unit-test (push) Has been cancelled
CI / bun-test (push) Has been cancelled
CI / adapter-test (push) Has been cancelled
CI / smoke-test (macos-latest) (push) Has been cancelled
CI / smoke-test (ubuntu-latest) (push) Has been cancelled
Security Audit / audit (push) Has been cancelled
Build Chrome Extension / build (push) Has been cancelled
Trigger Website Rebuild (Docs Updated) / dispatch (push) Has been cancelled
This commit is contained in:
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,857 @@
|
||||
# Daemon Lifecycle Redesign Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Replace the daemon's aggressive 5-minute idle timeout with a long-lived model (4h default) that requires both CLI inactivity AND Extension disconnection before exiting, plus add `daemon status/stop/restart` CLI commands.
|
||||
|
||||
**Architecture:** The daemon keeps its existing HTTP + WebSocket bridge architecture. We change the idle timeout logic to track two independent activity signals (CLI requests and Extension connection), add `/status` and `/shutdown` HTTP endpoints, reduce the Extension reconnect backoff cap, and register new CLI commands via Commander.js.
|
||||
|
||||
**Tech Stack:** Node.js, TypeScript, Commander.js, ws, Vitest
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
| File | Action | Responsibility |
|
||||
|------|--------|----------------|
|
||||
| `src/constants.ts` | Modify | Add `DEFAULT_DAEMON_IDLE_TIMEOUT` constant |
|
||||
| `src/daemon.ts` | Modify | Dual-condition idle timer, `/status` endpoint, `/shutdown` endpoint |
|
||||
| `src/daemon.test.ts` | Create | Unit tests for idle timer logic, `/status`, `/shutdown` |
|
||||
| `extension/src/protocol.ts` | Modify | Change `WS_RECONNECT_MAX_DELAY` from 60000 to 5000 |
|
||||
| `src/cli.ts` | Modify | Register `daemon` subcommand group |
|
||||
| `src/commands/daemon.ts` | Create | `status`, `stop`, `restart` subcommand implementations |
|
||||
| `src/commands/daemon.test.ts` | Create | Unit tests for daemon commands |
|
||||
| `src/browser/mcp.ts` | Modify | Better connection-waiting UX messages, 200ms poll interval |
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add `DEFAULT_DAEMON_IDLE_TIMEOUT` constant
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/constants.ts`
|
||||
|
||||
- [ ] **Step 1: Add the constant**
|
||||
|
||||
In `src/constants.ts`, add after the `DEFAULT_DAEMON_PORT` line:
|
||||
|
||||
```typescript
|
||||
/** Default idle timeout before daemon auto-exits (ms). */
|
||||
export const DEFAULT_DAEMON_IDLE_TIMEOUT = 4 * 60 * 60 * 1000; // 4 hours
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add src/constants.ts
|
||||
git commit -m "feat(daemon): add DEFAULT_DAEMON_IDLE_TIMEOUT constant (4 hours)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Implement dual-condition idle timer in daemon
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/daemon.ts:27,29-57,116-123,196-198,245-262,265-269`
|
||||
- Test: `src/daemon.test.ts` (create)
|
||||
|
||||
- [ ] **Step 1: Write failing tests for the new idle timer logic**
|
||||
|
||||
Create `src/daemon.test.ts`:
|
||||
|
||||
```typescript
|
||||
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
|
||||
|
||||
// We test the idle timer logic by extracting it into testable functions.
|
||||
// The daemon module has side effects (starts server), so we test the logic unit directly.
|
||||
|
||||
describe('IdleManager', () => {
|
||||
beforeEach(() => {
|
||||
vi.useFakeTimers();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
it('does not start timer when extension is connected', async () => {
|
||||
const { IdleManager } = await import('./daemon.js');
|
||||
const exit = vi.fn();
|
||||
const mgr = new IdleManager(300_000, exit); // 5 min for fast test
|
||||
|
||||
mgr.setExtensionConnected(true);
|
||||
mgr.onCliRequest();
|
||||
|
||||
vi.advanceTimersByTime(300_000 + 1000);
|
||||
expect(exit).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('starts timer when extension disconnects and CLI is idle', async () => {
|
||||
const { IdleManager } = await import('./daemon.js');
|
||||
const exit = vi.fn();
|
||||
const mgr = new IdleManager(300_000, exit);
|
||||
|
||||
mgr.onCliRequest(); // CLI was active
|
||||
mgr.setExtensionConnected(true);
|
||||
mgr.setExtensionConnected(false); // Extension disconnects
|
||||
|
||||
// Should not exit immediately — CLI was just active
|
||||
expect(exit).not.toHaveBeenCalled();
|
||||
|
||||
// Advance past timeout
|
||||
vi.advanceTimersByTime(300_000 + 1000);
|
||||
expect(exit).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('exits immediately on extension disconnect if CLI has been idle past timeout', async () => {
|
||||
const { IdleManager } = await import('./daemon.js');
|
||||
const exit = vi.fn();
|
||||
const mgr = new IdleManager(300_000, exit);
|
||||
|
||||
mgr.onCliRequest(); // Last CLI activity
|
||||
vi.advanceTimersByTime(400_000); // 400s elapsed — past 300s timeout
|
||||
|
||||
mgr.setExtensionConnected(true);
|
||||
mgr.setExtensionConnected(false);
|
||||
|
||||
expect(exit).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('resets timer on new CLI request', async () => {
|
||||
const { IdleManager } = await import('./daemon.js');
|
||||
const exit = vi.fn();
|
||||
const mgr = new IdleManager(300_000, exit);
|
||||
|
||||
mgr.onCliRequest();
|
||||
vi.advanceTimersByTime(200_000); // 200s elapsed
|
||||
mgr.onCliRequest(); // Reset
|
||||
|
||||
vi.advanceTimersByTime(200_000); // 200s more — only 200s since last request
|
||||
expect(exit).not.toHaveBeenCalled();
|
||||
|
||||
vi.advanceTimersByTime(100_001); // Now 300s+ since last request
|
||||
expect(exit).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('does not exit when timeout is 0 (disabled)', async () => {
|
||||
const { IdleManager } = await import('./daemon.js');
|
||||
const exit = vi.fn();
|
||||
const mgr = new IdleManager(0, exit);
|
||||
|
||||
mgr.onCliRequest();
|
||||
vi.advanceTimersByTime(24 * 60 * 60 * 1000); // 24 hours
|
||||
expect(exit).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('clears timer when extension connects', async () => {
|
||||
const { IdleManager } = await import('./daemon.js');
|
||||
const exit = vi.fn();
|
||||
const mgr = new IdleManager(300_000, exit);
|
||||
|
||||
mgr.onCliRequest();
|
||||
vi.advanceTimersByTime(200_000); // Timer running
|
||||
|
||||
mgr.setExtensionConnected(true); // Should clear timer
|
||||
vi.advanceTimersByTime(200_000); // Would have fired
|
||||
expect(exit).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
||||
```bash
|
||||
npx vitest run src/daemon.test.ts
|
||||
```
|
||||
|
||||
Expected: FAIL — `IdleManager` is not exported from `./daemon.js`
|
||||
|
||||
- [ ] **Step 3: Extract IdleManager class and refactor daemon.ts**
|
||||
|
||||
In `src/daemon.ts`, replace the idle timeout section (lines 27, 29-57) with:
|
||||
|
||||
Replace the `IDLE_TIMEOUT` constant (line 27):
|
||||
```typescript
|
||||
import { DEFAULT_DAEMON_PORT, DEFAULT_DAEMON_IDLE_TIMEOUT } from './constants.js';
|
||||
|
||||
const PORT = DEFAULT_DAEMON_PORT;
|
||||
const IDLE_TIMEOUT = DEFAULT_DAEMON_IDLE_TIMEOUT;
|
||||
```
|
||||
|
||||
Replace the idle timer state and `resetIdleTimer` function (lines 37, 49-57) with the `IdleManager` class:
|
||||
|
||||
```typescript
|
||||
/**
|
||||
* Manages daemon idle timeout with dual-condition logic:
|
||||
* exits only when BOTH CLI is idle AND Extension is disconnected.
|
||||
*/
|
||||
export class IdleManager {
|
||||
private _timer: ReturnType<typeof setTimeout> | null = null;
|
||||
private _lastCliRequestTime = Date.now();
|
||||
private _extensionConnected = false;
|
||||
private _timeoutMs: number;
|
||||
private _onExit: () => void;
|
||||
|
||||
constructor(timeoutMs: number, onExit: () => void) {
|
||||
this._timeoutMs = timeoutMs;
|
||||
this._onExit = onExit;
|
||||
}
|
||||
|
||||
/** Call when an HTTP request arrives from CLI */
|
||||
onCliRequest(): void {
|
||||
this._lastCliRequestTime = Date.now();
|
||||
this._resetTimer();
|
||||
}
|
||||
|
||||
/** Call when Extension WebSocket connects or disconnects */
|
||||
setExtensionConnected(connected: boolean): void {
|
||||
this._extensionConnected = connected;
|
||||
if (connected) {
|
||||
// Extension is alive — clear any pending exit timer
|
||||
this._clearTimer();
|
||||
} else {
|
||||
// Extension gone — check if CLI has also been idle long enough
|
||||
this._resetTimer();
|
||||
}
|
||||
}
|
||||
|
||||
private _clearTimer(): void {
|
||||
if (this._timer) {
|
||||
clearTimeout(this._timer);
|
||||
this._timer = null;
|
||||
}
|
||||
}
|
||||
|
||||
private _resetTimer(): void {
|
||||
this._clearTimer();
|
||||
|
||||
// Timeout disabled
|
||||
if (this._timeoutMs <= 0) return;
|
||||
|
||||
// Extension connected — don't start timer
|
||||
if (this._extensionConnected) return;
|
||||
|
||||
const elapsed = Date.now() - this._lastCliRequestTime;
|
||||
if (elapsed >= this._timeoutMs) {
|
||||
// CLI has been idle past the timeout already
|
||||
this._onExit();
|
||||
return;
|
||||
}
|
||||
|
||||
// Start timer for remaining duration
|
||||
this._timer = setTimeout(() => {
|
||||
this._onExit();
|
||||
}, this._timeoutMs - elapsed);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Then create the global `idleManager` instance after the class definition:
|
||||
|
||||
```typescript
|
||||
const idleManager = new IdleManager(IDLE_TIMEOUT, () => {
|
||||
console.error('[daemon] Idle timeout (no CLI requests + no Extension), shutting down');
|
||||
process.exit(0);
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Wire IdleManager into existing daemon code**
|
||||
|
||||
In the `handleRequest` function, replace `resetIdleTimer()` (line 142) with:
|
||||
```typescript
|
||||
idleManager.onCliRequest();
|
||||
```
|
||||
|
||||
In the `wss.on('connection')` handler (around line 196-198), add after `extensionWs = ws;`:
|
||||
```typescript
|
||||
idleManager.setExtensionConnected(true);
|
||||
```
|
||||
|
||||
In the `ws.on('close')` handler (around line 245-249), add after `extensionWs = null;`:
|
||||
```typescript
|
||||
idleManager.setExtensionConnected(false);
|
||||
```
|
||||
|
||||
In the `ws.on('error')` handler (around line 259-261), add after `extensionWs = null;`:
|
||||
```typescript
|
||||
idleManager.setExtensionConnected(false);
|
||||
```
|
||||
|
||||
In the `httpServer.listen` callback (line 268-269), replace `resetIdleTimer()` with:
|
||||
```typescript
|
||||
idleManager.onCliRequest(); // Start initial idle countdown
|
||||
```
|
||||
|
||||
Remove the old `resetIdleTimer` function and `idleTimer` variable entirely.
|
||||
|
||||
- [ ] **Step 5: Run tests to verify they pass**
|
||||
|
||||
```bash
|
||||
npx vitest run src/daemon.test.ts
|
||||
```
|
||||
|
||||
Expected: All 6 tests PASS
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add src/daemon.ts src/daemon.test.ts
|
||||
git commit -m "feat(daemon): replace fixed 5min timeout with dual-condition idle manager (4h default)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Add `/status` and `/shutdown` endpoints to daemon
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/daemon.ts:116-123`
|
||||
|
||||
- [ ] **Step 1: Add tests for /status and /shutdown endpoints**
|
||||
|
||||
Append to `src/daemon.test.ts`:
|
||||
|
||||
```typescript
|
||||
describe('/status endpoint', () => {
|
||||
it('returns daemon status with correct fields', async () => {
|
||||
// This is an integration test — tested via the daemon command tests.
|
||||
// Here we just verify the shape of the status response type.
|
||||
expect(true).toBe(true); // Placeholder — real coverage in Task 6
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
Note: The `/status` and `/shutdown` endpoints run inside the daemon process, which makes them hard to unit test in isolation. They are integration-tested via the `opencli daemon status/stop` commands in Task 6.
|
||||
|
||||
- [ ] **Step 2: Enhance the existing `/status` endpoint**
|
||||
|
||||
In `src/daemon.ts`, replace the existing `/status` handler (lines 116-123) with:
|
||||
|
||||
```typescript
|
||||
if (req.method === 'GET' && pathname === '/status') {
|
||||
const uptime = process.uptime();
|
||||
const mem = process.memoryUsage();
|
||||
jsonResponse(res, 200, {
|
||||
ok: true,
|
||||
pid: process.pid,
|
||||
uptime,
|
||||
extensionConnected: extensionWs?.readyState === WebSocket.OPEN,
|
||||
pending: pending.size,
|
||||
lastCliRequestTime: idleManager.lastCliRequestTime,
|
||||
memoryMB: Math.round(mem.rss / 1024 / 1024 * 10) / 10,
|
||||
port: PORT,
|
||||
});
|
||||
return;
|
||||
}
|
||||
```
|
||||
|
||||
Also add a public getter to `IdleManager`:
|
||||
|
||||
```typescript
|
||||
get lastCliRequestTime(): number {
|
||||
return this._lastCliRequestTime;
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add the `/shutdown` endpoint**
|
||||
|
||||
In `src/daemon.ts`, add before the `POST /command` handler:
|
||||
|
||||
```typescript
|
||||
if (req.method === 'POST' && pathname === '/shutdown') {
|
||||
jsonResponse(res, 200, { ok: true, message: 'Shutting down' });
|
||||
// Graceful shutdown after response is sent
|
||||
setTimeout(() => shutdown(), 100);
|
||||
return;
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run all tests**
|
||||
|
||||
```bash
|
||||
npx vitest run src/daemon.test.ts
|
||||
```
|
||||
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/daemon.ts src/daemon.test.ts
|
||||
git commit -m "feat(daemon): enhance /status endpoint, add /shutdown endpoint"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Reduce Extension WebSocket reconnect backoff cap
|
||||
|
||||
**Files:**
|
||||
- Modify: `extension/src/protocol.ts:57`
|
||||
|
||||
- [ ] **Step 1: Change the constant**
|
||||
|
||||
In `extension/src/protocol.ts`, change line 57:
|
||||
|
||||
```typescript
|
||||
/** Max reconnect delay (ms) — kept short since daemon is long-lived */
|
||||
export const WS_RECONNECT_MAX_DELAY = 5000;
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add extension/src/protocol.ts
|
||||
git commit -m "feat(extension): reduce WS reconnect backoff cap from 60s to 5s"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Implement `daemon status/stop/restart` CLI commands
|
||||
|
||||
**Files:**
|
||||
- Create: `src/commands/daemon.ts`
|
||||
- Modify: `src/cli.ts`
|
||||
|
||||
- [ ] **Step 1: Create daemon command module**
|
||||
|
||||
Create `src/commands/daemon.ts`:
|
||||
|
||||
```typescript
|
||||
/**
|
||||
* CLI commands for daemon lifecycle management:
|
||||
* opencli daemon status — show daemon state
|
||||
* opencli daemon stop — graceful shutdown
|
||||
* opencli daemon restart — stop + respawn
|
||||
*/
|
||||
|
||||
import chalk from 'chalk';
|
||||
import { DEFAULT_DAEMON_PORT } from '../constants.js';
|
||||
|
||||
const DAEMON_PORT = DEFAULT_DAEMON_PORT;
|
||||
const DAEMON_URL = `http://127.0.0.1:${DAEMON_PORT}`;
|
||||
|
||||
interface DaemonStatus {
|
||||
ok: boolean;
|
||||
pid: number;
|
||||
uptime: number;
|
||||
extensionConnected: boolean;
|
||||
pending: number;
|
||||
lastCliRequestTime: number;
|
||||
memoryMB: number;
|
||||
port: number;
|
||||
}
|
||||
|
||||
async function fetchStatus(): Promise<DaemonStatus | null> {
|
||||
try {
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), 2000);
|
||||
const res = await fetch(`${DAEMON_URL}/status`, {
|
||||
headers: { 'X-OpenCLI': '1' },
|
||||
signal: controller.signal,
|
||||
});
|
||||
clearTimeout(timer);
|
||||
if (!res.ok) return null;
|
||||
return await res.json() as DaemonStatus;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
async function requestShutdown(): Promise<boolean> {
|
||||
try {
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), 5000);
|
||||
const res = await fetch(`${DAEMON_URL}/shutdown`, {
|
||||
method: 'POST',
|
||||
headers: { 'X-OpenCLI': '1' },
|
||||
signal: controller.signal,
|
||||
});
|
||||
clearTimeout(timer);
|
||||
return res.ok;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
function formatUptime(seconds: number): string {
|
||||
const h = Math.floor(seconds / 3600);
|
||||
const m = Math.floor((seconds % 3600) / 60);
|
||||
if (h > 0) return `${h}h ${m}m`;
|
||||
if (m > 0) return `${m}m`;
|
||||
return `${Math.floor(seconds)}s`;
|
||||
}
|
||||
|
||||
function formatTimeSince(timestampMs: number): string {
|
||||
const seconds = (Date.now() - timestampMs) / 1000;
|
||||
if (seconds < 60) return `${Math.floor(seconds)}s ago`;
|
||||
const m = Math.floor(seconds / 60);
|
||||
if (m < 60) return `${m} min ago`;
|
||||
const h = Math.floor(m / 60);
|
||||
return `${h}h ${m % 60}m ago`;
|
||||
}
|
||||
|
||||
export async function daemonStatus(): Promise<void> {
|
||||
const status = await fetchStatus();
|
||||
if (!status) {
|
||||
console.log(`Daemon: ${chalk.dim('not running')}`);
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(`Daemon: ${chalk.green('running')} (PID ${status.pid})`);
|
||||
console.log(`Uptime: ${formatUptime(status.uptime)}`);
|
||||
console.log(`Extension: ${status.extensionConnected ? chalk.green('connected') : chalk.yellow('disconnected')}`);
|
||||
console.log(`Last CLI request: ${formatTimeSince(status.lastCliRequestTime)}`);
|
||||
console.log(`Memory: ${status.memoryMB} MB`);
|
||||
console.log(`Port: ${status.port}`);
|
||||
}
|
||||
|
||||
export async function daemonStop(): Promise<void> {
|
||||
const status = await fetchStatus();
|
||||
if (!status) {
|
||||
console.log(chalk.dim('Daemon is not running.'));
|
||||
return;
|
||||
}
|
||||
|
||||
const ok = await requestShutdown();
|
||||
if (ok) {
|
||||
console.log(chalk.green('Daemon stopped.'));
|
||||
} else {
|
||||
console.error(chalk.red('Failed to stop daemon.'));
|
||||
process.exitCode = 1;
|
||||
}
|
||||
}
|
||||
|
||||
export async function daemonRestart(): Promise<void> {
|
||||
const status = await fetchStatus();
|
||||
if (status) {
|
||||
const ok = await requestShutdown();
|
||||
if (!ok) {
|
||||
console.error(chalk.red('Failed to stop daemon.'));
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
// Wait for daemon to exit
|
||||
await new Promise(r => setTimeout(r, 500));
|
||||
}
|
||||
|
||||
// Import BrowserBridge to spawn a new daemon
|
||||
const { BrowserBridge } = await import('../browser/mcp.js');
|
||||
const bridge = new BrowserBridge();
|
||||
try {
|
||||
console.log('Starting daemon...');
|
||||
await bridge.connect({ timeout: 10 });
|
||||
console.log(chalk.green('Daemon restarted.'));
|
||||
} catch (err) {
|
||||
console.error(chalk.red(`Failed to restart daemon: ${err instanceof Error ? err.message : err}`));
|
||||
process.exitCode = 1;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Register daemon commands in cli.ts**
|
||||
|
||||
In `src/cli.ts`, add the import at the top:
|
||||
|
||||
```typescript
|
||||
import { daemonStatus, daemonStop, daemonRestart } from './commands/daemon.js';
|
||||
```
|
||||
|
||||
Add the daemon subcommand group before the `// ── External CLIs` section (around line 380):
|
||||
|
||||
```typescript
|
||||
// ── Built-in: daemon ──────────────────────────────────────────────────────
|
||||
const daemonCmd = program.command('daemon').description('Manage the opencli daemon');
|
||||
daemonCmd
|
||||
.command('status')
|
||||
.description('Show daemon status')
|
||||
.action(async () => { await daemonStatus(); });
|
||||
daemonCmd
|
||||
.command('stop')
|
||||
.description('Stop the daemon')
|
||||
.action(async () => { await daemonStop(); });
|
||||
daemonCmd
|
||||
.command('restart')
|
||||
.description('Restart the daemon')
|
||||
.action(async () => { await daemonRestart(); });
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run linter/type check**
|
||||
|
||||
```bash
|
||||
npx tsc --noEmit
|
||||
```
|
||||
|
||||
Expected: No errors
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add src/commands/daemon.ts src/cli.ts
|
||||
git commit -m "feat(daemon): add opencli daemon status/stop/restart commands"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Write tests for daemon commands
|
||||
|
||||
**Files:**
|
||||
- Create: `src/commands/daemon.test.ts`
|
||||
|
||||
- [ ] **Step 1: Write tests**
|
||||
|
||||
Create `src/commands/daemon.test.ts`:
|
||||
|
||||
```typescript
|
||||
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
|
||||
|
||||
// Mock fetch globally for all tests
|
||||
const mockFetch = vi.fn();
|
||||
vi.stubGlobal('fetch', mockFetch);
|
||||
|
||||
// Mock chalk to avoid ANSI in assertions
|
||||
vi.mock('chalk', () => ({
|
||||
default: {
|
||||
green: (s: string) => s,
|
||||
yellow: (s: string) => s,
|
||||
red: (s: string) => s,
|
||||
dim: (s: string) => s,
|
||||
},
|
||||
}));
|
||||
|
||||
describe('daemonStatus', () => {
|
||||
let consoleSpy: ReturnType<typeof vi.spyOn>;
|
||||
|
||||
beforeEach(() => {
|
||||
consoleSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
consoleSpy.mockRestore();
|
||||
mockFetch.mockReset();
|
||||
});
|
||||
|
||||
it('shows "not running" when daemon is unreachable', async () => {
|
||||
mockFetch.mockRejectedValue(new TypeError('fetch failed'));
|
||||
|
||||
const { daemonStatus } = await import('./daemon.js');
|
||||
await daemonStatus();
|
||||
|
||||
expect(consoleSpy).toHaveBeenCalledWith(expect.stringContaining('not running'));
|
||||
});
|
||||
|
||||
it('shows daemon info when running', async () => {
|
||||
mockFetch.mockResolvedValue({
|
||||
ok: true,
|
||||
json: async () => ({
|
||||
ok: true,
|
||||
pid: 12345,
|
||||
uptime: 7200,
|
||||
extensionConnected: true,
|
||||
pending: 0,
|
||||
lastCliRequestTime: Date.now() - 60_000,
|
||||
memoryMB: 12.3,
|
||||
port: 19825,
|
||||
}),
|
||||
});
|
||||
|
||||
const { daemonStatus } = await import('./daemon.js');
|
||||
await daemonStatus();
|
||||
|
||||
expect(consoleSpy).toHaveBeenCalledWith(expect.stringContaining('running'));
|
||||
expect(consoleSpy).toHaveBeenCalledWith(expect.stringContaining('12345'));
|
||||
});
|
||||
});
|
||||
|
||||
describe('daemonStop', () => {
|
||||
let consoleSpy: ReturnType<typeof vi.spyOn>;
|
||||
let consoleErrSpy: ReturnType<typeof vi.spyOn>;
|
||||
|
||||
beforeEach(() => {
|
||||
consoleSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
|
||||
consoleErrSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
consoleSpy.mockRestore();
|
||||
consoleErrSpy.mockRestore();
|
||||
mockFetch.mockReset();
|
||||
});
|
||||
|
||||
it('reports when daemon is not running', async () => {
|
||||
mockFetch.mockRejectedValue(new TypeError('fetch failed'));
|
||||
|
||||
const { daemonStop } = await import('./daemon.js');
|
||||
await daemonStop();
|
||||
|
||||
expect(consoleSpy).toHaveBeenCalledWith(expect.stringContaining('not running'));
|
||||
});
|
||||
|
||||
it('sends shutdown and reports success', async () => {
|
||||
// First call: fetchStatus
|
||||
// Second call: requestShutdown
|
||||
mockFetch
|
||||
.mockResolvedValueOnce({
|
||||
ok: true,
|
||||
json: async () => ({ ok: true, pid: 123, uptime: 100, extensionConnected: false, pending: 0, lastCliRequestTime: Date.now(), memoryMB: 10, port: 19825 }),
|
||||
})
|
||||
.mockResolvedValueOnce({ ok: true });
|
||||
|
||||
const { daemonStop } = await import('./daemon.js');
|
||||
await daemonStop();
|
||||
|
||||
expect(consoleSpy).toHaveBeenCalledWith(expect.stringContaining('stopped'));
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests**
|
||||
|
||||
```bash
|
||||
npx vitest run src/commands/daemon.test.ts
|
||||
```
|
||||
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add src/commands/daemon.test.ts
|
||||
git commit -m "test(daemon): add tests for daemon status/stop commands"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Improve CLI connection-waiting UX
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/browser/mcp.ts:58-118`
|
||||
|
||||
- [ ] **Step 1: Improve error messages and poll interval**
|
||||
|
||||
In `src/browser/mcp.ts`, replace the `_ensureDaemon` method (lines 58-118) with:
|
||||
|
||||
```typescript
|
||||
private async _ensureDaemon(timeoutSeconds?: number): Promise<void> {
|
||||
const effectiveSeconds = (timeoutSeconds && timeoutSeconds > 0) ? timeoutSeconds : Math.ceil(DAEMON_SPAWN_TIMEOUT / 1000);
|
||||
const timeoutMs = effectiveSeconds * 1000;
|
||||
|
||||
// Fast path: extension already connected
|
||||
if (await isExtensionConnected()) return;
|
||||
|
||||
// Daemon running but no extension — wait for extension with progress
|
||||
if (await isDaemonRunning()) {
|
||||
if (process.env.OPENCLI_VERBOSE || process.stderr.isTTY) {
|
||||
process.stderr.write('⏳ Waiting for Chrome extension to connect...\n');
|
||||
process.stderr.write(' Make sure Chrome is open and the OpenCLI extension is enabled.\n');
|
||||
}
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
while (Date.now() < deadline) {
|
||||
await new Promise(resolve => setTimeout(resolve, 200));
|
||||
if (await isExtensionConnected()) return;
|
||||
}
|
||||
throw new Error(
|
||||
'Daemon is running but the Browser Extension is not connected.\n' +
|
||||
'Please install and enable the opencli Browser Bridge extension in Chrome.',
|
||||
);
|
||||
}
|
||||
|
||||
// No daemon — spawn one
|
||||
const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||||
const parentDir = path.resolve(__dirname, '..');
|
||||
const daemonTs = path.join(parentDir, 'daemon.ts');
|
||||
const daemonJs = path.join(parentDir, 'daemon.js');
|
||||
const isTs = fs.existsSync(daemonTs);
|
||||
const daemonPath = isTs ? daemonTs : daemonJs;
|
||||
|
||||
if (process.env.OPENCLI_VERBOSE || process.stderr.isTTY) {
|
||||
process.stderr.write('⏳ Starting daemon...\n');
|
||||
}
|
||||
|
||||
const spawnArgs = isTs
|
||||
? [process.execPath, '--import', 'tsx/esm', daemonPath]
|
||||
: [process.execPath, daemonPath];
|
||||
|
||||
this._daemonProc = spawn(spawnArgs[0], spawnArgs.slice(1), {
|
||||
detached: true,
|
||||
stdio: 'ignore',
|
||||
env: { ...process.env },
|
||||
});
|
||||
this._daemonProc.unref();
|
||||
|
||||
// Wait for daemon + extension with faster polling
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
while (Date.now() < deadline) {
|
||||
await new Promise(resolve => setTimeout(resolve, 200));
|
||||
if (await isExtensionConnected()) return;
|
||||
}
|
||||
|
||||
if (await isDaemonRunning()) {
|
||||
throw new Error(
|
||||
'Daemon is running but the Browser Extension is not connected.\n' +
|
||||
'Please install and enable the opencli Browser Bridge extension in Chrome.',
|
||||
);
|
||||
}
|
||||
|
||||
throw new Error(
|
||||
'Failed to start opencli daemon. Try running manually:\n' +
|
||||
` node ${daemonPath}\n` +
|
||||
`Make sure port ${DEFAULT_DAEMON_PORT} is available.`,
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run existing browser tests to check for regressions**
|
||||
|
||||
```bash
|
||||
npx vitest run src/browser.test.ts
|
||||
```
|
||||
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add src/browser/mcp.ts
|
||||
git commit -m "feat(daemon): improve CLI connection-waiting UX with progress messages and 200ms polling"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 8: Run full test suite and verify
|
||||
|
||||
- [ ] **Step 1: Run type check**
|
||||
|
||||
```bash
|
||||
npx tsc --noEmit
|
||||
```
|
||||
|
||||
Expected: No errors
|
||||
|
||||
- [ ] **Step 2: Run all tests**
|
||||
|
||||
```bash
|
||||
npx vitest run
|
||||
```
|
||||
|
||||
Expected: All tests pass, no regressions
|
||||
|
||||
- [ ] **Step 3: Manual smoke test**
|
||||
|
||||
```bash
|
||||
# Check daemon status (should be "not running" if daemon isn't started)
|
||||
npx tsx src/main.ts daemon status
|
||||
|
||||
# Start daemon by running any browser command, then check status
|
||||
npx tsx src/main.ts daemon status
|
||||
|
||||
# Stop daemon
|
||||
npx tsx src/main.ts daemon stop
|
||||
|
||||
# Verify stopped
|
||||
npx tsx src/main.ts daemon status
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Final commit if any fixes needed**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "fix: address issues found during smoke testing"
|
||||
```
|
||||
@@ -0,0 +1,170 @@
|
||||
# Performance: Smart Wait & INTERCEPT Fix
|
||||
|
||||
**Date**: 2026-03-28
|
||||
**Status**: Approved
|
||||
|
||||
## Problem
|
||||
|
||||
Three distinct performance/correctness issues:
|
||||
|
||||
1. **INTERCEPT strategy semantic bug**: After `installInterceptor()` + `goto()`, adapters call `wait(N)` — which now uses `waitForDomStableJs` and returns early when the DOM settles. But DOM-settle != network capture. The API response may arrive *after* DOM is stable, causing `getInterceptedRequests()` to return an empty array.
|
||||
|
||||
2. **Blind `wait(N)` in adapters**: ~30 high-traffic adapters (Twitter family, Medium, Substack, etc.) call `wait(5)` waiting for React/Vue to hydrate. These should wait for a specific DOM element to appear, not a fixed cap.
|
||||
|
||||
3. **Daemon cold-start polling**: Fixed 300ms poll loop means ~600ms before first successful `isExtensionConnected()` check, even though the daemon is typically ready in 500–800ms.
|
||||
|
||||
## Design
|
||||
|
||||
### Layer 1 — `waitForCapture()` (correctness fix + perf)
|
||||
|
||||
Add `waitForCapture(timeout?: number): Promise<void>` to `IPage`.
|
||||
|
||||
Polls `window.__opencli_xhr.length > 0` every 100ms inside the browser tab. Resolves as soon as ≥1 capture arrives; rejects after `timeout` seconds.
|
||||
|
||||
```typescript
|
||||
// dom-helpers.ts
|
||||
export function waitForCaptureJs(maxMs: number): string {
|
||||
return `
|
||||
new Promise((resolve, reject) => {
|
||||
const deadline = Date.now() + ${maxMs};
|
||||
const check = () => {
|
||||
if ((window.__opencli_xhr || []).length > 0) return resolve('captured');
|
||||
if (Date.now() > deadline) return reject(new Error('No capture within ${maxMs / 1000}s'));
|
||||
setTimeout(check, 100);
|
||||
};
|
||||
check();
|
||||
})
|
||||
`;
|
||||
}
|
||||
```
|
||||
|
||||
`page.ts` and `cdp.ts` implement `waitForCapture()` by calling `waitForCaptureJs`.
|
||||
|
||||
**All INTERCEPT adapters** replace `wait(N)` → `waitForCapture(N+2)` (slightly longer timeout as safety margin).
|
||||
|
||||
`stepIntercept` in `pipeline/steps/intercept.ts` replaces its internal `wait(timeout)` with `waitForCapture(timeout)`.
|
||||
|
||||
**Expected gain**: 36kr hot/search: 6s → ~1–2s. Twitter search/followers: 5–8s → ~1–3s.
|
||||
|
||||
### Layer 2 — `wait({ selector })` (semantic precision)
|
||||
|
||||
Extend `WaitOptions` with `selector?: string`.
|
||||
|
||||
Add `waitForSelectorJs(selector, timeoutMs)` to `dom-helpers.ts` — polls `document.querySelector(selector)` every 100ms, resolves on first match, rejects on timeout.
|
||||
|
||||
```typescript
|
||||
// types.ts
|
||||
export interface WaitOptions {
|
||||
text?: string;
|
||||
selector?: string; // NEW
|
||||
time?: number;
|
||||
timeout?: number;
|
||||
}
|
||||
```
|
||||
|
||||
```typescript
|
||||
// dom-helpers.ts
|
||||
export function waitForSelectorJs(selector: string, timeoutMs: number): string {
|
||||
return `
|
||||
new Promise((resolve, reject) => {
|
||||
const deadline = Date.now() + ${timeoutMs};
|
||||
const check = () => {
|
||||
if (document.querySelector(${JSON.stringify(selector)})) return resolve('found');
|
||||
if (Date.now() > deadline) return reject(new Error('Selector not found: ' + ${JSON.stringify(selector)}));
|
||||
setTimeout(check, 100);
|
||||
};
|
||||
check();
|
||||
})
|
||||
`;
|
||||
}
|
||||
```
|
||||
|
||||
`page.ts` and `cdp.ts` handle `selector` branch in `wait()`.
|
||||
|
||||
**High-impact adapter changes**:
|
||||
|
||||
| Adapter | Old | New |
|
||||
|---------|-----|-----|
|
||||
| `twitter/*` (15 adapters) | `wait(5)` | `wait({ selector: '[data-testid="primaryColumn"]', timeout: 6 })` |
|
||||
| `twitter/reply.ts` | `wait(5)` | `wait({ selector: '[data-testid="tweetTextarea_0"]', timeout: 8 })` |
|
||||
| `medium/utils.ts` | `wait(5)` + inline 3s setTimeout | `wait({ selector: 'article', timeout: 8 })` + remove inline sleep |
|
||||
| `substack/utils.ts` | `wait(5)` × 2 | `wait({ selector: 'article', timeout: 8 })` |
|
||||
| `bloomberg/news.ts` | `wait(5)` | `wait({ selector: 'article', timeout: 6 })` |
|
||||
| `sinablog/utils.ts` | `wait(5)` | `wait({ selector: 'article, .article', timeout: 6 })` |
|
||||
| `producthunt` (already covered by layer 1) | — | — |
|
||||
|
||||
**Expected gain**: Twitter commands: 5s → ~0.5–2s. Medium: 8s → ~1–3s.
|
||||
|
||||
### Layer 3 — Daemon exponential backoff (cold-start)
|
||||
|
||||
Replace fixed 300ms poll in `_ensureDaemon()` (`browser/mcp.ts`) with exponential backoff:
|
||||
|
||||
```typescript
|
||||
// before
|
||||
while (Date.now() < deadline) {
|
||||
await new Promise(resolve => setTimeout(resolve, 300));
|
||||
if (await isExtensionConnected()) return;
|
||||
}
|
||||
|
||||
// after
|
||||
const backoffs = [50, 100, 200, 400, 800, 1500, 3000];
|
||||
let i = 0;
|
||||
while (Date.now() < deadline) {
|
||||
await new Promise(resolve => setTimeout(resolve, backoffs[Math.min(i++, backoffs.length - 1)]));
|
||||
if (await isExtensionConnected()) return;
|
||||
}
|
||||
```
|
||||
|
||||
**Expected gain**: First cold-start check succeeds at ~150ms instead of ~600ms.
|
||||
|
||||
## Files Changed
|
||||
|
||||
### New / Modified (framework)
|
||||
- `src/types.ts` — `WaitOptions.selector`, `IPage.waitForCapture()`
|
||||
- `src/browser/dom-helpers.ts` — `waitForCaptureJs()`, `waitForSelectorJs()`
|
||||
- `src/browser/page.ts` — `waitForCapture()`, `wait()` selector branch
|
||||
- `src/browser/cdp.ts` — `waitForCapture()`, `wait()` selector branch
|
||||
- `src/browser/mcp.ts` — exponential backoff in `_ensureDaemon()`
|
||||
- `src/pipeline/steps/intercept.ts` — use `waitForCapture()`
|
||||
|
||||
### Modified (adapters — Layer 1, INTERCEPT)
|
||||
- `clis/36kr/hot.ts`
|
||||
- `clis/36kr/search.ts`
|
||||
- `clis/twitter/search.ts`
|
||||
- `clis/twitter/followers.ts`
|
||||
- `clis/twitter/following.ts`
|
||||
- `clis/producthunt/hot.ts`
|
||||
- `clis/producthunt/browse.ts`
|
||||
|
||||
### Modified (adapters — Layer 2, selector)
|
||||
- `clis/twitter/reply.ts`
|
||||
- `clis/twitter/follow.ts`
|
||||
- `clis/twitter/unfollow.ts`
|
||||
- `clis/twitter/like.ts`
|
||||
- `clis/twitter/bookmark.ts`
|
||||
- `clis/twitter/unbookmark.ts`
|
||||
- `clis/twitter/block.ts`
|
||||
- `clis/twitter/unblock.ts`
|
||||
- `clis/twitter/hide-reply.ts`
|
||||
- `clis/twitter/notifications.ts`
|
||||
- `clis/twitter/profile.ts`
|
||||
- `clis/twitter/thread.ts`
|
||||
- `clis/twitter/timeline.ts`
|
||||
- `clis/twitter/delete.ts`
|
||||
- `clis/twitter/reply-dm.ts`
|
||||
- `clis/medium/utils.ts`
|
||||
- `clis/substack/utils.ts`
|
||||
- `clis/bloomberg/news.ts`
|
||||
- `clis/sinablog/utils.ts`
|
||||
|
||||
## Delivery Order
|
||||
|
||||
1. Layer 1 (`waitForCapture`) — correctness fix, highest ROI
|
||||
2. Layer 3 (backoff) — 3-line change, zero risk
|
||||
3. Layer 2 (`wait({ selector })`) — largest adapter surface, can be done per-site
|
||||
|
||||
## Testing
|
||||
|
||||
- Unit tests: `waitForCaptureJs`, `waitForSelectorJs` exported and tested in `dom-helpers.test.ts` (if exists) or new test file
|
||||
- Adapter tests: existing tests must continue to pass (mock `page.wait` / `page.waitForCapture`)
|
||||
- Run: `npx vitest run --project unit --project adapter`
|
||||
@@ -0,0 +1,207 @@
|
||||
# Daemon Lifecycle Redesign
|
||||
|
||||
## Problem
|
||||
|
||||
OpenCLI's daemon auto-exits after 5 minutes of idle time. During typical development
|
||||
cycles (write code → test → modify → test again), coding intervals frequently exceed
|
||||
5 minutes. Each restart incurs 2-4 seconds of overhead (process spawn + Extension
|
||||
WebSocket reconnection), creating a noticeable and frustrating delay.
|
||||
|
||||
The current design treats the daemon as a disposable process, but the actual cost
|
||||
profile doesn't justify this:
|
||||
|
||||
| Cost of staying alive | Cost of restarting |
|
||||
|-----------------------|--------------------|
|
||||
| ~12 MB memory, 0% CPU | 2-4 seconds delay per restart |
|
||||
|
||||
The restart cost far outweighs the idle cost.
|
||||
|
||||
## Solution
|
||||
|
||||
Replace the aggressive 5-minute fixed timeout with a long-lived daemon model. The
|
||||
daemon stays running for hours, exits only when truly abandoned, and reconnects to
|
||||
the Chrome Extension faster when needed.
|
||||
|
||||
Four changes:
|
||||
|
||||
1. Extend idle timeout from 5 minutes to 4 hours (configurable)
|
||||
2. Require dual idle condition: both no CLI requests AND no Extension connection
|
||||
3. Reduce Extension WebSocket reconnect backoff cap from 60s to 5s
|
||||
4. Add `opencli daemon status/stop/restart` commands
|
||||
|
||||
## Design
|
||||
|
||||
### Timeout Strategy
|
||||
|
||||
**Current behavior:** A single idle timer resets on each HTTP request. After 5
|
||||
minutes without a request, the daemon calls `process.exit(0)`.
|
||||
|
||||
**New behavior:** The daemon tracks two activity signals independently:
|
||||
|
||||
- **CLI activity:** timestamp of the last HTTP request from any CLI invocation
|
||||
- **Extension activity:** whether a WebSocket connection from the Chrome Extension
|
||||
is currently open
|
||||
|
||||
The exit countdown starts only when BOTH conditions are met simultaneously:
|
||||
|
||||
- No CLI request for `IDLE_TIMEOUT` duration
|
||||
- No Extension WebSocket connection
|
||||
|
||||
If either signal is active, the daemon stays alive. This means:
|
||||
|
||||
- A connected Extension keeps the daemon alive indefinitely (user has Chrome open,
|
||||
likely still working)
|
||||
- Recent CLI activity keeps the daemon alive even if Extension temporarily
|
||||
disconnects (Chrome restarting, Extension updating)
|
||||
|
||||
**Timeout value:** 4 hours by default.
|
||||
|
||||
```typescript
|
||||
const DEFAULT_IDLE_TIMEOUT = 4 * 60 * 60 * 1000; // 4 hours
|
||||
const IDLE_TIMEOUT = DEFAULT_IDLE_TIMEOUT;
|
||||
```
|
||||
|
||||
**Timer implementation:**
|
||||
|
||||
```
|
||||
resetIdleTimer():
|
||||
clear existing timer
|
||||
if Extension is connected:
|
||||
do not start timer (Extension connection keeps daemon alive)
|
||||
return
|
||||
start timer with IDLE_TIMEOUT duration
|
||||
on timeout: process.exit(0)
|
||||
|
||||
On CLI HTTP request:
|
||||
update lastRequestTime
|
||||
resetIdleTimer()
|
||||
|
||||
On Extension WebSocket connect:
|
||||
clear timer (Extension keeps daemon alive)
|
||||
|
||||
On Extension WebSocket disconnect:
|
||||
elapsed = now - lastRequestTime
|
||||
if elapsed >= IDLE_TIMEOUT:
|
||||
process.exit(0) // CLI has been idle long enough already
|
||||
else:
|
||||
start timer with (IDLE_TIMEOUT - elapsed) // count remaining time
|
||||
```
|
||||
|
||||
### Extension Fast Reconnect
|
||||
|
||||
**Current behavior:** When the Extension loses its WebSocket connection to the
|
||||
daemon, it reconnects with exponential backoff: 2s → 4s → 8s → 16s → 32s → 60s
|
||||
(capped). In the worst case, the Extension waits up to 60 seconds before attempting
|
||||
reconnection.
|
||||
|
||||
**New behavior:** Cap the backoff at 5 seconds instead of 60 seconds.
|
||||
|
||||
```typescript
|
||||
// extension/src/background.ts
|
||||
const WS_RECONNECT_MAX_DELAY = 5000; // was 60000
|
||||
```
|
||||
|
||||
Rationale: with a 4-hour daemon timeout, the daemon is almost always running. Long
|
||||
backoff intervals are unnecessary and only increase reconnection latency. A 5-second
|
||||
cap means the Extension reconnects within 5 seconds of the daemon becoming available.
|
||||
|
||||
### Daemon Management Commands
|
||||
|
||||
Add three new CLI commands for daemon lifecycle management:
|
||||
|
||||
**`opencli daemon status`**
|
||||
|
||||
Queries the daemon's `/status` endpoint (new) and displays:
|
||||
|
||||
```
|
||||
Daemon: running (PID 12345)
|
||||
Uptime: 2h 15m
|
||||
Extension: connected
|
||||
Last CLI request: 8 min ago
|
||||
Memory: 12.3 MB
|
||||
Port: 19825
|
||||
```
|
||||
|
||||
If daemon is not running:
|
||||
|
||||
```
|
||||
Daemon: not running
|
||||
```
|
||||
|
||||
**`opencli daemon stop`**
|
||||
|
||||
Sends a `POST /shutdown` request to the daemon, which triggers a graceful shutdown:
|
||||
reject pending requests with a shutdown message, close WebSocket connections, close
|
||||
HTTP server, then exit.
|
||||
|
||||
**`opencli daemon restart`**
|
||||
|
||||
Equivalent to `stop` followed by spawning a new daemon. Useful when the daemon gets
|
||||
into a bad state.
|
||||
|
||||
**Daemon-side endpoints:**
|
||||
|
||||
- `GET /status` — returns JSON with PID, uptime, extension connection state, last
|
||||
request time, memory usage
|
||||
- `POST /shutdown` — initiates graceful shutdown
|
||||
|
||||
Both endpoints require the same `X-OpenCLI` header as existing endpoints for CSRF
|
||||
protection.
|
||||
|
||||
### CLI Connection Experience
|
||||
|
||||
**Current behavior:** When daemon is running but Extension is not connected, the CLI
|
||||
silently polls every 300ms and eventually times out with a generic error.
|
||||
|
||||
**New behavior:** Show a progress indicator and actionable message:
|
||||
|
||||
```
|
||||
⏳ Waiting for Chrome extension to connect...
|
||||
Make sure Chrome is open and the OpenCLI extension is enabled.
|
||||
```
|
||||
|
||||
Poll interval reduced from 300ms to 200ms for slightly faster detection.
|
||||
|
||||
If the daemon is not running at all (connection refused), the CLI spawns it as before
|
||||
and shows:
|
||||
|
||||
```
|
||||
⏳ Starting daemon...
|
||||
```
|
||||
|
||||
## Files Changed
|
||||
|
||||
| File | Change | Estimated LOC |
|
||||
|------|--------|---------------|
|
||||
| `src/daemon.ts` | Dual-condition idle timeout, `/status` endpoint, `/shutdown` endpoint | ~40 |
|
||||
| `extension/src/background.ts` | `WS_RECONNECT_MAX_DELAY` 60000 → 5000 | 1 |
|
||||
| `src/browser/daemon-client.ts` | Better connection-waiting UX, 200ms poll interval | ~20 |
|
||||
| `src/commands/daemon.ts` (new) | `status`, `stop`, `restart` subcommands | ~80 |
|
||||
| `src/constants.ts` | `DEFAULT_IDLE_TIMEOUT` constant | 2 |
|
||||
|
||||
**Total: ~143 lines of new/changed code.**
|
||||
|
||||
## Backward Compatibility
|
||||
|
||||
- No breaking changes to CLI commands or Extension protocol
|
||||
- The daemon and extension use the fixed Browser Bridge port.
|
||||
- The only observable behavior change: daemon stays alive longer
|
||||
- New `daemon` subcommands are additive
|
||||
|
||||
## Testing
|
||||
|
||||
- Unit test: idle timer starts only when both CLI and Extension are idle
|
||||
- Unit test: idle timer is cleared when Extension connects
|
||||
- Unit test: `/status` returns correct state
|
||||
- Unit test: `/shutdown` triggers graceful exit
|
||||
- Integration test: daemon survives 10+ minutes without CLI requests while Extension
|
||||
is connected
|
||||
- Integration test: daemon exits after configured timeout when fully idle
|
||||
- Integration test: `opencli daemon status/stop/restart` work correctly
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- OS-level daemon management (launchd/systemd) — can be added later if needed
|
||||
- Daemon auto-update mechanism
|
||||
- Multi-daemon coordination
|
||||
- Persistent daemon state across restarts
|
||||
@@ -0,0 +1,144 @@
|
||||
# Browse Skill Testing Design
|
||||
|
||||
Two-layer testing framework for `opencli browse` commands and the
|
||||
Claude Code skill integration.
|
||||
|
||||
## Goal
|
||||
|
||||
Verify that `opencli browse` works reliably on real websites and that
|
||||
Claude Code can use the skill to complete browser tasks end-to-end.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
autoresearch/
|
||||
├── browse-tasks.json ← 59 task definitions with browse command sequences
|
||||
├── eval-browse.ts ← Layer 1: deterministic browse command testing
|
||||
├── eval-skill.ts ← Layer 2: Claude Code skill E2E testing
|
||||
├── run-browse.sh ← Launch Layer 1
|
||||
├── run-skill.sh ← Launch Layer 2
|
||||
├── baseline-browse.txt ← Layer 1 best score
|
||||
├── baseline-skill.txt ← Layer 2 best score
|
||||
└── results/ ← Per-run results (gitignored)
|
||||
```
|
||||
|
||||
## Layer 1: Deterministic Browse Command Testing
|
||||
|
||||
Tests `opencli browse` commands directly on real websites. No LLM
|
||||
involved — pure command reliability testing.
|
||||
|
||||
### How It Works
|
||||
|
||||
Each task defines a sequence of browse commands and a judge for the
|
||||
last command's output:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "hn-top-stories",
|
||||
"steps": [
|
||||
"opencli browse open https://news.ycombinator.com",
|
||||
"opencli browse eval \"JSON.stringify([...document.querySelectorAll('.titleline a')].slice(0,5).map(a=>({title:a.textContent,url:a.href})))\""
|
||||
],
|
||||
"judge": { "type": "arrayMinLength", "minLength": 5 }
|
||||
}
|
||||
```
|
||||
|
||||
### Execution
|
||||
|
||||
```bash
|
||||
./autoresearch/run-browse.sh
|
||||
```
|
||||
|
||||
- Runs all 59 tasks serially
|
||||
- Each task: execute steps → judge last step output → pass/fail
|
||||
- `opencli browse close` between tasks for clean state
|
||||
- Expected: ~2 minutes, $0 cost
|
||||
|
||||
### Task Categories
|
||||
|
||||
| Category | Count | Example |
|
||||
|----------|-------|---------|
|
||||
| extract | 9 | Open page, eval JS to extract data |
|
||||
| list | 10 | Open page, eval JS to extract array |
|
||||
| search | 6 | Open, type query, keys Enter, eval results |
|
||||
| nav | 7 | Open, click link, eval new page title |
|
||||
| scroll | 5 | Open, scroll, eval footer/hidden content |
|
||||
| form | 6 | Open, type into fields, eval field values |
|
||||
| complex | 6 | Multi-step: open → click → navigate → extract |
|
||||
| bench | 10 | Test set (various) |
|
||||
|
||||
## Layer 2: Claude Code Skill E2E Testing
|
||||
|
||||
Spawns Claude Code with the opencli-browser skill to complete tasks
|
||||
autonomously using browse commands.
|
||||
|
||||
### How It Works
|
||||
|
||||
```bash
|
||||
claude -p \
|
||||
--system-prompt "$(cat skills/opencli-browser/SKILL.md)" \
|
||||
--dangerously-skip-permissions \
|
||||
--allowedTools "Bash(opencli:*)" \
|
||||
--output-format json \
|
||||
"用 opencli browse 完成任务:Extract the top 5 stories from Hacker News with title and score. Start URL: https://news.ycombinator.com"
|
||||
```
|
||||
|
||||
### Execution
|
||||
|
||||
```bash
|
||||
./autoresearch/run-skill.sh
|
||||
```
|
||||
|
||||
- Runs all 59 tasks serially
|
||||
- Each task: spawn Claude Code → it uses browse commands autonomously → judge output
|
||||
- Expected: ~20 minutes, ~$5-10
|
||||
|
||||
### Judge
|
||||
|
||||
Both layers use the same judge types:
|
||||
|
||||
| Type | Description |
|
||||
|------|-------------|
|
||||
| `contains` | Output contains a substring |
|
||||
| `arrayMinLength` | Output is an array with ≥ N items |
|
||||
| `arrayFieldsPresent` | Array items have required fields |
|
||||
| `nonEmpty` | Output is non-empty |
|
||||
| `matchesPattern` | Output matches a regex |
|
||||
|
||||
## Output Format
|
||||
|
||||
```
|
||||
🔬 Layer 1: Browse Commands — 59 tasks
|
||||
|
||||
[1/59] extract-title-example... ✓ (0.5s)
|
||||
[2/59] hn-top-stories... ✓ (1.2s)
|
||||
...
|
||||
|
||||
Score: 55/59 (93%)
|
||||
Time: 2min
|
||||
Cost: $0
|
||||
|
||||
🔬 Layer 2: Skill E2E — 59 tasks
|
||||
|
||||
[1/59] extract-title-example... ✓ (8s, $0.01)
|
||||
[2/59] hn-top-stories... ✓ (15s, $0.08)
|
||||
...
|
||||
|
||||
Score: 52/59 (88%)
|
||||
Time: 20min
|
||||
Cost: $6.50
|
||||
```
|
||||
|
||||
## Constraints
|
||||
|
||||
- All 59 tasks run on real websites (no mocks)
|
||||
- Layer 1: zero LLM cost, ~2 min
|
||||
- Layer 2: ~$5-10 LLM cost, ~20 min
|
||||
- Results saved to `autoresearch/results/` (gitignored)
|
||||
- Baselines tracked in `baseline-browse.txt` and `baseline-skill.txt`
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- Layer 1 ≥ 90% (browse commands work on real sites)
|
||||
- Layer 2 ≥ 85% (Claude Code can use skill effectively)
|
||||
- Both layers cover all 8 task categories
|
||||
@@ -0,0 +1,41 @@
|
||||
# V2EX AutoResearch Test Suite Design
|
||||
|
||||
## Goal
|
||||
|
||||
Build a comprehensive test suite using V2EX (https://v2ex.com/) as the single target website to iteratively improve OpenCLI Browser's reliability and Claude Code skill effectiveness. Run 10 rounds of AutoResearch iteration (5 code-level + 5 SKILL.md-level).
|
||||
|
||||
## Test Suite Structure — 5 Layers, 40 Tasks
|
||||
|
||||
### Layer 1: Atomic (10 tasks)
|
||||
Single browser commands testing command-level reliability.
|
||||
|
||||
### Layer 2: Single Page (10 tasks)
|
||||
Meaningful extraction/interaction within one page.
|
||||
|
||||
### Layer 3: Multi-Step (10 tasks)
|
||||
Cross-page navigation + extraction combos.
|
||||
|
||||
### Layer 4: Write Operations (5 tasks)
|
||||
Login-required write operations (reply, favorite, thank).
|
||||
|
||||
### Layer 5: Complex Chain (5 tasks)
|
||||
Long chains: cross-post reference, multi-node comparison, full workflows.
|
||||
|
||||
## Test Infrastructure
|
||||
|
||||
- **v2ex-tasks.json** — Layer 1 deterministic tasks (browse commands + judge criteria)
|
||||
- **eval-v2ex.ts** — Runner for V2EX tasks (reuses eval-browse.ts pattern)
|
||||
- **v2ex-skill-tasks** — Layer 2 LLM E2E tasks embedded in eval runner
|
||||
- **presets/v2ex-reliability.ts** — AutoResearch preset for code optimization
|
||||
- **presets/v2ex-skill.ts** — AutoResearch preset for SKILL.md optimization
|
||||
|
||||
## AutoResearch Iteration Plan
|
||||
|
||||
- Rounds 1-5: Layer 1 preset → optimize src/browser/*.ts code
|
||||
- Rounds 6-10: Layer 2 preset → optimize skills/opencli-browser/SKILL.md
|
||||
- Alternating: fix code issues first, then improve LLM guidance
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- Layer 1 baseline → target 100% pass rate after 5 code iterations
|
||||
- Layer 2 baseline → target 100% pass rate after 5 SKILL.md iterations
|
||||
Reference in New Issue
Block a user