chore: import upstream snapshot with attribution
E2E Headed Chrome / e2e-headed (macos-15) (push) Has been cancelled
E2E Headed Chrome / e2e-headed (ubuntu-latest) (push) Has been cancelled
E2E Headed Chrome / e2e-headed (windows-latest) (push) Has been cancelled
CI / build (macos-latest) (push) Has been cancelled
CI / build (ubuntu-latest) (push) Has been cancelled
CI / build (windows-latest) (push) Has been cancelled
CI / unit-test (push) Has been cancelled
CI / bun-test (push) Has been cancelled
CI / adapter-test (push) Has been cancelled
CI / smoke-test (macos-latest) (push) Has been cancelled
CI / smoke-test (ubuntu-latest) (push) Has been cancelled
Security Audit / audit (push) Has been cancelled
Build Chrome Extension / build (push) Has been cancelled
Trigger Website Rebuild (Docs Updated) / dispatch (push) Has been cancelled
E2E Headed Chrome / e2e-headed (macos-15) (push) Has been cancelled
E2E Headed Chrome / e2e-headed (ubuntu-latest) (push) Has been cancelled
E2E Headed Chrome / e2e-headed (windows-latest) (push) Has been cancelled
CI / build (macos-latest) (push) Has been cancelled
CI / build (ubuntu-latest) (push) Has been cancelled
CI / build (windows-latest) (push) Has been cancelled
CI / unit-test (push) Has been cancelled
CI / bun-test (push) Has been cancelled
CI / adapter-test (push) Has been cancelled
CI / smoke-test (macos-latest) (push) Has been cancelled
CI / smoke-test (ubuntu-latest) (push) Has been cancelled
Security Audit / audit (push) Has been cancelled
Build Chrome Extension / build (push) Has been cancelled
Trigger Website Rebuild (Docs Updated) / dispatch (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,65 @@
|
||||
# AI Workflow
|
||||
|
||||
OpenCLI is designed for AI agents writing adapters. The workflow is built on a small set of browser primitives plus a skill that teaches the end-to-end loop.
|
||||
|
||||
## The Loop
|
||||
|
||||
From a new site URL to a passing `opencli browser verify` — one skill, one set of primitives:
|
||||
|
||||
```bash
|
||||
# 1. Pick up the skill (Claude Code)
|
||||
# skills/opencli-adapter-author/SKILL.md
|
||||
|
||||
# 2. Reconnaissance
|
||||
opencli browser analyze https://example.com
|
||||
# Fallback primitives when analyze says deeper inspection is needed:
|
||||
# opencli browser open https://example.com
|
||||
# opencli browser network # inspect XHR / fetch calls
|
||||
# opencli browser state # extract __INITIAL_STATE__ / __NEXT_DATA__
|
||||
|
||||
# 3. Scaffold + verify
|
||||
opencli browser init <site>/<name>
|
||||
opencli browser verify <site>/<name>
|
||||
```
|
||||
|
||||
The skill `opencli-adapter-author` walks through: coverage self-test → site recon → API discovery → field decoding → output design → adapter coding → verify → write-back to site memory.
|
||||
|
||||
See [skills/opencli-adapter-author/SKILL.md](https://github.com/jackwener/opencli/blob/main/skills/opencli-adapter-author/SKILL.md).
|
||||
|
||||
## Primitives
|
||||
|
||||
| Command | Purpose |
|
||||
|---------|---------|
|
||||
| `opencli doctor` | Sanity check: bridge, Chrome, signals |
|
||||
| `opencli browser analyze <url>` | One-shot site recon: anti-bot, pattern, nearest adapter, next step |
|
||||
| `opencli browser open <url>` | Open a tab in the Chrome session |
|
||||
| `opencli browser network` | List recent XHR / fetch calls |
|
||||
| `opencli browser state` | Page state: URL, title, interactive elements |
|
||||
| `opencli browser eval '<expr>'` | Evaluate JS in the page context (cookies + origin honored) |
|
||||
| `opencli browser init <site>/<name>` | Scaffold `~/.opencli/clis/<site>/<name>.js` |
|
||||
| `opencli browser verify <site>/<name>` | Run the adapter and print first rows |
|
||||
|
||||
No `explore` / `synthesize` / `generate` / `cascade` command. The skill drives the loop — the primitives are small and composable.
|
||||
|
||||
## Site Memory
|
||||
|
||||
Every site accumulates knowledge at `~/.opencli/sites/<site>/` (endpoints, field decode map, notes, response fixtures). The adapter-author skill reads memory on Step 2 and writes back on Step 12 — see `skills/opencli-adapter-author/references/site-memory.md` for the schema.
|
||||
|
||||
In-repo seeds for well-known sites live at `skills/opencli-adapter-author/references/site-memory/<site>.md` (eastmoney / xueqiu / bilibili / tonghuashun already covered).
|
||||
|
||||
## Authentication Strategies
|
||||
|
||||
Adapters declare one of:
|
||||
|
||||
1. **PUBLIC** — direct fetch, no credentials
|
||||
2. **COOKIE** — reuse Chrome session cookies (`browser: true` + `credentials: 'include'`)
|
||||
3. **INTERCEPT** — let the page make the request; capture the response
|
||||
4. **UI** — drive the authenticated browser UI when no stable API is available
|
||||
|
||||
Pick per the `coverage-matrix.md` and `api-discovery.md` references inside the skill.
|
||||
|
||||
## When Something Breaks
|
||||
|
||||
- Verify failure → run `opencli doctor`, then consult `skills/opencli-autofix/SKILL.md`
|
||||
- Field values wrong → jump back to `skills/opencli-adapter-author/references/field-decode-playbook.md`
|
||||
- Endpoint returns 401/403 → `api-discovery.md` §4 (token) / §5 (intercept)
|
||||
@@ -0,0 +1,163 @@
|
||||
# Architecture
|
||||
|
||||
OpenCLI is a command surface that sits on top of four major subsystems:
|
||||
|
||||
1. command discovery and registry
|
||||
2. execution and formatting
|
||||
3. browser / daemon / CDP connectivity
|
||||
4. adapter, plugin, and external CLI integration
|
||||
|
||||
## Runtime Shape
|
||||
|
||||
```text
|
||||
opencli CLI
|
||||
├─ command discovery / registry
|
||||
├─ execution / output
|
||||
├─ browser runtime
|
||||
│ ├─ Browser Bridge extension
|
||||
│ ├─ local daemon
|
||||
│ └─ direct CDP path
|
||||
├─ adapter loading
|
||||
│ ├─ built-in site adapters
|
||||
│ ├─ generated adapters
|
||||
│ └─ pipeline-backed adapters
|
||||
├─ plugin loading
|
||||
└─ external CLI passthrough
|
||||
```
|
||||
|
||||
## Core Modules
|
||||
|
||||
### CLI Surface
|
||||
|
||||
- `src/main.ts` — process entrypoint
|
||||
- `src/cli.ts` — top-level command tree and built-in command groups
|
||||
- `src/completion.ts` / `src/completion-fast.ts` — shell completion
|
||||
|
||||
### Discovery, Registry, Execution
|
||||
|
||||
- `src/discovery.ts` — discovers built-in adapters, generated adapters, plugins, and manifests
|
||||
- `src/registry.ts` — central command registry
|
||||
- `src/registry-api.ts` — adapter-facing registration helpers
|
||||
- `src/execution.ts` — argument validation, lazy loading, and command execution
|
||||
- `src/commanderAdapter.ts` — bridges registry metadata into Commander subcommands
|
||||
- `src/output.ts` — `table`, `json`, `yaml`, `md`, `csv` formatting
|
||||
- `src/serialization.ts` — registry and manifest serialization helpers
|
||||
|
||||
### Browser and Runtime
|
||||
|
||||
- `src/runtime.ts` — shared command runtime and target resolution
|
||||
- `src/daemon.ts` — lifecycle and bridge behavior for the local daemon
|
||||
- `src/doctor.ts` — browser bridge diagnostics
|
||||
- `src/observation/` — trace artifacts, redaction, and structured runtime evidence
|
||||
- `src/interceptor.ts` — interception helpers for browser-backed strategies
|
||||
- `src/browser/` — Browser Bridge connection and browser-side primitives
|
||||
|
||||
### Pipeline Engine
|
||||
|
||||
- `src/pipeline/executor.ts` — pipeline execution
|
||||
- `src/pipeline/template.ts` — template expansion
|
||||
- `src/pipeline/transform.ts` — transform helpers
|
||||
- `src/pipeline/steps/` — concrete steps such as:
|
||||
- `fetch`
|
||||
- `download`
|
||||
- `browser`
|
||||
- `intercept`
|
||||
- `tap`
|
||||
- `transform`
|
||||
|
||||
### Adapter and Extension Surfaces
|
||||
|
||||
- `clis/` — built-in site adapters
|
||||
- `src/plugin.ts` / `src/plugin-manifest.ts` / `src/plugin-scaffold.ts` — plugin install, metadata, scaffold
|
||||
- `src/external.ts` / `src/external-clis.yaml` — external CLI passthrough and installable tools
|
||||
- `src/electron-apps.ts` — desktop / Electron app support
|
||||
|
||||
## Command Sources
|
||||
|
||||
OpenCLI merges commands from multiple places into one registry:
|
||||
|
||||
| Source | Location | Examples |
|
||||
|---|---|---|
|
||||
| Built-in adapters | `clis/` | `twitter`, `bilibili`, `reddit`, `chatgpt-app` |
|
||||
| Generated / local adapters | `~/.opencli/clis/` | user-authored adapters |
|
||||
| Plugins | `~/.opencli/plugins/` | community-contributed commands |
|
||||
| External CLIs | `src/external-clis.yaml` + local registrations | `gh`, `docker`, `vercel` |
|
||||
|
||||
The user sees one unified command tree through `opencli list`.
|
||||
|
||||
## Connectivity Modes
|
||||
|
||||
### Browser Bridge mode
|
||||
|
||||
Primary path for browser-backed commands:
|
||||
|
||||
```text
|
||||
opencli process
|
||||
↔ local daemon
|
||||
↔ Browser Bridge extension
|
||||
↔ logged-in Chrome / Chromium
|
||||
```
|
||||
|
||||
This path is used for:
|
||||
|
||||
- cookie-backed websites
|
||||
- browser automation primitives
|
||||
- interactive browser verification
|
||||
|
||||
### Direct CDP mode
|
||||
|
||||
Used when OpenCLI talks directly to a Chrome or Electron debugging endpoint through `OPENCLI_CDP_ENDPOINT`.
|
||||
|
||||
Typical uses:
|
||||
|
||||
- remote Chrome
|
||||
- headless Chrome
|
||||
- Electron desktop adapters
|
||||
|
||||
## Authentication / Access Strategies
|
||||
|
||||
OpenCLI currently uses these access strategies:
|
||||
|
||||
| Strategy | Purpose |
|
||||
|---|---|
|
||||
| `public` | direct fetch with no login |
|
||||
| `cookie` | reuse browser session cookies |
|
||||
| `intercept` | capture the app's own network responses |
|
||||
| `ui` | DOM / accessibility driven interaction |
|
||||
|
||||
The key distinction is operational:
|
||||
|
||||
- `public` favors direct network access
|
||||
- `cookie`, `intercept`, `ui` depend on a live browser or desktop surface
|
||||
|
||||
## High-Risk Change Zones
|
||||
|
||||
Changes in these files usually affect broad command behavior:
|
||||
|
||||
- `src/cli.ts`
|
||||
- `src/commanderAdapter.ts`
|
||||
- `src/discovery.ts`
|
||||
- `src/execution.ts`
|
||||
- `src/runtime.ts`
|
||||
- `src/daemon.ts`
|
||||
- `src/plugin.ts`
|
||||
- `src/external.ts`
|
||||
- `src/pipeline/**`
|
||||
|
||||
These areas deserve targeted tests first, then broader validation when the change crosses module boundaries.
|
||||
|
||||
## Mental Model
|
||||
|
||||
The simplest accurate model is:
|
||||
|
||||
1. OpenCLI discovers command definitions.
|
||||
2. It registers them into one command registry.
|
||||
3. It resolves each invocation through execution + runtime.
|
||||
4. It reaches the target through one of:
|
||||
- network fetch
|
||||
- Browser Bridge
|
||||
- direct CDP
|
||||
- external CLI passthrough
|
||||
5. It formats the result into a stable output surface.
|
||||
|
||||
That is the architecture to preserve when refactoring.
|
||||
@@ -0,0 +1,120 @@
|
||||
# Contributing
|
||||
|
||||
Thanks for your interest in contributing to OpenCLI.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# 1. Fork & clone
|
||||
git clone git@github.com:<your-username>/opencli.git
|
||||
cd opencli
|
||||
|
||||
# 2. Install dependencies
|
||||
npm install
|
||||
|
||||
# 3. Build
|
||||
npm run build
|
||||
|
||||
# 4. Run a few checks
|
||||
npx tsc --noEmit
|
||||
npm run build
|
||||
|
||||
# 5. Link globally (optional, for testing `opencli` command)
|
||||
npm link
|
||||
```
|
||||
|
||||
## Adding a New Site Adapter
|
||||
|
||||
This is the most common type of contribution. All adapters use TypeScript with the `cli()` API.
|
||||
|
||||
Before you start:
|
||||
|
||||
- Prefer positional args for the command's primary subject (`search <query>`, `topic <id>`, `download <url>`). Reserve named flags for optional modifiers such as `--limit`, `--sort`, `--lang`, and `--output`.
|
||||
- Normalize expected adapter failures to `CliError` subclasses instead of raw `Error` whenever possible. Prefer `AuthRequiredError`, `EmptyResultError`, `CommandExecutionError`, `TimeoutError`, and `ArgumentError` so the top-level CLI can render better messages and hints.
|
||||
- If you add a new adapter or make a command newly discoverable, update the matching doc page and the user-facing indexes that expose it.
|
||||
|
||||
### Create the Adapter
|
||||
|
||||
Built-in adapters are authored in JavaScript. Create a file like `clis/<site>/<command>.js`:
|
||||
|
||||
```javascript
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
|
||||
cli({
|
||||
site: 'mysite',
|
||||
name: 'search',
|
||||
description: 'Search MySite',
|
||||
access: 'read', // 'read' | 'write'
|
||||
example: 'opencli mysite search <query> -f yaml',
|
||||
domain: 'www.mysite.com',
|
||||
strategy: Strategy.COOKIE,
|
||||
args: [
|
||||
{ name: 'query', positional: true, required: true, help: 'Search query' },
|
||||
{ name: 'limit', type: 'int', default: 10, help: 'Max results' },
|
||||
],
|
||||
columns: ['title', 'url', 'date'],
|
||||
|
||||
func: async (page, kwargs) => {
|
||||
const { query, limit = 10 } = kwargs;
|
||||
// ... browser automation logic
|
||||
if (!Array.isArray(data)) throw new CommandExecutionError('MySite returned an unexpected response');
|
||||
if (!data.length) throw new EmptyResultError('mysite search', 'Try a different keyword');
|
||||
return data.slice(0, Number(limit)).map((item) => ({
|
||||
title: item.title,
|
||||
url: item.url,
|
||||
date: item.created_at,
|
||||
}));
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
> TypeScript adapters are also supported — see [TypeScript Adapter](./ts-adapter).
|
||||
|
||||
### Validate Your Adapter
|
||||
|
||||
```bash
|
||||
opencli <site> <command> --limit 3 -f json # Test your command
|
||||
opencli <site> <command> -v # Verbose mode for debugging
|
||||
```
|
||||
|
||||
## Code Style
|
||||
|
||||
- **TypeScript strict mode** — avoid `any` where possible.
|
||||
- **ES Modules** — use `.js` extensions in imports (TypeScript output).
|
||||
- **Naming**: `kebab-case` for files, `camelCase` for variables/functions, `PascalCase` for types/classes.
|
||||
- **No default exports** — use named exports.
|
||||
- **Errors** — throw `CliError` subclasses for expected adapter failures; avoid raw `Error` for normal adapter control flow.
|
||||
|
||||
## Commit Convention
|
||||
|
||||
We use [Conventional Commits](https://www.conventionalcommits.org/):
|
||||
|
||||
```
|
||||
feat(twitter): add thread command
|
||||
fix(browser): handle CDP timeout gracefully
|
||||
docs: update CONTRIBUTING.md
|
||||
test(reddit): add e2e test for save command
|
||||
chore: bump vitest to v4
|
||||
```
|
||||
|
||||
## Submitting a Pull Request
|
||||
|
||||
1. Create a feature branch: `git checkout -b feat/mysite-trending`
|
||||
2. Make your changes and add tests when relevant
|
||||
3. Run the smallest check set that matches your change:
|
||||
```bash
|
||||
npx tsc --noEmit # Type check
|
||||
npm run build # Ensure dist stays healthy
|
||||
npx vitest run clis/<site>/<command>.test.js # Your adapter's tests
|
||||
npm test # Broader local gate when shared runtime changes justify it
|
||||
```
|
||||
4. Commit using conventional commit format
|
||||
5. Push and open a PR
|
||||
|
||||
If your PR adds a new adapter or changes user-facing commands, also verify:
|
||||
|
||||
- Adapter docs exist under `docs/adapters/`
|
||||
- `docs/adapters/index.md` is updated for new adapters
|
||||
- VitePress sidebar includes the new doc page
|
||||
- `README.md` / `README.zh-CN.md` stay aligned when command discoverability changes
|
||||
@@ -0,0 +1,368 @@
|
||||
# Documentation Audit — 2026-05
|
||||
|
||||
This document reviews the current long-form docs, README surfaces, and developer guides in `opencli`. It focuses on stale facts, internal contradictions, and documentation structure that now causes drift.
|
||||
|
||||
## Scope
|
||||
|
||||
- `README.md`
|
||||
- `README.zh-CN.md`
|
||||
- `docs/`
|
||||
- `skills/` references that are linked from user-facing docs
|
||||
|
||||
## Executive View
|
||||
|
||||
The docs are usable, but they are drifting in four visible ways:
|
||||
|
||||
1. **Hard-coded counts and feature claims are stale.**
|
||||
2. **Developer docs describe an older architecture and older test layout.**
|
||||
3. **English and Chinese docs are no longer updated with the same rigor.**
|
||||
4. **Some pages still describe deleted concepts or old workflows.**
|
||||
|
||||
The highest-value work is:
|
||||
|
||||
1. Fix the stale facts in `README*`, `docs/index.md`, `docs/zh/index.md`, and `docs/guide/getting-started.md`.
|
||||
2. Rewrite `docs/developer/testing.md` and `docs/developer/architecture.md` against current `main`.
|
||||
3. Make English and Chinese entry docs derive from the same source-of-truth checklist.
|
||||
4. Stop writing command/adapters counts by hand unless they are generated.
|
||||
|
||||
## Priority 0 — Clearly stale or incorrect
|
||||
|
||||
### 1. Adapter / site counts are stale across multiple entry points
|
||||
|
||||
Affected files:
|
||||
|
||||
- `README.md`
|
||||
- `README.zh-CN.md`
|
||||
- `docs/guide/getting-started.md`
|
||||
- `docs/comparison.md`
|
||||
|
||||
Current problems:
|
||||
|
||||
- `README.md` and `README.zh-CN.md` still say `90+` adapters.
|
||||
- `docs/guide/getting-started.md` still says `87+` pre-built adapters.
|
||||
- `docs/comparison.md` still says `87+` sites.
|
||||
|
||||
Current reality:
|
||||
|
||||
- `node dist/src/main.js list --format json | jq 'map(.site) | unique | length'` returns `106`.
|
||||
|
||||
Why this matters:
|
||||
|
||||
- These are the first pages people read.
|
||||
- The mismatch is easy to notice and weakens trust in the rest of the docs.
|
||||
- These values will keep drifting if we maintain them manually.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Replace all hard-coded counts with one of:
|
||||
- `100+`
|
||||
- `100+ sites`
|
||||
- `over 100 registered sites`
|
||||
- Best option: generate this number into docs at release time or avoid explicit counts entirely.
|
||||
|
||||
### 2. `docs/developer/testing.md` is materially out of date
|
||||
|
||||
Affected file:
|
||||
|
||||
- `docs/developer/testing.md`
|
||||
|
||||
Current problems:
|
||||
|
||||
- It says adapter tests live in `clis/**/*.test.{ts,js}`.
|
||||
- The file examples name adapter tests such as:
|
||||
- `clis/zhihu/download.test.ts`
|
||||
- `clis/twitter/timeline.test.ts`
|
||||
- `clis/reddit/read.test.ts`
|
||||
- `clis/bilibili/dynamic.test.ts`
|
||||
- Those files do not exist.
|
||||
- It says E2E coverage is `5` files.
|
||||
- Current reality is `11` E2E files.
|
||||
- It presents `npm test` as the main local gate, while current team rule is to prefer the smallest sufficient test set instead of default full-suite runs.
|
||||
|
||||
Current reality from the repo:
|
||||
|
||||
- `find src -name '*.test.ts' | wc -l` → `60`
|
||||
- `find clis -iregex '.*\\.test\\.(ts|js)$' | wc -l` → `0`
|
||||
- `find tests/e2e -name '*.test.ts' | wc -l` → `11`
|
||||
- `find tests/smoke -name '*.test.ts' | wc -l` → `1`
|
||||
|
||||
Why this matters:
|
||||
|
||||
- This page is the main developer testing contract.
|
||||
- A new contributor following it will get the wrong mental model of the test layout.
|
||||
- It encourages a heavier default test habit than the team currently wants.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Rewrite the page from current files, not from remembered structure.
|
||||
- Separate:
|
||||
- `fast local checks`
|
||||
- `targeted validation`
|
||||
- `full CI coverage`
|
||||
- Remove nonexistent adapter test examples.
|
||||
- Add a short rule:
|
||||
- local default = smallest sufficient validation
|
||||
- full-suite = broader refactor, shared runtime changes, or CI
|
||||
|
||||
### 3. `docs/developer/architecture.md` describes an older system shape
|
||||
|
||||
Affected file:
|
||||
|
||||
- `docs/developer/architecture.md`
|
||||
|
||||
Current problems:
|
||||
|
||||
- It refers to `src/browser.ts`, but that file does not exist.
|
||||
- The directory structure block says `src/clis/`, but adapters live at top-level `clis/`.
|
||||
- The architecture diagram is too simplified for the current system and omits important pieces such as:
|
||||
- `daemon.ts`
|
||||
- `external.ts`
|
||||
- `plugin.ts`
|
||||
- `electron-apps.ts`
|
||||
- update check / diagnostics / runtime detection paths
|
||||
- It says “3-tier authentication strategy” but lists `5` strategies.
|
||||
|
||||
Why this matters:
|
||||
|
||||
- This is the page people read to understand the project.
|
||||
- Once architecture docs are stale, all deeper docs become harder to trust.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Rewrite this page around current modules:
|
||||
- command discovery and registry
|
||||
- execution
|
||||
- browser / daemon bridge
|
||||
- external CLI integration
|
||||
- plugin system
|
||||
- desktop / CDP path
|
||||
- pipeline engine
|
||||
- Replace the static tree with a curated module map that matches current filenames.
|
||||
- Change “3-tier” to a neutral label like `authentication strategies`.
|
||||
|
||||
### 4. Home pages still mention deleted concepts
|
||||
|
||||
Affected files:
|
||||
|
||||
- `docs/index.md`
|
||||
- `docs/zh/index.md`
|
||||
|
||||
Current problems:
|
||||
|
||||
- Both home pages say:
|
||||
- `explore`
|
||||
- `synthesize`
|
||||
- `cascade`
|
||||
- `docs/developer/ai-workflow.md` explicitly says those commands do not exist and that the skill drives the loop.
|
||||
|
||||
Why this matters:
|
||||
|
||||
- The home page is currently teaching a product vocabulary that the actual CLI does not have.
|
||||
- This creates immediate confusion for users who go from docs to terminal.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Replace those phrases with current concepts:
|
||||
- `browser primitives`
|
||||
- `adapter-authoring skill`
|
||||
- `verify loop`
|
||||
- Keep the homepage aligned with `docs/developer/ai-workflow.md`.
|
||||
|
||||
### 5. Chinese getting-started page lists a deleted built-in command
|
||||
|
||||
Affected file:
|
||||
|
||||
- `docs/zh/guide/getting-started.md`
|
||||
|
||||
Current problem:
|
||||
|
||||
- It says built-in commands include `list、explore、validate...`
|
||||
- `explore` is not a current built-in command.
|
||||
|
||||
Why this matters:
|
||||
|
||||
- This is a hard user-facing error.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Replace the example list with current built-ins such as:
|
||||
- `list`
|
||||
- `validate`
|
||||
- `verify`
|
||||
- `browser`
|
||||
- `doctor`
|
||||
- `plugin`
|
||||
- `adapter`
|
||||
|
||||
## Priority 1 — Inconsistent or incomplete
|
||||
|
||||
### 6. Installation pages are inconsistent about runtime support and update flow
|
||||
|
||||
Affected files:
|
||||
|
||||
- `README.md`
|
||||
- `README.zh-CN.md`
|
||||
- `docs/guide/installation.md`
|
||||
- `docs/zh/guide/installation.md`
|
||||
|
||||
Current problems:
|
||||
|
||||
- `README.md` says Node `>= 21` or Bun `>= 1.0`.
|
||||
- `docs/guide/installation.md` and `docs/zh/guide/installation.md` only mention Node.
|
||||
- `README.md` documents skill refresh on update.
|
||||
- `docs/zh/guide/installation.md` only documents package update and omits skills refresh.
|
||||
|
||||
Why this matters:
|
||||
|
||||
- Entry docs should agree on install prerequisites and upgrade procedure.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Pick one official runtime support statement and reuse it everywhere.
|
||||
- If Bun is supported, add it consistently to guide pages.
|
||||
- Mirror the post-update skill refresh guidance in the install/update guides.
|
||||
|
||||
### 7. README and docs still use top-level tables and examples that will drift by hand
|
||||
|
||||
Affected files:
|
||||
|
||||
- `README.md`
|
||||
- `README.zh-CN.md`
|
||||
|
||||
Current problems:
|
||||
|
||||
- The “Built-in Commands” section is manually curated and already partially selective.
|
||||
- The surrounding copy still frames it like a broad current snapshot.
|
||||
|
||||
Why this matters:
|
||||
|
||||
- Manual command snapshots go stale quickly in a repo with active adapter growth.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Reframe the section as:
|
||||
- “Representative built-in commands”
|
||||
- “Sample sites”
|
||||
- Keep `opencli list` and `docs/adapters/index.md` as the full registry surface.
|
||||
|
||||
### 8. `docs/comparison.md` contains stale scale claims
|
||||
|
||||
Affected file:
|
||||
|
||||
- `docs/comparison.md`
|
||||
|
||||
Current problem:
|
||||
|
||||
- It still says `87+` sites.
|
||||
|
||||
Why this matters:
|
||||
|
||||
- Comparison pages shape market positioning.
|
||||
- Stale numbers make the project look less maintained than it is.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Remove exact numbers from comparison copy unless they are generated.
|
||||
|
||||
## Priority 2 — Structural drift risks
|
||||
|
||||
### 9. English and Chinese docs are drifting independently
|
||||
|
||||
Most visible examples:
|
||||
|
||||
- `docs/index.md` and `docs/zh/index.md` both kept the deleted `explore / synthesize / cascade` language.
|
||||
- `docs/zh/guide/getting-started.md` contains a stale built-in command example that should have been caught by parity review.
|
||||
- `README.md` and `README.zh-CN.md` both carry the same stale adapter count.
|
||||
|
||||
Why this keeps happening:
|
||||
|
||||
- We have mirrored content with no explicit parity checklist.
|
||||
- Updates land in one place and rely on memory for the rest.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Introduce a small doc parity checklist for any change that touches:
|
||||
- `README.md`
|
||||
- `README.zh-CN.md`
|
||||
- `docs/index.md`
|
||||
- `docs/zh/index.md`
|
||||
- `docs/guide/*`
|
||||
- `docs/zh/guide/*`
|
||||
- Add one PR checklist item:
|
||||
- “Did this change require an English/Chinese mirror update?”
|
||||
|
||||
### 10. Core product pages mix generated facts with narrative copy
|
||||
|
||||
Examples:
|
||||
|
||||
- command counts
|
||||
- site counts
|
||||
- test counts
|
||||
- lists of built-in commands
|
||||
|
||||
Why this matters:
|
||||
|
||||
- Numbers and command inventories drift faster than narrative guidance.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- For fast-changing facts:
|
||||
- generate them
|
||||
- or generalize them
|
||||
- Reserve hand-written docs for:
|
||||
- mental models
|
||||
- workflows
|
||||
- constraints
|
||||
- trade-offs
|
||||
|
||||
## Suggested rewrite order
|
||||
|
||||
### Pass 1 — Fix trust-breaking errors
|
||||
|
||||
1. `README.md`
|
||||
2. `README.zh-CN.md`
|
||||
3. `docs/index.md`
|
||||
4. `docs/zh/index.md`
|
||||
5. `docs/guide/getting-started.md`
|
||||
6. `docs/zh/guide/getting-started.md`
|
||||
7. `docs/comparison.md`
|
||||
|
||||
### Pass 2 — Rebuild the technical source-of-truth pages
|
||||
|
||||
1. `docs/developer/testing.md`
|
||||
2. `docs/developer/architecture.md`
|
||||
3. `docs/guide/installation.md`
|
||||
4. `docs/zh/guide/installation.md`
|
||||
|
||||
### Pass 3 — Prevent the next round of drift
|
||||
|
||||
1. Add a docs parity checklist to PR workflow.
|
||||
2. Remove exact counts from hand-written copy unless generated.
|
||||
3. Decide which pages are authoritative for:
|
||||
- install
|
||||
- browser bridge
|
||||
- testing
|
||||
- architecture
|
||||
- AI workflow
|
||||
|
||||
## Concrete edits I would make next
|
||||
|
||||
### Small fast edits
|
||||
|
||||
- Replace all `87+` / `90+` claims with `100+`.
|
||||
- Remove `explore / synthesize / cascade` from both home pages.
|
||||
- Remove `explore` from `docs/zh/guide/getting-started.md`.
|
||||
- Align install docs on Node/Bun support and skill refresh.
|
||||
|
||||
### Medium rewrites
|
||||
|
||||
- Rewrite `docs/developer/testing.md` from current filesystem state.
|
||||
- Rewrite `docs/developer/architecture.md` from current module boundaries.
|
||||
|
||||
### Process fix
|
||||
|
||||
- Add a lightweight “doc drift” checklist to PRs that touch command surface, runtime support, testing strategy, or adapter discovery.
|
||||
|
||||
## Bottom line
|
||||
|
||||
The docs do not need a ground-up rewrite. They need a focused trust repair pass on entry pages, then a source-of-truth rebuild for testing and architecture, then a small process change so counts and mirrored pages stop drifting.
|
||||
@@ -0,0 +1,157 @@
|
||||
# Testing Guide
|
||||
|
||||
> 面向开发者和 AI Agent 的当前测试参考手册。
|
||||
|
||||
## 测试结构
|
||||
|
||||
OpenCLI 当前测试主要分成四类:
|
||||
|
||||
| 类别 | 位置 | 当前规模 | 主要用途 |
|
||||
|---|---|---:|---|
|
||||
| 单元测试 | `src/**/*.test.ts` | 60 | 核心运行时、命令层、浏览器桥、输出、插件、诊断 |
|
||||
| E2E 测试 | `tests/e2e/*.test.ts` | 11 | 真实 CLI 入口、公开站点、浏览器命令、管理命令、输出格式 |
|
||||
| 烟雾测试 | `tests/smoke/*.test.ts` | 1 | 外部 API 与注册完整性健康检查 |
|
||||
| 步骤级测试 | `src/pipeline/steps/*.test.ts` | 已包含在单元测试内 | pipeline step 行为与边界情况 |
|
||||
|
||||
当前仓库里没有独立的 `clis/**/*.test.{ts,js}` adapter 测试树。adapter 相关验证主要分布在:
|
||||
|
||||
- `tests/e2e/`
|
||||
- `src/commanderAdapter.test.ts`
|
||||
- `src/registry.test.ts`
|
||||
- `src/execution.test.ts`
|
||||
- `src/validate.ts` / `opencli validate`
|
||||
|
||||
## 本地默认策略
|
||||
|
||||
本地默认跑最小充分验证,不要先跑全量。
|
||||
|
||||
推荐顺序:
|
||||
|
||||
1. 改动命令文案、输出格式、参数解析:
|
||||
- 跑对应单元测试
|
||||
- 跑一条真实 CLI 命令做 spot check
|
||||
2. 改动 adapter 发现、注册、验证逻辑:
|
||||
- 跑 `src/registry.test.ts`
|
||||
- 跑 `src/execution.test.ts`
|
||||
- 跑 `opencli validate`
|
||||
3. 改动 browser / daemon / runtime:
|
||||
- 跑对应 `src/*test.ts`
|
||||
- 必要时补一条 `tests/e2e/*` 或手动 `opencli browser ...` 验证
|
||||
4. 改动共享底层、跨多个模块、或 merge 前需要更高信心:
|
||||
- 再扩大到 `npm test`
|
||||
|
||||
## 常用命令
|
||||
|
||||
```bash
|
||||
# 类型检查
|
||||
npx tsc --noEmit
|
||||
|
||||
# 编译产物
|
||||
npm run build
|
||||
|
||||
# 跑一个目标测试文件
|
||||
npx vitest run src/<target>.test.ts
|
||||
|
||||
# 全量 vitest projects
|
||||
npm run test:all
|
||||
|
||||
# E2E
|
||||
npm run test:e2e
|
||||
|
||||
# 适配器注册 / schema 校验
|
||||
node dist/src/main.js validate
|
||||
```
|
||||
|
||||
如果你明确要跑 adapter project,也可以执行:
|
||||
|
||||
```bash
|
||||
npm run test:adapter
|
||||
```
|
||||
|
||||
## 当前 E2E 文件
|
||||
|
||||
当前 `tests/e2e/` 包含:
|
||||
|
||||
- `browser-auth.test.ts`
|
||||
- `browser-public.test.ts`
|
||||
- `cli.test.ts`
|
||||
- `extension-bridge.test.ts`
|
||||
- `formats.test.ts`
|
||||
- `list.test.ts`
|
||||
- `management.test.ts`
|
||||
- `public-commands.test.ts`
|
||||
- `recovery.test.ts`
|
||||
- `remote-chrome.test.ts`
|
||||
- `tab-targeting.test.ts`
|
||||
|
||||
如果这个列表变化,以仓库文件为准:
|
||||
|
||||
```bash
|
||||
find tests/e2e -name '*.test.ts' | sort
|
||||
```
|
||||
|
||||
## 当前值得优先覆盖的区域
|
||||
|
||||
以下改动最容易引入回归:
|
||||
|
||||
- `src/cli.ts`
|
||||
- `src/commanderAdapter.ts`
|
||||
- `src/discovery.ts`
|
||||
- `src/execution.ts`
|
||||
- `src/runtime.ts`
|
||||
- `src/daemon.ts`
|
||||
- `src/plugin.ts`
|
||||
- `src/external.ts`
|
||||
- `src/pipeline/**`
|
||||
|
||||
这类改动优先补:
|
||||
|
||||
- 精准单元测试
|
||||
- 一条真实 CLI 验证路径
|
||||
- 必要时再扩大到 `npm test`
|
||||
|
||||
## 手动验证建议
|
||||
|
||||
文档或命令面改动后,优先做 2 到 4 条真实命令 spot check,例如:
|
||||
|
||||
```bash
|
||||
node dist/src/main.js --help
|
||||
node dist/src/main.js list --format json
|
||||
node dist/src/main.js plugin --help
|
||||
node dist/src/main.js doctor --help
|
||||
```
|
||||
|
||||
浏览器相关改动再补:
|
||||
|
||||
```bash
|
||||
node dist/src/main.js browser --help
|
||||
node dist/src/main.js browser tab list
|
||||
```
|
||||
|
||||
## CI 角色
|
||||
|
||||
CI 负责更大范围的回归信心,本地负责最快闭环。
|
||||
|
||||
适合交给 CI 的内容:
|
||||
|
||||
- 更大的命令面回归
|
||||
- 多环境差异
|
||||
- E2E 稳定性
|
||||
- smoke 检查
|
||||
|
||||
适合本地优先做的内容:
|
||||
|
||||
- 参数解析
|
||||
- 输出格式
|
||||
- 注册与发现
|
||||
- 文档相关命令行为
|
||||
- 共享模块的小范围回归
|
||||
|
||||
## 更新这份文档的规则
|
||||
|
||||
当以下任一项变化时,顺手更新此页:
|
||||
|
||||
- `tests/e2e/` 文件列表
|
||||
- 默认本地测试命令
|
||||
- `package.json` 测试脚本
|
||||
- 共享运行时的高风险模块
|
||||
@@ -0,0 +1,156 @@
|
||||
# TypeScript Adapter Guide
|
||||
|
||||
Use TypeScript adapters when you need browser-side logic, multi-step flows, DOM manipulation, or complex data extraction that goes beyond simple API fetching.
|
||||
|
||||
## Basic Structure
|
||||
|
||||
```typescript
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
|
||||
cli({
|
||||
site: 'mysite',
|
||||
name: 'search',
|
||||
description: 'Search MySite',
|
||||
access: 'read', // 'read' | 'write'
|
||||
example: 'opencli mysite search <query> -f yaml',
|
||||
domain: 'www.mysite.com',
|
||||
strategy: Strategy.COOKIE, // PUBLIC | COOKIE | INTERCEPT | UI
|
||||
args: [
|
||||
{ name: 'query', required: true, help: 'Search query' },
|
||||
{ name: 'limit', type: 'int', default: 10, help: 'Max results' },
|
||||
],
|
||||
columns: ['title', 'url', 'date'],
|
||||
|
||||
func: async (page, kwargs) => {
|
||||
const { query, limit = 10 } = kwargs;
|
||||
|
||||
// Navigate and extract data
|
||||
await page.goto('https://www.mysite.com');
|
||||
|
||||
const data = await page.evaluate(async (q: string) => {
|
||||
const res = await fetch(`/api/search?q=${encodeURIComponent(q)}`, {
|
||||
credentials: 'include',
|
||||
});
|
||||
return (await res.json()).results;
|
||||
}, String(query));
|
||||
|
||||
if (!Array.isArray(data)) throw new CommandExecutionError('MySite returned an unexpected response');
|
||||
if (!data.length) throw new EmptyResultError('mysite search', 'Try a different keyword');
|
||||
|
||||
return data.slice(0, Number(limit)).map((item: any) => ({
|
||||
title: item.title,
|
||||
url: item.url,
|
||||
date: item.created_at,
|
||||
}));
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
## Access Metadata
|
||||
|
||||
Every adapter must declare `access: 'read' | 'write'`.
|
||||
|
||||
- Use `read` when the command only retrieves data from the target product or account.
|
||||
- Use `write` when the command changes remote product/account state, such as sending messages, publishing, liking, following, buying, deleting, creating remote assets, or starting paid/credit-consuming generation.
|
||||
- `download` and `export` commands are `read` when they only read remote data and write local files; local filesystem writes are a separate permission dimension.
|
||||
|
||||
Adapters may also declare `example` to override the canonical invocation shown in agent-facing help. Prefer YAML examples, e.g. `opencli mysite search <query> -f yaml`.
|
||||
|
||||
## Listing↔Detail ID Pairing (advisory)
|
||||
|
||||
If your site exposes both a listing-class command (`search` / `hot` / `top` /
|
||||
`recent` / ...) and a detail-class command (`read` / `paper` / `article` /
|
||||
`post` / `view` / ...), it's usually nicer for agents if listing rows surface
|
||||
an id-shaped column that round-trips into the detail command's positional
|
||||
arg. Without that, an agent can't follow up on a row without re-searching by
|
||||
title or scraping a URL out of band.
|
||||
|
||||
This is a **soft convention**, not a CI gate. Many legitimate listings
|
||||
genuinely don't pair (topic-string trending, profile-attribute rows,
|
||||
UI-only sessions). Use judgment per command, not a checklist.
|
||||
|
||||
Run `npm run advise:listing-id-pairing` to see candidate listings without an
|
||||
id column. See [Listing↔Detail ID Pairing](../conventions/listing-detail-id-pairing.md)
|
||||
for context, the full pattern table, and how to add an id to a listing.
|
||||
|
||||
## Strategy Types
|
||||
|
||||
| Strategy | Constant | Use Case |
|
||||
|----------|----------|----------|
|
||||
| Public | `Strategy.PUBLIC` | No auth needed |
|
||||
| Cookie | `Strategy.COOKIE` | Browser session cookies |
|
||||
| Intercept | `Strategy.INTERCEPT` | Capture browser requests/responses |
|
||||
| UI | `Strategy.UI` | Drive authenticated browser UI |
|
||||
|
||||
## Browser Session Reuse
|
||||
|
||||
Browser-backed commands are one-shot by default: each execution gets a fresh
|
||||
tab lease and releases it when the command returns. For interactive sites where
|
||||
successive commands should continue in the same page, opt into a persistent site
|
||||
session:
|
||||
|
||||
```typescript
|
||||
cli({
|
||||
site: 'mysite',
|
||||
name: 'ask',
|
||||
strategy: Strategy.COOKIE,
|
||||
siteSession: 'persistent',
|
||||
// ...
|
||||
});
|
||||
```
|
||||
|
||||
`siteSession: 'persistent'` makes commands for the same site share a stable
|
||||
adapter site tab and keeps that tab open until it is explicitly closed. Users
|
||||
can override the adapter default with `--site-session ephemeral` or force
|
||||
persistence with `--site-session persistent`.
|
||||
|
||||
## The `page` Object
|
||||
|
||||
The `page` parameter provides browser interaction methods:
|
||||
|
||||
- `page.goto(url)` — Navigate to a URL
|
||||
- `page.evaluate(fn, ...args)` — Execute a serializable function in the page context. Pass Node-side values through JSON-serializable args; the function cannot close over local variables.
|
||||
- `page.evaluate(script)` — Execute a raw JavaScript string in the page context. Prefer function form for new adapter code.
|
||||
- `page.waitForSelector(selector)` — Wait for an element
|
||||
- `page.click(selector)` — Click an element
|
||||
- `page.type(selector, text)` — Type text into an input
|
||||
|
||||
## The `kwargs` Object
|
||||
|
||||
Contains parsed CLI arguments as key-value pairs. Always destructure with defaults:
|
||||
|
||||
```typescript
|
||||
const { query, limit = 10, format = 'json' } = kwargs;
|
||||
```
|
||||
|
||||
For most search/read/detail commands, the main subject should be positional (`opencli mysite search "rust"`, `opencli mysite article 123`) instead of a named flag such as `--query` or `--id`. Keep named flags for optional modifiers.
|
||||
|
||||
## Error Handling
|
||||
|
||||
Prefer throwing `CliError` subclasses from `src/errors.ts` for expected adapter failures:
|
||||
|
||||
- `AuthRequiredError` for missing login / cookies
|
||||
- `EmptyResultError` for empty but valid responses
|
||||
- `CommandExecutionError` for unexpected API or browser failures
|
||||
- `TimeoutError` for site timeouts
|
||||
- `ArgumentError` for invalid user input
|
||||
|
||||
Avoid raw `Error` for normal adapter control flow. This keeps top-level CLI output consistent and preserves hints for users.
|
||||
|
||||
## AI-Assisted Development
|
||||
|
||||
Use the `opencli-adapter-author` skill plus the `opencli browser *` primitives to scaffold and verify adapters end-to-end:
|
||||
|
||||
```bash
|
||||
# Recon on the target site
|
||||
opencli browser open https://example.com
|
||||
opencli browser network
|
||||
opencli browser state
|
||||
|
||||
# Scaffold + verify
|
||||
opencli browser init mysite/trending
|
||||
opencli browser verify mysite/trending
|
||||
```
|
||||
|
||||
See [AI Workflow](/developer/ai-workflow) for the full loop and the adapter-author skill for the step-by-step runbook.
|
||||
@@ -0,0 +1,5 @@
|
||||
# YAML Adapter Guide (Deprecated)
|
||||
|
||||
> **YAML adapters are no longer supported.** All adapters now use TypeScript with the `cli()` API from `@jackwener/opencli/registry`.
|
||||
|
||||
See [Contributing Guide](./contributing) for how to write TypeScript adapters using the pipeline API or `func()`.
|
||||
Reference in New Issue
Block a user