Files
wehub-resource-sync 9b395f5cc3
E2E Headed Chrome / e2e-headed (macos-15) (push) Has been cancelled
E2E Headed Chrome / e2e-headed (ubuntu-latest) (push) Has been cancelled
E2E Headed Chrome / e2e-headed (windows-latest) (push) Has been cancelled
CI / build (macos-latest) (push) Has been cancelled
CI / build (ubuntu-latest) (push) Has been cancelled
CI / build (windows-latest) (push) Has been cancelled
CI / unit-test (push) Has been cancelled
CI / bun-test (push) Has been cancelled
CI / adapter-test (push) Has been cancelled
CI / smoke-test (macos-latest) (push) Has been cancelled
CI / smoke-test (ubuntu-latest) (push) Has been cancelled
Security Audit / audit (push) Has been cancelled
Build Chrome Extension / build (push) Has been cancelled
Trigger Website Rebuild (Docs Updated) / dispatch (push) Has been cancelled
chore: import upstream snapshot with attribution
2026-07-13 12:39:48 +08:00

170 lines
8.5 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Strategy Selection
SKILL.md 顶层已给出 strategy gate 的 enum、表格和必填字段。本文件展开**为什么**这套 ladder 是按"契约"而不是"接口高度"组织的,以及具体怎么用 `opencli browser analyze``api_candidates` 证据填 strategy note。
进入条件:你已经按 `site-recon.md` 跑过 `opencli browser analyze`、按 `api-discovery.md` 抓过候选 endpoint。本文件是写 note 之前的最后一站。
---
## 1. 核心模型:契约 vs 无契约
普遍假设 "API > DOM" — **数据不支持**
837 个内置 adapter 在 30 天观察窗(2026-04-20 → 2026-05-20)按 6 档 strategy 分类后的实测 fix 频率:
| Strategy | 契约级别 | fixes/adapter-year | 解读 |
|---|---|---|---|
| `PUBLIC_API` | stable | **1.18** | 一方文档化 API,最稳 |
| `COOKIE_API` | stable | 2.01 | 官方 web 接口 + 用户 cookie |
| `UI_SELECTOR` | visible-ui | 1.92 | DOM 的 a11y / semantic 约定也是契约 |
| `DOM_STATE` | visible-ui | 0.91 (N=11, 小样本) | hydration JSON 半契约 |
| `PAGE_FETCH` | **internal-unstable** | **8.41** | 站内未文档化 endpoint,最易漂 |
| `INTERCEPT` | **internal-unstable** | **8.69** | 拦截内部 XHR,签名/字段 silent drift |
含义:
-`PAGE_FETCH` / `INTERCEPT` 的 adapter 平均维护成本是 `PUBLIC_API`**~7-8 倍**
- `UI_SELECTOR` 在 1.92/year,跟 `COOKIE_API` 同档 — 不是"漂得最快"
- `DOM_STATE` 在 0.91/year 但 N=11 小样本,按 `UI_SELECTOR` 的近邻处理
**Selection bias caveat**`PAGE_FETCH` / `INTERCEPT` 高 fix 率部分来自 selection bias — 用这俩的本身就是难站(Twitter GraphQL、xhs signed URL)。但这不改变 practical implication:能用契约层就用契约层,别把稳定的 UI/DOM 实现盲目迁到无契约 endpoint。
数据观察窗局限:30 天窗口是近似不是长尾;`PAGE_FETCH`/`INTERCEPT`/`DOM_STATE` 样本量小(N=32/9/11),二期数据足时会单独评估 `DOM_STATE`
---
## 2. Ladder 心智模型
```
契约层(首选,互相平级,按 surface 适配):
PUBLIC_API ─┬─ COOKIE_API ─┬─ UI_SELECTOR ≈ DOM_STATE
(read) (write/click/upload)
无契约层(被迫才用,必须正向论证 8x 维护成本):
PAGE_FETCH ──── INTERCEPT
```
注意:**ladder 不是从上往下降级**。`UI_SELECTOR` 不是 `PUBLIC_API` 失败后的"惩罚选项"。如果数据/操作本来就是 UI 表面的事(publish、click、upload、表单),`UI_SELECTOR` 是首选,不需要为"为什么不是 API"过度辩护。
---
## 3. 怎么把 `api_candidates` 转化为 strategy note 证据
`opencli browser analyze <url>` 的输出里 `api_candidates[]` 字段,每条带:
```json
{
"url": "https://example.com/api/list",
"status": 200,
"contentType": "application/json",
"real_data_score": 0.82,
"verdict": "likely_data",
"reasons": ["json content-type", "non-empty top-level array", "3 business-like keys"],
"sample_paths": ["$.data.items:array(20)", "$.data.items[0].title:string"]
}
```
`verdict` 决策:
| Verdict | 含义 | strategy 信号 |
|---|---|---|
| `likely_data` (score ≥ 0.65) | 看起来是业务数据 | 优先 replay 这条做 `PUBLIC_API` / `COOKIE_API` 候选 |
| `maybe_data` (score 0.35-0.65) | 可能业务数据但有 telemetry / 空字段嫌疑 | replay 必须人工核对字段是不是目标数据 |
| `noise` | analytics / beacon / personalization | 不是 API 候选;Pattern A 不能基于这类条目成立 |
| `blocked` (401/403) | auth-gated | 先排 cookie / token / CSRF**不要**直接退到 `UI_SELECTOR` |
**关键**`real_data_score` 是证据,不是 strategy。你最终在 strategy note 里仍要写 replay 出来的 status / content-type / sample shape,不是把 score 直接当结论。
### 反例:booking #1680
```
Site: booking.com (酒店搜索)
analyze 输出:17 个 JSON XHR,原 Pattern A
但 api_candidates 全部 verdict=noiseanalytics + personalization + experiment
```
按 1.0.17 前的旧判定,agent 会按 Pattern A 写 `PAGE_FETCH` adapterreplay 拿到 noise data → adapter silent-fail。**新判定**`real_data_candidates = 0` → Pattern 落到 C → 提示 SSR HTML scrape → 正确的 strategy 是 `DOM_STATE` / `UI_SELECTOR`
`browser analyze``recommended_next_step` 也已更新为 "Inspect api_candidates, then replay the best endpoint" — 不再按 XHR count 推 API。
---
## 4. Strategy note 的关键字段填法
### `Contract` 字段
不是直接从 strategy enum 抄,而是反映"这个 source 有多稳"
- `stable`:一方文档化 API、官方 web 接口(PUBLIC_API、COOKIE_API
- `visible-ui`:用户可见的 DOM、a11y / semantic 标记(UI_SELECTOR、DOM_STATE
- `internal-unstable`:站内未文档化 endpoint、签名 / queryId 漂移、字段 silent renamePAGE_FETCH、INTERCEPT
### `Evidence` 三行
每行都是事实,不是猜测:
```md
- observed request/state: GET /api/v2/list (sample_paths: $.data.items:array(20), $.data.items[0].title:string)
- auth source: browser cookie (sessionid),无 CSRF
- replay result: 200 / application/json / 20 items / 非空
```
`observed request/state``DOM_STATE` 时写 state global key`window.__INITIAL_STATE__.feed.items`);在 `UI_SELECTOR` 时写 selector path 或 a11y locator`role=list[name="Trending"] > listitem`)。
### `If PAGE_FETCH or INTERCEPT` 三行论证
```md
Why PUBLIC_API / COOKIE_API are unavailable: <因为 a_bogus signature 必须 page runtime 生成 / 公开 API 缺少 since 字段 / 接口仅在登录态曝露但 cookie 透传会触发 anti-bot>
Why UI_SELECTOR / DOM_STATE are not safer: <因为数据是无限滚动 + 增量加载,DOM 一次只能拿 1 屏 / 因为目标是 write actionUI 无对应操作>
Why the maintenance cost is acceptable: <因为业务需求要 raw timeline cursor / 因为已经接受漂时 autofix 流程兜底>
```
**反模式**
- ❌ "因为 API 比 DOM 高级" — 不是论证,是假设
- ❌ "因为 selector 不可靠" — 数据不支持(UI_SELECTOR 跟 COOKIE_API 同档)
- ❌ "因为我看到 17 XHR" — 不是论证,是 booking #1680 反例
正确论证须基于:endpoint 的**真实**不可达 / 操作语义本质 / 维护成本承担方有明确接收方。
### `If UI_SELECTOR / DOM_STATE`
```md
- semantic anchor: <a11y role / data-testid / framework-stable class>
- typed error path: <selector 失配时抛 EmptyResultError / CommandExecutionError>
```
不需要"why not API"过度辩护。如果你能简短说一句"目标是 publish,没有公开 write API"或"数据在 SSR HTML 直接 inline 了"就够了。
---
## 5. 与其他 reference 的关系
| 文件 | 关系 |
|---|---|
| [`api-discovery.md`](./api-discovery.md) | §1-5 是 endpoint 发现的具体方法。本文件指它,但本文件管"用 endpoint 证据填 strategy note",那边管"怎么先找到 endpoint" |
| [`site-recon.md`](./site-recon.md) | Pattern A-E 是 site classification。Pattern A 命中 ≠ `PAGE_FETCH` 必然合适 — 还要看 `api_candidates` 是不是 `likely_data` |
| [`coverage-matrix.md`](./coverage-matrix.md) | 鉴权列已对齐 6 档 strategy enum |
| [`adapter-template.md`](./adapter-template.md) | 写代码模板。strategy note 应该在打开 template 之前已经定好 |
| [`success-rate-pitfalls.md`](./success-rate-pitfalls.md) | 11 种 silent failure 模式 — 多数发生在 strategy 选错时(比如把 noise endpoint 当业务数据) |
---
## 6. 反例案例库
### booking #1680 — Pattern A 误判
旧判定按 XHR count 推 Pattern A,实际 17 XHR 全是 analytics / personalization side channel。新 `verdict` 系统能识别为 `noise`,落到 Pattern C → SSR HTML scrape。
### Twitter GraphQL — PAGE_FETCH 高维护成本的典型
`queryId` 每隔 1-2 月漂一次,字段名 silent rename`legacy.user_screen_name``core.user.screen_name`)。30 天 9 个 fix PR。`Why the maintenance cost is acceptable` 的合理论证:业务需要 raw timeline cursor、autofix 流程已接住、fixed 时间窗口可控。
### xiaohongshu signed URL — INTERCEPT 必要场景
`a_bogus` signature 由 page runtime 即时生成,无法在 Node 端复现也不能拷贝 cookie 跨 origin replay。合理 strategy 是 `INTERCEPT`:触发 UI 让页面自己发请求,从 response 取数据。
### weread-official — PUBLIC_API 首选
WeRead 官方 Agent Gateway 有 Bearer auth + 文档化 schema。一方契约 + 不依赖 cookie / 不依赖浏览器 — 最理想的 strategy。维护成本最低。