commit 3a46cbf880cc06c8ac01794a3cc17ba2dabfe209 Author: wehub-resource-sync Date: Mon Jul 13 13:13:20 2026 +0800 chore: import upstream snapshot with attribution diff --git a/.github/FUNDING.yml b/.github/FUNDING.yml new file mode 100644 index 0000000..05ef389 --- /dev/null +++ b/.github/FUNDING.yml @@ -0,0 +1,4 @@ +# Shows a "Sponsor" button at the top of this repository. +# Docs: https://docs.github.com/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/displaying-a-sponsor-button-in-your-repository +buy_me_a_coffee: wenyuchiou +# github: WenyuChiou # uncomment once the GitHub Sponsors application is approved diff --git a/.github/ISSUE_TEMPLATE/bug-report.md b/.github/ISSUE_TEMPLATE/bug-report.md new file mode 100644 index 0000000..f59b4d9 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug-report.md @@ -0,0 +1,24 @@ +--- +name: 🐛 Bug Report / 內容問題 +about: 連結失效、資訊過時、星等錯誤、license 寫錯等 +title: "[bug] " +labels: bug +--- + + + +## 問題描述 / Issue + + +## 受影響的檔案 / Affected file + + +## 看到的內容 / What you saw + + +## 應該是什麼 / What it should be + + +## 其他資訊 / Other info +- [ ] 我已經確認連結真的失效（用 `curl -I` 或瀏覽器試過） +- [ ] 我看過 [`resources/style-guide.md`](../resources/style-guide.md) diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 0000000..0c9ffe7 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,13 @@ +# Issue 入口設定 — 把「不是 bug / 不是新增 entry」的提問導去 Discussions, +# 避免 issue tracker 被「我該怎麼開始」「stage X 卡住」這類問題灌爆。 +blank_issues_enabled: false +contact_links: + - name: 💬 提問 / 學習討論（Discussions Q&A） + url: https://github.com/WenyuChiou/awesome-agentic-ai-zh/discussions + about: 學習路線怎麼走、某個 stage / branch 卡住、想討論 agent 工具選型——這類「不是 bug、不是新增 project」的問題請開 Discussion，不要開 issue。 + - name: 📐 新增 project 前先讀策展標準（style-guide） + url: https://github.com/WenyuChiou/awesome-agentic-ai-zh/blob/main/resources/style-guide.md + about: 新增 project entry 的 schema、license 標註慣例、品質門檻都在這裡。讀完再用「Project Suggestion」範本開 issue。 + - name: 🔒 回報安全問題（Security） + url: https://github.com/WenyuChiou/awesome-agentic-ai-zh/blob/main/SECURITY.md + about: 惡意連結、範例程式碼供應鏈風險、外洩密鑰——請依 SECURITY.md 用私人管道回報，不要開公開 issue。 diff --git a/.github/ISSUE_TEMPLATE/project-suggestion.md b/.github/ISSUE_TEMPLATE/project-suggestion.md new file mode 100644 index 0000000..7fcd3ba --- /dev/null +++ b/.github/ISSUE_TEMPLATE/project-suggestion.md @@ -0,0 +1,45 @@ +--- +name: 💡 Project Suggestion / 推薦新 project +about: 建議在某個 stage 加入新 project +title: "[suggest] for Stage " +labels: suggestion +--- + + + +## Project 連結 / Project URL + + +## 建議放在哪個 stage / branch + + +## 為什麼這個 project 教這個 stage？/ Why does it teach this stage? + + +## 通過 [策展標準](../../CONTRIBUTING.md#策展標準) 嗎？ +- [ ] 最近 6 個月內有 commit（或明確標示 stable） +- [ ] 有 hello-world 文件，30 分鐘內能跑起來 +- [ ] License 明確（MIT / Apache-2 / BSD / 等） +- [ ] 維護者可信（知名組織 / 個人 / 公司） + +## 提議的 entry 草稿（選填）/ Draft entry (optional) + + +```markdown +### [Project Name](url) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| 語言 | Python | +| Stars | ★ Xk+ | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐ | + +**教什麼**：... + +**適合誰**：... +``` + +## 其他資訊 +- [ ] 我已經讀過 [`resources/style-guide.md`](../resources/style-guide.md) +- [ ] 我確認這個 project 不是 self-promotion（如果是，請標註利益關係） diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 0000000..de30c62 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,37 @@ + + +## PR 類型 / Type + +- [ ] 🆕 新增 project（Add new project entry） +- [ ] 🔧 修正錯誤（Fix error: stale link / wrong info） +- [ ] 🌏 翻譯（Translation: zh + en companion） +- [ ] 📝 內容改善（Content improvement） +- [ ] 🎨 風格 / 結構（Style / structure） +- [ ] 其他 / Other + +## 影響範圍 / Affected files + + +## 摘要 / Summary + + +## 如果是新增 project：策展標準 + +- [ ] 最近 6 個月內有 commit（或明確標示 stable） +- [ ] 有 hello-world 文件 +- [ ] License 明確 +- [ ] 維護者可信 +- [ ] 我可以一句話說明它**教這個 stage 的什麼** + +## 如果是翻譯 / 文字修正：style-guide check +- [ ] 沒有 zh-Hans 用詞（教程 / 視頻 / 軟件 / 用戶 / 網絡 / 接口 / 默認 / 函数 / 算法） +- [ ] 沒有 overclaim（the best / production-grade / 首選 / 全世界最好） +- [ ] License 標註符合 [慣例](../resources/style-guide.md#5-license-標註慣例) +- [ ] zh + en companion 兩邊都有更新（如果只動到一邊請說明為什麼） + +## 額外說明 / Additional notes + + +--- + +**Reviewer 會檢查**：策展理由是否成立、entry schema 是否符合 style guide、license 標註是否正確、zh/en 結構是否同步。 diff --git a/.github/TESTING-STATUS.md b/.github/TESTING-STATUS.md new file mode 100644 index 0000000..8e31585 --- /dev/null +++ b/.github/TESTING-STATUS.md @@ -0,0 +1,74 @@ +# Testing Status — 誠實揭露 + +> 這份是給 maintainer / 第一個跑各個 build 的人看的。**誠實地說明哪些 code 真的跑過、哪些只是 syntax check、哪些完全沒測**。 + +最後更新：2026-05-06 + +--- + +## ✅ 真的跑過、有觀察輸出 + +| 項目 | 狀態 | 證據 | +|---|---|---| +| `scripts/refresh-stars.py` | ✅ Verified | 在 main 上跑過 N 次，0 drift / 0 not-found 都有實際輸出 | +| `scripts/check-links.py --fast` | ✅ Verified | 跑過 120 GitHub URLs 全 OK | +| `gh api` repo 元資料抓取 | ✅ Verified | 152 個 entry 的 stars / license / pushed 都對證過至少一次 | +| Mermaid syntax | ✅ Verified | GitHub 上 render 看過正確（README hero） | +| CI banned-words / overclaim grep | ✅ Verified | 用相同 grep 邏輯本地跑過，0 violations | + +--- + +## ⚠️ 只做了 syntax check / 配置 validation，沒實際 end-to-end 跑 + +| 項目 | 狀態 | 缺什麼 | +|---|---|---| +| `scripts/build-pdf.sh` | ⚠️ Bash syntax OK | **沒實際跑過** pandoc + xelatex；沒驗證輸出的 PDF 真的能開、CJK 字型真的可用 | +| `scripts/build-mdbook.sh` | ⚠️ Bash syntax OK | 跑過一次但 mdbook-mermaid 失敗（已 fix 但沒重跑驗證） | +| `.github/workflows/lint.yml` | ⚠️ YAML valid | **沒在真 PR 上觸發過**——不知道 Linux runner 上 grep 行為跟本地 git-bash 是否一致 | +| `.github/workflows/docs.yml` | ⚠️ YAML valid · 本機 mkdocs build 綠 | 統一 Pages workflow（mkdocs `/` + mdBook `/book/`，取代已刪除的 deploy-book.yml）。mdBook 子路徑 base-url 尚未在 CI 端到端驗證（首次 deploy 後需實測 `/book/` 資產） | +| `walkthroughs/build-first-agent-in-7-steps.md` 的 Python 範例（~350 行）| ⚠️ 結構合理 | **完全沒實際跑過**——根據對 Anthropic SDK / LangGraph / Chroma / promptfoo 的理解寫出來，但沒從頭到尾 execute 一次。可能有：API 介面變動、套件版本相依、import path 微差 | +| `book.toml` mdBook 設定 | ⚠️ TOML valid | 沒實際 build 過完整 site | + +--- + +## ❌ 完全沒測（design / template，等實際使用才會發現問題） + +| 項目 | 為什麼沒測 | +|---|---| +| `scripts/build-pdf.sh` 跑出來的 PDF 視覺品質 | 需要 pandoc + xelatex + Noto Sans CJK 全套，本地環境沒裝 | +| GitHub Pages 上的 mdBook hosted 版 | repo Settings 還沒切到 GitHub Actions source（user 手動步驟） | +| 第一個 PDF release | 需要先跑 build-pdf.sh，沒做 | +| `.github/launch-checklist.md` 內所有「啟用 Discussions / 提交到 awesome lists / 寫 launch posts」項目 | 全部還沒做 | + +--- + +## 對社群貢獻者的建議 + +如果你是第一個真的要跑某個 build / workflow 的人： + +1. **跑 `bash scripts/build-pdf.sh` 之前**：先按 `scripts/README.md` 把字型裝好；輸出 PDF 之後實際開來看 CJK / mermaid block 有沒有正常 +2. **跑 `bash scripts/build-mdbook.sh` 之前**：先 `cargo install mdbook mdbook-mermaid` 並在 repo root 跑 `mdbook-mermaid install .`；推上去前先本地 `--serve` 看一下 +3. **試 walkthrough 的 Python**：建議用一個全新環境（venv），照 Stage 0 的一次性 install 跑完，遇到任何 import / API 不符的，**請開 issue + PR**——因為這就是「第一手實測」價值最高的時刻 +4. **觸發 CI lint workflow**：開個 throwaway PR 改 `stages/01-llm-basics.md`，故意加 `教程` 這個禁用詞，看 banned-words job 有沒有正常 fail。如果沒抓到，調整 grep 邏輯 +5. **Deploy book 第一次**：repo Settings → Pages → Source: GitHub Actions，然後 push 一次 commit 讓 workflow 跑。看 Actions tab 看結果 + +--- + +## 為什麼 maintainer 沒全部 test + +老實說： + +- **Build 工具鏈成本**：pandoc + xelatex + Noto Sans CJK 一套裝下來要 1-2 GB + 1-2 小時。不在 launch-blocking 路徑上時不值得本地裝 +- **AI walkthrough 的 LangGraph / Chroma 等套件**：版本日新月異；今天測完明天可能就過期。所以選擇用「**對著官方 API 文件寫，註明可能要對現在版本調整**」的策略 +- **CI workflow**：在真 PR 上才會觸發；沒第一個外部 PR 之前是 false-positive 還是 fully working 都看不出來 + +這份 repo 是 **「ship-able skeleton」**——所有結構都對、所有 metadata 都驗證過、所有 prose 都過 review，但**第一次實際跑 build / deploy / walkthrough 還是會發現坑**。 + +第一個踩到坑的人請開 issue + PR——這正是社群協作的價值所在。 + +--- + +## 修這份 testing status + +每次跑過某個項目後，把上面表格的 ⚠️ 改成 ✅ 並補「證據」欄。 +真實「跑過 + 有 observable output」才算 ✅，「我覺得 OK」不算。 diff --git a/.github/channel-partners.md b/.github/channel-partners.md new file mode 100644 index 0000000..36a3746 --- /dev/null +++ b/.github/channel-partners.md @@ -0,0 +1,135 @@ +# Channel Partners — Outreach Tracking + +> Single source of truth for **awesome-agentic-ai-zh** channel-partner outreach. +> Per-target pitch templates live in `.github/outreach/.md`. +> Maintainer: @WenyuChiou (个人 maintainer; rule: 1-2 sends/day max). + +--- + +## Status enum + +| Status | Meaning | +|---|---| +| `not contacted` | Pitch drafted in `outreach/.md`, nothing sent yet | +| `contacted` | Outbound sent (issue/PR/email) — awaiting response | +| `replied-positive` | Partner replied; discussion in progress; no commit yet | +| `replied-negative` | Partner declined or asked to redirect | +| `merged-or-listed` | Cross-link landed (PR merged / featured / listed) | +| `ghosted` | No reply in ≥ 2 weeks; one ping sent then dropped | +| `cooldown` | Don't contact for ≥ 30 days (over-asked, restructuring, etc.) | + +## Outreach matrix + +| # | Target | Channel | Status | Date contacted | Outcome | Date confirmed | Notes | +|---|---|---|---|---|---|---|---| +| 1 | [Datawhale](outreach/datawhale.md) | GitHub issue | not contacted | — | — | — | Already cite Hello-Agents Extra05/08 in our cookbook | +| 2 | [liyupi/ai-guide](outreach/liyupi.md) | GitHub PR | not contacted | — | — | — | ★13k mainland resource hub | +| 3 | [HuggingFace 中文社群](outreach/huggingface-zh.md) | HF community/discuss | not contacted | — | — | — | English ecosystem hub w/ growing zh segment | +| 4 | [LangChain (kyrolabs/awesome-langchain)](outreach/langchain-ai.md) | GitHub PR | not contacted | — | — | — | Stage 4 covers LangChain; §11 lists Langchain-Chatchat | +| 5 | [hesreallyhim/awesome-claude-code](outreach/awesome-claude-code.md) | GitHub **issue** | not contacted | — | — | — | ⚠️ Reorg STILL incomplete (verified 2026-05-21: README TOC is a placeholder; resources now live in `THE_RESOURCES_TABLE.csv` + a submission template). Keep parked — do not spend a send here yet | +| 6 | [punkpeye/awesome-mcp-servers](outreach/awesome-mcp-servers.md) | GitHub PR | contacted | 2026-05-09 | — | — | [PR #6135](https://github.com/punkpeye/awesome-mcp-servers/pull/6135). 2026-05-10: addressed bot name-check ([6f711ec](https://github.com/WenyuChiou/awesome-mcp-servers/commit/6f711ec3)) + replied to glama/emoji bot warnings ([comment](https://github.com/punkpeye/awesome-mcp-servers/pull/6135#issuecomment-4416517075)). Awaiting punkpeye human review. | +| 7 | [Zhipu BigModel community](outreach/zhipu.md) | dev community / 知乎 | not contacted | — | — | — | Inviting them to PR a Zhipu agent entry to §11 | +| 8 | [Moonshot Kimi](outreach/moonshot.md) | dev community / 知乎 | not contacted | — | — | — | Inviting them to PR a Kimi agent entry to §11 | +| 9 | [travisvn/awesome-claude-skills](https://github.com/travisvn/awesome-claude-skills) | GitHub PR | contacted | 2026-05-21 | — | — | [PR #754](https://github.com/travisvn/awesome-claude-skills/pull/754) opened 2026-05-21 — entry in `### Written Tutorials`, framed around Stage 5 (Claude Code ecosystem). Fit is moderate (Claude-Skills-specific list); awaiting maintainer review | +| 10 | [WangRongsheng/awesome-LLM-resources](https://github.com/WangRongsheng/awesome-LLM-resources) | GitHub PR | contacted | 2026-05-21 | — | — | [PR #121](https://github.com/WangRongsheng/awesome-LLM-resources/pull/121) opened 2026-05-21 — entry added to `## 课程 Course` (beside mlabonne/llm-course). Maintainer active; awaiting review | +| 11 | [AiHubCN/Awesome-Chinese-LLM](https://github.com/AiHubCN/Awesome-Chinese-LLM) | GitHub PR | not contacted | — | — | — | ★22k, pushed today, no license (yellow flag). Long TOC — need to browse README before deciding section. Lower priority due to license uncertainty | + +## Sequencing rule + +**Pace: 1-2 outbound sends per day.** Reasoning: + +- Replies need to be handled. If we batch-send all 8 in one day, we can't respond + to early-positive replies before they cool. +- Multiple open conversations dilute attention; one-at-a-time keeps quality. +- If 5 replies land in week 1, that's a good problem; if 0 land, we don't burn + all our cards before learning what's not working. + +Suggested first-week order (low-risk → high-risk, **revised 2026-05-09** +after upstream-target audit caught the awesome-claude-code reorg): + +1. **Day 1**: [#6 punkpeye/awesome-mcp-servers PR](outreach/awesome-mcp-servers.md) + — has `## Tutorials` section, ★86k repo, reciprocal cite already exists. + Lowest-risk concrete-action target. +2. **Day 2**: [#5 awesome-claude-code **issue**](outreach/awesome-claude-code.md) + — repo mid-reorg, no PR-able sections; open an issue parking the proposal + for when their new TOC lands. +3. **Day 3**: [#1 Datawhale](outreach/datawhale.md) — most strategic for zh-Hans + reach (we cite Hello-Agents Extra05/08). +4. **Day 4**: [#2 liyupi](outreach/liyupi.md) — high reach if accepted (★13k + resource hub). +5. **Day 5**: [#4 LangChain (kyrolabs/awesome-langchain)](outreach/langchain-ai.md). +6. **Day 6**: pause — review responses to date. +7. **Day 7+**: [#3 HuggingFace](outreach/huggingface-zh.md), then + [#7 Zhipu](outreach/zhipu.md), [#8 Moonshot](outreach/moonshot.md) only + after digesting earlier feedback. +8. **Day 8+ (added 2026-05-10 retroactively)**: targets 9-11 (`travisvn/awesome-claude-skills`, + `WangRongsheng/awesome-LLM-resources`, `AiHubCN/Awesome-Chinese-LLM`) — discovered they were + on `.github/launch-checklist.md` from day 1 but missing from this outreach matrix. Pitch + files not yet drafted; use awesome-mcp-servers template as base when ready. Prioritize + travisvn (cleanest fit, explicit Tutorials section). + +## Update protocol + +- Always update this matrix when contacted / received reply / closed. +- Use `git commit -m "outreach: status update for ()"` so the + log is greppable. +- Dates: ISO format `YYYY-MM-DD`. +- Notes: 1-2 lines max — full context lives in the per-target `outreach/.md`. + +## What NOT to do + +- ❌ Bulk-send same template to all 8 in one day — looks like spam +- ❌ Lead with star count (★525) — small to ★1k+ partners; lead with scope +- ❌ Promise things we won't ship (e.g., "we'll add X if you cross-link") +- ❌ Ping after one reply — give 5+ business days +- ❌ Pitch via Discord DM unless explicitly invited (follow each project's + preferred contact channel; Discord DM cold = annoying) +- ❌ Edit pitch templates without recording the change in the file's git history + +## Success indicators + +Order by signal strength (top = stronger): + +1. **Cross-link landed** in their canonical README / docs / awesome-list +2. **Public mention** (their tweet / post / blog cites us) +3. **Reciprocal listing** in their tutorials/learning section +4. **Soft acknowledgment** — they replied positively but no concrete action + +If by **2026-06-01** no signal #1-3 has landed across all 8: pause outreach, +audit the pitch tone (likely too founder-y, not enough technical specifics). + + +--- + +## English-audience launch (added 2026-05-17) + +Rationale: 14-day traffic is ~100% Chinese-social (Threads #1 external +referrer); zero English dev-channel inbound (no HN / Reddit / lobste.rs +/ newsletter). English content is 99.6% native (0.4% CJK measured) but +was under-promoted. Pre-req DONE this round: README positioning reframed +(trilingual / English fully maintained) + GitHub description/topics +de-zh-gated (commit b4bb862). Drafts ready — maintainer posts manually. + +| # | Target | Channel | Status | Draft | Notes | +|---|---|---|---|---|---| +| E1 | Hacker News | Show HN (one shot) | not contacted | [hacker-news.md](outreach/hacker-news.md) | Highest single-spike; weekday AM US-Eastern; author first-comment pre-empts "why a list"/"MT slop" | +| E2 | r/AI_Agents | Reddit self-post | not contacted | [reddit.md](outreach/reddit.md) | Primary sub; exact audience | +| E3 | r/LocalLLaMA | Reddit self-post | not contacted | [reddit.md](outreach/reddit.md) | Local-LLM-angle variant | +| E4 | r/ClaudeAI | Reddit self-post | not contacted | [reddit.md](outreach/reddit.md) | Ecosystem-depth variant | +| E5 | r/learnmachinelearning | Reddit self-post | not contacted | [reddit.md](outreach/reddit.md) | Resource-framing variant | +| E6 | TLDR AI / Ben's Bites / Latent Space / LWiAI | Newsletter tip | not contacted | [newsletters-en.md](outreach/newsletters-en.md) | Submit AFTER a HN/Reddit signal to reference | +| E7 | kyrolabs/awesome-agents · Shubhamsaboo/awesome-llm-apps | GitHub PR | not contacted | [awesome-lists-en.md](outreach/awesome-lists-en.md) | Passive; PR desc MUST state "trilingual/EN-maintained" or `-zh` gets mis-filed. (e2b-dev/awesome-ai-agents dropped 2026-05-21 — ~15-month-stale, see awesome-lists-en.md Don'ts) | + +### English sequencing rule + +1. **First**: README positioning + GitHub metadata (✅ done b4bb862) — + without it every English referral bounces on "this is Chinese-only". +2. Then **one** of E1/E2 (HN or r/AI_Agents) — not both same day; gauge response. +3. E3–E5 spaced 1 sub/day, each tailored (identical body = spam-flag). +4. E6 newsletters only AFTER an E1–E5 signal exists ("already trending"). +5. E7 awesome-list PRs anytime (passive, low-risk), excluding the two + already tracked above (punkpeye #6, travisvn #9). + +Same "What NOT to do" rules as the zh matrix apply: no star-count lead, +no overclaim, no upvote/star asks, reply to comments for ~24h, never +mass-paste identical text. diff --git a/.github/launch-checklist.md b/.github/launch-checklist.md new file mode 100644 index 0000000..e787fc3 --- /dev/null +++ b/.github/launch-checklist.md @@ -0,0 +1,84 @@ +# Launch checklist + +> 這份是 maintainer 內部用的——repo 從 Phase 5 ship 到開始**主動推廣**前要走完的一次性步驟。 + +--- + +## ✅ Pre-launch（已完成） + +- [x] Phase 1-5 內容 ship（134 entries、style guide、walkthrough、Mermaid 圖） +- [x] `.github/` issue + PR template +- [x] `resources/style-guide.md` zh + en +- [x] `scripts/check-links.py` + `refresh-stars.py` 可跑 +- [x] CI lint workflow（zh-Hans slip + overclaim 自動檢查；每月跑 link-rot + star-drift） + +## 🟡 Pre-launch（一次性手動 setup） + +- [ ] **GitHub Pages**：repo Settings → Pages → Source: **GitHub Actions** + - 啟用後，`.github/workflows/docs.yml` 推 main 會自動 build mkdocs（`/` 首頁）+ mdBook（`/book/`）並 deploy 到 `https://wenyuchiou.github.io/awesome-agentic-ai-zh/`（單一 workflow 擁有 Pages） +- [ ] **GitHub Discussions**：repo Settings → Features → enable Discussions + - Categories 建議： + - 🙋 Q&A — 學習問題 + - 💡 Project nominations — 推薦新 project（先討論再 PR） + - 📚 Stage discussion — 每個 stage 一個 thread + - 🎯 Show & tell — 走完 stage 的人 share 自己的成果 +- [ ] **第一次 PDF release**：本地跑 `bash scripts/build-pdf.sh`，把 `dist/awesome-agentic-ai-zh.pdf` 上傳到 GitHub Release v1.0 +- [ ] **GitHub Releases**：以 `phase-5` tag 為起點建第一個 release，附 PDF + +## 🟢 Soft launch（小範圍宣傳） + +- [ ] 跟 `WenyuChiou` 朋友圈分享（內測） +- [ ] 修 1-2 輪內測回饋（issue 處理） + +## 🚀 Public launch（推廣） + +### 提交到中文社群 awesome list + +- [ ] [`AiHubCN/Awesome-Chinese-LLM`](https://github.com/AiHubCN/Awesome-Chinese-LLM) — 開 PR 加進 catalog（教學資源 / 學習路線 section） +- [ ] [`WangRongsheng/awesome-LLM-resources`](https://github.com/WangRongsheng/awesome-LLM-resources) — 開 PR +- [ ] [`hesreallyhim/awesome-claude-code`](https://github.com/hesreallyhim/awesome-claude-code) — 開 PR（learning resources） +- [ ] [`travisvn/awesome-claude-skills`](https://github.com/travisvn/awesome-claude-skills) — 開 PR（learning resources） + +> 📡 **Channel partner outreach**：完整 outreach 計畫（8 個目標 × 3 種 pitch 變體 + 1-2 sends/day pacing）跟追蹤 matrix 在 [`.github/channel-partners.md`](channel-partners.md)。各 target 的 pitch 草稿在 [`.github/outreach/.md`](outreach/)。 + +### 寫 launch 文章 + +- [ ] **Threads 短版**（2-3 則）— 重點：134 個 project、跨 stage walkthrough、誠實時程 +- [ ] **dev.to 長版**（一篇 1000-1500 字）— 寫「為什麼平鋪 awesome 不夠用、結構化路線怎麼做」 +- [ ] （選）**個人部落格** — 同樣內容深度版 + +### 中文 LLM 社群 + +- [ ] Datawhale 微信社群（如果有 zh-TW 受眾） +- [ ] Hacker News（zh-TW 故事性夠強的話） +- [ ] r/LocalLLaMA、r/MachineLearning（看 reddit 接受度） + +## 🔁 Post-launch（持續） + +這份是**參考節奏，不是 SLA**——能做就做、忙起來放著也沒關係。社群開放型 repo 不需要強制定期維護。 + +- 有空時：review issue / 合併 PR +- 偶爾跑：CI 已設定每月自動跑 link rot + star drift（被動的、不用人工） +- 想做的時候：加幾個新 entry、清掉幾個 archived repo +- 不必排定期程：phase milestone、新增 branch 等大改——有 traction 訊號再做 + +--- + +## 📊 成功指標（不是目標、是訊號） + +排序大致按 fingerprint 強度——前面比後面更可靠。 + +1. **每月新 issue / PR 數量**（活躍社群訊號） +2. **stage maintainer 自薦數**（深度 engagement） +3. **被引用 / 被收錄到其他 awesome list 數** +4. **Star 數**（弱訊號，容易被刷；只看趨勢、不看絕對值） + +--- + +## 不該做的（deliberate "no"s） + +- ❌ 為了上熱門就刷 SEO 關鍵字 +- ❌ 為了 star 數加 low-quality entry +- ❌ 把 PR 合進 main 而不 review（即使是 typo 也要 review） +- ❌ 自家 repo 重新加回 catalog（先前刻意移除） +- ❌ 接 sponsor / affiliate link（會影響推薦獨立性） diff --git a/.github/outreach/_send-day-packages.md b/.github/outreach/_send-day-packages.md new file mode 100644 index 0000000..b61123c --- /dev/null +++ b/.github/outreach/_send-day-packages.md @@ -0,0 +1,237 @@ +# Send-Day Packages (ready to paste) + +> **What this file is**: the copy-paste operational source for outreach submissions. Open it on a send-day, pick ONE target, refresh the stats line, paste, submit. The per-target `.md` files in this folder hold the *positioning rationale* (why each target, pitch angle); **this** file holds the *exact content to send*. +> +> **Canonical numbers as of 2026-06-08** (baked into every package below): +> - Trilingual: **繁中 (canonical) / English / 简中** — all three hand-maintained, not machine-translated +> - **8 stages** (Stage 0 → Stage 8; Stage 5 + Stage 8 are shared hubs) · 2 tracks · 5 extension paths +> - **240+ curated resources** · MIT · CI lints links + anchors + banned-words on every PR +> - Repo: https://github.com/WenyuChiou/awesome-agentic-ai-zh · Docs: https://wenyuchiou.github.io/awesome-agentic-ai-zh/ +> - ★ ≈ **1.9k** — **refresh on the day you send** (`gh repo view WenyuChiou/awesome-agentic-ai-zh --json stargazerCount`). Stale stars in a PR read as careless. +> +> **Cadence (do NOT blast)**: one target per send-day, 1–2 sends/day max. Wait for a reply or ~1 week before the next. All submissions are done by you (maintainer identity); I prepare content only. + +--- + +## Status board + +| # | Target | ★ | Channel | Section | Lang | Fit | Status | +|---|---|---|---|---|---|---|---| +| A | Hannibal046/Awesome-LLM | 18k+ | PR | LLM Tutorials and Courses | en | good | ready (last batch) | +| B | HqWu-HITCS/Awesome-Chinese-LLM | 20k+ | PR | ### 7. LLM教程 | zh-Hans | good | ready (last batch) | +| C | kyrolabs/awesome-langchain | 9k+ | PR | Learn → Notebooks | en | good | ready | +| D | liyupi/ai-guide | 14k+ | PR | AI 学习路线 / 相关资源 | zh-Hans | good | ready | +| E | datawhalechina/hello-agents | 55k+ | **Issue** | (cross-link, not catalog) | zh-Hans | good | ready | +| F | Jenqyang/Awesome-AI-Agents | — | PR | Related | en | medium | ready (looser fit — see note) | +| — | AiHubCN/Awesome-Chinese-LLM | — | — | — | — | — | **SKIP** (fork of B) | + +**Nudges pending** (your existing open PRs — you run these): #121 WangRongsheng, #754 travisvn — see bottom. + +--- + +## A — Hannibal046/Awesome-LLM (PR) + +**Section**: `## LLM Tutorials and Courses` (or the closest "Other Awesome Lists" subsection if the maintainer prefers). +**Entry line**: + +```markdown +* [awesome-agentic-ai-zh](https://github.com/WenyuChiou/awesome-agentic-ai-zh) - Trilingual (zh-TW / English / zh-Hans) learning roadmap for agentic AI, from LLM basics to multi-agent systems, with 240+ curated resources and runnable examples. +``` + +**PR title**: `Add awesome-agentic-ai-zh (trilingual agentic-AI learning roadmap)` + +**PR body**: + +```markdown +Hi Hannibal046, + +Adding awesome-agentic-ai-zh to LLM Tutorials and Courses (move it if another spot fits better). + +It's a trilingual learning roadmap for agentic AI — Traditional Chinese (canonical), English, and Simplified Chinese, all three hand-maintained. The path runs from Stage 0 (what an LLM is, how tokens work) to Stage 8 (multi-agent orchestration, Computer Use / Browser Use / sandboxes), with 240+ curated resources and small runnable examples. MIT licensed; CI checks links and anchors on every PR. + +Thanks for maintaining Awesome-LLM. + +— Wenyu Chiou (individual maintainer) +``` + +--- + +## B — HqWu-HITCS/Awesome-Chinese-LLM (PR) + +**Section**: `### 7. LLM教程` (nested format — match exactly, this repo uses 项目名称 / 地址 / 简介). +**Entry block**: + +```markdown +* awesome-agentic-ai-zh: + * 地址:[https://github.com/WenyuChiou/awesome-agentic-ai-zh](https://github.com/WenyuChiou/awesome-agentic-ai-zh) + * 简介:三语(繁中 / English / 简中)agentic AI 学习地图,从 LLM 基础到多代理系统,8 个阶段 + 2 条学习路线 + 240+ curated 资源,附可跑的范例。MIT。 +``` + +**PR title**: `添加 awesome-agentic-ai-zh(三语 agentic AI 学习地图)到 7. LLM教程` + +**PR body**: + +```markdown +你好, + +想把 awesome-agentic-ai-zh 加到「7. LLM教程」。这是一份 agentic AI 的三语学习地图(繁中 canonical / 简中 / English,三语手工维护),8 个阶段从 LLM 基础排到多代理编排,每阶段标了预估时程、入门条件、该读什么,目前 240+ curated 资源,MIT 协议。 + +已按本项目格式提供链接与简介。觉得不合适直接关掉就好,谢谢维护这份清单。 + +— Wenyu(个人 maintainer) +``` + +--- + +## C — kyrolabs/awesome-langchain (PR) + +**Section**: `## Learn → ### Notebooks` — place **right after** the existing `liaokongVFX/LangChain-Chinese-Getting-Started-Guide` line (keeps the two zh learning resources adjacent). There is **no** "Tutorials" section; do not create one. + +**Diff**: + +```diff + - [LangChain Chinese Getting Started Guide](https://github.com/liaokongVFX/LangChain-Chinese-Getting-Started-Guide): Chinese LangChain Tutorial for Beginners ![GitHub Repo stars](https://img.shields.io/github/stars/liaokongVFX/LangChain-Chinese-Getting-Started-Guide?style=social) ++ - [WenyuChiou/awesome-agentic-ai-zh](https://github.com/WenyuChiou/awesome-agentic-ai-zh): Trilingual (zh-TW / zh-Hans / en) 8-stage learning roadmap for agentic AI. Stage 4 walks through LangChain, LangGraph, AutoGen, CrewAI, and Smolagents with prerequisites, time estimates, and hands-on exercises ![GitHub Repo stars](https://img.shields.io/github/stars/WenyuChiou/awesome-agentic-ai-zh?style=social) +``` + +**PR title**: `Add awesome-agentic-ai-zh (trilingual learning roadmap) to Learn → Notebooks` + +**PR body**: + +```markdown +Hi kyrolabs maintainers, + +Proposing [WenyuChiou/awesome-agentic-ai-zh](https://github.com/WenyuChiou/awesome-agentic-ai-zh) for **Learn → Notebooks**, next to the existing `liaokongVFX/LangChain-Chinese-Getting-Started-Guide` entry (same zh-learning surface). + +Why it fits: +- Trilingual (zh-TW canonical · zh-Hans · en — all three hand-maintained, not MT), which fills a gap for non-English learners. +- Stage 4 (Agent Frameworks) walks new developers through LangChain / LangGraph / AutoGen / CrewAI / Smolagents with prerequisites, time estimates, and hands-on exercises. +- The §11 catalog has 7 Chinese-ecosystem entries including `chatchat-space/Langchain-Chatchat` and the LangChain Chinese Getting Started Guide already in your list. + +Stats (refresh on send-day): ★1.9k, MIT licensed, rendered docs at https://wenyuchiou.github.io/awesome-agentic-ai-zh/. CI runs banned-word, link-rot, and anchor-integrity lints on every PR. + +If a different section works better, happy to redirect. Thanks for maintaining awesome-langchain. + +— Wenyu Chiou (individual maintainer) +``` + +> **Send-day stat refresh**: `gh repo view WenyuChiou/awesome-agentic-ai-zh --json stargazerCount,forkCount` + `gh api repos/WenyuChiou/awesome-agentic-ai-zh/traffic/views` if you want fresh visitor numbers. + +--- + +## D — liyupi/ai-guide (PR) + +**Section**: 「AI 学习路线」或「相关资源」(放哪边由 liyupi 决定). liyupi 偏好简体 + 大陆友善措辞,PR 用 zh-Hans。 + +**Entry line**: + +```markdown +- [WenyuChiou/awesome-agentic-ai-zh](https://github.com/WenyuChiou/awesome-agentic-ai-zh) — agentic AI 的三语(繁中 / English / 简中)8 阶段学习地图,从 Stage 0 的 LLM 基础走到 Stage 8 的多代理编排,240+ curated 资源 + 可跑的范例,MIT。和 ai-guide 互补:ai-guide 找 project、这份找学习顺序。 +``` + +**PR title**: `添加 awesome-agentic-ai-zh(三语 agentic AI 学习地图)到「AI 学习路线」` + +**PR body**: + +```markdown +你好 liyupi, + +想把 [awesome-agentic-ai-zh](https://github.com/WenyuChiou/awesome-agentic-ai-zh) 加到「AI 学习路线」或「相关资源」section(放哪边你决定)。 + +这份是 agentic AI 的三语学习地图(繁中 canonical / 简中 / English,三语手工维护,不是机翻),8 个阶段从 Stage 0 的 LLM 基础排到 Stage 8 的多代理编排,每阶段标了预估时程、入门条件、该读什么,目前 240+ curated 资源,MIT 协议。 + +定位上和 ai-guide 互补,不是取代:ai-guide 是资源大全,这份补的是「该按什么顺序学」。常见读者是想学但不知道先学哪个的工程师,走完阶段后回 ai-guide 找具体 project 用。 + +觉得不合适直接关掉就好,谢谢你做的 ai-guide。 + +— Wenyu(个人 maintainer) +``` + +--- + +## E — datawhalechina/hello-agents (GitHub **Issue**, not a PR) + +> This one is a **cross-link suggestion Issue** on the hello-agents repo, not a catalog entry. We already link Hello-Agents on our side (no strings); the issue just opens a reciprocal-link conversation. + +**Repo**: https://github.com/datawhalechina/hello-agents → Issues → New issue +**Issue title**: `Cross-link 建议:一份会把读者导向 Hello-Agents 的结构化学习路线` + +**Issue body**: + +```markdown +你好 Datawhale 团队, + +我在维护 [awesome-agentic-ai-zh](https://github.com/WenyuChiou/awesome-agentic-ai-zh) —— 一份 agentic AI 的三语(繁中 canonical / 简中 / English)8 阶段学习地图,240+ curated 资源,MIT。 + +我们这边已经把 Hello-Agents 放进了「走完前面阶段后的延伸阅读」(无条件,已经 ship)。读者主要是走完 Stage 4 之后想进 framework 跟 multi-agent 的人,Hello-Agents 正好是下一阶段最强的中文教材。 + +想问有没有可能做个双向 cross-link: +1. 我们已经 link 你们了。 +2. 如果你们觉得合适,能不能在 Hello-Agents 的 README 或 docs 里提一句「想看更完整的学习路线可以参考 awesome-agentic-ai-zh」? +3. 或是 reverse PR:我们在中文圈那一节加 Hello-Agents 的正式 entry,你们 review。 + +不合适也完全 OK。谢谢你们把 Hello-Agents 做出来,这几年中文 agentic AI 学习的公共财都是你们扛的。 + +— Wenyu(PhD candidate · Lehigh,个人 maintainer) +``` + +> Note: WeChat 是 Datawhale 主要互动 channel,但 GitHub issue 比较可追踪。一周没回就放着,他们团队很忙、不要 ping。 + +--- + +## F — Jenqyang/Awesome-AI-Agents (PR) + +> **Fit note (read before sending)**: this list has no Tutorials / Learning section. The closest home is `## Related`, but its visible subsection is "Paper-List Repo", which isn't a clean match for a learning roadmap. Send only if you're OK with the looser fit; the entry below leads with "happy to move it" so the maintainer can reslot. Check `CONTRIBUTING.md` in the repo first. + +**Section**: `## Related` (maintainer to confirm exact subsection). +**Entry line** (matches their `* [Name](URL) - desc ![stars]` format): + +```markdown +* [awesome-agentic-ai-zh](https://github.com/WenyuChiou/awesome-agentic-ai-zh) - Trilingual (Traditional Chinese / English / Simplified Chinese) 8-stage learning roadmap for agentic AI, from LLM basics to multi-agent systems, with 240+ curated resources and runnable examples. [![GitHub Repo stars](https://img.shields.io/github/stars/WenyuChiou/awesome-agentic-ai-zh?style=social)](https://github.com/WenyuChiou/awesome-agentic-ai-zh) +``` + +**PR title**: `Add awesome-agentic-ai-zh (trilingual agentic-AI learning roadmap) to Related` + +**PR body**: + +```markdown +Hi Jenqyang, + +Proposing awesome-agentic-ai-zh for the Related section — happy to move it wherever fits best, I wasn't sure which subsection is right. + +It's a trilingual learning roadmap for agentic AI: Traditional Chinese (canonical), English, and Simplified Chinese, all three hand-maintained rather than machine-translated. The structure runs from Stage 0 (what an LLM is, how tokens work) up to Stage 8 (multi-agent orchestration, Computer Use / Browser Use / sandboxes), with 240+ curated resources and small runnable examples. + +The gap it fills: most agent lists, including yours, are catalogs you reach for once you know what you want. This is the "where do I start, and in what order" layer for people who don't yet. MIT licensed; CI checks links and anchors on every PR. + +If it's not a fit, no problem at all. Thanks for maintaining the list. + +— Wenyu Chiou (individual maintainer) +``` + +--- + +## SKIP — AiHubCN/Awesome-Chinese-LLM + +**Do not submit.** It shares the repo name, the section structure (`7. LLM教程`), and the exact nested entry format with **HqWu-HITCS/Awesome-Chinese-LLM** (target B above). That's a fork/mirror signature. Submitting the same entry to both reads as duplicate/spam and dilutes the one that matters. Send B (the canonical HqWu-HITCS) only. If you ever confirm AiHubCN is genuinely independent **and** actively merges external PRs, reuse the B package verbatim — the format is identical. + +--- + +## Nudges (your existing open PRs — run on a send-day, ~1 week apart) + +Both are your own PRs, so these are yours to run. Polite single ping; if still no reply after another week, leave them. + +```bash +# PR #121 — WangRongsheng/awesome-LLM-resources (zh-Hans) +gh pr comment 121 --repo WangRongsheng/awesome-LLM-resources \ + --body "Hi,这个 PR 开了一段时间了,不知道有没有机会 review 一下?如果格式或 section 需要调整我随时改。谢谢!" + +# PR #754 — travisvn/awesome-claude-skills (en) +gh pr comment 754 --repo travisvn/awesome-claude-skills \ + --body "Gentle ping on this one — happy to adjust the entry or move it if a different section fits better. Thanks for maintaining the list!" +``` + +--- + +## After you submit + +Tell me which target + the PR/issue URL, and I'll update the matrix in `channel-partners.md` (status: submitted → date + link). Keep the one-at-a-time cadence so none of these read as a coordinated blast. diff --git a/.github/outreach/awesome-claude-code.md b/.github/outreach/awesome-claude-code.md new file mode 100644 index 0000000..191da62 --- /dev/null +++ b/.github/outreach/awesome-claude-code.md @@ -0,0 +1,100 @@ +# Outreach: hesreallyhim/awesome-claude-code + +> **Status**: not contacted · **Channel**: GitHub **issue** (NOT PR — see note below) +> **Primary lang**: en +> **Last updated**: 2026-05-09 +> **Repo**: https://github.com/hesreallyhim/awesome-claude-code (★ 47k+) + +> ⚠️ **REPO IS MID-REORG (verified 2026-05-09)** — their current README says +> "The previous Table of Contents was no longer fit for purpose, so a new +> organizational system is being prepared." There are no live sections to +> PR an entry into. **Don't open a PR.** Open an **issue** instead, parking +> the proposal for when the new TOC lands. + +**Why this target**: We already cite their list in our README's "Related projects" section (and in zh-Hans / zh-TW counterparts). Reciprocal listing is natural. They're the canonical Claude Code awesome-list with ★43k. + +**Pitch angle**: We're a **structured learning roadmap** complement to their flat catalog. Stage 5 of our roadmap is dedicated to the Claude Code ecosystem (MCP, Skills, Plugins). + +**Their counter-value**: Reciprocal cross-link; their readers get a learning order; we get exposure to Claude Code power users. + +--- + +## Variant 1 — Social post (X, ~280 chars) + +``` +For folks browsing @hesreallyhim's awesome-claude-code list — just published +an 8-stage trilingual learning roadmap that walks Stage 0 (foundations) → Stage 5 +(Claude Code: MCP / Skills / Plugins) → Stage 8 (production). + +★525 week 1 · 240+ curated projects · zh-TW / zh-Hans / en +🔗 github.com/WenyuChiou/awesome-agentic-ai-zh +``` + +## Variant 2 — GitHub **issue** (200-300 words) [USE THIS, NOT A PR] + +**Issue title**: Proposal: add a "Learning Resources" / "Tutorials" section in the new TOC — would awesome-agentic-ai-zh fit? + +**Issue body**: + +```markdown +Hi @hesreallyhim, + +Saw the reorg notice on the README — congrats on the cleanup, the previous +TOC had drifted hard. Holding off on opening a PR until the new TOC lands. + +Wanted to surface a proposal for when you're ready: + +I maintain [awesome-agentic-ai-zh](https://github.com/WenyuChiou/awesome-agentic-ai-zh) +— a trilingual (zh-TW canonical · zh-Hans · English) 8-stage learning roadmap +for agentic AI. **Stage 5 is dedicated entirely to the Claude Code ecosystem** +(MCP, Skills, Plugins, Hello-World walkthroughs, 65+ entry integration catalog +by use case). + +awesome-claude-code is already in our `Related projects` section +([our README](https://github.com/WenyuChiou/awesome-agentic-ai-zh/blob/main/README.md)) +— we cite you as the go-to flat catalog. We'd be the structured learning +complement. + +Two questions for the new TOC design: +1. Will the new TOC have a "Learning Resources" / "Tutorials" / "Curricula" + section? If yes, a one-line entry there would close the reciprocal loop. +2. If the new TOC keeps the list strictly tooling-only, totally understand + — happy to drop this. No follow-up needed in that case. + +No urgency. Reply when the new TOC is done. Thanks for maintaining this list +— it's a public good for the Claude Code community. + +Stats for credibility (week 1): 6,869 views / 3,185 unique visitors / 1,099 +clones / 408 unique cloners / 50 forks / 3 community contributors. MIT, +trilingual translation discipline, CI lint on every PR. + +— Wenyu (PhD candidate, individual maintainer) +``` + +## Variant 3 — DM / Twitter reply (150 words) + +``` +Hey @hesreallyhim — your awesome-claude-code list is already in our README's +"Related projects". Built a complement: a trilingual (zh-TW / zh-Hans / en) +8-stage learning roadmap, with Stage 5 dedicated to Claude Code (MCP, Skills, +Plugins, walkthroughs, 65+ integrations grouped by use case). + +★525 in week 1, MIT licensed. If a reciprocal link in awesome-claude-code's +Learning Resources section makes sense, just opened a PR +(github.com/hesreallyhim/awesome-claude-code/pull/). No worries if not. + +— Wenyu +``` + +--- + +## Notes + +- **Repo status as of 2026-05-09**: README is a placeholder ("Update in + progress"; TOC marked "I. TODO"). Maintainer @hesreallyhim is rebuilding + the organizational system. **OPEN AN ISSUE, NOT A PR.** +- The issue is a parking proposal — re-engage when the new TOC ships +- @hesreallyhim historically responsive (~3 days); during reorg, expect slower +- If they reply "the new TOC won't have a learning resources section" — thank + them, drop it, move on. Don't push. +- After they reply (positive or negative), update `.github/channel-partners.md` diff --git a/.github/outreach/awesome-lists-en.md b/.github/outreach/awesome-lists-en.md new file mode 100644 index 0000000..35a8a69 --- /dev/null +++ b/.github/outreach/awesome-lists-en.md @@ -0,0 +1,60 @@ +# Outreach draft — English awesome-lists (get listed) + +> **Status**: draft, not submitted. Passive reach (be listed in lists +> English builders already browse). Each is a PR to their list's +> learning/tutorials section. The `-zh` repo name reads as "Chinese-only" +> on these lists — the PR description must say "trilingual, English fully +> maintained" up front or it gets filed under a zh-only section. + +## Targets + +| Target | Section to PR into | Notes | +|---|---|---| +| [kyrolabs/awesome-agents](https://github.com/kyrolabs/awesome-agents) | learning / guides | Agent-specific. Verified live 2026-05-21 (★ 2.6k+, pushed 3d prior) — good fit | +| [Shubhamsaboo/awesome-llm-apps](https://github.com/Shubhamsaboo/awesome-llm-apps) | tutorials / learning | ★111k reach, but it is an *apps* list — confirm it has a learning/tutorials section before PR, else it gets rejected as out-of-scope | +| [punkpeye/awesome-mcp-servers](https://github.com/punkpeye/awesome-mcp-servers) | Tutorials | ALREADY in progress (PR #6135, see channel-partners.md #6) — don't double-submit | +| [travisvn/awesome-claude-skills](https://github.com/travisvn/awesome-claude-skills) | 📖 Tutorials & Guides | ALREADY tracked (channel-partners.md #9) — don't double-submit | + +## PR title + +``` +Add awesome-agentic-ai-zh — trilingual staged roadmap (LLM basics → multi-agent) +``` + +## PR description (entry + rationale) + +``` +Adding **awesome-agentic-ai-zh** to the [learning/tutorials] section. + +What it is: a staged learning roadmap for agentic AI (not a flat list) — +8 stages LLM-basics → multi-agent + Computer/Browser Use, 2 tracks +(use CLI agents vs build your own), 5 audience branches, 240+ curated +projects, runnable exercises. MIT. + +Why it fits this list: it's the "where do I start / in what order" +companion to the reference lists already here. + +Note on the name: the repo is `-zh` (Chinese-origin) but it is +**trilingual and the English edition is fully maintained** (~0.4% of +English lines carry any CJK; English-native required reading per stage; +CI-checked). Please file it under the general +learning/tutorials section, not a zh-only sub-section. + +Repo: https://github.com/WenyuChiou/awesome-agentic-ai-zh +Rendered site: https://wenyuchiou.github.io/awesome-agentic-ai-zh/ +``` + +## Suggested one-line list entry (match each list's existing format) + +``` +- [awesome-agentic-ai-zh](https://github.com/WenyuChiou/awesome-agentic-ai-zh) — Trilingual staged roadmap: LLM basics → multi-agent, 8 stages + 2 tracks + 240+ curated projects. MIT. +``` + +## Don'ts +- ❌ Don't PR to a list whose CONTRIBUTING forbids "roadmap/aggregate" entries — read it first. +- ❌ Don't double-submit to punkpeye / travisvn (already tracked). +- ❌ Don't PR to `e2b-dev/awesome-ai-agents` — verified 2026-05-21, last + push 2025-02-26 (~15 months stale / effectively abandoned); a PR there + will not be merged. Removed from the target table for this reason. +- ❌ Don't omit the "trilingual / English-maintained" note — the `-zh` + name otherwise gets it mis-filed or rejected as out-of-scope. diff --git a/.github/outreach/awesome-mcp-servers.md b/.github/outreach/awesome-mcp-servers.md new file mode 100644 index 0000000..71f21bf --- /dev/null +++ b/.github/outreach/awesome-mcp-servers.md @@ -0,0 +1,100 @@ +# Outreach: punkpeye/awesome-mcp-servers (PRIMARY) + +> **Status**: not contacted · **Channel**: GitHub PR +> **Primary lang**: en +> **Last updated**: 2026-05-09 +> **Primary repo**: https://github.com/punkpeye/awesome-mcp-servers (★86k+, MIT, has "Tutorials" section) +> **Secondary (skip for now)**: wong2/awesome-mcp-servers (★4k, MIT, **server-only policy** — no Tutorials section, off-policy for us) + +**Why this target**: We already cite both in our README's "Related projects" section (mutual benefit baked in). punkpeye is the canonical large MCP catalog and **has an explicit `## Tutorials` section** that fits us. wong2 is a stricter server-only fork — we'll skip that one to respect their list shape. + +**Pitch angle**: Their readers want to use MCP servers; we teach them how MCP works first (Stage 5.2 of our roadmap). Our §5.2 walkthrough → their flat catalog is a natural funnel. + +**Their counter-value**: Reciprocal cross-link; better onboarding for their ★86k readers. + +--- + +## Variant 1 — Social post (X, ~280 chars) + +``` +Browsing the awesome-mcp-servers catalog and unsure where to start? Stage 5.2 +of awesome-agentic-ai-zh walks through MCP from concept to first install in +~2 hours, then hands you off to wong2/awesome-mcp-servers for the actual +catalog browsing. + +★525 week 1 · MIT +🔗 github.com/WenyuChiou/awesome-agentic-ai-zh +``` + +## Variant 2 — GitHub PR (200-300 words) + +**Target file**: `README.md` — `## Tutorials` section +**PR title**: Add awesome-agentic-ai-zh to Tutorials — trilingual 8-stage learning roadmap + +**Diff** (insert in alphabetical or chronological position within `## Tutorials`): + +```diff ++ - [awesome-agentic-ai-zh](https://github.com/WenyuChiou/awesome-agentic-ai-zh) ++ — Trilingual (zh-TW · zh-Hans · en) 8-stage learning roadmap. Stage 5.2 is ++ a dedicated walkthrough of MCP (concept → first install → writing your ++ own server), with prerequisites and time estimates. Catalog includes 65+ ++ integrations grouped by use case. +``` + +**PR description**: + +```markdown +Hi @punkpeye, + +awesome-mcp-servers is already in our `Related projects` section +([README.md](https://github.com/WenyuChiou/awesome-agentic-ai-zh/blob/main/README.md)) +— we cite you as the primary catalog for MCP server discovery. + +Our repo is the **structured learning complement**: + +- Stage 5.2 of our roadmap is a **dedicated MCP walkthrough**: concept → + first install → writing your own server, with hands-on exercises and time + estimates +- After Stage 5.2, readers are sent to your catalog to find specific servers + for their stack +- Trilingual (zh-TW / zh-Hans / en), MIT, ★525 week 1 + +Targeting your `## Tutorials` section (line ~XX in README) since this is a +"how to learn MCP" resource, not a server. If a different section fits +better, just redirect — happy to update. + +Stats (week 1): 6,869 views / 3,185 unique / 1,099 clones / 408 unique cloners +/ 50 forks. CI runs banned-word audit + link-rot check on every PR. + +— Wenyu (PhD candidate, individual maintainer) +``` + +## Variant 3 — DM / Twitter (150 words) + +``` +@punkpeye — your awesome-mcp-servers list is already in our README's +"Related projects". I run awesome-agentic-ai-zh: a trilingual 8-stage +learning roadmap with Stage 5.2 dedicated to MCP (walkthrough → install → +writing your own server, with cost/time estimates). + +After Stage 5.2 our readers are sent to your catalog. Reciprocal link in +your Tutorials section would be natural — opened a PR (). Close it +if it doesn't fit. + +— Wenyu +``` + +--- + +## Notes + +- **Targeting punkpeye, not wong2** — punkpeye has a `## Tutorials` section + (★86k repo, very large reach); wong2 is server-only-policy by design (★4k, + no tutorials section, off-policy for our pitch) +- Confirm the line number / position of `## Tutorials` in punkpeye's README + before opening PR — alphabetical sort within the section is the convention +- punkpeye is responsive — PRs typically reviewed within ~7 days +- If they accept, mirror cross-cite by ensuring our README still references + them (already done as of 2026-05-09) +- If they redirect to a different section, follow their guidance — don't + push back diff --git a/.github/outreach/datawhale.md b/.github/outreach/datawhale.md new file mode 100644 index 0000000..450bf98 --- /dev/null +++ b/.github/outreach/datawhale.md @@ -0,0 +1,82 @@ +# Outreach: Datawhale (datawhalechina) + +> ⚠️ **Send content is now canonical in [`_send-day-packages.md`](_send-day-packages.md)** (package E — current numbers: 8 stages / 240+ resources). This file is kept for positioning rationale; do not paste its older issue/stats blocks directly. + +> **Status**: not contacted · **Channel**: GitHub issue + (later) WeChat group +> **Primary lang**: zh-Hans +> **Last updated**: 2026-05-09 +> **Decision-maker**: Datawhale 開源教學團隊 (open-source curriculum team) + +**Why this target**: Datawhale 是中國大陸最有影響力的 AI 教學社群之一；他們的 [`hello-agents`](https://github.com/datawhalechina/hello-agents) (★ 60k+) 在中文 agentic AI 圈子幾乎人人在用。我們的 Stage 5 cookbook 已經 cite 他們的 Extra05 / Extra08——cross-link 對雙方都加分。 + +**Pitch angle (我們對他們)**: 我們的 7 階段三語學習地圖把 Hello-Agents 放在 Stage 5 / 6 的位置——讀完我們前 4 階段的 LLM 基礎、prompt、context engineering 之後再進 Hello-Agents 會吸收得更好。我們等於是他們的「pre-flight」入口。 + +**Their counter-value (他們對我們)**: ★45k 的影響力；如果他們在 Hello-Agents README / docs 提我們一句「想看更完整的學習路線可以參考...」，能帶可觀流量。 + +--- + +## Variant 1 — Social post (Weibo / Threads / X，~280 字) + +> 「想用 Hello-Agents 但不確定該從哪裡入手？」 +> +> awesome-agentic-ai-zh 把 agentic AI 切成 7 階段（Stage 0 基礎 → Stage 7 production），每階段都標註預估時程跟入門條件。Stage 5/6 直接接到 @datawhalechina 的 Hello-Agents Extra05/08。 +> +> 三語（zh-TW / zh-Hans / en）· 145+ curated projects · MIT +> 👉 https://github.com/WenyuChiou/awesome-agentic-ai-zh + +## Variant 2 — GitHub issue (200-300 字) + +**Title**: Cross-link suggestion: structured learning path that points readers to Hello-Agents + +``` +Hi Datawhale 團隊！ + +我在維護 [awesome-agentic-ai-zh](https://github.com/WenyuChiou/awesome-agentic-ai-zh) +——一份中文 agentic AI 的 7 階段三語學習地圖（zh-TW canonical / zh-Hans / en，145+ +curated projects，MIT），第一週累積 ★525、3,185 unique visitors、1,099 clones。 + +我們的 Stage 5 cookbook 已經把 Hello-Agents 的 Extra05（記憶 + RAG 概覽）跟 Extra08 +（多代理）放進 reading list（[cookbook.md](https://github.com/WenyuChiou/awesome-agentic-ai-zh/blob/main/resources/cookbook.md)）， +作為走完前 4 階段 LLM 基礎之後的延伸閱讀。 + +**想 propose 一個雙向 cross-link**： + +1. 我們這邊已經 link 你們了（無條件，已經 ship） +2. 如果你們覺得合適——能不能在 Hello-Agents 的 README 或 docs 裡加一句「想看更 + 完整的 agentic AI 學習路線，可以參考 awesome-agentic-ai-zh」？ +3. 或是 reverse PR：我們在 §11 中文圈專用 加 Hello-Agents 的正式 entry（你們 + review）？ + +我們這邊的讀者主要從 Stage 4 之後想進 framework 跟 multi-agent，Hello-Agents +正好是下一階段最強的中文教材。如果不合適也完全 OK，謝謝你們把 Hello-Agents +做出來——它本身就是中文社群的公共財。 + +— Wenyu (PhD candidate · Lehigh CEE，個人 maintainer) +``` + +## Variant 3 — Email / WeChat DM (150 字) + +``` +Hi Datawhale 團隊好， + +我是 awesome-agentic-ai-zh 的維護者 Wenyu。這份 repo 是中文 agentic AI 的 7 階段 +三語學習地圖（145+ projects，三語齊全），上線一週 ★525。 + +我們 Stage 5 cookbook 已經把 Hello-Agents 的 Extra05/08 放進延伸閱讀清單。想跟你們 +聊聊有沒有可能 reciprocal cross-link 的可能——細節在我剛開的 [GitHub issue] +（連結）。 + +謝謝你們把 Hello-Agents 做出來，這幾年中文 agentic AI 學習的公共財都是你們扛的， +真的很感激。 + +— Wenyu +``` + +--- + +## Notes + +- **不要 promise**「我們會幫你們宣傳」之類的——只 offer 已經 ship 的 cross-link +- 如果他們同意 reverse PR 加 Hello-Agents 到 §11，記得用 `gh api` 確認 ★ 後加 +- WeChat 是 Datawhale 主要互動 channel，但 GitHub issue 比較 maintainable + 可追蹤 +- 如果一週沒回——OK，他們團隊很忙、不要 ping diff --git a/.github/outreach/hacker-news.md b/.github/outreach/hacker-news.md new file mode 100644 index 0000000..0887756 --- /dev/null +++ b/.github/outreach/hacker-news.md @@ -0,0 +1,74 @@ +# Outreach draft — Hacker News (Show HN) + +> **Status**: draft, not submitted. Maintainer reviews + posts manually. +> One shot — don't repost if it doesn't catch. Pick a weekday, ~08:00–10:00 +> US Eastern (HN morning), not Fri/weekend. + +## Why HN + +Largest single English dev-audience spike potential. Audience overlaps +exactly: people building with LLMs / agents who like structured depth. +Risk: HN is allergic to hype and to "another awesome-list". The draft +below leads with the concrete artifact and pre-empts the two predictable +top comments ("why another list" / "is the English LLM-translated"). + +## Title (pick one — no emoji, no hype, ≤ 80 chars) + +1. `Show HN: A trilingual, staged roadmap from LLM basics to multi-agent systems` +2. `Show HN: Agentic-AI learning roadmap – 8 stages, 240+ curated projects` +3. `Show HN: An opinionated path to learn agentic AI (not an awesome-list dump)` + +Recommended: **#1** (says what it is + the trilingual angle, no adjectives). + +## Body (paste into the text field, keep it short) + +``` +I built a structured learning roadmap for agentic AI because every +"awesome-list" I found was a flat link dump with no order — great as a +reference, useless as a path if you don't already know what you don't +know. + +This is sequenced: 8 stages from "what's a token" to multi-agent +orchestration + Computer/Browser Use, with explicit entry conditions and +a self-check at the end of each stage, plus two tracks (use existing CLI +agents vs. build your own) and 5 audience branches (researcher / +developer / teacher / knowledge worker / everyday user). + +It started as a Chinese-language project, but the English edition is +fully maintained, not a machine-translated afterthought (~0.4% of +English lines contain any CJK, almost all intentional term-mapping; CI +checks localization + anchor integrity). Rendered site (trilingual, +mkdocs): https://wenyuchiou.github.io/awesome-agentic-ai-zh/ +Repo: https://github.com/WenyuChiou/awesome-agentic-ai-zh + +It's MIT, ~240 curated projects each with star/audience/"what it +teaches/how to run", and small runnable exercises (1–5 per stage). Honest limitation: +it's opinionated (Claude-ecosystem-heavy in the later stages — MCP / +Skills / SDK), and the deep exercises point out to first-party cookbooks +rather than re-teaching them. Feedback on the sequencing + what's +missing is what I'm after. +``` + +## First-comment (post yourself, immediately, as the author) + +``` +Author here. Two things I expect to come up: + +1. "Why not just a list?" — the curation IS in there (240+ entries with + the usual metadata), but the value I was missing was ORDER + exit + criteria, so the spine is the stage sequence, not the list. + +2. "Is the English LLM-slop?" — fair worry for a zh-origin repo. It's + not a thin mirror: measured ~0.4% CJK across 64 English files, the + required-reading per stage is English-native primary sources + (Anthropic/OpenAI/HF docs), and structure is CI-gated. Tell me where + it reads translated and I'll fix it. + +Happy to take "this stage is wrong / out of date" specifics. +``` + +## Don'ts +- ❌ Don't say "the best / definitive / production-grade" (HN will pile on). +- ❌ Don't lead with the star count. +- ❌ Don't repost a flopped submission within weeks. +- ❌ Don't ask for upvotes/stars anywhere. diff --git a/.github/outreach/huggingface-zh.md b/.github/outreach/huggingface-zh.md new file mode 100644 index 0000000..12ec66b --- /dev/null +++ b/.github/outreach/huggingface-zh.md @@ -0,0 +1,97 @@ +# Outreach: HuggingFace 中文社群 + +> **Status**: not contacted · **Channel**: HF community post / Spaces card / discuss page +> **Primary lang**: en + zh-Hans (HF 國際化) +> **Last updated**: 2026-05-09 + +**Why this target**: HuggingFace 是英語 ML / agent 生態的中心，但中文社群也越來越大（中國團隊頻繁在 HF 發 model）。HF 沒有「learning roadmap for agentic AI」這個 slot——我們 trilingual + structured 的定位剛好填補。 + +**Pitch angle**: 我們不是模型 / dataset / Space，我們是「先把 agentic AI 學完再來用 HF」的入口。 + +**Their counter-value**: HF 流量 + 國際曝光；對英語讀者來說我們的 zh-TW / zh-Hans 段落是 bonus。 + +--- + +## Variant 1 — Social post (Twitter / X，~280 chars) + +``` +For folks asking "where do I start with agentic AI?" — built a trilingual +8-stage learning roadmap (zh-TW · zh-Hans · en) covering Stage 0 (foundations) +through Stage 8 (multi-agent production). 240+ curated projects with cost, +audience, and time estimates per stage. + +⭐ 525 in week 1 · MIT +🔗 github.com/WenyuChiou/awesome-agentic-ai-zh +``` + +## Variant 2 — HF Community Post / Discuss (200-300 words) + +**Title**: Trilingual 8-stage learning roadmap for agentic AI (Stage 0 → multi-agent production) + +**Body**: + +```markdown +Hi HF community, + +I've been building [awesome-agentic-ai-zh](https://github.com/WenyuChiou/awesome-agentic-ai-zh) +— a trilingual (zh-TW canonical · zh-Hans · English) 8-stage learning roadmap +for agentic AI: + +- **Stage 0**: Foundations (Python, git, CLI, REST APIs) +- **Stage 1-3**: LLM basics, prompt engineering, RAG, frameworks +- **Stage 4**: Agentic frameworks (LangChain, LangGraph, AutoGen, etc.) +- **Stage 5**: Claude Code ecosystem (MCP, Skills, Plugins) +- **Stage 6**: Multi-agent + memory + production hardening +- **Stage 7**: Production deployment + observability +- **Stage 8**: Agent Interfaces (Computer Use, Browser Use, Code Sandbox) + +Each stage has time estimates, prerequisites, hands-on exercises, and 240+ +curated projects across the catalog. The catalog includes MCP servers, Skills, +and integrations grouped by 16 use-case categories — including a section +specifically for the Chinese-language ecosystem (Coze, Qwen-Agent, LangChain +zh learning, etc.). + +Some HF community members may find it useful as a "**before you train your +first model**" structured path, especially folks asking where to start with +agents. Trilingual support is genuinely tested (not machine-translated). + +Stats (week 1): ★525 / 50 forks / 3,185 unique visitors / 408 unique cloners. +MIT, contributors welcome. + +Happy to take feedback if any HF maintainer thinks specific stages should +reference HF resources more directly (Stage 1 already cites HF transformers +and HF model cards). + +— Wenyu +``` + +## Variant 3 — Email / DM to specific HF maintainer (150 words) + +``` +Hi , + +I built awesome-agentic-ai-zh, a trilingual (zh-TW / zh-Hans / en) 8-stage +learning roadmap for agentic AI — covers foundations through multi-agent +production with 240+ curated projects, cost/time estimates per stage. ★525 ++ 3,185 unique visitors in week 1. + +Stage 1 (LLM basics) and Stage 4 (frameworks) reference HF transformers +and HF Hub fairly heavily. Wondering if there's a HF-side touchpoint that +makes sense — community blog post, discussion thread, or HF Learn linkage. + +No urgency, no expectation. If interested, happy to chat. If not, no offense +taken. + +— Wenyu +``` + +--- + +## Notes + +- HF community page: huggingface.co/posts (treat like a discussion forum) +- **Do not** tag random HF people; if pitching to maintainer, identify by past + ML/agent work specifically +- HF Learn: huggingface.co/learn — if our content ever gets featured, the + HF Learn audience is exactly our target +- For zh-Hans segment: HF has a 中文社群 page; can post there separately diff --git a/.github/outreach/langchain-ai.md b/.github/outreach/langchain-ai.md new file mode 100644 index 0000000..24a1d04 --- /dev/null +++ b/.github/outreach/langchain-ai.md @@ -0,0 +1,105 @@ +# Outreach: LangChain ecosystem (langchain-ai / kyrolabs/awesome-langchain) + +> ⚠️ **Send content is now canonical in [`_send-day-packages.md`](_send-day-packages.md)** (package C — current numbers: 8 stages / 240+ resources). This file is kept for positioning rationale; do not paste its older entry/stats blocks directly. + +> **Status**: not contacted · **Channel**: GitHub PR +> **Primary lang**: en (with zh as bonus) +> **Last updated**: 2026-05-26 (refreshed — stats, 8-stage structure, correct section target) +> **Repos**: +> - https://github.com/langchain-ai/langchain (main repo) +> - https://github.com/kyrolabs/awesome-langchain (community awesome list ★9k+) + +**Why this target**: LangChain is the gateway agent framework for ~80% of zh-language developers. Our Stage 4 covers it; our §11 catalog now includes Langchain-Chatchat (★37k) and the Chinese LangChain getting-started guide (which **already lives in the same section** we're targeting — see below). Cross-link is natural. + +**Pitch angle**: +- For `langchain-ai/langchain` itself: too big a target; aim instead at the **community awesome list** (`kyrolabs/awesome-langchain`). +- For `kyrolabs/awesome-langchain`: we're a multilingual learning-order complement to their flat catalog. +- **Target section confirmed (2026-05-26)**: `## Learn → ### Notebooks`. Precedent: `liaokongVFX/LangChain-Chinese-Getting-Started-Guide` already sits there. There is **no** "Tutorials & Learning Resources" section in the current README; do not propose one. + +**Their counter-value**: ★9k exposure to LangChain-curious developers worldwide. + +--- + +## Variant 1 — Social post (X / LinkedIn, ~280 chars) + +``` +LangChain learners often ask: "I have the docs, but where do I actually start?" + +Built an 8-stage trilingual learning roadmap (zh-TW · zh-Hans · en). Stage 4 +walks through LangChain / LangGraph / AutoGen / CrewAI / Smolagents with +prerequisites and time estimates. 145+ projects · MIT · ★1.7k. + +🔗 github.com/WenyuChiou/awesome-agentic-ai-zh +``` + +## Variant 2 — GitHub PR to kyrolabs/awesome-langchain (200-300 words) + +**PR title**: Add awesome-agentic-ai-zh (trilingual learning roadmap) to Learn → Notebooks + +**Diff** (against `## Learn → ### Notebooks`, after the `liaokongVFX/LangChain-Chinese-Getting-Started-Guide` line — keeps the two zh-ecosystem learning resources adjacent): + +```diff + - [LangChain Chinese Getting Started Guide](https://github.com/liaokongVFX/LangChain-Chinese-Getting-Started-Guide): Chinese LangChain Tutorial for Beginners ![GitHub Repo stars](https://img.shields.io/github/stars/liaokongVFX/LangChain-Chinese-Getting-Started-Guide?style=social) ++ - [WenyuChiou/awesome-agentic-ai-zh](https://github.com/WenyuChiou/awesome-agentic-ai-zh): Trilingual (zh-TW / zh-Hans / en) 8-stage learning roadmap for agentic AI — Stage 4 walks through LangChain, LangGraph, AutoGen, CrewAI, Smolagents with prerequisites, time estimates, and hands-on exercises ![GitHub Repo stars](https://img.shields.io/github/stars/WenyuChiou/awesome-agentic-ai-zh?style=social) +``` + +**PR description**: + +```markdown +Hi kyrolabs maintainers, + +Proposing addition of [WenyuChiou/awesome-agentic-ai-zh](https://github.com/WenyuChiou/awesome-agentic-ai-zh) to **Learn → Notebooks**, next to the existing `liaokongVFX/LangChain-Chinese-Getting-Started-Guide` entry (same zh-learning surface). + +**Why this is a good fit**: +- Trilingual (zh-TW canonical · zh-Hans · en — all three fully maintained, not MT) — fills a gap for non-English learners +- **Stage 4 (Agent Frameworks)** walks new developers through **LangChain / LangGraph / AutoGen / CrewAI / Smolagents** with prerequisites, time estimates, and hands-on exercises +- §11 of the catalog has 7 Chinese-ecosystem entries including `chatchat-space/Langchain-Chatchat` (★37k) and the LangChain Chinese Getting Started Guide that's already in your list +- Stage 5 covers the Claude Code / MCP / Skills layer; Stage 8 covers Agent Interfaces (Computer Use / Browser / Sandbox). Together with the catalog this is the complement-to-LangChain-docs that doesn't currently exist in zh + +**Stats (2026-05-26)**: ★1.7k · 191 forks · 5,090 unique visitors (14d) · 1,316 unique cloners (14d) · 3 community contributors. MIT licensed. Rendered docs at https://wenyuchiou.github.io/awesome-agentic-ai-zh/. CI runs banned-word + link-rot + anchor-integrity lints on every PR. + +If a different section or shape works better, happy to redirect. Thanks for maintaining awesome-langchain. + +— Wenyu Chiou (individual maintainer) +``` + +## Variant 3 — Email to LangChain DevRel (150 words) + +``` +Hi LangChain team, + +I built awesome-agentic-ai-zh — a trilingual (zh-TW / zh-Hans / en) 8-stage +learning roadmap for agentic AI. ★1.7k, 5k unique visitors / 14 days, heavy +zh-language community traction (top external referrer is Threads). + +Stage 4 walks new developers through LangChain → LangGraph → AutoGen → +CrewAI → Smolagents with prerequisites and time estimates per step. +Designed to bridge "I know Python" to "I can build a working agent." + +Two questions: +1. Is there a LangChain-side surface where this would fit (Learn, blog, + docs sidebar)? +2. Any specific LangChain features I should cover better in Stage 4? Open + to feedback. + +No expectation, just opening dialogue. + +— Wenyu +``` + +--- + +## Notes + +- **First target**: kyrolabs/awesome-langchain (community awesome list, lower + barrier to merge). **Section: `Learn → Notebooks`**, not "Tutorials" (no such + section exists in the current README — verified 2026-05-26). +- **Second target**: LangChain blog/docs (higher signal but harder to land) +- Avoid pitching `langchain-ai/langchain` itself directly — too big, signal is + drowned out +- LangSmith / LangGraph teams are separate — different DevRel; don't pitch all + three at once +- **Stat snapshot is per-PR-day** — refresh `★`, `forks`, `unique visitors`, + `clones` with `gh repo view --json stargazerCount,forkCount` + `gh api + repos/.../traffic/views,clones` on the day you submit. Stale stats in a PR + body read as careless. diff --git a/.github/outreach/liyupi.md b/.github/outreach/liyupi.md new file mode 100644 index 0000000..5034bba --- /dev/null +++ b/.github/outreach/liyupi.md @@ -0,0 +1,82 @@ +# Outreach: liyupi/ai-guide + +> ⚠️ **Send content is now canonical in [`_send-day-packages.md`](_send-day-packages.md)** (package D — current numbers: 8 stages / 240+ resources). This file is kept for positioning rationale; do not paste its older entry/stats blocks directly. + +> **Status**: not contacted · **Channel**: GitHub PR +> **Primary lang**: zh-Hans +> **Last updated**: 2026-05-09 +> **Repo**: https://github.com/liyupi/ai-guide (★ 16k+) + +**Why this target**: liyupi 的 ai-guide 是中國大陸最大的 AI 資源 hub 之一（★13k+），定位是 aggregator——「整理所有相關資源」。我們是結構化 7 階段學習地圖，剛好是他們資源海中缺的「**怎麼依序學**」這一塊。 + +**Pitch angle**: 我們不取代他們的 aggregator 功能，我們是他們資源使用者的「pre-reading 路線」。 + +**Their counter-value**: ★13k 的曝光；mainland zh 流量入口。 + +--- + +## Variant 1 — Social post (Weibo / X，~280 字) + +> 在 ai-guide 找到一堆好資源，但不知道該從哪一個開始？ +> +> awesome-agentic-ai-zh 把 agentic AI 切成 Stage 0 → Stage 7，每階段告訴你預估時程 +> + 入門條件 + 該讀什麼。三語齊全（zh-TW / zh-Hans / en），145+ curated projects。 +> +> 配著 ai-guide 一起看：用 ai-guide 找 project、用我們找順序。 +> 👉 https://github.com/WenyuChiou/awesome-agentic-ai-zh + +## Variant 2 — GitHub PR (200-300 字) + +**PR title**: Add awesome-agentic-ai-zh to "AI 学习路线" / "相关资源" section + +**File modified**: README.md（或者對應的 resource list 檔） + +```diff ++ - [`WenyuChiou/awesome-agentic-ai-zh`](https://github.com/WenyuChiou/awesome-agentic-ai-zh) ++ — 7 阶段三语学习地图（zh-TW canonical / zh-Hans / en）；从 Stage 0 基础到 ++ Stage 7 multi-agent production，每阶段附预估时程 + 入门条件 + 145+ curated ++ projects。MIT，跟 ai-guide 互补：ai-guide 找 project、awesome-agentic-ai-zh ++ 找学习顺序。 +``` + +**PR description**: + +``` +你好 liyupi 大佬！ + +我维护的 [awesome-agentic-ai-zh](https://github.com/WenyuChiou/awesome-agentic-ai-zh) +是一份中文 agentic AI 的 7 阶段三语学习地图（zh-TW canonical / zh-Hans / en，145+ +projects，MIT 协议），第一周 ★525、3,185 unique visitors、1,099 clones，主要流量来自 Threads / +X / 部分微信社群。 + +定位上跟 ai-guide 互补——ai-guide 是「大全」，我们是「学习顺序」。我们的读者经常 +是「想学但不知道该先学什么」的工程师，看完我们的 Stage 0–7 后，回到 ai-guide 找 +具体 project 用。 + +想 propose 加进 ai-guide 的「AI 学习路线」或「相关资源」section。如果觉得不合适 +请直接关掉，谢谢您！ + +— Wenyu (PhD candidate · Lehigh，个人 maintainer) +``` + +## Variant 3 — Email / Weibo DM (150 字) + +``` +你好 liyupi 大佬， + +我是 awesome-agentic-ai-zh 的维护者 Wenyu。这份是中文 agentic AI 的 7 阶段三语 +学习地图（145+ projects，三语齐全），上线一周 ★525。 + +定位跟 ai-guide 互补——ai-guide 找 project、我们找顺序。想加进你们的「相关资源」 +section，刚开了 PR（链接）。如果不合适请直接关掉，谢谢！ + +— Wenyu +``` + +--- + +## Notes + +- liyupi 的 PR 速度看心情——如果一週沒動就 ping 一下、沒回就放著 +- **不要**自吹「比 ai-guide 更好」之類的——我們是 complement、不是 replace +- 注意：liyupi 偏好簡體 + 中國大陸友善的措辭，PR 描述用 zh-Hans diff --git a/.github/outreach/moonshot.md b/.github/outreach/moonshot.md new file mode 100644 index 0000000..f61243f --- /dev/null +++ b/.github/outreach/moonshot.md @@ -0,0 +1,87 @@ +# Outreach: Moonshot Kimi 開發者頻道 + +> **Status**: not contacted · **Channel**: 開發者社群（Discord / 知乎 / GitHub） +> **Primary lang**: zh-Hans +> **Last updated**: 2026-05-09 +> **Their main surface**: https://kimi.moonshot.cn · https://github.com/MoonshotAI + +**Why this target**: 月之暗面 (Moonshot) 是大陸 frontier AI lab 之一（Kimi K2 / Kimi-Chat）。我們 §11 中文圈專用 沒有 Moonshot entry——他們的開源主要是 model paper / weights，沒有 agent SDK 形態的 canonical repo。 + +**Pitch angle (邀請式)**: 跟 Zhipu 同邏輯——§11 缺 Moonshot 的 agent / Skills 入口；邀請他們社群推薦合適的 PR。 + +**Their counter-value**: Kimi 開發者透過我們學整個 agentic 生態；他們在 zh community 的 visibility 提升。 + +--- + +## Variant 1 — Social post (Weibo / X，~280 字) + +``` +中文 agentic AI 學習地圖 awesome-agentic-ai-zh，§11 中文圈專用 收了 Qwen-Agent + +Coze——缺 Moonshot 的 entry。 + +如果月之暗面的同學 / 熱心開發者覺得有 Kimi 系列的 agent skill / SDK / cookbook +該收進來，歡迎 PR：github.com/WenyuChiou/awesome-agentic-ai-zh + +240+ projects · 三語齊全 · MIT · ★525 第一週 +``` + +## Variant 2 — Discussion / 知乎文章 (200-300 字) + +**Title**: 邀請：月之暗面 Kimi agent 生態，有合適的開源項目可以收進 awesome-agentic-ai-zh §11 嗎？ + +**Body**: + +```markdown +你好 Moonshot 社群， + +我维护 [awesome-agentic-ai-zh](https://github.com/WenyuChiou/awesome-agentic-ai-zh) +——一份中文 agentic AI 的 8 阶段三语学习地图（zh-TW canonical / zh-Hans / en，240+ +projects，MIT，★525 第一周）。 + +**§11 中文圈专用** 已经收了 Qwen-Agent / Coze / Langchain-Chatchat 等，但缺 +Moonshot Kimi 的 entry。我评估过的几个候选： + +- `MoonshotAI/Kimi-K2`：模型 paper / weights repo，不是 agent SDK 形态 +- 没看到官方的 `kimi-agent-sdk` / `kimi-skills` / `kimi-cookbook` canonical 仓库 + +想问问 Moonshot 社群： +1. 有没有官方或半官方的 Kimi agent / Skills / cookbook 仓库可以推荐？ +2. 如果有，欢迎 PR 到 §11。收录原则： + - agent / Skill / SDK / MCP-shaped（不只是模型 API） + - license 清楚 + - 最近 90 天有 commit + - 品质优于流行度（star 数不是门槛） + +社群的 chat-bot / agent 二次开发也算——只要是 Kimi 周边的合适学习资源都可以。 + +如果一时没合适项目，也 OK，先留个 thread 追踪 agent 生态成熟度。 + +— Wenyu (Lehigh CEE PhD candidate，个人 maintainer) +``` + +## Variant 3 — DM / 邮件 (150 字) + +``` +你好 Moonshot 社群， + +我是 awesome-agentic-ai-zh 的维护者 Wenyu。这份是中文 agentic AI 的 8 阶段三语 +学习地图（240+ projects，三语齐全，MIT，★525 第一周）。 + +§11 中文圈专用 收了 Qwen-Agent + Coze，缺 Kimi 的 entry。如果有官方推荐的 Kimi +agent / Skills / cookbook 仓库我应该收进来，请告诉我；或者直接 PR 到 §11。收录 +原则：agent-shaped + license 清楚 + 90 天内活跃。 + +谢谢！ +— Wenyu +``` + +--- + +## Notes + +- **同 Zhipu 邏輯**：邀請式而非推銷式 +- Moonshot 的開源相對少（主要是 paper + weights），如果真的沒合適 repo，**不 + 要硬 fit**——可以先放著，等他們釋出 agent SDK 再收 +- Moonshot 開發者社群入口：Discord / 知乎 / 飛書群（Kimi 開發者群） +- 不要與 Zhipu outreach 同日發送——避免「同一個人到處撒網」感 +- 監測：如果 Moonshot 後續釋出 `MoonshotAI/Kimi-Agent` 或類似——優先收進 §11 diff --git a/.github/outreach/newsletters-en.md b/.github/outreach/newsletters-en.md new file mode 100644 index 0000000..432296d --- /dev/null +++ b/.github/outreach/newsletters-en.md @@ -0,0 +1,59 @@ +# Outreach draft — English AI newsletters + +> **Status**: draft, not submitted. These are "tip / submission" blurbs, +> not articles. Submit via each newsletter's tip form/email. A learning +> roadmap is newsletter-friendly (evergreen, linkable, low-risk to +> feature). Best AFTER a HN/Reddit signal exists (newsletters like +> "already getting traction"). + +## Targets + submission route + +| Newsletter | Route | Notes | +|---|---|---| +| TLDR AI | tldr.tech "submit a link" / reply to an issue | Largest; wants a one-liner + link | +| Ben's Bites | bensbites submit form | Likes tools/resources builders can use today | +| Last Week in AI | skynettoday/LWiAI tip | Leans research+practitioner | +| Latent Space | latent.space (Discord/issue) | Practitioner, agent-heavy audience | +| C%2BAI / Rundown AI | their submit forms | Broad; keep it one sentence | + +## Blurb A — generic one-liner (TLDR AI / Rundown style) + +``` +A free, trilingual, staged roadmap for learning agentic AI — 8 stages +from LLM basics to multi-agent + Computer/Browser Use, 240+ curated +projects, runnable exercises, with entry conditions and a self-check per +stage. MIT. Rendered site: https://wenyuchiou.github.io/awesome-agentic-ai-zh/ +``` + +## Blurb B — 2–3 sentences (Ben's Bites / Latent Space style) + +``` +Most "awesome-AI-agents" lists are flat link dumps — useful to look +things up, useless as a path. This is a sequenced roadmap instead: 8 +stages (LLM basics → tool use → frameworks → Claude Code ecosystem → +memory/RAG → multi-agent → Computer/Browser Use), two tracks (use +existing CLI agents vs build your own), 5 audience branches, 240+ +curated projects with "what it teaches / how to run", and small runnable +exercises that work locally (Ollama) before any paid API. Trilingual and +MIT; the English edition is fully maintained, not machine-translated. +https://wenyuchiou.github.io/awesome-agentic-ai-zh/ +``` + +## Blurb C — "why your readers care" framing (editor pitch email) + +``` +Subject: Resource submission — staged agentic-AI learning roadmap (MIT, trilingual) + +Hi [name], your readers regularly ask "how do I actually start building +agents". This is a structured answer: a staged roadmap (not another flat +list) that goes LLM-basics → multi-agent with explicit prerequisites and +an end-of-stage self-check, plus a CLI-power-user track for people who +just want to USE agents rather than build them. Free/MIT, trilingual, +rendered docs site. Repo: https://github.com/WenyuChiou/awesome-agentic-ai-zh +Happy to give any extra context. — [you] +``` + +## Don'ts +- ❌ Mass-BCC the same email to all (personalise the [name]/[why]). +- ❌ Overclaim ("the definitive guide"). Editors cut hype. +- ❌ Submit the same week as a flop — wait for a signal to reference. diff --git a/.github/outreach/reddit.md b/.github/outreach/reddit.md new file mode 100644 index 0000000..7884ff8 --- /dev/null +++ b/.github/outreach/reddit.md @@ -0,0 +1,80 @@ +# Outreach draft — Reddit + +> **Status**: draft, not submitted. Maintainer posts manually, ONE +> subreddit per day max, tailored each time (cross-posting identical +> text reads as spam and gets auto-removed). Always disclose you're the +> author. Read each sub's rules + "self-promotion" policy first. + +## Targets (priority order) + +| Sub | Why | Rule note | +|---|---|---| +| r/AI_Agents | Exact audience (agent builders) | Self-promo tolerated if it's substantive + you engage | +| r/LocalLLaMA | Huge, builder-heavy, likes curation | No pure self-promo on weekends; lead with value | +| r/ClaudeAI | Later stages are Claude-ecosystem (MCP/Skills) | Fits; flair appropriately | +| r/learnmachinelearning | "How do I learn agents" asked daily | Post as a resource, not a launch | +| r/MachineLearning | Strict; only if framed as a resource, low priority | Needs heavy substance, mod-gated — optional | + +## r/AI_Agents (primary) + +**Title**: `A staged roadmap to learn agentic AI (not a flat awesome-list) — feedback wanted` + +**Body**: +``` +I kept seeing "awesome-X" lists for agents — useful as references, but +none gave an ORDER to actually learn in. So I built a sequenced roadmap +and I'd like this sub to poke holes in the sequencing. + +- 8 stages: LLM basics → prompt design → tool use → frameworks → + Claude Code ecosystem (MCP/Skills) → memory/RAG → multi-agent → + Computer/Browser Use. Each stage has entry conditions + an end self-check. +- 2 tracks: "use existing CLI agents" vs "build your own". +- 5 audience branches (researcher / dev / teacher / knowledge worker / + everyday user). +- 240+ curated projects (star / audience / what it teaches / how to run) + + small runnable exercises (1–5 per stage). MIT. + +Trilingual (the project is Chinese-origin but the English edition is +fully maintained, not MT slop). Rendered site: +https://wenyuchiou.github.io/awesome-agentic-ai-zh/ + +Honest bias: Claude-ecosystem-heavy in the later stages. What I want: +where is the stage order wrong, and what's a glaring omission? +``` + +## r/LocalLLaMA (variant — lead with the local-LLM angle) + +**Title**: `Trilingual agentic-AI roadmap — every stage's exercises run on Ollama/local first, Claude as the prod reference` + +**Body**: (same skeleton, swap first paragraph) +``` +Built a staged learn-path for agentic AI. Relevant to this sub +specifically: the hands-on exercises are dual-path — Ollama / local +runner first (llama.cpp, LocalAI, MLX listed), with Claude/Anthropic as +the production reference, so you can do the whole roadmap locally before +spending an API cent. +``` +(then the same 8-stages / 2-tracks / link / "honest bias" / "feedback +wanted" tail as r/AI_Agents) + +## r/ClaudeAI (variant — lead with the ecosystem depth) + +**Title**: `A learning path that actually covers the Claude Code ecosystem (MCP / Skills / Plugins / SDK), staged` + +**Body**: lead paragraph emphasising Stage 5/8 (Claude Code ecosystem + +Agent Interfaces) as the differentiator vs framework-only tutorials; +same tail. + +## r/learnmachinelearning (variant — resource framing, not launch) + +**Title**: `Resource: a free, staged roadmap from LLM basics to multi-agent (with exit self-checks)` + +**Body**: frame as "for the recurring 'how do I start with agents' +question" — emphasise Stage 0–2 foundation + the self-check gating; +same link; lighter on the build-track detail. + +## Don'ts +- ❌ Identical body across subs (auto-spam-flag). +- ❌ "Please star/upvote". +- ❌ Drive-by post then disappear — must reply to comments for ~24h. +- ❌ Posting to r/MachineLearning without resource framing (removal). diff --git a/.github/outreach/x-twitter.md b/.github/outreach/x-twitter.md new file mode 100644 index 0000000..1c57ae2 --- /dev/null +++ b/.github/outreach/x-twitter.md @@ -0,0 +1,130 @@ +# Outreach draft — X (Twitter) + +> **Status**: draft, not submitted. Maintainer reviews + posts manually. +> Identity-bound channel — do not delegate posting to an agent. + +## Why X + +Fastest broadcast for the EN agentic-AI community (the same crowd that +amplifies on HN / Reddit also lives here in shorter form). What X does +well: identity-signal-driven discovery (a PhD researcher posting a +curated learning artifact is read very differently from an anonymous +account), image-driven engagement (link cards + banner = ~2× CTR), and +quote-retweet amplification by mid-tier AI accounts. What X does poorly: +depth (280-char ceiling), context (no threading culture for awesome-list +type artifacts), and persistence (24-48h half-life). + +Risk: X is also where overclaim and self-promotion are punished hardest. +Lead with the artifact, not adjectives. + +## Pre-flight (verify before posting) + +These are already shipped (`44b1cbe`), but re-check the live preview +before you tweet — X's card scraper caches aggressively: + +- [ ] X card preview for `github.com/WenyuChiou/awesome-agentic-ai-zh` + shows EN-lead description (paste URL into a draft tweet first, eyeball + the card; if zh-TW-lead, the cache hasn't refreshed — wait ~1h or + re-share via the Pages URL) +- [ ] Your X bio matches the LinkedIn reframe (PhD, Civil & Environmental, + Lehigh · agent-based modeling · LLM / AI agent). Identity coherence + across platforms is the discovery multiplier on X +- [ ] `banner.en.png` is uploaded as the tweet image (drag-attach, not URL + auto-card — image attachment outperforms link card for first impression) + +## Tweet options (pick one; no emoji-spam, no hype, ≤ 280 chars) + +### A — EN-lead, broad AI audience (recommended) + +``` +An AI agent learning map — curates 240+ repos across the agentic-AI +ecosystem (MCP / skills / frameworks / agents), from LLM basics to +multi-agent. Trilingual (EN / 繁中 / 简中); branches for researcher / +dev / teacher / knowledge worker. + +github.com/WenyuChiou/awesome-agentic-ai-zh +``` + +### B — research-first (matches your LinkedIn reframe) + +``` +PhD student doing agent-based flood-adaptation modeling. Open-sourced +the AI agent learning map I built for my own learning — 240+ curated +repos across the agentic-AI ecosystem, LLM basics → multi-agent. +Trilingual (EN / 繁中 / 简中). + +github.com/WenyuChiou/awesome-agentic-ai-zh +``` + +### C — bilingual lead (CJK AI-Twitter circle: 寶玉 / AK / 向阳乔木) + +``` +做了個 AI Agent 學習地圖 —— 蒐集 240+ 個 repo(MCP / skills / frameworks / agents),從 LLM 基本概念一路走到 multi-agent 系統,三語維護(繁中 / 简中 / EN)。對 researcher / 開發者 / 老師 / 知識工作者各有分支。 + +An AI agent learning map — github.com/WenyuChiou/awesome-agentic-ai-zh +``` + +Recommended: **A** for the first push (EN-lead is where the audience is +and the meta tags are now aligned). Consider **C** as a separate post +~1 week later targeting CJK AI Twitter — different audience, different +time window, doesn't compete with the EN push. + +## Timing + +- **A (EN)**: weekday 08:00–10:00 US Eastern (US AI Twitter morning; + overlaps EU evening). Avoid Fri PM / weekends. +- **C (CJK)**: weekday 20:00–22:00 Beijing time (CN evening; overlaps TW + prime time). +- Don't post both same day — collisions hurt both. + +## First-reply (optional, post yourself ~5 min after main tweet) + +Use this only if engagement is picking up and a clarifier would unlock +more clicks. Don't pre-load it if the main tweet flops. + +``` +A few things that aren't obvious from the README: +- Stage 0 is for non-coders (web/desktop on-ramps before any CLI). +- Examples default to local Ollama, not paid APIs — the cloud path is + the alt, not the default. +- The English edition is fully maintained alongside the Chinese + canonical, gated by CI (anchor + locale checks). +``` + +## Image / link strategy + +- **Attach**: `resources/diagrams/banner.en.png` (the 2026-05-13 ChatGPT- + rendered EN banner, already in repo). +- **Link in tweet body**: `github.com/WenyuChiou/awesome-agentic-ai-zh` + — GitHub URL beats Pages URL because the ★ count is the trust signal + EN readers scan for. (Pages URL `wenyuchiou.github.io/awesome-agentic-ai-zh/en/` + is a fallback if you want to land EN readers on an English doc page + directly.) +- **No hashtags** in the main tweet — the audience is already targeted + by who follows you and who you quote-mention. Hashtags dilute the + reach signal more than they add discovery on X today. + +## Engagement tactics (light) + +- Quote-RT one of your own older posts about agent stuff with the new + URL, ~6h after main tweet, if first push got <20 likes (re-broadcast, + don't ask for boost) +- If a mid-tier AI account QTs you, reply with a *specific* follow-up + (a stage they'd find useful for their audience), not a generic + "thanks" +- If someone says "is the English just machine-translated?" — point them + to the CI lint config + the audit comment in the HN draft. Don't + defensively re-explain in the main thread. + +## Don'ts + +- ❌ Don't say "the best / definitive / world-class / production-grade" +- ❌ Don't lead with the ★ count (let the GitHub card show it) +- ❌ Don't @-mention famous AI accounts asking for amplification +- ❌ Don't post the same content to LinkedIn the same day (cross-platform + redundancy on the same hour smells like a launch campaign, not an + individual sharing). Stagger 24-48h +- ❌ Don't reuse the same tweet text after a flop — rewrite if going + for a second push +- ❌ Don't post C and A on the same day (different audiences, but + appearing twice in the same feed reads as spam) diff --git a/.github/outreach/zhipu.md b/.github/outreach/zhipu.md new file mode 100644 index 0000000..bf18ab9 --- /dev/null +++ b/.github/outreach/zhipu.md @@ -0,0 +1,86 @@ +# Outreach: Zhipu BigModel community (智譜) + +> **Status**: not contacted · **Channel**: 開發者社群（Discord / 知乎 / GitHub Discussions） +> **Primary lang**: zh-Hans +> **Last updated**: 2026-05-09 +> **Their main surface**: https://bigmodel.cn / https://github.com/MetaGLM + +**Why this target**: 智譜 (Zhipu) 是大陸頭部 AI lab 之一（GLM 系列模型）。我們的 §11 中文圈專用目前**沒有 Zhipu 的條目**——主要是因為他們官方的 agent SDK 沒有一個 active + Apache-licensed 的 canonical repo。 + +**Pitch angle (邀請式，不是推銷式)**: 我們是中文 agentic AI 的 8 階段學習地圖；§11 已經收了 Qwen-Agent + Coze，缺 Zhipu。**邀請他們 PR 一個官方推薦的 Zhipu agent 教材 / SDK / cookbook**——他們在我們 catalog 站 §11 的位置，他們的開發者透過我們學 Zhipu 之外的整個 agentic 生態。 + +**Their counter-value**: 中文圈的 entrant developers 有結構化路線可學；GLM/AutoGLM 的曝光。 + +--- + +## Variant 1 — Social post (Weibo / 知乎 / X，~280 字) + +``` +中文 agentic AI 學習地圖 awesome-agentic-ai-zh，§11 中文圈專用 已經收 @QwenLM 跟 +@coze-dev——缺 @zhipuai 的 entry。 + +如果智譜的同學 / 熱心開發者覺得有官方推薦的 GLM agent cookbook / SDK 該收進來， +歡迎 PR：github.com/WenyuChiou/awesome-agentic-ai-zh + +240+ projects · 三語齊全 · MIT · ★525 +``` + +## Variant 2 — Discussion / 知乎文章 (200-300 字) + +**Title**: 邀請：智譜 GLM agent 生態的官方 / 社群推薦項目，可以收進 awesome-agentic-ai-zh §11 嗎？ + +**Body**: + +```markdown +你好智譜社群， + +我维护 [awesome-agentic-ai-zh](https://github.com/WenyuChiou/awesome-agentic-ai-zh) +——一份中文 agentic AI 的 8 阶段三语学习地图（zh-TW canonical / zh-Hans / en，240+ +projects，MIT，★525 第一周）。 + +**§11 中文圈专用** 已经收了： +- QwenLM/Qwen-Agent（阿里巴巴） +- coze-dev/coze-studio + coze-loop（字节跳动） +- chatchat-space/Langchain-Chatchat +- liaokongVFX/LangChain-Chinese-Getting-Started-Guide + +**目前缺智谱 GLM 生态的 entry**——之前评估过 MetaGLM/glm-cookbook 但 246 天没更新， +不太符合 catalog 的「活跃维护」收录原则。 + +想问问智谱社群： +1. 有没有官方维护的 GLM agent SDK / cookbook / Skills 仓库可以推荐？ +2. 如果有合适的项目，欢迎大家直接 PR 到 §11。我们的收录原则在 + [resources/style-guide.md](https://github.com/WenyuChiou/awesome-agentic-ai-zh/blob/main/resources/style-guide.md) +3. 评估标准：MCP / Skills / agent-shaped、license 清楚（避免无 license）、最近 90 + 天有 commit、品质优于流行度（star 数不是门槛） + +如果一时没合适项目也 OK——agent 生态发展很快，先开个 thread 留言追踪。 + +— Wenyu (Lehigh CEE PhD candidate，个人 maintainer) +``` + +## Variant 3 — DM / 邮件 (150 字) + +``` +你好智谱社群， + +我是 awesome-agentic-ai-zh 的维护者 Wenyu。这份是中文 agentic AI 的 8 阶段三语 +学习地图（240+ projects，三语齐全，MIT，★525 第一周）。 + +§11 中文圈专用 已经收阿里 Qwen-Agent + 字节 Coze，缺智谱的 entry。如果有官方推荐 +的 GLM agent SDK / cookbook 我应该收进来，请告诉我；或者直接 PR 到 §11。收录原则 +在 resources/style-guide.md（agent-shaped + license 清楚 + 最近活跃）。 + +谢谢！ +— Wenyu +``` + +--- + +## Notes + +- **Tone**: 邀請式（"Want to be listed? Send PR"），不是 sales（"買我們服務"） +- 避免提 "★ 525" 太多次——對 ★1k+ partners 是小數字 +- 如果 Zhipu 沒有合適 repo，**不要硬 fit**——保留 §11 的品質 +- 如果有合適 PR 進來，注意 license 跟維護節奏（避免 archived / 246d-stale 的 MetaGLM/glm-cookbook 重蹈覆轍） +- Zhipu 中國大陸主要使用 知乎 + 微信 + 釘釘，GitHub Issues 是次要 channel diff --git a/.github/workflows/anchor-validator.yml b/.github/workflows/anchor-validator.yml new file mode 100644 index 0000000..6794b5b --- /dev/null +++ b/.github/workflows/anchor-validator.yml @@ -0,0 +1,36 @@ +name: Anchor Validator + +# 內部 markdown anchor 連結驗證——避免 stage 之間 cross-ref 在 rename / cleanup 後破掉。 +# 本 session 已遇過 3 次破 anchor（Stage 7 cleanup / Stage 5.6 rename / tracks/cli/A3）、 +# 因此把 anchor validate 升為 CI-level check。 + +on: + pull_request: + paths: + - '**.md' + schedule: + # 每月 1 號 UTC 04:00 跑（避開 lint.yml UTC 03:00） + - cron: '0 4 1 * *' + workflow_dispatch: + +permissions: + contents: read # explicit minimal permission (read-only repo content) + +jobs: + anchors: + name: Validate internal cross-file anchors + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + + - name: Setup Python + uses: actions/setup-python@v6 + with: + python-version: '3.11' + + - name: Validate anchors (strict mode) + # 2026-05 升級到 strict mode（commit 含 anchor cleanup batch、把 legacy 37 個 + # broken anchor 全修完）。剩下 1 個 .github/ISSUE_TEMPLATE/project-suggestion.md + # 找不到 CONTRIBUTING.md 是 separate issue（missing file、非 anchor 問題）、 + # script 已不算 broken anchor。 + run: python scripts/check-anchors.py --strict diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml new file mode 100644 index 0000000..c500da2 --- /dev/null +++ b/.github/workflows/docs.yml @@ -0,0 +1,109 @@ +name: Site (mkdocs + mdBook) → GitHub Pages + +# SINGLE owner of the GitHub Pages deployment. GitHub Pages has exactly +# one root per repo, so the mkdocs site (root /) and the mdBook (/book/) +# MUST be built and published by ONE workflow — two workflows both +# calling actions/deploy-pages race and clobber each other (that is +# exactly what happened when the old deploy-book.yml coexisted with +# this one; deploy-book.yml has been removed). +# +# Layout published: +# / → mkdocs-material trilingual site (canonical homepage) +# /en/ /zh-Hans/ → mkdocs locale builds (mkdocs-static-i18n) +# /book/ → mdBook (zh-TW long-form "book" packaging) +# +# Pages source must be "GitHub Actions" (Settings → Pages → Source). + +on: + push: + branches: [main] + paths: + - 'stages/**' + - 'tracks/**' + - 'branches/**' + - 'resources/**' + - 'walkthroughs/**' + - 'docs/**' + - 'examples/**' + - '*.md' + - 'mkdocs.yml' + - 'requirements-docs.txt' + - 'scripts/build-docs-tree.py' + - 'scripts/build-mdbook.sh' + - 'scripts/mkdocs_hooks.py' + - 'book.toml' + - '.github/workflows/docs.yml' + workflow_dispatch: + +concurrency: + group: pages + cancel-in-progress: false + +jobs: + build: + name: Build mkdocs (root) + mdBook (/book/) + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - uses: actions/checkout@v6 + + - name: Setup Python + uses: actions/setup-python@v6 + with: + python-version: '3.12' + + - name: Install docs dependencies + run: pip install -r requirements-docs.txt + + - name: Stage content tree + run: python scripts/build-docs-tree.py + + - name: Build mkdocs site (→ _build/site, the Pages root) + run: python -m mkdocs build + + - name: Install mdBook + run: | + MDBOOK_VERSION="0.4.52" + curl -sSL "https://github.com/rust-lang/mdBook/releases/download/v${MDBOOK_VERSION}/mdbook-v${MDBOOK_VERSION}-x86_64-unknown-linux-gnu.tar.gz" | tar -xz + chmod +x mdbook + echo "$PWD" >> "$GITHUB_PATH" + + - name: Build mdBook (subpath base-url → /book/) + # MDBOOK_OUTPUT__HTML__SITE_URL overrides book.toml's site-url + # (which is "/awesome-agentic-ai-zh/" for a root deploy) so every + # mdBook asset/link resolves under the /book/ subpath. No + # book.toml edit needed — env override is non-invasive. + env: + MDBOOK_OUTPUT__HTML__SITE_URL: /awesome-agentic-ai-zh/book/ + run: bash scripts/build-mdbook.sh + + - name: Merge mdBook into the mkdocs site under /book/ + run: | + set -euo pipefail + test -f _build/site/index.html # mkdocs root must exist + test -f book/dist/index.html # mdBook must have built + mkdir -p _build/site/book + cp -r book/dist/. _build/site/book/ + test -f _build/site/book/index.html + echo "merged: $(find _build/site/book -type f | wc -l) mdBook files under /book/" + + - name: Upload Pages artifact + uses: actions/upload-pages-artifact@v3 + with: + path: _build/site + + deploy: + name: Deploy to GitHub Pages + needs: build + runs-on: ubuntu-latest + permissions: + pages: write + id-token: write + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} + steps: + - name: Deploy + id: deployment + uses: actions/deploy-pages@v4 diff --git a/.github/workflows/freshness-2026.yml b/.github/workflows/freshness-2026.yml new file mode 100644 index 0000000..b87efb6 --- /dev/null +++ b/.github/workflows/freshness-2026.yml @@ -0,0 +1,46 @@ +name: 2026 Freshness Check + +# 每月掃 .md 找 stale model references (Claude 3.5 / GPT-4o / Gemini 2.0 / etc.) +# without proper '前身 / 歷史 / lineage' qualifier. 不在 PR 跑（會 noise 太多、可能誤判）。 +# +# Whitelist + stale patterns 維護在 scripts/freshness-models.yml（quarterly review）。 + +on: + schedule: + # 每月 1 號 UTC 05:00（避開 lint.yml 03:00 + anchor-validator 04:00） + - cron: '0 5 1 * *' + workflow_dispatch: + inputs: + strict: + description: 'Run in strict mode (fail CI on stale)' + required: false + default: 'false' + type: choice + options: ['false', 'true'] + +permissions: + contents: read # explicit minimal permission + +jobs: + freshness: + name: Detect stale model references + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + + - name: Setup Python + uses: actions/setup-python@v6 + with: + python-version: '3.11' + + - name: Install dependencies + run: pip install pyyaml + + - name: Scan for stale model refs + run: | + # Default: warn-only on schedule; manual dispatch can choose strict + if [ "${{ github.event.inputs.strict }}" = "true" ]; then + python scripts/check-2026-freshness.py + else + python scripts/check-2026-freshness.py --warn-only + fi diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml new file mode 100644 index 0000000..4cee533 --- /dev/null +++ b/.github/workflows/lint.yml @@ -0,0 +1,194 @@ +name: Lint (link rot + banned words + schema) + +on: + pull_request: + paths: + - '**.md' + - 'scripts/**' + - 'resources/**' + schedule: + # 每月 1 號 UTC 03:00 跑完整檢查（catch link rot + star drift） + - cron: '0 3 1 * *' + workflow_dispatch: + +jobs: + banned-words: + name: Banned-word audit (zh-Hans slips + overclaim phrases) + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + + - name: Check zh-Hans slips in zh-TW canonical files + run: | + # 只掃 zh-TW canonical 檔。 + # - --exclude="*.en.md" skip 英文 companion + # - --exclude="*.zh-Hans.md" skip 簡中 companion（裡面合法包含這些詞） + # resources/style-guide.md 跟 .github/ 是 policy doc，必然包含禁用詞做為 + # 「不要這樣寫」的範例，所以這個 step 也不掃 resources/。 + BANNED=("教程" "視頻" "軟件" "代碼" "用戶" "網絡" "默認" "函数" "演算法" "算法" "程序") + FOUND=0 + for word in "${BANNED[@]}"; do + if grep -rn --include="*.md" --exclude="*.en.md" --exclude="*.zh-Hans.md" \ + -F "$word" stages branches walkthroughs README.md CONTRIBUTING.md CONTRIBUTORS.md 2>/dev/null \ + | grep -v "(zh-Hans)" \ + | grep -v "zh-Hans repo" \ + | grep -v "中文（zh-Hans）"; then + echo "❌ Banned word found: $word" + FOUND=$((FOUND+1)) + fi + done + if [ $FOUND -gt 0 ]; then + echo "" + echo "Found $FOUND banned-word violations. See resources/style-guide.md §3." + exit 1 + fi + echo "✓ No banned words detected." + + - name: Check zh-TW residue in zh-Hans files (warning only) + run: | + # 反向檢查：zh-Hans 檔混入 zh-TW 用詞代表翻譯沒翻完。 + # **僅 warning 模式**——不擋 PR、不要求完美。zh-Hans 翻譯是漸進的， + # 殘留是常態；這個 check 只是提示哪幾個詞還可以順手修。 + # 不掃：style-guide / glossary（會引用對照詞當例子）、.ai/、book/。 + BANNED_TW=("使用者" "軟體" "硬體" "資訊" "品質" "資料" "專案" "腳本" "預設" "影片" "演算法" "應用程式" "網路" "連結" "服務" "範例" "範本" "設定" "註解" "終端機" "命令列" "編輯器" "瀏覽器" "客戶端" "伺服器" "倉儲" "必修閱讀" "飛書") + FOUND=0 + for word in "${BANNED_TW[@]}"; do + if grep -rn --include="*.zh-Hans.md" \ + -F "$word" . 2>/dev/null \ + | grep -v "^./resources/style-guide.zh-Hans.md" \ + | grep -v "^./resources/glossary.zh-Hans.md" \ + | grep -v "^./.ai/" \ + | grep -v "^./book/"; then + echo "::warning::zh-TW word found in zh-Hans file: $word" + FOUND=$((FOUND+1)) + fi + done + if [ $FOUND -gt 0 ]; then + echo "Note: $FOUND zh-TW residue word(s) detected in .zh-Hans.md. See .ai/zh-Hans_glossary.md. Not blocking the PR — fix when convenient." + else + echo "✓ No zh-TW residue." + fi + + - name: Check overclaim phrases (strict — blocking) + run: | + # 掃公開展示的內容檔（zh + en + zh-Hans）找 style-guide §3 列為禁用的 overclaim 用語。 + # 範圍涵蓋 stages/ branches/ walkthroughs/ tracks/ examples/ resources/（含 + # mcp-skills-catalog / cookbook / cli-agents-guide 等實際被讀者讀到的文件）+ 各 + # top-level 文件。 + # + # 路徑 hygiene： + # - `.github/` 整個資料夾不在掃描 paths（內含 outreach drafts 跟「Don't say...」 + # 明示禁用清單，本來就會引用禁用詞、不該被掃）。 + # - `resources/style-guide.md` + `resources/glossary.md` 透過 grep -v 後過濾 + # （它們在 resources/ 下、必然被 recurse 到，且為了當 policy 例子會列禁用詞）。 + # + # 用 `grep -Fi`（fixed string + case-insensitive）—— 2026-05-26 incident：上一輪 + # sweep 用 -F 是 case-sensitive、漏掉 examples/ 內 H2 標題的 `Production-grade` + # 大寫變體 5 處。這次升級為 -Fi。 + # + # Warn-only step history：原本有第二個 step 掃 `最完整的` 之類的灰色地帶用語、 + # 用 ::warning:: 不擋 PR。實測 corpus 有 30+ hit（多半是稱讚 Datawhale Hello-Agents + # 之類他人專案的 citation framing、非 self-promo overclaim），噪音大於價值、移除。 + # 如未來要為其他灰色詞加 warn 機制，pattern 可參考 git log 找 2026-05-26 此 step + # 的初版實作（drafted but removed pre-merge）。 + OVERCLAIMS=( + # === existing canonical bans === + "the most canonical" + "全世界最好的" + "業界最強" + "最緊迫" + # === style-guide §3 expansions (2026-05-26, P3-G from audit) === + "首選" # zh-TW "top choice" + "首选" # zh-Hans variant + "唯一選擇" # zh-TW "only choice" + "唯一选择" # zh-Hans + "業界最佳" # zh-TW "industry best" (sibling of 業界最強) + "业界最佳" # zh-Hans + # === audit P3-G English overclaim (case-insensitive matches all variants) === + "production-grade" # supersedes the older exact "Production-grade Chinese tutorial" + "world-class" + "best-in-class" + "cutting-edge" + "state-of-the-art" + "industry-leading" + ) + FOUND=0 + for phrase in "${OVERCLAIMS[@]}"; do + hits=$(grep -rIn --include="*.md" -Fi "$phrase" \ + stages branches walkthroughs tracks examples resources \ + README.md README.en.md README.zh-Hans.md \ + CONTRIBUTING.md CONTRIBUTING.en.md CONTRIBUTING.zh-Hans.md \ + CONTRIBUTORS.md \ + ROADMAP.md ROADMAP.en.md ROADMAP.zh-Hans.md \ + CAPSTONE.md CAPSTONE.en.md CAPSTONE.zh-Hans.md \ + PROGRESS.md PROGRESS.en.md PROGRESS.zh-Hans.md \ + RESOURCES.md RESOURCES.en.md RESOURCES.zh-Hans.md \ + 2>/dev/null \ + | grep -v "^resources/style-guide" \ + | grep -v "^resources/glossary" \ + || true) + if [ -n "$hits" ]; then + echo "❌ Overclaim phrase found: $phrase" + echo "$hits" + FOUND=$((FOUND+1)) + fi + done + if [ $FOUND -gt 0 ]; then + echo "" + echo "Found $FOUND overclaim category/categories. See resources/style-guide.md §3 'Overclaim 用語禁用'." + exit 1 + fi + echo "✓ No overclaim phrases detected." + + - name: Check zh-Hans mainland localization (blocking) + run: | + # Blocking gate: zh-Hans mirrors must stay mainland-localized. + # Catches future tw2s-only mirror syncs / hand edits that + # reintroduce Taiwan vocab (呼叫/程式/品质…) or 「」 quotes. + # stdlib-only script; ubuntu-latest ships python3 (no pip deps). + python3 scripts/zh-hans-localize.py --check + + link-rot: + name: Link rot check (GitHub URLs) + runs-on: ubuntu-latest + # 只在 schedule 跟 workflow_dispatch 跑——PR 跑會太慢 + if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' + steps: + - uses: actions/checkout@v6 + + - name: Setup Python + uses: actions/setup-python@v6 + with: + python-version: '3.11' + + - name: Install requests + run: pip install requests + + - name: Run link-check (--fast = GitHub URLs only) + run: python scripts/check-links.py --fast --quiet + + star-drift: + name: Star drift detection + runs-on: ubuntu-latest + if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' + steps: + - uses: actions/checkout@v6 + + - name: Setup Python + uses: actions/setup-python@v6 + with: + python-version: '3.11' + + - name: Install requests + run: pip install requests + + - name: Run refresh-stars (warn only, don't fail CI) + # gh CLI 已內建在 ubuntu-latest runner，GH_TOKEN 自動 set + # 高門檻 (50%) 才警告，避免每月誤報 + run: | + # 用 --check 才會在 drift 時退 1；用 || 接 ::warning:: 退 0 + if ! python scripts/refresh-stars.py --threshold 50 --check; then + echo "::warning::Star drift detected (>=50%). 跑 'python scripts/refresh-stars.py' 看哪些 entry 要更新。" + fi + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/mirror-sync-reminder.yml b/.github/workflows/mirror-sync-reminder.yml new file mode 100644 index 0000000..cb6d1b0 --- /dev/null +++ b/.github/workflows/mirror-sync-reminder.yml @@ -0,0 +1,57 @@ +name: Mirror Sync Reminder + +# 當 PR 改了 zh-TW canonical 但沒同步 .en.md / .zh-Hans.md mirror 時、 +# 自動在 PR 留一個 soft comment 提醒。不擋 PR——zh-TW 是 canonical、mirror sync 是 Path B。 + +on: + pull_request: + paths: + - 'stages/**.md' + - 'branches/**.md' + - 'tracks/**.md' + - 'resources/**.md' + - 'walkthroughs/**.md' + - 'README.md' + - 'CONTRIBUTING.md' + +jobs: + mirror-check: + name: Detect mirror sync gap + runs-on: ubuntu-latest + permissions: + pull-requests: write + steps: + - uses: actions/checkout@v6 + with: + # Need full history for accurate diff against base ref + fetch-depth: 0 + + - name: Setup Python + uses: actions/setup-python@v6 + with: + python-version: '3.11' + + - name: Detect mirror sync gap + id: detect + run: | + python scripts/check-mirror-sync.py \ + --pr-base origin/${{ github.base_ref }} + + - name: Comment on PR if gap detected + if: steps.detect.outputs.gap_detected == 'true' + # Pinned to v3.0.4 (specific release tag) — supply chain risk mitigation + # vs floating @v3. Upgrade by reviewing release notes + bumping tag. + # For tighter security (SHA pin), see: https://github.com/marocchino/sticky-pull-request-comment/releases + uses: marocchino/sticky-pull-request-comment@v3.0.4 + with: + # `header` makes the comment sticky — re-running the workflow + # updates the same comment instead of creating duplicates. + header: mirror-sync + path: .mirror-sync-comment.md + + - name: Delete comment if gap resolved + if: steps.detect.outputs.gap_detected == 'false' + uses: marocchino/sticky-pull-request-comment@v3.0.4 + with: + header: mirror-sync + delete: true diff --git a/.github/workflows/stage-template-check.yml b/.github/workflows/stage-template-check.yml new file mode 100644 index 0000000..89b4c8b --- /dev/null +++ b/.github/workflows/stage-template-check.yml @@ -0,0 +1,28 @@ +name: Stage Template Check + +# 驗 stages/*.md 有所有必要 H2 section（template 對齊）。 +# REQUIRED 缺則 fail PR、EXPECTED 缺則 warning（不擋）。 + +on: + pull_request: + paths: + - 'stages/**.md' + workflow_dispatch: + +permissions: + contents: read # explicit minimal permission + +jobs: + template: + name: Validate stage template structure + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + + - name: Setup Python + uses: actions/setup-python@v6 + with: + python-version: '3.11' + + - name: Validate stage template + run: python scripts/check-stage-template.py diff --git a/.github/workflows/weekly-catalog-refresh.yml b/.github/workflows/weekly-catalog-refresh.yml new file mode 100644 index 0000000..0ef3aa9 --- /dev/null +++ b/.github/workflows/weekly-catalog-refresh.yml @@ -0,0 +1,298 @@ +name: Weekly Catalog Refresh + +# 每週一 04:00 UTC（12:00 台北）刷新 GitHub star 數 + 檢查連結失效。 +# - star 數有變化 → 開 PR；若通過 sanity guard 則「自動 squash-merge」， +# maintainer 不需每週手動合。guard 不過 → PR 留開、上 needs-manual-review +# label + 貼原因,只有「異常週」才需要人看。 +# - broken link → 開 issue 讓 maintainer 處理（不自動修）。 +# +# 為何要 guard 而非裸 auto-merge：peter-evans 用 GITHUB_TOKEN 開的 PR +# GitHub 設計上不觸發 CI,所以 bot PR 沒有自動安全網。歷史上 PR-gate +# 攔過實際 bug（#17 漏 star-refresh.log、★ 空格被改壞）。此 inline guard +# 在 workflow 自己的 runner 跑,補上那個洞。 +# +# Guard 會擋下(→ 改人工 review,不自動合): +# - 刪檔 / 改名 / 非 .md 路徑變動 / log 檔混入 +# - 任何「★ 星數 token 以外」的內容被改(star-stripped diff 不一致) +# - 寫出 ★ 0(repo 多半被 archive/設私有,或 API 異常) +# - 變更行數 > 150 或變更檔數 > 40(疑似 mass API 異常 / runaway) +# - check-anchors --strict 壞掉 +# 註:auto-merge 需 main 上「沒有強制 required-review 的 branch protection」 +# (本 repo 單人維護、bot PR 無 CI,符合此前提)。若日後加了強制 review, +# auto-merge 會優雅地降級成「留 PR 等人工」,不會誤合。 +# +# 跟其他 cron 錯開：lint 03:00 daily / freshness 05:00 monthly / +# weekly-refresh 04:00 Mon（本檔）。 +# 手動觸發：Actions → Weekly Catalog Refresh → Run workflow。 +# 環境需求：GITHUB_TOKEN（auto-provided）。 + +on: + schedule: + - cron: '0 4 * * 1' + workflow_dispatch: + inputs: + threshold: + description: 'Star drift threshold (%) — only PR if any entry drifted more than this' + required: false + default: '10' + +permissions: + contents: write # create-pull-request 寫 branch + 自動 merge commit + pull-requests: write # 開 PR / gh pr merge / edit / comment + issues: write # broken link → open issue + +jobs: + star-refresh: + name: Refresh GitHub star counts + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + + - name: Setup Python + uses: actions/setup-python@v6 + with: + python-version: '3.11' + + - name: Install dependencies + run: pip install requests + + - name: Refresh star counts (apply mode) + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + # star-refresh.log 已在 .gitignore（PR #17 leak fix, ae76b7d） + python scripts/refresh-stars.py \ + --threshold "${{ github.event.inputs.threshold || '10' }}" \ + --apply 2>&1 | tee star-refresh.log + if git diff --quiet; then + echo "drift=false" >> "$GITHUB_OUTPUT" + echo "No star drift detected this week." + else + echo "drift=true" >> "$GITHUB_OUTPUT" + echo "Star drift detected." + git diff --stat + fi + id: refresh + + - name: Sanity guard (decides auto-merge vs manual review) + # 只在有 drift 時跑。本步驟永遠 exit 0(guard 不過 = 走人工 + # review 分支,不是 workflow 失敗)。結果寫 steps.guard.outputs.guard。 + if: steps.refresh.outputs.drift == 'true' + id: guard + run: | + set -uo pipefail + TMP="$(mktemp -d)" + fail() { + echo "guard=fail" >> "$GITHUB_OUTPUT" + { + echo "reason<> "$GITHUB_OUTPUT" + echo "::warning::GUARD FAIL — $1 (PR stays open for manual review)" + exit 0 + } + porc="$(git status --porcelain)" + echo "--- git status --porcelain ---"; echo "$porc" + + # (a) no deletions / renames + if echo "$porc" | grep -qE '^[ ]?D|^D[ ]'; then fail "a tracked file was deleted"; fi + if echo "$porc" | grep -qE '^R'; then fail "a tracked file was renamed"; fi + + # (b) every changed path must be a .md content file; no leak files + while IFS= read -r line; do + [ -z "$line" ] && continue + path="${line:3}" + case "$path" in + *.md) : ;; + *) fail "unexpected non-.md path changed: $path" ;; + esac + done <<< "$porc" + if echo "$porc" | grep -qE 'star-refresh\.log|link-report\.txt'; then + fail "diagnostic log file is staged" + fi + + # (c) ONLY the ★ star-count token may differ. Strip ★N tokens from + # removed + added line bodies; the sorted multisets must be + # identical. Any prose/structural change makes them differ. + # (Kills the "line merely contains ★ somewhere" bypass.) + STAR_RE='★ ?[0-9]+(\.[0-9]+)?[kKmM]?[+]?' + # `|| true`: under pipefail, grep exiting 1 on no-match must not + # red the step (degenerate drift=true / no ±lines) — keep going, + # an empty file just makes diff -q identical (→ guard=pass, which + # is correct: nothing substantive changed). + git diff -- '*.md' | grep -E '^-' | grep -vE '^---' | sed 's/^-//' | sed -E "s/${STAR_RE}//g" | sort > "$TMP/old" || true + git diff -- '*.md' | grep -E '^\+' | grep -vE '^\+\+\+' | sed 's/^+//' | sed -E "s/${STAR_RE}//g" | sort > "$TMP/new" || true + if ! diff -q "$TMP/old" "$TMP/new" >/dev/null; then + echo "--- non-★ content changed (star-stripped diff) ---" + diff "$TMP/old" "$TMP/new" | head -20 + fail "diff changed non-★ content (star-stripped line bodies differ)" + fi + + # (d) no zero/garbage star written (archived/private/API-error → ★ 0) + if git diff -- '*.md' | grep -E '^\+' | grep -vE '^\+\+\+' | grep -qE '★ 0([^0-9]|$)'; then + fail "a new ★ value is 0 (repo likely archived / private / API error)" + fi + + # (e) magnitude + file-count bounds (mass-zeroing / runaway rewrite) + changed="$(git diff --numstat | awk '{a+=$1; d+=$2} END{print a+d+0}')" + if [ "${changed:-0}" -gt 150 ]; then + fail "change magnitude ${changed} lines > 150 (suspicious for a star refresh)" + fi + repos="$(git diff --name-only -- '*.md' | wc -l | tr -d ' ')" + if [ "${repos:-0}" -gt 40 ]; then + fail "${repos} files changed > 40 (possible mass API anomaly)" + fi + + # (f) anchors still valid after the rewrite + if ! python scripts/check-anchors.py --strict; then + fail "check-anchors --strict failed after refresh" + fi + + echo "guard=pass" >> "$GITHUB_OUTPUT" + echo "GUARD PASS — only ★ annotations changed (${changed} lines, ${repos} files); anchors OK" + + - name: Create PR (drift detected) + # guard.outcome=='success' = guard step exited 0 (i.e. guard=pass OR + # guard=fail — both intended). If the guard step itself ERRORED + # (non-zero), outcome=='failure' → no PR is opened, the red job is + # the single clear signal (no orphan unlabeled PR). + if: steps.refresh.outputs.drift == 'true' && steps.guard.outcome == 'success' + id: cpr + uses: peter-evans/create-pull-request@v8 + with: + token: ${{ secrets.GITHUB_TOKEN }} + branch: auto/weekly-star-refresh + delete-branch: true + commit-message: | + chore(catalog): weekly auto-refresh of GitHub star counts + + Generated by .github/workflows/weekly-catalog-refresh.yml + title: "chore(catalog): weekly auto-refresh of GitHub star counts" + body: | + ## Weekly star count refresh + + Auto-generated by `weekly-catalog-refresh.yml` (`refresh-stars.py + --apply`). An in-workflow sanity guard decides what happens next: + + - **guard PASS** → auto-squash-merged (only `★` star annotations + changed; no log/non-`.md`/deleted/renamed files; no `★ 0`; + ≤150 lines / ≤40 files; anchors still valid). No human action. + - **guard FAIL** → this PR stays **open**, labelled + `needs-manual-review`, with a comment naming the failed check. + + ### Skip a week + Just close the PR — next Monday a fresh PR opens with current numbers. + + 🤖 Generated weekly by GitHub Actions + labels: | + automation + catalog-refresh + + - name: Auto-merge (guard passed) + if: >- + steps.refresh.outputs.drift == 'true' && + steps.guard.outputs.guard == 'pass' && + steps.cpr.outputs.pull-request-number + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + PR: ${{ steps.cpr.outputs.pull-request-number }} + run: | + set -uo pipefail + ensure_label() { + gh label create "needs-manual-review" --color "d93f0b" \ + --description "Auto-merge withheld by a guard; needs a human" 2>/dev/null || true + } + echo "Guard passed — waiting for PR #$PR mergeability, then squash-merge" + for i in $(seq 1 12); do + state="$(gh pr view "$PR" --json mergeable --jq .mergeable 2>/dev/null || echo UNKNOWN)" + echo "attempt $i: mergeable=$state" + if [ "$state" = "MERGEABLE" ]; then + gh pr merge "$PR" --squash --delete-branch + gh pr comment "$PR" --body "✅ Auto-merged by sanity guard (guard=pass — only ★ star annotations changed). 🤖 weekly-catalog-refresh" + exit 0 + fi + if [ "$state" = "CONFLICTING" ]; then + ensure_label + gh pr edit "$PR" --add-label "needs-manual-review" || true + gh pr comment "$PR" --body "⚠️ Guard passed but PR is CONFLICTING — manual resolution needed. 🤖 weekly-catalog-refresh" + exit 0 + fi + sleep 5 + done + ensure_label + gh pr edit "$PR" --add-label "needs-manual-review" || true + gh pr comment "$PR" --body "⚠️ Guard passed but GitHub did not confirm mergeability within ~60s — please merge manually. 🤖 weekly-catalog-refresh" + exit 0 + + - name: Flag for manual review (guard failed) + if: >- + steps.refresh.outputs.drift == 'true' && + steps.guard.outputs.guard == 'fail' && + steps.cpr.outputs.pull-request-number + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + PR: ${{ steps.cpr.outputs.pull-request-number }} + REASON: ${{ steps.guard.outputs.reason }} + run: | + set -uo pipefail + gh label create "needs-manual-review" --color "d93f0b" \ + --description "Auto-merge withheld by a guard; needs a human" 2>/dev/null || true + gh pr edit "$PR" --add-label "needs-manual-review" || true + # REASON passed via env (NOT inlined into shell) → no script injection + { + echo '⚠️ **Auto-merge withheld — sanity guard failed.**' + echo + echo 'Reason:' + echo '```' + printf '%s\n' "$REASON" + echo '```' + echo + echo 'Review the `Files changed` tab: if it is genuinely just star-count' + echo 'updates, merge manually; otherwise close it and investigate' + echo '`scripts/refresh-stars.py`.' + echo + echo '🤖 weekly-catalog-refresh guard' + } > "$RUNNER_TEMP/guard-comment.md" + gh pr comment "$PR" --body-file "$RUNNER_TEMP/guard-comment.md" + + link-check: + name: Scan for broken links + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + + - name: Setup Python + uses: actions/setup-python@v6 + with: + python-version: '3.11' + + - name: Install dependencies + run: pip install requests + + - name: Scan links (fast mode — GitHub repos only, weekly cadence) + run: | + python scripts/check-links.py --fast --quiet > link-report.txt 2>&1 || true + echo "--- broken links ---" + cat link-report.txt | head -50 + if grep -qE "FAIL|404|timeout|error" link-report.txt; then + echo "broken=true" >> "$GITHUB_OUTPUT" + else + echo "broken=false" >> "$GITHUB_OUTPUT" + fi + id: linkcheck + + - name: Open issue if broken links found + if: steps.linkcheck.outputs.broken == 'true' + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + existing=$(gh issue list --label "broken-links" --state open --json number --jq 'length') + if [ "$existing" -gt 0 ]; then + echo "An open broken-links issue already exists. Skipping new issue creation." + exit 0 + fi + gh issue create \ + --title "weekly link check: broken links detected ($(date -u +%Y-%m-%d))" \ + --label "broken-links,automation" \ + --body "$(printf '## Broken link report\n\n`check-links.py --fast` flagged the following URLs. Verify, then either fix the link, replace with a working alternative, or remove if the resource is genuinely gone.\n\n```\n%s\n```\n\n🤖 Generated weekly by GitHub Actions. To suppress next week, fix the listed links — next Monday the scan reruns.' "$(cat link-report.txt | head -100)")" diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..fb06ec7 --- /dev/null +++ b/.gitignore @@ -0,0 +1,34 @@ +# OS +.DS_Store +Thumbs.db + +# Editor +.vscode/ +.idea/ +*.swp + +# Local +.env +.cache/ + +# Generated +_build/ +node_modules/ +.ai/ +__pycache__/ + +# Phase 5 build artifacts (mdBook + PDF) +book/dist/ +book/src/ +dist/ +mermaid.min.js +mermaid-init.js + +# SVG artifacts from earlier cairosvg attempts (PIL is the canonical generator now) +resources/diagrams/stage5-stack*.svg + +# CI workflow scratch files (mirror sync reminder generates this temporarily) +.mirror-sync-comment.md +# weekly-catalog-refresh.yml diagnostic logs — must NOT leak into the auto-PR +star-refresh.log +link-report.txt diff --git a/CAPSTONE.en.md b/CAPSTONE.en.md new file mode 100644 index 0000000..14a899a --- /dev/null +++ b/CAPSTONE.en.md @@ -0,0 +1,97 @@ +# Capstone + +> [繁體中文](./CAPSTONE.md) | [简体中文](./CAPSTONE.zh-Hans.md) | **English** + +After finishing a track, **build something yourself** — this file is not a tutorial, not a walkthrough, and there is no model answer. Its purpose is to turn "I read the roadmap" into "I have something I can show + a grade I gave myself." + +**How to use this file**: +1. Pick **a problem you actually have** (work, research, daily life). Don't pick a toy problem — a capstone's value comes from being real. +2. Check your track's "Prerequisites" and confirm the required stages have each passed their "Self-check". +3. When done, **self-assess with the matching rubric** (4 levels: Not yet / Basic / Good / Excellent). Scoring honestly is more useful than scoring high. +4. Want feedback? Post the artifact + your self-assessment to [Discussion](https://github.com/WenyuChiou/awesome-agentic-ai-zh/discussions) for peer review (optional, not required). + +> What each stage teaches / what you need before it / how you know you've learned it — all of that stays with the stage file's "Learning objectives / Prerequisites / Self-check". This file only defines the **capstone** itself. + +--- + +## Track A Capstone — CLI Power User + +**Prerequisites**: Stage 0–2 + A1 + A2 + Stage 5 + A3 have each passed their self-check (Stage 8 is a shared hub across both tracks — recommended, but it does not gate capstone entry; the Track A capstone focuses on the CLI workflow). + +**Brief**: Assemble a CLI-agent workflow **you will reuse**, automating something you currently do by hand. + +**Requirements** (all mandatory): +- A CLI agent (Claude Code or equivalent) at the core +- At least **1** MCP server **or** a skill / command you wrote yourself +- A clear input → a usable artifact out (not "chatting with it") +- **Reproducible by someone else**: include `how to run` (install, configure, run, expected output) +- Handles at least **1 failure case** (missing input, API failure, what happens when the result is wrong) + +**Deliverables**: a folder / repo with the artifact + `README` + evidence of at least one real run (log / screenshot / output file) + a reflection under 150 words (where you got stuck, what you'd change next time). + +**Time**: 3–8 hours (not counting time spent learning the stages). + +### Track A scoring rubric (self-assessed, 4 levels) + +| Dimension | Not yet | Basic | Good | Excellent | +|---|---|---|---|---| +| Problem realism | Toy problem | Somewhat useful | Real, will reuse | Real, with a quantifiable metric (time saved / fewer errors etc.) | +| Tool use | Plain chat only | Used a CLI agent | + MCP or self-written skill/command | Multi-component, with justified choices | +| Reproducibility | Only runs for you | Steps incomplete | Others can run it from the README | One-click / fully automated with pre-checks | +| Robustness | Crashes on any error | Risks mentioned | Handles 1 failure | Multiple failures have fallbacks | +| Docs & reflection | None | Has a README | Clear README + reflection | Reflection points to a concrete next improvement | + +--- + +## Track B Capstone — Agent Builder + +**Prerequisites**: Stage 0–8 (including Stage 3, 4, 5, 6, 7, 7.5, 8) have each passed their self-check. + +**Brief**: Design, build, and **evaluate** a small system that solves a concrete problem. Pick one: +- **A. Multi-agent**: ≥ 2 cooperating agents with orchestration logic; or +- **B. RAG system**: a complete retrieval + generation pipeline. + +**Requirements** (all mandatory): +- Has tool use +- Has one outward interface (CLI / API / chat — any one, mapping to Stage 8) +- **Has explicit evaluation**: define ≥ 5 test cases yourself + measure a pass rate / qualitative assessment (this one is non-negotiable — the thing this curriculum most often skips is "verification") +- **Failure-mode analysis**: write down under what conditions it breaks and how you'd know +- An architecture sketch (a diagram or a paragraph describing components and data flow) + +**Deliverables**: a code repo + an architecture description + evaluation results (even just an N-cases / pass-rate table) + `README` + a reflection under 200 words (where the architecture call was wrong, what you'd do differently). + +**Time**: 8–20 hours (not counting time spent learning the stages). + +### Track B scoring rubric (self-assessed, 4 levels) + +| Dimension | Not yet | Basic | Good | Excellent | +|---|---|---|---|---| +| Problem definition | Vague | Has a goal | Clear scope, acceptance-checkable | Clear, and states why it's worth doing | +| Architecture | No design | Just runs | Justified multi-agent / RAG choice | Can articulate the trade-offs | +| Implementation correctness | Doesn't run | Main path runs | Handles edge cases | Stable and the code is readable | +| **Evaluation rigor** | Not tested | Tried it a few times by hand | ≥5 cases + pass rate | Has a baseline comparison / a rerunnable regression | +| Robustness & failure analysis | None | Risks mentioned | Concrete failure modes | Failures are detected + mitigated | +| Interface & docs | None | Runs | Interface + clear README | Others can use it directly | +| Reflection | None | One sentence | Concrete (names a specific problem in the architecture or component choice) | Points to an architecture-level next step | + +--- + +## Pick a brief by role (audience flavor) + +Same capstone — just swap in your scenario; no need to do a separate one: + +- 🔬 **researcher**: literature Q&A / experiment-log organization / data-preprocessing agent +- 💻 **developer**: a review/triage agent inside CI / repo Q&A / automated release notes +- 🎓 **teacher**: question generation + grading support / material rewriting / course Q&A (mind the academic-integrity boundary) +- 📊 **knowledge-worker**: meeting notes → action items / cross-document synthesis / first-draft weekly report +- 👥 **everyday-user**: personal-data consolidation / scheduling and reminders / repetitive-chore automation + +--- + +## How to show your work + +- Post to the matching [Discussion](https://github.com/WenyuChiou/awesome-agentic-ai-zh/discussions) category with a link to the artifact + your rubric self-assessment. +- Put it in your own portfolio / GitHub; describe it with concrete facts (what you did, what you measured), and **avoid** hype like "the strongest / the best in the world". +- Want to review someone else's: give feedback against that person's track rubric — about the work, not the person. + +> This file only defines the capstone and its scoring. Each stage's learning content and pass conditions stay with that stage's file. diff --git a/CAPSTONE.md b/CAPSTONE.md new file mode 100644 index 0000000..5bdcc7e --- /dev/null +++ b/CAPSTONE.md @@ -0,0 +1,97 @@ +# 結業專題 / Capstone + +> **繁體中文** | [简体中文](./CAPSTONE.zh-Hans.md) | [English](./CAPSTONE.en.md) + +走完一條軌道後,**自己做一個東西出來**——這份檔案不是教學、不是 walkthrough,沒有標準答案。它的用途是把「我讀完 roadmap」變成「我有一個能展示的作品 + 一份自己給自己的評分」。 + +**怎麼用這份檔案**: +1. 選你**真的有的一個問題**(工作上、研究上、生活上),別挑玩具題目——capstone 的價值來自真實。 +2. 對照你那條軌道的「進入條件」,確認該完成的 stage 都過了它的「自我檢查」。 +3. 做完後,用對應的 **rubric 自評**(四級:未達 / 基本 / 良好 / 優秀)。誠實打分比分數高更有用。 +4. 想要回饋?把成品 + 自評貼到 [Discussion](https://github.com/WenyuChiou/awesome-agentic-ai-zh/discussions) 找人 peer review(可選,不強制)。 + +> 每一站「學什麼 / 進入前要會什麼 / 怎麼算學會」一律以該 stage 檔案內的「學習目標 / 進入條件 / 自我檢查」為準;這份檔案只定義**結業專題**本身。 + +--- + +## Track A Capstone — CLI Power User + +**進入條件**:Stage 0–2 + A1 + A2 + Stage 5 + A3 都過了各自的自我檢查(Stage 8 是兩軌共用 hub,建議完成、但不擋 capstone 入場——Track A capstone 聚焦 CLI 工作流)。 + +**題目**:組一條**你會重複用**的 CLI-agent 工作流,把一件你現在手動做的事自動化。 + +**必要條件**(缺一不可): +- 用一個 CLI agent(Claude Code 或同類)當核心 +- 至少接 **1 個** MCP server **或** 自寫的 skill / command +- 有明確輸入 → 產出可用的成品(不是「跟它聊天」) +- **能被別人重跑**:附 `怎麼跑`(安裝、設定、執行、預期輸出) +- 處理至少 **1 種失敗情況**(輸入缺、API 失敗、結果不對時會怎樣) + +**交付物**:一個資料夾 / repo,含成品 + `README` + 至少一次真實執行的證據(log / 截圖 / 產出檔)+ 150 字內的反思(哪裡卡、下次怎麼改)。 + +**時間**:3–8 小時(不含學 stage 的時間)。 + +### Track A 評分 rubric(自評,四級) + +| 面向 | 未達 | 基本 | 良好 | 優秀 | +|---|---|---|---|---| +| 問題真實度 | 玩具題 | 有點用 | 真實、會重複用 | 真實且有可量化指標(省時 / 減錯次數等) | +| 工具運用 | 只用基本對話 | 用了 CLI agent | + MCP 或自寫 skill/command | 多元件協作且選型有理由 | +| 可重現 | 只有自己跑得動 | 步驟不全 | 別人照 README 跑得起來 | 一鍵 / 全自動且有前置檢查 | +| 韌性 | 一出錯就崩 | 有提到風險 | 處理 1 種失敗 | 多種失敗有 fallback | +| 文件與反思 | 無 | 有 README | README 清楚 + 反思 | 反思具體可指出下一步改進 | + +--- + +## Track B Capstone — Agent Builder + +**進入條件**:Stage 0–8(含 Stage 3、4、5、6、7、7.5、8)都過了各自的自我檢查。 + +**題目**:設計、實作、並**評測**一個解決具體問題的小型系統,二選一: +- **A. Multi-agent**:≥ 2 個分工協作的 agent,有編排邏輯;或 +- **B. RAG 系統**:檢索 + 生成的完整管線。 + +**必要條件**(缺一不可): +- 有 tool use +- 有一個對外 interface(CLI / API / chat 任一,對應 Stage 8) +- **有明確 evaluation**:自己定 ≥ 5 個測試案例 + 量出通過率 / 質性評估(這條不可省——這套課最容易跳過的就是「驗證」) +- **失敗模式分析**:寫清楚它在什麼情況會壞、你怎麼知道 +- 架構草圖(一張圖或一段文字說明元件與資料流) + +**交付物**:程式 repo + 架構說明 + evaluation 結果(哪怕只是 N 案例 / 通過率表)+ `README` + 200 字內反思(架構哪裡判斷錯、重來會怎麼改)。 + +**時間**:8–20 小時(不含學 stage 的時間)。 + +### Track B 評分 rubric(自評,四級) + +| 面向 | 未達 | 基本 | 良好 | 優秀 | +|---|---|---|---|---| +| 問題定義 | 模糊 | 有目標 | 範圍清楚、可驗收 | 清楚且說明為何值得做 | +| 架構 | 無設計 | 能跑就好 | multi-agent / RAG 選型有理由 | 設計權衡寫得出來 | +| 實作正確性 | 跑不動 | 主流程能跑 | 邊界情況也處理 | 穩定且程式可讀 | +| **評測嚴謹度** | 沒測 | 手動試幾次 | ≥5 案例 + 通過率 | 有 baseline 對照 / 回歸可重跑 | +| 韌性與失敗分析 | 無 | 提到風險 | 具體失敗模式 | 失敗有偵測 + 緩解 | +| 介面與文件 | 無 | 能跑 | interface + README 清楚 | 別人能直接用 | +| 反思 | 無 | 一句話 | 具體(指得出架構或元件選擇的具體問題) | 指得出架構級的下一步 | + +--- + +## 依身分選題(audience flavor) + +同一個 capstone,換你的場景就好——不用另外做: + +- 🔬 **researcher**:文獻問答 / 實驗 log 整理 / 資料前處理 agent +- 💻 **developer**:CI 內的 review/triage agent / repo 問答 / 自動化 release note +- 🎓 **teacher**:出題 + 評分輔助 / 教材改寫 / 課程問答(注意學術誠信邊界) +- 📊 **knowledge-worker**:會議記錄 → 行動項 / 跨文件彙整 / 週報初稿 +- 👥 **everyday-user**:個人資料彙整 / 行程與提醒 / 重複雜務自動化 + +--- + +## 怎麼展示 + +- 貼到 [Discussion](https://github.com/WenyuChiou/awesome-agentic-ai-zh/discussions) 的對應分類,附成品連結 + 你的 rubric 自評。 +- 放進你自己的 portfolio / GitHub;描述用具體事實(做了什麼、量到什麼),**避免**「最強 / 全世界最好」這類話術。 +- 想替別人 review:照對方那條軌道的 rubric 給回饋,對事不對人。 + +> 這份檔案只定義結業專題與評分標準。各 stage 的學習內容與通過條件,仍以該 stage 檔案為準。 diff --git a/CAPSTONE.zh-Hans.md b/CAPSTONE.zh-Hans.md new file mode 100644 index 0000000..e7daed0 --- /dev/null +++ b/CAPSTONE.zh-Hans.md @@ -0,0 +1,97 @@ +# 结业专题 / Capstone + +> [繁體中文](./CAPSTONE.md) | **简体中文** | [English](./CAPSTONE.en.md) + +走完一条轨道后,**自己做一个东西出来**——这份文件不是教学、不是 walkthrough,没有标准答案。它的用途是把“我读完 roadmap”变成“我有一个能展示的作品 + 一份自己给自己的评分”。 + +**怎么用这份文件**: +1. 选你**真的有的一个问题**(工作上、研究上、生活上),别挑玩具题目——capstone 的价值来自真实。 +2. 对照你那条轨道的“进入条件”,确认该完成的 stage 都过了它的“自我检查”。 +3. 做完后,用对应的 **rubric 自评**(四级:未达 / 基本 / 良好 / 优秀)。诚实打分比分数高更有用。 +4. 想要反馈?把成品 + 自评贴到 [Discussion](https://github.com/WenyuChiou/awesome-agentic-ai-zh/discussions) 找人 peer review(可选,不强制)。 + +> 每一站“学什么 / 进入前要会什么 / 怎么算学会”一律以该 stage 文件内的“学习目标 / 进入条件 / 自我检查”为准;这份文件只定义**结业专题**本身。 + +--- + +## Track A Capstone — CLI Power User + +**进入条件**:Stage 0–2 + A1 + A2 + Stage 5 + A3 都过了各自的自我检查(Stage 8 是两轨共用 hub,建议完成、但不挡 capstone 入场——Track A capstone 聚焦 CLI 工作流)。 + +**题目**:组一条**你会重复用**的 CLI-agent 工作流,把一件你现在手动做的事自动化。 + +**必要条件**(缺一不可): +- 用一个 CLI agent(Claude Code 或同类)当核心 +- 至少接 **1 个** MCP server **或** 自写的 skill / command +- 有明确输入 → 产出可用的成品(不是“跟它聊天”) +- **能被别人重跑**:附 `怎么跑`(安装、设置、执行、预期输出) +- 处理至少 **1 种失败情况**(输入缺、API 失败、结果不对时会怎样) + +**交付物**:一个文件夹 / repo,含成品 + `README` + 至少一次真实执行的证据(log / 截图 / 产出文件)+ 150 字内的反思(哪里卡、下次怎么改)。 + +**时间**:3–8 小时(不含学 stage 的时间)。 + +### Track A 评分 rubric(自评,四级) + +| 维度 | 未达 | 基本 | 良好 | 优秀 | +|---|---|---|---|---| +| 问题真实度 | 玩具题 | 有点用 | 真实、会重复用 | 真实且有可量化指标(省时 / 减错次数等) | +| 工具运用 | 只用基本对话 | 用了 CLI agent | + MCP 或自写 skill/command | 多组件协作且选型有理由 | +| 可重现 | 只有自己跑得动 | 步骤不全 | 别人照 README 跑得起来 | 一键 / 全自动且有前置检查 | +| 韧性 | 一出错就崩 | 有提到风险 | 处理 1 种失败 | 多种失败有 fallback | +| 文件与反思 | 无 | 有 README | README 清楚 + 反思 | 反思具体可指出下一步改进 | + +--- + +## Track B Capstone — Agent Builder + +**进入条件**:Stage 0–8(含 Stage 3、4、5、6、7、7.5、8)都过了各自的自我检查。 + +**题目**:设计、实作、并**评测**一个解决具体问题的小型系统,二选一: +- **A. Multi-agent**:≥ 2 个分工协作的 agent,有编排逻辑;或 +- **B. RAG 系统**:检索 + 生成的完整管线。 + +**必要条件**(缺一不可): +- 有 tool use +- 有一个对外 interface(CLI / API / chat 任一,对应 Stage 8) +- **有明确 evaluation**:自己定 ≥ 5 个测试案例 + 量出通过率 / 质性评估(这条不可省——这套课最容易跳过的就是“验证”) +- **失败模式分析**:写清楚它在什么情况会坏、你怎么知道 +- 架构草图(一张图或一段文字说明组件与数据流) + +**交付物**:程序 repo + 架构说明 + evaluation 结果(哪怕只是 N 案例 / 通过率表)+ `README` + 200 字内反思(架构哪里判断错、重来会怎么改)。 + +**时间**:8–20 小时(不含学 stage 的时间)。 + +### Track B 评分 rubric(自评,四级) + +| 维度 | 未达 | 基本 | 良好 | 优秀 | +|---|---|---|---|---| +| 问题定义 | 模糊 | 有目标 | 范围清楚、可验收 | 清楚且说明为何值得做 | +| 架构 | 无设计 | 能跑就好 | multi-agent / RAG 选型有理由 | 设计权衡写得出来 | +| 实作正确性 | 跑不动 | 主流程能跑 | 边界情况也处理 | 稳定且程序可读 | +| **评测严谨度** | 没测 | 手动试几次 | ≥5 案例 + 通过率 | 有 baseline 对照 / 回归可重跑 | +| 韧性与失败分析 | 无 | 提到风险 | 具体失败模式 | 失败有侦测 + 缓解 | +| 接口与文件 | 无 | 能跑 | interface + README 清楚 | 别人能直接用 | +| 反思 | 无 | 一句话 | 具体(指得出架构或组件选择的具体问题) | 指得出架构级的下一步 | + +--- + +## 依身份选题(audience flavor) + +同一个 capstone,换你的场景就好——不用另外做: + +- 🔬 **researcher**:文献问答 / 实验 log 整理 / 数据预处理 agent +- 💻 **developer**:CI 内的 review/triage agent / repo 问答 / 自动化 release note +- 🎓 **teacher**:出题 + 评分辅助 / 教材改写 / 课程问答(注意学术诚信边界) +- 📊 **knowledge-worker**:会议记录 → 行动项 / 跨文件汇整 / 周报初稿 +- 👥 **everyday-user**:个人数据汇整 / 行程与提醒 / 重复杂务自动化 + +--- + +## 怎么展示 + +- 贴到 [Discussion](https://github.com/WenyuChiou/awesome-agentic-ai-zh/discussions) 的对应分类,附成品链接 + 你的 rubric 自评。 +- 放进你自己的 portfolio / GitHub;描述用具体事实(做了什么、量到什么),**避免**“最强 / 全世界最好”这类话术。 +- 想替别人 review:照对方那条轨道的 rubric 给反馈,对事不对人。 + +> 这份文件只定义结业专题与评分标准。各 stage 的学习内容与通过条件,仍以该 stage 文件为准。 diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..6629e3a --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,198 @@ +# Changelog + +Last 14 days of substantive changes. Older history lives in `git log`. + +Format: `YYYY-MM-DD · category · 1-line summary (commit-sha)`. + +--- + +## 2026-07-13 + +- **content** · **Claude Fable 5 is back** — swept the repo's now-false "suspended / unavailable / use Opus 4.8" caveats. First-party verified (anthropic.com/news/redeploying-fable-5, 2026-06-30): the US export controls were lifted 2026-06-30 and Fable 5 was redeployed globally on 2026-07-01 (Claude Platform / Claude Code / Cowork; API rollout in progress; redeployed with a new safety classifier that blocks the flagged jailbreak and reroutes to Opus 4.8). Mythos 5 restored only for approved US organizations. Updated ~44 mentions across Stage 1 / 6 / 7 / 7.5 / 8, the glossary, examples/README, and the repo CLAUDE.md (tri-locale): suspension caveats → "restored 2026-07-01"; and since Fable 5 (Mythos-class, above Opus) is on top again, the now-false "Opus 4.8 is the current top usable Claude tier" claims were corrected to "Opus-class flagship" (kept only as a past-tense note where it records that Opus was the top usable tier *while Fable was suspended*). Also filled in Fable 5's now-known 1M context in the model pickers. Example pricing code AST-parses clean. Tri-locale; anchor / zh-Hans / switcher gates + code-reviewer pass. + +## 2026-07-12 + +- **content** · Stage 2 Exercise 2 (Few-Shot) now makes a **fair** zero-shot vs few-shot comparison (resolves #62, reported by @WMichstaBe). The zero-shot baseline had no task instruction (just `input: {text}\noutput:`), which conflated "telling the model the task" with "showing examples" — small models often read it as text continuation and scored ~0, overstating the few-shot gain. Both conditions now share the same `TASK` instruction and few-shot only *adds* examples, so the experiment isolates the effect of the examples. The fragile `assert c3 >= c0` (few-shot isn't guaranteed to beat zero-shot) was replaced with a completeness check plus an honest "net gain may be 0 or even negative" printout; the observation prose was reframed to few-shot's real value (pinning output format + judgment on ambiguous cases). Path A + Path B, tri-locale; example code AST-parses clean; gates + code-reviewer pass. + +## 2026-07-09 + +- **content** · Stage 7.5 gains a plain-language **"分工 / Division of labor"** subsection (tri-locale), sourced first-party from Anthropic's *Agentic coding and persistent returns to expertise* (2026-06-16) + the *2026 Agentic Coding Trends Report*: you decide **what** to build, the agent decides **how** (~70% of planning decisions are the human's; ~80% of execution is left to the agent). Ties into the stage's "work boundary" axis, the returns-to-expertise framing, and the human → agent-team extension (Stage 7). Written analogy-first (home-renovation) for non-engineers; every cited number is first-party-verified — the trends-report "delegation gap" figures were NOT first-party-confirmable, so deliberately omitted. Tri-locale; anchor / zh-Hans / switcher gates + code-reviewer pass. +- **site** · Removed the README **Star History** section. The star-history.com embedded chart no longer renders (GitHub restricted star-timeline data access in 2026; the anonymous `api.star-history.com/svg` embed now 503s for every repo, verified incl. `facebook/react`). Since the README header already carries a live stars badge, the section was redundant once the trend chart was gone, so it was dropped rather than kept as a duplicate badge. Tri-locale; gates + code-reviewer pass. +- **content** · **GPT-5.6 (Sol / Terra / Luna)** shipped and is no longer preview — rolled through Stage 1 (model table + legend), glossary (Context-Window + Frontier-Model), and Stage 6 (reasoning-table intro + GPT-5.5 row). All first-party verified against OpenAI's API model docs: **Sol** `gpt-5.6-sol` ($5/$30 per MTok), **Terra** `gpt-5.6-terra` ($2.50/$15), **Luna** `gpt-5.6-luna` ($1/$6); all three **1.05M context**, 128K max output, available in ChatGPT + Codex + API. **Fixes a real error**: the GPT row's Context read `~400k` (that was GPT-5.5's) — GPT-5.6 is 1.05M. The `(preview)` marker plus its now-unused legend clause were dropped per the table's maintenance convention (status resolved → delete the legend line); Stage 1 header month 2026-06 → 2026-07; the glossary frontier entry gains a 2026-07 cluster. Preview-vs-GA was checked carefully: the 2026-06-26 *preview* system card described the pre-launch limited-preview phase, whereas the API docs now list all three under "Frontier models" with pricing and **no** preview/limited badge (that page does badge "Deprecated" elsewhere, so the absence is meaningful). Tri-locale; anchor / zh-Hans / switcher / stage-template gates + code-reviewer pass. + +## 2026-07-01 + +- **content** · Claude **Sonnet 5** (released 2026-06-30) rolled through the repo, all first-party verified (platform.claude.com model docs + anthropic.com/news/claude-sonnet-5): `claude-sonnet-4-6` → `claude-sonnet-5` and `Sonnet 4.6` → `Sonnet 5` everywhere they name the current default Sonnet — Stage 1 model table + reading list + pricing example, glossary Context-Window / Computer-Use / Frontier-Model entries, Stage 8 Computer-Use row, setup-guide + examples/README model-picker, all walkthrough + stage-3/4 example CLI commands, the `branches/for-developer` Aider example (its two-generations-old `claude-sonnet-4-20250514` snapshot also bumped to `claude-sonnet-5`), the repo CLAUDE.md picker, and the `freshness-models.yml` whitelist (47 content files, 43 model-ID swaps). Verified specs carried, none invented: **1M context** (same as Opus 4.8), **$3/$15** standard ($2/$10 intro through 2026-08-31), "best speed×intelligence" positioning; Sonnet 5 supersedes Sonnet 4.6 (now Legacy). Historical `Sonnet 4.5` references (Stage 6 predecessor list, Stage 7 GAIA leaderboard) deliberately left intact. Tri-locale; anchor / zh-Hans / switcher gates + code-reviewer pass. +- **content** · Stage 2 (Prompt Engineering) + glossary now spell out **zero-shot / one-shot / few-shot** in plain language — terms Exercise 2 leaned on but never defined. The glossary entry `Few-shot / Zero-shot` → `Zero-shot / One-shot / Few-shot` gains the previously-missing **one-shot** (exactly 1 example) plus a one-line framing (the three differ only in how many examples you show the LLM); Stage 2 Exercise 2 gets a 3-bullet inline explainer tying the `3-shot` its code uses back to "few-shot". Also corrected a stray Traditional `分類` → `分类` in the zh-Hans exercise line. Tri-locale; anchor / zh-Hans / switcher gates + code-reviewer pass. + +## 2026-06-30 + +- **content** · Staleness audit Batch 1 (from the 2026-06-29 multi-agent repo audit): removed the phantom "Claude Mythos Preview" attribution on the Stage 7 WebArena benchmark row (→ "領先 model 未公布" — Mythos/Fable benchmarks were never published and access is suspended, so the cell contradicted the table's own caption); glossary Context-Window entry gains Grok 4.3 1M + Mistral Medium 3.5 256k for parity with the Frontier Model entry; cookbook "Claude 4.5+" → "Claude 4.8+"; A2A glossary entry refreshed to v1.0 (Linux Foundation governance, 150+ orgs, signed Agent Cards). All first-party verified. Tri-locale; gates pass. +- **content** · Staleness audit Batch 2 (model-ID / recommendation refresh, all first-party verified): Stage 6 Path-2 reasoning list + observation line, Stage 8 example comment, and setup-guide updated Gemini 3.1 Pro → 3.5 Flash and added xAI Grok 4.3 (GA) to the strongest-reasoning options; GPT-5.5 deliberately kept in runnable example code since GPT-5.6 is still limited preview (not GA); setup-guide free-tier corrected to "GPT-5.5 Instant (rate-limited)"; Stage 4 OpenAI Agents SDK "April 2026 update" reframed to past-tense (the built-in-sandbox / 7-provider claim verified accurate), AG2 v0.2-vs-v0.4 note softened. Tri-locale; gates pass. +- **layout** · Stage 1 model tables de-crammed: time-sensitive status/caveat text (Fable 5 suspension, license clauses, release dates, Arena rank) moved out of the data cells into one plain-language legend line per table; genuinely-old entries retired (GPT-5 / o-series, Gemini 3.1 Pro). Added an HTML-comment maintenance convention so the tables self-clean as models churn (new flagship = swap not append; resolved status = delete the legend line). Worst flagship cell dropped ~75 → ~33 chars. Tri-locale; plain-language; gates + code-reviewer (APPROVE) pass. +- **layout** · Stage 2 Curated-Projects table de-crammed (same pattern): the two worst cells (dspy / NirDiamant, ~150-182 → ~38-74 units) trimmed to a short reason + ★ / license; framework-not-tutorial + NOASSERTION caveats moved to one legend line. Tri-locale; gates pass. +- **layout** · Stage 1 Chinese-frontier table (the 7-provider / 7-col one, widest in the stage) split along the API-vs-open-weights line into two 6-col tables (① API-only: DeepSeek / Kimi / Hunyuan / MiniMax · ② open weights: Qwen / GLM / Yi); the License column folded into one plain legend per group. All 7 providers + license nuance preserved. Tri-locale; gates + column-count scan pass. +- **site** · MkDocs-Material UI upgrade (verified with a local `mkdocs build`, clean across all 3 locales): code-copy buttons, instant/SPA-style navigation + progress, navigation.sections, search.share, content tabs/tooltips/annotate; `:material-*:` icon support; new `docs/stylesheets/extra.css` (indigo brand color, grid-card hover, tighter rounded tables). A custom card landing page is a planned follow-up (blocked on the README-vs-index i18n conflict). +- **site** · Nav cleanup: the top nav was bilingual + inconsistent ("首頁 / Home", "Audience branches"…); switched to single-language source labels + i18n `nav_translations` so each locale shows only its own language (繁中 首頁 / en Home / 简中 首页), and dropped `navigation.sections` (over-expanded the sidebar). Verified with a local build, all 3 locales. +- **site** · Custom card landing page shipped (resolves the earlier README-vs-index blocker): `index.md` / `.en.md` / `.zh-Hans.md` is now the trilingual home — hero + stat cards + track/stage grid-cards. The README moved to an `/about/` page (staged as `about.md` so it no longer collides with `index` for the home slot), and `mkdocs_hooks.py` rewrites in-content `README.md` links to `about` at build time so they keep resolving (examples/README untouched). Verified locally: clean build, all 3 locale homes = landing, anchor / zh-Hans / switcher gates pass. +- **content** · Staleness audit Batch 3 (new harness-engineering frames + adds, all first-party sourced): Stage 7 gains a "feedback loops, not a more perfect prompt" subsection — the 4 feedback timings (tool returns / mid-run steering / end-of-turn acceptance / outer loop), anchored on Anthropic's planner→generator→evaluator harness post; Stage 7.5 gains "Harnesses expire: Model-Harness-Fit + the Bitter Lesson" (Sutton 2019); `deepagents` (LangChain, LangGraph-based, MIT, v0.6.12) added to Stage 4 framework resources + a plain glossary "Deep Agent" entry. Written analogy-first for non-engineers, jargon glossed inline; Codex `/goal` folded into N1's outer-loop row (no separate section). Tri-locale; gates pass. + +## 2026-06-29 + +- **content** · Stage 1 model table + glossary (frontier + Context-Window entries) + `scripts/freshness-models.yml` whitelist refreshed with late-June-2026 frontier models, all first-party verified: GPT row gains GPT-5.6 (Sol / Terra / Luna, **preview**); Gemini row → 3.5 Flash (3.5 Pro in dev); glossary frontier adds xAI Grok 4.3 (GA) + Mistral Medium 3.5 (open weights, preview), relabeled by half-month (Fable 5 suspension note retained). Preview-vs-GA marked; no fabricated benchmark / context numbers. Tri-locale; anchor / zh-Hans / switcher gates pass. +- **content** · Stage 6 reasoning-model table consistency follow-up: the "current (Jun 2026) frontier" intro + the GPT-5.5 and Gemini 3.1 Pro rows now flag that newer tiers exist (GPT-5.6 Sol / Terra / Luna **preview**; Gemini 3.5 Flash available, 3.5 Pro in dev). Existing verified rows and benchmarks (e.g. Gemini 3.1 Pro GPQA Diamond 94.3%) kept and correctly attributed; no preview-model benchmarks fabricated. Tri-locale; gates pass. + +## 2026-06-24 + +- **catalog** · Added DeusData/codebase-memory-mcp (★ 13.5k, MIT) to §5 Dev Collaboration — a code-intelligence MCP that indexes a codebase into a queryable knowledge graph (query structure / symbols / call paths instead of grep+read). Plain, non-marketing description (notes the re-index-after-edits + verify-load-bearing-claims caveats); tri-locale; §5 TOC count 7→9 (also corrects a pre-existing off-by-one); gates pass. + +## 2026-06-13 + +- **catalog** · Added 12 high-confidence repos (all gh-verified stars/license, none previously listed): microsoft/agent-framework (Stage 4); getzep/graphiti + lancedb/lancedb (Stage 6); comet-ml/opik, pydantic/logfire, NVIDIA-NeMo/Guardrails, BoundaryML/baml (Stage 7, incl. new Safety/Guardrails + Structured-Output rows); bytedance/UI-TARS-desktop + trycua/cua (Stage 8, new Computer Use Agent Stack); awslabs/mcp + ComposioHQ/composio (MCP/Skills catalog §6 / §12); microsoft/mcp-for-beginners (Stage 5.2 reading list). Tri-locale; per-section counts updated; anchor / zh-Hans / switcher gates pass. +- **content** · Stage 5 — plain-language orientation box in 5.1 (Claude Code = terminal agent for devs; Claude Cowork = desktop agent for non-coders; OpenAI parallels = Codex CLI / ChatGPT agent) so beginners see Claude Code is one *shape* among several; plus first-use plain glosses for heavy terms (harness / orchestration / scaffolding / control plane). Tri-locale; Cowork + ChatGPT agent verified first-party; anchor / zh-Hans / switcher gates pass. +- **content** · Reframed Claude Fable 5 across the roadmap after Anthropic suspended all access to Fable 5 + Mythos 5 on 2026-06-12 (US government export-control directive; [status](https://status.claude.com/) · [statement](https://www.anthropic.com/news/fable-mythos-access); no restoration timeline). Documentation tables (`CLAUDE.md` / `examples/` / stages 01·06·07·07.5·08 / glossary) now mark Fable 5 as suspended and currently unavailable, with Opus 4.8 as the current top usable Claude tier; recommendation pick-lists (Path-2 reasoning chooser, Computer Use vendor table, OmniParser / browser-use swap-lists) drop Fable 5 so no reader is pointed at an inaccessible model. Tri-locale; anchor / zh-Hans-localize / language-switcher gates all pass. Suspension verified against two first-party sources, no fabricated facts. +- **docs** · `CITATION.cff` version `2026.05.19` → `2026.06.13` (was stale vs the recent content batches). + +## 2026-06-12 + +- **content** · Claude Fable 5 (Mythos-class, `claude-fable-5`, GA 2026-06-09) added as the new top Claude tier across the trilingual roadmap — model tables in `CLAUDE.md` / `examples/` / stages 01·06·07·07.5·08 + glossary frontier entry; Opus 4.8 reframed as Opus-class flagship + Fable 5 safeguard-fallback. No fabricated context-window or benchmark numbers (Anthropic published none — marked "not yet published"). Also fixed a pre-existing `claude-opus-4-7` → `claude-opus-4-8` inconsistency (`12980b3`). +- **content** · Stage 5 — new **5.6 Dynamic Workflows** section after 5.5 Subagents (ecosystem-level intro + cross-link to the 7.5 deep-dive, no duplication); old 5.6 Source → 5.7, old 5.7 SDK → 5.8, all in-file refs + 7-Layer-map ranges + cross-file anchors (glossary / stages 03·06·07) relinked, tri-locale (`5044008`). +- **catalog** · `1weiho/open-slide` (★4.9k, MIT) added to §2 as an agent-native slide framework — ships Claude Code Skills, distinct from Stage 4 orchestration frameworks; tri-locale (`7d3fd5d`). +- **docs** · MCP/Skills catalog count made drift-proof — stale `62` → robust `65+`, category count reconciled to 15, across 33 files / all locales (`3782dd4`). Propagated the 7→8 stage reality into design notes / style-guide / reader docs (`39d397a`) and fixed outreach-draft count drift (`25785f0`). +- **outreach** · send-day copy-paste packages playbook for awesome-list submissions (`afd7a76`). +- **content** · per-chapter improvement audit (12-agent fan-out + skeptical filter) → 5 gap-fills, all tri-locale: Stage 3 lethal-trifecta security callout + MCP router note + glossary (`f3bde60`); Stage 1 next-token / sampling mental-model box (`1bd171f`); Stage 5 Hooks (L3 control layer) subsection (`9d2897f`); Stage 7 Loop Engineering note + glossary (`eb8e64c`). +- **catalog** · new Web Search / Retrieval category (exa-mcp + tavily-mcp) + Context7 in Dev-Collaboration; category count 15→16 (`b1718d3`). +- **content** · improvement-audit medium batch (6 more tri-locale gap-fills): Stage 3 structured outputs / JSON-mode (`93006a8`); Stage 2 reasoning-vs-CoT + Stage 5 MCP-in-2026 (Registry / FastMCP / security) + Stage 8 accessibility-tree & Playwright-MCP (`ea0633e`); Stage 6 RAG ingest-parsing + embedding-model selection + Stage 7 OTel GenAI conventions / pass^k·τ²-bench / MAST (`2fcfc6b`). + +--- + +## 2026-05-31 + +- **tooling** · pruned the ops-metric scripts that don't touch stars or URL validity (strategic-review action #1, scoped down per maintainer): removed `scripts/snapshot-traffic.py` (GitHub traffic snapshots), `scripts/refresh-outreach-status.py` (outreach-matrix drift), `scripts/check-catalog-staleness.py` (dormant-entry pinger), and the `docs/traffic/` snapshot dir. **KEPT** the weekly stars + URL auto-update (`weekly-catalog-refresh.yml` + `lint.yml`'s `star-drift` job) — the maintainer values the weekly cadence for star-count refresh and link-rot checking. All correctness + trilingual-parity guards intact (anchor / link-rot / mirror-sync / stage-template / banned-words / overclaim / zh-Hans-localize). + +--- + +## 2026-05-26 + +- **ci** · `lint.yml` overclaim check expanded (P3-G from audit) — promoted from case-sensitive exact-phrase to case-insensitive (`grep -Fi`), broadened scope to include `tracks/` `examples/` `resources/` (which the previous narrower scope missed — letting 5 uppercase `Production-grade` H2 headers in `examples/` slip through the earlier sweep). Strict-blocking list now includes all style-guide §3 phrases (`首選` / `首选` / `唯一選擇` / `唯一选择` / `業界最佳` / `业界最佳` / `業界最強` / `全世界最好的` / `最緊迫` / `the most canonical`) plus English equivalents (`production-grade` / `world-class` / `best-in-class` / `cutting-edge` / `state-of-the-art` / `industry-leading`). Corpus pre-cleaned across tri-locale before flipping to strict. +- **content** · overclaim residue swept across tri-locale (18 file edits) before the lint flip — 3 × `## Production-grade …` H2 headers in `examples/stage-{6,7}/` normalized to `## Production-ready …`; 3 × inline `首選` in `stages/05` / `stages/06` / `tracks/cli/A1` softened per style-guide §3; `tracks/cli/A1` `最完整的中文社群資源` → `中文社群資源豐富` (marketing → factual). +- **tooling** · `scripts/snapshot-traffic.py` shipped — captures weekly 14-day traffic window (views / clones / referrers / paths + point-in-time totals) to `docs/traffic/snapshots/YYYY-MM-DD.json` so historical trend survives the GitHub API's 14-day visibility limit. Each file ~5 KB. First snapshot included (`docs/traffic/snapshots/2026-05-26.json`). +- **tooling** · `scripts/refresh-outreach-status.py` shipped — reads `.github/channel-partners.md`, extracts PR URLs, queries `gh pr view`, reports drift between recorded status and live PR state (merged / closed / ghosted / approved). Report-only (text / markdown / json), `--check` for CI. Closes P2-F from the 2026-05-25 audit; P2-E closed by snapshot-traffic. + +--- + +## 2026-05-25 + +- **tooling** · `scripts/check-catalog-staleness.py` shipped — queries `gh api repos//` for `pushed_at` + `archived`, flags catalog entries dormant >= N months (default 12) or archived. Report-only (text / markdown / json). Initial run on the 247-repo catalog surfaced 17 stale entries: 5 archived (incl. `langchain-ai/langserve` archived 2026-05-05 still cited as live, `RooCodeInc/Roo-Code` archived 2026-05-15 in setup-guide) + 12 dormant (oldest: `microsoft/prompt-engine` 37 mo). +- **i18n** · Stage 1 + Stage 2 mirror schema resync — `## 🎯 Curated Projects` regenerated from canonical (en hand-translated · zh-Hans via opencc tw2s + zh-hans-localize vocab); −358 lines of stale H3-card format replaced with compact-table parity to canonical. Also normalized 5 Stage 1 .zh-Hans H2 titles back to canonical wording + emoji. Eliminates the forward-schema drift across all 8 stages. + +--- + +## 2026-05-19 + +- **catalog** · `microsoft/ai-agents-for-beginners` added to Stage 3 選讀/進階補充 as a parallel beginner course (explicitly *not* a substitute for the stage's hands-on practice), tri-locale (`2d83f72`, `94f2d73`). + +## 2026-05-18 + +- **catalog** · Kimi-K2 + GLM-4.5 added to §11 中文圈專用 — neutral schema, gh-verified Stars/License, tri-locale (`fd81f31`, `ad80845`). +- **ci** · weekly catalog-refresh PR now guarded auto-merge: sanity guard (star-token-only diff, ≤150 lines, anchors pass) → squash-merge, else label `needs-manual-review` (`3dc6ecd`). + +## 2026-05-17 + +- **docs** · per-track Capstone + 4-level self-assess rubric (`CAPSTONE`), tri-locale (`dbf1ef3`, `a31dde5`). +- **docs** · Pages unified — mkdocs at `/`, mdBook at `/book/`, one workflow; README's GitHub-only switcher stripped from rendered site (`5e59c7c`, `001d765`). +- **docs** · README positioning reframed (trilingual, English fully maintained); stale exercise-folder count corrected 27 → 23 (`b4bb862`, `24a87fe`). +- **outreach** · English-audience launch drafts — HN / Reddit / newsletters / awesome-lists (`b8f365b`). + +## 2026-05-16 + +- **governance** · CoC + SECURITY + CITATION.cff + issue-template config added, tri-locale mirrors (`9aa2963`, `84bc58f`). +- **docs** · public ROADMAP.md + learner PROGRESS.md tracker added, tri-locale (`e5cc310`, `3e628e9`). +- **docs** · GitHub Pages site (mkdocs-material, trilingual) + live docs-site badge (`498932c`, `ea4530f`). +- **i18n** · zh-Hans mainland-localization pass + Lint gate blocking Taiwan-vocab/「」 drift (`7f73b8a`, `805ae57`). +- **visuals** · final ASCII concept blocks replaced with generated PNGs — 10/10 complete (tri-locale) (`21a2bbf`). +- **ci** · actions bumped off deprecated Node20 ahead of June 2026 forced migration (`c6a8c19`). +- **outreach** · CONTRIBUTORS — @demo112 (#14) + @Rain120 (#18) (`7040738`). + +## 2026-05-15 + +- **content** · Stage 1 §主流 LLM 家族對比 (US 3 + China 7 + Western-OSS 4 + decision tree + benchmark + caveat) (`8f578bf`). +- **content** · Stage 5 §7-Layer Architecture Map (Claude primitives × 3 engineering disciplines) + embedded figures (`5f99bbb`, `1e5a12b`). +- **content** · subagent teaching deepened — dispatch who/how/what, vs Skill/Slash-Command disambiguation, advanced doc + figures (`009ddf9`, `21c555b`, `e8a919e`). +- **content** · 5 audience branches tableized (使用情境 / 流程 / Tier ladder) + academic-style polish, tri-locale (`184015b`, `6b7e5f6`). +- **i18n** · 97 broken outbound mirror anchors fixed + anchor-checker now enforces mirror files (`e1991a6`, `ab3a6d0`). + +## 2026-05-14 + +- **content** · NEW Stage 7.5 — Advanced Agentic Concepts (OpenAI Harness Engineering 5 principles, Why→What→How map, work-boundary diagram) (`4a6bf18`, `e2c1d11`). +- **content** · Track A3 §6 advanced-concept playbooks for daily CLI work (`876a457`). +- **visuals** · § (513×) and 🔄 (24×) symbols stripped across all user-facing docs; concept diagrams embedded as PNG × 3 locales (`29eb774`, `d04c224`). +- **catalog** · 4 Anthropic-related resources added across stages (`0af7fbc`). +- **ci** · weekly catalog-refresh workflow + `--apply` flag (`dc91a8b`). + +## 2026-05-13 + +- **content** · Stage 4/6/7 verified + merged to main (`cdb0ae3`); Stage 8 NEW — Agent Interfaces, §1-15 across 3 commits A/B/C (`b83c894`, `6c87a2f`, `069406f`). +- **content** · curation positioning crystallized — exercises reframed foundational/illustrative; repo = curation hub + simple cases, depth → hello-agents (`00dc046`, `0206dbc`). +- **content** · 精選 Projects consolidated to single 適合誰 tables across Stages 0-8 + Track A (`fd94d80`, `19a14a8`). +- **content** · Stage 5 expanded (§5.1-5.6: Claude Code basics, MCP/Plugin/Skill 定位, §5.5 Subagents, Harness Internals) (`2c3f1dd`, `f7de4e7`). +- **content** · Stage 6 RAG-first restructure + GraphRAG / Contextual Retrieval / Hybrid Search; 2026 frontier-model refresh (`f00e2c2`, `acbc9dc`). +- **ci** · 4 checks added — anchor validator, mirror-sync reminder, 2026 freshness, stage-template enforce (`a14c809`, `4491e6e`). +- **i18n** · 8-stage tri-locale mirror catch-up via Codex + Gemini delegation; 37 legacy anchors fixed, validator → strict (`8b39c75`, `706d257`). +- **catalog** · whale (DeepSeek terminal) + a-stock-data added to Chinese ecosystem (#14) (`3d375bd`). + +## 2026-05-12 + +- **content** · examples/ bootstrapped — Stage 1 (6) + 2 (4) + 3 (6) + 4 (5) + 6 (5) + 7 (5) inline starters + folder examples, tri-locale (`c1fcaa7`, `8051861`, `7d2c1b7`). +- **content** · dual-path examples — Ollama (default, cost-driven) alongside Anthropic; per-stage budget + LLM recommendation list (`bc37ad8`, `3fa5410`). +- **content** · tool-calling-tutor — installable Claude Code skill + Stage 5 §5.3 meta-example (`3584669`). +- **i18n** · diagrams renamed `.zh-Hans.png` per BCP 47 / W3C convention (`78797a3`). + +## 2026-05-11 + +- **accessibility** · `resources/setup-guide.md` (3 langs) — addresses the dev-fluency assumption gap that subagent audit flagged across 5 non-dev branches. 5 sections covering API key registration, Python install, hello-world, Claude Code first auth, SKILL.md primer (`3c88b2b`). Plus 15 branch-top callouts on all 5 audience branches. `resources/README.{en,zh-Hans}.md` created for trilingual parity. +- **accessibility** · README — promoted setup-guide pointer to top of Quick Start across all 3 langs (`ad47706`). Was buried in Related Resources where non-dev visitors hit technical walls before discovering it. +- **accessibility** · setup-guide opens with a 4-tier on-ramp (Web / Desktop / CLI / API) + official download URLs for Claude.ai, ChatGPT, Gemini, Le Chat, Claude Desktop, ChatGPT Desktop, LM Studio (`3c89952`). Replaces the abstract "decide two things" intro so non-dev readers see "just use claude.ai for free" as the first option, not "register API key → install Python". +- **accessibility** · setup-guide adds a 3rd tier between Desktop and CLI: **IDE with built-in AI** (Cursor, Windsurf, Cline, Continue, Roo Code, Zed, GitHub Copilot) with download URLs (`7e14093`). Distinguishes "AI sidekick while you write code" from "agent runs autonomous task in terminal". + +## 2026-05-10 + +- **funnel** · Stage 1 → Stage 2 callouts added across 3 langs to address visible drop in `traffic/popular/paths` (`0ee2a3a`) +- **outreach** · 3 awesome-list targets backfilled into channel-partners matrix from launch-checklist: `travisvn/awesome-claude-skills`, `WangRongsheng/awesome-LLM-resources`, `AiHubCN/Awesome-Chinese-LLM` (`90a6ad1`) +- **outreach** · PR #6135 to `punkpeye/awesome-mcp-servers` — addressed bot `name-check`, replied to non-applicable `glama-check` + `emoji-check` (`81a7313`) +- **content** · Cookbook Recipe 6 — **Local-LLM × CLI Agent walkthrough** (`5855852`). Bridges Stage 1 (local LLM) + Stage 5 (CLI agent) end-to-end. Explicitly notes Claude Code does **not** support local LLM as backend; routes readers to OpenCode / goose / Aider / Hermes instead. Stage 5 + cli-agents-guide also gain matching pointers. +- **catalog** · Hermes Agent (`NousResearch/hermes-agent` ★142k) added as 7th major CLI agent across `cli-agents-guide`, `tracks/cli/A1`, and 5 dependent files (`698f13a`). Differentiator: cloud-VM-native, model-neutral (200+ LLMs via OpenRouter / NIM / GLM / Kimi / etc.), self-improving skill loop. +- **i18n** · `*.zh-CN.md` → `*.zh-Hans.md` migration per BCP 47 / W3C compliance (`21b653d`). 25 files renamed, ~270 markdown lines updated, tooling (`sync-language-switchers.py`, `lint.yml`, `generate-stage5-stack.py`) migrated. Thanks [@xfq](https://github.com/xfq) (W3C i18n lead) for flagging in [#9](https://github.com/WenyuChiou/awesome-agentic-ai-zh/issues/9). Added to CONTRIBUTORS (`868691d`). +- **visuals** · English README hero (`banner.en.png`), Learning Map (`learning-map.en.png`), and Branch Decision Tree (`branch-decision-tree.en.png`) refreshed to ChatGPT-rendered versions (`c7edff8`, `4be6b88`, `6c03c58`). + +## 2026-05-09 + +- **outreach** · Day 1 PR sent: `punkpeye/awesome-mcp-servers#6135`, adding awesome-agentic-ai-zh to `## Tutorials` (`a0dc4d5`). Plan revised after upstream audit caught `hesreallyhim/awesome-claude-code` mid-reorg (Day 2 = issue not PR) (`708259c`). +- **outreach** · 8 channel-partner pitch templates created in `.github/outreach/` plus tracking matrix `.github/channel-partners.md` (`2f63745`). Targets: Datawhale, liyupi, HuggingFace, LangChain (kyrolabs), awesome-claude-code, awesome-mcp-servers, Zhipu, Moonshot. +- **catalog** · 11 中文圈專用 expanded from 2 → 7 entries: `QwenLM/Qwen-Agent`, `coze-dev/coze-studio`, `coze-dev/coze-loop`, `liaokongVFX/LangChain-Chinese-Getting-Started-Guide`, `chatchat-space/Langchain-Chatchat` (`4809039`). +- **funnel** · Stage 0 → Stage 1 callouts added (`3dfe761`). +- **ci** · zh-Hans companion files excluded from zh-TW banned-word audit (closes #7) (`3acc3f2`). + +## 2026-05-08 + +- **content** · `for-teacher` branch expanded with 3-tier teacher AI use-case framework (Chen 2020, Mittal 2024) via @scott0127 PR #6 (`cd1cad4`). +- **content** · Stage 6 unit guide: memory + RAG overview via @scott0127 PR #5. +- **content** · Branch decision tree (zh-Hans) added, English banner added, `for-developer` branch thickened 56 → 138 lines × 3 langs. + +## 2026-05-07 + +- **catalog** · 3 user-flagged gaps filled: `safishamsi/graphify`, `pbakaus/impeccable`, `netease-youdao/LobsterAI` + context-engineering and harness-engineering coverage. +- **content** · `resources/cookbook.md` added with 5 (now 6) step-by-step recipes covering Skill / MCP / Office / NotebookLM / Zotero / Local-LLM workflows. + +## 2026-05-06 + +- **launch** · Repo announced to bilingual community. Star count: 0 → 519 in week one. +- **content** · `learning-map.png` polished, README hero banner placement finalized. + +--- + +## Conventions + +- Each commit SHA is clickable: `https://github.com/WenyuChiou/awesome-agentic-ai-zh/commit/` +- Categories: `content` (stages/branches/tracks) · `docs` (project meta-docs: README/ROADMAP/PROGRESS/CAPSTONE/Pages site) · `governance` (CoC/SECURITY/CITATION/issue templates) · `accessibility` (on-ramp/setup friction) · `catalog` (mcp-skills-catalog entries) · `funnel` (cross-stage navigation) · `visuals` (diagrams/banners) · `i18n` (translation/locale) · `outreach` (channel partners) · `ci` (workflows/lint) · `launch` (one-time events) +- Maintained manually; not auto-generated. Updated alongside substantive commits. diff --git a/CITATION.cff b/CITATION.cff new file mode 100644 index 0000000..a540f3a --- /dev/null +++ b/CITATION.cff @@ -0,0 +1,31 @@ +cff-version: 1.2.0 +title: "awesome-agentic-ai-zh: A trilingual (English / 繁中 / 简中) learning roadmap for agentic AI" +message: "If you use this learning roadmap, please cite it using the metadata below." +type: software +authors: + - family-names: Chiou + given-names: Wenyu + alias: WenyuChiou + - name: "awesome-agentic-ai-zh contributors" +repository-code: "https://github.com/WenyuChiou/awesome-agentic-ai-zh" +url: "https://github.com/WenyuChiou/awesome-agentic-ai-zh" +abstract: >- + A community-curated, trilingual (Traditional Chinese / Simplified + Chinese / English) learning roadmap for agentic AI — covering LLM + basics, prompt design, tool use, agent frameworks, the Claude Code + ecosystem, memory/RAG, and multi-agent production, organized as + staged tracks and audience-specific branches. +license: MIT +version: "2026.06.13" +date-released: "2026-06-13" +keywords: + - agentic-ai + - ai-agents + - llm-agents + - claude-code + - claude-skills + - mcp + - model-context-protocol + - learning-roadmap + - trilingual + - tutorial diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..d9c2078 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,125 @@ +# Project Memory — awesome-agentic-ai-zh + +> Standing instructions for any AI agent (Claude, Codex, Gemini) working on this repo. Read this **first** before touching exercises or model recommendations. + +## 📍 Repo positioning — read before adding anything + +**This repo's role**: **learning roadmap + 240+ curated resources + simple illustrative cases.** + +**Benchmark for "what we are NOT"**: [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents) is the canonical chapter-length zh-TW depth tutorial (16 production capabilities, chapter format). **We don't compete with it; we route to it.** + +**Implications when contributing**: + +| Decision | Rule | +|---|---| +| New stage-level exercise folder | OK if it adds **a roadmap node + dual-path SDK demo + 1-line punchline**. 70-150 lines starter is the right size. | +| Expanding a starter beyond ~150 lines | **Push back**. If it's growing into chapter-length, add a 📚 callout pointing to hello-agents instead. | +| Adding a 5th `extension` to README | Diminishing return. Keep README tight (under ~200 lines); extra depth goes to the 📚 callout. | +| New resource (lib / paper / tool / framework) | Almost always YES — add to the relevant `精選 Projects` section or `resources/` catalog. Curation is the primary value. | +| New chapter-length tutorial inside this repo | **Push back**. If the topic deserves chapter-length, the right move is: write a 1-page summary + simple illustrative case + 📚 callout to a canonical source (hello-agents / Anthropic Cookbook / framework's own docs). | +| Trilingual mirror priority | zh-TW canonical first; en + zh-Hans mirror when capacity allows. Don't block shipping waiting for 3-lang. | + +**One-line summary**: **route → depth, not reinvent**. Every exercise folder ends with 📚 "want chapter-length? go to hello-agents X + [extra ref]". + +**Existing examples of this pattern** (as of 2026-05-13): +- All Stage 3 / 4 / 6 / 7 example READMEs have the 📚 callout (20 folders × 1 callout) +- Main README + 3-lang mirror have the positioning statement near 🎯 Why this exists section +- `tracks/cli/` is outline-only on purpose (CLI exercises are bash/markdown/config, not Python SDK; doesn't fit the dual-path frame — that's correct) + +## Canonical Ollama models (verified against user's `ollama list`) + +| Model tag | When to use | Notes | +|---|---|---| +| **`gemma4:e4b`** | Stage 1 + 2 (plain chat, prompt engineering) | Effective 4B params, ~7.5 GB download, CPU-friendly. **The `:e4b` tag matters** — NOT `gemma3n:e4b`, NOT `gemma3:4b`, NOT `gemma4:latest`. | +| **`gemma4:e2b`** | Low-RAM-machine alternative for Stage 1+2 | ~4 GB, runs on 4 GB RAM machines | +| **`qwen2.5:3b`** | Stage 3+ (tool use / agent / ReAct) | 1.9 GB, **reliable tool-use support** (OpenAI function-calling format), default for any agent / function-calling exercise | +| **`llama3.2:3b`** | `qwen2.5:3b` alternative for tool use | 2.0 GB, similar capability | +| **`mistral-nemo:12b`** | Higher-quality local fallback | 7.1 GB, closer-to-cloud quality | + +**Wrong tags I've used in error before** (now fixed across 13 files via `.ai/.../rename_gemma.py`): +- ❌ `gemma3:4b` — older naming, replaced 2026-05-12 +- ❌ `gemma3n:e4b` — wrong family, replaced 2026-05-12 +- ✅ `gemma4:e4b` — correct (per user's Ollama installation screenshot) + +If unsure, ask the user to run `ollama list` and verify. + +## Canonical Anthropic models + +| Model | Use case | Pricing (per 1M tokens) | +|---|---|---| +| **`claude-fable-5`** | Mythos-class (above Opus); suspended 2026-06-12, **restored 2026-07-01** (controls lifted 2026-06-30); the highest Claude tier | $10 input / $50 output | +| **`claude-haiku-4-5`** | Cheapest cloud option, OK for all exercises | $1 input / $5 output | +| **`claude-sonnet-5`** | Production default, agent development | $3 input / $15 output | +| **`claude-opus-4-8`** | Opus-class flagship; high quality, complex reasoning (Fable 5, restored 2026-07-01, is the tier above) | $5 input / $25 output | + +## Framing rules (do not violate) + +1. **Claude is the canonical / production reference** in documentation positioning. +2. **Ollama is the practice default** because of cost — students should not be blocked by API fees during learning. +3. **Every exercise must ship BOTH paths**: + - Path A (Ollama, `
`, primary practice runnable) + - Path B (Anthropic, `
`, optional cloud-quality comparison) +4. **Every exercise must mention budget explicitly** — single-run cost + total stage cost. +5. **Local LLMs must appear in any model recommendation list** — never list cloud-only options. + +## Exercise file conventions + +- `starter.py` = Ollama / OpenAI-compatible default (Path A) +- `starter_anthropic.py` = Anthropic SDK version (Path B) +- `test.py` = mock-based tests for the Ollama starter (OpenAI-compat response shape) +- `test_anthropic.py` = mock-based tests for the Anthropic starter (content-block shape) +- `requirements.txt` = both `openai` and `anthropic` pinned +- `README.md` = trilingual switcher + 怎麼跑（兩條 path）+ budget per path + walkthrough + common pitfalls +- Each starter ends with `# === 自我驗證 ===` block containing 2+ `assert` statements +- Each Python file headers Windows-cp950 UTF-8 reconfigure: + ```python + import sys + if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + ``` + +## Translation rules + +- **zh-TW canonical** (`.md` without language suffix). zh-Hans + en mirror. +- **Claude does translations** — do NOT delegate to Codex/Gemini. +- For zh-Hans bulk char conversion, use a per-char map (script in `.ai/2026/05/12/t2-trad-to-simp/`) then manual fix-up for remaining stragglers. + +## Codex delegation rules + +- Codex executes bulk batches (multiple exercises following an established pattern). +- Claude writes the pilot template + reviews codex output. +- Codex briefs must include the file structure (starter.py + starter_anthropic.py + test.py + test_anthropic.py + README.md + requirements.txt) and the framing rules above. +- Codex cannot commit (sandbox `.git` permission); Claude commits on its behalf per CLAUDE.md `~/.claude/CLAUDE.md` "agent boundary = commit boundary" rule. + +## Existing curriculum state (as of 2026-05-12) + +| Component | Status | +|---|---| +| Stage 0-3 inline exercises (3 langs) | ✅ Done — Path A Ollama / Path B Anthropic + budget callouts | +| Stage 3 folder `03-react-from-scratch` | ✅ Pilot rename done — `starter.py` (Ollama) + `starter_anthropic.py` (Anthropic) + dual test files | +| Stage 3 folders `02/04/05/06` | ✅ Phase 3 done (2026-05-12) — Ollama `starter.py` + rename existing → `starter_anthropic.py` + trilingual READMEs in dual-path style | +| Stage 1 folder `04-cross-provider` | ✅ Multi-provider (already includes Ollama via `call_ollama` in README) | +| Stage 1 folder `05-error-handling` | ✅ Phase 3 done (2026-05-12) — openai SDK exceptions + same retry wrapper, trilingual READMEs | +| Stage 3 doc inline simplified examples (練習 2-6) | ✅ Done (2026-05-12) — 5 new `
` blocks added inline (Path A 8-15 line cores), trilingual mirror, zh-Hans Trad-char drift fixed at lines 44/47/77/110/152 | +| `examples/stage-5/tool-calling-tutor/` skill | ✅ Done (2026-05-12) — installable Claude Code skill (frontmatter + 5-step body), 3 references (debug-flowchart / schema-evolution / sdk-diff), evals.json with 5 cases, trilingual READMEs + translations. Dual purpose: learner-aid + Stage 5 5.3 meta-example. Cross-referenced from stages/03 + stages/05 | +| Stage 4 (5 exercises) | ✅ Verified 2026-05-13 — ex1 LangGraph+CrewAI comparison, ex2 CrewAI multi-agent roles (CrewAI install fails on Python 3.14, code unmodified), ex3 LangGraph branching+HITL, ex4 Smolagents CodeAct, ex5 Pydantic AI typed output. 14 of 15 test suites verified green; ex2 CrewAI untestable on 3.14 due to tiktoken/regex wheel build failures | +| Stage 6 (5 exercises) | ✅ Verified 2026-05-13 — all 10 test suites green. Fixed 2 bugs: ChromaDB 'kb' collection name (needs 3-512 chars; renamed knowledge_base) + EphemeralClient state leak across test fixtures (added uuid suffix per test) | +| Stage 7 (5 exercises) | ✅ Verified 2026-05-13 — all 10 test suites green. Fixed 1 bug: eval test fake_agent operator precedence (and binds tighter than or) caused test_run_eval_aggregates to fail. FastAPI deploy includes Dockerfile | +| Track A1-A3 (12 CLI exercises) | 🟡 Outline complete (`tracks/cli/A{1,2,3}-*.md` × 3 langs, ~367 lines zh-TW; 12 numbered exercises documented end-to-end with goal / required-reading / hands-on / curated-projects / self-check). `examples/track-a/` folder intentionally NOT built — these exercises are bash + CLAUDE.md + slash command + MCP integration + GitHub Actions yml, **NOT** Python SDK code; the dual-path Ollama/Anthropic framing doesn't apply. Reference doc: [`resources/cli-agents-guide.md`](resources/cli-agents-guide.md) (148 lines). | +| Stage 5 (11 sub-exercises) | ⚪ Pending — different shape (bash / MCP / markdown / CLAUDE.md / SKILL.md / plugin.json authoring, not OpenAI SDK Python). 5.3 has 1 meta-example shipped: [`examples/stage-5/tool-calling-tutor/`](examples/stage-5/tool-calling-tutor/). Other sub- framing TBD — see [`docs/TESTING_PLAN.md`](docs/TESTING_PLAN.md). | +| `examples/README` LLM list + budget table | ✅ Done (3 langs) | +| Per-stage budget callouts | ✅ Done for Stage 1+2+3 (3 langs each) | + +## Known follow-up: pilot `03-react-from-scratch` README.en.md + README.zh-Hans.md drift + +The zh-TW `README.md` of `examples/stage-3/03-react-from-scratch/` already uses the dual-path layout (Path A primary / Path B optional + budget callouts + mock test mention for both backends). The `README.en.md` and `README.zh-Hans.md` siblings were NOT updated when the pilot's dual-path zh-TW README was written — they still describe the pre-dual-path layout (Anthropic-only `starter.py`, single `test.py`). After Phase 3 the other 5 folders all have aligned trilingual dual-path READMEs, so the pilot is now the lone outlier. Fix when revisiting Stage 3 docs polish — straight translation pass of the zh-TW README is enough. + +## Reference scripts (in `.ai/2026/05/12/`) + +- `t2-trad-to-simp/convert.py` — zh-TW → zh-Hans bulk char map (Stage 2) +- `t2-trad-to-simp/stage3_convert.py` — Stage 3 練習 1 inline section conversion +- `t2-trad-to-simp/en_swap.py` — Anthropic SDK → OpenAI SDK bulk substitution +- `t2-trad-to-simp/en_pathb_expand.py` — Compact 🦙 hint → full Path B `
` block +- `t2-trad-to-simp/rename_gemma.py` — `gemma3n:e4b` → `gemma4:e4b` (this commit's fix) + +Keep these scripts — they're reusable for T3+ work. diff --git a/CODE_OF_CONDUCT.en.md b/CODE_OF_CONDUCT.en.md new file mode 100644 index 0000000..bcde3d3 --- /dev/null +++ b/CODE_OF_CONDUCT.en.md @@ -0,0 +1,75 @@ +# Code of Conduct + +> [繁體中文](./CODE_OF_CONDUCT.md) | [简体中文](./CODE_OF_CONDUCT.zh-Hans.md) | **English** + +This project adopts [Contributor Covenant 2.1](https://www.contributor-covenant.org/version/2/1/code_of_conduct/) as its community code of conduct. + +## Our Pledge + +We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation. + +We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community. + +## Our Standards + +Examples of behavior that contributes to a positive environment for our community include: + +- Demonstrating empathy and kindness toward other people +- Being respectful of differing opinions, viewpoints, and experiences +- Giving and gracefully accepting constructive feedback +- Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience +- Focusing on what is best not just for us as individuals, but for the overall community + +Examples of unacceptable behavior include: + +- The use of sexualized language or imagery, and sexual attention or advances of any kind +- Trolling, insulting or derogatory comments, and personal or political attacks +- Public or private harassment +- Publishing others’ private information, such as a physical or email address, without their explicit permission +- Other conduct which could reasonably be considered inappropriate in a professional setting + +## Enforcement Responsibilities + +Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful. + +Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate. + +## Scope + +This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. + +## Enforcement + +Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the maintainer through the following channels: + +- Open an issue in this repo with a title prefixed by `[conduct]`, and **@WenyuChiou**; if the content is sensitive and should not be public, prefer GitHub DM to **@WenyuChiou** (this goes directly to the maintainer), or use GitHub's **"Report content"** mechanism to report it to the platform (this goes to GitHub Trust & Safety, not the maintainer) +- All complaints will be reviewed and investigated promptly and fairly + +All community leaders are obligated to respect the privacy and security of the reporter of any incident. + +## Enforcement Guidelines + +Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct: + +1. **Correction** - Community Impact: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community. + + Consequence: A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested. + +2. **Warning** - Community Impact: A violation through a single incident or series of actions. + + Consequence: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban. + +3. **Temporary Ban** - Community Impact: A serious violation of community standards, including sustained inappropriate behavior. + + Consequence: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban. + +4. **Permanent Ban** - Community Impact: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals. + + Consequence: A permanent ban from any sort of public interaction within the community. + +## Attribution + +This Code of Conduct is adapted from the Contributor Covenant, version 2.1, available at: + + +Community Impact Guidelines were inspired by [Mozilla’s code of conduct enforcement ladder](https://github.com/mozilla/diversity). diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md new file mode 100644 index 0000000..3b17c81 --- /dev/null +++ b/CODE_OF_CONDUCT.md @@ -0,0 +1,64 @@ +# 貢獻者公約 / Code of Conduct + +> **繁體中文** | [简体中文](./CODE_OF_CONDUCT.zh-Hans.md) | [English](./CODE_OF_CONDUCT.en.md) + +本專案採用 [Contributor Covenant 2.1](https://www.contributor-covenant.org/version/2/1/code_of_conduct/) 作為社群行為準則。 + +## 我們的承諾 + +為了營造開放且友善的環境,我們以貢獻者與維護者的身分承諾:讓參與本專案及社群的每一個人,都不會因為年齡、體型、可見或不可見的身心障礙、族裔、性徵、性別認同與表達、經驗水準、教育程度、社經地位、國籍、外貌、種族、宗教,或性取向而受到騷擾。 + +我們承諾以有助於建立開放、友善、多元、包容且健康社群的方式行動與互動。 + +## 我們的準則 + +有助於營造正面環境的行為包括: + +- 對他人展現同理與善意 +- 尊重不同的意見、觀點與經驗 +- 給予並優雅地接受建設性回饋 +- 為自己的錯誤負責、向受影響的人致歉,並從經驗中學習 +- 以整個社群的最大利益為重,而非只看個人 + +不可接受的行為包括: + +- 使用情色化的語言或影像,以及任何形式的性騷擾或性挑逗 +- 出言挑釁、侮辱或貶低性的言論,以及人身或政治攻擊 +- 公開或私下的騷擾 +- 未經明確許可,公開他人的私人資訊(例如真實姓名、地址、電子郵件) +- 其他在專業場合中可合理認定為不當的行為 + +## 執行責任 + +社群維護者有責任闡明並執行可接受行為的準則,對任何他們認為不當、具威脅性、冒犯或有害的行為,採取適當且公平的糾正措施。 + +維護者有權移除、編輯或拒絕不符合本準則的留言、commit、程式碼、wiki 編輯、issue 及其他貢獻,並在適當時說明理由。 + +## 適用範圍 + +本準則適用於所有專案空間,以及個人在公開場合代表專案或其社群的情況(例如使用官方信箱、透過官方社群帳號發文、或在線上/線下活動中擔任指定代表)。 + +## 執行 + +若遇到辱罵、騷擾或其他不可接受的行為,可透過以下管道向維護者檢舉: + +- 在本 repo 開一個標題以 `[conduct]` 開頭的 issue,並 **@WenyuChiou**;若內容敏感不宜公開,請優先透過 GitHub 私訊 **@WenyuChiou**(這會直接送到維護者),或使用 GitHub 的 **「Report content / 檢舉內容」** 機制向平台舉報(這是送到 GitHub 信任與安全團隊,非維護者) +- 所有檢舉都會被即時且公平地審視與處理 + +維護者有義務尊重檢舉者的隱私與安全。 + +## 執行準則 + +維護者會依以下「社群影響準則」判斷違規後果: + +1. **更正** — 私下書面提醒,說明違規之處,可能要求公開道歉。 +2. **警告** — 對單一事件或一連串行為提出警告,並在一段時間內限制互動。 +3. **暫時停權** — 一段時間內禁止與社群任何形式的互動。 +4. **永久停權** — 永久禁止參與社群。 + +## 歸屬 + +本準則改編自 [Contributor Covenant](https://www.contributor-covenant.org) 2.1 版,原文: + + +社群影響準則受 [Mozilla 的行為準則執行階梯](https://github.com/mozilla/diversity) 啟發。 diff --git a/CODE_OF_CONDUCT.zh-Hans.md b/CODE_OF_CONDUCT.zh-Hans.md new file mode 100644 index 0000000..2826397 --- /dev/null +++ b/CODE_OF_CONDUCT.zh-Hans.md @@ -0,0 +1,64 @@ +# 贡献者公约 / Code of Conduct + +> [繁體中文](./CODE_OF_CONDUCT.md) | **简体中文** | [English](./CODE_OF_CONDUCT.en.md) + +本项目采用 [Contributor Covenant 2.1](https://www.contributor-covenant.org/version/2/1/code_of_conduct/) 作为社区行为准则。 + +## 我们的承诺 + +为了营造开放且友善的环境，我们以贡献者与维护者的身份承诺：让参与本项目及社区的每一个人，都不会因为年龄、体型、可见或不可见的身心障碍、族裔、性征、性别认同与表达、经验水平、教育程度、社会经济地位、国籍、外貌、种族、宗教，或性取向而受到骚扰。 + +我们承诺以有助于建立开放、友善、多元、包容且健康社区的方式行动与互动。 + +## 我们的准则 + +有助于营造正面环境的行为包括： + +- 对他人展现同理与善意 +- 尊重不同的意见、观点与经验 +- 给予并优雅地接受建设性反馈 +- 为自己的错误负责、向受影响的人道歉，并从经验中学习 +- 以整个社区的最大利益为重，而非只看个人 + +不可接受的行为包括： + +- 使用情色化的语言或图像，以及任何形式的性骚扰或性挑逗 +- 出言挑衅、侮辱或贬低性的言论，以及人身或政治攻击 +- 公开或私下的骚扰 +- 未经明确许可，公开他人的私人信息（例如真实姓名、地址、电子邮件） +- 其他在专业场合中可合理认定为不当的行为 + +## 执行责任 + +社区维护者有责任阐明并执行可接受行为的准则，对任何他们认为不当、具威胁性、冒犯或有害的行为，采取适当且公平的纠正措施。 + +维护者有权移除、编辑或拒绝不符合本准则的留言、commit、代码、wiki 编辑、issue 及其他贡献，并在适当时说明理由。 + +## 适用范围 + +本准则适用于所有项目空间，以及个人在公开场合代表项目或其社区的情况（例如使用官方邮箱、通过官方社交账号发文、或在线上/线下活动中担任指定代表）。 + +## 执行 + +若遇到辱骂、骚扰或其他不可接受的行为，可通过以下渠道向维护者举报： + +- 在本 repo 开一个标题以 `[conduct]` 开头的 issue，并 **@WenyuChiou**；若内容敏感不宜公开，请优先通过 GitHub 私信 **@WenyuChiou**（这会直接送到维护者），或使用 GitHub 的 **“Report content / 举报内容”** 机制向平台举报（这是送到 GitHub Trust & Safety 团队，不是维护者） +- 所有举报都会被及时且公平地审视与处理 + +维护者有义务尊重举报者的隐私与安全。 + +## 执行准则 + +维护者会依以下“社区影响准则”判断违规后果： + +1. **更正** — 私下书面提醒，说明违规之处，可能要求公开道歉。 +2. **警告** — 对单一事件或一连串行为提出警告，并在一段时间内限制互动。 +3. **暂时封禁** — 一段时间内禁止与社区任何形式的互动。 +4. **永久封禁** — 永久禁止参与社区。 + +## 归属 + +本准则改编自 [Contributor Covenant](https://www.contributor-covenant.org) 2.1 版，原文： + + +社区影响准则受 [Mozilla 的行为准则执行阶梯](https://github.com/mozilla/diversity) 启发。 diff --git a/CONTRIBUTING.en.md b/CONTRIBUTING.en.md new file mode 100644 index 0000000..d8122bf --- /dev/null +++ b/CONTRIBUTING.en.md @@ -0,0 +1,117 @@ +# Contributing + +> [繁體中文](./CONTRIBUTING.md) | [简体中文](./CONTRIBUTING.zh-Hans.md) | **English** + + +Thanks for considering a contribution. **This is a curated learning roadmap, not an exhaustive catalog. Quality > quantity.** + +This repo is **designed for community-driven improvement** — one person can't keep pace with the AI agent ecosystem alone. The maintainer's quarterly review isn't enough; more eyes are needed. + +The catalog is split into **two tracks**: **Track A** (CLI Power User, `tracks/cli/A1-A3`) and **Track B** (Agent Builder, `stages/03-07`). When contributing, please indicate which track you're modifying — the two have different audiences. + +## 🚪 First-time contribution: 5 easy starting points + +Not sure where to start? Pick one you can finish in 30 minutes: + +1. **🐛 Report a stale entry**: run `python scripts/refresh-stars.py`, find repos with significant star drift, open an issue saying "this should be removed / updated" +2. **🔗 Fix one broken link**: hit a 404 reading stage X? Just PR the fix +3. **✍️ Fill in an entry's "Run it" section**: many entries lack install commands; if you've run it, add them +4. **🌏 Improve one English companion sentence**: pick any `.en.md`, compare to the zh version, fix one awkward translation +5. **💬 Add a personal note to an entry**: stuck on `Exercise 3`? Add a "Note: xxx" line + +None of these require reading the full style-guide first; they merge fast — perfect for a first PR. + +> 🧪 **Running the walkthrough / build script / CI workflow for the first time?** See [`.github/TESTING-STATUS.md`](.github/TESTING-STATUS.md) — an **honest disclosure** of what the maintainer has actually executed vs only syntax-checked vs not tested at all. Being the first to hit a bug and open an issue + PR is the highest-value contribution. + +## What We Accept + +### High-value PRs +- **Adding a project** to a stage with reasoning for why it teaches that stage +- **Translating** a stage page to 繁中 (Traditional Chinese only — we are NOT zh-Hans) +- **Flagging stale / unmaintained projects** (open an issue first) +- **Improving curation notes** on existing projects (clearer "what it teaches" explanations) +- **Reorganizing** within a stage if the current ordering doesn't match learning progression + +### Lower priority (still welcome) +- Typo fixes +- Link fixes (verify with `curl -I` first) +- Stage description polish + +### Not accepted +- Bulk additions of repos without curation reasoning +- Self-promotion without educational value +- Projects with no documentation +- Projects without clear license + +## How to Add a Project + +Each project in a stage page should follow this format: + +```markdown +### [Project Name](url) + +| Field | Value | +|---|---| +| Language | Python / TS / etc. | +| Stars | ★ k | +| License | MIT / Apache 2 / ... | +| Recommendation | ⭐⭐⭐⭐ | + +**What it teaches**: 1-sentence summary of the core learning. + +**Best for**: who should study this and why. + +**Notes**: 1-3 sentence personal evaluation. What's strong, what's weak, what to skip. + +**Run it**: +\`\`\`bash +# minimal install / first-run command +\`\`\` +``` + +## Curation Criteria + +A project worth listing must have: + +1. **Active maintenance**: commits within last 6 months OR explicit "stable, no longer maintained" notice +2. **Documented hello-world**: a reader should be able to run something within 30 minutes +3. **Clear license**: MIT, Apache 2, BSD, or comparable. Avoid no-license repos. +4. **Trustworthy maintainer**: well-known org, company, or individual with track record + +## Bilingual Style + +- **Traditional Chinese (zh-TW) is canonical**; English (`*.en.md`) is the companion. +- **No zh-Hans PRs accepted**. If you submit zh-Hans we'll ask you to convert. +- **Natural translation**, not word-for-word. Technical terms can stay in English where natural ("使用 LangGraph 建 multi-agent 系統"). +- **Full style rules: see [`resources/style-guide.md`](resources/style-guide.md)** (zh) or [`resources/style-guide.en.md`](resources/style-guide.en.md) (en) — banned words, entry schema, license conventions, writing style, recommendation star definitions all live there. Read before PR. + +## Process + +1. Open an issue first for new projects or bigger restructuring +2. PR with focused scope (one stage at a time) +3. Wait for review (typically 7 days) +4. Reviewer may ask for clarification on "why this teaches that stage" + +## Anti-patterns to Avoid + +- ❌ "leverage", "delve", "comprehensive", "robust" (LLM-tells) +- ❌ Hype framing ("revolutionary", "game-changing") +- ❌ Listing a project just because it's popular +- ❌ Long quotes from the project's own marketing copy + +## Becoming a Stage / Branch Maintainer + +Beyond one-shot PRs, we welcome **long-term maintainers** for specific stages +or branches — responsible for periodic review, triaging issues in that area, +and gating PRs. + +Self-nomination process: +1. Open an issue titled `[maintainer] Stage N — your-handle` or `[maintainer] for-X branch — your-handle` +2. State your time commitment (suggested: at least one quarter = 3 months) +3. Briefly describe your background in this area + +See [`CONTRIBUTORS.md`](CONTRIBUTORS.md) for the current maintainer roster. + +## License + +By contributing, you agree your work is licensed under MIT. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..5d1a19c --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,114 @@ +# 貢獻指南 + +> **繁體中文** | [简体中文](./CONTRIBUTING.zh-Hans.md) | [English](./CONTRIBUTING.en.md) + +謝謝你考慮貢獻。**這是一份精選的學習路線圖，不是百科目錄。品質 > 數量。** + +這個 repo **本來就是設計給社群一起改良的**——一個人 curate 永遠跟不上 AI agent 生態的變化速度。Maintainer 一個季度跑 1 次 review 不夠，需要更多眼睛看。 + +這份 catalog 分**兩條軌道**：**Track A**（CLI Power User，`tracks/cli/A1-A3`）跟 **Track B**（Agent Builder，`stages/03-07`）。貢獻時請註明你動的是哪條軌道——兩條的 audience 不一樣。 + +## 🚪 第一次貢獻：好上手的 5 個切入點 + +不確定從哪開始？挑一個你 30 分鐘內能做完的： + +1. **🐛 回報過時 entry**：跑 `python scripts/refresh-stars.py` 找星數差距大的 repo，開 issue 說「這個應該移除 / 更新」 +2. **🔗 修一個失效連結**：你看 stage X 時連結 404 了，直接 PR 改 +3. **✍️ 補一個 entry 的 `怎麼跑` section**：很多 entry 沒寫安裝指令，你跑過就補上 +4. **🌏 補英文 companion 沒翻好的句子**：找一個 `.en.md` 跟 zh 對照，你覺得翻得不順的地方改一行 +5. **💬 對某個 entry 加個人筆記**：你跑過 `練習 3` 卡某個地方，補一句「注意：xxx」 + +這 5 種都不用先讀完整份 style-guide，merge 速度也快——適合第一次貢獻、累積信心。 + +> 🧪 **想跑 walkthrough / build script / CI workflow 第一次？** 看 [`.github/TESTING-STATUS.md`](.github/TESTING-STATUS.md)——這份**誠實揭露**哪些 code maintainer 真的跑過、哪些只 syntax check、哪些完全沒測。第一個踩到坑的人開 issue + PR 是 highest-value contribution。 + +## 我們接受什麼 + +### 高價值 PR +- **新增 project** 到某個 stage，並說明為什麼這個 project 對應該階段的學習 +- **翻譯** 某個 stage 頁面成繁中（只要繁中——我們不收 zh-Hans） +- **標記停滯 / 失維護的 project**（請先開 issue） +- **改善現有 project 的策展備註**（讓「教什麼」說明更清楚） +- **重新整理** 某個 stage 內部順序，如果現在的順序不符合學習進程 + +### 較低優先（仍然歡迎） +- 錯字修正 +- 連結修正（請先用 `curl -I` 驗證） +- Stage 介紹文字優化 + +### 不接受 +- 沒有策展理由的批量加 repo +- 沒有教學價值的自我推銷 +- 沒文件的 project +- 沒明確 license 的 project + +## 怎麼新增一個 project + +每一個 project 在 stage 頁面內應該照這個格式： + +```markdown +### [Project Name](url) + +| 欄位 | 內容 | +|---|---| +| 語言 | Python / TS / etc. | +| Stars | ★ k | +| License | MIT / Apache 2 / ... | +| 推薦度 | ⭐⭐⭐⭐ | + +**教什麼**：核心學習一句話總結。 + +**適合誰**：誰應該讀這個、為什麼。 + +**備註**：1-3 句的個人評價。哪裡好、哪裡弱、哪裡可以跳。 + +**怎麼跑**： +\`\`\`bash +# 最小安裝 / 第一次跑的指令 +\`\`\` +``` + +## 策展標準 + +值得列入的 project 必須： + +1. **有維護**：最近 6 個月內有 commit，或明確標示「stable, no longer maintained」 +2. **有 hello-world 文件**：讀者應該能在 30 分鐘內把東西跑起來 +3. **明確 license**：MIT、Apache 2、BSD 或類似。避免沒 license 的 repo。 +4. **可信賴的維護者**：知名組織、公司，或有口碑的個人 + +## 雙語風格 + +- **繁中（Traditional Chinese, zh-TW）為正本**，英文版（`*.en.md`）是 companion。 +- **不接受 zh-Hans PR**。如果你交 zh-Hans 的 PR，我們會請你轉成繁中。 +- **自然翻譯**，不要逐字對譯。技術詞如果直接用英文比較自然，就保留英文（「使用 LangGraph 建 multi-agent 系統」）。 +- **完整風格規範請看 [`resources/style-guide.md`](resources/style-guide.md)**——禁用詞、entry schema、license 標註慣例、寫作風格、推薦星等定義都在裡面。PR 之前請先讀。 + +## 流程 + +1. 新 project 或大幅重組請先開 issue +2. 一次一個 stage，PR 範圍要聚焦 +3. 等審查（通常 7 天） +4. Reviewer 可能會問你「為什麼這個 project 教這個 stage」 + +## 要避免的反模式 + +- ❌ 「leverage」、「delve」、「comprehensive」、「robust」（LLM tell） +- ❌ 過度行銷（「revolutionary」、「game-changing」） +- ❌ 只因為熱門就列上來 +- ❌ 大段引用 project 自己的行銷文案 + +## 擔任 Stage / Branch 維護者 + +除了交一次性 PR，也歡迎擔任**特定 stage 或 branch 的長期維護者**——負責定期 review、處理該領域的 issue、把關該領域的 PR。 + +自薦流程： +1. 開一個 issue，標題 `[maintainer] Stage N — your-handle` 或 `[maintainer] for-X branch — your-handle` +2. 講清楚你願意 commit 多久（建議至少一季 = 3 個月） +3. 簡述你在這個領域的背景 + +詳見 [`CONTRIBUTORS.md`](CONTRIBUTORS.md)。每個 stage / branch 的 maintainer 名單都在那邊。 + +## License + +貢獻即代表你同意你的內容以 MIT 授權。 diff --git a/CONTRIBUTING.zh-Hans.md b/CONTRIBUTING.zh-Hans.md new file mode 100644 index 0000000..96ea79f --- /dev/null +++ b/CONTRIBUTING.zh-Hans.md @@ -0,0 +1,114 @@ +# 贡献指南 + +> [繁體中文](./CONTRIBUTING.md) | **简体中文** | [English](./CONTRIBUTING.en.md) + +谢谢你考虑贡献。**这是一份精选的学习路线图，不是百科目录。质量 > 数量。** + +这个 repo **本来就是设计给社群一起改良的**——一个人 curate 永远跟不上 AI agent 生态的变化速度。Maintainer 一个季度跑 1 次 review 不够，需要更多眼睛看。 + +这份 catalog 分**两条轨道**：**Track A**（CLI Power User，`tracks/cli/A1-A3`）跟 **Track B**（Agent Builder，`stages/03-07`）。贡献时请注明你动的是哪条轨道——两条的 audience 不一样。 + +## 🚪 第一次贡献：好上手的 5 个切入点 + +不确定从哪开始？挑一个你 30 分钟内能做完的： + +1. **🐛 回报过时 entry**：跑 `python scripts/refresh-stars.py` 找星数差距大的 repo，开 issue 说“这个应该移除 / 更新” +2. **🔗 修一个失效连结**：你看 stage X 时连结 404 了，直接 PR 改 +3. **✍️ 补一个 entry 的 `怎么跑` section**：很多 entry 没写安装指令，你跑过就补上 +4. **🌏 补英文 companion 没翻好的句子**：找一个 `.en.md` 跟 zh 对照，你觉得翻得不顺的地方改一行 +5. **💬 对某个 entry 加个人笔记**：你跑过 `练习 3` 卡某个地方，补一句“注意：xxx” + +这 5 种都不用先读完整份 style-guide，merge 速度也快——适合第一次贡献、累积信心。 + +> 🧪 **想跑 walkthrough / build script / CI workflow 第一次？** 看 [`.github/TESTING-STATUS.md`](.github/TESTING-STATUS.md)——这份**诚实揭露**哪些 code maintainer 真的跑过、哪些只 syntax check、哪些完全没测。第一个踩到坑的人开 issue + PR 是 highest-value contribution。 + +## 我们接受什么 + +### 高价值 PR +- **新增 project** 到某个 stage，并说明为什么这个 project 对应该阶段的学习 +- **翻译** 某个 stage 页面成繁中（只要繁中——我们不收 zh-Hans） +- **标记停滞 / 失维护的 project**（请先开 issue） +- **改善现有 project 的策展备注**（让“教什么”说明更清楚） +- **重新整理** 某个 stage 内部顺序，如果现在的顺序不符合学习进程 + +### 较低优先（仍然欢迎） +- 错字修正 +- 连结修正（请先用 `curl -I` 验证） +- Stage 介绍文字优化 + +### 不接受 +- 没有策展理由的批量加 repo +- 没有教学价值的自我推销 +- 没文件的 project +- 没明确 license 的 project + +## 怎么新增一个 project + +每一个 project 在 stage 页面内应该照这个格式： + +```markdown +### [Project Name](url) + +| 栏位 | 内容 | +|---|---| +| 语言 | Python / TS / etc. | +| Stars | ★ k | +| License | MIT / Apache 2 / ... | +| 推荐度 | ⭐⭐⭐⭐ | + +**教什么**：核心学习一句话总结。 + +**适合谁**：谁应该读这个、为什么。 + +**备注**：1-3 句的个人评价。哪里好、哪里弱、哪里可以跳。 + +**怎么跑**： +\`\`\`bash +# 最小安装 / 第一次跑的指令 +\`\`\` +``` + +## 策展标准 + +值得列入的 project 必须： + +1. **有维护**：最近 6 个月内有 commit，或明确标示“stable, no longer maintained” +2. **有 hello-world 文件**：读者应该能在 30 分钟内把东西跑起来 +3. **明确 license**：MIT、Apache 2、BSD 或类似。避免没 license 的 repo。 +4. **可信赖的维护者**：知名组织、公司，或有口碑的个人 + +## 双语风格 + +- **繁中（Traditional Chinese, zh-TW）为正本**，英文版（`*.en.md`）是 companion。 +- **不接受 zh-Hans PR**。如果你交 zh-Hans 的 PR，我们会请你转成繁中。 +- **自然翻译**，不要逐字对译。技术词如果直接用英文比较自然，就保留英文（“使用 LangGraph 建 multi-agent 系统”）。 +- **完整风格规范请看 [`resources/style-guide.zh-Hans.md`](resources/style-guide.zh-Hans.md)**——禁用词、entry schema、license 标注惯例、写作风格、推荐星等定义都在里面。PR 之前请先读。 + +## 流程 + +1. 新 project 或大幅重组请先开 issue +2. 一次一个 stage，PR 范围要聚焦 +3. 等审查（通常 7 天） +4. Reviewer 可能会问你“为什么这个 project 教这个 stage” + +## 要避免的反模式 + +- ❌ “leverage”、“delve”、“comprehensive”、“robust”（LLM tell） +- ❌ 过度行销（“revolutionary”、“game-changing”） +- ❌ 只因为热门就列上来 +- ❌ 大段引用 project 自己的行销文案 + +## 担任 Stage / Branch 维护者 + +除了交一次性 PR，也欢迎担任**特定 stage 或 branch 的长期维护者**——负责定期 review、处理该领域的 issue、把关该领域的 PR。 + +自荐流程： +1. 开一个 issue，标题 `[maintainer] Stage N — your-handle` 或 `[maintainer] for-X branch — your-handle` +2. 讲清楚你愿意 commit 多久（建议至少一季 = 3 个月） +3. 简述你在这个领域的背景 + +详见 [`CONTRIBUTORS.md`](CONTRIBUTORS.md)。每个 stage / branch 的 maintainer 名单都在那边。 + +## License + +贡献即代表你同意你的内容以 MIT 授权。 diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md new file mode 100644 index 0000000..e2cd41d --- /dev/null +++ b/CONTRIBUTORS.md @@ -0,0 +1,92 @@ +# 貢獻者 / Contributors + +謝謝每一個讓這份 repo 變更好的人。 + +--- + +## 🛠 Maintainer + +- [@WenyuChiou](https://github.com/WenyuChiou) — 創立、整體 curation、phase 1-5 主要作者 + +## 🌱 Stage 維護者 / Stage maintainers + +> 每個 stage 都歡迎社群志願者掛名**長期維護者**——有空時 review 一輪、處理該 stage 的 issue、把關 PR。沒有強制節奏。 +> +> 想擔任 maintainer？開個 issue 自薦就好，不用承諾固定 review 期程——能做幾次就幾次。 + +### 共用基礎（Stage 0-2） + +| Stage | Maintainer | 加入日期 | +|---|---|---| +| Stage 0 — 基礎準備 | （社群 PR 機會） | — | +| Stage 1 — LLM 入門 | （社群 PR 機會） | — | +| Stage 2 — Prompt 設計 | （社群 PR 機會） | — | + +### Track A — CLI Power User + +| Stage | Maintainer | 加入日期 | +|---|---|---| +| A1 — CLI Agent 入門 + 選擇 | （社群 PR 機會） | — | +| A2 — CLI Workflow Patterns | （社群 PR 機會） | — | +| A3 — Integration & Production | （社群 PR 機會） | — | + +### Track B — Agent Builder + +| Stage | Maintainer | 加入日期 | +|---|---|---| +| Stage 3 — Tool Use & Agent 入門 ⭐ | （社群 PR 機會） | — | +| Stage 4 — Agent 框架 | （社群 PR 機會） | — | +| Stage 5 — Claude Code 生態 ⭐⭐（兩條軌共用） | （社群 PR 機會） | — | +| Stage 6 — Memory · RAG · 進階 | （社群 PR 機會） | — | +| Stage 7 — 進階 Multi-Agent | （社群 PR 機會） | — | + +## 🌳 Branch 維護者 / Branch maintainers + +| Branch | Maintainer | 加入日期 | +|---|---|---| +| 🔬 for-researcher | （社群 PR 機會） | — | +| 💻 for-developer | （社群 PR 機會） | — | +| 🎓 for-teacher | **特別歡迎自薦**（學術引用待深化） | — | +| 📊 for-knowledge-worker | **特別歡迎自薦**（目前最薄） | — | +| 👥 for-everyday-users | （社群 PR 機會） | — | + +## 💬 內容貢獻者 / Content contributors + +> 每次 merged PR 或實質 issue 修正都會在這裡留名。第一次 contribution 不分大小都會列入。 + +| Contributor | 貢獻 / Contribution | First contribution | +|---|---|---| +| [@scott0127](https://github.com/scott0127) | **Stage 6 RAG 教材** — chunking strategies guide ([#2](https://github.com/WenyuChiou/awesome-agentic-ai-zh/pull/2))、unit guide overview ([#5](https://github.com/WenyuChiou/awesome-agentic-ai-zh/pull/5)) · **for-teacher 框架** — 3-tier 教師 AI 應用情境 + 學術引用 (Chen 2020 / Mittal 2024) ([#6](https://github.com/WenyuChiou/awesome-agentic-ai-zh/pull/6)) | 2026-05-08 | +| [@xfq](https://github.com/xfq) | **i18n correctness audit** — flagged BCP 47 / W3C compliance issue (`zh-CN` → `zh-Hans`) and provided the script subtag rationale ([#9](https://github.com/WenyuChiou/awesome-agentic-ai-zh/issues/9)). [W3C i18n lead](https://www.w3.org/International/). | 2026-05-10 | +| [@demo112](https://github.com/demo112) | **中國生態 catalog** — 新增 whale (DeepSeek terminal assistant) + a-stock-data (A 股工具包) 到 Chinese ecosystem ([#14](https://github.com/WenyuChiou/awesome-agentic-ai-zh/pull/14)) | 2026-05-14 | +| [@Rain120](https://github.com/Rain120) | **zh-Hans 大陸在地化** — 系統盤點 zh-Hans 鏡像的台灣用詞,促成 `scripts/zh-hans-localize.py`(curated 台→陸詞表 + 大陸引號,已套全 54 檔並進 lint CI gate;貢獻者掛 commit `7f73b8a` Co-Author)([#18](https://github.com/WenyuChiou/awesome-agentic-ai-zh/pull/18)) | 2026-05-16 | +| [@JunLin-Bobby](https://github.com/JunLin-Bobby) | **Stage 3 ReAct 練習 bug fix** — `trace.append` 移進 `for tc in tool_calls:` 迴圈內(原本在外層、一步多 tool 只記錄 `tool_calls[0]`)+ 為 question 內的 lookup key 加引號避免 LLM 跨語轉譯失敗(qwen2.5:3b + gemini-3-flash-preview 雙模型交叉驗證)([#26](https://github.com/WenyuChiou/awesome-agentic-ai-zh/pull/26)) | 2026-05-24 | +| [@thebrierfox](https://github.com/thebrierfox) | **§12 Finance hosted-MCP example** — 新增 YIELD INTELLIGENCE MCP (hosted remote server) 到 §12 其他常用、tri-locale。catalog 第一個用 style-guide §1 非 repo entry exemption 的 hosted-service entry (`形式 = hosted MCP server` 取代 Stars/License)、為 Stage 5 學習者示範 hosted vs self-hosted MCP 架構差異 ([#28](https://github.com/WenyuChiou/awesome-agentic-ai-zh/pull/28)) | 2026-05-26 | + +--- + +## 🤖 AI 工具貢獻 + +這個 repo 有相當部分的繁體中文翻譯、結構審查、license 驗證、跨檔一致性檢查由 AI 工具協助完成： +- **Claude (Anthropic)** — 主要 curation、結構設計、zh-TW 翻譯、跨 phase planning +- **Codex (OpenAI)** — 多輪審查（Phase 2-5 各一次 + cross-phase audit），抓出實質的 license 錯標、overclaim 用語、文件 drift 問題 +- **gh API** — 所有 entry 的 stars / license / pushed-at 都用 `gh api` 驗證過，避免幻覺 + +人工 review 仍是 ground truth——AI 找出**疑似問題**，最終決定（接受、拒絕、改寫）由 maintainer 拍板。 + +--- + +## 怎麼上這份名單 + +1. **Bug report / 連結修正 / 內容更新**：開 issue 或直接 PR +2. **新增 project entry**：照 [`resources/style-guide.md`](resources/style-guide.md) 的 schema 加，[PR template](.github/PULL_REQUEST_TEMPLATE.md) 會引導 checklist +3. **擔任 stage / branch maintainer**：開 issue 自薦，講清楚你願意 commit 多久（建議至少一季） +4. **改善 walkthroughs / scripts / 文件**：直接 PR + +每次合併 PR 後，maintainer 會在這份檔案加你的 GitHub handle + 貢獻摘要。 + +--- + +## License + +本檔案內容遵循 repo 的 MIT license。貢獻即視為同意以 MIT 授權你的內容。 diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..c711193 --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 Wenyu Chiou + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/PROGRESS.en.md b/PROGRESS.en.md new file mode 100644 index 0000000..44b592d --- /dev/null +++ b/PROGRESS.en.md @@ -0,0 +1,82 @@ +# Progress Tracker + +> [繁體中文](./PROGRESS.md) | [简体中文](./PROGRESS.zh-Hans.md) | **English** + +This is a checklist **for your own use**. You do not need to submit it, open a PR, or have anyone inspect it. Copy it (or fork the repo), tick your own progress, and see where you are and what comes next. + +**How to use it**: +1. Each stage's "Learning goals", "Entry requirements", and "Self-check" live in that stage file. This checklist is only an **overview + entry point** and does not repeat the content. +2. A stage's ✅ condition = you can pass the "**Self-check**" section at the end of that stage. Tick it only after passing, then move to the next stop. +3. You do not need to finish everything. Pick one track (Track A or B) + one audience branch to get started. + +> Not sure which track to choose? See the dual-track explanation in [`README.en.md`](README.en.md), or [`branches/DESIGN.md`](branches/DESIGN.md). If you get stuck, open a [Discussion](https://github.com/WenyuChiou/awesome-agentic-ai-zh/discussions). + +--- + +## Shared Foundations (required for both tracks) + +- [ ] **Stage 0 — Foundations** · [`stages/00-foundations.en.md`](stages/00-foundations.en.md) + ✅ Pass this stage's completion criteria (Stage 0 is a prerequisite gateway; see the stage file for the criteria) +- [ ] **Stage 1 — LLM Basics** · [`stages/01-llm-basics.en.md`](stages/01-llm-basics.en.md) + ✅ Pass this stage's "Self-check" +- [ ] **Stage 2 — Prompt Design** · [`stages/02-prompt-engineering.en.md`](stages/02-prompt-engineering.en.md) + ✅ Pass this stage's "Self-check" + +--- + +## Track A — CLI Power User + +> You want to **use** agent tools to get work done, and you do not necessarily want to build them yourself. + +- [ ] **A1 — CLI Agent Introduction + Selection** · [`tracks/cli/A1-cli-intro.en.md`](tracks/cli/A1-cli-intro.en.md) +- [ ] **A2 — CLI Workflow Patterns** · [`tracks/cli/A2-cli-workflow.en.md`](tracks/cli/A2-cli-workflow.en.md) +- [ ] **Stage 5 — Claude Code Ecosystem (shared by both tracks)** · [`stages/05-claude-code-ecosystem.en.md`](stages/05-claude-code-ecosystem.en.md) +- [ ] **A3 — Integration & Production** · [`tracks/cli/A3-cli-production.en.md`](tracks/cli/A3-cli-production.en.md) +- [ ] **Stage 8 — Agent Interfaces (shared by both tracks)** · [`stages/08-agent-interfaces.en.md`](stages/08-agent-interfaces.en.md) + +--- + +## Track B — Agent Builder + +> You want to **build** agents, frameworks, or multi-agent systems yourself. + +- [ ] **Stage 3 — Tool Use and Your First Agent** ⭐ · [`stages/03-tool-use-and-hello-agent.en.md`](stages/03-tool-use-and-hello-agent.en.md) +- [ ] **Stage 4 — Agent Frameworks** · [`stages/04-agent-frameworks.en.md`](stages/04-agent-frameworks.en.md) +- [ ] **Stage 5 — Claude Code Ecosystem** ⭐⭐(shared by both tracks)· [`stages/05-claude-code-ecosystem.en.md`](stages/05-claude-code-ecosystem.en.md) +- [ ] **Stage 6 — Context Management: RAG and Memory** · [`stages/06-memory-rag.en.md`](stages/06-memory-rag.en.md) +- [ ] **Stage 7 — Multi-Agent · Advanced Applications** · [`stages/07-multi-agent-production.en.md`](stages/07-multi-agent-production.en.md) +- [ ] **Stage 7.5 — Advanced Agentic Concepts** · [`stages/07.5-advanced-agentic-concepts.en.md`](stages/07.5-advanced-agentic-concepts.en.md) +- [ ] **Stage 8 — Agent Interfaces (shared by both tracks)** · [`stages/08-agent-interfaces.en.md`](stages/08-agent-interfaces.en.md) + +--- + +## Choose one audience branch (matching your role) + +> A branch is not "another course layer". It maps what you learned in the stages above to your real scenario. Pick just one. + +- [ ] 🔬 **for-researcher** · [`branches/for-researcher.en.md`](branches/for-researcher.en.md) +- [ ] 💻 **for-developer** · [`branches/for-developer.en.md`](branches/for-developer.en.md) +- [ ] 🎓 **for-teacher** · [`branches/for-teacher.en.md`](branches/for-teacher.en.md) +- [ ] 📊 **for-knowledge-worker** · [`branches/for-knowledge-worker.en.md`](branches/for-knowledge-worker.en.md) +- [ ] 👥 **for-everyday-users** · [`branches/for-everyday-users.en.md`](branches/for-everyday-users.en.md) + +--- + +## Capstone + +After finishing a track, build something you can show + self-assess. The brief, requirements, and 4-level rubric all live in [`CAPSTONE.en.md`](CAPSTONE.en.md) — this list is just the checkbox entry point; the standard is not duplicated. + +- [ ] **Track A Capstone** — assemble a reusable CLI-agent workflow · [`CAPSTONE.en.md`](CAPSTONE.en.md) +- [ ] **Track B Capstone** — build + **evaluate** a multi-agent / RAG system · [`CAPSTONE.en.md`](CAPSTONE.en.md) + +--- + +## Shortest viable path (if you only want one recommendation) + +Do not want to plan it yourself? Follow this path to reach "able to do hands-on work" with roughly the fewest detours: + +`Stage 0 → Stage 1 → Stage 2 →` choose a track `→`(Track A: `A1 → A2 → Stage 5 → A3`;Track B: `Stage 3 → Stage 4 → Stage 5 → Stage 6`)`→` your branch `→`(advanced, applies to Track B: `Stage 7 → 7.5 → 8`;Track A's Stage 8 is already in the main path above)`→` your track's **Capstone** (see [`CAPSTONE.en.md`](CAPSTONE.en.md)) + +--- + +> This checklist only tracks "where you are". For each stop, "what to learn / what you need before entering / how to know you learned it" is always defined by that stage file's "Learning goals / Entry requirements / Self-check" sections, to avoid splitting the same standard across two places. diff --git a/PROGRESS.md b/PROGRESS.md new file mode 100644 index 0000000..0a87d33 --- /dev/null +++ b/PROGRESS.md @@ -0,0 +1,82 @@ +# 學習進度追蹤 / Progress Tracker + +> **繁體中文** | [简体中文](./PROGRESS.zh-Hans.md) | [English](./PROGRESS.en.md) + +這是一份**給你自己用**的打勾清單——不用提交、不用 PR、沒有人會檢查。複製一份(或 fork repo)勾你自己的進度,知道走到哪、下一站是哪。 + +**怎麼用**: +1. 每個 stage 的「學習目標」「進入條件」「自我檢查」都在該 stage 檔案裡——這份清單只是**總覽 + 入口**,不重複內容。 +2. 一個 stage 的 ✅ 條件 = 你能通過該 stage 結尾的「**自我檢查**」那一節。通過了才勾,勾完往下一站。 +3. 不用全部做完。先選一條軌道(Track A 或 B)+ 一條你的 audience branch 就夠開始。 + +> 不確定選哪條?看 [`README.md`](README.md) 的雙軌說明,或 [`branches/DESIGN.md`](branches/DESIGN.md)。卡住開 [Discussion](https://github.com/WenyuChiou/awesome-agentic-ai-zh/discussions)。 + +--- + +## 共用基礎(兩條軌道都要) + +- [ ] **Stage 0 — 基礎準備** · [`stages/00-foundations.md`](stages/00-foundations.md) + ✅ 過該 stage 的通過條件(Stage 0 是 prerequisite gateway,通過條件見 stage 內說明) +- [ ] **Stage 1 — LLM 基礎** · [`stages/01-llm-basics.md`](stages/01-llm-basics.md) + ✅ 過該 stage 的「自我檢查」 +- [ ] **Stage 2 — Prompt 設計** · [`stages/02-prompt-engineering.md`](stages/02-prompt-engineering.md) + ✅ 過該 stage 的「自我檢查」 + +--- + +## Track A — CLI Power User + +> 你想「**用** agent 工具把工作做完」,不一定要自己 build。 + +- [ ] **A1 — CLI Agent 入門 + 選擇** · [`tracks/cli/A1-cli-intro.md`](tracks/cli/A1-cli-intro.md) +- [ ] **A2 — CLI Workflow Patterns** · [`tracks/cli/A2-cli-workflow.md`](tracks/cli/A2-cli-workflow.md) +- [ ] **Stage 5 — Claude Code 生態(兩軌共用)** · [`stages/05-claude-code-ecosystem.md`](stages/05-claude-code-ecosystem.md) +- [ ] **A3 — Integration & Production** · [`tracks/cli/A3-cli-production.md`](tracks/cli/A3-cli-production.md) +- [ ] **Stage 8 — Agent 操作介面(兩軌共用)** · [`stages/08-agent-interfaces.md`](stages/08-agent-interfaces.md) + +--- + +## Track B — Agent Builder + +> 你想「**自己 build** agent / 框架 / 多 agent 系統」。 + +- [ ] **Stage 3 — 工具使用與第一個 Agent** ⭐ · [`stages/03-tool-use-and-hello-agent.md`](stages/03-tool-use-and-hello-agent.md) +- [ ] **Stage 4 — Agent 框架** · [`stages/04-agent-frameworks.md`](stages/04-agent-frameworks.md) +- [ ] **Stage 5 — Claude Code 生態** ⭐⭐(兩軌共用)· [`stages/05-claude-code-ecosystem.md`](stages/05-claude-code-ecosystem.md) +- [ ] **Stage 6 — 上下文管理:RAG 與 Memory** · [`stages/06-memory-rag.md`](stages/06-memory-rag.md) +- [ ] **Stage 7 — Multi-Agent · 進階應用** · [`stages/07-multi-agent-production.md`](stages/07-multi-agent-production.md) +- [ ] **Stage 7.5 — 進階 Agentic 概念** · [`stages/07.5-advanced-agentic-concepts.md`](stages/07.5-advanced-agentic-concepts.md) +- [ ] **Stage 8 — Agent 操作介面(兩軌共用)** · [`stages/08-agent-interfaces.md`](stages/08-agent-interfaces.md) + +--- + +## 選一條 audience branch(對應你的身分) + +> Branch 不是「再上一層課」,是把上面 stage 學到的東西**對應到你的實際場景**。挑 1 條就好。 + +- [ ] 🔬 **for-researcher** · [`branches/for-researcher.md`](branches/for-researcher.md) +- [ ] 💻 **for-developer** · [`branches/for-developer.md`](branches/for-developer.md) +- [ ] 🎓 **for-teacher** · [`branches/for-teacher.md`](branches/for-teacher.md) +- [ ] 📊 **for-knowledge-worker** · [`branches/for-knowledge-worker.md`](branches/for-knowledge-worker.md) +- [ ] 👥 **for-everyday-users** · [`branches/for-everyday-users.md`](branches/for-everyday-users.md) + +--- + +## 結業專題 / Capstone + +走完一條軌道後,做一個能展示的作品 + 自評。題目、必要條件、四級評分 rubric 全在 [`CAPSTONE.md`](CAPSTONE.md)——這份清單只放打勾入口,標準不重複。 + +- [ ] **Track A Capstone** — 組一條會重複用的 CLI-agent 工作流 · [`CAPSTONE.md`](CAPSTONE.md) +- [ ] **Track B Capstone** — build + **評測** 一個 multi-agent / RAG 系統 · [`CAPSTONE.md`](CAPSTONE.md) + +--- + +## 一條最短可行路線(如果你只想要一個建議) + +不想自己排?照這個走,大約能在最少繞路下到「能動手做事」: + +`Stage 0 → Stage 1 → Stage 2 →` 選軌道 `→`(Track A: `A1 → A2 → Stage 5 → A3`;Track B: `Stage 3 → Stage 4 → Stage 5 → Stage 6`)`→` 你的 branch `→`(進階,Track B 適用:`Stage 7 → 7.5 → 8`;Track A 的 Stage 8 已在上方主線)`→` 你那軌的 **Capstone**(見 [`CAPSTONE.md`](CAPSTONE.md)) + +--- + +> 這份清單只追蹤「你走到哪」。每一站「學什麼 / 進入前要會什麼 / 怎麼算學會」一律以該 stage 檔案內的「學習目標 / 進入條件 / 自我檢查」為準——避免同一份標準散兩處。 diff --git a/PROGRESS.zh-Hans.md b/PROGRESS.zh-Hans.md new file mode 100644 index 0000000..1f4aaa2 --- /dev/null +++ b/PROGRESS.zh-Hans.md @@ -0,0 +1,82 @@ +# 学习进度追踪 / Progress Tracker + +> [繁體中文](./PROGRESS.md) | **简体中文** | [English](./PROGRESS.en.md) + +这是一份**给你自己用**的打勾清单——不用提交、不用 PR、没人会检查。复制一份（或 fork repo）勾你自己的进度，知道走到哪、下一站是哪。 + +**怎么用**： +1. 每个 stage 的“学习目标”“进入条件”“自我检查”都在该 stage 文件里——这份清单只是**总览 + 入口**，不重复内容。 +2. 一个 stage 的 ✅ 条件 = 你能通过该 stage 结尾的“**自我检查**”那一节。通过了才勾，勾完往下一站。 +3. 不用全部做完。先选一条轨道（Track A 或 B）+ 一条你的 audience branch 就够开始。 + +> 不确定选哪条？看 [`README.zh-Hans.md`](README.zh-Hans.md) 的双轨说明，或 [`branches/DESIGN.md`](branches/DESIGN.md)。卡住开 [Discussion](https://github.com/WenyuChiou/awesome-agentic-ai-zh/discussions)。 + +--- + +## 共用基础（两条轨道都要） + +- [ ] **Stage 0 — 基础准备** · [`stages/00-foundations.zh-Hans.md`](stages/00-foundations.zh-Hans.md) + ✅ 通过该 stage 的通过条件（Stage 0 是 prerequisite gateway，通过条件见 stage 内说明） +- [ ] **Stage 1 — LLM 基础** · [`stages/01-llm-basics.zh-Hans.md`](stages/01-llm-basics.zh-Hans.md) + ✅ 通过该 stage 的“自我检查” +- [ ] **Stage 2 — Prompt 设计** · [`stages/02-prompt-engineering.zh-Hans.md`](stages/02-prompt-engineering.zh-Hans.md) + ✅ 通过该 stage 的“自我检查” + +--- + +## Track A — CLI Power User + +> 你想“**用** agent 工具把工作做完”，不一定要自己 build。 + +- [ ] **A1 — CLI Agent 入门 + 选择** · [`tracks/cli/A1-cli-intro.zh-Hans.md`](tracks/cli/A1-cli-intro.zh-Hans.md) +- [ ] **A2 — CLI Workflow Patterns** · [`tracks/cli/A2-cli-workflow.zh-Hans.md`](tracks/cli/A2-cli-workflow.zh-Hans.md) +- [ ] **Stage 5 — Claude Code 生态(两轨共用)** · [`stages/05-claude-code-ecosystem.zh-Hans.md`](stages/05-claude-code-ecosystem.zh-Hans.md) +- [ ] **A3 — Integration & Production** · [`tracks/cli/A3-cli-production.zh-Hans.md`](tracks/cli/A3-cli-production.zh-Hans.md) +- [ ] **Stage 8 — Agent 操作界面(两轨共用)** · [`stages/08-agent-interfaces.zh-Hans.md`](stages/08-agent-interfaces.zh-Hans.md) + +--- + +## Track B — Agent Builder + +> 你想“**自己 build** agent / 框架 / 多 agent 系统”。 + +- [ ] **Stage 3 — 工具使用与第一个 Agent** ⭐ · [`stages/03-tool-use-and-hello-agent.zh-Hans.md`](stages/03-tool-use-and-hello-agent.zh-Hans.md) +- [ ] **Stage 4 — Agent 框架** · [`stages/04-agent-frameworks.zh-Hans.md`](stages/04-agent-frameworks.zh-Hans.md) +- [ ] **Stage 5 — Claude Code 生态** ⭐⭐(两轨共用)· [`stages/05-claude-code-ecosystem.zh-Hans.md`](stages/05-claude-code-ecosystem.zh-Hans.md) +- [ ] **Stage 6 — 上下文管理：RAG 与 Memory** · [`stages/06-memory-rag.zh-Hans.md`](stages/06-memory-rag.zh-Hans.md) +- [ ] **Stage 7 — Multi-Agent · 进阶应用** · [`stages/07-multi-agent-production.zh-Hans.md`](stages/07-multi-agent-production.zh-Hans.md) +- [ ] **Stage 7.5 — 进阶 Agentic 概念** · [`stages/07.5-advanced-agentic-concepts.zh-Hans.md`](stages/07.5-advanced-agentic-concepts.zh-Hans.md) +- [ ] **Stage 8 — Agent 操作界面(两轨共用)** · [`stages/08-agent-interfaces.zh-Hans.md`](stages/08-agent-interfaces.zh-Hans.md) + +--- + +## 选一条 audience branch（对应你的身份） + +> Branch 不是“再上一层课”，是把上面 stage 学到的东西**对应到你的实际场景**。挑 1 条就好。 + +- [ ] 🔬 **for-researcher** · [`branches/for-researcher.zh-Hans.md`](branches/for-researcher.zh-Hans.md) +- [ ] 💻 **for-developer** · [`branches/for-developer.zh-Hans.md`](branches/for-developer.zh-Hans.md) +- [ ] 🎓 **for-teacher** · [`branches/for-teacher.zh-Hans.md`](branches/for-teacher.zh-Hans.md) +- [ ] 📊 **for-knowledge-worker** · [`branches/for-knowledge-worker.zh-Hans.md`](branches/for-knowledge-worker.zh-Hans.md) +- [ ] 👥 **for-everyday-users** · [`branches/for-everyday-users.zh-Hans.md`](branches/for-everyday-users.zh-Hans.md) + +--- + +## 结业专题 / Capstone + +走完一条轨道后,做一个能展示的作品 + 自评。题目、必要条件、四级评分 rubric 全在 [`CAPSTONE.zh-Hans.md`](CAPSTONE.zh-Hans.md)——这份清单只放打勾入口,标准不重复。 + +- [ ] **Track A Capstone** — 组一条会重复用的 CLI-agent 工作流 · [`CAPSTONE.zh-Hans.md`](CAPSTONE.zh-Hans.md) +- [ ] **Track B Capstone** — build + **评测** 一个 multi-agent / RAG 系统 · [`CAPSTONE.zh-Hans.md`](CAPSTONE.zh-Hans.md) + +--- + +## 一条最短可行路线（如果你只想要一个建议） + +不想自己排？照这个走，大约能在最少绕路下到“能动手做事”： + +`Stage 0 → Stage 1 → Stage 2 →` 选轨道 `→`（Track A: `A1 → A2 → Stage 5 → A3`;Track B: `Stage 3 → Stage 4 → Stage 5 → Stage 6`）`→` 你的 branch `→`（进阶，Track B 适用：`Stage 7 → 7.5 → 8`;Track A 的 Stage 8 已在上方主线）`→` 你那轨的 **Capstone**（见 [`CAPSTONE.zh-Hans.md`](CAPSTONE.zh-Hans.md)） + +--- + +> 这份清单只追踪“你走到哪”。每一站“学什么 / 进入前要会什么 / 怎么算学会”一律以该 stage 文件内的“学习目标 / 进入条件 / 自我检查”为准——避免同一份标准散两处。 diff --git a/README.en.md b/README.en.md new file mode 100644 index 0000000..901bf2c --- /dev/null +++ b/README.en.md @@ -0,0 +1,324 @@ +
+ 繁體中文 | 简体中文 | English +
+ +
+ +![AI Agent Learning Roadmap](resources/diagrams/banner.en.png) + +# awesome-agentic-ai-zh + +
+ +[![License](https://img.shields.io/badge/license-MIT-blue)](LICENSE) +[![繁中](https://img.shields.io/badge/語言-繁體中文-red)](README.md) +[![简中](https://img.shields.io/badge/語言-简体中文-orange)](README.zh-Hans.md) +[![EN](https://img.shields.io/badge/lang-English-blue)](README.en.md) +![GitHub stars](https://img.shields.io/github/stars/WenyuChiou/awesome-agentic-ai-zh?logo=github) +![GitHub forks](https://img.shields.io/github/forks/WenyuChiou/awesome-agentic-ai-zh?logo=github) +[![Docs site](https://img.shields.io/badge/docs-Pages-2ea44f)](https://wenyuchiou.github.io/awesome-agentic-ai-zh/) + +> **Trilingual — the English edition is fully maintained, not a thin machine translation** (only ~0.4% of English lines carry any CJK, almost all intentional bilingual term-mapping). zh-TW is the curation source of truth (new content lands there first); the English and 简中 editions track the same structure, with CI checking localization correctness and anchor integrity across all three. + +**Learning roadmap + 240+ curated resources + simple illustrative cases** — three pillars helping you go from "I don't know where to start" to "I can design multi-agent systems". Structured **8-stage** path from LLM fundamentals to multi-agent orchestration, Computer Use / Browser Use / Code Sandbox. + +--- + +## 🎯 Why this exists + +**What this repo is**: **a learning roadmap + 240+ curated resources + simple illustrative cases** — three pillars helping AI / AI-agent learners go from "I don't know where to start" to "I can design multi-agent systems." + +Concretely: + +| Pillar | What it does | Scale | +|---|---|---| +| **Learning roadmap** | Organizes scattered high-quality projects, tutorials, and required reading into **8 stages** (including Stage 5 + Stage 8 as two shared hubs) + 2 tracks + 5 specialized branches, from zero to advanced | 8 stages, 2 tracks | +| **Resource curation** | Each stage curates **240+** projects (star rating, audience, what they teach, how to run) plus an MCP/Skill catalog covering the Chinese AI ecosystem (DeepSeek, Zhipu, Kimi, …) | 240+ projects, 65 MCP/Skill | +| **Simple illustrative cases** | Each stage ships 1-5 **foundational exercises** (70-150 line starter + dual-path Ollama/Anthropic SDK comparison + mock-based tests) | 23 exercise folders | + +After the main path, you go from "**LLM user**" to "**agent system builder**" — capable of designing multi-agent collaboration, writing your own MCP server, and shipping real agent systems. + +--- + +## 📋 Table of Contents + +- [🎯 Why this exists](#-why-this-exists) +- [📚 Quick Start](#-quick-start) +- [🗺️ Learning Map (Two Tracks)](#️-learning-map-two-tracks) +- [💡 How to Learn](#-how-to-learn) +- [📚 Related Resources](#-related-resources) +- [🤝 Contributing](#-contributing) +- [🙏 Acknowledgments](#-acknowledgments) +- [🎓 Citation](#-citation) +- [☕ Support this project](#-support-this-project) +- [License](#license) + +--- + +## 📚 Quick Start + +### 🚀 First time with AI agents / never written code before? + +Start here: **[`resources/setup-guide.en.md`](resources/setup-guide.en.md)** — 30-45 minutes from zero, walks you through getting an API key, installing Python, and running your first LLM hello-world. + +### Read online +- **[Learning Map (Two Tracks)](#️-learning-map-two-tracks)** — read this section to decide Track A or Track B +- **[Stage 0 Foundations](stages/00-foundations.en.md)** — already know Python / git / API? Skip straight to Stage 1 + +### Local clone +```bash +git clone https://github.com/WenyuChiou/awesome-agentic-ai-zh.git +cd awesome-agentic-ai-zh +# Start with stages/00-foundations.en.md +``` + +### ✨ What you get + +- 📖 **Fully free** — MIT-licensed, all content open +- 🗺️ **Two learning tracks** — Track A (CLI Power User) for "use existing CLIs"; Track B (Agent Builder) for "build your own". Shared Stages 0-2 foundation. +- 🛠️ **Foundational hands-on exercises** — 1-5 illustrative exercises per stage (specs + dual-path SDK comparison + success criteria). Positioned as **foundational + roadmap verification** — for chapter-length depth exercises see the hello-agents / Anthropic Cookbook callout in each stage +- 🎯 **240+ curated projects** — each with star rating, audience, what it teaches, how to run (incl. local LLM runners: Ollama, llama.cpp, LocalAI, MLX) +- 🌏 **Trilingual, fully maintained** — zh-TW (canonical) / 简中 / English; the English edition is complete, not a thin mirror +- 🎓 **Beyond frameworks: Claude Code ecosystem** — MCP / Skills / Plugins / SDK full stack +- 🔬 **5 specialized branches** — researcher / developer / teacher / knowledge worker / **everyday user** +- ⏱️ **Time commitment, stated upfront** — Track A 8-10 weeks / Track B 16-22 weeks minimum, 5-7 months realistic (5-8 hr/week part-time) + +--- + +## 🗺️ Learning Map (Two Tracks) + +![AI Agent Learning Map](resources/diagrams/learning-map.en.png) + +After **Stages 0-2 (shared foundations)**, pick a track based on your goal: + +- **Track A — CLI Power User**: you want to **USE** existing CLI agents (Claude Code, Codex, OpenCode, Gemini CLI, etc.) to get work done — not build agents from scratch. 3 sub-stages (A1-A3). +- **Track B — Agent Builder**: you want to **BUILD** your own agents — learn frameworks, write ReAct, design multi-agent systems. Stages 3-8 main path. + +The two tracks are **not mutually exclusive** — most people start with A to get hands-on, then come back to B for internals (or vice versa). Stage 5 (Claude Code Ecosystem) is used by both tracks. + +### Shared Foundations (Stages 0-2) + +| Stage | Topic | Key Content | Time | +|---|---|---|---| +| **0** | [Foundations](stages/00-foundations.en.md) | Python · CLI · git · API · JSON | 1-2 wks | +| **1** | [LLM Fundamentals](stages/01-llm-basics.en.md) | tokens · API · model comparison · local LLM | 1 wk | +| **2** | [Prompt Engineering](stages/02-prompt-engineering.en.md) | system prompts · few-shot · CoT | 1-2 wks | + +### Track A — CLI Power User (use CLIs to get work done) + +| Stage | Topic | Key Content | Time | +|---|---|---|---| +| **A1** | [CLI Agent Intro & Selection](tracks/cli/A1-cli-intro.en.md) | 7-CLI comparison · install · first run | 1 wk | +| **A2** | [CLI Workflow Patterns](tracks/cli/A2-cli-workflow.en.md) | CLAUDE.md · slash commands · multi-step decomposition | 1-2 wks | +| **A3** | [Integration & Production](tracks/cli/A3-cli-production.en.md) | MCP-into-CLI · CI automation · cost / observability | 1-2 wks | +| **+5** | [Stage 5 — Claude Code Ecosystem](stages/05-claude-code-ecosystem.en.md) (**Shared Hub**) | MCP · Skills · Plugins · Subagents; Track A reads 5.1-5.4 (5.5-5.7 optional) | 1-2 wks (Track A view) | +| **+8** | [Stage 8 — Agent Interfaces](stages/08-agent-interfaces.en.md) (**Shared Hub**) | Computer Use · Browser Use · Code Sandbox; Track A reads Track A usage | 1-2 wks (Track A view) | + +> **Track A total time**: includes Stages 0-2 (shared foundations) + A1-A3 + **Stage 5 + Stage 8 (two shared hubs) ≈ 8-10 weeks**. Core reference: [`resources/cli-agents-guide.en.md`](resources/cli-agents-guide.en.md). + +### Track B — Agent Builder (build agents from scratch) + +| Stage | Topic | Key Content | Time | +|---|---|---|---| +| **3** ⭐ | [Tool Use & Hello Agent](stages/03-tool-use-and-hello-agent.en.md) | function calling · ReAct · 5 hands-on exercises | 2-3 wks | +| **4** | [Agent Frameworks](stages/04-agent-frameworks.en.md) | LangGraph · AutoGen · CrewAI · Smolagents | 2-3 wks | +| **5** ⭐⭐ | [Claude Code Ecosystem](stages/05-claude-code-ecosystem.en.md) (**Shared Hub**, Track A also studies) | MCP · Skills · Plugins · Subagents | 3-4 wks (Track B view) | +| **6** | [Context Engineering: RAG and Memory](stages/06-memory-rag.en.md) | vector DB · long-term memory · contextual retrieval | 2 wks | +| **7** | [Multi-Agent · Productionization](stages/07-multi-agent-production.en.md) | multi-agent orchestration · eval · observability · advanced SDK | 2-4 wks | +| **7.5** | [Advanced Agentic Workflow Concepts](stages/07.5-advanced-agentic-concepts.en.md) (reading map) | work boundary · PAR loop · agent-as-judge · 12 advanced concepts + reading list | 1 wk (no code) | +| **8** ⭐⭐ | [Agent Interfaces](stages/08-agent-interfaces.en.md) (**Shared Hub**, Track A also studies) | Computer Use · Browser Use · Code Sandbox; 2024-2026 frontier | 2-3 wks (Track B view) | + +> **Track B total time**: minimum **16-22 weeks**, realistic **5-7 months** (5-8 hr/week part-time) + +> **Two shared hubs (used by both Track A + Track B)**: +> - **Stage 5** = Claude Code Ecosystem (MCP / Skills / Plugins / Subagents) — Track A learns MCP-into-CLI, Track B learns agent runtime structure +> - **Stage 8** = Agent Interfaces (Computer Use / Browser / Sandbox, 2024-2026 frontier) — Track A learns "how to use" for task delegation, Track B learns "how to build" with embedded interfaces + +> 💡 **Want a concrete cross-stage example?** [Build Your First AI Agent in 7 Steps](walkthroughs/build-first-agent-in-7-steps.en.md) — same Paper Summary Bot traced from Stage 1 through Stage 7, ~350 lines of executable code (**Track B**) + +After the main path, pick one of 5 specialized branches. **Not sure which?** + +![Branch decision tree](resources/diagrams/branch-decision-tree.en.png) + +> 💡 **The Everyday User branch can be read directly without walking the main path** — it's for people who want to use AI without writing code. + +| Branch | Best for | Topics | +|---|---|---| +| 🔬 [Researcher](branches/for-researcher.en.md) | Grad students, postdocs, PIs | Lit triage · paper writing · multi-agent review | +| 💻 [Developer](branches/for-developer.en.md) | Software engineers | Cursor · Aider · CLI delegation · code review | +| 🎓 [Teacher](branches/for-teacher.en.md) | Teachers, instructors | Lesson planning · slides · student feedback · privacy / ethics · prompt templates | +| 📊 [Knowledge Worker](branches/for-knowledge-worker.en.md) | Consultants, PMs, analysts | Email · meeting notes · report automation | +| 👥 [Everyday User](branches/for-everyday-users.en.md) | ChatGPT / Claude.ai users | Daily writing · learning · privacy · CLI agent intro | + +--- + +## 💡 How to Learn + +Welcome — future agent system builder. Some guidance before you start. + +This roadmap balances concepts with hands-on work, helping you **transform from an LLM user into an agent system builder**. It assumes **basic Python**. Before starting: + +- **Basic Python** — written functions, used APIs, can read JSON +- **Basic git** — clone, commit, push +- **Motivation to learn** — agents are the fastest-changing area in AI 2025+, and require sustained effort + +If anything's missing, do Stage 0; if not, **start at Stage 1**. + +The main path has 5 parts: + +- **Part 1 (Stages 0-2): Foundations & LLM Basics** — Python / git / API, what's an LLM, prompt design +- **Part 2 (Stages 3-4): Build Your Agent** — from tool use to agents, learn the major frameworks +- **Part 3 (Stage 5) Shared Hub** — Claude Code Ecosystem (MCP / Skills / Plugins / Subagents; used by both Track A + B) +- **Part 4 (Stages 6-7): Advanced Integration** — memory / RAG / multi-agent collaboration / harness engineering +- **Part 5 (Stage 8) Shared Hub** — Agent Interfaces (Computer Use / Browser Use / Code Sandbox, 2024-2026 frontier; used by both tracks) + +> 🔭 **Three layers of concept evolution**: **prompt engineering** (Stage 2 — how to write a single prompt) → **context engineering** (Stage 3 onward — how to dynamically assemble system prompt + memory + retrieved chunks + tool schema) → **harness engineering** (Stage 7 — agent loop / eval / observability / deploy as a complete production system). Three terms, three phases; you don't need to look elsewhere. See [`stages/02-prompt-engineering.en.md`](stages/02-prompt-engineering.en.md) "Beyond prompts: context engineering" and [`stages/07-multi-agent-production.en.md`](stages/07-multi-agent-production.en.md) Required Reading 5+6. + +After the main path (16-22 weeks for Track B, 8-10 weeks for Track A), pick a branch. + +The most important advice: **don't skip the hands-on exercises**. Each stage's exercises are "you can't learn this without doing it" — skim past them and you'll get stuck later. + +> 🎓 **How to actually use the exercises**: the `starter.py` in each exercise folder is a **complete solution**, not a TODO skeleton. If you clone, `cat starter.py`, and run `python test.py` to all-green, you'll think "I learned it" — but you haven't written a single line. **Correct learning loop**: `mv starter.py starter_reference.py`, look at signatures (not bodies), write your own, peek at the reference only after 20 min stuck. Full method + per-stage time budgets + escalation order in [`docs/HOW_TO_USE.md`](docs/HOW_TO_USE.md). + +Ready? [Start at Stage 0](stages/00-foundations.en.md). + +--- + +## 📚 Related Resources + +The full related-resources block (term definitions + daily-tool MCP/Skill highlights + awesome lists + Chinese-community resources) lives in **[RESOURCES.en.md](RESOURCES.en.md)** so this README stays focused. + +Common quick links, grouped by **scenario**: + +### 🚀 Onboarding / Environment + +| Your situation | Where | What's there | +|---|---|---| +| Never written code, first time with AI agents | [`resources/setup-guide.en.md`](resources/setup-guide.en.md) | 30-45 min from zero (API key, Python, first hello-world) | +| Not sure which LLM provider to pick | [`resources/setup-guide.en.md` A](resources/setup-guide.en.md#a--get-your-first-api-key-about-10-minutes) | Anthropic / OpenAI / DeepSeek / Kimi / NVIDIA NIM comparison | +| Topic-based awesome lists / Chinese community | [`RESOURCES.en.md` topic-based](RESOURCES.en.md#topic-based-awesome-lists) | 5-10 min skim | + +### 📖 Concepts / Terminology + +| Your situation | Where | What's there | +|---|---|---| +| Don't know a term (LLM / agent / RAG / token / MCP / Skill / vector DB…) | [`resources/glossary.en.md`](resources/glossary.en.md) | 30+ terms, 30-80 words each + which stage covers it | +| Why some agents live in terminal vs Telegram vs Jetson | [`resources/agent-paradigms.en.md`](resources/agent-paradigms.en.md) | 5 paradigms mental model + Hermes Agent / OpenClaw examples | +| MCP / Skills / Plugins glossary mapping | [`RESOURCES.en.md` three core terms](RESOURCES.en.md#three-core-terms-mcp--skills--plugins) | 1-page lookup | +| Certificate-granting online AI agent courses (EN + ZH) | [`resources/courses.en.md`](resources/courses.en.md) | 10 credible cert-granting courses, tiered; with an honest "completion cert ≠ a degree" caveat | + +### 🛠 Hands-on + +| Your situation | Where | What's there | +|---|---|---| +| Want to build Skill / MCP server / Word / Zotero / local LLM integration | [`resources/cookbook.en.md`](resources/cookbook.en.md) | 6 step-by-step recipes, 30-50 min each | +| Want to use subagents but do not know who to dispatch, how to dispatch, or what work to dispatch | [`resources/subagent-cookbook.en.md`](resources/subagent-cookbook.en.md) | 15 copy-paste dispatch recipes | +| Stuck on tool calling (LLM won't call / schema broken / ReAct won't stop) | [`examples/stage-5/tool-calling-tutor/`](examples/stage-5/tool-calling-tutor/) | Claude Code installable skill, 4-symptom diagnostic | +| How to use the hands-on exercises correctly (active vs passive mode) | [`docs/HOW_TO_USE.md`](docs/HOW_TO_USE.md) | 5-10 min read, applies to every stage | + +### 🔌 Daily tool integrations / Finding MCP servers + +| Your situation | Where | Scope | +|---|---|---| +| Connect to Notion / Obsidian / Excel / GitHub / etc. | [`RESOURCES.en.md` daily-tool integrations](RESOURCES.en.md#daily-tool-integrations-mcp-servers--skills) | 7-8 highlights | +| Full MCP server / Skill catalog (stars, categories) | [`resources/mcp-skills-catalog.en.md`](resources/mcp-skills-catalog.en.md) | 65+ entries, 16 categories | + +### 🔬 Research / Production + +| Your situation | Where | What's there | +|---|---|---| +| Research workflow + multi-LLM delegation skill pair | [`RESOURCES.en.md` research workflow](RESOURCES.en.md#research-workflow-by-the-repo-maintainer) | Maintainer's own Claude Code research skill set | +| CLI agent 7-way comparison + production combos | [`resources/cli-agents-guide.en.md`](resources/cli-agents-guide.en.md) | Track A's core reference, ~148 lines | +| Schema design rules (must-read for tool calling) | [`resources/schema-design-cheatsheet.en.md`](resources/schema-design-cheatsheet.en.md) | 5 golden rules + 5 anti-patterns | + +--- + +## 🤝 Contributing + +This repo is an AI learning document — if you've also curated great resources, contributions are very welcome: + +- 🐛 **Bug reports** — wrong content, broken links, stale info → open Issue +- 💡 **Suggestions** — missing stage / new project to add → open Issue to discuss +- 📝 **Improvements** — refine existing stage content, fix typos → direct PR +- ✍️ **Add a project** — 1-3 new projects per stage with "why this teaches that stage" rationale +- 🌏 **Translations** — improve the English edition or translate to other languages +- 🌱 **Become a Stage / Branch maintainer** — long-term review of a specific area, see [CONTRIBUTORS.md](CONTRIBUTORS.md) + +PR process and style rules: [CONTRIBUTING.md](CONTRIBUTING.md) + [resources/style-guide.en.md](resources/style-guide.en.md). + +> 📅 **Want to see what shipped recently?** → [`CHANGELOG.md`](CHANGELOG.md) (last 14 days). +> Internal phase rollout progress and launch checklist: [`.github/launch-checklist.md`](.github/launch-checklist.md) (maintainer-facing internal doc). + +--- + +## 💬 Advisory / Contact + +A free, open (MIT) learning edition — use it freely. + +Currently focused on advisory work: teams or companies needing **prompt review / audit** or **AI agent workflow consulting** are welcome to reach out (PhD student, limited availability): 📧 [wenyuchiou12@gmail.com](mailto:wenyuchiou12@gmail.com) + +--- + +## 🙏 Acknowledgments + +### Inspiration + +- [**Datawhale Hello-Agents**](https://github.com/datawhalechina/hello-agents) — the most thorough chapter-length agent tutorial in the Chinese-language ecosystem; inspired our chapter + progress structure. Every stage / exercise folder has a 📚 callout pointing to the relevant depth chapter. Special thanks. +- [**Datawhale community**](https://github.com/datawhalechina) — landmark Chinese ML learning community; multiple anchor projects come from them +- [**liyupi/ai-guide**](https://github.com/liyupi/ai-guide) — largest Chinese-language "AI mega-guide" + Vibe Coding tutorial (covers Agent Skills / RAG / MCP / A2A / Harness Engineering). This repo is a "structured roadmap"; ai-guide is a "breadth resource hub" — complementary + +### Related projects + +Other lists in the same space — useful to browse alongside this repo when hunting for specific tools: + +- [`wong2/awesome-mcp-servers`](https://github.com/wong2/awesome-mcp-servers) — categorized MCP server catalog +- [`punkpeye/awesome-mcp-servers`](https://github.com/punkpeye/awesome-mcp-servers) — another MCP server catalog +- [`hesreallyhim/awesome-claude-code`](https://github.com/hesreallyhim/awesome-claude-code) — Claude Code tools & plugins list + +These are pure catalogs (browse and pick). This repo is different in that it has a **learning order from Stage 0 all the way to production**. + +### Contributors + +[![Contributors](https://contrib.rocks/image?repo=WenyuChiou/awesome-agentic-ai-zh)](https://github.com/WenyuChiou/awesome-agentic-ai-zh/graphs/contributors) + +New contributors appear above automatically. Full list → [GitHub Contributors](https://github.com/WenyuChiou/awesome-agentic-ai-zh/graphs/contributors). + +### Personal + +- [@WenyuChiou](https://github.com/WenyuChiou) — Maintainer + +--- + +## 🎓 Citation + +If this learning roadmap helps your study or work, please cite: + +```bibtex +@misc{awesome_agentic_ai_zh_2026, + title = {awesome-agentic-ai-zh: A Structured Learning Roadmap for Agentic AI}, + author = {Chiou, Wenyu}, + year = {2026}, + url = {https://github.com/WenyuChiou/awesome-agentic-ai-zh}, + note = {8-stage learning path from prerequisites to Agent Interfaces (Computer Use / Browser Use / Code Sandbox), with curated projects + hello-X demos. Bilingual (zh-TW / English).} +} +``` + +--- + +## ☕ Support this project + +This learning map is free and open-source (MIT). If it helps you, a ⭐ Star means a lot — and if you'd like to support ongoing updates, you can buy the author a coffee: + +Buy Me A Coffee + +Or use the **❤ Sponsor** button at the top of the repo. (GitHub Sponsors is under review and will be added once approved.) + +--- + +## License + +MIT. Maintained by [@WenyuChiou](https://github.com/WenyuChiou). + +
+

⭐ If this repo helps you, please give it a Star — it matters for ongoing iteration

+
diff --git a/README.md b/README.md new file mode 100644 index 0000000..87e4c1b --- /dev/null +++ b/README.md @@ -0,0 +1,330 @@ +
+ 繁體中文 | 简体中文 | English +
+ +
+ +![AI Agent 學習路徑](resources/diagrams/banner.png) + +# awesome-agentic-ai-zh + +### 🤖 AI Agent 學習地圖 — 從基本 LLM 概念到自己打造多 agent 系統 + +

學習路線圖 + 240+ 資源 curation + 簡單 illustrative 案例
結構化 8 階段、從「LLM 是什麼、token 怎麼算」走到 multi-agent 編排、Computer Use / Browser Use / Sandbox

+ +[![License](https://img.shields.io/badge/license-MIT-blue?style=flat)](LICENSE) +[![繁中](https://img.shields.io/badge/語言-繁體中文-red?style=flat)](README.md) +[![简中](https://img.shields.io/badge/語言-简体中文-orange?style=flat)](README.zh-Hans.md) +[![EN](https://img.shields.io/badge/lang-English-blue?style=flat)](README.en.md) +![GitHub stars](https://img.shields.io/github/stars/WenyuChiou/awesome-agentic-ai-zh?style=flat&logo=github) +![GitHub forks](https://img.shields.io/github/forks/WenyuChiou/awesome-agentic-ai-zh?style=flat&logo=github) +[![線上文件站](https://img.shields.io/badge/線上文件站-Pages-2ea44f?style=flat)](https://wenyuchiou.github.io/awesome-agentic-ai-zh/) + +
+ +--- + +## 🎯 專案介紹 + +**本 repo 角色定位**：**學習路線圖 + 240+ 資源 curation + 簡單 illustrative 案例**——三件事為核心、幫想學 AI / AI agent 的人從「不知道從哪開始」走到「能設計多 agent 系統」。 + +具體做法： + +| 核心 | 做什麼 | 規模 | +|---|---|---| +| **學習路線圖** | 把網路散落的高品質專案、教材、必修閱讀，按**從零開始、循序漸進**整理成 **8 個階段**（含 Stage 5 + Stage 8 兩個共用 hub）+ 2 條學習路線 + 5 條延伸路徑 | 8 stages、2 tracks | +| **資源 curation** | 每階段精選 **240+** 個 project（含星等、適合誰、教什麼、怎麼跑），加上中文 AI 生態(DeepSeek / Zhipu / Kimi 等)MCP / Skill 完整 catalog | 240+ projects、65 MCP/Skill | +| **簡單 illustrative 案例** | 每階段附 1-5 個**基礎練習**（70-150 行 starter + dual-path Ollama/Anthropic SDK 對照 + mock-based test） | 23 個練習 folder | + +走完整條路線，你會從「**LLM 使用者**」進階到「**agent 系統建構者**」——能看懂 framework 在做什麼、能設計多 agent 協作、能寫自己的 MCP server。 + +> 📖 **關於中英文混用**：本專案保留 AI Agent 領域常見英文術語（Prompt Engineering / Context Engineering / Harness / MCP / Skills / RAG 等），因為官方文件、paper、GitHub repo 與 API 文件多以英文為主。每個重要概念會提供 **中文理解名 + 英文正式術語 + 一句白話定位**，讓讀者能先理解概念，再對接英文生態。完整對照見 [`resources/glossary.md`](resources/glossary.md)。 + +--- + +## 📋 目錄 + +- [🎯 專案介紹](#-專案介紹) +- [📚 快速開始](#-快速開始) + - [線上閱讀](#線上閱讀) + - [本地下載](#本地下載) + - [✨ 你會收穫什麼？](#-你會收穫什麼) +- [🗺️ 學習地圖（兩條學習路徑）](#️-學習地圖兩條學習路徑) +- [💡 如何學習](#-如何學習) +- [📚 相關資源](#-相關資源) +- [🤝 如何貢獻](#-如何貢獻) +- [🙏 致謝](#-致謝) +- [🎓 引用](#-引用) +- [☕ 支持這個專案](#-支持這個專案) +- [License](#license) + +--- + +## 📚 快速開始 + +### 🚀 第一次接觸 AI agent / 沒寫過 code？ + +先看 **[`resources/setup-guide.md`](resources/setup-guide.md)** — 30-45 分鐘從零帶你申請 API key、裝好 Python、跑出第一個 LLM hello-world。 + +### 線上閱讀 +- **[學習地圖（兩條學習路徑）](#️-學習地圖兩條學習路徑)** — 看完這節決定走 Track A 還 Track B +- **[Stage 0 基礎準備](stages/00-foundations.md)** — 已經會 Python / git / API 的人可以直接跳 Stage 1 + +### 本地下載 +```bash +git clone https://github.com/WenyuChiou/awesome-agentic-ai-zh.git +cd awesome-agentic-ai-zh +# 從 stages/00-foundations.md 開始 +``` + +### ✨ 你會收穫什麼？ + +- 📖 **完全免費** — MIT 授權，所有內容開放共學 +- 🗺️ **兩條學習路徑** — Track A（CLI Power User）給「想 USE 現成 CLI agent」的人；Track B（Agent Builder）給「想 BUILD 自己 agent」的人。共用 Stage 0-2 基礎 +- 🛠️ **基礎動手練習** — 每階段附 1-5 個 illustrative 練習（題目 + dual-path SDK 對照 + success criteria）。定位是**基礎入門 + 路線確認**——chapter-length 深度練習見對應 stage 的 hello-agents / Anthropic Cookbook callout +- 🎯 **精選 240+ 個 projects** — 每個都附星等推薦、適合誰、教什麼、怎麼跑（含本地 LLM 執行：Ollama、llama.cpp、LocalAI、MLX） +- 🌏 **三語完整維護** — 繁中(canonical)/ 簡中 / English,三版皆完整維護、英文非薄翻譯 +- 🎓 **不只「框架」、還有「Claude Code 生態」** — MCP / Skills / Plugins / SDK 完整堆疊 +- 🔬 **5 條依使用者分流的延伸路線** — 研究員 / 開發者 / 老師 / 知識工作者 / **日常使用者** +- ⏱️ **預估時程寫清楚** — Track A 8-10 週 / Track B 主幹最少 16-22 週、現實 5-7 個月（每週 5-8 hr） + +--- + +## 🗺️ 學習地圖（兩條學習路徑） + +![AI Agent 學習地圖](resources/diagrams/learning-map.png) + +走完 **Stage 0-2（共用基礎）** 之後，依你的目的選一條學習路徑： + +- **Track A — CLI Power User**：你想**用**現成的 CLI agent（Claude Code、Codex、OpenCode、Gemini CLI 等）把工作做順、效率拉高，不打算自己從零寫 agent。3 個 sub-stage（A1-A3）。 +- **Track B — Agent Builder**：你想**從零打造**自己的 agent——學 framework、寫 ReAct、設計 multi-agent。Stage 3-8 是主路線。 + +兩條學習路徑**不互斥**——多數人是先走 A 把 CLI 用起來，再回到 B 學內部運作；或反過來也行。Stage 5（Claude Code 生態）兩條路徑都會用到。 + +### 共用基礎（Stage 0-2） + +| Stage | 主題 | 關鍵內容 | 預估時程 | +|---|---|---|---| +| **0** | [基礎準備（Foundations）](stages/00-foundations.md) | Python · CLI · git · API · JSON | 1-2 週 | +| **1** | [LLM 基礎（LLM Basics）](stages/01-llm-basics.md) | token · API · 各家 LLM 比較 · 本地 LLM | 1 週 | +| **2** | [Prompt 設計（Prompt Engineering）](stages/02-prompt-engineering.md) | 系統 prompt · few-shot · CoT | 1-2 週 | + +### Track A — CLI Power User（想用 CLI 把事情做完） + +| Stage | 主題 | 關鍵內容 | 預估時程 | +|---|---|---|---| +| **A1** | [選一個 CLI Agent，開始用它做事（CLI Agent Intro & Selection）](tracks/cli/A1-cli-intro.md) | 7 主流 CLI 比較 · 安裝 · 第一次跑 | 1 週 | +| **A2** | [建立可重複使用的 CLI 工作流程（CLI Workflow Patterns）](tracks/cli/A2-cli-workflow.md) | CLAUDE.md · slash command · 多步驟拆解 | 1-2 週 | +| **A3** | [把 CLI Agent 接進真實工作流程（Integration & Production）](tracks/cli/A3-cli-production.md) | MCP 接 CLI · CI 自動化 · cost / observability | 1-2 週 | +| **+5** | [Stage 5 — Claude Code 生態](stages/05-claude-code-ecosystem.md)（**共用 hub**） | MCP · Skills · Plugins · Subagents、Track A 必看 5.1-5.4 / 選讀 5.5-5.7 | 1-2 週（Track A 視角）| +| **+8** | [Stage 8 — Agent Interfaces](stages/08-agent-interfaces.md)（**共用 hub**）| Computer Use · Browser Use · Code Sandbox、Track A 視角看 Track A 怎麼用 | 1-2 週（Track A 視角）| + +> **Track A 預估總時程**：含 Stage 0-2（共用基礎）+ A1-A3 + **Stage 5 + Stage 8（兩個共用 hub）= 約 8-10 週**。核心參考：[`resources/cli-agents-guide.md`](resources/cli-agents-guide.md)。 + +### Track B — Agent Builder（從零打造 agent） + +| Stage | 主題 | 關鍵內容 | 預估時程 | +|---|---|---|---| +| **3** ⭐ | [工具使用與第一個 Agent（Tool Use & Hello Agent）](stages/03-tool-use-and-hello-agent.md) | function calling · ReAct · 5 個動手練習 | 2-3 週 | +| **4** | [Agent 框架（Agent Frameworks）](stages/04-agent-frameworks.md) | LangGraph · AutoGen · CrewAI · Smolagents | 2-3 週 | +| **5** ⭐⭐ | [Claude Code 生態系（Claude Code Ecosystem）](stages/05-claude-code-ecosystem.md)（**共用 hub**、Track A 也學）| MCP · Skills · Plugins · Subagents | 3-4 週（Track B 視角）| +| **6** | [上下文管理（Context Engineering）：RAG 與 Memory](stages/06-memory-rag.md) | vector DB · long-term memory · contextual retrieval | 2 週 | +| **7** | [多 Agent 系統與穩定運作（Multi-Agent & Production）](stages/07-multi-agent-production.md) | multi-agent orchestration · eval · observability · SDK 進階 | 2-4 週 | +| **7.5** | [進階 Agentic Workflow 概念（Advanced Agentic Concepts）](stages/07.5-advanced-agentic-concepts.md)（reading map）| 工作邊界 · PAR loop · agent-as-judge · 12 個進階概念 + reading list | 1 週（不寫 code）| +| **8** ⭐⭐ | [Agent 操作介面（Agent Interfaces）](stages/08-agent-interfaces.md)（**共用 hub**、Track A 也學）| Computer Use · Browser Use · Code Sandbox、2024-2026 frontier | 2-3 週（Track B 視角）| + +> **Track B 預估總時程**：主幹最少 **16-22 週**、現實 **5-7 個月**（每週 5-8 hr 兼職） + +> **兩個共用 hub（Track A + Track B 都會用到）**： +> - **Stage 5** = Claude Code 生態（MCP / Skills / Plugins / Subagents）—— Track A 學 MCP 接 CLI、Track B 學 agent runtime 結構 +> - **Stage 8** = Agent Interfaces（Computer Use / Browser / Sandbox、2024-2026 frontier）—— Track A 學「**怎麼用**」委派任務、Track B 學「**怎麼 build**」embed 進 agent +> +> 兩個 hub 出現在兩條 track 內、視角不同、學的深度也不同（內文有 Track A / Track B 分視角段）。 + +> 💡 **想看跨 stage 的完整範例？** [7 步打造你的第一個 AI Agent](walkthroughs/build-first-agent-in-7-steps.md) — 同一個 Paper Summary Bot 從 Stage 1 一路寫到 Stage 7，~350 行真實程式碼（**Track B 適用**） + +走完主幹後，依你的身分挑一條延伸路線繼續走。**不確定挑哪條？** + +![Branch 決策樹](resources/diagrams/branch-decision-tree.png) + +> 💡 **「日常使用者」這條路線不必走完主幹就能直接讀**——是給「想用 AI、但不一定要寫 code」的人。 + +| 路線 | 適合誰 | 主題 | +|---|---|---| +| 🔬 [研究人員](branches/for-researcher.md) | 研究生、博後、PI | 文獻整理 · paper 寫作 · multi-agent review | +| 💻 [開發者](branches/for-developer.md) | 軟體工程師 | Cursor · Aider · CLI delegation · code review | +| 🎓 [教師](branches/for-teacher.md) | 老師、講師 | 備課 · 投影片 · 學生 feedback · 隱私 / 倫理 · prompt 範本 | +| 📊 [知識工作者](branches/for-knowledge-worker.md) | 顧問、PM、分析師 | Email · 會議紀錄 · report 自動化 | +| 👥 [日常使用者](branches/for-everyday-users.md) | ChatGPT / Claude.ai 使用者 | 寫信 · 學習 · 隱私場景 · CLI agent 入門 | + +--- + +## 💡 如何學習 + +這份路線圖兼顧概念與實作，目標是帶你**從 LLM 使用者一路走到 agent 系統建構者**。適合**有基本 Python 能力**的開發者、研究生、自學者。動手之前，先確認你有： + +- **基本 Python** — 寫過 function、用過 API、看得懂 JSON +- **基本 git** — clone、commit、push +- **想學的動機** — agent 是 2024-2026 變化最快的領域，需要持續投入（2026 仍每月推新 model / 新 framework） + +上面有缺的就從 Stage 0 補齊；都會了就**直接跳 Stage 1**。 + +主幹分 5 部分： + +- **Part 1（Stage 0-2）：基礎與 LLM 入門** — Python / git / API、什麼是 LLM、怎麼設計 prompt +- **Part 2（Stage 3-4）：建構你的 Agent** — 從 tool use 進化到 agent，學主流 framework +- **Part 3（Stage 5） 共用 hub** — Claude Code 生態系（MCP / Skills / Plugins / Subagents、Track A + B 都會用到） +- **Part 4（Stage 6-7）：進階整合** — memory / RAG / multi-agent 協作 / harness engineering +- **Part 5（Stage 8） 共用 hub** — Agent Interfaces（Computer Use / Browser Use / Code Sandbox、2024-2026 frontier、Track A + B 都會用到） + +> 🔭 **三層概念進化**：**prompt engineering**（Stage 2、單一 prompt 怎麼寫）→ **context engineering**（Stage 3 之後、怎麼動態組 system prompt + memory + retrieved chunks + tool schema）→ **harness engineering**（Stage 7、agent loop / eval / observability / deploy 整套包成 production system）。3 個術語對應 3 個 phase、不必另外找資源。詳見 [`stages/02-prompt-engineering.md#-進階context-engineering不是-prompt-engineering-了`](stages/02-prompt-engineering.md) 跟 [`stages/07-multi-agent-production.md`](stages/07-multi-agent-production.md) 必修閱讀 5+6。 + +走完主幹（14-19 週）後，依你的身分（研究員 / 開發者 / 老師 / 知識工作者 / 日常使用者）挑一條延伸路線繼續走。 + +最重要的一句話：**不要跳過 動手練習**。每個 stage 的 動手練習都是「不動手就學不會」的東西，光讀過去後面會卡住。 + +> 🎓 **動手練習怎麼用才對**：每個練習 folder 裡的 `starter.py` 是**完整解答**、不是 TODO skeleton。如果你 clone 下來直接 `cat starter.py` + `python test.py` pass、會誤以為「我學會了」、其實沒寫一行 code。**正確學習法**：`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫、卡住才回去對照。完整方法論 + 每個 stage 的時間預算 + 卡住處理流程看 [`docs/HOW_TO_USE.md`](docs/HOW_TO_USE.md)。 + +準備好了嗎？[從 Stage 0 開始](stages/00-foundations.md)。 + +--- + +## 📚 相關資源 + +完整的相關資源（用語說明 + 常用 MCP / Skill highlight + awesome lists + 中文社群）抽到 **[RESOURCES.md](RESOURCES.md)** 避免主頁過長。 + +直接看常用入口、依**情境**分組： + +### 🚀 入門 / 環境設定 + +| 你的狀況 | 去哪 | 內容 | +|---|---|---| +| 完全沒寫過 code、第一次接觸 AI agent | [`resources/setup-guide.md`](resources/setup-guide.md) | 30-45 分鐘從零裝好（API key、Python、第一個 hello-world） | +| 不知道挑哪個 LLM provider | [`resources/setup-guide.md` A](resources/setup-guide.md#a--申請第一個-api-key約-10-分鐘) | Anthropic / OpenAI / DeepSeek / Kimi / NVIDIA NIM 對照 | +| 同主題 awesome list / 中文社群 | [`RESOURCES.md` 同主題清單](RESOURCES.md#同主題的清單型-awesome-lists) | 5-10 分鐘逛一輪 | + +### 📖 概念 / 用語 + +| 你的狀況 | 去哪 | 內容 | +|---|---|---| +| 不懂某個詞（LLM / agent / RAG / token / MCP / Skill / 向量資料庫…） | [`resources/glossary.md`](resources/glossary.md) | 30+ 詞、每個 30-80 字 + 哪 stage 講細的 | +| 想搞懂 agent 為什麼有的在 terminal、有的在 Telegram、有的在 Jetson | [`resources/agent-paradigms.md`](resources/agent-paradigms.md) | 5 種 agent 型態 mental model + Hermes / OpenClaw 例子 | +| MCP / Skills / Plugins 用語對照 | [`RESOURCES.md` 三個核心用語](RESOURCES.md#三個核心用語mcp--skills--plugins) | 1 頁速查表 | +| 想找帶證書的線上 AI agent 課（英文 + 中文） | [`resources/courses.md`](resources/courses.md) | 10 門 credible、會發證書的課，分 tier；含「完成證書 ≠ 學歷」誠實 caveat | + +### 🛠 動手實作 + +| 你的狀況 | 去哪 | 內容 | +|---|---|---| +| 想動手寫 Skill / MCP server / 接 Word / Zotero / 本機 LLM | [`resources/cookbook.md`](resources/cookbook.md) | 6 個 step-by-step recipe、每個 30-50 分鐘 | +| 想用 subagent 但不知該派誰、怎麼派、派什麼工作 | [`resources/subagent-cookbook.md`](resources/subagent-cookbook.md) | 15 個複製貼上即用的 dispatch recipe | +| 自己寫 subagent / 組合多個 / debug 跑壞的（進階）| [`resources/subagent-advanced.md`](resources/subagent-advanced.md) | description 寫法 4 個 bug + composition 3 pattern + debug 5 切點 | +| 卡在 tool calling（LLM 不呼叫 / schema 寫不好 / ReAct loop 跑不停） | [`examples/stage-5/tool-calling-tutor/`](examples/stage-5/tool-calling-tutor/) | 可裝進 Claude Code 的 skill、4-symptom diagnostic | +| 動手練習怎麼正確使用（主動 vs 被動模式） | [`docs/HOW_TO_USE.md`](docs/HOW_TO_USE.md) | 5-10 分鐘讀完、配合每個 stage 用 | + +### 🔌 接日常工具 / 找 MCP server + +| 你的狀況 | 去哪 | 規模 | +|---|---|---| +| 接 Notion / Obsidian / Excel / GitHub 等工具 | [`RESOURCES.md` 接日常工具](RESOURCES.md#接日常工具常用-mcp-server--skill) | 7-8 個 highlight | +| 完整 MCP server / Skill 目錄（含星等、分類） | [`resources/mcp-skills-catalog.md`](resources/mcp-skills-catalog.md) | 65+ 條、16 大分類 | + +### 🔬 研究 / production 級 + +| 你的狀況 | 去哪 | 內容 | +|---|---|---| +| 研究 workflow + multi-LLM delegation skill | [`RESOURCES.md` 研究工作流](RESOURCES.md#研究工作流本-repo-維護者出品) | 本 repo 維護者出品的 Claude Code 研究 skill 對 | +| CLI agent 7 家對照 + production 搭配 | [`resources/cli-agents-guide.md`](resources/cli-agents-guide.md) | Track A 的核心參考、148 行 | +| Schema 設計規則（tool calling 必看） | [`resources/schema-design-cheatsheet.md`](resources/schema-design-cheatsheet.md) | 5 條黃金規則 + 5 個 anti-pattern | + +--- + +## 🤝 如何貢獻 + +這個 repo 是一個 AI 學習文件，如果你也有蒐集很好的資源，也歡迎貢獻： + +- 🐛 **回報 Bug** — 內容錯誤、連結失效、過時資訊 → 開 Issue +- 💡 **提建議** — 缺什麼 stage、該加哪個 project → 開 Issue 討論 +- 📝 **完善內容** — 改進現有 stage 內容、修 typo → 直接 PR +- ✍️ **新增 project** — 在某個 stage 加 1-3 個 project，並附上「為什麼這個 project 適合放這個 stage」的說明 +- 🌏 **翻譯** — 補英文 companion 沒翻到的段落，或翻成其他語言 +- 🌱 **擔任 Stage / Branch maintainer** — 長期 review 特定領域，詳見 [CONTRIBUTORS.md](CONTRIBUTORS.md) + +PR 流程跟 style 規範請看 [CONTRIBUTING.md](CONTRIBUTING.md) 跟 [resources/style-guide.md](resources/style-guide.md)。 + +> 📅 **想看最近 ship 了什麼** → [`CHANGELOG.md`](CHANGELOG.md)（最近 14 天）。 +> Maintainer 內部進度與 launch checklist 放在 [`.github/launch-checklist.md`](.github/launch-checklist.md)（內部文件）。 + +--- + +## 💬 顧問 / 聯絡 + +公開學習版（MIT），歡迎自由取用。 + +目前以顧問為主：團隊或公司若需 **prompt review / audit** 或 **AI agent workflow 諮詢**，歡迎來信（博士生、時間有限）：📧 [wenyuchiou12@gmail.com](mailto:wenyuchiou12@gmail.com) + +--- + +## 🙏 致謝 + +### Inspiration + +- [**Datawhale Hello-Agents**](https://github.com/datawhalechina/hello-agents) — 中文圈最完整的 chapter-length agent 教材，本 repo 的「章節 + 進度」結構受這份啟發；每個 stage / 練習 folder 都有 📚 callout 點過去深度章節。特別感謝。 +- [**Datawhale 社群**](https://github.com/datawhalechina) — 中文 ML 共學社群的標竿，本 repo 多個 anchor project 來自這裡 +- [**liyupi/ai-guide**](https://github.com/liyupi/ai-guide) — 中文圈最大「AI 資源大全」，跟 Vibe Coding 教學齊全（涵蓋 Agent Skills / RAG / MCP / A2A / Harness Engineering）。本 repo 是「結構化路線」、ai-guide 是「廣度資源庫」，互為補充 + +### 其他相關專案 + +同主題、不同切入角度的清單，搜資源時可以一起用： + +- [`wong2/awesome-mcp-servers`](https://github.com/wong2/awesome-mcp-servers) — MCP server 清單，按分類整理 +- [`punkpeye/awesome-mcp-servers`](https://github.com/punkpeye/awesome-mcp-servers) — 另一份 MCP server 清單 +- [`hesreallyhim/awesome-claude-code`](https://github.com/hesreallyhim/awesome-claude-code) — Claude Code 相關工具與 plugin 清單 + +這些是純清單形式（看到再挑），本 repo 的不同點是有「**從 Stage 0 一路走到 production 的學習順序**」。 + +### 貢獻者 + +[![Contributors](https://contrib.rocks/image?repo=WenyuChiou/awesome-agentic-ai-zh)](https://github.com/WenyuChiou/awesome-agentic-ai-zh/graphs/contributors) + +新貢獻者會自動出現在上方。完整列表 → [GitHub Contributors](https://github.com/WenyuChiou/awesome-agentic-ai-zh/graphs/contributors)。 + +### 個人 + +- [@WenyuChiou](https://github.com/WenyuChiou) — Maintainer + +--- + +## 🎓 引用 + +如果這個學習地圖對你的學習或工作有幫助，歡迎引用： + +```bibtex +@misc{awesome_agentic_ai_zh_2026, + title = {awesome-agentic-ai-zh: A Structured Learning Roadmap for Agentic AI}, + author = {Chiou, Wenyu}, + year = {2026}, + url = {https://github.com/WenyuChiou/awesome-agentic-ai-zh}, + note = {8-stage learning path from prerequisites to Agent Interfaces (Computer Use / Browser Use / Sandbox), with curated projects + hello-X demos. Bilingual (zh-TW / English).} +} +``` + +--- + +## ☕ 支持這個專案 + +這份學習地圖是免費、開源（MIT）。如果它對你有幫助，除了給個 ⭐ Star，也歡迎請作者喝杯咖啡、支持它持續更新： + +Buy Me A Coffee + +或直接點 repo 右上角的 **❤ Sponsor** 按鈕。（GitHub Sponsors 審核中，通過後會一併加上。） + +--- + +## License + +MIT。Maintained by [@WenyuChiou](https://github.com/WenyuChiou)。 + +
+

⭐ 如果這個 repo 對你有幫助，歡迎給個 Star — 這對作者繼續更新是很大的鼓勵

+
diff --git a/README.wehub.md b/README.wehub.md new file mode 100644 index 0000000..4191945 --- /dev/null +++ b/README.wehub.md @@ -0,0 +1,7 @@ +# WeHub 来源说明 + +- 原始项目：`WenyuChiou/awesome-agentic-ai-zh` +- 原始仓库：https://github.com/WenyuChiou/awesome-agentic-ai-zh +- 导入方式：上游默认分支的最新快照 +- 原作者、版权和许可证信息以原始仓库及本仓库 LICENSE 为准 +- 本文件仅用于记录来源，不代表 WeHub 是原项目作者 diff --git a/README.zh-Hans.md b/README.zh-Hans.md new file mode 100644 index 0000000..b543d3a --- /dev/null +++ b/README.zh-Hans.md @@ -0,0 +1,358 @@ +
+ 繁體中文 | 简体中文 | English +
+ +
+ +![AI Agent 学习路径](resources/diagrams/banner.zh-Hans.png) + +# awesome-agentic-ai-zh + +### 🤖 AI Agent 学习地图 — 从基础 LLM 概念到自己构建多 agent 系统 + +

学习路线图 + 240+ 资源 curation + 简单 illustrative 案例
结构化 8 阶段、从“LLM 是什么、token 怎么算”走到 multi-agent 编排、Computer Use / Browser Use / Sandbox

+ +[![License](https://img.shields.io/badge/license-MIT-blue?style=flat)](LICENSE) +[![繁中](https://img.shields.io/badge/語言-繁體中文-red?style=flat)](README.md) +[![简中](https://img.shields.io/badge/語言-简体中文-orange?style=flat)](README.zh-Hans.md) +[![EN](https://img.shields.io/badge/lang-English-blue?style=flat)](README.en.md) +![GitHub stars](https://img.shields.io/github/stars/WenyuChiou/awesome-agentic-ai-zh?style=flat&logo=github) +![GitHub forks](https://img.shields.io/github/forks/WenyuChiou/awesome-agentic-ai-zh?style=flat&logo=github) +[![在线文档站](https://img.shields.io/badge/在线文档站-Pages-2ea44f?style=flat)](https://wenyuchiou.github.io/awesome-agentic-ai-zh/) + +
+ +--- + +## 🎯 项目介绍 + +**本 repo 角色定位**：**学习路线图 + 240+ 资源 curation + 简单 illustrative 案例**——三件事为核心、帮想学 AI / AI agent 的人从“不知道从哪开始”走到“能设计多 agent 系统”。 + +具体做法： + +| 核心 | 做什么 | 规模 | +|---|---|---| +| **学习路线图** | 把网上散落的高质量项目、教材、必修阅读，按**从零开始、循序渐进**整理成 **8 个阶段**（含 Stage 5 + Stage 8 两个共用 hub）+ 2 条学习路线 + 5 条延伸路径 | 8 stages、2 tracks | +| **资源 curation** | 每阶段精选 **240+** 个 project（含星等、适合谁、教什么、怎么跑），加上中文 AI 生态(DeepSeek / Zhipu / Kimi 等)MCP / Skill 完整 catalog | 240+ projects、65 MCP/Skill | +| **简单 illustrative 案例** | 每阶段附 1-5 个**基础练习**（70-150 行 starter + dual-path Ollama/Anthropic SDK 对照 + mock-based test） | 23 个练习 folder | + +走完这条路线，你会从“**LLM 用户**”进阶到“**agent 系统构建者**”——能看懂 framework 在做什么、能设计多 agent 协作、能写自己的 MCP server。 + +--- + +## 📚 快速开始 + +### 🚀 第一次接触 AI agent / 没写过 code？ + +先看 **[`resources/setup-guide.zh-Hans.md`](resources/setup-guide.zh-Hans.md)** — 30-45 分钟从零带你申请 API key、装好 Python、跑出第一个 LLM hello-world。 + +### 在线阅读 +- **[学习地图（两条学习路径）](#️-学习地图两条学习路径)** — 看完这节决定走 Track A 还 Track B +- **[Stage 0 基础准备](stages/00-foundations.zh-Hans.md)** — 已经会 Python / git / API 的人可以直接跳 Stage 1 + +### 本地下载 +```bash +git clone https://github.com/WenyuChiou/awesome-agentic-ai-zh.git +cd awesome-agentic-ai-zh +# 从 stages/00-foundations.zh-Hans.md 开始 +``` + +### ✨ 你会收获什么？ + +- 📖 **完全免费** — MIT 授权，所有内容开放共学 +- 🗺️ **两条学习路径** — Track A（CLI Power User）给“想 USE 现成 CLI agent”的人；Track B（Agent Builder）给“想 BUILD 自己 agent”的人。共用 Stage 0-2 基础 +- 🛠️ **基础动手练习** — 每阶段附 1-5 个 illustrative 练习（题目 + dual-path SDK 对照 + success criteria）。定位是**基础入门 + 路线确认**——chapter-length 深度练习见对应 stage 的 hello-agents / Anthropic Cookbook callout +- 🎯 **精选 240+ 个 projects** — 每个都附星等推荐、适合谁、教什么、怎么跑（含本地 LLM 执行：Ollama、llama.cpp、LocalAI、MLX） +- 🌏 **三语完整维护** — 繁中(canonical)/ 简中 / English,三版皆完整维护、英文非薄翻译 +- 🎓 **不只“框架”、还有“Claude Code 生态”** — MCP / Skills / Plugins 完整堆叠 +- 🔬 **5 条依用户分流的延伸路线** — 研究员 / 开发者 / 老师 / 知识工作者 / 日常用户 +- ⏱️ **预估时程写清楚** — Track A 8-10 周 / Track B 主干最少 16-22 周、现实 5-7 个月（每周 5-8 hr） + +--- + +## 🗺️ 学习地图（两条学习路径） + +![AI Agent 学习地图](resources/diagrams/learning-map.zh-Hans.png) + +走完 **Stage 0-2（共用基础）** 之后，依你的目的选一条学习路径： + +- **Track A — CLI Power User**：你想**用**现成的 CLI agent（Claude Code、Codex、OpenCode、Gemini CLI 等）把工作做顺、效率拉高，不打算自己从零写 agent。3 个 sub-stage（A1-A3）。 +- **Track B — Agent Builder**: 你想**从零构建**自己的 agent——学 framework、写 ReAct、设计 multi-agent。Stage 3-8 是主路线。 + +两条学习路径**不互斥**——多数人是先走 A 把 CLI 用起来，再回到 B 学内部运作；或反过来也行。Stage 5（Claude Code 生态）两条路径都会用到。 + +### 共用基础（Stage 0-2） + +| Stage | 主题 | 关键内容 | 预估时程 | +|---|---|---|---| +| **0** | [基础准备（Foundations）](stages/00-foundations.zh-Hans.md) | Python · CLI · git · API · JSON | 1-2 周 | +| **1** | [LLM 基础（LLM Basics）](stages/01-llm-basics.zh-Hans.md) | token · API · 各家 LLM 比较 · 本地 LLM | 1 周 | +| **2** | [Prompt 设计（Prompt Engineering）](stages/02-prompt-engineering.zh-Hans.md) | 系统 prompt · few-shot · CoT | 1-2 周 | + +### Track A — CLI Power User（想用 CLI 把事情做完） + +| Stage | 主题 | 关键内容 | 预估时程 | +|---|---|---|---| +| **A1** | [选一个 CLI Agent，开始用它做事（CLI Agent Intro & Selection）](tracks/cli/A1-cli-intro.zh-Hans.md) | 7 个主流 CLI 比较 · 安装 · 第一次跑 | 1 周 | +| **A2** | [建立可重复使用的 CLI 工作流程（CLI Workflow Patterns）](tracks/cli/A2-cli-workflow.zh-Hans.md) | CLAUDE.md · slash command · 多步骤拆解 | 1-2 周 | +| **A3** | [把 CLI Agent 接进真实工作流程（Integration & Production）](tracks/cli/A3-cli-production.zh-Hans.md) | MCP 接 CLI · CI 自动化 · cost / observability | 1-2 周 | +| **+5** | [Stage 5 — Claude Code 生态系（Claude Code Ecosystem）](stages/05-claude-code-ecosystem.zh-Hans.md)（**共用 hub**）| MCP · Skills · Plugins · Subagents、Track A 必看 5.1-5.4 / 选读 5.5-5.7 | 1-2 周（Track A 视角）| +| **+8** | [Stage 8 — Agent 操作介面（Agent Interfaces）](stages/08-agent-interfaces.zh-Hans.md)（**共用 hub**）| Computer Use · Browser Use · Code Sandbox、Track A 视角看 Track A 怎么用 | 1-2 周（Track A 视角）| + +> **Track A 预估总时程**：含 Stage 0-2（共用基础）+ A1-A3 + **Stage 5 + Stage 8（两个共用 hub）= 约 8-10 周**。核心参考：[`resources/cli-agents-guide.zh-Hans.md`](resources/cli-agents-guide.zh-Hans.md)。 + +### Track B — Agent Builder（想从零构建 agent） + +| Stage | 主题 | 关键内容 | 预估时程 | +|---|---|---|---| +| **3** ⭐ | [工具使用与第一个 Agent（Tool Use & Hello Agent）](stages/03-tool-use-and-hello-agent.zh-Hans.md) | function calling · ReAct · 5 个动手练习 | 2-3 周 | +| **4** | [Agent 框架（Agent Frameworks）](stages/04-agent-frameworks.zh-Hans.md) | LangGraph · AutoGen · CrewAI · Smolagents | 2-3 周 | +| **5** ⭐⭐ | [Claude Code 生态系（Claude Code Ecosystem）](stages/05-claude-code-ecosystem.zh-Hans.md)（**共用 hub**、Track A 也学）| MCP · Skills · Plugins · Subagents | 3-4 周（Track B 视角）| +| **6** | [上下文管理（Context Engineering）：RAG 与 Memory](stages/06-memory-rag.zh-Hans.md) | vector DB · long-term memory · contextual retrieval | 2 周 | +| **7** | [多 Agent 系统与稳定运作（Multi-Agent & Production）](stages/07-multi-agent-production.zh-Hans.md) | multi-agent orchestration · eval · observability · SDK 进阶 | 2-4 周 | +| **7.5** | [进阶 Agentic Workflow 概念（Advanced Agentic Concepts）](stages/07.5-advanced-agentic-concepts.zh-Hans.md)（reading map）| 工作边界 · PAR loop · agent-as-judge · 12 个进阶概念 + reading list | 1 周（不写 code）| +| **8** ⭐⭐ | [Agent 操作介面（Agent Interfaces）](stages/08-agent-interfaces.zh-Hans.md)（**共用 hub**、Track A 也学）| Computer Use · Browser Use · Code Sandbox、2024-2026 frontier | 2-3 周（Track B 视角）| + +> **Track B 预估总时程**：主干最少 **16-22 周**、现实 **5-7 个月**（每周 5-8 hr 兼职） + +> **两个共用 hub（Track A + Track B 都会用到）**： +> - **Stage 5** = Claude Code 生态（MCP / Skills / Plugins / Subagents）—— Track A 学 MCP 接 CLI、Track B 学 agent runtime 结构 +> - **Stage 8** = Agent Interfaces（Computer Use / Browser / Sandbox、2024-2026 frontier）—— Track A 学“**怎么用**”委派任务、Track B 学“**怎么 build**”embed 进 agent + +> 💡 **想看跨 stage 的完整示例？** [7 步构建你的第一个 AI Agent](walkthroughs/build-first-agent-in-7-steps.zh-Hans.md) — 同一个 Paper Summary Bot 从 Stage 1 一路写到 Stage 7，~350 行真实代码（**Track B 适用**） + +走完主干（Track B 16-22 周 / Track A 8-10 周）后，依你的身份挑一条延伸路线继续走。**不确定挑哪条？** + +![Branch 决策树](resources/diagrams/branch-decision-tree.zh-Hans.png) + +> 💡 **“日常用户”这条路线不必走完主干就能直接读**——是给“想用 AI、但不一定要写 code”的人。 + +| 路线 | 适合谁 | 主题 | +|---|---|---| +| 🔬 [研究员](branches/for-researcher.zh-Hans.md) | 研究生、博后、PI | 文献整理 · paper 写作 · multi-agent review | +| 💻 [开发者](branches/for-developer.zh-Hans.md) | 软件工程师 | Cursor · Aider · CLI delegation · code review | +| 🎓 [老师](branches/for-teacher.zh-Hans.md) | 老师、讲师 | 备课 · 幻灯片 · 学生 feedback · 隐私 / 伦理 · prompt 范本 | +| 📊 [知识工作者](branches/for-knowledge-worker.zh-Hans.md) | 顾问、PM、分析师 | Email · 会议记录 · report 自动化 | +| 👥 [日常用户](branches/for-everyday-users.zh-Hans.md) | ChatGPT / Claude.ai 用户 | 写信 · 学习 · 隐私场景 · CLI agent 入门 | + +--- + +## 💡 如何学习 + +这份路线图兼顾概念与实作，目标是带你“从 LLM 用户一路走到 agent 系统构建者”。适合“有基本 Python 能力”的开发者、研究生、自学者。动手之前，先确认你有： + +- 基本 Python — 写过 function、用过 API、看得懂 JSON +- 基本 git — clone、commit、push +- 想学的动机 — agent 是 2025 年之后变化最快的领域，需要持续投入 + +上面有缺的就从 Stage 0 补齐；都会了就直接跳 Stage 1。 + +主干分 5 部分： + +- **Part 1（Stage 0-2）：基础与 LLM 入门** — Python / git / API、什么是 LLM、怎么设计 prompt +- **Part 2（Stage 3-4）：构建你的 Agent** — 从 tool use 进化到 agent，学主流 framework +- **Part 3（Stage 5） 共用 hub** — Claude Code 生态系（MCP / Skills / Plugins / Subagents、Track A + B 都会用到） +- **Part 4（Stage 6-7）：进阶集成** — memory / RAG / multi-agent 协作 / harness engineering +- **Part 5（Stage 8） 共用 hub** — Agent Interfaces（Computer Use / Browser Use / Code Sandbox、2024-2026 frontier、两条 track 都会用到） + +> 🔭 **三层概念进化**：**prompt engineering**（Stage 2、单一 prompt 怎么写）→ **context engineering**（Stage 3 之后、怎么动态组 system prompt + memory + retrieved chunks + tool schema）→ **harness engineering**（Stage 7、agent loop / eval / observability / deploy 整套包成 production system）。3 个术语对应 3 个 phase、不必另外找资源。详见 [`stages/02-prompt-engineering.zh-Hans.md`](stages/02-prompt-engineering.zh-Hans.md) 进阶：context engineering 跟 [`stages/07-multi-agent-production.zh-Hans.md`](stages/07-multi-agent-production.zh-Hans.md) 必修阅读 5+6。 + +走完主干（Track B 16-22 周 / Track A 8-10 周）后，依你的身份挑一条延伸路线继续走。 + +最重要的说一句话：**不要跳过 動手練習**。每个 stage 的 動手練習都是“不动手就学不会”的东西，光读过去后面会卡住。 + +> 🎓 **动手练习怎么用才对**：每个练习 folder 里的 `starter.py` 是**完整解答**、不是 TODO skeleton。如果你 clone 下来直接 `cat starter.py` + `python test.py` pass、会误以为“我学会了”、其实没写一行 code。**正确学习法**：`mv starter.py starter_reference.py`、看 signature 不看 body、自己重写、卡住才回去对照。完整方法论 + 每个 stage 的时间预算 + 卡住处理流程看 [`docs/HOW_TO_USE.md`](docs/HOW_TO_USE.md)。 + +准备好了吗？[从 Stage 0 开始](stages/00-foundations.zh-Hans.md)。 + +--- + +## 📚 相关资源 + +常用入口、依**情境**分组： + +### 🚀 入门 / 环境设定 + +| 你的状况 | 去哪 | 内容 | +|---|---|---| +| 完全没写过 code、第一次接触 AI agent | [`resources/setup-guide.zh-Hans.md`](resources/setup-guide.zh-Hans.md) | 30-45 分钟从零装好（API key、Python、第一个 hello-world） | +| 不知道挑哪个 LLM provider | [`resources/setup-guide.zh-Hans.md` A](resources/setup-guide.zh-Hans.md#a--申请第一个-api-key约-10-分钟) | Anthropic / OpenAI / DeepSeek / Kimi / NVIDIA NIM 对照 | + +### 📖 概念 / 用语 + +| 你的状况 | 去哪 | 内容 | +|---|---|---| +| 不懂某个词（LLM / agent / RAG / token / MCP / Skill / 向量数据库…） | [`resources/glossary.zh-Hans.md`](resources/glossary.zh-Hans.md) | 30+ 词、每个 30-80 字 + 哪 stage 讲细的 | +| 想搞懂 agent 为什么有的在 terminal、有的在 Telegram、有的在 Jetson | [`resources/agent-paradigms.zh-Hans.md`](resources/agent-paradigms.zh-Hans.md) | 5 种 agent 型态 mental model + Hermes / OpenClaw 例子 | +| MCP / Skills / Plugins 用语对照 | [`RESOURCES.zh-Hans.md` 三个核心用语](RESOURCES.zh-Hans.md#三个核心用语mcp--skills--plugins) | 1 页速查表 | +| 想找带证书的线上 AI agent 课（英文 + 中文） | [`resources/courses.zh-Hans.md`](resources/courses.zh-Hans.md) | 10 门 credible、会发证书的课，分 tier；并诚实标注完成证书不是学历 | + +### 🛠 动手实作 + +| 你的状况 | 去哪 | 内容 | +|---|---|---| +| 想动手写 Skill / MCP server / 接 Word / Zotero / 本机 LLM | [`resources/cookbook.zh-Hans.md`](resources/cookbook.zh-Hans.md) | 6 个 step-by-step recipe、每个 30-50 分钟 | +| 想用 subagent 但不知道该派谁、怎么派、派什么工作 | [`resources/subagent-cookbook.zh-Hans.md`](resources/subagent-cookbook.zh-Hans.md) | 15 个复制粘贴即用的 dispatch recipe | +| 卡在 tool calling（LLM 不调用 / schema 写不好 / ReAct loop 跑不停） | [`examples/stage-5/tool-calling-tutor/`](examples/stage-5/tool-calling-tutor/) | 可装进 Claude Code 的 skill、4-symptom diagnostic | +| 动手练习怎么正确使用（主动 vs 被动模式） | [`docs/HOW_TO_USE.md`](docs/HOW_TO_USE.md) | 5-10 分钟读完、配合每个 stage 用 | + +### 三个核心用语：MCP / Skills / Plugins + +README 跟各 stage 会频繁提到这三个 Claude Code 生态的关键词，先快速说明： + +- **MCP（Model Context Protocol）** — Anthropic 推的开放协议，让任何 LLM host（Claude Code、其他 IDE、自写 agent）都能用同一套接口去调用外部 tool server（文件、DB、API、自家服务）。把它想成“LLM 的 USB 接口”。详见 [Stage 5.2](stages/05-claude-code-ecosystem.zh-Hans.md#52--mcpmodel-context-protocol-基础)。 +- **Skills** — Claude Code 的“行为包”。一个 Skill 就是一份 `SKILL.md`，描述“在什么情境要做什么、可以调用哪些 MCP tool”。写好之后 Claude Code 会自动 discover。详见 [Stage 5.3](stages/05-claude-code-ecosystem.zh-Hans.md#53--skillsclaude-code-的行为层-claude-code-生态最关键的一层)。 +- **Plugins / Marketplaces** — 把 Skills、slash commands、hooks、MCP 设置打包成一个发布单位给 team 或社群安装。Marketplace 就是 plugin 的 catalog。详见 [Stage 5.4](stages/05-claude-code-ecosystem.zh-Hans.md#54--plugins-与-marketplaces)。 + +对, 应的 動手練習 练习都在 [Stage 5](stages/05-claude-code-ecosystem.zh-Hans.md)，Track A 的 [A3](tracks/cli/A3-cli-production.zh-Hans.md) 也会用到。 + +### 接日常工具：常用 MCP server / Skill + +把 Claude Code（或其他 CLI agent）接到你已经在用的 app，省掉手动切换的成本。下面几个是社群 / 官方比较成熟的： + +**笔记 / 知识库** + +- [**MarkusPfundstein/mcp-obsidian**](https://github.com/MarkusPfundstein/mcp-obsidian) ★ 3.9k+ — 通过 Obsidian REST API plugin 让 LLM 读写你的 Obsidian vault +- [**makenotion/notion-mcp-server**](https://github.com/makenotion/notion-mcp-server) ★ 4.4k+ — Notion **官方** MCP server，可查询／创建 page、database +- [**PleasePrompto/notebooklm-skill**](https://github.com/PleasePrompto/notebooklm-skill) ★ 7.3k+ — NotebookLM Skill（浏览器自动化），用 Claude Code 直接查你 NotebookLM 里的文件，回答带 citation +- [**teng-lin/notebooklm-py**](https://github.com/teng-lin/notebooklm-py) ★ 16k+ — 非官方 NotebookLM Python API + CLI，支持 Claude Code / Codex 等 agent 集成 + +**办公文件（Word / Excel / PowerPoint / PDF）** + +- [**anthropics/skills**](https://github.com/anthropics/skills) ★ 158k+ — Anthropic **官方** Skills 集合，docx / xlsx / pptx / pdf 处理直接内置 +- [**tfriedel/claude-office-skills**](https://github.com/tfriedel/claude-office-skills) ★ 725 — 增强版 Office skills（PPTX/DOCX/XLSX/PDF），含自动化 workflow + +**Google Workspace（Gmail / Docs / Drive / Calendar）** + +- [**taylorwilsdon/google_workspace_mcp**](https://github.com/taylorwilsdon/google_workspace_mcp) ★ 2.6k+ — 一个 server 包整套 Google Workspace（Gmail、Calendar、Docs、Sheets、Slides、Drive） + +**开发协作** + +- [**github/github-mcp-server**](https://github.com/github/github-mcp-server) ★ 29k+ — GitHub **官方** MCP，issue / PR / repo 操作 +- [**atlassian/atlassian-mcp-server**](https://github.com/atlassian/atlassian-mcp-server) ★ 810 — Atlassian **官方** Remote MCP（Jira、Confluence） +- [**jerhadf/linear-mcp-server**](https://github.com/jerhadf/linear-mcp-server) ★ 340+ — Linear MCP server +- [**korotovsky/slack-mcp-server**](https://github.com/korotovsky/slack-mcp-server) ★ 1.7k+ — Slack MCP，无 admin 权限也能用 + +**中文圈常用** + +- [**leemysw/feishu-docx**](https://github.com/leemysw/feishu-docx) ★ 235 — 飞书（Lark）docs / sheet / bitable ↔ Markdown，含 Claude Skills 支持 + +> 上面只是 highlight。**完整 65+ 个集成**（含数据库、浏览器自动化、Figma、Excalidraw、Cloudflare、Stripe…）：[`resources/mcp-skills-catalog.zh-Hans.md`](resources/mcp-skills-catalog.zh-Hans.md)。 + +> 想找更多 MCP server catalog？看 [`wong2/awesome-mcp-servers`](https://github.com/wong2/awesome-mcp-servers) / [`punkpeye/awesome-mcp-servers`](https://github.com/punkpeye/awesome-mcp-servers)（按分类整理）。**Canva** 的官方 MCP 还在 early access，社群版本不稳定，等成熟后再补上。 + +### 同主题的清单型 awesome lists + +这个 repo **不取代**清单型 awesome list — 你已经知道在找什么工具时，下面这些查起来更直接： + +**MCP 相关** + +- [**modelcontextprotocol/servers**](https://github.com/modelcontextprotocol/servers) — 官方 MCP reference servers（filesystem、github、sqlite、git、fetch、memory 等） +- [**wong2/awesome-mcp-servers**](https://github.com/wong2/awesome-mcp-servers) — 社群 MCP server 清单，按分类整理（150+ 个） +- [**punkpeye/awesome-mcp-servers**](https://github.com/punkpeye/awesome-mcp-servers) — 另一份 MCP server 清单 + +**Claude Code / Skills / Plugins 相关** + +- [**hesreallyhim/awesome-claude-code**](https://github.com/hesreallyhim/awesome-claude-code) — Claude Code 相关工具与 plugin 清单（整理中） +- [**travisvn/awesome-claude-skills**](https://github.com/travisvn/awesome-claude-skills) — Claude Skills 清单 +- [**anthropics/claude-plugins-official**](https://github.com/anthropics/claude-plugins-official) — Anthropic 官方 plugin 模板，要打包自己的 plugin 从这份开始 + +**中文圈常用** + +- [**datawhalechina/hello-agents**](https://github.com/datawhalechina/hello-agents) — Datawhale 系统性 agent 教学（zh-Hans） +- [**WangRongsheng/awesome-LLM-resources**](https://github.com/WangRongsheng/awesome-LLM-resources) — 完整的中文 LLM 资源整理（8k+ stars） +- [**AiHubCN/Awesome-Chinese-LLM**](https://github.com/AiHubCN/Awesome-Chinese-LLM) — 中文开源大模型整理 + +--- + +## 🤝 如何贡献 + +这个 repo 是一个 AI 学习文档，如果你也有收集很好的资源，也欢迎贡献： + +- 🐛 **汇报 Bug** — 内容错误、链接失效、过时信息 → 开 Issue +- 💡 **提建议** — 缺什么 stage、该加哪个 project → 开 Issue 讨论 +- 📝 **完善内容** — 改进现有 stage 内容、修 typo → 直接 PR +- ✍️ **新增 project** — 在某个 stage 加 1-3 个 project，并附上“为什么这个 project 适合放这个 stage”的说明 +- 🌏 **翻译** — 补英文 companion 没翻到的段落，或翻成其他语言 +- 🌱 **担任 Stage / Branch maintainer** — 长期 review 特定领域，详见 [CONTRIBUTING.md](CONTRIBUTING.md) 和 [resources/style-guide.zh-Hans.md](resources/style-guide.zh-Hans.md)。 + +PR 流程跟 style 规范请看 [CONTRIBUTING.md](CONTRIBUTING.md) 和 [resources/style-guide.zh-Hans.md](resources/style-guide.zh-Hans.md)。 + +> 📅 **想看最近 ship 了什么** → [`CHANGELOG.md`](CHANGELOG.md)（最近 14 天）。 +> Maintainer 内部进度与 launch checklist 放在 [.github/launch-checklist.md](.github/launch-checklist.md)（内部文件）。 + +--- + +## 💬 顾问 / 联系 + +公开学习版（MIT），欢迎自由取用。 + +目前以顾问为主：团队或公司若需 **prompt review / audit** 或 **AI agent workflow 咨询**，欢迎来信（博士生、时间有限）：📧 [wenyuchiou12@gmail.com](mailto:wenyuchiou12@gmail.com) + +--- + +## 🙏 致谢 + +### Inspiration + +- [**Datawhale Hello-Agents**](https://github.com/datawhalechina/hello-agents) — 中文圈最完整的 chapter-length agent 教材，本 repo 的“章节 + 进度”结构受这份启发；每个 stage / 练习 folder 都有 📚 callout 点过去深度章节。特别感谢。 +- [**Datawhale 社群**](https://github.com/datawhalechina) — 中文 ML 共学社群的标杆，本 repo 多个 anchor project 来自这里 +- [**liyupi/ai-guide**](https://github.com/liyupi/ai-guide) — 中文圈最大"AI 资源大全" + Vibe Coding 教学（涵盖 Agent Skills / RAG / MCP / A2A / Harness Engineering）。本 repo 是"结构化路线"、ai-guide 是"广度资源库"，互为补充 + +### 其他相关项目 + +同主题、不同切入角度的清单，搜资源时可以一起用： + +- [`wong2/awesome-mcp-servers`](https://github.com/wong2/awesome-mcp-servers) — MCP server 清单，按分类整理 +- [`punkpeye/awesome-mcp-servers`](https://github.com/punkpeye/awesome-mcp-servers) — 另一份 MCP server 清单 +- [`hesreallyhim/awesome-claude-code`](https://github.com/hesreallyhim/awesome-claude-code) — Claude Code 相关工具与 plugin 清单（整理中） +- [`travisvn/awesome-claude-skills`](https://github.com/travisvn/awesome-claude-skills) — Claude Skills 清单 +- [`anthropics/claude-plugins-official`](https://github.com/anthropics/claude-plugins-official) — Anthropic 官方 plugin 模板，要打包自己的 plugin 从这份开始 + +这些是纯清单形式（看到再挑），本 repo 的不同点是有“从 Stage 0 一路走到 production 的学习顺序”。 + +### 贡献者 + +[![Contributors](https://contrib.rocks/image?repo=WenyuChiou/awesome-agentic-ai-zh)](https://github.com/WenyuChiou/awesome-agentic-ai-zh/graphs/contributors) + +新贡献者会自动出现在上方。完整列表 → [GitHub Contributors](https://github.com/WenyuChiou/awesome-agentic-ai-zh/graphs/contributors)。 + +### 个人 + +- [@WenyuChiou](https://github.com/WenyuChiou) — Maintainer + +--- + +## 🎓 引用 + +如果这个学习地图对你的学习或工作有帮助，欢迎引用： + +```bibtex +@misc{awesome_agentic_ai_zh_2026, + title = {awesome-agentic-ai-zh: A Structured Learning Roadmap for Agentic AI}, + author = {Chiou, Wenyu}, + year = {2026}, + url = {https://github.com/WenyuChiou/awesome-agentic-ai-zh}, + note = {8-stage learning path from prerequisites to Agent Interfaces (Computer Use / Browser Use / Code Sandbox), with curated projects + hello-X demos. Bilingual (zh-TW / English).} +} +``` + +--- + +## ☕ 支持这个项目 + +这份学习地图是免费、开源（MIT）。如果它对你有帮助，除了给个 ⭐ Star，也欢迎请作者喝杯咖啡、支持它持续更新： + +Buy Me A Coffee + +或直接点 repo 右上角的 **❤ Sponsor** 按钮。（GitHub Sponsors 审核中，通过后会一并加上。） + +--- + +## License + +MIT。Maintained by [@WenyuChiou](https://github.com/WenyuChiou)。 + +
+

⭐ 如果这个 repo 对你有帮助，欢迎给个 Star — 这对作者继续更新是很大的鼓励

+
diff --git a/RESOURCES.en.md b/RESOURCES.en.md new file mode 100644 index 0000000..16ee514 --- /dev/null +++ b/RESOURCES.en.md @@ -0,0 +1,102 @@ +# Related Resources + +> [繁體中文](./RESOURCES.md) | [简体中文](./RESOURCES.zh-Hans.md) | **English** + +> [← Back to main README](README.en.md) + +This file collects: term definitions, daily-tool MCP/Skill highlights, topic-based awesome lists, Chinese-community resources. Pulled out of the main README to keep that page focused. + +> 💡 **Don't know a term?** (LLM, agent, RAG, token, vector DB, …) → [`resources/glossary.en.md`](resources/glossary.en.md) — 30+ common terms with 30–80-word definitions + +--- + +## Three core terms: MCP / Skills / Plugins + +The README and stages reference these three Claude Code ecosystem terms a lot. Quick definitions: + +- **MCP (Model Context Protocol)** — Anthropic's open protocol that lets any LLM host (Claude Code, other IDEs, your own agent) talk to any external tool server (filesystem, DB, API, your service) through one interface. Think "USB for LLMs". See [Stage 5.2](stages/05-claude-code-ecosystem.en.md#52--mcp-model-context-protocol--foundation). +- **Skills** — Claude Code's "behavior bundles". A Skill is a `SKILL.md` describing "in what context, do what, can call which MCP tools". Claude Code auto-discovers them. See [Stage 5.3](stages/05-claude-code-ecosystem.en.md#53--skills-claude-codes-behavior-layer--the-most-critical-layer-of-the-claude-code-ecosystem). +- **Plugins / Marketplaces** — package Skills, slash commands, hooks, and MCP configs into a distribution unit installable by your team or community. A marketplace is a catalog of plugins. See [Stage 5.4](stages/05-claude-code-ecosystem.en.md#54--plugins--marketplaces). + +Hands-on exercises live in [Stage 5](stages/05-claude-code-ecosystem.en.md), with Track A's [A3](tracks/cli/A3-cli-production.en.md) covering production integration. + +--- + +## Daily-tool integrations: MCP servers + Skills + +Connect Claude Code (or any other CLI agent) to the apps you already use, without window-hopping. Mature picks below: + +### Notes / Knowledge Base + +- [**MarkusPfundstein/mcp-obsidian**](https://github.com/MarkusPfundstein/mcp-obsidian) ★ 3.9k+ — Obsidian REST API plugin lets the LLM read/write your vault +- [**makenotion/notion-mcp-server**](https://github.com/makenotion/notion-mcp-server) ★ 4.4k+ — Notion **official** MCP, query/create pages, manipulate databases +- [**PleasePrompto/notebooklm-skill**](https://github.com/PleasePrompto/notebooklm-skill) ★ 7.3k+ — NotebookLM Skill, citation-backed answers from your uploaded docs +- [**teng-lin/notebooklm-py**](https://github.com/teng-lin/notebooklm-py) ★ 16k+ — unofficial NotebookLM Python API + CLI, plays well with Claude Code / Codex + +### Office Documents (Word / Excel / PowerPoint / PDF) + +- [**anthropics/skills**](https://github.com/anthropics/skills) ★ 158k+ — Anthropic **official** Skills with built-in docx / xlsx / pptx / pdf processing +- [**tfriedel/claude-office-skills**](https://github.com/tfriedel/claude-office-skills) ★ 725 — Office skills with automation workflows on top of the official ones + +### Google Workspace (Gmail / Docs / Drive / Calendar) + +- [**taylorwilsdon/google_workspace_mcp**](https://github.com/taylorwilsdon/google_workspace_mcp) ★ 2.6k+ — full Workspace stack (Gmail, Calendar, Docs, Sheets, Slides, Drive) in one server + +### Dev Collaboration + +- [**github/github-mcp-server**](https://github.com/github/github-mcp-server) ★ 29k+ — GitHub **official** MCP for issues / PRs / repos +- [**atlassian/atlassian-mcp-server**](https://github.com/atlassian/atlassian-mcp-server) ★ 810 — Atlassian **official** Remote MCP (Jira, Confluence) +- [**jerhadf/linear-mcp-server**](https://github.com/jerhadf/linear-mcp-server) ★ 340+ — Linear MCP +- [**korotovsky/slack-mcp-server**](https://github.com/korotovsky/slack-mcp-server) ★ 1.7k+ — Slack MCP, works without admin permissions + +### Research Workflow (by the repo maintainer) + +- [**WenyuChiou/ai-research-skills**](https://github.com/WenyuChiou/ai-research-skills) ★ 123 — 14 research-workflow skills as a 5-plugin marketplace +- [**WenyuChiou/research-hub**](https://github.com/WenyuChiou/research-hub) ★ 33 — Zotero + Obsidian + NotebookLM integration workspace +- [**WenyuChiou/zotero-skills**](https://github.com/WenyuChiou/zotero-skills) ★ 28 — Zotero CLI skill +- [**WenyuChiou/codex-delegate**](https://github.com/WenyuChiou/codex-delegate) ★ 57 + [**gemini-delegate-skill**](https://github.com/WenyuChiou/gemini-delegate-skill) ★ 34 — multi-LLM delegation pair + +### Chinese-language Ecosystem + +- [**leemysw/feishu-docx**](https://github.com/leemysw/feishu-docx) ★ 235 — Feishu (Lark) docs / sheet / bitable ↔ Markdown with Claude Skills support + +> The above is just the highlights. **Full 65+ entry catalog by category** (incl. databases, browser automation, Figma, Excalidraw, Cloudflare, Stripe, academic-writing / multi-LLM delegation, etc.) lives in [`resources/mcp-skills-catalog.en.md`](resources/mcp-skills-catalog.en.md). + +> Looking for more MCP server catalogs? See [`wong2/awesome-mcp-servers`](https://github.com/wong2/awesome-mcp-servers) / [`punkpeye/awesome-mcp-servers`](https://github.com/punkpeye/awesome-mcp-servers) (categorized). **Canva**'s official MCP is still early access — community versions are unstable; will add when stable. + +--- + +## Topic-based awesome lists + +This repo **doesn't replace** flat awesome lists. When you already know which tool you want, these are more direct: + +### MCP-related + +- [**modelcontextprotocol/servers**](https://github.com/modelcontextprotocol/servers) — official reference servers (filesystem, github, sqlite, git, fetch, memory, …) +- [**wong2/awesome-mcp-servers**](https://github.com/wong2/awesome-mcp-servers) — community MCP server catalog, by category (150+) +- [**punkpeye/awesome-mcp-servers**](https://github.com/punkpeye/awesome-mcp-servers) — another MCP server catalog + +### Claude Code / Skills / Plugins-related + +- [**hesreallyhim/awesome-claude-code**](https://github.com/hesreallyhim/awesome-claude-code) — Claude Code resources (currently restructuring) +- [**travisvn/awesome-claude-skills**](https://github.com/travisvn/awesome-claude-skills) — Claude Skills catalog +- [**anthropics/claude-plugins-official**](https://github.com/anthropics/claude-plugins-official) — Anthropic's official plugin marketplace template; start here when packaging your own plugin + +### Chinese-speaking community + +- [**datawhalechina/hello-agents**](https://github.com/datawhalechina/hello-agents) — Datawhale systematic agent tutorial (zh-Hans) +- [**WangRongsheng/awesome-LLM-resources**](https://github.com/WangRongsheng/awesome-LLM-resources) — comprehensive zh-Hans LLM resources (8k+ stars) +- [**AiHubCN/Awesome-Chinese-LLM**](https://github.com/AiHubCN/Awesome-Chinese-LLM) — open-source Chinese LLM catalog + +### Online courses / MOOCs (certificate comparison) + +- [**resources/courses.en.md**](resources/courses.en.md) — 10 credible, certificate-granting online AI agent courses (EN + ZH), tiered; with an honest "completion certificate ≠ a degree" caveat + +--- + +## What else? + +- Main README: [README.en.md](README.en.md) +- Full MCP / Skill catalog: [resources/mcp-skills-catalog.en.md](resources/mcp-skills-catalog.en.md) +- CLI agent comparison guide: [resources/cli-agents-guide.en.md](resources/cli-agents-guide.en.md) +- Style guide / contributing: [resources/style-guide.en.md](resources/style-guide.en.md), [CONTRIBUTING.en.md](CONTRIBUTING.en.md) diff --git a/RESOURCES.md b/RESOURCES.md new file mode 100644 index 0000000..dc79055 --- /dev/null +++ b/RESOURCES.md @@ -0,0 +1,104 @@ +# 相關資源 + +> **繁體中文** | [简体中文](./RESOURCES.zh-Hans.md) | [English](./RESOURCES.en.md) + +> [← 回主路線 README](README.md) + +這份檔案集中放：用語說明、常用 MCP / Skill 整合 highlight、同主題 awesome list、中文社群資源。從主 README 抽出來避免主頁過長。 + +> 💡 **不懂某個詞**（LLM、agent、RAG、token、向量資料庫⋯）→ [`resources/glossary.md`](resources/glossary.md)（用語小辭典，30 多個詞每個 30-80 字解釋） +> +> 🍳 **想動手做但不知怎麼開始**（寫 Skill / 寫 MCP server / 接 Word / 接 NotebookLM / 接 Zotero / 接本機 LLM）→ [`resources/cookbook.md`](resources/cookbook.md)（6 個 step-by-step recipe，每個 30-50 分鐘做完） + +--- + +## 三個核心用語：MCP / Skills / Plugins + +主 README 跟各 stage 會頻繁提到這三個 Claude Code 生態的關鍵詞，先快速說明： + +- **MCP（Model Context Protocol）** — Anthropic 推的開放協定，讓任何 LLM host（Claude Code、其他 IDE、自寫 agent）都能用同一套介面去呼叫外部 tool server（檔案、DB、API、自家服務）。把它想成「LLM 的 USB 接口」。詳見 [Stage 5.2](stages/05-claude-code-ecosystem.md#52--mcpmodel-context-protocol-基礎)。 +- **Skills** — Claude Code 的「行為包」。一個 Skill 就是一份 `SKILL.md`，描述「在什麼情境要做什麼、可以呼叫哪些 MCP tool」。寫好之後 Claude Code 會自動 discover。詳見 [Stage 5.3](stages/05-claude-code-ecosystem.md#53--skillsclaude-code-的行為層-claude-code-生態最關鍵的一層)。 +- **Plugins / Marketplaces** — 把 Skills、slash commands、hooks、MCP 設定打包成一個發佈單位給 team 或社群安裝。Marketplace 就是 plugin 的 catalog。詳見 [Stage 5.4](stages/05-claude-code-ecosystem.md#54--plugins-與-marketplaces)。 + +對應的 動手練習都在 [Stage 5](stages/05-claude-code-ecosystem.md)，Track A 的 [A3](tracks/cli/A3-cli-production.md) 也會用到。 + +--- + +## 接日常工具：常用 MCP server / Skill + +把 Claude Code（或其他 CLI agent）接到你已經在用的 app，省掉手動切換的成本。下面幾個是社群 / 官方比較成熟的： + +### 筆記 / 知識庫 + +- [**MarkusPfundstein/mcp-obsidian**](https://github.com/MarkusPfundstein/mcp-obsidian) ★ 3.9k+ — 透過 Obsidian REST API plugin 讓 LLM 讀寫你的 Obsidian vault +- [**makenotion/notion-mcp-server**](https://github.com/makenotion/notion-mcp-server) ★ 4.4k+ — Notion **官方** MCP server，可查詢／建立 page、database +- [**PleasePrompto/notebooklm-skill**](https://github.com/PleasePrompto/notebooklm-skill) ★ 7.3k+ — NotebookLM Skill（瀏覽器自動化），用 Claude Code 直接查你 NotebookLM 裡的文件，回答帶 citation +- [**teng-lin/notebooklm-py**](https://github.com/teng-lin/notebooklm-py) ★ 16k+ — 非官方 NotebookLM Python API + CLI，支援 Claude Code / Codex 等 agent 整合 + +### 辦公文件（Word / Excel / PowerPoint / PDF） + +- [**anthropics/skills**](https://github.com/anthropics/skills) ★ 158k+ — Anthropic **官方** Skills 集合，docx / xlsx / pptx / pdf 處理直接內建 +- [**tfriedel/claude-office-skills**](https://github.com/tfriedel/claude-office-skills) ★ 725 — 補強版 Office skills（PPTX/DOCX/XLSX/PDF），含自動化 workflow + +### Google Workspace（Gmail / Docs / Drive / Calendar） + +- [**taylorwilsdon/google_workspace_mcp**](https://github.com/taylorwilsdon/google_workspace_mcp) ★ 2.6k+ — 一個 server 包整套 Google Workspace（Gmail、Calendar、Docs、Sheets、Slides、Drive） + +### 開發協作 + +- [**github/github-mcp-server**](https://github.com/github/github-mcp-server) ★ 29k+ — GitHub **官方** MCP，issue / PR / repo 操作 +- [**atlassian/atlassian-mcp-server**](https://github.com/atlassian/atlassian-mcp-server) ★ 810 — Atlassian **官方** Remote MCP（Jira、Confluence） +- [**jerhadf/linear-mcp-server**](https://github.com/jerhadf/linear-mcp-server) ★ 340+ — Linear MCP server +- [**korotovsky/slack-mcp-server**](https://github.com/korotovsky/slack-mcp-server) ★ 1.7k+ — Slack MCP，無 admin 權限也能用 + +### 研究工作流（本 repo 維護者出品） + +- [**WenyuChiou/ai-research-skills**](https://github.com/WenyuChiou/ai-research-skills) ★ 123 — 14 個研究流程 skill，5-plugin marketplace +- [**WenyuChiou/research-hub**](https://github.com/WenyuChiou/research-hub) ★ 33 — Zotero + Obsidian + NotebookLM 整合 workspace +- [**WenyuChiou/zotero-skills**](https://github.com/WenyuChiou/zotero-skills) ★ 28 — Zotero CLI skill +- [**WenyuChiou/codex-delegate**](https://github.com/WenyuChiou/codex-delegate) ★ 57 + [**gemini-delegate-skill**](https://github.com/WenyuChiou/gemini-delegate-skill) ★ 34 — Multi-LLM delegation 對 + +### 中文圈常用 + +- [**leemysw/feishu-docx**](https://github.com/leemysw/feishu-docx) ★ 235 — 飛書（Lark）docs / sheet / bitable ↔ Markdown，含 Claude Skills 支援 + +> 上面只是 highlight。**完整 65+ 個整合的分類目錄**（含資料庫、瀏覽器自動化、Figma、Excalidraw、Cloudflare、Stripe、學術寫作 / Multi-LLM delegation 等）在 [`resources/mcp-skills-catalog.md`](resources/mcp-skills-catalog.md)。 + +> 想找更多 MCP server catalog？看 [`wong2/awesome-mcp-servers`](https://github.com/wong2/awesome-mcp-servers) / [`punkpeye/awesome-mcp-servers`](https://github.com/punkpeye/awesome-mcp-servers)（依分類整理）。**Canva** 的官方 MCP 還在 early access，社群版本不穩定，等成熟後再補上。 + +--- + +## 同主題的清單型 awesome lists + +本 repo **不取代**清單型 awesome list——你已經知道在找什麼工具時，下面這些查起來更直接： + +### MCP 相關 + +- [**modelcontextprotocol/servers**](https://github.com/modelcontextprotocol/servers) — 官方 MCP reference servers（filesystem、github、sqlite、git、fetch、memory 等） +- [**wong2/awesome-mcp-servers**](https://github.com/wong2/awesome-mcp-servers) — 社群 MCP server 清單，按分類整理（150+ 個） +- [**punkpeye/awesome-mcp-servers**](https://github.com/punkpeye/awesome-mcp-servers) — 另一份 MCP server 清單 + +### Claude Code / Skills / Plugins 相關 + +- [**hesreallyhim/awesome-claude-code**](https://github.com/hesreallyhim/awesome-claude-code) — Claude Code 相關資源清單（整理中） +- [**travisvn/awesome-claude-skills**](https://github.com/travisvn/awesome-claude-skills) — Claude Skills 清單 +- [**anthropics/claude-plugins-official**](https://github.com/anthropics/claude-plugins-official) — Anthropic 官方 plugin 範本，要打包自己的 plugin 從這份開始 + +### 中文社群必看 + +- [**datawhalechina/hello-agents**](https://github.com/datawhalechina/hello-agents) — Datawhale 系統性 agent 教學（zh-Hans） +- [**WangRongsheng/awesome-LLM-resources**](https://github.com/WangRongsheng/awesome-LLM-resources) — 完整的中文 LLM 資源整理（8k+ stars） +- [**AiHubCN/Awesome-Chinese-LLM**](https://github.com/AiHubCN/Awesome-Chinese-LLM) — 中文開源大模型整理 + +### 線上課程 / MOOC（帶證書對照） + +- [**resources/courses.md**](resources/courses.md) — 10 門 credible、會發證書的線上 AI agent 課（英文 + 中文），分 tier；含「完成證書 ≠ 學歷」的誠實 caveat + +--- + +## 還有什麼？ + +- 主 README：[README.md](README.md) +- 完整 MCP/Skill 目錄：[resources/mcp-skills-catalog.md](resources/mcp-skills-catalog.md) +- CLI agent 比較指南：[resources/cli-agents-guide.md](resources/cli-agents-guide.md) +- Style guide / 貢獻規範：[resources/style-guide.md](resources/style-guide.md)、[CONTRIBUTING.md](CONTRIBUTING.md) diff --git a/RESOURCES.zh-Hans.md b/RESOURCES.zh-Hans.md new file mode 100644 index 0000000..9d7e1a9 --- /dev/null +++ b/RESOURCES.zh-Hans.md @@ -0,0 +1,102 @@ +# 相关资源 + +> [繁體中文](./RESOURCES.md) | **简体中文** | [English](./RESOURCES.en.md) + +> [← 返回主路线 README](README.zh-Hans.md) + +这份文件集中放：用語说明、常用 MCP / Skill 集成 highlight、同主题 awesome list、中文社群资源。从主 README 抽出避免主页过长。 + +> 💡 **不懂某个词**（LLM、agent、RAG、token、向量数据库⋯）→ [`resources/glossary.zh-Hans.md`](resources/glossary.zh-Hans.md)（用語小词典，30 多个词每个 30-80 字解释） + +--- + +## 三个核心用语：MCP / Skills / Plugins + +主 README 跟各 stage 会频繁提到这三个 Claude Code 生态的关键词，先快速说明： + +- **MCP（Model Context Protocol）** — Anthropic 推的开放协定，让任何 LLM host（Claude Code、其他 IDE、自写 agent）都能用同一套接口去调用外部 tool server（文件、DB、API、自家系统）。把它想成“LLM 的 USB 接口”。详见 [Stage 5.2](stages/05-claude-code-ecosystem.zh-Hans.md#52--mcpmodel-context-protocol-基础)。 +- **Skills** — Claude Code 的“行为包”。一个 Skill 就是一份 `SKILL.zh-Hans.md`，描述“在什么场景要做什么、可以调用哪些 MCP tool”。写好之后 Claude Code 会自动 discover。详见 [Stage 5.3](stages/05-claude-code-ecosystem.zh-Hans.md#53--skillsclaude-code-的行为层-claude-code-生态最关键的一层)。 +- **Plugins / Marketplaces** — 把 Skills、slash commands、hooks、MCP 设置打包成一个发布单位给 team 或社群安装。Marketplace 就是 plugin 的 catalog。详见 [Stage 5.4](stages/05-claude-code-ecosystem.zh-Hans.md#54--plugins-与-marketplaces)。 + +对应的 **动手练习都在 [Stage 5](stages/05-claude-code-ecosystem.zh-Hans.md)，Track A 的 [A3](tracks/cli/A3-cli-production.zh-Hans.md) 也会用到。 + +--- + +## 接日常工具：常用 MCP server / Skill + +把 Claude Code（或其他 CLI agent）接到你已经在用的 app，省掉手动切换的成本。下面几个是社群 / 官方比较成熟的： + +### 笔记 / 知识库 + +- [**MarkusPfundstein/mcp-obsidian**](https://github.com/MarkusPfundstein/mcp-obsidian) ★ 3.9k+ — 透过 Obsidian REST API plugin 让 LLM 读写你的 Obsidian vault +- [**makenotion/notion-mcp-server**](https://github.com/makenotion/notion-mcp-server) ★ 4.4k+ — Notion **官方** MCP server，可查询／建立 page、database +- [**PleasePrompto/notebooklm-skill**](https://github.com/PleasePrompto/notebooklm-skill) ★ 7.3k+ — NotebookLM Skill（浏览器自动化），用 Claude Code 直接查你 NotebookLM 里的文件，回答带 citation +- [**teng-lin/notebooklm-py**](https://github.com/teng-lin/notebooklm-py) ★ 16k+ — 非官方 NotebookLM Python API + CLI，支持 Claude Code / Codex 等 agent 集成 + +### 办公文件（Word / Excel / PowerPoint / PDF） + +- [**anthropics/skills**](https://github.com/anthropics/skills) ★ 158k+ — Anthropic **官方** Skills 集合，docx / xlsx / pptx / pdf 处理直接内建 +- [**tfriedel/claude-office-skills**](https://github.com/tfriedel/claude-office-skills) ★ 725 — 补强版 Office skills（PPTX/DOCX/XLSX/PDF），含自动化 workflow + +### Google Workspace（Gmail / Docs / Drive / Calendar） + +- [**taylorwilsdon/google_workspace_mcp**](https://github.com/taylorwilsdon/google_workspace_mcp) ★ 2.6k+ — 一个 server 包整套 Google Workspace（Gmail、Calendar、Docs、Sheets、Slides、Drive） + +### 开发协作 + +- [**github/github-mcp-server**](https://github.com/github/github-mcp-server) ★ 29k+ — GitHub **官方** MCP，issue / PR / repo 操作 +- [**atlassian/atlassian-mcp-server**](https://github.com/atlassian/atlassian-mcp-server) ★ 810 — Atlassian **官方** Remote MCP（Jira、Confluence） +- [**jerhadf/linear-mcp-server**](https://github.com/jerhadf/linear-mcp-server) ★ 340+ — Linear MCP server +- [**korotovsky/slack-mcp-server**](https://github.com/korotovsky/slack-mcp-server) ★ 1.7k+ — Slack MCP，无 admin 权限也能用 + +### 研究工作流（本 repo 维护者出品） + +- [**WenyuChiou/ai-research-skills**](https://github.com/WenyuChiou/ai-research-skills) ★ 123 — 14 个研究流程 skill，5-plugin marketplace +- [**WenyuChiou/research-hub**](https://github.com/WenyuChiou/research-hub) ★ 33 — Zotero + Obsidian + NotebookLM 集成 workspace +- [**WenyuChiou/zotero-skills**](https://github.com/WenyuChiou/zotero-skills) ★ 28 — Zotero CLI skill +- [**WenyuChiou/codex-delegate**](https://github.com/WenyuChiou/codex-delegate) ★ 57 + [**gemini-delegate-skill**](https://github.com/WenyuChiou/gemini-delegate-skill) ★ 34 — Multi-LLM delegation 对 + +### 中文圈常用 + +- [**leemysw/feishu-docx**](https://github.com/leemysw/feishu-docx) ★ 235 — 飞书（Lark）docs / sheet / bitable ↔ Markdown，含 Claude Skills 支持 + +> 上面只是 highlight。**完整 65+ 个集成的分类目录**（含数据库、浏览器自动化、Figma、Excalidraw、Cloudflare、Stripe、学术写作 / Multi-LLM delegation 等）在 [`resources/mcp-skills-catalog.zh-Hans.md`](resources/mcp-skills-catalog.zh-Hans.md)。 + +> 想找更多 MCP server catalog？看 [`wong2/awesome-mcp-servers`](https://github.com/wong2/awesome-mcp-servers) / [`punkpeye/awesome-mcp-servers`](https://github.com/punkpeye/awesome-mcp-servers)（按分类整理）。**Canva** 的官方 MCP 还在 early access，社群版本不稳定，等成熟后再补上。 + +--- + +## 同主题的清单型 awesome lists + +本 repo **不取代清单型 awesome list——你已经知道在找什么工具时，下面这些查起来更直接： + +### MCP 相关 + +- [**modelcontextprotocol/servers**](https://github.com/modelcontextprotocol/servers) — 官方 MCP reference servers（filesystem、github、sqlite、git、fetch、memory 等） +- [**wong2/awesome-mcp-servers**](https://github.com/wong2/awesome-mcp-servers) — 社群 MCP server 清单，按分类整理（150+ 个） +- [**punkpeye/awesome-mcp-servers**](https://github.com/punkpeye/awesome-mcp-servers) — 另一份 MCP server 清单 + +### Claude Code / Skills / Plugins 相关 + +- [**hesreallyhim/awesome-claude-code**](https://github.com/hesreallyhim/awesome-claude-code) — Claude Code 相关资源清单（整理中） +- [**travisvn/awesome-claude-skills**](https://github.com/travisvn/awesome-claude-skills) — Claude Skills 清单 +- [**anthropics/claude-plugins-official**](https://github.com/anthropics/claude-plugins-official) — Anthropic 官方 plugin 范本，要打包自己的 plugin 从这份开始 + +### 中文圈必看 + +- [**datawhalechina/hello-agents**](https://github.com/datawhalechina/hello-agents) — Datawhale 系统性 agent 教学（zh-Hans） +- [**WangRongsheng/awesome-LLM-resources**](https://github.com/WangRongsheng/awesome-LLM-resources) — 完整的中文 LLM 资源整理（8k+ stars） +- [**AiHubCN/Awesome-Chinese-LLM**](https://github.com/AiHubCN/Awesome-Chinese-LLM) — 中文开源大模型整理 + +### 线上课程 / MOOC（带证书对照） + +- [**resources/courses.zh-Hans.md**](resources/courses.zh-Hans.md) — 10 门 credible、会发证书的线上 AI agent 课（英文 + 中文），分 tier；并诚实标注完成证书不是学历 + +--- + +## 还有什么？ + +- 主 README：[README.zh-Hans.md](README.zh-Hans.md) +- 完整 MCP/Skill 目录：[resources/mcp-skills-catalog.zh-Hans.md](resources/mcp-skills-catalog.zh-Hans.md) +- CLI agent 比较指南：[resources/cli-agents-guide.zh-Hans.md](resources/cli-agents-guide.zh-Hans.md) +- Style guide / 贡献规范：[resources/style-guide.zh-Hans.md](resources/style-guide.zh-Hans.md)、[CONTRIBUTING.zh-Hans.md](CONTRIBUTING.zh-Hans.md) diff --git a/ROADMAP.en.md b/ROADMAP.en.md new file mode 100644 index 0000000..66f2b53 --- /dev/null +++ b/ROADMAP.en.md @@ -0,0 +1,56 @@ +# Roadmap + +> [繁體中文](./ROADMAP.md) | [简体中文](./ROADMAP.zh-Hans.md) | **English** + +This repo is a **community-maintained learning roadmap**: no release date, no promised schedule. This document makes public “where we know things are not good enough and where we want to go next”, so people who want to contribute can pick one piece to start with, without reading the whole repo first just to know what is missing. + +> Want to work on one item? Open a [Discussion](https://github.com/WenyuChiou/awesome-agentic-ai-zh/discussions) first, or send a PR directly. For long-term stage / branch maintainers, see [`CONTRIBUTORS.md`](CONTRIBUTORS.md). For beginner entry points, see the “5 easy entry points” section in [`CONTRIBUTING.md`](CONTRIBUTING.md). + +**Status legend**: 🟢 In progress / always open to contributions · 🟡 Known gap, wanted · 🔵 Idea, pending discussion · ✅ Recently completed + +--- + +## Near-Term Gaps We Want to Fill + +### 🟡 Fill Out Hands-On Exercise Coverage +`examples/` currently covers Stage 1, 3, 4, 5, 6, and 7. **Missing**: Stage 2 (Prompt Design), Stage 8 (Agent Interfaces), plus Stage 0 (Foundation Concepts) and Stage 7.5 (Advanced Agentic Concepts), two existing stages that also have no corresponding hands-on examples. Each example should run in under 30 minutes and include `how to run` commands. + +### 🟡 Deepen the audience branch Files +5 audience branch files by length (zh-TW canonical, 2026-05 snapshot): for-knowledge-worker (143 lines, shortest) < for-developer (166) < for-everyday-users (179) < for-researcher (208) < for-teacher (224). **The shortest files, `for-knowledge-worker.md` / `for-developer.md`, need scenario coverage the most**. `for-teacher.md` is actually the longest, but `CONTRIBUTORS.md` still marks it as “especially open to self-nominations”: what is truly thin is the **academic citation depth for teacher scenarios** (currently only Chen 2020 / Mittal 2024), and more 3-tier teacher AI application scenarios + matching citations are welcome. + +### 🟡 Stage 2 / Stage 3 2026 freshness Touch-Up +A few small 2026 wording / model-reference updates have not been synced to mirrors yet (about 5 lines of diff × 2 locales). + +--- + +## In Progress / Always Open to Contributions + +- 🟢 **Outdated entry reports** — Run `python scripts/refresh-stars.py` to find repos with large star-count gaps, then open an issue or PR to annotate / remove them. +- 🟢 **Broken link fixes** — Monthly CI scans for link-rot, but direct PRs for anything found in real time are fastest. +- 🟢 **Complete `how to run` sections** — Many entries are missing installation / execution commands. If you have run one, add them. +- 🟢 **Mirror translation smoothing** — Compare `.en.md` / `.zh-Hans.md` against zh-TW, and fix one sentence that reads awkwardly. +- 🟢 **Long-term stage / branch maintainers** — Claim one stage or branch and review it when you have time. The slot table is in `CONTRIBUTORS.md`. + +--- + +## Infrastructure (maintainer in progress) + +- ✅ **Community health files** — `CODE_OF_CONDUCT.md` / `SECURITY.md` / `CITATION.cff` / issue-template routing (same batch as this roadmap). +- 🟢 **Learner progress layer** — `PROGRESS.md` self-check template + “Self check / Exit check” at the end of each stage (planned). +- 🔵 **Browsable docs site** — Render stages/tracks/branches into a website with navigation + search + language switching (GitHub Pages, under evaluation). +- 🟢 **Trilingual mirror parity** — `mirror-sync-reminder` + `check-mirror-sync.py` already guard PRs, and legacy drift is being cleaned continuously. +- 🟢 **Quality gates** — link-rot / star-drift / banned-word / anchor / zh-Hans localization CI is online and maintained. + +--- + +## Idea Box (pending discussion, not committed yet) + +- 🔵 **More audience branch files**: There are currently 5 (researcher / developer / teacher / knowledge-worker / everyday-users). Whether to split further (for example PM / designer / legal) depends on community needs. +- 🔵 **A third track?**: Today, Track A = CLI Power User (A1–A3 under `tracks/cli/`), and Track B = stages learning path (`stages/` directory Stage 0–8, **not** a standalone directory under `tracks/`). Whether to add a third track (for example “no-code agent only”) is pending discussion. +- 🔵 **Video / interactive supplements**: Whether a text-only learning roadmap should include minimal video walkthroughs depends on cost and maintenance load. + +To suggest an idea, open a [Discussion](https://github.com/WenyuChiou/awesome-agentic-ai-zh/discussions), not an issue (issues are for bugs / outdated entries / new projects). + +--- + +> This roadmap is not a contract. It reflects the direction “now”, and will change as the community contributes. The most reliable source for “what should happen next” is always open issues + Discussions. diff --git a/ROADMAP.md b/ROADMAP.md new file mode 100644 index 0000000..8054d29 --- /dev/null +++ b/ROADMAP.md @@ -0,0 +1,56 @@ +# 路線圖 / Roadmap + +> **繁體中文** | [简体中文](./ROADMAP.zh-Hans.md) | [English](./ROADMAP.en.md) + +這份 repo 是**社群維護的學習路線圖**——沒有發行日期、沒有承諾的時程。這份文件公開「我們知道哪裡還不夠好、接下來想往哪走」,讓想貢獻的人能挑一塊上手,而不用先讀完整個 repo 才知道缺什麼。 + +> 想動其中一項?開個 [Discussion](https://github.com/WenyuChiou/awesome-agentic-ai-zh/discussions) 講一聲,或直接 PR。擔任 stage / branch 長期維護者請看 [`CONTRIBUTORS.md`](CONTRIBUTORS.md)。新手切入點看 [`CONTRIBUTING.md`](CONTRIBUTING.md) 的「好上手的 5 個切入點」。 + +**狀態圖例**:🟢 進行中 / 隨時可貢獻 · 🟡 已知缺口、想做 · 🔵 想法、待討論 · ✅ 近期完成 + +--- + +## 近期想補的缺口 + +### 🟡 動手練習覆蓋補齊 +`examples/` 目前涵蓋 Stage 1、3、4、5、6、7。**缺**:Stage 2(Prompt 設計)、Stage 8(Agent Interfaces),以及 Stage 0(基礎概念)、Stage 7.5(進階 Agentic 概念)這兩個現有 stage 也都沒有對應的 hands-on 範例。每個範例要能在 30 分鐘內跑完、附 `怎麼跑` 指令。 + +### 🟡 audience branch 深化 +5 條 audience branch 篇幅(zh-TW canonical,2026-05 snapshot):for-knowledge-worker(143 行,最短)< for-developer(166)< for-everyday-users(179)< for-researcher(208)< for-teacher(224)。**篇幅最短的 `for-knowledge-worker.md` / `for-developer.md` 最需要補情境**。`for-teacher.md` 篇幅其實最長,但 `CONTRIBUTORS.md` 仍把它標「特別歡迎自薦」——它真正薄的是**教師情境的學術引用深度**(目前只有 Chen 2020 / Mittal 2024 兩筆),歡迎補更多 3-tier 教師 AI 應用情境 + 對應引用。 + +### 🟡 Stage 2 / Stage 3 2026 freshness 小修 +幾處 2026 用語 / 模型引用的小幅更新還沒同步到鏡像(約 5 行 diff × 2 locale)。 + +--- + +## 進行中 / 隨時可貢獻 + +- 🟢 **過時 entry 回報** — 跑 `python scripts/refresh-stars.py` 找星數差距大的 repo,開 issue 或 PR 標註 / 移除。 +- 🟢 **失效連結修正** — link-rot 每月 CI 會掃,但即時發現的直接 PR 最快。 +- 🟢 **`怎麼跑` section 補完** — 很多 entry 缺安裝 / 執行指令,你跑過就補。 +- 🟢 **鏡像翻譯順稿** — 對照 `.en.md` / `.zh-Hans.md` 與 zh-TW,改一句翻得不順的。 +- 🟢 **stage / branch 長期維護者** — 認領一個 stage 或 branch,有空時 review 一輪。名額表在 `CONTRIBUTORS.md`。 + +--- + +## 基礎建設(maintainer 進行中) + +- ✅ **社群健康檔** — `CODE_OF_CONDUCT.md` / `SECURITY.md` / `CITATION.cff` / issue-template 導流(本路線圖同批)。 +- 🟢 **學習者進度層** — `PROGRESS.md` 自我打勾範本 + 每個 stage 結尾「自我檢核 / Exit check」(規劃中)。 +- 🔵 **可瀏覽文件站** — 把 stages/tracks/branches 渲染成有導覽 + 搜尋 + 語言切換的網站(GitHub Pages,評估中)。 +- 🟢 **三語鏡像 parity** — `mirror-sync-reminder` + `check-mirror-sync.py` 已在 PR 時把關,持續清 legacy drift。 +- 🟢 **品質 gate** — link-rot / star-drift / banned-word / anchor / zh-Hans 在地化 CI 已上線並維護中。 + +--- + +## 想法箱(待討論,還沒承諾) + +- 🔵 **更多 audience branch**:目前 5 條(researcher / developer / teacher / knowledge-worker / everyday-users),是否要再分(例如 PM / 設計師 / 法務)看社群需求。 +- 🔵 **第三條軌道?**:目前 Track A = CLI Power User(`tracks/cli/` 的 A1–A3)、Track B = stages 學習路線(`stages/` 目錄 Stage 0–8,**不是** `tracks/` 下的獨立目錄)。是否要有第三條軌道(例如「只用 no-code agent」)待討論。 +- 🔵 **影音 / 互動補充**:純文字學習路線是否要配最小影音 walkthrough,成本與維護負擔待評估。 + +要提想法請開 [Discussion](https://github.com/WenyuChiou/awesome-agentic-ai-zh/discussions),不要開 issue(issue 留給 bug / 過時 entry / 新增 project)。 + +--- + +> 這份路線圖不是契約。它反映「現在」的方向,會隨社群投入而變。最可靠的「接下來要做什麼」永遠是 open issues + Discussions。 diff --git a/ROADMAP.zh-Hans.md b/ROADMAP.zh-Hans.md new file mode 100644 index 0000000..b41dce2 --- /dev/null +++ b/ROADMAP.zh-Hans.md @@ -0,0 +1,56 @@ +# 路线图 / Roadmap + +> [繁體中文](./ROADMAP.md) | **简体中文** | [English](./ROADMAP.en.md) + +这份 repo 是**社区维护的学习路线图**——没有发布日期、没有承诺的时间表。这份文件公开“我们知道哪里还不够好、接下来想往哪里走”，让想贡献的人能挑一块上手，而不用先读完整个 repo 才知道缺什么。 + +> 想动其中一项？开个 [Discussion](https://github.com/WenyuChiou/awesome-agentic-ai-zh/discussions) 说一声，或直接 PR。担任 stage / branch 长期维护者请看 [`CONTRIBUTORS.md`](CONTRIBUTORS.md)。新手切入点看 [`CONTRIBUTING.md`](CONTRIBUTING.md) 的“好上手的 5 个切入点”。 + +**状态图例**：🟢 进行中 / 随时可贡献 · 🟡 已知缺口、想做 · 🔵 想法、待讨论 · ✅ 近期完成 + +--- + +## 近期想补的缺口 + +### 🟡 动手练习覆盖补齐 +`examples/` 目前覆盖 Stage 1、3、4、5、6、7。**缺**：Stage 2（Prompt 设计）、Stage 8（Agent Interfaces），以及 Stage 0（基础概念）、Stage 7.5（进阶 Agentic 概念）这两个现有 stage 也都没有对应的 hands-on 示例。每个示例要能在 30 分钟内跑完，并附 `怎么跑` 命令。 + +### 🟡 audience branch 深化 +5 条 audience branch 篇幅（zh-TW canonical, 2026-05 snapshot）：for-knowledge-worker（143 行，最短）< for-developer（166）< for-everyday-users（179）< for-researcher（208）< for-teacher（224）。**篇幅最短的 `for-knowledge-worker.md` / `for-developer.md` 最需要补情境**。`for-teacher.md` 篇幅其实最长，但 `CONTRIBUTORS.md` 仍把它标为“特别欢迎自荐”——它真正薄的是**教师情境的学术引用深度**（目前只有 Chen 2020 / Mittal 2024 两笔），欢迎补更多 3-tier 教师 AI 应用情境 + 对应引用。 + +### 🟡 Stage 2 / Stage 3 2026 freshness 小修 +几处 2026 用语 / 模型引用的小幅更新还没同步到镜像（约 5 行 diff × 2 locale）。 + +--- + +## 进行中 / 随时可贡献 + +- 🟢 **过时 entry 反馈** — 跑 `python scripts/refresh-stars.py` 找星数差距大的 repo，开 issue 或 PR 标注 / 移除。 +- 🟢 **失效链接修正** — link-rot 每月 CI 会扫，但即时发现的直接 PR 最快。 +- 🟢 **`怎么跑` section 补完** — 很多 entry 缺安装 / 执行命令，你跑过就补。 +- 🟢 **镜像翻译顺稿** — 对照 `.en.md` / `.zh-Hans.md` 与 zh-TW，改一句翻得不顺的。 +- 🟢 **stage / branch 长期维护者** — 认领一个 stage 或 branch，有空时 review 一轮。名额表在 `CONTRIBUTORS.md`。 + +--- + +## 基础建设（maintainer 进行中） + +- ✅ **社区健康文件** — `CODE_OF_CONDUCT.md` / `SECURITY.md` / `CITATION.cff` / issue-template 导流（本路线图同批）。 +- 🟢 **学习者进度层** — `PROGRESS.md` 自我打勾模板 + 每个 stage 结尾“自我检查 / Exit check”（规划中）。 +- 🔵 **可浏览文件站** — 把 stages/tracks/branches 渲染成有导航 + 搜索 + 语言切换的网站（GitHub Pages，评估中）。 +- 🟢 **三语镜像 parity** — `mirror-sync-reminder` + `check-mirror-sync.py` 已在 PR 时把关，持续清 legacy drift。 +- 🟢 **质量 gate** — link-rot / star-drift / banned-word / anchor / zh-Hans 本地化 CI 已上线并维护中。 + +--- + +## 想法箱（待讨论，还没承诺） + +- 🔵 **更多 audience branch**：目前 5 条（researcher / developer / teacher / knowledge-worker / everyday-users），是否要再分（例如 PM / 设计师 / 法务）看社区需求。 +- 🔵 **第三条轨道？**：目前 Track A = CLI Power User（`tracks/cli/` 的 A1–A3）、Track B = stages 学习路线（`stages/` 目录 Stage 0–8，**不是** `tracks/` 下的独立目录）。是否要有第三条轨道（例如“只用 no-code agent”）待讨论。 +- 🔵 **视频 / 互动补充**：纯文字学习路线是否要配最小视频 walkthrough，成本与维护负担待评估。 + +要提想法请开 [Discussion](https://github.com/WenyuChiou/awesome-agentic-ai-zh/discussions)，不要开 issue（issue 留给 bug / 过时 entry / 新增项目）。 + +--- + +> 这份路线图不是契约。它反映“现在”的方向，会随社区投入而变。最可靠的“接下来要做什么”永远是 open issues + Discussions。 diff --git a/SECURITY.en.md b/SECURITY.en.md new file mode 100644 index 0000000..fa3e98a --- /dev/null +++ b/SECURITY.en.md @@ -0,0 +1,39 @@ +# Security Policy + +> [繁體中文](./SECURITY.md) | [简体中文](./SECURITY.zh-Hans.md) | **English** + +This repo is a **curated learning roadmap** (mainly Markdown learning materials plus a small amount of example code), not a software product that gets deployed. Even so, several categories of security issues are still worth reporting, especially because this material teaches readers to **actually run agents / MCP / tool-use**. + +## Supported Scope + +This project uses **rolling maintenance** and has no versioned releases. Only the latest state of the `main` branch will be fixed. + +## What Counts as a Security Issue + +Please report the following: + +- **Hijacked / malicious third-party links** — a repo link in the catalog points to a phishing site or malicious package, or the project has been taken over maliciously +- **Supply-chain risks in example code** — code in `examples/`, `walkthroughs/`, or `scripts/` leads readers toward unsafe dependencies, patterns that leak keys, or injectable commands +- **Leaked secrets in learning materials** — real API keys / tokens / credentials were accidentally pasted into the docs +- **Command-injection-style instructional content** — a piece of instruction leads readers to run something dangerous on their own machines (for example, executing an agent on untrusted input without sandboxing) + +**Not in scope** for this repo's security policy: vulnerabilities in third-party projects **themselves**. Please report those directly to the upstream project maintainers, and you may also open an issue to remind us to label or remove that entry. + +## How to Report + +**Please do not post exploitable details in a public issue.** + +1. **Preferred**: Use GitHub's **private vulnerability reporting** (the repo's **Security** tab → **Report a vulnerability**). Only maintainers can see it. +2. If that feature is unavailable: please DM **@WenyuChiou** on GitHub with the impact and the affected files / links, and we will continue privately. **Do not open a public issue** — even without exploit details, a public title and labels would expose that there is an unpatched issue. + +When reporting, please include as much as possible: affected files / links, a description of the issue, the possible impact, and (if any) suggested fixes. + +## Response Timeline + +This is a community-maintained educational project with no SLA, but we will handle reports as quickly as possible. In general: + +- **Acknowledgement**: within 7 days +- **Initial assessment**: within 14 days +- **Fix or annotation**: depends on severity; malicious links will be prioritized + +Thank you for helping make this material safer for all learners. diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 0000000..f88e860 --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,39 @@ +# 安全政策 / Security Policy + +> **繁體中文** | [简体中文](./SECURITY.zh-Hans.md) | [English](./SECURITY.en.md) + +這個 repo 是一份**精選學習路線圖**(主要是 Markdown 教材 + 少量範例程式碼),不是一個會部署的軟體產品。即便如此,有幾類安全問題仍然值得回報——尤其因為本教材教讀者**實際執行 agent / MCP / tool-use**。 + +## 支援範圍 + +本專案採**滾動式維護**,沒有版本化 release。只有 `main` 分支的最新狀態會被修正。 + +## 哪些算安全問題 + +請回報以下情況: + +- **被劫持 / 惡意的第三方連結** —— catalog 裡某個 repo 連結指向了釣魚站、惡意套件,或專案已被惡意接管 +- **範例程式碼的供應鏈風險** —— `examples/`、`walkthroughs/`、`scripts/` 裡的程式碼會引導不安全的相依套件、洩漏金鑰的寫法,或可被注入的指令 +- **教材中外洩的密鑰** —— 文件裡不小心貼出真實 API key / token / 憑證 +- **指令注入式的教學內容** —— 某段教學會誘導讀者在自己機器上跑出危險後果(例如未沙箱化地對 untrusted input 執行 agent) + +**不屬於**本 repo 安全範疇的:第三方專案**本身**的漏洞——那些請直接回報給上游專案的維護者,並可同時開 issue 提醒我們把該 entry 標註或移除。 + +## 如何回報 + +**請不要在公開 issue 裡貼出可被利用的細節。** + +1. **首選**:用 GitHub 的 **私人漏洞回報**(repo 的 **Security** 分頁 → **Report a vulnerability**)。只有維護者看得到。 +2. 若該功能不可用:請透過 GitHub 私訊 **@WenyuChiou**,說明問題影響與受影響的檔案 / 連結,我們會私下接續。**請勿開公開 issue**——即使不貼 exploit 細節,公開的標題與標籤也會對外暴露「有一個未修補的問題」。 + +回報時請盡量附上:受影響的檔案 / 連結、問題說明、可能的影響,以及(若有)修正建議。 + +## 處理時程 + +這是一個社群維護的教育專案,沒有 SLA,但我們會儘速處理。一般而言: + +- **確認收到**:7 天內 +- **初步評估**:14 天內 +- **修正或標註**:視嚴重程度而定;惡意連結會優先處理 + +謝謝你幫忙讓這份教材對所有學習者都更安全。 diff --git a/SECURITY.zh-Hans.md b/SECURITY.zh-Hans.md new file mode 100644 index 0000000..f72b109 --- /dev/null +++ b/SECURITY.zh-Hans.md @@ -0,0 +1,39 @@ +# 安全政策 / Security Policy + +> [繁體中文](./SECURITY.md) | **简体中文** | [English](./SECURITY.en.md) + +这个 repo 是一份**精选学习路线图**（主要是 Markdown 教材 + 少量示例代码），不是一个会部署的软件产品。即便如此，有几类安全问题仍然值得报告，尤其因为本教材教读者**实际运行 agent / MCP / tool-use**。 + +## 支持范围 + +本项目采用**滚动式维护**，没有版本化 release。只有 `main` 分支的最新状态会被修正。 + +## 哪些算安全问题 + +请报告以下情况： + +- **被劫持 / 恶意的第三方链接** —— catalog 里某个 repo 链接指向了钓鱼站、恶意包，或项目已被恶意接管 +- **示例代码的供应链风险** —— `examples/`、`walkthroughs/`、`scripts/` 里的代码会引导不安全的依赖包、泄露密钥的写法，或可被注入的命令 +- **教材中外泄的密钥** —— 文件里不小心贴出真实 API key / token / 凭证 +- **命令注入式的教学内容** —— 某段教学会诱导读者在自己机器上跑出危险后果（例如未沙箱化地对 untrusted input 运行 agent） + +**不属于**本 repo 安全范围的：第三方项目**本身**的漏洞。那些请直接报告给上游项目的维护者，并可同时开 issue 提醒我们把该 entry 标注或移除。 + +## 如何报告 + +**请不要在公开 issue 里贴出可被利用的细节。** + +1. **首选**：用 GitHub 的 **私人漏洞报告**（repo 的 **Security** 分页 → **Report a vulnerability**）。只有维护者看得到。 +2. 若该功能不可用：请通过 GitHub 私信 **@WenyuChiou**，说明问题影响与受影响的文件 / 链接，我们会私下接续。**请勿开公开 issue**——即使不贴 exploit 细节，公开的标题与标签也会对外暴露“有一个未修补的问题”。 + +报告时请尽量附上：受影响的文件 / 链接、问题说明、可能的影响，以及（若有）修正建议。 + +## 处理时程 + +这是一个社区维护的教育项目，没有 SLA，但我们会尽快处理。一般而言： + +- **确认收到**：7 天内 +- **初步评估**：14 天内 +- **修正或标注**：视严重程度而定；恶意链接会优先处理 + +谢谢你帮忙让这份教材对所有学习者都更安全。 diff --git a/book.toml b/book.toml new file mode 100644 index 0000000..8e33053 --- /dev/null +++ b/book.toml @@ -0,0 +1,39 @@ +[book] +title = "awesome-agentic-ai-zh" +description = "AI Agent 學習地圖 — 從第一行 LLM API 到自己打造多 agent 系統" +authors = ["Wenyu Chiou"] +language = "zh-TW" +multilingual = false +src = "book/src" + +[output.html] +default-theme = "light" +preferred-dark-theme = "navy" +git-repository-url = "https://github.com/WenyuChiou/awesome-agentic-ai-zh" +edit-url-template = "https://github.com/WenyuChiou/awesome-agentic-ai-zh/edit/main/{path}" +site-url = "/awesome-agentic-ai-zh/" + +[output.html.search] +enable = true +limit-results = 30 +use-boolean-and = true +boost-title = 2 +boost-hierarchy = 2 +boost-paragraph = 1 +expand = true + +[output.html.fold] +enable = true +level = 1 + +# Note: mdbook-mermaid preprocessor was removed. The repo no longer +# uses ```mermaid``` code blocks (replaced with PNG images + ASCII +# diagrams) and the v0.17.0 preprocessor was failing with "Unable to +# parse the input" on stale runs. To re-enable: install mdbook-mermaid, +# run `mdbook-mermaid install .`, and add back: +# additional-js = ["mermaid.min.js", "mermaid-init.js"] +# [preprocessor.mermaid] +# command = "mdbook-mermaid" + +[build] +build-dir = "book/dist" diff --git a/branches/DESIGN.md b/branches/DESIGN.md new file mode 100644 index 0000000..fc22a62 --- /dev/null +++ b/branches/DESIGN.md @@ -0,0 +1,157 @@ +# Branch 設計筆記 + +> 這份是給 maintainer 看的內部文件，**不是讀者面向的內容**。 +> +> 5 個 branch 怎麼分、entry 怎麼判斷該放哪、什麼時候要不要新開 branch——這些設計決定的記錄。新 maintainer 接手時看這份就懂為什麼是這樣分。 + +--- + +## 為什麼是 5 個 branch（不是 3 個或 10 個） + +### Branch 跟 Track 的關係 + +5 條 branch 設計成 **兩條軌道走完都接得上**： +- Track A 完成 A3 → 從 branches 選一條繼續 +- Track B 完成 Stage 7 → 從 branches 選一條繼續 +- Branch entry 的 curation 標準**不依軌道區分**——同一個工具不論是 Track A 用法（用現成 CLI）還是 Track B 用法（自己接 SDK），都放在對應 audience 的 branch 內 + +**例外：for-everyday-users branch 可以直接進入**——不一定要走完軌道。這條 branch 的目標讀者是「Claude.ai / ChatGPT 重度使用者，想用 AI 但不一定想 build」，他們可能根本不需要碰 Track A 或 B；branch 內也明確標示「不一定要走完整條主幹」。其他 4 條 branch（researcher / developer / teacher / knowledge-worker）預設讀者已走完一條軌道。 + +Branch maintainer 應該意識到：**進來看 branch 的讀者背景可能差很多**——剛走完 Track A 的人對 framework 內部不熟、剛走完 Track B 的人對 CLI 操作可能不熟、直接進 everyday-users 的人對 Stage 0-2 都可能跳過。Branch entry 的 prose 要盡量讓這幾種讀者都看得懂。 + +### 太少（≤3）的問題 +3 個會強行把多個 audience 塞同一條，譬如「professional」涵蓋 dev + researcher + knowledge worker——但他們的 pain point 完全不同。研究者要 grounded citation，開發者要 git-native，知識工作者要 email triage——硬擠成一條 branch 會讓 entry 互相 dilute。 + +### 太多（≥7）的問題 +audience 切太細會： +- 每個 branch 都很薄（沒幾個 entry），讀者覺得不被照顧 +- 邊界開始模糊（資料科學家 vs 機器學習工程師？產品經理 vs 顧問？） +- maintain 成本變高（要看的 branch 變多） + +### 5 是 sweet spot +4 個職業（research / dev / teach / knowledge work）覆蓋大部分專業場景；第 5 個 everyday users 收尾「不寫 code 的純使用者」這條沒被任何職業 branch 照顧到的 audience。 + +**判準**：每個 branch 都應該對應到一個**讀者一秒就能自我認領**的身份標籤。如果 maintainer 自己都要想 30 秒才能決定一個 entry 該放哪，就是 branch 切得不夠清楚。 + +--- + +## 5 個 audience 的核心 pain point + +每個 branch 都是回應一個具體 pain，不是涵蓋一整個職業生涯： + +| Branch | 核心 pain | branch 主要回應 | +|---|---|---| +| 🔬 研究人員 | 「我要 review 100 篇 paper、寫 lit review，但時間不夠」 | 文獻 RAG、Outline-driven 寫作、Zotero 整合 | +| 💻 開發者 | 「我有 10 個 PR 要 review、每個 codebase 都不同 convention」 | git-native CLI、IDE coding agent、code review skill | +| 🎓 教師 | 「備課要花 4 小時、我手上的 prompt 都太通用」 | 學科特化 prompt、課程素材、評量自動化 | +| 📊 知識工作者 | 「每天信箱 100 封、會議紀錄要轉成 action items、隔天還要寫 weekly report」 | Email triage、會議紀錄、自動化 workflow | +| 👥 日常使用者 | 「我不寫 code，但想用 AI 改善生活，不知道從哪開始」 | Tier 0 入門到 Tier 2 進階 CLI 的階梯式路線 | + +每個 branch 的 entry 選入都應該回到「能不能解決核心 pain」這個問題。如果不能，就是 entry 該放別的地方。 + +--- + +## Branch 之間的邊界 + +判斷一個 entry 該放哪個 branch，按這 3 條判準依序考慮： + +### 1. 主要 user persona +看上面 pain table——這個 entry 解決的是哪一個 audience 的 pain？通常很清楚。 + +### 2. 預期動手程度 +不寫 code 的工具 → 偏 everyday-users / knowledge-worker。CLI / SDK 工具 → 偏 developer。介於中間（譬如 ChatPaper 是命令列但對研究者友善）→ 看 #1 主要 persona。 + +### 3. 應用場景 +同一個工具在不同場景下歸類不同。例如： +- **Ollama**：給 everyday-users 是「隱私場景跑本地 LLM」（Tier 3），給 developer 是「開發 agent 的本地測試 backend」——但這份 catalog 把它放在 **Stage 1**（基礎設施層級），各 branch 從那裡引用。 +- **f/awesome-chatgpt-prompts**：放 for-teacher（給教師當教材參考）、也放 for-everyday-users（不寫 code 也能用的 prompt 庫）。 + +### 灰色地帶處理（同一 repo 出現在多 branch） + +**規則**：同一 repo 可以在多 branch 出現，但每處要有不同的 **framing**（適合誰、教什麼）。**推薦星等預設一致**——同一個工具的客觀價值不會因 audience 改變；除非有明確的 audience-specific 理由（譬如「進階度差太多」），且寫進 Notes 解釋。詳見 [`resources/style-guide.md`](../resources/style-guide.md) 2。 + +**範例**： +- `obra/superpowers` 出現在 Stage 5、for-developer、for-knowledge-worker、for-teacher + - Stage 5：作為 SKILL.md collection 範例 + - for-developer：作為 TDD / debug skill 來源 + - for-knowledge-worker：作為腦力激盪 / 規劃 skill + - for-teacher：作為通用寫作 skill + - **4 處都是 ⭐⭐⭐⭐**（這是規則的正例：framing 不同、評等一致） + +**反例（不該這樣做）**： +- `kaixindelele/ChatPaper` 只放 for-researcher，不放 for-everyday-users。原因：它是研究者專用流程（總結 / 翻譯 / 審稿回覆），everyday user 用不到也不該被推。 + +--- + +## 兩種 entry 結構：tier vs flat + +### Tier 結構（目前只用在 for-everyday-users） +![Branch tier 漸進結構](../resources/diagrams/branch-tier-progression.png) +**用 tier 的條件**：audience 內部「動手程度差很多」。Everyday users 從「打開 Claude.ai」到「跑 Ollama 本地 LLM」差距太大，不分 tier 會混亂。 + +### Flat 結構（其他 4 個 branch 都用這個） +單一個 list，照子主題分類（Coding Agents / Code Review / Workflow Tools 等）。 +**用 flat 的條件**：audience 內部相對同質——研究者多半願意動手用 CLI、開發者一定會寫 code，沒必要分 tier。 + +### 什麼時候從 flat 升級成 tier +觀察 issue / PR 是否反覆出現「**這個 entry 太進階 / 太簡單**」抱怨。若 ≥3 個讀者反映該 branch 內 entry 落差太大，考慮分 tier。 + +--- + +## 自我引用排除原則 + +`WenyuChiou/*` repo 一律不收（已從 catalog 移除 32 instances）。 + +### 例外（什麼條件下作者自己的 repo 才能加回去） +1. 該 repo 在某個 stage / branch 是**唯一夠用的選項**（沒其他社群替代） +2. 至少 2 個 stage maintainer 簽字同意 +3. 在 entry notes 明確標註「作者維護的 repo，含利益關係」 +4. 加一個「替代品」連結，方便讀者比較 + +**目前 0 個 entry 滿足這 4 條**——保持 0 個是健康狀態。 + +--- + +## 加新 branch 的決策樹 + +![加新 branch 決策樹](../resources/diagrams/add-branch-decision-flow.png) + +### 範例：要不要加 `for-data-scientists`？ +- pain 已被 for-researcher 涵蓋（文獻 RAG、實驗設計） +- audience scale 大，但跟 researcher 重疊高 +- 結論：不加 branch，但可以在 for-researcher 加「資料科學工具」 sub-section + +### 範例：要不要加 `for-product-managers`？ +- pain 已被 for-knowledge-worker 涵蓋（會議紀錄、report、跨 team 溝通） +- audience scale 大但邊界跟 knowledge-worker 模糊 +- 結論：不加 branch，在 for-knowledge-worker 加「產品經理」use case + +--- + +## 5 條 branch 的 maintenance 想法（不是 SLA） + +社群 repo 的維護是「能做就做」、不是排程義務。下面是大致方向： + +### Review 頻率 +- 沒有強制節奏。CI 已設定每月自動跑 link rot + star drift（被動的）。 +- 有空想動的人 → 跑 `python scripts/refresh-stars.py` 看哪些 entry 過時、`python scripts/check-links.py --fast` 看哪些連結壞掉。 + +### Entry 加入 / 移除節奏 +- 加入：看到值得收的就 PR。不必為了「衝量」主動找。 +- 移除：archived / 長期沒 commit / license 變奇怪 → 看到再標 ⚠️ 或 PR 拿掉。 + +### 跟 main path stages 的同步 +- Stage 改了某個 entry，branch 引用該 entry 的地方順手更新就好——沒做也不會壞。 + +### Maintainer 自薦 / 退場機制 +- 想擔任 maintainer 開 issue 自薦就好，不用承諾什麼具體期間。「我 review 一次」也算貢獻。 +- 退場：不需要 ceremony——維持沉默 2 個月，自動視為退場，新人接手 +- 詳見 [`CONTRIBUTORS.md`](../CONTRIBUTORS.md) + +--- + +## 不在這份的內容 + +- **個別 branch 的 entry 詳細**：見 `for-X.md` 本身 +- **stage 設計理由**：見 [`../stages/DESIGN.md`](../stages/DESIGN.md) +- **entry schema / 用詞規範**：見 [`../resources/style-guide.md`](../resources/style-guide.md) diff --git a/branches/for-developer.en.md b/branches/for-developer.en.md new file mode 100644 index 0000000..dd78ce1 --- /dev/null +++ b/branches/for-developer.en.md @@ -0,0 +1,166 @@ +# Extension Path: For Developers + +> [繁體中文](./for-developer.md) | [简体中文](./for-developer.zh-Hans.md) | **English** + +> 🚀 **First time installing Claude Code or writing `CLAUDE.md` / `SKILL.md`?** The quick setup guide is [`resources/setup-guide.en.md` D-E](../resources/setup-guide.en.md). Skip it if you already know this. + +> [← Back to main path README](../README.en.md) · Continue here after **Track A's A3** or **Track B's Stage 7**. Apply agentic AI to coding workflows. + +## Use Cases (Developer Scenarios × How AI Helps) + +The table below splits a developer's day into 7 common scenarios. Each has a different pain point, and each calls for a different level of AI tooling: + +| Scenario | Pain point | How AI helps | Recommended tools (light → heavy) | +|---|---|---|---| +| **AI pair programming** | You forget syntax mid-flow or cannot recall a method name | Autocomplete + rewrite + explanation | Cursor / Copilot → Claude Code | +| **Multi-file refactoring** | Changing one class risks missed references; cross-file rename is error-prone | Batch refactors while keeping style consistent across many files | Cursor → Claude Code → codex-delegate | +| **Code review (your own PR)** | Reviewing your own diff makes it easy to miss problems | Find bugs / smells and check edge cases | Claude Code / Cline → Continue (CI) | +| **Writing tests** | TDD cases are easy to miss; coverage falls short | Generate pytest cases from signatures / specs | Claude Code + Aider | +| **Debugging** | Logs are thin; stack traces are hard to interpret | Explain traces, generate hypotheses, run minimal repros | Claude Code | +| **Docs** | Docstrings / READMEs lag behind refactors | Generate docs from code and update docs alongside PRs | Claude Code | +| **CI / team automation** | Manual review is repetitive; style varies across people | Run automated review / lint in GitHub Actions | Claude Code Action + Continue | + +> 💡 **Individual vs team**: the first 6 rows are personal daily workflows. The final row (CI) is team governance. For teams under 5 people, AI automation in CI often has low ROI; you can defer it. + +## Curated Projects + +> **CLI agent comparison**: 7 major CLI agents (Claude Code / Codex / OpenCode / Gemini CLI / goose / Aider / Hermes Agent) compared side-by-side in [`resources/cli-agents-guide.en.md`](../resources/cli-agents-guide.en.md). New to CLI agents and want step-by-step onboarding → [`tracks/cli/A1-cli-intro.en.md`](../tracks/cli/A1-cli-intro.en.md) (Track A first stop). +> +> **MCP catalog**: Looking for integrations to wire CLI into daily tools (GitHub, Linear, Atlassian, Postgres, Playwright, Figma…) → [`resources/mcp-skills-catalog.en.md`](../resources/mcp-skills-catalog.en.md) (65+ entries by category). +> +> This page only lists tool entry points directly relevant to developer workflows. + +### Coding Agents + +#### [Cursor](https://www.cursor.com/) ⭐⭐⭐⭐⭐ +Editor-integrated AI pair-programming tool. Widely adopted in AI editor tools and a useful baseline for comparing other IDE agents. + +#### [Aider-AI/aider](https://github.com/Aider-AI/aider) ⭐⭐⭐⭐⭐ +★ 44k+ · Apache-2.0 — git-aware CLI pair-programmer. Edits files in your repo directly and writes commits for you. **The open-source reference for "git-native AI editing."** Model-agnostic. + +#### [anthropics/claude-code](https://github.com/anthropics/claude-code) ⭐⭐⭐⭐⭐ +★ 120k+ — Anthropic's official agentic coding assistant. Skills + plugins ecosystem. + +#### [cline/cline](https://github.com/cline/cline) ⭐⭐⭐⭐⭐ +★ 61k+ · Apache-2.0 — VS Code extension, autonomous in-IDE agent: tool use, browser, step-by-step approval. **The first pick for VS Code users wanting IDE-native agentic dev.** + +#### [continuedev/continue](https://github.com/continuedev/continue) ⭐⭐⭐⭐ +★ 33k+ · Apache-2.0 — source-controlled AI checks, enforceable in CI. Represents the **team / governance** angle on coding agents. + +#### [OpenHands (formerly OpenDevin)](https://github.com/All-Hands-AI/OpenHands) ⭐⭐⭐⭐ +★ 72k+ · MIT — open-source autonomous software development agent. More aggressive design than Aider / Claude Code — agent runs in its own sandbox and commits autonomously. Best for "throw a whole issue at it" scenarios. + +#### [block/goose](https://github.com/block/goose) ⭐⭐⭐⭐ +★ 43k+ · Apache-2.0 — Open-source, extensible AI agent that goes beyond code suggestions — install / execute / edit / test, with any LLM. Supports multiple LLM providers and MCP, ships as desktop app, CLI, and API. (Repo now resolves to `aaif-goose/goose`.) + +#### [RooCodeInc/Roo-Code](https://github.com/RooCodeInc/Roo-Code) ⭐⭐⭐⭐ +★ 23k+ · Apache-2.0 — VS Code coding agent with a "**team of specialized modes**" model. Different from Cline's single-agent flow. + +### Code Review + +#### [obra/superpowers](https://github.com/obra/superpowers) ⭐⭐⭐⭐ +20+ battle-tested skills including TDD patterns, debugging, collaboration patterns. Good source for code-review skill design. + +### Recommended Tools + +- [**yamadashy/repomix**](https://github.com/yamadashy/repomix) ⭐⭐⭐⭐⭐ ★ 26k+ — **Typical developer use case: package the whole codebase for a reviewer / refactor agent**. Outputs a single AI-friendly file (XML / Markdown / JSON) for Claude Code / Codex code review / refactoring. See the official README for technical details such as MCP server mode, tree-sitter compression, and secretlint filtering. **A must-have, daily-driver-grade tool for Track A.** + +## Workflows to Master (by frequency) + +| Frequency | Workflow | Steps (≤3) | Recommended tools | Best for | +|---|---|---|---|---| +| **Daily** | AI pair programming | (1) Open a branch
(2) Give the task to Claude Code and **ask for a plan first** (no code yet)
(3) Review plan → approve → code → review your own diff | Claude Code / Cursor / Cline | All developers | +| **Daily** | Git-native AI editing | (1) `aider`
(2) Ask in natural language
(3) review + commit / `/undo` | Aider | People who want a clean git flow | +| **Per PR** | Automated code review | (1) `.github/workflows/claude-review.yml`
(2) Capture git diff → run prompt → post back to PR
(3) human + AI review | Claude Code Action + Continue | Teams | +| **Per feature** | Test generation | (1) Provide function signature + docstring
(2) Ask AI for pytest cases, including edge cases
(3) Run coverage + intentionally break a bug to verify tests catch it | Claude Code / Aider | Test-writing phase | +| **Occasional** | Multi-file batch edits | (1) Claude writes a plan
(2) codex-delegate handles mechanical refactors
(3) Claude reviews the diff | Claude + codex-delegate | Refactors across 30+ files | + +> 💡 **Starter habit**: run "daily AI pairing" and "test generation" for a month first, then add automated PR review. + +### 3 Concrete Workflow Recipes + +**1. AI Pair Programming (daily cadence)** +1. Start a feature → `git checkout -b feature/xxx` +2. Hand the task to Claude Code / Cursor — **make it write a plan first** (don't dive into code) +3. Review the plan, course-correct → only then approve coding +4. After it's done: run tests + lint → review the diff yourself (**don't blind-accept**) +5. Write the commit message yourself, or have AI draft and edit before committing + +**2. Aider Git-Native Flow (closest "pair with AI" experience)** +```bash +# Inside the repo +aider --model anthropic/claude-sonnet-5 + +# Natural-language ask +> Add a timezone parameter to parse_date in utils.py, default UTC + +# Aider edits + commits automatically. To roll back: +> /undo # undoes the last AI commit +``` + +**3. PR-time Claude code review (GitHub Action)** + +`.github/workflows/claude-review.yml`: +```yaml +on: + pull_request: +jobs: + review: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + - name: Run Claude review + env: + ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} + run: | + # Use anthropics/claude-code-action or your own script + # Get git diff, run prompt, post results back to PR +``` +Reference: official [`anthropics/claude-code-action`](https://github.com/anthropics/claude-code-action) GitHub Action. + +## Common Pitfalls (Anti-patterns) + +| ❌ Don't | ✅ Do instead | +|---|---| +| Let AI push directly to main | Always go through PR → review → merge | +| Blind-accept large refactor diffs | Break into < 50 LOC chunks, review each | +| Hand `.env` / API keys to the AI | Use your tool's exclusion mechanism — Cursor `.cursorignore` / Aider `.aiderignore` / Claude Code `permissions.deny` in `.claude/settings.json` | +| Let AI run shell freely against production code | Sandbox + permission whitelist | +| Take AI-generated tests at face value | Run coverage + intentionally break a unit to see if tests catch it | +| Discover wrong direction after many commits | **Plan-first** mode: review the plan before any coding | + +## Tier Progression + +Recommended progression: + +| Tier | Tools | Best for | Learning cost | +|---|---|---|---| +| **Tier 0** | Cursor / Copilot / Claude.ai | IDE chat, autocomplete, no custom agents | 0 (if you can use an editor) | +| **Tier 1** | Claude Code / Cline / OpenCode + `CLAUDE.md` | CLI with file-system access, human-in-the-loop | 1-2 days | +| **Tier 2** | Custom Skills + MCP server | Packaging dev workflows as shared team skills | 1 week of setup | +| **Tier 3** | Auto-running agents in CI + production observability | [Stage 7](../stages/07-multi-agent-production.en.md) territory | Several weeks, governance required | + +> **Most individual developers can stay at Tier 0-1**. **Validate ROI before going Tier 2+**: it is only worth the investment if the team is large, the workflows repeat often, and failures are hard to reverse. + +## Other Branches Also Apply + +Branches that overlap heavily with developers: + +- **Doing ML research / writing papers** → [Researcher branch](./for-researcher.en.md) +- **Wire Notion / Linear / Atlassian / Postgres / Figma into your CLI** → [`resources/mcp-skills-catalog.en.md`](../resources/mcp-skills-catalog.en.md) +- **Author your own Skill / MCP server** → [Stage 5](../stages/05-claude-code-ecosystem.en.md) + [`resources/cookbook.en.md`](../resources/cookbook.en.md) +- **Schema design details** → [`resources/schema-design-cheatsheet.en.md`](../resources/schema-design-cheatsheet.en.md) +- **CLI from zero** → [Track A](../tracks/cli/A1-cli-intro.en.md) (A1 → A2 → A3) + +## Community Note + +Contributions especially welcome: + +- IDE-specific config templates (Cursor `.cursorrules`, Claude Code `CLAUDE.md` for Python / Go / Rust, etc.) +- Language-specific Skills (Python / TypeScript / Rust / Go best-practice patterns) +- CI / pre-commit hook integration case studies +- **Multi-developer team governance** — sharing Skills across devs, permission design, cost tracking + +See [CONTRIBUTING.md](../CONTRIBUTING.md). diff --git a/branches/for-developer.md b/branches/for-developer.md new file mode 100644 index 0000000..9844ee1 --- /dev/null +++ b/branches/for-developer.md @@ -0,0 +1,166 @@ +# 開發者延伸路線（For Developers） + +> **繁體中文** | [简体中文](./for-developer.zh-Hans.md) | [English](./for-developer.en.md) + +> 🚀 **第一次裝 Claude Code / 寫 `CLAUDE.md` / `SKILL.md`？** 快速 setup 指南在 [`resources/setup-guide.md` D-E](../resources/setup-guide.md)。已經熟可以跳過。 + +> [← 回主路線 README](../README.md) · 走完 **Track A 的 A3** 或 **Track B 的 Stage 7** 後從這裡接續。把 agentic AI 應用到開發流程上。 + +## 使用情境（開發場景 × AI 怎麼幫） + +下表把開發者一天會遇到的 7 個情境拆開——每個情境有不同的痛點，AI 工具也不同： + +| 場景 | 你常遇到的痛點 | AI 能幫的部分 | 推薦工具（從輕到重） | +|---|---|---|---| +| **AI 結對程式設計** | 寫到一半忘 syntax / 想到 method 名 | 自動補完 + 改寫 + 解釋 | Cursor / Copilot → Claude Code | +| **多檔重構** | 改一個 class 怕漏改、跨檔 rename 易錯 | batch refactor、改 50 個檔保持風格一致 | Cursor → Claude Code → codex-delegate | +| **Code review（自己 PR）** | review 自己的 diff 看不出問題 | 找 bug / smell、檢查 edge case | Claude Code / cline → Continue（CI） | +| **寫 test** | TDD 一直忘加 case、coverage 不足 | 從 signature / spec 生 pytest | Claude Code + Aider | +| **Debug** | log 不夠、stack trace 看不懂 | 解 trace、生 hypothesis、跑 minimal repro | Claude Code | +| **Doc** | docstring / README 沒人寫、refactor 後過期 | 從 code 生 doc、PR 對應改 doc | Claude Code | +| **CI / 團隊自動化** | 重複手動跑 review、跨人風格不一 | GitHub Action 自動跑 review / lint | Claude Code Action + Continue | + +> 💡 **個人 vs 團隊**：表中前 6 個是個人 daily workflow；最後 1 個（CI）是團隊規範。團隊規模 < 5 人時 CI 自動化的 ROI 不高、可先不上。 + +## 精選 Projects + +> **CLI agent 比較**：7 個主流 CLI agent（Claude Code / Codex / OpenCode / Gemini CLI / goose / Aider / Hermes Agent）的並列比較見 [`resources/cli-agents-guide.md`](../resources/cli-agents-guide.md)。第一次接觸 CLI agent 想要 step-by-step 入門 → [`tracks/cli/A1-cli-intro.md`](../tracks/cli/A1-cli-intro.md)（Track A 第一站）。 +> +> **MCP catalog**：要把 CLI 接到日常工具（GitHub、Linear、Atlassian、Postgres、Playwright、Figma 等）→ [`resources/mcp-skills-catalog.md`](../resources/mcp-skills-catalog.md)（65+ 個分類整理）。 +> +> 本頁只列**跟開發者 workflow 直接相關**的工具入口。 + +### Coding Agents + +#### [Cursor](https://www.cursor.com/) ⭐⭐⭐⭐⭐ +編輯器整合的 AI 結對程式設計工具。在 AI 編輯器類工具中採用度高、可作為比較其他 IDE agent 的基準。 + +#### [Aider-AI/aider](https://github.com/Aider-AI/aider) ⭐⭐⭐⭐⭐ +★ 44k+ · Apache-2.0 — git-aware 的 CLI pair-programmer。直接編輯你 repo 中的檔案，commit 都自動寫好。**「git-native AI 編輯流程」的開源範本**。模型不限。 + +#### [anthropics/claude-code](https://github.com/anthropics/claude-code) ⭐⭐⭐⭐⭐ +★ 120k+ — Anthropic 官方的 agentic coding 助理。有 Skills + plugin 生態系。 + +#### [cline/cline](https://github.com/cline/cline) ⭐⭐⭐⭐⭐ +★ 61k+ · Apache-2.0 — VS Code extension，autonomous in-IDE agent：tool use、browser、step-by-step approval。**VS Code 使用者要 IDE-native agentic dev 的好選項**。 + +#### [continuedev/continue](https://github.com/continuedev/continue) ⭐⭐⭐⭐ +★ 33k+ · Apache-2.0 — source-controlled AI checks，可以在 CI 強制執行。代表「**團隊 / governance**」這條角度的 coding agent。 + +#### [OpenHands (前身為 OpenDevin)](https://github.com/All-Hands-AI/OpenHands) ⭐⭐⭐⭐ +★ 72k+ · MIT — open source 的自主軟體開發 agent。設計上比 Aider / Claude Code 更激進——agent 自己跑 sandbox、自己 commit，適合「整個 issue 丟給它解」場景。 + +#### [block/goose](https://github.com/block/goose) ⭐⭐⭐⭐ +★ 43k+ · Apache-2.0 — 開源、可擴充的 AI agent，超出純 code suggestion——能 install / execute / edit / test，搭配任何 LLM。同時支援多家 LLM provider 跟 MCP，提供 desktop app、CLI、API 三種介面。（repo 現指向 `aaif-goose/goose`。） + +#### [RooCodeInc/Roo-Code](https://github.com/RooCodeInc/Roo-Code) ⭐⭐⭐⭐ +★ 23k+ · Apache-2.0 — VS Code 的 coding agent，採用「**多種專業 mode**」的設計，跟 Cline 的單一 agent flow 不同。VS Code 使用者要 multi-mode 替代方案的選擇。 + +### Code Review + +#### [obra/superpowers](https://github.com/obra/superpowers) ⭐⭐⭐⭐ +20+ 個經過實戰驗證的 skill，包括 TDD 模式、debug、協作模式。設計 code-review skill 時的好參考。 + +### 推薦工具 + +- [**yamadashy/repomix**](https://github.com/yamadashy/repomix) ⭐⭐⭐⭐⭐ ★ 26k+ — **典型開發者用途：打包整個 codebase 給 reviewer / refactor agent**。輸出單一 AI-friendly 檔案（XML / Markdown / JSON），方便 Claude Code / Codex 做 code review / refactoring。技術細節（MCP server mode、tree-sitter 壓縮、secretlint 過濾）見官方 README。**Track A 很值得當 daily driver 的工具。** + +## 必練流程（按使用頻率） + +| 頻率 | 流程 | 怎麼做（≤ 3 步） | 推薦工具 | 適合誰 | +|---|---|---|---|---| +| **每天** | AI 結對寫 code | (1) 開 branch
(2) 任務丟給 Claude Code、**先 plan**（不寫 code）
(3) Review plan → approve → 寫 code → 自己 review diff | Claude Code / Cursor / Cline | 全開發者 | +| **每天** | Git-native AI 編輯 | (1) `aider`
(2) 自然語言請求
(3) review + commit / `/undo` | Aider | 想要乾淨 git 流程的人 | +| **Per PR** | 自動 code review | (1) `.github/workflows/claude-review.yml`
(2) 抓 git diff → 跑 prompt → post 回 PR
(3) human + AI 雙審 | Claude Code Action + Continue | 團隊 | +| **Per feature** | 測試生成 | (1) 給 function signature + docstring
(2) 請 AI 生 pytest case（含 edge case）
(3) 跑覆蓋率 + 故意改 bug 看 test 抓不抓得到 | Claude Code / Aider | 寫 test 階段 | +| **不定期** | 多檔批次修改 | (1) Claude 寫 plan
(2) codex-delegate 跑機械式 refactor
(3) Claude review diff | Claude + codex-delegate | refactor 30+ 檔的時候 | + +> 💡 **新手起手式**：先做「每天 AI 結對」+「測試生成」兩條一個月、習慣後再上 PR 自動 review。 + +### 3 個具體 workflow recipe + +**1. AI 結對程式設計（每日節奏）** +1. 開新 feature → `git checkout -b feature/xxx` +2. 把任務丟給 Claude Code / Cursor，**先讓它寫 plan**（不直接寫 code） +3. Review plan、修正方向 → 才 approve 寫 code +4. 寫完跑 tests + lint → 自己 review diff（**不要 blind accept**） +5. Commit message 自己寫或 prompt 生草稿後改 + +**2. Aider git-native 流程（最像「跟 AI pair」）** +```bash +# 進入 repo 後 +aider --model anthropic/claude-sonnet-5 + +# 自然語言請求 +> 幫我把 utils.py 的 parse_date 加上時區參數，預設 UTC + +# Aider 會自動編輯 + commit。若不滿意： +> /undo # 退掉最後一次 AI commit +``` + +**3. PR 上的 Claude code review（GitHub Action）** + +`.github/workflows/claude-review.yml`： +```yaml +on: + pull_request: +jobs: + review: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + - name: Run Claude review + env: + ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} + run: | + # 用 anthropics/claude-code-action 或自寫 script + # 抓 git diff、跑 prompt、結果 post 回 PR +``` +參考 [`anthropics/claude-code-action`](https://github.com/anthropics/claude-code-action) 官方 GitHub Action。 + +## 常見踩坑（Anti-patterns） + +| ❌ 不要 | ✅ 改成 | +|---|---| +| 讓 AI 直接 push 到 main | 永遠 PR → review → merge | +| Blind accept 大規模 refactor diff | 拆成 < 50 LOC 改動，逐個 review | +| 把 .env / API key 丟給 AI 看 | 用工具對應的排除機制：Cursor `.cursorignore` / Aider `.aiderignore` / Claude Code 用 `.claude/settings.json` 的 `permissions.deny` | +| 讓 AI 在 production code 自由跑 shell | sandbox 限制、permission whitelist | +| 用 AI 生 test 後不檢查 assertion | 跑覆蓋率 + 故意改一個 bug 看 test 抓不抓得到 | +| 跨多個 commit 才發現方向錯 | **plan-first** 模式：先 review plan 再寫 code | + +## Tier 升級路徑 + +下表是建議的進階路徑： + +| Tier | 工具 | 適合誰 | 學習成本 | +|---|---|---|---| +| **Tier 0** | Cursor / Copilot / Claude.ai | IDE 內 chat、autocomplete、不自己寫 agent | 0（會用編輯器就行） | +| **Tier 1** | Claude Code / Cline / OpenCode + `CLAUDE.md` | CLI 接 file system、human-in-the-loop | 1-2 天上手 | +| **Tier 2** | 自寫 Skills + MCP server | 把 dev workflow 包成 skill 給團隊共用 | 1 週 setup | +| **Tier 3** | CI 自動跑 agent + production observability | 進到 [Stage 7](../stages/07-multi-agent-production.md) 領域 | 數週、需 governance | + +> **多數個人開發者可先停在 Tier 0-1**。**升級到 Tier 2+ 要先確認 ROI**——團隊夠大、流程夠重複、事故不可逆、才值得 invest。 + +## 也適用其他分支 + +開發者重疊度高的分支： + +- **要做 ML 研究 / 寫 paper** → [研究員分支](./for-researcher.md) +- **接 Notion / Linear / Atlassian / Postgres / Figma** 等 dev tool → [`resources/mcp-skills-catalog.md`](../resources/mcp-skills-catalog.md) +- **要寫自己的 Skill / MCP server** → [Stage 5](../stages/05-claude-code-ecosystem.md) + [`resources/cookbook.md`](../resources/cookbook.md) +- **想看 schema 設計細節** → [`resources/schema-design-cheatsheet.md`](../resources/schema-design-cheatsheet.md) +- **CLI 從零開始** → [Track A](../tracks/cli/A1-cli-intro.md)（A1 → A2 → A3） + +## 社群備註 + +特別歡迎以下貢獻： + +- IDE-specific 設定範本（Cursor `.cursorrules`、Claude Code `CLAUDE.md` for Python / Go / Rust 等） +- 程式語言特化 skill（Python / TypeScript / Rust / Go 各自的 best practice） +- CI / pre-commit hook 整合 case study +- **跨多人團隊用 AI dev 的 governance pattern**——多 dev 共用 Skills、permission 設計、cost tracking + +請見 [CONTRIBUTING.md](../CONTRIBUTING.md)。 diff --git a/branches/for-developer.zh-Hans.md b/branches/for-developer.zh-Hans.md new file mode 100644 index 0000000..0da75bd --- /dev/null +++ b/branches/for-developer.zh-Hans.md @@ -0,0 +1,166 @@ +# 开发者延伸路线（For Developers） + +> [繁體中文](./for-developer.md) | **简体中文** | [English](./for-developer.en.md) + +> 🚀 **第一次装 Claude Code / 写 `CLAUDE.md` / `SKILL.md`？** 快速 setup 指南在 [`resources/setup-guide.zh-Hans.md` D-E](../resources/setup-guide.zh-Hans.md)。已经熟可以跳过。 + +> [← 回主路线 README](../README.zh-Hans.md) · 走完 **Track A 的 A3** 或 **Track B 的 Stage 7** 后从这里接续。把 agentic AI 应用到开发流程上。 + +## 使用场景（开发场景 × AI 怎么帮） + +下表把开发者一天会遇到的 7 个场景拆开——每个场景有不同的痛点，AI 工具也不同： + +| 场景 | 你常遇到的痛点 | AI 能帮的部分 | 推荐工具（从轻到重） | +|---|---|---|---| +| **AI 结对编程** | 写到一半忘 syntax / 想不到 method 名 | 自动补全 + 改写 + 解释 | Cursor / Copilot → Claude Code | +| **多文件重构** | 改一个 class 怕漏改、跨文件 rename 容易错 | batch refactor、改 50 个文件仍保持风格一致 | Cursor → Claude Code → codex-delegate | +| **Code review（自己 PR）** | review 自己的 diff 看不出问题 | 找 bug / smell、检查 edge case | Claude Code / Cline → Continue（CI） | +| **写 test** | TDD 常忘加 case、coverage 不足 | 从 signature / spec 生成 pytest | Claude Code + Aider | +| **Debug** | log 不够、stack trace 看不懂 | 解释 trace、生成 hypothesis、跑 minimal repro | Claude Code | +| **Doc** | docstring / README 没人写、refactor 后过期 | 从 code 生成 doc、PR 对应改 doc | Claude Code | +| **CI / 团队自动化** | 重复手动跑 review、跨人风格不一 | GitHub Action 自动跑 review / lint | Claude Code Action + Continue | + +> 💡 **个人 vs 团队**：表中前 6 个是个人 daily workflow；最后 1 个（CI）是团队规范。团队规模 < 5 人时 CI 自动化的 ROI 不高，可以先不上。 + +## 精选 Projects + +> **CLI agent 比较**：7 个主流 CLI agent（Claude Code / Codex / OpenCode / Gemini CLI / goose / Aider / Hermes Agent）的并列比较见 [`resources/cli-agents-guide.zh-Hans.md`](../resources/cli-agents-guide.zh-Hans.md)。第一次接触 CLI agent 想要 step-by-step 入门 → [`tracks/cli/A1-cli-intro.zh-Hans.md`](../tracks/cli/A1-cli-intro.zh-Hans.md)（Track A 第一站）。 +> +> **MCP catalog**：要把 CLI 接到日常工具（GitHub、Linear、Atlassian、Postgres、Playwright、Figma 等）→ [`resources/mcp-skills-catalog.zh-Hans.md`](../resources/mcp-skills-catalog.zh-Hans.md)（65+ 个分类整理）。 +> +> 本页只列**跟开发者 workflow 直接相关**的工具入口。 + +### Coding Agents + +#### [Cursor](https://www.cursor.com/) ⭐⭐⭐⭐⭐ +编辑器集成的 AI 结对编程工具。在 AI 编辑器类工具中采用度高，可作为比较其他 IDE agent 的基准。 + +#### [Aider-AI/aider](https://github.com/Aider-AI/aider) ⭐⭐⭐⭐⭐ +★ 44k+ · Apache-2.0 — git-aware 的 CLI pair-programmer。直接编辑你 repo 中的文件，commit 都自动写好。**“git-native AI 编辑流程”的开源模板**。模型不限。 + +#### [anthropics/claude-code](https://github.com/anthropics/claude-code) ⭐⭐⭐⭐⭐ +★ 120k+ — Anthropic 官方的 agentic coding 助理。有 Skills + plugin 生态系。 + +#### [cline/cline](https://github.com/cline/cline) ⭐⭐⭐⭐⭐ +★ 61k+ · Apache-2.0 — VS Code extension，autonomous in-IDE agent：tool use、browser、step-by-step approval。**VS Code 用户想 IDE-native agentic dev 的好选项**。 + +#### [continuedev/continue](https://github.com/continuedev/continue) ⭐⭐⭐⭐ +★ 33k+ · Apache-2.0 — source-controlled AI checks，可以在 CI 强制执行。代表“**团队 / governance**”这条角度的 coding agent。 + +#### [OpenHands (前身为 OpenDevin)](https://github.com/All-Hands-AI/OpenHands) ⭐⭐⭐⭐ +★ 72k+ · MIT — open source 的自主软件开发 agent。设计上比 Aider / Claude Code 更激进——agent 自己跑 sandbox、自己 commit，适合“整个 issue 丢给它解”场景。 + +#### [block/goose](https://github.com/block/goose) ⭐⭐⭐⭐ +★ 43k+ · Apache-2.0 — 开源、可扩展的 AI agent，超出纯 code suggestion——能 install / execute / edit / test，搭配任何 LLM。同时支持多家 LLM provider 跟 MCP，提供 desktop app、CLI、API 三种接口。（repo 现指向 `aaif-goose/goose`。） + +#### [RooCodeInc/Roo-Code](https://github.com/RooCodeInc/Roo-Code) ⭐⭐⭐⭐ +★ 23k+ · Apache-2.0 — VS Code 的 coding agent，采用“**多种专业模式**”的设计，跟 Cline 的单一 agent flow 不同。VS Code 用户想 multi-mode 替代方案的选择。 + +### Code Review + +#### [obra/superpowers](https://github.com/obra/superpowers) ⭐⭐⭐⭐ +20+ 个经过实战验证的 skill，包括 TDD 模式、debug、协作模式。设计 code-review skill 时的好参考。 + +### 推荐工具 + +- [**yamadashy/repomix**](https://github.com/yamadashy/repomix) ⭐⭐⭐⭐⭐ ★ 26k+ — **典型开发者用途：打包整个 codebase 给 reviewer / refactor agent**。输出单个 AI-friendly 文件（XML / Markdown / JSON），方便 Claude Code / Codex 做 code review / refactoring。技术细节（MCP server mode、tree-sitter 压缩、secretlint 过滤）见官方 README。**Track A 的必备 daily-driver 工具。** + +## 必练流程（按使用频率） + +| 频率 | 流程 | 怎么做（≤ 3 步） | 推荐工具 | 适合谁 | +|---|---|---|---|---| +| **每天** | AI 结对写 code | (1) 开 branch
(2) 任务丢给 Claude Code、**先 plan**（不写 code）
(3) Review plan → approve → 写 code → 自己 review diff | Claude Code / Cursor / Cline | 全开发者 | +| **每天** | Git-native AI 编辑 | (1) `aider`
(2) 自然语言请求
(3) review + commit / `/undo` | Aider | 想要干净 git 流程的人 | +| **Per PR** | 自动 code review | (1) `.github/workflows/claude-review.yml`
(2) 抓 git diff → 跑 prompt → post 回 PR
(3) human + AI 双审 | Claude Code Action + Continue | 团队 | +| **Per feature** | 测试生成 | (1) 给 function signature + docstring
(2) 请 AI 生成 pytest case（含 edge case）
(3) 跑覆盖率 + 故意改 bug 看 test 抓不抓得到 | Claude Code / Aider | 写 test 阶段 | +| **不定期** | 多文件批量修改 | (1) Claude 写 plan
(2) codex-delegate 跑机械式 refactor
(3) Claude review diff | Claude + codex-delegate | refactor 30+ 文件的时候 | + +> 💡 **新手起手式**：先做“每天 AI 结对”+“测试生成”两条一个月，习惯后再上 PR 自动 review。 + +### 3 个具体 workflow recipe + +**1. AI 结对编程（每日节奏）** +1. 开新 feature → `git checkout -b feature/xxx` +2. 把任务丢给 Claude Code / Cursor，**先让它写 plan**（不直接写 code） +3. Review plan、修正方向 → 才 approve 写 code +4. 写完跑 tests + lint → 自己 review diff（**不要 blind accept**） +5. Commit message 自己写或 prompt 生草稿后改 + +**2. Aider git-native 流程（最像“跟 AI pair”）** +```bash +# 进入 repo 后 +aider --model anthropic/claude-sonnet-5 + +# 自然语言请求 +> 帮我把 utils.py 的 parse_date 加上时区参数，默认 UTC + +# Aider 会自动编辑 + commit。若不满意： +> /undo # 退掉最后一次 AI commit +``` + +**3. PR 上的 Claude code review（GitHub Action）** + +`.github/workflows/claude-review.yml`： +```yaml +on: + pull_request: +jobs: + review: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + - name: Run Claude review + env: + ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} + run: | + # 用 anthropics/claude-code-action 或自写 script + # 抓 git diff、跑 prompt、结果 post 回 PR +``` +参考 [`anthropics/claude-code-action`](https://github.com/anthropics/claude-code-action) 官方 GitHub Action。 + +## 常见踩坑（Anti-patterns） + +| ❌ 不要 | ✅ 改成 | +|---|---| +| 让 AI 直接 push 到 main | 永远 PR → review → merge | +| Blind accept 大规模 refactor diff | 拆成 < 50 LOC 改动，逐个 review | +| 把 .env / API key 丢给 AI 看 | 用工具对应的排除机制：Cursor `.cursorignore` / Aider `.aiderignore` / Claude Code 用 `.claude/settings.json` 的 `permissions.deny` | +| 让 AI 在 production code 自由跑 shell | sandbox 限制、permission whitelist | +| 用 AI 生 test 后不检查 assertion | 跑覆盖率 + 故意改一个 bug 看 test 抓不抓得到 | +| 跨多个 commit 才发现方向错 | **plan-first** 模式：先 review plan 再写 code | + +## Tier 升级路径 + +下表是建议的进阶路径： + +| Tier | 工具 | 适合谁 | 学习成本 | +|---|---|---|---| +| **Tier 0** | Cursor / Copilot / Claude.ai | IDE 内 chat、autocomplete、不自己写 agent | 0（会用编辑器就行） | +| **Tier 1** | Claude Code / Cline / OpenCode + `CLAUDE.md` | CLI 接 file system、human-in-the-loop | 1-2 天上手 | +| **Tier 2** | 自写 Skills + MCP server | 把 dev workflow 包成 skill 给团队共用 | 1 周 setup | +| **Tier 3** | CI 自动跑 agent + production observability | 进到 [Stage 7](../stages/07-multi-agent-production.zh-Hans.md) 领域 | 数周、需 governance | + +> **多数个人开发者可先停在 Tier 0-1**。**升级到 Tier 2+ 要先确认 ROI**——团队够大、流程够重复、事故不可逆，才值得 invest。 + +## 也适用其他分支 + +开发者重叠度高的分支： + +- **要做 ML 研究 / 写 paper** → [研究员分支](./for-researcher.zh-Hans.md) +- **接 Notion / Linear / Atlassian / Postgres / Figma** 等 dev tool → [`resources/mcp-skills-catalog.zh-Hans.md`](../resources/mcp-skills-catalog.zh-Hans.md) +- **要写自己的 Skill / MCP server** → [Stage 5](../stages/05-claude-code-ecosystem.zh-Hans.md) + [`resources/cookbook.zh-Hans.md`](../resources/cookbook.zh-Hans.md) +- **想看 schema 设计细节** → [`resources/schema-design-cheatsheet.zh-Hans.md`](../resources/schema-design-cheatsheet.zh-Hans.md) +- **CLI 从零开始** → [Track A](../tracks/cli/A1-cli-intro.zh-Hans.md)（A1 → A2 → A3） + +## 社群备注 + +特别欢迎以下贡献： + +- IDE-specific 设置范本（Cursor `.cursorrules`、Claude Code `CLAUDE.md` for Python / Go / Rust 等） +- 编程语言特化 skill（Python / TypeScript / Rust / Go 各自的 best practice） +- CI / pre-commit hook 集成 case study +- **跨多人团队用 AI dev 的 governance pattern**——多 dev 共用 Skills、permission 设计、cost tracking + +请见 [CONTRIBUTING.md](../CONTRIBUTING.md)。 diff --git a/branches/for-everyday-users.en.md b/branches/for-everyday-users.en.md new file mode 100644 index 0000000..455510b --- /dev/null +++ b/branches/for-everyday-users.en.md @@ -0,0 +1,176 @@ +# Extension Path: For Everyday Users + +> [繁體中文](./for-everyday-users.md) | [简体中文](./for-everyday-users.zh-Hans.md) | **English** + +> 🚀 **Everyday users can start directly at Tier 0** (web / mobile apps), **without any setup**. Only read [`resources/setup-guide.en.md` A-C](../resources/setup-guide.en.md) (about 30 minutes from zero) when you want to run a local LLM (Tier 3) or use CLI automation (Tier 2). + +> [← Back to main path README](../README.en.md) · You **don't have to walk the full main path** to start here — this branch is for people who **just want to USE AI, not build agents**. + +## Use Cases (Life Scenarios × How AI Helps) + +The table below splits everyday AI use into 7 common scenarios. Most of them are fully covered by web apps at Tier 0: + +| Scenario | Pain point | How AI helps | Recommended tools | +|---|---|---|---| +| **Writing email / cover letters** | Getting stuck on how to start | Drafting + tone edits + version comparison | Claude.ai / ChatGPT | +| **Learning new skills** | Materials feel formal; nobody is there to ask | Personalized tutoring, interruptible at any time | Claude.ai / ChatGPT | +| **Language practice** | No conversation partner; unclear grammar mistakes | Voice conversation and instant correction | ChatGPT Voice / Gemini | +| **Research / comparison** | Hard to know which source to trust | Multi-source search with citations | Perplexity | +| **Organizing life workflows** | Recipes / trips / todo lists are scattered | Consolidation + structure | Claude.ai / ChatGPT | +| **Batch file cleanup** | 100 PDFs / images with no clear grouping | Rename + classify + summarize | Claude Desktop / Claude Code | +| **Privacy-sensitive chat** | Medical / legal / financial notes should not go to the cloud | Run a local LLM | Ollama + qwen2.5 | + +> 💡 **Do not rush upgrades**: the first 5 scenarios can stay at Tier 0 (web). You only need Tier 1-3 when you repeat the same flow often or data absolutely cannot leave your machine. + +## Where to Start: 4 Tiers by "How Hands-On Are You?" + +``` +Tier 0: Web / Mobile App (recommended starting point) + ↓ +Tier 1: Desktop App (upgrade when you need to handle local files) + ↓ +Tier 2: CLI Agent (willing to learn a bit of command line; automate daily flows) + ↓ +Tier 3: Local LLM (privacy-sensitive, cost-sensitive, want offline) +``` + +**Most people stay at Tier 0 / Tier 1** — Tiers 2-3 are for special needs or learners. + +--- + +## 🎯 Curated Projects + +### Tier 0 — Web / Mobile App ⭐ Entry-level + +#### [Claude.ai](https://claude.ai) ⭐⭐⭐⭐⭐ +Anthropic's official interface. Best for long-form writing, in-depth discussion, complex questions — answer style is more restrained, less hallucination-prone. + +#### [ChatGPT](https://chatgpt.com) ⭐⭐⭐⭐⭐ +OpenAI's official interface. Largest ecosystem (GPTs, Custom Instructions, Voice mode). The standard general-purpose pick. + +#### [Gemini](https://gemini.google.com) ⭐⭐⭐⭐ +Google's offering. Long context — enough to read very long documents, roughly a thick book — is particularly useful for dropping in a whole PDF and asking questions; still check whether citations and summaries are correct. Integrated with Google services (Gmail, Docs). + +#### [Perplexity](https://perplexity.ai) ⭐⭐⭐⭐ +Search engine × LLM — every answer cites sources. Better than ChatGPT for "needs current info" scenarios. + +--- + +### Tier 1 — Desktop App + +#### [Claude Desktop](https://claude.ai/download) ⭐⭐⭐⭐⭐ +Beyond the web version: drag files in, read local files, retain long conversation context. **Also the gateway to AI-tool integration (MCP)** — you can connect Slack / Gmail / Calendar and operate them directly inside Claude. + +#### [ChatGPT Desktop](https://openai.com/chatgpt/desktop) ⭐⭐⭐⭐ +Desktop version of ChatGPT. Ask questions about screenshots, voice conversation, integrate with other apps. + +--- + +### Tier 2 — CLI Agents (advanced users willing to learn the command line) + +> These tools are positioned for developers but **everyday users can use them too** — e.g. batch-rename files, organize the Downloads folder, auto-write weekly reviews, summarize PDFs into Markdown. +> +> Want a detailed comparison? See [`resources/cli-agents-guide.en.md`](../resources/cli-agents-guide.en.md) — six major CLI agents side by side, recommendations by use case, common pitfalls, real-world setups. +> +> Want step-by-step onboarding? See [`tracks/cli/A1-cli-intro.en.md`](../tracks/cli/A1-cli-intro.en.md) — Track A first stop, from install to your first task. +> +> Want to wire your CLI agent to Notion / Obsidian / Excel / Google docs / etc.? See [`resources/mcp-skills-catalog.en.md`](../resources/mcp-skills-catalog.en.md) — 65+ MCP servers / Skills grouped by use case. + +#### [anthropics/claude-code](https://github.com/anthropics/claude-code) ⭐⭐⭐⭐⭐ +★ 120k+ — Anthropic's official CLI agent. Reads/writes files, runs commands, handles multi-step tasks. **The most beginner-friendly CLI tool for everyday users.** + +#### [openai/codex](https://github.com/openai/codex) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 80k+ | +| License | Apache-2.0 | + +**What it teaches**: OpenAI's terminal agent — it can help organize files, batch-process text, and run multi-step tasks from the command line; coding is only one use case. Same category as Claude Code, but uses OpenAI models. + +**Best for**: People who already subscribe to ChatGPT Plus / Pro and want to use the same account in the terminal. + +#### [sst/opencode](https://github.com/sst/opencode) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 155k+ | +| License | MIT | + +**What it teaches**: Open-source coding agent **not tied to any specific LLM provider** — use Claude, GPT, Gemini, or local Ollama, your choice. Community-maintained, fast iteration. + +**Best for**: Self-hosters; people who don't want vendor lock-in; anyone switching between multiple LLMs. + +#### [google-gemini/gemini-cli](https://github.com/google-gemini/gemini-cli) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 103k+ | +| License | Apache-2.0 | + +**What it teaches**: Google's official Gemini CLI agent. Brings Gemini's long context and Google ecosystem integration to the terminal. + +**Best for**: Heavy users of the Google ecosystem (Gmail, Drive, Docs). + +--- + +### Tier 3 — Local LLM (privacy / offline / cost) + +#### [Ollama](https://github.com/ollama/ollama) ⭐⭐⭐⭐⭐ +★ 170k+ — One command to run a local LLM. Use this when privacy-sensitive data (medical records, contracts, family conversations) shouldn't leave your machine. See [Stage 1 — Local LLM](../stages/01-llm-basics.en.md). + +#### [LM Studio](https://lmstudio.ai/) +Closed-source but the most beginner-friendly option — drag-and-drop UI, no command line. Mac / Windows / Linux. + +--- + +### Prompt Library + +#### [f/awesome-chatgpt-prompts](https://github.com/f/awesome-chatgpt-prompts) ⭐⭐⭐⭐ +★ 161k+ — Community-maintained prompt megacatalog. "Act as a translator / résumé consultant / chef..." in hundreds of roles. **When stuck on how to start, browse here.** + +--- + +## Required Reading + +1. [**Anthropic — How to write effective prompts**](https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/overview) — readable without code +2. [**OpenAI — Prompting Guide**](https://platform.openai.com/docs/guides/prompt-engineering) — the parallel official doc + +If you want to go deeper, see [Stage 2 — Prompt Engineering](../stages/02-prompt-engineering.en.md), which has a more systematic treatment. + +## Workflows You Can Build (by frequency) + +Use these 5 templates as starting points and adapt them to your own context: + +| Frequency | Workflow | Steps (≤3) | Recommended tools | +|---|---|---|---| +| **Daily** | Email triage | (1) Paste pending emails into Claude in the morning
(2) Ask it to classify "reply now / today / this week / skip"
(3) Draft replies for your review | Claude.ai / ChatGPT | +| **Daily** | Speaking practice | (1) Open ChatGPT Voice
(2) Practice English / Japanese conversation
(3) Ask it to flag grammar mistakes | ChatGPT Voice / Gemini | +| **Weekly** | Weekly journal | (1) Tell Claude what you did this week
(2) Ask for a journal + next week's priorities
(3) Save it to Obsidian / Notion | Claude.ai | +| **Occasional** | Batch file cleanup | (1) Run Claude Code in your Downloads folder
(2) Rename by date + topic
(3) Sort into subfolders | Claude Code | +| **Privacy scenario** | Local medical / legal / financial notes | (1) Run qwen2.5:7b in Ollama
(2) Organize personal notes without sending data to the cloud
(3) ⚠️ It protects **privacy**, not **correctness**: specific diagnoses / legal judgments / investment decisions still require professionals | Ollama + qwen2.5 | + +> 💡 **Starter habit**: run "daily email triage" and "speaking practice" for a month first, then add other workflows. + +## Tier Recommendations for Everyday Users + +Recommended progression: + +| Tier | Tools | Best for | Learning cost | +|---|---|---|---| +| **Tier 0** | Claude.ai / ChatGPT / Gemini / Perplexity (web) | 90% of scenarios: no install, no payment required | 0 (if you can use a browser) | +| **Tier 1** | Claude Desktop / ChatGPT Desktop + MCP | Local files, retained conversation history, Gmail / Notion integrations | 30 minutes | +| **Tier 2** | Claude Code / opencode (CLI) | Repeated automation needs, such as doing the same task 100 times daily | 1-2 days | +| **Tier 3** | Ollama local LLM | Privacy-sensitive data that cannot go to the cloud, API-cost sensitivity, offline use | Half a day | + +> **Do not let anyone push you to upgrade prematurely**. Tier 0 is enough for most people. Tiers 2-3 are tools, not status symbols. + +## Community Notes + +Contributions especially welcome: + +- Domain-specific prompt templates (cooking, fitness, language learning) +- Chinese-friendly chat tools (Chinese LLMs, localized wrappers) +- Privacy / safety best practices (what data is OK to send / what isn't) + +See [CONTRIBUTING.md](../CONTRIBUTING.md). diff --git a/branches/for-everyday-users.md b/branches/for-everyday-users.md new file mode 100644 index 0000000..510ce69 --- /dev/null +++ b/branches/for-everyday-users.md @@ -0,0 +1,179 @@ +# 日常使用者延伸路線（For Everyday Users） + +> **繁體中文** | [简体中文](./for-everyday-users.zh-Hans.md) | [English](./for-everyday-users.en.md) + +> 🚀 **日常使用者可直接從 Tier 0 開始**（網頁 / 手機 App）、**不需要任何 setup**。只有當你想跑本地 LLM（Tier 3）或用 CLI 自動化（Tier 2）時、才需要看 [`resources/setup-guide.md` A-C](../resources/setup-guide.md)（30 分鐘從零裝好）。 + +> [← 回主路線 README](../README.md) · 你**不一定要走完整條主幹**才能從這裡開始——這條分支是給「**只想 USE AI、不一定要 BUILD agent**」的人。 + +## 使用情境（生活場景 × AI 怎麼幫） + +下表把日常使用者一天會遇到的 7 個情境拆開——多數場景在網頁版（Tier 0）就能搞定： + +| 場景 | 你常遇到的痛點 | AI 能幫的部分 | 推薦工具 | +|---|---|---|---| +| **寫 email / cover letter** | 卡在「該怎麼開頭」 | 起草 + 改語氣 + 多版本對比 | Claude.ai / ChatGPT | +| **學新技能** | 教材太正式、沒人問問題 | 個人化 tutor、可隨時打斷問 | Claude.ai / ChatGPT | +| **練語言** | 沒對話對象、不知文法錯哪 | 語音對話、即時糾錯 | ChatGPT Voice / Gemini | +| **查資料 / 比較** | 不知該信哪個來源 | 多源搜尋 + 附引用 | Perplexity | +| **整理生活流程** | 食譜 / 行程 / 待辦清單散落 | 整合 + 結構化 | Claude.ai / ChatGPT | +| **批次整理檔案** | 100 個 PDF / 圖片不知怎麼分 | 重命名 + 分類 + 摘要 | Claude Desktop / Claude Code | +| **隱私敏感 chat** | 醫療 / 法律 / 財務筆記不想送雲 | 本地跑 LLM | Ollama + qwen2.5 | + +> 💡 **不要被催著升級**：前 5 個場景都可以停在 Tier 0（網頁版）。只有要「重複跑同一個流程」或「資料絕對不能送雲」才需要 Tier 1-3。 + +## 起步：你應該從哪一層進來？ + +按「**動手願意度**」分 4 層，從低到高： + +``` +Tier 0：網頁 / 手機 App（推薦從這裡開始） + ↓ +Tier 1：Desktop App（要處理本機檔案再升級） + ↓ +Tier 2：CLI Agent（願意學一點命令列，能自動化日常流程） + ↓ +Tier 3：本地 LLM（隱私敏感、API 費用敏感、想 offline） +``` + +**多數人停在 Tier 0 / Tier 1 就夠用了**——Tier 2-3 是給有特殊需求或想學的人。 + +--- + +## 🎯 精選 Projects + +### Tier 0 — 網頁 / 手機 App ⭐ 入門 + +#### [Claude.ai](https://claude.ai) ⭐⭐⭐⭐⭐ +Anthropic 官方介面。長文章、深度討論、複雜問題很適合用——回答風格較收斂、不太瞎掰。 + +#### [ChatGPT](https://chatgpt.com) ⭐⭐⭐⭐⭐ +OpenAI 官方介面。生態最廣（GPTs、Custom Instructions、Voice mode）。一般用途的標準選擇。 + +#### [Gemini](https://gemini.google.com) ⭐⭐⭐⭐ +Google 出品。長 context（一次能讀很長文件、約一本厚書的量）特別適合丟整本 PDF 進去問問題；仍要自己檢查引用與摘要是否正確。整合 Google 服務（Gmail、Docs）。 + +#### [Perplexity](https://perplexity.ai) ⭐⭐⭐⭐ +搜尋引擎 × LLM——每個答案都附引用來源。比 ChatGPT 適合「需要查最新資訊」的場景。 + +--- + +### Tier 1 — Desktop App + +#### [Claude Desktop](https://claude.ai/download) ⭐⭐⭐⭐⭐ +比網頁版多了：拖檔案進去、本機檔案讀取、保留長期對話脈絡。**也是進入 AI 工具整合生態（MCP）的入口**——可以接 Slack / Gmail / 行事曆，讓你在 Claude 裡直接操作這些服務。 + +#### [ChatGPT Desktop](https://openai.com/chatgpt/desktop) ⭐⭐⭐⭐ +ChatGPT 桌面版。可以對螢幕截圖問問題、語音對話、跟其他 App 整合。 + +--- + +### Tier 2 — CLI Agent（願意學命令列的進階使用者） + +> 這些工具雖然定位給開發者，但**日常使用者也能用**——例如批次重新命名檔案、整理下載資料夾、自動寫每週回顧、把 PDF 摘要存成 Markdown。 +> +> 想看詳細比較？見 [`resources/cli-agents-guide.md`](../resources/cli-agents-guide.md)（7 個主流 CLI agent 並列、依 use case 推薦、常見坑、實用搭配）。 +> +> 想要 step-by-step 上手？見 [`tracks/cli/A1-cli-intro.md`](../tracks/cli/A1-cli-intro.md)（Track A 第一站，從安裝到第一個任務）。 +> +> 想把 CLI agent 接到你的 Notion / Obsidian / Excel / Google 文件等日常工具？見 [`resources/mcp-skills-catalog.md`](../resources/mcp-skills-catalog.md)（按分類整理 65+ 個 MCP server / Skill）。 + +#### [anthropics/claude-code](https://github.com/anthropics/claude-code) ⭐⭐⭐⭐⭐ +★ 120k+ — Anthropic 官方的 CLI agent。能讀寫檔案、執行指令、做多步驟任務。**日常使用者最容易上手的 CLI 工具**。 + +#### [openai/codex](https://github.com/openai/codex) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 80k+ | +| License | Apache-2.0 | + +**教什麼**：OpenAI 出品的終端機 agent——可以在命令列幫你整理檔案、批次處理文字、執行多步驟任務；寫程式只是其中一種用途。跟 Claude Code 同類、但用的是 OpenAI 的模型。 + +**適合誰**：已經訂 ChatGPT Plus / Pro，想在終端機用同一個帳號做事的人。 + +#### [sst/opencode](https://github.com/sst/opencode) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 155k+ | +| License | MIT | + +**教什麼**：開源版的 coding agent，**不綁特定 LLM provider**——可以用 Claude、GPT、Gemini、本地 Ollama 任何一個。社群維護、迭代速度快。 + +**適合誰**：想 self-host、不想被 API provider 綁住，或要在多個 LLM 之間切換的人。 + +#### [google-gemini/gemini-cli](https://github.com/google-gemini/gemini-cli) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 103k+ | +| License | Apache-2.0 | + +**教什麼**：Google 官方的 Gemini CLI agent。把 Gemini 的長 context 跟 Google 生態整合到終端機。 + +**適合誰**：Google 生態的重度使用者（Gmail、Drive、Docs）。 + +--- + +### Tier 3 — 本地 LLM（隱私 / 離線 / 省錢） + +#### [Ollama](https://github.com/ollama/ollama) ⭐⭐⭐⭐⭐ +★ 170k+ — 一行指令跑本地 LLM。隱私敏感資料（病歷、合約、家人對話）不適合送去雲端時用這個。詳見 [Stage 1 — Local LLM 執行](../stages/01-llm-basics.md)。 + +#### [LM Studio](https://lmstudio.ai/) +非開源但對非開發者最友善——拖拉介面、不用 command line。Mac / Windows / Linux 都有。 + +--- + +### Prompt 素材庫 + +#### [f/awesome-chatgpt-prompts](https://github.com/f/awesome-chatgpt-prompts) ⭐⭐⭐⭐ +★ 161k+ — 社群維護的 prompt 大全。「act as 翻譯家 / 履歷顧問 / 廚師...」幾百種角色。**不知道怎麼開頭時從這裡找靈感**。 + +--- + +## 必修閱讀 + +1. [**Anthropic — How to write effective prompts**](https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/overview) — 不用程式碼也能讀的 prompt 寫法 +2. [**OpenAI — Prompting Guide**](https://platform.openai.com/docs/guides/prompt-engineering) — 對位的官方文件 +3. [**ChatGPT 怎麼用得最好（中文）**](https://www.runoob.com/) — 各家中文部落格的整理（runoob 等等） + +如果有興趣再深入，看 [Stage 2 — Prompt 設計](../stages/02-prompt-engineering.md)，那邊有正式系統性教學。 + +## 可以建的流程（按使用頻率） + +下表 5 條是模板、配合你自己的場景調整： + +| 頻率 | 流程 | 怎麼做（≤ 3 步） | 推薦工具 | +|---|---|---|---| +| **每天** | Email 分流 | (1) 早上把待回信件貼進 Claude
(2) 請它分類「立即回 / 今天回 / 這週回 / 不用回」
(3) 草擬回信讓你 review | Claude.ai / ChatGPT | +| **每天** | 練語言（口說） | (1) 打開 ChatGPT Voice 模式
(2) 對話練英 / 日
(3) 請它指出文法錯誤 | ChatGPT Voice / Gemini | +| **每週** | 週記整理 | (1) 跟 Claude 講這週做什麼
(2) 請它整理成週記 + 下週重點
(3) 存到 Obsidian / Notion | Claude.ai | +| **不定期** | 批次整理檔案 | (1) Claude Code 進 Downloads 資料夾
(2) 按日期 + 主題重命名
(3) 自動分到子資料夾 | Claude Code | +| **隱私場景** | 本地醫療 / 法律 / 財務筆記 | (1) Ollama 跑 qwen2.5:7b
(2) 整理個人筆記、資料不送雲
(3) ⚠️ 保護的是**隱私**、不是**正確性**——具體診斷 / 法律判斷 / 投資決策仍需專業人士 | Ollama + qwen2.5 | + +> 💡 **新手起手式**：先把「每天 Email 分流」+「練語言」做一個月、習慣 AI 在日常的位置、再加其他流程。 + +## 給日常使用者的層級建議 + +下表是建議的進階路徑： + +| Tier | 工具 | 適合誰 | 學習成本 | +|---|---|---|---| +| **Tier 0** | Claude.ai / ChatGPT / Gemini / Perplexity（網頁版） | 90% 的場景都在這裡——免安裝、免付費 | 0（會用瀏覽器就行） | +| **Tier 1** | Claude Desktop / ChatGPT Desktop + MCP | 要處理本機檔案、保留對話歷史、接 Gmail / Notion | 半小時裝好 | +| **Tier 2** | Claude Code / opencode（CLI） | 有重複自動化需求（每天做同樣的事 100 次） | 1-2 天上手 | +| **Tier 3** | Ollama 本地 LLM | 隱私敏感資料不能送雲、API 費用敏感、想 offline | 半天設定 | + +> **不要被人催著升級**——多數人 Tier 0 就夠用了。Tier 2-3 是工具、不是身份地位。 + +## 社群備註 + +這條分支也歡迎社群貢獻： + +- 推薦特定領域的 prompt template（料理、運動、學語言） +- 中文友善的 chat tools（國產 LLM、本地化 wrapper） +- 隱私 / 安全相關的最佳實踐（什麼資料能送 / 不能送） + +詳見 [CONTRIBUTING.md](../CONTRIBUTING.md)。 diff --git a/branches/for-everyday-users.zh-Hans.md b/branches/for-everyday-users.zh-Hans.md new file mode 100644 index 0000000..05c25c9 --- /dev/null +++ b/branches/for-everyday-users.zh-Hans.md @@ -0,0 +1,179 @@ +# 日常用户延伸路线（For Everyday Users） + +> [繁體中文](./for-everyday-users.md) | **简体中文** | [English](./for-everyday-users.en.md) + +> 🚀 **日常用户可直接从 Tier 0 开始**（网页 / 手机 App）、**不需要任何 setup**。只有当你想跑本地 LLM（Tier 3）或用 CLI 自动化（Tier 2）时，才需要看 [`resources/setup-guide.zh-Hans.md` A-C](../resources/setup-guide.zh-Hans.md)（30 分钟从零装好）。 + +> [← 回主路线 README](../README.zh-Hans.md) · 你**不一定要走完主干**才能从这里开始——这条分支是给“**只想 USE AI、不一定要 BUILD agent**”的人。 + +## 使用场景（生活场景 × AI 怎么帮） + +下表把日常用户一天会遇到的 7 个场景拆开——多数场景在网页版（Tier 0）就能搞定： + +| 场景 | 你常遇到的痛点 | AI 能帮的部分 | 推荐工具 | +|---|---|---|---| +| **写 email / cover letter** | 卡在“该怎么开头” | 起草 + 改语气 + 多版本对比 | Claude.ai / ChatGPT | +| **学新技能** | 教材太正式、没人问问题 | 个性化 tutor、可随时打断问 | Claude.ai / ChatGPT | +| **练语言** | 没对话对象、不知道语法错哪 | 语音对话、即时纠错 | ChatGPT Voice / Gemini | +| **查资料 / 比较** | 不知道该信哪个来源 | 多源搜索 + 附引用 | Perplexity | +| **整理生活流程** | 食谱 / 行程 / 待办清单散落 | 整合 + 结构化 | Claude.ai / ChatGPT | +| **批量整理文件** | 100 个 PDF / 图片不知道怎么分 | 重命名 + 分类 + 摘要 | Claude Desktop / Claude Code | +| **隐私敏感 chat** | 医疗 / 法律 / 财务笔记不想送云 | 本地跑 LLM | Ollama + qwen2.5 | + +> 💡 **不要被催着升级**：前 5 个场景都可以停在 Tier 0（网页版）。只有要“重复跑同一个流程”或“数据绝对不能送云”才需要 Tier 1-3。 + +## 起步：你应该从哪一层进入？ + +按“**动手意愿**”分 4 层，从低到高： + +``` +Tier 0：网页 / 手机 App（推荐从这里开始） + ↓ +Tier 1：Desktop App（要处理本地文件再升级） + ↓ +Tier 2：CLI Agent（愿意学一点命令行，能自动化日常流程） + ↓ +Tier 3：本地 LLM（隐私敏感、API 费用敏感、想 offline） +``` + +**多数人停在 Tier 0 / Tier 1 就够用了**——Tier 2-3 是给有特殊需求或想学的人。 + +--- + +## 🎯 精选 Projects + +### Tier 0 — 网页 / 手机 App ⭐ 入门 + +#### [Claude.ai](https://claude.ai) ⭐⭐⭐⭐⭐ +Anthropic 官方界面。长文章、深度讨论、复杂问题很适合用——回答风格较收敛、不太瞎掰。 + +#### [ChatGPT](https://chatgpt.com) ⭐⭐⭐⭐⭐ +OpenAI 官方界面。生态最广（GPTs、Custom Instructions、Voice mode）。一般用途的标准选择。 + +#### [Gemini](https://gemini.google.com) ⭐⭐⭐⭐ +Google 出品。长 context（一次能读很长文件、约一本厚书的量）特别适合丢整本 PDF 进去问问题；仍要自己检查引用与摘要是否正确。集成 Google 服务（Gmail、Docs）。 + +#### [Perplexity](https://perplexity.ai) ⭐⭐⭐⭐ +搜索引擎 × LLM——每个答案都附引用来源。比 ChatGPT 适合“需要查最新信息”的场景。 + +--- + +### Tier 1 — Desktop App + +#### [Claude Desktop](https://claude.ai/download) ⭐⭐⭐⭐⭐ +比网页版多了：拖文件进去、本地文件读取、保留长期对话脉络。**也是进入 AI 工具整合生态（MCP）的入口**——可以接 Slack / Gmail / 日历，让你在 Claude 里直接操作这些服务。 + +#### [ChatGPT Desktop](https://openai.com/chatgpt/desktop) ⭐⭐⭐⭐ +ChatGPT 桌面版。可以对屏幕截图问问题、语音对话、跟其他 App 集成。 + +--- + +### Tier 2 — CLI Agent（愿意学命令行的进阶用户） + +> 这些工具虽然定位给开发者，但**日常用户也能用**——例如批量重命名文件、整理下载文件夹、自动写每周回顾、把 PDF 摘要存成 Markdown。 +> +> 想看详细比较？见 [`resources/cli-agents-guide.zh-Hans.md`](../resources/cli-agents-guide.zh-Hans.md)（7 个主流 CLI agent 并列、依 use case 推荐、常见坑、实用搭配）。 +> +> 想要 step-by-step 上手？见 [`tracks/cli/A1-cli-intro.zh-Hans.md`](../tracks/cli/A1-cli-intro.zh-Hans.md)（Track A 第一站，从安装到第一个任务）。 +> +> 想把 CLI agent 接到你的 Notion / Obsidian / Excel / Google 文件等日常工具？见 [`resources/mcp-skills-catalog.zh-Hans.md`](../resources/mcp-skills-catalog.zh-Hans.md)（按分类整理 65+ 个 MCP server / Skill）。 + +#### [anthropics/claude-code](https://github.com/anthropics/claude-code) ⭐⭐⭐⭐⭐ +★ 120k+ — Anthropic 官方的 CLI agent。能读写文件、执行指令、做多步骤任务。**日常用户最容易上手的 CLI 工具**。 + +#### [openai/codex](https://github.com/openai/codex) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 80k+ | +| License | Apache-2.0 | + +**教什么**：OpenAI 出品的终端机 agent——可以在命令行帮你整理文件、批量处理文字、执行多步骤任务；写程序只是其中一种用途。跟 Claude Code 同类，但用的是 OpenAI 的模型。 + +**适合谁**：已经订 ChatGPT Plus / Pro，想在终端机用同一个账号做事的人。 + +#### [sst/opencode](https://github.com/sst/opencode) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 155k+ | +| License | MIT | + +**教什么**：开源版的 coding agent，**不绑定特定 LLM provider**——可以用 Claude、GPT、Gemini、本地 Ollama 任何一个。社群维护、迭代速度快。 + +**适合谁**：想 self-host、不想被 API provider 绑定，或要在多个 LLM 之间切换的人。 + +#### [google-gemini/gemini-cli](https://github.com/google-gemini/gemini-cli) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 103k+ | +| License | Apache-2.0 | + +**教什么**：Google 官方的 Gemini CLI agent。把 Gemini 的长 context 跟 Google 生态集成到终端机。 + +**适合谁**：Google 生态的重度用户（Gmail、Drive、Docs）。 + +--- + +### Tier 3 — 本地 LLM（隐私 / 离线 / 省钱） + +#### [Ollama](https://github.com/ollama/ollama) ⭐⭐⭐⭐⭐ +★ 170k+ — 一行指令跑本地 LLM。隐私敏感数据（病历、合约、家人对话）不适合送去云端时用这个。详见 [Stage 1 — Local LLM 执行](../stages/01-llm-basics.zh-Hans.md)。 + +#### [LM Studio](https://lmstudio.ai/) +非开源但对非开发者最友好——拖拉界面、不用 command line。Mac / Windows / Linux 都有。 + +--- + +### Prompt 素材库 + +#### [f/awesome-chatgpt-prompts](https://github.com/f/awesome-chatgpt-prompts) ⭐⭐⭐⭐ +★ 161k+ — 社群维护的 prompt 大全。“act as 翻译家 / 履历顾问 / 厨师...”几百种角色。**不知道怎么开头时从这里找灵感**。 + +--- + +## 必修阅读 + +1. [**Anthropic — How to write effective prompts**](https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/overview) — 不用代码也能读的 prompt 写法 +2. [**OpenAI — Prompting Guide**](https://platform.openai.com/docs/guides/prompt-engineering) — 对称的官方文件 +3. [**ChatGPT 怎么用得最好（中文）**](https://www.runoob.com/) — 各家中文博客的整理（runoob 等等） + +如果有兴趣再深入，看 [Stage 2 — Prompt 设计](../stages/02-prompt-engineering.zh-Hans.md)，那边有正式系统性教学。 + +## 可以建的流程（按使用频率） + +下表 5 条是模板，配合你自己的场景调整： + +| 频率 | 流程 | 怎么做（≤ 3 步） | 推荐工具 | +|---|---|---|---| +| **每天** | Email 分流 | (1) 早上把待回信件贴进 Claude
(2) 请它分类“立即回 / 今天回 / 这周回 / 不用回”
(3) 草拟回信让你 review | Claude.ai / ChatGPT | +| **每天** | 练语言（口说） | (1) 打开 ChatGPT Voice 模式
(2) 对话练英 / 日
(3) 请它指出语法错误 | ChatGPT Voice / Gemini | +| **每周** | 周记整理 | (1) 跟 Claude 讲这周做什么
(2) 请它整理成周记 + 下周重点
(3) 存到 Obsidian / Notion | Claude.ai | +| **不定期** | 批量整理文件 | (1) Claude Code 进 Downloads 文件夹
(2) 按日期 + 主题重命名
(3) 自动分到子文件夹 | Claude Code | +| **隐私场景** | 本地医疗 / 法律 / 财务笔记 | (1) Ollama 跑 qwen2.5:7b
(2) 整理个人笔记，数据不送云
(3) ⚠️ 保护的是**隐私**，不是**正确性**——具体诊断 / 法律判断 / 投资决策仍需专业人士 | Ollama + qwen2.5 | + +> 💡 **新手起手式**：先把“每天 Email 分流”+“练语言”做一个月，习惯 AI 在日常的位置，再加其他流程。 + +## 给日常用户的层级建议 + +下表是建议的进阶路径： + +| Tier | 工具 | 适合谁 | 学习成本 | +|---|---|---|---| +| **Tier 0** | Claude.ai / ChatGPT / Gemini / Perplexity（网页版） | 90% 的场景都在这里——免安装、免付费 | 0（会用浏览器就行） | +| **Tier 1** | Claude Desktop / ChatGPT Desktop + MCP | 要处理本地文件、保留对话历史、接 Gmail / Notion | 半小时装好 | +| **Tier 2** | Claude Code / opencode（CLI） | 有重复自动化需求（每天做同样的事 100 次） | 1-2 天上手 | +| **Tier 3** | Ollama 本地 LLM | 隐私敏感数据不能送云、API 费用敏感、想 offline | 半天设置 | + +> **不要被人催着升级**——多数人 Tier 0 就够用了。Tier 2-3 是工具，不是身份地位。 + +## 社群备注 + +这条分支也欢迎社群贡献： + +- 推荐特定领域的 prompt template（料理、运动、学语言） +- 中文友善的 chat tools（国产 LLM、本地化 wrapper） +- 隐私 / 安全相关的最佳实践（什么数据能送 / 不能送） + +详见 [CONTRIBUTING.md](../CONTRIBUTING.md)。 diff --git a/branches/for-knowledge-worker.en.md b/branches/for-knowledge-worker.en.md new file mode 100644 index 0000000..d9c83a4 --- /dev/null +++ b/branches/for-knowledge-worker.en.md @@ -0,0 +1,144 @@ +# Extension Path: For Knowledge Workers + +> [繁體中文](./for-knowledge-worker.md) | [简体中文](./for-knowledge-worker.zh-Hans.md) | **English** + +> 🚀 **No development background at all?** Most knowledge workers can start directly with Claude.ai / Claude Desktop, **without any setup**. Only read [`resources/setup-guide.en.md` A-D](../resources/setup-guide.en.md) (30-45 minutes from zero) when you need to connect an MCP server (such as Gmail / Notion) or use CLI automation. + +> [← Back to main path README](../README.en.md) · Continue here after **Track A's A3** or **Track B's Stage 7**. Apply agentic AI to office / knowledge work. + +## Use Cases (Office Scenarios × How AI Helps) + +The table below splits a knowledge worker's day into 7 common scenarios. Most of them are covered by Claude Desktop + MCP at Tier 1: + +| Scenario | Pain point | How AI helps | Recommended tools | +|---|---|---|---| +| **Email triage** | 100 messages a day; priority is hard to judge | Categorize + draft replies for your review | Claude Desktop + Gmail MCP | +| **Meetings → action items** | You forget half of a 30-minute meeting; action items are not captured | Transcript → key decisions + action items | Otter / Zoom transcript + Claude | +| **Cross-tool report aggregation** | Slack / Gmail / Notion each hold part of the picture | Pull metrics + synthesize + email summary | n8n / Make / Langflow | +| **Research / market intelligence** | Hard to know what to ask or who to trust | Multi-source search + cross-validation + memo | Perplexity + Claude | +| **Slack / messaging** | Tone is hard to calibrate in sensitive situations | Rewrite + adjust tone + produce alternatives | Claude.ai | +| **Notion / knowledge-base cleanup** | Notes are messy, unstructured, and hard to find | Retag + classify + auto-summarize | Claude Desktop + Notion MCP | +| **Documents / proposal drafts** | Specs and proposals get stuck | Outline → sections → polish | Claude.ai | + +> 💡 **MCP is central for knowledge workers**: new to MCP? Read [Stage 5.2 — MCP Foundation](../stages/05-claude-code-ecosystem.en.md#52--mcp-model-context-protocol--foundation). Looking for available MCP servers? See [`resources/mcp-skills-catalog.en.md`](../resources/mcp-skills-catalog.en.md). + +## Curated Projects + +> 💡 **Want to wire your AI agent to Notion / Gmail / Outlook / Slack / Excel / Lark?** Example: automatically turn Gmail messages into Notion todos. 65+ commonly-used office integration tools are listed in [`resources/mcp-skills-catalog.en.md`](../resources/mcp-skills-catalog.en.md) (grouped by use case). The section below stays focused on workflow / integration-platform-level tools. + +### Workflow Tools + +#### [n8n](https://github.com/n8n-io/n8n) ⭐⭐⭐⭐ +Self-hostable workflow automation platform with built-in AI integration; visual node-based editor. + +**Best for**: When you need glue between many SaaS tools (Slack + Gmail + Notion + AI). + +--- + +#### [Make.com](https://www.make.com/) (formerly Integromat) +Hosted workflow automation. Strong AI integration nodes. + +--- + +### Knowledge Worker Skills + +#### [obra/superpowers](https://github.com/obra/superpowers) ⭐⭐⭐⭐ + +Brainstorming, planning, and decision-making skills. + +--- + +### Knowledge Management / Personal AI + +#### [khoj-ai/khoj](https://github.com/khoj-ai/khoj) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 34k+ | +| License | AGPL-3.0 | + +**What it teaches**: Self-hosted "second brain" — chat with web + local docs, schedule automations, build custom agents. + +**Best for**: People wanting a self-hosted personal knowledge base + AI assistant. + +**Notes**: AGPL-3.0 license (copyleft). + +--- + +#### [lobehub/lobe-chat](https://github.com/lobehub/lobe-chat) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 76k+ | +| License | LobeHub Community License (Apache-2.0 base + commercial conditions) | + +**What it teaches**: Deployable multi-agent chat platform — plugin marketplace, knowledge bases, team collaboration. One representative option for self-hosted AI workspaces. + +**Best for**: Self-hosting a collaborative chat workspace. + +**Notes**: Commercial use needs to verify the LobeHub Community License's added conditions. + +--- + +#### [langflow-ai/langflow](https://github.com/langflow-ai/langflow) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 147k+ | +| License | MIT | + +**What it teaches**: Visual AI-agent design platform — useful for mapping customer support, report assembly, and data-query workflows into nodes. More agent-focused than n8n (n8n is generic workflow). API / MCP server deployment is an advanced note, not something you need to learn first. + +**Best for**: Knowledge workers who'd rather wire nodes than write Python; or anyone designing agent flows for team handoff. + +--- + +#### [Mintplex-Labs/anything-llm](https://github.com/Mintplex-Labs/anything-llm) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 60k+ | +| License | MIT | + +**What it teaches**: All-in-one private RAG workspace — upload documents, build agents, MCP-compatible, on-device by default. **A self-hosted alternative to NotebookLM**. + +**Best for**: Knowledge workers wanting a NotebookLM-style tool, self-hosted, without sending data to the cloud. + +--- + +### MCP Servers Useful for Knowledge Workers + +#### Communication MCP servers ⭐⭐⭐⭐ +Slack / Gmail / Discord etc. The original Anthropic-hosted reference servers were reorganized in 2025; community-maintained servers now live in [**punkpeye/awesome-mcp-servers**](https://github.com/punkpeye/awesome-mcp-servers#communication) and [**wong2/awesome-mcp-servers**](https://github.com/wong2/awesome-mcp-servers). Browse those lists for current Slack / Gmail / Drive / Calendar MCP servers. + +--- + +## Workflows You Can Build (by frequency) + +| Frequency | Workflow | Steps (≤3) | Recommended tools | Best for | +|---|---|---|---|---| +| **Daily** | Email triage | (1) Scan inbox
(2) Categorize into "now / today / this week / no reply"
(3) Draft replies for your review | Claude Desktop + Gmail MCP | All knowledge workers | +| **Per meeting** | Meetings → action items | (1) Capture transcript (Otter / Zoom)
(2) Have Claude extract "key decisions + action items"
(3) Assign + announce in Slack / email | Claude.ai + transcript tool | Managers / PMs | +| **Weekly** | Cross-tool report | (1) Pull metrics from N tools
(2) Synthesize with Claude / n8n
(3) Send email summary | n8n / Make / Langflow | People who send regular updates | +| **Occasional** | Research / market intelligence | (1) Clarify the question
(2) Search multiple sources + cross-validate
(3) Write a 1-2 page memo | Perplexity + Claude | Analysts / strategy roles | +| **Occasional** | Notion / knowledge-base cleanup | (1) Paste scattered notes into Claude
(2) Ask it to retag + classify
(3) Output structured Notion format | Claude Desktop + Notion MCP | Notion / Obsidian users | + +> 💡 **Starter habit**: run "daily email triage" for a month first so "open inbox, open Claude" becomes natural. Adding too many automations at once is hard to sustain. + +## Tier Recommendations + +Recommended progression: + +| Tier | Tools | Best for | Learning cost | +|---|---|---|---| +| **Tier 0** | Claude.ai / ChatGPT / Gemini / Perplexity (web) | Most knowledge workers start here | 0 (if you can use a browser) | +| **Tier 1** | Claude Desktop + MCP (Gmail / Notion / calendar) | Repeat workflows over local / cloud files | Half a day | +| **Tier 2** | n8n / Make / Langflow (automation platforms) | Connecting several SaaS tools without writing code | 1 week of setup | +| **Tier 3** | Claude Code / Codex / your own Python | Dev background or dev support, workflows ready for production deployment | Several weeks, overlaps with Track A | + +**Tier 3+ (CLI / SDK) is too heavy for most knowledge worker tasks**. Most people can stop at Tier 1-2. + +## Reading + +- [How I Turned Claude Code Into My Personal AI Agent OS](https://aimaker.substack.com/p/how-i-turned-claude-code-into-personal-ai-agent-operating-system-for-writing-research-complete-guide) — knowledge worker case study +- [**Anthropic — The Founder's Playbook**](https://claude.com/blog/the-founders-playbook) — Anthropic's 35-page startup playbook (2026-05-14); maps Idea / MVP / Launch / Scale onto 2026 AI capability diff --git a/branches/for-knowledge-worker.md b/branches/for-knowledge-worker.md new file mode 100644 index 0000000..b9152dc --- /dev/null +++ b/branches/for-knowledge-worker.md @@ -0,0 +1,144 @@ +# 知識工作者延伸路線（For Knowledge Workers） + +> **繁體中文** | [简体中文](./for-knowledge-worker.zh-Hans.md) | [English](./for-knowledge-worker.en.md) + +> 🚀 **完全沒開發背景？** 多數知識工作者可以直接從 Claude.ai / Claude Desktop 開始、**不需要任何 setup**。只有當你要接 MCP server（如 Gmail / Notion）或用 CLI 自動化時、才需要看 [`resources/setup-guide.md` A-D](../resources/setup-guide.md)（30-45 分鐘從零）。 + +> [← 回主路線 README](../README.md) · 走完 **Track A 的 A3** 或 **Track B 的 Stage 7** 後從這裡接續。把 agentic AI 應用到辦公室 / 知識工作上。 + +## 使用情境（辦公場景 × AI 怎麼幫） + +下表把知識工作者一天會遇到的 7 個情境拆開——多數場景在 Claude Desktop + MCP（Tier 1）就能搞定： + +| 場景 | 你常遇到的痛點 | AI 能幫的部分 | 推薦工具 | +|---|---|---|---| +| **Email 分流** | 每天 100 封看不完、優先順序錯 | 分類 + 草擬回信讓你 review | Claude Desktop + Gmail MCP | +| **會議 → 行動項目** | 聽完 30 分鐘忘一半、action item 沒記 | 逐字稿 → 主要決策 + 行動項目 | Otter / Zoom 逐字稿 + Claude | +| **跨工具報告整合** | Slack / Gmail / Notion 各一塊、要手動拉 | 自動拉指標 + 整合 + email summary | n8n / Make / Langflow | +| **研究 / 市場情報** | 不知問什麼問題、不知該信誰 | 多源搜尋 + 交叉驗證 + 備忘錄 | Perplexity + Claude | +| **Slack / 訊息** | 拿捏不準口氣、敏感場景 | 改寫 + 調語氣 + 多版本 | Claude.ai | +| **Notion / 知識庫整理** | 雜亂、沒架構、找不到舊筆記 | 重 tag + 分類 + 自動摘要 | Claude Desktop + Notion MCP | +| **文件 / 提案草稿** | spec / proposal 卡關 | 大綱 → 段落 → 潤色 | Claude.ai | + +> 💡 **MCP 是知識工作者的關鍵**：第一次接觸 MCP？看 [Stage 5.2 — MCP 基礎](../stages/05-claude-code-ecosystem.md#52--mcpmodel-context-protocol-基礎)；想知道有哪些 MCP server → [`resources/mcp-skills-catalog.md`](../resources/mcp-skills-catalog.md)。 + +## 精選 Projects + +> 💡 **想把 AI agent 接到 Notion / Gmail / Outlook / Slack / Excel / 飛書？**（例：把 Gmail 來信自動整理成 Notion 待辦）65+ 個常用辦公整合工具表見 [`resources/mcp-skills-catalog.md`](../resources/mcp-skills-catalog.md)（按使用情境分類）。下面這節保留 workflow / 整合平台級的工具。 + +### 工作流工具 + +#### [n8n](https://github.com/n8n-io/n8n) ⭐⭐⭐⭐ +可自架的工作流自動化平台，內建 AI 整合，採用視覺化節點式編輯器。 + +**適合誰**：要把多個 SaaS 工具串起來時（Slack + Gmail + Notion + AI）。 + +--- + +#### [Make.com](https://www.make.com/)（前身為 Integromat） +雲端代管的工作流自動化平台，AI 整合節點功能完整。 + +--- + +### 知識工作者 Skills + +#### [obra/superpowers](https://github.com/obra/superpowers) ⭐⭐⭐⭐ + +腦力激盪、規劃、決策類的 skill。 + +--- + +### 知識管理 / 個人 AI + +#### [khoj-ai/khoj](https://github.com/khoj-ai/khoj) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 34k+ | +| License | AGPL-3.0 | + +**教什麼**：自架的「第二大腦」——可以跟 web + 本地文件對話、排程自動化、自訂 agent。 + +**適合誰**：想自架個人知識庫 + AI assistant 的人。 + +**備註**：AGPL-3.0 license（傳染性開源）。 + +--- + +#### [lobehub/lobe-chat](https://github.com/lobehub/lobe-chat) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 76k+ | +| License | LobeHub Community License（基於 Apache-2.0 + 商用附加條款） | + +**教什麼**：可部署的多 agent 聊天平台——含 plugin marketplace、知識庫、團隊協作。可自架的 AI workspace 代表選項之一。 + +**適合誰**：要找可自架的協作 chat workspace。 + +**備註**：商用使用需確認 LobeHub Community License 的附加條款。 + +--- + +#### [langflow-ai/langflow](https://github.com/langflow-ai/langflow) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 147k+ | +| License | MIT | + +**教什麼**：視覺化 AI agent 設計平台——適合把客服、報告整理、資料查詢這類流程畫成節點。比 n8n 更專注在 agent 設計（n8n 是泛用工作流）。API / MCP server 部署是進階備註、不必一開始就學。 + +**適合誰**：寧可拉節點不寫 Python 的知識工作者，或要設計 agent 跟團隊溝通流程的人。 + +--- + +#### [Mintplex-Labs/anything-llm](https://github.com/Mintplex-Labs/anything-llm) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 60k+ | +| License | MIT | + +**教什麼**：all-in-one 的私有 RAG 工作平台——上傳文件、建 agent、相容 MCP、預設 on-device。**NotebookLM 的私有 self-hosted 替代方案**。 + +**適合誰**：知識工作者要私有部署、類 NotebookLM 的工具，避免把資料送到雲端。 + +--- + +### 對知識工作者有用的 MCP Server + +#### 通訊類 MCP server ⭐⭐⭐⭐ +Slack / Gmail / Discord 等。Anthropic 原本維護的 reference server 已於 2025 年重整；目前由社群維護的 server 集中在 [**punkpeye/awesome-mcp-servers**](https://github.com/punkpeye/awesome-mcp-servers#communication) 跟 [**wong2/awesome-mcp-servers**](https://github.com/wong2/awesome-mcp-servers)，要找最新的 Slack / Gmail / Drive / Calendar MCP server 可以從這兩個清單翻找。 + +--- + +## 可以建的流程（按使用頻率） + +| 頻率 | 流程 | 怎麼做（≤ 3 步） | 推薦工具 | 適合誰 | +|---|---|---|---|---| +| **每天** | Email 分流 | (1) 掃 inbox
(2) 分類成「立即 / 今天 / 這週 / 不用回」
(3) 草擬回信讓你 review | Claude Desktop + Gmail MCP | 全知識工作者 | +| **每次會議** | 會議 → 行動項目 | (1) 逐字稿（Otter / Zoom）
(2) Claude 抓「主要決策 + 行動項目」
(3) 指派 + Slack / email 公告 | Claude.ai + 逐字稿工具 | 主管 / PM | +| **每週** | 跨工具報告 | (1) 從 N 個工具拉指標
(2) Claude / n8n 整理
(3) email summary 寄出 | n8n / Make / Langflow | 要定期 update 老闆的人 | +| **不定期** | 研究 / 市場情報 | (1) 想清楚問題
(2) 多來源搜尋 + 交叉驗證
(3) 寫成 1-2 頁備忘錄 | Perplexity + Claude | 分析 / 策略職 | +| **不定期** | Notion / 知識庫重整 | (1) 把散落筆記貼進 Claude
(2) 請它重新 tag + 分類
(3) 輸出 Notion 結構化格式 | Claude Desktop + Notion MCP | 有 Notion / Obsidian 習慣的人 | + +> 💡 **新手起手式**：先把「每天 Email 分流」做一個月、養成「inbox 開 Claude」的習慣、再加其他流程。一次裝太多會養不起來。 + +## 層級建議 + +下表是建議的進階路徑： + +| Tier | 工具 | 適合誰 | 學習成本 | +|---|---|---|---| +| **Tier 0** | Claude.ai / ChatGPT / Gemini / Perplexity（網頁版） | 大多數知識工作者從這裡開始 | 0（會用瀏覽器就行） | +| **Tier 1** | Claude Desktop + MCP（Gmail / Notion / 行事曆） | 要對本機 / 雲端檔案重複跑流程 | 半天裝好 | +| **Tier 2** | n8n / Make / Langflow（自動化平台） | 要把多個 SaaS 工具串起來、不寫 code | 1 週 setup | +| **Tier 3** | Claude Code / Codex / 自己寫 Python | 有 dev 背景或團隊有 dev 支援、要做能上線部署的成果 | 數週、跟 Track A 重疊 | + +**Tier 3+（CLI / SDK）對多數知識工作者任務來說太重**——不要被別人慫恿過去。多數人停在 Tier 1-2 就夠。 + +## 閱讀 + +- [How I Turned Claude Code Into My Personal AI Agent OS](https://aimaker.substack.com/p/how-i-turned-claude-code-into-personal-ai-agent-operating-system-for-writing-research-complete-guide) — 知識工作者個案研究 +- [**Anthropic — The Founder's Playbook**](https://claude.com/blog/the-founders-playbook) — Anthropic 2026-05-14 發布的 35 頁 startup 指南;Idea / MVP / Launch / Scale 四階段對應到 2026 AI capability diff --git a/branches/for-knowledge-worker.zh-Hans.md b/branches/for-knowledge-worker.zh-Hans.md new file mode 100644 index 0000000..34b0dd2 --- /dev/null +++ b/branches/for-knowledge-worker.zh-Hans.md @@ -0,0 +1,144 @@ +# 知识工作者延伸路线（For Knowledge Workers） + +> [繁體中文](./for-knowledge-worker.md) | **简体中文** | [English](./for-knowledge-worker.en.md) + +> 🚀 **完全没有开发背景？** 多数知识工作者可以直接从 Claude.ai / Claude Desktop 开始、**不需要任何 setup**。只有当你要接 MCP server（如 Gmail / Notion）或用 CLI 自动化时，才需要看 [`resources/setup-guide.zh-Hans.md` A-D](../resources/setup-guide.zh-Hans.md)（30-45 分钟从零）。 + +> [← 回主路线 README](../README.zh-Hans.md) · 走完 **Track A 的 A3** 或 **Track B 的 Stage 7** 后从这里接续。把 agentic AI 应用到办公室 / 知识工作上。 + +## 使用场景（办公场景 × AI 怎么帮） + +下表把知识工作者一天会遇到的 7 个场景拆开——多数场景在 Claude Desktop + MCP（Tier 1）就能搞定： + +| 场景 | 你常遇到的痛点 | AI 能帮的部分 | 推荐工具 | +|---|---|---|---| +| **Email 分流** | 每天 100 封看不完、优先顺序错 | 分类 + 草拟回信让你 review | Claude Desktop + Gmail MCP | +| **会议 → 行动项目** | 听完 30 分钟忘一半、action item 没记 | 逐字稿 → 主要决策 + 行动项目 | Otter / Zoom 逐字稿 + Claude | +| **跨工具报告集成** | Slack / Gmail / Notion 各一块、要手动拉 | 自动拉指标 + 整合 + email summary | n8n / Make / Langflow | +| **研究 / 市场情报** | 不知道问什么问题、不知道该信谁 | 多源搜索 + 交叉验证 + 备忘录 | Perplexity + Claude | +| **Slack / 消息** | 拿捏不准语气、敏感场景 | 改写 + 调语气 + 多版本 | Claude.ai | +| **Notion / 知识库整理** | 杂乱、没架构、找不到旧笔记 | 重 tag + 分类 + 自动摘要 | Claude Desktop + Notion MCP | +| **文件 / 提案草稿** | spec / proposal 卡关 | 大纲 → 段落 → 润色 | Claude.ai | + +> 💡 **MCP 是知识工作者的关键**：第一次接触 MCP？看 [Stage 5.2 — MCP 基础](../stages/05-claude-code-ecosystem.zh-Hans.md#52--mcpmodel-context-protocol-基础)；想知道有哪些 MCP server → [`resources/mcp-skills-catalog.zh-Hans.md`](../resources/mcp-skills-catalog.zh-Hans.md)。 + +## 精选 Projects + +> 💡 **想把 AI agent 接到 Notion / Gmail / Outlook / Slack / Excel / 飞书？**（例：把 Gmail 来信自动整理成 Notion 待办）65+ 个常用办公集成工具表见 [`resources/mcp-skills-catalog.zh-Hans.md`](../resources/mcp-skills-catalog.zh-Hans.md)（按使用场景分类）。下面这节保留 workflow / 集成平台级的工具。 + +### 工作流工具 + +#### [n8n](https://github.com/n8n-io/n8n) ⭐⭐⭐⭐ +可自架的工作流自动化平台，内置 AI 集成，采用可视化节点式编辑器。 + +**适合谁**：要把多个 SaaS 工具串起来时（Slack + Gmail + Notion + AI）。 + +--- + +#### [Make.com](https://www.make.com/)（前身为 Integromat） +云端代管的工作流自动化平台，AI 集成节点功能完整。 + +--- + +### 知识工作者 Skills + +#### [obra/superpowers](https://github.com/obra/superpowers) ⭐⭐⭐⭐ + +头脑风暴、规划、决策类的 skill。 + +--- + +### 知识管理 / 个人 AI + +#### [khoj-ai/khoj](https://github.com/khoj-ai/khoj) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 34k+ | +| License | AGPL-3.0 | + +**教什么**：自架的“第二大脑”——可以跟 web + 本地文件对话、排程自动化、自定义 agent。 + +**适合谁**：想自架个人知识库 + AI assistant 的人。 + +**备注**：AGPL-3.0 license（传染性开源）。 + +--- + +#### [lobehub/lobe-chat](https://github.com/lobehub/lobe-chat) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 76k+ | +| License | LobeHub Community License（基于 Apache-2.0 + 商用附加条款） | + +**教什么**：可部署的多 agent 聊天平台——含 plugin marketplace、知识库、团队协作。可自架的 AI workspace 代表选项之一。 + +**适合谁**：想找可自架的协作 chat workspace。 + +**备注**：商用使用需确认 LobeHub Community License 的附加条款。 + +--- + +#### [langflow-ai/langflow](https://github.com/langflow-ai/langflow) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 147k+ | +| License | MIT | + +**教什么**：可视化 AI agent 设计平台——适合把客服、报告整理、资料查询这类流程画成节点。比 n8n 更专注于 agent 设计（n8n 是泛用工作流）。API / MCP server 部署是进阶备注，不必一开始就学。 + +**适合谁**：宁可拉节点不写 Python 的知识工作者，或要设计 agent 跟团队沟通流程的人。 + +--- + +#### [Mintplex-Labs/anything-llm](https://github.com/Mintplex-Labs/anything-llm) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 60k+ | +| License | MIT | + +**教什么**：all-in-one 的私有 RAG 工作平台——上传文件、建 agent、相容 MCP、预设 on-device。**NotebookLM 的私有 self-hosted 替代方案**。 + +**适合谁**：知识工作者要私有部署、类 NotebookLM 的工具，避免把数据送到云端。 + +--- + +### 对知识工作者有用的 MCP Server + +#### 通讯类 MCP server ⭐⭐⭐⭐ +Slack / Gmail / Discord 等。Anthropic 原本维护的 reference server 已于 2025 年重整；目前由社群维护的 server 集中在 [**punkpeye/awesome-mcp-servers**](https://github.com/punkpeye/awesome-mcp-servers#communication) 跟 [**wong2/awesome-mcp-servers**](https://github.com/wong2/awesome-mcp-servers)，要找最新的 Slack / Gmail / Drive / Calendar MCP server 可以从这两个清单翻找。 + +--- + +## 可以建的流程（按使用频率） + +| 频率 | 流程 | 怎么做（≤ 3 步） | 推荐工具 | 适合谁 | +|---|---|---|---|---| +| **每天** | Email 分流 | (1) 扫 inbox
(2) 分类成“立即 / 今天 / 这周 / 不用回”
(3) 草拟回信让你 review | Claude Desktop + Gmail MCP | 全知识工作者 | +| **每次会议** | 会议 → 行动项目 | (1) 逐字稿（Otter / Zoom）
(2) Claude 抓“主要决策 + 行动项目”
(3) 指派 + Slack / email 公告 | Claude.ai + 逐字稿工具 | 主管 / PM | +| **每周** | 跨工具报告 | (1) 从 N 个工具拉指标
(2) Claude / n8n 整理
(3) email summary 寄出 | n8n / Make / Langflow | 要定期 update 老板的人 | +| **不定期** | 研究 / 市场情报 | (1) 想清楚问题
(2) 多来源搜索 + 交叉验证
(3) 写成 1-2 页备忘录 | Perplexity + Claude | 分析 / 策略职 | +| **不定期** | Notion / 知识库重整 | (1) 把散落笔记贴进 Claude
(2) 请它重新 tag + 分类
(3) 输出 Notion 结构化格式 | Claude Desktop + Notion MCP | 有 Notion / Obsidian 习惯的人 | + +> 💡 **新手起手式**：先把“每天 Email 分流”做一个月，养成“inbox 开 Claude”的习惯，再加其他流程。一次装太多会养不起来。 + +## 层级建议 + +下表是建议的进阶路径： + +| Tier | 工具 | 适合谁 | 学习成本 | +|---|---|---|---| +| **Tier 0** | Claude.ai / ChatGPT / Gemini / Perplexity（网页版） | 大多数知识工作者从这里开始 | 0（会用浏览器就行） | +| **Tier 1** | Claude Desktop + MCP（Gmail / Notion / 日历） | 要对本机 / 云端文件重复跑流程 | 半天装好 | +| **Tier 2** | n8n / Make / Langflow（自动化平台） | 要把多个 SaaS 工具串起来、不写 code | 1 周 setup | +| **Tier 3** | Claude Code / Codex / 自己写 Python | 有 dev 背景或团队有 dev 支援、要做能上线部署的成果 | 数周、跟 Track A 重叠 | + +**Tier 3+（CLI / SDK）对多数知识工作者任务来说太重**——不要被别人怂恿过去。多数人停在 Tier 1-2 就够。 + +## 阅读 + +- [How I Turned Claude Code Into My Personal AI Agent OS](https://aimaker.substack.com/p/how-i-turned-claude-code-into-personal-ai-agent-operating-system-for-writing-research-complete-guide) — 知识工作者个案研究 +- [**Anthropic — The Founder's Playbook**](https://claude.com/blog/the-founders-playbook) — Anthropic 2026-05-14 发布的 35 页 startup 指南;Idea / MVP / Launch / Scale 四阶段对应到 2026 AI capability diff --git a/branches/for-researcher.en.md b/branches/for-researcher.en.md new file mode 100644 index 0000000..8af222c --- /dev/null +++ b/branches/for-researcher.en.md @@ -0,0 +1,208 @@ +# Extension Path: For Researchers + +> [繁體中文](./for-researcher.md) | [简体中文](./for-researcher.zh-Hans.md) | **English** + +> 🚀 **Computational researchers** (can run Python scripts, have an API key, and can use git) can jump into the advanced path directly. **Non-programming researchers** (humanities/social sciences, clinical research, literature-first work) can start with literature Q&A (NotebookLM) and Zotero AI tools, then read [`resources/setup-guide.en.md` A-C](../resources/setup-guide.en.md) when needed. + +> [← Back to main path README](../README.en.md) · Continue here after **Track A's A3** or **Track B's Stage 7**. Apply agentic AI to research workflows. + +## Use Cases + +Research days break into stages, and AI plays a different role at each stage. Use this table to orient yourself: + +| Stage | Common pain point | How AI helps | Recommended tools (light to heavy) | +|---|---|---|---| +| **Literature exploration** | You do not know the classic papers in a field | Recommendations + summaries + comparison | NotebookLM → paper-qa → gpt-researcher | +| **Close reading** | You lose the thread halfway through a PDF / miss the claim | Extract claims, figures, citations, and notes | Zotero + zotero-gpt → zotero-skills | +| **Research design** | The RQ is fuzzy, or the method choice is unclear | Clarifying dialogue and trade-off mapping | Claude.ai chat → ai-research-skills | +| **Experiments / coding** | Boilerplate repeats and plotting eats time | Write / edit code and batch refactor | Claude Code → codex-delegate | +| **Manuscript writing** | Drafts stall or sentences do not land | Outline → paragraphs → polishing | Claude.ai → gemini-delegate (long drafts) | +| **Revision / submission** | Journal requirements are easy to miss | banned-word / figure-text / submission checklist | academic-writing-skills | +| **Cross-paper synthesis** | Five papers need to talk to each other and context explodes | Read 1M tokens at once and organize the synthesis | gemini-delegate | + +> 💡 **Computational vs non-programming researchers**: the recommended tools run from light to heavy. Non-programming researchers can usually stop at the **first** tool in each row; computational researchers should move right only when they need automation. + +## Curated Projects + +> 💡 **Want to wire Claude Code into NotebookLM, Obsidian, Notion, Excel, PDF, Excalidraw, and other research tools?** 65+ integrations in [`resources/mcp-skills-catalog.en.md`](../resources/mcp-skills-catalog.en.md) (grouped by use case). The section below keeps research-specific tools and marketplaces. + +### Research Workflow Marketplaces + +#### [flonat/claude-research](https://github.com/flonat/claude-research) ⭐⭐⭐ + +Claude Code infrastructure for PhD researchers — skills, agents, hooks, rules for academic workflows. Strong LaTeX/bibliography focus. + +--- + +### Literature RAG / Q&A + +#### [Future-House/paper-qa](https://github.com/Future-House/paper-qa) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 8k+ | +| License | Apache-2.0 | + +**What it teaches**: PDF Q&A designed for **citation-grounded Q&A** — every answer includes sentence-level citations to reduce hallucination risk. Actual accuracy depends on document type; use the official benchmarks / papers as the reference. + +**Best for**: Researchers writing literature reviews who need "every answer must be traceable to its source." More rigorous than generic RAG. + +--- + +#### [assafelovic/gpt-researcher](https://github.com/assafelovic/gpt-researcher) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 27k+ | +| License | Apache-2.0 | + +**What it teaches**: Autonomous deep-research agent — planner + multi-source crawl + report synthesis. Give it a research topic, get a markdown / PDF brief out. + +**Best for**: Researchers who need to quickly scope new topics and produce research briefs. + +--- + +### Outline & Writing + +#### [stanford-oval/storm](https://github.com/stanford-oval/storm) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 28k+ | +| License | MIT | + +**What it teaches**: Multi-perspective outline-then-write pipeline — plain-language version: (1) simulate different perspectives asking questions, (2) organize those questions into an outline, then (3) generate a Wikipedia-style draft. From Stanford OVAL. + +**Best for**: Learning **outline-driven writing**. Great for producing topic briefs from scratch; the closest open-source analog to NotebookLM's structured report flow. + +**Notes**: Last push was over 6 months ago — verify the latest commit date before relying on it. + +--- + +#### [kaixindelele/ChatPaper](https://github.com/kaixindelele/ChatPaper) ⭐⭐⭐⭐⭐ (Chinese readers) + +| Field | Value | +|---|---| +| Language | Chinese + Python | +| Stars | ★ 19k+ | +| License | NOASSERTION (custom non-commercial) | + +**What it teaches**: Full arXiv workflow for Chinese researchers — paper summary + translation + polishing + review-response generation. Maintained by a Chinese team; defaults are friendly to Chinese-language workflows. + +**Best for**: Chinese graduate students looking for a Chinese-friendly entry-level paper workflow tool. + +**Notes**: License is custom non-commercial — read the original terms before any use; common practice is research / personal use, but you should verify the terms yourself. + +--- + +### Citation Manager Integrations + +#### [MuiseDestiny/zotero-gpt](https://github.com/MuiseDestiny/zotero-gpt) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 7k+ | +| License | AGPL-3.0 | + +**What it teaches**: A Zotero LLM plugin — chat with your library, summarize selections, generate inline notes. + +**Best for**: Heavy Zotero users who want AI inside their reading workflow without switching tools. + +**Notes**: AGPL-3.0 license (copyleft) — derivative products that ship modifications must follow the terms. + +--- + +### Multi-LLM Research Stack (Maintainer Setup) + +Some research tasks only need Claude (dialogue, design, review). Others waste Claude tokens (large code refactors, long-form drafts). The maintainer's actual setup is **Claude as planner / reviewer, Codex for code, and Gemini for long drafts**. Use this table to decide which model to use when: + +| Task type | Example | LLM to use | Why | +|---|---|---|---| +| Research design / hypothesis discussion | "Should this RQ use logistic vs survival?" | Claude.ai chat | Collaborative dialogue and context memory | +| Writing / editing code | "Add logging to 50 simulation scripts" | codex-delegate | Fast mechanical edits without burning Claude tokens | +| Long-form drafting (Chinese / English) | "Draft an 8-page paper section" | gemini-delegate | 1M context and strong long-form prose | +| Second opinion | "Ask Gemini to review my discussion section" | gemini-delegate | LLM-vs-LLM comparison makes Claude's own biases easier to spot | +| Pre-submission audit | "Run banned-word + figure-text checklist" | academic-writing-skills | Structured audit instead of ad hoc LLM judgment | + +#### Maintainer's 6 self-used research skills + +> ⚠️ **Disclosure**: The following 6 tools are research skills used day to day by the maintainer [@WenyuChiou](https://github.com/WenyuChiou) (Lehigh CEE PhD candidate) and published for people with similar needs. **They have not been independently evaluated by third parties**. Best fit: PhD dissertation writing and cross-paper literature organization. They may not fit your field. Full entries are in [`resources/mcp-skills-catalog.en.md` 13 + 14](../resources/mcp-skills-catalog.en.md#13-research-workflow-skills-academic--paper--lit). + +| Tool | Best for stage | One-liner | +|---|---|---| +| **[ai-research-skills](https://github.com/WenyuChiou/ai-research-skills)** ⭐⭐⭐⭐⭐ | Full pipeline | 14 research skills packaged as a 5-plugin marketplace; one command installs the set | +| **[research-hub](https://github.com/WenyuChiou/research-hub)** ⭐⭐⭐⭐ | Literature organization | Zotero + Obsidian + NotebookLM workspace with CLI / MCP / REST / dashboard interfaces | +| **[zotero-skills](https://github.com/WenyuChiou/zotero-skills)** ⭐⭐⭐⭐ | Reference management | Zotero CLI skill for search / add / classify / tag; complements zotero-gpt, which chats inside Zotero while this operates from outside | +| **[academic-writing-skills](https://github.com/WenyuChiou/academic-writing-skills)** ⭐⭐⭐ | Pre-submission | banned-word audit, figure-text coupling, and submission checklist; per-paper journal_format / style_overrides customization | +| **[codex-delegate](https://github.com/WenyuChiou/codex-delegate)** ⭐⭐⭐⭐⭐ | Coding | Standard Claude planner + Codex executor skill for batch refactor / boilerplate / migration work | +| **[gemini-delegate-skill](https://github.com/WenyuChiou/gemini-delegate-skill)** ⭐⭐⭐⭐ | Long drafts / synthesis | Claude planner + Gemini for 1M-context long-form writing / CJK / second opinions | + +--- + +### Multi-Agent for Research + +#### [langchain-ai/open_deep_research](https://github.com/langchain-ai/open_deep_research) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 11k+ | +| License | MIT | + +**What it teaches**: Open-source Deep Research — supports both single-agent and supervisor + multi-researcher architectures (the multi-agent path currently lives in `src/legacy/`), parallel search, citation-grounded report synthesis. A solid reference for "LLM agent that auto-produces a cited brief." + +**Best for**: Researchers building "agent auto-generates a cited brief" workflows. A solid open-source pick when you want a maintained reference implementation. + +**Notes**: Depends on LangGraph + search tools (API key required). + +--- + +#### [SakanaAI/AI-Scientist-v2](https://github.com/SakanaAI/AI-Scientist-v2) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 6k+ | +| License | The AI Scientist Source Code License (source-available, non-commercial + manuscript-disclosure clause) | + +**What it teaches**: End-to-end multi-agent science loop: ideate → code → experiment → write → peer-review. Sakana AI's research implementation of "AI writes a full ML paper." + +**Best for**: Researchers who want to see "what does a swarm of agents running a full research lifecycle look like." Architecture reference, not a production tool. + +**Notes**: Outputs are demo-level (not field-ready), ML/CS-domain bias. License is a custom source-available term (with a manuscript-disclosure clause) — read the LICENSE file before use. + +--- + +> Still missing: actively-maintained peer-review automation, conference-review pipelines. If you've built or know of one, please open a PR. + +## Required Reading + +1. [The Effortless Academic — Claude Code beginner guides](https://effortlessacademic.com/claude-code-and-cowork-for-academics-beginner-guide-part-1/) +2. [Pedro Sant'Anna — Researcher setup guide](https://paulgp.substack.com/p/getting-started-with-claude-code) + +## Workflows to Master + +The biggest mistake researchers make with AI is opening ChatGPT only when they get stuck. The key is making AI a daily tool by setting a cadence. The 7 workflows below are ordered by usage frequency and are routines the maintainer actually runs, not hypotheticals. + +| Frequency | Workflow | How to run it (≤ 3 steps) | Recommended tools | Best for | +|---|---|---|---|---| +| **Daily** | Literature inbox triage | (1) Put yesterday's papers into paper-qa
(2) Extract claims + a 4-5 line summary
(3) Move notes into Zotero / Obsidian | paper-qa + zotero-gpt | All researchers | +| **Daily** | Writing sprint (25 min) | (1) Give one paragraph to Claude.ai
(2) Run banned-word + figure-text audit
(3) Merge the revision into the main draft | Claude.ai + academic-writing-skills | Paper-writing stage | +| **Weekly** | Cross-paper synthesis | (1) Feed 5-10 PDFs to Gemini
(2) Ask where the papers disagree
(3) Turn the answer into a 1-page brief | gemini-delegate (1M context) | Computational researchers | +| **Weekly** | Zotero cleanup | (1) Mark unread / read
(2) Retag items
(3) Pull out PDFs that should be archived | zotero-skills or zotero-gpt | All researchers | +| **Monthly** | Research progress brief | (1) Pull recent notes from Obsidian + Zotero + NotebookLM
(2) Summarize 5 progress points
(3) Send to your advisor | research-hub | People using all 3 tools | +| **Per paper** | Final pre-submission audit | (1) banned-word audit
(2) figure-text coupling check
(3) submission checklist | academic-writing-skills | Final week before submission | +| **Per paper** | Multi-agent peer review | (1) Claude reviews logic / argument
(2) Codex checks code / table numbers
(3) Gemini reviews prose / clarity | codex-delegate + gemini-delegate | Pre-submission second opinion | + +> 💡 **Starter playbook**: run the daily inbox triage and writing sprint for one month first. Add advanced workflows only after the habit sticks. + +## Tier Recommendations + +Researchers do not need to install Claude Code on day one. This is the recommended progression: + +| Tier | Tools | Best for | Learning cost | +|---|---|---|---| +| **Tier 0** | Claude.ai web + NotebookLM | Non-programming researchers, humanities / social sciences, clinical research | 0 (browser skills are enough) | +| **Tier 1** | Claude Desktop + Zotero MCP / Obsidian MCP | Researchers already using Zotero / Obsidian | Half-day setup | +| **Tier 2** | Claude Code + ai-research-skills | Computational researchers who mostly write / edit code | 1-2 days to get started | +| **Tier 3** | Claude Code + codex-delegate + gemini-delegate + research-hub | People building a multi-LLM research pipeline across multiple tools | 1 week setup + ongoing tuning | + +**Most researchers can stop at Tier 1-2**. Tier 3 is worth it only when you have a lot of repeated workflows, such as running the same paper synthesis every week. diff --git a/branches/for-researcher.md b/branches/for-researcher.md new file mode 100644 index 0000000..1fcd1d6 --- /dev/null +++ b/branches/for-researcher.md @@ -0,0 +1,208 @@ +# 研究者延伸路線（For Researchers） + +> **繁體中文** | [简体中文](./for-researcher.zh-Hans.md) | [English](./for-researcher.en.md) + +> 🚀 **計算型研究者**（會跑 Python script、有 API key、會用 git）可直接進階；**非程式背景研究者**（人文社科、臨床研究、文獻為主）可先從文獻 Q&A（NotebookLM）、Zotero AI 工具開始、需要時再看 [`resources/setup-guide.md` A-C](../resources/setup-guide.md)。 + +> [← 回主路線 README](../README.md) · 走完 **Track A 的 A3** 或 **Track B 的 Stage 7** 後從這裡接續。把 agentic AI 應用到研究流程上。 + +## 使用情境（研究階段 × AI 怎麼幫） + +研究者一天分成幾個階段、AI 在每個階段的角色不同。下表幫你定位： + +| 階段 | 你常遇到的痛點 | AI 能幫的部分 | 推薦工具（從輕到重） | +|---|---|---|---| +| **文獻探索** | 不知道某個領域有哪些經典 paper | 推薦 + 摘要 + 比較 | NotebookLM → paper-qa → gpt-researcher | +| **文獻精讀** | PDF 翻一半就忘 / 抓不到 claim | 抓 claim、figure、citation、做筆記 | Zotero + zotero-gpt → zotero-skills | +| **研究設計** | RQ 模糊、不知選哪個 method | 對話釐清、列出 trade-off | Claude.ai 對話 → ai-research-skills | +| **實驗 / 寫程式** | 重複 boilerplate、寫 plot 浪費時間 | 寫 / 改 code、batch refactor | Claude Code → codex-delegate | +| **論文撰寫** | 草稿卡關、句子不通 | 大綱 → 段落 → 潤色 | Claude.ai → gemini-delegate（長稿） | +| **改稿 / 投稿** | 期刊規範一堆、容易漏 | banned-word / figure-text / submission checklist | academic-writing-skills | +| **跨 paper synthesis** | 5 篇 paper 互相對話、context 爆 | 1M token 一次讀完 + 整理 | gemini-delegate | + +> 💡 **計算型 vs 非程式背景**：表中「推薦工具」由輕到重——非程式背景研究者先停在每行**第一個**就夠了；計算型研究者要自動化才往後挑。 + +## 精選 Projects + +> 💡 **想把 Claude Code 接到 NotebookLM、Obsidian、Notion、Excel、PDF、Excalidraw 等研究常用工具？** 65+ 個整合在 [`resources/mcp-skills-catalog.md`](../resources/mcp-skills-catalog.md)（按使用情境分類）。下面這節保留「研究專屬」的工具與 marketplace。 + +### 研究流程 Marketplace + +#### [flonat/claude-research](https://github.com/flonat/claude-research) ⭐⭐⭐ + +給博士研究者的 Claude Code 基礎建設——學術流程用的 skill、agent、hook、規則。LaTeX / 文獻管理為主。 + +--- + +### 文獻 RAG / Q&A + +#### [Future-House/paper-qa](https://github.com/Future-House/paper-qa) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 8k+ | +| License | Apache-2.0 | + +**教什麼**：對 PDF 文件以 **citation-grounded Q&A** 為設計目標——每個答案附句子層級的引用、減少幻覺風險。實際準確率依文件類型而異、評測結果以官方 benchmark / paper 為準。 + +**適合誰**：寫文獻回顧、需要「查文獻時答案要可追溯」的研究者。比一般 RAG 更嚴謹。 + +--- + +#### [assafelovic/gpt-researcher](https://github.com/assafelovic/gpt-researcher) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 27k+ | +| License | Apache-2.0 | + +**教什麼**：自主 deep-research agent——planner + multi-source crawl + report 合成。給定一個研究主題，自動產出 markdown / PDF brief。 + +**適合誰**：要快速 scope 新題目、產 research brief 的研究者。 + +--- + +### 大綱與寫作 + +#### [stanford-oval/storm](https://github.com/stanford-oval/storm) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 28k+ | +| License | MIT | + +**教什麼**：multi-perspective outline-then-write pipeline——**白話三步**：(1) 先模擬不同觀點提出問題、(2) 把問題整理成大綱、(3) 最後生成 Wikipedia-style 草稿。Stanford OVAL 出品。 + +**適合誰**：想學「**outline-driven 寫作**」的人。從零產主題 brief 時的好工具，類似 NotebookLM structured report 流程的開源版。 + +**備註**：最後一次推送已超過 6 個月，使用前確認最新 commit 日期。 + +--- + +#### [kaixindelele/ChatPaper](https://github.com/kaixindelele/ChatPaper) ⭐⭐⭐⭐⭐（中文讀者） + +| 欄位 | 內容 | +|---|---| +| 語言 | 中文 + Python | +| Stars | ★ 19k+ | +| License | NOASSERTION（自訂條款，非商用） | + +**教什麼**：中文研究者向的 arXiv 全流程工具——論文總結 + 翻譯 + 潤色 + 審稿回覆生成。中國研究團隊維護，預設值對中文場景友善。 + +**適合誰**：中文研究生想找對中文友善的 paper 全流程入門工具。 + +**備註**：License 是自訂的非商用條款，使用前請先讀原始條款；研究或個人用途常見，但條款還是要自己看過確認。 + +--- + +### 文獻管理整合 + +#### [MuiseDestiny/zotero-gpt](https://github.com/MuiseDestiny/zotero-gpt) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 7k+ | +| License | AGPL-3.0 | + +**教什麼**：Zotero 的 LLM plugin——可以跟你的文獻庫對話、總結 selection、生成 inline notes。 + +**適合誰**：Zotero 重度使用者，想在閱讀流程裡直接接 AI 而不用切到別的工具。 + +**備註**：AGPL-3.0 license（傳染性開源）— 修改後要散布的衍生產品需遵守條款。 + +--- + +### Multi-LLM 研究組合（本 repo 維護者的研究 setup） + +研究流程裡有些任務 Claude 一個就夠（對話、設計、review），有些 Claude 做會浪費 token（大批 code refactor、長稿 draft）。維護者實際用的搭配是 **Claude 當 planner / reviewer、Codex 跑程式、Gemini 跑長稿**——下表列何時用哪個： + +| 任務類型 | 例子 | 用哪個 LLM | 為什麼 | +|---|---|---|---| +| 研究設計 / 假設討論 | 「這個 RQ 該用 logistic vs survival？」 | Claude.ai 對話 | 對話協作、context memory | +| 寫 / 改 code | 「50 個 simulation script 都加 logging」 | codex-delegate | 機械式編輯快、不燒 Claude token | +| 寫長稿（中英文） | 「draft 一個 8 頁 paper section」 | gemini-delegate | 1M context、長 prose 強項 | +| Second opinion | 「請 Gemini 看我的 discussion 段落」 | gemini-delegate | LLM-vs-LLM 對照、容易看出 Claude 自身偏誤 | +| 投稿前 audit | 「跑 banned-word + figure-text checklist」 | academic-writing-skills | structured audit、不靠 LLM 即興判斷 | + +#### 維護者自用的 6 個研究 skill + +> ⚠️ **揭露**：以下 6 個工具是維護者 [@WenyuChiou](https://github.com/WenyuChiou)（Lehigh CEE PhD candidate）日常在用的研究 skills、公開讓有相似需求的人用。**未經第三方獨立評測**——適合 PhD 學位寫作 / 跨 paper 文獻整理這類流程；不一定適合你的領域。詳細 entry 看 [`resources/mcp-skills-catalog.md` 13 + 14](../resources/mcp-skills-catalog.md#13-研究工作流-skills學術--paper--文獻)。 + +| 工具 | 適合階段 | 一句話 | +|---|---|---| +| **[ai-research-skills](https://github.com/WenyuChiou/ai-research-skills)** ⭐⭐⭐⭐⭐ | 全流程 | 14 個研究 skill 打包成 5-plugin marketplace、一個指令裝整套 | +| **[research-hub](https://github.com/WenyuChiou/research-hub)** ⭐⭐⭐⭐ | 文獻整理 | Zotero + Obsidian + NotebookLM 三工具整合 workspace、CLI / MCP / REST / dashboard 四介面 | +| **[zotero-skills](https://github.com/WenyuChiou/zotero-skills)** ⭐⭐⭐⭐ | 文獻管理 | Zotero CLI skill（搜 / 加 / 分類 / 標記）——跟 zotero-gpt 互補（後者在 Zotero 裡 chat、這份從外部操作） | +| **[academic-writing-skills](https://github.com/WenyuChiou/academic-writing-skills)** ⭐⭐⭐ | 投稿前 | banned-word audit、figure-text coupling、submission checklist；per-paper 可自訂 journal_format / style_overrides | +| **[codex-delegate](https://github.com/WenyuChiou/codex-delegate)** ⭐⭐⭐⭐⭐ | 寫程式 | Claude planner + Codex executor 的標準 skill——batch refactor / boilerplate / migration | +| **[gemini-delegate-skill](https://github.com/WenyuChiou/gemini-delegate-skill)** ⭐⭐⭐⭐ | 長稿 / synthesis | Claude planner + Gemini 寫 1M context 長文 / CJK / second-opinion | + +--- + +### Multi-Agent for Research + +#### [langchain-ai/open_deep_research](https://github.com/langchain-ai/open_deep_research) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 11k+ | +| License | MIT | + +**教什麼**：開源版的 Deep Research——支援單 agent 跟 supervisor + multi-researcher 兩種架構（multi-agent 那條目前在 `src/legacy/`）、平行搜尋、再合成成有引用的 report。是學「LLM agent 怎麼自動產出有引用 brief」的好參考。 + +**適合誰**：要打造「agent 自動產出有引用 brief」工作流程的研究者。是這個分類最 canonical 的開源選擇。 + +**備註**：依賴 LangGraph + 搜尋 tool（要 API key）。 + +--- + +#### [SakanaAI/AI-Scientist-v2](https://github.com/SakanaAI/AI-Scientist-v2) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 6k+ | +| License | The AI Scientist Source Code License（source-available，非商用 + 有 manuscript-disclosure 條款） | + +**教什麼**：端到端的 multi-agent 科學研究 loop：構想 → 寫程式 → 跑實驗 → 寫 paper → 互審。Sakana AI 的「AI 寫整篇 ML paper」研究實作。 + +**適合誰**：想看「多個 agent 跑完整研究 lifecycle 會長什麼樣」的研究者。研究架構參考、不是 production 工具。 + +**備註**：產出是 demo 等級（不是直接投稿用），ML / CS 領域偏多。License 是自訂的 source-available 條款（含 manuscript-disclosure 規定），使用前請先讀 LICENSE 檔。 + +--- + +> 還缺：peer-review 自動化、conference review pipeline 的活躍開源案例。如果你做過或知道有，歡迎開 PR。 + +## 必修閱讀 + +1. [The Effortless Academic — Claude Code beginner guides](https://effortlessacademic.com/claude-code-and-cowork-for-academics-beginner-guide-part-1/) +2. [Pedro Sant'Anna — Researcher setup guide](https://paulgp.substack.com/p/getting-started-with-claude-code) + +## 必練流程（按使用頻率） + +研究者用 AI 的最大誤區是「只在卡關才打開 ChatGPT」。把 AI 變成日常工具的關鍵是**設好頻率**——下表 7 條都是維護者自己每週都在跑的、不是空想。 + +| 頻率 | 流程 | 怎麼做（≤ 3 步） | 推薦工具 | 適合誰 | +|---|---|---|---|---| +| **每天** | 文獻 inbox 分流 | (1) 把昨天看到的 paper 丟 paper-qa
(2) 抓 claim + 4-5 行 summary
(3) 進 Zotero / Obsidian | paper-qa + zotero-gpt | 全研究者 | +| **每天** | 寫作 sprint（25 min） | (1) 寫一段給 Claude.ai
(2) 跑 banned-word + figure-text audit
(3) 改完進 main draft | Claude.ai + academic-writing-skills | 寫 paper 階段 | +| **每週** | 跨 paper synthesis | (1) 把 5-10 篇 PDF 餵 Gemini
(2) 問「這幾篇 disagree 在哪」
(3) 寫成 1 頁 brief | gemini-delegate（1M context） | 計算型 | +| **每週** | Zotero 整理 | (1) 標未讀 / 已讀
(2) 重 tag
(3) 抓出該歸檔的 PDF | zotero-skills 或 zotero-gpt | 全研究者 | +| **每月** | 研究進度 brief | (1) 從 Obsidian + Zotero + NotebookLM 抓近期筆記
(2) 整理出 5 個進度點
(3) 送指導教授 | research-hub | 同時用 3 工具的人 | +| **Per paper** | 投稿前 final audit | (1) banned-word audit
(2) figure-text coupling check
(3) submission checklist | academic-writing-skills | 投稿前 1 週 | +| **Per paper** | Multi-agent peer review | (1) Claude 看 logic / argument
(2) Codex 看 code / table 數字
(3) Gemini 看 prose / clarity | codex-delegate + gemini-delegate | 投稿前 second-opinion | + +> 💡 **新手起手式**：先做「每天 inbox 分流」+「寫作 sprint」兩條一個月、習慣後再加進階流程。一次裝太多會養不起來。 + +## 層級建議 + +研究者不需要一開始就裝 Claude Code。下表是建議的進階路徑： + +| Tier | 工具 | 適合誰 | 學習成本 | +|---|---|---|---| +| **Tier 0** | Claude.ai 網頁版 + NotebookLM | 非程式背景、人文社科、臨床研究 | 0（會用瀏覽器就行） | +| **Tier 1** | Claude Desktop + Zotero MCP / Obsidian MCP | 已有 Zotero / Obsidian 習慣的研究者 | 半天裝好 | +| **Tier 2** | Claude Code + ai-research-skills | 計算型研究者、寫 / 改程式為主 | 1-2 天上手 | +| **Tier 3** | Claude Code + codex-delegate + gemini-delegate + research-hub | 想跑 multi-LLM 研究 pipeline、跨多工具整合 | 1 週 setup + 持續調 | + +**多數研究者停在 Tier 1-2 就夠了**——Tier 3 是有大量重複流程（譬如每週跑同樣的 paper synthesis）才值得。 diff --git a/branches/for-researcher.zh-Hans.md b/branches/for-researcher.zh-Hans.md new file mode 100644 index 0000000..8a4d8f4 --- /dev/null +++ b/branches/for-researcher.zh-Hans.md @@ -0,0 +1,208 @@ +# 研究者延伸路线（For Researchers） + +> [繁體中文](./for-researcher.md) | **简体中文** | [English](./for-researcher.en.md) + +> 🚀 **计算型研究者**（会跑 Python script、有 API key、会用 git）可直接进阶；**非程序背景研究者**（人文社科、临床研究、文献为主）可先从文献 Q&A（NotebookLM）、Zotero AI 工具开始，需要时再看 [`resources/setup-guide.zh-Hans.md` A-C](../resources/setup-guide.zh-Hans.md)。 + +> [← 回主路线 README](../README.zh-Hans.md) · 走完 **Track A 的 A3** 或 **Track B 的 Stage 7** 后从这里接续。把 agentic AI 应用到研究流程上。 + +## 使用场景（研究阶段 × AI 怎么帮） + +研究者一天分成几个阶段，AI 在每个阶段的角色不同。下表帮你定位： + +| 阶段 | 你常遇到的痛点 | AI 能帮的部分 | 推荐工具（从轻到重） | +|---|---|---|---| +| **文献探索** | 不知道某个领域有哪些经典 paper | 推荐 + 摘要 + 比较 | NotebookLM → paper-qa → gpt-researcher | +| **文献精读** | PDF 翻一半就忘 / 抓不到 claim | 抓 claim、figure、citation、做笔记 | Zotero + zotero-gpt → zotero-skills | +| **研究设计** | RQ 模糊、不知选哪个 method | 对话厘清、列出 trade-off | Claude.ai 对话 → ai-research-skills | +| **实验 / 写代码** | 重复 boilerplate、写 plot 浪费时间 | 写 / 改 code、batch refactor | Claude Code → codex-delegate | +| **论文撰写** | 草稿卡关、句子不通 | 大纲 → 段落 → 润色 | Claude.ai → gemini-delegate（长稿） | +| **改稿 / 投稿** | 期刊规范一堆、容易漏 | banned-word / figure-text / submission checklist | academic-writing-skills | +| **跨 paper synthesis** | 5 篇 paper 互相对话、context 爆 | 1M token 一次读完 + 整理 | gemini-delegate | + +> 💡 **计算型 vs 非程序背景**：表中“推荐工具”由轻到重——非程序背景研究者先停在每行**第一个**就够了；计算型研究者要自动化才往后挑。 + +## 精选 Projects + +> 💡 **想把 Claude Code 接到 NotebookLM、Obsidian、Notion、Excel、PDF、Excalidraw 等研究常用工具？** 65+ 个集成在 [`resources/mcp-skills-catalog.zh-Hans.md`](../resources/mcp-skills-catalog.zh-Hans.md)（按使用场景分类）。下面这节保留“研究专属”的工具与 marketplace。 + +### 研究流程 Marketplace + +#### [flonat/claude-research](https://github.com/flonat/claude-research) ⭐⭐⭐ + +给博士研究者的 Claude Code 基础建设——学术流程用的 skill、agent、hook、规则。LaTeX / 文献管理为主。 + +--- + +### 文献 RAG / Q&A + +#### [Future-House/paper-qa](https://github.com/Future-House/paper-qa) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 8k+ | +| License | Apache-2.0 | + +**教什么**：对 PDF 文件以 **citation-grounded Q&A** 为设计目标——每个答案附句子层级的引用、减少幻觉风险。实际准确率依文件类型而异，评测结果以官方 benchmark / paper 为准。 + +**适合谁**：写文献回顾、需要“查文献时答案要可追溯”的研究者。比一般 RAG 更严谨。 + +--- + +#### [assafelovic/gpt-researcher](https://github.com/assafelovic/gpt-researcher) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 27k+ | +| License | Apache-2.0 | + +**教什么**：自主 deep-research agent——planner + multi-source crawl + report 合成。给定一个研究主题，自动产出 markdown / PDF brief。 + +**适合谁**：要快速 scope 新题目、产 research brief 的研究者。 + +--- + +### 大纲与写作 + +#### [stanford-oval/storm](https://github.com/stanford-oval/storm) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 28k+ | +| License | MIT | + +**教什么**：multi-perspective outline-then-write pipeline——**白话三步**：(1) 先模拟不同观点提出问题、(2) 把问题整理成大纲、(3) 最后生成 Wikipedia-style 草稿。Stanford OVAL 出品。 + +**适合谁**：想学“**outline-driven 写作**”的人。从零产主题 brief 时的好工具，类似 NotebookLM structured report 流程的开源版。 + +**备注**：最后一次推送已超过 6 个月，使用前确认最新 commit 日期。 + +--- + +#### [kaixindelele/ChatPaper](https://github.com/kaixindelele/ChatPaper) ⭐⭐⭐⭐⭐（中文读者） + +| 栏位 | 内容 | +|---|---| +| 语言 | 中文 + Python | +| Stars | ★ 19k+ | +| License | NOASSERTION（自定义条款，非商用） | + +**教什么**：中文研究者向的 arXiv 全流程工具——论文总结 + 翻译 + 润色 + 审稿回复生成。中国研究团队维护，默认值对中文场景友好。 + +**适合谁**：中文研究生想找对中文友好的 paper 全流程入门工具。 + +**备注**：License 是自定义的非商用条款，使用前请先读原始条款；研究或个人用途常见，但条款还是要自己看过确认。 + +--- + +### 文献管理集成 + +#### [MuiseDestiny/zotero-gpt](https://github.com/MuiseDestiny/zotero-gpt) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 7k+ | +| License | AGPL-3.0 | + +**教什么**：Zotero 的 LLM plugin——可以跟你的文献库对话、总结 selection、生成 inline notes。 + +**适合谁**：Zotero 重度用户，想在阅读流程里直接接 AI 而不用切到别的工具。 + +**备注**：AGPL-3.0 license（传染性开源）— 修改后要散布的衍生产品需遵守条款。 + +--- + +### Multi-LLM 研究组合（本 repo 维护者的研究 setup） + +研究流程里有些任务 Claude 一个就够（对话、设计、review），有些 Claude 做会浪费 token（大批 code refactor、长稿 draft）。维护者实际用的搭配是 **Claude 当 planner / reviewer、Codex 跑程序、Gemini 跑长稿**——下表列什么时候用哪个： + +| 任务类型 | 例子 | 用哪个 LLM | 为什么 | +|---|---|---|---| +| 研究设计 / 假设讨论 | “这个 RQ 该用 logistic vs survival？” | Claude.ai 对话 | 对话协作、context memory | +| 写 / 改 code | “50 个 simulation script 都加 logging” | codex-delegate | 机械式编辑快、不烧 Claude token | +| 写长稿（中英文） | “draft 一个 8 页 paper section” | gemini-delegate | 1M context、长 prose 强项 | +| Second opinion | “请 Gemini 看我的 discussion 段落” | gemini-delegate | LLM-vs-LLM 对照，容易看出 Claude 自身偏误 | +| 投稿前 audit | “跑 banned-word + figure-text checklist” | academic-writing-skills | structured audit，不靠 LLM 即兴判断 | + +#### 维护者自用的 6 个研究 skill + +> ⚠️ **披露**：以下 6 个工具是维护者 [@WenyuChiou](https://github.com/WenyuChiou)（Lehigh CEE PhD candidate）日常在用的研究 skills，公开让有相似需求的人用。**未经第三方独立评测**——适合 PhD 学位写作 / 跨 paper 文献整理这类流程；不一定适合你的领域。详细 entry 看 [`resources/mcp-skills-catalog.zh-Hans.md` 13 + 14](../resources/mcp-skills-catalog.zh-Hans.md#13-研究工作流-skills学术--paper--文献)。 + +| 工具 | 适合阶段 | 一句话 | +|---|---|---| +| **[ai-research-skills](https://github.com/WenyuChiou/ai-research-skills)** ⭐⭐⭐⭐⭐ | 全流程 | 14 个研究 skill 打包成 5-plugin marketplace，一个指令装整套 | +| **[research-hub](https://github.com/WenyuChiou/research-hub)** ⭐⭐⭐⭐ | 文献整理 | Zotero + Obsidian + NotebookLM 三工具集成 workspace，CLI / MCP / REST / dashboard 四种接口 | +| **[zotero-skills](https://github.com/WenyuChiou/zotero-skills)** ⭐⭐⭐⭐ | 文献管理 | Zotero CLI skill（搜 / 加 / 分类 / 标记）——跟 zotero-gpt 互补（后者在 Zotero 里 chat，这份从外部操作） | +| **[academic-writing-skills](https://github.com/WenyuChiou/academic-writing-skills)** ⭐⭐⭐ | 投稿前 | banned-word audit、figure-text coupling、submission checklist；per-paper 可定制 journal_format / style_overrides | +| **[codex-delegate](https://github.com/WenyuChiou/codex-delegate)** ⭐⭐⭐⭐⭐ | 写代码 | Claude planner + Codex executor 的标准 skill——batch refactor / boilerplate / migration | +| **[gemini-delegate-skill](https://github.com/WenyuChiou/gemini-delegate-skill)** ⭐⭐⭐⭐ | 长稿 / synthesis | Claude planner + Gemini 写 1M context 长文 / CJK / second-opinion | + +--- + +### Multi-Agent for Research + +#### [langchain-ai/open_deep_research](https://github.com/langchain-ai/open_deep_research) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 11k+ | +| License | MIT | + +**教什么**：开源版的 Deep Research——支持单 agent 跟 supervisor + multi-researcher 两种架构（multi-agent 那条目前在 `src/legacy/`）、平行搜索、再合成为有引用的 report。是学“LLM agent 怎么自动产出有引用 brief”的好参考。 + +**适合谁**：要打造“agent 自动产出有引用 brief”工作流程的研究者。是这个分类最 canonical 的开源选择。 + +**备注**：依赖 LangGraph + 搜索 tool（要 API key）。 + +--- + +#### [SakanaAI/AI-Scientist-v2](https://github.com/SakanaAI/AI-Scientist-v2) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 6k+ | +| License | The AI Scientist Source Code License（source-available，非商用 + 有 manuscript-disclosure 条款） | + +**教什么**：端到端的 multi-agent 科学研究 loop：构想 → 写代码 → 跑实验 → 写 paper → 互审。Sakana AI 的“AI 写整篇 ML paper”研究实践。 + +**适合谁**：想看“多个 agent 跑完整研究生命周期会长什么样”的研究者。研究架构参考、不是 production 工具。 + +**备注**：产出是 demo 级别（不是直接投稿用），ML / CS 领域偏多。License 是自定义的 source-available 条款（含 manuscript-disclosure 规定），使用前请先读 LICENSE 文件。 + +--- + +> 还缺：peer-review 自动化、conference review pipeline 的活跃开源案例。如果你做过或知道有，欢迎开 PR。 + +## 必修阅读 + +1. [The Effortless Academic — Claude Code beginner guides](https://effortlessacademic.com/claude-code-and-cowork-for-academics-beginner-guide-part-1/) +2. [Pedro Sant'Anna — Researcher setup guide](https://paulgp.substack.com/p/getting-started-with-claude-code) + +## 必练流程（按使用频率） + +研究者用 AI 的最大误区是“只在卡关才打开 ChatGPT”。把 AI 变成日常工具的关键是**设好频率**——下表 7 条都是维护者自己每周都在跑的，不是空想。 + +| 频率 | 流程 | 怎么做（≤ 3 步） | 推荐工具 | 适合谁 | +|---|---|---|---|---| +| **每天** | 文献 inbox 分流 | (1) 把昨天看到的 paper 丢 paper-qa
(2) 抓 claim + 4-5 行 summary
(3) 进 Zotero / Obsidian | paper-qa + zotero-gpt | 全研究者 | +| **每天** | 写作 sprint（25 min） | (1) 写一段给 Claude.ai
(2) 跑 banned-word + figure-text audit
(3) 改完进 main draft | Claude.ai + academic-writing-skills | 写 paper 阶段 | +| **每周** | 跨 paper synthesis | (1) 把 5-10 篇 PDF 喂 Gemini
(2) 问“这几篇 disagree 在哪”
(3) 写成 1 页 brief | gemini-delegate（1M context） | 计算型 | +| **每周** | Zotero 整理 | (1) 标未读 / 已读
(2) 重 tag
(3) 抓出该归档的 PDF | zotero-skills 或 zotero-gpt | 全研究者 | +| **每月** | 研究进度 brief | (1) 从 Obsidian + Zotero + NotebookLM 抓近期笔记
(2) 整理出 5 个进度点
(3) 送指导教授 | research-hub | 同时用 3 工具的人 | +| **Per paper** | 投稿前 final audit | (1) banned-word audit
(2) figure-text coupling check
(3) submission checklist | academic-writing-skills | 投稿前 1 周 | +| **Per paper** | Multi-agent peer review | (1) Claude 看 logic / argument
(2) Codex 看 code / table 数字
(3) Gemini 看 prose / clarity | codex-delegate + gemini-delegate | 投稿前 second-opinion | + +> 💡 **新手起手式**：先做“每天 inbox 分流”+“写作 sprint”两条一个月，习惯后再加进阶流程。一次装太多会养不起来。 + +## 层级建议 + +研究者不需要一开始就装 Claude Code。下表是建议的进阶路径： + +| Tier | 工具 | 适合谁 | 学习成本 | +|---|---|---|---| +| **Tier 0** | Claude.ai 网页版 + NotebookLM | 非程序背景、人文社科、临床研究 | 0（会用浏览器就行） | +| **Tier 1** | Claude Desktop + Zotero MCP / Obsidian MCP | 已有 Zotero / Obsidian 习惯的研究者 | 半天装好 | +| **Tier 2** | Claude Code + ai-research-skills | 计算型研究者、写 / 改程序为主 | 1-2 天上手 | +| **Tier 3** | Claude Code + codex-delegate + gemini-delegate + research-hub | 想跑 multi-LLM 研究 pipeline、跨多工具集成 | 1 周 setup + 持续调 | + +**多数研究者停在 Tier 1-2 就够了**——Tier 3 是有大量重复流程（比如每周跑同样的 paper synthesis）才值得。 diff --git a/branches/for-teacher.en.md b/branches/for-teacher.en.md new file mode 100644 index 0000000..2645f87 --- /dev/null +++ b/branches/for-teacher.en.md @@ -0,0 +1,226 @@ +# Extension Path: For Teachers / Educators + +> [繁體中文](./for-teacher.md) | [简体中文](./for-teacher.zh-Hans.md) | **English** + +> 🚀 **Most teachers can start directly with Claude.ai (web) + NotebookLM, without any setup**. Only read [`resources/setup-guide.en.md` A-C](../resources/setup-guide.en.md) (about 30 minutes to install what you need) when you need to automate repeated workflows (Tier 2+, such as generating 50 parent letters every week). + +> [← Back to main path README](../README.en.md) · Continue here after **Track A's A3** or **Track B's Stage 7**. Apply agentic AI to teaching workflows. + +## Use Cases + +Teacher-facing AI use cases can first be read as three branches: **lesson prep and class material creation**, **classroom and learning support**, and **other use cases**. + +This grouping follows common AI in Education discussions around administration, instruction, and learning, while also reflecting recent work on generative AI for material creation, feedback, and interactive support (Chen et al., 2020; Mittal et al., 2024). Start with teacher oversight principles and boundaries, then choose the branch that best matches your teaching need. + +![Teacher and AI agent use-case overview](../resources/diagrams/teacher-ai-use-cases-overview.jpg) + +### What Teachers Should Watch For When Using AI + +AI can prepare and assist, but it should not replace teacher judgment. Recent AI in Education and generative AI for education research also emphasizes clear learning goals, safety boundaries, and human review when teachers design AI agents (Chen et al., 2020; Mittal et al., 2024). + +- **Keep teacher judgment in the loop**: when student data, grades, or teaching decisions are involved, teachers remain responsible for final review. +- **Avoid giving answers too quickly**: if students interact with an AI agent, design the flow as Socratic dialogue so students explain their reasoning across multiple turns. +- **Align with learning goals**: use fixed prompts, checklists, or school-approved tools to constrain the AI's role and task, so student interaction stays tied to the lesson. +- **Rewrite student questions when needed**: for younger students, such as elementary or middle-school learners, rewrite unclear questions before sending them to the agent. + +### Lesson Prep and Class Material Creation + +These workflows help teachers prepare materials. The output should still be revised, selected, and checked by the teacher. + +- **Lesson plan generation**: turn curriculum standards, unit goals, and student levels into lesson outlines, time allocation, activity design, discussion prompts, and supplementary guides. +- **Quiz / rubric creation**: generate multiple-choice, short-answer, essay questions, answer keys, and scoring criteria from texts, textbook sections, or academic articles. +- **Slide deck preparation, curriculum mapping, and multimedia visualization**: turn textbook chapters or teacher notes into slide outlines, handout structures, weekly sequences, prerequisite knowledge, assessment checkpoints, images, 3D objects, video scripts, GIFs, or classroom presentation assets. +- **Student feedback synthesis and analysis**: summarize student answers, assignments, or class responses to identify common misconceptions, remediation needs, and next-step practice. +- **Multilingual material translation and adaptation**: rewrite or translate material for different languages, and generate text-to-speech assets when useful. +- **Materials for interactive games, activities, and virtual simulation scenarios**: prepare educational games, rhymes, task cards, role cards, scenario text, or simulation backgrounds; for actual interaction or activity design, see the next section on classroom and learning support. + +### Classroom and Learning Support + +![Classroom and learning support use cases](../resources/diagrams/teacher-ai-classroom-use-cases.jpg) + +These workflows help students understand, practice, and interact. AI acts more like a teaching assistant or activity support tool. Note that a single lesson does not need to include every element; choose the moments where an AI agent design actually fits the learning activity. + +- **Immersive learning and realistic scenario practice**: use realistic simulation, role-play, or speaking practice so students can rehearse in near-authentic contexts while lowering cognitive load and hesitation. +- **Curiosity and questioning support**: use Socratic follow-up questions and multi-turn interaction to help students ask clearer questions, explain their reasoning, and develop critical thinking and metacognition. +- **Instant grading and deeper feedback**: help students learn from mistakes by pointing out errors, explaining why they happen, and suggesting revisions instead of only giving a score or answer. +- **Intelligent tutoring and virtual teaching assistants**: answer questions, explain terminology, and provide hints so students receive appropriate support in and beyond class. +- **Adaptive teaching and dynamic paths**: provide difficulty-matched content based on student level, infer the zone of proximal development from learning performance, and offer suitable scaffolding or remediation materials. + +### Other Use Cases + +These use cases may not happen directly inside a lesson, but they shape teacher work, student support, and education-system operations. + +- **Special education support**: use speech-to-text, text-to-speech, and related tools to help students with different needs participate in class. +- **Parent-teacher communication and family learning**: summarize student progress and suggest home-based follow-up activities. +- **Administration and academic integrity**: summarize learning traces, generate reports, or support plagiarism and cheating-risk checks. +- **Career and skill-development guidance**: support career exploration, training-plan design, and weak-spot practice recommendations. +- **Teacher professional development**: summarize teaching methods, education-technology trends, and research insights. +- **Advanced research and business analysis**: support literature review, market-trend analysis, or business-plan drafting. +- **Privacy-preserving synthetic data**: generate anonymized synthetic data for research or system testing without directly exposing personal data. + +### References + +- Chen, L., Chen, P., & Lin, Z. (2020). [Artificial Intelligence in Education: A Review](https://doi.org/10.1109/ACCESS.2020.2988510). *IEEE Access*, 8, 75264-75278. +- Mittal, U., Sai, S., Chamola, V., & Sangwan, D. (2024). [A Comprehensive Review on Generative AI for Education](https://doi.org/10.1109/ACCESS.2024.3468368). *IEEE Access*, 12, 142733-142759. + +## Curated Projects + +### Teaching Workflow Skills + +(Most are not yet skill-marketplace packaged. This branch has the most room for community contribution — see CONTRIBUTING.md.) + +### Useful Building Blocks + +#### [obra/superpowers](https://github.com/obra/superpowers) ⭐⭐⭐⭐ +General writing / brainstorming skills. Adaptable for lesson prep. + +#### Advanced automation: [Claude Code](https://github.com/anthropics/claude-code) (with custom CLAUDE.md) ⭐⭐⭐⭐⭐ +★ 120k+ — **The basic teacher stack is Claude.ai (web) + NotebookLM + Google Classroom / LMS integrations**; start there. **Upgrade to Claude Code only when you already have repeatable batch workflows** (such as generating 50 parent letters every week or analyzing student feedback every semester), and expect to learn some CLI. + +### Teaching Course Materials (for teachers preparing classes) + +#### [huggingface/agents-course](https://github.com/huggingface/agents-course) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 28k+ | +| License | Apache-2.0 | + +**What it teaches**: Hugging Face's official agents curriculum — notebooks, exercises, certifications. A ready-made **AI agent teaching artifact**. + +**Best for**: Teachers running an "AI agents intro" workshop or class who want existing materials to teach from or adapt. + +**Notes**: This teaches *how to build agents* — it's not an "AI tutor for students" tool. + +--- + +#### [datawhalechina/llm-universe](https://github.com/datawhalechina/llm-universe) ⭐⭐⭐⭐ (Chinese) + +| Field | Value | +|---|---| +| Language | Chinese (zh-Hans) | +| Stars | ★ 13k+ | +| License | NOASSERTION | + +**What it teaches**: Datawhale's Chinese-language LLM application development course — RAG, agents, chapter exercises. A ready-made template for Chinese-speaking teachers preparing class material. + +**Best for**: Chinese-language teachers wanting a ready LLM curriculum to adapt to their students' level. + +**Notes**: Same caveat as `huggingface/agents-course` — it's "teach students to build LLM apps," not "AI assistant for the teacher." + +--- + +### Prompt Libraries + +#### [f/awesome-chatgpt-prompts](https://github.com/f/awesome-chatgpt-prompts) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 161k+ | +| License | NOASSERTION (CC0 / public-domain-style, but no SPDX) | + +**What it teaches**: Community-maintained prompt megacatalog — "act as X" templates covering hundreds of roles (teacher, interviewer, stand-up comedian, debater, ...). Teachers can use it as "prompt writing examples" to show students, or borrow specific prompts for in-class demos. + +**Best for**: Teachers introducing "prompt engineering" who want concrete examples of different writing styles to compare. + +**Notes**: Quality varies — treat as a sourcebook to pick from, not "use everything as-is." + +--- + +### Reading Material + +#### [The Effortless Academic — Beginner Guides](https://effortlessacademic.com/claude-code-and-cowork-for-academics-beginner-guide-part-1/) +Multi-part guide for academics adopting Claude Code, applicable to teachers. + +## Workflows You Can Build (by teaching stage) + +Use these 5 templates as starting points and adapt them to your subject: + +| Stage | Workflow | Steps (≤3) | Recommended tools | Caveat | +|---|---|---|---|---| +| **Before prep** | Lesson plan generator | (1) Curriculum + topic prompt → outline
(2) Outline → slides
(3) Slides → assessment items | Claude.ai / NotebookLM | Teacher final review | +| **During prep** | Rubric creation | (1) Provide student work samples + learning goals
(2) Ask AI for a 4-level rubric draft
(3) Teacher adjusts level boundaries | Claude.ai | Avoid vague terms like "high quality" | +| **Grading work** | Personalized feedback | (1) Student work + rubric → AI feedback draft
(2) Teacher reviews and edits each one
(3) Send back | Claude.ai | **AI assistance ≠ AI grading**; final grades must be human | +| **Class activity** | Scenario simulation | (1) Learning goal + role setup → dialogue script
(2) Run class practice
(3) Ask reflection questions | Claude.ai | Socratic follow-up, no direct answers; student input must contain **no PII** | +| **After class** | Personalized remediation material | (1) Summarize common student errors
(2) Generate short practice + hints by student level
(3) Add extension challenges | Claude.ai | Anonymize student data | + +> 💡 **Starter habit**: run the "before-prep lesson plan generator" for one semester first, then add rubric / feedback workflows. ⚠️ Any step involving student data or grading should be checked against the §Privacy + Ethics (Important) section below. + +### 3 Copy-Paste Prompt Templates + +**1. Lesson outline generator** (paste into Claude.ai): +``` +You are a [SUBJECT] teacher. I'm preparing a [DURATION]-minute class for +[GRADE] students on the topic "[TOPIC]". Prior knowledge: [SUMMARY]. +Produce: +1. Learning goals (3-4 bullets, use Bloom's taxonomy verbs) +2. Class outline with time allocation +3. 1 in-class activity / discussion prompt +4. 1 follow-up assessment item +Don't introduce content outside the topic I gave. +``` + +**2. Rubric draft**: +``` +I have a [ASSIGNMENT TYPE] for [GRADE] students on [TOPIC]. +Learning objectives: [2-3 bullets]. +Produce a 4-level rubric (Excellent / Proficient / Developing / Needs work) +with one paragraph per level across 4 dimensions: +content depth / organization / argumentation or calculation / clarity. +Make descriptions concrete and observable, not vague terms like "high quality". +``` + +**3. Student feedback synthesis**: +``` +Below are [N] student submission excerpts: +[PASTE TEXT] + +Please: +1. Summarize 3 common strengths across this batch +2. Summarize 3 common weaknesses +3. For the most common weakness, suggest 1-2 things to reinforce next class +Don't write per-student feedback — I'll do that myself. +``` + +## Privacy + Ethics (Important) + +Teachers using LLMs are different from regular users — **student data is involved**. Hard rules: + +- **Don't put student PII into public LLMs** (names, IDs, contact info, grades). Anonymize first ("Student A / B / C") +- **AI assistance ≠ AI grading**: drafting feedback / rubrics with LLM is fine, but **final grades require human judgment** — LLMs aren't reliable on complex evaluation yet +- **Disclose to students**: if class material is AI-assisted, disclose it (similar to declaring AI tool use in papers). Teaching integrity matters +- **Fact-check**: LLMs hallucinate citations, scholar names, research data. Domain content **must be verified** before class +- **Student work copyright**: don't bulk-upload student writing to third-party services for analysis — it may involve local privacy law, school policy, and third-party service terms. In the **United States**, also consider FERPA (student record protection); in the **European Union**, GDPR; and in **Taiwan**, the Personal Data Protection Act and school notices. Actual applicability depends on local law and school IT policy + +If your school / institution has an AI policy, **that takes priority** over this guide. + +## Tier Recommendations for Teachers + +Recommended progression. Most teachers should stay at Tier 0-1: + +| Tier | Tools | Best for | Learning cost | +|---|---|---|---| +| **Tier 0** | Claude.ai web chat | Occasional lesson prep, one-off tasks, item generation, writing emails. Copy the prompt template above and fill in the topic. | 0 (if you can use a browser) | +| **Tier 1** | Claude Desktop / [NotebookLM](https://notebooklm.google.com/) | Grading / organizing a semester's data, course mapping, bulk-importing reading list PDFs and querying them | 30 minutes | +| **Tier 2+** | Claude Code / CLI / SDK | Repeated automation, such as 30 student submissions every week → auto-generated draft feedback | 1 week; non-coders can ask school IT / a student RA to set it up | + +> **Most teachers can stop at Tier 0-1**. Once you're at Tier 2+, follow [Track A — CLI Power User](../tracks/cli/A1-cli-intro.en.md). + +## Other Branches Also Apply + +Many teachers are also researchers / knowledge workers. These branches overlap: + +- **Also doing research** (lit review, paper writing, references) → [Researcher branch](./for-researcher.en.md) +- **Reports / meeting notes / cross-tool integration** (Notion, Excel, email) → [Knowledge Worker branch](./for-knowledge-worker.en.md) +- **Connect AI to Notion / Obsidian / Lark / etc.** → [`resources/mcp-skills-catalog.en.md`](../resources/mcp-skills-catalog.en.md) + +## Community Note + +This branch is the smallest curated section currently. Contributions especially welcome: + +- Lesson plan generation skills +- Subject-specific prompt libraries (literature teacher's prompts, math teacher's prompts, language teacher's prompts...) +- Teacher-specific MCP servers (gradebook integrations, LMS connections like Canvas / Moodle / Google Classroom) +- **Subject + grade-level case studies** (e.g., "I used AI to teach middle-school math for a semester — here's my workflow") + +See [CONTRIBUTING.md](../CONTRIBUTING.md). diff --git a/branches/for-teacher.md b/branches/for-teacher.md new file mode 100644 index 0000000..4985696 --- /dev/null +++ b/branches/for-teacher.md @@ -0,0 +1,224 @@ +# 教師延伸路線（For Teachers / Educators） + +> **繁體中文** | [简体中文](./for-teacher.zh-Hans.md) | [English](./for-teacher.en.md) + +> 🚀 **大多數教師可直接從 Claude.ai（網頁版）+ NotebookLM 開始、不需要任何 setup**。只有當你要自動化重複流程（Tier 2+、例如每週生成 50 份家長信）時、才需要看 [`resources/setup-guide.md` A-C](../resources/setup-guide.md)（30 分鐘從零裝好需要的東西）。 + +> [← 回主路線 README](../README.md) · 走完 **Track A 的 A3** 或 **Track B 的 Stage 7** 後從這裡接續。把 agentic AI 應用到教學流程上。 + +## 使用情境 + +教師使用 AI 的情境可以先看成三個分支：**備課與上課素材製作**、**教學現場與學習輔助**、以及**其他應用場景**。 + +這樣的分類參考 AI in Education 文獻中常見的行政、教學與學習應用脈絡、也加入生成式 AI 在教材生成、回饋與互動支援上的近期討論（Chen et al., 2020；Mittal et al., 2024）。閱讀時建議先理解教師把關原則與使用邊界、再依自己的教學需求挑一個分支深入。 + +![教師與 AI agent 使用情境總覽](../resources/diagrams/teacher-ai-use-cases-overview.jpg) + +### 教師使用 AI 輔助時要注意什麼 + +AI 可以幫忙準備和輔助，但不應該直接取代教師判斷。近期 AI in Education 與生成式 AI 教育研究也提醒，教師設計 AI agent 時要保留清楚的教學目標、安全邊界與人工把關（Chen et al., 2020；Mittal et al., 2024）。 + +- **保留教師最後判斷**：牽涉學生資料、成績、教學決策等重大判斷時，教師仍要負責最後確認。 +- **避免直接給答案**：如果要讓學生與 AI agent 互動，可以設計成蘇格拉底式對話，在多輪互動中引導學生說出理由。 +- **貼合教學目標**：用固定提示詞、檢查清單、或學校核准的工具設定、限制 AI 的角色與任務、避免學生互動脫離課程目標。 +- **調整學生提問**：如果學生年齡較低，例如國小或國中，可以把學生問題先改寫成更清楚的提問，再交給 agent 回答。 + +### 備課與上課素材製作 + +這類情境偏向「幫老師準備材料」，輸出通常會被老師再改寫、挑選、檢查。 + +- **教案生成**：依課綱、單元目標與學生程度，整理課程大綱、時間分配、活動設計、討論提示與補充學習指南。 +- **Quiz / 評分量表（rubric）建立**：依文本、課文或學術文章，產生選擇題、簡答題、申論題、參考答案與評分規準。 +- **投影片準備、課程地圖、多媒體與視覺化素材**：把課本章節或教師筆記轉成投影片大綱、講義架構、週次安排、先備知識、評量節點、圖像、3D 物件、影片腳本、GIF 或課堂展示素材。 +- **學生回饋整理分析**：彙整學生作答、作業或課堂反應，找出常見迷思、需要補救的概念與下一步練習。 +- **多語系教材翻譯與轉化**：把教材改寫或翻譯成不同語言版本，也可以產生語音合成素材。 +- **互動式遊戲與活動、虛擬模擬場景的素材**：準備教學遊戲、押韻兒歌、任務卡、角色卡、情境文本或模擬場景背景；若要設計實際互動流程或課堂活動，請參考下一節「教學現場與學習輔助」。 + +### 教學現場與學習輔助 + +![教學現場與學習輔助應用場景](../resources/diagrams/teacher-ai-classroom-use-cases.jpg) + +這類情境偏向「幫學生理解、練習、互動」，AI 比較像教學助教或活動輔助工具。特別注意：不需要在單一教學活動中加入所有要素，而是挑選適合的環節加入 AI agent 設計。 + +- **沉浸式學習體驗與真實情境演練**：用真實情境模擬、角色扮演或外語口說模擬，讓學生在接近實作的情境中練習，降低認知負荷與退縮感。 +- **激發好奇心與提問能力**：透過蘇格拉底式追問與多輪互動，引導學生提出更清楚的問題、說明理由，進一步訓練批判性思考與後設認知。 +- **即時批改與深度回饋**：讓學生從錯誤中學習，AI 可以指出錯誤、說明原因、建議修正方向，而不是只給分數或答案。 +- **智慧家教與虛擬助教**：協助回答提問、解釋術語、給提示，讓學生在課堂內外都能獲得適度支援。 +- **適性教學與動態路徑**：依學生程度提供對應難度內容，並透過學習表現推測近側發展區，提供合適的鷹架與補救素材。 + +### 其他應用場景 + +這類情境不一定直接發生在課堂中，但會影響教師工作、學生支援與教育系統運作。 + +- **特殊教育支援**：透過語音轉文字、文字轉語音等方式，協助不同需求的學生參與課程。 +- **親師溝通與家庭教育**：整理學生進度報告，並提供家庭可延伸的輔助學習活動建議。 +- **行政管理與學術誠信**：整理學習軌跡、產生報告，或協助進行抄襲與作弊風險檢查。 +- **職涯與技能發展輔導**：協助職涯探索、培訓清單規劃，並依弱點推薦練習題。 +- **教師專業發展**：摘要教學方法、教育科技趨勢與研究重點，協助教師持續更新。 +- **高階研究分析**：輔助文獻分析、快速理解論文研究中提出的教學法或教育心理學。 +- **隱私保護與合成資料**：在不直接使用真實個資的前提下，產生匿名合成資料。 + +### 參考文獻 + +- Chen, L., Chen, P., & Lin, Z. (2020). [Artificial Intelligence in Education: A Review](https://doi.org/10.1109/ACCESS.2020.2988510). *IEEE Access*, 8, 75264-75278. +- Mittal, U., Sai, S., Chamola, V., & Sangwan, D. (2024). [A Comprehensive Review on Generative AI for Education](https://doi.org/10.1109/ACCESS.2024.3468368). *IEEE Access*, 12, 142733-142759. + +## 精選 Projects + +### 教學流程 Skills + +（大多數還沒有做成 skill marketplace。這個分支最有社群貢獻空間——見 CONTRIBUTING.md。） + +### 可用的基礎元件 + +#### [obra/superpowers](https://github.com/obra/superpowers) ⭐⭐⭐⭐ +通用的寫作 / 腦力激盪 skill。可改用在備課上。 + +#### 進階自動化：[Claude Code](https://github.com/anthropics/claude-code)（搭配自訂 CLAUDE.md）⭐⭐⭐⭐⭐ +★ 120k+ — **教師的基礎工具是 Claude.ai（網頁版）+ NotebookLM + Google Classroom / LMS 整合**、先從這裡開始。**只有當你已有會重複跑的批次流程**（如每週生成 50 份家長信、每學期跑學生反饋分析）才升級到 Claude Code、需要學一點 CLI。 + +### 教學課程素材（給教師備課用） + +#### [huggingface/agents-course](https://github.com/huggingface/agents-course) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 28k+ | +| License | Apache-2.0 | + +**教什麼**：Hugging Face 官方的 agent 課程——notebook、練習、結業認證。是一份**現成的「AI agent 教學」素材**。 + +**適合誰**：要在學校 / 工作坊開「AI agent 入門」課程的老師，可以直接拿來當教材或改編。 + +**備註**：注意這是「教 AI agent 怎麼建」的教材，不是「老師用 AI 教書」的工具。 + +--- + +#### [datawhalechina/llm-universe](https://github.com/datawhalechina/llm-universe) ⭐⭐⭐⭐（中文） + +| 欄位 | 內容 | +|---|---| +| 語言 | 中文（zh-Hans） | +| Stars | ★ 13k+ | +| License | NOASSERTION | + +**教什麼**：Datawhale 出品的中文 LLM 應用開發課程——含 RAG、agent、章節練習。中文教師備課的現成模板。 + +**適合誰**：中文教師要找現成可改的 LLM 教材底稿、再針對自己學生程度調整。 + +**備註**：跟 hf agents-course 一樣，是「教學生建 LLM 應用」的教材，不是「教師端的 AI 助教」。 + +--- + +### Prompt 素材庫 + +#### [f/awesome-chatgpt-prompts](https://github.com/f/awesome-chatgpt-prompts) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 161k+ | +| License | NOASSERTION（CC0 / public domain 風格，但未提供 SPDX） | + +**教什麼**：社群維護的 prompt 大全——「act as X」型樣板涵蓋幾百種角色（老師、面試官、stand-up comedian、辯論者⋯）。教師可以拿來當「prompt 寫法範例」教給學生，或直接借用其中合適的當作課堂示範。 + +**適合誰**：要教學生「prompt engineering」的老師，找現成例子比較不同寫法的差異。 + +**備註**：品質不一致——當作素材庫挑選用，不是「全部直接拿去教」。 + +--- + +### 閱讀材料 + +#### [The Effortless Academic — Beginner Guides](https://effortlessacademic.com/claude-code-and-cowork-for-academics-beginner-guide-part-1/) +寫給學術工作者導入 Claude Code 的多篇指南，教師也適用。 + +## 可以建的流程（按教學階段） + +下表 5 條是模板——配合你的學科自行調整： + +| 階段 | 流程 | 怎麼做（≤ 3 步） | 推薦工具 | 注意 | +|---|---|---|---|---| +| **備課前** | 教案生成器 | (1) 課綱 + 主題提示 → 大綱
(2) 大綱 → 投影片
(3) 投影片 → 評量題目 | Claude.ai / NotebookLM | 教師最後審 | +| **備課中** | Rubric 建立 | (1) 給學生作業樣本 + 學習目標
(2) 請 AI 草擬 4 級 rubric
(3) 教師調整級距 | Claude.ai | 避免「品質好」這種模糊詞 | +| **改作業** | 個別化回饋 | (1) 學生作業 + rubric → AI 寫回饋初稿
(2) 教師逐份審 + 改
(3) 寄回 | Claude.ai | **AI 輔助 ≠ AI 評分**，最終分數一定人工 | +| **課堂活動** | 情境模擬 | (1) 教學目標 + 角色設定 → 對話腳本
(2) 課堂演練
(3) 反思問題 | Claude.ai | 蘇格拉底式追問、不直接給答案；學生輸入**不含個資** | +| **課後補救** | 個別化補救教材 | (1) 整理學生常見錯誤
(2) 依學生程度 → 小練習 + 提示
(3) 延伸挑戰題 | Claude.ai | 注意學生個資匿名化 | + +> 💡 **新手起手式**：先做「備課前的教案生成器」一個學期、習慣後再加 rubric / 回饋流程。⚠️ 所有跟學生個資 / 評分相關的步驟都要回頭看下面的 §隱私 + 倫理（重要）章節。 + +### 3 個可直接複製的 prompt 範本 + +**1. 教案大綱生成**（複製到 Claude.ai 即可用）： +``` +你是一位 [學科] 老師。我要給 [年級] 學生上一堂 [時長] 分鐘的課，主題是「[主題]」。 +學生先備知識：[簡述]。請產出： +1. 學習目標（3-4 條，用 Bloom's taxonomy 動詞） +2. 課程大綱（含時間分配） +3. 1 個課堂活動 / 討論題 +4. 1 個課後評量題 +不要產生超出我給的主題範圍的內容。 +``` + +**2. Rubric 草稿生成**： +``` +我有一份 [作業類型] 作業，學生年級 [年級]，主題 [主題]。 +學習目標：[列 2-3 條]。 +請產出一份 4 級 rubric（卓越 / 熟練 / 發展中 / 待改進）， +每級在「內容深度」「組織結構」「論證 / 計算」「表達清晰度」4 個面向各給一段描述。 +描述要具體可觀察，不用「品質好」這種模糊詞。 +``` + +**3. 學生回饋整理**： +``` +以下是 [N] 份學生作業片段： +[貼上文本] + +請： +1. 摘要這批作業共同的 3 個強項 +2. 摘要 3 個共同弱點 +3. 針對最常見弱點，建議 1-2 個下次上課該加強的環節 +不要做個別化評語——我會自己針對個人寫。 +``` + +## 隱私 + 倫理（重要） + +教師端用 LLM 跟一般 user 不同，**牽涉學生資料**——以下是 hard rule： + +- **不要把學生個資丟進公開 LLM**（姓名、學號、聯絡方式、成績）。需要的話先匿名化（用「學生 A / B / C」） +- **AI 輔助 ≠ AI 評分**：用 LLM 草擬回饋 / rubric 沒問題，但**最終評分一定要人工把關**——LLM 對複雜思考的評估還不可靠 +- **告知學生**：如果課堂材料是 AI 輔助生成，建議向學生揭露（比照論文揭露 AI 工具使用）。教學誠信很重要 +- **檢查事實**：LLM 會編造引用、學者名字、研究資料。專業領域內容**必須核對**才能上課 +- **學生作品的著作權**：不要把學生作品用 LLM 大量分析後上傳到第三方 service、**可能涉及所在地個資法、學校政策、第三方服務條款**——在**美國**另需留意 FERPA（學生紀錄保護法）、在**歐盟**需留意 GDPR、在**台灣**則需注意《個資法》與校方公告。實際適用範圍請以該地法規與學校 IT 政策為準 + +如果你的學校 / 機構有 AI 使用政策，**那份比這份優先**。 + +## 給教師的層級建議 + +下表是建議的進階路徑——大多數教師應該停在 Tier 0-1： + +| Tier | 工具 | 適合誰 | 學習成本 | +|---|---|---|---| +| **Tier 0** | Claude.ai 網頁版聊天 | 偶爾備課、單次任務、出題、寫信。複製上面的 prompt 範本填入主題即可 | 0（會用瀏覽器就行） | +| **Tier 1** | Claude Desktop / [NotebookLM](https://notebooklm.google.com/) | 批改 / 整理一整學期資料、做課程地圖、整批匯入課本 PDF 後問問題 | 半小時裝好 | +| **Tier 2+** | Claude Code / CLI / SDK | 有重複自動化需求（例：每週收 30 份作業 → 自動生成回饋初稿） | 1 週上手；不熟程式可找學校 IT / 學生 RA 幫忙設定 | + +> **多數教師停在 Tier 0-1 就夠了**。升級到 Tier 2+ 就建議走 [Track A — CLI Power User](../tracks/cli/A1-cli-intro.md)。 + +## 也適用其他分支 + +很多老師同時是研究員 / 知識工作者，這幾個分支重疊： + +- **也做研究**（找文獻、寫 paper、整理 references）→ [研究員分支](./for-researcher.md) +- **要寫報告 / 整理會議記錄 / 跨工具整合**（Notion、Excel、Email）→ [知識工作者分支](./for-knowledge-worker.md) +- **要把 AI 接到 Notion / Obsidian / 飛書** 等日常工具 → [`resources/mcp-skills-catalog.md`](../resources/mcp-skills-catalog.md) + +## 社群備註 + +這個分支目前是精選內容最少的一塊。特別歡迎以下貢獻： + +- 教案生成 skill +- 學科專屬的 prompt library（國文老師的 prompts、數學老師的 prompts、英文老師的 prompts ⋯） +- 教師專屬的 MCP server（成績冊整合、LMS 串接如 Canvas / Moodle / Google Classroom） +- **某學科 + 某年級的完整 case study**（例如「我用 AI 帶國中數學一個學期，這是我的 workflow」） + +請見 [CONTRIBUTING.md](../CONTRIBUTING.md)。 diff --git a/branches/for-teacher.zh-Hans.md b/branches/for-teacher.zh-Hans.md new file mode 100644 index 0000000..b449eff --- /dev/null +++ b/branches/for-teacher.zh-Hans.md @@ -0,0 +1,224 @@ +# 教师延伸路线（For Teachers / Educators） + +> [繁體中文](./for-teacher.md) | **简体中文** | [English](./for-teacher.en.md) + +> 🚀 **大多数教师可直接从 Claude.ai（网页版）+ NotebookLM 开始，不需要任何 setup**。只有当你要自动化重复流程（Tier 2+，例如每周生成 50 份家长信）时，才需要看 [`resources/setup-guide.zh-Hans.md` A-C](../resources/setup-guide.zh-Hans.md)（30 分钟从零装好需要的东西）。 + +> [← 回主路线 README](../README.zh-Hans.md) · 走完 **Track A 的 A3** 或 **Track B 的 Stage 7** 后从这里接续。把 agentic AI 应用到教学流程上。 + +## 使用场景 + +教师使用 AI 的场景可以先看成三个分支：**备课与上课素材制作**、**教学现场与学习辅助**、以及**其他应用场景**。 + +这样的分类参考 AI in Education 文献中常见的行政、教学与学习应用脉络，也加入生成式 AI 在教材生成、反馈与互动支援上的近期讨论（Chen et al., 2020；Mittal et al., 2024）。阅读时建议先理解教师把关原则与使用边界，再依自己的教学需求挑一个分支深入。 + +![教师与 AI agent 使用场景总览](../resources/diagrams/teacher-ai-use-cases-overview.jpg) + +### 教师使用 AI 辅助时要注意什么 + +AI 可以帮忙准备和辅助，但不应该直接取代教师判断。近期 AI in Education 与生成式 AI 教育研究也提醒，教师设计 AI agent 时要保留清楚的教学目标、安全边界与人工把关（Chen et al., 2020；Mittal et al., 2024）。 + +- **保留教师最后判断**：牵涉学生数据、成绩、教学决策等重大判断时，教师仍要负责最后确认。 +- **避免直接给答案**：如果要让学生与 AI agent 互动，可以设计成苏格拉底式对话，在多轮互动中引导学生说出理由。 +- **贴合教学目标**：用固定提示词、检查清单、或学校核准的工具设置，限制 AI 的角色与任务，避免学生互动脱离课程目标。 +- **调整学生提问**：如果学生年龄较低，例如小学或初中，可以把学生问题先改写成更清楚的提问，再交给 agent 回答。 + +### 备课与上课素材制作 + +这类场景偏向“帮老师准备材料”，输出通常会被老师再改写、挑选、检查。 + +- **教案生成**：依课纲、单元目标与学生程度，整理课程大纲、时间分配、活动设计、讨论提示与补充学习指南。 +- **Quiz / 评分量表（rubric）建立**：依文本、课文或学术文章，产生选择题、简答题、申论题、参考答案与评分规准。 +- **幻灯片准备、课程地图、多媒体与可视化素材**：把课本章节或教师笔记转成幻灯片大纲、讲义架构、周次安排、先备知识、评估节点、图像、3D 对象、视频脚本、GIF 或课堂展示素材。 +- **学生反馈整理分析**：汇整学生作答、作业或课堂反应，找出常见迷思、需要补救的概念与下一步练习。 +- **多语系教材翻译与转化**：把教材改写或翻译成不同语言版本，也可以产生语音合成素材。 +- **互动式游戏与活动、虚拟模拟场景的素材**：准备教学游戏、押韵儿歌、任务卡、角色卡、情境文本或模拟场景背景；若要设计实际互动流程或课堂活动，请参考下一节“教学现场与学习辅助”。 + +### 教学现场与学习辅助 + +![教学现场与学习辅助应用场景](../resources/diagrams/teacher-ai-classroom-use-cases.jpg) + +这类场景偏向“帮学生理解、练习、互动”，AI 比较像教学助教或活动辅助工具。特别注意：不需要在单一教学活动中加入所有要素，而是挑选适合的环节加入 AI agent 设计。 + +- **沉浸式学习体验与真实情境演练**：用真实情境模拟、角色扮演或外语口说模拟，让学生在接近实作的情境中练习，降低认知负荷与退缩感。 +- **激发好奇心与提问能力**：透过苏格拉底式追问与多轮互动，引导学生提出更清楚的问题、说明理由，进一步训练批判性思考与后设认知。 +- **即时批改与深度反馈**：让学生从错误中学习，AI 可以指出错误、说明原因、建议修正方向，而不是只给分数或答案。 +- **智慧家教与虚拟助教**：协助回答提问、解释术语、给提示，让学生在课堂内外都能获得适度支援。 +- **适性教学与动态路径**：依学生程度提供对应难度内容，并透过学习表现推测近侧发展区，提供合适的鹰架与补救素材。 + +### 其他应用场景 + +这类场景不一定直接发生在课堂中，但会影响教师工作、学生支援与教育系统运作。 + +- **特殊教育支援**：透过语音转文字、文字转语音等方式，协助不同需求的学生参与课程。 +- **亲师沟通与家庭教育**：整理学生进度报告，并提供家庭可延伸的辅助学习活动建议。 +- **行政管理与学术诚信**：整理学习轨迹、产生报告，或协助进行抄袭与作弊风险检查。 +- **职涯与技能发展辅导**：协助职涯探索、培训清单规划，并依弱点推荐练习题。 +- **教师专业发展**：摘要教学方法、教育科技趋势与研究重点，协助教师持续更新。 +- **高阶研究分析**：辅助文献分析、快速理解论文研究中提出的教学法或教育心理学。 +- **隐私保护与合成数据**：在不直接使用真实个资的前提下，产生匿名合成数据。 + +### 参考文献 + +- Chen, L., Chen, P., & Lin, Z. (2020). [Artificial Intelligence in Education: A Review](https://doi.org/10.1109/ACCESS.2020.2988510). *IEEE Access*, 8, 75264-75278. +- Mittal, U., Sai, S., Chamola, V., & Sangwan, D. (2024). [A Comprehensive Review on Generative AI for Education](https://doi.org/10.1109/ACCESS.2024.3468368). *IEEE Access*, 12, 142733-142759. + +## 精选 Projects + +### 教学流程 Skills + +（大多数还没有做成 skill marketplace。这个分支最有社群贡献空间——见 CONTRIBUTING.md。） + +### 可用的基础组件 + +#### [obra/superpowers](https://github.com/obra/superpowers) ⭐⭐⭐⭐ +通用的写作 / 头脑风暴 skill。可改用在备课上。 + +#### 进阶自动化：[Claude Code](https://github.com/anthropics/claude-code)（搭配自定义 CLAUDE.md）⭐⭐⭐⭐⭐ +★ 120k+ — **教师的基础工具是 Claude.ai（网页版）+ NotebookLM + Google Classroom / LMS 集成**，先从这里开始。**只有当你已有会重复跑的批量流程**（如每周生成 50 份家长信、每学期跑学生反馈分析）才升级到 Claude Code，需要学一点 CLI。 + +### 教学课程素材（给教师备课用） + +#### [huggingface/agents-course](https://github.com/huggingface/agents-course) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 28k+ | +| License | Apache-2.0 | + +**教什么**：Hugging Face 官方的 agent 课程——notebook、练习、结业认证。是一份**现成的“AI agent 教学”素材**。 + +**适合谁**：要在学校 / 工作坊开“AI agent 入门”课程的老师，可以直接拿来当教材或改编。 + +**备注**：注意这是“教 AI agent 怎么建”的教材，不是“老师用 AI 教书”的工具。 + +--- + +#### [datawhalechina/llm-universe](https://github.com/datawhalechina/llm-universe) ⭐⭐⭐⭐（中文） + +| 栏位 | 内容 | +|---|---| +| 语言 | 中文（zh-Hans） | +| Stars | ★ 13k+ | +| License | NOASSERTION | + +**教什么**：Datawhale 出品的中文 LLM 应用开发课程——含 RAG、agent、章节练习。中文教师备课的现成模板。 + +**适合谁**：中文教师想找现成可改的 LLM 教材底稿、再针对自己学生程度调整。 + +**备注**：跟 hf agents-course 一样，是“教学生建 LLM 应用”的教材，不是“教师端的 AI 助教”。 + +--- + +### Prompt 素材库 + +#### [f/awesome-chatgpt-prompts](https://github.com/f/awesome-chatgpt-prompts) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 161k+ | +| License | NOASSERTION（CC0 / public domain 风格，但未提供 SPDX） | + +**教什么**：社群维护的 prompt 大全——“act as X”型模板涵盖几百种角色（老师、面试官、stand-up comedian、辩论者⋯）。教师可以拿来当“prompt engineering 写法示例”教给学生，或直接借用其中合适的当作课堂示范。 + +**适合谁**：要教学生“prompt engineering”的老师，找现成例子比较不同写法的差异。 + +**备注**：质量不一致——当作素材库挑选用，不是“全部直接拿去教”。 + +--- + +### 阅读材料 + +#### [The Effortless Academic — Beginner Guides](https://effortlessacademic.com/claude-code-and-cowork-for-academics-beginner-guide-part-1/) +写给学术工作者导入 Claude Code 的多篇指南，教师也适用。 + +## 可以建的流程（按教学阶段） + +下表 5 条是模板——配合你的学科自行调整： + +| 阶段 | 流程 | 怎么做（≤ 3 步） | 推荐工具 | 注意 | +|---|---|---|---|---| +| **备课前** | 教案生成器 | (1) 课纲 + 主题提示 → 大纲
(2) 大纲 → 幻灯片
(3) 幻灯片 → 评估题目 | Claude.ai / NotebookLM | 教师最后审 | +| **备课中** | Rubric 建立 | (1) 给学生作业样本 + 学习目标
(2) 请 AI 草拟 4 级 rubric
(3) 教师调整级距 | Claude.ai | 避免“质量好”这种模糊词 | +| **改作业** | 个性化反馈 | (1) 学生作业 + rubric → AI 写反馈初稿
(2) 教师逐份审 + 改
(3) 寄回 | Claude.ai | **AI 辅助 ≠ AI 评分**，最终分数一定人工 | +| **课堂活动** | 情境模拟 | (1) 教学目标 + 角色设定 → 对话脚本
(2) 课堂演练
(3) 反思问题 | Claude.ai | 苏格拉底式追问、不直接给答案；学生输入**不含个资** | +| **课后补救** | 个性化补救教材 | (1) 整理学生常见错误
(2) 依学生程度 → 小练习 + 提示
(3) 延伸挑战题 | Claude.ai | 注意学生个资匿名化 | + +> 💡 **新手起手式**：先做“备课前的教案生成器”一个学期，习惯后再加 rubric / 反馈流程。⚠️ 所有跟学生个资 / 评分相关的步骤都要回头看下面的 §隐私 + 伦理（重要）章节。 + +### 3 个可直接复制的 prompt 范本 + +**1. 教案大纲生成**（复制到 Claude.ai 即可用）： +``` +你是一位 [学科] 老师。我要给 [年级] 学生上一堂 [时长] 分钟的课，主题是「[主题]」。 +学生先备知识：[简述]。请产出： +1. 学习目标（3-4 条，用 Bloom's taxonomy 动词） +2. 课程大纲（含时间分配） +3. 1 个课堂活动 / 讨论题 +4. 1 个课后评估题 +不要产生超出我给的主题范围的内容。 +``` + +**2. Rubric 草稿生成**： +``` +我有一份 [作业类型] 作业，学生年级 [年级]，主题 [主题]。 +学习目标：[列 2-3 条]。 +请产出一份 4 级 rubric（卓越 / 熟练 / 发展中 / 待改进）， +每级在「内容深度」「组织结构」「论证 / 计算」「表达清晰度」4 个面向各给一段描述。 +描述要具体可观察，不用「质量好」这种模糊词。 +``` + +**3. 学生反馈整理**： +``` +以下是 [N] 份学生作业片段： +[贴上文本] + +请： +1. 摘要这批作业共同的 3 个强项 +2. 摘要 3 个共同弱点 +3. 针对最常见弱点，建议 1-2 个下次上课该加强的环节 +不要做个人化评语——我会自己针对个人写。 +``` + +## 隐私 + 伦理（重要） + +教师端用 LLM 跟一般 user 不同，**牵涉学生数据**——以下是 hard rule： + +- **不要把学生个资丢进公开 LLM**（姓名、学号、联系方式、成绩）。需要的话先匿名化（用“学生 A / B / C”） +- **AI 辅助 ≠ AI 评分**：用 LLM 草拟反馈 / rubric 没问题，但**最终评分一定要人工把关**——LLM 对复杂思考的评估还不可靠 +- **告知学生**：如果课堂材料是 AI 辅助生成，建议向学生揭露（比照论文揭露 AI 工具使用）。教学诚信很重要 +- **检查事实**：LLM 会编造引用、学者名字、研究数据。专业领域内容**必须核对**才能上课 +- **学生作品的著作权**：不要把学生作品用 LLM 大量分析后上传到第三方 service，**可能涉及所在地个资法、学校政策、第三方服务条款**——在**美国**另需留意 FERPA（学生记录保护法）、在**欧盟**需留意 GDPR、在**台湾**则需注意《个资法》与校方公告。实际适用范围请以该地法规与学校 IT 政策为准 + +如果你的学校 / 机构有 AI 使用政策，**那份比这份优先**。 + +## 给教师的层级建议 + +下表是建议的进阶路径——大多数教师应该停在 Tier 0-1： + +| Tier | 工具 | 适合谁 | 学习成本 | +|---|---|---|---| +| **Tier 0** | Claude.ai 网页版聊天 | 偶尔备课、单次任务、出题、写信。复制上面的 prompt 范本填入主题即可 | 0（会用浏览器就行） | +| **Tier 1** | Claude Desktop / [NotebookLM](https://notebooklm.google.com/) | 批改 / 整理一整学期数据、做课程地图、整批导入课本 PDF 后问问题 | 半小时装好 | +| **Tier 2+** | Claude Code / CLI / SDK | 有重复自动化需求（例：每周收 30 份作业 → 自动生成反馈初稿） | 1 周上手；不熟程序可找学校 IT / 学生 RA 帮忙设置 | + +> **多数教师停在 Tier 0-1 就够了**。升级到 Tier 2+ 就建议走 [Track A — CLI Power User](../tracks/cli/A1-cli-intro.zh-Hans.md)。 + +## 也适用其他分支 + +很多老师同时是研究员 / 知识工作者，这几个分支重叠： + +- **也做研究**（找文献、写 paper、整理 references）→ [研究员分支](./for-researcher.zh-Hans.md) +- **要写报告 / 整理会议记录 / 跨工具集成**（Notion、Excel、Email）→ [知识工作者分支](./for-knowledge-worker.zh-Hans.md) +- **要把 AI 接到 Notion / Obsidian / 飞书** 等日常工具 → [`resources/mcp-skills-catalog.zh-Hans.md`](../resources/mcp-skills-catalog.zh-Hans.md) + +## 社群备注 + +这个分支目前是精选内容最少的一块。特别欢迎以下贡献： + +- 教案生成 skill +- 学科专属的 prompt library（语文老师的 prompts、数学老师的 prompts、英文老师的 prompts ⋯） +- 教师专属的 MCP server（成绩册集成、LMS 串接如 Canvas / Moodle / Google Classroom） +- **某学科 + 某年级的完整 case study**（例如“我用 AI 带初中数学一个学期，这是我的 workflow”） + +请见 [CONTRIBUTING.md](../CONTRIBUTING.md)。 diff --git a/docs/HOW_TO_USE.md b/docs/HOW_TO_USE.md new file mode 100644 index 0000000..f1b4651 --- /dev/null +++ b/docs/HOW_TO_USE.md @@ -0,0 +1,155 @@ +# How to use this curriculum — 主動 vs 被動學習 + +> 給每個動手練習 folder 的 meta-instruction。如果你跳過這一頁、會把這套教材當 reference book 讀完、學到大概 60%。讀完這一頁、用對方法、學到 100%。 + +## 真實問題 + +每個練習 folder（譬如 `examples/stage-3/03-react-from-scratch/`）裡都有一個 **`starter.py`**——它**長得像 starter、其實是完整解答**。 + +如果你： + +```bash +git clone ... && cd examples/stage-3/03-react-from-scratch +cat starter.py # 看完整解答 +python starter.py # 跑通 +python test.py # 全 pass +``` + +你會以為「學會了」、其實**沒寫過一行 code**。 + +這是這份教材的最大設計缺陷。下面講怎麼繞過它。 + +## 兩種學習模式 + +### 🟢 主動模式（推薦、學到 100%） + +**步驟**： + +```bash +cd examples/stage-3/03-react-from-scratch/ + +# 1. 讀 README、了解這題在做什麼、預期 input / output +cat README.md + +# 2. 把 starter.py 改名（藏起來、等下對照用） +mv starter.py starter_reference.py + +# 3. 看 starter_reference.py 的「imports + function signatures」、不看 function body +head -50 starter_reference.py + +# 4. 自己寫一個新的 starter.py、function body 自己想 +$EDITOR starter.py + +# 5. 跑 test.py、看自己寫的能不能 pass +python test.py + +# 6. 卡住超過 20 分鐘？才打開 starter_reference.py 對照 +diff starter.py starter_reference.py + +# 7. 寫完一輪後、看 README 的 punchline + common pitfalls、跟你的 trial 對照 +``` + +**重點**： +- **看 signature、不看 body**。imports / TOOLS_SPEC / function names + arg types 可以看；裡面怎麼實作要自己想。 +- **卡 20 分鐘是健康的**。卡 1 小時也健康。卡 3 小時就回去看 reference、然後**默寫一遍**。 +- **test 通過 ≠ 學會**。test 通過代表 logic 對；學會代表你**講得出**為什麼這 13 行 ReAct loop 必要、為什麼 `tool_call_id` 要配對、為什麼要 `max_iter`。 + +### 🟡 被動模式（reference book、學到 60%） + +**步驟**： + +```bash +cd examples/stage-3/03-react-from-scratch/ +cat README.md +cat starter.py # 讀完整解答、理解每一行 +python test.py # 確認跑得起來 +``` + +**何時用**： +- 你**之前寫過 ReAct loop**、現在只是想看本 curriculum 是怎麼寫的、做 cross-reference +- 你在**找 pitfall reference**（譬如 production 出 bug、想看 curriculum 提過沒） +- 你是**講課老師**、要快速看完整套教材然後挑題目給學生 + +被動模式適合**已經會了**的人複習、不適合**沒寫過**的人入門。 + +## 為什麼這份教材的 starter.py 是完整解答（不是 TODO skeleton） + +短答：**v1 階段、為了快速 ship 完整可跑版本**。 + +長答：完整 starter.py 有 3 個好處（給維護者）： +1. **test 直接 pass**——確認 framework 整合沒漏東西 +2. **不會 outdated**——隨 framework 升級可以馬上 fix（不必同步維護 template） +3. **新手 onboard 快**——把 repo clone 下來就能跑、降低裝環境 friction + +但對學習者來講有 1 個大缺點：**容易被誤用成抄答案**。所以這份 HOW_TO_USE 文件存在、提醒你**自己改名、自己重寫**。 + +**v2 規劃**（未開始）：把 starter.py 分裂成 `starter_template.py`（TODO skeleton）+ `starter_reference.py`（完整解答）、test 預設打 template、學生 fill in、卡住才看 reference。這需要重做 ~20 個 folder、預計 v2 在 [`docs/TESTING_PLAN.md`](TESTING_PLAN.md) 之後排期。 + +## 每個 stage 怎麼用這份教材 + +| Stage | 主動模式時間預算 | 被動模式時間預算 | +|---|---|---| +| Stage 3（tool use + ReAct） | 5-8 hr（每練習 1-1.5 hr） | 1-2 hr（讀過去） | +| Stage 4（agent frameworks） | 8-12 hr（每練習 2 hr） | 2-3 hr | +| Stage 6（RAG + memory） | 8-12 hr | 2-3 hr | +| Stage 7（production） | 10-15 hr | 3-4 hr | + +**主動模式時間是被動的 4-5 倍**——這就是「卡住 + 修通」的時間成本、也是真學會的成本。如果你只有 1 週時間、選 1-2 個你覺得最重要的練習走**主動模式**、其他**被動模式**過。 + +## 我自己（curriculum 作者）跑驗證踩到的 bug + +跑 verification（2026-05-13）發現我寫的 starter / test **本身有 6 個 bug**： + +1. **operator precedence** in test (`and` 比 `or` 緊) +2. **ChromaDB collection name length** (Chroma 1.0 break、'kb' 太短) +3. **EphemeralClient state leak** 跨 test fixture +4. **i18n key mismatch**（test 用中文 query、starter db 用英文 key） +5. **Smolagents `@tool` 要求 Google-style docstring `Args:`** 區塊 +6. **Python 3.14 + tiktoken/regex 無 wheel**（CrewAI 在 3.14 裝不起來） + +**這對你的意義**：當你做主動模式、卡住時，**有可能不是你錯、是教材有 bug**。提 issue 上來、我會修。Bug 修在 commit [50c3bf8](https://github.com/WenyuChiou/awesome-agentic-ai-zh/commit/50c3bf8)。 + +## 練習 checkpoint（每練習做完問自己這 3 題） + +不要光看 starter.py 過去、問自己： + +1. **「為什麼」**：這份 code 為什麼這樣寫、不那樣寫？（譬如 ReAct loop 為什麼必須把 assistant response 接回 messages？沒接會怎樣？） +2. **「拿掉 X 會怎樣」**：拿掉 `max_iter`、拿掉 `tool_call_id`、拿掉 `cache_control`，runtime 會出什麼問題？ +3. **「production 怎麼改」**：這份 demo code 上 production 還缺什麼？（提示：observability / eval / retry / auth 通常都缺） + +回答得出來 = 真學會了。回答不出來 = 只是讀過。 + +## 進入條件：每個 Stage 開始前自我檢查 + +不要直接從 Stage 4 開始——除非 Stage 3 的 6 個練習你**每個都用主動模式寫過 1 次**。 + +- **Stage 4** 前：必須能不查文件寫出 13 行 ReAct loop（Stage 3 練習 3） +- **Stage 6** 前：必須能講出為什麼 schema 要寫 enum + required（Stage 3 練習 6） +- **Stage 7** 前：必須會用 mock 寫 LLM unit test（Stage 3 練習 5 + 任何 Stage 4） + +沒過 checkpoint 直接跳級、後面會卡住、回頭重做更慢。 + +## 如果你卡住 + +順序： + +1. **再讀一次 README 的 pitfall + punchline** — 80% 的卡住來自漏看某個關鍵設計 +2. **打開 `examples/stage-5/tool-calling-tutor/` skill**（裝進 Claude Code）— tool calling 相關的卡住、4-symptom triage 帶你診斷 +3. **看 `starter_reference.py`**（你改名藏起來的那個）— 對照你寫的差別、找出哪裡邏輯漏 +4. **看 GitHub issue** 有沒有人問過 +5. **開 issue** — 帶上你的 code + 你看到的錯誤、我會回 + +絕對不要：抄 `starter_reference.py` 就走。沒寫過 = 沒學會。 + +--- + +## 給維護者：v2 path + +v2 把 starter.py 拆成 template + reference 的計畫： + +- 每個 folder 多 2 個檔案：`starter_template.py`（TODO skeleton）+ `starter_reference.py`（answer） +- `test.py` 預設打 `starter_template.py`、有 env var 切到 reference 對照 +- README 多 1 段 "Learning mode" 解釋 +- 約 20 個 folder × 3 file changes = 60 個檔案 + +如果有人想接 v2、歡迎 PR。對應 issue / branch 等決定後開出來。 diff --git a/docs/TESTING_PLAN.md b/docs/TESTING_PLAN.md new file mode 100644 index 0000000..ff84d8f --- /dev/null +++ b/docs/TESTING_PLAN.md @@ -0,0 +1,114 @@ +# Testing Plan — T3+ Verification Log + +> Updated 2026-05-13. Verification is **done**; this doc is now a historical log. +> The branch `t3-stage-4-6-7-unverified` referenced in earlier versions has been +> fully merged into `main` and deleted. + +## ✅ Final state (everything on `main`) + +| Batch | What | How verified | Bugs fixed | +|---|---|---|---| +| Phase 3 — Stage 1 + 3 folder renames (6 folders) | `starter.py` (Ollama) / `starter_anthropic.py` / both test suites | `python test.py` + `python test_anthropic.py` per folder | 0 | +| Phase A — `stages/03-tool-use-and-hello-agent.md` inline `
` (練習 2-6) | 5 simplified inline blocks + zh-Hans drift | `wc -l` parity, `grep` no residual Trad chars | 0 | +| Phase B — `examples/stage-5/tool-calling-tutor/` skill | SKILL.md + 3 references + evals + trilingual READMEs | YAML frontmatter parses; evals.json valid JSON | 0 (live skill-install test still pending) | +| Phase C — cross-references | stages/03 + stages/05 + CLAUDE.md links | `grep -c` confirms 10 references across 7 files | 0 | +| **Stage 4 (5 ex)** | LangGraph + CrewAI + LangGraph workflow + Smolagents + Pydantic AI | 8/8 test suites verified green; ex2 CrewAI install-blocked on Python 3.14 (tiktoken/regex wheels) — code shipped unmodified | 3 (i18n key mismatch in ex3 + Smolagents docstring `Args:` requirement in ex4 + Pydantic AI version fallback in ex5 test) | +| **Stage 6 (5 ex)** | embeddings + ChromaDB + chunking + full RAG + long-term memory | 10/10 test suites verified green | 2 (ChromaDB `kb` collection name too short for Chroma 1.0+; `EphemeralClient` state leak across test fixtures) | +| **Stage 7 (5 ex)** | multi-agent debate + eval + observability + streaming/caching + FastAPI deploy | 10/10 test suites verified green | 1 (operator precedence: `and` binds tighter than `or` in fake_agent dispatcher) | + +**Total: 28/30 test files run green** + 1 install caveat (CrewAI on Python 3.14) + 1 pending live test (skill auto-load). + +**Total bugs fixed**: 6 — all in commit [`50c3bf8`](https://github.com/WenyuChiou/awesome-agentic-ai-zh/commit/50c3bf8). + +## 🟢 Pedagogy v1 also shipped (2026-05-13) + +Recognized late in the session: every `starter.py` is a **complete solution**, not a TODO skeleton. A learner who clones and runs `python test.py` passes without writing any code. + +v1 fix (doc-only, no code rename): +- `docs/HOW_TO_USE.md` — full active-vs-passive learning method (~200 lines, zh-TW) +- 22 exercise READMEs — 🎓 callout pointing to `mv starter.py starter_reference.py` shortcut + link to HOW_TO_USE +- Main README × 3 langs — surface the meta-instruction at the top-level + +Shipped in commits [`d598e37`](https://github.com/WenyuChiou/awesome-agentic-ai-zh/commit/d598e37) + [`2cf99fe`](https://github.com/WenyuChiou/awesome-agentic-ai-zh/commit/2cf99fe). + +## ⚠ Known caveats still on `main` + +1. **CrewAI exercise (Stage 4 ex2)** not tested on Python 3.14 — tiktoken + regex don't have wheels yet. Code shipped unchanged; users on Python 3.11/3.12/3.13 should be fine. Document at top of `examples/stage-4/02-multi-agent-roles/README.md` if needed for future learners. + +2. **tool-calling-tutor skill** not live-tested in Claude Code — only structural validation (YAML frontmatter parse + JSON evals validate). Manual install test: `cp -r examples/stage-5/tool-calling-tutor/{SKILL.md,references,evals} ~/.claude/skills/tool-calling-tutor/`, restart Claude Code, prompt 「為什麼 LLM 不呼叫我的 tool」. + +3. **starter.py = complete solution pedagogy gap** — flagged in `docs/HOW_TO_USE.md`. v2 would split into `starter_template.py` (TODO) + `starter_reference.py` (solution); v1 is doc-only meta-instruction. + +4. **Trilingual mirror of 🎓 callout incomplete** — v1 only added the 學習模式 callout to zh-TW READMEs. en + zh-Hans exercise READMEs still need the same callout. Low priority since most learners use zh-TW. + +5. **Pilot exercise drift** (pre-session, still open) — `examples/stage-3/03-react-from-scratch/README.en.md` + `.zh-Hans.md` are pre-dual-path; the zh-TW canonical is current. Stage 3 polish pass should fix. + +## 🔵 Stage 5 + Track A — current coverage + +### Track A1-A3 CLI track — **outline complete, no `examples/` folder by design** + +12 hands-on exercises documented across `tracks/cli/A{1,2,3}-*.md` × 3 langs (zh-TW canonical ~367 lines): + +| File | Lines (zh-TW) | Exercises | +|---|---|---| +| `tracks/cli/A1-cli-intro.md` | 107 | CLI-1 安裝 + 第一次跑 / CLI-2 CLAUDE.md / CLI-3 第二個 CLI 並用 / CLI-4 認證細節 | +| `tracks/cli/A2-cli-workflow.md` | 126 | CLI-5 production CLAUDE.md / CLI-6 slash command / CLI-7 多步驟拆解 / CLI-8 portable prompt | +| `tracks/cli/A3-cli-production.md` | 134 | CLI-9 MCP server 接 CLI / CLI-10 GitHub Actions / CLI-11 cost tracking / CLI-12 plugin 跨 team 分享 | + +**No `examples/track-a/` folder built — and this is intentional**. CLI exercises are: +- Bash commands (`ollama pull`, `claude` install, MCP-server install) +- Markdown authoring (CLAUDE.md, slash command `.md` files, SKILL.md) +- YAML / JSON config (GitHub Actions `.yml`, `plugin.json`, `marketplace.json`) +- **Not Python SDK code**, so the dual-path Ollama/Anthropic `starter.py` + `test.py` pattern doesn't apply. + +What learners do for Track A: follow each numbered exercise in the outline doc, on their own real repo (their work codebase, not a sample). The `tracks/cli/A*.md` files contain success criteria for self-check. + +**Core reference**: [`resources/cli-agents-guide.md`](../resources/cli-agents-guide.md) (148 lines) — 7-CLI comparison + decision rubric + common pitfalls. + +**Potential v2** (not committed): could ship `examples/track-a/` containing sample CLAUDE.md / `.claude/commands/review.md` / sample GHA workflow yml. Low priority — current outline is self-contained. + +### Stage 5 — partial coverage + +Stage 5 (`stages/05-claude-code-ecosystem.md`) has 4 sub-stages with hands-on exercises: + +| Sub-stage | Status | +|---|---| +| 5.1 Claude Code 基礎 | Outline only (in `stages/05-...md` 動手練習) | +| 5.2 MCP (Model Context Protocol) | Outline only; cookbook 2 covers building first MCP server | +| 5.3 Skills | Outline + **1 shipped meta-example**: [`examples/stage-5/tool-calling-tutor/`](../examples/stage-5/tool-calling-tutor/) (full SKILL.md + 3 references + evals.json, used as the Stage 5.3 authoring exemplar) | +| 5.4 Plugins & Marketplaces | Outline only | + +For v2, sub-stages 5.1 / 5.2 / 5.4 could ship sample artifacts (sample `CLAUDE.md`, MCP server skeleton, plugin.json). Similar to Track A v2 — low priority. + +## v2 path (deferred) + +Per `docs/HOW_TO_USE.md` "給維護者：v2 path": +- Split each `starter.py` → `starter_template.py` (TODO skeleton) + `starter_reference.py` (solution) +- Make `test.py` behavioral (input → output contract) instead of implementation-bound +- ~20 folders × 3 file changes = ~60 file changes +- Probably needs its own session + +## Historical: what was on the unverified branch + +Before verification, Stage 4 + 6 + 7 commits sat on branch `t3-stage-4-6-7-unverified` (rationale: framework deps not pip-installed at write time, API drift risk). After actual verification on 2026-05-13: + +``` +50c3bf8 fix(examples): 6 bugs found while verifying Stage 4/6/7 tests +9f60759 Stage 7 練習 5 (FastAPI deploy) +1a8ba16 Stage 7 練習 4 (streaming + caching) +128ca7a Stage 7 練習 3 (observability) +8119de0 Stage 7 練習 2 (eval) +5ff3ce3 Stage 7 練習 1 (multi-agent debate) +8150881 Stage 6 練習 5 (long-term memory) +7633874 Stage 6 練習 4 (full RAG pipeline) +7a8af9b Stage 6 練習 3 (chunking comparison) +b83a5e5 Stage 6 練習 2 (vector DB) +7d2c1b7 Stage 6 練習 1 (embeddings) +ab6d358 Stage 4 練習 5 (Pydantic AI) +6316d83 Stage 4 練習 4 (Smolagents CodeAct) +ea9c14a Stage 4 練習 3 (LangGraph branching) +dbe7c91 Stage 4 練習 2 (CrewAI multi-agent) +8051861 Stage 4 練習 1 (LangGraph + CrewAI) +``` + +All merged into `main` via [`cdb0ae3`](https://github.com/WenyuChiou/awesome-agentic-ai-zh/commit/cdb0ae3). Branch deleted from origin after merge. diff --git a/docs/stylesheets/extra.css b/docs/stylesheets/extra.css new file mode 100644 index 0000000..059baa6 --- /dev/null +++ b/docs/stylesheets/extra.css @@ -0,0 +1,98 @@ +/* awesome-agentic-ai-zh — site branding + landing-page styles. + Loaded via mkdocs.yml `extra_css`. Material defaults stay intact; + this refines brand color, the home hero, grid cards, and tables. */ + +/* ---- Brand color (indigo) ---- */ +:root { + --md-primary-fg-color: #4f46e5; + --md-primary-fg-color--light: #6366f1; + --md-primary-fg-color--dark: #4338ca; + --md-accent-fg-color: #6366f1; +} + +/* ---- Landing hero (index.md) ---- */ +.aaz-hero { + text-align: center; + padding: 2.5rem 1rem 1.5rem; +} +.aaz-hero .aaz-repo { + font-family: var(--md-code-font-family); + font-size: .72rem; + color: var(--md-default-fg-color--light); +} +.aaz-hero h1 { font-size: 2.1rem; font-weight: 700; margin: .35rem 0 .25rem; } +.aaz-hero .aaz-tagline { + font-size: 1.05rem; + color: var(--md-default-fg-color--light); + max-width: 34rem; + margin: .4rem auto 1.4rem; + line-height: 1.6; +} +.aaz-cta { + display: inline-flex; + gap: .6rem; + flex-wrap: wrap; + justify-content: center; + margin-bottom: 1rem; +} +.aaz-cta .md-button { border-radius: 24px; } +.aaz-langs { display: flex; gap: .4rem; justify-content: center; flex-wrap: wrap; } +.aaz-langs a { + font-size: .72rem; + color: var(--md-default-fg-color--light); + border: 1px solid var(--md-default-fg-color--lighter); + border-radius: 20px; + padding: .15rem .7rem; + text-decoration: none; + transition: color .15s ease, border-color .15s ease, background .15s ease; +} +.aaz-langs a:hover { + color: var(--md-accent-fg-color); + border-color: var(--md-accent-fg-color); + background: var(--md-default-fg-color--lightest); +} + +/* ---- Stat strip ---- */ +.aaz-stats { + display: grid; + grid-template-columns: repeat(4, 1fr); + gap: .75rem; + max-width: 40rem; + margin: 1.5rem auto; +} +.aaz-stat { + text-align: center; + padding: 1rem .4rem; + border-radius: 12px; + background: var(--md-default-fg-color--lightest); +} +.aaz-stat .aaz-num { font-size: 1.6rem; font-weight: 700; display: block; line-height: 1.2; } +.aaz-stat .aaz-lbl { font-size: .78rem; color: var(--md-default-fg-color--light); } +@media (max-width: 30rem) { + .aaz-stats { grid-template-columns: repeat(2, 1fr); } +} + +/* ---- Grid-card polish (Material `.grid.cards`) ---- */ +.md-typeset .grid.cards > ul > li { + border-radius: 12px; + transition: border-color .2s ease, box-shadow .2s ease, transform .2s ease; +} +.md-typeset .grid.cards > ul > li:hover { + border-color: var(--md-accent-fg-color); + box-shadow: 0 4px 18px rgba(0, 0, 0, .08); + transform: translateY(-2px); +} + +/* ---- Tables: tighter, rounded, hover ---- */ +.md-typeset table:not([class]) { + border-radius: 10px; + overflow: hidden; + font-size: .80rem; +} +.md-typeset table:not([class]) th { + background: var(--md-default-fg-color--lightest); + font-weight: 600; +} +.md-typeset table:not([class]) tr:hover td { + background: var(--md-default-fg-color--lightest); +} diff --git a/examples/README.en.md b/examples/README.en.md new file mode 100644 index 0000000..a8989ac --- /dev/null +++ b/examples/README.en.md @@ -0,0 +1,219 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# `examples/` — Runnable hands-on exercises + +> [← Back to main path README](../README.en.md) + +Every stage in the learning roadmap has a "Hands-on Exercises" section that tells you *what* to do. This folder adds the **actual runnable starter code** — copy → install deps → `python starter.py` → see expected output. + +## Directory layout + +``` +examples/ +├── stage-3/ # Tool Use & Agent intro +│ ├── 03-react-from-scratch/ # Exercise 3: ReAct from scratch +│ │ ├── starter.py # Main program (~70 LOC runnable) +│ │ ├── test.py # Self-check (pure assert, no pytest) +│ │ ├── README.md # 200-400-word walkthrough (+.zh-Hans.md +.en.md) +│ │ └── requirements.txt # Pinned deps +│ └── ... +├── stage-1/ +└── ... +``` + +Short exercises (≤30 LOC) stay inline as `
` blocks in the stage doc — no folder. Longer ones (>30 LOC) get their own folder so stage docs don't get bloated by code blocks. + +## How to run any example + +```bash +cd examples/stage-3/03-react-from-scratch +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... # Each example header lists the key it needs +python starter.py # Hits the real API to see output (~$0.001 in credits) +python test.py # Runs validation (mock-based, free) +``` + +## Design rules + +| Dimension | Rule | +|---|---| +| Program length | starter ≤80 LOC, split if longer | +| Dependencies | stdlib + ≤2 pip packages, pinned versions | +| Tests | Plain `assert`, no pytest; reader runs `python test.py` to see ✅ | +| Comments | Chinese (zh-TW primary), English variable / function names | +| Self-check | Every starter.py ends with a `# === Self-check ===` block | +| Environment vars | Header comment must list required keys | +| Free-tier friendly | Use the cheapest model (claude-haiku / Ollama); note how to switch to Sonnet | +| **Windows encoding** | **Every .py must reconfigure stdout to UTF-8** (see below) | + +### Windows cp950 encoding fix (mandatory in every starter.py / test.py) + +Windows consoles default to cp950 (Big5) and can't print emoji or non-Big5 Chinese. Add this right after imports in every `.py`: + +```python +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") +``` + +Without it, Windows readers running in PowerShell / cmd hit `UnicodeEncodeError: 'cp950' codec can't encode character '✅'`. + +## Three paths — **default is Ollama (cost-driven)** + +> 💰 **Why default to Ollama?** Running 1000 practice iterations on Sonnet costs ~$4; on haiku ~$0.25; on local Ollama $0. **API cost should not block learning.** Reserve cloud LLMs for "want to see high-quality answers / production deployment". + +Every exercise ships with all three paths: + +### Path A (**default, recommended**) — local Ollama +- Default `starter.py` / first inline `
` block uses a local model +- Requires [Ollama](https://ollama.com); pull a model based on the stage: + - **Stage 1 + 2** (plain chat / prompt eng): `ollama pull gemma4:e4b` (~7.5 GB; multimodal (text + image + audio); CPU-friendly) + - **Stage 3+** (tool use / agent): `ollama pull qwen2.5:3b` (1.9 GB; reliable tool-use support) +- $0, offline, fine for privacy-sensitive data +- SDK uses the `openai` package (OpenAI-compatible API) with `base_url="http://localhost:11434/v1"` +- Best for: all readers (this is the default recommendation) + +### Path B (optional) — Anthropic API (when you want cloud quality) +- Companion `starter_anthropic.py` (folder) or the second inline `
` block +- Requires `ANTHROPIC_API_KEY`; ~$0.001 per run (haiku) / ~$0.004 (sonnet) +- Higher answer quality and lower latency than local 3-4B Ollama models +- Best for: production-quality demands, long-context work, the Stage 7 production tier + +### Path C (verify logic, no API call) +- Every `test.py` uses `unittest.mock`; `python test.py` validates code logic without spending +- Complements A / B — mock first, then real call + +### Trade-offs + +| Dimension | A Ollama (default) | B Anthropic | C Mock | +|---|---|---|---| +| Cost per call | $0 | ~$0.001-0.004 | $0 | +| Requires | Ollama install | API key | nothing | +| Answer quality | medium (3-4B model) | high | canned, unrepresentative | +| Speed | 5-30 s/call (no GPU) | ~1-3 s/call | <0.1 s | +| Offline | ✅ | ❌ | ✅ | +| Privacy-sensitive data | ✅ | ❌ | ✅ | +| Stage 3+ tool use | ✅ (qwen2.5 / llama3.2) | ✅ | ✅ | +| Best for | **default, no budget pressure** | production upgrade | logic verification | + +→ **Recommended flow**: C first (validate logic, no cost), then A (see real model behaviour locally), then B at the Stage 7 production stage if cloud quality is needed. + +## Recommended LLM list + +> Local + cloud, user-perspective. +> 💡 You don't need to install every model — this table shows "which to use for practice" and "which to upgrade to for production". **Claude is the canonical / production reference; Ollama is the practice default.** + +### Local LLMs (practice default, via Ollama) + +| Model | Download | Recommended RAM | Stage | Tool-use | Speed (CPU/GPU) | Primary use | +|---|---|---|---|---|---|---| +| **`gemma4:e4b`** ⭐ | 7.5 GB | 8 GB | 1+2 | basic | slow / med | Stage 1-2 plain chat / prompt eng (default) | +| **`qwen2.5:3b`** ⭐ | 1.9 GB | 4 GB | 3+ | **reliable** | med / fast | Stage 3+ tool use / agent (default) | +| `llama3.2:3b` | 2.0 GB | 4 GB | 3+ | reliable | med / fast | qwen2.5:3b alternative | +| `mistral-nemo:12b` | 7.1 GB | 16 GB | 3+ | strong | slow / med | When you want closer-to-cloud quality | +| `qwen2.5:14b` | 9.0 GB | 16 GB | advanced | strong | slow / med | Larger-model comparison (GPU preferred) | +| `gemma4:e2b` | 4.0 GB | 4 GB | 1+2 | basic | med / fast | 4 GB-RAM-machine alternative | + +Install: `ollama pull ` + `ollama serve`. Hardware tuning details: [resources/cli-agents-guide.en.md](../resources/cli-agents-guide.en.md). + +### Cloud LLMs (canonical / production stack, via Anthropic) + +| Model | $/1M input | $/1M output | Context | Primary use | +|---|---|---|---|---| +| `claude-fable-5` | $10 | $50 | 1M | Mythos-class (above Opus); suspended 2026-06-12, **restored 2026-07-01** (export controls lifted); the highest Claude tier | +| **`claude-haiku-4-5`** ⭐ | $1 | $5 | 200k | Cheapest; fine for Stage 1-7 cloud-quality comparisons | +| **`claude-sonnet-5`** ⭐ | $3 | $15 | 1M | **Production default**; Stage 5+ agent development | +| `claude-opus-4-8` | $5 | $25 | 1M | Opus-class flagship; complex reasoning / long-context refactors | + +Subscription alternative: Claude Pro $20/month (includes Sonnet usage); Claude Max $100/month (includes Opus). Details: [resources/cli-agents-guide.en.md](../resources/cli-agents-guide.en.md). + +### Cloud LLM Chinese / open-source alternatives (region limits / budget / Chinese-language scenarios) + +> Can't or don't want to use Anthropic? These APIs are **all OpenAI-compatible** — change `base_url` and model name to run the same exercises. + +| Provider | Main model | $/1M input | $/1M output | OpenAI-compat? | Key selling point | +|---|---|---|---|---|---| +| **DeepSeek** ⭐ | `deepseek-chat` (V3) | $0.27 | $1.10 | ✅ | Cheapest cloud (4× cheaper than haiku $1/$5); strong CN & EN; free web at `chat.deepseek.com` | +| DeepSeek R1 | `deepseek-reasoner` | $0.55 | $2.19 | ✅ | Reasoning model (o1-class), still 1/30 the price of OpenAI o1 | +| **Moonshot Kimi** | `kimi-k2-turbo-preview` | $5-10 | $15-30 | ✅ | **1M-token context** (key selling point); good for large files / long conversations. Free web at `kimi.com` | +| **Qwen (Alibaba)** | `qwen-max` / `qwen-turbo` | $0.50-1.50 | $1.50-6 | ✅ (DashScope) | Native Chinese; **same models also run locally via Ollama** (cloud + local both work) | +| **GLM (ZhipuAI)** | `glm-4.5` / `glm-4-plus` | $0.30-2 | $1.50-9 | ✅ | China-native, has free tier. Free web `chatglm.cn` | +| **NVIDIA NIM** | Llama / Mistral / DeepSeek / Qwen etc. hosted | free tier 1000 credits | (same) | ✅ | **Hosts 10+ open models**; new accounts get credits; no local GPU needed. `build.nvidia.com` | + +**API endpoints (OpenAI SDK usage)**: + +```python +# DeepSeek +client = OpenAI(api_key=os.environ["DEEPSEEK_API_KEY"], base_url="https://api.deepseek.com/v1") +r = client.chat.completions.create(model="deepseek-chat", messages=[...]) + +# Moonshot Kimi (China endpoint; international uses .ai) +client = OpenAI(api_key=os.environ["MOONSHOT_API_KEY"], base_url="https://api.moonshot.cn/v1") +r = client.chat.completions.create(model="kimi-k2-turbo-preview", messages=[...]) + +# Qwen (Alibaba DashScope) +client = OpenAI(api_key=os.environ["DASHSCOPE_API_KEY"], + base_url="https://dashscope.aliyuncs.com/compatible-mode/v1") +r = client.chat.completions.create(model="qwen-turbo", messages=[...]) + +# GLM (ZhipuAI) +client = OpenAI(api_key=os.environ["ZHIPUAI_API_KEY"], base_url="https://open.bigmodel.cn/api/paas/v4") +r = client.chat.completions.create(model="glm-4.5-flash", messages=[...]) + +# NVIDIA NIM (hosted open-source) +client = OpenAI(api_key=os.environ["NVIDIA_API_KEY"], base_url="https://integrate.api.nvidia.com/v1") +r = client.chat.completions.create(model="meta/llama-3.3-70b-instruct", messages=[...]) +``` + +**How to pick**: + +| Scenario | Pick | Why | +|---|---|---| +| Mainland China, no cloud access | Ollama local / DeepSeek API | Local is free; DeepSeek has an in-China endpoint | +| Tight budget (< $1/month) | DeepSeek API | 4× cheaper than haiku; quality close | +| Large files / long-doc RAG | Moonshot Kimi | 1M-token context | +| Chinese-native task (classical Chinese, CN search) | Qwen / GLM | Higher Chinese training corpus ratio | +| Want to try 10+ open models without GPU | NVIDIA NIM | One key, play with Llama / Mixtral / Qwen / DeepSeek | +| Production agent (tool use) | Anthropic Claude (canonical) | This repo's Path B default; tool calling most reliable | + +### Budget estimate (completing all 54 exercises across Stage 1-7) + +| Learning path | Total time | Total cost | Best for | +|---|---|---|---| +| **All local Ollama** | ~30 hr (CPU) / ~10 hr (GPU) | **$0** | Budget-conscious, privacy needs, China-mainland no-cloud-access | +| **Mixed: local practice + haiku final review** ⭐ | ~30 hr | **$2-5** | **Recommended default** — practice locally, run final 1-2 iterations on haiku to see cloud quality | +| **All haiku** | ~10 hr | $5-15 | Want speed, budget allows, want full cloud experience | +| **All sonnet** | ~8 hr | $20-50 | Deep practice with higher-quality answers, want high-quality answers | +| **Mixed: sonnet + opus on hard problems** | ~8 hr | $30-80 | Already a production agent developer | + +> 🎯 **Beginner default**: run everything locally first; cap budget at $5. **Only consider upgrading to sonnet at the Stage 7 production tier.** + +## Index by stage + +| Stage | Exercises | Example location | +|---|---|---| +| 1 LLM basics | 6 | inline 4 + folder 2 (`examples/stage-1/`) | +| 2 Prompt engineering | 4 | all inline | +| **3 Tool use** | **6** | inline 1 + folder 5 (`examples/stage-3/`) | +| 4 Frameworks | 5 | all folder (`examples/stage-4/`) | +| 5 Claude Code ecosystem | 11 | inline 6 + folder 5 (`examples/stage-5/`) | +| 6 Memory/RAG | 5 | all folder (`examples/stage-6/`) | +| 7 Multi-agent | 5 | inline 1 + folder 4 (`examples/stage-7/`) | +| Track A1-A3 | 12 | all inline + 2 small folders (CLI-9 / CLI-10) | + +→ T1 scope: **Stage 3 全 6 exercises only** (remaining stages roll out per plan tiers). + +## Contributing / reporting issues + +If something doesn't run, output doesn't match expectations, or you want to add a new example: +- File an issue tagged `examples` +- Or open a PR following the "Design rules" table above + +## Why this split (instead of stuffing everything into stage docs) + +1. **Stage docs stay readable** — roadmap readers don't always want code, they want concepts; long code blocks break that +2. **Examples evolve independently** — SDK bumps, model rename, example needs its own commit without polluting the roadmap's git log +3. **Readers can clone one example** — `svn export` or `git clone --filter=tree:0` grabs a single folder +4. **Future CI** — example failures shouldn't block mdbook deploy; this split lets CI run examples conditionally diff --git a/examples/README.md b/examples/README.md new file mode 100644 index 0000000..d370740 --- /dev/null +++ b/examples/README.md @@ -0,0 +1,237 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# `examples/` — 動手練習可跑範例 + +> [← 回主路線 README](../README.md) + +學習地圖每個 stage 都有「動手練習」section、講「該做什麼」。這個資料夾補上**真的可以跑的範例 code**——複製 → 裝依賴 → `python starter.py` 看到預期輸出。 + +## 目錄結構 + +``` +examples/ +├── stage-3/ # Tool Use & Agent 入門 +│ ├── 03-react-from-scratch/ # 練習 3：從零實作 ReAct +│ │ ├── starter.py # 主程式（~70 行可跑） +│ │ ├── test.py # 自我驗證（pure assert、無 pytest） +│ │ ├── README.md # 200-400 字走查（+.zh-Hans.md +.en.md） +│ │ └── requirements.txt # 依賴釘版本 +│ └── ... +├── stage-1/ +└── ... +``` + +短的練習（≤30 LOC）直接以 `
` 收摺塞在 stage 檔內、不開資料夾。長的（>30 LOC）才開資料夾——避免 stage 檔被 code block 撐爆。 + +## 怎麼跑任一個範例 + +```bash +cd examples/stage-3/03-react-from-scratch +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... # 各範例頂端會說它要哪個 key +python starter.py # 跑真的 API 看輸出（會花一點點錢、約 $0.001） +python test.py # 跑驗證（用 mock、不花錢） +``` + +## 設計原則 + +| 維度 | 規則 | +|---|---| +| 程式長度 | starter ≤80 LOC、超過拆檔 | +| 依賴 | stdlib + 最多 2 個 pip 套件、釘版本 | +| 測試 | 純 `assert`、不用 pytest、reader 跑 `python test.py` 看 ✅ | +| 註解 | 中文（zh-TW 為主）、變數 / 函式名英文 | +| 自我驗證 | 每個 starter.py 結尾必有 `# === 自我驗證 ===` 區塊 | +| 環境變數 | 頂端註解寫清楚需要哪些 key | +| Free-tier 友善 | 用最便宜 model（claude-haiku / Ollama）、註解寫怎麼換 Sonnet | +| **Windows 編碼** | **每個 .py 頂端必須有 UTF-8 reconfigure**（見下） | + +### Windows cp950 編碼 fix（每個 starter.py / test.py 必加） + +Windows 預設 console 是 cp950（Big5）、印不出 emoji 跟非 Big5 中文。每個 `.py` 檔頂端 import 區後立刻加： + +```python +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") +``` + +否則 Windows reader 在 PowerShell / cmd 跑會炸 `UnicodeEncodeError: 'cp950' codec can't encode character '✅'`。 + +## 三條路徑 — **預設用 Ollama（成本考量）** + +> 💰 **為什麼默認 Ollama？** 練習場景跑 1000 次跑滿 Sonnet ~$4、跑 haiku ~$0.25、跑本機 Ollama $0。**學習階段不該被 API 成本卡住**。Cloud LLM 留給「想看高品質答案 / production deployment」時用。 + +每個練習都同時提供 3 條路徑： + +### Path A（**默認、推薦**）— Ollama 本機 +- 預設 `starter.py` / 第一個 inline `
` 用本機 LLM +- 需 [Ollama](https://ollama.com)、按 stage pull 對應 model： + - **Stage 1 + 2**（純 chat / prompt eng）：`ollama pull gemma4:e4b`（~7.5 GB、多模態、CPU 跑得動） + - **Stage 3+**（tool use / agent）：`ollama pull qwen2.5:3b`（1.9 GB、tool-use 支援穩定） +- 全程 $0、offline、隱私敏感資料 OK +- SDK 用 `openai` package（OpenAI-compatible API）、`base_url="http://localhost:11434/v1"` +- 適合：所有讀者（默認推這條） + +### Path B（選擇性）— Anthropic API（想看 cloud 高品質時） +- 對照 `starter_anthropic.py`（folder）或第二個 inline `
` 區塊 +- 需 `ANTHROPIC_API_KEY`、跑一輪約 $0.001（haiku）/ $0.004（sonnet） +- 答案品質 / latency 都比本機 Ollama 強 +- 適合：production 要求高品質、需要 long-context、Stage 7 production tier + +### Path C（驗邏輯、不打 API） +- 所有 `test.py` 都用 `unittest.mock`、`python test.py` 看程式邏輯有沒有寫對 +- 跟 Path A / B 互補：先 mock 驗邏輯、再 real call 確認 + +### 三條路的 Trade-off + +| 維度 | A Ollama（默認）| B Anthropic | C Mock | +|---|---|---|---| +| Cost / call | $0 | ~$0.001-0.004 | $0 | +| 需要 | Ollama install | API key | 無 | +| 答案品質 | 中（3-4B model） | 高 | 預設、看不出真實品質 | +| 速度 | 5-30s/call（無 GPU） | ~1-3s/call | <0.1s | +| Offline | ✅ | ❌ | ✅ | +| 隱私敏感資料 | ✅ | ❌ | ✅ | +| Stage 3+ tool use | ✅（qwen2.5 / llama3.2） | ✅ | ✅ | +| 適合 | **默認、無預算壓力** | production 升級 | 程式邏輯驗證 | + +→ **建議流程**：先 C 驗邏輯（不花錢）、再 A 本機跑看實際 model 行為、production 階段（Stage 7）再升 B 看 cloud 品質。 + +## 推薦 LLM 清單 + +> 本機 + cloud、user 視角。 +> 💡 不是要你全裝、是讓你看到「練習用哪個」「production 升級到哪個」。**Claude 是 canonical / production 主軸；Ollama 是練習默認**。 + +### 本機 LLM（練習默認、用 Ollama） + +| Model | 下載大小 | 建議 RAM | 對應 Stage | Tool-use | 速度（CPU/GPU） | 主用途 | +|---|---|---|---|---|---|---| +| **`gemma4:e4b`** ⭐ | 7.5 GB | 8 GB | 1+2 | 基本 | 慢 / 中 | Stage 1-2 純 chat / prompt eng（默認）| +| **`qwen2.5:3b`** ⭐ | 1.9 GB | 4 GB | 3+ | **穩定** | 中 / 快 | Stage 3+ tool use / agent（默認）| +| `llama3.2:3b` | 2.0 GB | 4 GB | 3+ | 穩定 | 中 / 快 | qwen2.5:3b 的替代 | +| `mistral-nemo:12b` | 7.1 GB | 16 GB | 3+ | 強 | 慢 / 中 | 想看更接近 cloud 品質 | +| `qwen2.5:14b` | 9.0 GB | 16 GB | 進階 | 強 | 慢 / 中 | 大 model 對照（需 GPU 偏好）| +| `gemma4:e2b` | 4.0 GB | 4 GB | 1+2 | 基本 | 中 / 快 | 4GB RAM 機器替代 | + +安裝：`ollama pull ` + `ollama serve`。詳細硬體配置看 [resources/cli-agents-guide.md](../resources/cli-agents-guide.md)。 + +### Cloud LLM（canonical / production 主軸、用 Anthropic） + +| Model | 每 1M input | 每 1M output | Context | 主用途 | +|---|---|---|---|---| +| `claude-fable-5` | $10 | $50 | 1M | Mythos 級（位階在 Opus 之上）；2026-06-12 暫停、**2026-07-01 恢復**（出口管制解除）——目前最高階的 Claude 層級 | +| **`claude-haiku-4-5`** ⭐ | $1 | $5 | 200k | 最便宜、Stage 1-7 練習 cloud 對照都 OK | +| **`claude-sonnet-5`** ⭐ | $3 | $15 | 1M | **production 默認**、Stage 5+ agent 開發 | +| `claude-opus-4-8` | $5 | $25 | 1M | Opus 級旗艦、複雜推理 / 長 context refactor | + +訂閱替代：Claude Pro $20/月含 Sonnet 用量、Claude Max $100/月含 Opus。詳細看 [resources/cli-agents-guide.md](../resources/cli-agents-guide.md)。 + +### Cloud LLM 中國 / 開源 alternatives（地區限制 / 預算敏感 / 中文場景） + +> 不能 / 不想用 Anthropic？這些 API **都 OpenAI-compatible**、改 `base_url` 跟 model name 就能跑本 repo 同一份練習。 + +| Provider | 主 model | 每 1M input | 每 1M output | OpenAI-compat? | 主賣點 | +|---|---|---|---|---|---| +| **DeepSeek** ⭐ | `deepseek-chat` (V3) | $0.27 | $1.10 | ✅ | 最便宜 cloud（比 haiku $1/$5 還便宜 4 倍）、中英文俱佳、含免費 web `chat.deepseek.com` | +| DeepSeek R1 | `deepseek-reasoner` | $0.55 | $2.19 | ✅ | 推理模型（o1 級）、價格仍只是 OpenAI o1 的 1/30 | +| **Moonshot Kimi** | `kimi-k2-turbo-preview` | $5-10 | $15-30 | ✅ | **1M token context**（賣點）、適合大檔案 / 長對話。web 版 `kimi.com` 免費 | +| **通義千問 Qwen** | `qwen-max` / `qwen-turbo` | $0.50-1.50 | $1.50-6 | ✅（DashScope）| 中文 native、**同 model 也能 Ollama 本機跑**（cloud + local 兩條路徑都通） | +| **智譜 GLM** | `glm-4.5` / `glm-4-plus` | $0.30-2 | $1.50-9 | ✅ | 中國 native、有 free tier。web `chatglm.cn` 免費 | +| **NVIDIA NIM** | Llama / Mistral / DeepSeek / Qwen 等 hosted | free tier 1000 credits | (同) | ✅ | **托管 10+ open model**、新帳號送 credits、不必本機 GPU。`build.nvidia.com` | + +**API endpoints（OpenAI SDK 接法）**： + +```python +# DeepSeek +client = OpenAI(api_key=os.environ["DEEPSEEK_API_KEY"], base_url="https://api.deepseek.com/v1") +r = client.chat.completions.create(model="deepseek-chat", messages=[...]) + +# Moonshot Kimi（中國 endpoint；海外用 .ai 結尾） +client = OpenAI(api_key=os.environ["MOONSHOT_API_KEY"], base_url="https://api.moonshot.cn/v1") +r = client.chat.completions.create(model="kimi-k2-turbo-preview", messages=[...]) + +# 通義千問 Qwen（Alibaba DashScope） +client = OpenAI(api_key=os.environ["DASHSCOPE_API_KEY"], + base_url="https://dashscope.aliyuncs.com/compatible-mode/v1") +r = client.chat.completions.create(model="qwen-turbo", messages=[...]) + +# 智譜 GLM +client = OpenAI(api_key=os.environ["ZHIPUAI_API_KEY"], base_url="https://open.bigmodel.cn/api/paas/v4") +r = client.chat.completions.create(model="glm-4.5-flash", messages=[...]) + +# NVIDIA NIM（hosted open-source） +client = OpenAI(api_key=os.environ["NVIDIA_API_KEY"], base_url="https://integrate.api.nvidia.com/v1") +r = client.chat.completions.create(model="meta/llama-3.3-70b-instruct", messages=[...]) +``` + +**怎麼挑**： + +| 情境 | 選 | 理由 | +|---|---|---| +| 中國大陸、無 cloud 訪問 | Ollama 本機 / DeepSeek API | 本機免費；DeepSeek 在中國有 endpoint | +| 預算極敏感（< $1/月） | DeepSeek API | 比 haiku 便宜 4 倍、品質接近 | +| 大檔案 / 長文檔 RAG | Moonshot Kimi | 1M token context 賣點 | +| 中文 native task（古文、中文搜索）| Qwen / GLM | 訓練語料中文佔比高 | +| 想試 10+ open model 沒 GPU | NVIDIA NIM | 一個 key 玩 Llama / Mixtral / Qwen / DeepSeek | +| Production agent（agent / tool use）| Anthropic Claude（canonical）| 本 repo Path B 默認、tool calling 最穩 | + +### 預算估算（跑完 Stage 1-7 全 54 練習） + +| 學習路徑 | 總時間 | 總成本 | 適合誰 | +|---|---|---|---| +| **全本機 Ollama** | ~30 hr (CPU) / ~10 hr (GPU) | **$0** | 預算敏感、隱私需求、中國大陸無 cloud 訪問 | +| **混合：本機練 + haiku 終驗** ⭐ | ~30 hr | **$2-5** | **推薦默認**：練習 local 跑、最後 1-2 次用 haiku 看 cloud 品質 | +| **全 haiku** | ~10 hr | $5-15 | 想快、預算允許、想看完整 cloud 體驗 | +| **全 sonnet** | ~8 hr | $20-50 | 深度練習、追求高品質答案 | +| **混合：sonnet 為主 + opus 難題** | ~8 hr | $30-80 | 已是 production agent 開發者 | + +> 🎯 **新手默認**：先全本機跑、預算上限 $5。**Stage 7 production tier 才考慮 sonnet 升級**。 + +### 怎麼從 Ollama 換到 Anthropic？ + +每個練習都有 `
` Path B 區塊或 `starter_anthropic.py`、改 3 行： + +```python +# 從這個（Path A 默認）： +from openai import OpenAI +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") +r = client.chat.completions.create(model="gemma4:e4b", ...) + +# 換成這個（Path B、若有 ANTHROPIC_API_KEY）： +import anthropic +client = anthropic.Anthropic() +r = client.messages.create(model="claude-haiku-4-5", ...) +``` + +主要差異：messages create 方法名、response shape（`choices[0].message.content` vs `content[0].text`）、tool spec wrap（OpenAI 多一層 `{"type": "function", "function": {...}}`）。詳細對照表見 [`resources/cli-agents-guide.md`](../resources/cli-agents-guide.md)。 + +## 對應 stage 索引 + +| Stage | 練習 | 範例位置 | +|---|---|---| +| 1 LLM 基礎 | 6 個 | inline 4 + folder 2（`examples/stage-1/`） | +| 2 Prompt eng | 4 個 | 全 inline | +| **3 Tool use** | **6 個** | inline 1 + folder 5（`examples/stage-3/`） | +| 4 Frameworks | 5 個 | 全 folder（`examples/stage-4/`） | +| 5 Claude Code 生態 | 11 個 | inline 6 + folder 5（`examples/stage-5/`） | +| 6 Memory/RAG | 5 個 | 全 folder（`examples/stage-6/`） | +| 7 Multi-agent | 5 個 | inline 1 + folder 4（`examples/stage-7/`） | +| Track A1-A3 | 12 個 | 全 inline、外加 2 個小 folder（CLI-9 / CLI-10） | + +→ T1 完成範圍：**只有 Stage 3 全部 6 個**（剩餘 stage 按 plan 分批推進）。 + +## 貢獻 / 報錯 + +跑不過、結果跟預期輸出對不上、或想補一個新練習： +- 開 issue 標 `examples` label +- 或直接 PR、follow 本資料夾「設計原則」表格的規則 + +## 為什麼這樣分（不直接全塞 stage 檔） + +1. **Stage 檔保持 readable**：學習地圖讀者不一定要看 code、只想理解 concept；長 code block 干擾閱讀流 +2. **範例可獨立演進**：API SDK 升版、model name 改、範例需要單獨 commit、不污染學習地圖 git log +3. **Reader 可以 clone 單一 example**：`svn export` 或 `git clone --filter=tree:0` 只抓一個資料夾 +4. **未來 CI**：example 失敗不應 block mdbook deploy；分開可讓 CI 有條件性檢查 diff --git a/examples/README.zh-Hans.md b/examples/README.zh-Hans.md new file mode 100644 index 0000000..088574e --- /dev/null +++ b/examples/README.zh-Hans.md @@ -0,0 +1,219 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# `examples/` — 动手练习可跑范例 + +> [← 回主路线 README](../README.zh-Hans.md) + +学习地图每个 stage 都有“动手练习”section、讲“该做什么”。这个资料夹补上**真的可以跑的范例 code**——复制 → 装依赖 → `python starter.py` 看到预期输出。 + +## 目录结构 + +``` +examples/ +├── stage-3/ # Tool Use & Agent 入门 +│ ├── 03-react-from-scratch/ # 练习 3：从零实现 ReAct +│ │ ├── starter.py # 主程式（~70 行可跑） +│ │ ├── test.py # 自我验证（pure assert、无 pytest） +│ │ ├── README.md # 200-400 字走查（+.zh-Hans.md +.en.md） +│ │ └── requirements.txt # 依赖钉版本 +│ └── ... +├── stage-1/ +└── ... +``` + +短的练习（≤30 LOC）直接以 `
` 收摺塞在 stage 档内、不开资料夹。长的（>30 LOC）才开资料夹——避免 stage 档被 code block 撑爆。 + +## 怎么跑任一个范例 + +```bash +cd examples/stage-3/03-react-from-scratch +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... # 各范例顶端会说它要哪个 key +python starter.py # 跑真的 API 看输出（会花一点点钱、约 $0.001） +python test.py # 跑验证（用 mock、不花钱） +``` + +## 设计原则 + +| 维度 | 规则 | +|---|---| +| 程序长度 | starter ≤80 LOC、超过拆档 | +| 依赖 | stdlib + 最多 2 个 pip 套件、钉版本 | +| 测试 | 纯 `assert`、不用 pytest、reader 跑 `python test.py` 看 ✅ | +| 注解 | 中文（zh-Hans 为主）、变数 / 函数名英文 | +| 自我验证 | 每个 starter.py 结尾必有 `# === 自我验证 ===` 区块 | +| 环境变数 | 顶端注解写清楚需要哪些 key | +| Free-tier 友善 | 用最便宜 model（claude-haiku / Ollama）、注解写怎么换 Sonnet | +| **Windows 编码** | **每个 .py 顶端必须有 UTF-8 reconfigure**（见下） | + +### Windows cp950 编码 fix（每个 starter.py / test.py 必加） + +Windows 预设 console 是 cp950（Big5）、印不出 emoji 跟非 Big5 中文。每个 `.py` 档顶端 import 区后立刻加： + +```python +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") +``` + +否则 Windows reader 在 PowerShell / cmd 跑会炸 `UnicodeEncodeError: 'cp950' codec can't encode character '✅'`。 + +## 三条路径 — **默认用 Ollama（成本考量）** + +> 💰 **为什么默认 Ollama？** 练习场景跑 1000 次跑满 Sonnet ~$4、跑 haiku ~$0.25、跑本机 Ollama $0。**学习阶段不该被 API 成本卡住**。Cloud LLM 留给“想看高质量答案 / production deployment”时用。 + +每个练习都同时提供 3 条路径： + +### Path A（**默认、推荐**）— Ollama 本机 +- 预设 `starter.py` / 第一个 inline `
` 用本机 LLM +- 需 [Ollama](https://ollama.com)、按 stage pull 对应 model： + - **Stage 1 + 2**（纯 chat / prompt eng）：`ollama pull gemma4:e4b`（~7.5 GB、多模態、CPU 跑得動） + - **Stage 3+**（tool use / agent）：`ollama pull qwen2.5:3b`（1.9 GB、tool-use 支持稳定） +- 全程 $0、offline、隐私敏感资料 OK +- SDK 用 `openai` package（OpenAI 兼容 API）、`base_url="http://localhost:11434/v1"` +- 适合：所有读者（默认推这条） + +### Path B（选择性）— Anthropic API（想看 cloud 高质量时） +- 对照 `starter_anthropic.py`（folder）或第二个 inline `
` 区块 +- 需 `ANTHROPIC_API_KEY`、跑一轮约 $0.001（haiku）/ $0.004（sonnet） +- 答案质量 / latency 都比本机 Ollama 强 +- 适合：production 要求高质量、需要 long-context、Stage 7 production tier + +### Path C（验逻辑、不打 API） +- 所有 `test.py` 都用 `unittest.mock`、`python test.py` 看程序逻辑有没有写对 +- 跟 Path A / B 互补：先 mock 验逻辑、再 real call 确认 + +### 三条路的 Trade-off + +| 维度 | A Ollama（默认）| B Anthropic | C Mock | +|---|---|---|---| +| Cost / call | $0 | ~$0.001-0.004 | $0 | +| 需要 | Ollama install | API key | 无 | +| 答案质量 | 中（3-4B model） | 高 | 预设、看不出真实质量 | +| 速度 | 5-30s/call（无 GPU） | ~1-3s/call | <0.1s | +| Offline | ✅ | ❌ | ✅ | +| 隐私敏感资料 | ✅ | ❌ | ✅ | +| Stage 3+ tool use | ✅（qwen2.5 / llama3.2） | ✅ | ✅ | +| 适合 | **默认、无预算压力** | production 升级 | 程序逻辑验证 | + +→ **建议流程**：先 C 验逻辑（不花钱）、再 A 本机跑看实际 model 行为、production 阶段（Stage 7）再升 B 看 cloud 质量。 + +## 推荐 LLM 清单 + +> 本机 + cloud、user 视角。 +> 💡 不是要你全装、是让你看到“练习用哪个”“production 升级到哪个”。**Claude 是 canonical / production 主轴；Ollama 是练习默认**。 + +### 本机 LLM（练习默认、用 Ollama） + +| Model | 下载大小 | 建议 RAM | 对应 Stage | Tool-use | 速度（CPU/GPU） | 主用途 | +|---|---|---|---|---|---|---| +| **`gemma4:e4b`** ⭐ | 7.5 GB | 8 GB | 1+2 | 基本 | 慢 / 中 | Stage 1-2 纯 chat / prompt eng（默认）| +| **`qwen2.5:3b`** ⭐ | 1.9 GB | 4 GB | 3+ | **稳定** | 中 / 快 | Stage 3+ tool use / agent（默认）| +| `llama3.2:3b` | 2.0 GB | 4 GB | 3+ | 稳定 | 中 / 快 | qwen2.5:3b 的替代 | +| `mistral-nemo:12b` | 7.1 GB | 16 GB | 3+ | 强 | 慢 / 中 | 想看更接近 cloud 质量 | +| `qwen2.5:14b` | 9.0 GB | 16 GB | 进阶 | 强 | 慢 / 中 | 大 model 对照（需 GPU 偏好）| +| `gemma4:e2b` | 4.0 GB | 4 GB | 1+2 | 基本 | 中 / 快 | 4GB RAM 机器替代 | + +安装：`ollama pull ` + `ollama serve`。详细硬件配置看 [resources/cli-agents-guide.zh-Hans.md](../resources/cli-agents-guide.zh-Hans.md)。 + +### Cloud LLM（canonical / production 主轴、用 Anthropic） + +| Model | 每 1M input | 每 1M output | Context | 主用途 | +|---|---|---|---|---| +| `claude-fable-5` | $10 | $50 | 1M | Mythos 级（位阶在 Opus 之上）；2026-06-12 暂停、**2026-07-01 恢复**（出口管制解除）——目前最高阶的 Claude 层级 | +| **`claude-haiku-4-5`** ⭐ | $1 | $5 | 200k | 最便宜、Stage 1-7 练习 cloud 对照都 OK | +| **`claude-sonnet-5`** ⭐ | $3 | $15 | 1M | **production 默认**、Stage 5+ agent 开发 | +| `claude-opus-4-8` | $5 | $25 | 1M | Opus 级旗舰、复杂推理 / 长 context refactor | + +订阅替代：Claude Pro $20/月含 Sonnet 用量、Claude Max $100/月含 Opus。详细看 [resources/cli-agents-guide.zh-Hans.md](../resources/cli-agents-guide.zh-Hans.md)。 + +### Cloud LLM 中国 / 开源 alternatives（地区限制 / 预算敏感 / 中文场景） + +> 不能 / 不想用 Anthropic？这些 API **都 OpenAI-compatible**、改 `base_url` 跟 model name 就能跑本 repo 同一份练习。 + +| Provider | 主 model | 每 1M input | 每 1M output | OpenAI-compat? | 主卖点 | +|---|---|---|---|---|---| +| **DeepSeek** ⭐ | `deepseek-chat` (V3) | $0.27 | $1.10 | ✅ | 最便宜 cloud（比 haiku $1/$5 还便宜 4 倍）、中英文俱佳、含免费 web `chat.deepseek.com` | +| DeepSeek R1 | `deepseek-reasoner` | $0.55 | $2.19 | ✅ | 推理模型（o1 级）、价格仍只是 OpenAI o1 的 1/30 | +| **Moonshot Kimi** | `kimi-k2-turbo-preview` | $5-10 | $15-30 | ✅ | **1M token context**（卖点）、适合大文件 / 长对话。web 版 `kimi.com` 免费 | +| **通义千问 Qwen** | `qwen-max` / `qwen-turbo` | $0.50-1.50 | $1.50-6 | ✅（DashScope）| 中文 native、**同 model 也能 Ollama 本机跑**（cloud + local 两条路径都通） | +| **智谱 GLM** | `glm-4.5` / `glm-4-plus` | $0.30-2 | $1.50-9 | ✅ | 中国 native、有 free tier。web `chatglm.cn` 免费 | +| **NVIDIA NIM** | Llama / Mistral / DeepSeek / Qwen 等 hosted | free tier 1000 credits | (同) | ✅ | **托管 10+ open model**、新账号送 credits、不必本机 GPU。`build.nvidia.com` | + +**API endpoints（OpenAI SDK 接法）**： + +```python +# DeepSeek +client = OpenAI(api_key=os.environ["DEEPSEEK_API_KEY"], base_url="https://api.deepseek.com/v1") +r = client.chat.completions.create(model="deepseek-chat", messages=[...]) + +# Moonshot Kimi（中国 endpoint；海外用 .ai 结尾） +client = OpenAI(api_key=os.environ["MOONSHOT_API_KEY"], base_url="https://api.moonshot.cn/v1") +r = client.chat.completions.create(model="kimi-k2-turbo-preview", messages=[...]) + +# 通义千问 Qwen（Alibaba DashScope） +client = OpenAI(api_key=os.environ["DASHSCOPE_API_KEY"], + base_url="https://dashscope.aliyuncs.com/compatible-mode/v1") +r = client.chat.completions.create(model="qwen-turbo", messages=[...]) + +# 智谱 GLM +client = OpenAI(api_key=os.environ["ZHIPUAI_API_KEY"], base_url="https://open.bigmodel.cn/api/paas/v4") +r = client.chat.completions.create(model="glm-4.5-flash", messages=[...]) + +# NVIDIA NIM（hosted open-source） +client = OpenAI(api_key=os.environ["NVIDIA_API_KEY"], base_url="https://integrate.api.nvidia.com/v1") +r = client.chat.completions.create(model="meta/llama-3.3-70b-instruct", messages=[...]) +``` + +**怎么挑**： + +| 情境 | 选 | 理由 | +|---|---|---| +| 中国大陆、无 cloud 访问 | Ollama 本机 / DeepSeek API | 本机免费；DeepSeek 在中国有 endpoint | +| 预算极敏感（< $1/月） | DeepSeek API | 比 haiku 便宜 4 倍、质量接近 | +| 大文件 / 长文档 RAG | Moonshot Kimi | 1M token context 卖点 | +| 中文 native task（古文、中文搜索）| Qwen / GLM | 训练语料中文占比高 | +| 想试 10+ open model 没 GPU | NVIDIA NIM | 一个 key 玩 Llama / Mixtral / Qwen / DeepSeek | +| Production agent（agent / tool use）| Anthropic Claude（canonical）| 本 repo Path B 默认、tool calling 最稳 | + +### 预算估算（跑完 Stage 1-7 全 54 练习） + +| 学习路径 | 总时间 | 总成本 | 适合谁 | +|---|---|---|---| +| **全本机 Ollama** | ~30 hr (CPU) / ~10 hr (GPU) | **$0** | 预算敏感、隐私需求、中国大陆无 cloud 访问 | +| **混合：本机练 + haiku 终验** ⭐ | ~30 hr | **$2-5** | **推荐默认**：练习 local 跑、最后 1-2 次用 haiku 看 cloud 质量 | +| **全 haiku** | ~10 hr | $5-15 | 想快、预算允许、想看完整 cloud 体验 | +| **全 sonnet** | ~8 hr | $20-50 | 深度练习、追求高质量答案 | +| **混合：sonnet 为主 + opus 难题** | ~8 hr | $30-80 | 已是 production agent 开发者 | + +> 🎯 **新手默认**：先全本机跑、预算上限 $5。**Stage 7 production tier 才考虑 sonnet 升级**。 + +## 对应 stage 索引 + +| Stage | 练习 | 范例位置 | +|---|---|---| +| 1 LLM 基础 | 6 个 | inline 4 + folder 2（`examples/stage-1/`） | +| 2 Prompt eng | 4 个 | 全 inline | +| **3 Tool use** | **6 个** | inline 1 + folder 5（`examples/stage-3/`） | +| 4 Frameworks | 5 个 | 全 folder（`examples/stage-4/`） | +| 5 Claude Code 生态 | 11 个 | inline 6 + folder 5（`examples/stage-5/`） | +| 6 Memory/RAG | 5 个 | 全 folder（`examples/stage-6/`） | +| 7 Multi-agent | 5 个 | inline 1 + folder 4（`examples/stage-7/`） | +| Track A1-A3 | 12 个 | 全 inline、外加 2 个小 folder（CLI-9 / CLI-10） | + +→ T1 完成范围：**只有 Stage 3 全部 6 个**（剩余 stage 按 plan 分批推进）。 + +## 贡献 / 报错 + +跑不过、结果跟预期输出对不上、或想补一个新练习： +- 开 issue 标 `examples` label +- 或直接 PR、follow 本资料夹“设计原则”表格的规则 + +## 为什么这样分（不直接全塞 stage 档） + +1. **Stage 档保持 readable**：学习地图读者不一定要看 code、只想理解 concept；长 code block 干扰阅读流 +2. **范例可独立演进**：API SDK 升版、model name 改、范例需要单独 commit、不污染学习地图 git log +3. **Reader 可以 clone 单一 example**：`svn export` 或 `git clone --filter=tree:0` 只抓一个资料夹 +4. **未来 CI**：example 失败不应 block mdbook deploy；分开可让 CI 有条件性检查 diff --git a/examples/stage-1/04-cross-provider/README.md b/examples/stage-1/04-cross-provider/README.md new file mode 100644 index 0000000..a039229 --- /dev/null +++ b/examples/stage-1/04-cross-provider/README.md @@ -0,0 +1,127 @@ +> **繁體中文** | [简体中文](./README.zh-Hans.md) | [English](./README.en.md) + +# 練習 4：Cross-Provider 比較（Claude / GPT / Gemini） + +對應 [Stage 1 — LLM 基礎](../../../stages/01-llm-basics.md) 練習 4。 +> 🎓 **學習模式**：這份 `starter.py` 是**完整解答**、不是 TODO skeleton。建議用**主動模式**——`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫一份 `starter.py`、跑 `python test.py` 驗證；卡 20 分鐘再回去對照 reference。完整方法論看 [`docs/HOW_TO_USE.md`](../../../docs/HOW_TO_USE.md)。 + + +## 為什麼要比較 + +同樣是「解釋 AGI vs narrow AI」這個 prompt、三家 LLM 回得不一樣： +- **Claude**：通常傾向先給結構（定義 → 例子）、語氣中性 +- **GPT**：傾向先給簡短答案、再展開（type-A 風格） +- **Gemini**：傾向 list / bullet 排列、example 多 + +跑一次自己看、比讀論文有感。順便量 token / 成本 / latency 三維。 + +## 怎麼跑 + +```bash +pip install -r requirements.txt + +# 至少設一個。沒設的會 skip、不會 crash +export ANTHROPIC_API_KEY=sk-ant-... +export OPENAI_API_KEY=sk-... +export GOOGLE_API_KEY=... + +python starter.py +``` + +預期看到（樣本）： + +``` +prompt: 用 1-2 句話解釋 AGI 跟 narrow AI 的差別。 +============================================================ +⚠ skip call_gemini（沒有對應 API key） + +[Anthropic / claude-haiku-4-5] latency=823ms in=21 out=58 +AGI（通用人工智慧）能跨領域學習與解題；narrow AI 只擅長單一任務... + +[OpenAI / gpt-5-mini] latency=612ms in=24 out=49 +Narrow AI 專精於特定任務（如下棋、辨識）、AGI 則具備... + +✅ 練習 4 通過 — 收到 2 家 provider 回應、可比較風格 / 長度 / 成本 +``` + +## 不花錢驗證程式邏輯 + +```bash +python test.py +``` + +4 個 test 都用 `unittest.mock.patch` 取代 SDK： + +``` +✅ test_skip_when_no_key +✅ test_compare_returns_only_valid_replies +✅ test_reply_dataclass_shape +✅ test_compare_one_provider_set + +🎉 全部通過 — Cross-provider 邏輯正確（skip-on-missing-key 已驗） +``` + +## 程式結構走查 + +| 段 | 在做什麼 | +|---|---| +| `Reply` dataclass | 統一三家 SDK 各自 Response 物件、抽出 4 個共通欄位（text/in/out/latency） | +| `call_claude / call_openai / call_gemini` | 各家 SDK 包裝、沒 key 就 return `None` | +| `compare(prompt)` | 跑三個 caller、跳過 None、回 valid replies list | +| `__main__` | 印對照表、自我驗證 | + +## 常見坑 + +1. **三家 SDK API shape 差很多**：Anthropic 用 `messages.create`、OpenAI 用 `chat.completions.create`、Google 用 `models.generate_content`。**用 dataclass 統一才能比較** +2. **Token 計算欄位名不一樣**：Anthropic 是 `input_tokens / output_tokens`、OpenAI 是 `prompt_tokens / completion_tokens`、Google 是 `prompt_token_count / candidates_token_count` +3. **沒設 key 應該 skip 而非 raise**：production code 一定要做這層 guard、production agent 不能因為一家 down 就全死 +4. **沒抓 latency**：跑完才知道哪家慢、production routing 需要這 data + +## 想加更多家？ + +OpenRouter / Mistral / Cohere / Groq 都是 OpenAI-compatible API、改 `base_url` 就接： + +```python +client = OpenAI( + base_url="https://api.groq.com/openai/v1", + api_key=os.environ["GROQ_API_KEY"], +) +``` + +## 🦙 Path B — 加上本機 Ollama 當第 4 家對照 + +`call_openai` 已經是 OpenAI-compatible client、把 `base_url` 跟 `model` 換掉就接 Ollama： + +```python +def call_ollama(prompt: str) -> Reply | None: + """本機 Ollama (gemma4:e4b 或 qwen2.5:3b)。沒裝就 return None、不 crash。""" + import requests + try: + requests.get("http://localhost:11434/api/tags", timeout=2) + except Exception: + return None # Ollama 沒跑 + from openai import OpenAI + client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + t0 = time.time() + r = client.chat.completions.create( + model="gemma4:e4b", + max_tokens=200, + messages=[{"role": "user", "content": prompt}], + ) + return Reply( + provider="Ollama-local", + model="gemma4:e4b", + text=r.choices[0].message.content or "", + in_tokens=r.usage.prompt_tokens, + out_tokens=r.usage.completion_tokens, + latency_ms=int((time.time() - t0) * 1000), + ) +``` + +把 `call_ollama` 加進 `compare()` 的 caller list、就能看 4 家對照（包括本機 free $0 model）。實測你會發現 gemma4:e4b 在 CPU 上的 latency 通常比 cloud 慢 5-10 倍、但 cost = 0。 + +## 延伸 + +- **成本對照** → 接 [`examples/stage-1/03-pricing/`](../) 的 PRICING dict、印 dollar cost column +- **同 prompt 跑 N 次取平均** → 在 `compare()` 內加 for-loop、看 latency stdev +- **加 quality eval** → 加第四家 LLM 當 judge、給每家回應打分（這在 Stage 7 練習 2 會教） diff --git a/examples/stage-1/04-cross-provider/requirements.txt b/examples/stage-1/04-cross-provider/requirements.txt new file mode 100644 index 0000000..c38214e --- /dev/null +++ b/examples/stage-1/04-cross-provider/requirements.txt @@ -0,0 +1,3 @@ +anthropic>=0.40,<1.0 +openai>=1.50,<2.0 +google-genai>=1.0,<2.0 diff --git a/examples/stage-1/04-cross-provider/starter.py b/examples/stage-1/04-cross-provider/starter.py new file mode 100644 index 0000000..ca0e9bd --- /dev/null +++ b/examples/stage-1/04-cross-provider/starter.py @@ -0,0 +1,132 @@ +""" +Stage 1 練習 4：Cross-Provider 比較 — starter.py + +同一個 prompt 同時送給 Claude / GPT / Gemini、印對照表。 +缺哪家 key 就 skip 哪家、不 crash。 + +跑法： + pip install -r requirements.txt + export ANTHROPIC_API_KEY=... # 至少設一個 + export OPENAI_API_KEY=... # （可選） + export GOOGLE_API_KEY=... # （可選） + python starter.py + +驗證： + python test.py （用 mock、不打 API） +""" + +from __future__ import annotations + +import os +import sys +import time +from dataclasses import dataclass + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + + +PROMPT = "用 1-2 句話解釋 AGI 跟 narrow AI 的差別。" + + +@dataclass +class Reply: + provider: str + model: str + text: str + in_tokens: int + out_tokens: int + latency_ms: int + + +def call_claude(prompt: str) -> Reply | None: + if not os.environ.get("ANTHROPIC_API_KEY"): + return None + import anthropic + + client = anthropic.Anthropic() + t0 = time.time() + msg = client.messages.create( + model="claude-haiku-4-5", + max_tokens=200, + messages=[{"role": "user", "content": prompt}], + ) + return Reply( + provider="Anthropic", + model="claude-haiku-4-5", + text=msg.content[0].text, + in_tokens=msg.usage.input_tokens, + out_tokens=msg.usage.output_tokens, + latency_ms=int((time.time() - t0) * 1000), + ) + + +def call_openai(prompt: str) -> Reply | None: + if not os.environ.get("OPENAI_API_KEY"): + return None + from openai import OpenAI + + client = OpenAI() + t0 = time.time() + r = client.chat.completions.create( + model="gpt-4o-mini", + max_tokens=200, + messages=[{"role": "user", "content": prompt}], + ) + return Reply( + provider="OpenAI", + model="gpt-4o-mini", + text=r.choices[0].message.content or "", + in_tokens=r.usage.prompt_tokens, + out_tokens=r.usage.completion_tokens, + latency_ms=int((time.time() - t0) * 1000), + ) + + +def call_gemini(prompt: str) -> Reply | None: + if not os.environ.get("GOOGLE_API_KEY"): + return None + from google import genai + + client = genai.Client() + t0 = time.time() + r = client.models.generate_content( + model="gemini-2.0-flash", + contents=prompt, + ) + usage = getattr(r, "usage_metadata", None) + return Reply( + provider="Google", + model="gemini-2.0-flash", + text=r.text, + in_tokens=getattr(usage, "prompt_token_count", 0) or 0, + out_tokens=getattr(usage, "candidates_token_count", 0) or 0, + latency_ms=int((time.time() - t0) * 1000), + ) + + +def compare(prompt: str) -> list[Reply]: + replies = [] + for fn in (call_claude, call_openai, call_gemini): + r = fn(prompt) + if r is None: + print(f"⚠ skip {fn.__name__}（沒有對應 API key）") + else: + replies.append(r) + return replies + + +if __name__ == "__main__": + print(f"prompt: {PROMPT}\n" + "=" * 60) + replies = compare(PROMPT) + + for r in replies: + print(f"\n[{r.provider} / {r.model}] latency={r.latency_ms}ms in={r.in_tokens} out={r.out_tokens}") + print(r.text) + + # === 自我驗證 === + assert len(replies) >= 1, "至少要有一家 provider 回應（請設一個 API key）" + for r in replies: + assert len(r.text) > 5, f"{r.provider} 回應太短" + assert r.in_tokens > 0, f"{r.provider} 沒 input token" + print(f"\n✅ 練習 4 通過 — 收到 {len(replies)} 家 provider 回應、可比較風格 / 長度 / 成本") diff --git a/examples/stage-1/04-cross-provider/test.py b/examples/stage-1/04-cross-provider/test.py new file mode 100644 index 0000000..989327a --- /dev/null +++ b/examples/stage-1/04-cross-provider/test.py @@ -0,0 +1,73 @@ +""" +Stage 1 練習 4 自我驗證：用 mock 取代三家 SDK、不打 API。 + +驗證內容： + - 沒設 key 的 provider 自動 skip、不會 crash + - 設了 key 的 provider 正常 call、Reply dataclass 結構正確 + - 至少一家 provider 才能 pass +""" + +from __future__ import annotations + +import os +import sys +from unittest.mock import patch + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter import Reply, call_claude, call_openai, call_gemini, compare + + +def test_skip_when_no_key(): + """沒設任何 key 時、三個 call_xxx 都該回 None。""" + with patch.dict(os.environ, {}, clear=True): + assert call_claude("hi") is None + assert call_openai("hi") is None + assert call_gemini("hi") is None + print("✅ test_skip_when_no_key") + + +def test_compare_returns_only_valid_replies(): + """compare() 跳過沒 key 的 provider、不會 raise。""" + with patch.dict(os.environ, {}, clear=True): + replies = compare("hi") + assert replies == [], "沒任何 key 時、replies 應為空 list" + print("✅ test_compare_returns_only_valid_replies") + + +def test_reply_dataclass_shape(): + """Reply 結構 ok。""" + r = Reply( + provider="X", model="x-1", text="hello", + in_tokens=10, out_tokens=20, latency_ms=500, + ) + assert r.provider == "X" + assert r.in_tokens + r.out_tokens == 30 + print("✅ test_reply_dataclass_shape") + + +def test_compare_one_provider_set(): + """模擬：只設 ANTHROPIC_API_KEY、call_claude 被 mock 成回固定 Reply。""" + fake_reply = Reply( + provider="Anthropic", model="claude-haiku-4-5", + text="AGI 跟 narrow AI 的差別…", + in_tokens=20, out_tokens=50, latency_ms=800, + ) + + with patch.dict(os.environ, {"ANTHROPIC_API_KEY": "sk-fake"}, clear=True), \ + patch("starter.call_claude", return_value=fake_reply): + replies = compare("test prompt") + + assert len(replies) == 1 + assert replies[0].provider == "Anthropic" + assert replies[0].in_tokens == 20 + print("✅ test_compare_one_provider_set") + + +if __name__ == "__main__": + test_skip_when_no_key() + test_compare_returns_only_valid_replies() + test_reply_dataclass_shape() + test_compare_one_provider_set() + print("\n🎉 全部通過 — Cross-provider 邏輯正確（skip-on-missing-key 已驗）") diff --git a/examples/stage-1/05-error-handling/README.en.md b/examples/stage-1/05-error-handling/README.en.md new file mode 100644 index 0000000..bf1e333 --- /dev/null +++ b/examples/stage-1/05-error-handling/README.en.md @@ -0,0 +1,116 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# Exercise 5: Error Handling + Retry Wrapper + +Corresponds to [Stage 1 — LLM Basics](../../../stages/01-llm-basics.en.md) Exercise 5. + +## Why this matters + +Production agents in Stages 3-8 will absolutely hit API errors: + +- Rate limit (429) — different subscription tiers, you can hit it anytime +- Network jitter (connection reset) — cross-DC / VPN happens daily +- Expired API key (401) — rotation out of sync +- Context overflow (400) — you appended too much history + +**Some errors should be retried (rate limit / network), others should not (key, context full).** Not distinguishing them is one of the most common production-agent mistakes. + +## How to run — two paths + +### Path A (default, free, local) + +```bash +pip install -r requirements.txt +ollama pull gemma4:e4b +ollama serve +python starter.py +``` + +Budget: **$0**. The local demo shows connection / context-window behavior. + +### Path B (Anthropic, real cloud errors) + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +Budget: ~**$0.0005** per run (only "scenario 2 normal call" actually hits the API). + +Expected output (Path A, local): + +``` +[Scenario 1] Pointing at a dead Ollama port + ✅ Caught APIConnectionError + 💡 Production handling: retry (network errors are typically transient) + +[Scenario 2] Normal call wrapped in with_retry (needs Ollama running) + ✅ Succeeded on first try: 👋 + +[Scenario 3] Prompt over context window (Ollama usually truncates or raises) + ⚠ Ollama didn't raise (it likely truncated). Cloud APIs would 400. + +✅ Exercise 5 passed — you understand which errors raise, when to retry, when to stop. $0/run +``` + +## Validate the logic without network failures (mock-based) + +```bash +python test.py # validates Path A (Ollama) retry wrapper +python test_anthropic.py # validates Path B (Anthropic) retry wrapper +``` + +6 tests use `unittest.mock` to fabricate errors + fake sleep (0 real seconds), validating the retry logic: + +``` +✅ test_no_retry_when_success_first_time +✅ test_retry_on_connection_error_then_success +✅ test_retry_on_rate_limit +✅ test_raise_after_max_attempts +✅ test_no_retry_on_auth_error +✅ test_exponential_backoff_delays + +🎉 All passed — retry wrapper logic correct +``` + +> **Local advantage**: Ollama can't hit a real `RateLimitError` (no quota), so the "rate limit demo" is invisible. But the mock-based tests cover the retry logic completely and reproduce in 0 seconds — which is precisely what makes the Ollama path ideal for **learning retry patterns: fast, free, deterministic**. + +## Program structure walkthrough + +| Section | What it does | +|---|---| +| `RETRIABLE = (APIConnectionError, RateLimitError)` | Whitelist: only these two retry; everything else raises | +| `with_retry(fn, ...)` | Exponential backoff wrapper: 1s, 2s, 4s, 8s + jitter | +| `demo_bad_key()` | Triggers a network / 401 error to inspect the raised exception | +| `demo_with_retry()` | Normal call wrapped in `with_retry`, expected to succeed on first try | +| `demo_too_long_prompt()` | Oversize prompt — see how the context-window limit surfaces | + +## Exception-class mapping between SDKs + +| Anthropic SDK | OpenAI SDK (Ollama) | Meaning | RETRIABLE? | +|---|---|---|---| +| `anthropic.APIConnectionError` | `openai.APIConnectionError` | Network down | ✅ | +| `anthropic.RateLimitError` | `openai.RateLimitError` | 429 throttle | ✅ | +| `anthropic.AuthenticationError` | `openai.AuthenticationError` | 401 bad key | ❌ | +| `anthropic.APIStatusError` | `openai.APIStatusError` | Generic HTTP error | Depends on status | + +The two SDKs use nearly identical class names; the retry logic is **fully backend-agnostic**. Swap SDKs = change the import line. + +## Common pitfalls + +1. **Blindly retry every exception** — you'll retry `AuthenticationError` 4 times for nothing. The RETRIABLE whitelist is the heart of this pattern +2. **No jitter** — 1000 workers rate-limited together, all retry after 1s, hit the limit again → deadlock. Add `random.uniform(0, 0.3)` to spread them +3. **`max_attempts` too high** — 8 retries = 1+2+4+8+16+32+64+128 = 255s, user gave up long ago. `max_attempts=4` usually suffices +4. **No attempt-count metric** — production must emit retry-count metrics; alert above threshold +5. **Ignoring `Retry-After`** — rate-limit responses include this header; SDKs handle it automatically, but a custom wrapper shouldn't ignore the hint + +## Extensions + +- **Better jitter** — try [decorrelated jitter](https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/) for stability +- **Circuit breaker** — after N consecutive failures, stop calling for a while +- **Use [tenacity](https://github.com/jd/tenacity)** — production code shouldn't roll its own retry; this starter is just to show what's inside +- **Finer error classification** — different backoffs for 429 / 503 / 502 / 500 +- **Stage 3 tool-level errors** — see [`../../stage-3/05-error-handling/`](../../stage-3/05-error-handling/) for structured tool errors that let the LLM decide within a ReAct loop diff --git a/examples/stage-1/05-error-handling/README.md b/examples/stage-1/05-error-handling/README.md new file mode 100644 index 0000000..659a01d --- /dev/null +++ b/examples/stage-1/05-error-handling/README.md @@ -0,0 +1,118 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 練習 5：Error Handling + Retry wrapper + +對應 [Stage 1 — LLM 基礎](../../../stages/01-llm-basics.md) 練習 5。 +> 🎓 **學習模式**：這份 `starter.py` 是**完整解答**、不是 TODO skeleton。建議用**主動模式**——`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫一份 `starter.py`、跑 `python test.py` 驗證；卡 20 分鐘再回去對照 reference。完整方法論看 [`docs/HOW_TO_USE.md`](../../../docs/HOW_TO_USE.md)。 + + +## 為什麼這題重要 + +Stage 3-8 的 production agent 一定會碰到 API 錯誤： + +- Rate limit（429）→ 雲端 API 訂閱階級不一樣、隨時可能撞到 +- 網路抖（connection reset）→ 跨機房 / VPN 是日常 +- API key 過期（401）→ rotate 沒同步 +- Context 過長（400）→ 你給太多歷史對話 + +**有些錯誤該 retry（rate limit / 網路）、有些不該（key 錯、context 滿）**。沒分清楚 = 寫 production agent 的常見坑。 + +## 怎麼跑 — 兩條路徑 + +### Path A（默認、本機免費） + +```bash +pip install -r requirements.txt +ollama pull gemma4:e4b +ollama serve +python starter.py +``` + +預算：**$0**。本機 demo 看 connection error / context window 的反應。 + +### Path B（Anthropic、想看真實 cloud 錯誤） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +預算：每次 ≈ **$0.0005**（只有「情境 2 正常 call」會打 API、claude-haiku-4-5）。 + +預期看到（Path A、本機）： + +``` +[情境 1] 故意連到不存在的 Ollama port + ✅ 抓到 APIConnectionError: APIConnectionError + 💡 production 處理: retry（網路錯通常是 transient） + +[情境 2] 正常 call、with_retry 包裝（需要 Ollama 在跑） + ✅ 成功、第一次就過: 👋 + +[情境 3] Prompt 超過 context window（Ollama 通常會截斷或 raise） + ⚠ Ollama 沒 raise（可能直接截斷 prompt）。Cloud API 通常會 400 + +✅ 練習 5 通過 — 你已了解 3 種錯誤如何 raise、知道何時該 retry 何時該 stop、$0/run +``` + +## 不花錢驗證程式邏輯（不需真的斷網） + +```bash +python test.py # 驗 Path A (Ollama) retry wrapper 邏輯 +python test_anthropic.py # 驗 Path B (Anthropic) retry wrapper 邏輯 +``` + +6 個 test 都用 `unittest.mock` 構造假錯誤 + 假 sleep（時間 0 秒）、驗證 retry 邏輯： + +``` +✅ test_no_retry_when_success_first_time +✅ test_retry_on_connection_error_then_success +✅ test_retry_on_rate_limit +✅ test_raise_after_max_attempts +✅ test_no_retry_on_auth_error +✅ test_exponential_backoff_delays + +🎉 全部通過 — retry wrapper 邏輯正確 +``` + +> **本機優勢**：Ollama 不會真的撞 RateLimitError（沒 quota），所以「rate limit demo」看不到。但 mock-based test 完整、retry 邏輯 0 秒可重現——這恰好是 Ollama path 適合理解 retry pattern 的地方：**快、免費、可重現**。 + +## 程式結構走查 + +| 段 | 在做什麼 | +|---|---| +| `RETRIABLE = (APIConnectionError, RateLimitError)` | 白名單：只 retry 這兩種、其他直接 raise | +| `with_retry(fn, ...)` | exponential backoff wrapper：1s, 2s, 4s, 8s + jitter | +| `demo_bad_key()` (Ollama) / `demo_bad_key()` (Anthropic) | 故意觸發網路 / 401 錯、看 exception 怎麼 raise | +| `demo_with_retry()` | 正常 call 包 with_retry、預期 1 次成功 | +| `demo_too_long_prompt()` | 超長 prompt、看 context window 反應 | + +## 兩個 SDK 的 exception class 對應表 + +| Anthropic SDK | OpenAI SDK (Ollama) | 含義 | RETRIABLE? | +|---|---|---|---| +| `anthropic.APIConnectionError` | `openai.APIConnectionError` | 網路斷 | ✅ | +| `anthropic.RateLimitError` | `openai.RateLimitError` | 429 限流 | ✅ | +| `anthropic.AuthenticationError` | `openai.AuthenticationError` | 401 key 錯 | ❌ | +| `anthropic.APIStatusError` | `openai.APIStatusError` | 一般 HTTP 錯 | 視 status code | + +兩個 SDK 的 exception class 名字幾乎一樣、retry 邏輯**完全跨 backend**。換 SDK 只要改 import 那一行。 + +## 常見坑 + +1. **無腦 retry 所有 exception**：會把 AuthenticationError 也 retry 一遍、浪費 4 倍時間最後 still 401。RETRIABLE 白名單是核心 +2. **Backoff 不加 jitter**：1000 個 worker 同時被 rate limit、同時 1s 後重試、再次 rate limit → 死循。加 `random.uniform(0, 0.3)` 打散 +3. **max_attempts 太高**：retry 8 次 = 1+2+4+8+16+32+64+128 = 255 秒、user 早就 give up。`max_attempts=4` 通常夠 +4. **沒記錄 attempt count**：production 一定要把 retry 次數加進 metric、超過 threshold 該 alert +5. **rate limit response 帶 `Retry-After` header**：API 告訴你等多久、SDK 已自動處理，但自寫 wrapper 別忽略這個 hint + +## 延伸 + +- **加 jitter strategy**：除了 uniform、可試 [decorrelated jitter](https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/)（更穩） +- **加 circuit breaker**：連續 N 次 retry 失敗、暫時 stop call +- **改用 [tenacity](https://github.com/jd/tenacity)** library：production 不要自己寫 retry、用成熟 lib +- **錯誤分類更細**：依 status code（429 / 503 / 502 / 500）給不同 backoff strategy +- **Stage 3 tool-level 錯誤**：看 [`../../stage-3/05-error-handling/`](../../stage-3/05-error-handling/)、結構化 tool error 讓 LLM 在 ReAct loop 裡決策 diff --git a/examples/stage-1/05-error-handling/README.zh-Hans.md b/examples/stage-1/05-error-handling/README.zh-Hans.md new file mode 100644 index 0000000..e80b2f7 --- /dev/null +++ b/examples/stage-1/05-error-handling/README.zh-Hans.md @@ -0,0 +1,116 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 练习 5：Error Handling + Retry wrapper + +对应 [Stage 1 — LLM 基础](../../../stages/01-llm-basics.zh-Hans.md) 练习 5。 + +## 为什么这题重要 + +Stage 3-8 的 production agent 一定会碰到 API 错误： + +- Rate limit（429）→ 云端 API 订阅级别不一样、随时可能撞到 +- 网络抖（connection reset）→ 跨机房 / VPN 是日常 +- API key 过期（401）→ rotate 没同步 +- Context 过长（400）→ 你给太多历史对话 + +**有些错误该 retry（rate limit / 网络）、有些不该（key 错、context 满）**。没分清楚 = 写 production agent 的常见坑。 + +## 怎么跑 — 两条路径 + +### Path A（默认、本机免费） + +```bash +pip install -r requirements.txt +ollama pull gemma4:e4b +ollama serve +python starter.py +``` + +预算：**$0**。本机 demo 看 connection error / context window 的反应。 + +### Path B（Anthropic、想看真实 cloud 错误） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +预算：每次 ≈ **$0.0005**（只有“情境 2 正常 call”会打 API、claude-haiku-4-5）。 + +预期看到（Path A、本机）： + +``` +[情境 1] 故意连到不存在的 Ollama port + ✅ 抓到 APIConnectionError: APIConnectionError + 💡 production 处理: retry（网路错通常是 transient） + +[情境 2] 正常 call、with_retry 包装（需要 Ollama 在跑） + ✅ 成功、第一次就过: 👋 + +[情境 3] Prompt 超过 context window（Ollama 通常会截断或 raise） + ⚠ Ollama 没 raise（可能直接截断 prompt）。Cloud API 通常会 400 + +✅ 练习 5 通过 — 你已了解 3 种错误如何 raise、知道何时该 retry 何时该 stop、$0/run +``` + +## 不花钱验证程序逻辑（不需真的断网） + +```bash +python test.py # 验 Path A (Ollama) retry wrapper 逻辑 +python test_anthropic.py # 验 Path B (Anthropic) retry wrapper 逻辑 +``` + +6 个 test 都用 `unittest.mock` 构造假错误 + 假 sleep（时间 0 秒）、验证 retry 逻辑： + +``` +✅ test_no_retry_when_success_first_time +✅ test_retry_on_connection_error_then_success +✅ test_retry_on_rate_limit +✅ test_raise_after_max_attempts +✅ test_no_retry_on_auth_error +✅ test_exponential_backoff_delays + +🎉 全部通过 — retry wrapper 逻辑正确 +``` + +> **本机优势**：Ollama 不会真的撞 RateLimitError（没 quota），所以“rate limit demo”看不到。但 mock-based test 完整、retry 逻辑 0 秒可重现——这恰好是 Ollama path 适合理解 retry pattern 的地方：**快、免费、可重现**。 + +## 程序结构走查 + +| 段 | 在做什么 | +|---|---| +| `RETRIABLE = (APIConnectionError, RateLimitError)` | 白名单：只 retry 这两种、其他直接 raise | +| `with_retry(fn, ...)` | exponential backoff wrapper：1s, 2s, 4s, 8s + jitter | +| `demo_bad_key()` (Ollama) / `demo_bad_key()` (Anthropic) | 故意触发网络 / 401 错、看 exception 怎么 raise | +| `demo_with_retry()` | 正常 call 包 with_retry、预期 1 次成功 | +| `demo_too_long_prompt()` | 超长 prompt、看 context window 反应 | + +## 两个 SDK 的 exception class 对应表 + +| Anthropic SDK | OpenAI SDK (Ollama) | 含义 | RETRIABLE? | +|---|---|---|---| +| `anthropic.APIConnectionError` | `openai.APIConnectionError` | 网络断 | ✅ | +| `anthropic.RateLimitError` | `openai.RateLimitError` | 429 限流 | ✅ | +| `anthropic.AuthenticationError` | `openai.AuthenticationError` | 401 key 错 | ❌ | +| `anthropic.APIStatusError` | `openai.APIStatusError` | 一般 HTTP 错 | 视 status code | + +两个 SDK 的 exception class 名字几乎一样、retry 逻辑**完全跨 backend**。换 SDK 只要改 import 那一行。 + +## 常见坑 + +1. **无脑 retry 所有 exception**：会把 AuthenticationError 也 retry 一遍、浪费 4 倍时间最后 still 401。RETRIABLE 白名单是核心 +2. **Backoff 不加 jitter**：1000 个 worker 同时被 rate limit、同时 1s 后重试、再次 rate limit → 死循。加 `random.uniform(0, 0.3)` 打散 +3. **max_attempts 太高**：retry 8 次 = 1+2+4+8+16+32+64+128 = 255 秒、user 早就 give up。`max_attempts=4` 通常够 +4. **没记录 attempt count**：production 一定要把 retry 次数加进 metric、超过 threshold 该 alert +5. **rate limit response 带 `Retry-After` header**：API 告诉你等多久、SDK 已自动处理，但自写 wrapper 别忽略这个 hint + +## 延伸 + +- **加 jitter strategy**：除了 uniform、可试 [decorrelated jitter](https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/)（更稳） +- **加 circuit breaker**：连续 N 次 retry 失败、暂时 stop call +- **改用 [tenacity](https://github.com/jd/tenacity)** library：production 不要自己写 retry、用成熟 lib +- **错误分类更细**：依 status code（429 / 503 / 502 / 500）给不同 backoff strategy +- **Stage 3 tool-level 错误**：看 [`../../stage-3/05-error-handling/`](../../stage-3/05-error-handling/)、结构化 tool error 让 LLM 在 ReAct loop 里决策 diff --git a/examples/stage-1/05-error-handling/requirements.txt b/examples/stage-1/05-error-handling/requirements.txt new file mode 100644 index 0000000..a4a9cdf --- /dev/null +++ b/examples/stage-1/05-error-handling/requirements.txt @@ -0,0 +1,2 @@ +openai>=1.50,<2.0 +anthropic>=0.40,<1.0 # 只 starter_anthropic.py 需要、Ollama path 不用 diff --git a/examples/stage-1/05-error-handling/starter.py b/examples/stage-1/05-error-handling/starter.py new file mode 100644 index 0000000..af07c80 --- /dev/null +++ b/examples/stage-1/05-error-handling/starter.py @@ -0,0 +1,137 @@ +""" +Stage 1 練習 5：Error Handling + Retry wrapper — Path A（Ollama 默認、本機免費）。 + +3 種錯誤情境 + 1 個 retry wrapper： + 1. API key 錯（401 AuthenticationError）→ 不要 retry、直接 raise + 2. Rate limit（429 RateLimitError）→ exponential backoff retry + 3. 網路錯（APIConnectionError）→ exponential backoff retry + +跑法： + pip install -r requirements.txt + ollama pull gemma4:e4b # Stage 1+2 預設、CPU-friendly + ollama serve # 預設 port 11434 + python starter.py + +驗證： + python test.py （mock 三種錯誤、不需真的斷網） + +想看 Anthropic 版本： + python starter_anthropic.py （需 ANTHROPIC_API_KEY） + +⚠️ 注意：本機 Ollama 不會真的撞 RateLimitError（沒 quota），所以「情境 2 rate limit」 +demo 看不到。但 `python test.py` 全部用 mock、retry 邏輯一樣可以完整驗證。 +這恰好是 Ollama path 反而更適合理解 retry pattern——快、免費、可重現。 +""" + +from __future__ import annotations + +import os +import random +import sys +import time +from typing import Any, Callable + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import ( + APIConnectionError, + APIStatusError, + AuthenticationError, + OpenAI, + RateLimitError, +) + +MODEL = os.environ.get("MODEL", "gemma4:e4b") + + +# === Retry wrapper === + +RETRIABLE = (APIConnectionError, RateLimitError) +MAX_ATTEMPTS = 4 +BASE_DELAY = 1.0 # 秒 + + +def with_retry(fn: Callable[[], Any], *, max_attempts: int = MAX_ATTEMPTS, base_delay: float = BASE_DELAY, sleep_fn=time.sleep) -> Any: + """ + Exponential backoff retry。 + - RETRIABLE 例外 → 等 base * 2^attempt 秒再試（含 jitter） + - 不 retriable 例外（譬如 AuthenticationError）→ 直接 raise、不浪費時間 + """ + last_exc = None + for attempt in range(max_attempts): + try: + return fn() + except RETRIABLE as e: # noqa: PERF203 + last_exc = e + if attempt == max_attempts - 1: + break + delay = base_delay * (2 ** attempt) + random.uniform(0, 0.3) + print(f" ⚠ attempt {attempt+1}/{max_attempts} fail ({type(e).__name__}); retry in {delay:.1f}s") + sleep_fn(delay) + raise last_exc # type: ignore[misc] + + +# === 3 個錯誤情境 demo === + +def demo_bad_key() -> None: + """情境 1: 故意用壞 base_url、看 APIConnectionError（Ollama 沒在跑時）。""" + print("\n[情境 1] 故意連到不存在的 Ollama port") + client = OpenAI(base_url="http://localhost:65535/v1", api_key="ollama") + try: + client.chat.completions.create( + model=MODEL, + max_tokens=10, + messages=[{"role": "user", "content": "hi"}], + ) + except APIConnectionError as e: + print(f" ✅ 抓到 APIConnectionError: {type(e).__name__}") + print(f" 💡 production 處理: retry（網路錯通常是 transient）") + + +def demo_with_retry() -> None: + """情境 2: 包 with_retry 跑一次正常 call、應該第 1 次就成功。""" + print("\n[情境 2] 正常 call、with_retry 包裝（需要 Ollama 在跑）") + client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + + def call(): + return client.chat.completions.create( + model=MODEL, + max_tokens=30, + messages=[{"role": "user", "content": "用一個 emoji 回答。"}], + ) + + try: + msg = with_retry(call) + print(f" ✅ 成功、第一次就過: {msg.choices[0].message.content}") + except APIConnectionError: + print(" ⚠ Ollama 沒在跑（port 11434 不通）。請先 `ollama serve`") + + +def demo_too_long_prompt() -> None: + """情境 3: 故意丟超大 prompt、看 context window 滿了怎樣。""" + print("\n[情境 3] Prompt 超過 context window（Ollama 通常會截斷或 raise）") + client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + huge_prompt = "重複很多次的 token。" * 200_000 # ~1M tokens + + try: + client.chat.completions.create( + model=MODEL, + max_tokens=10, + messages=[{"role": "user", "content": huge_prompt}], + ) + print(" ⚠ Ollama 沒 raise（可能直接截斷 prompt）。Cloud API 通常會 400") + except APIStatusError as e: + print(f" ✅ 抓到 APIStatusError: {e.status_code}") + print(f" 💡 production 處理: 在 client 端先 count token、超過就拒、別浪費 API call") + except APIConnectionError: + print(" ⚠ Ollama 沒在跑") + + +if __name__ == "__main__": + demo_bad_key() + demo_with_retry() + demo_too_long_prompt() + + # === 自我驗證 === + print("\n✅ 練習 5 通過 — 你已了解 3 種錯誤如何 raise、知道何時該 retry 何時該 stop、$0/run") diff --git a/examples/stage-1/05-error-handling/starter_anthropic.py b/examples/stage-1/05-error-handling/starter_anthropic.py new file mode 100644 index 0000000..eeb1e40 --- /dev/null +++ b/examples/stage-1/05-error-handling/starter_anthropic.py @@ -0,0 +1,121 @@ +""" +Stage 1 練習 5：Error Handling + Retry wrapper — Path B（Anthropic Claude）。 + +3 種錯誤情境 + 1 個 retry wrapper： + 1. API key 錯（401 AuthenticationError）→ 不要 retry、直接 raise + 2. Rate limit（429 RateLimitError）→ exponential backoff retry + 3. 網路錯（APIConnectionError）→ exponential backoff retry + +跑法： + pip install -r requirements.txt + export ANTHROPIC_API_KEY=sk-ant-... + python starter_anthropic.py + +預算：每次 ≈ $0.0005（只有「情境 2 正常 call」會真的打 API、claude-haiku-4-5）。 +Ollama 版本見 starter.py（本機 $0、用 openai SDK exception class）。 +""" + +from __future__ import annotations + +import os +import random +import sys +import time +from typing import Any, Callable + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic +from anthropic import ( + APIConnectionError, + APIStatusError, + AuthenticationError, + RateLimitError, +) + + +# === Retry wrapper === + +RETRIABLE = (APIConnectionError, RateLimitError) +MAX_ATTEMPTS = 4 +BASE_DELAY = 1.0 # 秒 + + +def with_retry(fn: Callable[[], Any], *, max_attempts: int = MAX_ATTEMPTS, base_delay: float = BASE_DELAY, sleep_fn=time.sleep) -> Any: + """ + Exponential backoff retry。 + - RETRIABLE 例外 → 等 base * 2^attempt 秒再試（含 jitter） + - 不 retriable 例外（譬如 AuthenticationError）→ 直接 raise、不浪費時間 + """ + last_exc = None + for attempt in range(max_attempts): + try: + return fn() + except RETRIABLE as e: # noqa: PERF203 + last_exc = e + if attempt == max_attempts - 1: + break # 最後一次失敗、break 出來 raise + delay = base_delay * (2 ** attempt) + random.uniform(0, 0.3) + print(f" ⚠ attempt {attempt+1}/{max_attempts} fail ({type(e).__name__}); retry in {delay:.1f}s") + sleep_fn(delay) + raise last_exc # type: ignore[misc] + + +# === 3 個錯誤情境 demo === + +def demo_bad_key() -> None: + """情境 1: 故意用壞 key、看 AuthenticationError 怎麼 raise。""" + print("\n[情境 1] 故意用壞 API key") + client = anthropic.Anthropic(api_key="sk-ant-FAKE-KEY-DO-NOT-USE") + try: + client.messages.create( + model="claude-haiku-4-5", + max_tokens=10, + messages=[{"role": "user", "content": "hi"}], + ) + except AuthenticationError as e: + print(f" ✅ 抓到 AuthenticationError: {e.status_code}") + print(f" 💡 production 處理: 立刻 alert、stop retry（key 不會自己變對）") + + +def demo_with_retry() -> None: + """情境 2: 包 with_retry 跑一次正常 call、應該第 1 次就成功。""" + print("\n[情境 2] 正常 call、with_retry 包裝") + client = anthropic.Anthropic() + + def call(): + return client.messages.create( + model="claude-haiku-4-5", + max_tokens=30, + messages=[{"role": "user", "content": "用一個 emoji 回答。"}], + ) + + msg = with_retry(call) + print(f" ✅ 成功、第一次就過: {msg.content[0].text}") + + +def demo_too_long_prompt() -> None: + """情境 3: 故意丟超大 prompt、看 context window 滿了怎樣。""" + print("\n[情境 3] Prompt 超過 context window") + client = anthropic.Anthropic() + huge_prompt = "重複很多次的 token。" * 300_000 # ~1.5M tokens、絕對超過任何 model + + try: + client.messages.create( + model="claude-haiku-4-5", + max_tokens=10, + messages=[{"role": "user", "content": huge_prompt}], + ) + except APIStatusError as e: + print(f" ✅ 抓到 APIStatusError: {e.status_code}") + print(f" 💡 production 處理: 在 client 端先 count token、超過就拒、別浪費 API call") + + +if __name__ == "__main__": + demo_bad_key() + demo_with_retry() + demo_too_long_prompt() + + # === 自我驗證 === + print("\n✅ 練習 5 通過 — 你已了解 3 種錯誤如何 raise、知道何時該 retry 何時該 stop") diff --git a/examples/stage-1/05-error-handling/test.py b/examples/stage-1/05-error-handling/test.py new file mode 100644 index 0000000..086cce6 --- /dev/null +++ b/examples/stage-1/05-error-handling/test.py @@ -0,0 +1,126 @@ +""" +Stage 1 練習 5 自我驗證 — Path A（Ollama starter.py）。 + +跑法： + python test.py + +驗證內容： + - with_retry 對 RETRIABLE 錯誤會 retry + - with_retry 對 non-retriable（譬如 AuthenticationError）直接 raise、不浪費 retry + - 超過 max_attempts 後 raise 最後一個 exception + - sleep 確實被叫（透過 mock sleep_fn） + - exponential backoff 真的指數增長 + +Anthropic 版本見 test_anthropic.py。 +""" + +from __future__ import annotations + +import sys +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import APIConnectionError, AuthenticationError, RateLimitError + +from starter import with_retry + + +def _make_connection_error(): + """openai.APIConnectionError 需要 request 參數、用 MagicMock 假裝。""" + return APIConnectionError(request=MagicMock()) + + +def _make_rate_limit_error(): + """openai.RateLimitError 需要 message + response + body。""" + return RateLimitError(message="rate limited", response=MagicMock(status_code=429), body=None) + + +def _make_auth_error(): + return AuthenticationError(message="bad key", response=MagicMock(status_code=401), body=None) + + +def test_no_retry_when_success_first_time(): + fn = MagicMock(return_value="ok") + sleep = MagicMock() + result = with_retry(fn, sleep_fn=sleep) + assert result == "ok" + assert fn.call_count == 1 + assert sleep.call_count == 0 + print("✅ test_no_retry_when_success_first_time") + + +def test_retry_on_connection_error_then_success(): + err = _make_connection_error() + fn = MagicMock(side_effect=[err, err, "ok"]) + sleep = MagicMock() + result = with_retry(fn, max_attempts=4, sleep_fn=sleep) + assert result == "ok" + assert fn.call_count == 3 + assert sleep.call_count == 2 + print("✅ test_retry_on_connection_error_then_success") + + +def test_retry_on_rate_limit(): + err = _make_rate_limit_error() + fn = MagicMock(side_effect=[err, "ok"]) + sleep = MagicMock() + result = with_retry(fn, sleep_fn=sleep) + assert result == "ok" + assert fn.call_count == 2 + print("✅ test_retry_on_rate_limit") + + +def test_raise_after_max_attempts(): + err = _make_connection_error() + fn = MagicMock(side_effect=[err, err, err, err]) + sleep = MagicMock() + try: + with_retry(fn, max_attempts=4, sleep_fn=sleep) + except APIConnectionError: + assert fn.call_count == 4 + assert sleep.call_count == 3 # 4 次 attempt 之間 sleep 3 次 + print("✅ test_raise_after_max_attempts") + return + raise AssertionError("應該 raise APIConnectionError") + + +def test_no_retry_on_auth_error(): + """AuthenticationError 不是 RETRIABLE、應該第一次就 raise、不 retry。""" + err = _make_auth_error() + fn = MagicMock(side_effect=err) + sleep = MagicMock() + try: + with_retry(fn, sleep_fn=sleep) + except AuthenticationError: + assert fn.call_count == 1, "AuthenticationError 不該被 retry" + assert sleep.call_count == 0 + print("✅ test_no_retry_on_auth_error") + return + raise AssertionError("應該 raise AuthenticationError") + + +def test_exponential_backoff_delays(): + """驗 sleep 的延遲時間隨 attempt 指數增長（base * 2^attempt）。""" + err = _make_connection_error() + fn = MagicMock(side_effect=[err, err, err, "ok"]) + delays = [] + sleep = MagicMock(side_effect=lambda d: delays.append(d)) + result = with_retry(fn, max_attempts=4, base_delay=1.0, sleep_fn=sleep) + assert result == "ok" + # 預期：attempt 0 後 sleep ~1s、attempt 1 後 ~2s、attempt 2 後 ~4s + assert 1.0 <= delays[0] < 1.5 + assert 2.0 <= delays[1] < 2.5 + assert 4.0 <= delays[2] < 4.5 + print("✅ test_exponential_backoff_delays") + + +if __name__ == "__main__": + test_no_retry_when_success_first_time() + test_retry_on_connection_error_then_success() + test_retry_on_rate_limit() + test_raise_after_max_attempts() + test_no_retry_on_auth_error() + test_exponential_backoff_delays() + print("\n🎉 全部通過 — Ollama path retry wrapper 邏輯正確（RETRIABLE 才 retry、exponential backoff 有效）") diff --git a/examples/stage-1/05-error-handling/test_anthropic.py b/examples/stage-1/05-error-handling/test_anthropic.py new file mode 100644 index 0000000..d9ad856 --- /dev/null +++ b/examples/stage-1/05-error-handling/test_anthropic.py @@ -0,0 +1,125 @@ +""" +Stage 1 練習 5 自我驗證 — Path B（Anthropic starter_anthropic.py）。 + +跑法： + python test_anthropic.py + +驗證內容： + - with_retry 對 RETRIABLE 錯誤會 retry + - with_retry 對 non-retriable（譬如 AuthenticationError）直接 raise、不浪費 retry + - 超過 max_attempts 後 raise 最後一個 exception + - sleep 確實被叫（透過 mock sleep_fn） + +Ollama 版本見 test.py（用 openai SDK 的 exception class）。 +""" + +from __future__ import annotations + +import sys +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from anthropic import APIConnectionError, AuthenticationError, RateLimitError + +from starter_anthropic import with_retry + + +def _make_connection_error(): + """anthropic.APIConnectionError 需要 request 參數、用 MagicMock 假裝。""" + return APIConnectionError(request=MagicMock()) + + +def _make_rate_limit_error(): + """anthropic.RateLimitError 需要 response 參數。""" + return RateLimitError(message="rate limited", response=MagicMock(status_code=429), body=None) + + +def _make_auth_error(): + return AuthenticationError(message="bad key", response=MagicMock(status_code=401), body=None) + + +def test_no_retry_when_success_first_time(): + fn = MagicMock(return_value="ok") + sleep = MagicMock() + result = with_retry(fn, sleep_fn=sleep) + assert result == "ok" + assert fn.call_count == 1 + assert sleep.call_count == 0 + print("✅ test_no_retry_when_success_first_time") + + +def test_retry_on_connection_error_then_success(): + err = _make_connection_error() + fn = MagicMock(side_effect=[err, err, "ok"]) + sleep = MagicMock() + result = with_retry(fn, max_attempts=4, sleep_fn=sleep) + assert result == "ok" + assert fn.call_count == 3 + assert sleep.call_count == 2 + print("✅ test_retry_on_connection_error_then_success") + + +def test_retry_on_rate_limit(): + err = _make_rate_limit_error() + fn = MagicMock(side_effect=[err, "ok"]) + sleep = MagicMock() + result = with_retry(fn, sleep_fn=sleep) + assert result == "ok" + assert fn.call_count == 2 + print("✅ test_retry_on_rate_limit") + + +def test_raise_after_max_attempts(): + err = _make_connection_error() + fn = MagicMock(side_effect=[err, err, err, err]) + sleep = MagicMock() + try: + with_retry(fn, max_attempts=4, sleep_fn=sleep) + except APIConnectionError: + assert fn.call_count == 4 + assert sleep.call_count == 3 # 4 次 attempt 之間 sleep 3 次 + print("✅ test_raise_after_max_attempts") + return + raise AssertionError("應該 raise APIConnectionError") + + +def test_no_retry_on_auth_error(): + """AuthenticationError 不是 RETRIABLE、應該第一次就 raise、不 retry。""" + err = _make_auth_error() + fn = MagicMock(side_effect=err) + sleep = MagicMock() + try: + with_retry(fn, sleep_fn=sleep) + except AuthenticationError: + assert fn.call_count == 1, "AuthenticationError 不該被 retry" + assert sleep.call_count == 0 + print("✅ test_no_retry_on_auth_error") + return + raise AssertionError("應該 raise AuthenticationError") + + +def test_exponential_backoff_delays(): + """驗 sleep 的延遲時間隨 attempt 指數增長（base * 2^attempt）。""" + err = _make_connection_error() + fn = MagicMock(side_effect=[err, err, err, "ok"]) + delays = [] + sleep = MagicMock(side_effect=lambda d: delays.append(d)) + result = with_retry(fn, max_attempts=4, base_delay=1.0, sleep_fn=sleep) + assert result == "ok" + # 預期：attempt 0 後 sleep ~1s、attempt 1 後 ~2s、attempt 2 後 ~4s + assert 1.0 <= delays[0] < 1.5 + assert 2.0 <= delays[1] < 2.5 + assert 4.0 <= delays[2] < 4.5 + print("✅ test_exponential_backoff_delays") + + +if __name__ == "__main__": + test_no_retry_when_success_first_time() + test_retry_on_connection_error_then_success() + test_retry_on_rate_limit() + test_raise_after_max_attempts() + test_no_retry_on_auth_error() + test_exponential_backoff_delays() + print("\n🎉 全部通過 — retry wrapper 邏輯正確（RETRIABLE 才 retry、exponential backoff 有效）") diff --git a/examples/stage-3/02-multi-tool-selection/README.en.md b/examples/stage-3/02-multi-tool-selection/README.en.md new file mode 100644 index 0000000..7aa2c55 --- /dev/null +++ b/examples/stage-3/02-multi-tool-selection/README.en.md @@ -0,0 +1,93 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# Exercise 2: Multi-Tool Selection + +Corresponds to [Stage 3 — Tool Use & Agent Intro](../../../stages/03-tool-use-and-hello-agent.en.md) Exercise 2. + +## Why this matters + +This exercise puts an LLM in front of three tools in a single turn: `web_search`, `calculator`, `calendar_lookup`. The point isn't tool quality — it's watching how schema `name` / `description` / `parameters` steer the model's choice. Writing schemas well is one of the highest-leverage things you do in Stage 3. + +## How to run — two paths + +### Path A (default, free, local) + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +Budget: **$0**. A single qwen2.5:3b tool call takes ~1-5s (CPU slower, GPU faster). + +### Path B (Anthropic, cloud-quality comparison) + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +Budget: ~**$0.0005** per run (claude-haiku-4-5). + +Expected output (Path A, local): + +``` +❓ Question: What is (19 * 42) - 8? Use the best available tool. (using Ollama qwen2.5:3b) + tool: calculator + tool_input: {'expression': '(19 * 42) - 8'} + observation: 790 +✅ Exercise 2 passed — you ran multi-tool selection locally on qwen2.5:3b, $0/run +``` + +## Validate the logic without API credits (mock-based) + +```bash +python test.py # validates Path A (Ollama) starter.py logic +python test_anthropic.py # validates Path B (Anthropic) starter_anthropic.py logic +``` + +Both test suites use `unittest.mock`, no real API call, $0/run. Path A uses the OpenAI-compat response shape; Path B uses Anthropic content blocks. + +## SDK differences between the two paths + +Three key differences (everything else is identical): + +| Part | Anthropic (Path B) | OpenAI-compat / Ollama (Path A) | +|---|---|---| +| Schema wrap | `tools=[{name, description, input_schema}, ...]` | `tools=[{"type": "function", "function": {name, description, parameters}}, ...]` | +| Reading tool call | `resp.content[i].type == "tool_use"` | `resp.choices[0].message.tool_calls[i]` | +| input format | `call.input` is already a dict | `call.function.arguments` is a JSON string — needs `json.loads(...)` | + +The selection **logic** is backend-agnostic — write a good schema and qwen2.5:3b picks the right tool too. This exercise is a great place to compare "on which questions does Claude pick the right tool but qwen2.5 doesn't?" — a clean way to feel the boundary of small models. + +## Common pitfalls + +The most common failure in multi-tool design is descriptions that read like documentation, not decision rules: + +- `calendar_lookup` described as "calendar" is ambiguous with `web_search`; "look up events for a specific date" is clearer +- `web_search` is for "external / recent / uncertain info", `calculator` for arithmetic — the clearer the boundary, the fewer wrong picks +- Small models (qwen2.5:3b) are **more sensitive** to description quality than Claude — the same schema where Claude might guess correctly can lead qwen astray + +## Want smarter answers? + +Default is `claude-haiku-4-5` (cheapest). Try Sonnet: + +```bash +MODEL=claude-sonnet-5 python starter_anthropic.py +``` + +Or on the Ollama path, swap to `qwen2.5:7b` (bigger, more stable, but slower): + +```bash +MODEL=qwen2.5:7b python starter.py +``` + +## Extensions + +- **Add more tools** — append one entry each to `TOOLS_SPEC` + `TOOL_IMPL` +- **Make it multi-turn ReAct** — wrap the single call in a `while` loop; see [`../03-react-from-scratch/`](../03-react-from-scratch/) +- **Dig into schema design** — see [`../06-schema-design/`](../06-schema-design/) for a bad vs good schema A/B diff --git a/examples/stage-3/02-multi-tool-selection/README.md b/examples/stage-3/02-multi-tool-selection/README.md new file mode 100644 index 0000000..9ffecba --- /dev/null +++ b/examples/stage-3/02-multi-tool-selection/README.md @@ -0,0 +1,100 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 練習 2：多工具選擇 + +對應 [Stage 3 — Tool Use & Agent 入門](../../../stages/03-tool-use-and-hello-agent.md) 練習 2。 +> 🎓 **學習模式**：這份 `starter.py` 是**完整解答**、不是 TODO skeleton。建議用**主動模式**——`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫一份 `starter.py`、跑 `python test.py` 驗證；卡 20 分鐘再回去對照 reference。完整方法論看 [`docs/HOW_TO_USE.md`](../../../docs/HOW_TO_USE.md)。 + +> 📚 **想要 chapter-length 深入版？** 本 folder 的 starter 是 70-150 行 illustrative 版、聚焦 `核心 pattern + 兩條 SDK path`，不是進階深度教材。深度教材推薦： +> - [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents) ⭐ 中文圈最完整、章節式 + 16 種 production 能力。**本練習對應 hello-agents 的 tool-calling / multi-tool dispatch 章節** +> - [Anthropic Tool Use Cookbook](https://github.com/anthropics/claude-cookbooks/tree/main/tool_use)（單工具→多工具→parallel 完整 notebook） +> - 完整 references 見 [Stage 3 精選 Projects](../../../stages/03-tool-use-and-hello-agent.md#-精選-projects) + + +## 為什麼這題重要 + +這個練習讓 LLM 在同一輪面對三個工具：`web_search`、`calculator`、`calendar_lookup`。重點不是工具本身強不強，而是觀察 schema 的 `name` / `description` / `parameters` 如何決定模型挑哪一個。寫清楚 schema、是 Stage 3 最值得花時間的子題。 + +## 怎麼跑 — 兩條路徑 + +### Path A（默認、本機免費） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +預算：**$0**。qwen2.5:3b 單輪 tool call ≈ 1-5 秒（CPU 慢、GPU 快）。 + +### Path B（Anthropic、想看 cloud 高品質） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +預算：每次 ≈ **$0.0005**（claude-haiku-4-5）。 + +預期看到（Path A、本機）： + +``` +❓ 問題：What is (19 * 42) - 8? Use the best available tool.（using Ollama qwen2.5:3b） + tool: calculator + tool_input: {'expression': '(19 * 42) - 8'} + observation: 790 +✅ 練習 2 通過 — 你已用本機 qwen2.5:3b 跑通 multi-tool selection、$0/run +``` + +## 不花錢驗證程式邏輯（mock-based） + +```bash +python test.py # 驗 Path A (Ollama) starter.py 邏輯 +python test_anthropic.py # 驗 Path B (Anthropic) starter_anthropic.py 邏輯 +``` + +兩條 test 都用 `unittest.mock`、不打真 API、$0/run。Path A 用 OpenAI-compat response shape、Path B 用 Anthropic content blocks。 + +## 兩條 path 的 SDK 差異 + +三個關鍵差異（其他完全一樣）： + +| 部分 | Anthropic（Path B） | OpenAI-compat / Ollama（Path A） | +|---|---|---| +| Schema 包法 | `tools=[{name, description, input_schema}, ...]` | `tools=[{"type": "function", "function": {name, description, parameters}}, ...]` | +| 抓 tool call | `resp.content[i].type == "tool_use"` | `resp.choices[0].message.tool_calls[i]` | +| input 格式 | `call.input` 是 dict（自動 parse） | `call.function.arguments` 是 JSON string、要 `json.loads(...)` | + +Tool selection **邏輯本身**跨 backend——schema 寫好、qwen2.5:3b 也會挑對 tool。這題很適合拿來對照 Claude vs qwen2.5「在哪幾題會挑錯」，是觀察小 model 邊界的好實驗。 + +## 容易踩坑 + +多工具選擇最常見的錯誤是 description 寫得太像「一般說明文件」，而不是「給模型做決策的判斷規則」： + +- `calendar_lookup` 描述只說「行事曆」就會跟 `web_search` 邊界模糊；明寫「查特定日期事件」才好 +- `web_search` 適合「外部 / 近期 / 不確定資訊」、`calculator` 只處理算式；邊界寫越清楚、模型越少誤判 +- 小 model（qwen2.5:3b）對 description 質量比 Claude **更敏感**——同一份 schema、Claude 可能還能猜對、qwen 直接挑錯 + +## 想看更聰明的答案？ + +預設用 `claude-haiku-4-5`（最便宜）。改成 sonnet： + +```bash +MODEL=claude-sonnet-5 python starter_anthropic.py +``` + +或在 Ollama path 換 `qwen2.5:7b`（更大、更穩、但慢）： + +```bash +MODEL=qwen2.5:7b python starter.py +``` + +## 延伸 + +- **加更多 tool**：在 `TOOLS_SPEC` + `TOOL_IMPL` 補一個 entry 即可 +- **改成多輪 ReAct**：把單輪 call 包進 while loop，看 [`../03-react-from-scratch/`](../03-react-from-scratch/) +- **schema 細節**：看 [`../06-schema-design/`](../06-schema-design/) 比較 bad / good schema 對選擇正確率的影響 diff --git a/examples/stage-3/02-multi-tool-selection/README.zh-Hans.md b/examples/stage-3/02-multi-tool-selection/README.zh-Hans.md new file mode 100644 index 0000000..3c12204 --- /dev/null +++ b/examples/stage-3/02-multi-tool-selection/README.zh-Hans.md @@ -0,0 +1,93 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 练习 2：多工具选择 + +对应 [Stage 3 — Tool Use & Agent 入门](../../../stages/03-tool-use-and-hello-agent.zh-Hans.md) 练习 2。 + +## 为什么这题重要 + +这个练习让 LLM 在同一轮面对三个工具：`web_search`、`calculator`、`calendar_lookup`。重点不是工具本身强不强，而是观察 schema 的 `name` / `description` / `parameters` 如何决定模型挑哪一个。把 schema 写清楚，是 Stage 3 最值得花时间的子题。 + +## 怎么跑 — 两条路径 + +### Path A（默认、本机免费） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +预算：**$0**。qwen2.5:3b 单轮 tool call ≈ 1-5 秒（CPU 慢、GPU 快）。 + +### Path B（Anthropic、想看 cloud 高质量） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +预算：每次 ≈ **$0.0005**（claude-haiku-4-5）。 + +预期看到（Path A、本机）： + +``` +❓ 问题：What is (19 * 42) - 8? Use the best available tool.（using Ollama qwen2.5:3b） + tool: calculator + tool_input: {'expression': '(19 * 42) - 8'} + observation: 790 +✅ 练习 2 通过 — 你已用本机 qwen2.5:3b 跑通 multi-tool selection、$0/run +``` + +## 不花钱验证程序逻辑（mock-based） + +```bash +python test.py # 验 Path A (Ollama) starter.py 逻辑 +python test_anthropic.py # 验 Path B (Anthropic) starter_anthropic.py 逻辑 +``` + +两条 test 都用 `unittest.mock`、不打真 API、$0/run。Path A 用 OpenAI-compat response shape、Path B 用 Anthropic content blocks。 + +## 两条 path 的 SDK 差异 + +三个关键差异（其他完全一样）： + +| 部分 | Anthropic（Path B） | OpenAI-compat / Ollama（Path A） | +|---|---|---| +| Schema 包法 | `tools=[{name, description, input_schema}, ...]` | `tools=[{"type": "function", "function": {name, description, parameters}}, ...]` | +| 抓 tool call | `resp.content[i].type == "tool_use"` | `resp.choices[0].message.tool_calls[i]` | +| input 格式 | `call.input` 是 dict（自动 parse） | `call.function.arguments` 是 JSON string、要 `json.loads(...)` | + +Tool selection **逻辑本身**跨 backend——schema 写好、qwen2.5:3b 也会挑对 tool。这题很适合拿来对照 Claude vs qwen2.5“在哪几题会挑错”，是观察小 model 边界的好实验。 + +## 容易踩坑 + +多工具选择最常见的错误是 description 写得太像“一般说明文档”，而不是“给模型做决策的判断规则”： + +- `calendar_lookup` 描述只说“行事历”就会跟 `web_search` 边界模糊；明写“查特定日期事件”才好 +- `web_search` 适合“外部 / 近期 / 不确定信息”、`calculator` 只处理算式；边界写越清楚、模型越少误判 +- 小 model（qwen2.5:3b）对 description 质量比 Claude **更敏感**——同一份 schema、Claude 可能还能猜对、qwen 直接挑错 + +## 想看更聪明的答案？ + +预设用 `claude-haiku-4-5`（最便宜）。改成 sonnet： + +```bash +MODEL=claude-sonnet-5 python starter_anthropic.py +``` + +或在 Ollama path 换 `qwen2.5:7b`（更大、更稳、但慢）： + +```bash +MODEL=qwen2.5:7b python starter.py +``` + +## 延伸 + +- **加更多 tool**：在 `TOOLS_SPEC` + `TOOL_IMPL` 补一个 entry 即可 +- **改成多轮 ReAct**：把单轮 call 包进 while loop，看 [`../03-react-from-scratch/`](../03-react-from-scratch/) +- **schema 细节**：看 [`../06-schema-design/`](../06-schema-design/) 比较 bad / good schema 对选择正确率的影响 diff --git a/examples/stage-3/02-multi-tool-selection/requirements.txt b/examples/stage-3/02-multi-tool-selection/requirements.txt new file mode 100644 index 0000000..a4a9cdf --- /dev/null +++ b/examples/stage-3/02-multi-tool-selection/requirements.txt @@ -0,0 +1,2 @@ +openai>=1.50,<2.0 +anthropic>=0.40,<1.0 # 只 starter_anthropic.py 需要、Ollama path 不用 diff --git a/examples/stage-3/02-multi-tool-selection/starter.py b/examples/stage-3/02-multi-tool-selection/starter.py new file mode 100644 index 0000000..24461d2 --- /dev/null +++ b/examples/stage-3/02-multi-tool-selection/starter.py @@ -0,0 +1,120 @@ +"""練習 2：多工具選擇 — Path A（Ollama 默認、本機免費）。 + +讓本機 qwen2.5:3b 在 3 個 tool（web_search / calculator / calendar_lookup）裡選一個。 +重點不是工具強不強，是觀察 schema 的 description / 參數 / required 如何引導模型選對。 + +跑法： + pip install -r requirements.txt + ollama pull qwen2.5:3b # Stage 3+ tool-use 默認 model + ollama serve # 預設 port 11434 + python starter.py + +驗證： + python test.py （用 mock、不打 API） + +想看 Anthropic Claude 版本： + python starter_anthropic.py （需 ANTHROPIC_API_KEY、$0.0005/run） +""" + +from __future__ import annotations + +import json +import os +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +MODEL = os.environ.get("MODEL", "qwen2.5:3b") # tool-use 穩定的 Ollama model + + +# === 1. Tools 定義（含實作）=== + +def web_search(query: str) -> str: + return f"search result: {query} -> Anthropic tool use docs and examples" + + +def calculator(expression: str) -> str: + allowed = set("0123456789.+-*/() ") + if any(ch not in allowed for ch in expression): + return "error: calculator only accepts basic arithmetic" + try: + return str(eval(expression, {"__builtins__": {}}, {})) # noqa: S307 + except Exception as exc: # noqa: BLE001 + return f"error: {exc}" + + +def calendar_lookup(date: str) -> str: + events = { + "2026-05-13": "10:00 Stage 3 review, 15:00 agent study group", + "tomorrow": "10:00 Stage 3 review, 15:00 agent study group", + } + return events.get(date.strip(), f"no events found for {date}") + + +# OpenAI-compat 的 tools schema 要包一層 {"type": "function", "function": {...}} +def _wrap(name: str, description: str, field: str, field_description: str) -> dict: + return { + "type": "function", + "function": { + "name": name, + "description": description, + "parameters": { + "type": "object", + "properties": {field: {"type": "string", "description": field_description}}, + "required": [field], + }, + }, + } + + +TOOLS_SPEC = [ + _wrap("web_search", "Search current or external information not in the prompt.", "query", "Search query"), + _wrap("calculator", "Evaluate basic arithmetic with +, -, *, /, and parentheses.", "expression", "Math expression"), + _wrap("calendar_lookup", "Look up events for a specific date or relative day.", "date", "Date to inspect"), +] + +TOOL_IMPL = { + "web_search": lambda args: web_search(args["query"]), + "calculator": lambda args: calculator(args["expression"]), + "calendar_lookup": lambda args: calendar_lookup(args["date"]), +} + + +# === 2. 單輪 tool selection === + +def run_tool_selection(question: str, client: Any = None) -> dict: + """單輪 call：LLM 看完 question + tools 後選一個 tool 呼叫，本地執行 observation 接回去。""" + client = client or OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + resp = client.chat.completions.create( + model=MODEL, + tools=TOOLS_SPEC, + messages=[{"role": "user", "content": question}], + ) + msg = resp.choices[0].message + text = msg.content or "" + tool_calls = msg.tool_calls or [] + if not tool_calls: + return {"tool": None, "thought": text, "observation": None} + call = tool_calls[0] + args = json.loads(call.function.arguments) + fn = TOOL_IMPL.get(call.function.name, lambda _: f"error: unknown tool {call.function.name}") + return {"tool": call.function.name, "tool_input": args, "thought": text, "observation": fn(args)} + + +# === 3. 自我驗證 === + +if __name__ == "__main__": + question = "What is (19 * 42) - 8? Use the best available tool." + print(f"❓ 問題：{question}（using Ollama {MODEL}）") + result = run_tool_selection(question) + print(f" tool: {result['tool']}") + print(f" tool_input: {result.get('tool_input')}") + print(f" observation: {result['observation']}") + + assert result["tool"] == "calculator", f"預期 calculator、得到 {result['tool']}" + assert result["observation"] and not result["observation"].startswith("error:") + print("✅ 練習 2 通過 — 你已用本機 qwen2.5:3b 跑通 multi-tool selection、$0/run") diff --git a/examples/stage-3/02-multi-tool-selection/starter_anthropic.py b/examples/stage-3/02-multi-tool-selection/starter_anthropic.py new file mode 100644 index 0000000..7201c65 --- /dev/null +++ b/examples/stage-3/02-multi-tool-selection/starter_anthropic.py @@ -0,0 +1,90 @@ +"""練習 2：多工具選擇 — Path B（Anthropic Claude）。 + +讓 Claude 在 3 個 tool（web_search / calculator / calendar_lookup）裡選一個執行。 +重點：tool schema 的 description 越精準、模型選對的機率越高。 + +跑法： + pip install -r requirements.txt + export ANTHROPIC_API_KEY=sk-ant-... + python starter_anthropic.py + +預算：每次 ≈ $0.0005（claude-haiku-4-5、單輪 call）。 +Ollama 版本見 starter.py（本機 $0）。 +""" + +from __future__ import annotations + +import os +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic + +MODEL = os.environ.get("MODEL", "claude-haiku-4-5") + + +def web_search(query: str) -> str: + return f"search result: {query} -> Anthropic tool use docs and examples" + + +def calculator(expression: str) -> str: + allowed = set("0123456789.+-*/() ") + if any(ch not in allowed for ch in expression): + return "error: calculator only accepts basic arithmetic" + try: + return str(eval(expression, {"__builtins__": {}}, {})) # noqa: S307 + except Exception as exc: # noqa: BLE001 + return f"error: {exc}" + + +def calendar_lookup(date: str) -> str: + events = {"2026-05-13": "10:00 Stage 3 review, 15:00 agent study group", "tomorrow": "10:00 Stage 3 review, 15:00 agent study group"} + return events.get(date.strip(), f"no events found for {date}") + + +def tool_schema(name: str, description: str, field: str, field_description: str) -> dict: + return {"name": name, "description": description, "input_schema": {"type": "object", "properties": {field: {"type": "string", "description": field_description}}, "required": [field]}} + + +TOOLS_SPEC = [ + tool_schema("web_search", "Search current or external information not in the prompt.", "query", "Search query"), + tool_schema("calculator", "Evaluate basic arithmetic with +, -, *, /, and parentheses.", "expression", "Math expression"), + tool_schema("calendar_lookup", "Look up events for a specific date or relative day.", "date", "Date to inspect"), +] + +TOOL_IMPL = { + "web_search": lambda args: web_search(args["query"]), + "calculator": lambda args: calculator(args["expression"]), + "calendar_lookup": lambda args: calendar_lookup(args["date"]), +} + + +def run_tool_selection(question: str, client: Any = None) -> dict: + client = client or anthropic.Anthropic() + resp = client.messages.create( + model=MODEL, + max_tokens=512, + tools=TOOLS_SPEC, + messages=[{"role": "user", "content": question}], + ) + text = " ".join(getattr(b, "text", "") for b in resp.content if getattr(b, "type", None) == "text") + tool_calls = [b for b in resp.content if getattr(b, "type", None) == "tool_use"] + if not tool_calls: + return {"tool": None, "thought": text, "observation": None} + call = tool_calls[0] + args = dict(call.input) + observation = TOOL_IMPL.get(call.name, lambda _: f"error: unknown tool {call.name}")(args) + return {"tool": call.name, "tool_input": args, "thought": text, "observation": observation} + + +if __name__ == "__main__": + result = run_tool_selection("What is (19 * 42) - 8? Use the best available tool.") + print(result) + + # === 自我檢查 === + assert result["tool"] == "calculator", f"expected calculator, got {result['tool']}" + assert result["observation"] and not result["observation"].startswith("error:") + print("Stage 3 exercise 2 starter check passed") diff --git a/examples/stage-3/02-multi-tool-selection/test.py b/examples/stage-3/02-multi-tool-selection/test.py new file mode 100644 index 0000000..e536c39 --- /dev/null +++ b/examples/stage-3/02-multi-tool-selection/test.py @@ -0,0 +1,101 @@ +"""練習 2 自我驗證 — Path A（Ollama starter.py）。 + +跑法： + python test.py + +驗證內容： + - tool 清單完整、calculator 邏輯正確 + - LLM 三種題目分別選到 calculator / calendar / web_search + - mock 用 OpenAI-compat shape（不需要 Anthropic SDK） + +Anthropic 版本 test 見 test_anthropic.py。 +""" + +from __future__ import annotations + +import json +import sys +from types import SimpleNamespace +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter import TOOLS_SPEC, calculator, run_tool_selection + + +# === Helpers for building mock OpenAI-compat responses === + +def make_tool_call(call_id: str, name: str, args: dict): + return SimpleNamespace( + id=call_id, + type="function", + function=SimpleNamespace(name=name, arguments=json.dumps(args)), + ) + + +def make_resp(content: str, tool_calls=None, finish_reason: str = "tool_calls"): + msg = SimpleNamespace(content=content, tool_calls=tool_calls) + return SimpleNamespace(choices=[SimpleNamespace(finish_reason=finish_reason, message=msg)]) + + +def mock_client_for(name: str, args: dict) -> MagicMock: + client = MagicMock() + client.chat.completions.create.return_value = make_resp( + f"I should use {name}.", + [make_tool_call("call_1", name, args)], + ) + return client + + +# === Tests === + +def test_tool_spec_contains_three_tools(): + names = [t["function"]["name"] for t in TOOLS_SPEC] + assert names == ["web_search", "calculator", "calendar_lookup"] + assert calculator("2 * (3 + 4)") == "14" + print("✅ test_tool_spec_contains_three_tools") + + +def test_llm_selects_calculator(): + client = mock_client_for("calculator", {"expression": "20 / 5"}) + result = run_tool_selection("What is 20 divided by 5?", client=client) + assert result["tool"] == "calculator" + assert result["observation"] == "4.0" + print("✅ test_llm_selects_calculator") + + +def test_llm_selects_calendar(): + client = mock_client_for("calendar_lookup", {"date": "tomorrow"}) + result = run_tool_selection("Do I have anything tomorrow?", client=client) + assert result["tool"] == "calendar_lookup" + assert "Stage 3 review" in result["observation"] + print("✅ test_llm_selects_calendar") + + +def test_llm_uses_search_instead_of_calendar_for_news(): + client = mock_client_for("web_search", {"query": "latest Claude tool use examples"}) + result = run_tool_selection("Find recent examples of Claude tool use.", client=client) + assert result["tool"] == "web_search" + assert "latest Claude tool use examples" in result["observation"] + print("✅ test_llm_uses_search_instead_of_calendar_for_news") + + +def test_no_tool_call_returns_none(): + client = MagicMock() + client.chat.completions.create.return_value = make_resp( + "I don't think any of these tools applies.", tool_calls=None, finish_reason="stop", + ) + result = run_tool_selection("Tell me a joke.", client=client) + assert result["tool"] is None + assert result["observation"] is None + print("✅ test_no_tool_call_returns_none") + + +if __name__ == "__main__": + test_tool_spec_contains_three_tools() + test_llm_selects_calculator() + test_llm_selects_calendar() + test_llm_uses_search_instead_of_calendar_for_news() + test_no_tool_call_returns_none() + print("\n🎉 全部通過 — Ollama path tool selection 邏輯正確") diff --git a/examples/stage-3/02-multi-tool-selection/test_anthropic.py b/examples/stage-3/02-multi-tool-selection/test_anthropic.py new file mode 100644 index 0000000..e69e4c6 --- /dev/null +++ b/examples/stage-3/02-multi-tool-selection/test_anthropic.py @@ -0,0 +1,75 @@ +"""Stage 3 練習 2 自我驗證 — Path B（Anthropic starter_anthropic.py）。 + +跑法： + python test_anthropic.py + +用 mock 取代 Anthropic client、不打真 API、$0/run。 +Ollama 版本見 test.py（OpenAI-compat shape）。 +""" + +from __future__ import annotations + +import sys +from types import SimpleNamespace +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter_anthropic import TOOLS_SPEC, calculator, run_tool_selection + + +def block_text(text: str): + return SimpleNamespace(type="text", text=text) + + +def block_tool_use(tool_id: str, name: str, inp: dict): + return SimpleNamespace(type="tool_use", id=tool_id, name=name, input=inp) + + +def make_resp(stop_reason: str, *blocks): + return SimpleNamespace(stop_reason=stop_reason, content=list(blocks)) + + +def mock_client_for(name: str, inp: dict) -> MagicMock: + client = MagicMock() + client.messages.create.return_value = make_resp( + "tool_use", + block_text(f"I should use {name}."), + block_tool_use("toolu_1", name, inp), + ) + return client + + +def test_tool_spec_contains_three_tools(): + assert [tool["name"] for tool in TOOLS_SPEC] == ["web_search", "calculator", "calendar_lookup"] + assert calculator("2 * (3 + 4)") == "14" + + +def test_llm_selects_calculator(): + client = mock_client_for("calculator", {"expression": "20 / 5"}) + result = run_tool_selection("What is 20 divided by 5?", client=client) + assert result["tool"] == "calculator" + assert result["observation"] == "4.0" + + +def test_llm_selects_calendar(): + client = mock_client_for("calendar_lookup", {"date": "tomorrow"}) + result = run_tool_selection("Do I have anything tomorrow?", client=client) + assert result["tool"] == "calendar_lookup" + assert "Stage 3 review" in result["observation"] + + +def test_llm_uses_search_instead_of_calendar_for_news(): + client = mock_client_for("web_search", {"query": "latest Claude tool use examples"}) + result = run_tool_selection("Find recent examples of Claude tool use.", client=client) + assert result["tool"] == "web_search" + assert "latest Claude tool use examples" in result["observation"] + + +if __name__ == "__main__": + test_tool_spec_contains_three_tools() + test_llm_selects_calculator() + test_llm_selects_calendar() + test_llm_uses_search_instead_of_calendar_for_news() + print("all pass") diff --git a/examples/stage-3/03-react-from-scratch/README.en.md b/examples/stage-3/03-react-from-scratch/README.en.md new file mode 100644 index 0000000..da38b63 --- /dev/null +++ b/examples/stage-3/03-react-from-scratch/README.en.md @@ -0,0 +1,105 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# Exercise 3: ReAct from Scratch (no framework) + +Corresponds to [Stage 3 — Tool Use & Agent Intro](../../../stages/03-tool-use-and-hello-agent.en.md) Exercise 3. + +## Why write it from scratch + +ReAct (Reasoning + Acting) is the foundational pattern of modern agents: + +``` +while not done: + thought = LLM reads current context and verbalizes the next step + action = LLM calls a tool + observation = tool result, fed back to the LLM +``` + +LangGraph / CrewAI hide this loop from you. **Writing it once yourself** is what teaches you: +- Why the `messages` array keeps growing +- How `tool_use_id` pairs with `tool_result` +- Why `stop_reason` is `tool_use` vs `end_turn` +- Why `max_iter` is a mandatory safety net + +All of that is covered in 70 lines of Python. + +## How to run + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter.py +``` + +Expected: + +``` +❓ Question: Divide 'Taipei population' by 'NYC population', 4 decimal places. +------------------------------------------------------------ +[step 0] thought: Let me look up Taipei's population... + tool: lookup_fact({'query': '台北人口'}) → 2602000 +[step 1] thought: Now NYC's... + tool: lookup_fact({'query': '紐約人口'}) → 8336000 +[step 2] thought: Compute the ratio... + tool: calculator({'expression': '2602000 / 8336000'}) → 0.3121... +[step 3] thought: The answer is 0.3122. +------------------------------------------------------------ +✅ Final answer: Taipei / NYC ≈ 0.3122 + Took 4 rounds. +✅ Exercise 3 passed — the ReAct loop chained lookup_fact and calculator on its own. +``` + +## Validate the logic without spending API credits + +```bash +python test.py +``` + +`test.py` uses `unittest.mock.MagicMock` to replace the Anthropic client and feed canned responses, validating your loop logic. Expected: + +``` +✅ test_calculator_basic +✅ test_calculator_rejects_eval_injection +✅ test_lookup_fact +✅ test_react_loop_single_tool_call +✅ test_react_loop_multi_step +✅ test_react_loop_respects_max_iter + +🎉 All tests passed — your ReAct loop logic is correct. +``` + +## Program structure walkthrough + +| Section | Lines | What it does | +|---|---|---| +| `tool_calculator` | ~30-40 | Safe calculator (whitelist filter, avoids `eval` injection) | +| `tool_lookup_fact` | ~42-50 | Fake fact lookup (teaching-only, avoids external API dep) | +| `TOOLS_SPEC` | ~52-75 | Tool schema that the LLM sees | +| `TOOL_IMPL` | ~77-80 | name → callable dispatch table | +| `react_loop` | ~85-130 | Main loop, with max_iter safety, `messages` accumulation, tool_result wiring | + +## Common pitfalls + +1. **Forgetting to append the assistant response to messages** — next round the LLM can't see what it just said, leading to infinite loops +2. **Not passing `tool_use_id` with tool_result** — the LLM can't pair results to calls +3. **`while True` without `max_iter`** — if a tool returns garbage the LLM may call it forever; safety net is mandatory +4. **Unfiltered eval** — `eval(user_input)` in calculator = RCE; use a whitelist or `ast.literal_eval` + +## Want smarter answers? + +Default model is `claude-haiku-4-5` (cheapest). Switch to Sonnet: + +```bash +MODEL=claude-sonnet-5 python starter.py +``` + +Or change `MODEL = ...` in `starter.py`. + +## Extensions + +- **Add more tools** — append one entry each to `TOOLS_SPEC` + `TOOL_IMPL` +- **Add streaming** — swap `client.messages.create(...)` for `with client.messages.stream(...) as s:`, print as it goes +- **Add prompt cache** — pass `cache_control={"type":"ephemeral"}` on `system=` or `tools=` to save 90% on repeat calls +- **Plug into [LangGraph](https://langchain-ai.github.io/langgraph/) or [Pydantic AI](https://ai.pydantic.dev/)** to see how frameworks hide these 70 lines diff --git a/examples/stage-3/03-react-from-scratch/README.md b/examples/stage-3/03-react-from-scratch/README.md new file mode 100644 index 0000000..7f7e722 --- /dev/null +++ b/examples/stage-3/03-react-from-scratch/README.md @@ -0,0 +1,130 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 練習 3：從零實作 ReAct（不用 framework） + +對應 [Stage 3 — Tool Use & Agent 入門](../../../stages/03-tool-use-and-hello-agent.md) 練習 3。 +> 🎓 **學習模式**：這份 `starter.py` 是**完整解答**、不是 TODO skeleton。建議用**主動模式**——`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫一份 `starter.py`、跑 `python test.py` 驗證；卡 20 分鐘再回去對照 reference。完整方法論看 [`docs/HOW_TO_USE.md`](../../../docs/HOW_TO_USE.md)。 + +> 📚 **想要 chapter-length 深入版？** 本 folder 的 starter 是 70-150 行 illustrative 版、聚焦 `核心 pattern + 兩條 SDK path`，不是進階深度教材。深度教材推薦： +> - [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents) ⭐ 中文圈最完整、章節式 + 16 種 production 能力。**本練習對應 hello-agents 的 ReAct 章節（搭配 [`learn_version` 分支](https://github.com/jjyaoao/HelloAgents/tree/learn_version)）** +> - [ReAct 原論文](https://arxiv.org/abs/2210.03629)（Yao et al. 2022 第 3 節） + [pguso/ai-agents-from-scratch](https://github.com/pguso/ai-agents-from-scratch)（本機 LLM 從零實作） +> - 完整 references 見 [Stage 3 精選 Projects](../../../stages/03-tool-use-and-hello-agent.md#-精選-projects) + + +## 為什麼從零寫 + +ReAct（Reasoning + Acting）是現代 agent 的基礎 pattern： + +``` +while not done: + thought = LLM 看完目前 context、講出下一步要做什麼 + action = LLM 呼叫一個 tool + observation = tool 執行結果、餵回去給 LLM +``` + +LangGraph / CrewAI 把這個 loop 藏起來了。你**自己寫過一次**才知道： +- 為什麼 messages array 一直長 +- tool_use_id 跟 tool_result 怎麼配對 +- stop_reason 為什麼是 `tool_use` 或 `end_turn` +- max_iter 為什麼是 safety net + +70 行 Python 全交代清楚。 + +## 怎麼跑 — 兩條路徑 + +### Path A（默認、本機免費） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +預算：**$0**。本機 qwen2.5:3b 跑 ReAct loop 4-6 輪 ≈ 30-120 秒（CPU 慢、GPU 快）。 + +### Path B（Anthropic、想看 cloud 高品質） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +預算：每次 ≈ **$0.001** (claude-haiku-4-5)。比本機快 5-15 倍、答案品質更穩。 + +預期看到（Path A、本機）： + +``` +❓ 問題：'台北人口' 除以 '紐約人口'、答案保留 4 位小數。 +------------------------------------------------------------ +[step 0] thought: 我先查台北人口... + tool: lookup_fact({'query': '台北人口'}) → 2602000 +[step 1] thought: 接著查紐約人口... + tool: lookup_fact({'query': '紐約人口'}) → 8336000 +[step 2] thought: 計算比例... + tool: calculator({'expression': '2602000 / 8336000'}) → 0.3121... +[step 3] thought: 答案是 0.3122 +------------------------------------------------------------ +✅ 最終答案：台北人口除以紐約人口約 0.3122 + 共 4 輪 +✅ 練習 3 通過 — ReAct loop 自己連用了 lookup_fact 跟 calculator +``` + +## 不花錢驗證程式邏輯（mock-based） + +```bash +python test.py # 驗 Path A (Ollama) starter.py 邏輯 +python test_anthropic.py # 驗 Path B (Anthropic) starter_anthropic.py 邏輯 +``` + +兩條 test 都用 `unittest.mock`、不打真 API、$0/run。Path A 用 OpenAI-compat response shape、Path B 用 Anthropic content blocks。 + +`test.py` 用 `unittest.mock.MagicMock` 取代 Anthropic client、塞固定 response、驗證你的 loop 邏輯。預期： + +``` +✅ test_calculator_basic +✅ test_calculator_rejects_eval_injection +✅ test_lookup_fact +✅ test_react_loop_single_tool_call +✅ test_react_loop_multi_step +✅ test_react_loop_respects_max_iter + +🎉 全部通過 — 你的 ReAct loop 邏輯正確 +``` + +## 程式結構走查 + +| 段 | 行 | 在做什麼 | +|---|---|---| +| `tool_calculator` | ~30-40 | 安全的計算器（whitelist 過濾、避免 `eval` 漏洞） | +| `tool_lookup_fact` | ~42-50 | 假事實庫（教學用、避免依賴外部 API） | +| `TOOLS_SPEC` | ~52-75 | tool schema 給 LLM 看 | +| `TOOL_IMPL` | ~77-80 | name → callable 對應表（dispatch） | +| `react_loop` | ~85-130 | 主迴圈、含 max_iter safety、`messages` 累積、tool result 接回去 | + +## 常見坑 + +1. **忘記把 assistant response 加進 messages**：下一輪 LLM 就看不到自己上一輪講過什麼、會 loop forever +2. **tool_result 沒帶 `tool_use_id`**：LLM 無法配對哪個 result 對應哪個 call +3. **`while True` 沒 max_iter**：tool 結果寫得不好、LLM 會無限呼叫；safety net 一定要設 +4. **eval 沒過濾**：calculator 直接 `eval(user_input)` = RCE 漏洞；用 whitelist 或 `ast.literal_eval` + +## 想看更聰明的答案？ + +預設用 `claude-haiku-4-5`（最便宜）。改成 sonnet： + +```bash +MODEL=claude-sonnet-5 python starter.py +``` + +或在 `starter.py` 改 `MODEL = ...` 那行。 + +## 延伸 + +- **加更多 tool**：在 `TOOLS_SPEC` + `TOOL_IMPL` 補一個 entry 即可 +- **加 streaming**：把 `client.messages.create(...)` 換成 `with client.messages.stream(...) as s:`、邊跑邊印 +- **加 prompt cache**：在 `system=` 或 `tools=` 帶 `cache_control={"type":"ephemeral"}` 重複 call 省 90% token +- **接 [LangGraph](https://langchain-ai.github.io/langgraph/) 或 [Pydantic AI](https://ai.pydantic.dev/) 看 framework 怎麼幫你藏掉這 70 行** diff --git a/examples/stage-3/03-react-from-scratch/README.zh-Hans.md b/examples/stage-3/03-react-from-scratch/README.zh-Hans.md new file mode 100644 index 0000000..08f779a --- /dev/null +++ b/examples/stage-3/03-react-from-scratch/README.zh-Hans.md @@ -0,0 +1,105 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 练习 3：从零实现 ReAct（不用 framework） + +对应 [Stage 3 — Tool Use & Agent 入门](../../../stages/03-tool-use-and-hello-agent.zh-Hans.md) 练习 3。 + +## 为什么从零写 + +ReAct（Reasoning + Acting）是现代 agent 的基础 pattern： + +``` +while not done: + thought = LLM 看完目前 context、讲出下一步要做什么 + action = LLM 调用一个 tool + observation = tool 执行结果、喂回去给 LLM +``` + +LangGraph / CrewAI 把这个 loop 藏起来了。你**自己写过一次**才知道： +- 为什么 messages array 一直长 +- tool_use_id 跟 tool_result 怎么配对 +- stop_reason 为什么是 `tool_use` 或 `end_turn` +- max_iter 为什么是 safety net + +70 行 Python 全交代清楚。 + +## 怎么跑 + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter.py +``` + +预期看到： + +``` +❓ 问题：'台北人口' 除以 '纽约人口'、答案保留 4 位小数。 +------------------------------------------------------------ +[step 0] thought: 我先查台北人口... + tool: lookup_fact({'query': '台北人口'}) → 2602000 +[step 1] thought: 接着查纽约人口... + tool: lookup_fact({'query': '纽约人口'}) → 8336000 +[step 2] thought: 计算比例... + tool: calculator({'expression': '2602000 / 8336000'}) → 0.3121... +[step 3] thought: 答案是 0.3122 +------------------------------------------------------------ +✅ 最终答案：台北人口除以纽约人口约 0.3122 + 共 4 轮 +✅ 练习 3 通过 — ReAct loop 自己连用了 lookup_fact 跟 calculator +``` + +## 不花钱验证程序逻辑 + +```bash +python test.py +``` + +`test.py` 用 `unittest.mock.MagicMock` 取代 Anthropic client、塞固定 response、验证你的 loop 逻辑。预期： + +``` +✅ test_calculator_basic +✅ test_calculator_rejects_eval_injection +✅ test_lookup_fact +✅ test_react_loop_single_tool_call +✅ test_react_loop_multi_step +✅ test_react_loop_respects_max_iter + +🎉 全部通过 — 你的 ReAct loop 逻辑正确 +``` + +## 程序结构走查 + +| 段 | 行 | 在做什么 | +|---|---|---| +| `tool_calculator` | ~30-40 | 安全的计算器（whitelist 过滤、避免 `eval` 漏洞） | +| `tool_lookup_fact` | ~42-50 | 假事实库（教学用、避免依赖外部 API） | +| `TOOLS_SPEC` | ~52-75 | tool schema 给 LLM 看 | +| `TOOL_IMPL` | ~77-80 | name → callable 对应表（dispatch） | +| `react_loop` | ~85-130 | 主循环、含 max_iter safety、`messages` 累积、tool result 接回去 | + +## 常见坑 + +1. **忘记把 assistant response 加进 messages**：下一轮 LLM 就看不到自己上一轮讲过什么、会 loop forever +2. **tool_result 没带 `tool_use_id`**：LLM 无法配对哪个 result 对应哪个 call +3. **`while True` 没 max_iter**：tool 结果写得不好、LLM 会无限调用；safety net 一定要设 +4. **eval 没过滤**：calculator 直接 `eval(user_input)` = RCE 漏洞；用 whitelist 或 `ast.literal_eval` + +## 想看更聪明的答案？ + +预设用 `claude-haiku-4-5`（最便宜）。改成 sonnet： + +```bash +MODEL=claude-sonnet-5 python starter.py +``` + +或在 `starter.py` 改 `MODEL = ...` 那行。 + +## 延伸 + +- **加更多 tool**：在 `TOOLS_SPEC` + `TOOL_IMPL` 补一个 entry 即可 +- **加 streaming**：把 `client.messages.create(...)` 换成 `with client.messages.stream(...) as s:`、边跑边印 +- **加 prompt cache**：在 `system=` 或 `tools=` 带 `cache_control={"type":"ephemeral"}` 重复 call 省 90% token +- **接 [LangGraph](https://langchain-ai.github.io/langgraph/) 或 [Pydantic AI](https://ai.pydantic.dev/) 看 framework 怎么帮你藏掉这 70 行** diff --git a/examples/stage-3/03-react-from-scratch/requirements.txt b/examples/stage-3/03-react-from-scratch/requirements.txt new file mode 100644 index 0000000..a4a9cdf --- /dev/null +++ b/examples/stage-3/03-react-from-scratch/requirements.txt @@ -0,0 +1,2 @@ +openai>=1.50,<2.0 +anthropic>=0.40,<1.0 # 只 starter_anthropic.py 需要、Ollama path 不用 diff --git a/examples/stage-3/03-react-from-scratch/starter.py b/examples/stage-3/03-react-from-scratch/starter.py new file mode 100644 index 0000000..ba1d20f --- /dev/null +++ b/examples/stage-3/03-react-from-scratch/starter.py @@ -0,0 +1,180 @@ +""" +練習 3：從零實作 ReAct（不用 framework）— starter.py（Path A、Ollama 默認） + +70 行 Python 把 Thought → Action → Observation 迴圈寫出來。 +不要 LangChain、不要 LangGraph，就是純 while loop。 + +跑法： + pip install -r requirements.txt + ollama pull qwen2.5:3b # Stage 3+ tool-use 默認 model + ollama serve # 預設 port 11434 + python starter.py + +驗證： + python test.py （test.py 跨 backend 通用、用 mock、不打 API） + +想看 Anthropic Claude 版本： + python starter_anthropic.py （需 ANTHROPIC_API_KEY、$0.001/run） +""" + +from __future__ import annotations + +import json +import os +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +MODEL = os.environ.get("MODEL", "qwen2.5:3b") # tool-use 穩定的 Ollama model + + +# === 1. Tools 定義（含實作）=== + +def tool_calculator(expression: str) -> str: + """安全的計算器：只允許 + - * / 跟數字。""" + allowed = set("0123456789.+-*/() ") + if any(c not in allowed for c in expression): + return f"error: 表達式含不允許字元（{expression}）" + try: + return str(eval(expression)) # noqa: S307 — 已用 whitelist + except Exception as e: # noqa: BLE001 + return f"error: {e}" + + +def tool_lookup_fact(query: str) -> str: + """假的事實查詢（教學專用、避免依賴外部 API）。""" + facts = { + "台北人口": "2602000", + "紐約人口": "8336000", + "光速": "299792458", # m/s + } + return facts.get(query.strip(), f"unknown: {query}") + + +# OpenAI-compatible 的 tools schema wrap 在 {"type":"function", "function":{...}} 裡 +TOOLS_SPEC = [ + { + "type": "function", + "function": { + "name": "calculator", + "description": "做基本算術運算（加減乘除）。輸入是表達式字串。", + "parameters": { + "type": "object", + "properties": { + "expression": {"type": "string", "description": "算術表達式"}, + }, + "required": ["expression"], + }, + }, + }, + { + "type": "function", + "function": { + "name": "lookup_fact", + "description": "查詢一個事實（人口 / 物理常數等）。", + "parameters": { + "type": "object", + "properties": { + "query": {"type": "string", "description": "查詢關鍵字"}, + }, + "required": ["query"], + }, + }, + }, +] + +TOOL_IMPL = { + "calculator": lambda inp: tool_calculator(inp["expression"]), + "lookup_fact": lambda inp: tool_lookup_fact(inp["query"]), +} + + +# === 2. ReAct loop === + +def react_loop(question: str, max_iter: int = 6, client: Any = None) -> dict: + """ + OpenAI-compatible ReAct loop。每輪： + 1. 問 LLM（含 tools） + 2. finish_reason: 'tool_calls' → 執行 tool、observation 接回、繼續 + 'stop' → 結束、最後 message 是答案 + 回傳 {final, trace, steps}。 + """ + client = client or OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + messages = [{"role": "user", "content": question}] + trace: list[dict] = [] + + for step in range(max_iter): + resp = client.chat.completions.create( + model=MODEL, + tools=TOOLS_SPEC, + messages=messages, + ) + msg = resp.choices[0].message + thought_text = msg.content or "" + tool_calls = msg.tool_calls or [] + + # 把 assistant message 加進 messages（OpenAI 格式） + assistant_entry: dict = {"role": "assistant", "content": thought_text} + if tool_calls: + assistant_entry["tool_calls"] = [ + { + "id": tc.id, + "type": "function", + "function": {"name": tc.function.name, "arguments": tc.function.arguments}, + } + for tc in tool_calls + ] + messages.append(assistant_entry) + + if resp.choices[0].finish_reason == "stop" or not tool_calls: + trace.append({"step": step, "thought": thought_text, "tool": None, "obs": None}) + return {"final": thought_text, "trace": trace, "steps": step + 1} + + # 執行 tool calls、observation 接回（OpenAI 用 role="tool"） + last_obs = "" + for tc in tool_calls: + fn = TOOL_IMPL.get(tc.function.name) + args = json.loads(tc.function.arguments) + obs = fn(args) if fn else f"error: unknown tool {tc.function.name}" + last_obs = obs + messages.append({ + "role": "tool", + "tool_call_id": tc.id, + "content": obs, + }) + + trace.append({ + "step": step, + "thought": thought_text, + "tool": tc.function.name, + "tool_input": json.loads(tc.function.arguments), + "obs": last_obs, + }) + + return {"final": None, "trace": trace, "steps": max_iter, "truncated": True} + + +# === 3. 自我驗證 === + +if __name__ == "__main__": + question = "'台北人口' 除以 '紐約人口'、答案保留 4 位小數。" + print(f"❓ 問題：{question}（using Ollama {MODEL}）") + print("-" * 60) + + result = react_loop(question, max_iter=5) + + for entry in result["trace"]: + print(f"[step {entry['step']}] thought: {(entry['thought'] or '')[:80]}...") + if entry["tool"]: + print(f" tool: {entry['tool']}({entry.get('tool_input')}) → {entry['obs']}") + print("-" * 60) + print(f"✅ 最終答案：{result['final']}") + print(f" 共 {result['steps']} 輪") + + # 寬鬆驗證（小 model 不一定精確到 4 位小數） + assert result.get("final") is not None or result.get("truncated"), "loop 應收尾或顯式 truncate" + print("✅ 練習 3 通過 — 你已用本機 qwen2.5:3b 跑通 ReAct + tool use、$0/run") diff --git a/examples/stage-3/03-react-from-scratch/starter_anthropic.py b/examples/stage-3/03-react-from-scratch/starter_anthropic.py new file mode 100644 index 0000000..958efa2 --- /dev/null +++ b/examples/stage-3/03-react-from-scratch/starter_anthropic.py @@ -0,0 +1,161 @@ +""" +練習 3：從零實作 ReAct（不用 framework）— starter.py + +目的：用 ~70 行 Python 把 Thought → Action → Observation 迴圈寫出來。 +不要 LangChain、不要 LangGraph，就是純 while loop。 + +跑法： + pip install -r requirements.txt + export ANTHROPIC_API_KEY=sk-ant-... + python starter.py + +驗證： + python test.py （用 mock、不花 API 錢） +""" + +from __future__ import annotations + +import json +import os +import sys +from typing import Any + +# Windows cp950 console 無法印 emoji / 中文、強制 UTF-8 +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic + +MODEL = os.environ.get("MODEL", "claude-haiku-4-5") # 想換 sonnet 改這行 + +# === 1. Tools 定義（含實作） === + +def tool_calculator(expression: str) -> str: + """安全的計算器：只允許 + - * / 跟數字。""" + allowed = set("0123456789.+-*/() ") + if any(c not in allowed for c in expression): + return f"error: 表達式含不允許字元（{expression}）" + try: + return str(eval(expression)) # noqa: S307 — 已用 whitelist + except Exception as e: # noqa: BLE001 + return f"error: {e}" + + +def tool_lookup_fact(query: str) -> str: + """假的事實查詢（避免依賴外部 API、教學專用）。""" + facts = { + "台北人口": "2602000", + "紐約人口": "8336000", + "光速": "299792458", # m/s + } + return facts.get(query.strip(), f"unknown: {query}") + + +TOOLS_SPEC = [ + { + "name": "calculator", + "description": "做基本算術運算（加減乘除）。輸入是表達式字串，例如 '3 * (5+2)'。", + "input_schema": { + "type": "object", + "properties": { + "expression": {"type": "string", "description": "算術表達式"}, + }, + "required": ["expression"], + }, + }, + { + "name": "lookup_fact", + "description": "查詢一個事實（人口 / 物理常數等）。輸入是查詢關鍵字、回傳一個字串答案或 'unknown: ...'。", + "input_schema": { + "type": "object", + "properties": { + "query": {"type": "string", "description": "查詢關鍵字（如「台北人口」、「光速」）"}, + }, + "required": ["query"], + }, + }, +] + +TOOL_IMPL = { + "calculator": lambda inp: tool_calculator(inp["expression"]), + "lookup_fact": lambda inp: tool_lookup_fact(inp["query"]), +} + + +# === 2. ReAct loop === + +def react_loop(question: str, max_iter: int = 6, client: Any = None) -> dict: + """ + 純 while 迴圈、每輪： + 1. 問 LLM（含 tools） + 2. 看 stop_reason：tool_use → 執行工具、把結果加進 messages、繼續 + end_turn → 完成、回傳最終答案 + 回傳 {final, trace}。trace 紀錄每輪 (thought, tool, obs)。 + """ + client = client or anthropic.Anthropic() + messages = [{"role": "user", "content": question}] + trace: list[dict] = [] + + for step in range(max_iter): + resp = client.messages.create( + model=MODEL, + max_tokens=1024, + tools=TOOLS_SPEC, + messages=messages, + ) + + # 收 thought（assistant 的 text block）跟 tool call + thought_text = " ".join(b.text for b in resp.content if b.type == "text") + tool_calls = [b for b in resp.content if b.type == "tool_use"] + + # 把 assistant 整個 response 加進 messages（給下一輪 context） + messages.append({"role": "assistant", "content": resp.content}) + + if resp.stop_reason == "end_turn" or not tool_calls: + # 已收尾、最後一段 text 就是答案 + trace.append({"step": step, "thought": thought_text, "tool": None, "obs": None}) + return {"final": thought_text, "trace": trace, "steps": step + 1} + + # 執行所有 tool call、把 observation 接回去 + tool_results = [] + last_obs = "" + for call in tool_calls: + fn = TOOL_IMPL.get(call.name) + obs = fn(call.input) if fn else f"error: unknown tool {call.name}" + last_obs = obs + tool_results.append({"type": "tool_result", "tool_use_id": call.id, "content": obs}) + messages.append({"role": "user", "content": tool_results}) + + trace.append({ + "step": step, + "thought": thought_text, + "tool": tool_calls[0].name, + "tool_input": dict(tool_calls[0].input), + "obs": last_obs, + }) + + # 跑滿 max_iter 還沒收尾 + return {"final": None, "trace": trace, "steps": max_iter, "truncated": True} + + +# === 3. 自我驗證（跑真 API） === + +if __name__ == "__main__": + question = "台北人口除以紐約人口、答案保留 4 位小數。" + print(f"❓ 問題：{question}") + print("-" * 60) + + result = react_loop(question, max_iter=5) + + for entry in result["trace"]: + print(f"[step {entry['step']}] thought: {entry['thought'][:80]}...") + if entry["tool"]: + print(f" tool: {entry['tool']}({entry.get('tool_input')}) → {entry['obs']}") + print("-" * 60) + print(f"✅ 最終答案：{result['final']}") + print(f" 共 {result['steps']} 輪") + + # === 自我驗證 === + assert result.get("final") is not None, "預期 react_loop 在 max_iter 內收尾" + assert "0.3" in (result["final"] or ""), f"預期答案含 0.3xxx（2602000/8336000≈0.3122）" + print("✅ 練習 3 通過 — ReAct loop 自己連用了 lookup_fact 跟 calculator") diff --git a/examples/stage-3/03-react-from-scratch/test.py b/examples/stage-3/03-react-from-scratch/test.py new file mode 100644 index 0000000..b421398 --- /dev/null +++ b/examples/stage-3/03-react-from-scratch/test.py @@ -0,0 +1,132 @@ +""" +練習 3 自我驗證 — Path A（Ollama starter.py）。 + +跑法： + python test.py + +驗證內容： + - react_loop 收到 tool_calls → 執行 tool → observation 接回 → 拿到 finish_reason='stop' + - react_loop 在 max_iter 內停（避免無限迴圈） + - tool_calculator / tool_lookup_fact 邏輯正確 + - 跨 backend：mock 用 OpenAI-compat shape（不需要 Anthropic SDK） + +Anthropic 版本 test 見 test_anthropic.py。 +""" + +from __future__ import annotations + +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import json +from types import SimpleNamespace +from unittest.mock import MagicMock + +from starter import react_loop, tool_calculator, tool_lookup_fact + + +# === Helpers for building mock OpenAI-compat responses === + +def make_tool_call(call_id: str, name: str, args: dict): + return SimpleNamespace( + id=call_id, + type="function", + function=SimpleNamespace(name=name, arguments=json.dumps(args)), + ) + + +def make_resp(finish_reason: str, content: str = "", tool_calls=None): + """Build a fake OpenAI chat.completions response.""" + msg = SimpleNamespace(content=content, tool_calls=tool_calls) + return SimpleNamespace(choices=[SimpleNamespace(finish_reason=finish_reason, message=msg)]) + + +# === Tests === + +def test_calculator_basic(): + assert tool_calculator("2 + 3") == "5" + assert tool_calculator("(10 + 2) / 4") == "3.0" + print("✅ test_calculator_basic") + + +def test_calculator_rejects_eval_injection(): + out = tool_calculator("__import__('os').system('ls')") + assert out.startswith("error:"), f"預期 reject、得到 {out}" + print("✅ test_calculator_rejects_eval_injection") + + +def test_lookup_fact(): + assert tool_lookup_fact("台北人口") == "2602000" + assert tool_lookup_fact("不存在的東西").startswith("unknown:") + print("✅ test_lookup_fact") + + +def test_react_loop_single_tool_call(): + """模擬：LLM 第 1 輪 tool_calls lookup_fact、第 2 輪 stop 給答案。""" + client = MagicMock() + client.chat.completions.create.side_effect = [ + make_resp("tool_calls", "我先查台北人口。", + [make_tool_call("call_1", "lookup_fact", {"query": "台北人口"})]), + make_resp("stop", "台北人口是 2602000 人。"), + ] + + result = react_loop("台北人口是多少？", max_iter=4, client=client) + + assert result["final"] == "台北人口是 2602000 人。" + assert result["steps"] == 2 + assert result["trace"][0]["tool"] == "lookup_fact" + assert result["trace"][0]["obs"] == "2602000" + print("✅ test_react_loop_single_tool_call") + + +def test_react_loop_multi_step(): + """模擬：lookup_fact x2 + calculator x1 + stop = 4 輪。""" + client = MagicMock() + client.chat.completions.create.side_effect = [ + make_resp("tool_calls", "查台北。", + [make_tool_call("c1", "lookup_fact", {"query": "台北人口"})]), + make_resp("tool_calls", "查紐約。", + [make_tool_call("c2", "lookup_fact", {"query": "紐約人口"})]), + make_resp("tool_calls", "算比例。", + [make_tool_call("c3", "calculator", {"expression": "2602000 / 8336000"})]), + make_resp("stop", "台北 / 紐約 約 0.3122。"), + ] + + result = react_loop("台北 / 紐約 人口比？", max_iter=6, client=client) + + assert result["final"] is not None + assert "0.3122" in result["final"] + assert result["steps"] == 4 + tool_seq = [t["tool"] for t in result["trace"] if t["tool"]] + assert tool_seq == ["lookup_fact", "lookup_fact", "calculator"] + print("✅ test_react_loop_multi_step") + + +def test_react_loop_respects_max_iter(): + """模擬：LLM 一直 tool_calls、永不收尾、應該在 max_iter 停。""" + client = MagicMock() + + def never_ending(**kwargs): + return make_resp("tool_calls", "再查一次。", + [make_tool_call("c", "lookup_fact", {"query": "光速"})]) + + client.chat.completions.create.side_effect = never_ending + + result = react_loop("永遠跑下去", max_iter=3, client=client) + + assert result.get("truncated") is True + assert result["steps"] == 3 + assert result["final"] is None + print("✅ test_react_loop_respects_max_iter") + + +if __name__ == "__main__": + test_calculator_basic() + test_calculator_rejects_eval_injection() + test_lookup_fact() + test_react_loop_single_tool_call() + test_react_loop_multi_step() + test_react_loop_respects_max_iter() + print("\n🎉 全部通過 — Ollama path ReAct loop 邏輯正確") diff --git a/examples/stage-3/03-react-from-scratch/test_anthropic.py b/examples/stage-3/03-react-from-scratch/test_anthropic.py new file mode 100644 index 0000000..1d0beb2 --- /dev/null +++ b/examples/stage-3/03-react-from-scratch/test_anthropic.py @@ -0,0 +1,134 @@ +""" +練習 3 自我驗證：用 mock 取代 anthropic client、不花 API 錢。 + +跑法： + python test.py + +驗證內容： + 1. react_loop 收到 tool_use → 執行 tool → 把 observation 接回去 → 拿到 end_turn + 2. react_loop 在 max_iter 內停（避免無限迴圈） + 3. tool_calculator / tool_lookup_fact 邏輯正確 +""" + +from __future__ import annotations + +import sys + +# Windows cp950 console 無法印 emoji / 中文、強制 UTF-8 +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from unittest.mock import MagicMock +from types import SimpleNamespace + +from starter_anthropic import react_loop, tool_calculator, tool_lookup_fact + + +# === Helpers for building mock Anthropic responses === + +def block_text(s: str): + return SimpleNamespace(type="text", text=s) + + +def block_tool_use(tool_id: str, name: str, inp: dict): + return SimpleNamespace(type="tool_use", id=tool_id, name=name, input=inp) + + +def make_resp(stop_reason: str, *blocks): + return SimpleNamespace(stop_reason=stop_reason, content=list(blocks)) + + +# === Tests === + +def test_calculator_basic(): + assert tool_calculator("2 + 3") == "5" + assert tool_calculator("(10 + 2) / 4") == "3.0" + print("✅ test_calculator_basic") + + +def test_calculator_rejects_eval_injection(): + out = tool_calculator("__import__('os').system('ls')") + assert out.startswith("error:"), f"預期 reject，得到 {out}" + print("✅ test_calculator_rejects_eval_injection") + + +def test_lookup_fact(): + assert tool_lookup_fact("台北人口") == "2602000" + assert tool_lookup_fact("不存在的東西") .startswith("unknown:") + print("✅ test_lookup_fact") + + +def test_react_loop_single_tool_call(): + """模擬：LLM 第 1 輪 tool_use lookup_fact、第 2 輪直接 end_turn 給答案。""" + client = MagicMock() + client.messages.create.side_effect = [ + make_resp( + "tool_use", + block_text("我先查台北人口。"), + block_tool_use("toolu_1", "lookup_fact", {"query": "台北人口"}), + ), + make_resp( + "end_turn", + block_text("台北人口是 2602000 人。"), + ), + ] + + result = react_loop("台北人口是多少？", max_iter=4, client=client) + + assert result["final"] == "台北人口是 2602000 人。" + assert result["steps"] == 2 + assert result["trace"][0]["tool"] == "lookup_fact" + assert result["trace"][0]["obs"] == "2602000" + print("✅ test_react_loop_single_tool_call") + + +def test_react_loop_multi_step(): + """模擬：lookup_fact x2 + calculator x1 + end_turn = 4 輪。""" + client = MagicMock() + client.messages.create.side_effect = [ + make_resp("tool_use", block_text("查台北。"), + block_tool_use("t1", "lookup_fact", {"query": "台北人口"})), + make_resp("tool_use", block_text("查紐約。"), + block_tool_use("t2", "lookup_fact", {"query": "紐約人口"})), + make_resp("tool_use", block_text("算比例。"), + block_tool_use("t3", "calculator", {"expression": "2602000 / 8336000"})), + make_resp("end_turn", block_text("台北 / 紐約 約 0.3122。")), + ] + + result = react_loop("台北 / 紐約 人口比？", max_iter=6, client=client) + + assert result["final"] is not None + assert "0.3122" in result["final"] + assert result["steps"] == 4 + tool_seq = [t["tool"] for t in result["trace"] if t["tool"]] + assert tool_seq == ["lookup_fact", "lookup_fact", "calculator"] + print("✅ test_react_loop_multi_step") + + +def test_react_loop_respects_max_iter(): + """模擬：LLM 一直 tool_use、永不收尾、應該在 max_iter 停。""" + client = MagicMock() + def never_ending(**kwargs): + return make_resp( + "tool_use", + block_text("再查一次。"), + block_tool_use("t", "lookup_fact", {"query": "光速"}), + ) + client.messages.create.side_effect = never_ending + + result = react_loop("永遠跑下去", max_iter=3, client=client) + + assert result.get("truncated") is True + assert result["steps"] == 3 + assert result["final"] is None + print("✅ test_react_loop_respects_max_iter") + + +if __name__ == "__main__": + test_calculator_basic() + test_calculator_rejects_eval_injection() + test_lookup_fact() + test_react_loop_single_tool_call() + test_react_loop_multi_step() + test_react_loop_respects_max_iter() + print("\n🎉 全部通過 — 你的 ReAct loop 邏輯正確") diff --git a/examples/stage-3/04-multi-step-reasoning/README.en.md b/examples/stage-3/04-multi-step-reasoning/README.en.md new file mode 100644 index 0000000..a26b849 --- /dev/null +++ b/examples/stage-3/04-multi-step-reasoning/README.en.md @@ -0,0 +1,101 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# Exercise 4: Multi-Step Reasoning + +Corresponds to [Stage 3 — Tool Use & Agent Intro](../../../stages/03-tool-use-and-hello-agent.en.md) Exercise 4. + +## Why this matters + +Extends the ReAct loop from Exercise 3 into a **3-5 step task**: look up Taipei population → look up NY population → divide → convert to percentage. The LLM plans the next step; the tools reliably execute small actions. Together they look like an agent that can complete a workflow. + +It's also a great place to feel **model scale vs multi-step stability**. Same loop, claude-haiku usually finishes in 4 steps; qwen2.5:3b may skip a step (e.g., forget to convert to percentage) or stop too early. + +## How to run — two paths + +### Path A (default, free, local) + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +Budget: **$0**. A 4-5 round loop takes ~30-120s on CPU. + +### Path B (Anthropic, cloud-quality comparison) + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +Budget: ~**$0.005** per run (claude-haiku-4-5, 5 rounds of accumulating messages). + +Expected output (Path A, local, ideal 4-step path): + +``` +❓ Question: Find Taipei population divided by New York population, then express it as a percentage. +------------------------------------------------------------ +[step 0] tool: lookup_population({'city': 'Taipei'}) → 2602000 +[step 1] tool: lookup_population({'city': 'New York'}) → 8336000 +[step 2] tool: divide({'a': 2602000, 'b': 8336000}) → 0.3122... +[step 3] tool: to_percentage({'ratio': 0.3122}) → 31.22 +------------------------------------------------------------ +✅ Final answer: Taipei is about 31.22% of New York's population. + Took 5 rounds. +✅ Exercise 4 passed — multi-step ReAct loop ran locally on qwen2.5:3b, $0/run +``` + +## Validate the logic without API credits (mock-based) + +```bash +python test.py # validates Path A (Ollama) starter.py logic +python test_anthropic.py # validates Path B (Anthropic) starter_anthropic.py logic +``` + +Both test suites use `unittest.mock`, no real API call, $0/run. + +## Conceptual reminders + +The core of multi-step tasks isn't "the model is good at math" — it's breaking a complex task into reliable small steps: + +- **Tools should be narrow and stable**: `divide(a, b)` does one thing; even `b=0` doesn't crash, it returns 0 +- **The LLM plans**: decides which tool to call next and when to stop +- **`max_iter=8` is a mandatory safety net**: prevents the model from looping forever without finishing +- **`messages` grows each round**: assistant response + tool_result are appended so the LLM can see history + +## What to watch on each path + +| Observation | Anthropic Claude haiku | Ollama qwen2.5:3b | +|---|---|---| +| Probability of completing 4 steps | High | Medium (may skip "to_percentage") | +| Step ordering | Stable | Can swap order | +| End-turn detection | Reliable `end_turn` | May add a redundant tool call before stopping | +| Cost per run | $0.005 | $0 | + +This is precisely the teaching point of Exercise 4 — **same ReAct loop, different model, which step breaks first**. When picking a production model, multi-step stability is as important as cost. + +## Want smarter answers? + +Default is `claude-haiku-4-5` (cheapest). Try Sonnet: + +```bash +MODEL=claude-sonnet-5 python starter_anthropic.py +``` + +Or on the Ollama path, swap to a larger model: + +```bash +MODEL=qwen2.5:7b python starter.py # 4.7 GB, more stable +MODEL=mistral-nemo:12b python starter.py # 7.1 GB, closer to cloud +``` + +## Extensions + +- **Add more tools** — append one entry each to `TOOLS_SPEC` + `TOOL_IMPL` +- **Add retry / error handling** — see [`../05-error-handling/`](../05-error-handling/) for tool failure patterns +- **Schema design** — see [`../06-schema-design/`](../06-schema-design/) for a bad vs good schema A/B diff --git a/examples/stage-3/04-multi-step-reasoning/README.md b/examples/stage-3/04-multi-step-reasoning/README.md new file mode 100644 index 0000000..dd4aabe --- /dev/null +++ b/examples/stage-3/04-multi-step-reasoning/README.md @@ -0,0 +1,108 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 練習 4：多步驟推理任務 + +對應 [Stage 3 — Tool Use & Agent 入門](../../../stages/03-tool-use-and-hello-agent.md) 練習 4。 +> 🎓 **學習模式**：這份 `starter.py` 是**完整解答**、不是 TODO skeleton。建議用**主動模式**——`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫一份 `starter.py`、跑 `python test.py` 驗證；卡 20 分鐘再回去對照 reference。完整方法論看 [`docs/HOW_TO_USE.md`](../../../docs/HOW_TO_USE.md)。 + +> 📚 **想要 chapter-length 深入版？** 本 folder 的 starter 是 70-150 行 illustrative 版、聚焦 `核心 pattern + 兩條 SDK path`，不是進階深度教材。深度教材推薦： +> - [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents) ⭐ 中文圈最完整、章節式 + 16 種 production 能力。**本練習對應 hello-agents 的 planning / multi-step workflow 章節** +> - [Anthropic — Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents)（什麼時候該拆步驟、什麼時候不要） +> - 完整 references 見 [Stage 3 精選 Projects](../../../stages/03-tool-use-and-hello-agent.md#-精選-projects) + + +## 為什麼這題重要 + +把練習 3 的 ReAct loop 延伸成 **3-5 步任務**：查台北人口 → 查紐約人口 → 相除 → 轉百分比。LLM 負責規劃下一步、工具負責可靠地執行小動作；兩者合起來才像能完成 workflow 的 agent。 + +這題也是觀察「**model 規模 vs 多步推理穩定度**」的好實驗。同樣 loop、claude-haiku 通常 4 步走完；qwen2.5:3b 可能中間漏一步（譬如忘了轉百分比），或停太早。 + +## 怎麼跑 — 兩條路徑 + +### Path A（默認、本機免費） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +預算：**$0**。多步 loop ≈ 30-120 秒（CPU、4-5 輪累積）。 + +### Path B（Anthropic、想看 cloud 高品質） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +預算：每次 ≈ **$0.005**（claude-haiku-4-5、5 輪 messages 累積、每輪 prompt 漸長）。 + +預期看到（Path A、本機，理想 4 步走完）： + +``` +❓ 問題：Find Taipei population divided by New York population, then express it as a percentage. +------------------------------------------------------------ +[step 0] tool: lookup_population({'city': 'Taipei'}) → 2602000 +[step 1] tool: lookup_population({'city': 'New York'}) → 8336000 +[step 2] tool: divide({'a': 2602000, 'b': 8336000}) → 0.3122... +[step 3] tool: to_percentage({'ratio': 0.3122}) → 31.22 +------------------------------------------------------------ +✅ 最終答案：Taipei is about 31.22% of New York's population. + 共 5 輪 +✅ 練習 4 通過 — 你已用本機 qwen2.5:3b 跑通多步 ReAct loop、$0/run +``` + +## 不花錢驗證程式邏輯（mock-based） + +```bash +python test.py # 驗 Path A (Ollama) starter.py 邏輯 +python test_anthropic.py # 驗 Path B (Anthropic) starter_anthropic.py 邏輯 +``` + +兩條 test 都用 `unittest.mock`、不打真 API、$0/run。Path A 用 OpenAI-compat response shape、Path B 用 Anthropic content blocks。 + +## 觀念提醒 + +多步任務的核心不是「模型很會算」、而是把複雜任務拆成可靠的小步： + +- **工具要窄而穩**：`divide(a, b)` 只做一件事、`b=0` 也不 crash 而是回 0 +- **LLM 負責規劃**：決定下一步要呼叫哪個工具、何時停 +- **`max_iter=8` 是必要安全網**：避免模型一直要求工具而沒收尾 +- **每輪 messages 一直長**：assistant response + tool_result 都接回去、LLM 才看得到歷史 + +## 兩個 path 觀察重點 + +| 觀察項 | Anthropic Claude haiku | Ollama qwen2.5:3b | +|---|---|---| +| 走完 4 步機率 | 高 | 中（可能漏「轉百分比」） | +| 中間步驟順序 | 穩定 | 可能跳序 | +| 收尾判斷 | 穩定 `end_turn` | 可能多跑一輪冗餘 tool call | +| 單次成本 | $0.005 | $0 | + +這恰好是 Stage 3 練習 4 的教學重點——**同樣 ReAct loop、不同 model、在哪一步開始崩**。Production 選 model 時、多步穩定度是 cost 之外的關鍵考量。 + +## 想看更聰明的答案？ + +預設用 `claude-haiku-4-5`（最便宜）。改成 sonnet： + +```bash +MODEL=claude-sonnet-5 python starter_anthropic.py +``` + +或 Ollama path 換更大 model： + +```bash +MODEL=qwen2.5:7b python starter.py # 4.7 GB、更穩 +MODEL=mistral-nemo:12b python starter.py # 7.1 GB、更接近 cloud +``` + +## 延伸 + +- **加更多 tool**：在 `TOOLS_SPEC` + `TOOL_IMPL` 補一個 entry 即可 +- **加 retry / error handling**：看 [`../05-error-handling/`](../05-error-handling/) 怎麼處理 tool 失敗 +- **schema 設計**：看 [`../06-schema-design/`](../06-schema-design/) 比較 bad / good schema diff --git a/examples/stage-3/04-multi-step-reasoning/README.zh-Hans.md b/examples/stage-3/04-multi-step-reasoning/README.zh-Hans.md new file mode 100644 index 0000000..84988cd --- /dev/null +++ b/examples/stage-3/04-multi-step-reasoning/README.zh-Hans.md @@ -0,0 +1,101 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 练习 4：多步骤推理任务 + +对应 [Stage 3 — Tool Use & Agent 入门](../../../stages/03-tool-use-and-hello-agent.zh-Hans.md) 练习 4。 + +## 为什么这题重要 + +把练习 3 的 ReAct loop 延伸成 **3-5 步任务**：查台北人口 → 查纽约人口 → 相除 → 转百分比。LLM 负责规划下一步、工具负责可靠地执行小动作；两者合起来才像能完成 workflow 的 agent。 + +这题也是观察“**model 规模 vs 多步推理稳定度**”的好实验。同样 loop、claude-haiku 通常 4 步走完；qwen2.5:3b 可能中间漏一步（譬如忘了转百分比），或停太早。 + +## 怎么跑 — 两条路径 + +### Path A（默认、本机免费） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +预算：**$0**。多步 loop ≈ 30-120 秒（CPU、4-5 轮累积）。 + +### Path B（Anthropic、想看 cloud 高质量） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +预算：每次 ≈ **$0.005**（claude-haiku-4-5、5 轮 messages 累积、每轮 prompt 渐长）。 + +预期看到（Path A、本机，理想 4 步走完）： + +``` +❓ 问题：Find Taipei population divided by New York population, then express it as a percentage. +------------------------------------------------------------ +[step 0] tool: lookup_population({'city': 'Taipei'}) → 2602000 +[step 1] tool: lookup_population({'city': 'New York'}) → 8336000 +[step 2] tool: divide({'a': 2602000, 'b': 8336000}) → 0.3122... +[step 3] tool: to_percentage({'ratio': 0.3122}) → 31.22 +------------------------------------------------------------ +✅ 最终答案：Taipei is about 31.22% of New York's population. + 共 5 轮 +✅ 练习 4 通过 — 你已用本机 qwen2.5:3b 跑通多步 ReAct loop、$0/run +``` + +## 不花钱验证程序逻辑（mock-based） + +```bash +python test.py # 验 Path A (Ollama) starter.py 逻辑 +python test_anthropic.py # 验 Path B (Anthropic) starter_anthropic.py 逻辑 +``` + +两条 test 都用 `unittest.mock`、不打真 API、$0/run。 + +## 观念提醒 + +多步任务的核心不是“模型很会算”、而是把复杂任务拆成可靠的小步： + +- **工具要窄而稳**：`divide(a, b)` 只做一件事、`b=0` 也不 crash 而是回 0 +- **LLM 负责规划**：决定下一步要调用哪个工具、何时停 +- **`max_iter=8` 是必要安全网**：避免模型一直要求工具而没收尾 +- **每轮 messages 一直长**：assistant response + tool_result 都接回去、LLM 才看得到历史 + +## 两个 path 观察重点 + +| 观察项 | Anthropic Claude haiku | Ollama qwen2.5:3b | +|---|---|---| +| 走完 4 步机率 | 高 | 中（可能漏“转百分比”） | +| 中间步骤顺序 | 稳定 | 可能跳序 | +| 收尾判断 | 稳定 `end_turn` | 可能多跑一轮冗余 tool call | +| 单次成本 | $0.005 | $0 | + +这恰好是 Stage 3 练习 4 的教学重点——**同样 ReAct loop、不同 model、在哪一步开始崩**。Production 选 model 时、多步稳定度是 cost 之外的关键考量。 + +## 想看更聪明的答案？ + +预设用 `claude-haiku-4-5`（最便宜）。改成 sonnet： + +```bash +MODEL=claude-sonnet-5 python starter_anthropic.py +``` + +或 Ollama path 换更大 model： + +```bash +MODEL=qwen2.5:7b python starter.py # 4.7 GB、更稳 +MODEL=mistral-nemo:12b python starter.py # 7.1 GB、更接近 cloud +``` + +## 延伸 + +- **加更多 tool**：在 `TOOLS_SPEC` + `TOOL_IMPL` 补一个 entry 即可 +- **加 retry / error handling**：看 [`../05-error-handling/`](../05-error-handling/) 怎么处理 tool 失败 +- **schema 设计**：看 [`../06-schema-design/`](../06-schema-design/) 比较 bad / good schema diff --git a/examples/stage-3/04-multi-step-reasoning/requirements.txt b/examples/stage-3/04-multi-step-reasoning/requirements.txt new file mode 100644 index 0000000..a4a9cdf --- /dev/null +++ b/examples/stage-3/04-multi-step-reasoning/requirements.txt @@ -0,0 +1,2 @@ +openai>=1.50,<2.0 +anthropic>=0.40,<1.0 # 只 starter_anthropic.py 需要、Ollama path 不用 diff --git a/examples/stage-3/04-multi-step-reasoning/starter.py b/examples/stage-3/04-multi-step-reasoning/starter.py new file mode 100644 index 0000000..4b3433a --- /dev/null +++ b/examples/stage-3/04-multi-step-reasoning/starter.py @@ -0,0 +1,182 @@ +"""練習 4：多步驟推理任務 — Path A（Ollama 默認、本機免費）。 + +把練習 3 的 ReAct loop 延伸成 3-5 步任務：查台北人口 → 查紐約人口 → 相除 → 轉百分比。 +重點：工具寫窄而穩、LLM 負責規劃下一步、max_iter 是 safety net。 + +跑法： + pip install -r requirements.txt + ollama pull qwen2.5:3b + ollama serve + python starter.py + +驗證： + python test.py （用 mock、不打 API） + +想看 Anthropic Claude 版本： + python starter_anthropic.py （需 ANTHROPIC_API_KEY、$0.005/run） + +⚠️ 注意：4 步推理對小 model 是挑戰。qwen2.5:3b 可能中間漏一步、或停太早。 +Claude haiku 比較穩——這恰好是教學重點：對比同樣 loop、不同 model 在哪一步崩。 +""" + +from __future__ import annotations + +import json +import os +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +MODEL = os.environ.get("MODEL", "qwen2.5:3b") + + +# === 1. Tools 定義（含實作）=== + +def lookup_population(city: str) -> str: + data = {"taipei": 2_602_000, "new york": 8_336_000, "empty city": 0} + return str(data.get(city.strip().lower(), 0)) + + +def divide(a: float, b: float) -> str: + b = float(b) + return "0" if b == 0 else str(float(a) / b) + + +def to_percentage(ratio: float) -> str: + return f"{float(ratio) * 100:.2f}" + + +def round_int(x: float) -> str: + return str(round(float(x))) + + +# OpenAI-compat 包一層 {"type": "function", "function": {...}} +TOOLS_SPEC = [ + { + "type": "function", + "function": { + "name": "lookup_population", + "description": "Return the population for a known city.", + "parameters": { + "type": "object", + "properties": {"city": {"type": "string"}}, + "required": ["city"], + }, + }, + }, + { + "type": "function", + "function": { + "name": "divide", + "description": "Divide a by b. Returns 0 instead of crashing when b is zero.", + "parameters": { + "type": "object", + "properties": {"a": {"type": "number"}, "b": {"type": "number"}}, + "required": ["a", "b"], + }, + }, + }, + { + "type": "function", + "function": { + "name": "to_percentage", + "description": "Convert a ratio such as 0.31 into a percentage number.", + "parameters": { + "type": "object", + "properties": {"ratio": {"type": "number"}}, + "required": ["ratio"], + }, + }, + }, + { + "type": "function", + "function": { + "name": "round_int", + "description": "Round a number to the nearest integer.", + "parameters": { + "type": "object", + "properties": {"x": {"type": "number"}}, + "required": ["x"], + }, + }, + }, +] + +TOOL_IMPL = { + "lookup_population": lambda i: lookup_population(i["city"]), + "divide": lambda i: divide(i["a"], i["b"]), + "to_percentage": lambda i: to_percentage(i["ratio"]), + "round_int": lambda i: round_int(i["x"]), +} + + +# === 2. ReAct loop（OpenAI-compat）=== + +def react_loop(question: str, max_iter: int = 8, client: Any = None) -> dict: + """OpenAI-compat 多步驟 ReAct loop。""" + client = client or OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + messages = [{"role": "user", "content": question}] + trace: list[dict] = [] + + for step in range(max_iter): + resp = client.chat.completions.create( + model=MODEL, + tools=TOOLS_SPEC, + messages=messages, + ) + msg = resp.choices[0].message + text = msg.content or "" + tool_calls = msg.tool_calls or [] + + assistant_entry: dict = {"role": "assistant", "content": text} + if tool_calls: + assistant_entry["tool_calls"] = [ + { + "id": tc.id, + "type": "function", + "function": {"name": tc.function.name, "arguments": tc.function.arguments}, + } + for tc in tool_calls + ] + messages.append(assistant_entry) + + if resp.choices[0].finish_reason == "stop" or not tool_calls: + trace.append({"step": step, "thought": text, "tool": None, "obs": None}) + return {"final": text, "trace": trace, "steps": step + 1} + + for tc in tool_calls: + fn = TOOL_IMPL.get(tc.function.name, lambda _: f"error: unknown tool {tc.function.name}") + args = json.loads(tc.function.arguments) + obs = fn(args) + messages.append({ + "role": "tool", + "tool_call_id": tc.id, + "content": obs, + }) + trace.append({"step": step, "thought": text, "tool": tc.function.name, "tool_input": args, "obs": obs}) + + return {"final": None, "trace": trace, "steps": max_iter, "truncated": True} + + +# === 3. 自我驗證 === + +if __name__ == "__main__": + question = "Find Taipei population divided by New York population, then express it as a percentage." + print(f"❓ 問題：{question}（using Ollama {MODEL}）") + print("-" * 60) + + result = react_loop(question) + for entry in result["trace"]: + if entry["tool"]: + print(f"[step {entry['step']}] tool: {entry['tool']}({entry.get('tool_input')}) → {entry['obs']}") + print("-" * 60) + print(f"✅ 最終答案：{result['final']}") + print(f" 共 {result['steps']} 輪") + + # 寬鬆驗證：小 model 不一定走完 4 步、但 loop 至少要收尾或顯式 truncate + assert result.get("final") is not None or result.get("truncated"), "loop 應收尾或 truncate" + print("✅ 練習 4 通過 — 你已用本機 qwen2.5:3b 跑通多步 ReAct loop、$0/run") diff --git a/examples/stage-3/04-multi-step-reasoning/starter_anthropic.py b/examples/stage-3/04-multi-step-reasoning/starter_anthropic.py new file mode 100644 index 0000000..020f8aa --- /dev/null +++ b/examples/stage-3/04-multi-step-reasoning/starter_anthropic.py @@ -0,0 +1,91 @@ +"""練習 4：多步驟推理任務 — Path B（Anthropic Claude）。 + +把練習 3 的 ReAct loop 延伸成 3-5 步任務：查台北人口 → 查紐約人口 → 相除 → 轉百分比。 +工具負責執行小動作、LLM 負責規劃下一步。 + +跑法： + pip install -r requirements.txt + export ANTHROPIC_API_KEY=sk-ant-... + python starter_anthropic.py + +預算：每次 ≈ $0.005（claude-haiku-4-5、5 輪 messages 累積）。 +Ollama 版本見 starter.py（本機 $0、但小 model 在多步任務上可能漏步）。 +""" + +from __future__ import annotations + +import os +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic + +MODEL = os.environ.get("MODEL", "claude-haiku-4-5") + + +def lookup_population(city: str) -> str: + data = {"taipei": 2_602_000, "new york": 8_336_000, "empty city": 0} + return str(data.get(city.strip().lower(), 0)) + + +def divide(a: float, b: float) -> str: + b = float(b) + return "0" if b == 0 else str(float(a) / b) + + +def to_percentage(ratio: float) -> str: + return f"{float(ratio) * 100:.2f}" + + +def round_int(x: float) -> str: + return str(round(float(x))) + + +TOOLS_SPEC = [ + {"name": "lookup_population", "description": "Return the population for a known city.", "input_schema": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}, + {"name": "divide", "description": "Divide a by b. Returns 0 instead of crashing when b is zero.", "input_schema": {"type": "object", "properties": {"a": {"type": "number"}, "b": {"type": "number"}}, "required": ["a", "b"]}}, + {"name": "to_percentage", "description": "Convert a ratio such as 0.31 into a percentage number.", "input_schema": {"type": "object", "properties": {"ratio": {"type": "number"}}, "required": ["ratio"]}}, + {"name": "round_int", "description": "Round a number to the nearest integer.", "input_schema": {"type": "object", "properties": {"x": {"type": "number"}}, "required": ["x"]}}, +] + +TOOL_IMPL = { + "lookup_population": lambda i: lookup_population(i["city"]), + "divide": lambda i: divide(i["a"], i["b"]), + "to_percentage": lambda i: to_percentage(i["ratio"]), + "round_int": lambda i: round_int(i["x"]), +} + + +def react_loop(question: str, max_iter: int = 8, client: Any = None) -> dict: + client = client or anthropic.Anthropic() + messages = [{"role": "user", "content": question}] + trace: list[dict] = [] + for step in range(max_iter): + resp = client.messages.create(model=MODEL, max_tokens=1024, tools=TOOLS_SPEC, messages=messages) + text = " ".join(getattr(b, "text", "") for b in resp.content if getattr(b, "type", None) == "text") + calls = [b for b in resp.content if getattr(b, "type", None) == "tool_use"] + messages.append({"role": "assistant", "content": resp.content}) + if resp.stop_reason == "end_turn" or not calls: + trace.append({"step": step, "thought": text, "tool": None, "obs": None}) + return {"final": text, "trace": trace, "steps": step + 1} + results = [] + for call in calls: + args = dict(call.input) + obs = TOOL_IMPL.get(call.name, lambda _: f"error: unknown tool {call.name}")(args) + results.append({"type": "tool_result", "tool_use_id": call.id, "content": obs}) + trace.append({"step": step, "thought": text, "tool": call.name, "tool_input": args, "obs": obs}) + messages.append({"role": "user", "content": results}) + return {"final": None, "trace": trace, "steps": max_iter, "truncated": True} + + +if __name__ == "__main__": + result = react_loop("Find Taipei population divided by New York population, then express it as a percentage.") + print(result) + + # === 自我檢查 === + assert result["final"] is not None, "expected the loop to reach end_turn" + assert any(str(n) in result["final"] for n in range(28, 35)), "expected a final answer near 31%" + print("Stage 3 exercise 4 starter check passed") diff --git a/examples/stage-3/04-multi-step-reasoning/test.py b/examples/stage-3/04-multi-step-reasoning/test.py new file mode 100644 index 0000000..ced487b --- /dev/null +++ b/examples/stage-3/04-multi-step-reasoning/test.py @@ -0,0 +1,99 @@ +"""練習 4 自我驗證 — Path A（Ollama starter.py）。 + +跑法： + python test.py + +驗證內容： + - divide / to_percentage 邊界正確（除 0、四捨五入） + - react_loop 跑完 4 步工具 + end_turn 給最終答案 + - mock 用 OpenAI-compat shape（不需要 Anthropic SDK） + +Anthropic 版本 test 見 test_anthropic.py。 +""" + +from __future__ import annotations + +import json +import sys +from types import SimpleNamespace +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter import divide, react_loop, to_percentage + + +def make_tool_call(call_id: str, name: str, args: dict): + return SimpleNamespace( + id=call_id, + type="function", + function=SimpleNamespace(name=name, arguments=json.dumps(args)), + ) + + +def make_resp(finish_reason: str, content: str = "", tool_calls=None): + msg = SimpleNamespace(content=content, tool_calls=tool_calls) + return SimpleNamespace(choices=[SimpleNamespace(finish_reason=finish_reason, message=msg)]) + + +def test_tools_handle_math_edges(): + assert divide(10, 2) == "5.0" + assert divide(10, 0) == "0" + assert to_percentage(0.3122) == "31.22" + print("✅ test_tools_handle_math_edges") + + +def test_population_ratio_uses_four_tool_steps(): + client = MagicMock() + client.chat.completions.create.side_effect = [ + make_resp("tool_calls", "Need Taipei.", [make_tool_call("t1", "lookup_population", {"city": "Taipei"})]), + make_resp("tool_calls", "Need NY.", [make_tool_call("t2", "lookup_population", {"city": "New York"})]), + make_resp("tool_calls", "Divide.", [make_tool_call("t3", "divide", {"a": 2602000, "b": 8336000})]), + make_resp("tool_calls", "Percentage.", [make_tool_call("t4", "to_percentage", {"ratio": 0.3122})]), + make_resp("stop", "Taipei is about 31% of New York by population."), + ] + result = react_loop("Compare Taipei and New York population.", client=client) + tools = [entry["tool"] for entry in result["trace"] if entry["tool"]] + assert tools == ["lookup_population", "lookup_population", "divide", "to_percentage"] + assert "31%" in result["final"] + assert result["steps"] == 5 + print("✅ test_population_ratio_uses_four_tool_steps") + + +def test_zero_population_path_still_finishes(): + client = MagicMock() + client.chat.completions.create.side_effect = [ + make_resp("tool_calls", "Numerator.", [make_tool_call("t1", "lookup_population", {"city": "Taipei"})]), + make_resp("tool_calls", "Denominator.", [make_tool_call("t2", "lookup_population", {"city": "Empty City"})]), + make_resp("tool_calls", "Safe divide.", [make_tool_call("t3", "divide", {"a": 2602000, "b": 0})]), + make_resp("tool_calls", "Percent.", [make_tool_call("t4", "to_percentage", {"ratio": 0})]), + make_resp("stop", "The denominator is zero, so the safe displayed percentage is 0%."), + ] + result = react_loop("Compare Taipei with an empty city.", client=client) + assert result["trace"][2]["obs"] == "0" + assert result["trace"][3]["obs"] == "0.00" + assert result["final"].endswith("0%.") + print("✅ test_zero_population_path_still_finishes") + + +def test_react_loop_respects_max_iter(): + """LLM 一直 tool_calls、永不收尾、應該在 max_iter 停。""" + client = MagicMock() + def never_ending(**kwargs): + return make_resp("tool_calls", "again", [make_tool_call("c", "lookup_population", {"city": "Taipei"})]) + client.chat.completions.create.side_effect = never_ending + + result = react_loop("never stop", max_iter=3, client=client) + assert result.get("truncated") is True + assert result["steps"] == 3 + assert result["final"] is None + print("✅ test_react_loop_respects_max_iter") + + +if __name__ == "__main__": + test_tools_handle_math_edges() + test_population_ratio_uses_four_tool_steps() + test_zero_population_path_still_finishes() + test_react_loop_respects_max_iter() + print("\n🎉 全部通過 — Ollama path 多步 ReAct loop 邏輯正確") diff --git a/examples/stage-3/04-multi-step-reasoning/test_anthropic.py b/examples/stage-3/04-multi-step-reasoning/test_anthropic.py new file mode 100644 index 0000000..ed81acf --- /dev/null +++ b/examples/stage-3/04-multi-step-reasoning/test_anthropic.py @@ -0,0 +1,75 @@ +"""Stage 3 練習 4 自我驗證 — Path B（Anthropic starter_anthropic.py）。 + +跑法： + python test_anthropic.py + +用 mock 取代 Anthropic client、不打真 API、$0/run。 +Ollama 版本見 test.py（OpenAI-compat shape）。 +""" + +from __future__ import annotations + +import sys +from types import SimpleNamespace +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter_anthropic import divide, react_loop, to_percentage + + +def block_text(text: str): + return SimpleNamespace(type="text", text=text) + + +def block_tool_use(tool_id: str, name: str, inp: dict): + return SimpleNamespace(type="tool_use", id=tool_id, name=name, input=inp) + + +def make_resp(stop_reason: str, *blocks): + return SimpleNamespace(stop_reason=stop_reason, content=list(blocks)) + + +def test_tools_handle_math_edges(): + assert divide(10, 2) == "5.0" + assert divide(10, 0) == "0" + assert to_percentage(0.3122) == "31.22" + + +def test_population_ratio_uses_four_tool_steps(): + client = MagicMock() + client.messages.create.side_effect = [ + make_resp("tool_use", block_text("Need Taipei population."), block_tool_use("t1", "lookup_population", {"city": "Taipei"})), + make_resp("tool_use", block_text("Need New York population."), block_tool_use("t2", "lookup_population", {"city": "New York"})), + make_resp("tool_use", block_text("Divide the two populations."), block_tool_use("t3", "divide", {"a": 2602000, "b": 8336000})), + make_resp("tool_use", block_text("Convert ratio to percent."), block_tool_use("t4", "to_percentage", {"ratio": 0.3122})), + make_resp("end_turn", block_text("Taipei is about 31% of New York by population.")), + ] + result = react_loop("Compare Taipei and New York population.", client=client) + tools = [entry["tool"] for entry in result["trace"] if entry["tool"]] + assert tools == ["lookup_population", "lookup_population", "divide", "to_percentage"] + assert "31%" in result["final"] + assert result["steps"] == 5 + + +def test_zero_population_path_still_finishes(): + client = MagicMock() + client.messages.create.side_effect = [ + make_resp("tool_use", block_text("Need numerator."), block_tool_use("t1", "lookup_population", {"city": "Taipei"})), + make_resp("tool_use", block_text("Need denominator."), block_tool_use("t2", "lookup_population", {"city": "Empty City"})), + make_resp("tool_use", block_text("Divide safely."), block_tool_use("t3", "divide", {"a": 2602000, "b": 0})), + make_resp("tool_use", block_text("Convert zero ratio."), block_tool_use("t4", "to_percentage", {"ratio": 0})), + make_resp("end_turn", block_text("The denominator is zero, so the safe displayed percentage is 0%.")), + ] + result = react_loop("Compare Taipei with an empty city.", client=client) + assert result["trace"][2]["obs"] == "0" + assert result["trace"][3]["obs"] == "0.00" + assert result["final"].endswith("0%.") + + +if __name__ == "__main__": + test_tools_handle_math_edges() + test_population_ratio_uses_four_tool_steps() + test_zero_population_path_still_finishes() + print("all pass") diff --git a/examples/stage-3/05-error-handling/README.en.md b/examples/stage-3/05-error-handling/README.en.md new file mode 100644 index 0000000..c05094e --- /dev/null +++ b/examples/stage-3/05-error-handling/README.en.md @@ -0,0 +1,100 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# Exercise 5: Tool Error Handling + +Corresponds to [Stage 3 — Tool Use & Agent Intro](../../../stages/03-tool-use-and-hello-agent.en.md) Exercise 5. + +## Why this matters + +Real agents rarely walk the happy path only: APIs time out, third parties go down, users send bad inputs. This exercise deliberately makes `fetch_weather(city)` return a **structured error** on the first call (`{"error": "network timeout", "retry_hint": "try again in 1s"}`) and succeed on the second; you observe how the ReAct loop hands the error observation back to the LLM and lets the model decide whether to retry, change the query, or give up. + +Core idea: **tool errors are data, not exceptions**. Return structured dicts, don't raise. + +## How to run — two paths + +### Path A (default, free, local) + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +Budget: **$0**. A 3-round loop takes ~10-60s. + +### Path B (Anthropic, cloud-quality comparison) + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +Budget: ~**$0.003** per run (claude-haiku-4-5, 3 rounds of accumulating messages). + +Expected output (Path A, local, ideal retry-then-succeed path): + +``` +❓ Question: Will it rain in Taipei today? (using Ollama qwen2.5:3b) +------------------------------------------------------------ +[step 0] tool: fetch_weather({'city': 'Taipei'}) → {'error': 'network timeout', 'retry_hint': 'try again in 1s'} +[step 1] tool: fetch_weather({'city': 'Taipei'}) → {'city': 'Taipei', 'forecast': 'rain', 'temperature_c': 24} +------------------------------------------------------------ +✅ Final answer: It will rain in Taipei today (24°C). +✅ Exercise 5 passed — tool errors are data, not exceptions, $0/run +``` + +## Validate the logic without API credits (mock-based) + +```bash +python test.py # validates Path A (Ollama) starter.py logic +python test_anthropic.py # validates Path B (Anthropic) starter_anthropic.py logic +``` + +Both test suites use `unittest.mock`, no real API call, $0/run. + +## Design reminders + +Errors should be structured data, so the LLM has context to make decisions: + +| Bad | Good | +|---|---| +| `raise Exception("failed")` | `return {"error": "network timeout", "retry_hint": "try again in 1s"}` | +| `return "failed"` | `return {"error": "...", "category": "transient", "retry_hint": "..."}` | +| Unbounded retry | `max_iter` safety + business-layer retry quota | + +Returning just `"failed"` leaves the model with nothing to act on. Adding `retry_hint`, error category, and recovery suggestions gives the model enough context to choose. And cap your retries — otherwise the agent loops forever on a broken tool. + +## What to watch on each path + +**Side observation**: small models (qwen2.5:3b) follow `retry_hint` less reliably than Claude — they might give up immediately or ignore the hint and repeat the same call. **That's exactly the teaching point**: in production, the same retry pattern produces different behaviors depending on how well a model reads structured errors — a real consideration when picking a model (we'll revisit in Stage 7). + +| Observation | Anthropic Claude haiku | Ollama qwen2.5:3b | +|---|---|---| +| Retries on `retry_hint` | High | Medium (may give up) | +| Graceful end after repeated failure | Stable | May retry a third time | +| Distinguishing transient vs permanent | Finer | Coarser | + +## Want smarter answers? + +Default is `claude-haiku-4-5` (cheapest). Try Sonnet: + +```bash +MODEL=claude-sonnet-5 python starter_anthropic.py +``` + +Or on the Ollama path, swap to a larger model: + +```bash +MODEL=qwen2.5:7b python starter.py +``` + +## Extensions + +- **Add a retry quota** — track `error_count` and give up after N +- **Add a circuit breaker** — after consecutive failures, stop calling for a while (avoids wave-after-wave on a broken downstream) +- **Classify errors** — transient (429 / connection) vs permanent (401 / 400) get different handling +- **Production tier** — see [`../../stage-1/05-error-handling/`](../../stage-1/05-error-handling/) for an API-level retry wrapper with exponential backoff + jitter diff --git a/examples/stage-3/05-error-handling/README.md b/examples/stage-3/05-error-handling/README.md new file mode 100644 index 0000000..bfaa79d --- /dev/null +++ b/examples/stage-3/05-error-handling/README.md @@ -0,0 +1,107 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 練習 5：Tool 錯誤處理 + +對應 [Stage 3 — Tool Use & Agent 入門](../../../stages/03-tool-use-and-hello-agent.md) 練習 5。 +> 🎓 **學習模式**：這份 `starter.py` 是**完整解答**、不是 TODO skeleton。建議用**主動模式**——`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫一份 `starter.py`、跑 `python test.py` 驗證；卡 20 分鐘再回去對照 reference。完整方法論看 [`docs/HOW_TO_USE.md`](../../../docs/HOW_TO_USE.md)。 + +> 📚 **想要 chapter-length 深入版？** 本 folder 的 starter 是 70-150 行 illustrative 版、聚焦 `核心 pattern + 兩條 SDK path`，不是進階深度教材。深度教材推薦： +> - [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents) ⭐ 中文圈最完整、章節式 + 16 種 production 能力。**本練習對應 hello-agents 的 Extra Chapter 錯誤處理 / circuit breaker** +> - [ 5 結構化錯誤回傳](../../../resources/schema-design-cheatsheet.md)（本 repo 既有 cheatsheet） +> - 完整 references 見 [Stage 3 精選 Projects](../../../stages/03-tool-use-and-hello-agent.md#-精選-projects) + + +## 為什麼這題重要 + +真實 agent 很少只走成功路徑：API 會 timeout、第三方服務暫時不可用、user 傳壞參數。這題故意讓 `fetch_weather(city)` 第一次回**結構化 error**（`{"error": "network timeout", "retry_hint": "try again in 1s"}`）、第二次才成功；觀察 ReAct loop 怎麼把 error observation 交回 LLM、讓模型自己決定 retry / 改 query / 放棄。 + +核心觀念：**tool error 是資料、不是 exception**。回傳結構化 dict、不要 raise。 + +## 怎麼跑 — 兩條路徑 + +### Path A（默認、本機免費） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +預算：**$0**。3 輪 loop ≈ 10-60 秒。 + +### Path B（Anthropic、想看 cloud 高品質） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +預算：每次 ≈ **$0.003**（claude-haiku-4-5、3 輪 messages 累積）。 + +預期看到（Path A、本機，理想 retry 走法）： + +``` +❓ 問題：Will it rain in Taipei today?（using Ollama qwen2.5:3b） +------------------------------------------------------------ +[step 0] tool: fetch_weather({'city': 'Taipei'}) → {'error': 'network timeout', 'retry_hint': 'try again in 1s'} +[step 1] tool: fetch_weather({'city': 'Taipei'}) → {'city': 'Taipei', 'forecast': 'rain', 'temperature_c': 24} +------------------------------------------------------------ +✅ 最終答案：It will rain in Taipei today (24°C). +✅ 練習 5 通過 — tool error 是 data 不是 exception、$0/run +``` + +## 不花錢驗證程式邏輯（mock-based） + +```bash +python test.py # 驗 Path A (Ollama) starter.py 邏輯 +python test_anthropic.py # 驗 Path B (Anthropic) starter_anthropic.py 邏輯 +``` + +兩條 test 都用 `unittest.mock`、不打真 API、$0/run。 + +## 設計提醒 + +錯誤也應該是結構化資料，讓 LLM 有 context 做決策： + +| Bad | Good | +|---|---| +| `raise Exception("failed")` | `return {"error": "network timeout", "retry_hint": "try again in 1s"}` | +| `return "failed"` | `return {"error": "...", "category": "transient", "retry_hint": "..."}` | +| 無限 retry | `max_iter` safety + 業務層 retry quota | + +只回傳 `"failed"` 讓模型不知道下一步；加入 `retry_hint`、錯誤類型與可恢復建議，模型才有足夠 context 做決策。retry 次數也要有限制，否則 agent 會在壞掉的工具前面無限打轉。 + +## 兩個 path 觀察重點 + +**附加觀察**：小 model（qwen2.5:3b）對 `retry_hint` 的 follow-up 可能不如 Claude 精細——可能會直接放棄、或無視 hint 重複同一個錯。**這恰好是教學點**：production 寫好 retry pattern 後，不同 model 對結構化 error 的「閱讀力」差距，是選 model 的考量之一（Stage 7 production tier 會再回來討論）。 + +| 觀察項 | Anthropic Claude haiku | Ollama qwen2.5:3b | +|---|---|---| +| 看到 retry_hint 就 retry | 高機率 | 中機率（可能直接放棄） | +| 連續失敗後 graceful end | 穩定 | 可能再 retry 第 3 次 | +| 錯誤類型分流（transient vs permanent） | 較細緻 | 較粗略 | + +## 想看更聰明的答案？ + +預設用 `claude-haiku-4-5`（最便宜）。改成 sonnet： + +```bash +MODEL=claude-sonnet-5 python starter_anthropic.py +``` + +或 Ollama path 換更大 model： + +```bash +MODEL=qwen2.5:7b python starter.py +``` + +## 延伸 + +- **加 retry quota**：在 loop 加 `error_count`、超過 N 次就放棄 +- **加 circuit breaker**：連續失敗、暫時 stop call（避免 wave-after-wave 打死下游） +- **錯誤分類**：transient（429 / connection）vs permanent（401 / 400）、不同處理 +- **Production 級**：看 [`../../stage-1/05-error-handling/`](../../stage-1/05-error-handling/) 的 API-level retry wrapper（exponential backoff + jitter） diff --git a/examples/stage-3/05-error-handling/README.zh-Hans.md b/examples/stage-3/05-error-handling/README.zh-Hans.md new file mode 100644 index 0000000..653cff4 --- /dev/null +++ b/examples/stage-3/05-error-handling/README.zh-Hans.md @@ -0,0 +1,100 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 练习 5：Tool 错误处理 + +对应 [Stage 3 — Tool Use & Agent 入门](../../../stages/03-tool-use-and-hello-agent.zh-Hans.md) 练习 5。 + +## 为什么这题重要 + +真实 agent 很少只走成功路径：API 会 timeout、第三方服务暂时不可用、user 传坏参数。这题故意让 `fetch_weather(city)` 第一次回**结构化 error**（`{"error": "network timeout", "retry_hint": "try again in 1s"}`）、第二次才成功；观察 ReAct loop 怎么把 error observation 交回 LLM、让模型自己决定 retry / 改 query / 放弃。 + +核心观念：**tool error 是数据、不是 exception**。回传结构化 dict、不要 raise。 + +## 怎么跑 — 两条路径 + +### Path A（默认、本机免费） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +预算：**$0**。3 轮 loop ≈ 10-60 秒。 + +### Path B（Anthropic、想看 cloud 高质量） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +预算：每次 ≈ **$0.003**（claude-haiku-4-5、3 轮 messages 累积）。 + +预期看到（Path A、本机，理想 retry 走法）： + +``` +❓ 问题：Will it rain in Taipei today?（using Ollama qwen2.5:3b） +------------------------------------------------------------ +[step 0] tool: fetch_weather({'city': 'Taipei'}) → {'error': 'network timeout', 'retry_hint': 'try again in 1s'} +[step 1] tool: fetch_weather({'city': 'Taipei'}) → {'city': 'Taipei', 'forecast': 'rain', 'temperature_c': 24} +------------------------------------------------------------ +✅ 最终答案：It will rain in Taipei today (24°C). +✅ 练习 5 通过 — tool error 是 data 不是 exception、$0/run +``` + +## 不花钱验证程序逻辑（mock-based） + +```bash +python test.py # 验 Path A (Ollama) starter.py 逻辑 +python test_anthropic.py # 验 Path B (Anthropic) starter_anthropic.py 逻辑 +``` + +两条 test 都用 `unittest.mock`、不打真 API、$0/run。 + +## 设计提醒 + +错误也应该是结构化数据，让 LLM 有 context 做决策： + +| Bad | Good | +|---|---| +| `raise Exception("failed")` | `return {"error": "network timeout", "retry_hint": "try again in 1s"}` | +| `return "failed"` | `return {"error": "...", "category": "transient", "retry_hint": "..."}` | +| 无限 retry | `max_iter` safety + 业务层 retry quota | + +只回传 `"failed"` 让模型不知道下一步；加入 `retry_hint`、错误类型与可恢复建议，模型才有足够 context 做决策。retry 次数也要有限制，否则 agent 会在坏掉的工具前面无限打转。 + +## 两个 path 观察重点 + +**附加观察**：小 model（qwen2.5:3b）对 `retry_hint` 的 follow-up 可能不如 Claude 精细——可能会直接放弃、或无视 hint 重复同一个错。**这恰好是教学点**：production 写好 retry pattern 后，不同 model 对结构化 error 的“阅读力”差距，是选 model 的考量之一（Stage 7 production tier 会再回来讨论）。 + +| 观察项 | Anthropic Claude haiku | Ollama qwen2.5:3b | +|---|---|---| +| 看到 retry_hint 就 retry | 高机率 | 中机率（可能直接放弃） | +| 连续失败后 graceful end | 稳定 | 可能再 retry 第 3 次 | +| 错误类型分流（transient vs permanent） | 较细致 | 较粗略 | + +## 想看更聪明的答案？ + +预设用 `claude-haiku-4-5`（最便宜）。改成 sonnet： + +```bash +MODEL=claude-sonnet-5 python starter_anthropic.py +``` + +或 Ollama path 换更大 model： + +```bash +MODEL=qwen2.5:7b python starter.py +``` + +## 延伸 + +- **加 retry quota**：在 loop 加 `error_count`、超过 N 次就放弃 +- **加 circuit breaker**：连续失败、暂时 stop call（避免 wave-after-wave 打死下游） +- **错误类型分类**：transient（429 / connection）vs permanent（401 / 400）、不同处理 +- **Production 级**：看 [`../../stage-1/05-error-handling/`](../../stage-1/05-error-handling/) 的 API-level retry wrapper（exponential backoff + jitter） diff --git a/examples/stage-3/05-error-handling/requirements.txt b/examples/stage-3/05-error-handling/requirements.txt new file mode 100644 index 0000000..a4a9cdf --- /dev/null +++ b/examples/stage-3/05-error-handling/requirements.txt @@ -0,0 +1,2 @@ +openai>=1.50,<2.0 +anthropic>=0.40,<1.0 # 只 starter_anthropic.py 需要、Ollama path 不用 diff --git a/examples/stage-3/05-error-handling/starter.py b/examples/stage-3/05-error-handling/starter.py new file mode 100644 index 0000000..74972a1 --- /dev/null +++ b/examples/stage-3/05-error-handling/starter.py @@ -0,0 +1,129 @@ +"""練習 5：Tool 錯誤處理 — Path A（Ollama 默認、本機免費）。 + +故意讓 fetch_weather 第一次回結構化 error、第二次才成功。觀察 ReAct loop 怎麼 +把 error observation 交回 LLM、讓模型自己決定 retry / 改 query / 放棄。 +重點：error 是結構化 dict（`{"error", "retry_hint"}`）、不是 Python exception。 + +跑法： + pip install -r requirements.txt + ollama pull qwen2.5:3b + ollama serve + python starter.py + +驗證： + python test.py （用 mock、不打 API） + +想看 Anthropic Claude 版本： + python starter_anthropic.py （需 ANTHROPIC_API_KEY、$0.003/run） + +⚠️ 注意：小 model 對 retry_hint 的 follow-up 較弱、可能直接放棄或無視 hint。 +這恰好是教學點——同樣 structured error pattern、不同 model 的「閱讀力」差。 +""" + +from __future__ import annotations + +import json +import os +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +MODEL = os.environ.get("MODEL", "qwen2.5:3b") +_failure_plan: list[bool] = [True, False] + + +def set_weather_failures(plan: list[bool]) -> None: + global _failure_plan + _failure_plan = list(plan) + + +def fetch_weather(city: str) -> dict: + """假的 weather API、用 _failure_plan 決定這次該失敗還是成功。""" + should_fail = _failure_plan.pop(0) if _failure_plan else False + if should_fail: + return {"error": "network timeout", "retry_hint": "try again in 1s"} + return {"city": city, "forecast": "rain", "temperature_c": 24} + + +TOOLS_SPEC = [ + { + "type": "function", + "function": { + "name": "fetch_weather", + "description": "Fetch current weather. If an error is returned, inspect retry_hint before retrying.", + "parameters": { + "type": "object", + "properties": {"city": {"type": "string", "description": "City name"}}, + "required": ["city"], + }, + }, + } +] + + +def react_loop(question: str, max_iter: int = 5, client: Any = None) -> dict: + """OpenAI-compat ReAct loop。tool error 是結構化 JSON、放進 tool_result 給 LLM 看。""" + client = client or OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + messages = [{"role": "user", "content": question}] + trace: list[dict] = [] + + for step in range(max_iter): + resp = client.chat.completions.create( + model=MODEL, + tools=TOOLS_SPEC, + messages=messages, + ) + msg = resp.choices[0].message + text = msg.content or "" + tool_calls = msg.tool_calls or [] + + assistant_entry: dict = {"role": "assistant", "content": text} + if tool_calls: + assistant_entry["tool_calls"] = [ + { + "id": tc.id, + "type": "function", + "function": {"name": tc.function.name, "arguments": tc.function.arguments}, + } + for tc in tool_calls + ] + messages.append(assistant_entry) + + if resp.choices[0].finish_reason == "stop" or not tool_calls: + trace.append({"step": step, "thought": text, "tool": None, "obs": None}) + return {"final": text, "trace": trace, "steps": step + 1} + + for tc in tool_calls: + args = json.loads(tc.function.arguments) + obs = fetch_weather(args["city"]) if tc.function.name == "fetch_weather" else {"error": "unknown tool"} + # OpenAI-compat 的 tool message content 接受字串、把 dict 序列化 + messages.append({ + "role": "tool", + "tool_call_id": tc.id, + "content": json.dumps(obs, ensure_ascii=False), + }) + trace.append({"step": step, "thought": text, "tool": tc.function.name, "tool_input": args, "obs": obs}) + + return {"final": None, "trace": trace, "steps": max_iter, "truncated": True} + + +if __name__ == "__main__": + set_weather_failures([True, False]) # 第一次失敗、第二次成功 + print(f"❓ 問題：Will it rain in Taipei today?（using Ollama {MODEL}）") + print("-" * 60) + + result = react_loop("Will it rain in Taipei today?") + for entry in result["trace"]: + if entry["tool"]: + print(f"[step {entry['step']}] tool: {entry['tool']}({entry.get('tool_input')}) → {entry['obs']}") + print("-" * 60) + print(f"✅ 最終答案：{result['final']}") + + # 寬鬆驗證：loop 至少要看到結構化 error 在 trace 裡（小 model 不一定 retry） + saw_error = any(isinstance(e["obs"], dict) and "error" in e["obs"] for e in result["trace"]) + assert saw_error, "預期至少一輪 tool 回傳結構化 error" + print("✅ 練習 5 通過 — 你已用本機 qwen2.5:3b 看見 tool error 在 ReAct loop 裡是 data 不是 exception、$0/run") diff --git a/examples/stage-3/05-error-handling/starter_anthropic.py b/examples/stage-3/05-error-handling/starter_anthropic.py new file mode 100644 index 0000000..8f57a7f --- /dev/null +++ b/examples/stage-3/05-error-handling/starter_anthropic.py @@ -0,0 +1,88 @@ +"""練習 5：Tool 錯誤處理 — Path B（Anthropic Claude）。 + +故意讓 fetch_weather 第一次回 error、第二次才成功。觀察 ReAct loop 怎麼把錯誤 +observation 交回 LLM、讓模型決定要 retry / 改 query / 放棄。錯誤回傳是 +結構化 dict（`{"error", "retry_hint"}`）、不是 Python exception。 + +跑法： + pip install -r requirements.txt + export ANTHROPIC_API_KEY=sk-ant-... + python starter_anthropic.py + +預算：每次 ≈ $0.003（claude-haiku-4-5、3 輪 messages 累積）。 +Ollama 版本見 starter.py（本機 $0、但小 model 對 retry hint 的閱讀力較弱）。 +""" + +from __future__ import annotations + +import json +import os +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic + +MODEL = os.environ.get("MODEL", "claude-haiku-4-5") +_failure_plan: list[bool] = [True, False] + + +def set_weather_failures(plan: list[bool]) -> None: + global _failure_plan + _failure_plan = list(plan) + + +def fetch_weather(city: str) -> dict: + should_fail = _failure_plan.pop(0) if _failure_plan else False + if should_fail: + return {"error": "network timeout", "retry_hint": "try again in 1s"} + return {"city": city, "forecast": "rain", "temperature_c": 24} + + +TOOLS_SPEC = [ + { + "name": "fetch_weather", + "description": "Fetch current weather. If an error is returned, inspect retry_hint before retrying.", + "input_schema": { + "type": "object", + "properties": {"city": {"type": "string", "description": "City name"}}, + "required": ["city"], + }, + } +] + + +def react_loop(question: str, max_iter: int = 5, client: Any = None) -> dict: + client = client or anthropic.Anthropic() + messages = [{"role": "user", "content": question}] + trace: list[dict] = [] + for step in range(max_iter): + resp = client.messages.create(model=MODEL, max_tokens=1024, tools=TOOLS_SPEC, messages=messages) + text = " ".join(getattr(b, "text", "") for b in resp.content if getattr(b, "type", None) == "text") + calls = [b for b in resp.content if getattr(b, "type", None) == "tool_use"] + messages.append({"role": "assistant", "content": resp.content}) + if resp.stop_reason == "end_turn" or not calls: + trace.append({"step": step, "thought": text, "tool": None, "obs": None}) + return {"final": text, "trace": trace, "steps": step + 1} + results = [] + for call in calls: + args = dict(call.input) + obs = fetch_weather(args["city"]) if call.name == "fetch_weather" else {"error": "unknown tool"} + results.append({"type": "tool_result", "tool_use_id": call.id, "content": json.dumps(obs, ensure_ascii=False)}) + trace.append({"step": step, "thought": text, "tool": call.name, "tool_input": args, "obs": obs}) + messages.append({"role": "user", "content": results}) + return {"final": None, "trace": trace, "steps": max_iter, "truncated": True} + + +if __name__ == "__main__": + set_weather_failures([True, False]) + result = react_loop("Will it rain in Taipei today?") + print(result) + + # === 自我檢查 === + assert result["trace"][0]["obs"]["error"] == "network timeout" + assert result["trace"][1]["obs"]["forecast"] == "rain" + assert result["final"] is not None + print("Stage 3 exercise 5 starter check passed") diff --git a/examples/stage-3/05-error-handling/test.py b/examples/stage-3/05-error-handling/test.py new file mode 100644 index 0000000..dfb8cc1 --- /dev/null +++ b/examples/stage-3/05-error-handling/test.py @@ -0,0 +1,100 @@ +"""練習 5 自我驗證 — Path A（Ollama starter.py）。 + +跑法： + python test.py + +驗證內容： + - fetch_weather failure plan 邏輯正確 + - react_loop 第一輪錯誤 + 第二輪 retry + 第三輪 end_turn + - 連續失敗也能 graceful end（不 crash） + - mock 用 OpenAI-compat shape + +Anthropic 版本 test 見 test_anthropic.py。 +""" + +from __future__ import annotations + +import json +import sys +from types import SimpleNamespace +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter import fetch_weather, react_loop, set_weather_failures + + +def make_tool_call(call_id: str, name: str, args: dict): + return SimpleNamespace( + id=call_id, + type="function", + function=SimpleNamespace(name=name, arguments=json.dumps(args)), + ) + + +def make_resp(finish_reason: str, content: str = "", tool_calls=None): + msg = SimpleNamespace(content=content, tool_calls=tool_calls) + return SimpleNamespace(choices=[SimpleNamespace(finish_reason=finish_reason, message=msg)]) + + +def test_fetch_weather_failure_plan(): + set_weather_failures([True, False]) + first = fetch_weather("Taipei") + second = fetch_weather("Taipei") + assert first["error"] == "network timeout" + assert second["forecast"] == "rain" + print("✅ test_fetch_weather_failure_plan") + + +def test_retry_then_success(): + set_weather_failures([True, False]) + client = MagicMock() + client.chat.completions.create.side_effect = [ + make_resp("tool_calls", "Check weather.", [make_tool_call("t1", "fetch_weather", {"city": "Taipei"})]), + make_resp("tool_calls", "Got retry_hint, retry once.", [make_tool_call("t2", "fetch_weather", {"city": "Taipei"})]), + make_resp("stop", "Taipei is rainy now."), + ] + result = react_loop("Will it rain in Taipei?", client=client) + tools = [entry["tool"] for entry in result["trace"] if entry["tool"]] + assert tools == ["fetch_weather", "fetch_weather"] + assert result["trace"][0]["obs"]["retry_hint"] == "try again in 1s" + assert result["trace"][1]["obs"]["forecast"] == "rain" + assert result["final"] == "Taipei is rainy now." + print("✅ test_retry_then_success") + + +def test_repeated_errors_can_end_gracefully(): + set_weather_failures([True, True]) + client = MagicMock() + client.chat.completions.create.side_effect = [ + make_resp("tool_calls", "Try weather API.", [make_tool_call("t1", "fetch_weather", {"city": "Taipei"})]), + make_resp("tool_calls", "Retry once.", [make_tool_call("t2", "fetch_weather", {"city": "Taipei"})]), + make_resp("stop", "Weather is unavailable after retry; please try again later."), + ] + result = react_loop("Will it rain in Taipei?", client=client) + assert result["trace"][0]["obs"]["error"] == "network timeout" + assert result["trace"][1]["obs"]["error"] == "network timeout" + assert "unavailable" in result["final"] + print("✅ test_repeated_errors_can_end_gracefully") + + +def test_unknown_tool_returns_structured_error(): + """模擬 LLM 呼叫不存在的 tool、確認 loop 不 crash、而是把 error 給 LLM 看。""" + set_weather_failures([]) + client = MagicMock() + client.chat.completions.create.side_effect = [ + make_resp("tool_calls", "I'll try a weird tool.", [make_tool_call("t1", "magic_tool", {"city": "Taipei"})]), + make_resp("stop", "OK, no magic tool here."), + ] + result = react_loop("Try magic.", client=client) + assert result["trace"][0]["obs"]["error"] == "unknown tool" + print("✅ test_unknown_tool_returns_structured_error") + + +if __name__ == "__main__": + test_fetch_weather_failure_plan() + test_retry_then_success() + test_repeated_errors_can_end_gracefully() + test_unknown_tool_returns_structured_error() + print("\n🎉 全部通過 — Ollama path tool error handling 邏輯正確") diff --git a/examples/stage-3/05-error-handling/test_anthropic.py b/examples/stage-3/05-error-handling/test_anthropic.py new file mode 100644 index 0000000..bcc8a09 --- /dev/null +++ b/examples/stage-3/05-error-handling/test_anthropic.py @@ -0,0 +1,76 @@ +"""Stage 3 練習 5 自我驗證 — Path B（Anthropic starter_anthropic.py）。 + +跑法： + python test_anthropic.py + +用 mock 取代 Anthropic client、不打真 API、$0/run。 +Ollama 版本見 test.py（OpenAI-compat shape）。 +""" + +from __future__ import annotations + +import sys +from types import SimpleNamespace +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter_anthropic import fetch_weather, react_loop, set_weather_failures + + +def block_text(text: str): + return SimpleNamespace(type="text", text=text) + + +def block_tool_use(tool_id: str, name: str, inp: dict): + return SimpleNamespace(type="tool_use", id=tool_id, name=name, input=inp) + + +def make_resp(stop_reason: str, *blocks): + return SimpleNamespace(stop_reason=stop_reason, content=list(blocks)) + + +def test_fetch_weather_failure_plan(): + set_weather_failures([True, False]) + first = fetch_weather("Taipei") + second = fetch_weather("Taipei") + assert first["error"] == "network timeout" + assert second["forecast"] == "rain" + + +def test_retry_then_success(): + set_weather_failures([True, False]) + client = MagicMock() + client.messages.create.side_effect = [ + make_resp("tool_use", block_text("Check weather."), block_tool_use("t1", "fetch_weather", {"city": "Taipei"})), + make_resp("tool_use", block_text("Tool returned retry_hint, retry once."), block_tool_use("t2", "fetch_weather", {"city": "Taipei"})), + make_resp("end_turn", block_text("Taipei is rainy now.")), + ] + result = react_loop("Will it rain in Taipei?", client=client) + tools = [entry["tool"] for entry in result["trace"] if entry["tool"]] + assert tools == ["fetch_weather", "fetch_weather"] + assert result["trace"][0]["obs"]["retry_hint"] == "try again in 1s" + assert result["trace"][1]["obs"]["forecast"] == "rain" + assert result["final"] == "Taipei is rainy now." + + +def test_repeated_errors_can_end_gracefully(): + set_weather_failures([True, True]) + client = MagicMock() + client.messages.create.side_effect = [ + make_resp("tool_use", block_text("Try weather API."), block_tool_use("t1", "fetch_weather", {"city": "Taipei"})), + make_resp("tool_use", block_text("Retry once."), block_tool_use("t2", "fetch_weather", {"city": "Taipei"})), + make_resp("end_turn", block_text("Weather is unavailable after retry; please try again later.")), + ] + result = react_loop("Will it rain in Taipei?", client=client) + assert result["trace"][0]["obs"]["error"] == "network timeout" + assert result["trace"][1]["obs"]["error"] == "network timeout" + assert "unavailable" in result["final"] + + +if __name__ == "__main__": + test_fetch_weather_failure_plan() + test_retry_then_success() + test_repeated_errors_can_end_gracefully() + print("all pass") diff --git a/examples/stage-3/06-schema-design/README.en.md b/examples/stage-3/06-schema-design/README.en.md new file mode 100644 index 0000000..7d6521e --- /dev/null +++ b/examples/stage-3/06-schema-design/README.en.md @@ -0,0 +1,84 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# Exercise 6: Function Schema Design (bad vs good) + +Corresponds to [Stage 3 — Tool Use & Agent Intro](../../../stages/03-tool-use-and-hello-agent.en.md) Exercise 6. + +## Why this matters + +Schemas are **part of the prompt** — and they're the part the model **leans on hardest** when choosing a tool. This exercise gives you `starter_bad` and `starter_good` for the same question: "Convert 32 Celsius to Fahrenheit." + +- **Bad schema**: short descriptions, every param as string, no `required`, no `enum` → LLM frequently misroutes temperature conversion to `process_data` +- **Good schema**: clear usage, `value: number`, `unit: enum["celsius", "fahrenheit"]`, all required fields listed → reliably routes to `convert_temperature` + +When you write a schema, don't aim for "a human can read this". Aim for "the model can use this to rule out the wrong tool". + +## How to run — two paths + +### Path A (default, free, local, 4 starters) + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve + +python starter_bad.py # watch a bad schema mislead qwen +python starter_good.py # watch a good schema lead qwen to the right tool +``` + +Budget: **$0**. + +### Path B (Anthropic, cloud-quality comparison) + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... + +python starter_bad_anthropic.py +python starter_good_anthropic.py +``` + +Budget: ~**$0.0005** per run (claude-haiku-4-5, single call). + +## Validate the logic without API credits (mock-based) + +```bash +python test.py # validates Path A (Ollama) starter_bad + starter_good +python test_anthropic.py # validates Path B (Anthropic) starter_*_anthropic +``` + +Each test suite also asserts on the schema structure directly (`good` has `required` + `enum`; `bad` doesn't) — not just on the LLM's choice. + +## Bad vs good schema A/B + +| Design dimension | Bad | Good | +|---|---|---| +| Description | "Process data." | "Use only to summarize structured JSON table rows. Do not use for temperature conversion." | +| Param types | All `string` | `number` / `array` / actual types | +| Required | None | `["value", "unit"]` | +| Enum constraint | None | `["celsius", "fahrenheit"]` | +| Error return | Plain string | Structured dict + retry_hint | + +## What to watch on each path (the teaching point) + +**Small models are more sensitive to schema quality than large ones** — so this exercise is **more pedagogically valuable on Ollama**: + +| Observation | Anthropic Claude haiku | Ollama qwen2.5:3b | +|---|---|---| +| Bad schema can still guess right | Medium-high | Low (almost always wrong) | +| Good schema picks correctly | Stable | Stable | +| Gap between bad and good | Small | Large | + +In other words: **time spent writing good schemas saves you the cost of upgrading the model**. Want to run a cheap model (qwen / mistral) in production? Your schemas need to be solid enough to run in production. + +## Further reading + +More schema design rules in [`resources/schema-design-cheatsheet.md`](../../../resources/schema-design-cheatsheet.md): clear usage, correct types, required fields, enum constraints, structured error returns. + +## Extensions + +- **Deliberately break the good schema** — remove one `enum` constraint and watch qwen start to miss +- **Add a third tool** — one with usage similar to but boundary-blurry with `convert_temperature`, and observe the LLM's choice +- **Combine with the structured-error pattern** from [`../05-error-handling/`](../05-error-handling/) — schema design + error handling is the combo for shipping to production diff --git a/examples/stage-3/06-schema-design/README.md b/examples/stage-3/06-schema-design/README.md new file mode 100644 index 0000000..ba2273c --- /dev/null +++ b/examples/stage-3/06-schema-design/README.md @@ -0,0 +1,91 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 練習 6：Function Schema 設計（bad vs good） + +對應 [Stage 3 — Tool Use & Agent 入門](../../../stages/03-tool-use-and-hello-agent.md) 練習 6。 +> 🎓 **學習模式**：這份 `starter.py` 是**完整解答**、不是 TODO skeleton。建議用**主動模式**——`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫一份 `starter.py`、跑 `python test.py` 驗證；卡 20 分鐘再回去對照 reference。完整方法論看 [`docs/HOW_TO_USE.md`](../../../docs/HOW_TO_USE.md)。 + +> 📚 **想要 chapter-length 深入版？** 本 folder 的 starter 是 70-150 行 illustrative 版、聚焦 `核心 pattern + 兩條 SDK path`，不是進階深度教材。深度教材推薦： +> - [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents) ⭐ 中文圈最完整、章節式 + 16 種 production 能力。**本練習對應 hello-agents 的 [Extra08 — 如何寫出好的 Skill](https://github.com/datawhalechina/hello-agents/blob/main/Extra-Chapter/Extra08-如何写出好的Skill.md)** +> - [OpenAI Function Calling guide](https://platform.openai.com/docs/guides/function-calling) + [](../../../resources/schema-design-cheatsheet.md) +> - 完整 references 見 [Stage 3 精選 Projects](../../../stages/03-tool-use-and-hello-agent.md#-精選-projects) + + +## 為什麼這題重要 + +Schema 是 **prompt 的一部分**、而且是模型做工具選擇時**最依賴**的 prompt。這題用 `starter_bad` 與 `starter_good` 對照同一題：「把攝氏 32 度換成華氏」。 + +- **Bad schema**：description 太短、參數都 string、沒 required、沒 enum → LLM 容易把溫度轉換丟給 `process_data` +- **Good schema**：用途明確、`value: number`、`unit: enum["celsius", "fahrenheit"]`、required 都列好 → 穩定選到 `convert_temperature` + +寫 schema 不要只想「人看得懂」、要想「模型能不能用它排除錯誤工具」。 + +## 怎麼跑 — 兩條路徑 + +### Path A（默認、本機免費、4 個 starter） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve + +python starter_bad.py # 觀察壞 schema 怎麼讓 qwen 挑錯 +python starter_good.py # 觀察好 schema 怎麼讓 qwen 挑對 +``` + +預算：**$0**。 + +### Path B（Anthropic、想看 cloud 高品質） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... + +python starter_bad_anthropic.py +python starter_good_anthropic.py +``` + +預算：每次 ≈ **$0.0005**（claude-haiku-4-5、單輪 call）。 + +## 不花錢驗證程式邏輯（mock-based） + +```bash +python test.py # 驗 Path A (Ollama) starter_bad + starter_good +python test_anthropic.py # 驗 Path B (Anthropic) starter_*_anthropic +``` + +兩條 test 都用 `unittest.mock`、不打真 API、$0/run。每組 test 都直接檢查 schema 結構（good 有 `required` + `enum`、bad 沒有），不只是看 LLM 怎麼選。 + +## Bad vs Good schema 對照 + +| 設計面向 | Bad | Good | +|---|---|---| +| Description | "Process data." | "Use only to summarize structured JSON table rows. Do not use for temperature conversion." | +| 參數型別 | 全部 `string` | `number` / `array` / 對應實際型別 | +| Required | 無 | `["value", "unit"]` | +| Enum 收斂 | 無 | `["celsius", "fahrenheit"]` | +| 失敗回傳 | 簡單字串 | 結構化 dict + retry_hint | + +## 兩個 path 的觀察重點（教學重點） + +**小 model 對 schema 質量的敏感度比大 model 高**——這題在 Ollama 上**反而更有教學意義**： + +| 觀察項 | Anthropic Claude haiku | Ollama qwen2.5:3b | +|---|---|---| +| Bad schema 仍能猜對 | 中-高機率 | 低機率（幾乎必錯） | +| Good schema 選對 | 穩定 | 穩定 | +| 差距 | 小 | 大 | + +換句話說：**寫 schema 的功夫、在小 model 上能省下換大 model 的成本**。Production 想用便宜 model（qwen / mistral）？schema 必須寫到能上線跑的程度。 + +## 延伸閱讀 + +更多 schema 設計規則對照 [`resources/schema-design-cheatsheet.md`](../../../resources/schema-design-cheatsheet.md)：清楚用途、正確型別、必填欄位、enum 收斂、結構化錯誤回傳。 + +## 延伸 + +- **故意改壞 good schema**：把一個 enum 拿掉、看 qwen 是否就開始挑錯 +- **加第三個工具**：寫一個跟 `convert_temperature` 用途相近但邊界模糊的 tool、看 LLM 怎麼挑 +- **接 [`../05-error-handling/`](../05-error-handling/) 的 structured error pattern**：結合 schema 設計 + 錯誤處理、production 級 diff --git a/examples/stage-3/06-schema-design/README.zh-Hans.md b/examples/stage-3/06-schema-design/README.zh-Hans.md new file mode 100644 index 0000000..daec03d --- /dev/null +++ b/examples/stage-3/06-schema-design/README.zh-Hans.md @@ -0,0 +1,84 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 练习 6：Function Schema 设计（bad vs good） + +对应 [Stage 3 — Tool Use & Agent 入门](../../../stages/03-tool-use-and-hello-agent.zh-Hans.md) 练习 6。 + +## 为什么这题重要 + +Schema 是 **prompt 的一部分**、而且是模型做工具选择时**最依赖**的 prompt。这题用 `starter_bad` 与 `starter_good` 对照同一题：“把摄氏 32 度换成华氏”。 + +- **Bad schema**：description 太短、参数都 string、没 required、没 enum → LLM 容易把温度转换丢给 `process_data` +- **Good schema**：用途明确、`value: number`、`unit: enum["celsius", "fahrenheit"]`、required 都列好 → 稳定选到 `convert_temperature` + +写 schema 不要只想“人看得懂”、要想“模型能不能用它排除错误工具”。 + +## 怎么跑 — 两条路径 + +### Path A（默认、本机免费、4 个 starter） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve + +python starter_bad.py # 观察坏 schema 怎么让 qwen 挑错 +python starter_good.py # 观察好 schema 怎么让 qwen 挑对 +``` + +预算：**$0**。 + +### Path B（Anthropic、想看 cloud 高质量） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... + +python starter_bad_anthropic.py +python starter_good_anthropic.py +``` + +预算：每次 ≈ **$0.0005**（claude-haiku-4-5、单轮 call）。 + +## 不花钱验证程序逻辑（mock-based） + +```bash +python test.py # 验 Path A (Ollama) starter_bad + starter_good +python test_anthropic.py # 验 Path B (Anthropic) starter_*_anthropic +``` + +两条 test 都用 `unittest.mock`、不打真 API、$0/run。每组 test 都直接检查 schema 结构（good 有 `required` + `enum`、bad 没有），不只是看 LLM 怎么选。 + +## Bad vs Good schema 对照 + +| 设计面向 | Bad | Good | +|---|---|---| +| Description | "Process data." | "Use only to summarize structured JSON table rows. Do not use for temperature conversion." | +| 参数类型 | 全部 `string` | `number` / `array` / 对应实际类型 | +| Required | 无 | `["value", "unit"]` | +| Enum 收敛 | 无 | `["celsius", "fahrenheit"]` | +| 失败回传 | 简单字符串 | 结构化 dict + retry_hint | + +## 两个 path 的观察重点（教学重点） + +**小 model 对 schema 质量的敏感度比大 model 高**——这题在 Ollama 上**反而更有教学意义**： + +| 观察项 | Anthropic Claude haiku | Ollama qwen2.5:3b | +|---|---|---| +| Bad schema 仍能猜对 | 中-高机率 | 低机率（几乎必错） | +| Good schema 选对 | 稳定 | 稳定 | +| 差距 | 小 | 大 | + +换句话说：**写 schema 的功夫、在小 model 上能省下换大 model 的成本**。Production 想用便宜 model（qwen / mistral）？schema 必须写到能上线跑的程度。 + +## 延伸阅读 + +更多 schema 设计规则对照 [`resources/schema-design-cheatsheet.md`](../../../resources/schema-design-cheatsheet.md)：清楚用途、正确类型、必填字段、enum 收敛、结构化错误回传。 + +## 延伸 + +- **故意改坏 good schema**：把一个 enum 拿掉、看 qwen 是否就开始挑错 +- **加第三个工具**：写一个跟 `convert_temperature` 用途相近但边界模糊的 tool、看 LLM 怎么挑 +- **接 [`../05-error-handling/`](../05-error-handling/) 的 structured error pattern**：结合 schema 设计 + 错误处理、production 级 diff --git a/examples/stage-3/06-schema-design/requirements.txt b/examples/stage-3/06-schema-design/requirements.txt new file mode 100644 index 0000000..db9d5be --- /dev/null +++ b/examples/stage-3/06-schema-design/requirements.txt @@ -0,0 +1,2 @@ +openai>=1.50,<2.0 +anthropic>=0.40,<1.0 # 只 starter_*_anthropic.py 需要、Ollama path 不用 diff --git a/examples/stage-3/06-schema-design/starter_bad.py b/examples/stage-3/06-schema-design/starter_bad.py new file mode 100644 index 0000000..c25d1b7 --- /dev/null +++ b/examples/stage-3/06-schema-design/starter_bad.py @@ -0,0 +1,93 @@ +"""練習 6：Schema 設計 — Bad schema（Path A、Ollama 默認）。 + +故意保留 anti-pattern：description 太模糊、參數都用 string、沒有 required、沒有 enum。 +小 model（qwen2.5:3b）對 schema 質量比大 model 更敏感——壞 schema 在 Claude haiku +上可能還能猜對、在 qwen2.5:3b 上幾乎必錯。對照 `starter_good.py`。 + +跑法： + pip install -r requirements.txt + ollama pull qwen2.5:3b + ollama serve + python starter_bad.py +""" + +from __future__ import annotations + +import json +import os +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +MODEL = os.environ.get("MODEL", "qwen2.5:3b") + + +def process_data(data: str = "") -> str: + return f"processed generic data: {data}" + + +def convert_temperature(value: str = "") -> str: + return f"converted something from: {value}" + + +# Anti-pattern: description 太短、params 都 string、無 required、無 enum +TOOLS_SPEC = [ + { + "type": "function", + "function": { + "name": "process_data", + "description": "Process data.", + "parameters": {"type": "object", "properties": {"data": {"type": "string"}}}, + }, + }, + { + "type": "function", + "function": { + "name": "convert_temperature", + "description": "Convert a value.", + "parameters": { + "type": "object", + "properties": {"value": {"type": "string"}, "unit": {"type": "string"}}, + }, + }, + }, +] + +TOOL_IMPL = { + "process_data": lambda i: process_data(i.get("data", "")), + "convert_temperature": lambda i: convert_temperature(i.get("value", "")), +} + + +def select_and_run(question: str, client: Any = None) -> dict: + client = client or OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + resp = client.chat.completions.create( + model=MODEL, + tools=TOOLS_SPEC, + messages=[{"role": "user", "content": question}], + ) + msg = resp.choices[0].message + tool_calls = msg.tool_calls or [] + if not tool_calls: + return {"tool": None, "tool_input": {}, "observation": None} + call = tool_calls[0] + args = json.loads(call.function.arguments) + return {"tool": call.function.name, "tool_input": args, "observation": TOOL_IMPL[call.function.name](args)} + + +if __name__ == "__main__": + question = "Convert 32 Celsius to Fahrenheit." + print(f"❓ 問題：{question}（using Ollama {MODEL}、BAD schema）") + result = select_and_run(question) + print(f" tool: {result['tool']}") + print(f" observation: {result['observation']}") + + # 寬鬆驗證：bad schema 不保證選對 tool、但至少要產出一個 tool call + assert result["tool"] is not None, "even bad schema should produce an observable selection" + if result["tool"] != "convert_temperature": + print("⚠ 小 model + 壞 schema → 預期會挑錯。對照 starter_good.py 看修正後的差異") + print("✅ Bad schema starter 跑通 — 對照 starter_good.py") diff --git a/examples/stage-3/06-schema-design/starter_bad_anthropic.py b/examples/stage-3/06-schema-design/starter_bad_anthropic.py new file mode 100644 index 0000000..68b28c7 --- /dev/null +++ b/examples/stage-3/06-schema-design/starter_bad_anthropic.py @@ -0,0 +1,73 @@ +"""練習 6：Schema 設計 — Bad schema（Path B、Anthropic Claude）。 + +故意保留 anti-pattern：description 太模糊、參數都用 string、沒有 required、沒有 enum。 +模型很容易把溫度轉換誤判給 `process_data`。對照 `starter_good_anthropic.py`。 + +跑法： + pip install -r requirements.txt + export ANTHROPIC_API_KEY=sk-ant-... + python starter_bad_anthropic.py + +預算：每次 ≈ $0.0005（claude-haiku-4-5、單輪 call）。 +Ollama 版本見 starter_bad.py。 +""" + +from __future__ import annotations + +import os +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic + +MODEL = os.environ.get("MODEL", "claude-haiku-4-5") + + +def process_data(data: str = "") -> str: + return f"processed generic data: {data}" + + +def convert_temperature(value: str = "") -> str: + return f"converted something from: {value}" + + +TOOLS_SPEC = [ + { + "name": "process_data", + "description": "Process data.", + "input_schema": {"type": "object", "properties": {"data": {"type": "string"}}}, + }, + { + "name": "convert_temperature", + "description": "Convert a value.", + "input_schema": {"type": "object", "properties": {"value": {"type": "string"}, "unit": {"type": "string"}}}, + }, +] + +TOOL_IMPL = { + "process_data": lambda i: process_data(i.get("data", "")), + "convert_temperature": lambda i: convert_temperature(i.get("value", "")), +} + + +def select_and_run(question: str, client: Any = None) -> dict: + client = client or anthropic.Anthropic() + resp = client.messages.create(model=MODEL, max_tokens=512, tools=TOOLS_SPEC, messages=[{"role": "user", "content": question}]) + calls = [b for b in resp.content if getattr(b, "type", None) == "tool_use"] + if not calls: + return {"tool": None, "tool_input": {}, "observation": None} + call = calls[0] + args = dict(call.input) + return {"tool": call.name, "tool_input": args, "observation": TOOL_IMPL[call.name](args)} + + +if __name__ == "__main__": + result = select_and_run("Convert 32 Celsius to Fahrenheit.") + print(result) + + # === 自我檢查 === + assert result["tool"] is not None, "even bad schema should produce an observable selection" + print("Stage 3 exercise 6 bad-schema starter check passed") diff --git a/examples/stage-3/06-schema-design/starter_good.py b/examples/stage-3/06-schema-design/starter_good.py new file mode 100644 index 0000000..f8fe803 --- /dev/null +++ b/examples/stage-3/06-schema-design/starter_good.py @@ -0,0 +1,109 @@ +"""練習 6：Schema 設計 — Good schema（Path A、Ollama 默認）。 + +清楚的工具用途、正確型別、必填欄位與 enum 收斂。小 model（qwen2.5:3b）也能穩定 +選到 `convert_temperature`。對照 `starter_bad.py`。 + +跑法： + pip install -r requirements.txt + ollama pull qwen2.5:3b + ollama serve + python starter_good.py +""" + +from __future__ import annotations + +import json +import os +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +MODEL = os.environ.get("MODEL", "qwen2.5:3b") + + +def process_data(data: list[dict], operation: str) -> dict: + if operation == "count_rows": + return {"rows": len(data)} + if operation == "list_columns": + return {"columns": sorted({key for row in data for key in row})} + return {"error": "unknown operation", "retry_hint": "use count_rows or list_columns"} + + +def convert_temperature(value: float, unit: str) -> dict: + if unit == "celsius": + return {"value": round(value * 9 / 5 + 32, 2), "unit": "fahrenheit"} + if unit == "fahrenheit": + return {"value": round((value - 32) * 5 / 9, 2), "unit": "celsius"} + return {"error": "unsupported unit", "retry_hint": "unit must be celsius or fahrenheit"} + + +# Good schema: 用途明確、正確型別、required、enum +TOOLS_SPEC = [ + { + "type": "function", + "function": { + "name": "process_data", + "description": "Use only to summarize structured JSON table rows. Do not use for temperature conversion.", + "parameters": { + "type": "object", + "properties": { + "data": {"type": "array", "items": {"type": "object"}, "description": "Rows to inspect"}, + "operation": {"type": "string", "enum": ["count_rows", "list_columns"]}, + }, + "required": ["data", "operation"], + }, + }, + }, + { + "type": "function", + "function": { + "name": "convert_temperature", + "description": "Use this when the user asks to convert temperatures between Fahrenheit and Celsius.", + "parameters": { + "type": "object", + "properties": { + "value": {"type": "number", "description": "Temperature value to convert"}, + "unit": {"type": "string", "enum": ["celsius", "fahrenheit"], "description": "Unit of the input value"}, + }, + "required": ["value", "unit"], + }, + }, + }, +] + +TOOL_IMPL = { + "process_data": lambda i: process_data(i["data"], i["operation"]), + "convert_temperature": lambda i: convert_temperature(i["value"], i["unit"]), +} + + +def select_and_run(question: str, client: Any = None) -> dict: + client = client or OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + resp = client.chat.completions.create( + model=MODEL, + tools=TOOLS_SPEC, + messages=[{"role": "user", "content": question}], + ) + msg = resp.choices[0].message + tool_calls = msg.tool_calls or [] + if not tool_calls: + return {"tool": None, "tool_input": {}, "observation": None} + call = tool_calls[0] + args = json.loads(call.function.arguments) + return {"tool": call.function.name, "tool_input": args, "observation": TOOL_IMPL[call.function.name](args)} + + +if __name__ == "__main__": + question = "Convert 32 Celsius to Fahrenheit." + print(f"❓ 問題：{question}（using Ollama {MODEL}、GOOD schema）") + result = select_and_run(question) + print(f" tool: {result['tool']}") + print(f" tool_input: {result.get('tool_input')}") + print(f" observation: {result['observation']}") + + assert result["tool"] == "convert_temperature", f"預期 convert_temperature、得到 {result['tool']}" + print("✅ Good schema starter 通過 — qwen2.5:3b 在清楚的 schema 上穩定挑對 tool、$0/run") diff --git a/examples/stage-3/06-schema-design/starter_good_anthropic.py b/examples/stage-3/06-schema-design/starter_good_anthropic.py new file mode 100644 index 0000000..c30d6f5 --- /dev/null +++ b/examples/stage-3/06-schema-design/starter_good_anthropic.py @@ -0,0 +1,90 @@ +"""練習 6：Schema 設計 — Good schema（Path B、Anthropic Claude）。 + +清楚的工具用途、正確型別、必填欄位與 enum 收斂。模型穩定選到 `convert_temperature`。 +對照 `starter_bad_anthropic.py`。 + +跑法： + pip install -r requirements.txt + export ANTHROPIC_API_KEY=sk-ant-... + python starter_good_anthropic.py + +預算：每次 ≈ $0.0005（claude-haiku-4-5、單輪 call）。 +Ollama 版本見 starter_good.py。 +""" + +from __future__ import annotations + +import os, sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic + +MODEL = os.environ.get("MODEL", "claude-haiku-4-5") + + +def process_data(data: list[dict], operation: str) -> dict: + if operation == "count_rows": + return {"rows": len(data)} + if operation == "list_columns": + return {"columns": sorted({key for row in data for key in row})} + return {"error": "unknown operation", "retry_hint": "use count_rows or list_columns"} + + +def convert_temperature(value: float, unit: str) -> dict: + if unit == "celsius": + return {"value": round(value * 9 / 5 + 32, 2), "unit": "fahrenheit"} + if unit == "fahrenheit": + return {"value": round((value - 32) * 5 / 9, 2), "unit": "celsius"} + return {"error": "unsupported unit", "retry_hint": "unit must be celsius or fahrenheit"} + + +TOOLS_SPEC = [ + { + "name": "process_data", + "description": "Use only to summarize structured JSON table rows. Do not use for temperature conversion.", + "input_schema": { + "type": "object", + "properties": { + "data": {"type": "array", "items": {"type": "object"}, "description": "Rows to inspect"}, + "operation": {"type": "string", "enum": ["count_rows", "list_columns"]}, + }, + "required": ["data", "operation"], + }, + }, + { + "name": "convert_temperature", + "description": "Use this when the user asks to convert temperatures between Fahrenheit and Celsius.", + "input_schema": { + "type": "object", + "properties": { + "value": {"type": "number", "description": "Temperature value to convert"}, + "unit": {"type": "string", "enum": ["celsius", "fahrenheit"], "description": "Unit of the input value"}, + }, + "required": ["value", "unit"], + }, + }, +] + +TOOL_IMPL = {"process_data": lambda i: process_data(i["data"], i["operation"]), "convert_temperature": lambda i: convert_temperature(i["value"], i["unit"])} + + +def select_and_run(question: str, client: Any = None) -> dict: + client = client or anthropic.Anthropic() + resp = client.messages.create(model=MODEL, max_tokens=512, tools=TOOLS_SPEC, messages=[{"role": "user", "content": question}]) + calls = [b for b in resp.content if getattr(b, "type", None) == "tool_use"] + if not calls: + return {"tool": None, "tool_input": {}, "observation": None} + call = calls[0] + args = dict(call.input) + return {"tool": call.name, "tool_input": args, "observation": TOOL_IMPL[call.name](args)} + + +if __name__ == "__main__": + result = select_and_run("Convert 32 Celsius to Fahrenheit.") + print(result) + # === 自我檢查 === + assert result["tool"] == "convert_temperature", f"expected convert_temperature, got {result['tool']}" + print("Stage 3 exercise 6 good-schema starter check passed") diff --git a/examples/stage-3/06-schema-design/test.py b/examples/stage-3/06-schema-design/test.py new file mode 100644 index 0000000..4aefed9 --- /dev/null +++ b/examples/stage-3/06-schema-design/test.py @@ -0,0 +1,80 @@ +"""練習 6 自我驗證 — Path A（Ollama starter_bad.py + starter_good.py）。 + +跑法： + python test.py + +驗證內容： + - bad schema 在 mock 下可能挑錯 tool（process_data） + - good schema 穩定挑到 convert_temperature + - good schema 有 required + enum、bad schema 沒有 + +Anthropic 版本 test 見 test_anthropic.py。 +""" + +from __future__ import annotations + +import json +import sys +from types import SimpleNamespace +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import starter_bad as bad +import starter_good as good + + +def make_tool_call(call_id: str, name: str, args: dict): + return SimpleNamespace( + id=call_id, + type="function", + function=SimpleNamespace(name=name, arguments=json.dumps(args)), + ) + + +def make_resp(content: str, tool_calls=None, finish_reason: str = "tool_calls"): + msg = SimpleNamespace(content=content, tool_calls=tool_calls) + return SimpleNamespace(choices=[SimpleNamespace(finish_reason=finish_reason, message=msg)]) + + +def test_bad_schema_can_select_wrong_tool(): + """模糊 schema 下、LLM 把溫度轉換誤丟給 process_data。""" + client = MagicMock() + client.chat.completions.create.return_value = make_resp( + "The schemas are vague, so I will process the text.", + [make_tool_call("t1", "process_data", {"data": "32 Celsius to Fahrenheit"})], + ) + result = bad.select_and_run("Convert 32 Celsius to Fahrenheit.", client=client) + assert result["tool"] == "process_data" + assert "processed generic data" in result["observation"] + print("✅ test_bad_schema_can_select_wrong_tool") + + +def test_good_schema_selects_temperature_tool(): + client = MagicMock() + client.chat.completions.create.return_value = make_resp( + "This is clearly a temperature conversion.", + [make_tool_call("t1", "convert_temperature", {"value": 32, "unit": "celsius"})], + ) + result = good.select_and_run("Convert 32 Celsius to Fahrenheit.", client=client) + assert result["tool"] == "convert_temperature" + assert result["observation"] == {"value": 89.6, "unit": "fahrenheit"} + print("✅ test_good_schema_selects_temperature_tool") + + +def test_good_schema_has_required_fields_and_enum(): + """直接檢查 schema 結構：good 有 required + enum、bad 沒有。""" + bad_temp = next(t for t in bad.TOOLS_SPEC if t["function"]["name"] == "convert_temperature") + good_temp = next(t for t in good.TOOLS_SPEC if t["function"]["name"] == "convert_temperature") + assert "required" not in bad_temp["function"]["parameters"] + assert good_temp["function"]["parameters"]["required"] == ["value", "unit"] + assert good_temp["function"]["parameters"]["properties"]["unit"]["enum"] == ["celsius", "fahrenheit"] + print("✅ test_good_schema_has_required_fields_and_enum") + + +if __name__ == "__main__": + test_bad_schema_can_select_wrong_tool() + test_good_schema_selects_temperature_tool() + test_good_schema_has_required_fields_and_enum() + print("\n🎉 全部通過 — Ollama path bad / good schema 對照邏輯正確") diff --git a/examples/stage-3/06-schema-design/test_anthropic.py b/examples/stage-3/06-schema-design/test_anthropic.py new file mode 100644 index 0000000..a1cdafb --- /dev/null +++ b/examples/stage-3/06-schema-design/test_anthropic.py @@ -0,0 +1,71 @@ +"""Stage 3 練習 6 自我驗證 — Path B（Anthropic starter_*_anthropic.py）。 + +跑法： + python test_anthropic.py + +用 mock 取代 Anthropic client、不打真 API、$0/run。 +Ollama 版本見 test.py（OpenAI-compat shape）。 +""" + +from __future__ import annotations + +import sys +from types import SimpleNamespace +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import starter_bad_anthropic as bad +import starter_good_anthropic as good + + +def block_text(text: str): + return SimpleNamespace(type="text", text=text) + + +def block_tool_use(tool_id: str, name: str, inp: dict): + return SimpleNamespace(type="tool_use", id=tool_id, name=name, input=inp) + + +def make_resp(stop_reason: str, *blocks): + return SimpleNamespace(stop_reason=stop_reason, content=list(blocks)) + + +def test_bad_schema_can_select_wrong_tool(): + client = MagicMock() + client.messages.create.return_value = make_resp( + "tool_use", + block_text("The schemas are vague, so I will process the text."), + block_tool_use("t1", "process_data", {"data": "32 Celsius to Fahrenheit"}), + ) + result = bad.select_and_run("Convert 32 Celsius to Fahrenheit.", client=client) + assert result["tool"] == "process_data" + assert "processed generic data" in result["observation"] + + +def test_good_schema_selects_temperature_tool(): + client = MagicMock() + client.messages.create.return_value = make_resp( + "tool_use", + block_text("This is clearly a temperature conversion."), + block_tool_use("t1", "convert_temperature", {"value": 32, "unit": "celsius"}), + ) + result = good.select_and_run("Convert 32 Celsius to Fahrenheit.", client=client) + assert result["tool"] == "convert_temperature" + assert result["observation"] == {"value": 89.6, "unit": "fahrenheit"} + + +def test_good_schema_has_required_fields_and_enum(): + bad_temp = next(tool for tool in bad.TOOLS_SPEC if tool["name"] == "convert_temperature") + good_temp = next(tool for tool in good.TOOLS_SPEC if tool["name"] == "convert_temperature") + assert "required" not in bad_temp["input_schema"] + assert good_temp["input_schema"]["required"] == ["value", "unit"] + assert good_temp["input_schema"]["properties"]["unit"]["enum"] == ["celsius", "fahrenheit"] + + +if __name__ == "__main__": + test_bad_schema_can_select_wrong_tool() + test_good_schema_selects_temperature_tool() + test_good_schema_has_required_fields_and_enum() + print("all pass") diff --git a/examples/stage-4/01-same-agent-two-frameworks/README.en.md b/examples/stage-4/01-same-agent-two-frameworks/README.en.md new file mode 100644 index 0000000..10f221e --- /dev/null +++ b/examples/stage-4/01-same-agent-two-frameworks/README.en.md @@ -0,0 +1,111 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# Exercise 1: Same Agent, Two Frameworks (LangGraph + CrewAI) + +Pairs with [Stage 4 — Agent Frameworks](../../../stages/04-agent-frameworks.en.md) Exercise 1. + +## Task + +A minimal search + summarize agent: + +- Given a query (e.g. "summarize Taipei") +- Agent uses a `search` tool to hit a knowledge base +- LLM summarizes the result in 1-2 sentences + +Built once in **LangGraph** and once in **CrewAI** — compare styles. + +## How to run — two paths + two frameworks + +### Path A (default, free, local) + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve + +python starter.py # LangGraph + Ollama +python starter_crewai.py # CrewAI + Ollama (comparison) +``` + +Budget: **$0**. + +### Path B (Anthropic, cloud-quality) + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py # LangGraph + Claude +``` + +Budget: ~**$0.001** per run (claude-haiku-4-5). + +## Validate the logic (mock-based) + +```bash +python test.py # LangGraph + mock LLM +python test_anthropic.py # starter_anthropic loads + ChatAnthropic constructs +python test_crewai.py # CrewAI tool + module loads +``` + +## Side-by-side framework comparison + +| Dimension | LangGraph | CrewAI | +|---|---|---| +| Core abstraction | `StateGraph` + node + edge | `Agent` + `Task` + `Crew` | +| Mental model | "How does state flow?" | "Who plays what role?" | +| Loop control | Explicit conditional edges | Hidden inside `Crew.kickoff()` | +| Lines of code (this task) | ~50 | ~25 | +| Debug path | Inspect graph state, time-travel | Verbose logs, hard to step | +| Best for | Complex branching, production, audit | Multi-agent prototypes, role-based tasks | +| Learning curve | Medium-high | Low | + +### LangGraph style (condensed) + +```python +g = StateGraph(State) +g.add_node("agent", agent_node) +g.add_node("tools", tool_node) +g.add_conditional_edges("agent", should_continue, {"tools": "tools", END: END}) +g.add_edge("tools", "agent") +``` + +"I tell the system explicitly: state shape, nodes, edges, branching via `should_continue`." + +### CrewAI style (condensed) + +```python +researcher = Agent(role="Researcher", goal="...", tools=[search], llm=MODEL) +task = Task(description=query, expected_output="...", agent=researcher) +crew = Crew(agents=[researcher], tasks=[task]) +crew.kickoff() +``` + +"I describe: who plays this role, what task, what tools. Framework decides how to run." + +## What to observe + +1. **Abstraction cost**: CrewAI hides more, writes less code; but stack depth grows when debugging +2. **Small-model friendliness**: LangGraph is more stable with qwen2.5:3b; CrewAI's denser prompts can confuse small models +3. **Controllability**: LangGraph exposes state transitions; CrewAI is "result-oriented" +4. **When to pick**: production / audit → LangGraph. Multi-agent prototypes / role-based → CrewAI + +## Common pitfalls + +- **LangGraph `bind_tools`**: must `llm.bind_tools([search])` to expose tool schema. Without it the model doesn't know the tool exists +- **CrewAI model spec**: needs LiteLLM format (`"ollama/qwen2.5:3b"`, not `"qwen2.5:3b"`). Misspell and framework silently falls back to OpenAI default +- **CrewAI return type**: `crew.kickoff()` returns a `CrewOutput` object; `str(result)` to get text. Bare `print(result)` may show repr + +## Want smarter answers? + +```bash +MODEL=claude-sonnet-5 python starter_anthropic.py # more stable +MODEL=qwen2.5:7b python starter.py # larger local model +``` + +## Extensions + +- **Streaming**: LangGraph `graph.stream(...)`, CrewAI `crew.kickoff(stream=True)` +- **Checkpointing**: LangGraph + `MemorySaver` for time-travel debug +- **Human-in-the-loop**: see Exercise 3 diff --git a/examples/stage-4/01-same-agent-two-frameworks/README.md b/examples/stage-4/01-same-agent-two-frameworks/README.md new file mode 100644 index 0000000..818b907 --- /dev/null +++ b/examples/stage-4/01-same-agent-two-frameworks/README.md @@ -0,0 +1,118 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 練習 1：同一個 agent、兩個 framework（LangGraph + CrewAI） + +對應 [Stage 4 — Agent Frameworks](../../../stages/04-agent-frameworks.md) 練習 1。 +> 🎓 **學習模式**：這份 `starter.py` 是**完整解答**、不是 TODO skeleton。建議用**主動模式**——`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫一份 `starter.py`、跑 `python test.py` 驗證；卡 20 分鐘再回去對照 reference。完整方法論看 [`docs/HOW_TO_USE.md`](../../../docs/HOW_TO_USE.md)。 + +> 📚 **想要 chapter-length 深入版？** 本 folder 的 starter 是 illustrative 版、聚焦核心 pattern + 兩條 SDK path，不是進階深度教材。深度教材推薦： +> - [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents) ⭐ 中文圈最完整、章節式 + 16 種 production 能力。**本練習對應 hello-agents 的 framework 對照 / orchestration 章節** +> - [LangGraph official tutorial](https://langchain-ai.github.io/langgraph/tutorials/) + [CrewAI 官方 docs](https://docs.crewai.com/) +> - 完整 references 見 [Stage 4 精選 Projects](../../../stages/04-agent-frameworks.md#-精選-projects) + + +## 任務 + +最簡單的 search + summarize agent： + +- 給一個 query（譬如「summarize Taipei」） +- Agent 用 `search` tool 拿 knowledge base 資料 +- LLM 把 search result 摘成 1-2 句 + +用 **LangGraph** 跟 **CrewAI** 各做一次、比較風格差異。 + +## 怎麼跑 — 兩條路徑 + 兩個 framework + +### Path A（默認、本機免費） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve + +python starter.py # LangGraph + Ollama +python starter_crewai.py # CrewAI + Ollama（對照） +``` + +預算：**$0**。 + +### Path B（Anthropic、想看 cloud 高品質） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py # LangGraph + Claude +``` + +預算：每次 ≈ **$0.001**（claude-haiku-4-5）。 + +## 不花錢驗證程式邏輯（mock-based） + +```bash +python test.py # LangGraph + mock LLM +python test_anthropic.py # starter_anthropic 可載入 + ChatAnthropic 可建構 +python test_crewai.py # CrewAI tool 邏輯 + 模組可載入 +``` + +## 兩個 framework 的並排比較 + +| 維度 | LangGraph | CrewAI | +|---|---|---| +| 核心抽象 | `StateGraph` + node + edge | `Agent` + `Task` + `Crew` | +| 思考方式 | 「狀態怎麼流動」 | 「角色怎麼分工」 | +| Loop 控制 | 顯式 conditional edge | 隱藏在 `Crew.kickoff()` 裡 | +| 程式碼行數（這題） | ~50 行 | ~25 行 | +| Debug 路徑 | 看 graph state、可 time-travel | 看 verbose log、不容易 step | +| 適合場景 | 複雜分支、production、需要 audit | 多 agent 雛形、role-based 任務 | +| 學習曲線 | 中-高 | 低 | + +### LangGraph 風格（精簡） + +```python +g = StateGraph(State) +g.add_node("agent", agent_node) +g.add_node("tools", tool_node) +g.add_conditional_edges("agent", should_continue, {"tools": "tools", END: END}) +g.add_edge("tools", "agent") +``` + +「我要顯式地告訴系統：狀態長這樣、節點互相連這樣、條件分支看 `should_continue`。」 + +### CrewAI 風格（精簡） + +```python +researcher = Agent(role="Researcher", goal="...", tools=[search], llm=MODEL) +task = Task(description=query, expected_output="...", agent=researcher) +crew = Crew(agents=[researcher], tasks=[task]) +crew.kickoff() +``` + +「我要描述：這個角色是誰、要完成什麼任務、有什麼工具。框架自己決定怎麼跑。」 + +## 觀察重點 + +1. **抽象代價**：CrewAI 隱藏的多、寫得少；要 debug 時 stack 比較深 +2. **小 model 友善度**：LangGraph 對 qwen2.5:3b 較穩；CrewAI 可能讓小 model 多繞幾步（因為 prompt 比較複雜） +3. **可控性**：LangGraph 你能看到每個 state 變化；CrewAI 偏向「結果導向」 +4. **何時選哪個**：production 級 / 需要 audit → LangGraph。多 agent 雛形 / role-based → CrewAI + +## 常見坑 + +- **LangGraph `bind_tools`**：要 `llm.bind_tools([search])` 才會把 tool schema 給 LLM。沒 bind 模型就不知道 tool 存在 +- **CrewAI LLM 設定**：要靠 LiteLLM 格式（譬如 `"ollama/qwen2.5:3b"`、不是 `"qwen2.5:3b"`）。錯一個字 framework 不會 raise、會直接連到 OpenAI 預設 +- **CrewAI 結果型別**：`crew.kickoff()` 回 `CrewOutput` 物件、`str(result)` 拿文字。直接 `print(result)` 有可能拿到 repr + +## 想看更聰明的答案？ + +```bash +MODEL=claude-sonnet-5 python starter_anthropic.py # 更穩 +MODEL=qwen2.5:7b python starter.py # 大本機 model +``` + +## 延伸 + +- **改成 streaming**：LangGraph `graph.stream(...)` 邊跑邊看 state、CrewAI `crew.kickoff(stream=True)` +- **加 checkpointing**：LangGraph 加 `MemorySaver` 就能 time-travel debug +- **加 human-in-the-loop**：練習 3 會做 diff --git a/examples/stage-4/01-same-agent-two-frameworks/README.zh-Hans.md b/examples/stage-4/01-same-agent-two-frameworks/README.zh-Hans.md new file mode 100644 index 0000000..561f82e --- /dev/null +++ b/examples/stage-4/01-same-agent-two-frameworks/README.zh-Hans.md @@ -0,0 +1,111 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 练习 1：同一个 agent、两个 framework（LangGraph + CrewAI） + +对应 [Stage 4 — Agent Frameworks](../../../stages/04-agent-frameworks.zh-Hans.md) 练习 1。 + +## 任务 + +最简单的 search + summarize agent： + +- 给一个 query（譬如“summarize Taipei”） +- Agent 用 `search` tool 拿 knowledge base 数据 +- LLM 把 search result 摘成 1-2 句 + +用 **LangGraph** 跟 **CrewAI** 各做一次、比较风格差异。 + +## 怎么跑 — 两条路径 + 两个 framework + +### Path A（默认、本机免费） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve + +python starter.py # LangGraph + Ollama +python starter_crewai.py # CrewAI + Ollama（对照） +``` + +预算：**$0**。 + +### Path B（Anthropic、想看 cloud 高质量） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py # LangGraph + Claude +``` + +预算：每次 ≈ **$0.001**（claude-haiku-4-5）。 + +## 不花钱验证程序逻辑（mock-based） + +```bash +python test.py # LangGraph + mock LLM +python test_anthropic.py # starter_anthropic 可载入 + ChatAnthropic 可构造 +python test_crewai.py # CrewAI tool 逻辑 + 模块可载入 +``` + +## 两个 framework 的并排比较 + +| 维度 | LangGraph | CrewAI | +|---|---|---| +| 核心抽象 | `StateGraph` + node + edge | `Agent` + `Task` + `Crew` | +| 思考方式 | “状态怎么流动” | “角色怎么分工” | +| Loop 控制 | 显式 conditional edge | 隐藏在 `Crew.kickoff()` 里 | +| 程序码行数（这题） | ~50 行 | ~25 行 | +| Debug 路径 | 看 graph state、可 time-travel | 看 verbose log、不容易 step | +| 适合场景 | 复杂分支、production、需要 audit | 多 agent 雏形、role-based 任务 | +| 学习曲线 | 中-高 | 低 | + +### LangGraph 风格（精简） + +```python +g = StateGraph(State) +g.add_node("agent", agent_node) +g.add_node("tools", tool_node) +g.add_conditional_edges("agent", should_continue, {"tools": "tools", END: END}) +g.add_edge("tools", "agent") +``` + +“我要显式地告诉系统：状态长这样、节点互相连这样、条件分支看 `should_continue`。” + +### CrewAI 风格（精简） + +```python +researcher = Agent(role="Researcher", goal="...", tools=[search], llm=MODEL) +task = Task(description=query, expected_output="...", agent=researcher) +crew = Crew(agents=[researcher], tasks=[task]) +crew.kickoff() +``` + +“我要描述：这个角色是谁、要完成什么任务、有什么工具。框架自己决定怎么跑。” + +## 观察重点 + +1. **抽象代价**：CrewAI 隐藏的多、写得少；要 debug 时 stack 比较深 +2. **小 model 友善度**：LangGraph 对 qwen2.5:3b 较稳；CrewAI 可能让小 model 多绕几步（因为 prompt 比较复杂） +3. **可控性**：LangGraph 你能看到每个 state 变化；CrewAI 偏向“结果导向” +4. **何时选哪个**：production 级 / 需要 audit → LangGraph。多 agent 雏形 / role-based → CrewAI + +## 常见坑 + +- **LangGraph `bind_tools`**：要 `llm.bind_tools([search])` 才会把 tool schema 给 LLM。没 bind 模型就不知道 tool 存在 +- **CrewAI LLM 设定**：要靠 LiteLLM 格式（譬如 `"ollama/qwen2.5:3b"`、不是 `"qwen2.5:3b"`）。错一个字 framework 不会 raise、会直接连到 OpenAI 预设 +- **CrewAI 结果类型**：`crew.kickoff()` 回 `CrewOutput` 对象、`str(result)` 拿文字。直接 `print(result)` 有可能拿到 repr + +## 想看更聪明的答案？ + +```bash +MODEL=claude-sonnet-5 python starter_anthropic.py # 更稳 +MODEL=qwen2.5:7b python starter.py # 大本机 model +``` + +## 延伸 + +- **改成 streaming**：LangGraph `graph.stream(...)` 边跑边看 state、CrewAI `crew.kickoff(stream=True)` +- **加 checkpointing**：LangGraph 加 `MemorySaver` 就能 time-travel debug +- **加 human-in-the-loop**：练习 3 会做 diff --git a/examples/stage-4/01-same-agent-two-frameworks/requirements.txt b/examples/stage-4/01-same-agent-two-frameworks/requirements.txt new file mode 100644 index 0000000..2bc8cca --- /dev/null +++ b/examples/stage-4/01-same-agent-two-frameworks/requirements.txt @@ -0,0 +1,5 @@ +langgraph>=0.2,<1.0 +langchain-openai>=0.2,<1.0 +langchain-anthropic>=0.2,<1.0 +langchain-core>=0.3,<1.0 +crewai>=0.80,<1.0 diff --git a/examples/stage-4/01-same-agent-two-frameworks/starter.py b/examples/stage-4/01-same-agent-two-frameworks/starter.py new file mode 100644 index 0000000..3e8ade7 --- /dev/null +++ b/examples/stage-4/01-same-agent-two-frameworks/starter.py @@ -0,0 +1,104 @@ +"""Stage 4 練習 1：同一個 agent、兩個 framework — LangGraph + Ollama（Path A、默認）。 + +任務：搜尋 + 摘要的小 agent。給一個 query、agent 用 search tool 拿資料、回一段摘要。 +這份用 LangGraph。對照的 CrewAI 版本見 starter_crewai.py。 + +跑法： + pip install -r requirements.txt + ollama pull qwen2.5:3b + ollama serve + python starter.py + +驗證： + python test.py + +預算：$0/run。對照 Anthropic 版見 starter_anthropic.py。 +""" + +from __future__ import annotations + +import os +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from langchain_core.messages import HumanMessage, ToolMessage +from langchain_core.tools import tool +from langchain_openai import ChatOpenAI +from langgraph.graph import END, StateGraph +from langgraph.graph.message import add_messages +from typing_extensions import Annotated, TypedDict + +MODEL = os.environ.get("MODEL", "qwen2.5:3b") + + +# === Tool === + +@tool +def search(query: str) -> str: + """Search a (fake, offline) knowledge base for a topic.""" + db = { + "taipei": "Taipei is the capital of Taiwan, population ~2.6M, known for night markets.", + "react agent": "ReAct (Reasoning + Acting) is an agent pattern: think → act → observe loop.", + } + return db.get(query.strip().lower(), f"no entry for {query}") + + +# === LangGraph 狀態定義 === + +class State(TypedDict): + messages: Annotated[list, add_messages] + + +def build_graph(llm: Any) -> Any: + """組一個 search → summarize 的圖：LLM 看 query 決定要不要呼叫 search、看到 tool result 後給摘要。""" + llm_with_tools = llm.bind_tools([search]) + + def agent_node(state: State): + return {"messages": [llm_with_tools.invoke(state["messages"])]} + + def tool_node(state: State): + msg = state["messages"][-1] + results = [] + for call in msg.tool_calls: + obs = search.invoke(call["args"]) + results.append(ToolMessage(content=obs, tool_call_id=call["id"])) + return {"messages": results} + + def should_continue(state: State) -> str: + return "tools" if state["messages"][-1].tool_calls else END + + g = StateGraph(State) + g.add_node("agent", agent_node) + g.add_node("tools", tool_node) + g.set_entry_point("agent") + g.add_conditional_edges("agent", should_continue, {"tools": "tools", END: END}) + g.add_edge("tools", "agent") + return g.compile() + + +def run(query: str, llm: Any = None) -> dict: + """Public entrypoint: 跑一輪 agent、回傳 final message + steps。""" + llm = llm or ChatOpenAI( + base_url="http://localhost:11434/v1", api_key="ollama", + model=MODEL, temperature=0, + ) + graph = build_graph(llm) + final_state = graph.invoke({"messages": [HumanMessage(content=query)]}) + return { + "final": final_state["messages"][-1].content, + "steps": len(final_state["messages"]), + } + + +if __name__ == "__main__": + query = "summarize what you know about Taipei" + print(f"❓ Query: {query}（using Ollama {MODEL}）") + print("-" * 60) + result = run(query) + print(f"✅ Final: {result['final']}") + print(f" Steps: {result['steps']}") + assert result["final"], "expected non-empty summary" + print("✅ 練習 1 通過 — LangGraph + Ollama qwen2.5:3b、$0/run") diff --git a/examples/stage-4/01-same-agent-two-frameworks/starter_anthropic.py b/examples/stage-4/01-same-agent-two-frameworks/starter_anthropic.py new file mode 100644 index 0000000..4b664a0 --- /dev/null +++ b/examples/stage-4/01-same-agent-two-frameworks/starter_anthropic.py @@ -0,0 +1,38 @@ +"""Stage 4 練習 1：同一個 agent、兩個 framework — LangGraph + Anthropic（Path B）。 + +跟 starter.py 同樣的 LangGraph 圖、只把 LLM backend 換成 Claude。 + +跑法： + pip install -r requirements.txt + export ANTHROPIC_API_KEY=sk-ant-... + python starter_anthropic.py + +預算：每次 ≈ $0.001（claude-haiku-4-5）。 +Ollama 版本見 starter.py。 +""" + +from __future__ import annotations + +import os +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from langchain_anthropic import ChatAnthropic + +from starter import run + +MODEL = os.environ.get("MODEL", "claude-haiku-4-5") + + +if __name__ == "__main__": + query = "summarize what you know about Taipei" + print(f"❓ Query: {query}（using Anthropic {MODEL}）") + print("-" * 60) + llm = ChatAnthropic(model=MODEL, temperature=0) + result = run(query, llm=llm) + print(f"✅ Final: {result['final']}") + print(f" Steps: {result['steps']}") + assert result["final"], "expected non-empty summary" + print("✅ 練習 1 (Anthropic path) 通過 — LangGraph + claude-haiku-4-5、≈$0.001/run") diff --git a/examples/stage-4/01-same-agent-two-frameworks/starter_crewai.py b/examples/stage-4/01-same-agent-two-frameworks/starter_crewai.py new file mode 100644 index 0000000..2592153 --- /dev/null +++ b/examples/stage-4/01-same-agent-two-frameworks/starter_crewai.py @@ -0,0 +1,77 @@ +"""Stage 4 練習 1：同一個 agent、CrewAI 版本對照（Ollama 預設）。 + +跟 starter.py (LangGraph) 同一個任務：search + summarize。 +看程式碼風格差異——CrewAI 用 Agent + Task 抽象、LangGraph 用 StateGraph + node。 + +跑法： + pip install -r requirements.txt # 含 crewai + ollama pull qwen2.5:3b + ollama serve + python starter_crewai.py + +預算：$0/run。要切 Anthropic 改 LLM_PROVIDER env var（CrewAI 用 LiteLLM 底層）。 + +⚠️ 注意：CrewAI 對小 model（qwen2.5:3b）較吃力，可能會多繞幾步。Claude / sonnet 較穩。 +""" + +from __future__ import annotations + +import os +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from crewai import Agent, Crew, Task +from crewai.tools import tool + +MODEL = os.environ.get("MODEL", "ollama/qwen2.5:3b") # LiteLLM 格式 +OLLAMA_BASE = os.environ.get("OLLAMA_API_BASE", "http://localhost:11434") + + +@tool("search") +def search(query: str) -> str: + """Search a (fake, offline) knowledge base for a topic.""" + db = { + "taipei": "Taipei is the capital of Taiwan, population ~2.6M, known for night markets.", + "react agent": "ReAct (Reasoning + Acting) is an agent pattern: think → act → observe loop.", + } + return db.get(query.strip().lower(), f"no entry for {query}") + + +def build_crew(query: str) -> Crew: + """CrewAI 風格：一個 agent + 一個 task。""" + os.environ["OPENAI_API_BASE"] = f"{OLLAMA_BASE}/v1" + os.environ["OPENAI_API_KEY"] = "ollama" + + researcher = Agent( + role="Researcher", + goal="Find and summarize the requested topic.", + backstory="You search a knowledge base and give concise summaries.", + tools=[search], + llm=MODEL, + verbose=False, + ) + task = Task( + description=query, + expected_output="A 1-2 sentence summary based on search results.", + agent=researcher, + ) + return Crew(agents=[researcher], tasks=[task], verbose=False) + + +def run(query: str) -> dict: + crew = build_crew(query) + result = crew.kickoff() + return {"final": str(result), "steps": None} + + +if __name__ == "__main__": + query = "summarize what you know about Taipei" + print(f"❓ Query: {query}（using CrewAI + Ollama {MODEL}）") + print("-" * 60) + result = run(query) + print(f"✅ Final: {result['final']}") + assert result["final"], "expected non-empty summary" + print("✅ CrewAI 版本通過 — 同樣任務、不同 framework、$0/run") + print(" 對照 starter.py（LangGraph）看程式碼風格差異") diff --git a/examples/stage-4/01-same-agent-two-frameworks/test.py b/examples/stage-4/01-same-agent-two-frameworks/test.py new file mode 100644 index 0000000..9cb5053 --- /dev/null +++ b/examples/stage-4/01-same-agent-two-frameworks/test.py @@ -0,0 +1,64 @@ +"""Stage 4 練習 1 自我驗證 — Path A（Ollama starter.py，LangGraph 版）。 + +跑法： + python test.py + +用 mock LLM 取代真實 API、不打網路。 +""" + +from __future__ import annotations + +import sys +from types import SimpleNamespace +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from langchain_core.messages import AIMessage + +from starter import build_graph, run, search + + +def test_search_tool_basic(): + assert "Taipei" in search.invoke({"query": "Taipei"}) + assert "no entry" in search.invoke({"query": "unknown_topic"}) + print("✅ test_search_tool_basic") + + +def test_run_with_mock_llm(): + """Mock LangChain LLM — agent 第一輪叫 search、第二輪收尾。""" + llm = MagicMock() + # 第一輪：tool_calls (search Taipei) + tool_call_msg = AIMessage( + content="", + tool_calls=[{"name": "search", "args": {"query": "Taipei"}, "id": "call_1", "type": "tool_call"}], + ) + # 第二輪：純文字 summary + final_msg = AIMessage(content="Taipei is the capital of Taiwan, population ~2.6M.") + llm.bind_tools.return_value = llm # 鏈式：bind_tools 回自己 + llm.invoke.side_effect = [tool_call_msg, final_msg] + + result = run("summarize Taipei", llm=llm) + assert "Taipei" in result["final"] + assert result["steps"] >= 3 # user + tool_call + tool_result + final + print("✅ test_run_with_mock_llm") + + +def test_run_skip_tool_when_known(): + """Mock：LLM 直接給答案、不呼叫 tool。""" + llm = MagicMock() + final_msg = AIMessage(content="Python is a programming language.", tool_calls=[]) + llm.bind_tools.return_value = llm + llm.invoke.return_value = final_msg + + result = run("what is Python", llm=llm) + assert "Python" in result["final"] + print("✅ test_run_skip_tool_when_known") + + +if __name__ == "__main__": + test_search_tool_basic() + test_run_with_mock_llm() + test_run_skip_tool_when_known() + print("\n🎉 全部通過 — LangGraph + Ollama 邏輯正確") diff --git a/examples/stage-4/01-same-agent-two-frameworks/test_anthropic.py b/examples/stage-4/01-same-agent-two-frameworks/test_anthropic.py new file mode 100644 index 0000000..1aeef2d --- /dev/null +++ b/examples/stage-4/01-same-agent-two-frameworks/test_anthropic.py @@ -0,0 +1,32 @@ +"""Stage 4 練習 1 自我驗證 — Path B（Anthropic starter_anthropic.py）。 + +starter_anthropic.py 只是換 LLM backend、不改 build_graph 邏輯，所以共用 test.py 的測試。 +這份只多驗：ChatAnthropic 可以被 build_graph 接受（不打 API、純 import 檢查）。 +""" + +from __future__ import annotations + +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + + +def test_anthropic_import_ok(): + """ChatAnthropic import 通過、無需 API key（建構式不打 API）。""" + from langchain_anthropic import ChatAnthropic + assert ChatAnthropic is not None + print("✅ test_anthropic_import_ok") + + +def test_starter_anthropic_module_ok(): + """starter_anthropic 模組可被 import、function 都存在。""" + import starter_anthropic + assert hasattr(starter_anthropic, "MODEL") + print("✅ test_starter_anthropic_module_ok") + + +if __name__ == "__main__": + test_anthropic_import_ok() + test_starter_anthropic_module_ok() + print("\n🎉 通過 — starter_anthropic.py 可載入（實際跑需要 ANTHROPIC_API_KEY）") diff --git a/examples/stage-4/01-same-agent-two-frameworks/test_crewai.py b/examples/stage-4/01-same-agent-two-frameworks/test_crewai.py new file mode 100644 index 0000000..2f95899 --- /dev/null +++ b/examples/stage-4/01-same-agent-two-frameworks/test_crewai.py @@ -0,0 +1,37 @@ +"""Stage 4 練習 1 CrewAI 對照版自我驗證。 + +只驗 import + tool function 邏輯。CrewAI 整個 Crew.kickoff() 太黑盒、不適合純 mock； +要實測請跑 starter_crewai.py 配 Ollama。 +""" + +from __future__ import annotations + +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter_crewai import search + + +def test_search_tool_basic(): + """CrewAI tool 包裝後仍可被 .run / .func 直接呼叫做單元測。""" + fn = getattr(search, "func", None) or getattr(search, "run", None) + if fn is None: + print("⚠ CrewAI tool 介面變動、跳過 unit test") + return + assert "Taipei" in fn("Taipei") + print("✅ test_search_tool_basic (CrewAI tool wrapper)") + + +def test_crewai_module_ok(): + import starter_crewai + assert hasattr(starter_crewai, "build_crew") + assert hasattr(starter_crewai, "MODEL") + print("✅ test_crewai_module_ok") + + +if __name__ == "__main__": + test_search_tool_basic() + test_crewai_module_ok() + print("\n🎉 通過 — CrewAI 對照版可載入（實測需要 Ollama 跑著）") diff --git a/examples/stage-4/02-multi-agent-roles/README.en.md b/examples/stage-4/02-multi-agent-roles/README.en.md new file mode 100644 index 0000000..871cb11 --- /dev/null +++ b/examples/stage-4/02-multi-agent-roles/README.en.md @@ -0,0 +1,129 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# Exercise 2: Multi-Agent Role Allocation (CrewAI) + +Pairs with [Stage 4 — Agent Frameworks](../../../stages/04-agent-frameworks.en.md) Exercise 2. + +## Task + +Three agents each own one step, collaborating to produce a blog intro: + +``` +Researcher → Writer → Critic + (find facts) (write) (verify, PASS/ISSUES) +``` + +Role-based pipelines are **CrewAI's sweet spot** — describe roles / goals / tasks, framework orchestrates. + +## How to run — two paths + +### Path A (default, free, local) + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +Budget: **$0**. Three agents sequential ≈ 30-90s on CPU with qwen2.5:3b. + +### Path B (Anthropic, cloud-quality) + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +Budget: ~**$0.005-0.01** per run (3 agents × short outputs, claude-haiku-4-5). + +## Validate the logic + +```bash +python test.py # tool logic + crew structure +python test_anthropic.py # starter_anthropic loads +``` + +CrewAI's `kickoff()` is too opaque for pure mock testing. These tests cover structure (3 agents + 3 tasks + sequential process + context dependencies) and tool logic. For full validation, run starter.py against Ollama. + +## Core CrewAI multi-agent concepts + +### Agent + +```python +researcher = Agent( + role="Researcher", + goal="...", # one line: what does "success" look like + backstory="...", # persona context, shapes the prompt + tools=[search], + llm=MODEL, +) +``` + +**Key**: `role` and `goal` dramatically affect prompt quality. Don't write "Agent" — write "Researcher who finds factual data". + +### Task + +```python +research_task = Task( + description="Search for X and report findings.", + expected_output="A 1-2 sentence factual entry.", + agent=researcher, +) +``` + +**Key**: `expected_output` is the "passing template" the LLM sees — be specific. "A 2-sentence intro paragraph" is 10× better than "Some text". + +### Context dependency + +```python +write_task = Task(..., context=[research_task]) # writer sees researcher's output +critic_task = Task(..., context=[research_task, write_task]) # critic sees both +``` + +**Key**: `context` is CrewAI's dataflow mechanism. `critic_task.context=[a, b]` means the critic sees the output of tasks a and b. + +### Sequential vs hierarchical process + +```python +Crew(..., process=Process.sequential) # linear walk-through +Crew(..., process=Process.hierarchical) # manager + workers, needs manager_llm +``` + +We use sequential (simplest, deterministic). Hierarchical lets a manager agent dispatch — useful for more complex tasks. + +## Observation across both paths + +| Observation | Anthropic Claude haiku | Ollama qwen2.5:3b | +|---|---|---| +| Researcher actually calls tool | Stable | Sometimes skips tool, makes things up | +| Writer cites researcher's output | Stable | May go off-memory, deviate from search result | +| Critic catches hallucinations | Sharper | Looser, may PASS when it shouldn't | +| Speed | 10-30s | 30-90s | +| Cost | $0.005-0.01 | $0 | + +**Punchline**: multi-agent is more model-quality-sensitive than single-agent — each agent can drop a step, errors compound by the time you reach the critic. Production multi-agent systems almost always use large models (or carefully fine-tuned small ones). + +## Common pitfalls + +- **`expected_output` too generic**: "Some output" gives the LLM no guide. "A 2-sentence blog intro paragraph in active voice" is 10× better +- **Missing `context`**: writer without `context=[research_task]` doesn't see researcher's output — it'll hallucinate +- **Small model + 3 agents**: qwen2.5:3b on 3-agent crews can take 1+ minute. Switch to `qwen2.5:7b` or Claude +- **`allow_delegation=True` use cautiously**: enables agents to call others, easy to loop. Default `False` for prototypes + +## Want smarter answers? + +```bash +MODEL=anthropic/claude-sonnet-5 python starter_anthropic.py # higher quality +MODEL=ollama/qwen2.5:7b python starter.py # larger local model +``` + +## Extensions + +- **Add a manager**: `process=Process.hierarchical` + `manager_llm=...` for dynamic delegation +- **Add memory**: CrewAI has `memory=True` for cross-task context +- **Streaming**: `crew.kickoff_for_each(...)` or `crew.kickoff_async(...)` +- **Human-in-the-loop**: see Exercise 3 (LangGraph) — CrewAI's HITL is weaker diff --git a/examples/stage-4/02-multi-agent-roles/README.md b/examples/stage-4/02-multi-agent-roles/README.md new file mode 100644 index 0000000..db09375 --- /dev/null +++ b/examples/stage-4/02-multi-agent-roles/README.md @@ -0,0 +1,136 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 練習 2：多 agent 角色分配（CrewAI） + +對應 [Stage 4 — Agent Frameworks](../../../stages/04-agent-frameworks.md) 練習 2。 +> 🎓 **學習模式**：這份 `starter.py` 是**完整解答**、不是 TODO skeleton。建議用**主動模式**——`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫一份 `starter.py`、跑 `python test.py` 驗證；卡 20 分鐘再回去對照 reference。完整方法論看 [`docs/HOW_TO_USE.md`](../../../docs/HOW_TO_USE.md)。 + +> 📚 **想要 chapter-length 深入版？** 本 folder 的 starter 是 illustrative 版、聚焦核心 pattern + 兩條 SDK path，不是進階深度教材。深度教材推薦： +> - [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents) ⭐ 中文圈最完整、章節式 + 16 種 production 能力。**本練習對應 hello-agents 的 multi-agent roles / Crew 章節** +> - [CrewAI Examples repo](https://github.com/crewAIInc/crewAI-examples)（官方 sequential / hierarchical 範本） +> - 完整 references 見 [Stage 4 精選 Projects](../../../stages/04-agent-frameworks.md#-精選-projects) + + +## 任務 + +3 個 agent 各自負責一段、合作完成一篇 blog intro： + +``` +Researcher → Writer → Critic + (找資料) (寫稿) (審稿、PASS/ISSUES) +``` + +這種「role-based pipeline」**CrewAI 最拿手**——你描述角色 / 目標 / 任務，框架自己 orchestrate。 + +## 怎麼跑 — 兩條路徑 + +### Path A（默認、本機免費） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +預算：**$0**。3 agent sequential ≈ 30-90 秒（CPU、qwen2.5:3b）。 + +### Path B（Anthropic、想看 cloud 高品質） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +預算：每次 ≈ **$0.005-0.01**（3 agent × 短輸出、claude-haiku-4-5）。 + +## 不花錢驗證程式邏輯 + +```bash +python test.py # tool 邏輯 + crew structure +python test_anthropic.py # starter_anthropic 載入 +``` + +CrewAI 整個 `kickoff()` 太黑盒、純 mock 困難。這份 test 只驗結構（3 agent + 3 task + sequential process + context dependencies）跟 tool 邏輯。實測請跑 starter.py。 + +## CrewAI multi-agent 核心觀念 + +### Agent + +```python +researcher = Agent( + role="Researcher", + goal="...", # 一句話講「成功」長什麼樣 + backstory="...", # 提供 persona context、影響 prompt + tools=[search], + llm=MODEL, +) +``` + +**重點**：`role` 跟 `goal` 影響 prompt 質量很大。不要寫「Agent」、要寫「Researcher who finds factual data」。 + +### Task + +```python +research_task = Task( + description="Search for X and report findings.", + expected_output="A 1-2 sentence factual entry.", + agent=researcher, +) +``` + +**重點**：`expected_output` 是給 LLM 看的「合格範本」、寫越具體越好（譬如「A 2-sentence intro paragraph」比「Some text」好 10 倍）。 + +### Context dependency + +```python +write_task = Task(..., context=[research_task]) # writer 看 researcher 結果 +critic_task = Task(..., context=[research_task, write_task]) # critic 同時看兩個 +``` + +**重點**：`context` 是 CrewAI 的 dataflow 機制。`critic_task.context=[a, b]` 表示 critic 看到 a, b 兩個 task 的 output。 + +### Sequential vs Hierarchical Process + +```python +Crew(..., process=Process.sequential) # 線性走完 +Crew(..., process=Process.hierarchical) # 多個 manager+worker、需設 manager_llm +``` + +這題用 sequential（最簡單、最 deterministic）。Hierarchical 是 ManagementAgent 派任務給其他 agent、適合更複雜場景。 + +## 兩個 path 觀察重點 + +| 觀察項 | Anthropic Claude haiku | Ollama qwen2.5:3b | +|---|---|---| +| Researcher 直接呼叫 tool | 穩 | 偶爾跳過 tool、自己編答案 | +| Writer 引用 Researcher 結果 | 穩 | 可能憑印象寫、偏離 search result | +| Critic 抓 hallucination | 較敏銳 | 較鬆、可能 PASS 過頭 | +| 速度 | 10-30 秒 | 30-90 秒 | +| 成本 | $0.005-0.01 | $0 | + +**教學 punchline**：multi-agent 對 model 質量比 single-agent 敏感——每個 agent 都可能漏一步、錯誤會累積到 critic 那邊。Production 多 agent 系統幾乎必用大 model（或細調過的小 model）。 + +## 常見坑 + +- **`expected_output` 太籠統**：寫「Some output」LLM 完全沒指引、隨便給。寫「A 2-sentence blog intro paragraph in active voice」效果差 10 倍 +- **`context` 漏設**：Writer 沒設 `context=[research_task]`、就拿不到 researcher 結果、會憑空寫 +- **小 model + 3 agent**：qwen2.5:3b 跑 3-agent crew 可能 1 分鐘+。換 `qwen2.5:7b` 或 Claude +- **`allow_delegation=True` 慎用**：開啟後 agent 可以叫其他 agent 幫忙、容易 loop。雛形階段建議 `False` + +## 想看更聰明的答案？ + +```bash +MODEL=anthropic/claude-sonnet-5 python starter_anthropic.py # 高品質 +MODEL=ollama/qwen2.5:7b python starter.py # 較大本機 model +``` + +## 延伸 + +- **加 manager**：`process=Process.hierarchical` + `manager_llm=...`、讓 manager agent 動態分配 +- **加 memory**：CrewAI 有 `memory=True`、讓 agent 跨 task 記住 context +- **改成 streaming**：`crew.kickoff_for_each(...)` 或 `crew.kickoff_async(...)` +- **加 human-in-the-loop**：練習 3 用 LangGraph 做、CrewAI 對 HITL 較弱 diff --git a/examples/stage-4/02-multi-agent-roles/README.zh-Hans.md b/examples/stage-4/02-multi-agent-roles/README.zh-Hans.md new file mode 100644 index 0000000..4d23f4e --- /dev/null +++ b/examples/stage-4/02-multi-agent-roles/README.zh-Hans.md @@ -0,0 +1,129 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 练习 2：多 agent 角色分配（CrewAI） + +对应 [Stage 4 — Agent Frameworks](../../../stages/04-agent-frameworks.zh-Hans.md) 练习 2。 + +## 任务 + +3 个 agent 各自负责一段、合作完成一篇 blog intro： + +``` +Researcher → Writer → Critic + (找资料) (写稿) (审稿、PASS/ISSUES) +``` + +这种“role-based pipeline”**CrewAI 最拿手**——你描述角色 / 目标 / 任务，框架自己 orchestrate。 + +## 怎么跑 — 两条路径 + +### Path A（默认、本机免费） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +预算：**$0**。3 agent sequential ≈ 30-90 秒（CPU、qwen2.5:3b）。 + +### Path B（Anthropic、想看 cloud 高质量） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +预算：每次 ≈ **$0.005-0.01**（3 agent × 短输出、claude-haiku-4-5）。 + +## 不花钱验证程序逻辑 + +```bash +python test.py # tool 逻辑 + crew structure +python test_anthropic.py # starter_anthropic 载入 +``` + +CrewAI 整个 `kickoff()` 太黑盒、纯 mock 困难。这份 test 只验结构（3 agent + 3 task + sequential process + context dependencies）跟 tool 逻辑。实测请跑 starter.py。 + +## CrewAI multi-agent 核心观念 + +### Agent + +```python +researcher = Agent( + role="Researcher", + goal="...", # 一句话讲「成功」长什么样 + backstory="...", # 提供 persona context、影响 prompt + tools=[search], + llm=MODEL, +) +``` + +**重点**：`role` 跟 `goal` 影响 prompt 质量很大。不要写“Agent”、要写“Researcher who finds factual data”。 + +### Task + +```python +research_task = Task( + description="Search for X and report findings.", + expected_output="A 1-2 sentence factual entry.", + agent=researcher, +) +``` + +**重点**：`expected_output` 是给 LLM 看的“合格范本”、写越具体越好（譬如“A 2-sentence intro paragraph”比“Some text”好 10 倍）。 + +### Context dependency + +```python +write_task = Task(..., context=[research_task]) # writer 看 researcher 结果 +critic_task = Task(..., context=[research_task, write_task]) # critic 同时看两个 +``` + +**重点**：`context` 是 CrewAI 的 dataflow 机制。`critic_task.context=[a, b]` 表示 critic 看到 a, b 两个 task 的 output。 + +### Sequential vs Hierarchical Process + +```python +Crew(..., process=Process.sequential) # 线性走完 +Crew(..., process=Process.hierarchical) # 多个 manager+worker、需设 manager_llm +``` + +这题用 sequential（最简单、最 deterministic）。Hierarchical 是 ManagementAgent 派任务给其他 agent、适合更复杂场景。 + +## 两个 path 观察重点 + +| 观察项 | Anthropic Claude haiku | Ollama qwen2.5:3b | +|---|---|---| +| Researcher 直接调用 tool | 稳 | 偶尔跳过 tool、自己编答案 | +| Writer 引用 Researcher 结果 | 稳 | 可能凭印象写、偏离 search result | +| Critic 抓 hallucination | 较敏锐 | 较松、可能 PASS 过头 | +| 速度 | 10-30 秒 | 30-90 秒 | +| 成本 | $0.005-0.01 | $0 | + +**教学 punchline**：multi-agent 对 model 质量比 single-agent 敏感——每个 agent 都可能漏一步、错误会累积到 critic 那边。Production 多 agent 系统几乎必用大 model（或细调过的小 model）。 + +## 常见坑 + +- **`expected_output` 太笼统**：写“Some output”LLM 完全没指引、随便给。写“A 2-sentence blog intro paragraph in active voice”效果差 10 倍 +- **`context` 漏设**：Writer 没设 `context=[research_task]`、就拿不到 researcher 结果、会凭空写 +- **小 model + 3 agent**：qwen2.5:3b 跑 3-agent crew 可能 1 分钟+。换 `qwen2.5:7b` 或 Claude +- **`allow_delegation=True` 慎用**：开启后 agent 可以叫其他 agent 帮忙、容易 loop。雏形阶段建议 `False` + +## 想看更聪明的答案？ + +```bash +MODEL=anthropic/claude-sonnet-5 python starter_anthropic.py # 高品质 +MODEL=ollama/qwen2.5:7b python starter.py # 较大本机 model +``` + +## 延伸 + +- **加 manager**：`process=Process.hierarchical` + `manager_llm=...`、让 manager agent 动态分配 +- **加 memory**：CrewAI 有 `memory=True`、让 agent 跨 task 记住 context +- **改成 streaming**：`crew.kickoff_for_each(...)` 或 `crew.kickoff_async(...)` +- **加 human-in-the-loop**：练习 3 用 LangGraph 做、CrewAI 对 HITL 较弱 diff --git a/examples/stage-4/02-multi-agent-roles/requirements.txt b/examples/stage-4/02-multi-agent-roles/requirements.txt new file mode 100644 index 0000000..e3308db --- /dev/null +++ b/examples/stage-4/02-multi-agent-roles/requirements.txt @@ -0,0 +1,2 @@ +crewai>=0.80,<1.0 +litellm>=1.50,<2.0 diff --git a/examples/stage-4/02-multi-agent-roles/starter.py b/examples/stage-4/02-multi-agent-roles/starter.py new file mode 100644 index 0000000..fe02e94 --- /dev/null +++ b/examples/stage-4/02-multi-agent-roles/starter.py @@ -0,0 +1,126 @@ +"""Stage 4 練習 2：多 agent 角色分配 — CrewAI + Ollama（Path A、默認）。 + +3 個 agent 各有角色： +- Researcher 找資料（用 search tool） +- Writer 寫稿（拿 researcher 的結果寫成 blog 段落） +- Critic 審稿（檢查 factual + tone） + +這個情境 CrewAI 最拿手——hierarchical / sequential delegation 是 CrewAI 的招牌。 + +跑法： + pip install -r requirements.txt + ollama pull qwen2.5:3b + ollama serve + python starter.py + +驗證： + python test.py + +預算：$0/run（Ollama）。對照 Anthropic 版見 starter_anthropic.py。 + +⚠️ 注意：3 個 agent 走 sequential、qwen2.5:3b 可能會繞較久（30-90 秒）。 +""" + +from __future__ import annotations + +import os +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from crewai import Agent, Crew, Process, Task +from crewai.tools import tool + +MODEL = os.environ.get("MODEL", "ollama/qwen2.5:3b") +OLLAMA_BASE = os.environ.get("OLLAMA_API_BASE", "http://localhost:11434") + + +# === Tool === + +@tool("search") +def search(query: str) -> str: + """Search a (fake, offline) knowledge base.""" + db = { + "react": "ReAct (Reasoning+Acting, Yao et al. 2022) is the foundational agent pattern: think→act→observe loop.", + "langgraph": "LangGraph is a graph-based agent orchestration framework by LangChain, focuses on state + checkpointing.", + "crewai": "CrewAI is a role-based multi-agent framework — define Agent/Task/Crew, run with kickoff().", + } + return db.get(query.strip().lower(), f"no entry for {query}") + + +# === Crew 設計 === + +def build_crew(topic: str, llm_model: str = MODEL) -> Crew: + os.environ["OPENAI_API_BASE"] = f"{OLLAMA_BASE}/v1" + os.environ["OPENAI_API_KEY"] = "ollama" + + researcher = Agent( + role="Researcher", + goal=f"Find concise factual info about {topic} from the knowledge base.", + backstory="You search a knowledge base and return raw factual entries.", + tools=[search], + llm=llm_model, + verbose=False, + allow_delegation=False, + ) + writer = Agent( + role="Writer", + goal=f"Write a 2-sentence blog intro about {topic}.", + backstory="You take the researcher's findings and write engaging blog copy.", + llm=llm_model, + verbose=False, + allow_delegation=False, + ) + critic = Agent( + role="Critic", + goal="Verify the writer's blog intro is factually grounded in the researcher's data + check tone.", + backstory="You're a strict editor who flags hallucinations and tone issues.", + llm=llm_model, + verbose=False, + allow_delegation=False, + ) + + research_task = Task( + description=f"Search for `{topic}` and report what you find.", + expected_output="A 1-2 sentence factual entry from the knowledge base.", + agent=researcher, + ) + write_task = Task( + description="Write a 2-sentence blog intro using the researcher's findings.", + expected_output="A 2-sentence intro paragraph.", + agent=writer, + context=[research_task], + ) + critic_task = Task( + description="Check if the writer's intro is factually grounded in the researcher's data. " + "Report PASS or list issues.", + expected_output="Either 'PASS: [intro]' or 'ISSUES: [list]'.", + agent=critic, + context=[research_task, write_task], + ) + + return Crew( + agents=[researcher, writer, critic], + tasks=[research_task, write_task, critic_task], + process=Process.sequential, + verbose=False, + ) + + +def run(topic: str, llm_model: str = MODEL) -> dict: + crew = build_crew(topic, llm_model=llm_model) + result = crew.kickoff() + return {"final": str(result), "topic": topic} + + +if __name__ == "__main__": + topic = "react" + print(f"❓ Topic: {topic}（using CrewAI + Ollama {MODEL}）") + print(f" 3 agents: Researcher → Writer → Critic（sequential）") + print("-" * 60) + result = run(topic) + print(f"✅ Final (critic's verdict):\n{result['final']}") + assert result["final"], "expected critic to produce a verdict" + print("\n✅ 練習 2 通過 — 3-agent crew 跑完、$0/run") diff --git a/examples/stage-4/02-multi-agent-roles/starter_anthropic.py b/examples/stage-4/02-multi-agent-roles/starter_anthropic.py new file mode 100644 index 0000000..b01e80f --- /dev/null +++ b/examples/stage-4/02-multi-agent-roles/starter_anthropic.py @@ -0,0 +1,34 @@ +"""Stage 4 練習 2：多 agent 角色分配 — CrewAI + Anthropic（Path B）。 + +CrewAI 用 LiteLLM 底層、改成 "anthropic/claude-haiku-4-5" 字串即可切 backend。 + +跑法： + pip install -r requirements.txt + export ANTHROPIC_API_KEY=sk-ant-... + python starter_anthropic.py + +預算：3 agent × 短輸出 ≈ $0.005-0.01/run。Claude 對 multi-agent 互動穩很多。 +""" + +from __future__ import annotations + +import os +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter import run + +MODEL = os.environ.get("MODEL", "anthropic/claude-haiku-4-5") # LiteLLM 格式 + + +if __name__ == "__main__": + topic = "react" + print(f"❓ Topic: {topic}（using CrewAI + Anthropic {MODEL}）") + print(f" 3 agents: Researcher → Writer → Critic（sequential）") + print("-" * 60) + result = run(topic, llm_model=MODEL) + print(f"✅ Final (critic's verdict):\n{result['final']}") + assert result["final"], "expected critic to produce a verdict" + print("\n✅ 練習 2 (Anthropic path) 通過 — Claude 跑 3-agent crew、≈$0.005-0.01/run") diff --git a/examples/stage-4/02-multi-agent-roles/test.py b/examples/stage-4/02-multi-agent-roles/test.py new file mode 100644 index 0000000..59136cd --- /dev/null +++ b/examples/stage-4/02-multi-agent-roles/test.py @@ -0,0 +1,44 @@ +"""Stage 4 練習 2 自我驗證 — CrewAI 模組可載入 + tool 邏輯正確。 + +CrewAI 整個 kickoff 太黑盒（內部走 LiteLLM、prompt 累積）、不適合純 mock。 +這份只驗：(1) 3 個 agent + 3 個 task 結構正確、(2) search tool 邏輯正確。 +要實測請跑 starter.py 配 Ollama。 +""" + +from __future__ import annotations + +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter import build_crew, search + + +def test_search_basic(): + fn = getattr(search, "func", None) or getattr(search, "run", None) + if fn is None: + print("⚠ CrewAI tool 介面變動、跳過") + return + assert "ReAct" in fn("react") + assert "no entry" in fn("nonexistent") + print("✅ test_search_basic") + + +def test_crew_structure(): + """確認 build_crew 出來的 Crew 真有 3 個 agent + 3 個 task、process 是 sequential。""" + crew = build_crew("react") + assert len(crew.agents) == 3, f"expected 3 agents, got {len(crew.agents)}" + roles = [a.role for a in crew.agents] + assert roles == ["Researcher", "Writer", "Critic"], f"unexpected roles: {roles}" + assert len(crew.tasks) == 3, f"expected 3 tasks, got {len(crew.tasks)}" + # critic_task 應該 context 包含 research_task 跟 write_task + critic_task = crew.tasks[2] + assert len(critic_task.context) == 2, "critic should depend on research + write" + print("✅ test_crew_structure") + + +if __name__ == "__main__": + test_search_basic() + test_crew_structure() + print("\n🎉 通過 — 3-agent crew 結構正確（實際 kickoff 需 Ollama 跑著）") diff --git a/examples/stage-4/02-multi-agent-roles/test_anthropic.py b/examples/stage-4/02-multi-agent-roles/test_anthropic.py new file mode 100644 index 0000000..b60b340 --- /dev/null +++ b/examples/stage-4/02-multi-agent-roles/test_anthropic.py @@ -0,0 +1,31 @@ +"""Stage 4 練習 2 自我驗證 — Path B import + module-load check。 + +CrewAI multi-agent kickoff 太黑盒、純 mock 困難。實測請跑 starter_anthropic.py 配 ANTHROPIC_API_KEY。 +""" + +from __future__ import annotations + +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + + +def test_starter_anthropic_loadable(): + import starter_anthropic + assert hasattr(starter_anthropic, "MODEL") + assert "anthropic/" in starter_anthropic.MODEL, f"預期 LiteLLM 格式 anthropic/...、得到 {starter_anthropic.MODEL}" + print("✅ test_starter_anthropic_loadable") + + +def test_run_function_imported(): + """starter_anthropic 重用 starter.run，並注入 anthropic model name。""" + from starter_anthropic import run + assert callable(run) + print("✅ test_run_function_imported") + + +if __name__ == "__main__": + test_starter_anthropic_loadable() + test_run_function_imported() + print("\n🎉 通過 — Anthropic path 可載入（實測需 ANTHROPIC_API_KEY）") diff --git a/examples/stage-4/03-graph-workflow/README.en.md b/examples/stage-4/03-graph-workflow/README.en.md new file mode 100644 index 0000000..02b18c9 --- /dev/null +++ b/examples/stage-4/03-graph-workflow/README.en.md @@ -0,0 +1,107 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# Exercise 3: Graph Workflow (LangGraph conditional branching + HITL) + +Pairs with [Stage 4 — Agent Frameworks](../../../stages/04-agent-frameworks.en.md) Exercise 3. + +## Task + +`classify → [search?] → respond → [HITL] → final` + +- **`classify_node`**: decides `needs_search` from the query +- **Conditional branch**: `needs_search=True` → `search`, otherwise `respond` +- **HITL checkpoint**: after `respond`, interrupt — wait for a human to write `approved` into state +- **`final_node`**: `approved=True` → PUBLISHED, else REJECTED + +This is **LangGraph's sweet spot**: graph state + checkpointing + `interrupt_before` is the signature combo. CrewAI's HITL support is weaker. + +## How to run + +```bash +pip install -r requirements.txt +python starter.py +``` + +Budget: **$0**. The nodes here are deterministic logic — no LLM calls. To wire in a real LLM, replace `respond_node` with `llm.invoke(...)`. + +```bash +python test.py # 6 tests for routing + HITL +python test_anthropic.py # Path B concept demo +``` + +## LangGraph structure (condensed) + +```python +g = StateGraph(State) +g.add_node("classify", classify_node) +g.add_node("search", search_node) +g.add_node("respond", respond_node) +g.add_node("final", final_node) + +g.add_edge(START, "classify") +g.add_conditional_edges("classify", should_search, {"search": "search", "respond": "respond"}) +g.add_edge("search", "respond") +g.add_edge("respond", "final") +g.add_edge("final", END) + +graph = g.compile(checkpointer=InMemorySaver(), interrupt_before=["final"]) +``` + +## How HITL works + +```python +# Phase 1: run up to (but not into) final — interrupt_before=["final"] pauses it +state_before = graph.invoke({"query": ...}, config={"configurable": {"thread_id": "demo"}}) +# Show state_before["draft"] to a human and ask "publish?" + +# Human's decision: write `approved` into state +graph.update_state(config, {"approved": True}) + +# Phase 2: resume from final (None means "no new input, use checkpoint") +state_after = graph.invoke(None, config=config) +``` + +**Key**: `interrupt_before=["final"]` tells the graph "stop before entering final". In production, wire this to a webhook / Slack / frontend button, and resume after the user approves. + +## Why this pattern matters + +| Scenario | Without HITL | With HITL | +|---|---|---| +| Agent sends email | Send directly (risky) | Show draft, human approves | +| Agent changes prod config | Apply directly | Dry-run, wait for approval | +| Agent issues refund | Auto-refund | Refund over $X waits for review | + +**Any production agent with side effects should include HITL.** LangGraph's `interrupt_before` is designed for this. + +## What to watch on each path + +Nodes are deterministic (no LLM), so Path A and Path B run identically. **The focus is the graph structure.** To wire in a real LLM: + +```python +# In respond_node: +from langchain_openai import ChatOpenAI # Path A +# from langchain_anthropic import ChatAnthropic # Path B +llm = ChatOpenAI(base_url="http://localhost:11434/v1", api_key="ollama", model="qwen2.5:3b") +draft = llm.invoke(state["query"]).content +return {"draft": draft} +``` + +## Common pitfalls + +- **No `checkpointer`**: `graph.compile(interrupt_before=[...])` without a checkpointer raises. You need one to pause/resume +- **`thread_id` mismatch**: phase-1 invoke + update_state + phase-2 invoke must share `config={"configurable": {"thread_id": "..."}}`; otherwise checkpoint not found +- **`interrupt_before` vs `interrupt_after`**: before = pause entering the node; after = pause after the node finishes. **HITL usually wants `before`** (so the human sees full state) +- **`conditional_edges` function must return a string**: `should_search`'s return value must be a key in the third dict of `add_conditional_edges` — can't return a literal node name + +## Want smarter answers? + +Wire a real LLM into `respond_node`. Production setups also swap the checkpointer to SQLite / Redis (`SqliteSaver` / `RedisSaver`) for persistence. + +## Extensions + +- **Add retry**: in `search_node` failures, retry via LangGraph's `error` edge +- **Multiple HITL stops**: `interrupt_before=["draft", "publish"]` +- **Time-travel debug**: `graph.get_state_history(config)` gives all checkpoints — fork from any of them +- **Streaming**: `for state in graph.stream(...)` to watch state evolve diff --git a/examples/stage-4/03-graph-workflow/README.md b/examples/stage-4/03-graph-workflow/README.md new file mode 100644 index 0000000..8aa0187 --- /dev/null +++ b/examples/stage-4/03-graph-workflow/README.md @@ -0,0 +1,114 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 練習 3：圖式 workflow（LangGraph 條件分支 + HITL） + +對應 [Stage 4 — Agent Frameworks](../../../stages/04-agent-frameworks.md) 練習 3。 +> 🎓 **學習模式**：這份 `starter.py` 是**完整解答**、不是 TODO skeleton。建議用**主動模式**——`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫一份 `starter.py`、跑 `python test.py` 驗證；卡 20 分鐘再回去對照 reference。完整方法論看 [`docs/HOW_TO_USE.md`](../../../docs/HOW_TO_USE.md)。 + +> 📚 **想要 chapter-length 深入版？** 本 folder 的 starter 是 illustrative 版、聚焦核心 pattern + 兩條 SDK path，不是進階深度教材。深度教材推薦： +> - [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents) ⭐ 中文圈最完整、章節式 + 16 種 production 能力。**本練習對應 hello-agents 的 graph workflow + HITL 章節** +> - [LangGraph HITL tutorial](https://langchain-ai.github.io/langgraph/tutorials/human-in-the-loop/) + [LangGraph time-travel docs](https://langchain-ai.github.io/langgraph/concepts/time-travel/) +> - 完整 references 見 [Stage 4 精選 Projects](../../../stages/04-agent-frameworks.md#-精選-projects) + + +## 任務 + +`classify → [search?] → respond → [HITL] → final` + +- **`classify_node`**：看 query 決定 `needs_search` +- **條件分支**：`needs_search=True` 走 `search` node、否則直接 `respond` +- **HITL checkpoint**：`respond_node` 後 interrupt、等人類在 state 改 `approved` +- **`final_node`**：`approved=True` → PUBLISHED、否則 REJECTED + +這題 **LangGraph 最拿手**：graph state + checkpointing + interrupt_before 是 LangGraph 招牌組合。CrewAI 對 HITL 支援較弱。 + +## 怎麼跑 + +```bash +pip install -r requirements.txt +python starter.py +``` + +預算：**$0**。這份 demo 的節點都是 deterministic 邏輯、不打 LLM；要實接 Claude / Ollama 在 `respond_node` 改成 `llm.invoke(...)` 即可。 + +```bash +python test.py # 6 個 test，驗 routing + HITL 邏輯 +python test_anthropic.py # Path B concept demo +``` + +## LangGraph 圖結構（精簡） + +```python +g = StateGraph(State) +g.add_node("classify", classify_node) +g.add_node("search", search_node) +g.add_node("respond", respond_node) +g.add_node("final", final_node) + +g.add_edge(START, "classify") +g.add_conditional_edges("classify", should_search, {"search": "search", "respond": "respond"}) +g.add_edge("search", "respond") +g.add_edge("respond", "final") +g.add_edge("final", END) + +graph = g.compile(checkpointer=InMemorySaver(), interrupt_before=["final"]) +``` + +## HITL 怎麼運作 + +```python +# 第一段：跑到 final 之前自動停（因為 interrupt_before=["final"]） +state_before = graph.invoke({"query": ...}, config={"configurable": {"thread_id": "demo"}}) +# 此時可以印 state_before["draft"]、問人類「要 publish 嗎？」 + +# 人類決定：把 approved 寫進 state +graph.update_state(config, {"approved": True}) + +# 第二段：繼續從 final 跑（None 表示「不給新 input、用 checkpoint」） +state_after = graph.invoke(None, config=config) +``` + +**關鍵**：`interrupt_before=["final"]` 告訴 graph「跑到 final 之前停」。Production 把它接到 webhook / Slack / 前端 button、等使用者按 approve 才繼續。 + +## 為什麼這個 pattern 重要 + +| 情境 | 不用 HITL | 用 HITL | +|---|---|---| +| Agent 發 email | 直接送出（風險） | 顯示草稿、人類按 approve | +| Agent 改 production 設定 | 直接套用 | dry-run 後等核准 | +| Agent 做退款 | 自動退 | 超過 $X 等審核 | + +**Production agent 凡是有 side effect 的、都該加 HITL**。LangGraph `interrupt_before` 就是為這個設計。 + +## 兩個 path 觀察重點 + +本練習節點都是 deterministic 邏輯、不打 LLM、所以 Path A / Path B 跑出來一致。**重點是學圖結構本身**。要實接 LLM： + +```python +# 在 respond_node 改： +from langchain_openai import ChatOpenAI # Path A +# from langchain_anthropic import ChatAnthropic # Path B +llm = ChatOpenAI(base_url="http://localhost:11434/v1", api_key="ollama", model="qwen2.5:3b") +draft = llm.invoke(state["query"]).content +return {"draft": draft} +``` + +## 常見坑 + +- **`checkpointer` 沒設**：`graph.compile(interrupt_before=[...])` 沒帶 checkpointer 會 raise。必須有 checkpointer 才能 pause/resume +- **`thread_id` 不一致**：第一段 invoke + update_state + 第二段 invoke 必須用同一個 `config={"configurable": {"thread_id": "..."}}`、否則 checkpoint 找不到 +- **`interrupt_before` vs `interrupt_after`**：before = 進這個 node 前停、after = 跑完這個 node 停。**HITL 通常用 before**（讓人類看完整 state 再決定） +- **conditional_edges 函數要回 string**：`should_search` return value 必須是 `add_conditional_edges` 第三個參數 dict 的 key、不能 return literal value 直接當 node name + +## 想看更聰明的答案？ + +把 `respond_node` 接 LLM、用真的 model 寫 draft（不再用 template）。Production setup 還會把 checkpointer 換成 SQLite / Redis（`SqliteSaver` / `RedisSaver`）做持久化。 + +## 延伸 + +- **加 retry**：在 `search_node` 失敗時 retry、用 LangGraph 的 `error` edge +- **加多個 HITL**：`interrupt_before=["draft", "publish"]` 兩個地方都暫停 +- **time-travel debug**：`graph.get_state_history(config)` 拿到所有 checkpoint、可以回到任一步 fork 新 thread +- **加 streaming**：`for state in graph.stream(...)` 邊跑邊看 state diff --git a/examples/stage-4/03-graph-workflow/README.zh-Hans.md b/examples/stage-4/03-graph-workflow/README.zh-Hans.md new file mode 100644 index 0000000..ce46556 --- /dev/null +++ b/examples/stage-4/03-graph-workflow/README.zh-Hans.md @@ -0,0 +1,107 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 练习 3：图式 workflow（LangGraph 条件分支 + HITL） + +对应 [Stage 4 — Agent Frameworks](../../../stages/04-agent-frameworks.zh-Hans.md) 练习 3。 + +## 任务 + +`classify → [search?] → respond → [HITL] → final` + +- **`classify_node`**：看 query 决定 `needs_search` +- **条件分支**：`needs_search=True` 走 `search` node、否则直接 `respond` +- **HITL checkpoint**：`respond_node` 后 interrupt、等人类在 state 改 `approved` +- **`final_node`**：`approved=True` → PUBLISHED、否则 REJECTED + +这题 **LangGraph 最拿手**：graph state + checkpointing + interrupt_before 是 LangGraph 招牌组合。CrewAI 对 HITL 支援较弱。 + +## 怎么跑 + +```bash +pip install -r requirements.txt +python starter.py +``` + +预算：**$0**。这份 demo 的节点都是 deterministic 逻辑、不打 LLM；要实接 Claude / Ollama 在 `respond_node` 改成 `llm.invoke(...)` 即可。 + +```bash +python test.py # 6 个 test，验 routing + HITL 逻辑 +python test_anthropic.py # Path B concept demo +``` + +## LangGraph 图结构（精简） + +```python +g = StateGraph(State) +g.add_node("classify", classify_node) +g.add_node("search", search_node) +g.add_node("respond", respond_node) +g.add_node("final", final_node) + +g.add_edge(START, "classify") +g.add_conditional_edges("classify", should_search, {"search": "search", "respond": "respond"}) +g.add_edge("search", "respond") +g.add_edge("respond", "final") +g.add_edge("final", END) + +graph = g.compile(checkpointer=InMemorySaver(), interrupt_before=["final"]) +``` + +## HITL 怎么运作 + +```python +# 第一段：跑到 final 之前自动停（因为 interrupt_before=["final"]） +state_before = graph.invoke({"query": ...}, config={"configurable": {"thread_id": "demo"}}) +# 此时可以印 state_before["draft"]、问人类「要 publish 吗？」 + +# 人类决定：把 approved 写进 state +graph.update_state(config, {"approved": True}) + +# 第二段：继续从 final 跑（None 表示「不给新 input、用 checkpoint」） +state_after = graph.invoke(None, config=config) +``` + +**关键**：`interrupt_before=["final"]` 告诉 graph“跑到 final 之前停”。Production 把它接到 webhook / Slack / 前端 button、等用户按 approve 才继续。 + +## 为什么这个 pattern 重要 + +| 情境 | 不用 HITL | 用 HITL | +|---|---|---| +| Agent 发 email | 直接送出（风险） | 显示草稿、人类按 approve | +| Agent 改 production 设定 | 直接套用 | dry-run 后等核准 | +| Agent 做退款 | 自动退 | 超过 $X 等审核 | + +**Production agent 凡是有 side effect 的、都该加 HITL**。LangGraph `interrupt_before` 就是为这个设计。 + +## 两个 path 观察重点 + +本练习节点都是 deterministic 逻辑、不打 LLM、所以 Path A / Path B 跑出来一致。**重点是学图结构本身**。要实接 LLM： + +```python +# 在 respond_node 改： +from langchain_openai import ChatOpenAI # Path A +# from langchain_anthropic import ChatAnthropic # Path B +llm = ChatOpenAI(base_url="http://localhost:11434/v1", api_key="ollama", model="qwen2.5:3b") +draft = llm.invoke(state["query"]).content +return {"draft": draft} +``` + +## 常见坑 + +- **`checkpointer` 没设**：`graph.compile(interrupt_before=[...])` 没带 checkpointer 会 raise。必须有 checkpointer 才能 pause/resume +- **`thread_id` 不一致**：第一段 invoke + update_state + 第二段 invoke 必须用同一个 `config={"configurable": {"thread_id": "..."}}`、否则 checkpoint 找不到 +- **`interrupt_before` vs `interrupt_after`**：before = 进这个 node 前停、after = 跑完这个 node 停。**HITL 通常用 before**（让人类看完整 state 再决定） +- **conditional_edges 函数要回 string**：`should_search` return value 必须是 `add_conditional_edges` 第三个参数 dict 的 key、不能 return literal value 直接当 node name + +## 想看更聪明的答案？ + +把 `respond_node` 接 LLM、用真的 model 写 draft（不再用 template）。Production setup 还会把 checkpointer 换成 SQLite / Redis（`SqliteSaver` / `RedisSaver`）做持久化。 + +## 延伸 + +- **加 retry**：在 `search_node` 失败时 retry、用 LangGraph 的 `error` edge +- **加多个 HITL**：`interrupt_before=["draft", "publish"]` 两个地方都暂停 +- **time-travel debug**：`graph.get_state_history(config)` 拿到所有 checkpoint、可以回到任一步 fork 新 thread +- **加 streaming**：`for state in graph.stream(...)` 边跑边看 state diff --git a/examples/stage-4/03-graph-workflow/requirements.txt b/examples/stage-4/03-graph-workflow/requirements.txt new file mode 100644 index 0000000..738aee8 --- /dev/null +++ b/examples/stage-4/03-graph-workflow/requirements.txt @@ -0,0 +1,5 @@ +langgraph>=0.2,<1.0 +typing-extensions>=4.10 +# starter_anthropic 概念展示用、不必裝；要實接 LLM 才裝下面 +# langchain-anthropic>=0.2,<1.0 +# langchain-openai>=0.2,<1.0 diff --git a/examples/stage-4/03-graph-workflow/starter.py b/examples/stage-4/03-graph-workflow/starter.py new file mode 100644 index 0000000..55dea5a --- /dev/null +++ b/examples/stage-4/03-graph-workflow/starter.py @@ -0,0 +1,137 @@ +"""Stage 4 練習 3：圖式 workflow — LangGraph + Ollama（Path A、默認）。 + +LangGraph 最拿手：條件分支 + human-in-the-loop + checkpointing。 + +任務：「研究 → 寫稿 → 人類審核 → 發佈」 +- classify_node 看 query 決定 search_needed +- 條件分支：need_search=True → search_node、否則直接 respond_node +- HITL：respond_node 前 interrupt、等人類 approve +- final_node 標記完成 + +跑法： + pip install -r requirements.txt + ollama pull qwen2.5:3b + ollama serve + python starter.py + +驗證： + python test.py +""" + +from __future__ import annotations + +import os +import sys +from typing import Any, Literal + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from langgraph.checkpoint.memory import InMemorySaver +from langgraph.graph import END, START, StateGraph +from typing_extensions import TypedDict + +MODEL = os.environ.get("MODEL", "qwen2.5:3b") + + +class State(TypedDict): + query: str + needs_search: bool + search_result: str + draft: str + approved: bool + final: str + + +# === Nodes === + +def classify_node(state: State) -> dict: + """看 query 決定要不要 search。簡化邏輯：含關鍵字「人口 / 天氣 / 最新」就 search。""" + q = state["query"].lower() + needs = any(k in q for k in ["人口", "weather", "天氣", "latest", "最新", "current"]) + return {"needs_search": needs} + + +def search_node(state: State) -> dict: + db = { + "taipei": "Taipei population ~2.6M (2024)", + "台北": "Taipei population ~2.6M (2024)", + "weather": "sunny 25°C", + "天氣": "sunny 25°C", + } + q = state["query"].lower() + for k, v in db.items(): + if k in q: + return {"search_result": v} + return {"search_result": "no data"} + + +def respond_node(state: State) -> dict: + """寫 draft。實際用 LLM，這裡簡化用 template 演示流程。""" + if state.get("search_result"): + return {"draft": f"Based on search: {state['search_result']}"} + return {"draft": f"Direct answer to: {state['query']}"} + + +def final_node(state: State) -> dict: + if state.get("approved"): + return {"final": f"PUBLISHED: {state['draft']}"} + return {"final": f"REJECTED (human said no): {state['draft']}"} + + +def should_search(state: State) -> Literal["search", "respond"]: + return "search" if state["needs_search"] else "respond" + + +# === Graph build === + +def build_graph(checkpointer: Any = None) -> Any: + g = StateGraph(State) + g.add_node("classify", classify_node) + g.add_node("search", search_node) + g.add_node("respond", respond_node) + g.add_node("final", final_node) + + g.add_edge(START, "classify") + g.add_conditional_edges("classify", should_search, {"search": "search", "respond": "respond"}) + g.add_edge("search", "respond") + g.add_edge("respond", "final") + g.add_edge("final", END) + + # HITL：在 final 之前 interrupt、讓人類在 state 改 approved + return g.compile(checkpointer=checkpointer, interrupt_before=["final"]) + + +def run(query: str, approve: bool = True) -> dict: + """跑一輪含 HITL 的 workflow。`approve` 模擬人類審核決定。""" + checkpointer = InMemorySaver() + graph = build_graph(checkpointer=checkpointer) + config = {"configurable": {"thread_id": "demo"}} + + # 第一段：跑到 final 之前停（interrupt_before=["final"]） + state_before = graph.invoke({"query": query, "approved": False}, config=config) + print(f" draft（等待 approval）: {state_before.get('draft', '')}") + + # HITL：人類審核——更新 state.approved + graph.update_state(config, {"approved": approve}) + + # 第二段：繼續跑完 + state_after = graph.invoke(None, config=config) + return state_after + + +if __name__ == "__main__": + print(f"❓ Workflow: classify → [search?] → respond → [HITL] → final（using {MODEL}）") + print("-" * 60) + + print("\n[Case 1] query 含「台北人口」、需要 search、人類 approve=True") + r1 = run("台北人口是多少？", approve=True) + print(f" final: {r1['final']}") + assert "PUBLISHED" in r1["final"] + + print("\n[Case 2] query 不需 search、人類 approve=False") + r2 = run("解釋一下 Python 是什麼", approve=False) + print(f" final: {r2['final']}") + assert "REJECTED" in r2["final"] + + print("\n✅ 練習 3 通過 — LangGraph 圖式 workflow + HITL checkpoint、$0/run") diff --git a/examples/stage-4/03-graph-workflow/starter_anthropic.py b/examples/stage-4/03-graph-workflow/starter_anthropic.py new file mode 100644 index 0000000..a7651e4 --- /dev/null +++ b/examples/stage-4/03-graph-workflow/starter_anthropic.py @@ -0,0 +1,34 @@ +"""Stage 4 練習 3：圖式 workflow — LangGraph（Path B、概念展示）。 + +這個 starter 的 workflow 不打 LLM API（節點都是 deterministic 邏輯 / template）、 +所以 Path A 跟 Path B **跑出來一樣**。重點是「**圖結構 + checkpointing + HITL** +邏輯本身**完全跨 backend**」。 + +實際 production 把 `respond_node` 換成真的 LLM call 即可——`ChatAnthropic(model="claude-haiku-4-5")` +或 `ChatOpenAI(base_url="...")` 都能塞進來。 + +跑法： + pip install -r requirements.txt + python starter_anthropic.py +""" + +from __future__ import annotations + +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter import run + +if __name__ == "__main__": + print("ℹ️ 這份 workflow 不打 LLM、純展示 LangGraph 圖結構 + HITL。") + print(" 要在節點裡接 Anthropic Claude，把 respond_node 改成：") + print(' from langchain_anthropic import ChatAnthropic') + print(' llm = ChatAnthropic(model="claude-haiku-4-5")') + print(' draft = llm.invoke(state["query"]).content') + print("-" * 60) + r = run("台北人口", approve=True) + print(f"final: {r['final']}") + assert r["final"] + print("✅ 練習 3 (Anthropic concept) 通過 — 圖結構跨 backend 一致") diff --git a/examples/stage-4/03-graph-workflow/test.py b/examples/stage-4/03-graph-workflow/test.py new file mode 100644 index 0000000..4c8a064 --- /dev/null +++ b/examples/stage-4/03-graph-workflow/test.py @@ -0,0 +1,74 @@ +"""Stage 4 練習 3 自我驗證 — LangGraph workflow（無 LLM、純圖邏輯）。 + +跑法： + python test.py + +驗證內容： + - classify_node 正確判斷 needs_search + - 條件分支正確 routing（search vs respond 兩條路） + - HITL：interrupt_before=['final'] + update_state(approved=...) 起作用 + - approve=True → PUBLISHED、approve=False → REJECTED +""" + +from __future__ import annotations + +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter import classify_node, run, search_node, should_search + + +def test_classify_needs_search(): + assert classify_node({"query": "台北人口"})["needs_search"] is True + assert classify_node({"query": "解釋一下 Python"})["needs_search"] is False + assert classify_node({"query": "current weather"})["needs_search"] is True + print("✅ test_classify_needs_search") + + +def test_search_node(): + out = search_node({"query": "taipei population"}) + assert "2.6M" in out["search_result"] + out2 = search_node({"query": "unknown topic"}) + assert "no data" in out2["search_result"] + print("✅ test_search_node") + + +def test_should_search_routing(): + assert should_search({"needs_search": True}) == "search" + assert should_search({"needs_search": False}) == "respond" + print("✅ test_should_search_routing") + + +def test_hitl_approve_true_publishes(): + """搜尋路徑 + 人類 approve=True → PUBLISHED。""" + result = run("台北人口", approve=True) + assert "PUBLISHED" in result["final"] + assert "2.6M" in result["draft"] # draft 引用了 search result + print("✅ test_hitl_approve_true_publishes") + + +def test_hitl_approve_false_rejects(): + """非搜尋路徑 + 人類 approve=False → REJECTED。""" + result = run("解釋 Python", approve=False) + assert "REJECTED" in result["final"] + assert "Direct answer" in result["draft"] + print("✅ test_hitl_approve_false_rejects") + + +def test_branching_skips_search(): + """非搜尋路徑時、search_result 應該不存在或空。""" + result = run("解釋 Python", approve=True) + assert not result.get("search_result"), f"預期無 search_result、得到 {result.get('search_result')}" + print("✅ test_branching_skips_search") + + +if __name__ == "__main__": + test_classify_needs_search() + test_search_node() + test_should_search_routing() + test_hitl_approve_true_publishes() + test_hitl_approve_false_rejects() + test_branching_skips_search() + print("\n🎉 全部通過 — LangGraph 圖結構 + 條件分支 + HITL 邏輯正確") diff --git a/examples/stage-4/03-graph-workflow/test_anthropic.py b/examples/stage-4/03-graph-workflow/test_anthropic.py new file mode 100644 index 0000000..f58dae1 --- /dev/null +++ b/examples/stage-4/03-graph-workflow/test_anthropic.py @@ -0,0 +1,34 @@ +"""Stage 4 練習 3 — Path B 概念展示：圖結構跨 backend 一致。 + +跑法：python test_anthropic.py + +這份只是 starter.py / test.py 的 Path B variant 確認——workflow 不依賴 LLM SDK、 +所以兩條 path 跑出來一致。實際 production 在節點接 Claude 時、只改 respond_node。 +""" + +from __future__ import annotations + +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter_anthropic import run + + +def test_concept_workflow_runs(): + result = run("台北人口", approve=True) + assert "PUBLISHED" in result["final"] + print("✅ test_concept_workflow_runs") + + +def test_starter_anthropic_module_ok(): + import starter_anthropic + assert hasattr(starter_anthropic, "run") + print("✅ test_starter_anthropic_module_ok") + + +if __name__ == "__main__": + test_concept_workflow_runs() + test_starter_anthropic_module_ok() + print("\n🎉 通過 — Path B concept demo 跑通") diff --git a/examples/stage-4/04-codeact-vs-json-tool/README.en.md b/examples/stage-4/04-codeact-vs-json-tool/README.en.md new file mode 100644 index 0000000..42070e4 --- /dev/null +++ b/examples/stage-4/04-codeact-vs-json-tool/README.en.md @@ -0,0 +1,114 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# Exercise 4: CodeAct vs JSON tool (Smolagents) + +Pairs with [Stage 4 — Agent Frameworks](../../../stages/04-agent-frameworks.en.md) Exercise 4. + +## Two agent-action paradigms + +| Path | How the agent acts | Example frameworks | +|---|---|---| +| **JSON tool** | LLM returns `{"name": "tool_x", "arguments": {...}}` | OpenAI function calling, LangGraph, CrewAI | +| **CodeAct** | LLM writes Python code, framework executes it | HuggingFace Smolagents | + +**This exercise solves the same task (population ratio) using CodeAct** — compare with the JSON-tool implementations in Exercises 1 / 3. + +## How to run — two paths + +### Path A (default, free, local) + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +Budget: **$0**. CodeAct is hard on small models — qwen2.5:3b sometimes produces syntax errors and iterates to fix. + +### Path B (Anthropic, cloud-quality) + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +Budget: ~**$0.005-0.02** per run (CodeAct typically multi-step, claude-haiku-4-5). Claude writes correct Python far more reliably than qwen2.5:3b. + +## Validate the logic + +```bash +python test.py # tool function + agent structure +python test_anthropic.py # Path B import check +``` + +## How CodeAct works + +The LLM doesn't return JSON — it returns a **Python code block**: + +``` +(user) Find Taipei population, divide by NYC, give ratio. + +(LLM response) +```python +pop_taipei = lookup_fact(query="Taipei population") # 2602000 +pop_nyc = lookup_fact(query="New York population") # 8336000 +ratio = calculator(expression=f"{pop_taipei}/{pop_nyc}") # 0.3122 +print(ratio) +``` + +(Smolagents runs the code in a sandbox and feeds the print output back to the LLM) +``` + +The framework provides a sandboxed Python interpreter; the agent imports tools, writes code, sees stdout, continues. + +## CodeAct vs JSON tool + +| Dimension | JSON tool | CodeAct | +|---|---|---| +| LLM output form | Structured JSON | Python code | +| Variable binding | LLM must remember / call again | Native variables (`pop_taipei = ...`) | +| Multi-step compute | One call per step | Multiple steps in one code block | +| Tokens per round | Fewer | More (code is longer) | +| Small-model friendliness | Better (stable JSON) | Worse (must produce valid Python) | +| Debug | Inspect tool calls | Inspect code execution log | +| Safety | Tool args validated | Sandboxed Python (watch eval/exec limits) | +| Best for | Single-step, clear boundaries | Multi-step computation, intermediate variables | + +**HuggingFace's stance**: CodeAct is closer to how humans solve problems — use a variable for intermediate results, don't re-fetch each step. + +## What to watch on each path + +| Observation | Anthropic Claude haiku | Ollama qwen2.5:3b | +|---|---|---| +| Valid Python syntax | Stable | Occasional syntax errors, iterates to fix | +| Variable naming / reuse | Natural | Tends to re-call tools instead of using vars | +| Correct multi-step ratio | High | Medium | +| Total steps | 1-2 | 3-5 (iterating fixes) | +| Cost | $0.005-0.02 | $0 | + +**Punchline**: CodeAct is **model-quality-sensitive** — the LLM has to write Python that's solid enough for production. **Small models favor JSON-tool over CodeAct** (Exercise 6 of Stage 3 confirms a similar pattern at the schema layer). + +## Common pitfalls + +- **`@tool` function docstring is part of the prompt**: Smolagents passes the docstring to the LLM as the tool description. **Bad docstring = LLM doesn't know when to use it.** +- **CodeAct sandbox**: by default Smolagents bans `import os`, `open`, etc. To allow specific modules, set `additional_authorized_imports=[...]` +- **`max_steps` too low**: CodeAct iterates; `max_steps=4` may not be enough. But too high = infinite loops. 4-8 is the empirical range +- **Small models produce syntax errors**: Smolagents feeds the error back to the LLM, but you waste tokens. Production wants a larger model + +## Want smarter answers? + +```bash +MODEL=anthropic/claude-sonnet-5 python starter_anthropic.py # most stable +MODEL=qwen2.5:7b python starter.py # larger local model +``` + +## Extensions + +- **More tools**: just `@tool`-decorate; Smolagents auto-extracts docstring as description +- **Try `ToolCallingAgent`**: Smolagents also offers JSON-tool-style agents. Compare side-by-side +- **Hugging Face Hub**: `HfApiModel` for HF inference (no need for local Ollama) +- **Read [Anthropic — Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents)**: Anthropic argues both paradigms are valid; pick by task diff --git a/examples/stage-4/04-codeact-vs-json-tool/README.md b/examples/stage-4/04-codeact-vs-json-tool/README.md new file mode 100644 index 0000000..b81c353 --- /dev/null +++ b/examples/stage-4/04-codeact-vs-json-tool/README.md @@ -0,0 +1,121 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 練習 4：CodeAct vs JSON tool（Smolagents） + +對應 [Stage 4 — Agent Frameworks](../../../stages/04-agent-frameworks.md) 練習 4。 +> 🎓 **學習模式**：這份 `starter.py` 是**完整解答**、不是 TODO skeleton。建議用**主動模式**——`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫一份 `starter.py`、跑 `python test.py` 驗證；卡 20 分鐘再回去對照 reference。完整方法論看 [`docs/HOW_TO_USE.md`](../../../docs/HOW_TO_USE.md)。 + +> 📚 **想要 chapter-length 深入版？** 本 folder 的 starter 是 illustrative 版、聚焦核心 pattern + 兩條 SDK path，不是進階深度教材。深度教材推薦： +> - [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents) ⭐ 中文圈最完整、章節式 + 16 種 production 能力。**本練習對應 hello-agents 的 CodeAct vs JSON tool 章節** +> - [Smolagents 官方 cookbook](https://github.com/huggingface/smolagents/tree/main/examples) + [QuantaLogic/quantalogic](https://github.com/quantalogic/quantalogic)（另一個 CodeAct framework） +> - 完整 references 見 [Stage 4 精選 Projects](../../../stages/04-agent-frameworks.md#-精選-projects) + + +## 兩種 agent action 路線對照 + +| 路線 | 怎麼 act | 範例 framework | +|---|---|---| +| **JSON tool** | LLM 回 `{"name": "tool_x", "arguments": {...}}` | OpenAI function calling、LangGraph、CrewAI | +| **CodeAct** | LLM 寫 Python code、直接執行 | HuggingFace Smolagents | + +**這題用 CodeAct 解同題（人口比例）、跟練習 1 / 3 的 JSON tool 路線對照**。 + +## 怎麼跑 — 兩條路徑 + +### Path A（默認、本機免費） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +預算：**$0**。CodeAct 對小 model 比較吃力——qwen2.5:3b 可能會產 syntax error、agent 自己迭代修。 + +### Path B（Anthropic、想看 cloud 高品質） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +預算：每次 ≈ **$0.005-0.02**（CodeAct 通常多步、claude-haiku-4-5）。Claude 寫 Python code 比 qwen2.5:3b 穩很多。 + +## 不花錢驗證程式邏輯 + +```bash +python test.py # tool function + agent 結構 +python test_anthropic.py # Path B 可載入檢查 +``` + +## CodeAct 是怎麼運作的 + +LLM 不回 JSON、而是**回 Python code block**： + +``` +（user）Find Taipei population, divide by NYC, give ratio. + +（LLM 回應） +```python +pop_taipei = lookup_fact(query="Taipei population") # 2602000 +pop_nyc = lookup_fact(query="New York population") # 8336000 +ratio = calculator(expression=f"{pop_taipei}/{pop_nyc}") # 0.3122 +print(ratio) +``` + +（Smolagents 執行這段 code、把 print 結果接回去給 LLM 繼續） +``` + +Framework 提供 sandboxed Python interpreter、agent 在裡面 import tool、寫 code、看 print 結果繼續。 + +## CodeAct vs JSON tool 對照 + +| 維度 | JSON tool | CodeAct | +|---|---|---| +| LLM 輸出形式 | 結構化 JSON | Python 程式碼 | +| 變數綁定 | LLM 要自己記得 / 重複呼叫 | 自然有 variable（`pop_taipei = ...`） | +| 多步運算 | 每步一次 LLM call | 一次寫好幾行 code | +| 一輪 token 數 | 較少 | 較多（code 較長） | +| 對小 model | 較友善（穩定的 JSON） | 較吃力（要產正確 Python） | +| Debug 友善 | tool call 看得清楚 | 看 code execution log | +| 安全考量 | tool args validated | Sandboxed Python（注意 eval/exec 限制） | +| 哪些題目擅長 | 單步、邊界明確 | 多步運算、需要中間 variable | + +**HuggingFace 的觀點**：CodeAct 更貼近「人類怎麼解問題」——你也是用變數記中間結果、不是每步都重新查。 + +## 兩個 path 觀察重點 + +| 觀察項 | Anthropic Claude haiku | Ollama qwen2.5:3b | +|---|---|---| +| 產正確 Python syntax | 穩 | 偶爾 syntax error、會自己迭代修 | +| 變數命名 / 重用 | 自然 | 容易重複呼叫 tool 而非用 variable | +| 多步 ratio 算對 | 高機率 | 中機率 | +| 步數 | 1-2 步 | 3-5 步（迭代修錯） | +| 成本 | $0.005-0.02 | $0 | + +**punchline**：CodeAct 是 **model 質量敏感** pattern——LLM 要會寫能在 production 跑的 Python。**小 model 在 JSON tool 路線比 CodeAct 路線優**（Stage 3 練習 6 也驗證過這點）。 + +## 常見坑 + +- **`@tool` 函式 docstring 是 prompt 的一部分**：Smolagents 把 docstring 當 tool description 給 LLM 看。**docstring 沒寫好、LLM 不知道何時用這 tool** +- **CodeAct sandbox**：Smolagents 預設禁 `import os`、`open` 等危險操作。要放行特定 module、設 `additional_authorized_imports=[...]` +- **`max_steps` 不夠**：CodeAct 跑多步、`max_steps=4` 可能不夠。但太大又會無限迴圈。經驗值 4-8 +- **小 model 寫的 code 有 syntax error**：Smolagents 會把 error 接回去讓 LLM 修、但會浪費 token。Production 用大 model 比較划算 + +## 想看更聰明的答案？ + +```bash +MODEL=anthropic/claude-sonnet-5 python starter_anthropic.py # 最穩 +MODEL=qwen2.5:7b python starter.py # 較大本機 model +``` + +## 延伸 + +- **加更多 tools**：`@tool` 裝飾函式即自動 wrap、Smolagents 自動拿 docstring 當 description +- **改 ToolCallingAgent**：Smolagents 也有非 CodeAct 的 `ToolCallingAgent`、用 JSON tool 路線。對照看 +- **接 Hugging Face Hub**：`HfApiModel` 直接打 HF inference（不必本機 ollama） +- **看 [Anthropic Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents)**：Anthropic 的觀點是兩條路線都合理、看任務 diff --git a/examples/stage-4/04-codeact-vs-json-tool/README.zh-Hans.md b/examples/stage-4/04-codeact-vs-json-tool/README.zh-Hans.md new file mode 100644 index 0000000..ffa06c0 --- /dev/null +++ b/examples/stage-4/04-codeact-vs-json-tool/README.zh-Hans.md @@ -0,0 +1,114 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 练习 4：CodeAct vs JSON tool（Smolagents） + +对应 [Stage 4 — Agent Frameworks](../../../stages/04-agent-frameworks.zh-Hans.md) 练习 4。 + +## 两种 agent action 路线对照 + +| 路线 | 怎么 act | 范例 framework | +|---|---|---| +| **JSON tool** | LLM 回 `{"name": "tool_x", "arguments": {...}}` | OpenAI function calling、LangGraph、CrewAI | +| **CodeAct** | LLM 写 Python code、直接执行 | HuggingFace Smolagents | + +**这题用 CodeAct 解同题（人口比例）、跟练习 1 / 3 的 JSON tool 路线对照**。 + +## 怎么跑 — 两条路径 + +### Path A（默认、本机免费） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +预算：**$0**。CodeAct 对小 model 比较吃力——qwen2.5:3b 可能会产 syntax error、agent 自己迭代修。 + +### Path B（Anthropic、想看 cloud 高质量） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +预算：每次 ≈ **$0.005-0.02**（CodeAct 通常多步、claude-haiku-4-5）。Claude 写 Python code 比 qwen2.5:3b 稳很多。 + +## 不花钱验证程序逻辑 + +```bash +python test.py # tool function + agent 结构 +python test_anthropic.py # Path B 可载入检查 +``` + +## CodeAct 是怎么运作的 + +LLM 不回 JSON、而是**回 Python code block**： + +``` +（user）Find Taipei population, divide by NYC, give ratio. + +（LLM 回应） +```python +pop_taipei = lookup_fact(query="Taipei population") # 2602000 +pop_nyc = lookup_fact(query="New York population") # 8336000 +ratio = calculator(expression=f"{pop_taipei}/{pop_nyc}") # 0.3122 +print(ratio) +``` + +（Smolagents 执行这段 code、把 print 结果接回去给 LLM 继续） +``` + +Framework 提供 sandboxed Python interpreter、agent 在里面 import tool、写 code、看 print 结果继续。 + +## CodeAct vs JSON tool 对照 + +| 维度 | JSON tool | CodeAct | +|---|---|---| +| LLM 输出形式 | 结构化 JSON | Python 程序码 | +| 变数绑定 | LLM 要自己记得 / 重复调用 | 自然有 variable（`pop_taipei = ...`） | +| 多步运算 | 每步一次 LLM call | 一次写好几行 code | +| 一轮 token 数 | 较少 | 较多（code 较长） | +| 对小 model | 较友善（稳定的 JSON） | 较吃力（要产正确 Python） | +| Debug 友善 | tool call 看得清楚 | 看 code execution log | +| 安全考量 | tool args validated | Sandboxed Python（注意 eval/exec 限制） | +| 哪些题目擅长 | 单步、边界明确 | 多步运算、需要中间 variable | + +**HuggingFace 的观点**：CodeAct 更贴近“人类怎么解问题”——你也是用变数记中间结果、不是每步都重新查。 + +## 两个 path 观察重点 + +| 观察项 | Anthropic Claude haiku | Ollama qwen2.5:3b | +|---|---|---| +| 产正确 Python syntax | 稳 | 偶尔 syntax error、会自己迭代修 | +| 变数命名 / 重用 | 自然 | 容易重复调用 tool 而非用 variable | +| 多步 ratio 算对 | 高机率 | 中机率 | +| 步数 | 1-2 步 | 3-5 步（迭代修错） | +| 成本 | $0.005-0.02 | $0 | + +**punchline**：CodeAct 是 **model 质量敏感** pattern——LLM 要会写能在 production 跑的 Python。**小 model 在 JSON tool 路线比 CodeAct 路线优**（Stage 3 练习 6 也验证过这点）。 + +## 常见坑 + +- **`@tool` 函数 docstring 是 prompt 的一部分**：Smolagents 把 docstring 当 tool description 给 LLM 看。**docstring 没写好、LLM 不知道何时用这 tool** +- **CodeAct sandbox**：Smolagents 预设禁 `import os`、`open` 等危险操作。要放行特定 module、设 `additional_authorized_imports=[...]` +- **`max_steps` 不够**：CodeAct 跑多步、`max_steps=4` 可能不够。但太大又会无限循环。经验值 4-8 +- **小 model 写的 code 有 syntax error**：Smolagents 会把 error 接回去让 LLM 修、但会浪费 token。Production 用大 model 比较划算 + +## 想看更聪明的答案？ + +```bash +MODEL=anthropic/claude-sonnet-5 python starter_anthropic.py # 最稳 +MODEL=qwen2.5:7b python starter.py # 较大本机 model +``` + +## 延伸 + +- **加更多 tools**：`@tool` 装饰函数即自动 wrap、Smolagents 自动拿 docstring 当 description +- **改 ToolCallingAgent**：Smolagents 也有非 CodeAct 的 `ToolCallingAgent`、用 JSON tool 路线。对照看 +- **接 Hugging Face Hub**：`HfApiModel` 直接打 HF inference（不必本机 ollama） +- **看 [Anthropic Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents)**：Anthropic 的观点是两条路线都合理、看任务 diff --git a/examples/stage-4/04-codeact-vs-json-tool/requirements.txt b/examples/stage-4/04-codeact-vs-json-tool/requirements.txt new file mode 100644 index 0000000..8910def --- /dev/null +++ b/examples/stage-4/04-codeact-vs-json-tool/requirements.txt @@ -0,0 +1,2 @@ +smolagents>=1.0,<2.0 +litellm>=1.50,<2.0 diff --git a/examples/stage-4/04-codeact-vs-json-tool/starter.py b/examples/stage-4/04-codeact-vs-json-tool/starter.py new file mode 100644 index 0000000..90ae94c --- /dev/null +++ b/examples/stage-4/04-codeact-vs-json-tool/starter.py @@ -0,0 +1,93 @@ +"""Stage 4 練習 4：CodeAct（Smolagents）vs JSON tool — Path A（Ollama 默認）。 + +CodeAct pattern：agent 產生 Python 程式碼當 action（不是 JSON tool call）。 +HuggingFace Smolagents 是這條路線的代表——LLM 寫 `result = calculator(...)` 一句一句執行。 + +跟練習 1 / 3 用的 JSON tool 路線對照： +- JSON tool: LLM 回 `{"name": "calculator", "arguments": {"expression": "..."}}` +- CodeAct: LLM 回 ```python\nresult = calculator(expression="...")\nprint(result)\n``` + +跑法： + pip install -r requirements.txt + ollama pull qwen2.5:3b + ollama serve + python starter.py + +驗證： + python test.py + +⚠️ 注意：Smolagents 對小 model 比較吃力（要產正確 Python syntax）。 +qwen2.5:3b 可能會產出有 syntax error 的 code、agent 自己迭代修。Claude 較穩。 +""" + +from __future__ import annotations + +import os +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from smolagents import CodeAgent, OpenAIServerModel, tool + +MODEL = os.environ.get("MODEL", "qwen2.5:3b") +OLLAMA_BASE = os.environ.get("OLLAMA_API_BASE", "http://localhost:11434/v1") + + +# === Tools === + +@tool +def calculator(expression: str) -> str: + """Safe arithmetic calculator (whitelist + - * / parentheses). + + Args: + expression: Arithmetic expression to evaluate, e.g. '2 + 3 * (4 - 1)'. + """ + allowed = set("0123456789.+-*/() ") + if any(c not in allowed for c in expression): + return "error: only basic arithmetic allowed" + try: + return str(eval(expression, {"__builtins__": {}}, {})) # noqa: S307 + except Exception as e: # noqa: BLE001 + return f"error: {e}" + + +@tool +def lookup_fact(query: str) -> str: + """Look up a fact (population / physical constants, etc.). + + Args: + query: Fact key to look up, e.g. 'taipei population' or 'speed of light'. + """ + db = { + "taipei population": "2602000", + "new york population": "8336000", + "speed of light": "299792458", + } + return db.get(query.strip().lower(), f"unknown: {query}") + + +def build_agent(model: Any = None) -> CodeAgent: + model = model or OpenAIServerModel( + model_id=MODEL, api_base=OLLAMA_BASE, api_key="ollama", + ) + return CodeAgent(tools=[calculator, lookup_fact], model=model, max_steps=4) + + +def run(question: str, model: Any = None) -> dict: + agent = build_agent(model=model) + result = agent.run(question) + return {"final": str(result)} + + +if __name__ == "__main__": + question = "Find Taipei population, divide by New York population, give the ratio." + print(f"❓ Question: {question}（using Smolagents + Ollama {MODEL}）") + print(" CodeAct pattern: agent will WRITE PYTHON CODE that calls tools") + print("-" * 60) + result = run(question) + print(f"\n✅ Final: {result['final']}") + assert result["final"], "expected non-empty answer" + print("✅ 練習 4 通過 — Smolagents CodeAct、$0/run") + print(" 對照練習 1 / 3 的 JSON tool 寫法、看兩種路線怎麼解同題") diff --git a/examples/stage-4/04-codeact-vs-json-tool/starter_anthropic.py b/examples/stage-4/04-codeact-vs-json-tool/starter_anthropic.py new file mode 100644 index 0000000..69f84c1 --- /dev/null +++ b/examples/stage-4/04-codeact-vs-json-tool/starter_anthropic.py @@ -0,0 +1,37 @@ +"""Stage 4 練習 4：CodeAct（Smolagents）— Path B（Anthropic Claude via LiteLLM）。 + +Smolagents 的 LiteLLMModel 可以打 Claude，model_id 用 LiteLLM 格式。 + +跑法： + pip install -r requirements.txt + export ANTHROPIC_API_KEY=sk-ant-... + python starter_anthropic.py + +預算：CodeAct 通常多步、每次 ≈ $0.005-0.02（claude-haiku-4-5）。 +Claude 寫 Python code 比 qwen2.5:3b 穩很多。 +""" + +from __future__ import annotations + +import os +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from smolagents import LiteLLMModel + +from starter import run + +MODEL = os.environ.get("MODEL", "anthropic/claude-haiku-4-5") + + +if __name__ == "__main__": + question = "Find Taipei population, divide by New York population, give the ratio." + print(f"❓ Question: {question}（using Smolagents + Anthropic {MODEL}）") + print("-" * 60) + model = LiteLLMModel(model_id=MODEL) + result = run(question, model=model) + print(f"\n✅ Final: {result['final']}") + assert result["final"] + print("✅ 練習 4 (Anthropic path) 通過 — Claude 寫 Python code 比小 model 穩很多") diff --git a/examples/stage-4/04-codeact-vs-json-tool/test.py b/examples/stage-4/04-codeact-vs-json-tool/test.py new file mode 100644 index 0000000..53696d7 --- /dev/null +++ b/examples/stage-4/04-codeact-vs-json-tool/test.py @@ -0,0 +1,65 @@ +"""Stage 4 練習 4 自我驗證 — CodeAct tool 邏輯 + agent 建構。 + +Smolagents 整個 agent.run() 會實際執行 Python code、純 mock 困難。 +這份只驗：(1) tool function 邏輯、(2) build_agent 可建構、(3) max_steps 等預設正確。 + +要實測請跑 starter.py 配 Ollama。 +""" + +from __future__ import annotations + +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter import build_agent, calculator, lookup_fact + + +def test_calculator_basic(): + fn = getattr(calculator, "func", None) or getattr(calculator, "forward", None) or calculator + # Smolagents @tool decorator wraps; the underlying callable can be reached via .forward or direct call + try: + result = fn("2 + 3") if callable(fn) and fn is not calculator else calculator("2 + 3") + except TypeError: + # If tool wrapper expects dict + result = calculator(expression="2 + 3") + assert "5" in str(result), f"expected 5, got {result}" + print("✅ test_calculator_basic") + + +def test_calculator_rejects_injection(): + try: + out = calculator(expression="__import__('os').system('ls')") + except TypeError: + out = calculator("__import__('os').system('ls')") + assert "error" in str(out) + print("✅ test_calculator_rejects_injection") + + +def test_lookup_fact_basic(): + try: + out = lookup_fact(query="Taipei population") + except TypeError: + out = lookup_fact("Taipei population") + assert "2602000" in str(out) + print("✅ test_lookup_fact_basic") + + +def test_build_agent_ok(): + """確認 build_agent 能組出 CodeAgent、且 max_steps 預設合理。""" + # 不真打 API：傳一個 mock model + class FakeModel: + def __call__(self, *a, **kw): return "" + agent = build_agent(model=FakeModel()) + assert hasattr(agent, "run"), "expected CodeAgent.run method" + assert agent.max_steps == 4, f"expected max_steps=4, got {agent.max_steps}" + print("✅ test_build_agent_ok") + + +if __name__ == "__main__": + test_calculator_basic() + test_calculator_rejects_injection() + test_lookup_fact_basic() + test_build_agent_ok() + print("\n🎉 通過 — Smolagents tool + agent 結構正確（實測需 Ollama）") diff --git a/examples/stage-4/04-codeact-vs-json-tool/test_anthropic.py b/examples/stage-4/04-codeact-vs-json-tool/test_anthropic.py new file mode 100644 index 0000000..9f2e9da --- /dev/null +++ b/examples/stage-4/04-codeact-vs-json-tool/test_anthropic.py @@ -0,0 +1,30 @@ +"""Stage 4 練習 4 — Path B 載入檢查。 + +Smolagents 整個 CodeAct 太重、純 mock 困難。實測請跑 starter_anthropic.py 配 ANTHROPIC_API_KEY。 +""" + +from __future__ import annotations + +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + + +def test_litellm_model_importable(): + from smolagents import LiteLLMModel + assert LiteLLMModel is not None + print("✅ test_litellm_model_importable") + + +def test_starter_anthropic_loadable(): + import starter_anthropic + assert hasattr(starter_anthropic, "MODEL") + assert "anthropic/" in starter_anthropic.MODEL + print("✅ test_starter_anthropic_loadable") + + +if __name__ == "__main__": + test_litellm_model_importable() + test_starter_anthropic_loadable() + print("\n🎉 通過 — Path B 可載入（實測需 ANTHROPIC_API_KEY）") diff --git a/examples/stage-4/05-typed-agent/README.en.md b/examples/stage-4/05-typed-agent/README.en.md new file mode 100644 index 0000000..43a528d --- /dev/null +++ b/examples/stage-4/05-typed-agent/README.en.md @@ -0,0 +1,139 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# Exercise 5: Type-Safe Agent (Pydantic AI structured output) + +Pairs with [Stage 4 — Agent Frameworks](../../../stages/04-agent-frameworks.en.md) Exercise 5. + +## Task + +Agent answers questions and is **forced** to return `AnswerWithConfidence`: + +```python +class AnswerWithConfidence(BaseModel): + answer: str + confidence: float = Field(ge=0.0, le=1.0) # runtime check + sources: list[str] +``` + +Pydantic AI lifts schema validation from the prompt layer (Stage 3 Exercise 6) **to the type layer** — if the LLM violates the schema, the framework auto-retries. Production teams use this to fight hallucination. + +## How to run — two paths + +### Path A (default, free, local) + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +Budget: **$0**. But qwen2.5:3b may retry several times to satisfy the schema, raising total token usage. + +### Path B (Anthropic, cloud-quality) + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +Budget: ~**$0.001** per run (claude-haiku-4-5, usually one shot, no retry). + +## Validate the logic + +```bash +python test.py # Pydantic schema validation + agent structure +python test_anthropic.py # Path B import check +``` + +`test.py` directly verifies `AnswerWithConfidence` raises `ValidationError` on invalid data (confidence > 1.0, wrong type, sources not a list) — no LLM calls, pure type-layer testing. + +## Why type-safe agents matter + +``` +Stage 3 Exercise 6: schema = JSON Schema in the prompt + LLM sees it, but what it returns is up to the LLM (may violate) + +Stage 4 Exercise 5: schema = Pydantic model in code + LLM violates → framework auto-raises → retry / fix + Final output is guaranteed to conform (runtime guarantee) +``` + +For production: + +| Need | Prompt-only schema | Pydantic AI | +|---|---|---| +| LLM drops a field | Your downstream code needs try/except | Auto-retry until conformant | +| Wrong type (confidence="high") | Downstream crash | Pydantic ValidationError, retry | +| Out of bound (confidence=1.5) | Downstream gets bad data | Reject, retry | +| LLM hallucinates extra fields | Silently accepted | Default ignore (configurable to strict) | + +**Bottom line**: production agents must use type-safe output. Stage 3 Exercise 6 teaches schema design; Stage 4 Exercise 5 teaches turning that schema into a runtime contract. + +## Core Pydantic AI concepts + +### Agent + output_type + +```python +agent = Agent( + model=..., + output_type=AnswerWithConfidence, # ← force LLM into this shape + system_prompt="..." +) +result = agent.run_sync(question) +answer: AnswerWithConfidence = result.output # validated object +``` + +**Key**: behind the scenes, the framework converts the Pydantic schema into structured-output instructions for the LLM, validates the response, and retries on failure. + +### Field constraints + +```python +confidence: float = Field(ge=0.0, le=1.0, description="...") +``` + +`ge` / `le` are Pydantic's numeric bounds. If the LLM returns 1.5, Pydantic raises ValidationError → retry. + +### Auto-retry + +```python +Agent(..., retries=3) # default 1, configurable +``` + +When Pydantic AI sees a ValidationError, it appends the error message back into the prompt and asks the LLM to retry. + +## What to watch on each path + +| Observation | Anthropic Claude haiku | Ollama qwen2.5:3b | +|---|---|---| +| One-shot schema correct | 90%+ | 50-70% | +| Average retries | 0-1 | 1-3 | +| Confidence bound respected | Stable | Occasional 1.5 / negative (rejected, retry) | +| Sources is a list | Stable | Occasional string (rejected) | +| Total token cost | Low (few retries) | High (multiple retries) | + +**Counterintuitive**: Path B (Claude) can actually have **lower total token cost** than Path A (qwen) — retries add up. Looking at the full bill, production teams pick the larger model. + +## Common pitfalls + +- **`output_type` too complex**: deeply nested models are hard for the LLM to produce in one shot. Aim for flat, ≤5 top-level fields +- **Missing `description`**: `Field(...)` without `description=` leaves the LLM guessing what the field is for +- **`retries=0`**: failure raises immediately, no chance to fix. Empirically `retries=1-3` works well +- **Small model + deep nesting**: qwen2.5:3b may retry many times and still fail. Upgrade or flatten + +## Want smarter answers? + +```bash +MODEL=claude-sonnet-5 python starter_anthropic.py # highest one-shot rate +MODEL=qwen2.5:7b python starter.py # larger local model +``` + +## Extensions + +- **Add tools**: Pydantic AI agents can have tools + structured output simultaneously via `@agent.tool` +- **Stream typed output**: `agent.run_stream(...)` validates as it streams +- **Cross-model comparison**: same schema across Claude / GPT / Gemini / local — see who's most stable +- **Production wiring**: Pydantic AI integrates with FastAPI; output doubles as your API response model diff --git a/examples/stage-4/05-typed-agent/README.md b/examples/stage-4/05-typed-agent/README.md new file mode 100644 index 0000000..bdc6e8a --- /dev/null +++ b/examples/stage-4/05-typed-agent/README.md @@ -0,0 +1,146 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 練習 5：型別安全 agent（Pydantic AI structured output） + +對應 [Stage 4 — Agent Frameworks](../../../stages/04-agent-frameworks.md) 練習 5。 +> 🎓 **學習模式**：這份 `starter.py` 是**完整解答**、不是 TODO skeleton。建議用**主動模式**——`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫一份 `starter.py`、跑 `python test.py` 驗證；卡 20 分鐘再回去對照 reference。完整方法論看 [`docs/HOW_TO_USE.md`](../../../docs/HOW_TO_USE.md)。 + +> 📚 **想要 chapter-length 深入版？** 本 folder 的 starter 是 illustrative 版、聚焦核心 pattern + 兩條 SDK path，不是進階深度教材。深度教材推薦： +> - [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents) ⭐ 中文圈最完整、章節式 + 16 種 production 能力。**本練習對應 hello-agents 的 structured output / type-safe 章節** +> - [Pydantic AI 官方 docs](https://ai.pydantic.dev/) + [Instructor library](https://github.com/instructor-ai/instructor)（另一條 typed-output 路線） +> - 完整 references 見 [Stage 4 精選 Projects](../../../stages/04-agent-frameworks.md#-精選-projects) + + +## 任務 + +Agent 回問題、**強制** return `AnswerWithConfidence`： + +```python +class AnswerWithConfidence(BaseModel): + answer: str + confidence: float = Field(ge=0.0, le=1.0) # runtime 驗證 0-1 + sources: list[str] +``` + +Pydantic AI 把 schema validation 從 prompt 層（Stage 3 練習 6）**提升到 type 層**——LLM 不照 schema、framework 自動 retry。Production team 用這個防 hallucinate。 + +## 怎麼跑 — 兩條路徑 + +### Path A（默認、本機免費） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +預算：**$0**。但 qwen2.5:3b 可能 retry 多次才產對 schema、總 token 較高。 + +### Path B（Anthropic、想看 cloud 高品質） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +預算：每次 ≈ **$0.001**（claude-haiku-4-5、通常一次過、無 retry）。 + +## 不花錢驗證程式邏輯 + +```bash +python test.py # Pydantic schema 驗證 + agent 結構 +python test_anthropic.py # Path B 載入檢查 +``` + +`test.py` 直接驗 `AnswerWithConfidence` 對非法資料（confidence > 1.0、type 不對、sources 不是 list）的 ValidationError——不需要打 LLM、純 type 層測試。 + +## 為什麼 type-safe agent 重要 + +``` +Stage 3 練習 6：schema = JSON Schema in prompt + LLM 看到、但回什麼是 LLM 決定（可能違反） + +Stage 4 練習 5：schema = Pydantic model in code + LLM 違反 → framework 自動 raise → retry / 修 + 最終 output 一定 conform（runtime 保證） +``` + +對 production： + +| 需求 | 純 prompt schema | Pydantic AI | +|---|---|---| +| LLM 偶爾少欄位 | 你的下游 code 要 try/except | 自動 retry 直到符合 | +| 型別錯（confidence="high"） | 下游 crash | Pydantic ValidationError、retry | +| 邊界錯（confidence=1.5） | 下游用錯誤值 | 拒絕、retry | +| LLM hallucinate 多餘欄位 | 接收 silently | 預設 ignore（可調 strict） | + +**結論**：production agent 必用 type-safe output。Stage 3 練習 6 教 schema 設計、Stage 4 練習 5 教把 schema 變成 runtime contract。 + +## Pydantic AI 核心觀念 + +### Agent + output_type + +```python +agent = Agent( + model=..., + output_type=AnswerWithConfidence, # ← 強制 LLM 回這個 shape + system_prompt="..." +) +result = agent.run_sync(question) +answer: AnswerWithConfidence = result.output # 已驗證的物件 +``` + +**重點**：framework 在背後把 Pydantic schema 轉成 LLM 的 structured output instruction、執行 validation、failure 時 retry。 + +### Field constraints + +```python +confidence: float = Field(ge=0.0, le=1.0, description="...") +``` + +`ge` / `le` 是 Pydantic 的 numeric bound。LLM 回 `1.5` 會被 ValidationError 擋下、retry。 + +### 自動 retry + +```python +Agent(..., retries=3) # default 1，可調 +``` + +Pydantic AI 看到 ValidationError、會把錯誤訊息塞回 prompt、要求 LLM 重產。 + +## 兩個 path 觀察重點 + +| 觀察項 | Anthropic Claude haiku | Ollama qwen2.5:3b | +|---|---|---| +| 一次產對 schema | 90%+ | 50-70% | +| 平均 retry 次數 | 0-1 | 1-3 | +| confidence 邊界遵守 | 穩 | 偶爾 1.5 / 負值（會被 reject + retry） | +| sources 是 list | 穩 | 偶爾 string、被 reject | +| 總 token 成本 | 低（少 retry） | 高（多 retry） | + +**反直覺結論**：Path B（Claude）的實際 token cost **可能比 Path A（qwen）低**——retry 成本累計起來。Production team 算總帳會選大 model。 + +## 常見坑 + +- **`output_type` 太複雜**：nested model 深、LLM 難一次寫對。production 建議扁平化、≤5 個 top-level 欄位 +- **缺 `description`**：`Field(...)` 沒寫 `description=`、LLM 看不到欄位用途、易誤填 +- **`retries=0`**：失敗就 raise、不給 LLM 修的機會。經驗值 `retries=1-3` +- **小 model + 深 nested**：qwen2.5:3b 可能 retry 多次仍不對。換大 model 或扁平 schema + +## 想看更聰明的答案？ + +```bash +MODEL=claude-sonnet-5 python starter_anthropic.py # 一次過機率最高 +MODEL=qwen2.5:7b python starter.py # 較大本機 model +``` + +## 延伸 + +- **加 tools**：Pydantic AI agent 可以同時有 tools + structured output、`@agent.tool` 裝飾函式 +- **stream typed output**：`agent.run_stream(...)` 邊跑邊驗 +- **跨 model 比較**：同一個 schema 跑 Claude / GPT / Gemini / 本機 model、看誰最穩 +- **接 production**：Pydantic AI 跟 FastAPI 整合很好、output 直接當 API response model diff --git a/examples/stage-4/05-typed-agent/README.zh-Hans.md b/examples/stage-4/05-typed-agent/README.zh-Hans.md new file mode 100644 index 0000000..3fa4090 --- /dev/null +++ b/examples/stage-4/05-typed-agent/README.zh-Hans.md @@ -0,0 +1,139 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 练习 5：型别安全 agent（Pydantic AI structured output） + +对应 [Stage 4 — Agent Frameworks](../../../stages/04-agent-frameworks.zh-Hans.md) 练习 5。 + +## 任务 + +Agent 回问题、**强制** return `AnswerWithConfidence`： + +```python +class AnswerWithConfidence(BaseModel): + answer: str + confidence: float = Field(ge=0.0, le=1.0) # runtime 验证 0-1 + sources: list[str] +``` + +Pydantic AI 把 schema validation 从 prompt 层（Stage 3 练习 6）**提升到 type 层**——LLM 不照 schema、framework 自动 retry。Production team 用这个防 hallucinate。 + +## 怎么跑 — 两条路径 + +### Path A（默认、本机免费） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +预算：**$0**。但 qwen2.5:3b 可能 retry 多次才产对 schema、总 token 较高。 + +### Path B（Anthropic、想看 cloud 高质量） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +预算：每次 ≈ **$0.001**（claude-haiku-4-5、通常一次过、无 retry）。 + +## 不花钱验证程序逻辑 + +```bash +python test.py # Pydantic schema 验证 + agent 结构 +python test_anthropic.py # Path B 载入检查 +``` + +`test.py` 直接验 `AnswerWithConfidence` 对非法数据（confidence > 1.0、type 不对、sources 不是 list）的 ValidationError——不需要打 LLM、纯 type 层测试。 + +## 为什么 type-safe agent 重要 + +``` +Stage 3 练习 6：schema = JSON Schema in prompt + LLM 看到、但回什么是 LLM 决定（可能违反） + +Stage 4 练习 5：schema = Pydantic model in code + LLM 违反 → framework 自动 raise → retry / 修 + 最终 output 一定 conform（runtime 保证） +``` + +对 production： + +| 需求 | 纯 prompt schema | Pydantic AI | +|---|---|---| +| LLM 偶尔少字段 | 你的下游 code 要 try/except | 自动 retry 直到符合 | +| 类型错（confidence="high"） | 下游 crash | Pydantic ValidationError、retry | +| 边界错（confidence=1.5） | 下游用错误值 | 拒绝、retry | +| LLM hallucinate 多余字段 | 接收 silently | 预设 ignore（可调 strict） | + +**结论**：production agent 必用 type-safe output。Stage 3 练习 6 教 schema 设计、Stage 4 练习 5 教把 schema 变成 runtime contract。 + +## Pydantic AI 核心观念 + +### Agent + output_type + +```python +agent = Agent( + model=..., + output_type=AnswerWithConfidence, # ← 强制 LLM 回这个 shape + system_prompt="..." +) +result = agent.run_sync(question) +answer: AnswerWithConfidence = result.output # 已验证的物件 +``` + +**重点**：framework 在背后把 Pydantic schema 转成 LLM 的 structured output instruction、执行 validation、failure 时 retry。 + +### Field constraints + +```python +confidence: float = Field(ge=0.0, le=1.0, description="...") +``` + +`ge` / `le` 是 Pydantic 的 numeric bound。LLM 回 `1.5` 会被 ValidationError 挡下、retry。 + +### 自动 retry + +```python +Agent(..., retries=3) # default 1，可调 +``` + +Pydantic AI 看到 ValidationError、会把错误讯息塞回 prompt、要求 LLM 重产。 + +## 两个 path 观察重点 + +| 观察项 | Anthropic Claude haiku | Ollama qwen2.5:3b | +|---|---|---| +| 一次产对 schema | 90%+ | 50-70% | +| 平均 retry 次数 | 0-1 | 1-3 | +| confidence 边界遵守 | 稳 | 偶尔 1.5 / 负值（会被 reject + retry） | +| sources 是 list | 稳 | 偶尔 string、被 reject | +| 总 token 成本 | 低（少 retry） | 高（多 retry） | + +**反直觉结论**：Path B（Claude）的实际 token cost **可能比 Path A（qwen）低**——retry 成本累计起来。Production team 算总帐会选大 model。 + +## 常见坑 + +- **`output_type` 太复杂**：nested model 深、LLM 难一次写对。production 建议扁平化、≤5 个 top-level 字段 +- **缺 `description`**：`Field(...)` 没写 `description=`、LLM 看不到字段用途、易误填 +- **`retries=0`**：失败就 raise、不给 LLM 修的机会。经验值 `retries=1-3` +- **小 model + 深 nested**：qwen2.5:3b 可能 retry 多次仍不对。换大 model 或扁平 schema + +## 想看更聪明的答案？ + +```bash +MODEL=claude-sonnet-5 python starter_anthropic.py # 一次过机率最高 +MODEL=qwen2.5:7b python starter.py # 较大本机 model +``` + +## 延伸 + +- **加 tools**：Pydantic AI agent 可以同时有 tools + structured output、`@agent.tool` 装饰函数 +- **stream typed output**：`agent.run_stream(...)` 边跑边验 +- **跨 model 比较**：同一个 schema 跑 Claude / GPT / Gemini / 本机 model、看谁最稳 +- **接 production**：Pydantic AI 跟 FastAPI 整合很好、output 直接当 API response model diff --git a/examples/stage-4/05-typed-agent/requirements.txt b/examples/stage-4/05-typed-agent/requirements.txt new file mode 100644 index 0000000..e59c4ec --- /dev/null +++ b/examples/stage-4/05-typed-agent/requirements.txt @@ -0,0 +1,2 @@ +pydantic-ai>=0.0.20,<2.0 +pydantic>=2.6,<3.0 diff --git a/examples/stage-4/05-typed-agent/starter.py b/examples/stage-4/05-typed-agent/starter.py new file mode 100644 index 0000000..45edc45 --- /dev/null +++ b/examples/stage-4/05-typed-agent/starter.py @@ -0,0 +1,87 @@ +"""Stage 4 練習 5：型別安全 agent — Pydantic AI + Ollama（Path A、默認）。 + +Pydantic AI 把「structured output」變成型別系統的一級成員： +- Agent 回傳必須 conform to Pydantic model +- runtime validation 自動跑（譬如 confidence 必須是 float、不是 string） +- 失敗時 LLM 會被告知 schema、retry 再 produce + +任務：問問題、agent 回 `{"answer": str, "confidence": float, "sources": [str]}`。 + +跑法： + pip install -r requirements.txt + ollama pull qwen2.5:3b + ollama serve + python starter.py + +驗證： + python test.py + +⚠️ 注意：Pydantic AI 對 model 的 instruction following 要求高。 +qwen2.5:3b 可能多 retry 才產對 schema、Claude haiku 通常一次過。 +""" + +from __future__ import annotations + +import os +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from pydantic import BaseModel, Field +from pydantic_ai import Agent +from pydantic_ai.models.openai import OpenAIModel +from pydantic_ai.providers.openai import OpenAIProvider + +MODEL = os.environ.get("MODEL", "qwen2.5:3b") +OLLAMA_BASE = os.environ.get("OLLAMA_API_BASE", "http://localhost:11434/v1") + + +# === Schema：agent 強制要回這個 shape === + +class AnswerWithConfidence(BaseModel): + """Structured answer the agent must produce.""" + answer: str = Field(description="The actual answer text.") + confidence: float = Field(ge=0.0, le=1.0, description="Confidence score 0-1.") + sources: list[str] = Field(description="Sources or references used.") + + +def build_agent(model: Any = None) -> Agent: + if model is None: + model = OpenAIModel( + MODEL, + provider=OpenAIProvider(base_url=OLLAMA_BASE, api_key="ollama"), + ) + return Agent( + model=model, + output_type=AnswerWithConfidence, + system_prompt=( + "You answer questions. ALWAYS return a structured answer with " + "an 'answer' text, a 'confidence' float (0.0 to 1.0), and a list of 'sources'. " + "If you don't know, set confidence low and explain in answer." + ), + ) + + +def run(question: str, model: Any = None) -> AnswerWithConfidence: + agent = build_agent(model=model) + result = agent.run_sync(question) + return result.output + + +if __name__ == "__main__": + question = "What is the population of Taipei?" + print(f"❓ Q: {question}（using Pydantic AI + Ollama {MODEL}）") + print("-" * 60) + answer = run(question) + print(f" answer: {answer.answer}") + print(f" confidence: {answer.confidence}") + print(f" sources: {answer.sources}") + + # Pydantic 已經保證 type 正確、這裡只 sanity check + assert isinstance(answer.confidence, float) + assert 0.0 <= answer.confidence <= 1.0 + assert isinstance(answer.sources, list) + print("\n✅ 練習 5 通過 — Pydantic AI 強制 structured output、$0/run") + print(" schema validation 在 runtime 自動跑、LLM 不能偷懶") diff --git a/examples/stage-4/05-typed-agent/starter_anthropic.py b/examples/stage-4/05-typed-agent/starter_anthropic.py new file mode 100644 index 0000000..781fd65 --- /dev/null +++ b/examples/stage-4/05-typed-agent/starter_anthropic.py @@ -0,0 +1,39 @@ +"""Stage 4 練習 5：型別安全 agent — Pydantic AI + Anthropic（Path B）。 + +Pydantic AI 對 Anthropic 是一級支援、用 'anthropic:claude-haiku-4-5' 字串即可建 model。 + +跑法： + pip install -r requirements.txt + export ANTHROPIC_API_KEY=sk-ant-... + python starter_anthropic.py + +預算：每次 ≈ $0.001（claude-haiku-4-5）。Claude 對 structured output 穩很多—— +schema validation retry 機率小、實際 token 成本通常比 qwen2.5:3b 還少。 +""" + +from __future__ import annotations + +import os +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from pydantic_ai.models.anthropic import AnthropicModel + +from starter import run + +MODEL = os.environ.get("MODEL", "claude-haiku-4-5") + + +if __name__ == "__main__": + question = "What is the population of Taipei?" + print(f"❓ Q: {question}（using Pydantic AI + Anthropic {MODEL}）") + print("-" * 60) + model = AnthropicModel(MODEL) + answer = run(question, model=model) + print(f" answer: {answer.answer}") + print(f" confidence: {answer.confidence}") + print(f" sources: {answer.sources}") + assert isinstance(answer.confidence, float) + print("\n✅ 練習 5 (Anthropic path) 通過 — Claude structured output 穩、≈$0.001/run") diff --git a/examples/stage-4/05-typed-agent/test.py b/examples/stage-4/05-typed-agent/test.py new file mode 100644 index 0000000..c625fe1 --- /dev/null +++ b/examples/stage-4/05-typed-agent/test.py @@ -0,0 +1,84 @@ +"""Stage 4 練習 5 自我驗證 — Pydantic AI schema 邏輯 + 型別約束。 + +跑法： + python test.py + +驗證內容： + - AnswerWithConfidence schema：必填欄位、confidence 邊界、type 約束 + - Pydantic 對非法資料會 raise ValidationError + - build_agent 接受 mock model、Agent 物件結構正確 +""" + +from __future__ import annotations + +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import pytest +from pydantic import ValidationError + +from starter import AnswerWithConfidence, build_agent + + +def test_valid_answer_constructs(): + a = AnswerWithConfidence(answer="Taipei pop is 2.6M", confidence=0.9, sources=["wiki"]) + assert a.confidence == 0.9 + assert a.sources == ["wiki"] + print("✅ test_valid_answer_constructs") + + +def test_confidence_out_of_range_rejected(): + """confidence > 1.0 應該被 Pydantic 拒絕。""" + try: + AnswerWithConfidence(answer="...", confidence=1.5, sources=[]) + except ValidationError: + print("✅ test_confidence_out_of_range_rejected") + return + raise AssertionError("expected ValidationError for confidence=1.5") + + +def test_confidence_must_be_float(): + """confidence 傳 string 應被 Pydantic 拒絕（schema validation 的核心）。""" + try: + AnswerWithConfidence(answer="...", confidence="high", sources=[]) # type: ignore + except ValidationError: + print("✅ test_confidence_must_be_float") + return + raise AssertionError("expected ValidationError for confidence='high'") + + +def test_sources_must_be_list(): + try: + AnswerWithConfidence(answer="...", confidence=0.5, sources="wiki") # type: ignore + except ValidationError: + print("✅ test_sources_must_be_list") + return + raise AssertionError("expected ValidationError for sources='wiki'") + + +def test_build_agent_ok(): + """build_agent 帶 mock model 可建構、output_type 正確。""" + class FakeModel: + pass + try: + agent = build_agent(model=FakeModel()) # pydantic-ai 不會在這裡打 API + except Exception as e: + # 某些版本 build 時就 validate model、放寬：能 import + AnswerWithConfidence 是 output_type 即可 + from starter import AnswerWithConfidence as AWC + print(f"⚠ build_agent 在某些版本需要 real model（{type(e).__name__}）、僅驗 schema 結構") + assert AWC.model_fields # pydantic v2 API + print("✅ test_build_agent_ok (schema-only check)") + return + assert agent is not None + print("✅ test_build_agent_ok") + + +if __name__ == "__main__": + test_valid_answer_constructs() + test_confidence_out_of_range_rejected() + test_confidence_must_be_float() + test_sources_must_be_list() + test_build_agent_ok() + print("\n🎉 全部通過 — Pydantic schema validation 邏輯正確") diff --git a/examples/stage-4/05-typed-agent/test_anthropic.py b/examples/stage-4/05-typed-agent/test_anthropic.py new file mode 100644 index 0000000..56eb03b --- /dev/null +++ b/examples/stage-4/05-typed-agent/test_anthropic.py @@ -0,0 +1,30 @@ +"""Stage 4 練習 5 — Path B 載入檢查。 + +實測請跑 starter_anthropic.py 配 ANTHROPIC_API_KEY。 +""" + +from __future__ import annotations + +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + + +def test_anthropic_model_importable(): + from pydantic_ai.models.anthropic import AnthropicModel + assert AnthropicModel is not None + print("✅ test_anthropic_model_importable") + + +def test_starter_anthropic_loadable(): + import starter_anthropic + assert hasattr(starter_anthropic, "MODEL") + assert "claude" in starter_anthropic.MODEL + print("✅ test_starter_anthropic_loadable") + + +if __name__ == "__main__": + test_anthropic_model_importable() + test_starter_anthropic_loadable() + print("\n🎉 通過 — Path B 可載入（實測需 ANTHROPIC_API_KEY）") diff --git a/examples/stage-5/tool-calling-tutor/README.en.md b/examples/stage-5/tool-calling-tutor/README.en.md new file mode 100644 index 0000000..6a05ddb --- /dev/null +++ b/examples/stage-5/tool-calling-tutor/README.en.md @@ -0,0 +1,137 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# tool-calling-tutor — Claude Code skill + +> What this skill does: when you're stuck on tool calling (LLM won't call, args wrong, ReAct loop runs forever, don't know how to write a schema), it auto-loads to walk you through a 4-symptom diagnostic + 5-step fix. + +Pairs with [Stage 3 — Tool Use & Agent Intro](../../../stages/03-tool-use-and-hello-agent.en.md). Also serves as the **bundled skill example** for [Stage 5 — Claude Code Ecosystem](../../../stages/05-claude-code-ecosystem.en.md) 5.3. + +## Why this skill exists + +Tool calling is the steepest learning curve in the curriculum — schema design + SDK response shape + ReAct loop are three mental models stacked at once. The Stage 3 doc covers the concepts, but **when you hit "this just doesn't work", you need interactive debugging**. + +This skill fills that gap: + +| Existing resource | Limit | What this skill adds | +|---|---|---| +| `stages/03-tool-use-and-hello-agent.en.md` | Covers 6 exercises, not interactive | Interactive triage: which symptom are you stuck on? | +| `resources/schema-design-cheatsheet.en.md` | 5 rules + 5 anti-patterns, prescriptive | Procedural: bad → good schema in 4 step-by-step iterations | +| `resources/glossary.en.md` 2 | 1-line definitions | Doesn't redefine, references | +| `examples/stage-3/02-06/` | Full runnable starters | Skill points at them as fork templates | + +## Dual purpose + +1. **For learners**: install as a personal debug assistant. When you prompt Claude Code with "why won't the LLM call my tool", the skill auto-loads and walks the 4-symptom diagnostic. +2. **As a Stage 5 5.3 meta-example**: when learning to write SKILL.md, study this one directly. Includes full frontmatter (with trigger phrases + Do NOT use for), `references/` design, and `evals/evals.json` example. + +## Install (30 seconds) + +### Option A: user-level (shared across all projects) + +```bash +mkdir -p ~/.claude/skills/tool-calling-tutor +cp SKILL.md ~/.claude/skills/tool-calling-tutor/ +cp -r references evals ~/.claude/skills/tool-calling-tutor/ +``` + +English speakers: `cp translations/SKILL.en.md ~/.claude/skills/tool-calling-tutor/SKILL.md` (canonical is zh-TW). +Simplified Chinese: `cp translations/SKILL.zh-Hans.md ~/.claude/skills/tool-calling-tutor/SKILL.md` + +### Option B: project-level (only triggers in this repo) + +```bash +mkdir -p .claude/skills/tool-calling-tutor +cp SKILL.md references/ evals/ .claude/skills/tool-calling-tutor/ +``` + +### Verify the install + +Restart Claude Code, then prompt: + +``` +Why won't the LLM call my tool? +``` + +Expected: Claude auto-loads the skill, first asks "are you on symptom (a)/(b)/(c)/(d)", then branches to the matching reference. + +## What's inside + +``` +tool-calling-tutor/ +├── SKILL.md # main skill file (zh-TW canonical) +├── README.md / .en.md / .zh-Hans.md # this file +├── references/ +│ ├── debug-flowchart.md # 4-symptom diagnostic +│ ├── schema-evolution.md # bad → good schema worked example +│ └── sdk-diff.md # Anthropic vs OpenAI-compat side-by-side +│ (each has .en.md / .zh-Hans.md translations) +├── translations/ +│ ├── SKILL.en.md # English version of SKILL.md +│ └── SKILL.zh-Hans.md # Simplified Chinese version +└── evals/ + └── evals.json # 5 test cases (promptfoo or manual) +``` + +## Run evals (optional) + +```bash +# Without promptfoo: just read evals/evals.json, paste each input into Claude, compare with expected_behavior +cat evals/evals.json +``` + +Batch with [promptfoo](https://github.com/promptfoo/promptfoo): + +```bash +npm install -g promptfoo +promptfoo eval -c evals/evals.json +``` + +## Relationship to other resources + +``` + ┌─────────────────────────────────┐ + │ Stage 3 doc + inline 練習 1-6 │ + │ (learn tool-calling concepts) │ + └────────────────┬─────────────────┘ + │ + ▼ + ┌─────────────────────────────────┐ + │ examples/stage-3/02-06/ │ + │ (full runnable starters) │ + └────────────────┬─────────────────┘ + │ fork template + ▼ + ┌─────────────────────────────────┐ + │ your tool-calling agent │ + │ ❓ stuck │ + └────────────────┬─────────────────┘ + │ skill loads + ▼ + ┌─────────────────────────────────┐ + │ tool-calling-tutor skill (this) │ + │ → 4-symptom triage │ + │ → references/ deep dive │ + │ → route to cookbook / Stage 4/7 │ + └─────────────────────────────────┘ +``` + +## What this skill does NOT handle + +| Situation | Route to | +|---|---| +| LangChain / LangGraph / CrewAI / Pydantic AI | Stage 4 | +| Building MCP server / client | `resources/cookbook.en.md` 2 | +| Production observability / cost tracking | Stage 7 | +| General prompt engineering | Stage 2 | + +## Extensions + +- **Customize trigger phrases**: add your own catch phrases to SKILL.md frontmatter `description` +- **Add your cases to references/**: open new sections in debug-flowchart for weird cases you've hit +- **Fork it**: this skill is designed as a Stage 5 5.3 meta-example — forking welcome + +## License + +Same as repo (MIT). Free to rewrite, fork, use commercially. diff --git a/examples/stage-5/tool-calling-tutor/README.md b/examples/stage-5/tool-calling-tutor/README.md new file mode 100644 index 0000000..d47e468 --- /dev/null +++ b/examples/stage-5/tool-calling-tutor/README.md @@ -0,0 +1,138 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# tool-calling-tutor — Claude Code skill + +> Skill 用途：當你卡在 tool calling（LLM 不呼叫、args 錯、ReAct loop 跑不停、schema 不知道怎麼寫），自動跳出來幫你 4-symptom 診斷 + 5-step 修法走查。 + +對應 [Stage 3 — Tool Use & Agent 入門](../../../stages/03-tool-use-and-hello-agent.md)，同時是 [Stage 5 — Claude Code Ecosystem](../../../stages/05-claude-code-ecosystem.md) 5.3 的**自帶 skill 範例**。 + +## 為什麼這個 skill 存在 + +Tool calling 是整個 curriculum 最陡的學習曲線——schema 設計、SDK response shape、ReAct loop 三個 mental model 疊在一起。Stage 3 doc 已經把概念講清楚，但**遇到「我這份就是不會跑」的時候、需要互動式 debug**。 + +這個 skill 補的就是這塊缺： + +| 已有資源 | 不足 | 這個 skill 補的 | +|---|---|---| +| `stages/03-tool-use-and-hello-agent.md` | 講 6 個練習、不互動 | 互動式 triage：你卡哪個 symptom？ | +| `resources/schema-design-cheatsheet.md` | 5 條規則 + 5 anti-pattern、prescriptive | 走步驟版：bad → good schema 怎麼 4 步改 | +| `resources/glossary.md` 2 | 1 行定義 | 不重複定義、引用為主 | +| `examples/stage-3/02-06/` | 完整可跑 starter | Skill 指過去當 fork template | + +## 雙重用途 + +1. **學習者用**：安裝後當 personal debug 助手。當你 prompt Claude Code「為什麼 LLM 不呼叫我的 tool」、skill 自動載入、走 4-symptom 診斷。 +2. **Stage 5 5.3 meta-example**：學 SKILL.md 怎麼寫的時候，直接看這份。包含完整 frontmatter（含 trigger phrases + Do NOT use for）、`references/` 設計、`evals/evals.json` 範例。 + +## 怎麼安裝（30 秒） + +### Option A：user 級（所有 project 共用） + +```bash +mkdir -p ~/.claude/skills/tool-calling-tutor +cp SKILL.md ~/.claude/skills/tool-calling-tutor/ +cp -r references evals ~/.claude/skills/tool-calling-tutor/ +``` + +繁體中文使用者：直接用 `SKILL.md`（canonical 是 zh-TW）。 +簡體中文：`cp translations/SKILL.zh-Hans.md ~/.claude/skills/tool-calling-tutor/SKILL.md` +English：`cp translations/SKILL.en.md ~/.claude/skills/tool-calling-tutor/SKILL.md` + +### Option B：project 級（只在這個 repo 觸發） + +```bash +mkdir -p .claude/skills/tool-calling-tutor +cp SKILL.md references/ evals/ .claude/skills/tool-calling-tutor/ +``` + +### 驗證安裝 + +重啟 Claude Code、然後 prompt： + +``` +為什麼 LLM 不呼叫我的 tool？ +``` + +預期：Claude 自動載入 skill、先問你「是 (a)/(b)/(c)/(d) 哪個 symptom」、然後 branch 到對應 reference。 + +## 包含什麼 + +``` +tool-calling-tutor/ +├── SKILL.md # 主 skill 檔（zh-TW canonical） +├── README.md / .en.md / .zh-Hans.md # 你正在看的這份 +├── references/ +│ ├── debug-flowchart.md # 4-symptom 診斷流程 +│ ├── schema-evolution.md # bad → good schema 4-step worked example +│ └── sdk-diff.md # Anthropic vs OpenAI-compat 並排對照 +│ （以上每份都有 .en.md / .zh-Hans.md 翻譯） +├── translations/ +│ ├── SKILL.en.md # SKILL.md 英文版（給英語使用者裝） +│ └── SKILL.zh-Hans.md # SKILL.md 簡體版 +└── evals/ + └── evals.json # 5 個 test cases（promptfoo / 手動皆可） +``` + +## 跑 evals（選擇性） + +```bash +# 不裝 promptfoo 也可以、直接眼看 evals/evals.json 的 input 拿去問 Claude、對照 expected_behavior +cat evals/evals.json +``` + +如果要 batch 跑、裝 [promptfoo](https://github.com/promptfoo/promptfoo)： + +```bash +npm install -g promptfoo +promptfoo eval -c evals/evals.json +``` + +## 跟其他資源的關係 + +``` + ┌─────────────────────────────────┐ + │ Stage 3 doc + 練習 1-6 inline │ + │ (學 tool calling 概念) │ + └────────────────┬─────────────────┘ + │ + ▼ + ┌─────────────────────────────────┐ + │ examples/stage-3/02-06/ │ + │ (完整可跑 starter + test) │ + └────────────────┬─────────────────┘ + │ fork template + ▼ + ┌─────────────────────────────────┐ + │ 你的 tool-calling agent │ + │ ❓ 卡住了 │ + └────────────────┬─────────────────┘ + │ 載入 + ▼ + ┌─────────────────────────────────┐ + │ tool-calling-tutor skill (這個) │ + │ → 4-symptom triage │ + │ → references/ deep dive │ + │ → 路由到 cookbook / Stage 4/7 │ + └─────────────────────────────────┘ +``` + +## 不處理什麼 + +| 情境 | 路 | +|---|---| +| LangChain / LangGraph / CrewAI / Pydantic AI | Stage 4 | +| 寫 MCP server / client | `resources/cookbook.md` 2 | +| Production observability / cost tracking | Stage 7 | +| 一般 prompt engineering | Stage 2 | + +## 延伸 + +- **改 trigger phrases**：在 SKILL.md frontmatter `description` 加你自己常用的觸發句 +- **加你的 case 到 references/**：debug-flowchart 裡開新 Section、把你碰到的 weird case 記下來 +- **fork 成你的版本**：這個 skill 設計就是 Stage 5 5.3 的 meta-example、歡迎 fork + +## License + +跟 repo 一致（MIT）。Skill body 改寫、fork、商用都 OK。 diff --git a/examples/stage-5/tool-calling-tutor/README.zh-Hans.md b/examples/stage-5/tool-calling-tutor/README.zh-Hans.md new file mode 100644 index 0000000..a6973c9 --- /dev/null +++ b/examples/stage-5/tool-calling-tutor/README.zh-Hans.md @@ -0,0 +1,138 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# tool-calling-tutor — Claude Code skill + +> Skill 用途：当你卡在 tool calling（LLM 不调用、args 错、ReAct loop 跑不停、schema 不知道怎么写），自动跳出来帮你 4-symptom 诊断 + 5-step 修法走查。 + +对应 [Stage 3 — Tool Use & Agent 入门](../../../stages/03-tool-use-and-hello-agent.zh-Hans.md)，同时是 [Stage 5 — Claude Code Ecosystem](../../../stages/05-claude-code-ecosystem.zh-Hans.md) 5.3 的**自带 skill 范例**。 + +## 为什么这个 skill 存在 + +Tool calling 是整个 curriculum 最陡的学习曲线——schema 设计、SDK response shape、ReAct loop 三个 mental model 叠在一起。Stage 3 doc 已经把概念讲清楚，但**遇到“我这份就是不会跑”的时候、需要互动式 debug**。 + +这个 skill 补的就是这块缺： + +| 已有资源 | 不足 | 这个 skill 补的 | +|---|---|---| +| `stages/03-tool-use-and-hello-agent.zh-Hans.md` | 讲 6 个练习、不互动 | 互动式 triage：你卡哪个 symptom？ | +| `resources/schema-design-cheatsheet.zh-Hans.md` | 5 条规则 + 5 anti-pattern、prescriptive | 走步骤版：bad → good schema 怎么 4 步改 | +| `resources/glossary.zh-Hans.md` 2 | 1 行定义 | 不重复定义、引用为主 | +| `examples/stage-3/02-06/` | 完整可跑 starter | Skill 指过去当 fork template | + +## 双重用途 + +1. **学习者用**：安装后当 personal debug 助手。当你 prompt Claude Code“为什么 LLM 不调用我的 tool”、skill 自动载入、走 4-symptom 诊断。 +2. **Stage 5 5.3 meta-example**：学 SKILL.md 怎么写的时候，直接看这份。包含完整 frontmatter（含 trigger phrases + Do NOT use for）、`references/` 设计、`evals/evals.json` 范例。 + +## 怎么安装（30 秒） + +### Option A：user 级（所有 project 共用） + +```bash +mkdir -p ~/.claude/skills/tool-calling-tutor +cp SKILL.md ~/.claude/skills/tool-calling-tutor/ +cp -r references evals ~/.claude/skills/tool-calling-tutor/ +``` + +简体中文用户：`cp translations/SKILL.zh-Hans.md ~/.claude/skills/tool-calling-tutor/SKILL.md`（canonical 是 zh-TW）。 +繁体中文：直接用 `SKILL.md`。 +English：`cp translations/SKILL.en.md ~/.claude/skills/tool-calling-tutor/SKILL.md` + +### Option B：project 级（只在这个 repo 触发） + +```bash +mkdir -p .claude/skills/tool-calling-tutor +cp SKILL.md references/ evals/ .claude/skills/tool-calling-tutor/ +``` + +### 验证安装 + +重启 Claude Code、然后 prompt： + +``` +为什么 LLM 不调用我的 tool？ +``` + +预期：Claude 自动载入 skill、先问你“是 (a)/(b)/(c)/(d) 哪个 symptom”、然后 branch 到对应 reference。 + +## 包含什么 + +``` +tool-calling-tutor/ +├── SKILL.md # 主 skill 档（zh-TW canonical） +├── README.md / .en.md / .zh-Hans.md # 你正在看的这份 +├── references/ +│ ├── debug-flowchart.md # 4-symptom 诊断流程 +│ ├── schema-evolution.md # bad → good schema 4-step worked example +│ └── sdk-diff.md # Anthropic vs OpenAI-compat 并排对照 +│ （以上每份都有 .en.md / .zh-Hans.md 翻译） +├── translations/ +│ ├── SKILL.en.md # SKILL.md 英文版（给英语用户装） +│ └── SKILL.zh-Hans.md # SKILL.md 简体版 +└── evals/ + └── evals.json # 5 个 test cases（promptfoo / 手动皆可） +``` + +## 跑 evals（选择性） + +```bash +# 不装 promptfoo 也可以、直接眼看 evals/evals.json 的 input 拿去问 Claude、对照 expected_behavior +cat evals/evals.json +``` + +如果要 batch 跑、装 [promptfoo](https://github.com/promptfoo/promptfoo)： + +```bash +npm install -g promptfoo +promptfoo eval -c evals/evals.json +``` + +## 跟其他资源的关系 + +``` + ┌─────────────────────────────────┐ + │ Stage 3 doc + 练习 1-6 inline │ + │ (学 tool calling 概念) │ + └────────────────┬─────────────────┘ + │ + ▼ + ┌─────────────────────────────────┐ + │ examples/stage-3/02-06/ │ + │ (完整可跑 starter + test) │ + └────────────────┬─────────────────┘ + │ fork template + ▼ + ┌─────────────────────────────────┐ + │ 你的 tool-calling agent │ + │ ❓ 卡住了 │ + └────────────────┬─────────────────┘ + │ 载入 + ▼ + ┌─────────────────────────────────┐ + │ tool-calling-tutor skill (这个) │ + │ → 4-symptom triage │ + │ → references/ deep dive │ + │ → 路由到 cookbook / Stage 4/7 │ + └─────────────────────────────────┘ +``` + +## 不处理什么 + +| 情境 | 路 | +|---|---| +| LangChain / LangGraph / CrewAI / Pydantic AI | Stage 4 | +| 写 MCP server / client | `resources/cookbook.zh-Hans.md` 2 | +| Production observability / cost tracking | Stage 7 | +| 一般 prompt engineering | Stage 2 | + +## 延伸 + +- **改 trigger phrases**：在 SKILL.md frontmatter `description` 加你自己常用的触发句 +- **加你的 case 到 references/**：debug-flowchart 里开新 Section、把你碰到的 weird case 记下来 +- **fork 成你的版本**：这个 skill 设计就是 Stage 5 5.3 的 meta-example、欢迎 fork + +## License + +跟 repo 一致（MIT）。Skill body 改写、fork、商用都 OK。 diff --git a/examples/stage-5/tool-calling-tutor/SKILL.md b/examples/stage-5/tool-calling-tutor/SKILL.md new file mode 100644 index 0000000..78808a3 --- /dev/null +++ b/examples/stage-5/tool-calling-tutor/SKILL.md @@ -0,0 +1,107 @@ +--- +name: tool-calling-tutor +description: When the user is building a tool-calling agent and gets stuck — "為什麼 LLM 不呼叫我的 tool", "我這 schema 哪裡寫壞", "tool 被呼叫但 args 不對", "ReAct loop 跑不停", "the LLM won't call my tool", "help me design a function schema", "debug this tool-use behavior". Walks them through a 4-branch diagnostic + 5-step schema design walkthrough, with references to bad/good schema A/B and SDK-diff cheatsheet. Do NOT use for: pure LangChain / LangGraph / CrewAI framework questions (route to Stage 4 frameworks), MCP server building (route to cookbook 2), production agent observability (route to Stage 7). +--- + +# Tool Calling Tutor + +You are now in the **tool-calling debugging** context. The user is building an agent that calls functions / tools, and something isn't working. Your job is to walk them through diagnosis + fix, not to write code for them. + +## Step 1 — Triage（first thing you do） + +When the user mentions tool calling problems, ask **which of these 4 symptoms** they're hitting (one question, multiple choice): + +1. **(a) LLM 不呼叫我的 tool** — 模型直接用自然語言回答、完全沒觸發 tool_calls +2. **(b) Tool 被呼叫、但參數錯** — 呼叫對 tool，但 `arguments` 不對（型別錯、缺欄位、值不合理） +3. **(c) ReAct loop 跑不停 / 漏步** — 多步 loop 無限循環，或者中間漏一個 tool 沒呼叫 +4. **(d) 我從零開始、還沒寫 schema** — 用戶要新做一個 tool、想知道 schema 怎麼設計 + +**不要猜**——讓使用者明確選一個。每個 branch 走的 reference 不同。 + +## Step 2 — Branch by symptom + +### (a) LLM 不呼叫 tool → 看 description 與工具邊界 + +最常見 3 個原因（按優先順序問）： + +1. **`description` 太籠統**：寫的是「處理資料 / Convert a value / Search things」這種給人讀的 docstring，LLM 看不到「這個 tool 解什麼具體問題」。看 [`references/debug-flowchart.md`](references/debug-flowchart.md) Section A。 +2. **多 tool 邊界互相重疊**：兩個 tool 的 description 都能套到 user query、LLM 選不出來、乾脆都不選。 +3. **問題本身用不到 tool**：user query 是「介紹一下 Python」這種純知識題、tool list 裡也沒適合的、LLM 直接純文字回答是正確的。 + +**怎麼修**：把 `description` 從「**做什麼**」改寫成「**何時用**」。對照 [`references/schema-evolution.md`](references/schema-evolution.md) 的 bad → good A/B。 + +### (b) Tool 被呼叫、但參數錯 → 看 parameters schema + +最常見 3 個原因： + +1. **參數型別全用 `string`**：`{"value": {"type": "string"}}` LLM 不知道要傳 number。改成 `{"type": "number"}`。 +2. **沒有 `required`**：模型可能漏傳必填欄位。明列 `"required": ["value", "unit"]`。 +3. **enum 該用沒用**：`unit: string` 讓 LLM 傳 `"C"` `"Celsius"` `"celsius"` 都有可能。改 `"enum": ["celsius", "fahrenheit"]`。 + +**對照** [`references/schema-evolution.md`](references/schema-evolution.md) 的 4 個改進。 + +### (c) ReAct loop 跑不停 / 漏步 → 看 control flow + +跑不停的 3 個典型原因： + +1. **忘記把 assistant response 加回 `messages`**——下輪 LLM 看不到自己上輪講過什麼、會無限重複 +2. **`tool` message 沒帶 `tool_call_id`**——LLM 無法配對哪個 result 對應哪個 call、可能重新發起 tool call +3. **沒設 `max_iter` safety net**——當 tool 結果寫得不好、LLM 會無限呼叫 + +漏步（多步任務中間少一步）的原因： + +1. **Model 不夠強**：qwen2.5:3b 在 4-step task 上會漏「轉百分比」這種子步驟。試 `MODEL=qwen2.5:7b` 或 `MODEL=claude-haiku-4-5`。 +2. **Tool description 沒講「必要前置」**：譬如 `to_percentage` 應該寫「Convert a ratio (e.g., 0.31) into percentage. Call this LAST after dividing.」明示順序。 + +**對照可跑範例** → [`../../stage-3/03-react-from-scratch/`](../../stage-3/03-react-from-scratch/) 跟 [`../../stage-3/04-multi-step-reasoning/`](../../stage-3/04-multi-step-reasoning/) 的完整 starter。 + +### (d) 從零設計 schema → 走 5 步法 + +對任何新 tool，按這 5 步： + +1. **Define**：一句話講這個 tool 做什麼（不超過 15 字）。寫不出來 = tool scope 太大、要拆。 +2. **Describe（LLM 視角）**：把 description 寫成「**Use this when the user asks to / mentions / wants** ...」格式，不是「This function ...」。 +3. **Type**：每個 param 用正確 type — `number` / `boolean` / `array` / `object`，不要全 `string`。 +4. **Constrain**：`required` 列必填欄位；模糊邊界用 `enum` 收斂；`description` 補欄位用途。 +5. **Error pattern**：tool 失敗回傳 `{"error": "...", "retry_hint": "..."}` 結構化 dict，**不要 `raise`**——production 的 retry 由 LLM 決定。 + +**Fork template**：直接 copy [`../../stage-3/02-multi-tool-selection/starter.py`](../../stage-3/02-multi-tool-selection/starter.py)（單輪 tool）或 [`../../stage-3/03-react-from-scratch/starter.py`](../../stage-3/03-react-from-scratch/starter.py)（多輪 loop）的 `TOOLS_SPEC` + `TOOL_IMPL` 結構、改成你的 tool。 + +## Step 3 — SDK 差異提醒 + +使用者可能在 Anthropic / OpenAI / Ollama 之間切換、SDK shape 不同。看 [`references/sdk-diff.md`](references/sdk-diff.md) 的 3 行對照表。**不要假設使用者知道——主動問一次「你用哪個 SDK」**。 + +## Step 4 — Mock test first（強烈建議） + +每個 tool-calling 程式都應該有 mock-based test、不打真 API： + +- Path A (Ollama) 用 OpenAI-compat response shape mock +- Path B (Anthropic) 用 content block mock + +完整 mock pattern 對照 [`../../stage-3/03-react-from-scratch/test.py`](../../stage-3/03-react-from-scratch/test.py)。**先把 test 跑通、再連真的 LLM**——可以省下 80% 的 debug 時間。 + +## Step 5 — When to escalate / route away + +這個 skill **不**處理： + +- **LangChain / LangGraph / CrewAI / Pydantic AI** 等 framework 問題 → 路 Stage 4 +- **MCP server / client** 設計 → 路 [`resources/cookbook.md` 2 寫你的第一個 MCP server](../../../resources/cookbook.md) +- **Production 監控 / observability / cost tracking** → 路 Stage 7 +- **Prompt engineering 一般技巧** → 路 Stage 2 + +碰到這些情境、直接告訴使用者「這個 skill 處理 tool-use mechanics、你這個問題需要 Stage X、建議去看 ...」、不要硬吃下去。 + +## Don't + +- **不要直接幫使用者寫一整份 starter.py**——他們需要練 mental model、不是拿到答案 copy-paste。指他們 fork [`../../stage-3/`](../../stage-3/) 的 starter、改 TOOLS_SPEC 就好。 +- **不要跳過 Step 1 triage**——4 個 symptom 的修法不同，沒問清楚就猜會浪費時間。 +- **不要假設 user 用 Claude**——Path A 默認是 Ollama qwen2.5:3b，先問再答。 +- **不要把 schema-design 規則背一遍**——`resources/schema-design-cheatsheet.md` 已經寫好，指過去就行。 + +## References + +- [`references/debug-flowchart.md`](references/debug-flowchart.md) — "為什麼 LLM 不呼叫我的 tool" 4-symptom 診斷 +- [`references/schema-evolution.md`](references/schema-evolution.md) — Bad → Good schema worked example（4 個改進步驟） +- [`references/sdk-diff.md`](references/sdk-diff.md) — Anthropic vs OpenAI-compat 並排表 +- [`resources/schema-design-cheatsheet.md`](../../../resources/schema-design-cheatsheet.md) — 5 條黃金規則 + 5 個 anti-pattern（curriculum 既有資源） +- [`resources/glossary.md` 2](../../../resources/glossary.md) — Agent / Tool Use / ReAct 名詞定義 diff --git a/examples/stage-5/tool-calling-tutor/evals/evals.json b/examples/stage-5/tool-calling-tutor/evals/evals.json new file mode 100644 index 0000000..679a045 --- /dev/null +++ b/examples/stage-5/tool-calling-tutor/evals/evals.json @@ -0,0 +1,55 @@ +{ + "$schema": "https://json.schemastore.org/promptfoo-config.json", + "description": "tool-calling-tutor skill evals — 5 test cases covering each of the 4 triage branches + a routing-away case", + "evals": [ + { + "name": "symptom_a_llm_not_calling", + "input": "我寫了一個 tool 叫 get_weather、tools=[get_weather] 也傳進去了、可是 Claude 直接回了一段文字、根本沒呼叫 tool。哪裡錯了？", + "expected_behavior": [ + "Should first ask which of 4 symptoms (a/b/c/d) the user is hitting", + "After user confirms (a), should reference references/debug-flowchart.md Section A", + "Should mention the 5 common causes: description too generic, overlapping boundaries, query doesn't need tool, missing schema wrapper, model too small", + "Should ask user to inspect description before guessing" + ] + }, + { + "name": "symptom_b_wrong_args", + "input": "Tool 有被呼叫但 args 不對。user 說 'Convert 32 Celsius'、結果 value 傳成 '32 Celsius' 而不是 32。", + "expected_behavior": [ + "Should identify this as symptom (b)", + "Should reference references/schema-evolution.md", + "Should suggest 3 fixes: change type to number, add required, add enum for unit", + "Should NOT just write a fixed schema for the user — should walk them through the 4 iterations" + ] + }, + { + "name": "symptom_c_loop_no_end", + "input": "ReAct loop 跑滿 max_iter=10 都沒收尾、LLM 一直叫同一個 tool。", + "expected_behavior": [ + "Should identify this as symptom (c)", + "Should reference references/debug-flowchart.md Section C", + "Should check: (1) is assistant response appended to messages? (2) does tool message have tool_call_id? (3) does tool return self-contained result?", + "Should mention max_iter safety net is mandatory, not optional" + ] + }, + { + "name": "symptom_d_design_from_scratch", + "input": "我想做一個 tool 讓 LLM 可以查 GitHub repo 的 star 數、但完全不知道 schema 怎麼寫。", + "expected_behavior": [ + "Should identify this as symptom (d)", + "Should walk through the 5-step recipe (define, describe, type, constrain, error pattern)", + "Should point user to fork examples/stage-3/02-multi-tool-selection/starter.py as template", + "Should remind to ask 'which SDK are you using' before showing code" + ] + }, + { + "name": "route_away_langgraph", + "input": "我用 LangGraph 寫了一個 agent、StateGraph 的 add_edge 有點問題、能幫我看嗎？", + "expected_behavior": [ + "Should NOT engage with LangGraph-specific debugging", + "Should route user to Stage 4 — Agent Frameworks", + "Should mention this skill handles tool-use mechanics, not framework-specific issues" + ] + } + ] +} diff --git a/examples/stage-5/tool-calling-tutor/references/debug-flowchart.en.md b/examples/stage-5/tool-calling-tutor/references/debug-flowchart.en.md new file mode 100644 index 0000000..45bb790 --- /dev/null +++ b/examples/stage-5/tool-calling-tutor/references/debug-flowchart.en.md @@ -0,0 +1,164 @@ +# Debug Flowchart: "Why won't the LLM call my tool?" + +> [繁體中文](./debug-flowchart.md) | [简体中文](./debug-flowchart.zh-Hans.md) | **English** + + +> 4-symptom diagnostic. Pairs with SKILL.md Step 2. + +## Section A — Symptom (a): LLM doesn't trigger tool_calls at all + +Most common: you pass `tools=[...]`, the LLM responds in plain text, no tool selected. + +### Quick check (30 seconds) + +Tick each of these: + +``` +[ ] resp.choices[0].finish_reason == "tool_calls"? +[ ] Or "stop"? ← if "stop", the LLM actively chose not to call a tool +[ ] Is resp.choices[0].message.tool_calls None / empty? +[ ] Is resp.choices[0].message.content a long block of text? ← LLM answered from knowledge +``` + +### 5 common causes (by frequency) + +#### 1. `description` is too generic (70% of cases) + +```python +# ❌ LLM can't tell when to use it +{"name": "get_data", "description": "Get data."} + +# ✅ LLM knows immediately when this applies +{"name": "get_weather", "description": "Use this when the user asks about current weather, forecast, or temperature for a specific city."} +``` + +**Fix**: rewrite descriptions to start with "Use this when..." and list 2-3 trigger situations. + +#### 2. Multiple tools have overlapping boundaries (15%) + +Both tool descriptions match the same query — LLM can't pick — so it picks neither: + +```python +# ❌ Overlapping +tool_a = {"name": "search", "description": "Find information."} +tool_b = {"name": "lookup", "description": "Look up data."} +# user: "look up Taipei's population" → LLM can't pick + +# ✅ Mutually exclusive +tool_a = {"name": "web_search", "description": "Use for current/external info not in knowledge: news, weather, prices."} +tool_b = {"name": "fact_lookup", "description": "Use for static facts: populations, physical constants, capital cities."} +``` + +**Fix**: add "Do NOT use for ..." to each tool to spell out the negative boundary. + +#### 3. The query genuinely doesn't need a tool (10%) + +```python +# user: "What is Python?" +# tools: [calculator, weather_lookup] +# Correct behavior: LLM answers from knowledge, doesn't select a tool +``` + +**Not a bug** — sanity-check whether the query actually needs a tool. + +#### 4. Tool schema structure is wrong (3%) + +```python +# ❌ OpenAI-compat missing the wrapper +TOOLS = [{"name": "x", "description": "...", "parameters": {...}}] + +# ✅ OpenAI-compat needs the wrapper +TOOLS = [{"type": "function", "function": {"name": "x", "description": "...", "parameters": {...}}}] +``` + +SDKs usually raise; Ollama sometimes swallows it and replies in plain text. + +#### 5. Model is too small (2%) + +`gemma4:e4b` / 1.5B-class models have unstable tool-calling support. **Stage 3+ defaults to `qwen2.5:3b`** or `llama3.2:3b`. + +```bash +ollama pull qwen2.5:3b +MODEL=qwen2.5:3b python starter.py +``` + +## Section B — Symptom (b): tool called, but args are wrong + +```python +# user: "Convert 32 Celsius to Fahrenheit" +# expected: convert_temperature(value=32, unit="celsius") +# actual: convert_temperature(value="32 Celsius", unit="") ← wrong +``` + +### 3 causes + fixes + +| Cause | Symptom | Fix | +|---|---|---| +| All params `string` | `value: "32"` instead of `32` | `parameters.value.type = "number"` | +| No `required` | `unit` missing | `"required": ["value", "unit"]` | +| No `enum` | `unit: "C" / "Celsius" / "celsius"` appearing randomly | `"enum": ["celsius", "fahrenheit"]` | + +Full A/B in [`schema-evolution.en.md`](schema-evolution.en.md). + +## Section C — Symptom (c): ReAct loop won't terminate + +```python +# Hits max_iter=10, never end_turn +``` + +### 3 causes + +#### 1. Forgot to append assistant response to messages + +```python +# ❌ infinite loop +for step in range(5): + resp = client.chat.completions.create(...) + # missing: messages.append({"role": "assistant", ...}) + if resp.tool_calls: + obs = run_tool(...) + messages.append({"role": "tool", "content": obs}) +# Next round LLM can't see its previous thought — infinite repeat + +# ✅ fix +messages.append({"role": "assistant", "content": msg.content, "tool_calls": msg.tool_calls}) +``` + +#### 2. Tool message missing `tool_call_id` + +```python +# ❌ LLM can't pair result with call +messages.append({"role": "tool", "content": obs}) + +# ✅ +messages.append({"role": "tool", "tool_call_id": tc.id, "content": obs}) +``` + +#### 3. Tool returns garbage — LLM doesn't know what "done" looks like + +```python +# user: "Check Taipei weather" +# tool returns "ok" ← LLM can't tell if this is the answer or a placeholder + +# ✅ tool results must be self-contained +return {"city": "Taipei", "forecast": "rain", "temperature_c": 24} +``` + +## Section D — Symptom (c, sub): ReAct loop skips a step + +Multi-step task misses an intermediate tool call: + +``` +[step 0] lookup_population(city=Taipei) → 2602000 +[step 1] lookup_population(city=NewYork) → 8336000 +[step 2] divide(2602000, 8336000) → 0.3122 +[step 3] end_turn: "The answer is 0.3122" ← skipped to_percentage +``` + +### Fixes + +1. **Upgrade model**: `qwen2.5:7b` or `claude-haiku-4-5` substantially improves multi-step stability +2. **Description encodes ordering**: `{"name": "to_percentage", "description": "Convert a ratio to percentage. **Call this LAST after dividing.**"}` +3. **Add chain-of-thought prompt**: prepend user message with "Plan the steps first, then execute one by one." + +Full runnable comparison → [`../../stage-3/04-multi-step-reasoning/`](../../../stage-3/04-multi-step-reasoning/) diff --git a/examples/stage-5/tool-calling-tutor/references/debug-flowchart.md b/examples/stage-5/tool-calling-tutor/references/debug-flowchart.md new file mode 100644 index 0000000..32dea01 --- /dev/null +++ b/examples/stage-5/tool-calling-tutor/references/debug-flowchart.md @@ -0,0 +1,164 @@ +# Debug Flowchart：「為什麼 LLM 不呼叫我的 tool」 + +> **繁體中文** | [简体中文](./debug-flowchart.zh-Hans.md) | [English](./debug-flowchart.en.md) + + +> 4-symptom 診斷流程。對應 SKILL.md Step 2。 + +## Section A — Symptom (a)：LLM 完全不觸發 tool_calls + +最常見：LLM 看 `tools=[...]`、但回了純文字、沒選任何 tool。 + +### Quick check（30 秒） + +把這 4 個逐項勾一遍： + +``` +[ ] resp.choices[0].finish_reason == "tool_calls"？ +[ ] 還是 "stop"？ ← 如果是 stop、LLM 主動選擇不呼叫 tool +[ ] resp.choices[0].message.tool_calls 是 None / 空 list？ +[ ] resp.choices[0].message.content 是長文字？ ← LLM 用知識回答了 +``` + +### 5 個常見原因（按發生頻率排序） + +#### 1. `description` 太籠統（70% 的 case） + +```python +# ❌ LLM 看不出何時用 +{"name": "get_data", "description": "Get data."} + +# ✅ LLM 一看就知道何時用 +{"name": "get_weather", "description": "Use this when the user asks about current weather, forecast, or temperature for a specific city."} +``` + +**修法**：description 句首改成 "Use this when..."、列 2-3 種觸發情境。 + +#### 2. 多 tool 邊界互相重疊（15%） + +兩個 tool 的 description 都能套到同一個 query、LLM 選不出來、乾脆都不選： + +```python +# ❌ 邊界重疊 +tool_a = {"name": "search", "description": "Find information."} +tool_b = {"name": "lookup", "description": "Look up data."} +# user: "幫我查台北人口" → LLM 不知道哪個 + +# ✅ 邊界互斥 +tool_a = {"name": "web_search", "description": "Use for current/external info not in knowledge: news, weather, prices."} +tool_b = {"name": "fact_lookup", "description": "Use for static facts: populations, physical constants, capital cities."} +``` + +**修法**：每個 tool 加「Do NOT use for ...」明寫負面邊界。 + +#### 3. user query 根本不需要 tool（10%） + +```python +# user: "什麼是 Python？" +# tools: [calculator, weather_lookup] +# 正確行為：LLM 用知識回答、不選 tool +``` + +**這不是 bug**——驗證一下 user query 是否真的需要 tool。 + +#### 4. Tool schema 結構錯誤（3%） + +```python +# ❌ OpenAI-compat 缺外包 +TOOLS = [{"name": "x", "description": "...", "parameters": {...}}] + +# ✅ OpenAI-compat 要包一層 +TOOLS = [{"type": "function", "function": {"name": "x", "description": "...", "parameters": {...}}}] +``` + +SDK 通常會 raise error；但用 Ollama 偶爾會吞掉、純文字回答。 + +#### 5. Model 太小（2%） + +`gemma4:e4b` / 1.5B 級 model 對 tool calling 支援不穩。**Stage 3+ 默認用 `qwen2.5:3b`** 或 `llama3.2:3b`。 + +```bash +ollama pull qwen2.5:3b +MODEL=qwen2.5:3b python starter.py +``` + +## Section B — Symptom (b)：tool 被呼叫、但 args 錯 + +```python +# user: "Convert 32 Celsius to Fahrenheit" +# 預期：convert_temperature(value=32, unit="celsius") +# 實際：convert_temperature(value="32 Celsius", unit="") ← 不對 +``` + +### 3 個原因 + 修法 + +| 原因 | 觀察 | 修法 | +|---|---|---| +| param 全 string | `value: "32"` 而非 `32` | `parameters.value.type = "number"` | +| 缺 `required` | `unit` 沒傳 | `"required": ["value", "unit"]` | +| enum 缺 | `unit: "C" / "Celsius" / "celsius"` 都出現 | `"enum": ["celsius", "fahrenheit"]` | + +詳細 A/B 看 [`schema-evolution.md`](schema-evolution.md)。 + +## Section C — Symptom (c)：ReAct loop 跑不停 + +```python +# 跑滿 max_iter=10、never end_turn +``` + +### 3 個原因 + +#### 1. 忘記把 assistant response 接回 messages + +```python +# ❌ 跑不停 +for step in range(5): + resp = client.chat.completions.create(...) + # 下面忘了 messages.append({"role": "assistant", ...}) + if resp.tool_calls: + obs = run_tool(...) + messages.append({"role": "tool", "content": obs}) +# 下輪 LLM 看不到自己上輪的 thought、無限重複 + +# ✅ 修 +messages.append({"role": "assistant", "content": msg.content, "tool_calls": msg.tool_calls}) +``` + +#### 2. tool message 沒帶 `tool_call_id` + +```python +# ❌ LLM 無法配對 +messages.append({"role": "tool", "content": obs}) + +# ✅ +messages.append({"role": "tool", "tool_call_id": tc.id, "content": obs}) +``` + +#### 3. tool 結果是 garbage、LLM 不知道什麼是「完成」 + +```python +# user: "查台北天氣" +# tool 回傳 "ok" ← LLM 不知道這是答案還是 placeholder + +# ✅ tool 結果要 self-contained +return {"city": "Taipei", "forecast": "rain", "temperature_c": 24} +``` + +## Section D — Symptom (c, sub)：ReAct loop 漏步 + +多步任務中間少一個 tool call： + +``` +[step 0] lookup_population(city=Taipei) → 2602000 +[step 1] lookup_population(city=NewYork) → 8336000 +[step 2] divide(2602000, 8336000) → 0.3122 +[step 3] end_turn: "答案是 0.3122" ← 漏了 to_percentage +``` + +### 修法 + +1. **換大 model**：`qwen2.5:7b` 或 `claude-haiku-4-5`、多步穩定度顯著提升 +2. **Description 明示順序**：`{"name": "to_percentage", "description": "Convert a ratio to percentage. **Call this LAST after dividing.**"}` +3. **加 chain-of-thought prompt**：user message 開頭加「Plan the steps first, then execute one by one.」 + +完整對照可跑範例 → [`../../stage-3/04-multi-step-reasoning/`](../../../stage-3/04-multi-step-reasoning/) diff --git a/examples/stage-5/tool-calling-tutor/references/debug-flowchart.zh-Hans.md b/examples/stage-5/tool-calling-tutor/references/debug-flowchart.zh-Hans.md new file mode 100644 index 0000000..3a893b0 --- /dev/null +++ b/examples/stage-5/tool-calling-tutor/references/debug-flowchart.zh-Hans.md @@ -0,0 +1,164 @@ +# Debug Flowchart：“为什么 LLM 不调用我的 tool” + +> [繁體中文](./debug-flowchart.md) | **简体中文** | [English](./debug-flowchart.en.md) + + +> 4-symptom 诊断流程。对应 SKILL.md Step 2。 + +## Section A — Symptom (a)：LLM 完全不触发 tool_calls + +最常见：LLM 看 `tools=[...]`、但回了纯文字、没选任何 tool。 + +### Quick check（30 秒） + +把这 4 个逐项勾一遍： + +``` +[ ] resp.choices[0].finish_reason == "tool_calls"？ +[ ] 还是 "stop"？ ← 如果是 stop、LLM 主动选择不调用 tool +[ ] resp.choices[0].message.tool_calls 是 None / 空 list？ +[ ] resp.choices[0].message.content 是长文字？ ← LLM 用知识回答了 +``` + +### 5 个常见原因（按发生频率排序） + +#### 1. `description` 太笼统（70% 的 case） + +```python +# ❌ LLM 看不出何时用 +{"name": "get_data", "description": "Get data."} + +# ✅ LLM 一看就知道何时用 +{"name": "get_weather", "description": "Use this when the user asks about current weather, forecast, or temperature for a specific city."} +``` + +**修法**：description 句首改成 "Use this when..."、列 2-3 种触发情境。 + +#### 2. 多 tool 边界互相重叠（15%） + +两个 tool 的 description 都能套到同一个 query、LLM 选不出来、干脆都不选： + +```python +# ❌ 边界重叠 +tool_a = {"name": "search", "description": "Find information."} +tool_b = {"name": "lookup", "description": "Look up data."} +# user: "帮我查台北人口" → LLM 不知道哪个 + +# ✅ 边界互斥 +tool_a = {"name": "web_search", "description": "Use for current/external info not in knowledge: news, weather, prices."} +tool_b = {"name": "fact_lookup", "description": "Use for static facts: populations, physical constants, capital cities."} +``` + +**修法**：每个 tool 加“Do NOT use for ...”明写负面边界。 + +#### 3. user query 根本不需要 tool（10%） + +```python +# user: "什么是 Python？" +# tools: [calculator, weather_lookup] +# 正确行为：LLM 用知识回答、不选 tool +``` + +**这不是 bug**——验证一下 user query 是否真的需要 tool。 + +#### 4. Tool schema 结构错误（3%） + +```python +# ❌ OpenAI-compat 缺外包 +TOOLS = [{"name": "x", "description": "...", "parameters": {...}}] + +# ✅ OpenAI-compat 要包一层 +TOOLS = [{"type": "function", "function": {"name": "x", "description": "...", "parameters": {...}}}] +``` + +SDK 通常会 raise error；但用 Ollama 偶尔会吞掉、纯文字回答。 + +#### 5. Model 太小（2%） + +`gemma4:e4b` / 1.5B 级 model 对 tool calling 支援不稳。**Stage 3+ 默认用 `qwen2.5:3b`** 或 `llama3.2:3b`。 + +```bash +ollama pull qwen2.5:3b +MODEL=qwen2.5:3b python starter.py +``` + +## Section B — Symptom (b)：tool 被调用、但 args 错 + +```python +# user: "Convert 32 Celsius to Fahrenheit" +# 预期：convert_temperature(value=32, unit="celsius") +# 实际：convert_temperature(value="32 Celsius", unit="") ← 不对 +``` + +### 3 个原因 + 修法 + +| 原因 | 观察 | 修法 | +|---|---|---| +| param 全 string | `value: "32"` 而非 `32` | `parameters.value.type = "number"` | +| 缺 `required` | `unit` 没传 | `"required": ["value", "unit"]` | +| enum 缺 | `unit: "C" / "Celsius" / "celsius"` 都出现 | `"enum": ["celsius", "fahrenheit"]` | + +详细 A/B 看 [`schema-evolution.zh-Hans.md`](schema-evolution.zh-Hans.md)。 + +## Section C — Symptom (c)：ReAct loop 跑不停 + +```python +# 跑满 max_iter=10、never end_turn +``` + +### 3 个原因 + +#### 1. 忘记把 assistant response 接回 messages + +```python +# ❌ 跑不停 +for step in range(5): + resp = client.chat.completions.create(...) + # 下面忘了 messages.append({"role": "assistant", ...}) + if resp.tool_calls: + obs = run_tool(...) + messages.append({"role": "tool", "content": obs}) +# 下轮 LLM 看不到自己上轮的 thought、无限重复 + +# ✅ 修 +messages.append({"role": "assistant", "content": msg.content, "tool_calls": msg.tool_calls}) +``` + +#### 2. tool message 没带 `tool_call_id` + +```python +# ❌ LLM 无法配对 +messages.append({"role": "tool", "content": obs}) + +# ✅ +messages.append({"role": "tool", "tool_call_id": tc.id, "content": obs}) +``` + +#### 3. tool 结果是 garbage、LLM 不知道什么是“完成” + +```python +# user: "查台北天气" +# tool 回传 "ok" ← LLM 不知道这是答案还是 placeholder + +# ✅ tool 结果要 self-contained +return {"city": "Taipei", "forecast": "rain", "temperature_c": 24} +``` + +## Section D — Symptom (c, sub)：ReAct loop 漏步 + +多步任务中间少一个 tool call： + +``` +[step 0] lookup_population(city=Taipei) → 2602000 +[step 1] lookup_population(city=NewYork) → 8336000 +[step 2] divide(2602000, 8336000) → 0.3122 +[step 3] end_turn: "答案是 0.3122" ← 漏了 to_percentage +``` + +### 修法 + +1. **换大 model**：`qwen2.5:7b` 或 `claude-haiku-4-5`、多步稳定度显著提升 +2. **Description 明示顺序**：`{"name": "to_percentage", "description": "Convert a ratio to percentage. **Call this LAST after dividing.**"}` +3. **加 chain-of-thought prompt**：user message 开头加“Plan the steps first, then execute one by one.” + +完整对照可跑范例 → [`../../stage-3/04-multi-step-reasoning/`](../../../stage-3/04-multi-step-reasoning/) diff --git a/examples/stage-5/tool-calling-tutor/references/schema-evolution.en.md b/examples/stage-5/tool-calling-tutor/references/schema-evolution.en.md new file mode 100644 index 0000000..ce48181 --- /dev/null +++ b/examples/stage-5/tool-calling-tutor/references/schema-evolution.en.md @@ -0,0 +1,152 @@ +# Schema Evolution: bad schema improved to good (worked example) + +> [繁體中文](./schema-evolution.md) | [简体中文](./schema-evolution.zh-Hans.md) | **English** + + +> Same tool (temperature conversion), 4 improvement steps. Pairs with SKILL.md Step 2(d). Fills the procedural gap that [`resources/schema-design-cheatsheet.en.md`](../../../../resources/schema-design-cheatsheet.en.md) (which is prescriptive) doesn't cover. + +## Iteration 0: original bad schema + +```python +{ + "name": "convert", + "description": "Convert a value.", + "parameters": { + "type": "object", + "properties": { + "value": {"type": "string"}, + "unit": {"type": "string"} + } + } +} +``` + +### Observed behavior (running qwen2.5:3b) + +```python +# user: "Convert 32 Celsius to Fahrenheit" +# LLM behavior (average over many runs): +# - 40% picks convert, args = {"value": "32 Celsius", "unit": ""} ← wrong type +# - 30% picks convert, args = {"value": "32", "unit": "C"} ← inconsistent unit +# - 20% picks a different tool (process_data, etc.) ← unclear boundary +# - 10% doesn't call any tool, responds in plain text ← description too generic +``` + +**Success rate ≈ 0%.** Claude haiku gets it right ~60-70% (still unstable). + +## Iteration 1: fix the description + +```python +# ❌ before +"description": "Convert a value." + +# ✅ after (clear "when to use") +"description": "Use this when the user asks to convert temperatures between Fahrenheit and Celsius." +``` + +### New behavior + +- LLM correctly triggers tool ~60% of the time (up from ~30%; args still off) +- Unit format still wrong + +**This step fixes "will the LLM call it" — args still need work.** + +## Iteration 2: fix parameter type + +```python +# ❌ before +"value": {"type": "string"} + +# ✅ after +"value": {"type": "number", "description": "Temperature value to convert"} +``` + +### New behavior + +- `value` is now `32` (number) instead of `"32"` / `"32 Celsius"` +- `unit` still sometimes missing or wrong format (`"C"` vs `"celsius"`) + +**Type pinned; field completeness and enum still need work.** + +## Iteration 3: add `required` + +```python +"parameters": { + "type": "object", + "properties": { + "value": {"type": "number", "description": "Temperature value to convert"}, + "unit": {"type": "string"} + }, + "required": ["value", "unit"] # ✅ NEW +} +``` + +### New behavior + +- LLM no longer skips `unit` +- Still occasionally sends `"C"` / `"Celsius"` / `"celsius"` (case/abbreviation drift) + +**Mandatory fields pinned; fuzzy boundaries still need an enum.** + +## Iteration 4: add `enum` + +```python +"unit": { + "type": "string", + "enum": ["celsius", "fahrenheit"], # ✅ NEW + "description": "Unit of the input value" +} +``` + +### Final schema + +```python +{ + "name": "convert_temperature", # ✅ also more specific name + "description": "Use this when the user asks to convert temperatures between Fahrenheit and Celsius.", + "parameters": { + "type": "object", + "properties": { + "value": {"type": "number", "description": "Temperature value to convert"}, + "unit": { + "type": "string", + "enum": ["celsius", "fahrenheit"], + "description": "Unit of the input value" + } + }, + "required": ["value", "unit"] + } +} +``` + +### Behavior + +- qwen2.5:3b 95%+ correct +- Claude haiku 99%+ + +## Cost vs benefit of the 4 changes + +| Iteration | What changed | Code delta | Accuracy lift (qwen) | +|---|---|---|---| +| 1 | description | 1 line | 0% → 60% | +| 2 | type: number | 1 line | 60% → 75% | +| 3 | required | 1 line | 75% → 85% | +| 4 | enum | 1 line | 85% → 95%+ | + +**4 lines of code take accuracy from ~0% to 95%+.** That's the ROI of schema design. + +## Why this matters **more** on small models + +``` +Accuracy (same query × 1000 runs): + BAD schema GOOD schema diff +Claude haiku 60% 99% +39% +qwen2.5:3b 0% 95% +95% +gemma4:e4b 0% 80% +80% +``` + +**Takeaway**: time spent writing good schemas **saves you the cost of upgrading the model**. Want a cheap production model? Your schemas must be solid enough to run in production. + +## See the full comparison example + +→ [`../../stage-3/06-schema-design/`](../../../stage-3/06-schema-design/): contains `starter_bad.py` + `starter_good.py` + trilingual READMEs (runnable both Path A Ollama and Path B Anthropic). diff --git a/examples/stage-5/tool-calling-tutor/references/schema-evolution.md b/examples/stage-5/tool-calling-tutor/references/schema-evolution.md new file mode 100644 index 0000000..6913fd2 --- /dev/null +++ b/examples/stage-5/tool-calling-tutor/references/schema-evolution.md @@ -0,0 +1,152 @@ +# Schema Evolution：壞 schema 改到好（worked example） + +> **繁體中文** | [简体中文](./schema-evolution.zh-Hans.md) | [English](./schema-evolution.en.md) + + +> 同一個工具（溫度轉換）、4 個改進步驟。對應 SKILL.md Step 2(d)、補 [`resources/schema-design-cheatsheet.md`](../../../../resources/schema-design-cheatsheet.md) 缺的 procedural 走法。 + +## Iteration 0：原始壞 schema + +```python +{ + "name": "convert", + "description": "Convert a value.", + "parameters": { + "type": "object", + "properties": { + "value": {"type": "string"}, + "unit": {"type": "string"} + } + } +} +``` + +### 觀察行為（用 qwen2.5:3b 跑） + +```python +# user: "Convert 32 Celsius to Fahrenheit" +# LLM 行為（多次測試平均）： +# - 40% 挑到 convert、args = {"value": "32 Celsius", "unit": ""} ← 型別錯 +# - 30% 挑到 convert、args = {"value": "32", "unit": "C"} ← unit 不一致 +# - 20% 挑到別的 tool（process_data 等） ← 邊界不清 +# - 10% 完全沒呼叫 tool、純文字回答 ← description 太籠統 +``` + +**正確率 ≈ 0%**。Claude haiku 大概 60-70% 能猜對（但仍不穩定）。 + +## Iteration 1：修 description + +```python +# ❌ 之前 +"description": "Convert a value." + +# ✅ 之後（明確「何時用」） +"description": "Use this when the user asks to convert temperatures between Fahrenheit and Celsius." +``` + +### 新行為 + +- LLM 正確 trigger tool 機率：60%（從 ~70% 提升、但 args 仍錯） +- 仍然挑錯 unit 格式 + +**這一步搞定「LLM 願不願意呼叫」、但 args 還沒固定。** + +## Iteration 2：修 parameter type + +```python +# ❌ 之前 +"value": {"type": "string"} + +# ✅ 之後 +"value": {"type": "number", "description": "Temperature value to convert"} +``` + +### 新行為 + +- `value` 改傳 `32`（number）而非 `"32"` / `"32 Celsius"` +- 仍偶爾漏傳 `unit`、或傳 `"C"` 而非 `"celsius"` + +**型別固定、但欄位完整性與 enum 還沒處理。** + +## Iteration 3：加 `required` + +```python +"parameters": { + "type": "object", + "properties": { + "value": {"type": "number", "description": "Temperature value to convert"}, + "unit": {"type": "string"} + }, + "required": ["value", "unit"] # ✅ NEW +} +``` + +### 新行為 + +- LLM 不再漏傳 `unit` +- 仍偶爾傳 `"C"` / `"Celsius"` / `"celsius"`（大小寫 / 縮寫不一致） + +**必填欄位固定、但模糊邊界還沒收斂。** + +## Iteration 4：加 `enum` + +```python +"unit": { + "type": "string", + "enum": ["celsius", "fahrenheit"], # ✅ NEW + "description": "Unit of the input value" +} +``` + +### 最終 schema + +```python +{ + "name": "convert_temperature", # ✅ 也改具體 name + "description": "Use this when the user asks to convert temperatures between Fahrenheit and Celsius.", + "parameters": { + "type": "object", + "properties": { + "value": {"type": "number", "description": "Temperature value to convert"}, + "unit": { + "type": "string", + "enum": ["celsius", "fahrenheit"], + "description": "Unit of the input value" + } + }, + "required": ["value", "unit"] + } +} +``` + +### 行為 + +- qwen2.5:3b 正確率 95%+ +- Claude haiku 99%+ + +## 4 個改進的 cost vs benefit + +| Iteration | 改了什麼 | 程式碼變動量 | 正確率提升（qwen） | +|---|---|---|---| +| 1 | description | 1 line | 0% → 60% | +| 2 | type: number | 1 line | 60% → 75% | +| 3 | required | 1 line | 75% → 85% | +| 4 | enum | 1 line | 85% → 95%+ | + +**4 行程式碼、把正確率從 ~0% 推到 95%+**。這就是 schema 設計的 ROI。 + +## 為什麼這在小 model 上**更重要** + +``` +正確率（同一份 query × 1000 次）： + BAD schema GOOD schema diff +Claude haiku 60% 99% +39% +qwen2.5:3b 0% 95% +95% +gemma4:e4b 0% 80% +80% +``` + +**結論**：寫好 schema 的功夫**省下換大 model 的 $$**。Production 想用便宜 model？schema 必須能上線跑。 + +## 看完整對照範例 + +→ [`../../stage-3/06-schema-design/`](../../../stage-3/06-schema-design/)：含 `starter_bad.py` + `starter_good.py` + 4 lang README 對照可跑版本。 diff --git a/examples/stage-5/tool-calling-tutor/references/schema-evolution.zh-Hans.md b/examples/stage-5/tool-calling-tutor/references/schema-evolution.zh-Hans.md new file mode 100644 index 0000000..ba1b441 --- /dev/null +++ b/examples/stage-5/tool-calling-tutor/references/schema-evolution.zh-Hans.md @@ -0,0 +1,152 @@ +# Schema Evolution：坏 schema 改到好（worked example） + +> [繁體中文](./schema-evolution.md) | **简体中文** | [English](./schema-evolution.en.md) + + +> 同一个工具（温度转换）、4 个改进步骤。对应 SKILL.md Step 2(d)、补 [`resources/schema-design-cheatsheet.zh-Hans.md`](../../../../resources/schema-design-cheatsheet.zh-Hans.md) 缺的 procedural 走法。 + +## Iteration 0：原始坏 schema + +```python +{ + "name": "convert", + "description": "Convert a value.", + "parameters": { + "type": "object", + "properties": { + "value": {"type": "string"}, + "unit": {"type": "string"} + } + } +} +``` + +### 观察行为（用 qwen2.5:3b 跑） + +```python +# user: "Convert 32 Celsius to Fahrenheit" +# LLM 行为（多次测试平均）： +# - 40% 挑到 convert、args = {"value": "32 Celsius", "unit": ""} ← 类型错 +# - 30% 挑到 convert、args = {"value": "32", "unit": "C"} ← unit 不一致 +# - 20% 挑到别的 tool（process_data 等） ← 边界不清 +# - 10% 完全没调用 tool、纯文字回答 ← description 太笼统 +``` + +**正确率 ≈ 0%**。Claude haiku 大概 60-70% 能猜对（但仍不稳定）。 + +## Iteration 1：修 description + +```python +# ❌ 之前 +"description": "Convert a value." + +# ✅ 之后（明确「何时用」） +"description": "Use this when the user asks to convert temperatures between Fahrenheit and Celsius." +``` + +### 新行为 + +- LLM 正确 trigger tool 机率：60%（从 ~70% 提升、但 args 仍错） +- 仍然挑错 unit 格式 + +**这一步搞定“LLM 愿不愿意调用”、但 args 还没固定。** + +## Iteration 2：修 parameter type + +```python +# ❌ 之前 +"value": {"type": "string"} + +# ✅ 之后 +"value": {"type": "number", "description": "Temperature value to convert"} +``` + +### 新行为 + +- `value` 改传 `32`（number）而非 `"32"` / `"32 Celsius"` +- 仍偶尔漏传 `unit`、或传 `"C"` 而非 `"celsius"` + +**类型固定、但字段完整性与 enum 还没处理。** + +## Iteration 3：加 `required` + +```python +"parameters": { + "type": "object", + "properties": { + "value": {"type": "number", "description": "Temperature value to convert"}, + "unit": {"type": "string"} + }, + "required": ["value", "unit"] # ✅ NEW +} +``` + +### 新行为 + +- LLM 不再漏传 `unit` +- 仍偶尔传 `"C"` / `"Celsius"` / `"celsius"`（大小写 / 缩写不一致） + +**必填字段固定、但模糊边界还没收敛。** + +## Iteration 4：加 `enum` + +```python +"unit": { + "type": "string", + "enum": ["celsius", "fahrenheit"], # ✅ NEW + "description": "Unit of the input value" +} +``` + +### 最终 schema + +```python +{ + "name": "convert_temperature", # ✅ 也改具体 name + "description": "Use this when the user asks to convert temperatures between Fahrenheit and Celsius.", + "parameters": { + "type": "object", + "properties": { + "value": {"type": "number", "description": "Temperature value to convert"}, + "unit": { + "type": "string", + "enum": ["celsius", "fahrenheit"], + "description": "Unit of the input value" + } + }, + "required": ["value", "unit"] + } +} +``` + +### 行为 + +- qwen2.5:3b 正确率 95%+ +- Claude haiku 99%+ + +## 4 个改进的 cost vs benefit + +| Iteration | 改了什么 | 程序码变动量 | 正确率提升（qwen） | +|---|---|---|---| +| 1 | description | 1 line | 0% → 60% | +| 2 | type: number | 1 line | 60% → 75% | +| 3 | required | 1 line | 75% → 85% | +| 4 | enum | 1 line | 85% → 95%+ | + +**4 行程序码、把正确率从 ~0% 推到 95%+**。这就是 schema 设计的 ROI。 + +## 为什么这在小 model 上**更重要** + +``` +正确率（同一份 query × 1000 次）： + BAD schema GOOD schema diff +Claude haiku 60% 99% +39% +qwen2.5:3b 0% 95% +95% +gemma4:e4b 0% 80% +80% +``` + +**结论**：写好 schema 的功夫**省下换大 model 的 $$**。Production 想用便宜 model？schema 必须能上线跑。 + +## 看完整对照范例 + +→ [`../../stage-3/06-schema-design/`](../../../stage-3/06-schema-design/)：含 `starter_bad.py` + `starter_good.py` + 4 lang README 对照可跑版本。 diff --git a/examples/stage-5/tool-calling-tutor/references/sdk-diff.en.md b/examples/stage-5/tool-calling-tutor/references/sdk-diff.en.md new file mode 100644 index 0000000..b2ad7a7 --- /dev/null +++ b/examples/stage-5/tool-calling-tutor/references/sdk-diff.en.md @@ -0,0 +1,133 @@ +# SDK Diff: Anthropic vs OpenAI-compat (Ollama / OpenAI / Together / most open-source) + +> [繁體中文](./sdk-diff.md) | [简体中文](./sdk-diff.zh-Hans.md) | **English** + + +> Same ReAct loop, 3 key differences between SDKs. Pairs with SKILL.md Step 3. + +## TL;DR comparison + +| Part | Anthropic SDK | OpenAI-compat SDK | +|---|---|---| +| **Tool schema wrap** | `tools=[{name, description, input_schema}]` | `tools=[{"type": "function", "function": {name, description, parameters}}]` | +| **Schema field name** | `input_schema` | `parameters` | +| **Reading tool call** | `[b for b in resp.content if b.type == "tool_use"]` | `resp.choices[0].message.tool_calls` | +| **Args format** | `call.input` is already a dict | `call.function.arguments` is a JSON string — needs `json.loads(...)` | +| **Stop detection** | `resp.stop_reason == "end_turn"` | `resp.choices[0].finish_reason == "stop"` | +| **Assistant turn append** | `messages.append({"role": "assistant", "content": resp.content})` | `messages.append({"role": "assistant", "content": msg.content, "tool_calls": msg.tool_calls})` | +| **Tool result append** | `messages.append({"role": "user", "content": [{"type": "tool_result", "tool_use_id": call.id, "content": obs}]})` | `messages.append({"role": "tool", "tool_call_id": tc.id, "content": obs})` | +| **Exception class** | `anthropic.RateLimitError` / `anthropic.APIConnectionError` | `openai.RateLimitError` / `openai.APIConnectionError` | + +## Side-by-side single-turn tool call + +### Anthropic + +```python +import anthropic + +client = anthropic.Anthropic() + +TOOLS = [{ + "name": "get_weather", + "description": "Use this when the user asks about current weather.", + "input_schema": { + "type": "object", + "properties": {"city": {"type": "string"}}, + "required": ["city"] + } +}] + +resp = client.messages.create( + model="claude-haiku-4-5", + max_tokens=512, + tools=TOOLS, + messages=[{"role": "user", "content": "Is it raining in Taipei?"}] +) + +# Read tool call +calls = [b for b in resp.content if b.type == "tool_use"] +city = calls[0].input["city"] # already a dict +``` + +### OpenAI-compat (Ollama) + +```python +from openai import OpenAI +import json + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +TOOLS = [{ + "type": "function", + "function": { + "name": "get_weather", + "description": "Use this when the user asks about current weather.", + "parameters": { # ← different name + "type": "object", + "properties": {"city": {"type": "string"}}, + "required": ["city"] + } + } +}] + +resp = client.chat.completions.create( + model="qwen2.5:3b", + tools=TOOLS, + messages=[{"role": "user", "content": "Is it raining in Taipei?"}] +) + +# Read tool call +tc = resp.choices[0].message.tool_calls[0] +args = json.loads(tc.function.arguments) # ← needs json.loads +city = args["city"] +``` + +## Side-by-side ReAct loop (multi-turn) + +### Anthropic full loop + +```python +messages = [{"role": "user", "content": "..."}] +for step in range(5): + resp = client.messages.create(model=MODEL, max_tokens=1024, tools=TOOLS, messages=messages) + messages.append({"role": "assistant", "content": resp.content}) # full content list appended + if resp.stop_reason == "end_turn": + break + tool_results = [] + for call in [b for b in resp.content if b.type == "tool_use"]: + obs = TOOL_IMPL[call.name](call.input) + tool_results.append({"type": "tool_result", "tool_use_id": call.id, "content": obs}) + messages.append({"role": "user", "content": tool_results}) # tool results wrapped in user message +``` + +### OpenAI-compat full loop + +```python +messages = [{"role": "user", "content": "..."}] +for step in range(5): + resp = client.chat.completions.create(model=MODEL, tools=TOOLS, messages=messages) + msg = resp.choices[0].message + messages.append({"role": "assistant", "content": msg.content, "tool_calls": msg.tool_calls}) + if not msg.tool_calls: + break + for tc in msg.tool_calls: + args = json.loads(tc.function.arguments) + obs = TOOL_IMPL[tc.function.name](args) + messages.append({"role": "tool", "tool_call_id": tc.id, "content": obs}) # dedicated role +``` + +## 4 easy-to-trip-on spots + +1. **`parameters` vs `input_schema`**: most common copy-paste trap — pasting an Anthropic schema directly into OpenAI-compat silently fails (Ollama doesn't raise; it just doesn't call the tool). +2. **`call.input` vs `json.loads(arguments)`**: forgetting `json.loads` on the OpenAI-compat side gives you a string instead of a dict — KeyError. +3. **Different role for tool result**: Anthropic uses `role="user"` + `[{"type": "tool_result", ...}]`; OpenAI-compat uses `role="tool"` + plain string content. +4. **`stop_reason` vs `finish_reason`**: both exist but with different field names and values. Anthropic `"end_turn"` / `"tool_use"`; OpenAI `"stop"` / `"tool_calls"`. + +## Full comparison examples + +Every Stage 3 exercise ships both starters: + +- `starter.py` = OpenAI-compat / Ollama +- `starter_anthropic.py` = Anthropic + +Compare any of [`../../stage-3/02-multi-tool-selection/`](../../../stage-3/02-multi-tool-selection/) ~ [`../../stage-3/06-schema-design/`](../../../stage-3/06-schema-design/). diff --git a/examples/stage-5/tool-calling-tutor/references/sdk-diff.md b/examples/stage-5/tool-calling-tutor/references/sdk-diff.md new file mode 100644 index 0000000..1e5c45c --- /dev/null +++ b/examples/stage-5/tool-calling-tutor/references/sdk-diff.md @@ -0,0 +1,133 @@ +# SDK Diff：Anthropic vs OpenAI-compat（Ollama / OpenAI / Together / 多數開源） + +> **繁體中文** | [简体中文](./sdk-diff.zh-Hans.md) | [English](./sdk-diff.en.md) + + +> 同一個 ReAct loop、不同 SDK 的 3 個關鍵差異。對應 SKILL.md Step 3。 + +## TL;DR 對照表 + +| 部分 | Anthropic SDK | OpenAI-compat SDK | +|---|---|---| +| **Tool schema 包法** | `tools=[{name, description, input_schema}]` | `tools=[{"type": "function", "function": {name, description, parameters}}]` | +| **Schema 欄位名** | `input_schema` | `parameters` | +| **抓 tool call** | `[b for b in resp.content if b.type == "tool_use"]` | `resp.choices[0].message.tool_calls` | +| **Args 格式** | `call.input` 已是 dict | `call.function.arguments` 是 JSON string、要 `json.loads(...)` | +| **判完成** | `resp.stop_reason == "end_turn"` | `resp.choices[0].finish_reason == "stop"` | +| **Assistant 接回** | `messages.append({"role": "assistant", "content": resp.content})` | `messages.append({"role": "assistant", "content": msg.content, "tool_calls": msg.tool_calls})` | +| **Tool result 接回** | `messages.append({"role": "user", "content": [{"type": "tool_result", "tool_use_id": call.id, "content": obs}]})` | `messages.append({"role": "tool", "tool_call_id": tc.id, "content": obs})` | +| **Exception class** | `anthropic.RateLimitError` / `anthropic.APIConnectionError` | `openai.RateLimitError` / `openai.APIConnectionError` | + +## 並排程式碼對照（單輪 tool call） + +### Anthropic + +```python +import anthropic + +client = anthropic.Anthropic() + +TOOLS = [{ + "name": "get_weather", + "description": "Use this when the user asks about current weather.", + "input_schema": { + "type": "object", + "properties": {"city": {"type": "string"}}, + "required": ["city"] + } +}] + +resp = client.messages.create( + model="claude-haiku-4-5", + max_tokens=512, + tools=TOOLS, + messages=[{"role": "user", "content": "台北現在有下雨嗎？"}] +) + +# 抓 tool call +calls = [b for b in resp.content if b.type == "tool_use"] +city = calls[0].input["city"] # 已是 dict +``` + +### OpenAI-compat（Ollama） + +```python +from openai import OpenAI +import json + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +TOOLS = [{ + "type": "function", + "function": { + "name": "get_weather", + "description": "Use this when the user asks about current weather.", + "parameters": { # ← 名字不同 + "type": "object", + "properties": {"city": {"type": "string"}}, + "required": ["city"] + } + } +}] + +resp = client.chat.completions.create( + model="qwen2.5:3b", + tools=TOOLS, + messages=[{"role": "user", "content": "台北現在有下雨嗎？"}] +) + +# 抓 tool call +tc = resp.choices[0].message.tool_calls[0] +args = json.loads(tc.function.arguments) # ← 要 json.loads +city = args["city"] +``` + +## 並排 ReAct loop（多輪） + +### Anthropic 完整 loop + +```python +messages = [{"role": "user", "content": "..."}] +for step in range(5): + resp = client.messages.create(model=MODEL, max_tokens=1024, tools=TOOLS, messages=messages) + messages.append({"role": "assistant", "content": resp.content}) # 整個 content list 直接接 + if resp.stop_reason == "end_turn": + break + tool_results = [] + for call in [b for b in resp.content if b.type == "tool_use"]: + obs = TOOL_IMPL[call.name](call.input) + tool_results.append({"type": "tool_result", "tool_use_id": call.id, "content": obs}) + messages.append({"role": "user", "content": tool_results}) # tool results 包在 user message +``` + +### OpenAI-compat 完整 loop + +```python +messages = [{"role": "user", "content": "..."}] +for step in range(5): + resp = client.chat.completions.create(model=MODEL, tools=TOOLS, messages=messages) + msg = resp.choices[0].message + messages.append({"role": "assistant", "content": msg.content, "tool_calls": msg.tool_calls}) + if not msg.tool_calls: + break + for tc in msg.tool_calls: + args = json.loads(tc.function.arguments) + obs = TOOL_IMPL[tc.function.name](args) + messages.append({"role": "tool", "tool_call_id": tc.id, "content": obs}) # 自己的 role +``` + +## 4 個容易踩錯的地方 + +1. **`parameters` vs `input_schema`**：複製貼上時最常踩——抄 Anthropic schema 直接給 OpenAI-compat 用會 silent fail（Ollama 不會 raise、就是不呼叫 tool）。 +2. **`call.input` vs `json.loads(arguments)`**：OpenAI-compat 忘記 `json.loads` 會拿到 string 不是 dict、KeyError。 +3. **Tool result 接回 role 不同**：Anthropic 用 `role="user"` + `[{"type": "tool_result", ...}]`；OpenAI-compat 用 `role="tool"` + 純 string content。 +4. **`stop_reason` vs `finish_reason`**：兩個都存在、但欄位名跟值不同。Anthropic `"end_turn"` / `"tool_use"`；OpenAI `"stop"` / `"tool_calls"`。 + +## 完整對照範例 + +每個 Stage 3 練習都同時 ship 兩個 starter： + +- `starter.py` = OpenAI-compat / Ollama +- `starter_anthropic.py` = Anthropic + +對照 [`../../stage-3/02-multi-tool-selection/`](../../../stage-3/02-multi-tool-selection/) ~ [`../../stage-3/06-schema-design/`](../../../stage-3/06-schema-design/) 任一個 folder。 diff --git a/examples/stage-5/tool-calling-tutor/references/sdk-diff.zh-Hans.md b/examples/stage-5/tool-calling-tutor/references/sdk-diff.zh-Hans.md new file mode 100644 index 0000000..74a4dae --- /dev/null +++ b/examples/stage-5/tool-calling-tutor/references/sdk-diff.zh-Hans.md @@ -0,0 +1,133 @@ +# SDK Diff：Anthropic vs OpenAI-compat（Ollama / OpenAI / Together / 多数开源） + +> [繁體中文](./sdk-diff.md) | **简体中文** | [English](./sdk-diff.en.md) + + +> 同一个 ReAct loop、不同 SDK 的 3 个关键差异。对应 SKILL.md Step 3。 + +## TL;DR 对照表 + +| 部分 | Anthropic SDK | OpenAI-compat SDK | +|---|---|---| +| **Tool schema 包法** | `tools=[{name, description, input_schema}]` | `tools=[{"type": "function", "function": {name, description, parameters}}]` | +| **Schema 字段名** | `input_schema` | `parameters` | +| **抓 tool call** | `[b for b in resp.content if b.type == "tool_use"]` | `resp.choices[0].message.tool_calls` | +| **Args 格式** | `call.input` 已是 dict | `call.function.arguments` 是 JSON string、要 `json.loads(...)` | +| **判完成** | `resp.stop_reason == "end_turn"` | `resp.choices[0].finish_reason == "stop"` | +| **Assistant 接回** | `messages.append({"role": "assistant", "content": resp.content})` | `messages.append({"role": "assistant", "content": msg.content, "tool_calls": msg.tool_calls})` | +| **Tool result 接回** | `messages.append({"role": "user", "content": [{"type": "tool_result", "tool_use_id": call.id, "content": obs}]})` | `messages.append({"role": "tool", "tool_call_id": tc.id, "content": obs})` | +| **Exception class** | `anthropic.RateLimitError` / `anthropic.APIConnectionError` | `openai.RateLimitError` / `openai.APIConnectionError` | + +## 并排程序码对照（单轮 tool call） + +### Anthropic + +```python +import anthropic + +client = anthropic.Anthropic() + +TOOLS = [{ + "name": "get_weather", + "description": "Use this when the user asks about current weather.", + "input_schema": { + "type": "object", + "properties": {"city": {"type": "string"}}, + "required": ["city"] + } +}] + +resp = client.messages.create( + model="claude-haiku-4-5", + max_tokens=512, + tools=TOOLS, + messages=[{"role": "user", "content": "台北现在有下雨吗？"}] +) + +# 抓 tool call +calls = [b for b in resp.content if b.type == "tool_use"] +city = calls[0].input["city"] # 已是 dict +``` + +### OpenAI-compat（Ollama） + +```python +from openai import OpenAI +import json + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +TOOLS = [{ + "type": "function", + "function": { + "name": "get_weather", + "description": "Use this when the user asks about current weather.", + "parameters": { # ← 名字不同 + "type": "object", + "properties": {"city": {"type": "string"}}, + "required": ["city"] + } + } +}] + +resp = client.chat.completions.create( + model="qwen2.5:3b", + tools=TOOLS, + messages=[{"role": "user", "content": "台北现在有下雨吗？"}] +) + +# 抓 tool call +tc = resp.choices[0].message.tool_calls[0] +args = json.loads(tc.function.arguments) # ← 要 json.loads +city = args["city"] +``` + +## 并排 ReAct loop（多轮） + +### Anthropic 完整 loop + +```python +messages = [{"role": "user", "content": "..."}] +for step in range(5): + resp = client.messages.create(model=MODEL, max_tokens=1024, tools=TOOLS, messages=messages) + messages.append({"role": "assistant", "content": resp.content}) # 整个 content list 直接接 + if resp.stop_reason == "end_turn": + break + tool_results = [] + for call in [b for b in resp.content if b.type == "tool_use"]: + obs = TOOL_IMPL[call.name](call.input) + tool_results.append({"type": "tool_result", "tool_use_id": call.id, "content": obs}) + messages.append({"role": "user", "content": tool_results}) # tool results 包在 user message +``` + +### OpenAI-compat 完整 loop + +```python +messages = [{"role": "user", "content": "..."}] +for step in range(5): + resp = client.chat.completions.create(model=MODEL, tools=TOOLS, messages=messages) + msg = resp.choices[0].message + messages.append({"role": "assistant", "content": msg.content, "tool_calls": msg.tool_calls}) + if not msg.tool_calls: + break + for tc in msg.tool_calls: + args = json.loads(tc.function.arguments) + obs = TOOL_IMPL[tc.function.name](args) + messages.append({"role": "tool", "tool_call_id": tc.id, "content": obs}) # 自己的 role +``` + +## 4 个容易踩错的地方 + +1. **`parameters` vs `input_schema`**：复制贴上时最常踩——抄 Anthropic schema 直接给 OpenAI-compat 用会 silent fail（Ollama 不会 raise、就是不调用 tool）。 +2. **`call.input` vs `json.loads(arguments)`**：OpenAI-compat 忘记 `json.loads` 会拿到 string 不是 dict、KeyError。 +3. **Tool result 接回 role 不同**：Anthropic 用 `role="user"` + `[{"type": "tool_result", ...}]`；OpenAI-compat 用 `role="tool"` + 纯 string content。 +4. **`stop_reason` vs `finish_reason`**：两个都存在、但字段名跟值不同。Anthropic `"end_turn"` / `"tool_use"`；OpenAI `"stop"` / `"tool_calls"`。 + +## 完整对照范例 + +每个 Stage 3 练习都同时 ship 两个 starter： + +- `starter.py` = OpenAI-compat / Ollama +- `starter_anthropic.py` = Anthropic + +对照 [`../../stage-3/02-multi-tool-selection/`](../../../stage-3/02-multi-tool-selection/) ~ [`../../stage-3/06-schema-design/`](../../../stage-3/06-schema-design/) 任一个 folder。 diff --git a/examples/stage-5/tool-calling-tutor/translations/SKILL.en.md b/examples/stage-5/tool-calling-tutor/translations/SKILL.en.md new file mode 100644 index 0000000..7241e90 --- /dev/null +++ b/examples/stage-5/tool-calling-tutor/translations/SKILL.en.md @@ -0,0 +1,107 @@ +--- +name: tool-calling-tutor +description: When the user is building a tool-calling agent and gets stuck — "why won't the LLM call my tool", "what's wrong with my schema", "tool was called but the args are wrong", "ReAct loop won't terminate", "help me design a function schema", "debug this tool-use behavior". Walks them through a 4-branch diagnostic + 5-step schema design walkthrough, with references to bad/good schema A/B and an SDK-diff cheatsheet. Do NOT use for: pure LangChain / LangGraph / CrewAI framework questions (route to Stage 4 frameworks), MCP server building (route to cookbook 2), production agent observability (route to Stage 7). +--- + +# Tool Calling Tutor + +You are now in the **tool-calling debugging** context. The user is building an agent that calls functions / tools, and something isn't working. Your job is to walk them through diagnosis + fix, not to write code for them. + +## Step 1 — Triage (the first thing you do) + +When the user mentions a tool-calling problem, ask **which of these 4 symptoms** they're hitting (one multiple-choice question): + +1. **(a) LLM won't call my tool** — model answers in natural language, no `tool_calls` triggered +2. **(b) Tool is called, but args are wrong** — right tool, but `arguments` are off (wrong type, missing field, nonsensical value) +3. **(c) ReAct loop won't stop / skips a step** — multi-step loop runs forever, or it skips a tool call in the middle +4. **(d) I'm starting from scratch, haven't written the schema** — user wants to build a new tool and design the schema + +**Don't guess** — make them pick one explicitly. Each branch leads to a different reference. + +## Step 2 — Branch by symptom + +### (a) LLM doesn't call the tool → fix description and tool boundaries + +The three most common causes (ask in this order): + +1. **`description` is too generic**: writing "Process data / Convert a value / Search things" like a human-facing docstring — the LLM can't tell *when* this tool applies. See [`references/debug-flowchart.en.md`](../references/debug-flowchart.en.md) Section A. +2. **Multiple tools have overlapping boundaries**: both descriptions match the user query — LLM can't pick — so it picks neither. +3. **The query genuinely doesn't need a tool**: "Tell me about Python" doesn't need any tool; pure text response is correct. + +**Fix**: rewrite `description` from "**what it does**" to "**when to use it**". Compare with [`references/schema-evolution.en.md`](../references/schema-evolution.en.md) for the bad → good A/B. + +### (b) Tool called, args wrong → fix the parameters schema + +Three common causes: + +1. **All params typed as `string`**: `{"value": {"type": "string"}}` — the LLM doesn't know to pass a number. Change to `{"type": "number"}`. +2. **No `required`**: the model can skip a mandatory field. List `"required": ["value", "unit"]`. +3. **Missing `enum`**: `unit: string` lets the LLM pass `"C"` / `"Celsius"` / `"celsius"` at random. Switch to `"enum": ["celsius", "fahrenheit"]`. + +See [`references/schema-evolution.en.md`](../references/schema-evolution.en.md) for the 4-step improvement. + +### (c) ReAct loop won't stop / skips → check control flow + +Three typical reasons a loop won't stop: + +1. **Forgot to append assistant response to `messages`** — next round, the LLM can't see what it just said, infinite repeat +2. **`tool` message missing `tool_call_id`** — LLM can't pair which result goes with which call, may re-issue the call +3. **No `max_iter` safety net** — if a tool returns garbage, LLM keeps calling + +Reasons for skipped steps in a multi-step task: + +1. **Model not strong enough**: qwen2.5:3b on a 4-step task may skip "convert to percentage". Try `MODEL=qwen2.5:7b` or `MODEL=claude-haiku-4-5`. +2. **Tool description omits the prerequisite ordering**: e.g., `to_percentage` should say "Convert a ratio (e.g., 0.31) into percentage. Call this LAST after dividing." Make the order explicit. + +**Compare runnable examples** → [`../../stage-3/03-react-from-scratch/`](../../../stage-3/03-react-from-scratch/) and [`../../stage-3/04-multi-step-reasoning/`](../../../stage-3/04-multi-step-reasoning/) full starters. + +### (d) Designing from scratch → follow the 5-step recipe + +For any new tool, do these 5 steps: + +1. **Define**: one sentence on what this tool does (≤15 words). Can't write it = scope too big, split it. +2. **Describe (from the LLM's POV)**: write the description as "**Use this when the user asks to / mentions / wants** ...", not "This function ...". +3. **Type**: give each param the correct type — `number` / `boolean` / `array` / `object`. Don't default everything to `string`. +4. **Constrain**: list mandatory fields in `required`; use `enum` to collapse fuzzy boundaries; describe each field. +5. **Error pattern**: on failure, return `{"error": "...", "retry_hint": "..."}` as a structured dict — **don't `raise`**. In production, retry is the LLM's decision. + +**Fork template**: copy [`../../stage-3/02-multi-tool-selection/starter.py`](../../../stage-3/02-multi-tool-selection/starter.py) (single-turn) or [`../../stage-3/03-react-from-scratch/starter.py`](../../../stage-3/03-react-from-scratch/starter.py) (multi-turn loop) — keep the `TOOLS_SPEC` + `TOOL_IMPL` structure, swap in your tool. + +## Step 3 — SDK differences reminder + +The user might be switching between Anthropic / OpenAI / Ollama — the SDK shape differs. See [`references/sdk-diff.en.md`](../references/sdk-diff.en.md) for the 3-line diff table. **Don't assume — ask "which SDK are you using" explicitly.** + +## Step 4 — Mock test first (strongly recommended) + +Every tool-calling program should have mock-based tests that don't hit a real API: + +- Path A (Ollama) — mock the OpenAI-compat response shape +- Path B (Anthropic) — mock content blocks + +Full mock pattern → [`../../stage-3/03-react-from-scratch/test.py`](../../../stage-3/03-react-from-scratch/test.py). **Get tests passing before hooking up a real LLM** — saves ~80% of debug time. + +## Step 5 — When to escalate / route away + +This skill does **NOT** handle: + +- **LangChain / LangGraph / CrewAI / Pydantic AI** framework questions → Stage 4 +- **MCP server / client** design → [`resources/cookbook.md` 2](../../../../resources/cookbook.md) +- **Production monitoring / observability / cost tracking** → Stage 7 +- **General prompt engineering** → Stage 2 + +If the user asks about any of these, tell them "this skill handles tool-use mechanics; your question needs Stage X — see ..." and route — don't try to absorb it. + +## Don't + +- **Don't just write a complete `starter.py` for them** — they need to build the mental model, not copy-paste an answer. Point them to fork [`../../stage-3/`](../../../stage-3/) starters and swap in their `TOOLS_SPEC`. +- **Don't skip Step 1 triage** — the 4 symptoms have different fixes; guessing wastes time. +- **Don't assume the user is using Claude** — Path A default is Ollama qwen2.5:3b. Ask, then answer. +- **Don't recite all the schema-design rules** — `resources/schema-design-cheatsheet.en.md` already has them; just point. + +## References + +- [`references/debug-flowchart.en.md`](../references/debug-flowchart.en.md) — 4-symptom diagnostic for "why won't the LLM call my tool" +- [`references/schema-evolution.en.md`](../references/schema-evolution.en.md) — Bad → good schema worked example (4 improvements) +- [`references/sdk-diff.en.md`](../references/sdk-diff.en.md) — Anthropic vs OpenAI-compat side-by-side +- [`resources/schema-design-cheatsheet.en.md`](../../../../resources/schema-design-cheatsheet.en.md) — 5 golden rules + 5 anti-patterns (existing curriculum resource) +- [`resources/glossary.en.md` 2](../../../../resources/glossary.en.md) — Agent / Tool Use / ReAct term definitions diff --git a/examples/stage-5/tool-calling-tutor/translations/SKILL.zh-Hans.md b/examples/stage-5/tool-calling-tutor/translations/SKILL.zh-Hans.md new file mode 100644 index 0000000..6ec8b14 --- /dev/null +++ b/examples/stage-5/tool-calling-tutor/translations/SKILL.zh-Hans.md @@ -0,0 +1,107 @@ +--- +name: tool-calling-tutor +description: When the user is building a tool-calling agent and gets stuck — "为什么 LLM 不调用我的 tool", "我这 schema 哪里写坏", "tool 被调用但 args 不对", "ReAct loop 跑不停", "the LLM won't call my tool", "help me design a function schema", "debug this tool-use behavior". Walks them through a 4-branch diagnostic + 5-step schema design walkthrough, with references to bad/good schema A/B and SDK-diff cheatsheet. Do NOT use for: pure LangChain / LangGraph / CrewAI framework questions (route to Stage 4 frameworks), MCP server building (route to cookbook 2), production agent observability (route to Stage 7). +--- + +# Tool Calling Tutor + +You are now in the **tool-calling debugging** context. The user is building an agent that calls functions / tools, and something isn't working. Your job is to walk them through diagnosis + fix, not to write code for them. + +## Step 1 — Triage（first thing you do） + +When the user mentions tool calling problems, ask **which of these 4 symptoms** they're hitting (one question, multiple choice): + +1. **(a) LLM 不调用我的 tool** — 模型直接用自然语言回答、完全没触发 tool_calls +2. **(b) Tool 被调用、但参数错** — 调用对 tool，但 `arguments` 不对（类型错、缺字段、值不合理） +3. **(c) ReAct loop 跑不停 / 漏步** — 多步 loop 无限循环，或者中间漏一个 tool 没调用 +4. **(d) 我从零开始、还没写 schema** — 用户要新做一个 tool、想知道 schema 怎么设计 + +**不要猜**——让用户明确选一个。每个 branch 走的 reference 不同。 + +## Step 2 — Branch by symptom + +### (a) LLM 不调用 tool → 看 description 与工具边界 + +最常见 3 个原因（按优先顺序问）： + +1. **`description` 太笼统**：写的是“处理数据 / Convert a value / Search things”这种给人读的 docstring，LLM 看不到“这个 tool 解什么具体问题”。看 [`references/debug-flowchart.zh-Hans.md`](../references/debug-flowchart.zh-Hans.md) Section A。 +2. **多 tool 边界互相重叠**：两个 tool 的 description 都能套到 user query、LLM 选不出来、干脆都不选。 +3. **问题本身用不到 tool**：user query 是“介绍一下 Python”这种纯知识题、tool list 里也没适合的、LLM 直接纯文字回答是正确的。 + +**怎么修**：把 `description` 从“**做什么**”改写成“**何时用**”。对照 [`references/schema-evolution.zh-Hans.md`](../references/schema-evolution.zh-Hans.md) 的 bad → good A/B。 + +### (b) Tool 被调用、但参数错 → 看 parameters schema + +最常见 3 个原因： + +1. **参数类型全用 `string`**：`{"value": {"type": "string"}}` LLM 不知道要传 number。改成 `{"type": "number"}`。 +2. **没有 `required`**：模型可能漏传必填字段。明列 `"required": ["value", "unit"]`。 +3. **enum 该用没用**：`unit: string` 让 LLM 传 `"C"` `"Celsius"` `"celsius"` 都有可能。改 `"enum": ["celsius", "fahrenheit"]`。 + +**对照** [`references/schema-evolution.zh-Hans.md`](../references/schema-evolution.zh-Hans.md) 的 4 个改进。 + +### (c) ReAct loop 跑不停 / 漏步 → 看 control flow + +跑不停的 3 个典型原因： + +1. **忘记把 assistant response 加回 `messages`**——下轮 LLM 看不到自己上轮讲过什么、会无限重复 +2. **`tool` message 没带 `tool_call_id`**——LLM 无法配对哪个 result 对应哪个 call、可能重新发起 tool call +3. **没设 `max_iter` safety net**——当 tool 结果写得不好、LLM 会无限调用 + +漏步（多步任务中间少一步）的原因： + +1. **Model 不够强**：qwen2.5:3b 在 4-step task 上会漏“转百分比”这种子步骤。试 `MODEL=qwen2.5:7b` 或 `MODEL=claude-haiku-4-5`。 +2. **Tool description 没讲“必要前置”**：譬如 `to_percentage` 应该写“Convert a ratio (e.g., 0.31) into percentage. Call this LAST after dividing.”明示顺序。 + +**对照可跑范例** → [`../../stage-3/03-react-from-scratch/`](../../../stage-3/03-react-from-scratch/) 跟 [`../../stage-3/04-multi-step-reasoning/`](../../../stage-3/04-multi-step-reasoning/) 的完整 starter。 + +### (d) 从零设计 schema → 走 5 步法 + +对任何新 tool，按这 5 步： + +1. **Define**：一句话讲这个 tool 做什么（不超过 15 字）。写不出来 = tool scope 太大、要拆。 +2. **Describe（LLM 视角）**：把 description 写成“**Use this when the user asks to / mentions / wants** ...”格式，不是“This function ...”。 +3. **Type**：每个 param 用正确 type — `number` / `boolean` / `array` / `object`，不要全 `string`。 +4. **Constrain**：`required` 列必填字段；模糊边界用 `enum` 收敛；`description` 补字段用途。 +5. **Error pattern**：tool 失败回传 `{"error": "...", "retry_hint": "..."}` 结构化 dict，**不要 `raise`**——production 的 retry 由 LLM 决定。 + +**Fork template**：直接 copy [`../../stage-3/02-multi-tool-selection/starter.py`](../../../stage-3/02-multi-tool-selection/starter.py)（单轮 tool）或 [`../../stage-3/03-react-from-scratch/starter.py`](../../../stage-3/03-react-from-scratch/starter.py)（多轮 loop）的 `TOOLS_SPEC` + `TOOL_IMPL` 结构、改成你的 tool。 + +## Step 3 — SDK 差异提醒 + +用户可能在 Anthropic / OpenAI / Ollama 之间切换、SDK shape 不同。看 [`references/sdk-diff.zh-Hans.md`](../references/sdk-diff.zh-Hans.md) 的 3 行对照表。**不要假设用户知道——主动问一次“你用哪个 SDK”**。 + +## Step 4 — Mock test first（强烈建议） + +每个 tool-calling 程序都应该有 mock-based test、不打真 API： + +- Path A (Ollama) 用 OpenAI-compat response shape mock +- Path B (Anthropic) 用 content block mock + +完整 mock pattern 对照 [`../../stage-3/03-react-from-scratch/test.py`](../../../stage-3/03-react-from-scratch/test.py)。**先把 test 跑通、再连真的 LLM**——可以省下 80% 的 debug 时间。 + +## Step 5 — When to escalate / route away + +这个 skill **不**处理： + +- **LangChain / LangGraph / CrewAI / Pydantic AI** 等 framework 问题 → 路 Stage 4 +- **MCP server / client** 设计 → 路 [`resources/cookbook.md` 2 写你的第一个 MCP server](../../../../resources/cookbook.md) +- **Production 监控 / observability / cost tracking** → 路 Stage 7 +- **Prompt engineering 一般技巧** → 路 Stage 2 + +碰到这些情境、直接告诉用户“这个 skill 处理 tool-use mechanics、你这个问题需要 Stage X、建议去看 ...”、不要硬吃下去。 + +## Don't + +- **不要直接帮用户写一整份 starter.py**——他们需要练 mental model、不是拿到答案 copy-paste。指他们 fork [`../../stage-3/`](../../../stage-3/) 的 starter、改 TOOLS_SPEC 就好。 +- **不要跳过 Step 1 triage**——4 个 symptom 的修法不同，没问清楚就猜会浪费时间。 +- **不要假设 user 用 Claude**——Path A 默认是 Ollama qwen2.5:3b，先问再答。 +- **不要把 schema-design 规则背一遍**——`resources/schema-design-cheatsheet.zh-Hans.md` 已经写好，指过去就行。 + +## References + +- [`references/debug-flowchart.zh-Hans.md`](../references/debug-flowchart.zh-Hans.md) — "为什么 LLM 不调用我的 tool" 4-symptom 诊断 +- [`references/schema-evolution.zh-Hans.md`](../references/schema-evolution.zh-Hans.md) — Bad → Good schema worked example（4 个改进步骤） +- [`references/sdk-diff.zh-Hans.md`](../references/sdk-diff.zh-Hans.md) — Anthropic vs OpenAI-compat 并排表 +- [`resources/schema-design-cheatsheet.zh-Hans.md`](../../../../resources/schema-design-cheatsheet.zh-Hans.md) — 5 条黄金规则 + 5 个 anti-pattern（curriculum 既有资源） +- [`resources/glossary.zh-Hans.md` 2](../../../../resources/glossary.zh-Hans.md) — Agent / Tool Use / ReAct 名词定义 diff --git a/examples/stage-6/01-embeddings/README.en.md b/examples/stage-6/01-embeddings/README.en.md new file mode 100644 index 0000000..992c868 --- /dev/null +++ b/examples/stage-6/01-embeddings/README.en.md @@ -0,0 +1,95 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# Exercise 1: Embeddings + Nearest Neighbors + +Pairs with [Stage 6 — Memory & RAG](../../../stages/06-memory-rag.en.md) Exercise 1. + +## Task + +Embed 100 sentences, then for a query find the top-k most similar. Observe what cosine similarity ranking means. + +## How to run — two paths + +### Path A (default, free, local) + +```bash +pip install -r requirements.txt +python starter.py # downloads ~80 MB on first run +``` + +Budget: **$0**. `sentence-transformers/all-MiniLM-L6-v2` runs on CPU, ~100 sentences in < 1 second. + +### Path B (cloud embedding, comparison, very cheap) + +```bash +pip install -r requirements.txt +export OPENAI_API_KEY=sk-... +python starter_anthropic.py +``` + +Budget: ~**$0.00002** per run (text-embedding-3-small, 100 sentences). + +> 💡 **Anthropic doesn't provide an embedding API** — they officially recommend [Voyage AI](https://www.voyageai.com/). This demo uses OpenAI (most common); swapping to Voyage is a client swap. + +## Validate the logic + +```bash +python test.py # mock SentenceTransformer, no download +python test_anthropic.py # mock OpenAI client, validate normalize +``` + +## Core concepts + +```python +# 1. Encode → vector +sent_vecs = model.encode(sentences, normalize_embeddings=True) # 100 × 384 vec +q_vec = model.encode([query], normalize_embeddings=True)[0] # 384 vec + +# 2. Cosine similarity = dot product (because normalized) +sims = sent_vecs @ q_vec # 100 similarity scores + +# 3. Top-k +top_idx = np.argsort(-sims)[:top_k] +``` + +**Why normalize**: normalized vectors' dot product equals cosine similarity directly (range [-1, 1]) — no need to recompute norms. Standard vector DB trick. + +## Local vs cloud embedding + +| Dimension | sentence-transformers (local) | OpenAI text-embedding-3-small (cloud) | +|---|---|---| +| Dims | 384 | 1536 | +| Speed (100 sents, CPU) | < 1s | 1-2s (incl. network) | +| Cost | $0 | $0.00002 / 100 sentences | +| Multilingual | OK (`paraphrase-multilingual-MiniLM-L12-v2`) | Strong | +| Long context (>512 tokens) | Truncated | Strong | +| Determinism | 100% | 99% (API has minor noise) | + +**Bottom line**: personal / small data / local experimentation — sentence-transformers is plenty. Heavy multilingual / long docs / SaaS — go cloud. + +## Common pitfalls + +- **No normalization**: cosine ≠ dot product; compute `sim = dot(a,b) / (|a||b|)` yourself +- **Mixed precision**: sentence-transformers defaults to fp32; fp16 quantization (memory savings) shifts similarities 1-2% +- **Don't compare vectors across models**: MiniLM and OpenAI are different semantic spaces; cosines aren't comparable +- **Tiny queries**: 1-2 word queries embed poorly; use full sentences + +## Want better embeddings? + +```bash +# Larger local model (better accuracy, slower) +# In starter.py change MODEL_NAME to: +# "sentence-transformers/all-mpnet-base-v2" # 768 dims, ↑ accuracy +# "sentence-transformers/paraphrase-multilingual-..." # multilingual + +# Higher-quality cloud +EMBED_MODEL=text-embedding-3-large python starter_anthropic.py # 3072 dims, $$ +``` + +## Extensions + +- **BM25 + embedding hybrid**: combine keyword and semantic — common in production +- **Add a reranker**: feed top-k to a cross-encoder (`cross-encoder/ms-marco-MiniLM-L-6-v2`) — big precision lift +- **Plug into Exercise 2 vector DB**: store in Chroma so you don't re-embed each run diff --git a/examples/stage-6/01-embeddings/README.md b/examples/stage-6/01-embeddings/README.md new file mode 100644 index 0000000..5599696 --- /dev/null +++ b/examples/stage-6/01-embeddings/README.md @@ -0,0 +1,102 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 練習 1：Embeddings + nearest neighbors + +對應 [Stage 6 — Memory & RAG](../../../stages/06-memory-rag.md) 練習 1。 +> 🎓 **學習模式**：這份 `starter.py` 是**完整解答**、不是 TODO skeleton。建議用**主動模式**——`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫一份 `starter.py`、跑 `python test.py` 驗證；卡 20 分鐘再回去對照 reference。完整方法論看 [`docs/HOW_TO_USE.md`](../../../docs/HOW_TO_USE.md)。 + +> 📚 **想要 chapter-length 深入版？** 本 folder 的 starter 是 illustrative 版、聚焦核心 pattern + 兩條 SDK path，不是進階深度教材。深度教材推薦： +> - [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents) ⭐ 中文圈最完整、章節式 + 16 種 production 能力。**本練習對應 hello-agents 的 embedding 章節** +> - [sentence-transformers official docs](https://www.sbert.net/) + [MTEB leaderboard](https://huggingface.co/spaces/mteb/leaderboard)（embedding model 評分排行） +> - 完整 references 見 [Stage 6 精選 Projects](../../../stages/06-memory-rag.md#-精選-projects範本--spec--範例-collection) + + +## 任務 + +把 100 個句子做 embedding、給一個 query、找出 top-k 最相近的句子。觀察 cosine similarity 排序意義。 + +## 怎麼跑 — 兩條路徑 + +### Path A（默認、本機免費） + +```bash +pip install -r requirements.txt +python starter.py # 第一次自動下載 model (~80 MB) +``` + +預算：**$0**。`sentence-transformers/all-MiniLM-L6-v2` 模型在 CPU 跑、約 100 句 < 1 秒。 + +### Path B（cloud embedding，對照、極便宜） + +```bash +pip install -r requirements.txt +export OPENAI_API_KEY=sk-... +python starter_anthropic.py +``` + +預算：每次 ≈ **$0.00002**（text-embedding-3-small、100 句）。 + +> 💡 **Anthropic 沒提供 embedding API**——官方推薦 [Voyage AI](https://www.voyageai.com/)。這份用 OpenAI 示範（最常見），改 Voyage 只需換 client。 + +## 不花錢驗證程式邏輯 + +```bash +python test.py # mock SentenceTransformer、不下載 model +python test_anthropic.py # mock OpenAI client、驗 normalize 邏輯 +``` + +## 核心觀念 + +```python +# 1. Encode → vector +sent_vecs = model.encode(sentences, normalize_embeddings=True) # 100 × 384 vec +q_vec = model.encode([query], normalize_embeddings=True)[0] # 384 vec + +# 2. Cosine similarity = dot product (因為 normalized) +sims = sent_vecs @ q_vec # 100 個 similarity score + +# 3. Top-k +top_idx = np.argsort(-sims)[:top_k] +``` + +**為什麼 normalize**：normalized vector 的 dot product 直接等於 cosine similarity（範圍 [-1, 1]）、不用每次重算 norm。是 vector DB 通用技巧。 + +## 本機 vs cloud embedding 對照 + +| 維度 | sentence-transformers (本機) | OpenAI text-embedding-3-small (cloud) | +|---|---|---| +| 維度 | 384 | 1536 | +| 速度（100 句、CPU） | < 1 秒 | 1-2 秒（含網路） | +| 成本 | $0 | $0.00002 / 100 sentences | +| Multilingual | OK（多語版見 `paraphrase-multilingual-MiniLM-L12-v2`） | 強 | +| Long context（>512 token） | 截斷 | 強 | +| 一致性（同 input 同 output） | 100% | 99%（API 偶爾微擾） | + +**結論**：個人 / 小資料 / 本機實驗、用 sentence-transformers 完全夠。大量 multilingual / 長文檔 / SaaS、用 cloud。 + +## 常見坑 + +- **沒 normalize**：cosine similarity ≠ dot product、要 `sim = dot(a,b) / (|a||b|)` 自己算 +- **Mixed precision**：sentence-transformers 預設 fp32、若用 fp16 量化（省記憶體）相似度會差 1-2% +- **不同模型 vector 不能比**：MiniLM 跟 OpenAI 是兩個語意空間、不要把 cosine sim 直接比 +- **太短 query**：1-2 字 query embedding 不穩、結果可能跳很遠。query 至少要句子 + +## 想看更好的 embedding？ + +```bash +# 本機更大 model（精度更好、速度較慢） +# 把 starter.py 的 MODEL_NAME 改成： +# "sentence-transformers/all-mpnet-base-v2" # 768 維、accuracy↑ +# "sentence-transformers/paraphrase-multilingual-..." # 多語 + +# Cloud 高精度 +EMBED_MODEL=text-embedding-3-large python starter_anthropic.py # 3072 維、$$ +``` + +## 延伸 + +- **改成 BM25 + embedding hybrid**：keyword 跟 semantic 各取優、production 常用 +- **加 reranker**：top-k 拿來丟 cross-encoder（`cross-encoder/ms-marco-MiniLM-L-6-v2`）做 reranking、精度大躍進 +- **接練習 2 vector DB**：放到 Chroma 裡能跑萬筆規模、不必每次重 embedding diff --git a/examples/stage-6/01-embeddings/README.zh-Hans.md b/examples/stage-6/01-embeddings/README.zh-Hans.md new file mode 100644 index 0000000..eefe5da --- /dev/null +++ b/examples/stage-6/01-embeddings/README.zh-Hans.md @@ -0,0 +1,95 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 练习 1：Embeddings + nearest neighbors + +对应 [Stage 6 — Memory & RAG](../../../stages/06-memory-rag.zh-Hans.md) 练习 1。 + +## 任务 + +把 100 个句子做 embedding、给一个 query、找出 top-k 最相近的句子。观察 cosine similarity 排序意义。 + +## 怎么跑 — 两条路径 + +### Path A（默认、本机免费） + +```bash +pip install -r requirements.txt +python starter.py # 第一次自动下载 model (~80 MB) +``` + +预算：**$0**。`sentence-transformers/all-MiniLM-L6-v2` 模型在 CPU 跑、约 100 句 < 1 秒。 + +### Path B（cloud embedding，对照、极便宜） + +```bash +pip install -r requirements.txt +export OPENAI_API_KEY=sk-... +python starter_anthropic.py +``` + +预算：每次 ≈ **$0.00002**（text-embedding-3-small、100 句）。 + +> 💡 **Anthropic 没提供 embedding API**——官方推荐 [Voyage AI](https://www.voyageai.com/)。这份用 OpenAI 示范（最常见），改 Voyage 只需换 client。 + +## 不花钱验证程序逻辑 + +```bash +python test.py # mock SentenceTransformer、不下载 model +python test_anthropic.py # mock OpenAI client、验 normalize 逻辑 +``` + +## 核心观念 + +```python +# 1. Encode → vector +sent_vecs = model.encode(sentences, normalize_embeddings=True) # 100 × 384 vec +q_vec = model.encode([query], normalize_embeddings=True)[0] # 384 vec + +# 2. Cosine similarity = dot product (因为 normalized) +sims = sent_vecs @ q_vec # 100 个 similarity score + +# 3. Top-k +top_idx = np.argsort(-sims)[:top_k] +``` + +**为什么 normalize**：normalized vector 的 dot product 直接等于 cosine similarity（范围 [-1, 1]）、不用每次重算 norm。是 vector DB 通用技巧。 + +## 本机 vs cloud embedding 对照 + +| 维度 | sentence-transformers (本机) | OpenAI text-embedding-3-small (cloud) | +|---|---|---| +| 维度 | 384 | 1536 | +| 速度（100 句、CPU） | < 1 秒 | 1-2 秒（含网络） | +| 成本 | $0 | $0.00002 / 100 sentences | +| Multilingual | OK（多语版见 `paraphrase-multilingual-MiniLM-L12-v2`） | 强 | +| Long context（>512 token） | 截断 | 强 | +| 一致性（同 input 同 output） | 100% | 99%（API 偶尔微扰） | + +**结论**：个人 / 小数据 / 本机实验、用 sentence-transformers 完全够。大量 multilingual / 长文档 / SaaS、用 cloud。 + +## 常见坑 + +- **没 normalize**：cosine similarity ≠ dot product、要 `sim = dot(a,b) / (|a||b|)` 自己算 +- **Mixed precision**：sentence-transformers 预设 fp32、若用 fp16 量化（省记忆体）相似度会差 1-2% +- **不同模型 vector 不能比**：MiniLM 跟 OpenAI 是两个语意空间、不要把 cosine sim 直接比 +- **太短 query**：1-2 字 query embedding 不稳、结果可能跳很远。query 至少要句子 + +## 想看更好的 embedding？ + +```bash +# 本机更大 model（精度更好、速度较慢） +# 把 starter.py 的 MODEL_NAME 改成： +# "sentence-transformers/all-mpnet-base-v2" # 768 维、accuracy↑ +# "sentence-transformers/paraphrase-multilingual-..." # 多语 + +# Cloud 高精度 +EMBED_MODEL=text-embedding-3-large python starter_anthropic.py # 3072 维、$$ +``` + +## 延伸 + +- **改成 BM25 + embedding hybrid**：keyword 跟 semantic 各取优、production 常用 +- **加 reranker**：top-k 拿来丢 cross-encoder（`cross-encoder/ms-marco-MiniLM-L-6-v2`）做 reranking、精度大跃进 +- **接练习 2 vector DB**：放到 Chroma 里能跑万笔规模、不必每次重 embedding diff --git a/examples/stage-6/01-embeddings/requirements.txt b/examples/stage-6/01-embeddings/requirements.txt new file mode 100644 index 0000000..1ba447d --- /dev/null +++ b/examples/stage-6/01-embeddings/requirements.txt @@ -0,0 +1,3 @@ +sentence-transformers>=2.7,<4.0 +numpy>=1.24 +openai>=1.50,<2.0 # Path B 用 OpenAI embeddings；不裝也能跑 Path A diff --git a/examples/stage-6/01-embeddings/starter.py b/examples/stage-6/01-embeddings/starter.py new file mode 100644 index 0000000..9e6414c --- /dev/null +++ b/examples/stage-6/01-embeddings/starter.py @@ -0,0 +1,77 @@ +"""Stage 6 練習 1：Embeddings — Path A（本機 sentence-transformers、$0）。 + +把 100 個句子轉成 vector、查 nearest neighbors。理解 cosine similarity / distance。 + +跑法： + pip install -r requirements.txt + python starter.py # 第一次會自動下載 model (~80MB) + +驗證： + python test.py + +預算：$0（model 跑在 CPU）。Path B 用 cloud embedding（OpenAI / Voyage）見 starter_anthropic.py。 +""" + +from __future__ import annotations + +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import numpy as np +from sentence_transformers import SentenceTransformer + +MODEL_NAME = "sentence-transformers/all-MiniLM-L6-v2" + + +SENTENCES = [ + "The cat sat on the mat.", + "A dog is chasing the ball in the park.", + "Taipei is the capital of Taiwan.", + "Python is a programming language.", + "Machine learning models learn patterns from data.", + "Neural networks are inspired by the human brain.", + "Coffee shops in Taipei often serve great pour-over.", + "JavaScript runs in web browsers.", + "The weather forecast says it will rain tomorrow.", + "Cats and dogs are common household pets.", +] + [f"Random filler sentence number {i}." for i in range(90)] + + +def embed(texts: list[str], model: Any = None) -> np.ndarray: + """Encode a list of strings to vectors (L2-normalized).""" + model = model or SentenceTransformer(MODEL_NAME) + return model.encode(texts, normalize_embeddings=True, convert_to_numpy=True) + + +def find_nearest(query: str, sentences: list[str], top_k: int = 3, model: Any = None) -> list[dict]: + """Returns top-k nearest sentences with cosine similarity scores.""" + model = model or SentenceTransformer(MODEL_NAME) + sent_vecs = embed(sentences, model=model) + q_vec = embed([query], model=model)[0] + # cosine similarity = dot product (because normalized) + sims = sent_vecs @ q_vec + top_idx = np.argsort(-sims)[:top_k] + return [ + {"sentence": sentences[i], "similarity": float(sims[i]), "rank": int(rank)} + for rank, i in enumerate(top_idx, 1) + ] + + +if __name__ == "__main__": + query = "I love programming in Python" + print(f"❓ Query: {query}") + print(f" Embedding {len(SENTENCES)} sentences with {MODEL_NAME}...") + print("-" * 60) + + results = find_nearest(query, SENTENCES, top_k=5) + for r in results: + print(f" #{r['rank']} sim={r['similarity']:.3f}: {r['sentence']}") + + # Validate top-1 is semantically related + assert "Python" in results[0]["sentence"] or "programming" in results[0]["sentence"].lower(), \ + f"預期 top-1 跟 programming 相關、得到 {results[0]['sentence']}" + print("\n✅ 練習 1 通過 — sentence-transformers 在本機跑通、$0/run") + print(" 觀察：cosine similarity 越高、語意越近") diff --git a/examples/stage-6/01-embeddings/starter_anthropic.py b/examples/stage-6/01-embeddings/starter_anthropic.py new file mode 100644 index 0000000..bd181be --- /dev/null +++ b/examples/stage-6/01-embeddings/starter_anthropic.py @@ -0,0 +1,62 @@ +"""Stage 6 練習 1：Embeddings — Path B（cloud embedding 對照）。 + +Anthropic 沒提供官方 embedding API、官方推薦 Voyage AI。OpenAI 也是常見選擇。 +這份示範用 OpenAI 的 text-embedding-3-small 對照本機 sentence-transformers。 + +跑法： + pip install -r requirements.txt + export OPENAI_API_KEY=sk-... # OR set VOYAGE_API_KEY + python starter_anthropic.py + +預算：每次 ≈ $0.00002 per 100 sentences（text-embedding-3-small）。極便宜。 +但**對 RAG demo / 個人實驗，本機 sentence-transformers 完全夠用**——cloud embedding 主要差別在 multilingual 跟 long-context 場景。 +""" + +from __future__ import annotations + +import os +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import numpy as np + +MODEL = os.environ.get("EMBED_MODEL", "text-embedding-3-small") + + +def embed_openai(texts: list[str], client: Any = None) -> np.ndarray: + """Encode via OpenAI embeddings API.""" + from openai import OpenAI + client = client or OpenAI() + resp = client.embeddings.create(model=MODEL, input=texts) + vecs = np.array([d.embedding for d in resp.data]) + # Normalize for cosine similarity + norms = np.linalg.norm(vecs, axis=1, keepdims=True) + return vecs / norms + + +def find_nearest_openai(query: str, sentences: list[str], top_k: int = 3) -> list[dict]: + from openai import OpenAI + client = OpenAI() + sent_vecs = embed_openai(sentences, client=client) + q_vec = embed_openai([query], client=client)[0] + sims = sent_vecs @ q_vec + top_idx = np.argsort(-sims)[:top_k] + return [ + {"sentence": sentences[i], "similarity": float(sims[i]), "rank": int(rank)} + for rank, i in enumerate(top_idx, 1) + ] + + +if __name__ == "__main__": + from starter import SENTENCES + query = "I love programming in Python" + print(f"❓ Query: {query}（using OpenAI {MODEL}）") + print("-" * 60) + results = find_nearest_openai(query, SENTENCES, top_k=5) + for r in results: + print(f" #{r['rank']} sim={r['similarity']:.3f}: {r['sentence']}") + print("\n✅ 練習 1 (Path B cloud) 通過 — OpenAI embeddings ≈$0.00002/run") + print(" 對照本機版本（starter.py）、看 ranking 差異") diff --git a/examples/stage-6/01-embeddings/test.py b/examples/stage-6/01-embeddings/test.py new file mode 100644 index 0000000..61229a2 --- /dev/null +++ b/examples/stage-6/01-embeddings/test.py @@ -0,0 +1,92 @@ +"""Stage 6 練習 1 自我驗證 — 不下載 model、用 mock。 + +跑法： + python test.py + +驗證內容： + - find_nearest 排序正確（相關句子 sim 最高） + - L2-normalized vector 的 dot product = cosine similarity + - top_k 邊界 +""" + +from __future__ import annotations + +import sys +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import numpy as np + +from starter import SENTENCES, find_nearest + + +def make_fake_model(vec_map: dict) -> MagicMock: + """Mock SentenceTransformer：給定 text → vec 對應表。""" + m = MagicMock() + def encode(texts, **kw): + out = np.array([vec_map.get(t, np.zeros(4)) for t in texts]) + # Normalize（模擬 normalize_embeddings=True） + norms = np.linalg.norm(out, axis=1, keepdims=True) + norms[norms == 0] = 1.0 + return out / norms + m.encode.side_effect = encode + return m + + +def test_top_1_is_most_similar(): + """構造 4 個 sentence、其中第 2 個跟 query 完全同向、應排第 1。""" + sentences = ["a", "b", "c", "d"] + # 故意做：b 跟 query 同向（→ similarity 1.0） + vecs = { + "a": np.array([1, 0, 0, 0], dtype=float), + "b": np.array([0, 1, 0, 0], dtype=float), + "c": np.array([0, 0, 1, 0], dtype=float), + "d": np.array([0, 0, 0, 1], dtype=float), + "query for b": np.array([0, 1, 0, 0], dtype=float), + } + model = make_fake_model(vecs) + results = find_nearest("query for b", sentences, top_k=2, model=model) + assert results[0]["sentence"] == "b" + assert results[0]["similarity"] == 1.0 + print("✅ test_top_1_is_most_similar") + + +def test_top_k_respected(): + sentences = ["a", "b", "c", "d", "e"] + vecs = {s: np.random.rand(4) for s in sentences + ["q"]} + model = make_fake_model(vecs) + results = find_nearest("q", sentences, top_k=3, model=model) + assert len(results) == 3 + assert {r["rank"] for r in results} == {1, 2, 3} + print("✅ test_top_k_respected") + + +def test_similarity_in_minus_one_to_one(): + """L2-normalized → cosine sim ∈ [-1, 1]。""" + sentences = ["a", "b"] + vecs = { + "a": np.array([1, 0, 0, 0], dtype=float), + "b": np.array([-1, 0, 0, 0], dtype=float), + "q": np.array([1, 0, 0, 0], dtype=float), + } + model = make_fake_model(vecs) + results = find_nearest("q", sentences, top_k=2, model=model) + sims = [r["similarity"] for r in results] + assert all(-1.0 - 1e-6 <= s <= 1.0 + 1e-6 for s in sims), f"sim 越界: {sims}" + print("✅ test_similarity_in_minus_one_to_one") + + +def test_corpus_size(): + """SENTENCES corpus 應該有 100 個。""" + assert len(SENTENCES) == 100, f"預期 100 個句子、得到 {len(SENTENCES)}" + print("✅ test_corpus_size") + + +if __name__ == "__main__": + test_top_1_is_most_similar() + test_top_k_respected() + test_similarity_in_minus_one_to_one() + test_corpus_size() + print("\n🎉 全部通過 — embedding 邏輯正確（無需下載 model）") diff --git a/examples/stage-6/01-embeddings/test_anthropic.py b/examples/stage-6/01-embeddings/test_anthropic.py new file mode 100644 index 0000000..f31ccd0 --- /dev/null +++ b/examples/stage-6/01-embeddings/test_anthropic.py @@ -0,0 +1,46 @@ +"""Stage 6 練習 1 — Path B 載入檢查 + OpenAI mock 對照。 + +跑法： + python test_anthropic.py +""" + +from __future__ import annotations + +import sys +from types import SimpleNamespace +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import numpy as np + +from starter_anthropic import embed_openai + + +def test_embed_openai_with_mock(): + """Mock OpenAI client、確認 embed_openai 正確 normalize。""" + fake_embeddings = [ + SimpleNamespace(embedding=[1.0, 0.0, 0.0]), + SimpleNamespace(embedding=[0.0, 2.0, 0.0]), + ] + client = MagicMock() + client.embeddings.create.return_value = SimpleNamespace(data=fake_embeddings) + + vecs = embed_openai(["a", "b"], client=client) + # Both should be unit length after normalize + assert abs(np.linalg.norm(vecs[0]) - 1.0) < 1e-6 + assert abs(np.linalg.norm(vecs[1]) - 1.0) < 1e-6 + print("✅ test_embed_openai_with_mock") + + +def test_starter_anthropic_loadable(): + import starter_anthropic + assert hasattr(starter_anthropic, "MODEL") + print("✅ test_starter_anthropic_loadable") + + +if __name__ == "__main__": + test_embed_openai_with_mock() + test_starter_anthropic_loadable() + print("\n🎉 通過 — Path B 可載入 + normalize 正確") diff --git a/examples/stage-6/02-vector-db/README.en.md b/examples/stage-6/02-vector-db/README.en.md new file mode 100644 index 0000000..bff4c11 --- /dev/null +++ b/examples/stage-6/02-vector-db/README.en.md @@ -0,0 +1,101 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# Exercise 2: Vector DB (ChromaDB) + semantic vs keyword + +Pairs with [Stage 6 — Memory & RAG](../../../stages/06-memory-rag.en.md) Exercise 2. + +## Task + +Index 8 docs into Chroma; compare semantic (vector) vs keyword (substring) retrieval on the same query. + +## How to run + +```bash +pip install -r requirements.txt +python starter.py # auto-downloads embedding model on first run +``` + +Budget: **$0**. In-memory mode; released after process exits. + +```bash +python test.py # 5 tests for index/query/ranking +python test_anthropic.py # Path B concept demo (same as starter) +``` + +## When to use a vector DB + +| Scenario | List + cosine | ChromaDB | +|---|---|---| +| < 100 docs | ✅ enough | overkill | +| 100-10K docs | Slow (re-embed each query) | ✅ persistent + indexed | +| 10K+ docs | No | ✅ (consider Qdrant / Weaviate at huge scale) | +| Persistence | Re-embed | ✅ SQLite backend | +| Filter / metadata | DIY | ✅ where clause | +| Hybrid search | DIY | ✅ built-in BM25 + vector | + +**Rule of thumb**: experimentation = `EphemeralClient`; production = `PersistentClient(path=...)`. + +## Semantic vs keyword + +``` +Query: "where to drink good coffee in Asian cities" + +📝 Keyword (substring) → misses doc 3 + Query doesn't have the exact word "coffee" + +🔍 Semantic (vector) → hits doc 3 + "Coffee shops in Taipei often serve pour-over..." + Semantic alignment, not literal match +``` + +| Dimension | Keyword | Semantic | +|---|---|---| +| Synonyms ("car" vs "auto") | Miss | Catch | +| Rephrasings | Miss | Catch | +| Typos | Miss | Catch (small) | +| Exact proper nouns | Strong | Occasionally confused | +| Negation (NOT) | Easy | Hard (embeddings don't grok negation) | +| Speed | Fast | Medium (need to embed the query) | +| Production | BM25 + vector **hybrid** | Same | + +**Production takeaway**: use both — hybrid search is best practice. Chroma 0.4+ has BM25 + vector built in. + +## Chroma API + +```python +client = chromadb.EphemeralClient() # in-memory; PersistentClient(path=...) for disk +embed_fn = embedding_functions.SentenceTransformerEmbeddingFunction(model_name="...") +collection = client.get_or_create_collection(name="demo", embedding_function=embed_fn) + +collection.add(ids=[...], documents=[...], metadatas=[{"category": "..."}, ...]) +collection.query(query_texts=[query], n_results=3, where={"category": "tech"}) +collection.upsert(...) +collection.delete(ids=[...]) +``` + +## Common pitfalls + +- **Duplicate ids in `.add()`**: raises. Use `.upsert()` or check `.get()["ids"]` first +- **Rebuilding the collection each query**: don't! `PersistentClient` indexes once +- **`n_results` too high**: no reranker — large k pulls in noise. 3-10 typically +- **Filter confusion**: `where={"category": "tech"}` is metadata; `where_document={"$contains": "..."}` is content +- **Inconsistent embedding function**: indexing with model A and querying with model B breaks retrieval. Chroma binds embedding_function to the collection to prevent this + +## Production-ready alternatives + +```bash +# Persistent +collection = build_collection(path="./chroma_db") + +# Cloud embeddings (higher quality) +embed_fn = embedding_functions.OpenAIEmbeddingFunction(api_key=..., model_name="text-embedding-3-small") +``` + +## Extensions + +- **Metadata filter**: `collection.query(query_texts=[q], where={"category": "food"})` +- **Hybrid search**: BM25 + vector via Chroma 0.4+ or external `rank_bm25` +- **Swap to Qdrant / Weaviate** at production scale +- **Plug into Exercise 4**: full RAG pipeline reuses this collection diff --git a/examples/stage-6/02-vector-db/README.md b/examples/stage-6/02-vector-db/README.md new file mode 100644 index 0000000..8905316 --- /dev/null +++ b/examples/stage-6/02-vector-db/README.md @@ -0,0 +1,116 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 練習 2：Vector DB（ChromaDB）+ semantic vs keyword + +對應 [Stage 6 — Memory & RAG](../../../stages/06-memory-rag.md) 練習 2。 +> 🎓 **學習模式**：這份 `starter.py` 是**完整解答**、不是 TODO skeleton。建議用**主動模式**——`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫一份 `starter.py`、跑 `python test.py` 驗證；卡 20 分鐘再回去對照 reference。完整方法論看 [`docs/HOW_TO_USE.md`](../../../docs/HOW_TO_USE.md)。 + +> 📚 **想要 chapter-length 深入版？** 本 folder 的 starter 是 illustrative 版、聚焦核心 pattern + 兩條 SDK path，不是進階深度教材。深度教材推薦： +> - [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents) ⭐ 中文圈最完整、章節式 + 16 種 production 能力。**本練習對應 hello-agents 的 vector store 章節** +> - [ChromaDB official tutorial](https://docs.trychroma.com/) + [Qdrant / Weaviate / Pinecone 對照](https://github.com/zilliztech/VectorDBBench)（production scale benchmark） +> - 完整 references 見 [Stage 6 精選 Projects](../../../stages/06-memory-rag.md#-精選-projects範本--spec--範例-collection) + + +## 任務 + +把 8 個 doc 存進 Chroma、用同一個 query 比較 semantic（vector）跟 keyword（substring）兩種 retrieval。 + +## 怎麼跑 + +```bash +pip install -r requirements.txt +python starter.py # 第一次自動下載 embedding model +``` + +預算：**$0**。Chroma in-memory mode、跑完就 release。 + +```bash +python test.py # 5 個 test、驗 index/query/ranking +python test_anthropic.py # Path B concept demo（同 starter） +``` + +## Vector DB 為什麼 vs 為什麼不 + +| 情境 | List comprehension + cosine | ChromaDB | +|---|---|---| +| < 100 doc | ✅ 簡單夠用 | overkill | +| 100-10K doc | 慢（每次 query 都 re-embed all） | ✅ 持久 + index | +| 10K+ doc | 不行 | ✅（或考慮 production 用的 Qdrant / Weaviate） | +| Persistence | ❌ 每次重 embed | ✅ SQLite 後端 | +| Filter / metadata | 自己寫 | ✅ where clause | +| Hybrid search | 自己寫 | ✅ 有 BM25 + vector | + +**經驗法則**：練習用 in-memory（`EphemeralClient`）、production 用 `PersistentClient(path=...)`。 + +## Semantic vs Keyword 對照 + +``` +Query: "where to drink good coffee in Asian cities" + +📝 Keyword (substring) → 漏 doc 3 + 因為 query 沒有「coffee」精確字串 + （只要 query 換字面、就 miss） + +🔍 Semantic (vector) → 抓到 doc 3 + "Coffee shops in Taipei often serve pour-over..." + 語意對齊、不靠字面 +``` + +| 維度 | Keyword search | Semantic search (vector) | +|---|---|---| +| 同義詞 | 漏（"car" 跟 "auto" 不同） | 抓得到 | +| 換字面 | 漏 | 抓得到 | +| 拼字錯 | 漏 | 抓得到（小錯） | +| 精確專有名詞 | 強 | 偶爾混淆 | +| 否定（NOT） | 簡單 | 困難（embedding 不擅長） | +| 速度 | 快 | 中（要 embed query） | +| Production 用法 | BM25 + vector **hybrid** | 同左 | + +**Production 結論**：兩條路都要、**hybrid search** 是 best practice。Chroma 0.4+ 內建 BM25 + vector hybrid（`query_texts` 加 `where` filter）。 + +## Chroma 核心 API + +```python +client = chromadb.EphemeralClient() # in-memory；PersistentClient(path=...) 用磁碟 +embed_fn = embedding_functions.SentenceTransformerEmbeddingFunction(model_name="...") +collection = client.get_or_create_collection(name="demo", embedding_function=embed_fn) + +# Add docs（自動 embed） +collection.add(ids=[...], documents=[...], metadatas=[{"category": "..."}, ...]) + +# Query +collection.query(query_texts=[query], n_results=3, where={"category": "tech"}) + +# Update / Delete +collection.upsert(...) +collection.delete(ids=[...]) +``` + +## 常見坑 + +- **重複 id `.add()`**：Chroma raise；用 `.upsert()` 或先檢查 `.get()["ids"]` +- **每次 query 重 build collection**：別這樣！`PersistentClient` 持久化、只 index 一次 +- **`n_results` 太大**：Chroma 沒返回 reranker、k 太大就會吃到 noise。經驗值 3-10 +- **Filter 用錯**：`where={"category": "tech"}` 是 metadata filter、`where_document={"$contains": "..."}` 是 doc 內容 filter +- **embedding function 一致性**：index 時用 model A、query 時用 model B、retrieval 會錯。Chroma 把 embedding_function 綁在 collection 上避免 + +## 想看實際在 production 跑的選擇？ + +```bash +# Persistent mode +# In starter.py: +collection = build_collection(path="./chroma_db") # 寫到磁碟 + +# Cloud embedding（更高精度） +# Replace SentenceTransformerEmbeddingFunction with: +embed_fn = embedding_functions.OpenAIEmbeddingFunction(api_key=..., model_name="text-embedding-3-small") +``` + +## 延伸 + +- **加 metadata filter**：`collection.query(query_texts=[q], where={"category": "food"})` +- **Hybrid search**：BM25 + vector、可以用 Chroma 0.4+ 或自己接 `rank_bm25` library +- **Swap to Qdrant / Weaviate**：production scale、需要 distributed +- **接練習 4**：完整 RAG pipeline 用同一個 collection diff --git a/examples/stage-6/02-vector-db/README.zh-Hans.md b/examples/stage-6/02-vector-db/README.zh-Hans.md new file mode 100644 index 0000000..b05df1c --- /dev/null +++ b/examples/stage-6/02-vector-db/README.zh-Hans.md @@ -0,0 +1,99 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 练习 2：Vector DB（ChromaDB）+ semantic vs keyword + +对应 [Stage 6 — Memory & RAG](../../../stages/06-memory-rag.zh-Hans.md) 练习 2。 + +## 任务 + +把 8 个 doc 存进 Chroma、用同一个 query 比较 semantic（vector）跟 keyword（substring）两种 retrieval。 + +## 怎么跑 + +```bash +pip install -r requirements.txt +python starter.py # 第一次自动下载 embedding model +``` + +预算：**$0**。Chroma in-memory mode、跑完就 release。 + +```bash +python test.py # 5 个 test、验 index/query/ranking +python test_anthropic.py # Path B concept demo（同 starter） +``` + +## Vector DB 为什么 vs 为什么不 + +| 情境 | List comprehension + cosine | ChromaDB | +|---|---|---| +| < 100 doc | ✅ 简单够用 | overkill | +| 100-10K doc | 慢（每次 query 都 re-embed all） | ✅ 持久 + index | +| 10K+ doc | 不行 | ✅（或考虑 production 用的 Qdrant / Weaviate） | +| Persistence | ❌ 每次重 embed | ✅ SQLite 后端 | +| Filter / metadata | 自己写 | ✅ where clause | +| Hybrid search | 自己写 | ✅ 有 BM25 + vector | + +**经验法则**：练习用 in-memory（`EphemeralClient`）、production 用 `PersistentClient(path=...)`。 + +## Semantic vs Keyword 对照 + +``` +Query: "where to drink good coffee in Asian cities" + +📝 Keyword (substring) → 漏 doc 3 + 因为 query 没有「coffee」精确字串 + +🔍 Semantic (vector) → 抓到 doc 3 + "Coffee shops in Taipei often serve pour-over..." + 语意对齐、不靠字面 +``` + +| 维度 | Keyword search | Semantic search (vector) | +|---|---|---| +| 同义词 | 漏（"car" 跟 "auto" 不同） | 抓得到 | +| 换字面 | 漏 | 抓得到 | +| 拼字错 | 漏 | 抓得到（小错） | +| 精确专有名词 | 强 | 偶尔混淆 | +| 否定（NOT） | 简单 | 困难（embedding 不擅长） | +| 速度 | 快 | 中（要 embed query） | +| Production 用法 | BM25 + vector **hybrid** | 同左 | + +**Production 结论**：两条路都要、**hybrid search** 是 best practice。Chroma 0.4+ 内建 BM25 + vector hybrid。 + +## Chroma 核心 API + +```python +client = chromadb.EphemeralClient() # in-memory；PersistentClient(path=...) 用磁盘 +embed_fn = embedding_functions.SentenceTransformerEmbeddingFunction(model_name="...") +collection = client.get_or_create_collection(name="demo", embedding_function=embed_fn) + +collection.add(ids=[...], documents=[...], metadatas=[{"category": "..."}, ...]) +collection.query(query_texts=[query], n_results=3, where={"category": "tech"}) +collection.upsert(...) +collection.delete(ids=[...]) +``` + +## 常见坑 + +- **重复 id `.add()`**：Chroma raise；用 `.upsert()` 或先检查 `.get()["ids"]` +- **每次 query 重 build collection**：别这样！`PersistentClient` 持久化、只 index 一次 +- **`n_results` 太大**：Chroma 没返回 reranker、k 太大就会吃到 noise。经验值 3-10 +- **Filter 用错**：`where={"category": "tech"}` 是 metadata filter、`where_document={"$contains": "..."}` 是 doc 内容 filter +- **embedding function 一致性**：index 时用 model A、query 时用 model B、retrieval 会错。Chroma 把 embedding_function 绑在 collection 上避免 + +## 想看实际在 production 跑的选择？ + +```bash +collection = build_collection(path="./chroma_db") # 持久化 +# 或 cloud embedding： +embed_fn = embedding_functions.OpenAIEmbeddingFunction(api_key=..., model_name="text-embedding-3-small") +``` + +## 延伸 + +- **加 metadata filter**：`collection.query(query_texts=[q], where={"category": "food"})` +- **Hybrid search**：BM25 + vector +- **Swap to Qdrant / Weaviate**：production scale +- **接练习 4**：完整 RAG pipeline 用同一个 collection diff --git a/examples/stage-6/02-vector-db/requirements.txt b/examples/stage-6/02-vector-db/requirements.txt new file mode 100644 index 0000000..09c85f9 --- /dev/null +++ b/examples/stage-6/02-vector-db/requirements.txt @@ -0,0 +1,2 @@ +chromadb>=0.4,<2.0 +sentence-transformers>=2.7,<4.0 diff --git a/examples/stage-6/02-vector-db/starter.py b/examples/stage-6/02-vector-db/starter.py new file mode 100644 index 0000000..742617a --- /dev/null +++ b/examples/stage-6/02-vector-db/starter.py @@ -0,0 +1,110 @@ +"""Stage 6 練習 2：Vector DB（ChromaDB）— Path A（本機、$0）。 + +把 embedding 存進 Chroma、做 semantic query、對照 keyword search。 + +跑法： + pip install -r requirements.txt + python starter.py + +驗證： + python test.py + +預算：$0。Chroma 跑在本機 SQLite 後端、規模到萬筆都 OK。 +""" + +from __future__ import annotations + +import os +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import chromadb +from chromadb.utils import embedding_functions + +DOCS = [ + {"id": "1", "text": "Taipei is the capital of Taiwan, known for night markets and tech.", "category": "city"}, + {"id": "2", "text": "Python is a high-level programming language with dynamic typing.", "category": "tech"}, + {"id": "3", "text": "Coffee shops in Taipei often serve pour-over with single-origin beans.", "category": "food"}, + {"id": "4", "text": "Machine learning models learn patterns from training data.", "category": "tech"}, + {"id": "5", "text": "Night markets in Taiwan offer street food like stinky tofu.", "category": "food"}, + {"id": "6", "text": "JavaScript is mainly used for web frontend development.", "category": "tech"}, + {"id": "7", "text": "Tokyo is the capital of Japan with a population of about 14 million.", "category": "city"}, + {"id": "8", "text": "Neural networks use layers of weighted connections.", "category": "tech"}, +] + + +def build_collection(path: str = ":memory:") -> Any: + """建立 / 取回 chroma collection。path=':memory:' 表 in-memory。""" + if path == ":memory:": + client = chromadb.EphemeralClient() + else: + client = chromadb.PersistentClient(path=path) + embed_fn = embedding_functions.SentenceTransformerEmbeddingFunction( + model_name="sentence-transformers/all-MiniLM-L6-v2", + ) + collection = client.get_or_create_collection( + name="demo", embedding_function=embed_fn, + ) + return collection + + +def index_docs(collection: Any, docs: list[dict]) -> None: + """Bulk add docs to collection.""" + if collection.count() > 0: + # Avoid duplicate add—Chroma raises on duplicate id + existing = set(collection.get()["ids"]) + new = [d for d in docs if d["id"] not in existing] + else: + new = docs + if not new: + return + collection.add( + ids=[d["id"] for d in new], + documents=[d["text"] for d in new], + metadatas=[{"category": d["category"]} for d in new], + ) + + +def semantic_query(collection: Any, query: str, top_k: int = 3) -> list[dict]: + """Semantic search.""" + result = collection.query(query_texts=[query], n_results=top_k) + return [ + {"id": result["ids"][0][i], "text": result["documents"][0][i], + "distance": result["distances"][0][i]} + for i in range(len(result["ids"][0])) + ] + + +def keyword_query(docs: list[dict], query: str, top_k: int = 3) -> list[dict]: + """簡化 keyword search（substring match、無 BM25）。""" + q = query.lower() + matches = [d for d in docs if any(w in d["text"].lower() for w in q.split() if len(w) > 2)] + return matches[:top_k] + + +if __name__ == "__main__": + print("Building Chroma collection (in-memory)...") + collection = build_collection() + index_docs(collection, DOCS) + print(f" Indexed {collection.count()} docs.\n") + + query = "where to drink good coffee in Asian cities" + print(f"❓ Query: {query}") + print("-" * 60) + + print("\n🔍 Semantic search (vector):") + for r in semantic_query(collection, query, top_k=3): + print(f" d={r['distance']:.3f} [{r['id']}] {r['text']}") + + print("\n📝 Keyword search (substring):") + for r in keyword_query(DOCS, query, top_k=3): + print(f" [{r['id']}] {r['text']}") + + # Semantic 應該抓到 doc 3 (coffee shops Taipei)、keyword 因為 query 無「coffee」精確匹配可能 miss + sem_top = semantic_query(collection, query, top_k=1) + assert sem_top[0]["id"] in {"3", "1"}, f"預期 coffee 或 Taipei 句、得到 {sem_top[0]}" + print("\n✅ 練習 2 通過 — Chroma vector DB 跑通、$0/run") + print(" 觀察：query 沒「coffee」字眼、keyword 漏；semantic 抓得到") diff --git a/examples/stage-6/02-vector-db/starter_anthropic.py b/examples/stage-6/02-vector-db/starter_anthropic.py new file mode 100644 index 0000000..edc120a --- /dev/null +++ b/examples/stage-6/02-vector-db/starter_anthropic.py @@ -0,0 +1,26 @@ +"""Stage 6 練習 2：Vector DB — Path B 概念展示。 + +Chroma 本身是本機 vector DB、不依賴 LLM provider。Path B 跟 Path A 跑出來一樣。 +RAG 流水線裡的 LLM 步驟（generation）會在練習 4 才出現——這題只測 retrieval。 + +要切 cloud-grade embedding（OpenAI / Voyage）見 starter.py 註解： + 把 embed_fn 換成 OpenAIEmbeddingFunction(api_key=..., model_name=...) +""" + +from __future__ import annotations + +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter import DOCS, build_collection, index_docs, semantic_query + +if __name__ == "__main__": + print("ℹ️ Vector DB（Chroma）跨 LLM provider 一致。Path A == Path B。") + print(" Cloud embedding 替換在 build_collection() 註解、不必跑 LLM。\n") + collection = build_collection() + index_docs(collection, DOCS) + for r in semantic_query(collection, "coffee in Asian cities", top_k=2): + print(f" {r}") + print("✅ 練習 2 (concept demo) 通過") diff --git a/examples/stage-6/02-vector-db/test.py b/examples/stage-6/02-vector-db/test.py new file mode 100644 index 0000000..85d5631 --- /dev/null +++ b/examples/stage-6/02-vector-db/test.py @@ -0,0 +1,75 @@ +"""Stage 6 練習 2 自我驗證 — Chroma collection + query 邏輯。 + +跑法： + python test.py + +第一次跑會自動下載 ~80MB embedding model。後續 cached、$0。 +""" + +from __future__ import annotations + +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter import DOCS, build_collection, index_docs, keyword_query, semantic_query + + +def test_docs_corpus_shape(): + assert len(DOCS) == 8 + assert all({"id", "text", "category"} <= d.keys() for d in DOCS) + print("✅ test_docs_corpus_shape") + + +def test_index_and_count(): + collection = build_collection() + index_docs(collection, DOCS) + assert collection.count() == 8, f"預期 8 doc、得到 {collection.count()}" + print("✅ test_index_and_count") + + +def test_semantic_query_ranks_relevant_first(): + """coffee 相關 query 應該 retrieve doc 3（coffee shops）排前。""" + collection = build_collection() + index_docs(collection, DOCS) + results = semantic_query(collection, "where to drink good coffee", top_k=3) + top_ids = [r["id"] for r in results] + assert "3" in top_ids, f"預期 doc 3（coffee shops）出現在 top-3、得到 {top_ids}" + print("✅ test_semantic_query_ranks_relevant_first") + + +def test_keyword_vs_semantic_difference(): + """指出兩個 search 方法差異： + keyword 對「python programming」會抓 doc 2； + semantic 對「dynamic language」（無 python 字眼）也能抓到 doc 2。""" + collection = build_collection() + index_docs(collection, DOCS) + + # Semantic 抓「dynamic language」 + sem_results = semantic_query(collection, "dynamic typed language", top_k=2) + sem_ids = [r["id"] for r in sem_results] + assert "2" in sem_ids, f"semantic 應抓到 doc 2、得到 {sem_ids}" + + # Keyword 對同 query「dynamic typed language」未必抓到（因為 doc 2 是「Python is a high-level...dynamic typing」） + # 至少驗 keyword 不會跨語意：「coffee in Asia」keyword 抓 coffee 句 + kw_results = keyword_query(DOCS, "coffee", top_k=3) + assert any("coffee" in r["text"].lower() for r in kw_results) + print("✅ test_keyword_vs_semantic_difference") + + +def test_top_k_respected(): + collection = build_collection() + index_docs(collection, DOCS) + assert len(semantic_query(collection, "test", top_k=2)) == 2 + assert len(semantic_query(collection, "test", top_k=5)) == 5 + print("✅ test_top_k_respected") + + +if __name__ == "__main__": + test_docs_corpus_shape() + test_index_and_count() + test_semantic_query_ranks_relevant_first() + test_keyword_vs_semantic_difference() + test_top_k_respected() + print("\n🎉 全部通過 — Chroma vector DB 邏輯正確") diff --git a/examples/stage-6/02-vector-db/test_anthropic.py b/examples/stage-6/02-vector-db/test_anthropic.py new file mode 100644 index 0000000..29fdd7c --- /dev/null +++ b/examples/stage-6/02-vector-db/test_anthropic.py @@ -0,0 +1,22 @@ +"""Stage 6 練習 2 — Path B 載入檢查。 + +Vector DB 跨 LLM provider 一致、Path B 跟 Path A 跑出來相同。 +""" + +from __future__ import annotations + +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + + +def test_module_loadable(): + import starter_anthropic + assert hasattr(starter_anthropic, "build_collection") + print("✅ test_module_loadable") + + +if __name__ == "__main__": + test_module_loadable() + print("\n🎉 通過 — Path B concept demo 可載入") diff --git a/examples/stage-6/03-chunking-comparison/README.en.md b/examples/stage-6/03-chunking-comparison/README.en.md new file mode 100644 index 0000000..fe90017 --- /dev/null +++ b/examples/stage-6/03-chunking-comparison/README.en.md @@ -0,0 +1,105 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# Exercise 3: Chunking Comparison (fixed / paragraph / heading-aware) + +Pairs with [Stage 6 — Memory & RAG](../../../stages/06-memory-rag.en.md) Exercise 3. + +## Task + +Same document → 3 chunking strategies → 5 queries → see which yields the best retrieval. + +| Strategy | How | Good for | +|---|---|---| +| **fixed-length** | N chars per chunk + overlap | Plain text (logs, chat history) | +| **paragraph** | Split on double newline | Prose / articles | +| **heading-aware** | Split on `# / ##` | Structured docs (README, wiki, specs) | + +## How to run + +```bash +pip install -r requirements.txt +python starter.py # first run downloads embedding model +``` + +Budget: **$0** (fully local). + +```bash +python test.py # 4 tests for chunking logic +python test_anthropic.py # Path B concept demo +``` + +## Strategy comparison + +### Fixed-length + +```python +def chunk_fixed(text, chunk_size=200, overlap=40): + # 200 chars per chunk, 40-char overlap + # Simple but splits sentences +``` + +Pros: 1-line implementation, works on any text. Cons: breaks sentences mid-flow. + +### Paragraph-based + +```python +def chunk_paragraphs(text): + return text.split("\n\n") +``` + +Pros: preserves semantic units. Cons: paragraphs vary in length (10 chars vs 500 chars), throws off embeddings. + +### Heading-aware + +```python +def chunk_headings(text): + return re.split(r"\n(?=#{1,3} )", text) +``` + +Pros: each chunk includes its heading (extra retrieval signal), structure preserved. Cons: requires markdown / structured docs — won't work on plain text or PDFs without headings. + +## Observations + +For query "How much does the MRT cost?": + +- **fixed-length** may cut "EasyCard is the universal payment. A single ride..." across chunks (MRT cost detached from context) +- **paragraph** retrieves the full Transit paragraph including NT$20-65 +- **heading-aware** retrieves the entire `## Transit` section — embedding for "MRT cost" lands closer to the heading + +For "What food can I try?": + +- **fixed-length** hits a chunk containing Food info but with bad boundaries +- **paragraph** hits one of the Food paragraphs +- **heading-aware** returns the whole `## Food` section — Shilin / dan bing / coffee info in one shot + +**Punchline**: **chunking strategy caps RAG quality** — content the retriever misses, no LLM can answer. + +## Production considerations + +- **Chunk size**: empirically 200-1000 chars (depends on embedding model's token limit; MiniLM is 512) +- **Overlap**: 10-20% on fixed-length to avoid splitting sentences +- **Reranker**: retrieve top-20, rerank with a cross-encoder, keep top-5 — big precision boost +- **Metadata**: tag each chunk `{"section": "Transit"}` for filter queries +- **Hybrid hierarchy**: split by heading first, then sub-split each section by fixed-length + +## Common pitfalls + +- **Chunks too large**: embedding averages out details, retrieval misses precise paragraphs +- **Chunks too small**: insufficient context, LLM can't form holistic answers — need more top-k +- **Testing on toy queries**: production queries are different. **Always validate chunking with real queries** +- **PDF chunking with markdown logic**: PDFs have columns / footnotes / tables — use `pdfplumber` or `unstructured` +- **CJK text byte-sliced mid-character**: UTF-8 byte slice breaks multi-byte chars. Use character-level slicing + +## Smarter chunking options + +- **`LangChain RecursiveCharacterTextSplitter`**: tries `\n\n`, then `\n`, then `.`, then char — auto-fallback +- **`LlamaIndex SemanticSplitter`**: uses embeddings to find semantic boundaries (most accurate, slowest) +- **`unstructured`**: full pipeline for PDF / DOCX / HTML + +## Extensions + +- **Grid-search chunk_size**: across 5 queries, find the size with highest mean similarity +- **Metadata filter**: tag chunks by section, query "only in Food section" +- **Plug into Exercise 4**: feed retrieval results to an LLM for full RAG diff --git a/examples/stage-6/03-chunking-comparison/README.md b/examples/stage-6/03-chunking-comparison/README.md new file mode 100644 index 0000000..bfaf2da --- /dev/null +++ b/examples/stage-6/03-chunking-comparison/README.md @@ -0,0 +1,115 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 練習 3：Chunking 對照（fixed / paragraph / heading-aware） + +對應 [Stage 6 — Memory & RAG](../../../stages/06-memory-rag.md) 練習 3。 +> 🎓 **學習模式**：這份 `starter.py` 是**完整解答**、不是 TODO skeleton。建議用**主動模式**——`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫一份 `starter.py`、跑 `python test.py` 驗證；卡 20 分鐘再回去對照 reference。完整方法論看 [`docs/HOW_TO_USE.md`](../../../docs/HOW_TO_USE.md)。 + +> 📚 **想要 chapter-length 深入版？** 本 folder 的 starter 是 illustrative 版、聚焦核心 pattern + 兩條 SDK path，不是進階深度教材。深度教材推薦： +> - [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents) ⭐ 中文圈最完整、章節式 + 16 種 production 能力。**本練習對應 hello-agents 的 RAG chunking 章節** +> - [LangChain RecursiveCharacterTextSplitter docs](https://python.langchain.com/docs/how_to/recursive_text_splitter/) + [LlamaIndex SemanticSplitter](https://docs.llamaindex.ai/en/stable/api_reference/node_parsers/semantic_splitter/) +> - 完整 references 見 [Stage 6 精選 Projects](../../../stages/06-memory-rag.md#-精選-projects範本--spec--範例-collection) + + +## 任務 + +同一份文件 → 3 種切法、跑 5 個 query、看哪種切法 retrieval 最準。 + +| 策略 | 怎麼切 | 適合 | +|---|---|---| +| **fixed-length** | 每 N 字一段、有 overlap | 純文字（log / chat history） | +| **paragraph** | 雙 newline 為界 | 散文 / 文章 | +| **heading-aware** | 看 `# / ##` 切 | structured doc（README / wiki / spec） | + +## 怎麼跑 + +```bash +pip install -r requirements.txt +python starter.py # 第一次自動下載 embedding model +``` + +預算：**$0**（純本機）。 + +```bash +python test.py # 4 個 test、驗 chunking 邏輯 +python test_anthropic.py # Path B concept demo +``` + +## 3 種策略對照 + +### Fixed-length + +```python +def chunk_fixed(text, chunk_size=200, overlap=40): + # 每 200 字一塊、40 字重疊 + # 簡單但會切壞句子 +``` + +**優點**：實作 1 行、適用所有文字 +**缺點**：切到一半句子、語意被打斷 + +### Paragraph-based + +```python +def chunk_paragraphs(text): + return text.split("\n\n") +``` + +**優點**：保語意完整 +**缺點**：段落長度不一（10 字 vs 500 字混在一起、embedding 不準） + +### Heading-aware + +```python +def chunk_headings(text): + return re.split(r"\n(?=#{1,3} )", text) +``` + +**優點**：每個 chunk 含 heading（給 retrieval 額外 signal）、structure 清楚 +**缺點**：要 markdown / structured doc。PDF / 純文字無 heading 就不行 + +## 觀察重點 + +對 query「How much does the MRT cost?」： + +- **fixed-length**：可能切到「...EasyCard is the universal payment. A single ride...」（MRT cost 跟前後 chunk 切開了） +- **paragraph**：抓到完整 Transit 段、含 NT$20-65 +- **heading-aware**：抓到完整 ## Transit section、含 heading「Transit」（embedding 對「MRT cost」的相似度更高） + +對 query「What food can I try?」： + +- **fixed-length**：抓到含 Food section 的 chunk、但可能 boundary 不對 +- **paragraph**：抓到 Food 內某個段落 +- **heading-aware**：抓到 `## Food` 整段、含 Shilin / dan bing / coffee 資訊一次到位 + +**punchline**：**chunking 策略決定 RAG 上限**——retrieval 抓不到的內容、再強的 LLM 也答不出。 + +## Production 額外考量 + +- **Chunk size**：經驗 200-1000 字（取決於 embedding model 的 token limit、MiniLM 是 512 token） +- **Overlap**：fixed-length 用 10-20% overlap、避免句子被切壞 +- **Reranker**：撈 top-20、丟 cross-encoder rerank 留 top-5、精度大躍進 +- **Metadata**：每個 chunk 加 `{"section": "Transit"}` 等 metadata、query 時可 filter +- **混合策略**：先 heading 切大塊、每塊內再 fixed-length 切小塊（兩層 hierarchy） + +## 常見坑 + +- **Chunk 太大**：embedding 平均掉細節、retrieval 抓不到精準段落 +- **Chunk 太小**：context 不足、LLM 看不到 holistic 答案。需要更多 top-k 補 +- **沒測過真實 query**：開發時用 toy query 驗、production query 完全不同。**永遠用真實 query 校 chunking** +- **PDF chunking 直接套 markdown 邏輯**：PDF 有 column / footnote / table、要用 `pdfplumber` 或 `unstructured` 庫處理 +- **CJK 文本切到一半字元**：UTF-8 byte slice 會切壞 multi-byte char。用 character-level slice + +## 想看更聰明的 chunking？ + +- **`LangChain RecursiveCharacterTextSplitter`**：自動 fallback、先 try `\n\n`、再 `\n`、再 `.`、再 char +- **`LlamaIndex SemanticSplitter`**：用 embedding 找語意斷點切（最準、慢） +- **`unstructured`**：對 PDF / DOCX / HTML 全套處理 + +## 延伸 + +- **改變 chunk_size 跑 grid search**：對 5 個 query、看哪個 size 平均 sim 最高 +- **加 metadata filter**：每個 chunk 標 section、query 時可指定「只在 Food section 搜」 +- **接練習 4 完整 RAG**：選好 chunking 策略後、丟給 LLM 生答案 diff --git a/examples/stage-6/03-chunking-comparison/README.zh-Hans.md b/examples/stage-6/03-chunking-comparison/README.zh-Hans.md new file mode 100644 index 0000000..0b5b305 --- /dev/null +++ b/examples/stage-6/03-chunking-comparison/README.zh-Hans.md @@ -0,0 +1,98 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 练习 3：Chunking 对照（fixed / paragraph / heading-aware） + +对应 [Stage 6 — Memory & RAG](../../../stages/06-memory-rag.zh-Hans.md) 练习 3。 + +## 任务 + +同一份文档 → 3 种切法、跑 5 个 query、看哪种切法 retrieval 最准。 + +| 策略 | 怎么切 | 适合 | +|---|---|---| +| **fixed-length** | 每 N 字一段、有 overlap | 纯文字（log / chat history） | +| **paragraph** | 双 newline 为界 | 散文 / 文章 | +| **heading-aware** | 看 `# / ##` 切 | structured doc（README / wiki / spec） | + +## 怎么跑 + +```bash +pip install -r requirements.txt +python starter.py # 第一次自动下载 embedding model +``` + +预算：**$0**（纯本机）。 + +```bash +python test.py # 4 个 test、验 chunking 逻辑 +python test_anthropic.py # Path B concept demo +``` + +## 3 种策略对照 + +### Fixed-length + +```python +def chunk_fixed(text, chunk_size=200, overlap=40): + # 每 200 字一块、40 字重叠 +``` + +**优点**：实作 1 行、适用所有文字；**缺点**：切到一半句子、语意被打断 + +### Paragraph-based + +```python +def chunk_paragraphs(text): + return text.split("\n\n") +``` + +**优点**：保语意完整；**缺点**：段落长度不一（10 字 vs 500 字、embedding 不准） + +### Heading-aware + +```python +def chunk_headings(text): + return re.split(r"\n(?=#{1,3} )", text) +``` + +**优点**：每个 chunk 含 heading、structure 清楚；**缺点**：要 markdown / structured doc + +## 观察重点 + +对 query“How much does the MRT cost?”： + +- **fixed-length**：可能切到“...EasyCard is the universal payment. A single ride...”（MRT cost 跟前后 chunk 切开了） +- **paragraph**：抓到完整 Transit 段、含 NT$20-65 +- **heading-aware**：抓到完整 ## Transit section、含 heading“Transit” + +**punchline**：**chunking 策略决定 RAG 上限**——retrieval 抓不到的内容、再强的 LLM 也答不出。 + +## Production 额外考量 + +- **Chunk size**：经验 200-1000 字（取决于 embedding model 的 token limit） +- **Overlap**：fixed-length 用 10-20% overlap、避免句子被切坏 +- **Reranker**：捞 top-20、丢 cross-encoder rerank 留 top-5、精度大跃进 +- **Metadata**：每个 chunk 加 `{"section": "Transit"}` 等 metadata、query 时可 filter +- **混合策略**：先 heading 切大块、每块内再 fixed-length 切小块（两层 hierarchy） + +## 常见坑 + +- **Chunk 太大**：embedding 平均掉细节、retrieval 抓不到精准段落 +- **Chunk 太小**：context 不足、LLM 看不到 holistic 答案 +- **没测过真实 query**：开发时用 toy query 验、production query 完全不同 +- **PDF chunking 直接套 markdown 逻辑**：PDF 有 column / footnote / table、要用 `pdfplumber` 或 `unstructured` +- **CJK 文本切到一半字符**：UTF-8 byte slice 会切坏 multi-byte char + +## 想看更聪明的 chunking？ + +- **`LangChain RecursiveCharacterTextSplitter`**：自动 fallback +- **`LlamaIndex SemanticSplitter`**：用 embedding 找语意断点切 +- **`unstructured`**：对 PDF / DOCX / HTML 全套处理 + +## 延伸 + +- **改变 chunk_size 跑 grid search**：对 5 个 query、看哪个 size 平均 sim 最高 +- **加 metadata filter**：每个 chunk 标 section、query 时可指定“只在 Food section 搜” +- **接练习 4 完整 RAG**：选好 chunking 策略后、丢给 LLM 生答案 diff --git a/examples/stage-6/03-chunking-comparison/requirements.txt b/examples/stage-6/03-chunking-comparison/requirements.txt new file mode 100644 index 0000000..390f904 --- /dev/null +++ b/examples/stage-6/03-chunking-comparison/requirements.txt @@ -0,0 +1,2 @@ +sentence-transformers>=2.7,<4.0 +numpy>=1.24 diff --git a/examples/stage-6/03-chunking-comparison/starter.py b/examples/stage-6/03-chunking-comparison/starter.py new file mode 100644 index 0000000..1413e57 --- /dev/null +++ b/examples/stage-6/03-chunking-comparison/starter.py @@ -0,0 +1,133 @@ +"""Stage 6 練習 3：Chunking 對照 — 3 種切法 A/B（純 Python、$0）。 + +同一份文件 → 3 種切法： + 1. fixed-length（每 N 字一段、可加 overlap） + 2. paragraph-based（按段落切、保語意完整） + 3. heading-aware（看 # / ## 切、保章節 hierarchy） + +對 5 個真實 query 比較 top-k 結果、看哪種切法撈到正確 context。 + +跑法： + pip install -r requirements.txt + python starter.py + +驗證： + python test.py +""" + +from __future__ import annotations + +import re +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import numpy as np +from sentence_transformers import SentenceTransformer + +EMBED_MODEL = "sentence-transformers/all-MiniLM-L6-v2" + + +# 範例 markdown 文件（含 heading + paragraph） +SAMPLE_DOC = """# Taipei Travel Guide + +## Food + +Taipei is famous for night markets. Shilin Night Market is the most famous, with stinky tofu and bubble tea stalls. Local breakfast often features dan bing (egg crepe) and youtiao with hot soy milk. + +For coffee, Taipei has many third-wave specialty shops. Fika Fika and GABEE are well-known. Prices are roughly NT$150-250 per cup. + +## Transit + +The MRT (metro) covers most tourist areas. EasyCard is the universal payment. A single ride costs NT$20-65. Buses are cheaper but slower. + +Bicycles are also popular — YouBike is the city's bike-share. Cost is NT$10 per 30 min. + +## Weather + +Taipei summer is hot and humid, often 32-35°C with afternoon thunderstorms. Winter is mild (15-20°C) but can be rainy for weeks. The best time to visit is October-November or March-April. +""" + + +# === 3 個 chunking strategy === + +def chunk_fixed(text: str, chunk_size: int = 200, overlap: int = 40) -> list[str]: + """固定長度切（with overlap）。簡單但會切壞句子 / 段落。""" + chunks = [] + i = 0 + while i < len(text): + chunks.append(text[i:i + chunk_size]) + i += chunk_size - overlap + return [c.strip() for c in chunks if c.strip()] + + +def chunk_paragraphs(text: str) -> list[str]: + """按段落切（雙 newline 為界）。保語意完整、但段落長短不一。""" + return [p.strip() for p in text.split("\n\n") if p.strip()] + + +def chunk_headings(text: str) -> list[str]: + """Heading-aware：按 # / ## 切、每個 section 含 heading + body、有 hierarchy 標記。""" + sections = re.split(r"\n(?=#{1,3} )", text) + return [s.strip() for s in sections if s.strip()] + + +# === Retrieval comparison === + +def embed_chunks(chunks: list[str], model: SentenceTransformer) -> np.ndarray: + return model.encode(chunks, normalize_embeddings=True, convert_to_numpy=True) + + +def top_k_for_query(query: str, chunks: list[str], chunk_vecs: np.ndarray, + model: SentenceTransformer, k: int = 2) -> list[dict]: + q_vec = model.encode([query], normalize_embeddings=True, convert_to_numpy=True)[0] + sims = chunk_vecs @ q_vec + top_idx = np.argsort(-sims)[:k] + return [{"chunk": chunks[i], "similarity": float(sims[i])} for i in top_idx] + + +def compare_strategies(doc: str, queries: list[str], k: int = 2) -> dict: + """3 種 chunking、跑同樣 query、回傳對比結果。""" + model = SentenceTransformer(EMBED_MODEL) + + strategies = { + "fixed_200_overlap_40": chunk_fixed(doc, chunk_size=200, overlap=40), + "paragraphs": chunk_paragraphs(doc), + "headings": chunk_headings(doc), + } + + results = {} + for name, chunks in strategies.items(): + vecs = embed_chunks(chunks, model) + per_query = [] + for q in queries: + per_query.append({"query": q, "top": top_k_for_query(q, chunks, vecs, model, k=k)}) + results[name] = {"chunk_count": len(chunks), "queries": per_query} + return results + + +if __name__ == "__main__": + queries = [ + "What food can I try at night markets?", + "How much does the MRT cost?", + "Best time of year to visit?", + "How to rent bicycles?", + "Any coffee shops?", + ] + print("Document: Taipei Travel Guide (3 headings)") + print(f"Queries: {len(queries)}\n") + + out = compare_strategies(SAMPLE_DOC, queries, k=2) + for name, data in out.items(): + print(f"=== {name} ({data['chunk_count']} chunks) ===") + for entry in data["queries"]: + best = entry["top"][0] + print(f" Q: {entry['query']}") + print(f" → sim={best['similarity']:.3f}: {best['chunk'][:80]}...") + print() + + # Sanity: heading-aware 通常 chunk 數 = 4（title + 3 section）、最少 noise + assert out["headings"]["chunk_count"] <= 5, "heading chunks 應該少而 self-contained" + print("✅ 練習 3 通過 — 3 種 chunking 對照、$0/run") + print(" 觀察：heading-aware 對 sectional query 最準；fixed-length 容易切壞句子") diff --git a/examples/stage-6/03-chunking-comparison/starter_anthropic.py b/examples/stage-6/03-chunking-comparison/starter_anthropic.py new file mode 100644 index 0000000..13582c9 --- /dev/null +++ b/examples/stage-6/03-chunking-comparison/starter_anthropic.py @@ -0,0 +1,21 @@ +"""Stage 6 練習 3：Chunking — Path B concept demo（同 starter）。 + +Chunking 是純資料前處理、不依賴 LLM。Path A == Path B。 +要把 retrieval 結果丟給 Claude 生答案、見 練習 4 完整 RAG。 +""" + +from __future__ import annotations + +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter import SAMPLE_DOC, compare_strategies + +if __name__ == "__main__": + print("ℹ️ Chunking 跨 LLM provider 一致、Path A == Path B。") + out = compare_strategies(SAMPLE_DOC, ["food at night markets"], k=1) + for name, data in out.items(): + print(f" {name}: {data['chunk_count']} chunks") + print("✅ 練習 3 (concept demo) 通過") diff --git a/examples/stage-6/03-chunking-comparison/test.py b/examples/stage-6/03-chunking-comparison/test.py new file mode 100644 index 0000000..771c991 --- /dev/null +++ b/examples/stage-6/03-chunking-comparison/test.py @@ -0,0 +1,57 @@ +"""Stage 6 練習 3 自我驗證 — 3 種 chunking strategy 的純邏輯測試（不打 embed）。 +""" + +from __future__ import annotations + +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter import SAMPLE_DOC, chunk_fixed, chunk_headings, chunk_paragraphs + + +def test_fixed_length_chunks(): + chunks = chunk_fixed("a" * 500, chunk_size=200, overlap=40) + # 500 字、step=160（200-40）→ 應該 ~ 3-4 chunk + assert 3 <= len(chunks) <= 4, f"預期 3-4 chunk、得到 {len(chunks)}" + assert all(len(c) <= 200 for c in chunks) + print("✅ test_fixed_length_chunks") + + +def test_paragraphs_split_on_double_newline(): + text = "Para one.\n\nPara two.\n\nPara three." + chunks = chunk_paragraphs(text) + assert chunks == ["Para one.", "Para two.", "Para three."] + print("✅ test_paragraphs_split_on_double_newline") + + +def test_headings_keep_section_together(): + chunks = chunk_headings(SAMPLE_DOC) + # Sample 有 1 個 # 標題 + 3 個 ## 標題、4 個 section + assert len(chunks) == 4, f"預期 4 section、得到 {len(chunks)}" + # 每個 section 都應該以 # 開頭 + assert all(c.startswith("#") for c in chunks) + # Food section 應該包含「night markets」 + food = next(c for c in chunks if c.startswith("## Food")) + assert "night markets" in food.lower() + print("✅ test_headings_keep_section_together") + + +def test_chunk_count_difference(): + """3 種切法應該產生不同 chunk 數、印證它們真的不同。""" + fixed_n = len(chunk_fixed(SAMPLE_DOC, chunk_size=200, overlap=40)) + para_n = len(chunk_paragraphs(SAMPLE_DOC)) + head_n = len(chunk_headings(SAMPLE_DOC)) + # heading-aware 通常最少（每個 section 1 chunk） + assert head_n <= para_n, "heading 應 <= paragraphs" + print(f" fixed: {fixed_n}, paragraphs: {para_n}, headings: {head_n}") + print("✅ test_chunk_count_difference") + + +if __name__ == "__main__": + test_fixed_length_chunks() + test_paragraphs_split_on_double_newline() + test_headings_keep_section_together() + test_chunk_count_difference() + print("\n🎉 全部通過 — 3 種 chunking strategy 邏輯正確") diff --git a/examples/stage-6/03-chunking-comparison/test_anthropic.py b/examples/stage-6/03-chunking-comparison/test_anthropic.py new file mode 100644 index 0000000..80120f9 --- /dev/null +++ b/examples/stage-6/03-chunking-comparison/test_anthropic.py @@ -0,0 +1,19 @@ +"""Stage 6 練習 3 — Path B concept demo 載入檢查。""" + +from __future__ import annotations + +import sys + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + + +def test_loadable(): + import starter_anthropic + assert hasattr(starter_anthropic, "compare_strategies") + print("✅ test_loadable") + + +if __name__ == "__main__": + test_loadable() + print("\n🎉 通過") diff --git a/examples/stage-6/04-full-rag-pipeline/README.en.md b/examples/stage-6/04-full-rag-pipeline/README.en.md new file mode 100644 index 0000000..9783cfd --- /dev/null +++ b/examples/stage-6/04-full-rag-pipeline/README.en.md @@ -0,0 +1,122 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# Exercise 4: Full RAG Pipeline + +Pairs with [Stage 6 — Memory & RAG](../../../stages/06-memory-rag.en.md) Exercise 4. + +## Task + +Tie Exercises 1-3 together: + +``` +doc → chunk_doc → embed → ChromaDB → top_k retrieve → LLM generation +``` + +Sample KB is a company onboarding doc with 4 sections (vacation / remote / expenses / tech stack). + +## How to run — two paths + +### Path A (default, free, local) + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +Budget: **$0**. + +### Path B (Anthropic, cloud-quality answers) + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +Budget: ~**$0.001** per run. + +## Validate the logic + +```bash +python test.py # 5 tests — mock LLM, exercise full pipeline +python test_anthropic.py # Anthropic mock +``` + +`test.py` uses a mock LLM through the full pipeline (chunking → retrieval → generation), confirming the prompt actually includes context and `generate` sees the retrieval output. + +## RAG in 4 steps + +```python +def rag(query, doc): + collection = build_kb(doc) # 1. chunk + embed + index (one-time) + contexts = retrieve(collection, q) # 2. top-k semantic search + answer = generate(q, contexts) # 3. LLM reads context, answers + return {"contexts": contexts, "answer": answer} +``` + +Each step has independent trade-offs: + +| Step | Main knob | Affects | +|---|---|---| +| chunk | size / overlap / strategy | retrieval ceiling | +| embed | model size / multilingual | retrieval precision | +| retrieve | top_k / metadata filter / reranker | recall vs precision | +| generate | prompt / model / temperature | answer quality | + +## Generation prompt pattern + +```python +prompt = f"""Answer the user's question based ONLY on the context below. +If the context doesn't contain the answer, say "I don't have that information". + +Context: +{context_text} + +Question: {query} + +Answer:""" +``` + +**Three key instructions**: +1. `based ONLY on context` — prevents hallucination +2. `if missing → say so` — gives the LLM an out, no forced answers +3. Context then Question — models prefer this layout + +## Path comparison + +| Observation | Anthropic Claude haiku | Ollama qwen2.5:3b | +|---|---|---| +| Grounding in context | Stable (sticks to context) | Sometimes drifts, fills with general knowledge | +| "I don't have that info" rate | High (follows rules) | Low (forces an answer) | +| Fluency | High | Medium | +| Multi-context integration | Good | Sometimes only looks at the first | +| Speed | 1-3s | 5-15s on CPU | +| Cost | $0.001 | $0 | + +**Production reality**: RAG quality = retrieval quality × generation quality. Retrieval miss → LLM hallucinates; retrieval good but LLM weak → low-quality answer. Stage 7 production often uses local / mid-size for retrieval and Claude / GPT for generation. + +## Common pitfalls + +- **No "only based on context" instruction**: LLM goes off-script, fills from training data — uncontrolled +- **`top_k` too high**: long context, attention diffuses, wrong answers +- **`top_k` too low**: misses key sections, can't answer +- **Context after the question**: LLMs weight the start of the prompt more; put context first +- **No eval for "say unknown when you can't answer"**: production needs 5-10 eval cases for this + +## Production-ready RAG + +- **Persistent ChromaDB**: `chromadb.PersistentClient(path=...)` to skip re-indexing +- **Reranker**: retrieve top-20, cross-encoder rerank, keep top-3 +- **Citation**: prompt "cite which context section you used", LLM tags [chunk_0] +- **Streaming**: `client.chat.completions.create(stream=True)` +- **LangGraph integration**: turn retrieve → generate into graph nodes with fallback path + +## Extensions + +- **Query rewriting**: LLM rewrites the user query into something better for retrieval (HyDE pattern) +- **Multi-hop RAG**: first retrieve gives partial answer, use partial answer to retrieve more +- **Plug into Exercise 5 long-term memory**: dialogue history also goes into vector store diff --git a/examples/stage-6/04-full-rag-pipeline/README.md b/examples/stage-6/04-full-rag-pipeline/README.md new file mode 100644 index 0000000..4dcca7f --- /dev/null +++ b/examples/stage-6/04-full-rag-pipeline/README.md @@ -0,0 +1,129 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 練習 4：完整 RAG 流水線 + +對應 [Stage 6 — Memory & RAG](../../../stages/06-memory-rag.md) 練習 4。 +> 🎓 **學習模式**：這份 `starter.py` 是**完整解答**、不是 TODO skeleton。建議用**主動模式**——`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫一份 `starter.py`、跑 `python test.py` 驗證；卡 20 分鐘再回去對照 reference。完整方法論看 [`docs/HOW_TO_USE.md`](../../../docs/HOW_TO_USE.md)。 + +> 📚 **想要 chapter-length 深入版？** 本 folder 的 starter 是 illustrative 版、聚焦核心 pattern + 兩條 SDK path，不是進階深度教材。深度教材推薦： +> - [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents) ⭐ 中文圈最完整、章節式 + 16 種 production 能力。**本練習對應 hello-agents 的 完整 RAG 流水線章節** +> - [LlamaIndex RAG tutorial](https://docs.llamaindex.ai/en/stable/understanding/rag/) + [LangChain RAG cookbook](https://python.langchain.com/docs/tutorials/rag/) +> - 完整 references 見 [Stage 6 精選 Projects](../../../stages/06-memory-rag.md#-精選-projects範本--spec--範例-collection) + + +## 任務 + +把練習 1-3 串起來： + +``` +doc → chunk_doc → embed → ChromaDB → top_k retrieve → LLM 生答案 +``` + +範例 KB 是公司 onboarding 文件、4 個 section（vacation / remote / expenses / tech stack）。 + +## 怎麼跑 — 兩條路徑 + +### Path A（默認、本機免費） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +預算：**$0**。 + +### Path B（Anthropic、想看 cloud 答案質量） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +預算：每次 ≈ **$0.001**。 + +## 不花錢驗證程式邏輯 + +```bash +python test.py # 5 個 test、mock LLM 驗整條 pipeline +python test_anthropic.py # Anthropic mock +``` + +`test.py` 用 mock LLM 跑整個 RAG pipeline（chunking → retrieval → generation）、確認 prompt 真的帶上 context、確認 generate 拿到 retrieval 結果。 + +## RAG 4 個步驟 + +```python +def rag(query, doc): + collection = build_kb(doc) # 1. chunk + embed + index（一次性） + contexts = retrieve(collection, q) # 2. top-k 語意搜 + answer = generate(q, contexts) # 3. LLM 看 context 回答 + return {"contexts": contexts, "answer": answer} +``` + +每一步都有獨立 trade-off： + +| 步驟 | 主要 knob | 影響 | +|---|---|---| +| chunk | size / overlap / strategy | retrieval 上限 | +| embed | model 大小 / multilingual | retrieval 精度 | +| retrieve | top_k / metadata filter / reranker | recall vs precision | +| generate | prompt 寫法 / model / temperature | 答案質量 | + +## Generate prompt 經典 pattern + +```python +prompt = f"""Answer the user's question based ONLY on the context below. +If the context doesn't contain the answer, say "I don't have that information". + +Context: +{context_text} + +Question: {query} + +Answer:""" +``` + +**3 個關鍵 instruction**： +1. `based ONLY on context` — 防 hallucinate +2. `if doesn't contain → say so` — 給 LLM 退路、不強答 +3. Context + Question 順序固定 — 模型訓練偏好這個 layout + +## 兩個 path 觀察重點 + +| 觀察項 | Anthropic Claude haiku | Ollama qwen2.5:3b | +|---|---|---| +| 答案 grounding | 穩（緊扣 context） | 偶爾發散、用知識補答 | +| "I don't have that info" 機率 | 高（守規則） | 低（強答） | +| 答案 fluency | 高 | 中 | +| 多 context 整合 | 好 | 偶爾只看第一個 | +| 速度 | 1-3 秒 | 5-15 秒（CPU） | +| 成本 | $0.001 | $0 | + +**production 觀察**：RAG 質量 = retrieval quality × generation quality。**retrieval 漏 = LLM 無中生有；retrieval 對但 LLM 弱 = 答案不準**。Stage 7 production 通常 retrieval 走本機 / 中模型、generation 用 Claude / GPT。 + +## 常見坑 + +- **prompt 沒講「only based on context」**：LLM 會自由發揮、用訓練資料補答、不可控 +- **`top_k` 設太大**：context 太長、LLM 注意力分散、可能答錯 +- **`top_k` 設太小**：context 漏關鍵段、LLM 無法答 +- **prompt 把 context 放後面**：LLM 較重視 prompt 開頭、context 應該在 question 之前 +- **沒驗證「答錯就 say I don't know」**：production 加 5-10 個「答不出來該說 unknown」的 eval case + +## 想看實際在 production 跑的 RAG？ + +- **Persistent ChromaDB**：`chromadb.PersistentClient(path=...)` 不重新 index +- **Reranker**：retrieve top-20、cross-encoder rerank、留 top-3 +- **Citation**：prompt 改成「cite which context section you used」、LLM 標 [chunk_0] +- **Streaming**：`client.chat.completions.create(stream=True)` 邊跑邊印 +- **接 LangGraph**：把 retrieve → generate 變 graph node、加 fallback path + +## 延伸 + +- **加 query rewriting**：先讓 LLM 把 user query 改寫成更適合 retrieval 的版本（HyDE pattern） +- **Multi-hop RAG**：第一輪 retrieve 拿到部分答案、用部分答案再 retrieve 補完整 +- **接練習 5 long-term memory**：對話 history 也丟進 vector store、跨輪對話記住事情 diff --git a/examples/stage-6/04-full-rag-pipeline/README.zh-Hans.md b/examples/stage-6/04-full-rag-pipeline/README.zh-Hans.md new file mode 100644 index 0000000..1a0193c --- /dev/null +++ b/examples/stage-6/04-full-rag-pipeline/README.zh-Hans.md @@ -0,0 +1,120 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 练习 4：完整 RAG 流水线 + +对应 [Stage 6 — Memory & RAG](../../../stages/06-memory-rag.zh-Hans.md) 练习 4。 + +## 任务 + +把练习 1-3 串起来： + +``` +doc → chunk_doc → embed → ChromaDB → top_k retrieve → LLM 生答案 +``` + +范例 KB 是公司 onboarding 文档、4 个 section（vacation / remote / expenses / tech stack）。 + +## 怎么跑 — 两条路径 + +### Path A（默认、本机免费） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +预算：**$0**。 + +### Path B（Anthropic、想看 cloud 答案质量） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +预算：每次 ≈ **$0.001**。 + +## 不花钱验证程序逻辑 + +```bash +python test.py # 5 个 test、mock LLM 验整条 pipeline +python test_anthropic.py # Anthropic mock +``` + +## RAG 4 个步骤 + +```python +def rag(query, doc): + collection = build_kb(doc) # 1. chunk + embed + index（一次性） + contexts = retrieve(collection, q) # 2. top-k 语意搜 + answer = generate(q, contexts) # 3. LLM 看 context 回答 + return {"contexts": contexts, "answer": answer} +``` + +每一步都有独立 trade-off： + +| 步骤 | 主要 knob | 影响 | +|---|---|---| +| chunk | size / overlap / strategy | retrieval 上限 | +| embed | model 大小 / multilingual | retrieval 精度 | +| retrieve | top_k / metadata filter / reranker | recall vs precision | +| generate | prompt 写法 / model / temperature | 答案质量 | + +## Generate prompt 经典 pattern + +```python +prompt = f"""Answer the user's question based ONLY on the context below. +If the context doesn't contain the answer, say "I don't have that information". + +Context: +{context_text} + +Question: {query} + +Answer:""" +``` + +**3 个关键 instruction**： +1. `based ONLY on context` — 防 hallucinate +2. `if doesn't contain → say so` — 给 LLM 退路、不强答 +3. Context + Question 顺序固定 — 模型训练偏好这个 layout + +## 两个 path 观察重点 + +| 观察项 | Anthropic Claude haiku | Ollama qwen2.5:3b | +|---|---|---| +| 答案 grounding | 稳（紧扣 context） | 偶尔发散、用知识补答 | +| "I don't have that info" 机率 | 高（守规则） | 低（强答） | +| 答案 fluency | 高 | 中 | +| 多 context 整合 | 好 | 偶尔只看第一个 | +| 速度 | 1-3 秒 | 5-15 秒（CPU） | +| 成本 | $0.001 | $0 | + +**production 观察**：RAG 质量 = retrieval quality × generation quality。**retrieval 漏 = LLM 无中生有；retrieval 对但 LLM 弱 = 答案不准**。Stage 7 production 通常 retrieval 走本机 / 中模型、generation 用 Claude / GPT。 + +## 常见坑 + +- **prompt 没讲“only based on context”**：LLM 会自由发挥、用训练数据补答、不可控 +- **`top_k` 设太大**：context 太长、LLM 注意力分散、可能答错 +- **`top_k` 设太小**：context 漏关键段、LLM 无法答 +- **prompt 把 context 放后面**：LLM 较重视 prompt 开头、context 应该在 question 之前 +- **没验证“答错就 say I don't know”**：production 加 5-10 个“答不出来该说 unknown”的 eval case + +## 想看实际在 production 跑的 RAG？ + +- **Persistent ChromaDB**：`chromadb.PersistentClient(path=...)` 不重新 index +- **Reranker**：retrieve top-20、cross-encoder rerank、留 top-3 +- **Citation**：prompt 改成“cite which context section you used”、LLM 标 [chunk_0] +- **Streaming**：`client.chat.completions.create(stream=True)` 边跑边印 +- **接 LangGraph**：把 retrieve → generate 变 graph node、加 fallback path + +## 延伸 + +- **加 query rewriting**：先让 LLM 把 user query 改写成更适合 retrieval 的版本（HyDE pattern） +- **Multi-hop RAG**：第一轮 retrieve 拿到部分答案、用部分答案再 retrieve 补完整 +- **接练习 5 long-term memory**：对话 history 也丢进 vector store、跨轮对话记住事情 diff --git a/examples/stage-6/04-full-rag-pipeline/requirements.txt b/examples/stage-6/04-full-rag-pipeline/requirements.txt new file mode 100644 index 0000000..be0870d --- /dev/null +++ b/examples/stage-6/04-full-rag-pipeline/requirements.txt @@ -0,0 +1,4 @@ +chromadb>=0.4,<2.0 +sentence-transformers>=2.7,<4.0 +openai>=1.50,<2.0 +anthropic>=0.40,<1.0 diff --git a/examples/stage-6/04-full-rag-pipeline/starter.py b/examples/stage-6/04-full-rag-pipeline/starter.py new file mode 100644 index 0000000..80c7c0d --- /dev/null +++ b/examples/stage-6/04-full-rag-pipeline/starter.py @@ -0,0 +1,123 @@ +"""Stage 6 練習 4：完整 RAG 流水線 — Path A（Ollama 默認、$0）。 + +document → chunk → embed → top-k retrieve → LLM 生答案 + +跑法： + pip install -r requirements.txt + ollama pull qwen2.5:3b + ollama serve + python starter.py + +驗證： + python test.py + +預算：$0/run。Path B 用 Claude 生答案、見 starter_anthropic.py。 +""" + +from __future__ import annotations + +import os +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import chromadb +from chromadb.utils import embedding_functions +from openai import OpenAI + +MODEL = os.environ.get("MODEL", "qwen2.5:3b") +OLLAMA_BASE = os.environ.get("OLLAMA_API_BASE", "http://localhost:11434/v1") +EMBED_MODEL = "sentence-transformers/all-MiniLM-L6-v2" + + +# Sample knowledge base +KB_DOC = """# Company Onboarding + +## Vacation Policy + +Full-time employees get 15 days of paid vacation per year, accruing monthly. Unused days roll over up to 5 days into the next year. Vacation must be requested at least 1 week in advance via the HR portal. + +## Remote Work + +Hybrid model: 3 days in office, 2 days remote per week. Fully remote requires manager approval and HR review. + +## Expenses + +Submit receipts via Concur within 30 days. Meals over $50 need pre-approval. Travel is booked through corporate travel portal only. + +## Tech Stack + +We use Python 3.11+ for backend, React 18 for frontend. Code reviews are mandatory; minimum 1 approver before merging to main. +""" + + +def chunk_doc(text: str) -> list[str]: + """Simple heading-aware chunking.""" + import re + sections = re.split(r"\n(?=#{1,3} )", text) + return [s.strip() for s in sections if s.strip()] + + +def build_kb(doc: str) -> Any: + client = chromadb.EphemeralClient() + embed_fn = embedding_functions.SentenceTransformerEmbeddingFunction(model_name=EMBED_MODEL) + collection = client.get_or_create_collection(name="knowledge_base", embedding_function=embed_fn) + chunks = chunk_doc(doc) + collection.add( + ids=[f"chunk_{i}" for i in range(len(chunks))], + documents=chunks, + ) + return collection + + +def retrieve(collection: Any, query: str, top_k: int = 2) -> list[str]: + result = collection.query(query_texts=[query], n_results=top_k) + return result["documents"][0] + + +def generate(query: str, contexts: list[str], llm: Any = None) -> str: + """Use LLM to answer query given retrieved contexts.""" + llm = llm or OpenAI(base_url=OLLAMA_BASE, api_key="ollama") + context_text = "\n\n---\n\n".join(contexts) + prompt = f"""Answer the user's question based ONLY on the context below. If the context doesn't contain the answer, say "I don't have that information". + +Context: +{context_text} + +Question: {query} + +Answer:""" + resp = llm.chat.completions.create( + model=MODEL, + messages=[{"role": "user", "content": prompt}], + ) + return resp.choices[0].message.content or "" + + +def rag(query: str, doc: str = KB_DOC, llm: Any = None, top_k: int = 2) -> dict: + """Full RAG pipeline.""" + collection = build_kb(doc) + contexts = retrieve(collection, query, top_k=top_k) + answer = generate(query, contexts, llm=llm) + return {"query": query, "contexts": contexts, "answer": answer} + + +if __name__ == "__main__": + queries = [ + "How many vacation days do I get?", + "Can I work fully remote?", + "What's the Python version?", + ] + for q in queries: + print(f"\n❓ {q}") + result = rag(q) + print(f" contexts retrieved: {len(result['contexts'])}") + print(f" answer: {result['answer']}") + + # Sanity: vacation query should retrieve the vacation section + result = rag("How many vacation days do I get?") + assert any("15 days" in c for c in result["contexts"]), \ + f"預期 retrieve 到含 15 days 的 chunk、得到 {result['contexts']}" + print("\n✅ 練習 4 通過 — 完整 RAG pipeline 跑通、$0/run（using Ollama {MODEL})") diff --git a/examples/stage-6/04-full-rag-pipeline/starter_anthropic.py b/examples/stage-6/04-full-rag-pipeline/starter_anthropic.py new file mode 100644 index 0000000..8dc0cbd --- /dev/null +++ b/examples/stage-6/04-full-rag-pipeline/starter_anthropic.py @@ -0,0 +1,66 @@ +"""Stage 6 練習 4：完整 RAG — Path B（Anthropic Claude）。 + +跟 starter.py 同一個 pipeline、只把 generate() 的 LLM 換成 Claude。 +Chunking + retrieval（vector DB）跨 backend 一致。 + +跑法： + pip install -r requirements.txt + export ANTHROPIC_API_KEY=sk-ant-... + python starter_anthropic.py + +預算：每次 ≈ $0.001（claude-haiku-4-5、短 context + 短答案）。 +Production RAG 通常用 cloud LLM 生成、答案質量明顯比 qwen2.5:3b 高。 +""" + +from __future__ import annotations + +import os +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic + +from starter import KB_DOC, build_kb, retrieve + +MODEL = os.environ.get("MODEL", "claude-haiku-4-5") + + +def generate_anthropic(query: str, contexts: list[str], client: Any = None) -> str: + client = client or anthropic.Anthropic() + context_text = "\n\n---\n\n".join(contexts) + prompt = f"""Answer the user's question based ONLY on the context below. If the context doesn't contain the answer, say "I don't have that information". + +Context: +{context_text} + +Question: {query} + +Answer:""" + resp = client.messages.create( + model=MODEL, + max_tokens=300, + messages=[{"role": "user", "content": prompt}], + ) + return " ".join(b.text for b in resp.content if b.type == "text") + + +def rag_anthropic(query: str, doc: str = KB_DOC, top_k: int = 2) -> dict: + collection = build_kb(doc) + contexts = retrieve(collection, query, top_k=top_k) + answer = generate_anthropic(query, contexts) + return {"query": query, "contexts": contexts, "answer": answer} + + +if __name__ == "__main__": + queries = [ + "How many vacation days do I get?", + "What's the Python version?", + ] + for q in queries: + print(f"\n❓ {q}") + result = rag_anthropic(q) + print(f" answer: {result['answer']}") + print(f"\n✅ 練習 4 (Anthropic) 通過 — Claude {MODEL}、≈$0.001/run") diff --git a/examples/stage-6/04-full-rag-pipeline/test.py b/examples/stage-6/04-full-rag-pipeline/test.py new file mode 100644 index 0000000..9476015 --- /dev/null +++ b/examples/stage-6/04-full-rag-pipeline/test.py @@ -0,0 +1,74 @@ +"""Stage 6 練習 4 自我驗證 — 用 mock LLM 跑完整 RAG pipeline。 +""" + +from __future__ import annotations + +import sys +from types import SimpleNamespace +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter import KB_DOC, build_kb, chunk_doc, generate, rag, retrieve + + +def make_mock_llm(answer: str = "Mock answer."): + llm = MagicMock() + msg = SimpleNamespace(content=answer) + choice = SimpleNamespace(message=msg) + llm.chat.completions.create.return_value = SimpleNamespace(choices=[choice]) + return llm + + +def test_chunk_doc_produces_sections(): + chunks = chunk_doc(KB_DOC) + # Sample has 4 ## sections + 1 # title → expect 5 + assert len(chunks) == 5, f"預期 5 個 chunk、得到 {len(chunks)}" + print("✅ test_chunk_doc_produces_sections") + + +def test_retrieve_finds_relevant_section(): + collection = build_kb(KB_DOC) + contexts = retrieve(collection, "vacation days", top_k=1) + assert "15 days" in contexts[0], f"預期含 '15 days'、得到 {contexts[0]}" + print("✅ test_retrieve_finds_relevant_section") + + +def test_generate_uses_context(): + llm = make_mock_llm("You get 15 days of vacation per year.") + contexts = ["Full-time employees get 15 days of paid vacation."] + answer = generate("How many vacation days?", contexts, llm=llm) + # 確認 prompt 帶上了 context（mock 看 create() 收到的 messages） + call_args = llm.chat.completions.create.call_args + prompt = call_args.kwargs["messages"][0]["content"] + assert "15 days" in prompt, "預期 prompt 含 context" + assert "How many vacation days?" in prompt + assert "15 days" in answer + print("✅ test_generate_uses_context") + + +def test_rag_full_pipeline_with_mock_llm(): + llm = make_mock_llm("Python 3.11+.") + result = rag("What Python version?", llm=llm) + assert result["query"] == "What Python version?" + assert len(result["contexts"]) > 0 + assert "Python 3.11" in result["answer"] + print("✅ test_rag_full_pipeline_with_mock_llm") + + +def test_rag_top_k_retrieval(): + """確認 retrieve 真的拿了 top_k 個 chunk。""" + llm = make_mock_llm("Answer.") + result = rag("vacation", llm=llm, top_k=3) + assert len(result["contexts"]) == 3 + print("✅ test_rag_top_k_retrieval") + + +if __name__ == "__main__": + test_chunk_doc_produces_sections() + test_retrieve_finds_relevant_section() + test_generate_uses_context() + test_rag_full_pipeline_with_mock_llm() + test_rag_top_k_retrieval() + print("\n🎉 全部通過 — RAG pipeline 完整邏輯正確") diff --git a/examples/stage-6/04-full-rag-pipeline/test_anthropic.py b/examples/stage-6/04-full-rag-pipeline/test_anthropic.py new file mode 100644 index 0000000..185db65 --- /dev/null +++ b/examples/stage-6/04-full-rag-pipeline/test_anthropic.py @@ -0,0 +1,35 @@ +"""Stage 6 練習 4 — Path B 自我驗證（mock Anthropic client）。 +""" + +from __future__ import annotations + +import sys +from types import SimpleNamespace +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter_anthropic import generate_anthropic + + +def make_anthropic_client(answer: str): + client = MagicMock() + block = SimpleNamespace(type="text", text=answer) + client.messages.create.return_value = SimpleNamespace(content=[block]) + return client + + +def test_generate_anthropic_uses_context(): + client = make_anthropic_client("Python 3.11+.") + answer = generate_anthropic("Python version?", ["We use Python 3.11+."], client=client) + call_args = client.messages.create.call_args + prompt = call_args.kwargs["messages"][0]["content"] + assert "Python 3.11" in prompt + assert answer == "Python 3.11+." + print("✅ test_generate_anthropic_uses_context") + + +if __name__ == "__main__": + test_generate_anthropic_uses_context() + print("\n🎉 Path B mock test 通過") diff --git a/examples/stage-6/05-long-term-memory/README.en.md b/examples/stage-6/05-long-term-memory/README.en.md new file mode 100644 index 0000000..ff07a45 --- /dev/null +++ b/examples/stage-6/05-long-term-memory/README.en.md @@ -0,0 +1,110 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# Exercise 5: Long-term Memory (remember across turns) + +Pairs with [Stage 6 — Memory & RAG](../../../stages/06-memory-rag.en.md) Exercise 5. + +## Task + +Agent remembers things across conversation turns. Implementation: + +``` +Turn 1: user: "I live in Taipei and prefer Python." + → maybe_remember_fact() catches "I + verb...", stores in vector store +Turn 2: user: "What's 2+2?" → recall returns nothing relevant, pure arithmetic +Turn 3: user: "Recommend a language for me." + → recall pulls "prefer Python", drops it into the system prompt → LLM recommends Python +``` + +This is **RAG's other side** — what you retrieve isn't documents, it's conversation history. + +## How to run + +### Path A (default, free, local) + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +Budget: **$0**. + +### Path B (Anthropic) + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +Budget: ~**$0.001** per run. + +## Validate the logic + +```bash +python test.py # 5 tests with mock LLM +python test_anthropic.py # Anthropic mock +``` + +## MemoryStore + chat flow + +```python +class MemoryStore: + def remember(self, fact: str) -> str: # add to vector store + def recall(self, query: str) -> list: # top-k semantic search + +def chat(user_msg, memory): + memories = memory.recall(user_msg, top_k=3) # 1. fetch relevant memories + system = f"Relevant memories: {memories}" # 2. put them in system prompt + return llm.invoke(system + user_msg) # 3. LLM uses them +``` + +**Key**: don't store memory as "context window history" (would blow the limit) — store as "semantic search index" — LLM only sees relevant ones. + +## vs. plain chat history + +| Dimension | Chat history (in-context) | Vector memory store | +|---|---|---| +| Stored where | messages array | ChromaDB | +| Capacity | Context window (200k-2M tokens) | Unbounded (millions of facts) | +| Cross-session | ❌ Session ends, gone | ✅ Persistent | +| Retrieval | All history in prompt (eats tokens) | top-k semantic search (precise) | +| Good for | Short conversations, single session | Long-term user relationships, multi-session, persona | + +**Production**: use both — last N turns in-context, beyond N + cross-session via vector memory. + +## What's a "memory-worthy fact" + +This demo uses a heuristic: user says "I + verb..." → store. Production is more sophisticated: + +1. **Explicit trigger**: user says "remember that..." +2. **Profile facts**: location / language / role / preferences +3. **Past decisions**: how the agent handled some situation before +4. **Negative feedback**: "don't suggest X" must persist +5. **LLM-extracted**: each turn, use an LLM to extract facts (mem0 / Letta / MemGPT all do this) + +## Common pitfalls + +- **Add to memory every turn**: vector store explodes. Filter with fact extraction +- **No dedup**: user says "I live in Taipei" 5 turns in a row, 5 copies stored. Add dedup (similarity > 0.95 = duplicate) +- **No forget / update mechanism**: user moved — "I now live in Tokyo". Old "Taipei" memory? Need a supersede concept +- **No context size control**: top-k too big, context bloats, LLM distracted +- **Privacy / GDPR**: user requests deletion; need `forget(user_id)` API + +## Production-ready tools + +- **[mem0](https://github.com/mem0ai/mem0)**: full memory pipeline — auto-fact-extraction, forgetting, user-scoped namespaces +- **[Letta (formerly MemGPT)](https://github.com/letta-ai/letta)**: two-tier memory (working + archival), OS-paging concept for LLMs +- **CrewAI memory**: built-in short / long-term memory +- **LangGraph checkpointer + persistent storage**: thread-level memory out of the box + +## Extensions + +- **Dedup**: `if similarity(new, existing) > 0.95: skip` +- **LLM-based fact extraction**: each turn, a small LLM extracts facts — beats heuristics +- **user_id scoping**: MemoryStore takes `user_id` filter so users don't contaminate each other +- **Plug into [mem0](https://github.com/mem0ai/mem0)**: don't roll your own memory pipeline in production diff --git a/examples/stage-6/05-long-term-memory/README.md b/examples/stage-6/05-long-term-memory/README.md new file mode 100644 index 0000000..633fff0 --- /dev/null +++ b/examples/stage-6/05-long-term-memory/README.md @@ -0,0 +1,125 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 練習 5：Long-term Memory（跨輪對話記得事情） + +對應 [Stage 6 — Memory & RAG](../../../stages/06-memory-rag.md) 練習 5。 +> 🎓 **學習模式**：這份 `starter.py` 是**完整解答**、不是 TODO skeleton。建議用**主動模式**——`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫一份 `starter.py`、跑 `python test.py` 驗證；卡 20 分鐘再回去對照 reference。完整方法論看 [`docs/HOW_TO_USE.md`](../../../docs/HOW_TO_USE.md)。 + +> 📚 **想要 chapter-length 深入版？** 本 folder 的 starter 是 illustrative 版、聚焦核心 pattern + 兩條 SDK path，不是進階深度教材。深度教材推薦： +> - [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents) ⭐ 中文圈最完整、章節式 + 16 種 production 能力。**本練習對應 hello-agents 的 long-term memory 章節** +> - [mem0](https://github.com/mem0ai/mem0)（auto fact extraction + forgetting）+ [Letta / MemGPT](https://github.com/letta-ai/letta)（兩級 memory pattern） +> - 完整 references 見 [Stage 6 精選 Projects](../../../stages/06-memory-rag.md#-精選-projects範本--spec--範例-collection) + + +## 任務 + +Agent 跨多輪對話記得 user 說過的事情。實作方式： + +``` +Turn 1: user: "I live in Taipei and prefer Python." + → maybe_remember_fact() 抓到「I + ...」格式、存進 vector store +Turn 2: user: "What's 2+2?" → recall 拿不到 relevant memory、純算術 +Turn 3: user: "Recommend a language for me." + → recall 拿到「prefer Python」、塞進 system prompt → LLM 知道推薦 Python +``` + +這是 **RAG 的另一面**——retrieve 對象從外部文件變成「對話歷史」。 + +## 怎麼跑 — 兩條路徑 + +### Path A（默認、本機免費） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +預算：**$0**。 + +### Path B（Anthropic） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +預算：每次 ≈ **$0.001**。 + +## 不花錢驗證程式邏輯 + +```bash +python test.py # 5 個 test、mock LLM +python test_anthropic.py # Anthropic mock +``` + +## MemoryStore + chat 流程 + +```python +class MemoryStore: + def remember(self, fact: str) -> str: # add to vector store + def recall(self, query: str) -> list: # top-k semantic search + +def chat(user_msg, memory): + memories = memory.recall(user_msg, top_k=3) # 1. 撈相關 memory + system = f"Relevant memories: {memories}" # 2. 塞進 system prompt + return llm.invoke(system + user_msg) # 3. LLM 看 memory 回答 +``` + +**核心**：memory 不是當「context window history」存（會爆），而是當「semantic search index」存——LLM 只看到相關的幾條。 + +## 跟普通 chat history 的差別 + +| 維度 | Chat history (in-context) | Vector memory store | +|---|---|---| +| 存放位置 | messages array | ChromaDB | +| 容量 | context window 限制（200k-2M token） | 不限（millions of facts） | +| 跨 session | ❌ session 結束就沒 | ✅ persistent | +| 撈出方式 | 所有 history 都進 prompt（吃 token） | top-k semantic search（精準） | +| 適合 | 短對話、單 session | 長 user 關係、多 session、persona | + +**Production 用法**：兩個都用——recent N 輪當 in-context、超過 N 輪 + 跨 session 用 vector memory。 + +## 什麼是「memory-worthy fact」 + +這份用 heuristic：user 說「I + verb...」就存。Production 通常更複雜： + +1. **Explicit trigger**：user 說「remember that...」 +2. **Profile facts**：location / language / role / preferences +3. **Past decisions**：上次 agent 怎麼處理某情境 +4. **Negative feedback**：「don't suggest X」要存 +5. **LLM-extracted**：每輪結束、用 LLM 自己抓 facts（mem0 / Letta / MemGPT 都這麼做） + +## 兩個 path 觀察重點 + +| 觀察項 | Anthropic Claude haiku | Ollama qwen2.5:3b | +|---|---|---| +| 把 memory 融進回答 | 自然（cite memory） | 偶爾 ignore memory、用通用答案 | +| 「無相關 memory」時不強用 | 守規則 | 較鬆 | +| 多 memory 整合 | 好 | 中 | + +## 常見坑 + +- **每輪都 add to memory**：vector store 會爆。要有「fact extraction」過濾、只存值得記的 +- **沒 dedup**：使用者多輪重複講「I live in Taipei」、會存 5 條一樣的。要加 dedup 邏輯（similarity > 0.95 視為重複） +- **Forget / update 機制缺失**：使用者搬家了「I now live in Tokyo」、舊 memory「Taipei」要怎麼處理？production 要支援「supersede」概念 +- **沒 context size 控制**：recall top-k 太大、context 爆，LLM 也分心 +- **隱私 / GDPR**：使用者要求刪除個人資訊、要有 `forget(user_id)` API + +## Production-ready 工具 + +- **[mem0](https://github.com/mem0ai/mem0)**：full memory pipeline、auto-fact-extraction、forgetting、user-scoped namespace +- **[Letta (formerly MemGPT)](https://github.com/letta-ai/letta)**：兩級 memory（working memory + archival storage）、把 OS 的 paging 觀念搬到 LLM +- **CrewAI memory**：framework 內建 short / long term memory +- **LangGraph checkpointer + persistent storage**：自帶 thread-level memory + +## 延伸 + +- **加 dedup**：`if similarity(new, existing) > 0.95: skip` +- **加 LLM-based fact extraction**：每輪結束用小 LLM 抓 facts、不靠 heuristic +- **加 user_id scoping**：MemoryStore 加 `user_id` filter、不同 user 互不污染 +- **接 [mem0](https://github.com/mem0ai/mem0)**：production 不要自己寫 memory pipeline、用成熟 lib diff --git a/examples/stage-6/05-long-term-memory/README.zh-Hans.md b/examples/stage-6/05-long-term-memory/README.zh-Hans.md new file mode 100644 index 0000000..3163d06 --- /dev/null +++ b/examples/stage-6/05-long-term-memory/README.zh-Hans.md @@ -0,0 +1,110 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 练习 5：Long-term Memory（跨轮对话记得事情） + +对应 [Stage 6 — Memory & RAG](../../../stages/06-memory-rag.zh-Hans.md) 练习 5。 + +## 任务 + +Agent 跨多轮对话记得 user 说过的事情。实作方式： + +``` +Turn 1: user: "I live in Taipei and prefer Python." + → maybe_remember_fact() 抓到「I + ...」格式、存进 vector store +Turn 2: user: "What's 2+2?" → recall 拿不到 relevant memory、纯算术 +Turn 3: user: "Recommend a language for me." + → recall 拿到「prefer Python」、塞进 system prompt → LLM 知道推荐 Python +``` + +这是 **RAG 的另一面**——retrieve 对象从外部文档变成“对话历史”。 + +## 怎么跑 + +### Path A（默认、本机免费） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +预算：**$0**。 + +### Path B（Anthropic） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +预算：每次 ≈ **$0.001**。 + +## 不花钱验证程序逻辑 + +```bash +python test.py # 5 个 test、mock LLM +python test_anthropic.py # Anthropic mock +``` + +## MemoryStore + chat 流程 + +```python +class MemoryStore: + def remember(self, fact: str) -> str: # add to vector store + def recall(self, query: str) -> list: # top-k semantic search + +def chat(user_msg, memory): + memories = memory.recall(user_msg, top_k=3) # 1. 捞相关 memory + system = f"Relevant memories: {memories}" # 2. 塞进 system prompt + return llm.invoke(system + user_msg) # 3. LLM 看 memory 回答 +``` + +**核心**：memory 不是当“context window history”存（会爆），而是当“semantic search index”存——LLM 只看到相关的几条。 + +## 跟普通 chat history 的差别 + +| 维度 | Chat history (in-context) | Vector memory store | +|---|---|---| +| 存放位置 | messages array | ChromaDB | +| 容量 | context window 限制（200k-2M token） | 不限（millions of facts） | +| 跨 session | ❌ session 结束就没 | ✅ persistent | +| 捞出方式 | 所有 history 都进 prompt（吃 token） | top-k semantic search（精准） | +| 适合 | 短对话、单 session | 长 user 关系、多 session、persona | + +**Production 用法**：两个都用——recent N 轮当 in-context、超过 N 轮 + 跨 session 用 vector memory。 + +## 什么是“memory-worthy fact” + +这份用 heuristic：user 说“I + verb...”就存。Production 通常更复杂： + +1. **Explicit trigger**：user 说“remember that...” +2. **Profile facts**：location / language / role / preferences +3. **Past decisions**：上次 agent 怎么处理某情境 +4. **Negative feedback**：“don't suggest X”要存 +5. **LLM-extracted**：每轮结束、用 LLM 自己抓 facts（mem0 / Letta / MemGPT 都这么做） + +## 常见坑 + +- **每轮都 add to memory**：vector store 会爆。要有“fact extraction”过滤、只存值得记的 +- **没 dedup**：用户多轮重复讲“I live in Taipei”、会存 5 条一样的。要加 dedup 逻辑（similarity > 0.95 视为重复） +- **Forget / update 机制缺失**：用户搬家了“I now live in Tokyo”、旧 memory“Taipei”要怎么处理？production 要支持“supersede”概念 +- **没 context size 控制**：recall top-k 太大、context 爆，LLM 也分心 +- **隐私 / GDPR**：用户要求删除个人信息、要有 `forget(user_id)` API + +## Production-ready 工具 + +- **[mem0](https://github.com/mem0ai/mem0)**：full memory pipeline、auto-fact-extraction、forgetting、user-scoped namespace +- **[Letta (formerly MemGPT)](https://github.com/letta-ai/letta)**：两级 memory（working memory + archival storage） +- **CrewAI memory**：framework 内建 short / long term memory +- **LangGraph checkpointer + persistent storage**：自带 thread-level memory + +## 延伸 + +- **加 dedup**：`if similarity(new, existing) > 0.95: skip` +- **加 LLM-based fact extraction**：每轮结束用小 LLM 抓 facts、不靠 heuristic +- **加 user_id scoping**：MemoryStore 加 `user_id` filter、不同 user 互不污染 +- **接 [mem0](https://github.com/mem0ai/mem0)**：production 不要自己写 memory pipeline、用成熟 lib diff --git a/examples/stage-6/05-long-term-memory/requirements.txt b/examples/stage-6/05-long-term-memory/requirements.txt new file mode 100644 index 0000000..be0870d --- /dev/null +++ b/examples/stage-6/05-long-term-memory/requirements.txt @@ -0,0 +1,4 @@ +chromadb>=0.4,<2.0 +sentence-transformers>=2.7,<4.0 +openai>=1.50,<2.0 +anthropic>=0.40,<1.0 diff --git a/examples/stage-6/05-long-term-memory/starter.py b/examples/stage-6/05-long-term-memory/starter.py new file mode 100644 index 0000000..2284b42 --- /dev/null +++ b/examples/stage-6/05-long-term-memory/starter.py @@ -0,0 +1,128 @@ +"""Stage 6 練習 5：Long-term memory — Path A（Ollama + ChromaDB、$0）。 + +Agent 跨多輪對話記得事情。實作：每輪結束、把使用者說過的「事實」存進 vector store； +下一輪先 retrieve 相關 memory、塞進 prompt。 + +跑法： + pip install -r requirements.txt + ollama pull qwen2.5:3b + ollama serve + python starter.py + +驗證： + python test.py +""" + +from __future__ import annotations + +import os +import sys +import uuid +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import chromadb +from chromadb.utils import embedding_functions +from openai import OpenAI + +MODEL = os.environ.get("MODEL", "qwen2.5:3b") +OLLAMA_BASE = os.environ.get("OLLAMA_API_BASE", "http://localhost:11434/v1") + + +# === Memory store === + +class MemoryStore: + """Long-term memory backed by ChromaDB vector search.""" + + def __init__(self, collection_name: str = "memory"): + client = chromadb.EphemeralClient() + embed_fn = embedding_functions.SentenceTransformerEmbeddingFunction( + model_name="sentence-transformers/all-MiniLM-L6-v2", + ) + self.collection = client.get_or_create_collection( + name=collection_name, embedding_function=embed_fn, + ) + + def remember(self, fact: str) -> str: + """Add a fact to memory. Returns the memory id.""" + mid = str(uuid.uuid4()) + self.collection.add(ids=[mid], documents=[fact]) + return mid + + def recall(self, query: str, top_k: int = 3) -> list[str]: + """Retrieve top-k relevant memories.""" + if self.collection.count() == 0: + return [] + result = self.collection.query( + query_texts=[query], n_results=min(top_k, self.collection.count()), + ) + return result["documents"][0] + + def all(self) -> list[str]: + return self.collection.get()["documents"] + + +# === Conversational agent with memory === + +def chat(user_msg: str, memory: MemoryStore, llm: Any = None, history: list = None) -> dict: + """One conversation turn with memory recall + storage.""" + llm = llm or OpenAI(base_url=OLLAMA_BASE, api_key="ollama") + history = history or [] + + # 1. Recall relevant memories + memories = memory.recall(user_msg, top_k=3) + memory_text = "\n".join(f"- {m}" for m in memories) if memories else "(no relevant memories)" + + # 2. Build prompt with memory context + system = f"""You are a helpful assistant with long-term memory. Use the relevant memories below to personalize answers. + +Relevant memories: +{memory_text} +""" + messages = [{"role": "system", "content": system}] + history + [{"role": "user", "content": user_msg}] + + # 3. LLM responds + resp = llm.chat.completions.create(model=MODEL, messages=messages) + answer = resp.choices[0].message.content or "" + + return {"answer": answer, "recalled": memories, "user_msg": user_msg} + + +def maybe_remember_fact(user_msg: str, memory: MemoryStore) -> str | None: + """Heuristic: if user says 'I am / I like / I live', store as a fact.""" + triggers = ["I am ", "I like ", "I live ", "My favorite ", "I work ", "I prefer "] + for t in triggers: + if t.lower() in user_msg.lower(): + mid = memory.remember(user_msg) + return mid + return None + + +if __name__ == "__main__": + memory = MemoryStore() + + # Turn 1: user shares facts (these go into memory) + turn1 = "I live in Taipei and I prefer Python over JavaScript." + print(f"\n[Turn 1] user: {turn1}") + fact_id = maybe_remember_fact(turn1, memory) + if fact_id: + print(f" (remembered as {fact_id[:8]}...)") + + # Turn 2: an unrelated question (memory still benign) + r2 = chat("What's 2 + 2?", memory) + print(f"\n[Turn 2] user: {r2['user_msg']}") + print(f" recalled: {r2['recalled']}") + print(f" answer: {r2['answer']}") + + # Turn 3: user asks for a recommendation — memory should kick in + r3 = chat("Recommend a programming language for me to learn.", memory) + print(f"\n[Turn 3] user: {r3['user_msg']}") + print(f" recalled: {r3['recalled']}") + print(f" answer: {r3['answer']}") + + # Sanity: turn 3 應該 recall 到「prefer Python」這條 memory + assert any("Python" in m for m in r3["recalled"]), \ + f"預期 recall 到 Python preference、得到 {r3['recalled']}" + print("\n✅ 練習 5 通過 — long-term memory 跨輪 retrieve 成功、$0/run") diff --git a/examples/stage-6/05-long-term-memory/starter_anthropic.py b/examples/stage-6/05-long-term-memory/starter_anthropic.py new file mode 100644 index 0000000..8aab89b --- /dev/null +++ b/examples/stage-6/05-long-term-memory/starter_anthropic.py @@ -0,0 +1,55 @@ +"""Stage 6 練習 5：Long-term memory — Path B（Anthropic）。 + +把 generate 改成 Claude；memory store（vector DB）跨 backend 一致。 + +跑法： + pip install -r requirements.txt + export ANTHROPIC_API_KEY=sk-ant-... + python starter_anthropic.py +""" + +from __future__ import annotations + +import os +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic + +from starter import MemoryStore, maybe_remember_fact + +MODEL = os.environ.get("MODEL", "claude-haiku-4-5") + + +def chat_anthropic(user_msg: str, memory: MemoryStore, client: Any = None) -> dict: + client = client or anthropic.Anthropic() + memories = memory.recall(user_msg, top_k=3) + memory_text = "\n".join(f"- {m}" for m in memories) if memories else "(no relevant memories)" + + system = f"""You are a helpful assistant with long-term memory. Use the relevant memories below to personalize answers. + +Relevant memories: +{memory_text} +""" + resp = client.messages.create( + model=MODEL, + max_tokens=300, + system=system, + messages=[{"role": "user", "content": user_msg}], + ) + answer = " ".join(b.text for b in resp.content if b.type == "text") + return {"answer": answer, "recalled": memories, "user_msg": user_msg} + + +if __name__ == "__main__": + memory = MemoryStore() + maybe_remember_fact("I live in Taipei and I prefer Python over JavaScript.", memory) + maybe_remember_fact("I work as a data scientist.", memory) + + r = chat_anthropic("Recommend a programming language for me to learn.", memory) + print(f"recalled: {r['recalled']}") + print(f"answer: {r['answer']}") + print(f"\n✅ 練習 5 (Anthropic path) 通過 — Claude {MODEL}、≈$0.001/run") diff --git a/examples/stage-6/05-long-term-memory/test.py b/examples/stage-6/05-long-term-memory/test.py new file mode 100644 index 0000000..c8294b2 --- /dev/null +++ b/examples/stage-6/05-long-term-memory/test.py @@ -0,0 +1,86 @@ +"""Stage 6 練習 5 自我驗證 — MemoryStore + maybe_remember_fact 邏輯。 +""" + +from __future__ import annotations + +import sys +import uuid +from types import SimpleNamespace +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter import MemoryStore, chat, maybe_remember_fact + + +def fresh_memory(): + """每個 test 用獨立 collection 避免 EphemeralClient 跨 test 共享 state。""" + return MemoryStore(collection_name=f"test_{uuid.uuid4().hex[:8]}") + + +def make_mock_llm(answer: str = "ok"): + llm = MagicMock() + msg = SimpleNamespace(content=answer) + llm.chat.completions.create.return_value = SimpleNamespace( + choices=[SimpleNamespace(message=msg)] + ) + return llm + + +def test_memory_remember_and_recall(): + mem = fresh_memory() + mem.remember("User prefers Python over JavaScript.") + mem.remember("User lives in Taipei.") + mem.remember("Bananas are yellow.") # unrelated + + recalled = mem.recall("what programming language does the user like", top_k=2) + assert any("Python" in m for m in recalled), f"預期 recall Python、得到 {recalled}" + print("✅ test_memory_remember_and_recall") + + +def test_memory_empty_recall(): + mem = fresh_memory() + assert mem.recall("anything") == [] + print("✅ test_memory_empty_recall") + + +def test_maybe_remember_fact_triggers_on_self_statements(): + mem = fresh_memory() + fid = maybe_remember_fact("I live in Taipei", mem) + assert fid is not None + assert mem.collection.count() == 1 + print("✅ test_maybe_remember_fact_triggers_on_self_statements") + + +def test_maybe_remember_fact_skips_non_self_statements(): + mem = fresh_memory() + fid = maybe_remember_fact("What's the weather?", mem) + assert fid is None + assert mem.collection.count() == 0 + print("✅ test_maybe_remember_fact_skips_non_self_statements") + + +def test_chat_uses_memory_in_prompt(): + """chat 應該把 recalled memory 塞進 system prompt。""" + mem = fresh_memory() + mem.remember("User prefers Python over JavaScript.") + + llm = make_mock_llm("I recommend Python.") + result = chat("Recommend a programming language.", mem, llm=llm) + + # mock 看 create() 收到的 messages + call_args = llm.chat.completions.create.call_args + system_msg = call_args.kwargs["messages"][0]["content"] + assert "Python over JavaScript" in system_msg, "system prompt 應含 memory" + assert "Python" in result["recalled"][0] + print("✅ test_chat_uses_memory_in_prompt") + + +if __name__ == "__main__": + test_memory_remember_and_recall() + test_memory_empty_recall() + test_maybe_remember_fact_triggers_on_self_statements() + test_maybe_remember_fact_skips_non_self_statements() + test_chat_uses_memory_in_prompt() + print("\n🎉 全部通過 — long-term memory 邏輯正確") diff --git a/examples/stage-6/05-long-term-memory/test_anthropic.py b/examples/stage-6/05-long-term-memory/test_anthropic.py new file mode 100644 index 0000000..62b5a21 --- /dev/null +++ b/examples/stage-6/05-long-term-memory/test_anthropic.py @@ -0,0 +1,34 @@ +"""Stage 6 練習 5 — Path B mock test。 +""" + +from __future__ import annotations + +import sys +from types import SimpleNamespace +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter import MemoryStore +from starter_anthropic import chat_anthropic + + +def test_chat_anthropic_uses_memory_in_system_prompt(): + mem = MemoryStore() + mem.remember("User prefers Python.") + + client = MagicMock() + block = SimpleNamespace(type="text", text="I recommend Python.") + client.messages.create.return_value = SimpleNamespace(content=[block]) + + result = chat_anthropic("Recommend a language.", mem, client=client) + call_args = client.messages.create.call_args + system_msg = call_args.kwargs["system"] + assert "User prefers Python." in system_msg + print("✅ test_chat_anthropic_uses_memory_in_system_prompt") + + +if __name__ == "__main__": + test_chat_anthropic_uses_memory_in_system_prompt() + print("\n🎉 Path B mock test 通過") diff --git a/examples/stage-7/01-multi-agent-debate/README.en.md b/examples/stage-7/01-multi-agent-debate/README.en.md new file mode 100644 index 0000000..326f3b2 --- /dev/null +++ b/examples/stage-7/01-multi-agent-debate/README.en.md @@ -0,0 +1,98 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# Exercise 1: Multi-Agent Debate + +Pairs with [Stage 7 — Multi-Agent & Production](../../../stages/07-multi-agent-production.en.md) Exercise 1. + +## Task + +Three agents (PRO + CON + Judge) debate the same question: + +![Multi-agent debate: PRO / CON / Judge](../../../resources/diagrams/multi-agent-debate-flow.en.png) + +PRO and CON are called **independently** — they don't see each other's arguments (prevents bias propagation). The Judge sees both and decides. + +## Why this pattern matters + +- **Reduces single-LLM bias**: one LLM tends to bake in a stance and ignore counterarguments +- **Strengthens reasoning**: forcing both sides to articulate produces cleaner traces +- **Auditability**: high-stakes production decisions (policy / medical / legal review) need trails +- **Disagreement = signal**: when agents disagree, the question may be ambiguous or the model uncertain + +## How to run + +### Path A (default, free, local) + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +Budget: **$0**. Three LLM calls × CPU ≈ 15-45s. + +### Path B (Anthropic) + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +Budget: ~**$0.003** per run (3 calls × short prompts × claude-haiku-4-5). + +## Validate the logic + +```bash +python test.py # 3 tests with mock LLM, verify judge sees pro+con +python test_anthropic.py +``` + +## Key design points + +```python +# Same model, different system prompts +pro = llm_call(system="argue PRO position", user=question) +con = llm_call(system="argue CON position", user=question) + +# Judge sees question + pro + con +judge = llm_call( + system="neutral judge, output WINNER=PRO or WINNER=CON", + user=f"Question: {question}\n\nPRO: {pro}\n\nCON: {con}", +) +``` + +**Key**: PRO and CON are **independent calls**. Don't pass PRO's output into CON — CON would then react to PRO rather than think independently, amplifying bias. + +## Production-ready variants + +- **N-way debate**: 3+ agents holding different perspectives (e.g., "engineer / PM / customer view") +- **Iterative debate**: PRO and CON see each other and rebut for N rounds; first to concede loses +- **Different models**: PRO uses Claude, CON uses GPT, Judge uses Gemini — cross-model debate finds blind spots +- **Self-consistency**: run the debate 3 times, see how stable the Judge's verdict is + +## Path observations + +| Observation | Anthropic Claude | Ollama qwen2.5:3b | +|---|---|---| +| PRO / CON hold their positions | Stable | Sometimes both turn "balanced" — no clear stance | +| Judge outputs clear WINNER | Stable | Occasionally skips the `WINNER=` format | +| Reasoning quality | High | Medium | +| Cost | $0.003 | $0 | + +## Common pitfalls + +- **Identical system prompt for PRO and CON**: outputs converge, debate is meaningless +- **Fixed PRO-then-CON order in Judge prompt**: may bias toward whichever comes first (recency / primacy). Production should shuffle +- **No structured Judge output**: without `WINNER=PRO or CON` format, downstream parsing is painful +- **Prompts too short**: 1-sentence PRO and CON give the Judge nothing to weigh + +## Extensions + +- **Plug into [LangGraph](https://langchain-ai.github.io/langgraph/)**: PRO/CON become parallel nodes, Judge a join +- **Use [AutoGen](https://github.com/microsoft/autogen)**: AutoGen has first-class multi-agent debate support +- **Add confidence**: Judge outputs `confidence 0-1`; low confidence escalates to a human +- **Plug into eval (Exercise 2)**: run debate on 50 cases vs. single-agent baseline diff --git a/examples/stage-7/01-multi-agent-debate/README.md b/examples/stage-7/01-multi-agent-debate/README.md new file mode 100644 index 0000000..7aae773 --- /dev/null +++ b/examples/stage-7/01-multi-agent-debate/README.md @@ -0,0 +1,105 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 練習 1：Multi-Agent 辯論 + +對應 [Stage 7 — Multi-Agent & Production](../../../stages/07-multi-agent-production.md) 練習 1。 +> 🎓 **學習模式**：這份 `starter.py` 是**完整解答**、不是 TODO skeleton。建議用**主動模式**——`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫一份 `starter.py`、跑 `python test.py` 驗證；卡 20 分鐘再回去對照 reference。完整方法論看 [`docs/HOW_TO_USE.md`](../../../docs/HOW_TO_USE.md)。 + +> 📚 **想要 chapter-length 深入版？** 本 folder 的 starter 是 illustrative 版、聚焦核心 pattern + 兩條 SDK path，不是進階深度教材。深度教材推薦： +> - [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents) ⭐ 中文圈最完整、章節式 + 16 種 production 能力。**本練習對應 hello-agents 的 multi-agent collaboration / debate 章節** +> - [Anthropic — Building Effective Agents debate](https://www.anthropic.com/engineering/building-effective-agents) + [Microsoft AutoGen multi-agent docs](https://microsoft.github.io/autogen/) +> - 完整 references 見 [Stage 7 精選 Projects](../../../stages/07-multi-agent-production.md#-精選-projects範本--sdk--工具-collection) + + +## 任務 + +3 個 agent（PRO + CON + Judge）對同問題辯論： + +![Multi-agent debate：PRO / CON / Judge](../../../resources/diagrams/multi-agent-debate-flow.png) + +PRO 跟 CON **獨立** call、互不看到對方論點（避免 bias propagation）；Judge 看完兩邊再裁決。 + +## 為什麼這個 pattern 重要 + +- **降低 single-LLM bias**：單一 LLM 給的答案常帶 stance、不主動指出反面 +- **強化 reasoning**：兩個 LLM 強迫 articulate 各自立場、reasoning trace 更清楚 +- **可解釋性**：production 高風險決策（policy / 醫療 / 法律 review）有 audit trail +- **錯誤偵測**：兩 agent 互不同意時、可能就是答案有歧義 / model 不確定 + +## 怎麼跑 + +### Path A（默認、本機免費） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +預算：**$0**。3 個 LLM call × CPU ≈ 15-45 秒。 + +### Path B（Anthropic） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +預算：每次 ≈ **$0.003**（3 LLM call × short prompt × claude-haiku-4-5）。 + +## 不花錢驗證程式邏輯 + +```bash +python test.py # 3 個 test、mock 3 LLM call、驗 judge 看到 pro+con +python test_anthropic.py +``` + +## 重要設計細節 + +```python +# pro / con 用同一個 model、不同 system prompt +pro = llm_call(system="argue PRO position", user=question) +con = llm_call(system="argue CON position", user=question) + +# judge 看「question + pro + con」做裁決 +judge = llm_call( + system="neutral judge, output WINNER=PRO or WINNER=CON", + user=f"Question: {question}\n\nPRO: {pro}\n\nCON: {con}", +) +``` + +**Key**：pro / con **獨立 call**——不要把 pro 結果丟給 con。如果 con 看到 pro、會傾向反駁 pro 而非獨立思考、bias 反而加強。 + +## Production-ready 變形 + +- **N-way debate**：3+ agent 各持不同立場（e.g. "engineer / PM / customer view"） +- **Iterative debate**：pro 跟 con 互看 N 輪、看誰先放棄 +- **Different models**：pro 用 Claude、con 用 GPT、judge 用 Gemini——cross-model debate 找盲點 +- **Self-consistency check**：跑 3 次 debate、看 judge 結果穩定度 + +## 兩個 path 觀察重點 + +| 觀察項 | Anthropic Claude | Ollama qwen2.5:3b | +|---|---|---| +| pro / con 持立場 | 穩 | 偶爾兩邊都講「平衡 view」、立場不堅定 | +| judge 給明確 WINNER | 穩 | 偶爾不給 WINNER= 格式 | +| reasoning 質量 | 高 | 中 | +| 成本 | $0.003 | $0 | + +## 常見坑 + +- **PRO 跟 CON 用同一個 system prompt**：模型答案會同質、debate 意義消失 +- **Judge 看 pro/con 順序固定**：可能 bias 第一個（recency / primacy effect）。production 可以隨機 shuffle +- **沒 structured judge output**：不寫 `WINNER=PRO or CON` 格式、後續 parsing 困難 +- **太短 prompt**：pro / con 各只給 1 句、judge 沒材料 + +## 延伸 + +- **接 [LangGraph](https://langchain-ai.github.io/langgraph/)**：pro/con 變 parallel node、judge 變 join node +- **接 [AutoGen](https://github.com/microsoft/autogen)**：AutoGen 對 multi-agent debate 有專門支援 +- **加 confidence**：judge 多 output confidence 0-1、low confidence 才把 case escalate 給人 +- **接 eval（練習 2）**：跑 debate 在 50 個 case、跟 single-agent baseline 比準確率 diff --git a/examples/stage-7/01-multi-agent-debate/README.zh-Hans.md b/examples/stage-7/01-multi-agent-debate/README.zh-Hans.md new file mode 100644 index 0000000..df44c10 --- /dev/null +++ b/examples/stage-7/01-multi-agent-debate/README.zh-Hans.md @@ -0,0 +1,98 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 练习 1：Multi-Agent 辩论 + +对应 [Stage 7 — Multi-Agent & Production](../../../stages/07-multi-agent-production.zh-Hans.md) 练习 1。 + +## 任务 + +3 个 agent（PRO + CON + Judge）对同问题辩论： + +![Multi-agent debate：PRO / CON / Judge](../../../resources/diagrams/multi-agent-debate-flow.zh-Hans.png) + +PRO 跟 CON **独立** call、互不看到对方论点（避免 bias propagation）；Judge 看完两边再裁决。 + +## 为什么这个 pattern 重要 + +- **降低 single-LLM bias**：单一 LLM 给的答案常带 stance、不主动指出反面 +- **强化 reasoning**：两个 LLM 强迫 articulate 各自立场、reasoning trace 更清楚 +- **可解释性**：production 高风险决策（policy / 医疗 / 法律 review）有 audit trail +- **错误侦测**：两 agent 互不同意时、可能就是答案有歧义 / model 不确定 + +## 怎么跑 + +### Path A（默认、本机免费） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +预算：**$0**。3 个 LLM call × CPU ≈ 15-45 秒。 + +### Path B（Anthropic） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +预算：每次 ≈ **$0.003**（3 LLM call × short prompt × claude-haiku-4-5）。 + +## 不花钱验证程序逻辑 + +```bash +python test.py # 3 个 test、mock 3 LLM call、验 judge 看到 pro+con +python test_anthropic.py +``` + +## 重要设计细节 + +```python +# pro / con 用同一个 model、不同 system prompt +pro = llm_call(system="argue PRO position", user=question) +con = llm_call(system="argue CON position", user=question) + +# judge 看「question + pro + con」做裁决 +judge = llm_call( + system="neutral judge, output WINNER=PRO or WINNER=CON", + user=f"Question: {question}\n\nPRO: {pro}\n\nCON: {con}", +) +``` + +**Key**：pro / con **独立 call**——不要把 pro 结果丢给 con。如果 con 看到 pro、会倾向反驳 pro 而非独立思考、bias 反而加强。 + +## Production-ready 变形 + +- **N-way debate**：3+ agent 各持不同立场（e.g. "engineer / PM / customer view"） +- **Iterative debate**：pro 跟 con 互看 N 轮、看谁先放弃 +- **Different models**：pro 用 Claude、con 用 GPT、judge 用 Gemini——cross-model debate 找盲点 +- **Self-consistency check**：跑 3 次 debate、看 judge 结果稳定度 + +## 两个 path 观察重点 + +| 观察项 | Anthropic Claude | Ollama qwen2.5:3b | +|---|---|---| +| pro / con 持立场 | 稳 | 偶尔两边都讲“平衡 view”、立场不坚定 | +| judge 给明确 WINNER | 稳 | 偶尔不给 WINNER= 格式 | +| reasoning 质量 | 高 | 中 | +| 成本 | $0.003 | $0 | + +## 常见坑 + +- **PRO 跟 CON 用同一个 system prompt**：模型答案会同质、debate 意义消失 +- **Judge 看 pro/con 顺序固定**：可能 bias 第一个（recency / primacy effect）。production 可以随机 shuffle +- **没 structured judge output**：不写 `WINNER=PRO or CON` 格式、后续 parsing 困难 +- **太短 prompt**：pro / con 各只给 1 句、judge 没材料 + +## 延伸 + +- **接 [LangGraph](https://langchain-ai.github.io/langgraph/)**：pro/con 变 parallel node、judge 变 join node +- **接 [AutoGen](https://github.com/microsoft/autogen)**：AutoGen 对 multi-agent debate 有专门支援 +- **加 confidence**：judge 多 output confidence 0-1、low confidence 才把 case escalate 给人 +- **接 eval（练习 2）**：跑 debate 在 50 个 case、跟 single-agent baseline 比准确率 diff --git a/examples/stage-7/01-multi-agent-debate/requirements.txt b/examples/stage-7/01-multi-agent-debate/requirements.txt new file mode 100644 index 0000000..7c268c1 --- /dev/null +++ b/examples/stage-7/01-multi-agent-debate/requirements.txt @@ -0,0 +1,2 @@ +openai>=1.50,<2.0 +anthropic>=0.40,<1.0 diff --git a/examples/stage-7/01-multi-agent-debate/starter.py b/examples/stage-7/01-multi-agent-debate/starter.py new file mode 100644 index 0000000..bd0e7c7 --- /dev/null +++ b/examples/stage-7/01-multi-agent-debate/starter.py @@ -0,0 +1,65 @@ +"""Stage 7 練習 1：Multi-Agent 辯論 — Path A（Ollama 默認、$0）。 + +2 個 agent 對同一個問題持相反立場辯論、第 3 個 judge agent 評分。 +這個 pattern（debate / peer review）可以**降低單一 LLM 的 bias**——production +高賭注決策（policy / 醫療 / 法律 review）常用。 + +跑法： + pip install -r requirements.txt + ollama pull qwen2.5:3b + ollama serve + python starter.py +""" + +from __future__ import annotations + +import os +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +MODEL = os.environ.get("MODEL", "qwen2.5:3b") +OLLAMA_BASE = os.environ.get("OLLAMA_API_BASE", "http://localhost:11434/v1") + + +def llm_call(system: str, user: str, llm: Any = None) -> str: + llm = llm or OpenAI(base_url=OLLAMA_BASE, api_key="ollama") + resp = llm.chat.completions.create( + model=MODEL, + messages=[{"role": "system", "content": system}, {"role": "user", "content": user}], + ) + return resp.choices[0].message.content or "" + + +def debate(question: str, llm: Any = None) -> dict: + """3-agent debate：pro / con / judge。""" + pro_argument = llm_call( + system="You argue the PRO position on the user's question. Be concise (2-3 sentences).", + user=question, llm=llm, + ) + con_argument = llm_call( + system="You argue the CON position on the user's question. Be concise (2-3 sentences).", + user=question, llm=llm, + ) + judge_verdict = llm_call( + system="You are a neutral judge. Read both arguments below and pick the stronger one. " + "Reply with: WINNER=PRO or WINNER=CON, then 1-sentence reasoning.", + user=f"Question: {question}\n\nPRO: {pro_argument}\n\nCON: {con_argument}", + llm=llm, + ) + return {"question": question, "pro": pro_argument, "con": con_argument, "judge": judge_verdict} + + +if __name__ == "__main__": + q = "Should small teams use a framework (LangGraph/CrewAI) or build agents from scratch?" + print(f"❓ Question: {q}\n") + result = debate(q) + print(f"PRO: {result['pro']}\n") + print(f"CON: {result['con']}\n") + print(f"Judge: {result['judge']}") + assert "WINNER" in result["judge"].upper() + print("\n✅ 練習 1 通過 — 3-agent debate 跑通、$0/run") diff --git a/examples/stage-7/01-multi-agent-debate/starter_anthropic.py b/examples/stage-7/01-multi-agent-debate/starter_anthropic.py new file mode 100644 index 0000000..532d3d5 --- /dev/null +++ b/examples/stage-7/01-multi-agent-debate/starter_anthropic.py @@ -0,0 +1,57 @@ +"""Stage 7 練習 1：Multi-Agent 辯論 — Path B（Anthropic Claude）。 + +Multi-agent debate 對 model 質量敏感——3 個 agent 都要能持住自己的立場、Claude 比 qwen 穩。 + +跑法： + pip install -r requirements.txt + export ANTHROPIC_API_KEY=sk-ant-... + python starter_anthropic.py +""" + +from __future__ import annotations + +import os +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic + +MODEL = os.environ.get("MODEL", "claude-haiku-4-5") + + +def llm_call_anthropic(system: str, user: str, client: Any = None) -> str: + client = client or anthropic.Anthropic() + resp = client.messages.create( + model=MODEL, max_tokens=300, system=system, + messages=[{"role": "user", "content": user}], + ) + return " ".join(b.text for b in resp.content if b.type == "text") + + +def debate_anthropic(question: str, client: Any = None) -> dict: + pro = llm_call_anthropic( + "You argue the PRO position on the user's question. Be concise (2-3 sentences).", + question, client=client, + ) + con = llm_call_anthropic( + "You argue the CON position on the user's question. Be concise (2-3 sentences).", + question, client=client, + ) + judge = llm_call_anthropic( + "You are a neutral judge. Reply with: WINNER=PRO or WINNER=CON, then 1-sentence reasoning.", + f"Question: {question}\n\nPRO: {pro}\n\nCON: {con}", + client=client, + ) + return {"question": question, "pro": pro, "con": con, "judge": judge} + + +if __name__ == "__main__": + q = "Should small teams use a framework or build from scratch?" + print(f"❓ {q}\n") + r = debate_anthropic(q) + for k in ("pro", "con", "judge"): + print(f"{k.upper()}: {r[k]}\n") + print(f"✅ 練習 1 (Anthropic) 通過 — Claude {MODEL}、≈$0.003/run（3 個 LLM call）") diff --git a/examples/stage-7/01-multi-agent-debate/test.py b/examples/stage-7/01-multi-agent-debate/test.py new file mode 100644 index 0000000..5b83237 --- /dev/null +++ b/examples/stage-7/01-multi-agent-debate/test.py @@ -0,0 +1,66 @@ +"""Stage 7 練習 1 自我驗證 — mock 3 個 LLM call。""" + +from __future__ import annotations + +import sys +from types import SimpleNamespace +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter import debate, llm_call + + +def make_llm(responses: list[str]): + llm = MagicMock() + llm.chat.completions.create.side_effect = [ + SimpleNamespace(choices=[SimpleNamespace(message=SimpleNamespace(content=r))]) + for r in responses + ] + return llm + + +def test_llm_call_basic(): + llm = make_llm(["ok"]) + out = llm_call("be brief", "hi", llm=llm) + assert out == "ok" + print("✅ test_llm_call_basic") + + +def test_debate_3_calls(): + """debate 應該打 3 次 LLM（pro, con, judge）、且 judge 看到兩邊 argument。""" + llm = make_llm([ + "PRO: frameworks save time.", + "CON: frameworks add lock-in.", + "WINNER=PRO. Time savings outweigh lock-in for small teams.", + ]) + result = debate("Should small teams use a framework?", llm=llm) + assert llm.chat.completions.create.call_count == 3 + assert "frameworks save time" in result["pro"] + assert "frameworks add lock-in" in result["con"] + assert "WINNER=PRO" in result["judge"] + + # 第 3 個 call（judge）的 user prompt 應該包含 pro + con + third_call = llm.chat.completions.create.call_args_list[2] + judge_user = third_call.kwargs["messages"][1]["content"] + assert "frameworks save time" in judge_user + assert "frameworks add lock-in" in judge_user + print("✅ test_debate_3_calls") + + +def test_debate_independent_pro_con(): + """確認 pro / con 各自獨立 call、不互相看到（避免 bias propagation）。""" + llm = make_llm(["pro arg", "con arg", "WINNER=PRO."]) + debate("test", llm=llm) + # 第 1 個 call (pro) 的 user prompt 應該 = question、不含 con + first = llm.chat.completions.create.call_args_list[0] + assert first.kwargs["messages"][1]["content"] == "test" + print("✅ test_debate_independent_pro_con") + + +if __name__ == "__main__": + test_llm_call_basic() + test_debate_3_calls() + test_debate_independent_pro_con() + print("\n🎉 全部通過 — multi-agent debate 邏輯正確") diff --git a/examples/stage-7/01-multi-agent-debate/test_anthropic.py b/examples/stage-7/01-multi-agent-debate/test_anthropic.py new file mode 100644 index 0000000..243cf82 --- /dev/null +++ b/examples/stage-7/01-multi-agent-debate/test_anthropic.py @@ -0,0 +1,29 @@ +"""Stage 7 練習 1 — Anthropic mock test。""" + +from __future__ import annotations + +import sys +from types import SimpleNamespace +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter_anthropic import debate_anthropic + + +def test_debate_anthropic_3_calls(): + client = MagicMock() + responses = ["pro", "con", "WINNER=CON. Better reasoning."] + client.messages.create.side_effect = [ + SimpleNamespace(content=[SimpleNamespace(type="text", text=r)]) for r in responses + ] + result = debate_anthropic("Q?", client=client) + assert client.messages.create.call_count == 3 + assert "WINNER=CON" in result["judge"] + print("✅ test_debate_anthropic_3_calls") + + +if __name__ == "__main__": + test_debate_anthropic_3_calls() + print("\n🎉 通過") diff --git a/examples/stage-7/02-eval/README.en.md b/examples/stage-7/02-eval/README.en.md new file mode 100644 index 0000000..2e602c3 --- /dev/null +++ b/examples/stage-7/02-eval/README.en.md @@ -0,0 +1,127 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# Exercise 2: Eval Pipeline ("pytest for LLMs") + +Pairs with [Stage 7 — Multi-Agent & Production](../../../stages/07-multi-agent-production.en.md) Exercise 2. + +## Task + +Write 5 eval cases for a production agent, run a baseline, track regression. **Without eval, you ship blind.** + +The 5 cases cover: +1-2. **Math** (deterministic answers) +3-4. **Geography** (factual recall) +5. **Grounding test** (fake word "flrgglemerk" — agent should say "don't know", not hallucinate) + +Two evaluators: + +| Method | When | Cost | +|---|---|---| +| **String match** | Deterministic substring expected | $0, instant | +| **LLM-as-judge** | Open-ended answers (recommendation / explanation) | One extra LLM call | + +## How to run + +### Path A (default, free, local) + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +### Path B (Anthropic) + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +Budget: 5 cases × 1 call ≈ **$0.003** (Claude haiku). + +## Validate the logic + +```bash +python test.py # 7 tests: evaluators + run_eval aggregation +python test_anthropic.py # Anthropic agent mock +``` + +## Production value of eval + +``` +Without eval: + PR merge → ship → user complains → only then you discover the regression + +With eval: + PR → run eval → pass_rate drops 95% → 70% → block merge + → find which cases regressed → fix prompt / model / retry → recover +``` + +**Pin a baseline**: capture the initial pass_rate (e.g., 80%) — every ship must not regress. + +## Classic eval shape + +```python +eval_cases = [ + {"id": ..., "input": ..., "expected_substring": ..., "instruction": ...}, + ... +] + +def run_eval(cases, agent_fn, eval_fn): + results = [...] + return {"pass_count": ..., "pass_rate": ...} +``` + +**Three keys**: +1. **`id` required** — pinpoint which case regressed +2. **`expected_substring` not full match** — LLM answers have variability +3. **Eval function decoupled from agent** — swap evaluators against the same cases + +## When to use LLM-as-judge + +| Scenario | Substring | LLM-as-judge | +|---|---|---| +| "2+2=?" | ✅ "4" | overkill | +| "summarize this article" | ❌ no fixed substring | ✅ | +| "is the tone professional?" | ❌ | ✅ | +| "count tokens used" | ✅ regex | overkill | + +**Empirical rule**: 80% of cases use substring + heuristics; 20% use LLM-as-judge (more cost / latency). + +## Production-ready tools + +- **[promptfoo](https://github.com/promptfoo/promptfoo)**: YAML config + CLI runner + diff reports +- **[Anthropic Workbench eval](https://console.anthropic.com/workbench/evals)**: official UI, prompts as code +- **[LangSmith](https://smith.langchain.com/)**: LangChain ecosystem eval + observability +- **[Weights & Biases Weave](https://wandb.ai/site/weave)**: generic LLM eval framework +- **[Braintrust](https://www.braintrust.dev/)**: cross-model / version A/B, dashboards built for production use + +## Path observations + +| Observation | Anthropic Claude | Ollama qwen2.5:3b | +|---|---|---| +| Math pass rate | ~100% | ~80% | +| Geography pass rate | ~100% | ~70-90% | +| Grounding test (flrgglemerk) | Stays grounded, says don't know | Occasionally fabricates | +| Overall pass_rate | 95-100% | 70-85% | + +**Takeaway**: production should build a 50-200-case eval set against your specific use case to decide which model. + +## Common pitfalls + +- **Eval set too small (< 10)**: noise dominates, regressions invisible +- **Eval set too close to training data**: model memorizes, real user queries fail +- **No grounding test**: production hallucination is the deadliest bug — always test "should say I don't know" +- **`expected_substring` too strict**: "The capital is Tokyo, Japan." as expected, "Tokyo" as answer = fail. Match only key tokens +- **LLM-as-judge bias**: same model as agent + judge → self-preference. Use a different model for judge + +## Extensions + +- **Track regression**: write `{"date": ..., "pass_rate": ...}` to sqlite, plot trend +- **CI integration**: GitHub Actions runs eval, `pass_rate < 90%` blocks merge +- **A/B model comparison**: same eval, run qwen / Claude / GPT, compare accuracy +- **Connect to observability (Exercise 3)**: eval failures → alert diff --git a/examples/stage-7/02-eval/README.md b/examples/stage-7/02-eval/README.md new file mode 100644 index 0000000..ef51467 --- /dev/null +++ b/examples/stage-7/02-eval/README.md @@ -0,0 +1,134 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 練習 2：Eval Pipeline（"pytest for LLMs"） + +對應 [Stage 7 — Multi-Agent & Production](../../../stages/07-multi-agent-production.md) 練習 2。 +> 🎓 **學習模式**：這份 `starter.py` 是**完整解答**、不是 TODO skeleton。建議用**主動模式**——`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫一份 `starter.py`、跑 `python test.py` 驗證；卡 20 分鐘再回去對照 reference。完整方法論看 [`docs/HOW_TO_USE.md`](../../../docs/HOW_TO_USE.md)。 + +> 📚 **想要 chapter-length 深入版？** 本 folder 的 starter 是 illustrative 版、聚焦核心 pattern + 兩條 SDK path，不是進階深度教材。深度教材推薦： +> - [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents) ⭐ 中文圈最完整、章節式 + 16 種 production 能力。**本練習對應 hello-agents 的 eval / regression testing 章節** +> - [promptfoo](https://github.com/promptfoo/promptfoo) + [Anthropic Workbench Evals](https://console.anthropic.com/workbench/evals)（官方 eval UI） +> - 完整 references 見 [Stage 7 精選 Projects](../../../stages/07-multi-agent-production.md#-精選-projects範本--sdk--工具-collection) + + +## 任務 + +為 production agent 寫 5 個 eval case、跑 baseline、追 regression。**Production 沒 eval = 沒 confidence ship**。 + +5 個 case 涵蓋： +1-2. **Math**（deterministic 答案） +3-4. **Geography**（factual recall） +5. **Grounding test**（fake word「flrgglemerk」、agent 應該說 "don't know"、不該 hallucinate） + +兩種 evaluator： + +| 方式 | 何時用 | 成本 | +|---|---|---| +| **string match** | 答案有 deterministic substring（數字、專有名詞） | $0、極快 | +| **LLM-as-judge** | 開放式答案（recommendation / explanation） | 多 1 個 LLM call | + +## 怎麼跑 + +### Path A（默認、本機免費） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +### Path B（Anthropic） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +預算：5 cases × 1 call ≈ **$0.003**（Claude haiku）。 + +## 不花錢驗證程式邏輯 + +```bash +python test.py # 7 個 test：evaluator + run_eval aggregation +python test_anthropic.py # Anthropic agent mock +``` + +## Eval 的 production 價值 + +``` +Without eval： + PR merge → 上 prod → user 抱怨 → 你才知道 LLM 行為變了 + +With eval： + PR → run eval → pass_rate 從 95% 掉到 70% → block merge + → 找出哪些 case regression → 改 prompt / model / retry → recover +``` + +**Eval pin baseline**：第一次跑、記住 pass_rate（譬如 80%）；每次 ship 前確認沒掉。 + +## 經典 eval 結構 + +```python +eval_cases = [ + {"id": ..., "input": ..., "expected_substring": ..., "instruction": ...}, + ... +] + +def run_eval(cases, agent_fn, eval_fn): + results = [...] + return {"pass_count": ..., "pass_rate": ...} +``` + +**3 個關鍵**： +1. **`id` 必要**：方便定位是哪一題 regress +2. **`expected_substring` 而非 full match**：LLM 答案有 variability、用 substring 才穩 +3. **Eval function 跟 agent 分離**：可以換不同 evaluator 對同一份 cases + +## LLM-as-judge 何時用 + +| 情境 | 用 substring | 用 LLM-as-judge | +|---|---|---| +| 「2+2=?」答案 | ✅ "4" | overkill | +| 「summarize this article」 | ❌ 沒固定 substring | ✅ | +| 「is the tone professional?」 | ❌ | ✅ | +| 「count tokens used」 | ✅ 用 regex | overkill | + +**Production 經驗**：80% case 用 substring + heuristic、20% 用 LLM-as-judge（cost / latency 較高）。 + +## Production-ready tools + +- **[promptfoo](https://github.com/promptfoo/promptfoo)**：YAML config + CLI runner + diff report +- **[Anthropic Workbench eval](https://console.anthropic.com/workbench/evals)**：官方 eval UI、prompts as code +- **[LangSmith](https://smith.langchain.com/)**：LangChain ecosystem 的 eval + observability 一條龍 +- **[Weights & Biases Weave](https://wandb.ai/site/weave)**：generic LLM eval framework +- **[Braintrust](https://www.braintrust.dev/)**：跨 model / version A/B、上線部署用的 dashboards + +## 兩個 path 觀察重點 + +| 觀察項 | Anthropic Claude | Ollama qwen2.5:3b | +|---|---|---| +| math case pass rate | ~100% | ~80% | +| geo case pass rate | ~100% | ~70-90% | +| grounding test (flrgglemerk) | 守規則說 don't know | 偶爾編答案 | +| 整體 pass_rate | 95-100% | 70-85% | + +**結論**：production 應該針對「自家 use case」建 50-200 case 的 eval set、看自家任務上的 pass rate、決定該用哪個 model。 + +## 常見坑 + +- **Eval set 太小（< 10 cases）**：noise 高、看不出 regression +- **Eval set 太靠近 training data**：model 死背、實際 user query 表現完全不同 +- **沒 grounding test**：production agent hallucination 是最致命的 bug、必須有「答不出來該說 unknown」case +- **`expected_substring` 太嚴**：寫「The capital is Tokyo, Japan.」當 expected、LLM 寫「Tokyo」會 fail。應該只匹配關鍵字 +- **LLM-as-judge bias**：用同一個 model 既當 agent 又當 judge、容易 self-preferenced。Production 用不同 model 當 judge + +## 延伸 + +- **加 regression 追蹤**：每次 ship 把 `{"date": ..., "pass_rate": ...}` 存進 sqlite、畫趨勢圖 +- **CI 整合**：GitHub Actions 自動跑 eval、`pass_rate < 90%` 就 block merge +- **A/B model 對照**：同一份 eval、跑 qwen vs Claude vs GPT、看誰準 +- **接觀察 (練習 3)**：eval failure 串到 observability、自動 alert diff --git a/examples/stage-7/02-eval/README.zh-Hans.md b/examples/stage-7/02-eval/README.zh-Hans.md new file mode 100644 index 0000000..ec6d192 --- /dev/null +++ b/examples/stage-7/02-eval/README.zh-Hans.md @@ -0,0 +1,127 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 练习 2：Eval Pipeline（"pytest for LLMs"） + +对应 [Stage 7 — Multi-Agent & Production](../../../stages/07-multi-agent-production.zh-Hans.md) 练习 2。 + +## 任务 + +为 production agent 写 5 个 eval case、跑 baseline、追 regression。**Production 没 eval = 没 confidence ship**。 + +5 个 case 涵盖： +1-2. **Math**（deterministic 答案） +3-4. **Geography**（factual recall） +5. **Grounding test**（fake word“flrgglemerk”、agent 应该说 "don't know"、不该 hallucinate） + +两种 evaluator： + +| 方式 | 何时用 | 成本 | +|---|---|---| +| **string match** | 答案有 deterministic substring（数字、专有名词） | $0、极快 | +| **LLM-as-judge** | 开放式答案（recommendation / explanation） | 多 1 个 LLM call | + +## 怎么跑 + +### Path A（默认、本机免费） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +### Path B（Anthropic） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +预算：5 cases × 1 call ≈ **$0.003**（Claude haiku）。 + +## 不花钱验证程序逻辑 + +```bash +python test.py # 7 个 test：evaluator + run_eval aggregation +python test_anthropic.py # Anthropic agent mock +``` + +## Eval 的 production 价值 + +``` +Without eval： + PR merge → 上 prod → user 抱怨 → 你才知道 LLM 行为变了 + +With eval： + PR → run eval → pass_rate 从 95% 掉到 70% → block merge + → 找出哪些 case regression → 改 prompt / model / retry → recover +``` + +**Eval pin baseline**：第一次跑、记住 pass_rate（譬如 80%）；每次 ship 前确认没掉。 + +## 经典 eval 结构 + +```python +eval_cases = [ + {"id": ..., "input": ..., "expected_substring": ..., "instruction": ...}, + ... +] + +def run_eval(cases, agent_fn, eval_fn): + results = [...] + return {"pass_count": ..., "pass_rate": ...} +``` + +**3 个关键**： +1. **`id` 必要**：方便定位是哪一题 regress +2. **`expected_substring` 而非 full match**：LLM 答案有 variability、用 substring 才稳 +3. **Eval function 跟 agent 分离**：可以换不同 evaluator 对同一份 cases + +## LLM-as-judge 何时用 + +| 情境 | 用 substring | 用 LLM-as-judge | +|---|---|---| +| “2+2=?”答案 | ✅ "4" | overkill | +| “summarize this article” | ❌ 没固定 substring | ✅ | +| “is the tone professional?” | ❌ | ✅ | +| “count tokens used” | ✅ 用 regex | overkill | + +**Production 经验**：80% case 用 substring + heuristic、20% 用 LLM-as-judge（cost / latency 较高）。 + +## Production-ready tools + +- **[promptfoo](https://github.com/promptfoo/promptfoo)**：YAML config + CLI runner + diff report +- **[Anthropic Workbench eval](https://console.anthropic.com/workbench/evals)**：官方 eval UI、prompts as code +- **[LangSmith](https://smith.langchain.com/)**：LangChain ecosystem 的 eval + observability 一条龙 +- **[Weights & Biases Weave](https://wandb.ai/site/weave)**：generic LLM eval framework +- **[Braintrust](https://www.braintrust.dev/)**：跨 model / version A/B、上线部署用的 dashboards + +## 两个 path 观察重点 + +| 观察项 | Anthropic Claude | Ollama qwen2.5:3b | +|---|---|---| +| math case pass rate | ~100% | ~80% | +| geo case pass rate | ~100% | ~70-90% | +| grounding test (flrgglemerk) | 守规则说 don't know | 偶尔编答案 | +| 整体 pass_rate | 95-100% | 70-85% | + +**结论**：production 应该针对“自家 use case”建 50-200 case 的 eval set、看自家任务上的 pass rate、决定该用哪个 model。 + +## 常见坑 + +- **Eval set 太小（< 10 cases）**：noise 高、看不出 regression +- **Eval set 太靠近 training data**：model 死背、实际 user query 表现完全不同 +- **没 grounding test**：production agent hallucination 是最致命的 bug、必须有“答不出来该说 unknown”case +- **`expected_substring` 太严**：写“The capital is Tokyo, Japan.”当 expected、LLM 写“Tokyo”会 fail。应该只匹配关键字 +- **LLM-as-judge bias**：用同一个 model 既当 agent 又当 judge、容易 self-preferenced。Production 用不同 model 当 judge + +## 延伸 + +- **加 regression 追踪**：每次 ship 把 `{"date": ..., "pass_rate": ...}` 存进 sqlite、画趋势图 +- **CI 整合**：GitHub Actions 自动跑 eval、`pass_rate < 90%` 就 block merge +- **A/B model 对照**：同一份 eval、跑 qwen vs Claude vs GPT、看谁准 +- **接观察 (练习 3)**：eval failure 串到 observability、自动 alert diff --git a/examples/stage-7/02-eval/requirements.txt b/examples/stage-7/02-eval/requirements.txt new file mode 100644 index 0000000..7c268c1 --- /dev/null +++ b/examples/stage-7/02-eval/requirements.txt @@ -0,0 +1,2 @@ +openai>=1.50,<2.0 +anthropic>=0.40,<1.0 diff --git a/examples/stage-7/02-eval/starter.py b/examples/stage-7/02-eval/starter.py new file mode 100644 index 0000000..178e93c --- /dev/null +++ b/examples/stage-7/02-eval/starter.py @@ -0,0 +1,106 @@ +"""Stage 7 練習 2：Eval — Path A（Ollama 默認、$0）。 + +Eval pipeline = 「pytest for LLMs」。為 production agent 寫 5-10 個 eval case、跑 baseline、 +追蹤 regression。Production 沒 eval = 沒 confidence ship。 + +兩種 evaluator： +1. **String match**：output 含特定 keyword（簡單、無 LLM cost） +2. **LLM-as-judge**：用 LLM 評分（複雜但 flexible） + +跑法： + pip install -r requirements.txt + ollama pull qwen2.5:3b + ollama serve + python starter.py +""" + +from __future__ import annotations + +import os +import sys +from typing import Any, Callable + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +MODEL = os.environ.get("MODEL", "qwen2.5:3b") +OLLAMA_BASE = os.environ.get("OLLAMA_API_BASE", "http://localhost:11434/v1") + + +# === Eval cases === + +EVAL_CASES = [ + {"id": "math_1", "input": "What is 2 + 2?", "expected_substring": "4"}, + {"id": "math_2", "input": "What is 10 * 5?", "expected_substring": "50"}, + {"id": "geo_1", "input": "What is the capital of Japan?", "expected_substring": "Tokyo"}, + {"id": "geo_2", "input": "What is the capital of France?", "expected_substring": "Paris"}, + {"id": "ground_1", "input": "What is 'flrgglemerk'?", + "expected_substring": "don't know", "instruction": "If you don't know, say so."}, +] + + +# === Agent under test === + +def agent_answer(question: str, llm: Any = None, instruction: str = "") -> str: + llm = llm or OpenAI(base_url=OLLAMA_BASE, api_key="ollama") + system = "Answer concisely (1-2 sentences). " + instruction + resp = llm.chat.completions.create( + model=MODEL, + messages=[{"role": "system", "content": system}, {"role": "user", "content": question}], + ) + return resp.choices[0].message.content or "" + + +# === Evaluators === + +def eval_substring(output: str, case: dict) -> bool: + return case["expected_substring"].lower() in output.lower() + + +def eval_llm_as_judge(output: str, case: dict, judge_llm: Any = None) -> bool: + """Use an LLM to judge whether output answers the question correctly.""" + judge_llm = judge_llm or OpenAI(base_url=OLLAMA_BASE, api_key="ollama") + prompt = f"""Given a user question and an AI's answer, decide if the answer is correct. Reply with ONLY 'PASS' or 'FAIL'. + +Question: {case['input']} +Expected to contain: {case['expected_substring']} +AI Answer: {output} + +Verdict:""" + resp = judge_llm.chat.completions.create( + model=MODEL, + messages=[{"role": "user", "content": prompt}], + ) + verdict = (resp.choices[0].message.content or "").upper() + return "PASS" in verdict and "FAIL" not in verdict.replace("FAIL?", "") + + +# === Eval runner === + +def run_eval(cases: list[dict], agent_fn: Callable, eval_fn: Callable, **agent_kwargs) -> dict: + results = [] + for case in cases: + instruction = case.get("instruction", "") + output = agent_fn(case["input"], instruction=instruction, **agent_kwargs) + passed = eval_fn(output, case) + results.append({"id": case["id"], "passed": passed, "output": output[:80]}) + passes = sum(1 for r in results if r["passed"]) + return {"results": results, "pass_count": passes, "total": len(results), + "pass_rate": passes / len(results)} + + +if __name__ == "__main__": + print(f"Running eval on {len(EVAL_CASES)} cases (using {MODEL})...\n") + + print("=== Evaluator: string match ===") + out = run_eval(EVAL_CASES, agent_answer, eval_substring) + for r in out["results"]: + mark = "✅" if r["passed"] else "❌" + print(f" {mark} [{r['id']}] {r['output']}") + print(f" Pass: {out['pass_count']}/{out['total']} ({out['pass_rate']:.0%})") + + assert out["total"] == 5 + print(f"\n✅ 練習 2 通過 — eval pipeline 跑通、$0/run") + print(f" 觀察：production 應該 pin baseline pass rate、每次 ship 前確認沒 regression") diff --git a/examples/stage-7/02-eval/starter_anthropic.py b/examples/stage-7/02-eval/starter_anthropic.py new file mode 100644 index 0000000..09b4099 --- /dev/null +++ b/examples/stage-7/02-eval/starter_anthropic.py @@ -0,0 +1,43 @@ +"""Stage 7 練習 2：Eval — Path B（Claude）。 + +跟 starter.py 同流程、agent + judge 都用 Claude。 + +跑法： + pip install -r requirements.txt + export ANTHROPIC_API_KEY=sk-ant-... + python starter_anthropic.py +""" + +from __future__ import annotations + +import os +import sys +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic + +from starter import EVAL_CASES, eval_substring, run_eval + +MODEL = os.environ.get("MODEL", "claude-haiku-4-5") + + +def agent_answer_anthropic(question: str, instruction: str = "", client: Any = None) -> str: + client = client or anthropic.Anthropic() + system = "Answer concisely (1-2 sentences). " + instruction + resp = client.messages.create( + model=MODEL, max_tokens=200, system=system, + messages=[{"role": "user", "content": question}], + ) + return " ".join(b.text for b in resp.content if b.type == "text") + + +if __name__ == "__main__": + out = run_eval(EVAL_CASES, agent_answer_anthropic, eval_substring) + for r in out["results"]: + mark = "✅" if r["passed"] else "❌" + print(f" {mark} [{r['id']}] {r['output']}") + print(f"\nPass: {out['pass_count']}/{out['total']} ({out['pass_rate']:.0%})") + print(f"✅ 練習 2 (Anthropic) 通過 — {MODEL}、5 cases × ≈$0.0005 = ≈$0.003/run") diff --git a/examples/stage-7/02-eval/test.py b/examples/stage-7/02-eval/test.py new file mode 100644 index 0000000..c48fa50 --- /dev/null +++ b/examples/stage-7/02-eval/test.py @@ -0,0 +1,91 @@ +"""Stage 7 練習 2 自我驗證 — eval pipeline + 兩種 evaluator。""" + +from __future__ import annotations + +import sys +from types import SimpleNamespace +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter import EVAL_CASES, eval_llm_as_judge, eval_substring, run_eval + + +def test_eval_substring_pass(): + case = {"id": "x", "input": "?", "expected_substring": "Tokyo"} + assert eval_substring("The capital is Tokyo.", case) is True + print("✅ test_eval_substring_pass") + + +def test_eval_substring_fail(): + case = {"id": "x", "input": "?", "expected_substring": "Tokyo"} + assert eval_substring("The capital is Beijing.", case) is False + print("✅ test_eval_substring_fail") + + +def test_eval_substring_case_insensitive(): + case = {"id": "x", "input": "?", "expected_substring": "tokyo"} + assert eval_substring("The capital is TOKYO.", case) is True + print("✅ test_eval_substring_case_insensitive") + + +def test_eval_llm_as_judge_pass(): + """Mock judge replies PASS。""" + judge = MagicMock() + judge.chat.completions.create.return_value = SimpleNamespace( + choices=[SimpleNamespace(message=SimpleNamespace(content="PASS"))] + ) + case = {"id": "x", "input": "What's the capital?", "expected_substring": "Tokyo"} + assert eval_llm_as_judge("Tokyo!", case, judge_llm=judge) is True + print("✅ test_eval_llm_as_judge_pass") + + +def test_eval_llm_as_judge_fail(): + judge = MagicMock() + judge.chat.completions.create.return_value = SimpleNamespace( + choices=[SimpleNamespace(message=SimpleNamespace(content="FAIL"))] + ) + case = {"id": "x", "input": "?", "expected_substring": "Tokyo"} + assert eval_llm_as_judge("Beijing", case, judge_llm=judge) is False + print("✅ test_eval_llm_as_judge_fail") + + +def test_run_eval_aggregates_correctly(): + """Mock agent 答對 4/5、ground_1（fake word）故意 hallucinate fail、驗 aggregation。""" + def fake_agent(question, instruction="", **kw): + if "2 + 2" in question: + return "The answer is 4." + if "10 * 5" in question: + return "50" + if "Japan" in question: + return "Tokyo" + if "France" in question: + return "Paris" + # ground_1: fake agent hallucinates instead of saying "don't know" — fail + return "I made something up" + + out = run_eval(EVAL_CASES, fake_agent, eval_substring) + assert out["total"] == 5 + assert out["pass_count"] == 4, f"預期 4 pass（4 個有 substring）+ 1 fail（ground_1）、得到 {out['pass_count']}" + failed = [r for r in out["results"] if not r["passed"]] + assert len(failed) == 1 and failed[0]["id"] == "ground_1" + print("✅ test_run_eval_aggregates_correctly") + + +def test_eval_cases_corpus_shape(): + assert len(EVAL_CASES) == 5 + for c in EVAL_CASES: + assert {"id", "input", "expected_substring"} <= c.keys() + print("✅ test_eval_cases_corpus_shape") + + +if __name__ == "__main__": + test_eval_substring_pass() + test_eval_substring_fail() + test_eval_substring_case_insensitive() + test_eval_llm_as_judge_pass() + test_eval_llm_as_judge_fail() + test_run_eval_aggregates_correctly() + test_eval_cases_corpus_shape() + print("\n🎉 全部通過 — eval pipeline 邏輯正確") diff --git a/examples/stage-7/02-eval/test_anthropic.py b/examples/stage-7/02-eval/test_anthropic.py new file mode 100644 index 0000000..bf13505 --- /dev/null +++ b/examples/stage-7/02-eval/test_anthropic.py @@ -0,0 +1,27 @@ +"""Stage 7 練習 2 — Path B Anthropic mock test。""" + +from __future__ import annotations + +import sys +from types import SimpleNamespace +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter_anthropic import agent_answer_anthropic + + +def test_agent_answer_anthropic_mock(): + client = MagicMock() + client.messages.create.return_value = SimpleNamespace( + content=[SimpleNamespace(type="text", text="Tokyo")] + ) + out = agent_answer_anthropic("Capital of Japan?", client=client) + assert "Tokyo" in out + print("✅ test_agent_answer_anthropic_mock") + + +if __name__ == "__main__": + test_agent_answer_anthropic_mock() + print("\n🎉 通過") diff --git a/examples/stage-7/03-observability/README.en.md b/examples/stage-7/03-observability/README.en.md new file mode 100644 index 0000000..fb8af33 --- /dev/null +++ b/examples/stage-7/03-observability/README.en.md @@ -0,0 +1,134 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# Exercise 3: Observability (4 production telemetry primitives) + +Pairs with [Stage 7 — Multi-Agent & Production](../../../stages/07-multi-agent-production.en.md) Exercise 3. + +## Task + +Production agents need 4 telemetry primitives: + +1. **Latency** — per-step timing (p50/p95/p99) +2. **Token usage** — input/output (cost tracking) +3. **Trace** — every step of a multi-step agent (debug + audit) +4. **Errors** — exceptions + retry count + +Implementation: `TraceContext` + `trace_span` context manager wrapping LLM calls. + +## How to run + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +Budget: **$0** (Path A). Path B with Claude: ~$0.0001/run. + +```bash +python test.py # 5 tests +python test_anthropic.py +``` + +## The 4 primitives + +### Latency (via contextmanager) + +```python +@contextmanager +def trace_span(ctx, name, **extras): + t0 = time.perf_counter() + try: + yield + finally: + latency_ms = (time.perf_counter() - t0) * 1000 + ctx.add_span(name, latency_ms, **extras) + +with trace_span(ctx, "search_step"): + result = expensive_search(query) +``` + +### Token usage + +```python +resp = client.messages.create(...) +ctx.add_tokens(input_t=resp.usage.input_tokens, output_t=resp.usage.output_tokens) +``` + +- **Anthropic**: `usage.input_tokens` / `usage.output_tokens` are precise +- **OpenAI/Ollama**: `usage.prompt_tokens` / `usage.completion_tokens`; Ollama occasionally omits usage + +### Trace + +```python +ctx = TraceContext("req_42") +with trace_span(ctx, "search"): ... +with trace_span(ctx, "llm_call"): ... +print(ctx.summary()) # full request timeline +``` + +### Errors + +```python +@contextmanager +def trace_span(ctx, name): + try: + yield + except Exception as e: + ctx.add_error(f"{name}: {e}") + raise # critical: re-raise, don't swallow +``` + +## Production tools (don't roll your own) + +These primitives are for learning. In production use OpenTelemetry + a managed platform: + +- **[Langfuse](https://langfuse.com/)** — open source, self-hostable, tracing + eval + prompt management +- **[LangSmith](https://smith.langchain.com/)** — LangChain ecosystem +- **[Helicone](https://www.helicone.ai/)** — proxy mode, zero code change +- **[Arize Phoenix](https://github.com/Arize-ai/phoenix)** — open source, OpenTelemetry-native +- **[Datadog LLM Observability](https://www.datadoghq.com/product/llm-observability/)** — integrates with APM +- **[Anthropic API Console](https://console.anthropic.com/)** — built-in Claude cost dashboard + +## Production checklist + +For every production agent you must be able to answer: + +``` +[ ] What's the p50 / p95 / p99 latency? +[ ] Average tokens per request? ($) +[ ] Which step is slowest? +[ ] Error rate? Most common error? +[ ] Retry success rate? +[ ] Cost/request trend (monthly)? +[ ] Which queries get wrong answers? (connects to eval, Exercise 2) +``` + +Can't answer = no observability. + +## Path observations + +| Observation | Anthropic Claude | Ollama qwen2.5:3b | +|---|---|---| +| `usage.tokens` precision | ✅ Complete (incl. cache_*) | ⚠ Sometimes missing | +| Cost tracking | Direct: tokens × pricing | $0 but GPU time has cost | +| Latency source | Network + queue + inference | Pure inference | +| Production observation | Anthropic console | Self-host prometheus/grafana | + +## Common pitfalls + +- **No token tracking**: a month into production you can't forecast cost +- **Spans too coarse**: only logging "agent_call" hides the bottleneck (search vs rerank vs generate) +- **Swallowed errors**: context manager eats exception, caller thinks success +- **Production using `print()`**: use structured logging (JSON / OpenTelemetry), ship to cloud +- **No sampling**: high QPS = trace backend overwhelmed; sample (e.g., 10% of traces, 100% of errors) + +## Extensions + +- **OpenTelemetry**: replace `trace_span` with `tracer.start_as_current_span(...)` — ship to Jaeger / Datadog +- **Langfuse SDK**: 3-line integration with Anthropic Claude, automatic tracing +- **Prometheus metrics**: counter (request_count), histogram (latency), gauge (active_sessions) +- **Wire to eval (Exercise 2)**: eval failures auto-alert to Slack diff --git a/examples/stage-7/03-observability/README.md b/examples/stage-7/03-observability/README.md new file mode 100644 index 0000000..f3b9bcc --- /dev/null +++ b/examples/stage-7/03-observability/README.md @@ -0,0 +1,144 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 練習 3：Observability（4 個 production telemetry） + +對應 [Stage 7 — Multi-Agent & Production](../../../stages/07-multi-agent-production.md) 練習 3。 +> 🎓 **學習模式**：這份 `starter.py` 是**完整解答**、不是 TODO skeleton。建議用**主動模式**——`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫一份 `starter.py`、跑 `python test.py` 驗證；卡 20 分鐘再回去對照 reference。完整方法論看 [`docs/HOW_TO_USE.md`](../../../docs/HOW_TO_USE.md)。 + +> 📚 **想要 chapter-length 深入版？** 本 folder 的 starter 是 illustrative 版、聚焦核心 pattern + 兩條 SDK path，不是進階深度教材。深度教材推薦： +> - [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents) ⭐ 中文圈最完整、章節式 + 16 種 production 能力。**本練習對應 hello-agents 的 observability / tracing 章節（Extra Chapter）** +> - [Langfuse](https://github.com/langfuse/langfuse) + [Arize Phoenix](https://github.com/Arize-ai/phoenix)（OpenTelemetry-native） +> - 完整 references 見 [Stage 7 精選 Projects](../../../stages/07-multi-agent-production.md#-精選-projects範本--sdk--工具-collection) + + +## 任務 + +Production agent 必備 4 個 telemetry： + +1. **Latency**：每個 step 多久（p50/p95/p99） +2. **Token usage**：input / output（追 cost） +3. **Trace**：multi-step agent 每一步（debug + audit） +4. **Errors**：exception + retry count + +實作：`TraceContext` + `trace_span` context manager + 在 LLM call 之間 instrument。 + +## 怎麼跑 + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +預算：**$0**（Path A）。Path B 用 Claude：~$0.0001/run。 + +```bash +python test.py # 5 個 test +python test_anthropic.py +``` + +## 4 個 primitive + +### Latency（用 contextmanager） + +```python +@contextmanager +def trace_span(ctx, name, **extras): + t0 = time.perf_counter() + try: + yield + finally: + latency_ms = (time.perf_counter() - t0) * 1000 + ctx.add_span(name, latency_ms, **extras) + +# usage: +with trace_span(ctx, "search_step"): + result = expensive_search(query) +``` + +### Token usage + +```python +resp = client.messages.create(...) +ctx.add_tokens(input_t=resp.usage.input_tokens, output_t=resp.usage.output_tokens) +``` + +**Anthropic**：`usage.input_tokens` / `usage.output_tokens` 精確 +**OpenAI / Ollama**：`usage.prompt_tokens` / `usage.completion_tokens`、Ollama 偶爾不返回 usage + +### Trace + +```python +ctx = TraceContext("req_42") +with trace_span(ctx, "search"): + ... +with trace_span(ctx, "llm_call"): + ... +print(ctx.summary()) # 看整個 request 的 timeline +``` + +### Errors + +```python +@contextmanager +def trace_span(ctx, name): + try: + yield + except Exception as e: + ctx.add_error(f"{name}: {e}") + raise # ← 重要：raise 出去、不要吞 exception +``` + +## Production tools（不要自己寫） + +實作 primitive 是學原理。Production 用 OpenTelemetry + 託管平台： + +- **[Langfuse](https://langfuse.com/)**：open-source、self-host、tracing + eval + prompt management 一條龍 +- **[LangSmith](https://smith.langchain.com/)**：LangChain ecosystem +- **[Helicone](https://www.helicone.ai/)**：proxy mode、零 code change +- **[Arize Phoenix](https://github.com/Arize-ai/phoenix)**：open-source、OpenTelemetry-native +- **[Datadog LLM Observability](https://www.datadoghq.com/product/llm-observability/)**：integrate with general APM +- **[Anthropic API Console](https://console.anthropic.com/)**：Claude usage / cost 內建 dashboard + +## Production checklist + +對每個 production agent 至少要回答： + +``` +[ ] p50 / p95 / p99 latency 多少？ +[ ] 每 request 平均花多少 token？($) +[ ] 哪幾個 step 最慢？ +[ ] 錯誤率多少？哪一類錯最多？ +[ ] retry 後 success rate 多少？ +[ ] cost / request 趨勢（每月）？ +[ ] 哪些 query 答錯？(連到 eval、練習 2) +``` + +回答不出來 = 沒 observability。 + +## 兩個 path 觀察重點 + +| 觀察項 | Anthropic Claude | Ollama qwen2.5:3b | +|---|---|---| +| `usage.tokens` 精確度 | ✅ 完整（含 cache_*） | ⚠ 偶爾沒返回 | +| Cost tracking | 直接 cost = token × pricing | $0、但 GPU 時間有成本 | +| Latency 來源 | Network + queue + inference | 純 inference | +| Production deploy 觀察 | 看 Anthropic console | 自己跑 prometheus / grafana | + +## 常見坑 + +- **Token usage 沒記**：上線一個月才發現 cost 漏記、無法 forecast +- **Span 不夠細**：只記 "agent_call" 整體、沒拆 "search" / "rerank" / "generate"、debug 時看不到瓶頸 +- **Error swallowed**：context manager 吃掉 exception、上層以為成功 +- **production 直接 print()**：應該用 structured logging（JSON / OpenTelemetry）、寫去 cloud +- **沒 sample 機制**：高 QPS 全量 trace 會塞爆 backend、要 sampling（譬如 10% trace + 100% error） + +## 延伸 + +- **OpenTelemetry 整合**：把 `trace_span` 改成 `tracer.start_as_current_span(...)` 就能丟去 Jaeger / Datadog +- **Langfuse SDK**：3 行接上 Anthropic Claude、自動 trace +- **Prometheus metrics**：counter（request_count）、histogram（latency）、gauge（active_sessions） +- **接 eval（練習 2）**：eval failure 自動 alert 到 Slack diff --git a/examples/stage-7/03-observability/README.zh-Hans.md b/examples/stage-7/03-observability/README.zh-Hans.md new file mode 100644 index 0000000..2ff192a --- /dev/null +++ b/examples/stage-7/03-observability/README.zh-Hans.md @@ -0,0 +1,134 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 练习 3：Observability（4 个 production telemetry） + +对应 [Stage 7 — Multi-Agent & Production](../../../stages/07-multi-agent-production.zh-Hans.md) 练习 3。 + +## 任务 + +Production agent 必备 4 个 telemetry： + +1. **Latency**：每个 step 多久（p50/p95/p99） +2. **Token usage**：input / output（追 cost） +3. **Trace**：multi-step agent 每一步（debug + audit） +4. **Errors**：exception + retry count + +实作：`TraceContext` + `trace_span` context manager + 在 LLM call 之间 instrument。 + +## 怎么跑 + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +预算：**$0**（Path A）。Path B 用 Claude：~$0.0001/run。 + +```bash +python test.py # 5 个 test +python test_anthropic.py +``` + +## 4 个 primitive + +### Latency（用 contextmanager） + +```python +@contextmanager +def trace_span(ctx, name, **extras): + t0 = time.perf_counter() + try: + yield + finally: + latency_ms = (time.perf_counter() - t0) * 1000 + ctx.add_span(name, latency_ms, **extras) + +with trace_span(ctx, "search_step"): + result = expensive_search(query) +``` + +### Token usage + +```python +resp = client.messages.create(...) +ctx.add_tokens(input_t=resp.usage.input_tokens, output_t=resp.usage.output_tokens) +``` + +- **Anthropic**：`usage.input_tokens` / `usage.output_tokens` 精确 +- **OpenAI/Ollama**：`usage.prompt_tokens` / `usage.completion_tokens`、Ollama 偶尔不返回 usage + +### Trace + +```python +ctx = TraceContext("req_42") +with trace_span(ctx, "search"): ... +with trace_span(ctx, "llm_call"): ... +print(ctx.summary()) # 看整个 request 的 timeline +``` + +### Errors + +```python +@contextmanager +def trace_span(ctx, name): + try: + yield + except Exception as e: + ctx.add_error(f"{name}: {e}") + raise # 重要：raise 出去、不要吞 exception +``` + +## Production tools（不要自己写） + +实作 primitive 是学原理。Production 用 OpenTelemetry + 托管平台： + +- **[Langfuse](https://langfuse.com/)**：open-source、self-host、tracing + eval + prompt management 一条龙 +- **[LangSmith](https://smith.langchain.com/)**：LangChain ecosystem +- **[Helicone](https://www.helicone.ai/)**：proxy mode、零 code change +- **[Arize Phoenix](https://github.com/Arize-ai/phoenix)**：open-source、OpenTelemetry-native +- **[Datadog LLM Observability](https://www.datadoghq.com/product/llm-observability/)**：integrate with general APM +- **[Anthropic API Console](https://console.anthropic.com/)**：Claude usage / cost 内建 dashboard + +## Production checklist + +对每个 production agent 至少要回答： + +``` +[ ] p50 / p95 / p99 latency 多少？ +[ ] 每 request 平均花多少 token？($) +[ ] 哪几个 step 最慢？ +[ ] 错误率多少？哪一类错最多？ +[ ] retry 后 success rate 多少？ +[ ] cost / request 趋势（每月）？ +[ ] 哪些 query 答错？(连到 eval、练习 2) +``` + +回答不出来 = 没 observability。 + +## 两个 path 观察重点 + +| 观察项 | Anthropic Claude | Ollama qwen2.5:3b | +|---|---|---| +| `usage.tokens` 精确度 | ✅ 完整（含 cache_*） | ⚠ 偶尔没返回 | +| Cost tracking | 直接 cost = token × pricing | $0、但 GPU 时间有成本 | +| Latency 来源 | Network + queue + inference | 纯 inference | +| Production deploy 观察 | 看 Anthropic console | 自己跑 prometheus / grafana | + +## 常见坑 + +- **Token usage 没记**：上线一个月才发现 cost 漏记、无法 forecast +- **Span 不够细**：只记 "agent_call" 整体、没拆 "search" / "rerank" / "generate"、debug 时看不到瓶颈 +- **Error swallowed**：context manager 吃掉 exception、上层以为成功 +- **production 直接 print()**：应该用 structured logging（JSON / OpenTelemetry）、写去 cloud +- **没 sample 机制**：高 QPS 全量 trace 会塞爆 backend、要 sampling（譬如 10% trace + 100% error） + +## 延伸 + +- **OpenTelemetry 整合**：把 `trace_span` 改成 `tracer.start_as_current_span(...)` 就能丢去 Jaeger / Datadog +- **Langfuse SDK**：3 行接上 Anthropic Claude、自动 trace +- **Prometheus metrics**：counter（request_count）、histogram（latency）、gauge（active_sessions） +- **接 eval（练习 2）**：eval failure 自动 alert 到 Slack diff --git a/examples/stage-7/03-observability/requirements.txt b/examples/stage-7/03-observability/requirements.txt new file mode 100644 index 0000000..7c268c1 --- /dev/null +++ b/examples/stage-7/03-observability/requirements.txt @@ -0,0 +1,2 @@ +openai>=1.50,<2.0 +anthropic>=0.40,<1.0 diff --git a/examples/stage-7/03-observability/starter.py b/examples/stage-7/03-observability/starter.py new file mode 100644 index 0000000..4646c18 --- /dev/null +++ b/examples/stage-7/03-observability/starter.py @@ -0,0 +1,145 @@ +"""Stage 7 練習 3：Observability — Path A（Ollama 默認、$0）。 + +Production agent 必備 4 個 telemetry： +1. **Latency**：每個 LLM call 多久（追 p50/p95/p99） +2. **Token usage**：input / output / total tokens（追 cost） +3. **Trace**：multi-step agent 的每一步（debug + audit） +4. **Errors**：捕 exception + retry count + +跑法： + pip install -r requirements.txt + ollama pull qwen2.5:3b + ollama serve + python starter.py +""" + +from __future__ import annotations + +import json +import logging +import os +import sys +import time +from contextlib import contextmanager +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +MODEL = os.environ.get("MODEL", "qwen2.5:3b") +OLLAMA_BASE = os.environ.get("OLLAMA_API_BASE", "http://localhost:11434/v1") + +# Structured logger — production 應該寫去 cloud (Datadog / CloudWatch / Loki) +logging.basicConfig( + level=logging.INFO, + format='%(asctime)s [%(levelname)s] %(message)s', +) +logger = logging.getLogger("agent.observability") + + +# === 4 個 telemetry primitives === + +class TraceContext: + """Per-request trace context. Production 通常用 OpenTelemetry。""" + + def __init__(self, request_id: str): + self.request_id = request_id + self.spans: list[dict] = [] + self.total_tokens = {"input": 0, "output": 0} + self.errors: list[str] = [] + + def add_span(self, name: str, latency_ms: float, **extras): + span = {"name": name, "latency_ms": latency_ms, **extras} + self.spans.append(span) + logger.info(f"[{self.request_id}] span={name} ms={latency_ms:.1f} {json.dumps(extras)}") + + def add_tokens(self, input_t: int, output_t: int): + self.total_tokens["input"] += input_t + self.total_tokens["output"] += output_t + + def add_error(self, msg: str): + self.errors.append(msg) + logger.error(f"[{self.request_id}] error={msg}") + + def summary(self) -> dict: + total_ms = sum(s["latency_ms"] for s in self.spans) + return { + "request_id": self.request_id, + "total_latency_ms": total_ms, + "span_count": len(self.spans), + "input_tokens": self.total_tokens["input"], + "output_tokens": self.total_tokens["output"], + "error_count": len(self.errors), + } + + +@contextmanager +def trace_span(ctx: TraceContext, name: str, **extras): + """Context manager 計時 + 記 span。""" + t0 = time.perf_counter() + err = None + try: + yield + except Exception as e: + err = str(e) + ctx.add_error(f"{name}: {err}") + raise + finally: + latency_ms = (time.perf_counter() - t0) * 1000 + ctx.add_span(name, latency_ms, error=err, **extras) + + +# === Instrumented agent === + +def observable_agent(question: str, ctx: TraceContext, llm: Any = None) -> str: + """LLM call wrapped in trace span + token logging。""" + llm = llm or OpenAI(base_url=OLLAMA_BASE, api_key="ollama") + + with trace_span(ctx, "llm_call", model=MODEL): + resp = llm.chat.completions.create( + model=MODEL, + messages=[{"role": "user", "content": question}], + ) + + # Token usage（OpenAI / Ollama 都有 usage 欄位、不一定每個 Ollama 版本支援） + usage = getattr(resp, "usage", None) + if usage: + ctx.add_tokens( + input_t=getattr(usage, "prompt_tokens", 0), + output_t=getattr(usage, "completion_tokens", 0), + ) + + return resp.choices[0].message.content or "" + + +def multi_step_agent(question: str, llm: Any = None) -> dict: + """模擬 multi-step agent：search → reason → answer、每步 traced。""" + ctx = TraceContext(request_id=f"req_{int(time.time()*1000)}") + + with trace_span(ctx, "search"): + time.sleep(0.05) # simulate tool call latency + search_result = "Fake search result for: " + question + + answer = observable_agent(f"Based on '{search_result}', answer: {question}", ctx, llm=llm) + + return { + "answer": answer, + "trace_summary": ctx.summary(), + "spans": ctx.spans, + } + + +if __name__ == "__main__": + print("Running instrumented agent...\n") + result = multi_step_agent("What's 2+2?") + print(f"\n📊 Trace summary:") + for k, v in result["trace_summary"].items(): + print(f" {k}: {v}") + print(f"\n📝 Spans:") + for s in result["spans"]: + print(f" {s['name']}: {s['latency_ms']:.1f}ms") + + assert result["trace_summary"]["span_count"] >= 2 # search + llm_call + print(f"\n✅ 練習 3 通過 — 觀察 4 個 telemetry primitive、$0/run") diff --git a/examples/stage-7/03-observability/starter_anthropic.py b/examples/stage-7/03-observability/starter_anthropic.py new file mode 100644 index 0000000..6fad0eb --- /dev/null +++ b/examples/stage-7/03-observability/starter_anthropic.py @@ -0,0 +1,47 @@ +"""Stage 7 練習 3：Observability — Path B（Anthropic、含 token usage 詳細）。 + +Anthropic SDK 的 resp.usage 欄位精確（input_tokens / output_tokens / cache_*）、 +比 Ollama 完整。Production 用 Claude 通常 token tracking 更精準。 + +跑法： + pip install -r requirements.txt + export ANTHROPIC_API_KEY=sk-ant-... + python starter_anthropic.py +""" + +from __future__ import annotations + +import os +import sys +import time +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic + +from starter import TraceContext, trace_span + +MODEL = os.environ.get("MODEL", "claude-haiku-4-5") + + +def observable_agent_anthropic(question: str, ctx: TraceContext, client: Any = None) -> str: + client = client or anthropic.Anthropic() + with trace_span(ctx, "llm_call", model=MODEL): + resp = client.messages.create( + model=MODEL, max_tokens=300, + messages=[{"role": "user", "content": question}], + ) + # Anthropic usage 是精確的 + ctx.add_tokens(input_t=resp.usage.input_tokens, output_t=resp.usage.output_tokens) + return " ".join(b.text for b in resp.content if b.type == "text") + + +if __name__ == "__main__": + ctx = TraceContext(request_id=f"req_{int(time.time()*1000)}") + answer = observable_agent_anthropic("What's 2+2?", ctx) + print(f"answer: {answer}") + print(f"trace: {ctx.summary()}") + # Claude haiku ≈ $1/$5 per M tokens、簡單 query usage ≈ 10-30 token 雙向 + print(f"\n✅ 練習 3 (Anthropic) 通過 — token usage 精確、≈$0.0001/run") diff --git a/examples/stage-7/03-observability/test.py b/examples/stage-7/03-observability/test.py new file mode 100644 index 0000000..73da9b4 --- /dev/null +++ b/examples/stage-7/03-observability/test.py @@ -0,0 +1,88 @@ +"""Stage 7 練習 3 自我驗證 — TraceContext + trace_span 邏輯。""" + +from __future__ import annotations + +import sys +import time +from types import SimpleNamespace +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter import TraceContext, multi_step_agent, observable_agent, trace_span + + +def test_trace_context_basic(): + ctx = TraceContext("req_1") + ctx.add_span("test_span", 12.5, kind="test") + ctx.add_tokens(input_t=100, output_t=50) + s = ctx.summary() + assert s["span_count"] == 1 + assert s["input_tokens"] == 100 + assert s["output_tokens"] == 50 + assert s["total_latency_ms"] == 12.5 + print("✅ test_trace_context_basic") + + +def test_trace_span_records_latency(): + ctx = TraceContext("req_2") + with trace_span(ctx, "sleep_10ms"): + time.sleep(0.01) + assert ctx.spans[0]["latency_ms"] >= 5 + assert ctx.spans[0]["name"] == "sleep_10ms" + print("✅ test_trace_span_records_latency") + + +def test_trace_span_records_error(): + ctx = TraceContext("req_3") + try: + with trace_span(ctx, "fail_span"): + raise ValueError("boom") + except ValueError: + pass + assert len(ctx.errors) == 1 + assert "boom" in ctx.errors[0] + assert ctx.spans[0].get("error") == "boom" + print("✅ test_trace_span_records_error") + + +def test_observable_agent_records_tokens(): + llm = MagicMock() + msg = SimpleNamespace(content="hi") + resp = SimpleNamespace( + choices=[SimpleNamespace(message=msg)], + usage=SimpleNamespace(prompt_tokens=20, completion_tokens=5), + ) + llm.chat.completions.create.return_value = resp + + ctx = TraceContext("req_4") + out = observable_agent("hi?", ctx, llm=llm) + assert out == "hi" + assert ctx.total_tokens["input"] == 20 + assert ctx.total_tokens["output"] == 5 + assert len(ctx.spans) == 1 + print("✅ test_observable_agent_records_tokens") + + +def test_multi_step_agent_has_multiple_spans(): + llm = MagicMock() + msg = SimpleNamespace(content="42") + resp = SimpleNamespace( + choices=[SimpleNamespace(message=msg)], + usage=SimpleNamespace(prompt_tokens=10, completion_tokens=3), + ) + llm.chat.completions.create.return_value = resp + + result = multi_step_agent("What's 2+2?", llm=llm) + assert result["trace_summary"]["span_count"] >= 2 # search + llm_call + print("✅ test_multi_step_agent_has_multiple_spans") + + +if __name__ == "__main__": + test_trace_context_basic() + test_trace_span_records_latency() + test_trace_span_records_error() + test_observable_agent_records_tokens() + test_multi_step_agent_has_multiple_spans() + print("\n🎉 全部通過 — observability primitive 邏輯正確") diff --git a/examples/stage-7/03-observability/test_anthropic.py b/examples/stage-7/03-observability/test_anthropic.py new file mode 100644 index 0000000..a567834 --- /dev/null +++ b/examples/stage-7/03-observability/test_anthropic.py @@ -0,0 +1,32 @@ +"""Stage 7 練習 3 — Anthropic mock test。""" + +from __future__ import annotations + +import sys +from types import SimpleNamespace +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter import TraceContext +from starter_anthropic import observable_agent_anthropic + + +def test_observable_anthropic_records_precise_tokens(): + client = MagicMock() + client.messages.create.return_value = SimpleNamespace( + content=[SimpleNamespace(type="text", text="42")], + usage=SimpleNamespace(input_tokens=15, output_tokens=2), + ) + ctx = TraceContext("req_1") + out = observable_agent_anthropic("2+2?", ctx, client=client) + assert "42" in out + assert ctx.total_tokens["input"] == 15 + assert ctx.total_tokens["output"] == 2 + print("✅ test_observable_anthropic_records_precise_tokens") + + +if __name__ == "__main__": + test_observable_anthropic_records_precise_tokens() + print("\n🎉 通過") diff --git a/examples/stage-7/04-sdk-advanced/README.en.md b/examples/stage-7/04-sdk-advanced/README.en.md new file mode 100644 index 0000000..c6614cf --- /dev/null +++ b/examples/stage-7/04-sdk-advanced/README.en.md @@ -0,0 +1,137 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# Exercise 4: Advanced SDK (streaming + prompt caching) + +Pairs with [Stage 7 — Multi-Agent & Production](../../../stages/07-multi-agent-production.en.md) Exercise 4. + +## Two SDK features production needs + +1. **Streaming** — send tokens to UI as they're generated; user sees first token in 0.3-1s instead of waiting for full answer +2. **Prompt caching** (Anthropic-only) — repeated long system prompts / tools / context save 90% cost + +## How to run + +### Path A (default, free, local, streaming demo) + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +### Path B (Anthropic, streaming + caching) + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +Budget: streaming + caching demo ≈ **$0.005** (2 calls + cached ~2000 tokens). + +## Validate the logic + +```bash +python test.py # 3 tests, mock OpenAI streaming +python test_anthropic.py # mock Anthropic streaming + cache_control +``` + +## Streaming + +### OpenAI / Ollama + +```python +stream = client.chat.completions.create( + model=..., messages=[...], + stream=True, +) +for chunk in stream: + delta = chunk.choices[0].delta.content + if delta: + print(delta, end="", flush=True) +``` + +### Anthropic + +```python +with client.messages.stream( + model=..., max_tokens=300, messages=[...] +) as stream: + for text in stream.text_stream: + print(text, end="", flush=True) +``` + +**UX impact**: non-streaming = 5s wait for the answer; streaming = first token in 0.5s. **Perception is dramatically different.** + +## Prompt caching (Anthropic-only) + +```python +resp = client.messages.create( + model="claude-haiku-4-5", + system=[ + { + "type": "text", + "text": "[2000-token reference material...]", + "cache_control": {"type": "ephemeral"}, + } + ], + messages=[{"role": "user", "content": "..."}] +) +``` + +First call: `cache_creation_input_tokens=2000` (25% write premium) +Subsequent calls within 5 min: `cache_read_input_tokens=2000` (10% cost = 90% off) + +**When to use**: +- Long system prompts called repeatedly (chatbots) +- Tool schemas reused across calls (multi-tool agents) +- Same document queried multiple times (RAG on a fixed doc) + +**When not**: +- Every prompt is unique +- Fewer than 1 call per 5 min (cache expires) + +## Production math + +For an agent at 1000 req/min with 5000-token system prompt: + +| Mode | Input cost / req | Monthly (30 days) | +|---|---|---| +| No caching | 5000 × $1/M = $0.005 | $216,000 | +| With caching | 500 × $1/M = $0.0005 | $21,600 | + +**90% savings** — this is why production agents universally use caching. + +## Path observations + +| Observation | Anthropic Claude | Ollama qwen2.5:3b | +|---|---|---| +| Streaming | ✅ smooth | ✅ smooth | +| First-token latency | 0.3-0.8s | 0.5-2s (CPU) | +| Prompt caching | ✅ 90% off | ❌ no API | +| Production fit | Full caching + streaming | Mostly for dev / local demo | + +## Common pitfalls + +### Streaming +- **Forgetting `flush=True`**: buffered output, user still waits +- **Not handling None deltas**: first/last chunks may have `delta.content is None` — skip them +- **Mid-stream disconnection**: catch + restart +- **Token counting**: streaming responses may not include `usage` — tokenize or sum chunks yourself + +### Prompt caching +- **`cache_control` in the wrong place**: attach to the segment you want cached. Can cache system + tools + first few messages simultaneously +- **Cache key includes model name**: switching haiku → sonnet invalidates +- **5-minute TTL**: low-QPS scenarios expire often, pay 25% premium without saving +- **Minimum 1024 tokens**: shorter content won't cache + +## Extensions + +- **Streaming + tool use**: tool_use blocks also stream; use `event_type` to dispatch +- **Anthropic Batch API**: non-realtime work, batch costs 50% less, 24h turnaround (great for eval, bulk processing) +- **Files API**: upload 100MB+ docs, combine with cache_control +- **OpenAI Responses API**: OpenAI also has prompt caching (different API, automatic) — different rules +- **Wire to observability (Exercise 3)**: log `cache_read_input_tokens` to track cache hit rate diff --git a/examples/stage-7/04-sdk-advanced/README.md b/examples/stage-7/04-sdk-advanced/README.md new file mode 100644 index 0000000..13195a1 --- /dev/null +++ b/examples/stage-7/04-sdk-advanced/README.md @@ -0,0 +1,144 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 練習 4：SDK 進階（streaming + prompt caching） + +對應 [Stage 7 — Multi-Agent & Production](../../../stages/07-multi-agent-production.md) 練習 4。 +> 🎓 **學習模式**：這份 `starter.py` 是**完整解答**、不是 TODO skeleton。建議用**主動模式**——`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫一份 `starter.py`、跑 `python test.py` 驗證；卡 20 分鐘再回去對照 reference。完整方法論看 [`docs/HOW_TO_USE.md`](../../../docs/HOW_TO_USE.md)。 + +> 📚 **想要 chapter-length 深入版？** 本 folder 的 starter 是 illustrative 版、聚焦核心 pattern + 兩條 SDK path，不是進階深度教材。深度教材推薦： +> - [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents) ⭐ 中文圈最完整、章節式 + 16 種 production 能力。**本練習對應 hello-agents 的 進階 SDK feature 章節** +> - [Anthropic Prompt Caching docs](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching) + [Anthropic Batch API](https://docs.anthropic.com/en/docs/build-with-claude/batch-processing) +> - 完整 references 見 [Stage 7 精選 Projects](../../../stages/07-multi-agent-production.md#-精選-projects範本--sdk--工具-collection) + + +## Production 兩個必備 SDK feature + +1. **Streaming** — 邊產 token 邊送 UI、user 0.3-1 秒就看到第一個字（不必等完整答案） +2. **Prompt caching**（Anthropic-only）— 重複 long system prompt / tools / context 省 90% cost + +## 怎麼跑 + +### Path A（默認、本機免費、streaming demo） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +### Path B（Anthropic、streaming + caching） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +預算：streaming demo + caching demo ≈ **$0.005**（2 call + cached ~2000 token）。 + +## 不花錢驗證程式邏輯 + +```bash +python test.py # 3 個 test、mock OpenAI streaming +python test_anthropic.py # mock Anthropic streaming + cache_control +``` + +## Streaming 怎麼用 + +### OpenAI / Ollama + +```python +stream = client.chat.completions.create( + model=..., messages=[...], + stream=True, # ← key +) +for chunk in stream: + delta = chunk.choices[0].delta.content + if delta: + print(delta, end="", flush=True) +``` + +### Anthropic + +```python +with client.messages.stream( + model=..., max_tokens=300, messages=[...] +) as stream: + for text in stream.text_stream: + print(text, end="", flush=True) +``` + +**UX 觀察**：non-streaming 5 秒才看到答案 / streaming 0.5 秒看到第一個字。**user perception 差很大**。 + +## Prompt Caching 怎麼用（Anthropic-only） + +```python +resp = client.messages.create( + model="claude-haiku-4-5", + system=[ + { + "type": "text", + "text": "[2000-token reference material...]", + "cache_control": {"type": "ephemeral"}, # ← key + } + ], + messages=[{"role": "user", "content": "..."}] +) +``` + +第一次寫入：`cache_creation_input_tokens=2000`（25% premium） +之後 5 分鐘內：`cache_read_input_tokens=2000`（10% cost = 90% off） + +**何時用**： +- Long system prompt 重複 call（聊天機器人） +- Tool schema 重複（multi-tool agent） +- Document context 重複問（RAG with same doc） + +**不用的時候**： +- 每次 prompt 都不同 +- 5 分鐘內 call 次數 < 1（cache 過期） + +## Production 算盤 + +對 1000 req/min 的 agent、prompt 含 5000 token system prompt： + +| 模式 | Input cost / req | 月 cost（30 天） | +|---|---|---| +| 無 caching | 5000 × $1/M = $0.005 | $216,000 | +| 有 caching | 500 × $1/M = $0.0005 | $21,600 | + +**省 90%**——這就是為什麼 production agent 一律用 caching。 + +## 兩個 path 觀察重點 + +| 觀察項 | Anthropic Claude | Ollama qwen2.5:3b | +|---|---|---| +| Streaming | ✅ smooth | ✅ smooth | +| First token latency | 0.3-0.8s | 0.5-2s (CPU) | +| Prompt caching | ✅ 90% off | ❌ 無此 API | +| 適合 production 用法 | 全套 caching + streaming | 主要為 dev / 本機 demo | + +## 常見坑 + +### Streaming +- **忘記 `flush=True`**：buffered output、user 還是要等 +- **沒處理 None delta**：開頭 / 結尾 chunk 可能 `delta.content is None`、要 skip +- **錯誤處理**：streaming 中途斷線、要 catch + restart +- **Token counting**：streaming response 不一定有 `usage`、要自己 tokenize 或 sum chunks + +### Prompt caching +- **`cache_control` 放錯位置**：要在「想 cache 的那段」、不是整個 system。可同時 cache system + tools + 前面幾條 messages +- **Cache key 含 model name**：換 model（haiku → sonnet）cache 失效 +- **5 分鐘 TTL**：低 QPS 場景 cache 經常過期、白付 25% premium 沒省到 +- **Minimum 1024 tokens**：太短的 content cache 不會生效 + +## 延伸 + +- **Streaming + tool use**：tool_use block 也能 stream、用 `event_type` 判斷 +- **Anthropic Batch API**：非即時任務丟 batch、省 50% cost、24 小時內回（適合 eval、bulk processing） +- **Files API**：100MB+ 文件直接 upload、cache_control 一起用 +- **OpenAI Responses API**：OpenAI 也有 prompt caching（不同 API、自動 cache）、條件不同 +- **接 observability（練習 3）**：cache_read_input_tokens 記到 telemetry、追 cache hit rate diff --git a/examples/stage-7/04-sdk-advanced/README.zh-Hans.md b/examples/stage-7/04-sdk-advanced/README.zh-Hans.md new file mode 100644 index 0000000..d3528ae --- /dev/null +++ b/examples/stage-7/04-sdk-advanced/README.zh-Hans.md @@ -0,0 +1,137 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 练习 4：SDK 进阶（streaming + prompt caching） + +对应 [Stage 7 — Multi-Agent & Production](../../../stages/07-multi-agent-production.zh-Hans.md) 练习 4。 + +## Production 两个必备 SDK feature + +1. **Streaming** — 边产 token 边送 UI、user 0.3-1 秒就看到第一个字（不必等完整答案） +2. **Prompt caching**（Anthropic-only）— 重复 long system prompt / tools / context 省 90% cost + +## 怎么跑 + +### Path A（默认、本机免费、streaming demo） + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve +python starter.py +``` + +### Path B（Anthropic、streaming + caching） + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +python starter_anthropic.py +``` + +预算：streaming demo + caching demo ≈ **$0.005**（2 call + cached ~2000 token）。 + +## 不花钱验证程序逻辑 + +```bash +python test.py # 3 个 test、mock OpenAI streaming +python test_anthropic.py # mock Anthropic streaming + cache_control +``` + +## Streaming 怎么用 + +### OpenAI / Ollama + +```python +stream = client.chat.completions.create( + model=..., messages=[...], + stream=True, +) +for chunk in stream: + delta = chunk.choices[0].delta.content + if delta: + print(delta, end="", flush=True) +``` + +### Anthropic + +```python +with client.messages.stream( + model=..., max_tokens=300, messages=[...] +) as stream: + for text in stream.text_stream: + print(text, end="", flush=True) +``` + +**UX 观察**：non-streaming 5 秒才看到答案 / streaming 0.5 秒看到第一个字。**user perception 差很大**。 + +## Prompt Caching 怎么用（Anthropic-only） + +```python +resp = client.messages.create( + model="claude-haiku-4-5", + system=[ + { + "type": "text", + "text": "[2000-token reference material...]", + "cache_control": {"type": "ephemeral"}, + } + ], + messages=[{"role": "user", "content": "..."}] +) +``` + +第一次写入：`cache_creation_input_tokens=2000`（25% premium） +之后 5 分钟内：`cache_read_input_tokens=2000`（10% cost = 90% off） + +**何时用**： +- Long system prompt 重复 call（聊天机器人） +- Tool schema 重复（multi-tool agent） +- Document context 重复问（RAG with same doc） + +**不用的时候**： +- 每次 prompt 都不同 +- 5 分钟内 call 次数 < 1（cache 过期） + +## Production 算盘 + +对 1000 req/min 的 agent、prompt 含 5000 token system prompt： + +| 模式 | Input cost / req | 月 cost（30 天） | +|---|---|---| +| 无 caching | 5000 × $1/M = $0.005 | $216,000 | +| 有 caching | 500 × $1/M = $0.0005 | $21,600 | + +**省 90%**——这就是为什么 production agent 一律用 caching。 + +## 两个 path 观察重点 + +| 观察项 | Anthropic Claude | Ollama qwen2.5:3b | +|---|---|---| +| Streaming | ✅ smooth | ✅ smooth | +| First token latency | 0.3-0.8s | 0.5-2s (CPU) | +| Prompt caching | ✅ 90% off | ❌ 无此 API | +| 适合 production 用法 | 全套 caching + streaming | 主要为 dev / 本机 demo | + +## 常见坑 + +### Streaming +- **忘记 `flush=True`**：buffered output、user 还是要等 +- **没处理 None delta**：开头 / 结尾 chunk 可能 `delta.content is None`、要 skip +- **错误处理**：streaming 中途断线、要 catch + restart +- **Token counting**：streaming response 不一定有 `usage`、要自己 tokenize 或 sum chunks + +### Prompt caching +- **`cache_control` 放错位置**：要在“想 cache 的那段”、不是整个 system。可同时 cache system + tools + 前面几条 messages +- **Cache key 含 model name**：换 model（haiku → sonnet）cache 失效 +- **5 分钟 TTL**：低 QPS 场景 cache 经常过期、白付 25% premium 没省到 +- **Minimum 1024 tokens**：太短的 content cache 不会生效 + +## 延伸 + +- **Streaming + tool use**：tool_use block 也能 stream、用 `event_type` 判断 +- **Anthropic Batch API**：非实时任务丢 batch、省 50% cost、24 小时内回（适合 eval、bulk processing） +- **Files API**：100MB+ 文件直接 upload、cache_control 一起用 +- **OpenAI Responses API**：OpenAI 也有 prompt caching（不同 API、自动 cache）、条件不同 +- **接 observability（练习 3）**：cache_read_input_tokens 记到 telemetry、追 cache hit rate diff --git a/examples/stage-7/04-sdk-advanced/requirements.txt b/examples/stage-7/04-sdk-advanced/requirements.txt new file mode 100644 index 0000000..7c268c1 --- /dev/null +++ b/examples/stage-7/04-sdk-advanced/requirements.txt @@ -0,0 +1,2 @@ +openai>=1.50,<2.0 +anthropic>=0.40,<1.0 diff --git a/examples/stage-7/04-sdk-advanced/starter.py b/examples/stage-7/04-sdk-advanced/starter.py new file mode 100644 index 0000000..144a89b --- /dev/null +++ b/examples/stage-7/04-sdk-advanced/starter.py @@ -0,0 +1,74 @@ +"""Stage 7 練習 4：SDK 進階 — Path A（Ollama 默認、$0、streaming）。 + +Production agent 兩個必備 SDK 進階 feature： +1. **Streaming**：邊產 token 邊回 UI（user perception 從 5 秒 → 0.5 秒就看到 token） +2. **Prompt caching**：Anthropic-specific、重複 long context 省 90% cost（Path B 才有） + +跑法： + pip install -r requirements.txt + ollama pull qwen2.5:3b + ollama serve + python starter.py +""" + +from __future__ import annotations + +import os +import sys +import time +from typing import Any, Iterator + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +MODEL = os.environ.get("MODEL", "qwen2.5:3b") +OLLAMA_BASE = os.environ.get("OLLAMA_API_BASE", "http://localhost:11434/v1") + + +def stream_response(prompt: str, llm: Any = None) -> Iterator[str]: + """Yield each token chunk as it arrives.""" + llm = llm or OpenAI(base_url=OLLAMA_BASE, api_key="ollama") + stream = llm.chat.completions.create( + model=MODEL, + messages=[{"role": "user", "content": prompt}], + stream=True, + ) + for chunk in stream: + delta = chunk.choices[0].delta.content + if delta: + yield delta + + +def stream_to_string(prompt: str, llm: Any = None) -> dict: + """Helper：consume streaming generator + 統計 latency。""" + t0 = time.perf_counter() + first_token_at = None + chunks = [] + for delta in stream_response(prompt, llm=llm): + if first_token_at is None: + first_token_at = time.perf_counter() - t0 + chunks.append(delta) + total_latency = time.perf_counter() - t0 + return { + "text": "".join(chunks), + "first_token_ms": (first_token_at or 0) * 1000, + "total_latency_ms": total_latency * 1000, + "chunk_count": len(chunks), + } + + +if __name__ == "__main__": + prompt = "Explain what a Python list comprehension is in 3 sentences." + print(f"❓ {prompt}\n") + print("(streaming token by token...)\n") + + t0 = time.perf_counter() + for delta in stream_response(prompt): + print(delta, end="", flush=True) + total = time.perf_counter() - t0 + + print(f"\n\n📊 Total: {total:.2f}s") + print(f"✅ 練習 4 通過 — streaming SDK 跑通、$0/run") + print(" UX 觀察：streaming 讓 user 早在 0.3-1 秒就看到第一個 token，不必等完整答案") diff --git a/examples/stage-7/04-sdk-advanced/starter_anthropic.py b/examples/stage-7/04-sdk-advanced/starter_anthropic.py new file mode 100644 index 0000000..1268593 --- /dev/null +++ b/examples/stage-7/04-sdk-advanced/starter_anthropic.py @@ -0,0 +1,86 @@ +"""Stage 7 練習 4：SDK 進階 — Path B（Anthropic streaming + prompt caching）。 + +Anthropic 額外的兩個 production 殺手 feature： +1. **Streaming**：跟 OpenAI 類似、但 content blocks 結構不同 +2. **Prompt caching**：cache_control={"type": "ephemeral"}、重複 long context（system / tools） + 省 90% cost。第一次寫入有 25% premium、之後 5 分鐘內每次省 90%。 + +跑法： + pip install -r requirements.txt + export ANTHROPIC_API_KEY=sk-ant-... + python starter_anthropic.py +""" + +from __future__ import annotations + +import os +import sys +import time +from typing import Any, Iterator + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic + +MODEL = os.environ.get("MODEL", "claude-haiku-4-5") + + +def stream_anthropic(prompt: str, client: Any = None) -> Iterator[str]: + client = client or anthropic.Anthropic() + with client.messages.stream( + model=MODEL, + max_tokens=300, + messages=[{"role": "user", "content": prompt}], + ) as stream: + for text in stream.text_stream: + yield text + + +def cached_query(question: str, large_system_prompt: str, client: Any = None) -> dict: + """示範 prompt caching：第一次 call 沒 cache、第二次 hit cache、看 usage 差異。""" + client = client or anthropic.Anthropic() + + resp = client.messages.create( + model=MODEL, + max_tokens=200, + system=[ + { + "type": "text", + "text": large_system_prompt, + "cache_control": {"type": "ephemeral"}, # ← key + } + ], + messages=[{"role": "user", "content": question}], + ) + + usage = resp.usage + return { + "answer": " ".join(b.text for b in resp.content if b.type == "text"), + "input_tokens": usage.input_tokens, + "cache_creation_input_tokens": getattr(usage, "cache_creation_input_tokens", 0), + "cache_read_input_tokens": getattr(usage, "cache_read_input_tokens", 0), + "output_tokens": usage.output_tokens, + } + + +if __name__ == "__main__": + # Demo 1: streaming + print("=== Streaming demo ===") + print("(token by token...)\n") + t0 = time.perf_counter() + for delta in stream_anthropic("Explain Python list comprehension in 3 sentences."): + print(delta, end="", flush=True) + print(f"\n[took {time.perf_counter()-t0:.2f}s]\n") + + # Demo 2: prompt caching（同 system prompt 2 次、第二次省 90% input cost） + print("=== Prompt caching demo ===") + big_system = "You are a helpful assistant. " + ("This is reference material. " * 200) # ~2000 tokens + r1 = cached_query("What's 2+2?", big_system) + print(f"Call 1 (cache miss): input={r1['input_tokens']}, cache_create={r1['cache_creation_input_tokens']}, cache_read={r1['cache_read_input_tokens']}") + r2 = cached_query("What's 3+3?", big_system) + print(f"Call 2 (cache hit): input={r2['input_tokens']}, cache_create={r2['cache_creation_input_tokens']}, cache_read={r2['cache_read_input_tokens']}") + + if r2["cache_read_input_tokens"] > 0: + print(f"\n✅ Cache 命中！第二次只算 {r2['input_tokens']} input + {r2['cache_read_input_tokens']} cache-read（cache-read 只算 10% 價）") + print(f"\n✅ 練習 4 (Anthropic) 通過 — streaming + prompt caching、Claude {MODEL}") diff --git a/examples/stage-7/04-sdk-advanced/test.py b/examples/stage-7/04-sdk-advanced/test.py new file mode 100644 index 0000000..28226b4 --- /dev/null +++ b/examples/stage-7/04-sdk-advanced/test.py @@ -0,0 +1,56 @@ +"""Stage 7 練習 4 自我驗證 — streaming + stream_to_string。""" + +from __future__ import annotations + +import sys +from types import SimpleNamespace +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter import stream_response, stream_to_string + + +def fake_streaming_llm(chunks: list[str]): + """Mock OpenAI streaming response — iterable of chunks.""" + def make_chunk(delta): + return SimpleNamespace(choices=[SimpleNamespace(delta=SimpleNamespace(content=delta))]) + llm = MagicMock() + llm.chat.completions.create.return_value = iter([make_chunk(c) for c in chunks]) + return llm + + +def test_stream_response_yields_chunks(): + llm = fake_streaming_llm(["Hello", " ", "world"]) + out = list(stream_response("hi", llm=llm)) + assert out == ["Hello", " ", "world"] + print("✅ test_stream_response_yields_chunks") + + +def test_stream_to_string_aggregates(): + llm = fake_streaming_llm(["a", "b", "c"]) + result = stream_to_string("q", llm=llm) + assert result["text"] == "abc" + assert result["chunk_count"] == 3 + print("✅ test_stream_to_string_aggregates") + + +def test_stream_to_string_skips_empty_deltas(): + """Streaming 偶爾有 None delta（initial / final chunk）、要 skip。""" + def make_chunk(delta): + return SimpleNamespace(choices=[SimpleNamespace(delta=SimpleNamespace(content=delta))]) + llm = MagicMock() + llm.chat.completions.create.return_value = iter([ + make_chunk(None), make_chunk("a"), make_chunk(""), make_chunk("b"), make_chunk(None), + ]) + result = stream_to_string("q", llm=llm) + assert result["text"] == "ab" + print("✅ test_stream_to_string_skips_empty_deltas") + + +if __name__ == "__main__": + test_stream_response_yields_chunks() + test_stream_to_string_aggregates() + test_stream_to_string_skips_empty_deltas() + print("\n🎉 全部通過 — streaming 邏輯正確") diff --git a/examples/stage-7/04-sdk-advanced/test_anthropic.py b/examples/stage-7/04-sdk-advanced/test_anthropic.py new file mode 100644 index 0000000..3f1e21a --- /dev/null +++ b/examples/stage-7/04-sdk-advanced/test_anthropic.py @@ -0,0 +1,52 @@ +"""Stage 7 練習 4 — Anthropic streaming + cached_query mock.""" + +from __future__ import annotations + +import sys +from types import SimpleNamespace +from unittest.mock import MagicMock + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from starter_anthropic import cached_query, stream_anthropic + + +def test_stream_anthropic_yields_text(): + """Mock Anthropic streaming context manager.""" + stream_obj = MagicMock() + stream_obj.text_stream = iter(["Hello", " world"]) + stream_obj.__enter__ = MagicMock(return_value=stream_obj) + stream_obj.__exit__ = MagicMock(return_value=False) + + client = MagicMock() + client.messages.stream.return_value = stream_obj + + out = list(stream_anthropic("hi", client=client)) + assert out == ["Hello", " world"] + print("✅ test_stream_anthropic_yields_text") + + +def test_cached_query_passes_cache_control(): + """確認 cache_control 真的傳進去 system param。""" + client = MagicMock() + client.messages.create.return_value = SimpleNamespace( + content=[SimpleNamespace(type="text", text="ok")], + usage=SimpleNamespace( + input_tokens=10, output_tokens=2, + cache_creation_input_tokens=2000, cache_read_input_tokens=0, + ), + ) + result = cached_query("Q?", "big system prompt", client=client) + call_kwargs = client.messages.create.call_args.kwargs + system_arg = call_kwargs["system"] + assert isinstance(system_arg, list) + assert system_arg[0]["cache_control"] == {"type": "ephemeral"} + assert result["cache_creation_input_tokens"] == 2000 + print("✅ test_cached_query_passes_cache_control") + + +if __name__ == "__main__": + test_stream_anthropic_yields_text() + test_cached_query_passes_cache_control() + print("\n🎉 通過 — streaming + caching API contract 正確") diff --git a/examples/stage-7/05-deploy/Dockerfile b/examples/stage-7/05-deploy/Dockerfile new file mode 100644 index 0000000..163dbb8 --- /dev/null +++ b/examples/stage-7/05-deploy/Dockerfile @@ -0,0 +1,22 @@ +FROM python:3.11-slim + +WORKDIR /app + +COPY requirements.txt . +RUN pip install --no-cache-dir -r requirements.txt + +COPY starter.py starter_anthropic.py ./ + +# Default to Ollama path — override APP_MODULE for Anthropic +ENV APP_MODULE=starter:app + +EXPOSE 8000 + +CMD ["sh", "-c", "uvicorn ${APP_MODULE} --host 0.0.0.0 --port 8000"] + +# Build: +# docker build -t agent-api . +# Run (Ollama path; needs ollama on host): +# docker run -p 8000:8000 -e OLLAMA_API_BASE=http://host.docker.internal:11434/v1 agent-api +# Run (Anthropic path): +# docker run -p 8000:8000 -e APP_MODULE=starter_anthropic:app -e ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY agent-api diff --git a/examples/stage-7/05-deploy/README.en.md b/examples/stage-7/05-deploy/README.en.md new file mode 100644 index 0000000..e290199 --- /dev/null +++ b/examples/stage-7/05-deploy/README.en.md @@ -0,0 +1,125 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# Exercise 5: Deploy (FastAPI + Docker) + +Pairs with [Stage 7 — Multi-Agent & Production](../../../stages/07-multi-agent-production.en.md) Exercise 5. + +## Task + +Package the agent as a production-style HTTP API: + +- FastAPI app with `/health` + `/chat` endpoints +- Structured logging with `request_id` +- Proper HTTP status codes (200 / 422 / 429 / 503 / 500) +- Pydantic schema validation (FastAPI free) +- Dockerfile (covers both Ollama and Anthropic deploys) + +## How to run + +### Local Ollama + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve + +uvicorn starter:app --reload --port 8000 + +# In another shell: +curl -X POST http://localhost:8000/chat \ + -H 'Content-Type: application/json' \ + -d '{"message": "hi"}' +``` + +### Local Anthropic + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +uvicorn starter_anthropic:app --reload --port 8000 +``` + +### Docker + +```bash +docker build -t agent-api . + +# Ollama path (host must run ollama) +docker run -p 8000:8000 \ + -e OLLAMA_API_BASE=http://host.docker.internal:11434/v1 \ + agent-api + +# Anthropic path +docker run -p 8000:8000 \ + -e APP_MODULE=starter_anthropic:app \ + -e ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY \ + agent-api +``` + +## Validate without starting the server + +```bash +python test.py # 5 tests via fastapi.TestClient +python test_anthropic.py # 3 tests (incl. 429 rate limit) +``` + +`fastapi.TestClient` uses in-process ASGI — no real port, no Docker. + +## Production essentials + +| Element | Why | In this starter | +|---|---|---| +| `/health` endpoint | K8s liveness/readiness probes | ✅ | +| `request_id` per call | Trace / debug | ✅ uuid4 | +| Structured logging | ELK / Datadog / Loki parseable | ✅ JSON-like format | +| Pydantic schema validation | Malformed JSON → 422 automatically | ✅ FastAPI built-in | +| Specific exception → HTTP status | 503 ≠ 500 — client knows whether to retry | ✅ APIConnectionError → 503 | +| Token tracking in response | Cost / token usage transparency | ✅ Path B includes input/output tokens | + +## Status codes + +| Situation | HTTP code | Client should | +|---|---|---| +| LLM answered | 200 | Use answer | +| Missing `message` field | 422 | Fix request, don't retry | +| Anthropic rate limit (429) | 429 | Exponential backoff retry | +| LLM service disconnected | 503 | Retry (transient) | +| Other unexpected | 500 | Log + alert, don't auto-retry | + +## Deploy targets + +| Target | Good for | Watch out | +|---|---|---| +| **Local uvicorn** | Dev | 1 worker, not for prod | +| **Docker + uvicorn** | Small prod | Add `--workers N`, put nginx in front | +| **K8s** | Scalable prod | Use `/health` for liveness/readiness | +| **AWS Lambda + API Gateway** | Sporadic traffic | Slow cold starts, fits light agents | +| **Cloud Run / Fargate** | Mid-scale prod | Scale-to-zero, simple | +| **Anthropic Computer Use / Skills** | Very specific use cases | See Stage 5 | + +## Common pitfalls + +- **No health check**: load balancer can't detect dead instances +- **Heavy `/health`**: calling the LLM to verify = wasted cost + slow startup gets you killed +- **Missing `request_id`**: traces scattered across logs, can't correlate +- **All errors → 500**: client can't distinguish transient (retry) vs permanent. Use specific codes +- **Synchronous LLM call in `def`**: FastAPI blocks the event loop. Use `async def` + `await client.messages.create(...)` or a thread pool +- **No rate limiting**: attackers or buggy clients explode your LLM bill. Add `slowapi` / nginx rate limit +- **Hard-coded secrets**: API key in code = git leak. Use env vars + secret manager + +## Connecting back to earlier exercises + +- **Exercise 3 observability**: add `TraceContext` to endpoint, log latency / tokens / errors per request +- **Exercise 2 eval**: post-deploy CI eval, `pass_rate < 90%` triggers rollback +- **Exercise 4 caching**: cache_control on the system prompt — 90% cost cut immediately +- **Stage 6 RAG**: endpoint wires up vector DB + memory store + +## Extensions + +- **Streaming endpoint**: `@app.post("/chat/stream")` with `StreamingResponse` + SSE format +- **Auth**: FastAPI `Depends(verify_token)` + JWT / API key +- **Cost limit**: per-user / per-day token cap, reject above limit +- **OpenTelemetry**: `tracer.start_as_current_span("chat_endpoint")` ships traces to Datadog +- **K8s manifests**: Deployment + Service + HPA + ConfigMap diff --git a/examples/stage-7/05-deploy/README.md b/examples/stage-7/05-deploy/README.md new file mode 100644 index 0000000..236be29 --- /dev/null +++ b/examples/stage-7/05-deploy/README.md @@ -0,0 +1,132 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 練習 5：Deploy（FastAPI + Docker） + +對應 [Stage 7 — Multi-Agent & Production](../../../stages/07-multi-agent-production.md) 練習 5。 +> 🎓 **學習模式**：這份 `starter.py` 是**完整解答**、不是 TODO skeleton。建議用**主動模式**——`mv starter.py starter_reference.py`、看 signature 不看 body、自己重寫一份 `starter.py`、跑 `python test.py` 驗證；卡 20 分鐘再回去對照 reference。完整方法論看 [`docs/HOW_TO_USE.md`](../../../docs/HOW_TO_USE.md)。 + +> 📚 **想要 chapter-length 深入版？** 本 folder 的 starter 是 illustrative 版、聚焦核心 pattern + 兩條 SDK path，不是進階深度教材。深度教材推薦： +> - [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents) ⭐ 中文圈最完整、章節式 + 16 種 production 能力。**本練習對應 hello-agents 的 production deploy / harness 章節** +> - [FastAPI official tutorial](https://fastapi.tiangolo.com/tutorial/) + [awesome-harness-engineering](https://github.com/ai-boost/awesome-harness-engineering)（harness pattern 全集） +> - 完整 references 見 [Stage 7 精選 Projects](../../../stages/07-multi-agent-production.md#-精選-projects範本--sdk--工具-collection) + + +## 任務 + +把 agent 包進 production-style HTTP API： + +- FastAPI app with `/health` + `/chat` endpoints +- Structured logging with request_id +- Proper HTTP status codes (200 / 422 / 429 / 503 / 500) +- Pydantic schema validation (FastAPI 自動驗） +- Dockerfile（含 Ollama 跟 Anthropic 兩個 deploy 模式） + +## 怎麼跑 + +### Local Ollama + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve + +uvicorn starter:app --reload --port 8000 + +# 另一個 shell: +curl -X POST http://localhost:8000/chat \ + -H 'Content-Type: application/json' \ + -d '{"message": "hi"}' +``` + +### Local Anthropic + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +uvicorn starter_anthropic:app --reload --port 8000 +``` + +### Docker + +```bash +docker build -t agent-api . + +# Ollama path（需 host 跑著 ollama） +docker run -p 8000:8000 \ + -e OLLAMA_API_BASE=http://host.docker.internal:11434/v1 \ + agent-api + +# Anthropic path +docker run -p 8000:8000 \ + -e APP_MODULE=starter_anthropic:app \ + -e ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY \ + agent-api +``` + +## 不啟 server 驗證 + +```bash +python test.py # 5 個 test、用 fastapi.TestClient +python test_anthropic.py # 3 個 test（含 429 rate limit） +``` + +`fastapi.TestClient` 用 in-process ASGI、不開真 port、不用 docker。 + +## Production 必備 + +| 元素 | 為什麼 | 在這份 starter | +|---|---|---| +| `/health` endpoint | K8s liveness / readiness probe | ✅ | +| `request_id` per call | trace / debug 必備 | ✅ uuid4 | +| Structured logging | ELK / Datadog / Loki 看得懂 | ✅ JSON-like format | +| Pydantic schema validation | malformed JSON 自動 422 | ✅ FastAPI 內建 | +| Specific exception → HTTP status | 503 ≠ 500，client 知道該不該 retry | ✅ APIConnectionError → 503 | +| Token tracking response | cost / token usage 透明 | ✅ Path B 含 input_tokens / output_tokens | + +## Status code 對照 + +| 情況 | HTTP code | client 該怎樣 | +|---|---|---| +| LLM 答了 | 200 | 用答案 | +| user 沒傳 message field | 422 | 修 request、別 retry | +| Anthropic rate limit (429) | 429 | exponential backoff retry | +| LLM 服務斷線 (APIConnectionError) | 503 | retry（transient） | +| 其他 unexpected | 500 | log + alert、別自動 retry | + +## Deploy targets + +| Target | 適合 | 注意 | +|---|---|---| +| **Local uvicorn** | dev | 1 worker、不適 production | +| **Docker + uvicorn** | small prod | 加 `--workers N`、reverse proxy（nginx）前面 | +| **K8s** | scalable prod | liveness/readiness probe 用 `/health` | +| **AWS Lambda + API Gateway** | sporadic traffic | cold start 慢、適合輕量 agent | +| **Cloud Run / Fargate** | 中規模 prod | scale-to-zero、簡單 | +| **Anthropic Computer Use / Skills** | very specific use cases | 看 Stage 5 | + +## 常見坑 + +- **沒 health check**：load balancer 不知道 instance 死了、流量繼續送 +- **`/health` 太重**：去打 LLM 確認 = 耗 cost、且 cold start 慢就被踢 +- **`request_id` 沒記**：trace 散在各 log 裡找不到對應 +- **All errors → 500**：client 無法分辨 transient（retry）vs permanent（don't retry）。要分 status code +- **synchronous LLM call**：FastAPI 用 `def` 而非 `async def`、會 block event loop。Production 應該用 `async def` + `await client.messages.create(...)` 或 thread pool +- **No rate limiting**：被攻擊或 client bug 會打爆 LLM bill。前面加 `slowapi` / nginx rate limit +- **Hard-coded secret**：API key 直接寫 code = git 流出。用 env var + secret manager + +## 接前面 stages + +- **練習 3 observability**：把 `TraceContext` 加進 endpoint、每 request 記 latency / tokens / errors +- **練習 2 eval**：deploy 後跑 CI eval、`pass_rate < 90%` 就 rollback +- **練習 4 caching**：把 system prompt 加 `cache_control`、production cost 立刻減 90% +- **Stage 6 RAG**：endpoint 接 vector DB + memory store + +## 延伸 + +- **加 streaming endpoint**：`@app.post("/chat/stream")` 配 `StreamingResponse` + SSE format +- **加 auth**：FastAPI `Depends(verify_token)` + JWT / API key +- **加 cost limit**：每 user / day 上限 X token、超過 reject +- **接 OpenTelemetry**：`tracer.start_as_current_span("chat_endpoint")` 自動丟去 Datadog +- **K8s manifests**：Deployment + Service + HPA + ConfigMap diff --git a/examples/stage-7/05-deploy/README.zh-Hans.md b/examples/stage-7/05-deploy/README.zh-Hans.md new file mode 100644 index 0000000..78a49f1 --- /dev/null +++ b/examples/stage-7/05-deploy/README.zh-Hans.md @@ -0,0 +1,120 @@ +
+ 繁體中文 | 简体中文 | English +
+ +# 练习 5：Deploy（FastAPI + Docker） + +对应 [Stage 7 — Multi-Agent & Production](../../../stages/07-multi-agent-production.zh-Hans.md) 练习 5。 + +## 任务 + +把 agent 包进 production-style HTTP API： + +- FastAPI app with `/health` + `/chat` endpoints +- Structured logging with request_id +- Proper HTTP status codes (200 / 422 / 429 / 503 / 500) +- Pydantic schema validation (FastAPI 自动验） +- Dockerfile（含 Ollama 跟 Anthropic 两个 deploy 模式） + +## 怎么跑 + +### Local Ollama + +```bash +pip install -r requirements.txt +ollama pull qwen2.5:3b +ollama serve + +uvicorn starter:app --reload --port 8000 + +curl -X POST http://localhost:8000/chat \ + -H 'Content-Type: application/json' \ + -d '{"message": "hi"}' +``` + +### Local Anthropic + +```bash +pip install -r requirements.txt +export ANTHROPIC_API_KEY=sk-ant-... +uvicorn starter_anthropic:app --reload --port 8000 +``` + +### Docker + +```bash +docker build -t agent-api . + +docker run -p 8000:8000 \ + -e OLLAMA_API_BASE=http://host.docker.internal:11434/v1 \ + agent-api + +docker run -p 8000:8000 \ + -e APP_MODULE=starter_anthropic:app \ + -e ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY \ + agent-api +``` + +## 不启 server 验证 + +```bash +python test.py # 5 个 test、用 fastapi.TestClient +python test_anthropic.py # 3 个 test（含 429 rate limit） +``` + +## Production 必备 + +| 元素 | 为什么 | 在这份 starter | +|---|---|---| +| `/health` endpoint | K8s liveness / readiness probe | ✅ | +| `request_id` per call | trace / debug 必备 | ✅ uuid4 | +| Structured logging | ELK / Datadog / Loki 看得懂 | ✅ | +| Pydantic schema validation | malformed JSON 自动 422 | ✅ FastAPI 内建 | +| Specific exception → HTTP status | 503 ≠ 500，client 知道该不该 retry | ✅ APIConnectionError → 503 | +| Token tracking response | cost / token usage 透明 | ✅ Path B 含 input_tokens / output_tokens | + +## Status code 对照 + +| 情况 | HTTP code | client 该怎样 | +|---|---|---| +| LLM 答了 | 200 | 用答案 | +| user 没传 message field | 422 | 修 request、别 retry | +| Anthropic rate limit (429) | 429 | exponential backoff retry | +| LLM 服务断线 (APIConnectionError) | 503 | retry（transient） | +| 其他 unexpected | 500 | log + alert、别自动 retry | + +## Deploy targets + +| Target | 适合 | 注意 | +|---|---|---| +| **Local uvicorn** | dev | 1 worker、不适 production | +| **Docker + uvicorn** | small prod | 加 `--workers N`、reverse proxy（nginx）前面 | +| **K8s** | scalable prod | liveness/readiness probe 用 `/health` | +| **AWS Lambda + API Gateway** | sporadic traffic | cold start 慢、适合轻量 agent | +| **Cloud Run / Fargate** | 中规模 prod | scale-to-zero、简单 | +| **Anthropic Computer Use / Skills** | very specific use cases | 看 Stage 5 | + +## 常见坑 + +- **没 health check**：load balancer 不知道 instance 死了、流量继续送 +- **`/health` 太重**：去打 LLM 确认 = 耗 cost、且 cold start 慢就被踢 +- **`request_id` 没记**：trace 散在各 log 里找不到对应 +- **All errors → 500**：client 无法分辨 transient（retry）vs permanent（don't retry）。要分 status code +- **synchronous LLM call**：FastAPI 用 `def` 而非 `async def`、会 block event loop。Production 应该用 `async def` + `await client.messages.create(...)` 或 thread pool +- **No rate limiting**：被攻击或 client bug 会打爆 LLM bill。前面加 `slowapi` / nginx rate limit +- **Hard-coded secret**：API key 直接写 code = git 流出。用 env var + secret manager + +## 接前面 stages + +- **练习 3 observability**：把 `TraceContext` 加进 endpoint、每 request 记 latency / tokens / errors +- **练习 2 eval**：deploy 后跑 CI eval、`pass_rate < 90%` 就 rollback +- **练习 4 caching**：把 system prompt 加 `cache_control`、production cost 立刻减 90% +- **Stage 6 RAG**：endpoint 接 vector DB + memory store + +## 延伸 + +- **加 streaming endpoint**：`@app.post("/chat/stream")` 配 `StreamingResponse` + SSE format +- **加 auth**：FastAPI `Depends(verify_token)` + JWT / API key +- **加 cost limit**：每 user / day 上限 X token、超过 reject +- **接 OpenTelemetry**：`tracer.start_as_current_span("chat_endpoint")` 自动丢去 Datadog +- **K8s manifests**：Deployment + Service + HPA + ConfigMap diff --git a/examples/stage-7/05-deploy/requirements.txt b/examples/stage-7/05-deploy/requirements.txt new file mode 100644 index 0000000..141af4a --- /dev/null +++ b/examples/stage-7/05-deploy/requirements.txt @@ -0,0 +1,6 @@ +fastapi>=0.110,<1.0 +uvicorn[standard]>=0.27,<1.0 +pydantic>=2.6,<3.0 +httpx>=0.27,<1.0 # TestClient 需要 +openai>=1.50,<2.0 +anthropic>=0.40,<1.0 diff --git a/examples/stage-7/05-deploy/starter.py b/examples/stage-7/05-deploy/starter.py new file mode 100644 index 0000000..1d59d41 --- /dev/null +++ b/examples/stage-7/05-deploy/starter.py @@ -0,0 +1,101 @@ +"""Stage 7 練習 5：Deploy — Path A（FastAPI + Ollama、$0）。 + +把 agent 包進 FastAPI HTTP endpoint、production-style health check + structured logging。 +本地端 `uvicorn starter:app` 就能跑、Docker / k8s / Lambda 都能 lift。 + +跑法： + pip install -r requirements.txt + ollama pull qwen2.5:3b + ollama serve + uvicorn starter:app --reload --port 8000 + # 另一個 shell: curl -X POST http://localhost:8000/chat -H 'Content-Type: application/json' -d '{"message": "hi"}' + +驗證（不必啟 server）: + python test.py +""" + +from __future__ import annotations + +import logging +import os +import sys +import time +import uuid +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from fastapi import FastAPI, HTTPException +from openai import APIConnectionError, OpenAI +from pydantic import BaseModel + +MODEL = os.environ.get("MODEL", "qwen2.5:3b") +OLLAMA_BASE = os.environ.get("OLLAMA_API_BASE", "http://localhost:11434/v1") + +logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s") +logger = logging.getLogger("agent.api") + + +# === Schemas === + +class ChatRequest(BaseModel): + message: str + max_tokens: int = 300 + + +class ChatResponse(BaseModel): + request_id: str + answer: str + latency_ms: float + model: str + + +# === Agent === + +def agent_call(message: str, max_tokens: int, llm: Any = None) -> str: + llm = llm or OpenAI(base_url=OLLAMA_BASE, api_key="ollama") + resp = llm.chat.completions.create( + model=MODEL, + max_tokens=max_tokens, + messages=[{"role": "user", "content": message}], + ) + return resp.choices[0].message.content or "" + + +# === FastAPI app === + +app = FastAPI(title="Agent API", version="0.1.0") + + +@app.get("/health") +def health(): + """K8s / load balancer 用這個探測。Production 通常多檢查：DB / cache / upstream LLM ping。""" + return {"status": "ok", "model": MODEL} + + +@app.post("/chat", response_model=ChatResponse) +def chat(req: ChatRequest): + request_id = str(uuid.uuid4()) + t0 = time.perf_counter() + logger.info(f"[{request_id}] chat request: {req.message[:80]}") + try: + answer = agent_call(req.message, req.max_tokens) + except APIConnectionError as e: + logger.error(f"[{request_id}] upstream LLM unavailable: {e}") + raise HTTPException(status_code=503, detail="LLM service unavailable") from e + except Exception as e: # noqa: BLE001 + logger.error(f"[{request_id}] unexpected error: {e}") + raise HTTPException(status_code=500, detail="Internal error") from e + + latency_ms = (time.perf_counter() - t0) * 1000 + logger.info(f"[{request_id}] chat done in {latency_ms:.0f}ms") + return ChatResponse( + request_id=request_id, answer=answer, + latency_ms=latency_ms, model=MODEL, + ) + + +if __name__ == "__main__": + import uvicorn + uvicorn.run(app, host="0.0.0.0", port=8000) diff --git a/examples/stage-7/05-deploy/starter_anthropic.py b/examples/stage-7/05-deploy/starter_anthropic.py new file mode 100644 index 0000000..ac12a4b --- /dev/null +++ b/examples/stage-7/05-deploy/starter_anthropic.py @@ -0,0 +1,96 @@ +"""Stage 7 練習 5：Deploy — Path B（FastAPI + Anthropic）。 + +跟 starter.py 同 HTTP API、agent_call 改成 anthropic SDK。 + +跑法： + pip install -r requirements.txt + export ANTHROPIC_API_KEY=sk-ant-... + uvicorn starter_anthropic:app --reload --port 8000 + curl -X POST http://localhost:8000/chat -H 'Content-Type: application/json' -d '{"message": "hi"}' + +Production cost：1 chat ≈ $0.001（haiku、短 prompt）。 +""" + +from __future__ import annotations + +import logging +import os +import sys +import time +import uuid +from typing import Any + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic +from fastapi import FastAPI, HTTPException +from pydantic import BaseModel + +MODEL = os.environ.get("MODEL", "claude-haiku-4-5") +logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s") +logger = logging.getLogger("agent.api") + + +class ChatRequest(BaseModel): + message: str + max_tokens: int = 300 + + +class ChatResponse(BaseModel): + request_id: str + answer: str + latency_ms: float + model: str + input_tokens: int + output_tokens: int + + +def agent_call_anthropic(message: str, max_tokens: int, client: Any = None) -> dict: + client = client or anthropic.Anthropic() + resp = client.messages.create( + model=MODEL, max_tokens=max_tokens, + messages=[{"role": "user", "content": message}], + ) + return { + "answer": " ".join(b.text for b in resp.content if b.type == "text"), + "input_tokens": resp.usage.input_tokens, + "output_tokens": resp.usage.output_tokens, + } + + +app = FastAPI(title="Agent API (Anthropic)", version="0.1.0") + + +@app.get("/health") +def health(): + return {"status": "ok", "model": MODEL} + + +@app.post("/chat", response_model=ChatResponse) +def chat(req: ChatRequest): + request_id = str(uuid.uuid4()) + t0 = time.perf_counter() + logger.info(f"[{request_id}] chat request") + try: + result = agent_call_anthropic(req.message, req.max_tokens) + except anthropic.APIConnectionError as e: + raise HTTPException(status_code=503, detail="LLM unavailable") from e + except anthropic.RateLimitError as e: + raise HTTPException(status_code=429, detail="Rate limited") from e + except Exception as e: # noqa: BLE001 + logger.error(f"[{request_id}] error: {e}") + raise HTTPException(status_code=500, detail="Internal error") from e + + latency_ms = (time.perf_counter() - t0) * 1000 + logger.info(f"[{request_id}] done in {latency_ms:.0f}ms, tokens={result['input_tokens']}+{result['output_tokens']}") + return ChatResponse( + request_id=request_id, answer=result["answer"], + latency_ms=latency_ms, model=MODEL, + input_tokens=result["input_tokens"], output_tokens=result["output_tokens"], + ) + + +if __name__ == "__main__": + import uvicorn + uvicorn.run(app, host="0.0.0.0", port=8000) diff --git a/examples/stage-7/05-deploy/test.py b/examples/stage-7/05-deploy/test.py new file mode 100644 index 0000000..9444fb0 --- /dev/null +++ b/examples/stage-7/05-deploy/test.py @@ -0,0 +1,83 @@ +"""Stage 7 練習 5 自我驗證 — FastAPI endpoint + error handling。 +""" + +from __future__ import annotations + +import sys +from types import SimpleNamespace +from unittest.mock import MagicMock, patch + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from fastapi.testclient import TestClient +from openai import APIConnectionError + +from starter import app, agent_call + +client = TestClient(app) + + +def test_health_endpoint(): + r = client.get("/health") + assert r.status_code == 200 + body = r.json() + assert body["status"] == "ok" + assert "model" in body + print("✅ test_health_endpoint") + + +def test_chat_endpoint_happy_path(): + """Mock 上游 LLM、確認 /chat 回 ChatResponse shape。""" + fake_resp = SimpleNamespace( + choices=[SimpleNamespace(message=SimpleNamespace(content="Hello!"))] + ) + with patch("starter.OpenAI") as mock_openai: + mock_openai.return_value.chat.completions.create.return_value = fake_resp + r = client.post("/chat", json={"message": "hi"}) + assert r.status_code == 200 + body = r.json() + assert body["answer"] == "Hello!" + assert "request_id" in body + assert body["latency_ms"] >= 0 + print("✅ test_chat_endpoint_happy_path") + + +def test_chat_returns_503_on_upstream_unavailable(): + """模擬 Ollama down、回 503 而非 500。""" + with patch("starter.OpenAI") as mock_openai: + mock_openai.return_value.chat.completions.create.side_effect = APIConnectionError( + request=MagicMock() + ) + r = client.post("/chat", json={"message": "hi"}) + assert r.status_code == 503 + assert "unavailable" in r.json()["detail"].lower() + print("✅ test_chat_returns_503_on_upstream_unavailable") + + +def test_chat_validates_input_schema(): + """缺 message field、應該 422 Unprocessable Entity（FastAPI 自動驗 Pydantic schema）。""" + r = client.post("/chat", json={}) + assert r.status_code == 422 + print("✅ test_chat_validates_input_schema") + + +def test_agent_call_unit(): + """agent_call function 單元測試（無 HTTP）。""" + fake_resp = SimpleNamespace( + choices=[SimpleNamespace(message=SimpleNamespace(content="ok"))] + ) + llm = MagicMock() + llm.chat.completions.create.return_value = fake_resp + out = agent_call("hi", max_tokens=100, llm=llm) + assert out == "ok" + print("✅ test_agent_call_unit") + + +if __name__ == "__main__": + test_health_endpoint() + test_chat_endpoint_happy_path() + test_chat_returns_503_on_upstream_unavailable() + test_chat_validates_input_schema() + test_agent_call_unit() + print("\n🎉 全部通過 — FastAPI agent endpoint 邏輯正確") diff --git a/examples/stage-7/05-deploy/test_anthropic.py b/examples/stage-7/05-deploy/test_anthropic.py new file mode 100644 index 0000000..d3a0921 --- /dev/null +++ b/examples/stage-7/05-deploy/test_anthropic.py @@ -0,0 +1,56 @@ +"""Stage 7 練習 5 — Anthropic FastAPI endpoint test。""" + +from __future__ import annotations + +import sys +from types import SimpleNamespace +from unittest.mock import MagicMock, patch + +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic +from fastapi.testclient import TestClient + +from starter_anthropic import app + +client = TestClient(app) + + +def test_health_anthropic(): + r = client.get("/health") + assert r.status_code == 200 + print("✅ test_health_anthropic") + + +def test_chat_anthropic_happy_path(): + fake_resp = SimpleNamespace( + content=[SimpleNamespace(type="text", text="Hi!")], + usage=SimpleNamespace(input_tokens=10, output_tokens=2), + ) + with patch("starter_anthropic.anthropic.Anthropic") as mock_cls: + mock_cls.return_value.messages.create.return_value = fake_resp + r = client.post("/chat", json={"message": "hello"}) + assert r.status_code == 200 + body = r.json() + assert body["answer"] == "Hi!" + assert body["input_tokens"] == 10 + assert body["output_tokens"] == 2 + print("✅ test_chat_anthropic_happy_path") + + +def test_chat_anthropic_429_on_rate_limit(): + with patch("starter_anthropic.anthropic.Anthropic") as mock_cls: + mock_cls.return_value.messages.create.side_effect = anthropic.RateLimitError( + message="rate limited", response=MagicMock(status_code=429), body=None, + ) + r = client.post("/chat", json={"message": "hi"}) + assert r.status_code == 429 + print("✅ test_chat_anthropic_429_on_rate_limit") + + +if __name__ == "__main__": + test_health_anthropic() + test_chat_anthropic_happy_path() + test_chat_anthropic_429_on_rate_limit() + print("\n🎉 通過 — Anthropic FastAPI endpoint 正確") diff --git a/index.en.md b/index.en.md new file mode 100644 index 0000000..31c8970 --- /dev/null +++ b/index.en.md @@ -0,0 +1,123 @@ +--- +hide: + - navigation + - toc +--- + +
+github.com / WenyuChiou / awesome-agentic-ai-zh + +# The AI Agent learning map + +

From "what is an LLM and how are tokens counted" all the way to building your own multi-agent systems.

+ +
+[:material-rocket-launch: Start learning](stages/00-foundations.md){ .md-button .md-button--primary } +[:material-map-outline: Full roadmap](ROADMAP.md){ .md-button } +
+ +
繁體中文简体中文English
+
+ +
+
8stages
+
240+projects
+
23exercises
+
3languages
+
+ +## Pick a learning track + +
+ +- :material-console:{ .lg .middle } __Track A — CLI power user__ + + --- + + Push CLI agents like Claude Code to their limit: workflows, productionization. + + [:octicons-arrow-right-24: Start at A1](tracks/cli/A1-cli-intro.md) + +- :material-robot:{ .lg .middle } __Track B — Agent builder__ + + --- + + From tool calls all the way to multi-agent systems and your own MCP server. + + [:octicons-arrow-right-24: Start at Stage 3](stages/03-tool-use-and-hello-agent.md) + +
+ +## Eight stages, step by step + +
+ +- :material-message-text:{ .lg .middle } __Stage 1 — LLM basics__ + + --- + + Tokens, context, choosing a model. + + [:octicons-arrow-right-24: Open](stages/01-llm-basics.md) + +- :material-pencil:{ .lg .middle } __Stage 2 — Prompt engineering__ + + --- + + Say what you want so the model delivers reliably. + + [:octicons-arrow-right-24: Open](stages/02-prompt-engineering.md) + +- :material-tools:{ .lg .middle } __Stage 3 — Tool use__ + + --- + + Give the LLM tools; write your first agent. + + [:octicons-arrow-right-24: Open](stages/03-tool-use-and-hello-agent.md) + +- :material-view-grid:{ .lg .middle } __Stage 4 — Agent frameworks__ + + --- + + LangGraph, AutoGen, Agents SDK — which to pick. + + [:octicons-arrow-right-24: Open](stages/04-agent-frameworks.md) + +- :material-console:{ .lg .middle } __Stage 5 — Claude Code ecosystem__ + + --- + + CLI agents, MCP, Skills, subagents. + + [:octicons-arrow-right-24: Open](stages/05-claude-code-ecosystem.md) + +- :material-database:{ .lg .middle } __Stage 6 — Memory & RAG__ + + --- + + Let an agent remember and retrieve. + + [:octicons-arrow-right-24: Open](stages/06-memory-rag.md) + +- :material-account-group:{ .lg .middle } __Stage 7 — Multi-agent__ + + --- + + Harness, multi-agent collaboration, production. + + [:octicons-arrow-right-24: Open](stages/07-multi-agent-production.md) + +- :material-power-plug:{ .lg .middle } __Stage 8 — Agent interfaces__ + + --- + + Computer Use, Browser Use, Sandbox. + + [:octicons-arrow-right-24: Open](stages/08-agent-interfaces.md) + +
+ +--- + +Trilingual, with hands-on exercises in every stage. For the full intro and table of contents → [project overview](README.md). diff --git a/index.md b/index.md new file mode 100644 index 0000000..64c87c2 --- /dev/null +++ b/index.md @@ -0,0 +1,123 @@ +--- +hide: + - navigation + - toc +--- + +
+github.com / WenyuChiou / awesome-agentic-ai-zh + +# AI Agent 學習地圖 + +

從「LLM 是什麼、token 怎麼算」,一路走到自己打造多 agent 系統。

+ +
+[:material-rocket-launch: 開始學習](stages/00-foundations.md){ .md-button .md-button--primary } +[:material-map-outline: 完整路線圖](ROADMAP.md){ .md-button } +
+ +
繁體中文简体中文English
+
+ +
+
8階段
+
240+精選專案
+
23動手練習
+
3語言
+
+ +## 選一條學習路線 + +
+ +- :material-console:{ .lg .middle } __Track A — CLI 高手__ + + --- + + 把 Claude Code 這類 CLI agent 用到極致:工作流、production 化。 + + [:octicons-arrow-right-24: 從 A1 開始](tracks/cli/A1-cli-intro.md) + +- :material-robot:{ .lg .middle } __Track B — Agent 建構者__ + + --- + + 從工具呼叫,一路寫到多 agent 協作、寫自己的 MCP server。 + + [:octicons-arrow-right-24: 從 Stage 3 開始](stages/03-tool-use-and-hello-agent.md) + +
+ +## 8 個階段,循序漸進 + +
+ +- :material-message-text:{ .lg .middle } __Stage 1 — LLM 基礎__ + + --- + + token、context、模型怎麼挑。 + + [:octicons-arrow-right-24: 進入](stages/01-llm-basics.md) + +- :material-pencil:{ .lg .middle } __Stage 2 — Prompt 設計__ + + --- + + 把需求講清楚,讓模型穩定產出。 + + [:octicons-arrow-right-24: 進入](stages/02-prompt-engineering.md) + +- :material-tools:{ .lg .middle } __Stage 3 — 工具呼叫__ + + --- + + 讓 LLM 會用工具,寫出第一個 agent。 + + [:octicons-arrow-right-24: 進入](stages/03-tool-use-and-hello-agent.md) + +- :material-view-grid:{ .lg .middle } __Stage 4 — Agent 框架__ + + --- + + LangGraph、AutoGen、Agents SDK 怎麼選。 + + [:octicons-arrow-right-24: 進入](stages/04-agent-frameworks.md) + +- :material-console:{ .lg .middle } __Stage 5 — Claude Code 生態__ + + --- + + CLI agent、MCP、Skills、subagent。 + + [:octicons-arrow-right-24: 進入](stages/05-claude-code-ecosystem.md) + +- :material-database:{ .lg .middle } __Stage 6 — 記憶 / RAG__ + + --- + + 讓 agent 記得住、查得到。 + + [:octicons-arrow-right-24: 進入](stages/06-memory-rag.md) + +- :material-account-group:{ .lg .middle } __Stage 7 — 多 Agent__ + + --- + + harness、多 agent 協作、production 化。 + + [:octicons-arrow-right-24: 進入](stages/07-multi-agent-production.md) + +- :material-power-plug:{ .lg .middle } __Stage 8 — Agent 介面__ + + --- + + Computer Use、Browser Use、Sandbox。 + + [:octicons-arrow-right-24: 進入](stages/08-agent-interfaces.md) + +
+ +--- + +三語對照、每階段附動手練習。想看完整介紹與目錄 → [專案說明](README.md)。 diff --git a/index.zh-Hans.md b/index.zh-Hans.md new file mode 100644 index 0000000..cf11ed8 --- /dev/null +++ b/index.zh-Hans.md @@ -0,0 +1,123 @@ +--- +hide: + - navigation + - toc +--- + +
+github.com / WenyuChiou / awesome-agentic-ai-zh + +# AI Agent 学习地图 + +

从“LLM 是什么、token 怎么算”,一路走到自己打造多 agent 系统。

+ +
+[:material-rocket-launch: 开始学习](stages/00-foundations.md){ .md-button .md-button--primary } +[:material-map-outline: 完整路线图](ROADMAP.md){ .md-button } +
+ +
繁體中文简体中文English
+
+ +
+
8阶段
+
240+精选项目
+
23动手练习
+
3语言
+
+ +## 选一条学习路线 + +
+ +- :material-console:{ .lg .middle } __Track A — CLI 高手__ + + --- + + 把 Claude Code 这类 CLI agent 用到极致:工作流、production 化。 + + [:octicons-arrow-right-24: 从 A1 开始](tracks/cli/A1-cli-intro.md) + +- :material-robot:{ .lg .middle } __Track B — Agent 建构者__ + + --- + + 从工具调用,一路写到多 agent 协作、写自己的 MCP server。 + + [:octicons-arrow-right-24: 从 Stage 3 开始](stages/03-tool-use-and-hello-agent.md) + +
+ +## 8 个阶段,循序渐进 + +
+ +- :material-message-text:{ .lg .middle } __Stage 1 — LLM 基础__ + + --- + + token、context、模型怎么挑。 + + [:octicons-arrow-right-24: 进入](stages/01-llm-basics.md) + +- :material-pencil:{ .lg .middle } __Stage 2 — Prompt 设计__ + + --- + + 把需求讲清楚,让模型稳定产出。 + + [:octicons-arrow-right-24: 进入](stages/02-prompt-engineering.md) + +- :material-tools:{ .lg .middle } __Stage 3 — 工具调用__ + + --- + + 让 LLM 会用工具,写出第一个 agent。 + + [:octicons-arrow-right-24: 进入](stages/03-tool-use-and-hello-agent.md) + +- :material-view-grid:{ .lg .middle } __Stage 4 — Agent 框架__ + + --- + + LangGraph、AutoGen、Agents SDK 怎么选。 + + [:octicons-arrow-right-24: 进入](stages/04-agent-frameworks.md) + +- :material-console:{ .lg .middle } __Stage 5 — Claude Code 生态__ + + --- + + CLI agent、MCP、Skills、subagent。 + + [:octicons-arrow-right-24: 进入](stages/05-claude-code-ecosystem.md) + +- :material-database:{ .lg .middle } __Stage 6 — 记忆 / RAG__ + + --- + + 让 agent 记得住、查得到。 + + [:octicons-arrow-right-24: 进入](stages/06-memory-rag.md) + +- :material-account-group:{ .lg .middle } __Stage 7 — 多 Agent__ + + --- + + harness、多 agent 协作、production 化。 + + [:octicons-arrow-right-24: 进入](stages/07-multi-agent-production.md) + +- :material-power-plug:{ .lg .middle } __Stage 8 — Agent 界面__ + + --- + + Computer Use、Browser Use、Sandbox。 + + [:octicons-arrow-right-24: 进入](stages/08-agent-interfaces.md) + +
+ +--- + +三语对照、每阶段附动手练习。想看完整介绍与目录 → [项目说明](README.md)。 diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 0000000..f7f2c63 --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,174 @@ +site_name: awesome-agentic-ai-zh +site_description: A trilingual learning roadmap for agentic AI (English / 繁中 / 简中) — 8 stages from LLM basics to multi-agent systems, with hands-on exercises and curated projects. +site_url: https://wenyuchiou.github.io/awesome-agentic-ai-zh/ +repo_url: https://github.com/WenyuChiou/awesome-agentic-ai-zh +repo_name: WenyuChiou/awesome-agentic-ai-zh +edit_uri: edit/main/ + +# Content lives at the REPO ROOT (stages/ tracks/ branches/ resources/ +# + root *.md), not a docs/ subfolder. mkdocs forbids docs_dir being +# the parent of mkdocs.yml, so `scripts/build-docs-tree.py` stages the +# whitelisted content into _build/docs/ first (run it before `mkdocs +# build` — the GitHub Actions workflow does this automatically). +docs_dir: _build/docs +site_dir: _build/site + +extra_css: + - docs/stylesheets/extra.css + +exclude_docs: | + *.py + *.sh + +theme: + name: material + language: zh-TW + features: + - navigation.tabs + - navigation.top + - navigation.indexes + - navigation.footer + - navigation.instant + - navigation.instant.progress + - navigation.tracking + - search.suggest + - search.highlight + - search.share + - content.action.edit + - content.code.copy + - content.code.annotate + - content.tabs.link + - content.tooltips + - toc.follow + palette: + - media: "(prefers-color-scheme: light)" + scheme: default + primary: indigo + accent: indigo + toggle: + icon: material/weather-night + name: 切換深色 / Dark mode + - media: "(prefers-color-scheme: dark)" + scheme: slate + primary: indigo + accent: indigo + toggle: + icon: material/weather-sunny + name: 切換淺色 / Light mode + +plugins: + - search + - i18n: + docs_structure: suffix + fallback_to_default: true + reconfigure_material: true + reconfigure_search: true + languages: + - locale: zh-TW + default: true + name: 繁體中文 + build: true + - locale: en + name: English + build: true + nav_translations: + 首頁: Home + 專案說明: Project overview + 怎麼用: How to use + 進度: Progress + 結業專題: Capstone + 路線圖: Roadmap + 書本版: Book (mdBook) + 共用基礎: Foundations + Track A — CLI: Track A — CLI power user + Track B — Agent: Track B — Agent builder + 讀者分流: Audience branches + 實作: Walkthroughs + 資源: Resources + 貢獻: Contributing + - locale: zh-Hans + name: 简体中文 + build: true + nav_translations: + 首頁: 首页 + 專案說明: 项目说明 + 怎麼用: 怎么用 + 進度: 进度 + 結業專題: 结业专题 + 路線圖: 路线图 + 書本版: 书本版 + 共用基礎: 共用基础 + Track A — CLI: Track A — CLI 高手 + Track B — Agent: Track B — Agent 建构者 + 讀者分流: 读者分流 + 實作: 实作 + 資源: 资源 + 貢獻: 贡献 + # Material ships no `zh-Hans.html` UI partial (it predates strict + # BCP-47 — uses `zh` for Simplified). Map this locale's chrome to + # Material's `zh`. Applied via apply_user_overrides (runs AFTER + # the plugin's forced language=locale, so this wins). + theme: + language: zh + +markdown_extensions: + - admonition + - attr_list + - md_in_html + - tables + - toc: + permalink: true + - pymdownx.details + - pymdownx.superfences + - pymdownx.tasklist: + custom_checkbox: true + - pymdownx.emoji: + emoji_index: !!python/name:material.extensions.emoji.twemoji + emoji_generator: !!python/name:material.extensions.emoji.to_svg + +# Build-time hooks. mkdocs_hooks.on_page_markdown strips the README's +# GitHub-only inline language switcher (its ./README.*.md links 404 +# on the rendered site; Material's header selector is the in-site one). +hooks: + - scripts/mkdocs_hooks.py + +nav: + - 首頁: index.md + - 專案說明: about.md + - 怎麼用: docs/HOW_TO_USE.md + - 進度: PROGRESS.md + - 結業專題: CAPSTONE.md + - 路線圖: ROADMAP.md + - 書本版: https://wenyuchiou.github.io/awesome-agentic-ai-zh/book/ + - 共用基礎: + - Stage 0 — Foundations: stages/00-foundations.md + - Stage 1 — LLM Basics: stages/01-llm-basics.md + - Stage 2 — Prompt Engineering: stages/02-prompt-engineering.md + - Track A — CLI: + - A1 — CLI Intro: tracks/cli/A1-cli-intro.md + - A2 — CLI Workflow: tracks/cli/A2-cli-workflow.md + - A3 — CLI Production: tracks/cli/A3-cli-production.md + - Track B — Agent: + - Stage 3 — Tool Use & Hello Agent: stages/03-tool-use-and-hello-agent.md + - Stage 4 — Agent Frameworks: stages/04-agent-frameworks.md + - Stage 5 — Claude Code Ecosystem: stages/05-claude-code-ecosystem.md + - Stage 6 — Memory & RAG: stages/06-memory-rag.md + - Stage 7 — Multi-Agent: stages/07-multi-agent-production.md + - Stage 7.5 — Advanced Agentic: stages/07.5-advanced-agentic-concepts.md + - Stage 8 — Agent Interfaces: stages/08-agent-interfaces.md + - 讀者分流: + - Branch design: branches/DESIGN.md + - for-researcher: branches/for-researcher.md + - for-developer: branches/for-developer.md + - for-teacher: branches/for-teacher.md + - for-knowledge-worker: branches/for-knowledge-worker.md + - for-everyday-users: branches/for-everyday-users.md + - 實作: + - Build your first agent in 7 steps: walkthroughs/build-first-agent-in-7-steps.md + - 資源: RESOURCES.md + - 貢獻: + - Contributing guide: CONTRIBUTING.md + - Code of Conduct: CODE_OF_CONDUCT.md + - Security policy: SECURITY.md + - Contributors: CONTRIBUTORS.md + - Changelog: CHANGELOG.md diff --git a/requirements-docs.txt b/requirements-docs.txt new file mode 100644 index 0000000..f123fa2 --- /dev/null +++ b/requirements-docs.txt @@ -0,0 +1,6 @@ +# Docs site (GitHub Pages) build dependencies — used by .github/workflows/docs.yml +# and for local preview: pip install -r requirements-docs.txt && mkdocs serve +# Explicit base pin so a future mkdocs 2.x can't silently break CI. +mkdocs>=1.6,<2 +mkdocs-material==9.5.44 +mkdocs-static-i18n==1.2.3 diff --git a/resources/README.en.md b/resources/README.en.md new file mode 100644 index 0000000..4dc87c6 --- /dev/null +++ b/resources/README.en.md @@ -0,0 +1,100 @@ +# `resources/` Index + +
+ 繁體中文 | 简体中文 | English +
+ +> This is the repo's **reference area**: supplementary material that sits outside the main path and is meant to be opened when needed. Each file has a distinct role. + +--- + +## 7 References + When to Read Each + +| File | Role | When to Read | Lines | +|---|---|---|---| +| [`glossary.en.md`](glossary.en.md) | **30-second term lookup** | You hit terms like LLM / RAG / token / agent / vector DB / streaming / batch API while reading a stage | ~210 | +| [`cli-agents-guide.en.md`](cli-agents-guide.en.md) | **7 CLI agents compared** | First time choosing among Claude Code / Codex / OpenCode / Gemini CLI / goose / Aider / Hermes Agent | ~134 | +| [`mcp-skills-catalog.en.md`](mcp-skills-catalog.en.md) | **65+ integration catalog** | You want Claude Code connected to Notion / Obsidian / Excel / Postgres / Slack / other real tools | ~775 | +| [`schema-design-cheatsheet.en.md`](schema-design-cheatsheet.en.md) | **5 function-schema rules + 5 anti-patterns** | You are writing a tool schema / MCP server schema / function calling and the LLM picks the wrong tool or arguments | ~159 | +| [`cookbook.en.md`](cookbook.en.md) | **6 step-by-step recipes** | You want to build a first Skill / MCP server / Office integration / NotebookLM flow / Zotero flow / local LLM in 30-50 minutes | ~620 | +| [`setup-guide.en.md`](setup-guide.en.md) | **From-zero setup guide** | No dev background; first time creating an API key, installing Python, or using Claude Code | ~400 | +| [`style-guide.en.md`](style-guide.en.md) | **Format and wording rules before PRs** | You want to contribute to the repo, add entries, or improve translations | ~338 | + +Together these are about ~2500 lines of reference. That sounds large, but **each file is read at a different moment**. You do not read all of them at once; you open the relevant one for 30 seconds to 45 minutes. + +--- + +## Entry Points: "What Am I Trying to Do?" + +### 🆕 I Have Never Written Code / This Is My First AI Agent Setup + +→ [`setup-guide.en.md`](setup-guide.en.md) (30-45 minutes from zero) + +### 🆕 I Am Just Starting to Learn AI Agents + +You do not need any reference first. **Start with the main [README](../README.en.md) → [Stage 0](../stages/00-foundations.en.md)**. When a term is unclear, come back to [`glossary.en.md`](glossary.en.md). + +### 🛠 I Need to Choose a CLI Agent + +→ [`cli-agents-guide.en.md`](cli-agents-guide.en.md) (CLI comparison + recommendations by use case) + +### 🔌 I Want to Connect Claude Code to Tool X (Notion / Excel / Postgres / etc.) + +→ [`mcp-skills-catalog.en.md`](mcp-skills-catalog.en.md) (65+ integrations in 16 categories) + +### 🍳 I Want to Build My First Skill / MCP Server / Word Integration + +→ [`cookbook.en.md`](cookbook.en.md) (6 step-by-step recipes) + +### 📐 I Wrote a Tool Schema and the LLM Is Not Following It + +→ [`schema-design-cheatsheet.en.md`](schema-design-cheatsheet.en.md) (5 rules + 5 anti-patterns) + +### 📚 I Hit an Unclear Term While Reading a Stage + +→ [`glossary.en.md`](glossary.en.md) (30-80 words per term + which stage goes deeper) + +### 🤝 I Want to Send a PR / Translate / Add a New Entry + +→ [`style-guide.en.md`](style-guide.en.md) + [`../CONTRIBUTING.en.md`](../CONTRIBUTING.en.md) + +--- + +## Duplication? + +Duplication is intentional only where it helps navigation. The roles stay separate: + +- **glossary** is a 30-second lookup, stage text is a 3-5 minute read, and cookbook is a 30-50 minute build. +- **schema-design-cheatsheet** overlaps with cookbook 2, but the cheatsheet explains schema rules while the cookbook gets a server running. +- **cli-agents-guide** is a comparison reference; **mcp-skills-catalog** is a tool integration catalog. +- **setup-guide** is for people starting from zero; Stage 0 assumes you are ready to follow a learning path. + +--- + +## Language Coverage + +| File | zh-TW (canonical) | zh-Hans | English | +|---|---|---|---| +| glossary | ✅ | ✅ | ✅ | +| cli-agents-guide | ✅ | ✅ | ✅ | +| mcp-skills-catalog | ✅ | ✅ | ✅ | +| schema-design-cheatsheet | ✅ | ✅ | ✅ | +| cookbook | ✅ | ✅ | ✅ | +| setup-guide | ✅ | ✅ | ✅ | +| style-guide | ✅ | ✅ | ✅ | + +--- + +## Standards for Adding a New Reference + +A new reference file should not be added casually. It must: + +1. **Avoid duplicating any existing role** in the table above. +2. **Solve a problem the main path cannot cover well**. If 50 lines in Stage X would cover it, put it in that stage. +3. **Be expected to receive 3+ cross-references** from stages or branches. If it only serves one stage, keep it in that stage. + +Possible future references: + +- `cost-calculator-guide.md`: cross-provider pricing. Stage 1 covers enough for now. +- `troubleshooting-guide.md`: common error runbook. Existing material is enough until more community reports arrive. +- `prompt-patterns-guide.md`: CoT / few-shot template library. Stage 2 already covers the basics; a deeper version can wait for community PRs. diff --git a/resources/README.md b/resources/README.md new file mode 100644 index 0000000..4778c48 --- /dev/null +++ b/resources/README.md @@ -0,0 +1,101 @@ +# `resources/` 索引 + +
+ 繁體中文 | 简体中文 | English +
+ +> 本 repo 的「**reference 區**」——非主路線、需要時來查的補充材料。每份檔案有明確角色，不重複。 + +--- + +## 7 份 reference + 各自的「什麼時候看」 + +| 檔案 | 角色 | 什麼時候看 | 行數 | +|---|---|---|---| +| [`glossary.md`](glossary.md) | **30 秒查詞典** | 看 stage 內文時遇到 LLM / RAG / token / agent / vector DB / streaming / batch API 不知道什麼意思 | ~210 | +| [`cli-agents-guide.md`](cli-agents-guide.md) | **7 個 CLI agent 比較** | 第一次選 CLI agent（Claude Code / Codex / OpenCode / Gemini CLI / goose / Aider / Hermes Agent）不知道挑哪個 | ~134 | +| [`mcp-skills-catalog.md`](mcp-skills-catalog.md) | **65+ 個整合 catalog** | 想把 Claude Code 接 Notion / Obsidian / Excel / Postgres / Slack / 等等實際工具 | ~775 | +| [`schema-design-cheatsheet.md`](schema-design-cheatsheet.md) | **function schema 設計 5 規則 + 5 anti-pattern** | 寫 tool schema / MCP server schema / function calling，發現 LLM 選錯 tool / 傳錯參數 | ~159 | +| [`cookbook.md`](cookbook.md) | **6 個 step-by-step recipe** | 想 30-50 分鐘做出第一個 Skill / MCP server / 接 Office / 接 NotebookLM / 接 Zotero / 接本機 LLM | ~620 | +| [`setup-guide.md`](setup-guide.md) | **從零開始的 setup 指南** | 完全沒 dev 背景、第一次申請 API key / 裝 Python / 用 Claude Code | ~400 | +| [`style-guide.md`](style-guide.md) | **送 PR 前的格式 / 用詞規範** | 要對 repo 貢獻、寫 entry / 翻譯 | ~338 | + +合計 ~2500 行 reference。看起來不少，但**每份檔案讀的時機不同**——你不會一次全讀，只在對應情境查 30 秒到 45 分鐘。 + +--- + +## 怎麼進來：以「我現在要做什麼」分類 + +### 🆕 我完全沒寫過 code / 第一次接觸 AI agent + +→ [`setup-guide.md`](setup-guide.md)（30-45 分鐘從零裝好） + +### 🆕 我剛開始學 AI agent + +不需要先讀任何 reference。**直接從主路線 [README](../README.md) → [Stage 0](../stages/00-foundations.md) 開始**。遇到不懂的詞回來查 [`glossary.md`](glossary.md) 就好。 + +### 🛠 我要選 CLI agent + +→ [`cli-agents-guide.md`](cli-agents-guide.md)（6 個 CLI 比較 + 依 use case 推薦） + +### 🔌 我要把 Claude Code 接 X 工具（Notion / Excel / Postgres 等） + +→ [`mcp-skills-catalog.md`](mcp-skills-catalog.md)（65+ 個整合分 16 類） + +### 🍳 我想動手寫第一個 Skill / MCP server / 接 Word 等 + +→ [`cookbook.md`](cookbook.md)（6 個 step-by-step recipe） + +### 📐 我寫 tool schema 但 LLM 不照我意思做 + +→ [`schema-design-cheatsheet.md`](schema-design-cheatsheet.md)（5 規則 + 5 anti-pattern） + +### 📚 我看 stage 內文遇到不懂的詞 + +→ [`glossary.md`](glossary.md)（每詞 30-80 字解釋 + 哪 stage 講細） + +### 🤝 我想送 PR / 翻譯 / 加新 entry + +→ [`style-guide.md`](style-guide.md) + [`../CONTRIBUTING.md`](../CONTRIBUTING.md) + +--- + +## 重複 / 重疊？ + +刻意避免重複。每份 reference 跟主路線 / 其他 reference 的關係： + +- **glossary** 是「30 秒查」、stage 內文是「3-5 分鐘讀」、cookbook 是「30-50 分鐘做」——三層深度，不重疊 +- **schema-design-cheatsheet** 跟 cookbook 2（寫 MCP server）有交集——cheatsheet 講「**寫 schema 的規則**」、cookbook 講「**怎麼把 server 跑起來**」。看哪個取決於你卡在哪 +- **cli-agents-guide** 是 reference table；catalog 是 plug-in tools——兩個層級不同 +- **跟 [Hello-Agents](https://github.com/datawhalechina/hello-agents) 的關係**：Hello-Agents 是中文圈最完整的 agent 教材，深度高。我們是「**roadmap + curated catalog + 動手 recipe**」的角度，不取代它。Stage 5.3 / cookbook 1 都明確 cross-ref Hello-Agents Extra08「如何寫出好的 Skill」當深度補充 + +--- + +## 三語覆蓋 + +| 檔案 | zh-TW（canonical） | zh-Hans | English | +|---|---|---|---| +| glossary | ✅ | ✅ | ✅ | +| cli-agents-guide | ✅ | ✅ | ✅ | +| mcp-skills-catalog | ✅ | ✅ | ✅ | +| schema-design-cheatsheet | ✅ | ✅ | ✅ | +| cookbook | ✅ | ✅ | ✅ | +| setup-guide | ✅ | ✅ | ✅ | +| style-guide | ✅ | ✅ | ✅ | + +--- + +## 加新 reference 檔的標準 + +不是隨便加。新檔案必須： + +1. **不重複既有任何檔的角色**（看上面的「角色」欄） +2. **解決 main path 解決不了的問題**——如果 Stage X 的內文加 50 行就能 cover，就放 stage 裡別開新檔 +3. **預期會被 ≥ 3 個 stage 或 branch cross-ref**——只服務一個 stage 的內容，放那個 stage 就好 + +近期考慮過、但**沒加**的（可選 future work）： +- `cost-calculator-guide.md`（cross-provider 計價）—— 現在 [Stage 1](../stages/01-llm-basics.md) 有提到，等需求明顯再開 +- `troubleshooting-guide.md`（常見錯誤 runbook）—— 現有資料夠應付，等社群回報多了再開 +- `prompt-patterns-guide.md`（CoT / few-shot 範本庫）—— 現在 [Stage 2](../stages/02-prompt-engineering.md) 有，深度版等社群 PR + +社群想加可以開 issue 討論。 diff --git a/resources/README.zh-Hans.md b/resources/README.zh-Hans.md new file mode 100644 index 0000000..2e8a6c0 --- /dev/null +++ b/resources/README.zh-Hans.md @@ -0,0 +1,100 @@ +# `resources/` 索引 + +
+ 繁體中文 | 简体中文 | English +
+ +> 这是本 repo 的 **reference 区**：不在主路线里、需要时再查的补充材料。每份文件都有明确角色，不重复。 + +--- + +## 7 份 reference + 各自的“什么时候看” + +| 文件 | 角色 | 什么时候看 | 行数 | +|---|---|---|---| +| [`glossary.zh-Hans.md`](glossary.zh-Hans.md) | **30 秒查词典** | 看 stage 内容时遇到 LLM / RAG / token / agent / vector DB / streaming / batch API 不知道什么意思 | ~210 | +| [`cli-agents-guide.zh-Hans.md`](cli-agents-guide.zh-Hans.md) | **7 个 CLI agent 比较** | 第一次选 CLI agent（Claude Code / Codex / OpenCode / Gemini CLI / goose / Aider / Hermes Agent）不知道挑哪个 | ~134 | +| [`mcp-skills-catalog.zh-Hans.md`](mcp-skills-catalog.zh-Hans.md) | **65+ 个集成 catalog** | 想把 Claude Code 接 Notion / Obsidian / Excel / Postgres / Slack / 等实际工具 | ~775 | +| [`schema-design-cheatsheet.zh-Hans.md`](schema-design-cheatsheet.zh-Hans.md) | **function schema 设计 5 规则 + 5 anti-pattern** | 写 tool schema / MCP server schema / function calling，发现 LLM 选错 tool / 传错参数 | ~159 | +| [`cookbook.zh-Hans.md`](cookbook.zh-Hans.md) | **6 个 step-by-step recipe** | 想 30-50 分钟做出第一个 Skill / MCP server / 接 Office / 接 NotebookLM / 接 Zotero / 接本机 LLM | ~620 | +| [`setup-guide.zh-Hans.md`](setup-guide.zh-Hans.md) | **从零开始的 setup 指南** | 完全没有 dev 背景、第一次申请 API key / 装 Python / 用 Claude Code | ~400 | +| [`style-guide.zh-Hans.md`](style-guide.zh-Hans.md) | **送 PR 前的格式 / 用词规范** | 要对 repo 贡献、写 entry / 翻译 | ~338 | + +合计约 ~2500 行 reference。看起来不少，但**每份文件阅读的时机不同**——你不会一次全读，只在对应场景查 30 秒到 45 分钟。 + +--- + +## 怎么进来：按“我现在要做什么”分类 + +### 🆕 我完全没写过 code / 第一次接触 AI agent + +→ [`setup-guide.zh-Hans.md`](setup-guide.zh-Hans.md)（30-45 分钟从零装好） + +### 🆕 我刚开始学 AI agent + +不需要先读任何 reference。**直接从主路线 [README](../README.zh-Hans.md) → [Stage 0](../stages/00-foundations.zh-Hans.md) 开始**。遇到不懂的词回来查 [`glossary.zh-Hans.md`](glossary.zh-Hans.md) 就好。 + +### 🛠 我要选 CLI agent + +→ [`cli-agents-guide.zh-Hans.md`](cli-agents-guide.zh-Hans.md)（CLI 比较 + 按 use case 推荐） + +### 🔌 我要把 Claude Code 接 X 工具（Notion / Excel / Postgres 等） + +→ [`mcp-skills-catalog.zh-Hans.md`](mcp-skills-catalog.zh-Hans.md)（65+ 个集成分 16 类） + +### 🍳 我想动手写第一个 Skill / MCP server / 接 Word 等 + +→ [`cookbook.zh-Hans.md`](cookbook.zh-Hans.md)（6 个 step-by-step recipe） + +### 📐 我写 tool schema 但 LLM 不照我意思做 + +→ [`schema-design-cheatsheet.zh-Hans.md`](schema-design-cheatsheet.zh-Hans.md)（5 规则 + 5 anti-pattern） + +### 📚 我看 stage 内容遇到不懂的词 + +→ [`glossary.zh-Hans.md`](glossary.zh-Hans.md)（每个词 30-80 字解释 + 哪个 stage 讲细） + +### 🤝 我想送 PR / 翻译 / 加新 entry + +→ [`style-guide.zh-Hans.md`](style-guide.zh-Hans.md) + [`../CONTRIBUTING.md`](../CONTRIBUTING.md) + +--- + +## 重复 / 重叠？ + +只在有助于导航时保留少量重叠，各文件角色仍然分开： + +- **glossary** 是“30 秒查”，stage 内容是“3-5 分钟读”，cookbook 是“30-50 分钟做”。 +- **schema-design-cheatsheet** 跟 cookbook 2 有交集，但 cheatsheet 讲 schema 规则，cookbook 讲怎么把 server 跑起来。 +- **cli-agents-guide** 是比较 reference；**mcp-skills-catalog** 是工具集成目录。 +- **setup-guide** 给从零开始的人；Stage 0 默认你已经准备好进入学习路线。 + +--- + +## 三语覆盖 + +| 文件 | zh-TW（canonical） | zh-Hans | English | +|---|---|---|---| +| glossary | ✅ | ✅ | ✅ | +| cli-agents-guide | ✅ | ✅ | ✅ | +| mcp-skills-catalog | ✅ | ✅ | ✅ | +| schema-design-cheatsheet | ✅ | ✅ | ✅ | +| cookbook | ✅ | ✅ | ✅ | +| setup-guide | ✅ | ✅ | ✅ | +| style-guide | ✅ | ✅ | ✅ | + +--- + +## 加新 reference 文件的标准 + +不是随便加。新文件必须： + +1. **不重复既有任何文件的角色**（看上面的“角色”栏）。 +2. **解决 main path 解决不了的问题**。如果 Stage X 的内容加 50 行就能 cover，就放 stage 里别开新文件。 +3. **预期会被 3 个以上 stage 或 branch cross-ref**。只服务一个 stage 的内容，放那个 stage 就好。 + +近期考虑过但没加的： + +- `cost-calculator-guide.md`：cross-provider 计价。现在 Stage 1 有提到，等需求明显再开。 +- `troubleshooting-guide.md`：常见错误 runbook。现有资料够应付，等社群反馈多了再开。 +- `prompt-patterns-guide.md`：CoT / few-shot 模板库。现在 Stage 2 有基础内容，深度版等社群 PR。 diff --git a/resources/agent-paradigms.en.md b/resources/agent-paradigms.en.md new file mode 100644 index 0000000..f5a9ab1 --- /dev/null +++ b/resources/agent-paradigms.en.md @@ -0,0 +1,203 @@ +> [繁體中文](./agent-paradigms.md) | [简体中文](./agent-paradigms.zh-Hans.md) | **English** + +# 5 Agent Paradigms — Where does your agent live, and who does it serve? + +> [← Back to main README](../README.en.md) + +> 📌 **This is a mental-model reference**. After reading you'll understand: "Why do Claude Code, Hermes Agent, and OpenClaw all call themselves 'agents' but feel completely different to use?" +> If you already know which one you want → [`resources/cli-agents-guide.en.md`](cli-agents-guide.en.md) (7-CLI side-by-side comparison) or [`resources/cookbook.en.md`](cookbook.en.md) (step-by-step deployment). + +The word "agent" gets used loosely. Cursor is an agent. Claude Code is an agent. Hermes Agent — the one chatting with you on Telegram — is an agent. OpenClaw running on a Jetson board in your apartment is also an agent. But these four feel completely different in practice — because they belong to **different paradigms**. The difference isn't which LLM family they use; it's **where the agent runs, what interface you use to talk to it, and whether it needs an internet connection**. + +Once you understand the paradigm, moving a use case from Type 2 to Type 4 isn't "switching tools" — it's **switching how you think**. + +--- + +## One table to anchor everything + +| Type | Examples | Where the agent runs | How you reach it | LLM | Offline OK? | Monthly cost (rough) | +|---|---|---|---|---|---|---| +| **1. IDE-coupled** | Cursor / Cline / Continue | Inside your IDE | IDE sidebar | Multi-provider | ❌ | $0-20 | +| **2. Terminal pair-programmer** | Claude Code / Codex / Gemini CLI | Your terminal | Terminal REPL | Single-family | ❌ | $20 sub or API | +| **3. BYO-LLM CLI** | Aider / OpenCode / goose | Your terminal | Terminal REPL | Bring your own API key | ❌ | API usage | +| **4. Cloud-deployed** | **Hermes Agent** | $5 VPS / Modal | **Telegram / Slack / any chat app** | 200+ provider routing | ❌ | $5 server + API | +| **5. Edge-deployed** | **OpenClaw / ClawBox** | Jetson board / Raspberry Pi | local chat / SSH | **Local Ollama** (Qwen / Llama / Mistral) | **✅** | One-time €549, then 0 | + +→ Types 4 and 5 are both **"deployed autonomous agents"** — the agent **doesn't live on your laptop**, it runs out there 24×7 serving you. Type 4 lives in the cloud; Type 5 lives on edge hardware. Types 1-3 are **"co-located agents"** — the agent shares your laptop and stops when you walk away. + +--- + +## Type 1: IDE-coupled — "sidebar pair-programmer" + +**Examples**: [Cursor](https://cursor.com) / [Windsurf](https://codeium.com/windsurf) / [Cline](https://cline.bot) / [Continue](https://continue.dev) / [Zed](https://zed.dev) + +**Hero example**: +You're writing a React component in Cursor. Editor on the left, Cursor sidebar on the right. You select a chunk of code, hit `Cmd+K`, and Cursor rewrites it in place. You see the inline diff and accept/reject. + +**Why this type exists**: When you write code, **your eyes need to be on the code** — you can't pivot to a separate terminal for chat. IDE-coupled agents keep the LLM next to your sightline, preserving visual context. + +**Good for**: lots of small edits, less exploration; side-by-side coding; visual diffs. +**Not for**: agent autonomously running multi-step tasks (a sidebar isn't a great workspace); non-coding tasks. + +--- + +## Type 2: Terminal pair-programmer — "the Claude Code paradigm" + +**Examples**: [Claude Code](https://github.com/anthropics/claude-code) / [Codex](https://github.com/openai/codex) / [Gemini CLI](https://github.com/google-gemini/gemini-cli) + +**Hero example**: +You open Claude Code in a terminal and type "refactor the entire auth module, swap callbacks for async/await, run the tests." Claude Code reads files, edits them, runs pytest, and reports back. Five to ten minutes of streaming output, autonomous throughout. + +**Why this type exists**: Claude Code and Codex turn the whole terminal into the agent's workspace. The agent has full access to the file system, shell, and git — it can complete multi-step tasks autonomously. More autonomous than Type 1. + +**Distinguishing trait**: Subscription pricing ($20/month covers the whole month, no per-token billing); locked to a specific LLM family (Claude Code = Claude only). + +**Good for**: agentic tasks; long refactors; paper writing; anything beyond a 1-2 step prompt. +**Not for**: cost-comparing across LLM providers; non-coding/writing scenarios; offline work. + +--- + +## Type 3: BYO-LLM CLI — "Type 2's mental model, multi-provider" + +**Examples**: [Aider](https://aider.chat) / [OpenCode](https://github.com/sst/opencode) / [goose](https://block.github.io/goose) / [Hermes Agent](https://github.com/NousResearch/hermes-agent)* + +**Hero example**: +You want to use DeepSeek-R1 to write code (10× cheaper than Claude Opus). Aider with `--model deepseek/deepseek-reasoner` + an `OPENROUTER_API_KEY` does it — git-aware, automatic commit messages, the same workflow as Type 2. + +**Difference from Type 2**: Type 2 locks you into one LLM family. Type 3 takes any OpenAI-compatible endpoint with your own API key. + +**Distinguishing trait**: cost-sensitive; multi-provider comparison; self-hosted LLMs (Ollama / vLLM) are fair game. + +**Good for**: experimenting across LLMs; saving cost; local LLMs; not wanting vendor lock-in. +**Not for**: people who find setup intimidating (you manage API keys and provider configs). + +*Hermes Agent belongs to both Type 3 (CLI mode) and Type 4 (cloud mode) depending on how you deploy it. More detail below. + +--- + +## Type 4: Cloud-deployed — exemplar: Hermes Agent + +**Exemplar**: [Hermes Agent](https://github.com/NousResearch/hermes-agent) (Nous Research, ★ 193k+, MIT) + +**Hero example**: +You're on the subway, you open Telegram on your phone, and you message your Hermes bot: "Pull today's arXiv ML papers, give me 3 highlights, send the result back to Telegram." Hermes — running on your $5 DigitalOcean VPS — receives the message, decides to use GPT-5 (paper discovery) + Claude Opus (summary) + Gemini Flash (compression to 3 bullet points), executes the pipeline, and sends the result back. You never touched your laptop. + +**5 distinctive features**: + +1. **Multi-platform chat interface**: Telegram / Discord / Slack / WhatsApp / Signal all work as entry points. Whichever platform you ping from is the platform the agent replies on. +2. **Multi-LLM routing (200+ model neutral)**: OpenRouter + NVIDIA NIM + Zhipu GLM + Kimi + Xiaomi MiMo + MiniMax + HF + OpenAI + Anthropic + Google. **A single conversation can span multiple LLMs**. +3. **24/7 availability**: the agent doesn't depend on your laptop; it lives on a cloud VPS, always reachable. +4. **Built-in cron**: routines like "every day at 9am, do X and send Y" are first-class. +5. **Self-improving skills** (experimental, not yet independently audited): the more you interact, the more the agent generalizes into reusable skills that accumulate across sessions. + +**Why this type exists**: When the agent is **a personal assistant** rather than a pair programmer, it shouldn't be tied to your laptop. Type 4 turns the agent into a 24×7 service. + +**Distinguishing trait**: ~$5/month VPS hosting + API costs; China-region LLM support (GLM / Kimi) — a useful backup to switch to when US services are flaky. + +**Trade-offs**: +- ⚠️ Self-improving skills are a new capability with no independent security audit yet — don't enable it for high-stakes tasks (medical / legal / payments) +- You lose IDE/terminal-style direct filesystem manipulation; you adopt a chat-first workflow +- You need self-host fluency (Linux / Docker / systemd basics) + +**Good for**: cross-platform notifications; 24/7 routines (daily paper scan / stock-watching / reminders); China-region LLMs; multi-LLM cost optimization; workflows that shouldn't depend on your laptop being open. +**Not for**: pure code writing (Type 2 is more native); people who don't want to self-host; high-reliability production tasks. + +--- + +## Type 5: Edge-deployed — exemplar: OpenClaw / ClawBox + +**Exemplar**: [OpenClaw](https://www.jetson-ai-lab.com/tutorials/openclaw/) (community, Jetson ecosystem) / [ClawBox](https://openclawhardware.dev/) (€549 pre-installed Jetson kit, 67 TOPS) + +**Hero example**: +You run a law firm. You need AI to help organize a client's medical records + medical notes + physician testimony into a chronology. **But this data absolutely cannot go to the cloud**. So you buy a ClawBox (NVIDIA Jetson Orin Nano + OpenClaw + Ollama + Qwen 3.5 7B pre-installed), put it on your firm's network, and SSH in to work with it. All data stays inside this €549 box — zero telemetry, zero API calls, fully auditable. + +**5 distinctive features**: + +1. **Hardware-specific**: NVIDIA Jetson series (Orin Nano 8 GB, Thor 128 GB) or Raspberry Pi. GPU-accelerated, edge inference. +2. **Local LLM only**: Ollama backend, running Qwen 3.5 2B-7B / Llama / Mistral / Gemma or other open-weight models. **No cloud API calls whatsoever**. +3. **Zero cloud dependency / fully auditable**: localhost-bound, works in network-isolated environments, no telemetry. +4. **Edge-optimized memory**: semantic search memory files under 10 MB, persistent cross-session memory (e.g. [openclaw-memory-enhancer](https://github.com/henryfcb/openclaw-memory-enhancer)). +5. **Physical AI bridge**: drives physical devices (robots / sensors / smart home) — the agent works across physical + digital environments. + +**Why this type exists**: When data **cannot leave the local machine** (medical / legal / defense / privacy-sensitive), cloud-deployed isn't an option. Type 5 puts the agent fully on-device, trading €549 of hardware for 0 cloud cost and 0 data exposure. + +**Distinguishing trait**: one-time hardware investment, then API cost goes to zero; lives inside NVIDIA's edge hardware ecosystem; Jetson Thor can run a 30B model. + +**Trade-offs**: +- Model size is bounded by edge hardware (Orin Nano tops out at 7B, Thor at 30B) +- Setup is more involved than cloud (you need NVIDIA Jetson familiarity, JetPack, Docker, Ollama) +- No 24/7 cross-platform convenience like cloud-deployed + +**Good for**: privacy-sensitive data; offline-first; home AI box (smart home); physical AI (robots); long-term ownership without recurring API bills. +**Not for**: people uncomfortable with Linux / NVIDIA tooling; needing frontier models (GPT-5 / Claude Opus); unwilling to spend €549. + +--- + +## Subagent — “Spawning an Agent Inside an Agent Runtime” + +The 5 types above describe **where the agent runs** (IDE / Terminal / any CLI / Cloud / Edge). A **subagent** is another dimension: **while an agent is executing a task, it spawns another agent to handle a subtask**. + +There are two main implementation paths: + +| Path | How it starts | Examples | +|---|---|---| +| **Framework-based** (Stage 4) | `pip install langgraph / crewai / autogen` + Python orchestration code | LangGraph / CrewAI / AutoGen / Swarm / Strands | +| **Claude Code native** (Stage 5.5) | Write `.claude/agents/.md`; invoke it from the main session with the Task tool | Claude Code subagents + Claude Agent SDK | + +**The difference is runtime ownership**: +- Framework path: your own Python process runs the orchestrator, and each sub-agent is an object inside your program +- Claude path: Claude Code spawns a new agent instance itself; parent / child share the Claude runtime, and the parent only sees the child’s final result (context is isolated automatically) + +**Which should you choose?** If you need to mix LLM providers (GPT + Claude + Gemini) or embed multi-agent orchestration into another application, choose the framework path. If you are already committed to Claude Code and staying inside the Claude ecosystem, choose the subagent path (much less boilerplate). + +See the full comparison table at the [start of Stage 5.5](../stages/05-claude-code-ecosystem.en.md#55--subagents-claude-codes-native-multi-agent-mechanism--2025-new-feature); **to jump straight into 15 daily dispatch recipes** → [`subagent-cookbook.en.md`](./subagent-cookbook.en.md) (each includes a scenario + which subagent to use + a copy-paste prompt template). + +--- + +## Cross-paradigm combinations (the power-user pattern) + +Real power users often run **2 or 3 types simultaneously**, each handling what it's best at: + +![Personal power-user multi-type workflow](../resources/diagrams/power-user-multi-type-workflow.en.png) + +**Why this combination**: +- Type 2 handles code (terminal is the most natural interface) +- Type 4 handles routines + cross-platform (works when your laptop is closed) +- Type 5 handles privacy (data cannot leave the machine) + +--- + +## Decision tree (text form) + +![Agent paradigm decision tree](../resources/diagrams/agent-paradigm-decision-tree.en.png) + +--- + +## Links to existing stages / branches + +- **Learn Type 2 hands-on** → [Stage 5: Claude Code Ecosystem](../stages/05-claude-code-ecosystem.en.md) +- **See the 7-CLI detailed comparison** (Type 2 + Type 3) → [`resources/cli-agents-guide.en.md`](cli-agents-guide.en.md) +- **Compare IDE-coupled tools** (Type 1) → [`branches/for-developer.en.md`](../branches/for-developer.en.md) +- **Step-by-step Hermes deployment** → [`resources/cookbook.en.md` Recipe 6](cookbook.en.md) (Hermes + Ollama walkthrough) +- **Jetson + OpenClaw setup** → [Jetson AI Lab tutorial](https://www.jetson-ai-lab.com/tutorials/openclaw/) + [Seeed Studio wiki](https://wiki.seeedstudio.com/local_openclaw_on_recomputer_jetson/) + +--- + +## How I personally use these + +- **Daily development**: Type 2 (Claude Code, subscription) +- **Paper monitoring**: still manual for now (a weekly arXiv scan by hand) — Type 4 Hermes is on the list to try +- **Research vault**: Claude Code on my laptop calls the [research-hub](https://github.com/WenyuChiou/research-hub) pipeline (Type 2 mode) +- **No Type 5 yet**: my data isn't sensitive enough yet to justify going fully offline + +Once you try Type 4 or Type 5 in practice, come back and add your own use case to this reference. + +--- + +## References + +- [Jetson AI Lab: OpenClaw tutorial](https://www.jetson-ai-lab.com/tutorials/openclaw/) +- [ClawBox hardware](https://openclawhardware.dev/) +- [NVIDIA: Jetson Generative AI at the Edge](https://blogs.nvidia.com/blog/jetson-generative-ai-edge-oss/) +- [Hermes Agent (NousResearch)](https://github.com/NousResearch/hermes-agent) +- [claw-spark: One-click setup for Jetson / DGX Spark / RTX](https://github.com/theshiphq/claw-spark) diff --git a/resources/agent-paradigms.md b/resources/agent-paradigms.md new file mode 100644 index 0000000..a66063a --- /dev/null +++ b/resources/agent-paradigms.md @@ -0,0 +1,203 @@ +> **繁體中文** | [简体中文](./agent-paradigms.zh-Hans.md) | [English](./agent-paradigms.en.md) + +# Agent 5 種型態 — 你的 agent 跑在哪、為誰服務？ + +> [← 回主路線 README](../README.md) + +> 📌 **這份是 mental model reference**。看完之後你會知道：「同樣叫 agent、為什麼 Claude Code、Hermes Agent、OpenClaw 用起來完全不同感受？」 +> 已經知道想用哪個 → [`resources/cli-agents-guide.md`](cli-agents-guide.md)（7 CLI 並排比較）或 [`resources/cookbook.md`](cookbook.md)（step-by-step 部署）。 + +「Agent」一詞被用得很泛。Cursor 是 agent、Claude Code 是 agent、Telegram 上跟你聊天的 Hermes 也是 agent、家裡 Jetson 板子跑的 OpenClaw 也是 agent。但這 4 個東西用起來完全不同感受 —— 因為它們屬於**不同 paradigm**。差別不在 LLM 是哪家、而在 **agent 跑在哪、你用什麼介面跟它互動、需不需要連網**。 + +理解 paradigm 之後你才知道：搬一個 use case 從 Type 2 到 Type 4 不是「換工具」、是**換思考方式**。 + +--- + +## 一張表先建立認知 + +| Type | 代表 | Agent 跑在哪 | 你用什麼介面 | LLM | 離線 OK？ | 月成本（粗估）| +|---|---|---|---|---|---|---| +| **1. IDE-coupled** | Cursor / Cline / Continue | 你 IDE 內 | IDE sidebar | 多 provider | ❌ | $0-20 | +| **2. Terminal pair-programmer** | Claude Code / Codex / Gemini CLI | 你 terminal | terminal REPL | 綁特定家 | ❌ | $20 訂閱 或 API 用量 | +| **3. BYO-LLM CLI** | Aider / OpenCode / goose | 你 terminal | terminal REPL | 自帶 API key | ❌ | API 用量 | +| **4. Cloud-deployed** | **Hermes Agent** | $5 VPS / Modal | **Telegram / Slack / 任一 chat app** | 200+ provider routing | ❌ | $5 server + API | +| **5. Edge-deployed** | **OpenClaw / ClawBox** | Jetson 板子 / Raspberry Pi | local chat / SSH | **本機 Ollama**（Qwen / Llama / Mistral）| **✅** | 一次硬體 €549、之後 0 | + +→ 4 跟 5 都是「**deployed autonomous agent**」（agent **不在你 laptop 前**、跑在外面 24×7 serve 你）。4 在 cloud、5 在 edge。剩下的 1-3 是「**co-located agent**」（agent 跟你一起在 laptop 上、你走它停）。 + +--- + +## Type 1: IDE-coupled — 「sidebar pair-programmer」 + +**代表**：[Cursor](https://cursor.com) / [Windsurf](https://codeium.com/windsurf) / [Cline](https://cline.bot) / [Continue](https://continue.dev) / [Zed](https://zed.dev) + +**Hero example**： +你在 Cursor 寫一個 React component。左邊 editor、右邊 Cursor sidebar 聊天。你選一段 code 按 `Cmd+K`、Cursor 就地改寫。改完之後你看 inline diff、accept/reject。 + +**為什麼這型存在**：寫 code 的時候你**眼睛要看 code**、不能去 terminal 對話。IDE-coupled agent 把 LLM 放在你視線旁邊、保留視覺 context。 + +**適合**：edit 多、explore 少；side-by-side coding；需要 visual diff。 +**不適合**：需要 agent 自己跑多步驟（agent 在 sidebar 不太自由）；non-coding task。 + +--- + +## Type 2: Terminal pair-programmer — 「Claude Code paradigm」 + +**代表**：[Claude Code](https://github.com/anthropics/claude-code) / [Codex](https://github.com/openai/codex) / [Gemini CLI](https://github.com/google-gemini/gemini-cli) + +**Hero example**： +你在 terminal 開 Claude Code、輸入「refactor 整個 auth module、把 callback 改成 async/await、跑 tests」。Claude Code 自己讀檔、改檔、跑 pytest、報告結果。整個過程 5-10 分鐘、你看 streaming output。 + +**為什麼這型存在**：Claude Code / Codex 把整個 terminal 變成 agent 的 workspace。agent 有 file system / shell / git 完整 access、可以自主完成多步驟 task。比 Type 1 更 autonomous。 + +**特色**：訂閱制（$20/月可用整月、不算 token）；綁定特定 LLM 家族（Claude Code = Claude only）。 + +**適合**：agentic task；長 refactor；paper writing；任何 1-2 step 之上的工作。 +**不適合**：跨多家 LLM 比較成本；非 coding/writing 場景；offline。 + +--- + +## Type 3: BYO-LLM CLI — 「multi-provider 同 mental model」 + +**代表**：[Aider](https://aider.chat) / [OpenCode](https://github.com/sst/opencode) / [goose](https://block.github.io/goose) / [Hermes Agent](https://github.com/NousResearch/hermes-agent)* + +**Hero example**： +你想用 DeepSeek-V4-Pro（2026-04 preview、開源 MIT、前身 R1 reasoning lineage 已併入主線）寫 code（比 Claude Opus 便宜 10×）。Aider 設 `--model deepseek/deepseek-reasoner` + `OPENROUTER_API_KEY` 就能跑、git-aware、commit message 自動寫。 + +**跟 Type 2 的差別**：Type 2 綁特定家、Type 3 你帶 API key、任何 OpenAI-compatible endpoint 都行。 + +**特色**：cost-sensitive；多 provider 比較；自架 LLM（Ollama / vLLM）也能用。 + +**適合**：實驗多家 LLM；省 cost；本機 LLM；不想被一家綁。 +**不適合**：怕 setup 複雜（要管 API key、provider config）。 + +*Hermes Agent 既屬於 Type 3（CLI mode）也屬於 Type 4（cloud mode）—— 取決於你怎麼部署。下面細講。 + +--- + +## Type 4: Cloud-deployed — 例：Hermes Agent + +**代表**：[Hermes Agent](https://github.com/NousResearch/hermes-agent)（Nous Research、★ 193k+、MIT） + +**Hero example**： +你坐在地鐵、手機開 Telegram、對 Hermes bot 說「整理今天 arXiv ML 新 paper、給我 3 個 highlights、傳回 Telegram」。Hermes agent 在你 $5 DigitalOcean VPS 上跑、收訊息、決定該用 GPT-5（找 paper）+ Claude Opus（寫 summary）+ Gemini Flash（壓縮成 3 條）、執行完傳結果回 Telegram。整個過程你沒碰 laptop。 + +**5 個 distinctive feature**： + +1. **Multi-platform chat interface**：Telegram / Discord / Slack / WhatsApp / Signal 都能當入口。你在哪個平台 ping、agent 就在哪回。 +2. **Multi-LLM routing（200+ model neutral）**：OpenRouter + NVIDIA NIM + 智譜 GLM + Kimi + 小米 MiMo + MiniMax + HF + OpenAI + Anthropic + Google。**同一 conversation 內可跨 LLM**。 +3. **24/7 在線**：agent 不依賴你 laptop、cloud VPS host、任何時刻可用。 +4. **Built-in cron**：「每天 9am 抓 X 給我 Y」這種 routine 直接內建。 +5. **自我學習技能**（實驗中、尚未獨立審計）：agent 跟你互動久了、會自動歸納出可重用的 skill、跨 session 累積演化。 + +**為什麼這型存在**：當 agent 是「**個人助理**」而不是「pair programmer」時、它不該綁你 laptop。Type 4 把 agent 變成 24×7 service。 + +**特色**：deployment cost ~$5/月 VPS + API；中國圈 LLM 支援（GLM / Kimi）—— 國際服務中斷時可以改用這些接力。 + +**Trade-off**： +- ⚠️ 自我學習技能是新功能、還沒經過獨立安全檢驗；用在會造成嚴重後果的任務（醫療 / 法律 / 金流）前先別開 +- 失去 IDE / terminal 的 file system 直接讀寫便利、變成 chat-first workflow +- 需要會 self-host VPS（Linux / docker / systemd 基礎） + +**適合**：跨平台通知；24/7 routine（每天抓 paper / 看股票 / 提醒）；中國圈 LLM；多 LLM cost optimization；非 laptop-bound 工作流。 +**不適合**：純寫 code（Type 2 native）；不想 self-host；對 production reliability 要求高。 + +--- + +## Type 5: Edge-deployed — 例：OpenClaw / ClawBox + +**代表**：[OpenClaw](https://www.jetson-ai-lab.com/tutorials/openclaw/)（社群、Jetson 生態） / [ClawBox](https://openclawhardware.dev/)（€549 預裝 Jetson 套件、67 TOPS） + +**Hero example**： +你是法律事務所、要 AI 幫你整理當事人病歷 + 醫療記錄 + 醫師證詞、產出時序表。**但這些資料絕對不能上 cloud**。你買一台 ClawBox（NVIDIA Jetson Orin Nano + 預裝 OpenClaw + Ollama + Qwen 3.5 7B）、放在事務所網路內、SSH 進去跟它工作。所有資料只在這台 €549 的盒子裡、無 telemetry、無 API call、完全可審計。 + +**5 個 distinctive feature**： + +1. **Hardware-specific**：NVIDIA Jetson 系列（Orin Nano 8 GB、Thor 128 GB）或 Raspberry Pi。GPU 加速、邊緣推論。 +2. **本機 LLM only**：Ollama backend、跑 Qwen 3.5 2B-7B / Llama / Mistral / Gemma 等 open-weight。**沒有任何 cloud API call**。 +3. **零雲端依賴 / 完全可審計**：localhost-bound、network-isolated 可用、無 telemetry。 +4. **Edge-optimized memory**：semantic search memory file < 10 MB、跨 session 記憶（例：[openclaw-memory-enhancer](https://github.com/henryfcb/openclaw-memory-enhancer)）。 +5. **Physical AI bridge**：可控物理 device（robot / sensor / smart home）—— agent 跨 physical + digital 環境。 + +**為什麼這型存在**：當資料**不能離開本機**時（醫療 / 法律 / 軍工 / 隱私敏感）、cloud-deployed 不是選項。Type 5 把 agent 完全 on-device、用 €549 換 0 cloud cost + 0 data exposure。 + +**特色**：一次硬體投資、之後 API 0 元；對應 NVIDIA 邊緣硬體生態；Jetson Thor 跑 30B model 也 OK。 + +**Trade-off**： +- 模型受邊緣 hardware 限制（Orin Nano 跑 7B 上限、Thor 才到 30B） +- Setup 比 cloud 複雜（要會 NVIDIA Jetson 環境、JetPack、Docker、Ollama） +- 沒有 cloud-deployed 的 24/7 跨平台便利 + +**適合**：隱私敏感資料；offline-first；家用 AI box（smart home）；physical AI（robot）；長期持有、不想付 API recurring cost。 +**不適合**：不會 Linux / NVIDIA 環境；需要前沿 model（GPT-5 / Claude Opus）；不想花 €549。 + +--- + +## Subagent — 「在 agent runtime 裡再 spawn agent」 + +上面 5 個 type 講的是 **agent 跑在哪裡**（IDE / Terminal / 任意 CLI / Cloud / Edge）。**Subagent** 是另一個維度：**一個 agent 在執行任務時、spawn 出另一個 agent 跑子任務**。 + +主要兩種實作路徑： + +| 路徑 | 怎麼啟動 | 代表 | +|---|---|---| +| **Framework-based**（Stage 4） | `pip install langgraph / crewai / autogen` + Python orchestration code | LangGraph / CrewAI / AutoGen / Swarm / Strands | +| **Claude Code 原生**（Stage 5.5） | 寫 `.claude/agents/.md`、主 session 用 Task tool invoke | Claude Code subagent + Claude Agent SDK | + +**差別在 runtime ownership**： +- Framework path：你用 Python 寫一支主程式（orchestrator）來調度、每個 sub-agent 都是這支程式裡的物件 +- Claude path：Claude Code 自動建立新的子 agent、主 agent 只拿到子 agent 的最終結果、不用管它的內部過程（context 自動隔離、互不干擾） + +**選哪個**：要跨 LLM provider（GPT + Claude + Gemini 混用）或要把 multi-agent 包進別的應用程式 → framework path。已 commit Claude Code、只在 Claude 生態 → subagent path（少很多 boilerplate）。 + +完整對照表見 [Stage 5.5 開頭](../stages/05-claude-code-ecosystem.md#55--subagentsclaude-code-原生-multi-agent-機制-2025-新功能)；**想直接看 15 個 daily dispatch recipe** → [`subagent-cookbook.md`](./subagent-cookbook.md)（每個含情境 + 用哪個 subagent + 複製即用的 prompt 範本）。 + +--- + +## 跨型態組合（power user pattern） + +真實 user 常常**同時用 2-3 個 type**、各做擅長的事： + +![個人 power-user 多 type workflow](../resources/diagrams/power-user-multi-type-workflow.png) + +**為什麼這樣搭**： +- Type 2 處理 code（terminal 介面最自然） +- Type 4 處理 routine + 跨平台（laptop 沒開時也工作） +- Type 5 處理隱私（不可上 cloud） + +--- + +## Decision tree（簡化文字版） + +![選哪個 agent type 決策樹](../resources/diagrams/agent-paradigm-decision-tree.png) + +--- + +## 跟既有 stage / branch 的連結 + +- **想學 Type 2 上手** → [Stage 5: Claude Code 生態](../stages/05-claude-code-ecosystem.md) +- **想看 7 CLI 詳細並排比較**（Type 2 + Type 3）→ [`resources/cli-agents-guide.md`](cli-agents-guide.md) +- **想看 IDE-coupled 對比**（Type 1）→ [`branches/for-developer.md`](../branches/for-developer.md) +- **想 step-by-step 部署 Hermes** → [`resources/cookbook.md` Recipe 6](cookbook.md)（含 Hermes + Ollama walkthrough） +- **想搞 Jetson + OpenClaw** → [Jetson AI Lab tutorial](https://www.jetson-ai-lab.com/tutorials/openclaw/) + [Seeed Studio wiki](https://wiki.seeedstudio.com/local_openclaw_on_recomputer_jetson/) + +--- + +## 我自己怎麼用 + +- **每天主開發**：Type 2（Claude Code、訂閱制） +- **paper monitoring**：暫時手動（每週手動掃 arXiv）—— 之後想試 Type 4 Hermes 自動化 +- **research vault**：Claude Code 在 laptop 內呼叫 [research-hub](https://github.com/WenyuChiou/research-hub) pipeline（Type 2 模式） +- **沒接觸 Type 5**：目前資料沒到「不能上 cloud」的敏感程度 + +Type 4 / Type 5 你之後玩了、可以再回來補這份 reference 自己的 use case。 + +--- + +## References + +- [Jetson AI Lab: OpenClaw tutorial](https://www.jetson-ai-lab.com/tutorials/openclaw/) +- [ClawBox hardware](https://openclawhardware.dev/) +- [NVIDIA: Jetson Generative AI at the Edge](https://blogs.nvidia.com/blog/jetson-generative-ai-edge-oss/) +- [Hermes Agent (NousResearch)](https://github.com/NousResearch/hermes-agent) +- [claw-spark: One-click setup for Jetson / DGX Spark / RTX](https://github.com/theshiphq/claw-spark) diff --git a/resources/agent-paradigms.zh-Hans.md b/resources/agent-paradigms.zh-Hans.md new file mode 100644 index 0000000..812cb6e --- /dev/null +++ b/resources/agent-paradigms.zh-Hans.md @@ -0,0 +1,203 @@ +> [繁體中文](./agent-paradigms.md) | **简体中文** | [English](./agent-paradigms.en.md) + +# Agent 5 种型态 — 你的 agent 跑在哪、为谁服务？ + +> [← 回主路线 README](../README.zh-Hans.md) + +> 📌 **这份是 mental model reference**。看完之后你会知道：“同样叫 agent、为什么 Claude Code、Hermes Agent、OpenClaw 用起来完全不同感受？” +> 已经知道想用哪个 → [`resources/cli-agents-guide.zh-Hans.md`](cli-agents-guide.zh-Hans.md)（7 CLI 并排比较）或 [`resources/cookbook.zh-Hans.md`](cookbook.zh-Hans.md)（step-by-step 部署）。 + +“Agent”一词被用得很泛。Cursor 是 agent、Claude Code 是 agent、Telegram 上跟你聊天的 Hermes 也是 agent、家里 Jetson 板子跑的 OpenClaw 也是 agent。但这 4 个东西用起来完全不同感受 —— 因为它们属于**不同 paradigm**。差别不在 LLM 是哪家、而在 **agent 跑在哪、你用什么界面跟它互动、需不需要联网**。 + +理解 paradigm 之后你才知道:搬一个 use case 从 Type 2 到 Type 4 不是“换工具”、是**换思考方式**。 + +--- + +## 一张表先建立认知 + +| Type | 代表 | Agent 跑在哪 | 你用什么界面 | LLM | 离线 OK? | 月成本（粗估）| +|---|---|---|---|---|---|---| +| **1. IDE-coupled** | Cursor / Cline / Continue | 你 IDE 内 | IDE sidebar | 多 provider | ❌ | $0-20 | +| **2. Terminal pair-programmer** | Claude Code / Codex / Gemini CLI | 你 terminal | terminal REPL | 绑特定家 | ❌ | $20 订阅 或 API 用量 | +| **3. BYO-LLM CLI** | Aider / OpenCode / goose | 你 terminal | terminal REPL | 自带 API key | ❌ | API 用量 | +| **4. Cloud-deployed** | **Hermes Agent** | $5 VPS / Modal | **Telegram / Slack / 任一 chat app** | 200+ provider routing | ❌ | $5 server + API | +| **5. Edge-deployed** | **OpenClaw / ClawBox** | Jetson 板子 / Raspberry Pi | local chat / SSH | **本机 Ollama**（Qwen / Llama / Mistral）| **✅** | 一次硬件 €549、之后 0 | + +→ 4 跟 5 都是“**deployed autonomous agent**”（agent **不在你 laptop 前**、跑在外面 24×7 serve 你）。4 在 cloud、5 在 edge。剩下的 1-3 是“**co-located agent**”（agent 跟你一起在 laptop 上、你走它停）。 + +--- + +## Type 1: IDE-coupled — “sidebar pair-programmer” + +**代表**:[Cursor](https://cursor.com) / [Windsurf](https://codeium.com/windsurf) / [Cline](https://cline.bot) / [Continue](https://continue.dev) / [Zed](https://zed.dev) + +**Hero example**: +你在 Cursor 写一个 React component。左边 editor、右边 Cursor sidebar 聊天。你选一段 code 按 `Cmd+K`、Cursor 就地改写。改完之后你看 inline diff、accept/reject。 + +**为什么这型存在**:写 code 的时候你**眼睛要看 code**、不能去 terminal 对话。IDE-coupled agent 把 LLM 放在你视线旁边、保留视觉 context。 + +**适合**:edit 多、explore 少;side-by-side coding;需要 visual diff。 +**不适合**:需要 agent 自己跑多步骤（agent 在 sidebar 不太自由）;non-coding task。 + +--- + +## Type 2: Terminal pair-programmer — “Claude Code paradigm” + +**代表**:[Claude Code](https://github.com/anthropics/claude-code) / [Codex](https://github.com/openai/codex) / [Gemini CLI](https://github.com/google-gemini/gemini-cli) + +**Hero example**: +你在 terminal 开 Claude Code、输入“refactor 整个 auth module、把 callback 改成 async/await、跑 tests”。Claude Code 自己读档、改档、跑 pytest、报告结果。整个过程 5-10 分钟、你看 streaming output。 + +**为什么这型存在**:Claude Code / Codex 把整个 terminal 变成 agent 的 workspace。agent 有 file system / shell / git 完整 access、可以自主完成多步骤 task。比 Type 1 更 autonomous。 + +**特色**:订阅制（$20/月可用整月、不算 token）;绑定特定 LLM 家族（Claude Code = Claude only）。 + +**适合**:agentic task;长 refactor;paper writing;任何 1-2 step 之上的工作。 +**不适合**:跨多家 LLM 比较成本;非 coding/writing 场景;offline。 + +--- + +## Type 3: BYO-LLM CLI — “multi-provider 同 mental model” + +**代表**:[Aider](https://aider.chat) / [OpenCode](https://github.com/sst/opencode) / [goose](https://block.github.io/goose) / [Hermes Agent](https://github.com/NousResearch/hermes-agent)* + +**Hero example**: +你想用 DeepSeek-R1 写 code（比 Claude Opus 便宜 10×）。Aider 设 `--model deepseek/deepseek-reasoner` + `OPENROUTER_API_KEY` 就能跑、git-aware、commit message 自动写。 + +**跟 Type 2 的差别**:Type 2 绑特定家、Type 3 你带 API key、任何 OpenAI-compatible endpoint 都行。 + +**特色**:cost-sensitive;多 provider 比较;自架 LLM（Ollama / vLLM）也能用。 + +**适合**:实验多家 LLM;省 cost;本机 LLM;不想被一家绑。 +**不适合**:怕 setup 复杂（要管 API key、provider config）。 + +*Hermes Agent 既属于 Type 3（CLI mode）也属于 Type 4（cloud mode）—— 取决于你怎么部署。下面细讲。 + +--- + +## Type 4: Cloud-deployed — 例:Hermes Agent + +**代表**:[Hermes Agent](https://github.com/NousResearch/hermes-agent)（Nous Research、★ 193k+、MIT） + +**Hero example**: +你坐在地铁、手机开 Telegram、对 Hermes bot 说“整理今天 arXiv ML 新 paper、给我 3 个 highlights、传回 Telegram”。Hermes agent 在你 $5 DigitalOcean VPS 上跑、收讯息、决定该用 GPT-5（找 paper）+ Claude Opus（写 summary）+ Gemini Flash（压缩成 3 条）、执行完传结果回 Telegram。整个过程你没碰 laptop。 + +**5 个 distinctive feature**: + +1. **Multi-platform chat interface**:Telegram / Discord / Slack / WhatsApp / Signal 都能当入口。你在哪个平台 ping、agent 就在哪回。 +2. **Multi-LLM routing（200+ model neutral）**:OpenRouter + NVIDIA NIM + 智谱 GLM + Kimi + 小米 MiMo + MiniMax + HF + OpenAI + Anthropic + Google。**同一 conversation 内可跨 LLM**。 +3. **24/7 在线**:agent 不依赖你 laptop、cloud VPS host、任何时刻可用。 +4. **Built-in cron**:“每天 9am 抓 X 给我 Y”这种 routine 直接内建。 +5. **自我学习技能**（实验中、尚未独立审计）:agent 跟你互动久了、会自动归纳出可复用的 skill、跨 session 累积演化。 + +**为什么这型存在**:当 agent 是“**个人助理**”而不是“pair programmer”时、它不该绑你 laptop。Type 4 把 agent 变成 24×7 service。 + +**特色**:deployment cost ~$5/月 VPS + API;中国圈 LLM 支持（GLM / Kimi）—— 国际服务中断时可以改用这些接力。 + +**Trade-off**: +- ⚠️ 自我学习技能是新功能、还没经过独立安全检验；用在会造成严重后果的任务（医疗 / 法律 / 金流）前先别开 +- 失去 IDE / terminal 的 file system 直接读写便利、变成 chat-first workflow +- 需要会 self-host VPS（Linux / docker / systemd 基础） + +**适合**:跨平台通知;24/7 routine（每天抓 paper / 看股票 / 提醒）;中国圈 LLM;多 LLM cost optimization;非 laptop-bound 工作流。 +**不适合**:纯写 code（Type 2 native）;不想 self-host;对 production reliability 要求高。 + +--- + +## Type 5: Edge-deployed — 例:OpenClaw / ClawBox + +**代表**:[OpenClaw](https://www.jetson-ai-lab.com/tutorials/openclaw/)（社群、Jetson 生态） / [ClawBox](https://openclawhardware.dev/)（€549 预装 Jetson 套件、67 TOPS） + +**Hero example**: +你是法律事务所、要 AI 帮你整理当事人病历 + 医疗记录 + 医师证词、产出时序表。**但这些资料绝对不能上 cloud**。你买一台 ClawBox（NVIDIA Jetson Orin Nano + 预装 OpenClaw + Ollama + Qwen 3.5 7B）、放在事务所网络内、SSH 进去跟它工作。所有资料只在这台 €549 的盒子里、无 telemetry、无 API call、完全可审计。 + +**5 个 distinctive feature**: + +1. **Hardware-specific**:NVIDIA Jetson 系列（Orin Nano 8 GB、Thor 128 GB）或 Raspberry Pi。GPU 加速、边缘推论。 +2. **本机 LLM only**:Ollama backend、跑 Qwen 3.5 2B-7B / Llama / Mistral / Gemma 等 open-weight。**没有任何 cloud API call**。 +3. **零云端依赖 / 完全可审计**:localhost-bound、network-isolated 可用、无 telemetry。 +4. **Edge-optimized memory**:semantic search memory file < 10 MB、跨 session 记忆（例:[openclaw-memory-enhancer](https://github.com/henryfcb/openclaw-memory-enhancer)）。 +5. **Physical AI bridge**:可控物理 device（robot / sensor / smart home）—— agent 跨 physical + digital 环境。 + +**为什么这型存在**:当资料**不能离开本机**时（医疗 / 法律 / 军工 / 隐私敏感）、cloud-deployed 不是选项。Type 5 把 agent 完全 on-device、用 €549 换 0 cloud cost + 0 data exposure。 + +**特色**:一次硬件投资、之后 API 0 元;对应 NVIDIA 边缘硬件生态;Jetson Thor 跑 30B model 也 OK。 + +**Trade-off**: +- 模型受边缘 hardware 限制（Orin Nano 跑 7B 上限、Thor 才到 30B） +- Setup 比 cloud 复杂（要会 NVIDIA Jetson 环境、JetPack、Docker、Ollama） +- 没有 cloud-deployed 的 24/7 跨平台便利 + +**适合**:隐私敏感资料;offline-first;家用 AI box（smart home）;physical AI（robot）;长期持有、不想付 API recurring cost。 +**不适合**:不会 Linux / NVIDIA 环境;需要前沿 model（GPT-5 / Claude Opus）;不想花 €549。 + +--- + +## Subagent — “在 agent runtime 里再 spawn agent” + +上面 5 个 type 讲的是 **agent 跑在哪里**（IDE / Terminal / 任意 CLI / Cloud / Edge）。**Subagent** 是另一个维度：**一个 agent 在执行任务时，spawn 出另一个 agent 跑子任务**。 + +主要两种实作路径： + +| 路径 | 怎么启动 | 代表 | +|---|---|---| +| **Framework-based**（Stage 4） | `pip install langgraph / crewai / autogen` + Python orchestration code | LangGraph / CrewAI / AutoGen / Swarm / Strands | +| **Claude Code 原生**（Stage 5.5） | 写 `.claude/agents/.md`，主 session 用 Task tool invoke | Claude Code subagent + Claude Agent SDK | + +**差别在 runtime ownership**： +- Framework path：你用 Python 写一支主程序（orchestrator）来调度，每个 sub-agent 都是这支程序里的对象 +- Claude path：Claude Code 自动建立新的子 agent，主 agent 只拿到子 agent 的最终结果、不用管它的内部过程（context 自动隔离、互不干扰） + +**选哪个**：要跨 LLM provider（GPT + Claude + Gemini 混用）或要把 multi-agent 包进别的应用程序 → framework path。已 commit Claude Code、只在 Claude 生态 → subagent path（少很多 boilerplate）。 + +完整对照表见 [Stage 5.5 开头](../stages/05-claude-code-ecosystem.zh-Hans.md#55--subagentsclaude-code-原生-multi-agent-机制-2025-新功能)；**想直接看 15 个 daily dispatch recipe** → [`subagent-cookbook.zh-Hans.md`](./subagent-cookbook.zh-Hans.md)（每个含场景 + 用哪个 subagent + 复制即用的 prompt 模板）。 + +--- + +## 跨型态组合（power user pattern） + +真实 user 常常**同时用 2-3 个 type**、各做擅长的事: + +![个人 power-user 多 type workflow](../resources/diagrams/power-user-multi-type-workflow.zh-Hans.png) + +**为什么这样搭**: +- Type 2 处理 code（terminal 界面最自然） +- Type 4 处理 routine + 跨平台（laptop 没开时也工作） +- Type 5 处理隐私（不可上 cloud） + +--- + +## Decision tree（简化文字版） + +![选哪个 agent type 决策树](../resources/diagrams/agent-paradigm-decision-tree.zh-Hans.png) + +--- + +## 跟既有 stage / branch 的连结 + +- **想学 Type 2 上手** → [Stage 5: Claude Code 生态](../stages/05-claude-code-ecosystem.zh-Hans.md) +- **想看 7 CLI 详细并排比较**（Type 2 + Type 3）→ [`resources/cli-agents-guide.zh-Hans.md`](cli-agents-guide.zh-Hans.md) +- **想看 IDE-coupled 对比**（Type 1）→ [`branches/for-developer.zh-Hans.md`](../branches/for-developer.zh-Hans.md) +- **想 step-by-step 部署 Hermes** → [`resources/cookbook.zh-Hans.md` Recipe 6](cookbook.zh-Hans.md)（含 Hermes + Ollama walkthrough） +- **想搞 Jetson + OpenClaw** → [Jetson AI Lab tutorial](https://www.jetson-ai-lab.com/tutorials/openclaw/) + [Seeed Studio wiki](https://wiki.seeedstudio.com/local_openclaw_on_recomputer_jetson/) + +--- + +## 我自己怎么用 + +- **每天主开发**:Type 2（Claude Code、订阅制） +- **paper monitoring**:暂时手动（每周手动扫 arXiv）—— 之后想试 Type 4 Hermes 自动化 +- **research vault**:Claude Code 在 laptop 内调用 [research-hub](https://github.com/WenyuChiou/research-hub) pipeline（Type 2 模式） +- **没接触 Type 5**:目前资料没到“不能上 cloud”的敏感程度 + +Type 4 / Type 5 你之后玩了、可以再回来补这份 reference 自己的 use case。 + +--- + +## References + +- [Jetson AI Lab: OpenClaw tutorial](https://www.jetson-ai-lab.com/tutorials/openclaw/) +- [ClawBox hardware](https://openclawhardware.dev/) +- [NVIDIA: Jetson Generative AI at the Edge](https://blogs.nvidia.com/blog/jetson-generative-ai-edge-oss/) +- [Hermes Agent (NousResearch)](https://github.com/NousResearch/hermes-agent) +- [claw-spark: One-click setup for Jetson / DGX Spark / RTX](https://github.com/theshiphq/claw-spark) diff --git a/resources/cli-agents-guide.en.md b/resources/cli-agents-guide.en.md new file mode 100644 index 0000000..0b017c0 --- /dev/null +++ b/resources/cli-agents-guide.en.md @@ -0,0 +1,148 @@ +> [繁體中文](./cli-agents-guide.md) | [简体中文](./cli-agents-guide.zh-Hans.md) | **English** + +# CLI Agents Comparison Guide + +> [← Back to main path README](../README.en.md) + +> 📌 **This is a reference doc** (depth comparison, selection logic, pitfalls, recommended setups). +> First time touching CLI agents, want step-by-step onboarding → see [`tracks/cli/A1-cli-intro.en.md`](../tracks/cli/A1-cli-intro.en.md) (Track A first stop). +> First want to understand "why does one agent live in a terminal, another in Telegram, another on a Jetson board?" mental model → see [`resources/agent-paradigms.en.md`](agent-paradigms.en.md) (5 agent paradigms). +> Already using one, want to decide / compare / upgrade → stay here. + +A cross-branch reference shared by Track A (A1-A3) + all 5 specialized branches: **how to choose between Claude Code / Codex / OpenCode / Gemini CLI / goose / Aider / Hermes Agent?** Every branch references CLI agents but no single branch "owns" this comparison, so it lives in `resources/`. + +--- + +## 📋 7 Major Terminal CLI Agents + +Only terminal-based CLI agents are included. IDE-based agents (Cursor / Cline / Continue) live in [for-developer](../branches/for-developer.en.md). The first 6 numbers verified via `gh api` on 2026-05-06; Hermes Agent verified on 2026-05-10. + +| Tool | Provider | License | Primary LLM | Auth / Pricing | Stars | +|---|---|---|---|---|---| +| [Claude Code](https://github.com/anthropics/claude-code) | Anthropic (official) | NOASSERTION | Claude | Claude subscription **OR** Anthropic Console API key | ★ 132k+ | +| [Codex](https://github.com/openai/codex) | OpenAI (official) | Apache-2.0 | GPT family | ChatGPT account sign-in **OR** OpenAI API key | ★ 89k+ | +| [OpenCode](https://github.com/sst/opencode) | community (repo now at `anomalyco/opencode`) | MIT | Any (multi-provider) | BYO API key, or built-in OpenCode Zen hosted | ★ 171k+ | +| [Gemini CLI](https://github.com/google-gemini/gemini-cli) | Google (official) | Apache-2.0 | Gemini | Generous free tier, paid above quota | ★ 103k+ | +| [goose](https://github.com/block/goose) | Agentic AI Foundation (repo now at `aaif-goose/goose`) | Apache-2.0 | 15+ providers (incl. Ollama) | BYO API key, or existing Claude / ChatGPT / Gemini subscription via ACP | ★ 47k+ | +| [Aider](https://github.com/Aider-AI/aider) | Aider-AI (community) | Apache-2.0 | Any | BYO API key | ★ 44k+ | +| [Hermes Agent](https://github.com/NousResearch/hermes-agent) | Nous Research | MIT | 200+ via OpenRouter / NVIDIA NIM / Zhipu GLM / Kimi / Xiaomi MiMo / MiniMax / HF / OpenAI | BYO API key (multi-provider) | ★ 193k+ | + +--- + +## 🎯 Which to Pick? Decide by Use Case + +### Writing papers / literature / research +**Top pick**: Claude Code (long context, strong reasoning, good hallucination resistance). Gemini CLI is the alternative — its million-token context fits whole-PDF / whole-dataset workflows. + +### Writing code / refactoring a codebase +**Top pick**: Aider (git-native — auto-commits each change, easy to revert) or Claude Code. OpenCode fits when you need to switch between LLMs. + +### Privacy / offline / no cloud +**Top pick**: goose or OpenCode + local Ollama. Both support BYO LLM and connect to `http://localhost:11434/v1` (Ollama default). + +### Already subscribed to ChatGPT Plus / Pro +**Top pick**: Codex — same account, no separate billing. + +### Want 1M-token long context + Google ecosystem +**Top pick**: Gemini CLI. Generous free tier and long context are the differentiators. Note: Google service integration (Gmail / Drive / Docs) goes through MCP extensions, not built-in connectors — same setup pattern as other CLIs. + +### Want to avoid vendor lock-in +**Top pick**: OpenCode > goose > Aider. None tie you to a specific provider; models are swappable. + +### First time installing a CLI agent — wanting easiest start +**Top pick**: Claude Code. Broad ecosystem, CLAUDE.md mechanism for version-controlled prompts, plenty of community resources when you hit issues. + +### Want it running on a cloud VM, talking to it via Telegram / Slack / Discord, with mainland China LLMs as primary +**Top pick**: Hermes Agent. Three differentiators: +- **Decoupled from your laptop** — agent runs on a $5 VPS / Modal serverless / Vercel Sandbox; you message it from Telegram / Discord / Slack / WhatsApp / Signal +- **Model-neutral** — supports GLM / Kimi / Xiaomi MiMo / MiniMax, matching the 11 Chinese-ecosystem catalog entries +- **Built-in self-improving skill loop + cron scheduler** — agent autonomously generates skills from interaction, refines them across sessions, runs scheduled jobs unattended +- ⚠️ Self-evolving skills is a frontier feature with no independent audit yet; for production tasks, start with low-stakes experiments + +--- + +## 📝 Portable Prompts Across CLIs + +If you want prompts that work across CLI tools (or want to switch without rewriting), follow these principles: + +1. **Specify file paths explicitly** — "modify `src/auth.py`" beats "modify that auth file" +2. **Ask for multi-step breakdowns** — "first list a plan, then act after I confirm" works in every CLI +3. **Avoid CLI-specific magic** — `/init` `/compact` are Claude-Code-specific; OpenCode doesn't have them +4. **Use `.cursorrules` / `CLAUDE.md` / `AGENTS.md` for persistent preferences** — Claude Code reads `CLAUDE.md`, Codex reads `AGENTS.md`, OpenCode reads `OPENCODE.md`, **content can be the same** +5. **State review scope clearly** — "review only my diff" vs "review the whole repo" + +Cross-CLI prompts are usually 5-10% more verbose than CLI-specific ones, but the upside is you can switch tools without rewriting. + +--- + +## ⚠️ Common Pitfalls + +### File path handling +- Windows uses backslashes (`C:\Users\...`); most CLIs translate internally but sometimes get confused +- Recommendation: in git-bash / WSL use forward slashes, avoid weird quoting + +### Git integration differences +- **Aider** auto-commits every change (by design, not a bug) +- **Claude Code / Codex / OpenCode / goose** don't auto-commit by default — manual or via prompt + +### Default sandbox (each CLI varies; verify against official docs before use) +- **Claude Code**: bash writes default to cwd; reads broader (except deny-rule paths) +- **Codex**: in version-controlled folders, `Auto` (workspace-write + on-request escalation) is recommended; in non-git folders, `read-only` +- **goose / OpenCode**: relatively permissive — add explicit sandbox / approval rules; don't rely on defaults + +### Token cost accumulation +- Running a `grep` on a large codebase can consume 100k+ tokens +- Summarizing a long PDF can hit 500k tokens (Gemini handles this; other tools need to be cost-aware) +- Recommendation: estimate cost before each operation; set a monthly cap + +### Multi-CLI session interference +- Two CLIs in the same repo (e.g. Claude Code + Aider) can race-condition file edits +- Recommendation: one repo, one CLI (unless you genuinely need parallelism) + +--- + +## 🔧 Real-World Setups + +Three common combinations; pick one that fits: + +### Setup A: Claude Code primary + OpenCode backup +- Claude Code handles 90% of daily work (code, docs, debug) +- OpenCode + Ollama for privacy-sensitive data (medical, financial) +- One prompt, runs in either + +### Setup B: Codex (GPT) + Aider (Claude) mix +- Codex handles small tasks within ChatGPT Plus quota +- Aider with Claude API key handles big refactors (git-native commit convenient) +- Separate billing, no interference + +### Setup C: Gemini CLI primary (long-context scenarios) +- Whole PDF / whole codebase fed at once +- Add Aider for scenarios needing precise git diff +- Fits scholars, knowledge workers + +### Setup D: Hermes Agent + Local Ollama (multi-platform + mainland China LLMs + offline) +- **Hermes Agent** runs on a low-cost VPS or your own machine as a multi-platform agent gateway +- **LLM endpoint** can be Ollama (`http://localhost:11434/v1`), or swapped to providers such as z.ai GLM / Kimi +- **Chat entrypoint** can be Telegram / Slack / Discord; Hermes routes platform messages into the agent workflow +- **When you want zero Anthropic / OpenAI dependency**, this setup fits offline, privacy-sensitive, and low-cost repeat experiments +- Step-by-step walkthrough: [`resources/cookbook.md` Recipe 6](cookbook.en.md#6-local-llm--cli-agent-quick-walkthrough) + +--- + +## Linking Back to Branches + +Different audiences have different CLI needs: + +- **[for-developer](../branches/for-developer.en.md)**: also see IDE-based agents (Cursor, Cline, Continue) +- **[for-everyday-users](../branches/for-everyday-users.en.md)** Tier 2: CLI is the advanced option; try Tier 0 / 1 (Web / Desktop App) first +- **[for-researcher](../branches/for-researcher.en.md)**: also see paper-specific tools (paper-qa, gpt-researcher, ChatPaper) +- **[for-knowledge-worker](../branches/for-knowledge-worker.en.md)**: also see workflow automation (n8n, Make) +- **[for-teacher](../branches/for-teacher.en.md)**: CLI is advanced for teachers; start with prompt libraries + +--- + +## Maintenance Notes + +- 7 CLI tools' stars / license / pushed_at auto-refreshed weekly by the `weekly-catalog-refresh` CI (manual run: `python scripts/refresh-stars.py`) +- The CLI market moves fast — new tools require evaluation before inclusion (bar: 30k+ stars, actively maintained, true CLI not IDE) +- The comparison table deliberately leaves out "strengths / weaknesses" columns — avoiding subjective bias and letting the use-case section + readers' own judgment do that work diff --git a/resources/cli-agents-guide.md b/resources/cli-agents-guide.md new file mode 100644 index 0000000..1d79313 --- /dev/null +++ b/resources/cli-agents-guide.md @@ -0,0 +1,148 @@ +> **繁體中文** | [简体中文](./cli-agents-guide.zh-Hans.md) | [English](./cli-agents-guide.en.md) + +# CLI Agents 比較指南 + +> [← 回主路線 README](../README.md) + +> 📌 **這份是 reference doc**（深度比較、選擇邏輯、坑、推薦搭配）。 +> 第一次接觸 CLI agent、想要 step-by-step 上手 → 看 [`tracks/cli/A1-cli-intro.md`](../tracks/cli/A1-cli-intro.md)（Track A 第一站）。 +> 想先理解「為什麼有的 agent 在 terminal、有的在 Telegram、有的在 Jetson」這層 mental model → 看 [`resources/agent-paradigms.md`](agent-paradigms.md)（5 種 agent 型態）。 +> 已經在用、想決定 / 比較 / 升級 → 留在這份。 + +跨 5 個 branch + Track A 共用的參考——**Claude Code / Codex / OpenCode / Gemini CLI / goose / Aider / Hermes Agent 之間怎麼挑？** Track A（A1-A3）的 CLI workflow 設計、5 條 branch 內的 CLI 引用都連到這份；每個 branch 都會用到 CLI agent，但沒有一個 branch 真的「擁有」這份比較，所以放在 `resources/`。 + +--- + +## 📋 7 個主流 CLI agent + +只列在 terminal 跑的（IDE-based 如 Cursor / Cline / Continue 不在這份；那些放在 [for-developer](../branches/for-developer.md)）。前 6 個數字 `gh api` 驗證於 2026-05-06；Hermes Agent 驗證於 2026-05-10。 + +| 工具 | 提供者 | License | 主推 LLM | 認證 / 計費 | Stars | +|---|---|---|---|---|---| +| [Claude Code](https://github.com/anthropics/claude-code) | Anthropic（官方） | NOASSERTION | Claude | Claude 訂閱 **或** Anthropic Console API key | ★ 132k+ | +| [Codex](https://github.com/openai/codex) | OpenAI（官方） | Apache-2.0 | GPT 系列 | ChatGPT 帳號登入 **或** OpenAI API key | ★ 89k+ | +| [OpenCode](https://github.com/sst/opencode) | 社群（repo 已遷至 `anomalyco/opencode`） | MIT | 任意（多 provider） | BYO API key 或 OpenCode Zen 內建 hosted | ★ 171k+ | +| [Gemini CLI](https://github.com/google-gemini/gemini-cli) | Google（官方） | Apache-2.0 | Gemini | 免費額度寬，超出收費 | ★ 103k+ | +| [goose](https://github.com/block/goose) | Agentic AI Foundation（repo 已遷至 `aaif-goose/goose`） | Apache-2.0 | 15+ provider（含 Ollama） | BYO API key 或既有 Claude / ChatGPT / Gemini 訂閱（ACP） | ★ 47k+ | +| [Aider](https://github.com/Aider-AI/aider) | Aider-AI（社群） | Apache-2.0 | 任意 | BYO API key | ★ 44k+ | +| [Hermes Agent](https://github.com/NousResearch/hermes-agent) | Nous Research | MIT | 200+ via OpenRouter / NVIDIA NIM / 智譜 GLM / Kimi / 小米 MiMo / MiniMax / HF / OpenAI | BYO API key（多 provider） | ★ 193k+ | + +--- + +## 🎯 該選哪個？依 use case 決定 + +### 寫 paper / 文獻 / 研究 +**首推**：Claude Code（長 context、reasoning 強、擋幻覺扎實）。Gemini CLI 是備選——它的百萬 token 適合丟整本 PDF / dataset 進去問。 + +### 寫 code / 改 codebase +**首推**：Aider（git-native——每次改完自動 commit，方便 revert）或 Claude Code。OpenCode 適合需要在多 LLM 間切的場景。 + +### 隱私 / offline / 不送雲端 +**首推**：goose 或 OpenCode + 本地 Ollama。兩個都支援 BYO LLM，可以接 `http://localhost:11434/v1`（Ollama 預設）。 + +### 已訂 ChatGPT Plus / Pro +**首推**：Codex——同一個帳號就能用，不另外付費。 + +### 用 Google 生態 + 想要 1M token 長 context +**首推**：Gemini CLI。免費額度寬、長 context 是強項。注意：Google 服務（Gmail / Drive / Docs）的整合靠 MCP 擴充，不是內建——跟其他 CLI 一樣需要安裝 MCP server。 + +### 不想被 vendor lock-in +**首推**：OpenCode > goose > Aider。三個都不綁特定 provider，模型可換。 + +### 第一次裝 CLI agent，先試手感 +**首推**：Claude Code。生態廣泛、CLAUDE.md 機制讓 prompt 可以版本控制、出問題時社群資源多。 + +### 想跑在 cloud VM、用 Telegram / Slack 等多平台跟它聊 + 用中國大陸 LLM +**首推**：Hermes Agent。差異化在三件事： +- **不綁 laptop**——agent 跑在 $5 VPS / Modal serverless，你從 Telegram / Discord / Slack / WhatsApp / Signal 任一個介面對話 +- **多 LLM 中性**——支援 GLM / Kimi / 小米 MiMo / MiniMax，剛好對應 11 中文圈生態 +- **內建 self-improving skill loop + cron 排程**——agent 跟你互動久了會自動生成 skill，跨 session 持續優化 +- ⚠️ skill 自動演化是 frontier feature，目前缺獨立審計；對 production 任務建議先在低風險場景試 + +--- + +## 📝 跨 CLI 都通用的 prompt 寫法 + +如果想讓 prompt 在不同 CLI 之間 portable（或想隨時換工具不重寫），照這幾條原則： + +1. **明確指定檔案路徑**——「修改 `src/auth.py`」比「修改那個 auth 檔」好 +2. **要求多步驟拆解**——`先列 plan、確認後再動手`，所有 CLI 都吃這個結構 +3. **避免依賴特定 CLI 的 magic 指令**——`/init` `/compact` 是 Claude Code 專屬，OpenCode 沒有 +4. **用 `.cursorrules` / `CLAUDE.md` / `AGENTS.md` 記持續性偏好**——Claude Code 用 `CLAUDE.md`，Codex 用 `AGENTS.md`，OpenCode 用 `OPENCODE.md`，**內容可以一樣** +5. **明確要 review 的 scope**——「只 review 我這次的 diff」vs 「review 整個 repo」 + +跨 CLI 寫的 prompt 通常會比 CLI-specific prompt 麻煩 5-10%，但好處是切換工具時不用重寫。 + +--- + +## ⚠️ 常見坑 + +### File path 處理 +- Windows 路徑用反斜線（`C:\Users\...`），多數 CLI 內部會轉，但有時會搞混 +- 建議：在 git-bash / WSL 下用 forward slash，避免奇怪 quoting + +### git 整合差異 +- **Aider** 自動 commit 每次改動（這是它的設計，不是 bug） +- **Claude Code / Codex / OpenCode / goose** 預設不自動 commit，需要手動或 prompt 要求 + +### Sandbox 預設值（每個 CLI 文件略有差異，使用前請對照官方文件） +- **Claude Code**：bash 寫入預設限定 cwd，讀取範圍較廣（被 deny rule 排除的除外） +- **Codex**：版本控制目錄建議 `Auto`（workspace-write + on-request 提權）；非 git 目錄建議 `read-only` +- **goose / OpenCode**：相對寬鬆——建議自己加 sandbox / approval 設定，不要靠預設 + +### Token cost 累積 +- 在大 codebase 上跑 `grep` 一次可能消耗 10 萬+ token +- 在大 PDF 上摘要可能 50 萬 token（Gemini 適合，其他要 cost-aware） +- 建議：每次操作前估 cost；訂 monthly cap + +### 多 CLI session 互相干擾 +- 同一個 repo 開兩個 CLI（譬如 Claude Code + Aider），改檔可能 race condition +- 建議：一個 repo 一個 CLI（除非真的有並行需求） + +--- + +## 🔧 實用搭配（real-world setup） + +下面 3 個常見搭配，挑一個合的場景： + +### Setup A：Claude Code 主推 + OpenCode 備援 +- Claude Code 處理日常 90%（寫 code、寫 doc、debug） +- OpenCode 接 Ollama，處理隱私資料（醫療紀錄、財務分析） +- 一個 prompt 寫一次，兩邊都能跑 + +### Setup B：Codex（GPT）+ Aider（Claude）混用 +- Codex 處理 ChatGPT Plus 額度內的小事 +- Aider 接 Claude API key 處理大重構（git-native commit 方便） +- 兩個帳單分開算、互不影響 + +### Setup C：Gemini CLI 主推（給長 context 場景） +- 整本 PDF / 整個 codebase 一次餵進去 +- 加 Aider 處理需要精準 git diff 的場景 +- 適合學者、知識工作者 + +### Setup D：Hermes Agent + 本機 Ollama（多平台 + 中國大陸 LLM + offline） +- **Hermes Agent** 跑在 $5 VPS 或自己的機器上，當作多平台 agent gateway +- **LLM endpoint** 用 Ollama（`http://localhost:11434/v1`），也可以改接 z.ai GLM / Kimi 等 provider +- **聊天入口** 用 Telegram / Slack / Discord；Hermes 負責把平台訊息轉進 agent workflow +- **完全不想接 Anthropic / OpenAI** 時，這條路線適合做離線、隱私資料、低成本重複實驗 +- Step-by-step 做法看 [`resources/cookbook.md` Recipe 6](cookbook.md#6-本機-llm--cli-agent-快速-walkthrough) + +--- + +## 從這份指南連回各 branch + +不同 audience 對 CLI 的需求不一樣： + +- **[for-developer](../branches/for-developer.md)**：除了 CLI，也看 IDE-based agents（Cursor、Cline、Continue） +- **[for-everyday-users](../branches/for-everyday-users.md)** Tier 2：CLI 是進階選項，先試 Tier 0 / 1 的 Web / Desktop App +- **[for-researcher](../branches/for-researcher.md)**：除了 CLI，也看 paper-specific 工具（paper-qa、gpt-researcher、ChatPaper） +- **[for-knowledge-worker](../branches/for-knowledge-worker.md)**：除了 CLI，也看 workflow 自動化（n8n、Make） +- **[for-teacher](../branches/for-teacher.md)**：CLI 對教師偏進階；建議先看 prompt 素材庫 + +--- + +## 維護備註 + +- 7 個 CLI 的 stars / license / pushed_at 由 `weekly-catalog-refresh` CI 每週自動更新（手動可跑 `python scripts/refresh-stars.py`） +- CLI 工具市場變化快——新工具出現要評估是否加入這份比較（門檻：> 30k stars + 維護中 + 真的 CLI 不是 IDE） +- 比較表格的「強項 / 弱項」欄位刻意沒填——避免產生主觀 bias，讓 use case section 跟讀者自己的判斷做這件事 diff --git a/resources/cli-agents-guide.zh-Hans.md b/resources/cli-agents-guide.zh-Hans.md new file mode 100644 index 0000000..345499f --- /dev/null +++ b/resources/cli-agents-guide.zh-Hans.md @@ -0,0 +1,146 @@ +> [繁體中文](./cli-agents-guide.md) | **简体中文** | [English](./cli-agents-guide.en.md) + +# CLI Agents 比较指南 + +> [← 回主路线 README](../README.md) + +> 📌 **这份是 reference doc**（深度比较、选择逻辑、坑、推荐搭配）。 +> 第一次接触 CLI agent、想要 step-by-step 上手 → 看 [`tracks/cli/A1-cli-intro.zh-Hans.md`](../tracks/cli/A1-cli-intro.zh-Hans.md)（Track A 第一站）。 +> 想先理解“为什么有的 agent 在 terminal、有的在 Telegram、有的在 Jetson”这层 mental model → 看 [`resources/agent-paradigms.zh-Hans.md`](agent-paradigms.zh-Hans.md)（5 种 agent 型态）。 +> 已经在用、想决定 / 比较 / 升级 → 留在这份。 + +跨 5 个 branch + Track A 共用的参考——**Claude Code / Codex / OpenCode / Gemini CLI / goose / Aider / Hermes Agent 之间怎么挑？** Track A（A1-A3）的 CLI workflow 设计、5 条 branch 内的 CLI 引用都连到这份；每个 branch 都会用到 CLI agent，但没有一个 branch 真的“拥有”这份比较，所以放在 `resources/`。 + +## 📋 7 个主流 CLI agent + +只列在 terminal 跑的（IDE-based 如 Cursor / Cline / Continue 不在这份；那些放在 [for-developer](../branches/for-developer.zh-Hans.md)）。前 6 个数字 `gh api` 验证于 2026-05-06；Hermes Agent 验证于 2026-05-10。 + +| 工具 | 提供者 | License | 主推 LLM | 认证 / 计费 | Stars | +|---|---|---|---|---|---| +| [Claude Code](https://github.com/anthropics/claude-code) | Anthropic（官方） | NOASSERTION | Claude | Claude 订阅 **或** Anthropic Console API key | ★ 132k+ | +| [Codex](https://github.com/openai/codex) | OpenAI（官方） | Apache-2.0 | GPT 系列 | ChatGPT 账号登录 **或** OpenAI API key | ★ 89k+ | +| [OpenCode](https://github.com/sst/opencode) | 社群（repo 已迁至 `anomalyco/opencode`） | MIT | 任意（多 provider） | BYO API key 或 OpenCode Zen 内建 hosted | ★ 171k+ | +| [Gemini CLI](https://github.com/google-gemini/gemini-cli) | Google（官方） | Apache-2.0 | Gemini | 免费额度宽，超出收费 | ★ 103k+ | +| [goose](https://github.com/block/goose) | Agentic AI Foundation（repo 已迁至 `aaif-goose/goose`） | Apache-2.0 | 15+ provider（含 Ollama） | BYO API key 或既有 Claude / ChatGPT / Gemini 订阅（ACP） | ★ 47k+ | +| [Aider](https://github.com/Aider-AI/aider) | Aider-AI（社群） | Apache-2.0 | 任意 | BYO API key | ★ 44k+ | +| [Hermes Agent](https://github.com/NousResearch/hermes-agent) | Nous Research | MIT | 200+ via OpenRouter / NVIDIA NIM / 智谱 GLM / Kimi / 小米 MiMo / MiniMax / HF / OpenAI | BYO API key（多 provider） | ★ 193k+ | + +--- + +## 🎯 该选哪个？依 use case 决定 + +### 写 paper / 文献 / 研究 +**首推**：Claude Code（长 context、reasoning 强、挡幻觉扎实）。Gemini CLI 是备选——它的百万 token 适合丢整本 PDF / dataset 进去问。 + +### 写 code / 改 codebase +**首推**：Aider（git-native——每次改完自动 commit，方便 revert）或 Claude Code。OpenCode 适合需要在多 LLM 间切的场景。 + +### 隐私 / offline / 不送云端 +**首推**：goose 或 OpenCode + 本地 Ollama。两个都支持 BYO LLM，可以接 `http://localhost:11434/v1`（Ollama 默认）。 + +### 已订 ChatGPT Plus / Pro +**首推**：Codex——同一个账号就能用，不另外付费。 + +### 用 Google 生态 + 想要 1M token 长 context +**首推**：Gemini CLI。免费额度宽、长 context 是强项。注意：Google 服务（Gmail / Drive / Docs）的集成靠 MCP 扩展，不是内建——跟其他 CLI 一样需要安装 MCP server。 + +### 不想被 vendor lock-in +**首推**：OpenCode > goose > Aider。三个都不绑特定 provider，模型可换。 + +### 第一次装 CLI agent，先试手感 +**首推**：Claude Code。生态广泛、CLAUDE.md 机制让 prompt 可以版本控制、出问题时社群资源多。 + +### 想跑在 cloud VM、用 Telegram / Slack 等多平台跟它聊 + 用中国大陆 LLM +**首推**：Hermes Agent。差异化在三件事： +- **不绑 laptop**——agent 跑在 $5 VPS / Modal serverless，你从 Telegram / Discord / Slack / WhatsApp / Signal 任一个介面对话 +- **多 LLM 中性**——支持 GLM / Kimi / 小米 MiMo / MiniMax，刚好对应 11 中文圈生态 +- **内建 self-improving skill loop + cron 排程**——agent 跟你互动久了会自动生成 skill，跨 session 持续优化 +- ⚠️ skill 自动演化是 frontier feature，目前缺独立审计；对 production 任务建议先在低风险场景试 + +--- + +## 📝 跨 CLI 都通用的 prompt 写法 + +如果想让 prompt 在不同 CLI 之间 portable（或想随时换工具不重写），照这几条原则： + +1. **明确指定文件路径**——“修改 `src/auth.py`”比“修改那个 auth 档”好 +2. **要求多步骤拆解**——`先列 plan、确认后再动手`，所有 CLI 都吃这个结构 +3. **避免依赖特定 CLI 的 magic 指令**——`/init` `/compact` 是 Claude Code 专属，OpenCode 没有 +4. **用 `.cursorrules` / `CLAUDE.md` / `AGENTS.md` 记持续性偏好**——Claude Code 用 `CLAUDE.md`，Codex 用 `AGENTS.md`，OpenCode 用 `OPENCODE.md`，**内容可以一样** +5. **明确要 review 的 scope**——“只 review 我这次的 diff”vs “review 整个 repo” + +跨 CLI 写的 prompt 通常会比 CLI-specific prompt 麻烦 5-10%，但好处是切换工具时不用重写。 + +--- + +## ⚠️ 常见坑 + +### File path 处理 +- Windows 路径用反斜线（`C:\Users\...`），多数 CLI 内部会转，但有时会搞混 +- 建议：在 git-bash / WSL 下用 forward slash，避免奇怪 quoting + +### git 集成差异 +- **Aider** 自动 commit 每次改动（这是它的设计，不是 bug） +- **Claude Code / Codex / OpenCode / goose** 默认不自动 commit，需要手动或 prompt 要求 + +### Sandbox 默认值（每个 CLI 文件略有差异，使用前请对照官方文件） +- **Claude Code**：bash 写入默认限定 cwd，读取范围较广（被 deny rule 排除的除外） +- **Codex**：版本控制目录建议 `Auto`（workspace-write + on-request 提权）；非 git 目录建议 `read-only` +- **goose / OpenCode**：相对宽松——建议自己加 sandbox / approval 设置，不要靠默认 + +### Token cost 累积 +- 在大 codebase 上跑 `grep` 一次可能消耗 10 万+ token +- 在大 PDF 上摘要可能 50 万 token（Gemini 适合，其他要 cost-aware） +- 建议：每次操作前估 cost；订 monthly cap + +### 多 CLI session 互相干扰 +- 同一个 repo 开两个 CLI（譬如 Claude Code + Aider），改档可能 race condition +- 建议：一个 repo 一个 CLI（除非真的有并行需求） + +--- + +## 🔧 实用搭配（real-world setup） + +下面 3 个常见搭配，挑一个合的场景： + +### Setup A：Claude Code 主推 + OpenCode 备援 +- Claude Code 处理日常 90%（写 code、写 doc、debug） +- OpenCode 接 Ollama，处理隐私数据（医疗记录、财务分析） +- 一个 prompt 写一次，两边都能跑 + +### Setup B：Codex（GPT）+ Aider（Claude）混用 +- Codex 处理 ChatGPT Plus 额度内的小事 +- Aider 接 Claude API key 处理大重构（git-native commit 方便） +- 两个账单分开算、互不影响 + +### Setup C：Gemini CLI 主推（给长 context 场景） +- 整本 PDF / 整个 codebase 一次喂进去 +- 加 Aider 处理需要精确 git diff 的场景 +- 适合学者、知识工作者 + +### Setup D：Hermes Agent + 本地 Ollama（多平台 + 中国大陆 LLM + offline） +- **Hermes Agent** 跑在 $5 VPS 或自己的机器上，当作多平台 agent gateway +- **LLM endpoint** 用 Ollama（`http://localhost:11434/v1`），也可以改接 z.ai GLM / Kimi 等 provider +- **聊天入口** 用 Telegram / Slack / Discord；Hermes 负责把平台消息转进 agent workflow +- **完全不想接 Anthropic / OpenAI** 时，这条路线适合做离线、隐私资料、低成本重复实验 +- Step-by-step 做法看 [`resources/cookbook.md` Recipe 6](cookbook.zh-Hans.md#6-本地-llm--cli-agent-快速-walkthrough) + +--- + +## 从这份指南连回各 branch + +不同 audience 对 CLI 的需求不一样： + +- **[for-developer](../branches/for-developer.zh-Hans.md)**：除了 CLI，也看 IDE-based agents（Cursor、Cline、Continue） +- **[for-everyday-users](../branches/for-everyday-users.zh-Hans.md)** Tier 2：CLI 是进阶选项，先试 Tier 0 / 1 的 Web / Desktop App +- **[for-researcher](../branches/for-researcher.zh-Hans.md)**：除了 CLI，也看 paper-specific 工具（paper-qa、gpt-researcher、ChatPaper） +- **[for-knowledge-worker](../branches/for-knowledge-worker.zh-Hans.md)**：除了 CLI，也看 workflow 自动化（n8n、Make） +- **[for-teacher](../branches/for-teacher.zh-Hans.md)**：CLI 对教师偏进阶；建议先看 prompt 素材库 + +--- + +## 维护备注 + +- 7 个 CLI 的 stars / license / pushed_at 由 `weekly-catalog-refresh` CI 每周自动更新（手动可跑 `python scripts/refresh-stars.py`） +- CLI 工具市场变化快——新工具出现要评估是否加入这份比较（门槛：> 30k stars + 维护中 + 真的 CLI 不是 IDE） +- 比较表格的“强项 / 弱项”栏位刻意没填——避免产生主观 bias，让 use case section 跟读者自己的判断做这件事 diff --git a/resources/cookbook.en.md b/resources/cookbook.en.md new file mode 100644 index 0000000..82d303c --- /dev/null +++ b/resources/cookbook.en.md @@ -0,0 +1,619 @@ +# Cookbook — Turn Concepts into Executable Recipes + +> [繁體中文](./cookbook.md) | [简体中文](./cookbook.zh-Hans.md) | **English** + +> Stage 5 (Claude Code Ecosystem) talks about "Concepts" and "Available Tools" with [`mcp-skills-catalog.md`](mcp-skills-catalog.md). This cookbook fills in the gap in between: "**How to build it**". Each recipe is a step-by-step guide + sample code + common pitfalls, designed to be completed in about 30-50 minutes. +> +> This is not a reference or a tutorial—it's a recipe. Pick the one you need and start cooking. + +--- + +## 📋 Table of Contents + +1. [Write Your First Skill (SKILL.md Anatomy)](#1-write-your-first-skill) +2. [Write Your First MCP Server (Python SDK)](#2-write-your-first-mcp-server) +3. [Office Docs Workflow](#3-office-docs-workflow) +4. [NotebookLM Workflow](#4-notebooklm-workflow) +5. [Zotero Workflow](#5-zotero-workflow) +6. [Local LLM + CLI Agent Quick Walkthrough](#6-local-llm--cli-agent-quick-walkthrough) + +--- + +## 1. Write Your First Skill + +> A Skill is a folder containing `SKILL.md`, which Claude Code discovers automatically upon startup and loads contextually. The minimum viable version can run with as few as 50 lines of code. + +### Why + +The difference between writing a Skill and adding a few instructions within a prompt lies in: +- Skills are **per-domain**, meaning they don't pollute all conversations. +- They can be packaged and shared across projects or teams. +- Claude decides when to load them (based on whether the description matches the context). + +### Steps + +#### Step 1: Create the Skill Folder + +You can place skills in two locations (depending on whether you want user-level or project-level scope): + +```bash +# User-level (shared across all projects) +mkdir -p ~/.claude/skills/my-first-skill +cd ~/.claude/skills/my-first-skill + +# Or Project-level (triggered only within this repo) +mkdir -p .claude/skills/my-first-skill +cd .claude/skills/my-first-skill +``` + +#### Step 2: Write `SKILL.md` + +A minimal, working template: + +```markdown +--- +name: my-first-skill +description: When the user asks for [SPECIFIC SITUATION], use this skill to [WHAT IT DOES]. Examples include [2-3 trigger phrases]. Do NOT use for [WHAT IT'S NOT FOR]. +--- + +# My First Skill + +You are now in the [domain] context. + +## When the user asks X, do these steps: + +1. First, [action A] +2. Then, [action B] +3. Verify with [check] + +## Don't do: + +- [anti-pattern 1] +- [anti-pattern 2] + +## Reference + +- (optional) link to a doc / paper / API spec +``` + +A concrete example: "Organize Python imports by PEP 8 order" + +```markdown +--- +name: python-import-organizer +description: When the user pastes Python code or asks to clean up imports / format code / sort imports, organize the imports following PEP 8 + isort order: stdlib first, then third-party, then local. Do NOT use for non-Python code. +--- + +# Python Import Organizer + +When the user wants Python imports cleaned up: + +1. Group imports into 3 sections: stdlib / third-party / local +2. Within each group, sort alphabetically +3. Add a blank line between groups +4. Remove unused imports (only if user explicitly asks; otherwise just sort) + +## Don't: +- Don't change function code, only the import block +- Don't auto-remove imports without asking +``` + +#### Step 3: Test + +```bash +# Restart Claude Code (to re-discover skills) +# Provide a trigger phrase in the conversation +# e.g., "Help me organize the imports in this Python code." +# Observe if Claude follows the steps in SKILL.md +``` + +#### Step 4 (Advanced): Add Evals + +Add `evals/evals.json` within the skill folder: + +```json +{ + "evals": [ + { + "input": "Organize the imports in this Python code: import os +import requests +from mypackage import foo", + "expected_behavior": ["Group by stdlib / third-party / local", "Sort alphabetically"] + } + ] +} +``` + +You can then use tools like promptfoo for batch testing. + +### Common Pitfalls + +| Symptom | Cause | Solution | +|---|---|---| +| Claude never triggers my skill | `description` is too generic, not matching user queries | Add 2-3 specific trigger phrases to `description` (e.g., "when the user asks X / Y / Z") | +| Triggers but behaves incorrectly | Skill steps in `SKILL.md` are too abstract | Change to a numbered list, with each step being a clear action | +| Triggers when it shouldn't | `description` is too broad, matching unrelated queries | Add "Do NOT use for X" to narrow down the scope | + +### Further Reading + +- See [Stage 5.3](../stages/05-claude-code-ecosystem.en.md#53--skills-claude-codes-behavior-layer--the-most-critical-layer-of-the-claude-code-ecosystem) for a detailed explanation of Skill anatomy. +- Refer to the official skill templates in [`anthropics/skills`](https://github.com/anthropics/skills) (for docx / xlsx / pptx, etc.) for examples. +- Package multiple skills into a plugin → [Stage 5.4](../stages/05-claude-code-ecosystem.en.md#54--plugins--marketplaces) + +--- + +## 2. Write Your First MCP Server + +> An MCP server is a standalone process that provides tools / resources / prompts to an LLM host (Claude Desktop / Claude Code). The minimal runnable version is less than 50 lines of Python. + +### Why + +- Skills are for Claude's "role + rules"; MCPs are for Claude's "**external functions**". +- Skills cannot read files or call APIs; MCPs can (any tool you can script). +- Skills only run within Claude Code; MCPs can be used by any LLM host (including custom agents). + +### Steps + +#### Step 1: Install the Official SDK + +```bash +pip install mcp +``` + +#### Step 2: Write `server.py` + +A minimal template for an echo tool: + +```python +# server.py +from mcp.server import Server +from mcp.server.stdio import stdio_server +from mcp.types import Tool, TextContent + +app = Server("hello-mcp") + +@app.list_tools() +async def list_tools() -> list[Tool]: + return [ + Tool( + name="echo", + description="Echo the input text back to the user.", + inputSchema={ + "type": "object", + "properties": { + "text": { + "type": "string", + "description": "Text to echo back", + } + }, + "required": ["text"], + }, + ) + ] + +@app.call_tool() +async def call_tool(name: str, arguments: dict) -> list[TextContent]: + if name == "echo": + return [TextContent(type="text", text=f"Echo: {arguments['text']}")] + raise ValueError(f"Unknown tool: {name}") + +async def main(): + async with stdio_server() as (read, write): + await app.run(read, write, app.create_initialization_options()) + +if __name__ == "__main__": + import asyncio + asyncio.run(main()) +``` + +#### Step 3: Configure in Claude Desktop / Code + +**Claude Desktop**: Edit `~/Library/Application Support/Claude/claude_desktop_config.json` (macOS) or `%APPDATA%\Claude\claude_desktop_config.json` (Windows): + +```json +{ + "mcpServers": { + "hello-mcp": { + "command": "python", + "args": ["/absolute/path/to/server.py"] + } + } +} +``` + +**Claude Code**: Use the `claude mcp add` command: + +```bash +claude mcp add hello-mcp python /absolute/path/to/server.py +``` + +#### Step 4: Restart Claude Desktop / Code and Test + +``` +You ask: echo "hello world" to me +Claude replies (with a tool call icon): Echo: hello world +``` + +### Common Pitfalls + +| Symptom | Cause | Solution | +|---|---|---| +| Claude Desktop doesn't see the tool | `server.py` failed to start | Run `python server.py` directly in the terminal and check `stderr` for errors | +| Tool is listed but call fails | Incorrect `inputSchema` format (missing `required` fields, wrong `type`) | Refer to [`schema-design-cheatsheet.md`](schema-design-cheatsheet.en.md) | +| Claude doesn't proactively call the tool | `description` is too generic | Refine `description` to be specific trigger phrases like "When the user asks X, use this tool" | +| stdio vs. SSE? | `stdio` is for local desktop integration; `SSE` is for remote/web | Always use `stdio` for the first server. | + +### Further Reading + +- See [Stage 5.2](../stages/05-claude-code-ecosystem.en.md#52--mcp-model-context-protocol--foundation) for a full introduction to MCP. +- Refer to the official examples in [`modelcontextprotocol/servers`](https://github.com/modelcontextprotocol/servers) (e.g., filesystem, github, sqlite, time). +- For production servers, see [Stage 5.2 "Practice: MCP in production"](../stages/05-claude-code-ecosystem.en.md#52--mcp-model-context-protocol--foundation) and the `~/.claude/skills/` examples in [`anthropics/claude-code`](https://github.com/anthropics/claude-code). + +--- + +## 3. Office Docs Workflow + +> Use Claude to read and write Word / Excel / PowerPoint / PDF files without installing extra tools—the official [`anthropics/skills`](https://github.com/anthropics/skills) repo already includes them. + +### Why + +Common scenarios include: +- Generating a Word / PPT document from Markdown / an outline. +- Summarizing / extracting data from multiple PDFs / Excel files. +- Editing received `.docx` files (e.g., adding track changes, reformatting). +- Cross-referencing tables to generate reports. + +You don't need to parse XML or find tutorials for `python-docx` / `openpyxl` – `anthropics/skills` has it covered. + +### Steps + +#### Step 1: Install Skills + +The simplest way is to clone the official Anthropic skills repo to your user-level skill directory: + +```bash +# User-level (for all projects) +git clone https://github.com/anthropics/skills.git ~/.claude/skills/anthropic-skills +``` + +Alternatively, use `claude plugin install` if it's packaged as a plugin. + +#### Step 2: Restart Claude Code + +- `skills/docx/` → Read/write DOCX files +- `skills/xlsx/` → Read/write Excel files +- `skills/pptx/` → Read/write PowerPoint files +- `skills/pdf/` → Read PDFs + +Claude will automatically load the appropriate skill based on your query. + +#### Step 3: Practical Prompt Templates + +**Generate PPT from Outline**: +``` +Read my outline.md, and generate a PPT based on this structure: +- 1 title slide +- 1 slide per H2, with bullet points condensed from H3 content +- 1 conclusion slide + +Save as ./output/presentation.pptx +``` + +**Read Excel to Summarize Data**: +``` +Read the first sheet of ./data/sales-2023.xlsx, calculate the total Q4 sales for each region, +and write it into ./output/q4-summary.md using a markdown table format. +``` + +**Edit DOCX**: +``` +Read ./doc/draft.docx, change all instances of "使用者" to "用户" (zh-Hans translation), +and save as ./doc/draft.zh-Hans.docx, preserving the original track changes. +``` + +**Read PDF and Extract Information**: +``` +Read ./papers/research.pdf, extract the abstract, main contributions, and limitations, +and write each into separate markdown sections in ./notes/research-summary.md. +``` + +### Common Pitfalls + +| Symptom | Cause | Solution | +|---|---|---| +| Skill not triggered | Incorrect repo path | Ensure `SKILL.md` is at a level like `~/.claude/skills/anthropic-skills/skills/docx/SKILL.md` | +| Generated PPT has ugly styling | No design reference provided | Add "Use ./template.pptx as a style reference" to the prompt | +| Large PDFs are not fully read | Context window limitation | Use [`SylphxAI/pdf-reader-mcp`](https://github.com/SylphxAI/pdf-reader-mcp) (5-10x faster) | +| Excel formulas are lost | `docx` skill doesn't handle formulas | Prompt explicitly "preserve formulas, do not hard-code values" before opening the file | + +### Further Reading + +- Catalog 2 [`mcp-skills-catalog.en.md` 2 Office Documents](mcp-skills-catalog.en.md#2-office-documents-word--excel--powerpoint--pdf): Enhanced office skills / dedicated MCP for Excel / PPT. +- Office workflow in Chinese: [`leemysw/feishu-docx`](https://github.com/leemysw/feishu-docx) for Feishu / Lark docs ↔ Markdown. + +--- + +## 4. NotebookLM Workflow + +> NotebookLM is Google's RAG-on-your-docs tool. **Claude Code does not have official NotebookLM integration**, but there are two mature community solutions. + +### Why + +NotebookLM's strengths: +- Automatically indexes up to 50 uploaded PDFs. +- Provides Q&A with citations (each answer links to the source document and page number). +- Generates summaries, mind maps, or podcast-style audio overviews. + +Its weakness: It's used via the NotebookLM web interface, disconnecting it from your other workflows (Claude Code, Obsidian, Zotero). + +Two solutions bridge this gap: +1. **PleasePrompto/notebooklm-skill** (Skill, browser automation) +2. **teng-lin/notebooklm-py** (Python API + CLI) + +### Choosing Between the Two Solutions + +| Scenario | Choose This | Why | +|---|---|---| +| Occasionally query NotebookLM from Claude Code | `PleasePrompto/notebooklm-skill` | Single prompt in Claude Code to run; simple setup. | +| Batch operations (e.g., create 100 notebooks, import documents in bulk) | `teng-lin/notebooklm-py` | Python API for programmatic execution. | +| Avoid breaking due to Google policy changes | (Wait for an official Google API) | Both solutions are unofficial and subject to breaking changes. | + +### Solution A: PleasePrompto/notebooklm-skill + +#### Step 1: Clone to skills Directory + +```bash +git clone https://github.com/PleasePrompto/notebooklm-skill ~/.claude/skills/notebooklm +``` + +#### Step 2: First Run Requires Google Login (Browser Automation) + +Follow the repo's README to set up OAuth or login cookies. + +#### Step 3: Practical Prompts + +``` +Search my NotebookLM notebooks for ones related to "LLM Agents 2024". +Find all paragraphs mentioning "tool use" and organize them into a comparison table, +including the filename and page number for each source. +``` + +### Solution B: teng-lin/notebooklm-py + +```bash +pip install notebooklm-py +``` + +Example: + +```python +from notebooklm import NotebookLM +nlm = NotebookLM() # OAuth flow + +# Create a notebook +nb = nlm.create_notebook("My Research") + +# Batch import PDFs +for pdf in glob.glob("papers/*.pdf"): + nb.add_source(pdf) + +# Q&A +answer = nb.query("What are the main contributions?") +print(answer.text) +print(answer.citations) +``` + +### Common Pitfalls + +| Symptom | Cause | Solution | +|---|---|---| +| Suddenly stops working | Google changed its internal API | Check the issue tracker; wait for community updates | +| Q&A answers are vague | Too many sources uploaded, retrieval is inaccurate | Split into multiple notebooks (each with < 50 sources) | +| Poor Chinese support | Default UI set to English | Change NotebookLM settings to zh-Hant | + +### Further Reading + +- Catalog 1 [`mcp-skills-catalog.en.md` 1 Notes / Knowledge Base](mcp-skills-catalog.en.md#1-notes--knowledge-base) +- Complete research workspace: Integrate NotebookLM + Zotero + Obsidian using [`WenyuChiou/research-hub`](https://github.com/WenyuChiou/research-hub). + +--- + +## 5. Zotero Workflow + +> Zotero manages your literature. With [`WenyuChiou/zotero-skills`](https://github.com/WenyuChiou/zotero-skills), Claude Code can directly search, add, categorize, and tag references. + +### Why + +Classic pain points in the research workflow: +- "Where is that paper?" — Zotero has it, but requires switching windows. +- "Give me summaries of all papers discussing transformers." — Requires manual selection, export, then feeding to an LLM. +- "What tags should I add to this paper?" — Manual process. + +`zotero-skills` turns these into single prompts within Claude Code. + +### Difference from zotero-gpt + +| Tool | Role | Best For | +|---|---|---| +| [`MuiseDestiny/zotero-gpt`](https://github.com/MuiseDestiny/zotero-gpt) | Zotero plugin (chat **inside** Zotero) | Asking LLM questions while reading papers, without switching windows. | +| [`WenyuChiou/zotero-skills`](https://github.com/WenyuChiou/zotero-skills) | Claude Code skill (operates Zotero from **outside**) | Primarily using Claude Code for paper writing / literature review. | + +They are complementary and not mutually exclusive; you can install both. + +### Steps + +#### Step 1: Enable Zotero Local API + +Zotero's desktop app doesn't enable the API by default. Enable it: +- **Edit → Preferences → Advanced → Config Editor** +- Find `extensions.zotero.httpServer.enabled` and set it to `true`. +- Find `extensions.zotero.httpServer.port`; the default is `23119`. + +#### Step 2: Clone zotero-skills + +```bash +git clone https://github.com/WenyuChiou/zotero-skills ~/.claude/skills/zotero-skills +``` + +Follow the repo's README for setup, including API key configuration for write operations via the Web API. + +#### Step 3: Practical Prompts + +**Search Literature**: +``` +Search my Zotero library for all papers published after 2023 related to multi-agent systems, +sort them by cited count, and output as a markdown table. +``` + +**Automatic Categorization**: +``` +Review the 50 papers in my "Inbox" collection in Zotero, automatically create sub-collections based on topics +(e.g., "RAG", "Tool Use", "Multi-Agent"), and move the papers into them. +``` + +**Tagging Papers**: +``` +Read this paper in my Zotero (after reviewing its attached PDF), +extract 5 keywords from the abstract to use as tags. +``` + +**Organize Citations for Paper Writing**: +``` +My paper draft is in ./paper/v3.tex. Find all \cite{} entries, compare them against my Zotero library, +and export any missing BibTeX entries as a .bib file for me. +``` + +### Common Pitfalls + +| Symptom | Cause | Solution | +|---|---|---| +| Skill triggers but query fails | Zotero not running / API not enabled | Run the Zotero desktop app + confirm port 23119 is listening | +| Write operations (add/move) fail | Local API is read-only; requires Web API | Configure the Web API key ([zotero.org/settings/keys](https://www.zotero.org/settings/keys)) | +| Collection structure becomes messy | Prompt for auto-categorization lacked directory structure context | Provide Claude with the existing collection tree in the prompt before asking it to categorize. | + +### Further Reading + +- Complete research workspace: Integrate Zotero + Obsidian + NotebookLM using [`WenyuChiou/research-hub`](https://github.com/WenyuChiou/research-hub). +- Academic paper writing: [`WenyuChiou/academic-writing-skills`](https://github.com/WenyuChiou/academic-writing-skills). +- Collection of 14 research workflow skills: [`WenyuChiou/ai-research-skills`](https://github.com/WenyuChiou/ai-research-skills). + +--- + +## 6. Local LLM + CLI Agent Quick Walkthrough + +> In about 30 minutes, connect Stage 1's local model setup to a Stage 5 CLI agent: useful for offline work, privacy-sensitive files, and experiments where you do not want to spend API quota. + +### Why + +Stage 1 teaches local LLM runtimes such as Ollama / llama.cpp / vLLM. Stage 5 teaches the Claude Code, MCP, Skills, and Plugins ecosystem. The common misunderstanding between them: **Claude Code is not a local LLM runner**. Claude Code requires Anthropic OAuth / API credentials; it cannot directly switch its model endpoint to Ollama or another local endpoint. + +If your goal is "local LLM + CLI agent", choose a CLI that supports BYO LLM instead. **OpenCode / goose / Aider / Hermes Agent** can connect to an OpenAI-compatible endpoint or an Ollama provider. This recipe gives you a short path to validate the model, the agent, and one real task. + +### Steps + +#### Step 1: Ollama + model (10 minutes) + +```bash +# Install Ollama: https://ollama.com +ollama pull qwen2.5:3b +# On 16GB+ RAM, you can also try: ollama pull qwen2.5:7b +ollama serve +``` + +Confirm the OpenAI-compatible API responds: + +```bash +curl http://localhost:11434/v1/chat/completions \ + -H "Content-Type: application/json" \ + -d '{"model":"qwen2.5:3b","messages":[{"role":"user","content":"Explain ReAct agents in 3 sentences."}]}' +``` + +#### Step 2: Pick one CLI agent and connect it to Ollama (10 minutes) + +**OpenCode**: good when you want provider switching plus local models. + +```bash +npm install -g opencode-ai +opencode auth login # choose Ollama, set endpoint to http://localhost:11434/v1 +opencode +``` + +**goose**: has an Ollama provider and is straightforward for local-agent trials. + +```bash +# Install instructions: https://block.github.io/goose +goose configure # choose Ollama, set model to qwen2.5:3b +goose session start +``` + +**Aider**: git-native, useful for small code edits inside a repository. + +```bash +pip install aider-chat +aider --model ollama/qwen2.5:3b --no-show-model-warnings +``` + +**Hermes Agent**: useful on a VPS when Telegram / Slack / Discord should be the agent front door. + +```bash +curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash +hermes model set ollama:qwen2.5:3b +hermes +``` + +#### Step 3: Run one real small task (10 minutes) + +Do not stop at "hello world". Pick a task that touches files, summarization, tables, or search: + +- Find 5 PDFs in `~/Downloads`, then extract one-sentence summary and method for each paper. +- Read the first 3 columns of `data.csv`, output a Markdown table, and flag column issues. +- Search `~/notes/` for paragraphs from the last 7 days mentioning `agent safety`, then turn them into a checklist. + +Watch three things: + +- **Speed**: small local models are often 2-5x slower than API models. +- **Quality**: 3B / 7B models usually trail Claude on reasoning, long context, and complex code. +- **Cost**: token cost is `$0`, but you spend local RAM / VRAM and power. + +#### Step 4: Compare with Claude Code (5 minutes) + +| Dimension | Claude Code | OpenCode + Ollama | +|---|---|---| +| LLM | Anthropic hosted | Local model | +| Cost | Subscription or per-token | `$0` token cost | +| Speed | Usually steadier | Hardware-dependent, often 2-5x slower | +| Privacy | Content goes to Anthropic | Content stays local | +| Reasoning ceiling | Stronger with Claude 4.8+ | Depends on the local model | +| Best use case | Complex codebases, long context, reliable reasoning | Private files, offline demos, low-cost repetition | + +### Important Limitation: Claude Code Cannot Directly Use a Local LLM + +Claude Code currently requires Anthropic OAuth / API credentials and has no official setting for replacing its model with Ollama or a local endpoint. You may see proxy or API-shim experiments online, but that is not the official supported path; stability and compatibility are yours to validate. + +For local LLM work, treat "Claude Code" and "BYO-LLM CLI agents" as separate tools: use Claude Code when you need Claude's quality; use OpenCode / goose / Aider / Hermes for local, offline, privacy-sensitive, or low-cost experiments. + +### Common Pitfalls + +| Problem | Cause | Fix | +|---|---|---| +| `connection refused` | Ollama server is not running in the background | Run `ollama serve` in another terminal | +| Model output is fragmented or weak | The 3B model is too small | Try `qwen2.5:7b` or `deepseek-r1:7b` | +| CLI agent does not edit files | Local model is too weak, or prompt is underspecified | Narrow the task, name the files, define success criteria | +| Memory / OOM | Model exceeds RAM / VRAM | Start with `qwen2.5:3b`, then move to 7B; enable swap if needed | + +### Further Reading + +- Stage 1 [Local LLM exercise](../stages/01-llm-basics.en.md#exercise-6-local-llm): Ollama / llama.cpp / vLLM tradeoffs +- [`cli-agents-guide.md`](cli-agents-guide.en.md): how to choose among 7 CLI agents +- Hermes Agent README: multi-platform gateway setup for Telegram / Discord / Slack and providers + +--- + +## Can't Find the Recipe You Need? + +- See [Stage 5](../stages/05-claude-code-ecosystem.md) for the full concept. +- See [`mcp-skills-catalog.md`](mcp-skills-catalog.en.md) for a comprehensive list of tools. +- See [`schema-design-cheatsheet.md`](schema-design-cheatsheet.en.md) for details on writing tool schemas. +- See [`cli-agents-guide.md`](cli-agents-guide.en.md) for a comparison of 7 popular CLI agents. + +Want a new recipe? Open an issue or submit a PR. Recipe format: **Why + Steps + Sample Prompt + Common Pitfalls + Further Reading**. diff --git a/resources/cookbook.md b/resources/cookbook.md new file mode 100644 index 0000000..bf9dc62 --- /dev/null +++ b/resources/cookbook.md @@ -0,0 +1,620 @@ +# Cookbook — 把概念變成可執行 recipe + +> **繁體中文** | [简体中文](./cookbook.zh-Hans.md) | [English](./cookbook.en.md) + +> Stage 5（Claude Code 生態）跟 [`mcp-skills-catalog.md`](mcp-skills-catalog.md) 講「概念」跟「有哪些工具」。這份 cookbook 補中間缺的：「**怎麼動手做出來**」。每個 recipe 是一份 step-by-step + sample code + 常見 pitfall，~30-50 分鐘做完一個。 +> +> 不是 reference 也不是 tutorial——是 recipe，挑你需要的那道煮就好。 + +--- + +## 📋 目錄 + +1. [寫你的第一個 Skill（SKILL.md anatomy）](#1-寫你的第一個-skill) +2. [寫你的第一個 MCP server（Python SDK）](#2-寫你的第一個-mcp-server) +3. [Word / Excel / PowerPoint workflow](#3-office-docs-workflow) +4. [NotebookLM workflow](#4-notebooklm-workflow) +5. [Zotero workflow](#5-zotero-workflow) +6. [本機 LLM + CLI Agent 快速 walkthrough](#6-本機-llm--cli-agent-快速-walkthrough) + +--- + +## 1. 寫你的第一個 Skill + +> Skill = 一個資料夾含 `SKILL.md`，Claude Code 啟動時自動 discover、按情境自動載入。最小 viable 版本 50 行就能跑。 +> +> 📚 **這份是「30 分鐘跑出第一個」實作版。想看「Skill 怎麼寫得好」深度討論** → [Hello-Agents Extra08：如何寫出好的 Skill](https://github.com/datawhalechina/hello-agents/blob/main/Extra-Chapter/Extra08-如何写出好的Skill.md)（中文最完整的 Skill 最佳實踐，討論 description 寫法、references / scripts 設計等）。兩份互補：先用本 recipe 跑出第一個，再讀那份 polish 寫法。 + +### 為什麼 + +寫 Skill 跟「在 prompt 裡加幾段 instruction」差別在於： +- Skill 是 **per-domain** 的，不會污染所有 conversation +- 可以打包跨 project / team 共用 +- Claude 自己決定何時載入（看 description match 不 match） + +### 步驟 + +#### Step 1：建立 skill 資料夾 + +兩個位置可以放（看你要 user 級還是 project 級）： + +```bash +# user 級（所有 project 共用） +mkdir -p ~/.claude/skills/my-first-skill +cd ~/.claude/skills/my-first-skill + +# 或 project 級（只在這個 repo 觸發） +mkdir -p .claude/skills/my-first-skill +cd .claude/skills/my-first-skill +``` + +#### Step 2：寫 `SKILL.md` + +最小可 work 的範本： + +```markdown +--- +name: my-first-skill +description: When the user asks for [SPECIFIC SITUATION], use this skill to [WHAT IT DOES]. Examples include [2-3 trigger phrases]. Do NOT use for [WHAT IT'S NOT FOR]. +--- + +# My First Skill + +You are now in the [domain] context. + +## When the user asks X, do these steps: + +1. First, [action A] +2. Then, [action B] +3. Verify with [check] + +## Don't do: + +- [anti-pattern 1] +- [anti-pattern 2] + +## Reference + +- (optional) link to a doc / paper / API spec +``` + +具體例子：「整理 Python 程式碼的 import 順序」 + +```markdown +--- +name: python-import-organizer +description: When the user pastes Python code or asks to clean up imports / format code / sort imports, organize the imports following PEP 8 + isort order: stdlib first, then third-party, then local. Do NOT use for non-Python code. +--- + +# Python Import Organizer + +When the user wants Python imports cleaned up: + +1. Group imports into 3 sections: stdlib / third-party / local +2. Within each group, sort alphabetically +3. Add a blank line between groups +4. Remove unused imports (only if user explicitly asks; otherwise just sort) + +## Don't: +- Don't change function code, only the import block +- Don't auto-remove imports without asking +``` + +#### Step 3：測試 + +```bash +# 重啟 Claude Code（讓它重新 discover skills） +# 在 conversation 裡丟一個觸發句 +# e.g.「幫我整理一下這段 Python 的 imports」 +# 觀察 Claude 有沒有按照 SKILL.md 的步驟做 +``` + +#### Step 4（進階）：加 evals + +在 skill folder 內加 `evals/evals.json`： + +```json +{ + "evals": [ + { + "input": "整理一下這段 Python 的 imports: import os\nimport requests\nfrom mypackage import foo", + "expected_behavior": ["按 stdlib / third-party / local 分組", "alphabetical 排序"] + } + ] +} +``` + +之後可以用 promptfoo 之類工具 batch 跑。 + +### 常見 pitfall + +| 症狀 | 原因 | 解法 | +|---|---|---| +| Claude 從不觸發我的 skill | description 寫得太籠統，匹配不到 user query | description 加 2-3 個具體 trigger phrase（"when the user asks X / Y / Z"） | +| 觸發了但行為不對 | SKILL.md 步驟太抽象 | 改成 numbered list、每步明確動作 | +| 觸發了不該觸發 | description 太寬，匹配到不相關 query | 加 "Do NOT use for X" 收斂 | + +### 進一步 + +- 看 [Stage 5.3](../stages/05-claude-code-ecosystem.md#53--skillsclaude-code-的行為層-claude-code-生態最關鍵的一層) 的 Skill anatomy 詳解 +- 看 [`anthropics/skills`](https://github.com/anthropics/skills) 官方 skill 範本（docx / xlsx / pptx 等）的寫法 +- 多個 skill 打包成 plugin → [Stage 5.4](../stages/05-claude-code-ecosystem.md#54--plugins-與-marketplaces) + +--- + +## 2. 寫你的第一個 MCP server + +> MCP server = 一個獨立 process，跑起來提供 tool / resource / prompt 給 LLM host（Claude Desktop / Claude Code）。最小可 run 版 < 50 行 Python。 + +### 為什麼 + +- Skill 是給 Claude 的「角色 + 規則」；MCP 是給 Claude 的「**外部 function**」 +- Skill 不能讀檔、不能呼叫 API；MCP 可以（任何 tool 你寫得出來） +- Skill 只在 Claude Code 跑；MCP 任何 LLM host（包括 Cursor、自寫 agent）都能接 + +### 步驟 + +#### Step 1：安裝官方 SDK + +```bash +pip install mcp +``` + +#### Step 2：寫 `server.py` + +最小範本——一個會回 echo 的 tool： + +```python +# server.py +from mcp.server import Server +from mcp.server.stdio import stdio_server +from mcp.types import Tool, TextContent + +app = Server("hello-mcp") + +@app.list_tools() +async def list_tools() -> list[Tool]: + return [ + Tool( + name="echo", + description="Echo the input text back to the user.", + inputSchema={ + "type": "object", + "properties": { + "text": { + "type": "string", + "description": "Text to echo back", + } + }, + "required": ["text"], + }, + ) + ] + +@app.call_tool() +async def call_tool(name: str, arguments: dict) -> list[TextContent]: + if name == "echo": + return [TextContent(type="text", text=f"Echo: {arguments['text']}")] + raise ValueError(f"Unknown tool: {name}") + +async def main(): + async with stdio_server() as (read, write): + await app.run(read, write, app.create_initialization_options()) + +if __name__ == "__main__": + import asyncio + asyncio.run(main()) +``` + +#### Step 3：在 Claude Desktop / Code 設定 + +**Claude Desktop**：編輯 `~/Library/Application Support/Claude/claude_desktop_config.json`（macOS）或 `%APPDATA%\Claude\claude_desktop_config.json`（Windows）： + +```json +{ + "mcpServers": { + "hello-mcp": { + "command": "python", + "args": ["/絕對路徑/到/server.py"] + } + } +} +``` + +**Claude Code**：用 `claude mcp add` 指令： + +```bash +claude mcp add hello-mcp python /絕對路徑/到/server.py +``` + +#### Step 4：重啟 Claude Desktop / Code、測試 + +``` +你問：echo "hello world" 給我 +Claude 回（會顯示 tool call icon）：Echo: hello world +``` + +### 常見 pitfall + +| 症狀 | 原因 | 解法 | +|---|---|---| +| Claude Desktop 沒看到 tool | server.py 啟動失敗 | terminal 直接 `python server.py` 跑、看 stderr 哪裡爆 | +| tool 列出但 call 失敗 | inputSchema 格式錯（required 漏寫、type 寫錯） | 看 [`schema-design-cheatsheet.md`](schema-design-cheatsheet.md) | +| Claude 不主動叫 tool | description 太籠統 | description 改成「When the user asks X, use this tool」式的具體 trigger | +| stdio 跟 SSE 哪個用？ | local desktop integration 用 stdio；remote / web 用 SSE | 第一個 server 一律用 stdio | + +### 進一步 + +- 看 [Stage 5.2](../stages/05-claude-code-ecosystem.md#52--mcpmodel-context-protocol-基礎) 的 MCP 完整介紹 +- 看 [`modelcontextprotocol/servers`](https://github.com/modelcontextprotocol/servers) 官方範例（filesystem、github、sqlite、time 等） +- 寫 production server 看 [Stage 5.2「練習：MCP in production」](../stages/05-claude-code-ecosystem.md#52--mcpmodel-context-protocol-基礎) 跟 [`anthropics/claude-code`](https://github.com/anthropics/claude-code) 的 `~/.claude/skills/` + +--- + +## 3. Office Docs Workflow + +> 用 Claude 讀寫 Word / Excel / PowerPoint / PDF 不用裝額外 tool——[`anthropics/skills`](https://github.com/anthropics/skills) 官方 repo 已經內建好。 + +### 為什麼 + +最常見場景： +- 把 Markdown / 大綱 → 自動生成 Word / PPT +- 讀一堆 PDF / Excel → 整理摘要 / 提取數字 +- 改別人傳來的 docx → 加 track changes、或重排格式 +- 把表格 cross-reference 寫成報告 + +不需要自己 parse XML、不需要找 python-docx / openpyxl 教學——anthropics/skills 已經包好。 + +### 步驟 + +#### Step 1：安裝 skills + +最簡單：clone Anthropic 官方 skills repo 到 user-level skill 目錄： + +```bash +# user 級（所有 project 用） +git clone https://github.com/anthropics/skills.git ~/.claude/skills/anthropic-skills +``` + +或者用 `claude plugin install`（如果有打包成 plugin）。 + +#### Step 2：重啟 Claude Code + +- skills/docx/ → docx 讀寫 +- skills/xlsx/ → Excel 讀寫 +- skills/pptx/ → PowerPoint 讀寫 +- skills/pdf/ → PDF 讀 + +Claude 會根據 user query 自動載入合適的 skill。 + +#### Step 3：實用 prompt 範本 + +**從大綱生 PPT**： +``` +讀我寫的 outline.md，照這個結構生一份 PPT： +- 封面 1 頁 +- 每個 H2 一頁，bullet points 從 H3 內容濃縮 +- 結語 1 頁 + +存成 ./output/presentation.pptx +``` + +**讀 Excel 整理數字**： +``` +讀 ./data/sales-2023.xlsx 第一張 sheet，把每個 region 的 Q4 總額算出來， +寫進 ./output/q4-summary.md（用 markdown table 格式）。 +``` + +**改 docx**： +``` +讀 ./doc/draft.docx，把繁中詞彙轉成簡中（譬如「軟體」→「软件」）， +存成 ./doc/draft.zh-Hans.docx，保留原本的 track changes。 +``` + +**讀 PDF 提取資訊**： +``` +讀 ./papers/research.pdf，把 abstract、main contributions、limitations +分別寫進三個 markdown section，存到 ./notes/research-summary.md。 +``` + +### 常見 pitfall + +| 症狀 | 原因 | 解法 | +|---|---|---| +| skill 沒被觸發 | repo 路徑放錯 | 確認 SKILL.md 在 `~/.claude/skills/anthropic-skills/skills/docx/SKILL.md` 這種層級 | +| pptx 生出來樣式醜 | 沒給設計參考 | prompt 加「參考 ./template.pptx 的樣式」 | +| 大 PDF 讀不完 | context 爆 | 改用 [`SylphxAI/pdf-reader-mcp`](https://github.com/SylphxAI/pdf-reader-mcp)（5-10× 快） | +| Excel 公式被吃掉 | docx skill 不處理 formulas | 開檔前 prompt 明說「保留 formula 不要 hard-code」 | + +### 進一步 + +- catalog 2 [`mcp-skills-catalog.md` 2 辦公文件](mcp-skills-catalog.md#2-辦公文件word--excel--powerpoint--pdf)：補強版 office skill / Excel / PPT 專用 MCP +- 中文圈 office workflow：[`leemysw/feishu-docx`](https://github.com/leemysw/feishu-docx) 飛書 / Lark docs ↔ Markdown + +--- + +## 4. NotebookLM Workflow + +> NotebookLM 是 Google 的 RAG-on-your-docs 工具。**Claude Code 沒有官方 NotebookLM 整合**，但社群有 2 個成熟方案。 + +### 為什麼 + +NotebookLM 強的地方： +- 上傳 50 份 PDF 自動建索引 +- Q&A 帶 citation（每個答案都標出來自哪份文件第幾頁） +- 生成 summary / mind map / podcast-style audio overview + +弱點：要在 NotebookLM 網頁裡用，跟你的其他 workflow（Claude Code、Obsidian、Zotero）斷開。 + +兩個方案橋接： +1. **PleasePrompto/notebooklm-skill**（Skill，browser automation） +2. **teng-lin/notebooklm-py**（Python API + CLI） + +### 兩個方案怎麼選 + +| 場景 | 選哪個 | 為什麼 | +|---|---|---| +| 偶爾從 Claude Code 查一下 NotebookLM | `PleasePrompto/notebooklm-skill` | Claude Code 內 prompt 一句話就跑、setup 簡單 | +| 批次操作（建 100 個 notebook、批次匯入文件） | `teng-lin/notebooklm-py` | Python API，可程式化跑 | +| 不想 Google 政策變動就壞 | （等 Google 出官方 API） | 兩個都是 unofficial、會有風險 | + +### 方案 A：PleasePrompto/notebooklm-skill + +#### Step 1：clone 到 skills 目錄 + +```bash +git clone https://github.com/PleasePrompto/notebooklm-skill ~/.claude/skills/notebooklm +``` + +#### Step 2：第一次跑會要 Google login（瀏覽器自動化） + +照 repo README 設定 OAuth / 登入 cookie。 + +#### Step 3：實用 prompt + +``` +查我 NotebookLM 內「LLM Agents 2024」這個 notebook， +找出所有提到 "tool use" 的段落，整理成一份比較表， +帶上每個來源文件名跟頁數。 +``` + +### 方案 B：teng-lin/notebooklm-py + +```bash +pip install notebooklm-py +``` + +範例： + +```python +from notebooklm import NotebookLM +nlm = NotebookLM() # OAuth 流程 + +# 建一個 notebook +nb = nlm.create_notebook("My Research") + +# 批次匯入 PDF +for pdf in glob.glob("papers/*.pdf"): + nb.add_source(pdf) + +# Q&A +answer = nb.query("What are the main contributions?") +print(answer.text) +print(answer.citations) +``` + +### 常見 pitfall + +| 症狀 | 原因 | 解法 | +|---|---|---| +| 突然不能用 | Google 改了內部 API | 檢查 issue tracker、等社群更新 | +| Q&A 答案模糊 | 上傳文件太多、retrieve 失準 | 拆成幾個 notebook（每個 < 50 source）| +| 中文支援不好 | 預設 UI 設成英文 | NotebookLM 設定改 zh-Hant | + +### 進一步 + +- catalog 1 [`mcp-skills-catalog.md` 1 筆記 / 知識庫](mcp-skills-catalog.md#1-筆記--知識庫) +- 完整 research workspace：用 [`WenyuChiou/research-hub`](https://github.com/WenyuChiou/research-hub) 整合 NotebookLM + Zotero + Obsidian + +--- + +## 5. Zotero Workflow + +> Zotero 管文獻，加上 [`WenyuChiou/zotero-skills`](https://github.com/WenyuChiou/zotero-skills) 後 Claude Code 能直接搜 / 加 / 分類 / 標 references。 + +### 為什麼 + +研究流程經典痛點： +- 「我那篇 paper 在哪？」——Zotero 有，但要切換視窗 +- 「給我所有講 transformer 的 paper 摘要」——要自己 select、export、丟給 LLM +- 「這篇 paper 該打什麼 tag？」——人工 + +zotero-skills 把這些變成 Claude Code 內一句 prompt 就跑。 + +### 跟 zotero-gpt 差別 + +| 工具 | 角色 | 適合 | +|---|---|---| +| [`MuiseDestiny/zotero-gpt`](https://github.com/MuiseDestiny/zotero-gpt) | Zotero plugin（在 Zotero **內部** chat） | 邊讀 paper 邊問 LLM、不切換視窗 | +| [`WenyuChiou/zotero-skills`](https://github.com/WenyuChiou/zotero-skills) | Claude Code skill（從 **外部** 操作 Zotero） | 寫 paper / 整理文獻時，Claude Code 為主 | + +互補不衝突，可以兩個都裝。 + +### 步驟 + +#### Step 1：開啟 Zotero local API + +Zotero 桌面版預設不開 API。打開： +- **Edit → Preferences → Advanced → Config Editor** +- 找 `extensions.zotero.httpServer.enabled`，設 `true` +- 找 `extensions.zotero.httpServer.port`，預設 `23119` + +#### Step 2：clone zotero-skills + +```bash +git clone https://github.com/WenyuChiou/zotero-skills ~/.claude/skills/zotero-skills +``` + +照 repo README 設定（包含 API key 給 Web API 寫操作用）。 + +#### Step 3：實用 prompt + +**搜文獻**： +``` +搜我 Zotero library 內所有 2023 年之後、跟 multi-agent 相關的 paper， +按 cited count 排序、輸出成 markdown table。 +``` + +**自動分類**： +``` +看我 collection "Inbox" 裡的 50 篇 paper，按主題自動建 sub-collection +（譬如 "RAG"、"Tool Use"、"Multi-Agent"），把 paper 移進去。 +``` + +**標 tag**： +``` +讀我 Zotero 內這篇 paper（attached PDF 看完）， +從 abstract 提取 5 個 keyword 當 tag 加上去。 +``` + +**寫 paper 引用整理**： +``` +我的 paper draft 在 ./paper/v3.tex， +找出所有 \cite{} 對應的 BibTeX entry，跟 Zotero library 對比， +把缺的 export 出 .bib 給我。 +``` + +### 常見 pitfall + +| 症狀 | 原因 | 解法 | +|---|---|---| +| skill 觸發但 query 失敗 | Zotero 沒在跑 / API 沒開 | 開 Zotero 桌面版 + 確認 port 23119 listening | +| 寫操作（add / move）失敗 | local API 是 read-only，要用 Web API | 設定 Web API key（[zotero.org/settings/keys](https://www.zotero.org/settings/keys)） | +| collection 結構亂 | 自動分類 prompt 沒給目錄結構 | prompt 給 Claude 看現有 collection tree、再決定怎麼分 | + +### 進一步 + +- 完整 research workspace：[`WenyuChiou/research-hub`](https://github.com/WenyuChiou/research-hub) 整合 Zotero + Obsidian + NotebookLM +- 學術論文寫作：[`WenyuChiou/academic-writing-skills`](https://github.com/WenyuChiou/academic-writing-skills) +- 14 個研究流程 skill 集：[`WenyuChiou/ai-research-skills`](https://github.com/WenyuChiou/ai-research-skills) + +--- + +## 6. 本機 LLM + CLI Agent 快速 walkthrough + +> 30 分鐘把 Stage 1 的本機模型接到 Stage 5 的 CLI agent：離線、隱私資料、不想用 API 額度時，可以先用這條路線做 end-to-end 驗證。 + +### 為什麼 + +Stage 1 教你用 Ollama / llama.cpp / vLLM 跑本機 LLM；Stage 5 教 Claude Code、MCP、Skills、Plugins 的 agent 生態。中間常見的誤會是：**Claude Code 不是本機 LLM runner**。Claude Code 需要 Anthropic OAuth / API key，不能直接把 model endpoint 改成 Ollama 或其他本機 endpoint。 + +如果目標是「本機 LLM + CLI agent」，選擇支援 BYO LLM 的 CLI 會更直接：**OpenCode / goose / Aider / Hermes Agent** 都能接 OpenAI-compatible endpoint 或 Ollama provider。這個 recipe 用一個短流程讓你先跑通 model、agent、任務三件事。 + +### 步驟 + +#### Step 1：Ollama + model（10 分鐘） + +```bash +# 安裝 Ollama：https://ollama.com +ollama pull qwen2.5:3b +# RAM 16GB+ 可以改試：ollama pull qwen2.5:7b +ollama serve +``` + +確認 OpenAI-compatible API 有回應： + +```bash +curl http://localhost:11434/v1/chat/completions \ + -H "Content-Type: application/json" \ + -d '{"model":"qwen2.5:3b","messages":[{"role":"user","content":"用 3 句話解釋 ReAct agent。"}]}' +``` + +#### Step 2：選一個 CLI agent 接 Ollama（10 分鐘） + +**OpenCode**：適合想切換多 provider、又要接本機模型的人。 + +```bash +npm install -g opencode-ai +opencode auth login # provider 選 Ollama，endpoint 設 http://localhost:11434/v1 +opencode +``` + +**goose**：內建 Ollama provider，適合先做本機 agent 試跑。 + +```bash +# 安裝方式看 https://block.github.io/goose +goose configure # provider 選 Ollama，model 設 qwen2.5:3b +goose session start +``` + +**Aider**：git-native，適合在 repo 內做小型程式碼修改。 + +```bash +pip install aider-chat +aider --model ollama/qwen2.5:3b --no-show-model-warnings +``` + +**Hermes Agent**：適合跑在 VPS，讓 Telegram / Slack / Discord 變成 agent 入口。 + +```bash +curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash +hermes model set ollama:qwen2.5:3b +hermes +``` + +#### Step 3：跑一個真實小任務（10 分鐘） + +不要只問「hello world」。挑一個會碰到檔案、摘要、表格或搜尋的小任務： + +- 從 `~/Downloads` 找 5 個 PDF，抽出每篇 paper 的 1 句 summary 與 method。 +- 讀 `data.csv` 前 3 欄，輸出 Markdown table 並指出欄位問題。 +- 搜 `~/notes/` 裡 7 天內提到 `agent safety` 的段落，整理成 checklist。 + +觀察三件事： + +- **Speed**：本機小模型常比 API 慢 2-5 倍。 +- **Quality**：3B / 7B 模型的 reasoning、長 context、複雜程式碼能力通常不如 Claude。 +- **Cost**：token 成本是 `$0`，但會吃本機 RAM / VRAM 與電力。 + +#### Step 4：跟 Claude Code 的差異（5 分鐘） + +| 面向 | Claude Code | OpenCode + Ollama | +|---|---|---| +| LLM | Anthropic hosted | 本機模型 | +| 成本 | 訂閱或 per-token | `$0` token cost | +| 速度 | 通常較穩 | 看硬體，常慢 2-5 倍 | +| 隱私 | 內容送 Anthropic | 內容留在本機 | +| Reasoning 上限 | Claude 4.8+ 較強 | 取決於本機模型 | +| 適合 use case | 複雜 codebase、長 context、可靠推理 | 隱私資料、離線 demo、低成本反覆試 | + +### 重要限制：Claude Code 不能直接用本機 LLM + +Claude Code 目前需要 Anthropic OAuth / API key，沒有官方設定可以把模型切成 Ollama 或本機 endpoint。網路上可能有 proxy 或 API shim 做實驗，但這不是官方支援路徑，穩定性與相容性要自己承擔。 + +要用本機 LLM，建議把「Claude Code」跟「支援 BYO LLM 的 CLI agent」分開看：Claude Code 用在需要 Claude 品質的工作；OpenCode / goose / Aider / Hermes 用在本機、離線、隱私或低成本實驗。 + +### 常見 pitfall + +| 問題 | 原因 | 解法 | +|---|---|---| +| `connection refused` | Ollama server 沒在背景跑 | 開另一個 terminal 跑 `ollama serve` | +| model output 斷句怪、邏輯弱 | 3B 模型能力有限 | 改用 `qwen2.5:7b` 或 `deepseek-r1:7b` | +| CLI agent 沒改到檔案 | 本機模型太弱，或 prompt 太模糊 | 縮小任務、指定檔案與成功條件 | +| memory / OOM | 模型吃掉 RAM / VRAM | 先用 `qwen2.5:3b`，再升到 7B；必要時開 swap | + +### 進一步 + +- Stage 1 [Local LLM 練習](../stages/01-llm-basics.md#練習-6local-llm)：Ollama / llama.cpp / vLLM 的差異 +- [`cli-agents-guide.md`](cli-agents-guide.md)：7 個 CLI agent 怎麼選 +- Hermes Agent README：多平台 gateway（Telegram / Discord / Slack）與 provider 設定 + +--- + +## 找不到你要的 recipe？ + +- 看 [Stage 5](../stages/05-claude-code-ecosystem.md) 完整概念 +- 看 [`mcp-skills-catalog.md`](mcp-skills-catalog.md) 完整工具清單 +- 看 [`schema-design-cheatsheet.md`](schema-design-cheatsheet.md) 寫 tool schema 的細節 +- 看 [`cli-agents-guide.md`](cli-agents-guide.md) 7 個主流 CLI agent 比較 + +要新 recipe → 開 issue 或直接 PR 一份。recipe 格式：**為什麼 + 步驟 + 範本 prompt + 常見 pitfall + 進一步**。 diff --git a/resources/cookbook.zh-Hans.md b/resources/cookbook.zh-Hans.md new file mode 100644 index 0000000..7a0986a --- /dev/null +++ b/resources/cookbook.zh-Hans.md @@ -0,0 +1,622 @@ +# Cookbook — 把概念变成可执行的 recipe + +> [繁體中文](./cookbook.md) | **简体中文** | [English](./cookbook.en.md) + +> Stage 5（Claude Code 生态）跟 [`mcp-skills-catalog.md`](mcp-skills-catalog.zh-Hans.md) 讲“概念”跟“有哪些工具”。这份 cookbook 补中间缺的：“**怎么动手做出来**”。每个 recipe 是一份 step-by-step + sample code + 常见 pitfall，~30-50 分钟做完一个。 +> +> 不是 reference 也不是 tutorial——是 recipe，挑你需要的那道煮就好。 + +--- + +## 📋 目录 + +1. [写你的第一个 Skill（SKILL.md anatomy）](#1-写你的第一个-skill) +2. [写你的第一个 MCP server（Python SDK）](#2-写你的第一个-mcp-server) +3. [Word / Excel / PowerPoint workflow](#3-office-docs-workflow) +4. [NotebookLM workflow](#4-notebooklm-workflow) +5. [Zotero workflow](#5-zotero-workflow) +6. [本地 LLM + CLI Agent 快速 walkthrough](#6-本地-llm--cli-agent-快速-walkthrough) + +--- + +## 1. 写你的第一个 Skill + +> Skill = 一个文件夹含 `SKILL.md`，Claude Code 启动时自动 discover、按情境自动加载。最小 viable 版本 50 行就能跑。 +> +> 📚 **这份是“30 分钟跑出第一个”实作版。想看“Skill 怎么写得好”深度讨论** → [Hello-Agents Extra08：如何写出好的 Skill](https://github.com/datawhalechina/hello-agents/blob/main/Extra-Chapter/Extra08-如何写出好的Skill.md)（中文最完整的 Skill 最佳实践，讨论 description 写法、references / scripts 设计等）。两份互补：先用本 recipe 跑出第一个，再读那份 polish 写法。 + +### 为什么 + +写 Skill 跟“在 prompt 里加几段 instruction”差别在于： +- Skill 是 **per-domain** 的，不会污染所有 conversation +- 可以打包跨 project / team 共用 +- Claude 自己决定何时加载（看 description match 不 match） + +### 步骤 + +#### Step 1：创建 skill 文件夹 + +两个位置可以放（看你要 user 级还是 project 级）： + +```bash +# user 级（所有 project 共用） +mkdir -p ~/.claude/skills/my-first-skill +cd ~/.claude/skills/my-first-skill + +# 或 project 级（只在这个 repo 触发） +mkdir -p .claude/skills/my-first-skill +cd .claude/skills/my-first-skill +``` + +#### Step 2：写 `SKILL.md` + +最小可 work 的模板： + +```markdown +--- +name: my-first-skill +description: When the user asks for [SPECIFIC SITUATION], use this skill to [WHAT IT DOES]. Examples include [2-3 trigger phrases]. Do NOT use for [WHAT IT'S NOT FOR]. +--- + +# My First Skill + +You are now in the [domain] context. + +## When the user asks X, do these steps: + +1. First, [action A] +2. Then, [action B] +3. Verify with [check] + +## Don't do: + +- [anti-pattern 1] +- [anti-pattern 2] + +## Reference + +- (optional) link to a doc / paper / API spec +``` + +具体例子：“整理 Python 代码的 import 顺序” + +```markdown +--- +name: python-import-organizer +description: When the user pastes Python code or asks to clean up imports / format code / sort imports, organize the imports following PEP 8 + isort order: stdlib first, then third-party, then local. Do NOT use for non-Python code. +--- + +# Python Import Organizer + +When the user wants Python imports cleaned up: + +1. Group imports into 3 sections: stdlib / third-party / local +2. Within each group, sort alphabetically +3. Add a blank line between groups +4. Remove unused imports (only if user explicitly asks; otherwise just sort) + +## Don't: +- Don't change function code, only the import block +- Don't auto-remove imports without asking +``` + +#### Step 3：测试 + +```bash +# 重启 Claude Code（让它重新 discover skills） +# 在 conversation 里丢一个触发句 +# e.g.「帮我整理一下这段 Python 的 imports」 +# 观察 Claude 有没有按照 SKILL.md 的步骤做 +``` + +#### Step 4（进阶）：加 evals + +在 skill folder 内加 `evals/evals.json`： + +```json +{ + "evals": [ + { + "input": "整理一下这段 Python 的 imports: import os +import requests +from mypackage import foo", + "expected_behavior": ["按 stdlib / third-party / local 分组", "alphabetical 排序"] + } + ] +} +``` + +之后可以用 promptfoo 之类工具 batch 跑。 + +### 常见 pitfall + +| 症状 | 原因 | 解法 | +|---|---|---| +| Claude 从不触发我的 skill | description 写得太笼统，匹配不到 user query | description 加 2-3 个具体 trigger phrase（"when the user asks X / Y / Z"） | +| 触发了但行为不对 | SKILL.md 步骤太抽象 | 改成 numbered list、每步明确动作 | +| 触发了不该触发 | description 太宽，匹配到不相关 query | 加 "Do NOT use for X" 收敛 | + +### 进一步 + +- 看 [Stage 5.3](../stages/05-claude-code-ecosystem.zh-Hans.md#53--skillsclaude-code-的行为层-claude-code-生态最关键的一层) 的 Skill anatomy 详解 +- 看 [`anthropics/skills`](https://github.com/anthropics/skills) 官方 skill 模板（docx / xlsx / pptx 等）的写法 +- 多个 skill 打包成 plugin → [Stage 5.4](../stages/05-claude-code-ecosystem.zh-Hans.md#54--plugins-与-marketplaces) + +--- + +## 2. 写你的第一个 MCP server + +> MCP server = 一个独立 process，跑起来提供 tool / resource / prompt 给 LLM host（Claude Desktop / Claude Code）。最小可 run 版 < 50 行 Python。 + +### 为什么 + +- Skill 是给 Claude 的“角色 + 规则”；MCP 是给 Claude 的“**外部 function**” +- Skill 不能读档、不能调用 API；MCP 可以（任何 tool 你写得出来） +- Skill 只在 Claude Code 跑；MCP 任何 LLM host（包括 Cursor、自写 agent）都能接 + +### 步骤 + +#### Step 1：安装官方 SDK + +```bash +pip install mcp +``` + +#### Step 2：写 `server.py` + +最小模板——一个会回 echo 的 tool： + +```python +# server.py +from mcp.server import Server +from mcp.server.stdio import stdio_server +from mcp.types import Tool, TextContent + +app = Server("hello-mcp") + +@app.list_tools() +async def list_tools() -> list[Tool]: + return [ + Tool( + name="echo", + description="Echo the input text back to the user.", + inputSchema={ + "type": "object", + "properties": { + "text": { + "type": "string", + "description": "Text to echo back", + } + }, + "required": ["text"], + }, + ) + ] + +@app.call_tool() +async def call_tool(name: str, arguments: dict) -> list[TextContent]: + if name == "echo": + return [TextContent(type="text", text=f"Echo: {arguments['text']}")] + raise ValueError(f"Unknown tool: {name}") + +async def main(): + async with stdio_server() as (read, write): + await app.run(read, write, app.create_initialization_options()) + +if __name__ == "__main__": + import asyncio + asyncio.run(main()) +``` + +#### Step 3：在 Claude Desktop / Code 设置 + +**Claude Desktop**：编辑 `~/Library/Application Support/Claude/claude_desktop_config.json`（macOS）或 `%APPDATA%\Claude\claude_desktop_config.json`（Windows）： + +```json +{ + "mcpServers": { + "hello-mcp": { + "command": "python", + "args": ["/绝对路径/到/server.py"] + } + } +} +``` + +**Claude Code**：用 `claude mcp add` 指令： + +```bash +claude mcp add hello-mcp python /绝对路径/到/server.py +``` + +#### Step 4：重启 Claude Desktop / Code、测试 + +``` +你问：echo "hello world" 给我 +Claude 回（会显示 tool call icon）：Echo: hello world +``` + +### 常见 pitfall + +| 症状 | 原因 | 解法 | +|---|---|---| +| Claude Desktop 没看到 tool | server.py 启动失败 | 终端直接 `python server.py` 跑、看 stderr 哪里爆 | +| tool 列出但 call 失败 | inputSchema 格式错（required 漏写、type 写错） | 看 [`schema-design-cheatsheet.md`](schema-design-cheatsheet.zh-Hans.md) | +| Claude 不主动叫 tool | description 太笼统 | description 改成“When the user asks X, use this tool”式的具体 trigger | +| stdio 跟 SSE 哪个用？ | local desktop integration 用 stdio；remote / web 用 SSE | 第一个 server 一律用 stdio | + +### 进一步 + +- 看 [Stage 5.2](../stages/05-claude-code-ecosystem.zh-Hans.md#52--mcpmodel-context-protocol-基础) 的 MCP 完整介绍 +- 看 [`modelcontextprotocol/servers`](https://github.com/modelcontextprotocol/servers) 官方示例（filesystem、github、sqlite、time 等） +- 写 production server 看 [Stage 5.2“练习：MCP in production”](../stages/05-claude-code-ecosystem.zh-Hans.md#52--mcpmodel-context-protocol-基础) 跟 [`anthropics/claude-code`](https://github.com/anthropics/claude-code) 的 `~/.claude/skills/` + +--- + +## 3. Office Docs Workflow + +> 用 Claude 读写 Word / Excel / PowerPoint / PDF 不用装额外 tool——[`anthropics/skills`](https://github.com/anthropics/skills) 官方 repo 已经内建好。 + +### 为什么 + +最常见场景： +- 把 Markdown / 大纲 → 自动生成 Word / PPT +- 读一堆 PDF / Excel → 整理摘要 / 提取数字 +- 改别人传来的 docx → 加 track changes、或重排格式 +- 把表格 cross-reference 写成报告 + +不需要自己 parse XML、不需要找 python-docx / openpyxl 教学——anthropics/skills 已经包好。 + +### 步骤 + +#### Step 1：安装 skills + +最简单：clone Anthropic 官方 skills repo 到 user-level skill 目录： + +```bash +# user 级（所有 project 用） +git clone https://github.com/anthropics/skills.git ~/.claude/skills/anthropic-skills +``` + +或者用 `claude plugin install`（如果有打包成 plugin）。 + +#### Step 2：重启 Claude Code + +- skills/docx/ → docx 读写 +- skills/xlsx/ → Excel 读写 +- skills/pptx/ → PowerPoint 读写 +- skills/pdf/ → PDF 读 + +Claude 会根据 user query 自动加载合适的 skill。 + +#### Step 3：实用 prompt 模板 + +**从大纲生 PPT**： +``` +读我写的 outline.md，照这个结构生一份 PPT： +- 封面 1 页 +- 每个 H2 一页，bullet points 从 H3 内容浓縮 +- 结语 1 页 + +存成 ./output/presentation.pptx +``` + +**读 Excel 整理数字**： +``` +读 ./data/sales-2023.xlsx 第一张 sheet，把每个 region 的 Q4 总额算出来， +写进 ./output/q4-summary.md（用 markdown table 格式）。 +``` + +**改 docx**： +``` +读 ./doc/draft.docx，把繁中词汇转成简中（譬如“软体”→“软件”）， +存成 ./doc/draft.zh-Hans.docx，保留原本的 track changes。 +``` + +**读 PDF 提取信息**： +``` +读 ./papers/research.pdf，把 abstract、main contributions、limitations +分别写进三个 markdown section，存到 ./notes/research-summary.md。 +``` + +### 常见 pitfall + +| 症状 | 原因 | 解法 | +|---|---|---| +| skill 没被触发 | repo 路径放错 | 确认 SKILL.md 在 `~/.claude/skills/anthropic-skills/skills/docx/SKILL.md` 这种层级 | +| pptx 生出来样式丑 | 没给设计参考 | prompt 加“参考 ./template.pptx 的样式” | +| 大 PDF 读不完 | context 爆 | 改用 [`SylphxAI/pdf-reader-mcp`](https://github.com/SylphxAI/pdf-reader-mcp)（5-10× 快） | +| Excel 公式被吃掉 | docx skill 不处理 formulas | 开档前 prompt 明说“保留 formula 不要 hard-code” | + +### 进一步 + +- catalog 2 [`mcp-skills-catalog.md` 2 办公文件](mcp-skills-catalog.zh-Hans.md#2-办公文件word--excel--powerpoint--pdf)：补强版 office skill / Excel / PPT 专用 MCP +- 中文圈 office workflow：[`leemysw/feishu-docx`](https://github.com/leemysw/feishu-docx) 飞书 / Lark docs ↔ Markdown + +--- + +## 4. NotebookLM Workflow + +> NotebookLM 是 Google 的 RAG-on-your-docs 工具。**Claude Code 没有官方 NotebookLM 集成**，但社群有 2 个成熟方案。 + +### 为什么 + +NotebookLM 强的地方： +- 上传 50 份 PDF 自动建索引 +- Q&A 带 citation（每个答案都标出来自哪份文件第几页） +- 生成 summary / mind map / podcast-style audio overview + +弱点：要在 NotebookLM 网页里用，跟你的其他 workflow（Claude Code、Obsidian、Zotero）断开。 + +两个方案桥接： +1. **PleasePrompto/notebooklm-skill**（Skill，browser automation） +2. **teng-lin/notebooklm-py**（Python API + CLI） + +### 两个方案怎么选 + +| 场景 | 选哪个 | 为什么 | +|---|---|---| +| 偶尔从 Claude Code 查一下 NotebookLM | `PleasePrompto/notebooklm-skill` | Claude Code 内 prompt 一句话就跑、setup 简单 | +| 批次操作（建 100 个 notebook、批次导入文件） | `teng-lin/notebooklm-py` | Python API，可程序化跑 | +| 不想 Google 政策变动就坏 | （等 Google 出官方 API） | 两个都是 unofficial、会有风险 | + +### 方案 A：PleasePrompto/notebooklm-skill + +#### Step 1：clone 到 skills 目录 + +```bash +git clone https://github.com/PleasePrompto/notebooklm-skill ~/.claude/skills/notebooklm +``` + +#### Step 2：第一次跑会要 Google login（浏览器自动化） + +照 repo README 设置 OAuth / 登录 cookie。 + +#### Step 3：实用 prompt + +``` +查我 NotebookLM 内“LLM Agents 2024”这个 notebook， +找出所有提到 "tool use" 的段落，整理成一份比较表， +带上每个来源文件名跟页数。 +``` + +### 方案 B：teng-lin/notebooklm-py + +```bash +pip install notebooklm-py +``` + +示例： + +```python +from notebooklm import NotebookLM +nlm = NotebookLM() # OAuth 流程 + +# 建一个 notebook +nb = nlm.create_notebook("My Research") + +# 批次导入 PDF +for pdf in glob.glob("papers/*.pdf"): + nb.add_source(pdf) + +# Q&A +answer = nb.query("What are the main contributions?") +print(answer.text) +print(answer.citations) +``` + +### 常见 pitfall + +| 症状 | 原因 | 解法 | +|---|---|---| +| 突然不能用 | Google 改了内部 API | 检查 issue tracker、等社群更新 | +| Q&A 答案模糊 | 上传文件太多、retrieve 失准 | 拆成几个 notebook（每个 < 50 source）| +| 中文支持不好 | 预设 UI 设成英文 | NotebookLM 设置改 zh-Hant | + +### 进一步 + +- catalog 1 [`mcp-skills-catalog.md` 1 笔记 / 知识库](mcp-skills-catalog.zh-Hans.md#1-笔记--知识库) +- 完整 research workspace：用 [`WenyuChiou/research-hub`](https://github.com/WenyuChiou/research-hub) 集成 NotebookLM + Zotero + Obsidian + +--- + +## 5. Zotero Workflow + +> Zotero 管文献，加上 [`WenyuChiou/zotero-skills`](https://github.com/WenyuChiou/zotero-skills) 后 Claude Code 能直接搜 / 加 / 分类 / 标 references。 + +### 为什么 + +研究流程经典痛点： +- “我那篇 paper 在哪？”——Zotero 有，但要切换窗口 +- “给我所有讲 transformer 的 paper 摘要”——要自己 select、export、丢给 LLM +- “这篇 paper 该打什么 tag？”——人工 + +zotero-skills 把这些变成 Claude Code 内一句 prompt 就跑。 + +### 跟 zotero-gpt 差别 + +| 工具 | 角色 | 适合 | +|---|---|---| +| [`MuiseDestiny/zotero-gpt`](https://github.com/MuiseDestiny/zotero-gpt) | Zotero plugin（在 Zotero **内部** chat） | 边读 paper 边问 LLM、不切换窗口 | +| [`WenyuChiou/zotero-skills`](https://github.com/WenyuChiou/zotero-skills) | Claude Code skill（从 **外部** 操作 Zotero） | 写 paper / 整理文献时，Claude Code 为主 | + +互补不冲突，可以两个都装。 + +### 步骤 + +#### Step 1：开启 Zotero local API + +Zotero 桌面版默认不开 API。打开： +- **Edit → Preferences → Advanced → Config Editor** +- 找 `extensions.zotero.httpServer.enabled`，设 `true` +- 找 `extensions.zotero.httpServer.port`，默认 `23119` + +#### Step 2：clone zotero-skills + +```bash +git clone https://github.com/WenyuChiou/zotero-skills ~/.claude/skills/zotero-skills +``` + +照 repo README 设置（包含 API key 给 Web API 写操作用）。 + +#### Step 3：实用 prompt + +**搜文献**： +``` +搜我 Zotero library 内所有 2023 年之后、跟 multi-agent 相关的 paper， +按 cited count 排序、输出成 markdown table。 +``` + +**自动分类**： +``` +看我 collection "Inbox" 里的 50 篇 paper，按主题自动建 sub-collection +（譬如 "RAG"、"Tool Use"、"Multi-Agent"），把 paper 移进去。 +``` + +**标 tag**： +``` +读我 Zotero 内这篇 paper（attached PDF 看完）， +从 abstract 提取 5 个 keyword 当 tag 加上去。 +``` + +**写 paper 引用整理**： +``` +我的 paper draft 在 ./paper/v3.tex， +找出所有 \cite{} 对应的 BibTeX entry，跟 Zotero library 对比， +把缺的 export 出 .bib 给我。 +``` + +### 常见 pitfall + +| 症状 | 原因 | 解法 | +|---|---|---| +| skill 触发但 query 失败 | Zotero 没在跑 / API 没开 | 开 Zotero 桌面版 + 确认 port 23119 listening | +| 写操作（add / move）失败 | local API 是 read-only，要用 Web API | 设置 Web API key（[zotero.org/settings/keys](https://www.zotero.org/settings/keys)） | +| collection 结构乱 | 自动分类 prompt 没给目录结构 | prompt 给 Claude 看现有 collection tree、再决定怎么分 | + +### 进一步 + +- 完整 research workspace：[`WenyuChiou/research-hub`](https://github.com/WenyuChiou/research-hub) 集成 Zotero + Obsidian + NotebookLM +- 学术论文写作：[`WenyuChiou/academic-writing-skills`](https://github.com/WenyuChiou/academic-writing-skills) +- 14 个研究流程 skill 集：[`WenyuChiou/ai-research-skills`](https://github.com/WenyuChiou/ai-research-skills) + +--- + +## 6. 本地 LLM + CLI Agent 快速 walkthrough + +> 30 分钟把 Stage 1 的本地模型接到 Stage 5 的 CLI agent：离线、隐私资料、不想用 API 额度时，可以先用这条路线做 end-to-end 验证。 + +### 为什么 + +Stage 1 教你用 Ollama / llama.cpp / vLLM 跑本地 LLM；Stage 5 教 Claude Code、MCP、Skills、Plugins 的 agent 生态。中间常见的误会是：**Claude Code 不是本地 LLM runner**。Claude Code 需要 Anthropic OAuth / API key，不能直接把 model endpoint 改成 Ollama 或其他本地 endpoint。 + +如果目标是“本地 LLM + CLI agent”，选择支持 BYO LLM 的 CLI 会更直接：**OpenCode / goose / Aider / Hermes Agent** 都能接 OpenAI-compatible endpoint 或 Ollama provider。这个 recipe 用一个短流程让你先跑通 model、agent、任务三件事。 + +### 步骤 + +#### Step 1：Ollama + model（10 分钟） + +```bash +# 安装 Ollama：https://ollama.com +ollama pull qwen2.5:3b +# RAM 16GB+ 可以改试：ollama pull qwen2.5:7b +ollama serve +``` + +确认 OpenAI-compatible API 有响应： + +```bash +curl http://localhost:11434/v1/chat/completions \ + -H "Content-Type: application/json" \ + -d '{"model":"qwen2.5:3b","messages":[{"role":"user","content":"用 3 句话解释 ReAct agent。"}]}' +``` + +#### Step 2：选一个 CLI agent 接 Ollama（10 分钟） + +**OpenCode**：适合想切换多 provider、又要接本地模型的人。 + +```bash +npm install -g opencode-ai +opencode auth login # provider 选 Ollama，endpoint 设 http://localhost:11434/v1 +opencode +``` + +**goose**：内置 Ollama provider，适合先做本地 agent 试跑。 + +```bash +# 安装方式看 https://block.github.io/goose +goose configure # provider 选 Ollama，model 设 qwen2.5:3b +goose session start +``` + +**Aider**：git-native，适合在 repo 内做小型代码修改。 + +```bash +pip install aider-chat +aider --model ollama/qwen2.5:3b --no-show-model-warnings +``` + +**Hermes Agent**：适合跑在 VPS，让 Telegram / Slack / Discord 变成 agent 入口。 + +```bash +curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash +hermes model set ollama:qwen2.5:3b +hermes +``` + +#### Step 3：跑一个真实小任务（10 分钟） + +不要只问“hello world”。挑一个会碰到文件、摘要、表格或搜索的小任务： + +- 从 `~/Downloads` 找 5 个 PDF，抽出每篇 paper 的 1 句 summary 与 method。 +- 读 `data.csv` 前 3 列，输出 Markdown table 并指出列问题。 +- 搜 `~/notes/` 里 7 天内提到 `agent safety` 的段落，整理成 checklist。 + +观察三件事： + +- **Speed**：本地小模型常比 API 慢 2-5 倍。 +- **Quality**：3B / 7B 模型的 reasoning、长 context、复杂代码能力通常不如 Claude。 +- **Cost**：token 成本是 `$0`，但会吃本地 RAM / VRAM 与电力。 + +#### Step 4：跟 Claude Code 的差异（5 分钟） + +| 面向 | Claude Code | OpenCode + Ollama | +|---|---|---| +| LLM | Anthropic hosted | 本地模型 | +| 成本 | 订阅或 per-token | `$0` token cost | +| 速度 | 通常较稳 | 看硬件，常慢 2-5 倍 | +| 隐私 | 内容送 Anthropic | 内容留在本地 | +| Reasoning 上限 | Claude 4.8+ 较强 | 取决于本地模型 | +| 适合 use case | 复杂 codebase、长 context、可靠推理 | 隐私资料、离线 demo、低成本反复试 | + +### 重要限制：Claude Code 不能直接用本地 LLM + +Claude Code 目前需要 Anthropic OAuth / API key，没有官方设置可以把模型切成 Ollama 或本地 endpoint。网上可能有 proxy 或 API shim 做实验，但这不是官方支持路径，稳定性与兼容性要自己承担。 + +要用本地 LLM，建议把“Claude Code”和“支持 BYO LLM 的 CLI agent”分开看：Claude Code 用在需要 Claude 质量的工作；OpenCode / goose / Aider / Hermes 用在本地、离线、隐私或低成本实验。 + +### 常见 pitfall + +| 问题 | 原因 | 解法 | +|---|---|---| +| `connection refused` | Ollama server 没在后台跑 | 开另一个 terminal 跑 `ollama serve` | +| model output 断句怪、逻辑弱 | 3B 模型能力有限 | 改用 `qwen2.5:7b` 或 `deepseek-r1:7b` | +| CLI agent 没改到文件 | 本地模型太弱，或 prompt 太模糊 | 缩小任务、指定文件与成功条件 | +| memory / OOM | 模型吃掉 RAM / VRAM | 先用 `qwen2.5:3b`，再升到 7B；必要时开 swap | + +### 进一步 + +- Stage 1 [Local LLM 练习](../stages/01-llm-basics.zh-Hans.md#练习-6local-llm)：Ollama / llama.cpp / vLLM 的差异 +- [`cli-agents-guide.md`](cli-agents-guide.zh-Hans.md)：7 个 CLI agent 怎么选 +- Hermes Agent README：多平台 gateway（Telegram / Discord / Slack）与 provider 设置 + +--- + +## 找不到你要的 recipe？ + +- 看 [Stage 5](../stages/05-claude-code-ecosystem.zh-Hans.md) 完整概念 +- 看 [`mcp-skills-catalog.md`](mcp-skills-catalog.zh-Hans.md) 完整工具清单 +- 看 [`schema-design-cheatsheet.md`](schema-design-cheatsheet.zh-Hans.md) 写 tool schema 的细节 +- 看 [`cli-agents-guide.md`](cli-agents-guide.zh-Hans.md) 7 个主流 CLI agent 比较 + +要新 recipe → 开 issue 或直接 PR 一份。recipe 格式：**为什么 + 步骤 + 范本 prompt + 常见 pitfall + 进一步**。 diff --git a/resources/courses.en.md b/resources/courses.en.md new file mode 100644 index 0000000..86c01b5 --- /dev/null +++ b/resources/courses.en.md @@ -0,0 +1,95 @@ +> [繁體中文](./courses.md) | [简体中文](./courses.zh-Hans.md) | **English** + +# Online AI Agent Courses (Certificate Comparison) + +> [← Back to main path README](../README.en.md) + +> 📌 **This is a reference catalog, not a ranking.** This repo is a hands-on learning roadmap and **does not replace** structured online courses — the courses below are parallel video / interactive on-ramps. Want to build → back to [stages](../README.en.md); want to look up a term → [`resources/glossary.en.md`](glossary.en.md). This page lists **only courses that grant a certificate**. + +> ⚠️ **Read this first.** A Certificate of Completion proves **you participated in and finished the course** — not mastery, and **not** an accredited credential or academic credit. Employers usually weigh things in this order: "is the content relevant to the role > how rigorous is the assessment > what did you actually build > who issued it" — the certificate itself ranks near the bottom. Treat it as **evidence of self-directed learning + a structure to stay motivated**, not as "a certificate gets you hired." The "Certificate" column below states facts only, no value judgment. + +--- + +## ⚡ Quick pick (choose one in 30 seconds) + +- **No idea where to start** → do [Hugging Face — AI Agents Course](https://huggingface.co/learn/agents-course) first: 🆓 free, framework-neutral, and the certificate is gated behind a quiz + assignments. The recommended starting point on this page. +- **Want a certificate for free** → Hugging Face, [Weights & Biases — AI Engineering: Agents](https://wandb.ai/site/courses/agents/), [Anthropic Academy](https://anthropic.skilljar.com/) (Claude / MCP focus). +- **Want university / big-vendor backing for your resume (paid)** → [IBM — RAG and Agentic AI (Professional Certificate)](https://www.coursera.org/professional-certificates/ibm-rag-and-agentic-ai), [Vanderbilt — AI Agent Developer](https://www.coursera.org/specializations/ai-agents). +- **Want to go end-to-end on one framework (LangChain / LangGraph)** → [Coursera — Agentic AI Engineering (Edureka)](https://www.coursera.org/specializations/agentic-ai-engineering). +- **Want a Chinese-language course** → [NVIDIA DLI (Chinese)](https://www.nvidia.cn/training/instructor-led-workshops/building-agentic-ai-applications-with-llms/) (best fit), Alibaba Cloud ACA, Huawei HCIA-AI. + > ⚠️ **Chinese-language, certificate-granting, agent-specific courses are currently all paid** (NVIDIA DLI / Alibaba / Huawei). To get a certificate **for free**, you currently take the English courses above (some have community subtitles). + +**Legend**: 🆓 free (certificate included) · 💰 paid · 🆓→💰 free to audit, paid certificate. + +--- + +## How to read this list + +- **Learning value** (credible instructor + hands-on + recent content) and **certificate value** are two different axes. This list is ordered mainly by learning value; the certificate is a heavily-annotated secondary signal. +- ⭐ ratings follow [style-guide §2](style-guide.en.md#2-recommendation-star-definitions): ⭐⭐⭐⭐⭐ must-do … ⭐ niche. +- **A paid Professional Certificate / Specialization (IBM, universities) usually carries more recognition than a free completion certificate** — but neither is a degree, and the "X% career outcomes" figure on Coursera is marketing. +- **The agent field moves fast**: discount the recency of any course older than 12–18 months against current framework versions (smolagents / LangGraph / MCP). + +--- + +## 🌍 English courses + +### tier-1 (highly credible + hands-on + recent) + +| Course (link) | Cost | Who it's for | What it teaches | Certificate | +|---|---|---|---|---| +| [Hugging Face — AI Agents Course](https://huggingface.co/learn/agents-course) ⭐⭐⭐⭐⭐ | 🆓 | Anyone wanting free, framework-neutral hands-on material | Hands-on with smolagents / LangGraph / LlamaIndex + observability / eval; build and benchmark an agent | Free, two levels: Fundamentals (Unit 1 + quiz ≥80%); Certificate of Completion (plus assignments + a final challenge). Issued directly by Hugging Face | +| [DeepLearning.AI — Agentic AI](https://www.deeplearning.ai/courses/agentic-ai/) ⭐⭐⭐⭐⭐ | 🆓→💰 | Developers with intermediate Python + basic LLM/API concepts | Four core design patterns: reflection, tool use, planning, multi-agent (31 videos + 8 graded assignments) | Free to audit (no certificate); certificate requires paid Pro (~$25–30/mo) plus passing assessments. Instructor: Andrew Ng. Chinese companion: [`datawhalechina/agentic-ai`](https://github.com/datawhalechina/agentic-ai) | +| [Weights & Biases — AI Engineering: Agents](https://wandb.ai/site/courses/agents/) ⭐⭐⭐⭐ | 🆓 | Developers who want agents that are "evaluated and shippable" | Built with the OpenAI team; reasoning models → agents, tool/memory/planning architecture, orchestrator-worker multi-agent, reproducible eval on accuracy/latency/cost (~2 hours) | Free Certificate of Completion (issued by W&B AI Academy) | +| [IBM — RAG and Agentic AI (Professional Certificate)](https://www.coursera.org/professional-certificates/ibm-rag-and-agentic-ai) ⭐⭐⭐⭐ | 💰 | People who want a big-vendor Professional Certificate | RAG + agentic AI, hands-on, across a multi-course Professional Certificate | Paid (Coursera Plus; financial aid available). Issued by IBM — more recognized than a generic completion certificate | +| [Vanderbilt University — AI Agent Developer](https://www.coursera.org/specializations/ai-agents) ⭐⭐⭐⭐ | 💰 | People who want university backing and a systematic path | Designing, building and refining agent software; Python agentic apps | Paid (Coursera Plus; financial aid available). Vanderbilt University Specialization Certificate | + +### tier-2 (solid, with a specific caveat) + +| Course (link) | Cost | Who it's for | What it teaches | Certificate | +|---|---|---|---|---| +| [Coursera — Agentic AI Engineering (Edureka)](https://www.coursera.org/specializations/agentic-ai-engineering) ⭐⭐⭐ | 💰 | People who want to go end-to-end on LangChain / LangGraph / MCP | 4 courses: LangChain ecosystem, LCEL, ReAct/memory, LangGraph multi-agent, MCP deployment, eval | Paid Specialization Certificate. Caveat: Edureka is a commercial training provider (not a university/lab), so credibility is moderate; for stronger backing in the same space pick IBM / Vanderbilt above | +| [Anthropic Academy](https://anthropic.skilljar.com/) ⭐⭐⭐⭐ | 🆓 | People building agents on the Claude / MCP stack | Claude Code, Claude API, MCP, Agent Skills; 17 self-paced courses across 5 tracks | Free official certificate (issued via Skilljar, includes a quiz, email sign-up, LinkedIn-shareable). Caveat: vendor-specific (Claude/MCP) — a supplement to, not a replacement for, a framework-neutral foundation course | + +> Worth knowing but not listed as a main entry: [LangChain Academy — Intro to LangGraph](https://academy.langchain.com/courses/intro-to-langgraph) (🆓 free LangGraph completion certificate, single-vendor). Cloud vendors (Google Cloud / AWS) mostly issue skill badges for their agentic paths — these are not the same thing as their paid **professional certification exams**; don't conflate the two. + +--- + +## 🀄 Chinese-language courses + +> **Gap-first fact**: certificate-granting, agent-specific courses **in Chinese are currently all paid**, and most are **big-vendor certifications** (tied to their own stack); a **zh-TW-native + certificate + agent-specific** course barely exists. To get a certificate for free, Chinese-speaking learners currently still take the English courses above (Hugging Face / W&B / Anthropic are all free). Of the three below, the NVIDIA DLI Chinese course is the best fit. + +### tier-1 (highly credible) + +| Course (link) | Cost | Who it's for | What it teaches | Certificate | +|---|---|---|---|---| +| [NVIDIA DLI — Building Agentic AI with LLMs (Chinese)](https://www.nvidia.cn/training/instructor-led-workshops/building-agentic-ai-applications-with-llms/) ⭐⭐⭐⭐ | 💰 | People who want to build agent systems hands-on, in Chinese | Building agentic systems with LLMs: deep reasoning, retrieval, tool calling, multi-agent, LangGraph, deployment considerations (8 hours) | Paid (~¥3,500/person, instructor-led, scheduled), includes 6 months of cloud labs. NVIDIA DLI certificate after passing the test. Caveat: relatively expensive and requires scheduling | + +### tier-2 (solid, with a specific caveat) + +| Course (link) | Cost | Who it's for | What it teaches | Certificate | +|---|---|---|---|---| +| [Alibaba Cloud — LLM ACA Certification + Bailian Agent Clouder](https://edu.aliyun.com/certification) ⭐⭐⭐ | 💰 | People building agents on the Alibaba Cloud (Bailian/Tongyi) stack who want in-ecosystem recognition | LLM engineering; the Clouder series includes a "building agent apps on the Bailian platform" module | Paid official certification (real-name required). Caveat: recognition is vendor-scoped (tied to Bailian/Tongyi), not academically transferable; zh-Hans only | +| [Huawei — HCIA-AI (LLM applications track)](https://e.huawei.com/cn/talent/cert/) ⭐⭐⭐ | 💰 | People who want Huawei-ecosystem recognition and a systematic intro | The 2026 V1.0 syllabus covers AI fundamentals, LLM knowledge, LLM applications, and compute-center solutions | Paid official certification (exam-based). Caveat: recognition skews toward the Huawei ecosystem and the mainland China job market; zh-Hans. HCIP/HCIE are deeper extensions | + +> Other Chinese options kept as notes, not main entries: imooc.com "AI Agent Full-Stack Engineer" (a commercial bootcamp — judge by syllabus; limited external recognition); the Ministry of Education "AI course completion certificate" (an official e-certificate, but broad AI literacy rather than agent engineering). **Excluded**: cert-mills (generic AIGC certs with no assessment and no credible issuer). + +--- + +## ⚠️ About "getting a certificate" — the full caveat (please relay this faithfully to readers) + +1. **A completion certificate is not an accredited credential.** It reflects participation and effort, not mastery; it is not the same as credit-bearing coursework or a degree. This list **never** presents these certificates as qualifications. +2. **Paid ≠ automatically recognized, free ≠ automatically lightweight.** Big-vendor / university paid Professional Certificates (IBM, Vanderbilt) usually carry more recognition; but free courses can be rigorously assessed too (Hugging Face's certificate is gated behind a quiz + assignments + a challenge). What matters is the issuer and the assessment, not the price. +3. **These certificates demonstrate exposure and effort, not competence.** The honest framing is "evidence of self-directed learning," not "proof you can build a shippable agent." +4. **A certificate is most useful for screening, and rarely works alone.** One from a credible source can help you pass an initial filter, but it almost never guarantees an offer — it only works alongside a portfolio. +5. **What you build matters more than what you collect.** Employers want evidence you can do the work (a GitHub repo, a deployed agent, open-source contributions). A course's real payoff is the artifact it forces you to build, not the PDF. +6. **Don't collect scattered badges — move along one coherent skill set.** Five unrelated intro certificates are far less convincing than one coherent path that demonstrates a single skill set — which is exactly why you should treat courses as "steps in a roadmap," not a trophy wall. +7. **Recency caveat.** Agent frameworks and best practices turn over fast, so an older cohort's certificate may represent stale knowledge. Check the course's vintage. + +--- + +## Maintenance notes + +- **Last verified: 2026-05.** Course details (especially certificate terms and free/paid status) drift quickly — treat each course's official page as the source of truth, and flag stale entries visibly rather than letting them silently go wrong. +- Bar for adding a new course: identifiable instructor/institution + a real assessment or a first-party-issued certificate + recent content. **No** cert-mills (marketplace-style completion certificates carry near-zero external recognition). +- Trilingual sync: every add/remove, star change, or certificate-terms change must be applied to `courses.md` + `courses.en.md` + `courses.zh-Hans.md`. diff --git a/resources/courses.md b/resources/courses.md new file mode 100644 index 0000000..1b0ad21 --- /dev/null +++ b/resources/courses.md @@ -0,0 +1,95 @@ +> **繁體中文** | [简体中文](./courses.zh-Hans.md) | [English](./courses.en.md) + +# 線上 AI Agent 課程（帶證書對照） + +> [← 回主路線 README](../README.md) + +> 📌 **這份是 reference catalog，不是排名。** 本 repo 是動手做的學習路線，**不取代**結構化線上課程——下面這些課程可當作平行的影片／互動式入門對照。想動手做 → 回 [stages](../README.md)；想先查用語 → [`resources/glossary.md`](glossary.md)。本頁只收**會發證書**的課程。 + +> ⚠️ **先讀這段，再往下看。** 完成證書（Certificate of Completion）證明的是**你參與並完成了課程**，不是 mastery，也**不等於** accredited 學歷或學分。雇主看的順序通常是「內容跟職務相不相關 > 評量嚴不嚴 > 你做出什麼作品 > 機構背書」，證書本身排在後面。把它當成**自主學習的證據 + 維持動機的結構**，不是「拿了證書就能找到工作」。下面「證書」欄只陳述事實，不做價值判斷。 + +--- + +## ⚡ 快速選（30 秒挑一門） + +- **完全沒方向** → 先做 [Hugging Face — AI Agents Course](https://huggingface.co/learn/agents-course)：🆓 免費、框架中立、證書要過 quiz + 作業才拿得到。本頁最推薦的起點。 +- **想免費就拿證書** → Hugging Face、[Weights & Biases — AI Engineering: Agents](https://wandb.ai/site/courses/agents/)、[Anthropic Academy](https://anthropic.skilljar.com/)（Claude / MCP 方向）。 +- **想要履歷用的大學／大廠背書（付費）** → [IBM — RAG and Agentic AI（Professional Certificate）](https://www.coursera.org/professional-certificates/ibm-rag-and-agentic-ai)、[Vanderbilt — AI Agent Developer](https://www.coursera.org/specializations/ai-agents)。 +- **想沿一個框架（LangChain / LangGraph）做到底** → [Coursera — Agentic AI Engineering（Edureka）](https://www.coursera.org/specializations/agentic-ai-engineering)。 +- **想用中文** → [NVIDIA DLI 中文](https://www.nvidia.cn/training/instructor-led-workshops/building-agentic-ai-applications-with-llms/)（最貼題）、阿里雲 ACA、華為 HCIA-AI。 + > ⚠️ **中文、帶證書、又 agent 專門的課，目前基本都要付費**（NVIDIA DLI / 阿里 / 華為）。想**免費**拿證書，現階段得走上面的英文課（部分有社群字幕）。 + +**圖例**：🆓 免費（含證書）· 💰 付費 · 🆓→💰 旁聽免費、證書付費。 + +--- + +## 怎麼讀這份清單 + +- **學習價值**（講師可信 + 動手做 + 內容夠新）和**證書價值**是兩條不同的軸。本清單主要照學習價值排，證書當成大量加註的次要信號。 +- ⭐ 星等照 [style-guide §2](style-guide.md#2-推薦星等定義)：⭐⭐⭐⭐⭐ 必做 … ⭐ 利基。 +- **付費 Professional Certificate / Specialization（IBM、大學）認可度，通常高於免費完成證書**；但兩者都不是學位，Coursera 那種「X% career outcomes」是行銷數字。 +- **Agent 領域變化快**：超過 12–18 個月的課，對框架版本（smolagents / LangGraph / MCP）要打 recency 折扣。 + +--- + +## 🌍 英文課程 + +### tier-1（高度可信 + 動手做 + 夠新） + +| 課程（連結） | 成本 | 適合誰 | 教什麼 | 證書 | +|---|---|---|---|---| +| [Hugging Face — AI Agents Course](https://huggingface.co/learn/agents-course) ⭐⭐⭐⭐⭐ | 🆓 | 想用免費、框架中立教材動手做的人 | smolagents / LangGraph / LlamaIndex 三家動手 + observability / eval；建並 benchmark 一個 agent | 免費，兩級：Fundamentals（Unit 1 + quiz ≥80%）；Certificate of Completion（再加作業 + 最終挑戰）。HF 直接簽發 | +| [DeepLearning.AI — Agentic AI](https://www.deeplearning.ai/courses/agentic-ai/) ⭐⭐⭐⭐⭐ | 🆓→💰 | 有中階 Python + 基本 LLM/API 概念的開發者 | 四個核心設計模式：reflection、tool use、planning、multi-agent（31 影片 + 8 評量作業） | 旁聽免費（無證書）；證書需付費 Pro（約 $25–30/月）並完成評量。講師 Andrew Ng。中文對照：[`datawhalechina/agentic-ai`](https://github.com/datawhalechina/agentic-ai) | +| [Weights & Biases — AI Engineering: Agents](https://wandb.ai/site/courses/agents/) ⭐⭐⭐⭐ | 🆓 | 想學「會評估、可上線」agent 的開發者 | 與 OpenAI 團隊合作；reasoning model 建 agent、tool/memory/planning、orchestrator-worker 多 agent、用 accuracy/latency/cost 做可複現 eval（約 2 小時） | 免費完成證書（W&B AI Academy 簽發） | +| [IBM — RAG and Agentic AI（Professional Certificate）](https://www.coursera.org/professional-certificates/ibm-rag-and-agentic-ai) ⭐⭐⭐⭐ | 💰 | 想要大廠 Professional Certificate 的人 | RAG + agentic AI 實作，多課程組成的 Professional Certificate | 付費（Coursera Plus；可申請助學金）。IBM 簽發，認可度高於一般完成證書 | +| [Vanderbilt 大學 — AI Agent Developer](https://www.coursera.org/specializations/ai-agents) ⭐⭐⭐⭐ | 💰 | 想要大學背書、系統性學 agent 開發的人 | 設計、打造、調校 agent 軟體；Python agentic 應用 | 付費（Coursera Plus；可申請助學金）。Vanderbilt 大學 Specialization Certificate | + +### tier-2（紮實，但有特定 caveat） + +| 課程（連結） | 成本 | 適合誰 | 教什麼 | 證書 | +|---|---|---|---|---| +| [Coursera — Agentic AI Engineering（Edureka）](https://www.coursera.org/specializations/agentic-ai-engineering) ⭐⭐⭐ | 💰 | 想沿 LangChain / LangGraph / MCP 一路做完的人 | 4 課：LangChain 生態、LCEL、ReAct/memory、LangGraph 多 agent、MCP 部署、eval | 付費 Specialization Certificate。caveat：Edureka 是商業培訓機構（非大學/lab），可信度中等；想要更高背書選上面 IBM / Vanderbilt | +| [Anthropic Academy](https://anthropic.skilljar.com/) ⭐⭐⭐⭐ | 🆓 | 在 Claude / MCP stack 上做 agent 的人 | Claude Code、Claude API、MCP、Agent Skills；17 門自學課、5 條學習軌 | 免費官方證書（Skilljar 簽發，含 quiz，email 註冊即可，可加 LinkedIn）。caveat：vendor-specific（Claude/MCP），補充而非取代框架中立的基礎課 | + +> 也想了解但未列為主 entry：[LangChain Academy — Intro to LangGraph](https://academy.langchain.com/courses/intro-to-langgraph)（🆓 免費 LangGraph 完成證書，single-vendor）。雲廠商（Google Cloud / AWS）的 agentic 路線多半給 skill badge，跟它們各自的付費**專業認證考試**不是同一回事，別混為一談。 + +--- + +## 🀄 中文課程 + +> **gap-first 事實**：會發證書、又 agent 專門的中文課**目前都要付費**，且多是**大廠 vendor 認證**（綁自家 stack）；**zh-TW 原生 + 帶證書 + agent 專門的幾乎不存在**。想免費拿證書，繁中／簡中學習者現階段通常還是走上面的英文課（Hugging Face / W&B / Anthropic 皆免費）。下面三門以 NVIDIA DLI 中文版最貼題。 + +### tier-1（高度可信） + +| 課程（連結） | 成本 | 適合誰 | 教什麼 | 證書 | +|---|---|---|---|---| +| [NVIDIA DLI — 使用大語言模型建構代理式 AI（中文）](https://www.nvidia.cn/training/instructor-led-workshops/building-agentic-ai-applications-with-llms/) ⭐⭐⭐⭐ | 💰 | 想用中文、跟著動手做 agent 系統的人 | 用 LLM 建 agentic 系統：deep reasoning、檢索、tool 呼叫、多 agent、LangGraph、上線部署考量（8 小時） | 付費（約 ¥3500／人，講師帶領、排程制），含 6 個月雲端 lab。完成測驗後拿 NVIDIA DLI 證書。caveat：價格較高、需排課 | + +### tier-2（紮實，但有特定 caveat） + +| 課程（連結） | 成本 | 適合誰 | 教什麼 | 證書 | +|---|---|---|---|---| +| [阿里雲 — 大模型 ACA 認證 + 百煉智能體 Clouder](https://edu.aliyun.com/certification) ⭐⭐⭐ | 💰 | 在阿里雲（百煉/通義）stack 上做 agent、想要圈內認可的人 | 大模型工程；Clouder 系列含「基於百煉平台構建智能體應用」模組 | 付費官方認證（需實名）。caveat：認可是 vendor-scoped（綁百煉/通義）、非學術可轉移；zh-Hans 限定 | +| [華為 — HCIA-AI（大模型應用方向）](https://e.huawei.com/cn/talent/cert/) ⭐⭐⭐ | 💰 | 想要華為生態認可、系統性入門的人 | 2026 V1.0 大綱含人工智慧基礎、大模型知識、大模型應用、智算中心方案 | 付費官方認證（考試制）。caveat：認可偏華為生態與大陸就業市場；zh-Hans。HCIP/HCIE 為更深延伸 | + +> 其他中文選項以備註處理，不列為主 entry：慕課網「AI Agent 全棧開發工程師」（商業 bootcamp，judge by syllabus、外部認可有限）、教育部「人工智慧課程修讀證書」（官方電子證書、偏綜合素養非 agent 工程）。**不收** cert-mill（無評量、無可信簽發方的泛 AIGC 認證）。 + +--- + +## ⚠️ 關於「拿證書」這件事——完整 caveat（請務必照實轉述給讀者） + +1. **完成證書不是 accredited 學歷。** 它代表參與和投入，不代表 mastery；跟學分課、學位是兩回事。本清單**絕不**把這些證書講成資格認證。 +2. **付費 ≠ 一定被認可，免費 ≠ 一定沒份量。** 大廠／大學的付費 Professional Certificate（IBM、Vanderbilt）認可度通常高些；但免費課也可能評量嚴格（Hugging Face 的證書 gate 在 quiz + 作業 + 挑戰）。看的是簽發方和評量，不是價格。 +3. **這些證書實際證明 exposure 與 effort，不是 competence。** 誠實說法是「自主學習的證據」，不是「能打造可上線 agent 的證明」。 +4. **證書最有用的場景是 screening，且很少單獨起作用。** 來自可信來源的證書能幫你過初篩，但幾乎不保證 offer，要搭作品集才有效。 +5. **你做出的作品，比你拿到的證書更重要。** 雇主要的是你能做事的證據（GitHub repo、deploy 的 agent、開源貢獻）。一門課真正的 payoff 是它逼你做出的 artifact，不是那張 PDF。 +6. **別蒐集零散 badge，要往一條連貫的 skill set 走。** 五張不相關的入門證書，遠不如一條展示同一 skill set 的連貫路徑可信——這也是把課程當「roadmap 的步驟」而非「獎盃牆」的理由。 +7. **Recency caveat。** Agent 框架與最佳實務汰換很快，舊 cohort 的證書可能代表過時的知識。看課程的 vintage。 + +--- + +## 維護備註 + +- **最後核對：2026-05。** 課程資訊（尤其證書條件、免費／付費）漂移很快——以各課官網為準，stale 的可見地標註而非默默錯誤。 +- 加新課的門檻：講師／機構可辨識 + 真的有評量或第一方簽發的證書 + 內容夠新。**不收** cert-mill（marketplace 型完成證書外部認可近乎零）。 +- 三語同步：每次增刪／改星等／改證書條件，都要套到 `courses.md` + `courses.en.md` + `courses.zh-Hans.md`。 diff --git a/resources/courses.zh-Hans.md b/resources/courses.zh-Hans.md new file mode 100644 index 0000000..a74cd5d --- /dev/null +++ b/resources/courses.zh-Hans.md @@ -0,0 +1,95 @@ +> [繁體中文](./courses.md) | **简体中文** | [English](./courses.en.md) + +# 线上 AI Agent 课程（带证书对照） + +> [← 回主路线 README](../README.zh-Hans.md) + +> 📌 **这份是 reference catalog，不是排名。** 本 repo 是动手做的学习路线，**不取代**结构化线上课程——下面这些课程可当作平行的影片／互动式入门对照。想动手做 → 回 [stages](../README.zh-Hans.md)；想先查用语 → [`resources/glossary.zh-Hans.md`](glossary.zh-Hans.md)。本页只收**会发证书**的课程。 + +> ⚠️ **先读这段，再往下看。** 完成证书（Certificate of Completion）证明的是**你参与并完成了课程**，不是 mastery，也**不等于** accredited 学历或学分。雇主看的顺序通常是“内容跟职务相不相关 > 评量严不严 > 你做出什么作品 > 机构背书”，证书本身排在后面。把它当成**自主学习的证据 + 维持动机的结构**，不是“拿了证书就能找到工作”。下面“证书”栏只陈述事实，不做价值判断。 + +--- + +## ⚡ 快速选（30 秒挑一门） + +- **完全没方向** → 先做 [Hugging Face — AI Agents Course](https://huggingface.co/learn/agents-course)：🆓 免费、框架中立、证书要过 quiz + 作业才拿得到。本页最推荐的起点。 +- **想免费就拿证书** → Hugging Face、[Weights & Biases — AI Engineering: Agents](https://wandb.ai/site/courses/agents/)、[Anthropic Academy](https://anthropic.skilljar.com/)（Claude / MCP 方向）。 +- **想要履历用的大学／大厂背书（付费）** → [IBM — RAG and Agentic AI（Professional Certificate）](https://www.coursera.org/professional-certificates/ibm-rag-and-agentic-ai)、[Vanderbilt — AI Agent Developer](https://www.coursera.org/specializations/ai-agents)。 +- **想沿一个框架（LangChain / LangGraph）做到底** → [Coursera — Agentic AI Engineering（Edureka）](https://www.coursera.org/specializations/agentic-ai-engineering)。 +- **想用中文** → [NVIDIA DLI 中文](https://www.nvidia.cn/training/instructor-led-workshops/building-agentic-ai-applications-with-llms/)（最贴题）、阿里云 ACA、华为 HCIA-AI。 + > ⚠️ **中文、带证书、又 agent 专门的课，目前基本都要付费**（NVIDIA DLI / 阿里 / 华为）。想**免费**拿证书，现阶段得走上面的英文课（部分有社群字幕）。 + +**图例**：🆓 免费（含证书）· 💰 付费 · 🆓→💰 旁听免费、证书付费。 + +--- + +## 怎么读这份清单 + +- **学习价值**（讲师可信 + 动手做 + 内容够新）和**证书价值**是两条不同的轴。本清单主要照学习价值排，证书当成大量加注的次要信号。 +- ⭐ 星等照 [style-guide §2](style-guide.zh-Hans.md#2-推荐星等定义)：⭐⭐⭐⭐⭐ 必做 … ⭐ 利基。 +- **付费 Professional Certificate / Specialization（IBM、大学）认可度，通常高于免费完成证书**；但两者都不是学位，Coursera 那种“X% career outcomes”是行销数字。 +- **Agent 领域变化快**：超过 12–18 个月的课，对框架版本（smolagents / LangGraph / MCP）要打 recency 折扣。 + +--- + +## 🌍 英文课程 + +### tier-1（高度可信 + 动手做 + 够新） + +| 课程（连结） | 成本 | 适合谁 | 教什么 | 证书 | +|---|---|---|---|---| +| [Hugging Face — AI Agents Course](https://huggingface.co/learn/agents-course) ⭐⭐⭐⭐⭐ | 🆓 | 想用免费、框架中立教材动手做的人 | smolagents / LangGraph / LlamaIndex 三家动手 + observability / eval；建并 benchmark 一个 agent | 免费，两级：Fundamentals（Unit 1 + quiz ≥80%）；Certificate of Completion（再加作业 + 最终挑战）。HF 直接签发 | +| [DeepLearning.AI — Agentic AI](https://www.deeplearning.ai/courses/agentic-ai/) ⭐⭐⭐⭐⭐ | 🆓→💰 | 有中阶 Python + 基本 LLM/API 概念的开发者 | 四个核心设计模式：reflection、tool use、planning、multi-agent（31 影片 + 8 评量作业） | 旁听免费（无证书）；证书需付费 Pro（约 $25–30/月）并完成评量。讲师 Andrew Ng。中文对照：[`datawhalechina/agentic-ai`](https://github.com/datawhalechina/agentic-ai) | +| [Weights & Biases — AI Engineering: Agents](https://wandb.ai/site/courses/agents/) ⭐⭐⭐⭐ | 🆓 | 想学“会评估、可上线”agent 的开发者 | 与 OpenAI 团队合作；reasoning model 建 agent、tool/memory/planning、orchestrator-worker 多 agent、用 accuracy/latency/cost 做可复现 eval（约 2 小时） | 免费完成证书（W&B AI Academy 签发） | +| [IBM — RAG and Agentic AI（Professional Certificate）](https://www.coursera.org/professional-certificates/ibm-rag-and-agentic-ai) ⭐⭐⭐⭐ | 💰 | 想要大厂 Professional Certificate 的人 | RAG + agentic AI 实作，多课程组成的 Professional Certificate | 付费（Coursera Plus；可申请助学金）。IBM 签发，认可度高于一般完成证书 | +| [Vanderbilt 大学 — AI Agent Developer](https://www.coursera.org/specializations/ai-agents) ⭐⭐⭐⭐ | 💰 | 想要大学背书、系统性学 agent 开发的人 | 设计、打造、调校 agent 软件；Python agentic 应用 | 付费（Coursera Plus；可申请助学金）。Vanderbilt 大学 Specialization Certificate | + +### tier-2（扎实，但有特定 caveat） + +| 课程（连结） | 成本 | 适合谁 | 教什么 | 证书 | +|---|---|---|---|---| +| [Coursera — Agentic AI Engineering（Edureka）](https://www.coursera.org/specializations/agentic-ai-engineering) ⭐⭐⭐ | 💰 | 想沿 LangChain / LangGraph / MCP 一路做完的人 | 4 课：LangChain 生态、LCEL、ReAct/memory、LangGraph 多 agent、MCP 部署、eval | 付费 Specialization Certificate。caveat：Edureka 是商业培训机构（非大学/lab），可信度中等；想要更高背书选上面 IBM / Vanderbilt | +| [Anthropic Academy](https://anthropic.skilljar.com/) ⭐⭐⭐⭐ | 🆓 | 在 Claude / MCP stack 上做 agent 的人 | Claude Code、Claude API、MCP、Agent Skills；17 门自学课、5 条学习轨 | 免费官方证书（Skilljar 签发，含 quiz，email 注册即可，可加 LinkedIn）。caveat：vendor-specific（Claude/MCP），补充而非取代框架中立的基础课 | + +> 也想了解但未列为主 entry：[LangChain Academy — Intro to LangGraph](https://academy.langchain.com/courses/intro-to-langgraph)（🆓 免费 LangGraph 完成证书，single-vendor）。云厂商（Google Cloud / AWS）的 agentic 路线多半给 skill badge，跟它们各自的付费**专业认证考试**不是同一回事，别混为一谈。 + +--- + +## 🀄 中文课程 + +> **gap-first 事实**：会发证书、又 agent 专门的中文课**目前都要付费**，且多是**大厂 vendor 认证**（绑自家 stack）；**zh-TW 原生 + 带证书 + agent 专门的几乎不存在**。想免费拿证书，繁中／简中学习者现阶段通常还是走上面的英文课（Hugging Face / W&B / Anthropic 皆免费）。下面三门以 NVIDIA DLI 中文版最贴题。 + +### tier-1（高度可信） + +| 课程（连结） | 成本 | 适合谁 | 教什么 | 证书 | +|---|---|---|---|---| +| [NVIDIA DLI — 使用大语言模型建构代理式 AI（中文）](https://www.nvidia.cn/training/instructor-led-workshops/building-agentic-ai-applications-with-llms/) ⭐⭐⭐⭐ | 💰 | 想用中文、跟着动手做 agent 系统的人 | 用 LLM 建 agentic 系统：deep reasoning、检索、tool 调用、多 agent、LangGraph、上线部署考量（8 小时） | 付费（约 ¥3500／人，讲师带领、排程制），含 6 个月云端 lab。完成测验后拿 NVIDIA DLI 证书。caveat：价格较高、需排课 | + +### tier-2（扎实，但有特定 caveat） + +| 课程（连结） | 成本 | 适合谁 | 教什么 | 证书 | +|---|---|---|---|---| +| [阿里云 — 大模型 ACA 认证 + 百炼智能体 Clouder](https://edu.aliyun.com/certification) ⭐⭐⭐ | 💰 | 在阿里云（百炼/通义）stack 上做 agent、想要圈内认可的人 | 大模型工程；Clouder 系列含“基于百炼平台构建智能体应用”模组 | 付费官方认证（需实名）。caveat：认可是 vendor-scoped（绑百炼/通义）、非学术可转移；zh-Hans 限定 | +| [华为 — HCIA-AI（大模型应用方向）](https://e.huawei.com/cn/talent/cert/) ⭐⭐⭐ | 💰 | 想要华为生态认可、系统性入门的人 | 2026 V1.0 大纲含人工智慧基础、大模型知识、大模型应用、智算中心方案 | 付费官方认证（考试制）。caveat：认可偏华为生态与大陆就业市场；zh-Hans。HCIP/HCIE 为更深延伸 | + +> 其他中文选项以备注处理，不列为主 entry：慕课网“AI Agent 全栈开发工程师”（商业 bootcamp，judge by syllabus、外部认可有限）、教育部“人工智慧课程修读证书”（官方电子证书、偏综合素养非 agent 工程）。**不收** cert-mill（无评量、无可信签发方的泛 AIGC 认证）。 + +--- + +## ⚠️ 关于“拿证书”这件事——完整 caveat（请务必照实转述给读者） + +1. **完成证书不是 accredited 学历。** 它代表参与和投入，不代表 mastery；跟学分课、学位是两回事。本清单**绝不**把这些证书讲成资格认证。 +2. **付费 ≠ 一定被认可，免费 ≠ 一定没份量。** 大厂／大学的付费 Professional Certificate（IBM、Vanderbilt）认可度通常高些；但免费课也可能评量严格（Hugging Face 的证书 gate 在 quiz + 作业 + 挑战）。看的是签发方和评量，不是价格。 +3. **这些证书实际证明 exposure 与 effort，不是 competence。** 诚实说法是“自主学习的证据”，不是“能打造可上线 agent 的证明”。 +4. **证书最有用的场景是 screening，且很少单独起作用。** 来自可信来源的证书能帮你过初筛，但几乎不保证 offer，要搭作品集才有效。 +5. **你做出的作品，比你拿到的证书更重要。** 雇主要的是你能做事的证据（GitHub repo、deploy 的 agent、开源贡献）。一门课真正的 payoff 是它逼你做出的 artifact，不是那张 PDF。 +6. **别搜集零散 badge，要往一条连贯的 skill set 走。** 五张不相关的入门证书，远不如一条展示同一 skill set 的连贯路径可信——这也是把课程当“roadmap 的步骤”而非“奖杯墙”的理由。 +7. **Recency caveat。** Agent 框架与最佳实务汰换很快，旧 cohort 的证书可能代表过时的知识。看课程的 vintage。 + +--- + +## 维护备注 + +- **最后核对：2026-05。** 课程资讯（尤其证书条件、免费／付费）漂移很快——以各课官网为准，stale 的可见地标注而非默默错误。 +- 加新课的门槛：讲师／机构可辨识 + 真的有评量或第一方签发的证书 + 内容够新。**不收** cert-mill（marketplace 型完成证书外部认可近乎零）。 +- 三语同步：每次增删／改星等／改证书条件，都要套到 `courses.md` + `courses.en.md` + `courses.zh-Hans.md`。 diff --git a/resources/diagrams/add-branch-decision-flow.png b/resources/diagrams/add-branch-decision-flow.png new file mode 100644 index 0000000..72addb5 Binary files /dev/null and b/resources/diagrams/add-branch-decision-flow.png differ diff --git a/resources/diagrams/agent-guardrail-patterns.en.png b/resources/diagrams/agent-guardrail-patterns.en.png new file mode 100644 index 0000000..fc7dfac Binary files /dev/null and b/resources/diagrams/agent-guardrail-patterns.en.png differ diff --git a/resources/diagrams/agent-guardrail-patterns.png b/resources/diagrams/agent-guardrail-patterns.png new file mode 100644 index 0000000..d80b86f Binary files /dev/null and b/resources/diagrams/agent-guardrail-patterns.png differ diff --git a/resources/diagrams/agent-guardrail-patterns.zh-Hans.png b/resources/diagrams/agent-guardrail-patterns.zh-Hans.png new file mode 100644 index 0000000..5ad9914 Binary files /dev/null and b/resources/diagrams/agent-guardrail-patterns.zh-Hans.png differ diff --git a/resources/diagrams/agent-paradigm-decision-tree.en.png b/resources/diagrams/agent-paradigm-decision-tree.en.png new file mode 100644 index 0000000..c19c96e Binary files /dev/null and b/resources/diagrams/agent-paradigm-decision-tree.en.png differ diff --git a/resources/diagrams/agent-paradigm-decision-tree.png b/resources/diagrams/agent-paradigm-decision-tree.png new file mode 100644 index 0000000..a1390e4 Binary files /dev/null and b/resources/diagrams/agent-paradigm-decision-tree.png differ diff --git a/resources/diagrams/agent-paradigm-decision-tree.zh-Hans.png b/resources/diagrams/agent-paradigm-decision-tree.zh-Hans.png new file mode 100644 index 0000000..c59e43e Binary files /dev/null and b/resources/diagrams/agent-paradigm-decision-tree.zh-Hans.png differ diff --git a/resources/diagrams/ai-ml-llm-agent-hierarchy.en.png b/resources/diagrams/ai-ml-llm-agent-hierarchy.en.png new file mode 100644 index 0000000..b9dbb71 Binary files /dev/null and b/resources/diagrams/ai-ml-llm-agent-hierarchy.en.png differ diff --git a/resources/diagrams/ai-ml-llm-agent-hierarchy.png b/resources/diagrams/ai-ml-llm-agent-hierarchy.png new file mode 100644 index 0000000..d0367b9 Binary files /dev/null and b/resources/diagrams/ai-ml-llm-agent-hierarchy.png differ diff --git a/resources/diagrams/ai-ml-llm-agent-hierarchy.zh-Hans.png b/resources/diagrams/ai-ml-llm-agent-hierarchy.zh-Hans.png new file mode 100644 index 0000000..b336361 Binary files /dev/null and b/resources/diagrams/ai-ml-llm-agent-hierarchy.zh-Hans.png differ diff --git a/resources/diagrams/banner.en.png b/resources/diagrams/banner.en.png new file mode 100644 index 0000000..fdcbcc6 Binary files /dev/null and b/resources/diagrams/banner.en.png differ diff --git a/resources/diagrams/banner.png b/resources/diagrams/banner.png new file mode 100644 index 0000000..344cbd3 Binary files /dev/null and b/resources/diagrams/banner.png differ diff --git a/resources/diagrams/banner.zh-Hans.png b/resources/diagrams/banner.zh-Hans.png new file mode 100644 index 0000000..77ba918 Binary files /dev/null and b/resources/diagrams/banner.zh-Hans.png differ diff --git a/resources/diagrams/branch-decision-tree.en.png b/resources/diagrams/branch-decision-tree.en.png new file mode 100644 index 0000000..d3d575c Binary files /dev/null and b/resources/diagrams/branch-decision-tree.en.png differ diff --git a/resources/diagrams/branch-decision-tree.png b/resources/diagrams/branch-decision-tree.png new file mode 100644 index 0000000..f98b7a6 Binary files /dev/null and b/resources/diagrams/branch-decision-tree.png differ diff --git a/resources/diagrams/branch-decision-tree.zh-Hans.png b/resources/diagrams/branch-decision-tree.zh-Hans.png new file mode 100644 index 0000000..7df0473 Binary files /dev/null and b/resources/diagrams/branch-decision-tree.zh-Hans.png differ diff --git a/resources/diagrams/branch-tier-progression.png b/resources/diagrams/branch-tier-progression.png new file mode 100644 index 0000000..151411e Binary files /dev/null and b/resources/diagrams/branch-tier-progression.png differ diff --git a/resources/diagrams/chunking-strategies.jpg b/resources/diagrams/chunking-strategies.jpg new file mode 100644 index 0000000..8e95cae Binary files /dev/null and b/resources/diagrams/chunking-strategies.jpg differ diff --git a/resources/diagrams/claude-architecture-map.en.png b/resources/diagrams/claude-architecture-map.en.png new file mode 100644 index 0000000..f5a3662 Binary files /dev/null and b/resources/diagrams/claude-architecture-map.en.png differ diff --git a/resources/diagrams/claude-architecture-map.png b/resources/diagrams/claude-architecture-map.png new file mode 100644 index 0000000..6cc9781 Binary files /dev/null and b/resources/diagrams/claude-architecture-map.png differ diff --git a/resources/diagrams/claude-architecture-map.zh-Hans.png b/resources/diagrams/claude-architecture-map.zh-Hans.png new file mode 100644 index 0000000..404e9d0 Binary files /dev/null and b/resources/diagrams/claude-architecture-map.zh-Hans.png differ diff --git a/resources/diagrams/concept-cluster.en.png b/resources/diagrams/concept-cluster.en.png new file mode 100644 index 0000000..4e2ba81 Binary files /dev/null and b/resources/diagrams/concept-cluster.en.png differ diff --git a/resources/diagrams/concept-cluster.png b/resources/diagrams/concept-cluster.png new file mode 100644 index 0000000..fa302de Binary files /dev/null and b/resources/diagrams/concept-cluster.png differ diff --git a/resources/diagrams/concept-cluster.zh-Hans.png b/resources/diagrams/concept-cluster.zh-Hans.png new file mode 100644 index 0000000..391da60 Binary files /dev/null and b/resources/diagrams/concept-cluster.zh-Hans.png differ diff --git a/resources/diagrams/concept-prompts.md b/resources/diagrams/concept-prompts.md new file mode 100644 index 0000000..84db643 --- /dev/null +++ b/resources/diagrams/concept-prompts.md @@ -0,0 +1,484 @@ +# Stage 7.5 概念圖 — ChatGPT image-gen prompts（3 圖 × 3 語言） + +> 把下面的 prompt 整段複製貼到 ChatGPT（含 DALL-E / 原生 image-gen model），生成對應的概念圖 PNG。 + +> 📋 **v2 更新**：加上 icon 描述（每個元素附 emoji icon、視覺辨識度更高）+ 中英文 bilingual 術語（中文 prompt 用「中文 (English)」格式介紹專有詞、英文 prompt 純英文）+ 明確「no source attribution」規則。 + +> 📂 **批次上傳檔案**：對應的 9 個 `.txt` 在 `C:\Users\wenyu\Downloads\awesome-agentic-ai-zh-images\stage7.5\`、可直接拖進 ChatGPT。 + +--- + +## 圖 A — 12 進階概念 cluster map + +### zh-TW prompt + +``` +生成一張清晰的技術示意圖、主題：「12 個進階 Agentic AI 概念分類地圖 (12 Advanced Agentic AI Concepts — Cluster Map)」。 + +整體設計：簡潔技術 blog 風（參考 Anthropic engineering blog），2D 表格 / grid 結構、配合視覺圖示 (icon) 讓內容好讀。 + +橫軸（X-axis、由左至右）= 4 個 agent stack 層次、每個加上一個 icon： + 1. Service 層 (Service layer) — 圖示：🎼 + 2. Repo 層 (Repo layer) — 圖示：💾 + 3. Config 層 (Config layer) — 圖示：⚙️ + 4. Types 層 (Types layer) — 圖示：📋 + +縱軸（Y-axis、由上至下）= 4 個問題類型、每個加 icon + 一句白話副標： + 1. 編排類 (Orchestration) — 🎭、副標「agent 怎麼分工協作 (how agents coordinate)」 + 2. 反思類 (Reflection) — 🪞、副標「agent 怎麼自我修正 (how agents self-correct)」 + 3. 治理類 (Governance) — 🛡️、副標「agent 自主權邊界 (autonomy boundaries)」 + 4. 韌性類 (Resilience) — 💪、副標「agent 預防失敗 (failure prevention)」 + +在對應格子放概念名稱 + 編號 + 小 icon： + - Service × 編排：1. Work Boundary (跨層 *) 🌐 / 3. Speculative Parallel ⚡ / 10. Self-organizing Teams 🐝 + - Service × 反思：5. Plan-Act-Reflect 🔁 / 4. Agent-as-Judge ⚖️ + - Service × 治理：6. Hierarchical Task Decomposition 🪜 + - Types × 編排：2. Contract Hand-offs 🤝 + - Types × 反思：11. Spec-driven Development 📐 + - Config × 治理：7. Autonomy Gradients 🎚️ / 12. Graceful Degradation 🛡️↓ + - Config × 韌性：8. Cost-aware Budget Gates 💰 / 9. Failure Injection 💉 + - Repo × 反思 格子註記：「memory 由 Stage 6 提供 (memory provided by Stage 6)」、不放概念名稱 + +視覺風格： +- Sans-serif 字體（如 Inter）、字夠大易讀 +- 配色：2-3 色（hover blue 主色 + neutral grey + 強調 orange） +- Icon 明顯但不擾人、能視覺辨識 +- 比例 16:9 適合 README banner +- 留白多、版面不要塞滿 + +絕對不要 (DO NOT)： +- 不要任何「Source: ...」「by ...」「Credits: ...」「Reference: ...」標籤 +- 不要 Anthropic / OpenAI / Cognition 等 vendor logo 或文字 +- 不要任何來源 / 出處 / 引用 attribution +- 不要 3D 透視、漸層陰影、卡通插圖 +- 不要 emoji 以外的裝飾性 illustration +``` + +### zh-Hans prompt + +``` +生成一张清晰的技术示意图、主题：「12 个进阶 Agentic AI 概念分类地图 (12 Advanced Agentic AI Concepts — Cluster Map)」。 + +整体设计：简洁技术 blog 风（参考 Anthropic engineering blog），2D 表格 / grid 结构、配合视觉图示 (icon) 让内容好读。 + +横轴（X-axis、由左至右）= 4 个 agent stack 层次、每个加上一个 icon： + 1. Service 层 (Service layer) — 图示：🎼 + 2. Repo 层 (Repo layer) — 图示：💾 + 3. Config 层 (Config layer) — 图示：⚙️ + 4. Types 层 (Types layer) — 图示：📋 + +纵轴（Y-axis、由上至下）= 4 个问题类型、每个加 icon + 一句白话副标： + 1. 编排类 (Orchestration) — 🎭、副标「agent 怎么分工协作 (how agents coordinate)」 + 2. 反思类 (Reflection) — 🪞、副标「agent 怎么自我修正 (how agents self-correct)」 + 3. 治理类 (Governance) — 🛡️、副标「agent 自主权边界 (autonomy boundaries)」 + 4. 韧性类 (Resilience) — 💪、副标「agent 预防失败 (failure prevention)」 + +在对应格子放概念名称 + 编号 + 小 icon： + - Service × 编排：1. Work Boundary (跨层 *) 🌐 / 3. Speculative Parallel ⚡ / 10. Self-organizing Teams 🐝 + - Service × 反思：5. Plan-Act-Reflect 🔁 / 4. Agent-as-Judge ⚖️ + - Service × 治理：6. Hierarchical Task Decomposition 🪜 + - Types × 编排：2. Contract Hand-offs 🤝 + - Types × 反思：11. Spec-driven Development 📐 + - Config × 治理：7. Autonomy Gradients 🎚️ / 12. Graceful Degradation 🛡️↓ + - Config × 韧性：8. Cost-aware Budget Gates 💰 / 9. Failure Injection 💉 + - Repo × 反思 格子注记：「memory 由 Stage 6 提供 (memory provided by Stage 6)」、不放概念名称 + +视觉风格： +- Sans-serif 字体（如 Inter）、字够大易读 +- 配色：2-3 色（hover blue 主色 + neutral grey + 强调 orange） +- Icon 明显但不扰人、能视觉辨识 +- 比例 16:9 适合 README banner +- 留白多、版面不要塞满 + +绝对不要 (DO NOT)： +- 不要任何「Source: ...」「by ...」「Credits: ...」「Reference: ...」标签 +- 不要 Anthropic / OpenAI / Cognition 等 vendor logo 或文字 +- 不要任何来源 / 出处 / 引用 attribution +- 不要 3D 透视、渐层阴影、卡通插图 +- 不要 emoji 以外的装饰性 illustration +``` + +### en prompt + +``` +Generate a clean technical diagram titled "12 Advanced Agentic AI Concepts — Cluster Map". + +Overall design: clean tech blog aesthetic (reference Anthropic engineering blog), 2D table/grid layout with visual icons to aid scanning. + +X-axis (left to right) = 4 agent stack layers, each with an icon: + 1. Service layer — icon: 🎼 + 2. Repo layer — icon: 💾 + 3. Config layer — icon: ⚙️ + 4. Types layer — icon: 📋 + +Y-axis (top to bottom) = 4 problem categories, each with an icon + plain-language subtitle: + 1. Orchestration — 🎭, subtitle "how agents coordinate" + 2. Reflection — 🪞, subtitle "how agents self-correct" + 3. Governance — 🛡️, subtitle "autonomy boundaries" + 4. Resilience — 💪, subtitle "failure prevention" + +Place concept names + numbers + small icons in the matching cells: + - Service × Orchestration: 1. Work Boundary (cross-layer *) 🌐 / 3. Speculative Parallel ⚡ / 10. Self-organizing Teams 🐝 + - Service × Reflection: 5. Plan-Act-Reflect 🔁 / 4. Agent-as-Judge ⚖️ + - Service × Governance: 6. Hierarchical Task Decomposition 🪜 + - Types × Orchestration: 2. Contract Hand-offs 🤝 + - Types × Reflection: 11. Spec-driven Development 📐 + - Config × Governance: 7. Autonomy Gradients 🎚️ / 12. Graceful Degradation 🛡️↓ + - Config × Resilience: 8. Cost-aware Budget Gates 💰 / 9. Failure Injection 💉 + - Repo × Reflection cell annotation: "memory provided by Stage 6" (no concept name) + +Visual style: +- Sans-serif typography (e.g., Inter), font size large enough to read +- Color palette: 2-3 colors (primary hover blue + neutral grey + accent orange) +- Icons clear but not intrusive, visually identifiable +- 16:9 aspect ratio, suitable as a README banner +- Generous whitespace, do not pack the layout + +DO NOT: +- No "Source: ..." / "by ..." / "Credits: ..." / "Reference: ..." labels +- No vendor logos or names (Anthropic / OpenAI / Cognition / etc.) +- No attribution or citation of any kind +- No 3D perspective, gradient shadows, or cartoon illustration +- No decorative illustration beyond the listed emoji icons +``` + +--- + +## 圖 B — Reading decision tree + +### zh-TW prompt + +``` +生成一張清晰的技術示意圖、主題：「Agentic AI 進階閱讀決策樹 (Advanced Reading Decision Tree)」。 + +整體設計：top-down 決策樹、配合視覺圖示讓 5 條分支好辨識。 + +頂端標題（icon 🤔）：「你現在卡什麼問題？ (What problem are you stuck on right now?)」 + +往下分出 5 條分支、橫向排列、每條獨特顏色 + icon： + +1. 紅色分支 🌱 + 問題：「不知道 agent 怎麼開始 (Don't know how to start with agents)」 + ↓ + 📖 Anthropic Building Effective Agents (~20 min) + +2. 綠色分支 🤝 + 問題：「multi-agent 要不要用、怎麼開 (Multi-agent: when and how)」 + ↓ + 📖 Cognition Don't Build Multi-Agents (~10 min) + +3. 黃色分支 🧠 + 問題：「context 沒效率 (Context inefficient)」 + ↓ + 📖 Anthropic Skills + CLAUDE.md memory docs (~15 min) + +4. 藍色分支 🧪 + 問題：「eval 怎麼寫 / 自動驗證 (How to write evals / auto-verify)」 + ↓ + 📖 Hamel Husain Evals blog (~15 min) + +5. 紫色分支 🚀 + 問題：「想跟上 frontier 現況 (Catch up on frontier)」 + ↓ + 📖 LangGraph Plan-Execute / Voyager / Self-Discover paper (~30 min/篇) + +底部小字：「規則 (Rule)：每分支最多挑 2 篇深讀、不要 broad-scan (pick at most 2 per branch)」 + +視覺風格： +- 方框 + 連線、不要過多圓角 +- 文章名稱用粗體、加 📖 icon +- Sans-serif、比例 4:3 或正方形 +- 5 條分支顏色明顯區分 +- 每條分支頂端的 icon 視覺辨識度高（不只是 emoji 文字） + +絕對不要 (DO NOT)： +- 不要任何「Source: ...」「by Anthropic / Cognition AI / Hamel Husain / LangGraph」等 attribution +- 不要 vendor logo / 公司商標 +- 不要 mind-map 那種繽紛樹枝樹葉風格 +- 不要 emoji 之外的卡通插圖 +- 不要 3D 渲染、漸層陰影 +``` + +### zh-Hans prompt + +``` +生成一张清晰的技术示意图、主题：「Agentic AI 进阶阅读决策树 (Advanced Reading Decision Tree)」。 + +整体设计：top-down 决策树、配合视觉图示让 5 条分支好辨识。 + +顶端标题（icon 🤔）：「你现在卡什么问题？ (What problem are you stuck on right now?)」 + +往下分出 5 条分支、横向排列、每条独特颜色 + icon： + +1. 红色分支 🌱 + 问题：「不知道 agent 怎么开始 (Don't know how to start with agents)」 + ↓ + 📖 Anthropic Building Effective Agents (~20 min) + +2. 绿色分支 🤝 + 问题：「multi-agent 要不要用、怎么开 (Multi-agent: when and how)」 + ↓ + 📖 Cognition Don't Build Multi-Agents (~10 min) + +3. 黄色分支 🧠 + 问题：「context 没效率 (Context inefficient)」 + ↓ + 📖 Anthropic Skills + CLAUDE.md memory docs (~15 min) + +4. 蓝色分支 🧪 + 问题：「eval 怎么写 / 自动验证 (How to write evals / auto-verify)」 + ↓ + 📖 Hamel Husain Evals blog (~15 min) + +5. 紫色分支 🚀 + 问题：「想跟上 frontier 现况 (Catch up on frontier)」 + ↓ + 📖 LangGraph Plan-Execute / Voyager / Self-Discover paper (~30 min/篇) + +底部小字：「规则 (Rule)：每分支最多挑 2 篇深读、不要 broad-scan (pick at most 2 per branch)」 + +视觉风格： +- 方框 + 连线、不要过多圆角 +- 文章名称用粗体、加 📖 icon +- Sans-serif、比例 4:3 或正方形 +- 5 条分支颜色明显区分 +- 每条分支顶端的 icon 视觉辨识度高（不只是 emoji 文字） + +绝对不要 (DO NOT)： +- 不要任何「Source: ...」「by Anthropic / Cognition AI / Hamel Husain / LangGraph」等 attribution +- 不要 vendor logo / 公司商标 +- 不要 mind-map 那种缤纷树枝树叶风格 +- 不要 emoji 之外的卡通插图 +- 不要 3D 渲染、渐层阴影 +``` + +### en prompt + +``` +Generate a clean technical diagram titled "Agentic AI Advanced Reading Decision Tree". + +Overall design: top-down decision tree with visual icons making the 5 branches easy to distinguish. + +Top heading (with icon 🤔): "What problem are you stuck on right now?" + +Branch down into 5 columns, laid out horizontally, each with a distinct color + icon: + +1. Red branch 🌱 + Problem: "Don't know how to start with agents" + ↓ + 📖 Anthropic Building Effective Agents (~20 min) + +2. Green branch 🤝 + Problem: "Multi-agent: when and how" + ↓ + 📖 Cognition Don't Build Multi-Agents (~10 min) + +3. Yellow branch 🧠 + Problem: "Context inefficient" + ↓ + 📖 Anthropic Skills + CLAUDE.md memory docs (~15 min) + +4. Blue branch 🧪 + Problem: "How to write evals / auto-verify" + ↓ + 📖 Hamel Husain Evals blog (~15 min) + +5. Purple branch 🚀 + Problem: "Catch up on the frontier" + ↓ + 📖 LangGraph Plan-Execute / Voyager / Self-Discover paper (~30 min each) + +Footer: "Rule: pick at most 2 papers per branch — do not broad-scan." + +Visual style: +- Rectangular boxes + lines, minimal rounded corners +- Article names in bold, with a 📖 icon +- Sans-serif, 4:3 or square aspect ratio +- The 5 branches have distinct colors +- Branch-top icons are visually identifiable (not just emoji as plain text) + +DO NOT: +- No "Source: ..." / "by Anthropic" / "Cognition AI" / "Hamel Husain" / "LangGraph" attribution +- No vendor logos / company marks +- No colorful mind-map style with tree-branch / leaf illustrations +- No cartoon illustrations beyond the listed emoji icons +- No 3D rendering, gradient shadows +``` + +--- + +## 圖 C — F11-F14 failure-mode lifecycle + +### zh-TW prompt + +``` +生成一張清晰的技術示意圖、主題：「Agent failure mode 進化循環 (Failure-Mode Evolution Cycle)」。 + +整體設計：左/中央 5 步驟循環 (cycle / loop) + 右側 F14 具體例子、每個步驟有獨特 icon。 + +主圖（左 / 中央）：5 步驟循環、順時針或從上往下： + + ① 💥 發現 incident (Discover incident) + — dogfood 或 production fail + ↓ + ② 📝 文件化 (Document) + — docs/observed-failure-modes.md + — 命名為 F-N (named F-N、如 F11 / F12 / F13 / F14) + ↓ + ③ 📋 加進 preset YAML check (Codify) + — acceptance-gate preset YAML + ↓ + ④ 🤖✅ 自動 catch (Auto-catch) + — 下次同樣情境自動被抓 + ↓ + ⑤ 📜 mandatory rule (Mandatory invocation rule) + — CLAUDE.md / SKILL.md 規則 + +從 ⑤ 拉一條箭頭回到 ①、循環持續、加註「持續循環 (continuous loop)」。 + +中間或側邊加文字：「不是寫死所有規則、是每出一次包就 codify 一次 (codify each incident once, don't pre-write every rule)」。 + +右側獨立框：F14 具體例子（直線箭頭）： + 💥 F14 incident (2026-05-13) + ↓ + 📝 observed-failure-modes.md F14 + ↓ + 📋 multi-locale-mirror-sync.yml + check + ↓ + 🤖✅ Retrospective test → 9+ real drift caught + ↓ + 📜 CLAUDE.md mandatory rule + +視覺風格： +- 主圖（5-step loop）+ F14 線性 example 右側、用線分隔 +- 配色：incident 紅、document blue、preset green、auto-catch orange、CLAUDE.md purple +- Sans-serif、比例 16:9 +- 簡潔 tech blog 風 + +絕對不要 (DO NOT)： +- 不要任何「Source: ...」「by agent-collab-skills」「Anthropic Skills 機制」等 attribution +- 不要 vendor logo / 公司商標 +- 不要齒輪 / 發條 / 鐘錶 visual metaphor +- 不要 emoji 之外的卡通插圖 +- 不要 3D 渲染、漸層陰影 +``` + +### zh-Hans prompt + +``` +生成一张清晰的技术示意图、主题：「Agent failure mode 进化循环 (Failure-Mode Evolution Cycle)」。 + +整体设计：左/中央 5 步骤循环 (cycle / loop) + 右侧 F14 具体例子、每个步骤有独特 icon。 + +主图（左 / 中央）：5 步骤循环、顺时针或从上往下： + + ① 💥 发现 incident (Discover incident) + — dogfood 或 production fail + ↓ + ② 📝 文档化 (Document) + — docs/observed-failure-modes.md + — 命名为 F-N (named F-N、如 F11 / F12 / F13 / F14) + ↓ + ③ 📋 加进 preset YAML check (Codify) + — acceptance-gate preset YAML + ↓ + ④ 🤖✅ 自动 catch (Auto-catch) + — 下次同样情境自动被抓 + ↓ + ⑤ 📜 mandatory rule (Mandatory invocation rule) + — CLAUDE.md / SKILL.md 规则 + +从 ⑤ 拉一条箭头回到 ①、循环持续、加注「持续循环 (continuous loop)」。 + +中间或侧边加文字：「不是写死所有规则、是每出一次包就 codify 一次 (codify each incident once, don't pre-write every rule)」。 + +右侧独立框：F14 具体例子（直线箭头）： + 💥 F14 incident (2026-05-13) + ↓ + 📝 observed-failure-modes.md F14 + ↓ + 📋 multi-locale-mirror-sync.yml + check + ↓ + 🤖✅ Retrospective test → 9+ real drift caught + ↓ + 📜 CLAUDE.md mandatory rule + +视觉风格： +- 主图（5-step loop）+ F14 线性 example 右侧、用线分隔 +- 配色：incident 红、document blue、preset green、auto-catch orange、CLAUDE.md purple +- Sans-serif、比例 16:9 +- 简洁 tech blog 风 + +绝对不要 (DO NOT)： +- 不要任何「Source: ...」「by agent-collab-skills」「Anthropic Skills 机制」等 attribution +- 不要 vendor logo / 公司商标 +- 不要齿轮 / 发条 / 钟表 visual metaphor +- 不要 emoji 之外的卡通插图 +- 不要 3D 渲染、渐层阴影 +``` + +### en prompt + +``` +Generate a clean technical diagram titled "Agent Failure-Mode Evolution Cycle". + +Overall design: 5-step cycle / loop on the left/center + F14 concrete example on the right, each step with a distinct icon. + +Main diagram (left / center): 5-step cycle, clockwise or top-down: + + (1) 💥 Discover incident + — from dogfood or production failure + ↓ + (2) 📝 Document + — in docs/observed-failure-modes.md + — name it F-N (e.g. F11 / F12 / F13 / F14) + ↓ + (3) 📋 Codify as preset YAML check + — in the acceptance-gate preset YAML + ↓ + (4) 🤖✅ Auto-catch + — the next time the same situation appears, it is caught automatically + ↓ + (5) 📜 Mandatory invocation rule + — written into CLAUDE.md / SKILL.md + +An arrow from (5) back to (1), with the caption "continuous loop". + +Side caption (between or beside the loop): "Codify each incident once — don't pre-write every rule." + +Right-side separate box: F14 concrete example (linear arrows): + 💥 F14 incident (2026-05-13) + ↓ + 📝 observed-failure-modes.md F14 + ↓ + 📋 multi-locale-mirror-sync.yml + new check + ↓ + 🤖✅ Retrospective test → 9+ real drifts caught + ↓ + 📜 CLAUDE.md mandatory rule + +Visual style: +- Main 5-step loop + linear F14 example on the right, separated by a divider line +- Color palette: incident red, document blue, preset green, auto-catch orange, CLAUDE.md purple +- Sans-serif typography +- 16:9 aspect ratio +- Clean tech blog aesthetic + +DO NOT: +- No "Source: ..." / "by agent-collab-skills" / "Anthropic Skills mechanism" attribution +- No vendor logos / company marks +- No gear / clockwork / clock visual metaphors +- No cartoon illustration beyond the listed emoji icons +- No 3D rendering, gradient shadows +``` + +--- + +## 用法建議 + +1. 開 ChatGPT（DALL-E 或任何有 image gen 能力的 model） +2. 把對應語言的 prompt **整段**複製貼上、或把 9 個 `.txt` 拖進去叫它 batch 處理 +3. 生成後存成 `resources/diagrams/concept-cluster..png` / `reading-decision-tree..png` / `failure-lifecycle..png` +4. 嵌入 Stage 7.5 對應 section（取代或補充既有 ASCII 圖） + +> 💡 **生成不滿意？** prompt 末尾「不要」list 加新項目（例如「不要藍色」「字太小、再放大」）regenerate。通常 2-3 次能拿到可用版本。 diff --git a/resources/diagrams/failure-lifecycle.en.png b/resources/diagrams/failure-lifecycle.en.png new file mode 100644 index 0000000..7bd5d1b Binary files /dev/null and b/resources/diagrams/failure-lifecycle.en.png differ diff --git a/resources/diagrams/failure-lifecycle.png b/resources/diagrams/failure-lifecycle.png new file mode 100644 index 0000000..1d76905 Binary files /dev/null and b/resources/diagrams/failure-lifecycle.png differ diff --git a/resources/diagrams/failure-lifecycle.zh-Hans.png b/resources/diagrams/failure-lifecycle.zh-Hans.png new file mode 100644 index 0000000..7872a36 Binary files /dev/null and b/resources/diagrams/failure-lifecycle.zh-Hans.png differ diff --git a/resources/diagrams/learning-map.en.png b/resources/diagrams/learning-map.en.png new file mode 100644 index 0000000..1a1358f Binary files /dev/null and b/resources/diagrams/learning-map.en.png differ diff --git a/resources/diagrams/learning-map.png b/resources/diagrams/learning-map.png new file mode 100644 index 0000000..c6ecf69 Binary files /dev/null and b/resources/diagrams/learning-map.png differ diff --git a/resources/diagrams/learning-map.zh-Hans.png b/resources/diagrams/learning-map.zh-Hans.png new file mode 100644 index 0000000..311429d Binary files /dev/null and b/resources/diagrams/learning-map.zh-Hans.png differ diff --git a/resources/diagrams/multi-agent-debate-flow.en.png b/resources/diagrams/multi-agent-debate-flow.en.png new file mode 100644 index 0000000..78e1de9 Binary files /dev/null and b/resources/diagrams/multi-agent-debate-flow.en.png differ diff --git a/resources/diagrams/multi-agent-debate-flow.png b/resources/diagrams/multi-agent-debate-flow.png new file mode 100644 index 0000000..353568e Binary files /dev/null and b/resources/diagrams/multi-agent-debate-flow.png differ diff --git a/resources/diagrams/multi-agent-debate-flow.zh-Hans.png b/resources/diagrams/multi-agent-debate-flow.zh-Hans.png new file mode 100644 index 0000000..dc546e7 Binary files /dev/null and b/resources/diagrams/multi-agent-debate-flow.zh-Hans.png differ diff --git a/resources/diagrams/multi-llm-delegation-composition.en.png b/resources/diagrams/multi-llm-delegation-composition.en.png new file mode 100644 index 0000000..fea8c43 Binary files /dev/null and b/resources/diagrams/multi-llm-delegation-composition.en.png differ diff --git a/resources/diagrams/multi-llm-delegation-composition.png b/resources/diagrams/multi-llm-delegation-composition.png new file mode 100644 index 0000000..28eaf99 Binary files /dev/null and b/resources/diagrams/multi-llm-delegation-composition.png differ diff --git a/resources/diagrams/power-user-multi-type-workflow.en.png b/resources/diagrams/power-user-multi-type-workflow.en.png new file mode 100644 index 0000000..34c36d4 Binary files /dev/null and b/resources/diagrams/power-user-multi-type-workflow.en.png differ diff --git a/resources/diagrams/power-user-multi-type-workflow.png b/resources/diagrams/power-user-multi-type-workflow.png new file mode 100644 index 0000000..acaaea4 Binary files /dev/null and b/resources/diagrams/power-user-multi-type-workflow.png differ diff --git a/resources/diagrams/power-user-multi-type-workflow.zh-Hans.png b/resources/diagrams/power-user-multi-type-workflow.zh-Hans.png new file mode 100644 index 0000000..71998f2 Binary files /dev/null and b/resources/diagrams/power-user-multi-type-workflow.zh-Hans.png differ diff --git a/resources/diagrams/principle-dependency.en.png b/resources/diagrams/principle-dependency.en.png new file mode 100644 index 0000000..16ff1e0 Binary files /dev/null and b/resources/diagrams/principle-dependency.en.png differ diff --git a/resources/diagrams/principle-dependency.png b/resources/diagrams/principle-dependency.png new file mode 100644 index 0000000..121d748 Binary files /dev/null and b/resources/diagrams/principle-dependency.png differ diff --git a/resources/diagrams/principle-dependency.zh-Hans.png b/resources/diagrams/principle-dependency.zh-Hans.png new file mode 100644 index 0000000..b63986a Binary files /dev/null and b/resources/diagrams/principle-dependency.zh-Hans.png differ diff --git a/resources/diagrams/prompt-context-harness-stack.en.png b/resources/diagrams/prompt-context-harness-stack.en.png new file mode 100644 index 0000000..e3d35fa Binary files /dev/null and b/resources/diagrams/prompt-context-harness-stack.en.png differ diff --git a/resources/diagrams/prompt-context-harness-stack.png b/resources/diagrams/prompt-context-harness-stack.png new file mode 100644 index 0000000..ff1ac32 Binary files /dev/null and b/resources/diagrams/prompt-context-harness-stack.png differ diff --git a/resources/diagrams/prompt-context-harness-stack.zh-Hans.png b/resources/diagrams/prompt-context-harness-stack.zh-Hans.png new file mode 100644 index 0000000..35a3cc1 Binary files /dev/null and b/resources/diagrams/prompt-context-harness-stack.zh-Hans.png differ diff --git a/resources/diagrams/rag-pipeline-overview.jpg b/resources/diagrams/rag-pipeline-overview.jpg new file mode 100644 index 0000000..6cd6c1f Binary files /dev/null and b/resources/diagrams/rag-pipeline-overview.jpg differ diff --git a/resources/diagrams/reading-decision-tree.en.png b/resources/diagrams/reading-decision-tree.en.png new file mode 100644 index 0000000..016cb7c Binary files /dev/null and b/resources/diagrams/reading-decision-tree.en.png differ diff --git a/resources/diagrams/reading-decision-tree.png b/resources/diagrams/reading-decision-tree.png new file mode 100644 index 0000000..dc5469c Binary files /dev/null and b/resources/diagrams/reading-decision-tree.png differ diff --git a/resources/diagrams/reading-decision-tree.zh-Hans.png b/resources/diagrams/reading-decision-tree.zh-Hans.png new file mode 100644 index 0000000..2a8ed94 Binary files /dev/null and b/resources/diagrams/reading-decision-tree.zh-Hans.png differ diff --git a/resources/diagrams/reflexion-persistent-memory-loop.en.png b/resources/diagrams/reflexion-persistent-memory-loop.en.png new file mode 100644 index 0000000..98c8833 Binary files /dev/null and b/resources/diagrams/reflexion-persistent-memory-loop.en.png differ diff --git a/resources/diagrams/reflexion-persistent-memory-loop.png b/resources/diagrams/reflexion-persistent-memory-loop.png new file mode 100644 index 0000000..6c6e07d Binary files /dev/null and b/resources/diagrams/reflexion-persistent-memory-loop.png differ diff --git a/resources/diagrams/reflexion-persistent-memory-loop.zh-Hans.png b/resources/diagrams/reflexion-persistent-memory-loop.zh-Hans.png new file mode 100644 index 0000000..6541203 Binary files /dev/null and b/resources/diagrams/reflexion-persistent-memory-loop.zh-Hans.png differ diff --git a/resources/diagrams/stack-4layer.en.png b/resources/diagrams/stack-4layer.en.png new file mode 100644 index 0000000..2f09963 Binary files /dev/null and b/resources/diagrams/stack-4layer.en.png differ diff --git a/resources/diagrams/stack-4layer.png b/resources/diagrams/stack-4layer.png new file mode 100644 index 0000000..436f986 Binary files /dev/null and b/resources/diagrams/stack-4layer.png differ diff --git a/resources/diagrams/stack-4layer.zh-Hans.png b/resources/diagrams/stack-4layer.zh-Hans.png new file mode 100644 index 0000000..3b30a5e Binary files /dev/null and b/resources/diagrams/stack-4layer.zh-Hans.png differ diff --git a/resources/diagrams/stage5-stack.en.png b/resources/diagrams/stage5-stack.en.png new file mode 100644 index 0000000..161ebb6 Binary files /dev/null and b/resources/diagrams/stage5-stack.en.png differ diff --git a/resources/diagrams/stage5-stack.png b/resources/diagrams/stage5-stack.png new file mode 100644 index 0000000..6676635 Binary files /dev/null and b/resources/diagrams/stage5-stack.png differ diff --git a/resources/diagrams/stage5-stack.zh-Hans.png b/resources/diagrams/stage5-stack.zh-Hans.png new file mode 100644 index 0000000..35c9b70 Binary files /dev/null and b/resources/diagrams/stage5-stack.zh-Hans.png differ diff --git a/resources/diagrams/subagent-4-stage-flow.en.png b/resources/diagrams/subagent-4-stage-flow.en.png new file mode 100644 index 0000000..71e74b1 Binary files /dev/null and b/resources/diagrams/subagent-4-stage-flow.en.png differ diff --git a/resources/diagrams/subagent-4-stage-flow.png b/resources/diagrams/subagent-4-stage-flow.png new file mode 100644 index 0000000..26c092a Binary files /dev/null and b/resources/diagrams/subagent-4-stage-flow.png differ diff --git a/resources/diagrams/subagent-4-stage-flow.zh-Hans.png b/resources/diagrams/subagent-4-stage-flow.zh-Hans.png new file mode 100644 index 0000000..f7953b9 Binary files /dev/null and b/resources/diagrams/subagent-4-stage-flow.zh-Hans.png differ diff --git a/resources/diagrams/subagent-composition-patterns.en.png b/resources/diagrams/subagent-composition-patterns.en.png new file mode 100644 index 0000000..eb63529 Binary files /dev/null and b/resources/diagrams/subagent-composition-patterns.en.png differ diff --git a/resources/diagrams/subagent-composition-patterns.png b/resources/diagrams/subagent-composition-patterns.png new file mode 100644 index 0000000..dcbb0ce Binary files /dev/null and b/resources/diagrams/subagent-composition-patterns.png differ diff --git a/resources/diagrams/subagent-composition-patterns.zh-Hans.png b/resources/diagrams/subagent-composition-patterns.zh-Hans.png new file mode 100644 index 0000000..2d008d5 Binary files /dev/null and b/resources/diagrams/subagent-composition-patterns.zh-Hans.png differ diff --git a/resources/diagrams/subagent-vs-skill.en.png b/resources/diagrams/subagent-vs-skill.en.png new file mode 100644 index 0000000..b5f786c Binary files /dev/null and b/resources/diagrams/subagent-vs-skill.en.png differ diff --git a/resources/diagrams/subagent-vs-skill.png b/resources/diagrams/subagent-vs-skill.png new file mode 100644 index 0000000..fae2d1f Binary files /dev/null and b/resources/diagrams/subagent-vs-skill.png differ diff --git a/resources/diagrams/subagent-vs-skill.zh-Hans.png b/resources/diagrams/subagent-vs-skill.zh-Hans.png new file mode 100644 index 0000000..718a878 Binary files /dev/null and b/resources/diagrams/subagent-vs-skill.zh-Hans.png differ diff --git a/resources/diagrams/teacher-ai-classroom-use-cases.jpg b/resources/diagrams/teacher-ai-classroom-use-cases.jpg new file mode 100644 index 0000000..d59b2db Binary files /dev/null and b/resources/diagrams/teacher-ai-classroom-use-cases.jpg differ diff --git a/resources/diagrams/teacher-ai-use-cases-overview.jpg b/resources/diagrams/teacher-ai-use-cases-overview.jpg new file mode 100644 index 0000000..8df1654 Binary files /dev/null and b/resources/diagrams/teacher-ai-use-cases-overview.jpg differ diff --git a/resources/glossary.en.md b/resources/glossary.en.md new file mode 100644 index 0000000..a081869 --- /dev/null +++ b/resources/glossary.en.md @@ -0,0 +1,449 @@ +# Glossary + +> [繁體中文](./glossary.md) | [简体中文](./glossary.zh-Hans.md) | **English** + +> The roadmap leans heavily on terms like LLM, RAG, MCP, agent. Look up unfamiliar ones here in 30 seconds, then go back to reading the stage. +> +> Each entry gives **the smallest usable definition** (30-80 words + which stage covers it in depth) — not Wikipedia. + +## 🌐 Unified terminology table (English ↔ Chinese, consistent across stages) + +This table is the project's **enforced naming convention** — every stage uses the same Chinese readability label for each English term. If you see drift inside a stage, please file an issue. + +| English term | Chinese readability label | Primary stage | +|---|---|---| +| Prompt Engineering | Prompt 設計 / Prompt 设计 | Stage 2 | +| Context Engineering | 上下文管理 | Stage 6 | +| Harness Engineering | Agent 執行系統設計 / Agent 执行系统设计 | Stage 7 | +| Tool Use | 工具使用 | Stage 3 | +| Function Calling | 函式 / 函数 / 工具呼叫 | Stage 3 | +| Structured Output | 結構化輸出 / 结构化输出 | Stage 3 | +| Agent Loop | Agent 執行迴圈 / Agent 执行循环 | Stage 3 | +| Framework | 框架 | Stage 4 | +| Orchestration | 協調與編排 / 协调与编排 | Stage 4 / 7 | +| Handoff | 任務交接 / 任务交接 | Stage 7 | +| Supervisor / Worker | 協調者 / 執行者 (协调者 / 执行者) | Stage 7 | +| Runtime | 執行層 / 执行层 | Stage 7 | +| Scaffolding | 支撐架構 / 支撑架构 | Stage 7 | +| Observability | 觀測與紀錄 / 观测与记录 | Stage 7 | +| Telemetry | 運行紀錄 / 运行记录 | Stage 7 | +| Eval | 效果評估 / 效果评估 | Stage 7 | +| Evaluation Harness | 評估框架 / 评估框架 | Stage 7 | +| Production | 可穩定使用 / 上線化 (可稳定使用 / 上线化) | Stage 7 | +| Production-grade | 可長期穩定使用的 / 可长期稳定使用的 | Stage 7 | +| Deployment | 部署 | Stage 7 | +| Cost Tracking | 成本追蹤 / 成本追踪 | Stage 7 | +| Latency | 延遲 / 等待時間 (延迟 / 等待时间) | Stage 7 | +| Vector DB | 向量資料庫 / 向量数据库 | Stage 6 | +| Retrieval | 檢索 / 检索 | Stage 6 | +| Reranking | 重排序 | Stage 6 | +| Long Context | 長上下文 / 长上下文 | Stage 6 | +| Fine-tuning | 模型微調 / 模型微调 | Stage 6 | +| Agent Interfaces | Agent 操作介面 / Agent 操作界面 | Stage 8 | +| Code Sandbox | 隔離程式執行環境 / 隔离程序执行环境 | Stage 8 | +| Cold Start | 啟動延遲 / 启动延迟 | Stage 8 | +| Reward Hacking | 鑽評分漏洞 / 钻评分漏洞 | Stage 7 / 8 | + +→ For full definitions, see the sections below. + +--- + +## 1. Basic concepts + +### LLM (Large Language Model) + +GPT, Claude, Gemini — models that take text in and produce text out. Fundamentally a pure function: input prompt → output text. **They don't browse the web, they don't remember past conversations** — those need to be wired up externally. + +📍 Detail: [Stage 1](../stages/01-llm-basics.en.md) + +### Token + +LLMs see **tokens** (sub-word units), not characters. Roughly 1 English word ≈ 1.3 tokens, 1 Chinese character ≈ 1.5–2 tokens. LLM pricing and context windows are measured in tokens. "1M-token context" ≈ 750k English words. + +📍 Detail: [Stage 1](../stages/01-llm-basics.en.md) + +### Context Window + +The maximum tokens an LLM can "see" in one call. **2026 frontier**: Claude Sonnet 5 / Opus 4.8 1M, GPT-5.6 1.05M, Gemini 3.5 Flash 1M (Pro series up to 2M), xAI Grok 4.3 1M, Mistral Medium 3.5 256k. **Bigger isn't always better** — beyond a length the LLM gets "Lost in the Middle". + +### Prompt + +The text you feed an LLM. **Prompt engineering** = designing that text to get good answers. Basic structure: system prompt (role/rules) + user prompt (the actual ask). + +📍 Detail: [Stage 2](../stages/02-prompt-engineering.en.md) + +### Zero-shot / One-shot / Few-shot + +Put "a few worked examples" in the prompt for the LLM to copy — the only difference between the three terms is **how many examples you give**: + +- **Zero-shot** (0 examples): just ask, no examples at all. +- **One-shot** (1 example): give **1** input → output example first, then ask. +- **Few-shot** (a handful): give **2–5** input → output examples first. **Few-shot usually improves accuracy a lot**, especially for strict formatting. + +### Chain-of-Thought (CoT) + +Make the LLM "think before answering" — have it output the reasoning process before the conclusion. **Two common forms**: + +- **Few-shot CoT** ([Wei et al. 2022](https://arxiv.org/abs/2201.11903)): put a few examples with reasoning steps into the prompt, and the LLM imitates that style of thinking +- **Zero-shot CoT** ([Kojima et al. 2022](https://arxiv.org/abs/2205.11916)): add "Let's think step by step" at the end of the prompt to trigger a reasoning trace + +**Accuracy usually improves**, at the cost of more tokens. Few-shot usually beats zero-shot. + +--- + +## 2. Agents / Tool Use + +### Agent + +A system centered on an LLM that can **perceive state → make decisions → take actions → observe results** in a **loop**, repeating until the goal is completed. **Three core elements**: + +- **LLM** (reasoning / planning / deciding) +- **Actions** (ways to do things — not limited to function calls. This can include writing and running code (CodeAct), operating a browser (computer use), retrieving from a KB (RAG retrieval), calling an MCP server, or pure planning / task decomposition) +- **Loop** (the heartbeat — the fundamental difference between an agent and plain LLM Q&A) + +The difference is this: plain LLM = Q&A; agent = the three elements + a continuing loop until the goal is reached or the budget runs out. **ReAct is one agent pattern, not the definition of an agent** — CodeAct, computer-use, and planning agents are all agents. + +📍 Detail: [Stage 3](../stages/03-tool-use-and-hello-agent.en.md) + +### Tool Use / Function Calling + +Lets the LLM call functions you defined (DB lookup, math, browser, …). Instead of plain text, the LLM returns `{"function": "search", "args": {…}}`. Your code executes it and feeds the result back to the LLM. + +**The concept is the same; the API schema differs**: +- **Anthropic "Tool Use"**: uses `input_schema` (JSON Schema directly) +- **OpenAI / Ollama "Function Calling"**: wraps it in an outer `{"type": "function", "function": {...}}` +- The token representation the LLM sees differs internally, so when writing a cross-vendor SDK you need to map them correctly + +📍 Detail: [Stage 3](../stages/03-tool-use-and-hello-agent.en.md) +📍 How to write good schemas: [Function Schema Design cheatsheet](schema-design-cheatsheet.en.md) + +### ReAct (Reasoning + Acting) + +The classic agent pattern: **Thought → Action (call tool) → Observation (see result) → Thought ...** loop until done. Most agent frameworks implement this internally. + +📍 Detail: [Stage 3](../stages/03-tool-use-and-hello-agent.en.md) + +### Structured Output + +Make the LLM output **JSON or another fixed schema** instead of free text. All major LLM APIs have `response_format` or similar. Agent frameworks rely on it for LLM ↔ code communication. + +### Agent Loop + +The "LLM → tool → result → LLM" repeated cycle. Termination: LLM says "done" / step budget exhausted / cost cap hit. + +### Self-Refine (Basic reflection / no memory) + +The agent evaluates the previous round's output and changes the next round's behavior — an "Actor answers → Critic finds issues → Actor reads feedback and answers again" single-session loop. **It does not need a persistent memory layer**; it is purely a reasoning-loop mechanism, a sibling pattern to ReAct. Production agents (Cursor / Cline / Claude Code) run variants of this every day. + +Representative paper: [Self-Refine (Madaan 2023)](https://arxiv.org/abs/2303.17651). **For the full Reflexion version** (with episodic memory), see 3 Memory / Retrieval / RAG. + +📍 Detail + routing: [Stage 3 Reflection](../stages/03-tool-use-and-hello-agent.en.md#-reflection-reflexion--self-refine--concept--routing) + +--- + +## 3. Memory / Retrieval / RAG + +### Memory — Two Orthogonal Axes + +"Memory" often gets lumped together, but there are actually **2 orthogonal classification axes**: + +- **Time axis**: short-term (current conversation) vs long-term (persistent across sessions) +- **Content axis** (CoALA framework): **Working** (scratch space) / **Episodic** (past experiences) / **Semantic** (factual knowledge) / **Procedural** (how to do things) + +→ The two axes do not conflict: long-term memory can contain **at the same time** episodic memory (what the user said last time), semantic memory (facts from the company's knowledge base), and procedural memory (tool sequences that worked before). + +📍 Detail: [Stage 6 What is Memory + How to Design It](../stages/06-memory-rag.en.md#-what-is-memory--how-to-design-it) + [Stage 6 CoALA Framework](../stages/06-memory-rag.en.md#advanced-coala-framework--a-4-layer-taxonomy-for-agent-memory) + +### RAG (Retrieval-Augmented Generation) + +Two-stage architectural pattern: + +1. **Ingest** (one-time / periodic): document → chunk → embed → store in a vector store (build a retrievable KB) +2. **Query** (every user question): embed the question → semantic search (or hybrid + BM25) → top-K chunks → put them into the prompt → LLM answers + +**Solves the problem that the LLM does not know your private / changing / stale data**. Retrieval is **not limited to dense embeddings** — the production default is hybrid (dense + BM25) + reranker. + +📍 Detail: [Stage 6](../stages/06-memory-rag.en.md) +📍 Paper: [Lewis et al. 2020](https://arxiv.org/abs/2005.11401) + +### Reflexion (Full reflection / with episodic memory) + +Unlike Self-Refine (2 Agents), Reflexion **requires a persistent episodic memory store** — after each trial, the agent **writes a reflection summary into memory**, then retrieves it into the prompt at the start of the next trial. **Accumulating lessons across trials** is the essence of Reflexion (not a single-session loop). + +It is placed in 3 instead of 2 because it is **fundamentally a memory pattern** — the episodic memory store is core, not optional. + +Representative paper: [Reflexion (Shinn 2023)](https://arxiv.org/abs/2303.11366). + +📍 Detail: [Stage 6 Advanced: Full Reflexion with Persistent Memory](../stages/06-memory-rag.en.md#-advanced-full-reflexion-with-persistent-memory--track-b-elective) + +### Embedding + +Turn text / images into N-dimensional **vectors** so that things with similar meanings are close together. This roadmap defaults to **dense embeddings** (dense vectors produced by sentence-transformers / OpenAI ada-002, etc.); there are also **sparse embeddings** (BM25 / SPLADE, etc., based on lexical token matching) — production RAG often uses both together for hybrid search. + +📍 Detail: [Stage 6](../stages/06-memory-rag.en.md) + +### Vector DB + +The storage layer for storing + efficiently querying embeddings. **The main query type = approximate nearest-neighbor (ANN)** — the whole point of a vector DB is that ANN is hundreds of times faster than brute-force cosine scanning. Examples: Pinecone / Chroma / Qdrant / Weaviate / pgvector. + +📍 Detail: [Stage 6](../stages/06-memory-rag.en.md) + +### Semantic Search + +Use embeddings to compare "meaning similarity" rather than "exact string match". "How do I charge an EV" can retrieve "electric car battery tutorial". Traditional keyword search (BM25, etc.) can't do this. + +### Chunking + +Splitting long documents into embedding-friendly small pieces (typically 200–1000 tokens). **Chunk strategy directly affects RAG quality** — too small loses context, too long blurs relevance. Common: fixed-size, by paragraph, by structure (heading-based). + +### Hybrid Search + +Run semantic search and keyword search together, merge and rerank. Usually beats either alone. Production-grade RAG default. + +### Reranking + +After first-pass retrieval pulls top-50, use a more expensive but more accurate model (cross-encoder) to rerank to top-5 for the LLM. Cohere Rerank, bge-reranker, etc. + +### Contextual Retrieval + +Anthropic 2024 method — embed each chunk together with a summary of the document it came from, so "this chunk taken alone makes no sense" doesn't break retrieval. + +📍 Detail: [Stage 6](../stages/06-memory-rag.en.md) + +### Fine-tuning + +Re-train the model on your own data, baking knowledge or behavior into the weights (unlike RAG — RAG injects data into the context at inference time and never changes the weights). Good for making the model reliably learn a **format / style / domain vocabulary**; **not** for stuffing in "the latest facts" (that is RAG's job — facts fine-tuned in go stale and are hard to update). In most agent scenarios, **try prompt + RAG first**, and only consider fine-tuning if that genuinely is not enough. + +📍 Detail: [Stage 6](../stages/06-memory-rag.en.md) + +--- + +## 4. Multi-Agent + +### Multi-Agent + +Multiple agents collaborating on one task. Common patterns: + +- **Supervisor + Worker**: one agent plans/dispatches, others execute. +- **Swarm**: peer agents, no fixed supervisor. +- **Debate**: agents argue different positions, then form consensus. + +📍 Detail: [Stage 7](../stages/07-multi-agent-production.en.md) + +### Handoff + +One agent transfers a task to another. Adds "how to pass context" and "who handles failure" beyond a plain function call. + +### A2A (Agent-to-Agent) Protocol + +An agent ↔ agent communication protocol started by Google and now governed by the Linux Foundation. Sibling to MCP, but for agent-to-agent rather than agent-to-tool. Reached **v1.0** in 2026 (150+ organizations, plus identity verification so one agent can confirm another really is who it claims to be). + +--- + +## 5. Claude Code Ecosystem + +### MCP (Model Context Protocol) + +Anthropic's open protocol, introduced in 2024, that lets any LLM host (Claude Code, Cursor, your own agent) connect to external tool servers through one interface. Think "**USB for LLMs**". + +**Technically it standardizes 3 primitives**: +- **Tools**: functions an LLM can call (read DB / search web / send email…) +- **Resources**: data an LLM can read (file contents, API responses, DB rows…) +- **Prompts**: reusable prompt templates (triggered inside the host with `/`) + +**Architecture**: server / client pattern — the tool server runs locally or remotely, and the LLM host connects as the client. The server exposes those primitives over one of three transports: stdio / SSE / HTTP. + +📍 Detail: [Stage 5.2](../stages/05-claude-code-ecosystem.en.md#52--mcp-model-context-protocol--foundation) + +### Skills / SKILL.md + +Claude Code's "behavior bundles". A Skill = a folder containing `SKILL.md` (describing "what to do in what situations, and which tools can be called") + optional reference files / scripts. + +**Trigger mechanism** (many people do not know this, but it matters): before Claude Code handles each message, it scans the **frontmatter `description` field** of every available skill — if it matches the current situation, the corresponding SKILL.md is auto-loaded. **So the quality of the description directly determines whether the skill gets triggered.** In practice, starting with "Use when ..." works best. + +📍 Detail: [Stage 5.3](../stages/05-claude-code-ecosystem.en.md#53--skills-claude-codes-behavior-layer--the-most-critical-layer-of-the-claude-code-ecosystem) + +### Plugin / Marketplace + +Package multiple Skills + slash commands + hooks + MCP configs into one shippable unit. A **Marketplace** is a catalog of plugins; users `claude plugin install` to grab community-built ones. + +📍 Detail: [Stage 5.4](../stages/05-claude-code-ecosystem.en.md#54--plugins--marketplaces) + +### Slash Command + +Commands inside Claude Code starting with `/` (`/help`, `/compact`, `/plan`, etc.). Custom-definable — drop a prompt into `.claude/commands/.md` and it becomes `/name`. + +### CLAUDE.md + +A markdown file at project root that Claude Code reads on every launch. Project-level rules / conventions / context (language, code style, files to avoid, etc.). + +### Hooks + +Scripts that run before or after specific Claude Code events. **The official system supports 7 event types**: + +| Hook | Trigger time | Typical use | +|---|---|---| +| `PreToolUse` | **Before** a tool call | Block dangerous operations (`rm -rf`, destructive ops), rewrite parameters | +| `PostToolUse` | **After** a tool call | Logging, auto-format files that were just written | +| `UserPromptSubmit` | When the user submits a message | Add context (git status / current time) | +| `Notification` | When Claude Code emits a notification | Desktop toast / Slack ping | +| `Stop` | When the session ends | Auto-commit / cleanup | +| `PreCompact` | Before auto-compact | Promote important decisions into memory | +| `PostCompact` | After compact | Check what context got compressed | + +Configuration: add a `"hooks"` block in `.claude/settings.json` and point it at your script path. + +### Deep Agent + +An agent design that comes "fully equipped" — beyond just calling tools, it has built-in planning (a todo list), long-term memory (a filesystem), division of labor via subagents, and loadable skills. Contrast: a plain agent is just an LLM plus a few tools. Reference implementation: LangChain's [deepagents](https://github.com/langchain-ai/deepagents). + +--- + +### Subagent + +A spawned agent from the main Claude Code session, with its own context window, dedicated to a specific task. E.g. "spin up a code-reviewer subagent for this diff." + +How to set up: put frontmatter + system prompt + tool whitelist in `.claude/agents/.md`. The main session invokes it with the Task tool (automatic parallel / sequential). **Compared with framework-based multi-agent**: a subagent does not require LangGraph / CrewAI or similar frameworks; markdown is enough, but it is tied to the Claude Code runtime. Full guide: [Stage 5.5](../stages/05-claude-code-ecosystem.en.md#55--subagents-claude-codes-native-multi-agent-mechanism--2025-new-feature); **15 copy-paste dispatch recipes** → [`subagent-cookbook.en.md`](./subagent-cookbook.en.md). + +--- + +## 6. Production / Eval / Cost + +### Eval (Evaluation Framework) + +Run a test set against your agent and quantify accuracy / latency / cost. **A production agent without eval has no tests.** Common: promptfoo, LangSmith, langfuse evals. + +📍 Detail: [Stage 7](../stages/07-multi-agent-production.en.md) + +### Observability + +Capture every internal step (which LLM call, which tool, what result). Lets you replay when bugs hit. Common: langfuse, Helicone, weave. + +📍 Detail: [Stage 7](../stages/07-multi-agent-production.en.md) + +### Prompt Caching + +LLM caches the prefix of a prompt; on repeat, only the new suffix is billed at full price (Anthropic 90% off cached, OpenAI 50% off). Long-context repeated queries save a lot. + +### Streaming + +LLM returns tokens as they're generated (one at a time) instead of waiting for the full response. Better UX (looks like typing); technically uses SSE or chunked transfer. **Default for production interactive apps**. Trade-offs: client must handle partial responses; ReAct tool-call parsing waits for stream end. + +### Batch API + +Bundle a large number of LLM requests for delayed (≤24h) processing. **Typically 50% off (Anthropic, OpenAI)**. Good for non-interactive: batch summarization, batch classification, eval suites, ETL pipelines. **Don't use for interactive chat** — latency unacceptable. + +### Token Cost / Inference Cost + +Per LLM call: input tokens × input price + output tokens × output price. Costs of an agent's ReAct loop add up fast — a single grep over a large codebase can run 100k tokens. + +### Guardrails + +Rule layer that prevents the LLM from doing bad things — block prompt injection, PII leakage, harmful output, etc. NeMo Guardrails, Guardrails AI, etc. + +### Prompt Injection + +Hiding malicious instructions inside content the LLM will read (web pages, documents, tool results) so it ignores its real task and does what the attacker wants. Root cause: the LLM can't tell "instructions" apart from "instructions smuggled inside data". Defenses: least privilege, isolating untrusted content, human review of high-risk actions. Related: lethal trifecta, Guardrails. + +### Lethal Trifecta + +Simon Willison's framing: an agent becomes exploitable when it has all three of (1) access to private data, (2) exposure to untrusted content, (3) the ability to communicate externally — at which point prompt injection can make it steal and exfiltrate data. The defense is to break at least one leg (commonly: cut external comms, or isolate untrusted input). + +--- + +## 7. Buzzwords / Loose Terms + +### CLI Agent + +Agents that run in a terminal (Claude Code, Codex, Aider, Gemini CLI, etc.). Versus IDE-bound (Cursor, Continue) or web-based (ChatGPT, Claude.ai). + +📍 Detail: [Track A A1](../tracks/cli/A1-cli-intro.en.md), [`resources/cli-agents-guide.en.md`](cli-agents-guide.en.md) + +### BYO API Key (Bring Your Own) + +Tool that supports user-provided API keys instead of bundled subscriptions. Aider / OpenCode / goose are BYO; Claude Code / Codex default to subscriptions. + +### Local LLM / On-Device + +Models running on your own hardware (Ollama, llama.cpp, MLX, LocalAI, etc.). Data stays local — privacy-friendly but capabilities lag frontier models. + +📍 Detail: [Stage 1](../stages/01-llm-basics.en.md) + +### Quantization + +Compress model weights from fp16 down to int8 / int4 to save memory and increase speed at small accuracy cost. Local LLM users see this constantly (Q4_K_M, Q8_0, etc.). + +### Hallucination + +The LLM "confidently asserts something false" — invents APIs, fabricates numbers and presents them as fact. Every production agent needs defenses (RAG / structured output / eval / guardrails). + +### Frontier Model + +The current top tier (**2026-07**: OpenAI **GPT-5.6** (Sol / Terra / Luna — three tiers, 1.05M context, available in ChatGPT, Codex, and the API); **2026-06 (late)**: Claude Sonnet 5 (best speed-intelligence balance, close to Opus 4.8 but cheaper), Google Gemini 3.5 Flash, xAI Grok 4.3, Mistral Medium 3.5 (open weights, preview); **2026-06 (early)**: Claude Fable 5 (Mythos-class, above Opus) shipped, was suspended 2026-06-12 by a US export-control directive, but **the controls were lifted 2026-06-30 and [Fable 5 was redeployed globally 2026-07-01](https://www.anthropic.com/news/redeploying-fable-5)** (Mythos 5 restored only for approved US orgs); **2026-05**: GPT-5.5, Claude Opus 4.8 (Opus-class flagship), Gemini 3.1 Pro, DeepSeek-V4-Pro, etc.). Use frontier for hard reasoning; use cheap small models for simple classification / translation to save cost. + +### Context Engineering + +The discipline of engineering **what information goes into the context window on each LLM call** — dynamically assembling RAG retrieval results, memory, tool definitions, and conversation history into the context the model can see. Karpathy 2025: the delicate art of putting **just the right information for the next step** into the window. The key question is *what goes in the window*, not "how many calls are involved." **The next layer above prompt engineering** — prompt engineering shapes **strings**; context engineering shapes **information**. + +📍 Detail: [Stage 2 closing](../stages/02-prompt-engineering.en.md) / [Stage 6](../stages/06-memory-rag.en.md) / [Stage 7](../stages/07-multi-agent-production.en.md) +📍 Further: [`Meirtz/Awesome-Context-Engineering`](https://github.com/Meirtz/Awesome-Context-Engineering) + +### Harness Engineering + +The discipline of engineering the **execution and control layer around the model** — everything that is not model weights and not just the prompt string itself: agent loop / tool registry / context manager / permissions / safety layer / memory layer / eval / observability / retry / circuit breaker, etc. Simon Willison 2025: **coding agent = LLM + harness**. Addy Osmani: harness = all the code that is not the model itself. [OpenAI also used the term "Harness Engineering" in February 2026](https://openai.com/index/harness-engineering). Claude Code, Cursor, OpenCode, etc. are harnesses. **A framework wraps an LLM into an agent; a harness wraps an agent into a product that can actually go live.** + +Contrast: +- **Framework** (Stage 4) defines the **API**: what the interface you call looks like +- **Harness** (this term) defines the **runtime**: how it runs, how it recovers, how it is observed + +📍 Discipline-level concept (**8 core components** / prompt→context→harness three-layer engineering split / framework vs harness): [Stage 7 Harness Engineering](../stages/07-multi-agent-production.en.md) +📍 Reference implementation case study (reading Claude Code source): [Stage 5 5.7](../stages/05-claude-code-ecosystem.en.md) +📍 Further: [`anthropics/claude-agent-sdk-python`](https://github.com/anthropics/claude-agent-sdk-python), [`ai-boost/awesome-harness-engineering`](https://github.com/ai-boost/awesome-harness-engineering), [`ZhangHanDong/harness-engineering-from-cc-to-ai-coding`](https://github.com/ZhangHanDong/harness-engineering-from-cc-to-ai-coding) + +### Loop Engineering + +The fourth discipline after prompt → context → harness engineering: designing and tuning an agent's iteration loop itself (goal, tools, context management, termination logic, error handling) so long-running, multi-step, cross-session execution stays reliable and on-target. Related: harness, Dynamic Workflows, ReAct. + +--- + +## 8. Agent Interfaces + +### Computer Use (screen-level agent) + +An agent operates real desktop apps via **screenshot → vision → coordinates → simulated mouse/keyboard** — no API needed, the agent uses the screen like a human. Representative: Anthropic Claude Computer Use (Opus 4.8 / Sonnet 5), OpenAI Codex desktop, Google Gemini in Chrome. **Anthropic public beta opened Oct 2024; OSWorld benchmark reached 76.26% (superhuman) by May 2026**. + +📍 Full coverage + 4-vendor comparison: [Stage 8 Computer Use](../stages/08-agent-interfaces.en.md) + +### Browser Use (web-level agent) + +An agent operates web pages, primarily via **DOM-aware navigation** (direct CSS selector queries) with vision fallback. Closed-source: Atlas / Comet / Dia / Gemini in Chrome. OSS leader: [browser-use](https://github.com/browser-use/browser-use) (★ 95k+). + +📍 Full coverage + 5-vendor comparison + OSS frameworks: [Stage 8 Browser Use](../stages/08-agent-interfaces.en.md) + +### Sandbox (code execution isolation) + +Runs agent-written code in an isolated environment instead of the host — avoids `rm -rf /`, internet data exfiltration, credential theft. Representatives: E2B (Firecracker microVM), Daytona (container), Modal (GPU sandbox), Vercel, Cloudflare. **OpenAI Agents SDK natively supports these as of April 2026**. + +📍 Full 9-row terminology glossary + 7-vendor comparison: [Stage 8 Code Sandbox](../stages/08-agent-interfaces.en.md) + +### microVM (micro Virtual Machine) + +A slimmed-down VM with minimal footprint, < 100ms startup, yet still has an **independent kernel** — sits between Docker containers (fast + weak isolation) and full VMs (slow + strong isolation). **Most agent sandboxes choose microVM**. Implementation example: Firecracker (AWS, used by E2B). + +📍 Full comparison: [Stage 8 terminology glossary](../stages/08-agent-interfaces.en.md) + +### Firecracker + +AWS's open-source microVM, written in Rust, **the underlying technology of AWS Lambda** and E2B sandbox isolation. Provides strong isolation + fast startup. + +### gVisor + +Google's "user-space kernel" — intercepts syscalls and emulates them itself, no hypervisor required. Sits between containers and VMs. + +--- + +## Term not here? + +- Read the actual stage content: [Stage 5.2 MCP](../stages/05-claude-code-ecosystem.en.md#52--mcp-model-context-protocol--foundation) / [5.3 Skills](../stages/05-claude-code-ecosystem.en.md#53--skills-claude-codes-behavior-layer--the-most-critical-layer-of-the-claude-code-ecosystem) / [5.4 Plugins](../stages/05-claude-code-ecosystem.en.md#54--plugins--marketplaces) +- Required reading lists in [Stage 1](../stages/01-llm-basics.en.md) / [Stage 6](../stages/06-memory-rag.en.md) / [Stage 7](../stages/07-multi-agent-production.en.md) / [Stage 8](../stages/08-agent-interfaces.en.md) +- Missing? Open an issue or PR a new entry. diff --git a/resources/glossary.md b/resources/glossary.md new file mode 100644 index 0000000..5daf974 --- /dev/null +++ b/resources/glossary.md @@ -0,0 +1,453 @@ +# 用語小辭典（Glossary） + +> **繁體中文** | [简体中文](./glossary.zh-Hans.md) | [English](./glossary.en.md) + +> 本路線圖會大量出現「LLM」、「RAG」、「MCP」、「agent」這類詞。讀到不懂的詞先在這裡查 30 秒，再回去讀 stage 內容。 +> +> 每個詞**只給最小可用的解釋**（30-80 字 + 在哪一個 stage 講細的）——不是維基百科。 + +## 🌐 統一詞彙對照表（中英對照、跨 stage 一致） + +本表是專案內**強制統一的命名約定**——所有 stage 用同一個中文理解名。如果你在 stage 內看到不一致，請報 issue。 + +| 英文術語 | 中文理解名 | 主要 stage | +|---|---|---| +| Prompt Engineering | Prompt 設計 | Stage 2 | +| Context Engineering | 上下文管理 | Stage 6 | +| Harness Engineering | Agent 執行系統設計 | Stage 7 | +| Tool Use | 工具使用 | Stage 3 | +| Function Calling | 函式 / 工具呼叫 | Stage 3 | +| Structured Output | 結構化輸出 | Stage 3 | +| Agent Loop | Agent 執行迴圈 | Stage 3 | +| Framework | 框架 | Stage 4 | +| Orchestration | 協調與編排 | Stage 4 / 7 | +| Handoff | 任務交接 | Stage 7 | +| Supervisor / Worker | 協調者 / 執行者 | Stage 7 | +| Runtime | 執行層 | Stage 7 | +| Scaffolding | 支撐架構 | Stage 7 | +| Observability | 觀測與紀錄 | Stage 7 | +| Telemetry | 運行紀錄 | Stage 7 | +| Eval | 效果評估 | Stage 7 | +| Evaluation Harness | 評估框架 | Stage 7 | +| Production | 可穩定使用 / 上線化 | Stage 7 | +| Production-grade | 可長期穩定使用的 | Stage 7 | +| Deployment | 部署 | Stage 7 | +| Cost Tracking | 成本追蹤 | Stage 7 | +| Latency | 延遲 / 等待時間 | Stage 7 | +| Vector DB | 向量資料庫 | Stage 6 | +| Retrieval | 檢索 | Stage 6 | +| Reranking | 重排序 | Stage 6 | +| Long Context | 長上下文 | Stage 6 | +| Fine-tuning | 模型微調 | Stage 6 | +| Agent Interfaces | Agent 操作介面 | Stage 8 | +| Code Sandbox | 隔離程式執行環境 | Stage 8 | +| Cold Start | 啟動延遲 | Stage 8 | +| Reward Hacking | 鑽評分漏洞 | Stage 7 / 8 | + +→ 詳細定義請看下面各區塊。 + +--- + +## 1. 基本概念 + +### LLM（Large Language Model，大語言模型） + +GPT、Claude、Gemini 這類「給文字、回文字」的模型。本身是純函式：input prompt → output text。它**不會自己上網、不會記住上次對話**——這些都要外接系統來做。 + +📍 詳細：[Stage 1](../stages/01-llm-basics.md) + +### Token + +LLM 看到的不是「字」，是 **token**（次字單位）。中文 1 個字 ≈ 1.5-2 token，英文 1 個 word ≈ 1.3 token。LLM 計費跟 context window 都以 token 計。「100 萬 token context」≈ 75 萬中文字。 + +📍 詳細：[Stage 1](../stages/01-llm-basics.md) + +### Context Window（上下文視窗） + +LLM 一次能「看」多少 token。**2026 frontier**：Claude Sonnet 5 / Opus 4.8 1M、GPT-5.6 1.05M、Gemini 3.5 Flash 1M（Pro 系列上看 2M）、xAI Grok 4.3 1M、Mistral Medium 3.5 256k。**不是越大越好**——超過某個長度後 LLM 會「在中間遺漏」（Lost in the Middle）。 + +### Prompt（提示詞） + +你給 LLM 的輸入文字。**Prompt engineering** 就是設計這段輸入讓 LLM 給好答案。System prompt（角色設定）+ user prompt（這次的問題）是基本結構。 + +📍 詳細：[Stage 2](../stages/02-prompt-engineering.md) + +### Zero-shot / One-shot / Few-shot + +在 prompt 裡放「幾個示範例子」讓 LLM 照著做——這三個詞的差別只在**你給幾個範例**： + +- **Zero-shot**（0 個範例）：直接問、不給任何範例。 +- **One-shot**（1 個範例）：先給 **1 個** input → output 範例再問。 +- **Few-shot**（少數幾個）：給 **2-5 個** input → output 範例後再問。**Few-shot 通常顯著提升準確度**，特別是格式要求嚴的任務。 + +### Chain-of-Thought（CoT，思維鏈） + +要 LLM「先想再答」——讓它輸出推理過程再給結論。**兩種形式**： + +- **Few-shot CoT**（原始 paper、[Wei et al. 2022](https://arxiv.org/abs/2201.11903)）：在 prompt 裡放幾個含推理步驟的範例、LLM 模仿著想 +- **Zero-shot CoT**（[Kojima et al. 2022](https://arxiv.org/abs/2205.11916)）：prompt 結尾加「Let's think step by step」就觸發 reasoning trace + +**準確度通常會提升**、代價是 token 數變多。Few-shot 通常比 zero-shot 準。 + +--- + +## 2. Agent / 工具使用 + +### Agent（代理人） + +以 LLM 為核心、能在**迴圈**中**感知狀態 → 做決策 → 採取行動 → 觀察結果**、重複到完成目標的系統。**核心三要素**： + +- **LLM**（推理 / 規劃 / decide） +- **Actions**（做事的手段——不限於 function call。可以是寫程式碼執行（CodeAct）、操作瀏覽器（computer use）、查 KB（RAG retrieval）、call MCP server、純規劃分解任務等） +- **Loop**（心跳——agent 跟純 LLM Q&A 的根本差別） + +差別在於：純 LLM = Q&A、agent = 三要素 + 持續迴圈直到目標達成或耗盡 budget。**ReAct 是其中一種 agent pattern、不是 agent 的定義**——CodeAct、computer-use、planning agent 都是 agent。 + +📍 詳細：[Stage 3](../stages/03-tool-use-and-hello-agent.md) + +### Tool Use / Function Calling + +讓 LLM 呼叫你定義好的 function（查 DB、算數學、開瀏覽器…）。LLM 回的不是文字而是 `{"function": "search", "args": {...}}`、你的程式去執行、把結果再丟回 LLM。 + +**兩個詞概念相同、API schema 不一樣**： +- **Anthropic「Tool Use」**：schema 用 `input_schema`（JSON Schema 直接放） +- **OpenAI / Ollama「Function Calling」**：包一層 `{"type": "function", "function": {...}}` 外層 +- LLM 內部接收的 token 表達不同、寫 SDK 跨家時要記得對應好 + +📍 詳細：[Stage 3](../stages/03-tool-use-and-hello-agent.md) +📍 schema 怎麼寫好：[Function Schema 設計 cheatsheet](schema-design-cheatsheet.md) + +### ReAct（Reasoning + Acting） + +最經典的 agent pattern：**Thought（想）→ Action（叫工具）→ Observation（看結果）→ Thought ...** 一直 loop 到答得出來。多數 agent framework 內部都實作這個。 + +📍 詳細：[Stage 3](../stages/03-tool-use-and-hello-agent.md) + +### Structured Output（結構化輸出） + +要 LLM 輸出 **JSON / 其他固定 schema**，而不是自由文字。各家 LLM API 都有 `response_format` 或類似旗標支援。Agent 框架幾乎都靠這個跟 LLM 溝通。 + +### Agent Loop + +「LLM → tool → 結果 → LLM」這個重複的循環。Loop 結束條件可能是：LLM 說「I'm done」、跑超過 N 步、超出 budget。 + +### Self-Refine（基本版反思 / 無記憶） + +agent 自我評估上一回合輸出、改下一回合的 pattern——「Actor 出答案 → Critic 找問題 → Actor 看 feedback 再答」的 single-session loop。**不需要持久記憶層**，純粹是 reasoning loop 機制、是 ReAct 的 sibling pattern。production agent（Cursor / Cline / Claude Code）每天在跑這個變種。 + +代表 paper：[Self-Refine (Madaan 2023)](https://arxiv.org/abs/2303.17651)。**完整版 Reflexion**（含 episodic memory）見 3 Memory / Retrieval / RAG（不同層的東西）。 + +📍 詳細 + 路由：[Stage 3 反思](../stages/03-tool-use-and-hello-agent.md#-反思reflexion--self-refine-概念--路由) + +--- + +## 3. Memory / Retrieval / RAG + +### Memory（記憶）— 兩種正交分類軸 + +「memory」常被混為一談、其實有 **2 種正交分類軸**： + +- **時效軸**：short-term（當前對話） vs long-term（跨 session 持久） +- **內容軸**（CoALA framework）：**Working**（暫存）/ **Episodic**（過去經歷）/ **Semantic**（事實知識）/ **Procedural**（怎麼做） + +→ 兩軸不互斥：long-term memory 裡可以**同時**有 episodic（user 上次說了什麼）+ semantic（公司知識庫事實）+ procedural（用過的 tool sequence）。 + +📍 詳細：[Stage 6 Memory 是什麼 + 怎麼設計](../stages/06-memory-rag.md#-memory-是什麼--怎麼設計) + [Stage 6 CoALA framework](../stages/06-memory-rag.md#進階coala-framework--agent-memory-的-4-層-taxonomy) + +### RAG（Retrieval-Augmented Generation） + +兩階段架構模式： + +1. **Ingest**（一次性 / 定期）：document → chunk → embed → 存進 vector store（建可檢索的 KB） +2. **Query**（每次 user 問問題）：question embed → semantic search（或 hybrid + BM25）→ top-K chunks → 塞進 prompt → LLM 答 + +**解決 LLM 不知道你私有 / 變動 / 過期資料**。Retrieval **不限於 dense embedding**——production 標配是 hybrid（dense + BM25）+ reranker。 + +📍 詳細：[Stage 6](../stages/06-memory-rag.md) +📍 paper：[Lewis et al. 2020](https://arxiv.org/abs/2005.11401) + +### Reflexion（完整版反思 / 帶 episodic memory） + +跟 Self-Refine（2 Agent）不同：Reflexion **需要持久 episodic memory store**——agent 跑完 trial 後**寫一段 reflection summary 進 memory**、下一次 trial 開始時 retrieve 進 prompt。**跨 trial 累積教訓**是 Reflexion 的本質（不是 single-session loop）。 + +放在 3 而非 2 Agent 因為它**本質是 memory pattern**——episodic memory store 是核心、不是 optional。 + +代表 paper：[Reflexion (Shinn 2023)](https://arxiv.org/abs/2303.11366)。 + +📍 詳細：[Stage 6 進階：帶持久記憶的 Reflexion 完整版](../stages/06-memory-rag.md#-進階帶持久記憶的-reflexion-完整版--track-b-選讀) + +### Embedding（嵌入） + +把文字 / 圖片轉成 N 維**向量**、讓「意思接近的東西距離近」。本路線圖預設指 **dense embedding**（稠密向量、sentence-transformers / OpenAI ada-002 等產生）；另有 **sparse embedding**（BM25 / SPLADE 等、用字面 token 比對）——production RAG 兩者並用做 hybrid search。 + +📍 詳細：[Stage 6](../stages/06-memory-rag.md) + +### Vector DB（向量資料庫） + +存 + 高效查 embedding 的儲存層。**主要查詢類型 = approximate nearest-neighbor (ANN)**——所以 Vector DB 存在的意義就是「ANN 比直接 cosine 全掃快幾百倍」。代表：Pinecone / Chroma / Qdrant / Weaviate / pgvector。 + +📍 詳細：[Stage 6](../stages/06-memory-rag.md) + +### Semantic Search（語意搜尋） + +用 embedding 比較「意思相似」而不是「字串完全相同」。「電動車怎麼充電」可以撈到「EV charging tutorial」。傳統關鍵字搜尋（BM25 等）做不到這個。 + +### Chunking（切塊） + +把長文件切成適合 embedding 的小段（通常 200-1000 token）。**切法直接影響 RAG 品質**——切太碎丟脈絡、切太長相關度模糊。常見策略：固定大小、按段落、按結構（heading）。 + +### Hybrid Search（混合搜尋） + +語意搜尋 + 關鍵字搜尋一起用，再 merge 排序。多半比單一方法準。上線部署 RAG 的標配。 + +### Reranking（重新排序） + +第一輪 retrieval 撈 top-50，再用更貴但更準的模型（cross-encoder）重排成 top-5 給 LLM。Cohere Rerank、bge-reranker 等。 + +### Contextual Retrieval + +Anthropic 2024 提的方法——chunk 加上「整份文件的脈絡摘要」一起 embed，避免「這 chunk 拿出來看不知道是哪份文件講的」問題。 + +📍 詳細：[Stage 6](../stages/06-memory-rag.md) + +### Fine-tuning（模型微調） + +拿你自己的資料**再訓練**模型、把知識或行為「燒進」權重裡（跟 RAG 不同——RAG 是 inference 時才把資料塞進 context、不改權重）。適合讓模型穩定學會某種**格式 / 風格 / 領域用語**；**不適合**拿來塞「最新事實」（那是 RAG 的活，fine-tune 進去的事實會過期又難更新）。多數 agent 場景**先試 prompt + RAG**，真的不夠才考慮 fine-tune。 + +📍 詳細：[Stage 6](../stages/06-memory-rag.md) + +--- + +## 4. Multi-Agent + +### Multi-Agent（多 agent） + +多個 agent 互相協作完成一個任務。常見 pattern： + +- **Supervisor + Worker**：一個 agent 規劃 / 分派、其他執行 +- **Swarm（群集）**：平等的 agent 群，沒有固定 supervisor +- **Debate（辯論）**：多個 agent 各持立場、最後 consensus + +📍 詳細：[Stage 7](../stages/07-multi-agent-production.md) + +### Handoff + +一個 agent 把任務交給另一個 agent。比直接 function call 多了「context 怎麼傳」、「失敗誰處理」的問題。 + +### A2A（Agent-to-Agent）Protocol + +Google 發起、現由 Linux Foundation 治理的 agent 之間溝通協定，類似 MCP 但用於 agent ↔ agent（不是 agent ↔ tool）。2026 已達 **v1.0**（已有 150+ 組織採用，並加入身分驗證、讓一個 agent 能確認對方是不是真的它），是 MCP（agent↔tool）的姊妹標準。 + +--- + +## 5. Claude Code 生態 + +### MCP（Model Context Protocol） + +Anthropic 2024 推的開放協定、讓任何 LLM host（Claude Code、Cursor、自寫 agent）用同一套介面接外部 tool server。把它想成「**LLM 的 USB 接口**」。 + +**技術上標準化 3 種 primitives**： +- **Tools**：LLM 可呼叫的 function（read DB / search web / send email…） +- **Resources**：LLM 可讀取的資料（檔案內容、API response、DB rows…） +- **Prompts**：可複用的 prompt 模板（給 user 在 host 內 `/` 觸發） + +**架構**：server / client 模式——tool server 跑在本機或遠端、LLM host 當 client 連接。Server 用 stdio / SSE / HTTP 三種 transport 之一暴露 primitives。 + +📍 詳細：[Stage 5.2](../stages/05-claude-code-ecosystem.md#52--mcpmodel-context-protocol-基礎) + +### Skills / SKILL.md + +Claude Code 的「行為包」。一個 Skill = 一個資料夾含 `SKILL.md`（描述「在什麼情境要做什麼、可呼叫哪些 tool」）+ 可選的 reference files / scripts。 + +**觸發機制**（很多人不知道、很關鍵）：Claude Code 每次處理你訊息**前**、會掃所有可用 skill 的 **frontmatter `description` 欄位**——匹配當下情境就把對應 SKILL.md 自動載入。**所以 description 寫得好不好直接決定 skill 會不會被觸發**。寫法：以「Use when ...」開頭最有效。 + +📍 詳細：[Stage 5.3](../stages/05-claude-code-ecosystem.md#53--skillsclaude-code-的行為層-claude-code-生態最關鍵的一層) + +### Plugin / Marketplace + +把多個 Skills + slash commands + hooks + MCP 設定打包成一個發布單位。**Marketplace** 就是 plugin 的目錄，社群可以 `claude plugin install` 安裝別人寫好的。 + +📍 詳細：[Stage 5.4](../stages/05-claude-code-ecosystem.md#54--plugins-與-marketplaces) + +### Slash Command + +Claude Code 內以 `/` 開頭的指令（`/help`、`/compact`、`/plan` 等）。可以自訂——把一段 prompt 存到 `.claude/commands/.md` 就變成 `/name`。 + +### CLAUDE.md + +放在 project root 的 markdown 檔，Claude Code 每次啟動都會讀。寫 project 級的規則 / 規範 / context（用什麼語言、coding style、別動哪些檔等）。 + +### Hooks + +在 Claude Code 特定事件前後執行的 script。**官方支援 7 種事件類型**： + +| Hook | 觸發時機 | 典型用途 | +|---|---|---| +| `PreToolUse` | 工具呼叫**前** | 攔截危險操作（rm -rf、destructive op）、改參數 | +| `PostToolUse` | 工具呼叫**後** | log 記錄、auto-format 寫完的檔 | +| `UserPromptSubmit` | user 訊息送出時 | 加 context（git status / 當前時間）| +| `Notification` | Claude Code 通知時 | 桌面 toast / Slack ping | +| `Stop` | session 結束時 | 自動 commit / 清理 | +| `PreCompact` | 自動 compact 前 | 把重要決定 promote 到 memory | +| `PostCompact` | compact 後 | 確認哪些 context 被壓縮 | + +寫法：`.claude/settings.json` 加 `"hooks"` 區塊、指 script 路徑。 + +### Deep Agent（深度 agent） + +「自帶完整配備」的 agent 設計——不只會呼叫工具，還內建規劃（待辦清單）、長期記憶（檔案系統）、子 agent 分工、可載入的 skills。對照：陽春 agent 只有 LLM + 幾個工具。代表實作：LangChain 的 [deepagents](https://github.com/langchain-ai/deepagents)。 + +--- + +### Subagent（子 agent） + +主 Claude Code session 之外，spawn 出來跑特定任務的 agent。有自己的 context window。例如「給我一個 code-reviewer subagent 看看 diff」。 + +寫法：在 `.claude/agents/.md` 放 frontmatter + system prompt + tool whitelist。主 session 用 Task tool invoke（自動 parallel / sequential）。**跟 framework-based multi-agent 對照**：subagent 不需要裝 LangGraph / CrewAI 等 framework、直接寫 markdown 即可；但綁 Claude Code runtime。完整教學見 [Stage 5.5](../stages/05-claude-code-ecosystem.md#55--subagentsclaude-code-原生-multi-agent-機制-2025-新功能)；**15 個複製貼上即用的 dispatch recipe** → [`subagent-cookbook.md`](./subagent-cookbook.md)；**自己寫 / 組合 / debug 進階主題** → [`subagent-advanced.md`](./subagent-advanced.md)。 + +--- + +## 6. Production / Eval / Cost + +### Eval（評估框架） + +針對 agent 跑一組 test case，量化它的準確度 / latency / cost。**production agent 沒有 eval 等於沒有測試**。常見工具：promptfoo、LangSmith、langfuse evals。 + +📍 詳細：[Stage 7](../stages/07-multi-agent-production.md) + +### Observability + +把 agent 內部跑的每一步（哪個 LLM call、哪個 tool、什麼結果）都記下來。出 bug 時能 replay。常見：langfuse、Helicone、weave。 + +📍 詳細：[Stage 7](../stages/07-multi-agent-production.md) + +### Prompt Caching + +LLM 把 prompt 前綴 cache 起來，下次同前綴只算 cache hit 的便宜價（Anthropic 90% off、OpenAI 50% off）。Long context + 重複 query 的場景可以省很多錢。 + +### Streaming（串流輸出） + +LLM 邊生邊回（一個 token 一個 token），不是等全部生完才丟整段回來。讀者體驗較好（像在打字）；技術上用 SSE 或 chunked transfer。**production 互動式應用幾乎都開**。代價：客戶端要能 handle partial response、ReAct 內 tool call 解析要等到 stream 結束。 + +### Batch API（批次 API） + +把大量 LLM 請求打包送（不要求即時），24 小時內回。**Anthropic / OpenAI 通常打 5 折**。適合非互動場景：批次摘要、批次分類、eval 跑大量 test case、ETL pipeline。**互動式 chat 不能用**——延遲對使用者體驗來說太久。 + +### Token Cost / Inference Cost + +每次 LLM 呼叫的成本 = input tokens × input price + output tokens × output price。Agent 跑 ReAct loop 的成本可以累積很快——大 codebase grep 一次可能花 10 萬 token。 + +### Guardrails + +防 LLM 做壞事的規則層——擋掉 prompt injection、PII 外流、有害輸出等。NeMo Guardrails、Guardrails AI 等。 + +### Prompt Injection（提示注入） + +把惡意指令藏在 LLM 會讀到的內容裡（網頁、文件、工具回傳），誘導它無視原任務、改做攻擊者要的事。根因：LLM 分不清「指令」與「資料裡夾帶的指令」。防法：最小權限、隔離不可信內容、高風險動作人審。相關：lethal trifecta、Guardrails。 + +### Lethal Trifecta（致命三角） + +Simon Willison 提出：agent 同時有（1）存取私密資料、（2）接觸不可信內容、（3）對外通訊三種能力時，就可能被 prompt injection 操控去偷資料外傳。防法是打斷至少一環（常見：切斷對外通訊或隔離不可信輸入）。 + +--- + +## 7. 用詞 / Buzzword + +### CLI Agent + +跑在終端機的 agent（Claude Code、Codex、Aider、Gemini CLI 等）。對比於跑在 IDE 內（Cursor、Continue）或 web 上（ChatGPT、Claude.ai）。 + +📍 詳細：[Track A A1](../tracks/cli/A1-cli-intro.md)、[`resources/cli-agents-guide.md`](cli-agents-guide.md) + +### BYO API Key（Bring Your Own） + +工具支援你自己提供 API key 而不是綁訂閱。Aider / OpenCode / goose 等 CLI 都是 BYO；Claude Code / Codex 預設是訂閱制。 + +### Local LLM / On-Device + +模型跑在你自己機器上（Ollama、llama.cpp、MLX、LocalAI 等），資料不外傳。隱私 OK 但能力比 frontier 模型有差。 + +📍 詳細：[Stage 1](../stages/01-llm-basics.md) + +### Quantization（量化） + +把模型權重從 fp16 壓到 int8 / int4，省記憶體跟速度，代價是準確度小幅降低。Local LLM 用戶常碰到（Q4_K_M、Q8_0 等）。 + +### Hallucination（幻覺） + +LLM 「自信地說錯」——把不存在的 API 編出來、把錯的數字當成事實寫。所有 production agent 都要防這個（用 RAG / structured output / eval / guardrails）。 + +### Frontier Model + +當下最頂的模型（**2026-07**：OpenAI **GPT-5.6**（Sol / Terra / Luna 三級、1.05M context、ChatGPT / Codex / API 皆可用）；**2026-06 後半**：Claude Sonnet 5（速度×智慧的最佳平衡、接近 Opus 4.8 但更便宜）、Google Gemini 3.5 Flash、xAI Grok 4.3、Mistral Medium 3.5（開源權重、preview）；**2026-06 前半**：Claude Fable 5（Mythos-class，定位在 Opus 之上）發布，2026-06-12 曾被美國出口管制暫停，但 **出口管制 2026-06-30 解除、[Fable 5 於 2026-07-01 全球恢復](https://www.anthropic.com/news/redeploying-fable-5)**（Mythos 5 僅對核准美國組織恢復）；**2026-05**：GPT-5.5、Claude Opus 4.8（Opus-class 旗艦）、Gemini 3.1 Pro、DeepSeek-V4-Pro 等）。一般智慧任務用 frontier；簡單分類 / 翻譯用便宜的小模型省錢。 + +### Context Engineering + +工程「**每次 LLM call 時、context window 裡裝什麼資訊**」的學科——動態把 RAG retrieve 結果、memory、tool definitions、對話 history 組裝成 LLM 看得到的 context。Karpathy 2025：「填進 window 的資訊**剛好對下一步有用**的精細藝術」。重點是 *what goes in the window*、不是「跨幾次 call」。**Prompt engineering 的下一層**——前者工程**字串**、後者工程**資訊**。 + +📍 詳細：[Stage 2 結尾](../stages/02-prompt-engineering.md) / [Stage 6](../stages/06-memory-rag.md) / [Stage 7](../stages/07-multi-agent-production.md) +📍 延伸：[`Meirtz/Awesome-Context-Engineering`](https://github.com/Meirtz/Awesome-Context-Engineering) + +### Harness Engineering + +工程「**模型外圍的執行與控制層**」——所有不是 model weights、也不是 prompt string 本身的工程元件：agent loop / tool registry / context manager / permissions / safety layer / memory layer / eval / observability / retry / circuit breaker 等。Simon Willison 2025：「**coding agent = LLM + harness**」、Addy Osmani：「harness = 所有不是 model 本身的程式碼」。[OpenAI 2026-02 也使用 "Harness Engineering" 這個說法](https://openai.com/index/harness-engineering)。Claude Code、Cursor、OpenCode 等 CLI agent 都是 harness。**framework 把 LLM 包成 agent、harness 把 agent 包成可上線使用的產品**。 + +對比： +- **Framework**（Stage 4）規範 **API**：你呼叫的介面長什麼樣 +- **Harness**（本詞）規範 **runtime**：怎麼跑、怎麼 recovery、怎麼觀測 + +📍 **學科級概念**（**8 個核心元件** / prompt→context→harness 三層工程分工 / framework vs harness）：[Stage 7 Harness Engineering](../stages/07-multi-agent-production.md#-harness-engineering--production-agent-runtime-的工程設計--本-stage-核心概念) +📍 **Reference implementation case study**（讀 Claude Code source）：[Stage 5 5.7](../stages/05-claude-code-ecosystem.md#57--claude-code-source-解剖reference-harness-implementation-track-b-必看) +📍 延伸：[`anthropics/claude-agent-sdk-python`](https://github.com/anthropics/claude-agent-sdk-python)、[`ai-boost/awesome-harness-engineering`](https://github.com/ai-boost/awesome-harness-engineering)、[`ZhangHanDong/harness-engineering-from-cc-to-ai-coding`](https://github.com/ZhangHanDong/harness-engineering-from-cc-to-ai-coding) + +### Loop Engineering（迴圈工程） + +prompt engineering → context engineering → harness engineering 之後的第四層：設計 / 調校 agent 的「迭代迴圈」本身——目標、工具、context 管理、終止條件、錯誤處理，讓長時間（數百步、跨 session）運行仍可靠、可控、不跑偏。相關：harness、Dynamic Workflows、ReAct。 + +--- + +## 8. Agent Interfaces + +### Computer Use（螢幕級 agent） + +Agent 透過 **screenshot → vision → 算座標 → 模擬鍵鼠** 操作真實桌面 app——不靠 API、直接像人類用螢幕。代表：Anthropic Claude Computer Use（Opus 4.8 / Sonnet 5）/ OpenAI Codex desktop / Google Gemini in Chrome。**2024-10 Anthropic 公開 beta 開啟、2026 OSWorld 達 76.26% superhuman**。 + +📍 完整解說 + 4 強對比：[Stage 8 Computer Use](../stages/08-agent-interfaces.md#-computer-use--螢幕級-agent) + +### Browser Use（web 級 agent） + +Agent 操作網頁、主要用 **DOM-aware navigation**（直接 query CSS selector）+ 必要時 vision fallback。代表閉源：Atlas / Comet / Dia / Gemini in Chrome。代表 OSS：[browser-use](https://github.com/browser-use/browser-use)（★ 95k+）。 + +📍 完整解說 + 5 強對比 + OSS 框架：[Stage 8 Browser Use](../stages/08-agent-interfaces.md#-browser-use--web-級-agent) + +### Sandbox（程式碼隔離環境） + +讓 agent 寫的 code 在隔離環境跑、不在 host 機器——避免 agent `rm -rf /` / 連 internet 泄資料 / 偷 credentials 等災難。代表：E2B（Firecracker microVM）/ Daytona（Container）/ Modal（GPU sandbox）/ Vercel / Cloudflare。**OpenAI Agents SDK 2026-04 內建支援這些 provider**。 + +📍 完整 9-row 術語小辭典（含 microVM / Container 差異）+ 7 強對比：[Stage 8 Code Sandbox](../stages/08-agent-interfaces.md#-code-execution-sandbox--隔離環境含術語小辭典) + +### microVM（micro Virtual Machine） + +VM 的精簡版、極小 footprint、啟動 < 100ms 但仍**獨立 kernel**——介於 Docker container（快 + 弱隔離）跟 full VM（慢 + 強隔離）之間。**Agent sandbox 多半選 microVM**。代表實作：[Firecracker](#firecracker)（AWS、E2B 用）。 + +📍 完整對比：[Stage 8 術語小辭典](../stages/08-agent-interfaces.md#-隔離技術術語小辭典) + +### Firecracker + +AWS 開源的 microVM、Rust 寫、**AWS Lambda 底層** + E2B sandbox 用它做 isolation。強隔離 + 快啟動兼顧。 + +📍 [Stage 8 術語小辭典](../stages/08-agent-interfaces.md#-隔離技術術語小辭典) + +### gVisor + +Google 寫的「用戶空間 kernel」、攔截 syscall 自己模擬、**不用 hypervisor**——介於 container 跟 VM。 + +📍 [Stage 8 術語小辭典](../stages/08-agent-interfaces.md#-隔離技術術語小辭典) + +--- + +## 找不到的詞？ + +- 看 [Stage 5.2 — MCP](../stages/05-claude-code-ecosystem.md#52--mcpmodel-context-protocol-基礎) / [5.3 — Skills](../stages/05-claude-code-ecosystem.md#53--skillsclaude-code-的行為層-claude-code-生態最關鍵的一層) / [5.4 — Plugins](../stages/05-claude-code-ecosystem.md#54--plugins-與-marketplaces) 的內文 +- 看 [Stage 1](../stages/01-llm-basics.md) / [Stage 6](../stages/06-memory-rag.md) / [Stage 7](../stages/07-multi-agent-production.md) / [Stage 8](../stages/08-agent-interfaces.md) 的延伸閱讀清單 +- 找不到的詞 → 開 issue 或直接 PR 加進這份小辭典 diff --git a/resources/glossary.zh-Hans.md b/resources/glossary.zh-Hans.md new file mode 100644 index 0000000..26e1105 --- /dev/null +++ b/resources/glossary.zh-Hans.md @@ -0,0 +1,445 @@ +# 术语小词典（Glossary） + +> [繁體中文](./glossary.md) | **简体中文** | [English](./glossary.en.md) + +> 本路线图会大量出现“LLM”、“RAG”、“MCP”、“agent”这类词。读到不懂的词先在这里查 30 秒，再回去读 stage 内容。 +> +> 每个词**只给最小可用的解释**（30-80 字 + 在哪一个 stage 讲细的）——不是维基百科。 + +## 🌐 统一词汇对照表（中英对照、跨 stage 一致） + +本表是项目内**强制统一的命名约定**——所有 stage 用同一个中文理解名。如果你在 stage 内看到不一致，请报 issue。 + +| 英文术语 | 中文理解名 | 主要 stage | +|---|---|---| +| Prompt Engineering | Prompt 设计 | Stage 2 | +| Context Engineering | 上下文管理 | Stage 6 | +| Harness Engineering | Agent 执行系统设计 | Stage 7 | +| Tool Use | 工具使用 | Stage 3 | +| Function Calling | 函数 / 工具调用 | Stage 3 | +| Structured Output | 结构化输出 | Stage 3 | +| Agent Loop | Agent 执行循环 | Stage 3 | +| Framework | 框架 | Stage 4 | +| Orchestration | 协调与编排 | Stage 4 / 7 | +| Handoff | 任务交接 | Stage 7 | +| Supervisor / Worker | 协调者 / 执行者 | Stage 7 | +| Runtime | 执行层 | Stage 7 | +| Scaffolding | 支撑架构 | Stage 7 | +| Observability | 观测与记录 | Stage 7 | +| Telemetry | 运行记录 | Stage 7 | +| Eval | 效果评估 | Stage 7 | +| Evaluation Harness | 评估框架 | Stage 7 | +| Production | 可稳定使用 / 上线化 | Stage 7 | +| Production-grade | 可长期稳定使用的 | Stage 7 | +| Deployment | 部署 | Stage 7 | +| Cost Tracking | 成本追踪 | Stage 7 | +| Latency | 延迟 / 等待时间 | Stage 7 | +| Vector DB | 向量数据库 | Stage 6 | +| Retrieval | 检索 | Stage 6 | +| Reranking | 重排序 | Stage 6 | +| Long Context | 长上下文 | Stage 6 | +| Fine-tuning | 模型微调 | Stage 6 | +| Agent Interfaces | Agent 操作界面 | Stage 8 | +| Code Sandbox | 隔离程序执行环境 | Stage 8 | +| Cold Start | 启动延迟 | Stage 8 | +| Reward Hacking | 钻评分漏洞 | Stage 7 / 8 | + +→ 详细定义请看下面各区块。 + +--- + +## 1. 基本概念 + +### LLM（Large Language Model，大语言模型） + +GPT、Claude、Gemini 这类“给文字、回文字”的模型。本身是纯函数：input prompt → output text。它**不会自己上网、不会记住上次对话**——这些都要外接系统来做。 + +📍 详细：[Stage 1](../stages/01-llm-basics.zh-Hans.md) + +### Token + +LLM 看到的不是“字”，是 **token**（次字单位）。中文 1 个字 ≈ 1.5-2 token，英文 1 个 word ≈ 1.3 token。LLM 计费跟 context window 都以 token 计。“100 万 token context”≈ 75 万中文字。 + +📍 详细：[Stage 1](../stages/01-llm-basics.zh-Hans.md) + +### Context Window（上下文视窗） + +LLM 一次能“看”多少 token。**2026 frontier**：Claude Sonnet 5 / Opus 4.8 1M、GPT-5.6 1.05M、Gemini 3.5 Flash 1M（Pro 系列上看 2M）、xAI Grok 4.3 1M、Mistral Medium 3.5 256k。**不是越大越好**——超过某个长度后 LLM 会“在中间遗漏”（Lost in the Middle）。 + +### Prompt（提示词） + +你给 LLM 的输入文字。**Prompt engineering** 就是设计这段输入让 LLM 给好答案。System prompt（角色设定）+ user prompt（这次的问题）是基本结构。 + +📍 详细：[Stage 2](../stages/02-prompt-engineering.zh-Hans.md) + +### Zero-shot / One-shot / Few-shot + +在 prompt 里放“几个示范例子”让 LLM 照着做——这三个词的差别只在**你给几个范例**： + +- **Zero-shot**（0 个范例）：直接问、不给任何范例。 +- **One-shot**（1 个范例）：先给 **1 个** input → output 范例再问。 +- **Few-shot**（少数几个）：给 **2-5 个** input → output 范例后再问。**Few-shot 通常显著提升准确度**，特别是格式要求严的任务。 + +### Chain-of-Thought（CoT，思维链） + +要 LLM“先想再答”——让它先输出推理过程，再给结论。**常见有两种形式**： + +- **Few-shot CoT**（原始 paper、[Wei et al. 2022](https://arxiv.org/abs/2201.11903)）：在 prompt 里放几个带推理步骤的例子，让 LLM 模仿着想 +- **Zero-shot CoT**（[Kojima et al. 2022](https://arxiv.org/abs/2205.11916)）：在 prompt 结尾加上“Let's think step by step”来触发 reasoning trace + +**准确度通常会提升**，代价是 token 数变多。Few-shot 通常比 zero-shot 更准。 + +--- + +## 2. Agent / 工具使用 + +### Agent（代理人） + +以 LLM 为核心、能在**循环**中**感知状态 → 做决策 → 采取行动 → 观察结果**、重复直到完成目标的系统。**核心三要素**： + +- **LLM**（推理 / 规划 / decide） +- **Actions**（做事的手段——不限于 function call。可以是写代码执行（CodeAct）、操作浏览器（computer use）、查 KB（RAG retrieval）、call MCP server、纯规划拆任务等） +- **Loop**（心跳——agent 跟纯 LLM Q&A 的根本差别） + +差别在于：纯 LLM = Q&A；agent = 三要素 + 持续循环，直到目标达成或预算耗尽。**ReAct 是其中一种 agent pattern，不是 agent 的定义**——CodeAct、computer-use、planning agent 都是 agent。 + +📍 详细：[Stage 3](../stages/03-tool-use-and-hello-agent.zh-Hans.md) + +### Tool Use / Function Calling + +让 LLM 调用你定义好的 function（查 DB、算数学、开浏览器…）。LLM 回的不是文字而是 `{"function": "search", "args": {...}}`，你的程序去执行、把结果再丢回 LLM。 + +**两个词概念相同，但 API schema 不一样**： +- **Anthropic 的 "Tool Use"**：schema 用 `input_schema`（直接放 JSON Schema） +- **OpenAI / Ollama 的 "Function Calling"**：外面再包一层 `{"type": "function", "function": {...}}` +- LLM 内部接收到的 token 表达也不同，写跨厂商 SDK 时要记得对应好 + +📍 详细：[Stage 3](../stages/03-tool-use-and-hello-agent.zh-Hans.md) +📍 schema 怎么写好：[Function Schema 设计 cheatsheet](schema-design-cheatsheet.zh-Hans.md) + +### ReAct（Reasoning + Acting） + +最经典的 agent pattern：**Thought（想）→ Action（叫工具）→ Observation（看结果）→ Thought ...** 一直 loop 到答得出来。多数 agent framework 内部都实作这个。 + +📍 详细：[Stage 3](../stages/03-tool-use-and-hello-agent.zh-Hans.md) + +### Structured Output（结构化输出） + +要 LLM 输出 **JSON / 其他固定 schema**，而不是自由文字。各家 LLM API 都有 `response_format` 或类似旗标支持。Agent 框架几乎都靠这个跟 LLM 沟通。 + +### Agent Loop + +“LLM → tool → 结果 → LLM”这个重复的循环。Loop 结束条件可能是：LLM 说“I'm done”、跑超过 N 步、超出 budget。 + +### Self-Refine（基础版反思 / 无记忆） + +agent 自我评估上一轮输出、修改下一轮的做法——“Actor 出答案 → Critic 找问题 → Actor 看 feedback 再答”的 single-session loop。**不需要持久记忆层**，本质上就是 reasoning loop 机制，是 ReAct 的 sibling pattern。production agent（Cursor / Cline / Claude Code）每天都在跑这种变体。 + +代表 paper：[Self-Refine (Madaan 2023)](https://arxiv.org/abs/2303.17651)。**完整版 Reflexion**（含 episodic memory）见 3 Memory / Retrieval / RAG（这是不同层的东西）。 + +📍 详细 + 路由：[Stage 3 反思](../stages/03-tool-use-and-hello-agent.zh-Hans.md#-反思reflexion--self-refine-概念--路由) + +--- + +## 3. Memory / Retrieval / RAG + +### Memory（记忆）— 两种正交分类轴 + +“memory”常被混在一起讲，其实有 **2 种正交分类轴**： + +- **时效轴**：short-term（当前对话）vs long-term（跨 session 持久） +- **内容轴**（CoALA framework）：**Working**（暂存）/ **Episodic**（过去经历）/ **Semantic**（事实知识）/ **Procedural**（怎么做） + +→ 两条轴并不冲突：long-term memory 里可以**同时**有 episodic（user 上次说了什么）+ semantic（公司知识库事实）+ procedural（用过的 tool sequence）。 + +📍 详细：[Stage 6 Memory 是什么 + 如何设计](../stages/06-memory-rag.zh-Hans.md#-memory-是什么--怎么设计) + [Stage 6 CoALA Framework](../stages/06-memory-rag.zh-Hans.md#进阶coala-framework--agent-memory-的-4-层分类法) + +### RAG（Retrieval-Augmented Generation） + +两阶段架构模式： + +1. **Ingest**（一次性 / 定期）：document → chunk → embed → 存进 vector store（建一个可检索的 KB） +2. **Query**（每次 user 提问）：question embed → semantic search（或 hybrid + BM25）→ top-K chunks → 塞进 prompt → LLM 回答 + +**解决的是 LLM 不知道你的私有 / 变动 / 过期资料**。Retrieval **不只限于 dense embedding**——production 默认配置通常是 hybrid（dense + BM25）+ reranker。 + +📍 详细：[Stage 6](../stages/06-memory-rag.zh-Hans.md) +📍 paper：[Lewis et al. 2020](https://arxiv.org/abs/2005.11401) + +### Reflexion（完整版反思 / 带 episodic memory） + +跟 Self-Refine（2 Agent）不同：Reflexion **需要持久 episodic memory store**——agent 跑完一次 trial 后，会**写一段 reflection summary 进 memory**，下一次 trial 开始时再检索进 prompt。**跨 trial 累积教训**才是 Reflexion 的本质（不是 single-session loop）。 + +放在 3 而不是 2 Agent，是因为它**本质上是 memory pattern**——episodic memory store 是核心，不是 optional。 + +代表 paper：[Reflexion (Shinn 2023)](https://arxiv.org/abs/2303.11366)。 + +📍 详细：[Stage 6 进阶：带持久记忆的 Reflexion 完整版](../stages/06-memory-rag.zh-Hans.md#-进阶带持久记忆的-reflexion-完整版--track-b-选读) + +### Embedding（嵌入） + +把文字 / 图片转成 N 维**向量**，让“意思接近”的东西距离更近。本路线图默认指 **dense embedding**（稠密向量，由 sentence-transformers / OpenAI ada-002 等产生）；另外也有 **sparse embedding**（BM25 / SPLADE 等，按字面 token 匹配）——production RAG 往往两者一起用来做 hybrid search。 + +📍 详细：[Stage 6](../stages/06-memory-rag.zh-Hans.md) + +### Vector DB（向量数据库） + +存储 + 高效查询 embedding 的存储层。**主要查询类型 = approximate nearest-neighbor (ANN)**——Vector DB 存在的意义就是 ANN 比直接做 cosine 全扫快几百倍。代表：Pinecone / Chroma / Qdrant / Weaviate / pgvector。 + +📍 详细：[Stage 6](../stages/06-memory-rag.zh-Hans.md) + +### Semantic Search（语义搜索） + +用 embedding 比较“意思相似”而不是“字符串完全相同”。“电动车怎么充电”可以捞到“EV charging tutorial”。传统关键字搜索（BM25 等）做不到这个。 + +### Chunking（切块） + +把长文件切成适合 embedding 的小段（通常 200-1000 token）。**切法直接影响 RAG 质量**——切太碎丢脉络、切太长相关度模糊。常见策略：固定大小、按段落、按结构（heading）。 + +### Hybrid Search（混合搜索） + +语义搜索 + 关键字搜索一起用，再 merge 排序。多半比单一方法准。上线部署 RAG 的标配。 + +### Reranking（重新排序） + +第一轮 retrieval 捞 top-50，再用更贵但更准的模型（cross-encoder）重排成 top-5 给 LLM。Cohere Rerank、bge-reranker 等。 + +### Contextual Retrieval + +Anthropic 2024 提的方法——chunk 加上“整份文件的脉络摘要”一起 embed，避免“这 chunk 拿出来看不知道是哪份文件讲的”问题。 + +📍 详细：[Stage 6](../stages/06-memory-rag.zh-Hans.md) + +### Fine-tuning（模型微调） + +拿你自己的资料**再训练**模型、把知识或行为“烧进”权重里（跟 RAG 不同——RAG 是 inference 时才把资料塞进 context、不改权重）。适合让模型稳定学会某种**格式 / 风格 / 领域用语**；**不适合**拿来塞“最新事实”（那是 RAG 的活，fine-tune 进去的事实会过期又难更新）。多数 agent 场景**先试 prompt + RAG**，真的不够才考虑 fine-tune。 + +📍 详细：[Stage 6](../stages/06-memory-rag.zh-Hans.md) + +--- + +## 4. Multi-Agent（多 agent） + +### Multi-Agent（多 agent） + +多个 agent 互相协作完成一个任务。常见 pattern： + +- **Supervisor + Worker**：一个 agent 规划 / 分派、其他执行 +- **Swarm（群集）**：平等的 agent 群，没有固定 supervisor +- **Debate（辩论）**：多个 agent 各持立场、最后 consensus + +📍 详细：[Stage 7](../stages/07-multi-agent-production.zh-Hans.md) + +### Handoff + +一个 agent 把任务交给另一个 agent。比直接 function call 多了“context 怎么传”、“失败谁处理”的问题。 + +### A2A（Agent-to-Agent）Protocol + +Google 发起、现由 Linux Foundation 治理的 agent 之间沟通协定，类似 MCP 但用于 agent ↔ agent（不是 agent ↔ tool）。2026 已达 **v1.0**（已有 150+ 组织采用，并加入身分验证、让一个 agent 能确认对方是不是真的它），是 MCP（agent↔tool）的姊妹标准。 + +--- + +## 5. Claude Code 生态 + +### MCP（Model Context Protocol） + +Anthropic 在 2024 推出的开放协定，让任何 LLM host（Claude Code、Cursor、自写 agent）都能用同一套接口连接外部 tool server。把它想成“**LLM 的 USB 接口**”。 + +**技术上标准化了 3 种 primitives**： +- **Tools**：LLM 可调用的 function（read DB / search web / send email…） +- **Resources**：LLM 可读取的数据（文件内容、API response、DB rows…） +- **Prompts**：可复用的 prompt 模板（给用户在 host 内用 `/` 触发） + +**架构**：server / client 模式——tool server 跑在本地或远端，LLM host 当 client 连接。Server 通过 stdio / SSE / HTTP 三种 transport 之一暴露这些 primitives。 + +📍 详细：[Stage 5.2](../stages/05-claude-code-ecosystem.zh-Hans.md#52--mcpmodel-context-protocol-基础) + +### Skills / SKILL.md + +Claude Code 的“行为包”。一个 Skill = 一个文件夹，里面有 `SKILL.md`（描述“在什么情境要做什么、可调用哪些 tool”）+ 可选的 reference files / scripts。 + +**触发机制**（很多人不知道，但很关键）：Claude Code 每次处理你消息**前**，都会扫描所有可用 skill 的 **frontmatter `description` 字段**——如果匹配当前情境，就会自动载入对应的 SKILL.md。**所以 description 写得好不好，直接决定 skill 会不会被触发。** 实务上以 “Use when ...” 开头最有效。 + +📍 详细：[Stage 5.3](../stages/05-claude-code-ecosystem.zh-Hans.md#53--skillsclaude-code-的行为层-claude-code-生态最关键的一层) + +### Plugin / Marketplace + +把多个 Skills + slash commands + hooks + MCP 设置打包成一个发布单位。**Marketplace** 就是 plugin 的目录，社群可以 `claude plugin install` 安装别人写好的。 + +📍 详细：[Stage 5.4](../stages/05-claude-code-ecosystem.zh-Hans.md#54--plugins-与-marketplaces) + +### Slash Command + +Claude Code 内以 `/` 开头的指令（`/help`、`/compact`、`/plan` 等）。可以自定义——把一段 prompt 存到 `.claude/commands/.md` 就变成 `/name`。 + +### CLAUDE.md + +放在 project root 的 markdown 档，Claude Code 每次启动都会读。写 project 级的规则 / 规范 / context（用什么语言、coding style、别动哪些档等）。 + +### Hooks + +在 Claude Code 特定事件前后执行的 script。**官方支持 7 种事件类型**： + +| Hook | 触发时机 | 典型用途 | +|---|---|---| +| `PreToolUse` | 工具调用**前** | 拦截危险操作（`rm -rf`、destructive op）、改参数 | +| `PostToolUse` | 工具调用**后** | 记 log、自动格式化刚写好的文件 | +| `UserPromptSubmit` | user 提交消息时 | 加 context（git status / 当前时间） | +| `Notification` | Claude Code 发通知时 | 桌面 toast / Slack ping | +| `Stop` | session 结束时 | 自动 commit / 清理 | +| `PreCompact` | 自动 compact 前 | 把重要决定提升到 memory | +| `PostCompact` | compact 后 | 确认哪些 context 被压缩 | + +写法：在 `.claude/settings.json` 里加 `"hooks"` 区块，指向 script 路径。 + +### Deep Agent（深度 agent） + +“自带完整配备”的 agent 设计——不只会调用工具，还内建规划（待办清单）、长期记忆（文件系统）、子 agent 分工、可加载的 skills。对照：简单的 agent 只有 LLM + 几个工具。代表实作：LangChain 的 [deepagents](https://github.com/langchain-ai/deepagents)。 + +--- + +### Subagent（子 agent） + +主 Claude Code session 之外，spawn 出来跑特定任务的 agent。有自己的 context window。例如“给我一个 code-reviewer subagent 看看 diff”。 + +写法：在 `.claude/agents/.md` 放 frontmatter + system prompt + tool whitelist。主 session 用 Task tool invoke（自动 parallel / sequential）。**跟 framework-based multi-agent 对照**：subagent 不需要装 LangGraph / CrewAI 等 framework，直接写 markdown 即可；但绑定 Claude Code runtime。完整教学见 [Stage 5.5](../stages/05-claude-code-ecosystem.zh-Hans.md#55--subagentsclaude-code-原生-multi-agent-机制-2025-新功能)；**15 个复制粘贴即用的 dispatch recipe** → [`subagent-cookbook.zh-Hans.md`](./subagent-cookbook.zh-Hans.md)。 + +--- + +## 6. Production / Eval / Cost + +### Eval（评估框架） + +针对 agent 跑一组 test case，量化它的准确度 / latency / cost。**production agent 没有 eval 等于没有测试**。常见工具：promptfoo、LangSmith、langfuse evals。 + +📍 详细：[Stage 7](../stages/07-multi-agent-production.zh-Hans.md) + +### Observability + +把 agent 内部跑的每一步（哪个 LLM call、哪个 tool、什么结果）都记下来。出 bug 时能 replay。常见：langfuse、Helicone、weave。 + +📍 详细：[Stage 7](../stages/07-multi-agent-production.zh-Hans.md) + +### Prompt Caching + +LLM 把 prompt 前缀 cache 起来，下次同前缀只算 cache hit 的便宜价（Anthropic 90% off、OpenAI 50% off）。Long context + 重复 query 的场景可以省很多钱。 + +### Token Cost / Inference Cost + +每次 LLM 调用的成本 = input tokens × input price + output tokens × output price。Agent 跑 ReAct loop 的成本可以累积很快——大 codebase grep 一次可能花 10 万 token。 + +### Guardrails + +防 LLM 做坏事的规则层——挡掉 prompt injection、PII 外流、有害输出等。NeMo Guardrails、Guardrails AI 等。 + +### Prompt Injection（提示注入） + +把恶意指令藏在 LLM 会读到的内容里（网页、文档、工具返回），诱导它无视原任务、改做攻击者要的事。根因：LLM 分不清“指令”与“数据里夹带的指令”。防法：最小权限、隔离不可信内容、高风险动作人审。相关：lethal trifecta、Guardrails。 + +### Lethal Trifecta（致命三角） + +Simon Willison 提出：agent 同时有（1）访问私密数据、（2）接触不可信内容、（3）对外通讯三种能力时，就可能被 prompt injection 操控去偷数据外传。防法是打断至少一环（常见：切断对外通讯或隔离不可信输入）。 + +--- + +## 7. 用词 / Buzzword + +### CLI Agent + +跑在终端机的 agent（Claude Code、Codex、Aider、Gemini CLI 等）。对比于跑在 IDE 内（Cursor、Continue）或 web 上（ChatGPT、Claude.ai）。 + +📍 详细：[Track A A1](../tracks/cli/A1-cli-intro.zh-Hans.md)、[`resources/cli-agents-guide.zh-Hans.md`](cli-agents-guide.zh-Hans.md) + +### BYO API Key（Bring Your Own） + +工具支援你自己提供 API key 而不是绑订阅。Aider / OpenCode / goose 等 CLI 都是 BYO；Claude Code / Codex 预设是订阅制。 + +### Local LLM / On-Device + +模型跑在你自*己*机器上（Ollama、llama.cpp、MLX、LocalAI 等），数据不外传。隐私 OK 但能力比 frontier 模型有差。 + +📍 详细：[Stage 1](../stages/01-llm-basics.zh-Hans.md) + +### Quantization（量化） + +把模型权重从 fp16 压到 int8 / int4，省内存跟速度，代价是准确度小幅降低。Local LLM 用户常碰到（Q4_K_M、Q8_0 等）。 + +### Hallucination（幻觉） + +LLM “自信地说错”——把不存在的 API 编出来、把错的数字当成事实写。所有 production agent 都要防这个（用 RAG / structured output / eval / guardrails）。 + +### Frontier Model + +当下最顶的模型（**2026-07**：OpenAI **GPT-5.6**（Sol / Terra / Luna 三级、1.05M context、ChatGPT / Codex / API 都能用）；**2026-06 后半**：Claude Sonnet 5（速度×智慧的最佳平衡、接近 Opus 4.8 但更便宜）、Google Gemini 3.5 Flash、xAI Grok 4.3、Mistral Medium 3.5（开源权重、preview）；**2026-06 前半**：Claude Fable 5（Mythos-class，定位在 Opus 之上）发布，2026-06-12 曾被美国出口管制暂停，但 **出口管制 2026-06-30 解除、[Fable 5 于 2026-07-01 全球恢复](https://www.anthropic.com/news/redeploying-fable-5)**（Mythos 5 仅对核准美国组织恢复）；**2026-05**：GPT-5.5、Claude Opus 4.8（Opus-class 旗舰）、Gemini 3.1 Pro、DeepSeek-V4-Pro 等）。一般智慧任务用 frontier；简单分类 / 翻译用便宜的小模型省钱。 + +### Context Engineering + +工程 **每次 LLM call 时，context window 里装什么信息** 的学科——动态把 RAG retrieve 结果、memory、tool definitions、对话 history 组装成模型看得到的 context。Karpathy 2025：把 **刚好对下一步有用的信息** 填进窗口的精细艺术。重点是 *what goes in the window*，不是“跨了几次 call”。**Prompt engineering 的下一层**——前者工程 **字符串**，后者工程 **信息**。 + +📍 详细：[Stage 2 结尾](../stages/02-prompt-engineering.zh-Hans.md) / [Stage 6](../stages/06-memory-rag.zh-Hans.md) / [Stage 7](../stages/07-multi-agent-production.zh-Hans.md) +📍 延伸：[`Meirtz/Awesome-Context-Engineering`](https://github.com/Meirtz/Awesome-Context-Engineering) + +### Harness Engineering + +工程 **模型外面的执行与控制层**——所有不是 model weights、也不是 prompt string 本身的工程元件：agent loop / tool registry / context manager / permissions / safety layer / memory layer / eval / observability / retry / circuit breaker 等。Simon Willison 2025：**coding agent = LLM + harness**。Addy Osmani：harness = 所有不是 model 本身的代码。[OpenAI 也在 2026-02 使用了 "Harness Engineering" 这个说法](https://openai.com/index/harness-engineering)。Claude Code、Cursor、OpenCode 等 CLI agent 都是 harness。**framework 把 LLM 包成 agent，harness 把 agent 包成可上线使用的产品**。 + +对比： +- **Framework**（Stage 4）规范 **API**：你调用的接口长什么样 +- **Harness**（本词）规范 **runtime**：怎么跑、怎么 recovery、怎么观测 + +📍 学科级概念（**8 个核心元件** / prompt→context→harness 三层工程分工 / framework vs harness）：[Stage 7 Harness Engineering](../stages/07-multi-agent-production.zh-Hans.md) +📍 Reference implementation case study（读 Claude Code source）：[Stage 5 5.7](../stages/05-claude-code-ecosystem.zh-Hans.md) +📍 延伸：[`anthropics/claude-agent-sdk-python`](https://github.com/anthropics/claude-agent-sdk-python)、[`ai-boost/awesome-harness-engineering`](https://github.com/ai-boost/awesome-harness-engineering)、[`ZhangHanDong/harness-engineering-from-cc-to-ai-coding`](https://github.com/ZhangHanDong/harness-engineering-from-cc-to-ai-coding) + +### Loop Engineering（循环工程） + +prompt engineering → context engineering → harness engineering 之后的第四层：设计 / 调校 agent 的“迭代循环”本身——目标、工具、context 管理、终止条件、错误处理，让长时间（数百步、跨 session）运行仍可靠、可控、不跑偏。相关：harness、Dynamic Workflows、ReAct。 + +--- + +## 8. Agent Interfaces + +### Computer Use（屏幕级 agent） + +Agent 通过 **screenshot → vision → 算坐标 → 模拟键鼠** 操作真实桌面 app——不靠 API、直接像人类用屏幕。代表：Anthropic Claude Computer Use（Opus 4.8 / Sonnet 5）/ OpenAI Codex desktop / Google Gemini in Chrome。**2024-10 Anthropic 公开 beta 开启、2026 OSWorld 达 76.26% superhuman**。 + +📍 完整解说 + 4 强对比：[Stage 8 Computer Use](../stages/08-agent-interfaces.zh-Hans.md) + +### Browser Use（web 级 agent） + +Agent 操作网页、主要用 **DOM-aware navigation**（直接 query CSS selector）+ 必要时 vision fallback。代表闭源：Atlas / Comet / Dia / Gemini in Chrome。代表 OSS：[browser-use](https://github.com/browser-use/browser-use)（★ 95k+）。 + +📍 完整解说 + 5 强对比 + OSS 框架：[Stage 8 Browser Use](../stages/08-agent-interfaces.zh-Hans.md) + +### Sandbox（程序代码隔离环境） + +让 agent 写的 code 在隔离环境跑、不在 host 机器——避免 agent `rm -rf /` / 连 internet 泄资料 / 偷 credentials 等灾难。代表：E2B（Firecracker microVM）/ Daytona（Container）/ Modal（GPU sandbox）/ Vercel / Cloudflare。**OpenAI Agents SDK 2026-04 内建支持这些 provider**。 + +📍 完整 9-row 术语小词典（含 microVM / Container 差异）+ 7 强对比：[Stage 8 Code Sandbox](../stages/08-agent-interfaces.zh-Hans.md) + +### microVM（micro Virtual Machine） + +VM 的精简版、极小 footprint、启动 < 100ms 但仍**独立 kernel**——介于 Docker container（快 + 弱隔离）跟 full VM（慢 + 强隔离）之间。**Agent sandbox 多半选 microVM**。代表实现：[Firecracker](#firecracker)（AWS、E2B 用）。 + +📍 完整对比：[Stage 8 术语小词典](../stages/08-agent-interfaces.zh-Hans.md) + +### Firecracker + +AWS 开源的 microVM、Rust 写、**AWS Lambda 底层** + E2B sandbox 用它做 isolation。强隔离 + 快启动兼顾。 + +📍 [Stage 8 术语小词典](../stages/08-agent-interfaces.zh-Hans.md) + +### gVisor + +Google 写的“用户空间 kernel”、拦截 syscall 自己模拟、**不用 hypervisor**——介于 container 跟 VM。 + +📍 [Stage 8 术语小词典](../stages/08-agent-interfaces.zh-Hans.md) + +--- + +## 找不到的词？ + +- 看 [Stage 5.2 — MCP](../stages/05-claude-code-ecosystem.zh-Hans.md#52--mcpmodel-context-protocol-基础) / [5.3 — Skills](../stages/05-claude-code-ecosystem.zh-Hans.md#53--skillsclaude-code-的行为层-claude-code-生态最关键的一层) / [5.4 — Plugins](../stages/05-claude-code-ecosystem.zh-Hans.md#54--plugins-与-marketplaces) 的内文 +- 看 [Stage 1](../stages/01-llm-basics.zh-Hans.md) / [Stage 6](../stages/06-memory-rag.zh-Hans.md) / [Stage 7](../stages/07-multi-agent-production.zh-Hans.md) / [Stage 8](../stages/08-agent-interfaces.zh-Hans.md) 的延伸阅读清单 +- 找不到的词 → 开 issue 或直接 PR 加进这份小词典 diff --git a/resources/mcp-skills-catalog.en.md b/resources/mcp-skills-catalog.en.md new file mode 100644 index 0000000..ed8715b --- /dev/null +++ b/resources/mcp-skills-catalog.en.md @@ -0,0 +1,1065 @@ +# MCP / Skills Integration Catalog + +> [繁體中文](./mcp-skills-catalog.md) | [简体中文](./mcp-skills-catalog.zh-Hans.md) | **English** + +> Connect Claude Code (or any other CLI agent) to the apps you already use, without window-hopping. This page is a curated index of 65+ MCP servers / Claude Skills / integrations grouped by use case (incl. research-workflow + multi-LLM-delegation dedicated sections). + +--- + +## How to use this catalog + +- **Looking for a specific tool's MCP**: jump to the relevant section below +- **Want to know what MCP / Skills / Plugins are**: see [RESOURCES.en.md "Three core terms"](../RESOURCES.en.md#three-core-terms-mcp--skills--plugins) first, then [Stage 5 — Claude Code Ecosystem](../stages/05-claude-code-ecosystem.en.md) +- **Want hands-on exercises (install + test)**: see [Stage 5.2 (MCP)](../stages/05-claude-code-ecosystem.en.md#52--mcp-model-context-protocol--foundation) and [Stage 5.3 (Skills)](../stages/05-claude-code-ecosystem.en.md#53--skills-claude-codes-behavior-layer--the-most-critical-layer-of-the-claude-code-ecosystem) + +### Inclusion direction (not strict rules) + +- **Official first**: Anthropic / vendor-published MCP / Skill usually ranks higher +- **Stars are a hint, not a gate**: community repos around 100+ tend to be maintained, but "niche but useful" repos are welcome via PR with a sentence explaining why +- **Metadata when possible**: pull stars / license via `gh api`; refresh whenever +- **Avoid (not forbidden)**: archived, long-stale, unclear-license repos — niche tools can be exceptions + +### Index + +1. [Notes / Knowledge Base](#1-notes--knowledge-base) (7) +2. [Office Documents (Word / Excel / PowerPoint / PDF)](#2-office-documents-word--excel--powerpoint--pdf) (7) +3. [Google Workspace](#3-google-workspace) (2) +4. [Microsoft 365](#4-microsoft-365) (3) +5. [Dev Collaboration (GitHub / Atlassian / Slack…)](#5-dev-collaboration-github--atlassian--slack) (9) +6. [Databases](#6-databases) (8) +7. [Browser Automation / Web Scraping](#7-browser-automation--web-scraping) (4) +8. [Design (Figma / Excalidraw)](#8-design-figma--excalidraw) (3) +9. [Monitoring / Observability](#9-monitoring--observability) (3) +10. [Media / Streaming (YouTube / Spotify)](#10-media--streaming-youtube--spotify) (3) +11. [Chinese-language Ecosystem](#11-chinese-language-ecosystem) (9) +12. [Other Common (Cloudflare / Stripe…)](#12-other-common-cloudflare--stripe) (4) +13. [Research Workflow Skills](#13-research-workflow-skills-academic--paper--lit) (4) +14. [Multi-LLM Delegation Skills](#14-multi-llm-delegation-skills) (3) +15. [Finance / Trading Agents](#15-finance--trading-agents) (2) +16. [Web Search / Retrieval](#16-web-search--retrieval) (2) + +--- + +## 1. Notes / Knowledge Base + +### [makenotion/notion-mcp-server](https://github.com/makenotion/notion-mcp-server) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 4k+ | +| License | NOASSERTION | +| Rating | ⭐⭐⭐⭐⭐ (**official**) | + +**What it does**: Notion's official MCP server — query pages, create pages, manipulate databases. +**Audience**: heavy Notion users for note-taking / project management / wikis — let the LLM pull data and write pages directly. +**Notes**: requires Notion integration token; supports both read-only and read-write modes. + +### [MarkusPfundstein/mcp-obsidian](https://github.com/MarkusPfundstein/mcp-obsidian) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 3.5k+ | +| License | MIT | +| Rating | ⭐⭐⭐⭐ (community, most popular) | + +**What it does**: read/write your Obsidian vault via the Obsidian REST API community plugin. +**Audience**: heavy Obsidian users wanting Claude Code to organize daily notes, auto-link, search across files. +**Notes**: requires the [Local REST API](https://github.com/coddingtonbear/obsidian-local-rest-api) plugin in Obsidian. + +### [PleasePrompto/notebooklm-skill](https://github.com/PleasePrompto/notebooklm-skill) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 6k+ | +| License | NOASSERTION | +| Rating | ⭐⭐⭐⭐ | + +**What it does**: a Claude Code Skill that uses browser automation to query NotebookLM, with citation-backed answers. +**Audience**: people who manage papers / research notes in NotebookLM but want to query from Claude Code in one prompt. +**Notes**: requires Google account auth. + +### [teng-lin/notebooklm-py](https://github.com/teng-lin/notebooklm-py) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 12k+ | +| License | NOASSERTION | +| Rating | ⭐⭐⭐⭐ | + +**What it does**: unofficial NotebookLM Python API + CLI + agentic skill; broader feature set than the skill above, including capabilities the web UI doesn't expose. +**Audience**: people doing programmatic / batch operations on NotebookLM (auto-create notebooks, bulk-import documents). +**Notes**: unofficial; may break with Google policy changes — check the issue tracker before relying on it. + +### [ergut/mcp-logseq](https://github.com/ergut/mcp-logseq) ⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 264 | +| License | MIT | +| Rating | ⭐⭐⭐ | + +**What it does**: read/write Logseq graph via Logseq's Local HTTP API. +**Audience**: Logseq users automating daily journals, cross-page links, backlink queries. +**Notes**: enable Logseq's HTTP API (Settings → Features → HTTP API). + +### [skridlevsky/graphthulhu](https://github.com/skridlevsky/graphthulhu) ⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 147 | +| License | MIT | +| Rating | ⭐⭐⭐ (covers both Logseq + Obsidian) | + +**What it does**: 39 tools across navigation, search, analysis, writing, journals, flashcards, whiteboards. +**Audience**: people using both Logseq and Obsidian who don't want two MCP servers. +**Notes**: community project; broad tool surface but each tool is relatively basic. + +### [ankimcp/anki-mcp-server](https://github.com/ankimcp/anki-mcp-server) ⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 254 | +| License | MIT | +| Rating | ⭐⭐⭐ | + +**What it does**: create / query / batch-edit Anki decks via AnkiConnect. +**Audience**: people using Anki for languages / medicine / law — let the LLM auto-generate cards from study material. +**Notes**: requires Anki Desktop + the [AnkiConnect](https://ankiweb.net/shared/info/2055492159) addon. + +--- + +## 2. Office Documents (Word / Excel / PowerPoint / PDF) + +### [anthropics/skills](https://github.com/anthropics/skills) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 129k+ | +| License | NOASSERTION | +| Rating | ⭐⭐⭐⭐⭐ (**official**, must-install) | + +**What it does**: Anthropic's official Agent Skills repo — includes docx / xlsx / pptx / pdf processing skills. +**Audience**: every Claude Code user — `claude skill install` and Claude can read/write Office files directly. +**Notes**: this is a Skills collection, not an MCP; lives in the Stage 5.3 Skills system. + +### [haris-musa/excel-mcp-server](https://github.com/haris-musa/excel-mcp-server) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 3.8k+ | +| License | MIT | +| Rating | ⭐⭐⭐⭐⭐ (most popular community Excel MCP) | + +**What it does**: Excel file manipulation MCP — read / write / modify cells, formulas, sheets. +**Audience**: people working with Excel reports daily who want LLM-driven data filling and cleanup. +**Notes**: Python-based, depends on openpyxl. + +### [GongRzhe/Office-PowerPoint-MCP-Server](https://github.com/GongRzhe/Office-PowerPoint-MCP-Server) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 1.7k+ | +| License | MIT | +| Rating | ⭐⭐⭐⭐ | + +**What it does**: PPT manipulation via python-pptx — create decks, edit slides, insert images, change layouts. +**Audience**: people who want LLMs to auto-generate decks from outlines / Markdown (consultants, lecturers, students). +**Notes**: overlaps with `anthropics/skills`'s pptx skill; use this when the official one isn't enough. + +### [1weiho/open-slide](https://github.com/1weiho/open-slide) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 4.9k+ | +| License | MIT | +| Rating | ⭐⭐⭐⭐ (agent-native slide framework) | + +**What it does**: a React slide framework built for coding agents — describe a deck in natural language and let Claude Code / Codex / Cursor write the React; ships two Claude Code Skills (`/create-slide`, `/slide-authoring`). +**Audience**: people who want agents to produce decks as code (git-versionable) — a different route from PowerPoint-MCP's .pptx output. +**Notes**: TypeScript / React / Vite; scaffold with `npx @open-slide/cli init`. It's an agent-native tool (agents author with it), not a Stage 4 agent-building / orchestration framework. + +### [SylphxAI/pdf-reader-mcp](https://github.com/SylphxAI/pdf-reader-mcp) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 688 | +| License | MIT | +| Rating | ⭐⭐⭐⭐ (high-throughput PDF) | + +**What it does**: high-speed PDF parsing MCP, ~5-10× faster than `anthropics/skills`'s pdf skill (per their claim). +**Audience**: people doing batch reads of papers / contracts / reports. +**Notes**: parallel processing; noticeable on large PDFs. + +### [tfriedel/claude-office-skills](https://github.com/tfriedel/claude-office-skills) ⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 590 | +| License | NOASSERTION | +| Rating | ⭐⭐⭐ (Office skill add-on) | + +**What it does**: extends `anthropics/skills` with Office workflows it doesn't cover (automation, advanced formatting). +**Audience**: people who find the official docx/xlsx/pptx skills too coarse-grained. +**Notes**: complements `anthropics/skills`, not a replacement. + +### [kreuzberg-dev/kreuzberg](https://github.com/kreuzberg-dev/kreuzberg) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 8.2k+ | +| License | NOASSERTION | +| Rating | ⭐⭐⭐⭐ | + +**What it does**: 97+ document format parser framework, Rust core. Provides MCP server + REST API + CLI. +**Audience**: cross-format batch parsing engineers who care about throughput. +**Notes**: covers obscure formats like HWP, ODT, etc., not just PDF / Office. + +--- + +## 3. Google Workspace + +### [taylorwilsdon/google_workspace_mcp](https://github.com/taylorwilsdon/google_workspace_mcp) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 2.3k+ | +| License | MIT | +| Rating | ⭐⭐⭐⭐⭐ (one server, all of Google) | + +**What it does**: Gmail, Calendar, Docs, Sheets, Slides, Drive, Chat, Forms, Tasks, Search — all in one MCP server. +**Audience**: heavy Google Workspace users — replying to email, scheduling, writing docs, manipulating sheets, all from one server. +**Notes**: OAuth setup is a bit involved but only needs to be done once; most complete coverage of Google's tools. + +### [xing5/mcp-google-sheets](https://github.com/xing5/mcp-google-sheets) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 844 | +| License | MIT | +| Rating | ⭐⭐⭐⭐ (Sheets-only) | + +**What it does**: focused Google Sheets / Drive integration — create sheets, edit cells, query formulas. +**Audience**: people using only Google Sheets who don't want the full Workspace MCP. +**Notes**: narrower scope than `google_workspace_mcp`, but simpler setup. + +--- + +## 4. Microsoft 365 + +### [Softeria/ms-365-mcp-server](https://github.com/Softeria/ms-365-mcp-server) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 681 | +| License | MIT | +| Rating | ⭐⭐⭐⭐ (full M365) | + +**What it does**: M365 + Office services via Microsoft Graph API — Outlook, Teams, OneDrive, SharePoint. +**Audience**: enterprise M365 users wanting LLM-driven email replies, calendar lookups, OneDrive operations. +**Notes**: requires Azure AD app registration; corporate IT policies may block this. + +### [ryaker/outlook-mcp](https://github.com/ryaker/outlook-mcp) ⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 363 | +| License | NOASSERTION | +| Rating | ⭐⭐⭐ (Outlook only) | + +**What it does**: Outlook mail / calendar via Graph API. +**Audience**: people who only need Outlook, not the rest of M365. +**Notes**: narrower scope than `ms-365-mcp-server`. + +### [merill/lokka](https://github.com/merill/lokka) ⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 244 | +| License | MIT | +| Rating | ⭐⭐⭐ | + +**What it does**: M365 + Microsoft Graph admin operations — Entra (AD), Intune, etc. +**Audience**: M365 system admins managing tenants / users / policies. +**Notes**: more useful for IT admins than end users. + +--- + +## 5. Dev Collaboration (GitHub / Atlassian / Slack…) + +### [github/github-mcp-server](https://github.com/github/github-mcp-server) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 29.5k+ | +| License | MIT | +| Rating | ⭐⭐⭐⭐⭐ (**official**) | + +**What it does**: GitHub's official MCP — issues / PRs / repos / Actions / Codespaces. +**Audience**: every GitHub user; once Claude Code is wired up, PR review, issue triage, release notes all work. +**Notes**: **must-install for Track A's A3 Exercise CLI-9**. + +### [sooperset/mcp-atlassian](https://github.com/sooperset/mcp-atlassian) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 5.1k+ | +| License | MIT | +| Rating | ⭐⭐⭐⭐⭐ (most popular community Atlassian) | + +**What it does**: Confluence + Jira in one MCP, more flexible than the official remote. +**Audience**: people using Atlassian who find the official remote server too restrictive. +**Notes**: pick this OR `atlassian/atlassian-mcp-server` (official) depending on your IT policy. + +### [atlassian/atlassian-mcp-server](https://github.com/atlassian/atlassian-mcp-server) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 650+ | +| License | Apache-2.0 | +| Rating | ⭐⭐⭐⭐ (**official**) | + +**What it does**: Atlassian's official Remote MCP, secure connection to Jira / Confluence. +**Audience**: companies with enterprise Atlassian + IT policies requiring official tooling. +**Notes**: remote model with official SLA. + +### [korotovsky/slack-mcp-server](https://github.com/korotovsky/slack-mcp-server) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 1.6k+ | +| License | MIT | +| Rating | ⭐⭐⭐⭐ (no admin permissions needed) | + +**What it does**: Slack MCP — DMs, group DMs, channel messages, with built-in history fetch logic. +**Audience**: individual users (not Slack admins) who still want LLM-Slack integration. +**Notes**: doesn't need admin tokens; uses user-level OAuth. + +### [jerhadf/linear-mcp-server](https://github.com/jerhadf/linear-mcp-server) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 344 | +| License | NOASSERTION | +| Rating | ⭐⭐⭐⭐ | + +**What it does**: Linear (issue tracker) MCP — query issues, create issues, change status. +**Audience**: developers managing sprints / backlogs in Linear. +**Notes**: requires Linear API key. + +### [SaseQ/discord-mcp](https://github.com/SaseQ/discord-mcp) ⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 298 | +| License | MIT | +| Rating | ⭐⭐⭐ | + +**What it does**: Discord MCP — read/write channel messages, manage servers. +**Audience**: maintainers running OSS / community Discord servers. +**Notes**: requires Discord bot token; watch rate limits. + +### [safishamsi/graphify](https://github.com/safishamsi/graphify) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 44k+ | +| License | MIT | +| Rating | ⭐⭐⭐⭐⭐ | + +**What it does**: AI coding skill that turns codebases / SQL schemas / R scripts / shell scripts / docs / papers / images / videos into a queryable knowledge graph. Works across Claude Code, Codex, OpenCode, Cursor, Gemini CLI. +**Audience**: engineers / researchers analyzing large codebases, tracking cross-file references, or asking questions across "app code + DB schema + infra" together. +**Notes**: cross-cutting tool — fits both dev collaboration (understanding existing codebases) and research workflow (turning any artifact into a graph). When stuck on a big codebase, use graphify to extract structure, then feed it back to Claude for reasoning. + +### [upstash/context7](https://github.com/upstash/context7) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 57k | +| License | MIT | +| Rating | ⭐⭐⭐⭐⭐ (must-have for coding) | + +**What it does**: pulls up-to-date library / framework docs into the agent's context so it stops hallucinating outdated APIs — one of the most-installed coding MCPs. +**Audience**: developers who hit "the LLM wrote code against a stale version of this library" — Context7 feeds the agent current docs on demand. +**Notes**: among the highest-starred coding MCPs; reach for it whenever the agent's library knowledge lags the version you're actually on. + +### [DeusData/codebase-memory-mcp](https://github.com/DeusData/codebase-memory-mcp) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 13.5k+ | +| License | MIT | +| Rating | ⭐⭐⭐⭐⭐ (code intelligence) | + +**What it does**: indexes a codebase into a queryable knowledge graph, so a coding agent can ask about structure / symbols / call paths instead of repeatedly grepping and reading files. Single static binary, 158 languages. +**Audience**: people running coding agents on large or unfamiliar repos who want fast orientation and lower token use. +**Notes**: re-index after big edits, since the graph can go stale; treat its answers as a fast first pass and verify load-bearing claims (who-calls-X / is-this-dead) against the actual code. + +--- + +## 6. Databases + +### [googleapis/mcp-toolbox](https://github.com/googleapis/mcp-toolbox) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 15k+ | +| License | Apache-2.0 | +| Rating | ⭐⭐⭐⭐⭐ (**Google official**, multi-DB) | + +**What it does**: cross-DB MCP server — MySQL / PostgreSQL / Cloud SQL / Spanner / BigQuery. +**Audience**: engineers running databases on Google Cloud, or anyone needing multi-engine support. +**Notes**: open-source + Google-maintained; solid choice for production use. + +### [bytebase/dbhub](https://github.com/bytebase/dbhub) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 2.7k+ | +| License | MIT | +| Rating | ⭐⭐⭐⭐⭐ (community multi-DB) | + +**What it does**: zero-dependency, token-efficient multi-DB MCP — Postgres, MySQL, SQL Server, MariaDB, SQLite. +**Audience**: engineers who don't want the Google Cloud SDK and need cross-OSS-DB support. +**Notes**: overlaps with `googleapis/mcp-toolbox` but lighter weight. + +### [supabase-community/supabase-mcp](https://github.com/supabase-community/supabase-mcp) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 2.7k+ | +| License | Apache-2.0 | +| Rating | ⭐⭐⭐⭐⭐ (**Supabase official-community**) | + +**What it does**: connect Supabase (Postgres, Auth, Storage, Edge Functions) to LLMs. +**Audience**: full-stack devs using Supabase as backend. +**Notes**: official community-maintained. + +### [timescale/pg-aiguide](https://github.com/timescale/pg-aiguide) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 1.7k+ | +| License | Apache-2.0 | +| Rating | ⭐⭐⭐⭐ (Postgres coding aid) | + +**What it does**: MCP server + Claude plugin to help LLMs write better PostgreSQL code. +**Audience**: Postgres-heavy SQL writers / DBAs. +**Notes**: focused on "LLM writes better SQL", not just query execution. + +### [benborla/mcp-server-mysql](https://github.com/benborla/mcp-server-mysql) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 1.6k+ | +| License | MIT | +| Rating | ⭐⭐⭐⭐ (read-only MySQL) | + +**What it does**: read-only MySQL MCP — let the LLM see schemas, run queries. +**Audience**: scenarios where the LLM should analyze production DBs but never modify them. +**Notes**: read-only is a safety feature, not a limitation. + +### [mongodb-js/mongodb-mcp-server](https://github.com/mongodb-js/mongodb-mcp-server) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 1k+ | +| License | Apache-2.0 | +| Rating | ⭐⭐⭐⭐ (**MongoDB official**) | + +**What it does**: MongoDB and MongoDB Atlas Cluster MCP server. +**Audience**: engineers using MongoDB / Atlas. +**Notes**: `mongodb-js` is MongoDB's official GitHub org. + +### [redis/mcp-redis](https://github.com/redis/mcp-redis) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 504 | +| License | MIT | +| Rating | ⭐⭐⭐⭐ (**Redis official**) | + +**What it does**: official Redis MCP — natural-language operations on Redis and Redis Stack (Vector / Search / JSON). +**Audience**: people using Redis as cache / vector DB / queue. +**Notes**: officially maintained; includes vector search. + +### [awslabs/mcp](https://github.com/awslabs/mcp) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 9.3k+ | +| License | Apache-2.0 | +| Rating | ⭐⭐⭐⭐ (**AWS official**) | + +**What it does**: AWS's first-party MCP servers (Lambda / S3 / DynamoDB / CloudWatch / Cost Explorer and more). +**Audience**: teams on AWS who want agents to query / operate their cloud. +**Notes**: officially maintained by AWS; uses your existing AWS login (CLI profiles / IAM roles), no separate token to manage. + +--- + +## 7. Browser Automation / Web Scraping + +### [microsoft/playwright-mcp](https://github.com/microsoft/playwright-mcp) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 34k+ | +| License | Apache-2.0 | +| Rating | ⭐⭐⭐⭐⭐ (**Microsoft official**) | + +**What it does**: Playwright MCP server — let the LLM open browsers, click buttons, fill forms, scrape pages. +**Audience**: anyone doing E2E automation, cross-site integration, scraping behind logins. +**Notes**: official Playwright; most robust. **First choice for Claude Code + web automation**. + +### [ChromeDevTools/chrome-devtools-mcp](https://github.com/ChromeDevTools/chrome-devtools-mcp) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 38k+ | +| License | Apache-2.0 | +| Rating | ⭐⭐⭐⭐⭐ (**Chrome official**) | + +**What it does**: expose Chrome DevTools to coding agents — performance, network, console traces all available to the LLM. +**Audience**: developers debugging frontend bugs, doing web performance analysis. +**Notes**: pairs perfectly with Playwright MCP — one drives, one observes. + +### [firecrawl/firecrawl-mcp-server](https://github.com/firecrawl/firecrawl-mcp-server) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 6.2k+ | +| License | MIT | +| Rating | ⭐⭐⭐⭐⭐ (**Firecrawl official**) | + +**What it does**: Firecrawl's official MCP — large-scale web scraping + search + structured extraction. +**Audience**: people scraping large amounts of web data for training / RAG / research. +**Notes**: requires Firecrawl API key (has a free tier). + +### [browserbase/mcp-server-browserbase](https://github.com/browserbase/mcp-server-browserbase) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 3.3k+ | +| License | Apache-2.0 | +| Rating | ⭐⭐⭐⭐ (**Browserbase official**) | + +**What it does**: Browserbase's official MCP, paired with Stagehand for cloud-based browser automation. +**Audience**: people whose local browser automation is too heavy / who need parallel cloud sessions. +**Notes**: commercial service (free tier exists); complementary to Playwright MCP (local vs cloud). + +--- + +## 8. Design (Figma / Excalidraw) + +### [GLips/Figma-Context-MCP](https://github.com/GLips/Figma-Context-MCP) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 14.6k+ | +| License | MIT | +| Rating | ⭐⭐⭐⭐⭐ (most popular Figma MCP) | + +**What it does**: feed Figma layout info to coding agents — read design files, expose component structure, let Cursor / Claude Code generate matching React components. +**Audience**: front-end devs going from Figma designs to component code. +**Notes**: requires Figma access token; must-install for design-to-code workflows. + +### [excalidraw/excalidraw-mcp](https://github.com/excalidraw/excalidraw-mcp) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 4.3k+ | +| License | NOASSERTION | +| Rating | ⭐⭐⭐⭐⭐ (**Excalidraw official**) | + +**What it does**: streamable Excalidraw MCP — let LLMs draw architecture diagrams and flowcharts directly. +**Audience**: anyone writing design docs / system architecture / flowcharts who wants Claude to draw from text. +**Notes**: official Excalidraw; output imports straight into Excalidraw for editing. + +### [yctimlin/mcp_excalidraw](https://github.com/yctimlin/mcp_excalidraw) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 1.9k+ | +| License | MIT | +| Rating | ⭐⭐⭐⭐ (alternative Excalidraw) | + +**What it does**: MCP server + Claude Code Skill, real-time canvas sync, create / edit / export. +**Audience**: people who need real-time canvas sync and programmatic operation. +**Notes**: complementary to the official; community-maintained. + +### [pbakaus/impeccable](https://github.com/pbakaus/impeccable) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 25k+ | +| License | Apache-2.0 | +| Rating | ⭐⭐⭐⭐⭐ | + +**What it does**: "**The design language that makes your AI harness better at design.**" A vocabulary / pattern set that helps AI agents produce UI / visual output that escapes the generic "AI-generated" feel. +**Audience**: developers using AI to generate UI / mockups / visual designs but getting generic results; front-end + AI workflows. +**Notes**: not an MCP server or Skill bundle — it's a **design language** reference. Feed AI the higher-quality design vocabulary and it produces better output. + +--- + +## 9. Monitoring / Observability + +### [grafana/mcp-grafana](https://github.com/grafana/mcp-grafana) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 3k+ | +| License | Apache-2.0 | +| Rating | ⭐⭐⭐⭐⭐ (**Grafana official**) | + +**What it does**: Grafana's official MCP — query dashboards / metrics / alerts from the LLM. +**Audience**: SREs / DevOps using Grafana for metrics. +**Notes**: "why did this dashboard line drop?" — ask, and the LLM pulls metrics for the answer. + +### [getsentry/sentry-mcp](https://github.com/getsentry/sentry-mcp) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 677 | +| License | NOASSERTION | +| Rating | ⭐⭐⭐⭐ (**Sentry official**) | + +**What it does**: query Sentry error events / issues / traces from LLMs. +**Audience**: engineers using Sentry for production errors. +**Notes**: "show me last week's stack trace for this error" works directly in Claude Code. + +### [winor30/mcp-server-datadog](https://github.com/winor30/mcp-server-datadog) ⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 142 | +| License | Apache-2.0 | +| Rating | ⭐⭐⭐ (community Datadog) | + +**What it does**: Datadog API MCP — monitors / logs / metrics. +**Audience**: Datadog users while there's no official Datadog MCP yet. +**Notes**: likely to be replaced once Datadog ships an official MCP. + +--- + +## 10. Media / Streaming (YouTube / Spotify) + +### [varunneal/spotify-mcp](https://github.com/varunneal/spotify-mcp) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 599 | +| License | MIT | +| Rating | ⭐⭐⭐⭐ | + +**What it does**: connect LLMs to Spotify — play tracks, manage playlists, query history. +**Audience**: anyone integrating playback control or text → music workflows with Claude Code. +**Notes**: requires Spotify Premium (API restriction). + +### [kimtaeyoon83/mcp-server-youtube-transcript](https://github.com/kimtaeyoon83/mcp-server-youtube-transcript) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 534 | +| License | MIT | +| Rating | ⭐⭐⭐⭐ (YouTube transcripts) | + +**What it does**: pull YouTube video transcripts into the LLM for summary / translation / RAG. +**Audience**: people using video as study material, batch-summarizing YouTube content. +**Notes**: depends on YouTube auto-captions; non-English transcripts are hit-or-miss. + +### [ZubeidHendricks/youtube-mcp-server](https://github.com/ZubeidHendricks/youtube-mcp-server) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 510 | +| License | NOASSERTION | +| Rating | ⭐⭐⭐⭐ (full YouTube API) | + +**What it does**: full YouTube API MCP — beyond transcripts, also video management, Shorts, analytics. +**Audience**: YouTube creators automating channel management. +**Notes**: requires YouTube Data API key + OAuth. + +--- + +## 11. Chinese-language Ecosystem + +### [leemysw/feishu-docx](https://github.com/leemysw/feishu-docx) ⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 193 | +| License | MIT | +| Rating | ⭐⭐⭐ | + +**What it does**: bidirectional Feishu (Lark) docs / sheet / bitable ↔ Markdown, with OAuth 2.0, CLI, TUI, Claude Skills. +**Audience**: Chinese-language users on Feishu / Lark wanting to bridge Lark content with Claude Code. +**Notes**: currently one of the few MCP / Skill options in the Chinese ecosystem; WeChat / DingTalk don't have standalone MCPs yet (they live inside chatbot frameworks). + +### [netease-youdao/LobsterAI](https://github.com/netease-youdao/LobsterAI) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 5k+ | +| License | MIT | +| Rating | ⭐⭐⭐⭐ | + +**What it does**: NetEase Youdao's "24/7 all-scenario AI agent" — workflow automation, cross-app coordination, file processing. Chinese-native. +**Audience**: Chinese-language users wanting an alternative to Claude Code / OpenAI Operator-class all-in-one agents; scenarios needing tight integration with mainland Chinese services (NetEase, DingTalk, etc.). +**Notes**: product-style agent (not a Skill / MCP); substitutes for Claude Code / Codex rather than complementing them. + +### [QwenLM/Qwen-Agent](https://github.com/QwenLM/Qwen-Agent) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 16k+ | +| License | Apache-2.0 | +| Rating | ⭐⭐⭐⭐⭐ | + +**What it does**: Alibaba's official Qwen agent framework — RAG, tool use, code interpreter, multi-agent, MCP-compatible. Defaults to Qwen models but swappable to other LLMs. +**Audience**: developers using Qwen / Tongyi as primary LLM; teams that want a Chinese-native agent framework (examples + docs are bilingual but Chinese-first). +**Notes**: MCP compatibility is the highlight — plugs into Claude Code-style hosts directly; active maintenance (last commit 2026-03). + +### [coze-dev/coze-studio](https://github.com/coze-dev/coze-studio) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 20k+ | +| License | Apache-2.0 | +| Rating | ⭐⭐⭐⭐⭐ | + +**What it does**: open-source release of ByteDance Coze — no-code agent builder (workflow / plugin / knowledge / memory), self-hosted or cloud. +**Audience**: teams building agents without writing code; engineers wanting a reference implementation of an enterprise agent platform (RAG, workflow, memory, plugin system). +**Notes**: built on Coze's in-house Eino framework; connects to OpenAI / Claude / Qwen / domestic Chinese LLMs. Powers both the international (coze.com) and mainland (coze.cn) products. + +### [coze-dev/coze-loop](https://github.com/coze-dev/coze-loop) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 5k+ | +| License | Apache-2.0 | +| Rating | ⭐⭐⭐⭐ | + +**What it does**: Coze's agent observability + evaluation platform — trace, debug, eval, prompt management. The back half of the agent dev lifecycle. +**Audience**: teams whose agents are running in production and need monitoring; developers wanting to see how "agent eval / observability" can be designed. +**Notes**: peer to LangSmith / Arize Phoenix; OSS release is self-hostable. + +### [liaokongVFX/LangChain-Chinese-Getting-Started-Guide](https://github.com/liaokongVFX/LangChain-Chinese-Getting-Started-Guide) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 8.9k+ | +| License | unspecified | +| Rating | ⭐⭐⭐⭐ | + +**What it does**: Chinese-language LangChain getting-started guide — covers basics, prompts, memory, agents, chains, and applied examples. The earliest and most complete LangChain Chinese learning resource. +**Audience**: Chinese-language users who want LangChain but find the English docs heavy; readers who want to understand LangChain's design before committing to the framework. +**Notes**: no formal license (content is openly readable); LangChain itself moves fast — some APIs in the guide may diverge from the latest version. + +### [chatchat-space/Langchain-Chatchat](https://github.com/chatchat-space/Langchain-Chatchat) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 37k+ | +| License | Apache-2.0 | +| Rating | ⭐⭐⭐⭐ | + +**What it does**: LangChain-based open-source knowledge-base QA system — local deployment, supports multiple vector stores, end-to-end RAG example. +**Audience**: Chinese teams who want RAG without building it from scratch; scenarios requiring local-only deployment (no cloud LLM). +**Notes**: ★ 37k makes it the most popular RAG implementation in the Chinese ecosystem; maintenance has slowed (last commit 2025-11). For new projects, fork and evaluate as a reference, not a turnkey base. + +### [usewhale/whale](https://github.com/usewhale/whale) ⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 117 | +| License | MIT | +| Rating | ⭐⭐⭐ | + +**What it does**: Terminal AI coding assistant optimized for DeepSeek models — supports MCP server integration, Claude-style Skills, conversation caching, written in Go. +**Audience**: Chinese developers who use DeepSeek as their primary LLM; those who want a terminal tool without the full Claude Code stack. +**Notes**: One of the few open-source tools with DeepSeek-specific optimization; MCP + Skills dual support allows incremental capability expansion. + +### [simonlin1212/a-stock-data](https://github.com/simonlin1212/a-stock-data) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 492 | +| License | Apache-2.0 | +| Rating | ⭐⭐⭐⭐ | + +**What it does**: China A-share market data toolkit — a single SKILL.md file wrapping 8 data sources (mootdx, EastMoney, akshare, iwencai, etc.) with 21 endpoints, directly usable by AI coding assistants. +**Audience**: Chinese developers using Claude Code / Codex / OpenClaw for investment research or quantitative analysis; those who don't want to build data-fetching logic from scratch. +**Notes**: Installable with a single `curl` + `pip install`; highest-starred community Skill for Chinese A-share data. Compatible with Claude Code, Codex, and OpenClaw. + +> Looking for WeChat / DingTalk integrations? Today the mainstream is chatbot frameworks (e.g., zhayujie/CowAgent), not pure MCP servers. Will add when proper MCPs emerge. + +### [MoonshotAI/Kimi-K2](https://github.com/MoonshotAI/Kimi-K2) ⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 10.7k+ | +| License | Modified MIT | +| Rating | ⭐⭐⭐ | + +**What it does**: Moonshot's Kimi K2 open-weight LLM series — open weights + OpenAI/Anthropic-compatible API, oriented toward agentic / coding / long-horizon tasks; usable as a backend model for an agent stack. +**Audience**: Chinese developers who want to run agent / coding workflows on a domestic open model, or to self-host open weights. +**Notes**: License is Modified MIT (standard MIT + added large-scale-commercial clauses) — read the original LICENSE before commercial use; weights are also available on Hugging Face. + +### [zai-org/GLM-4.5](https://github.com/zai-org/GLM-4.5) ⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 4.3k+ | +| License | Apache-2.0 | +| Rating | ⭐⭐⭐ | + +**What it does**: Zhipu (Z.ai)'s GLM-4.5 open model — positioned as Agentic, Reasoning, and Coding (ARC) foundation models; open weights + API, usable as a backend for agent / tool use / coding. +**Audience**: Chinese developers evaluating domestic open agentic models, or who need weights under a permissive license (Apache-2.0). +**Notes**: zai-org is Zhipu's open-source org; the same series also has GLM-4 (★ 7k+) for context; weights are on Hugging Face. + +--- + +## 12. Other Common (Cloudflare / Stripe…) + +### [cloudflare/mcp-server-cloudflare](https://github.com/cloudflare/mcp-server-cloudflare) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 3.7k+ | +| License | Apache-2.0 | +| Rating | ⭐⭐⭐⭐⭐ (**Cloudflare official**) | + +**What it does**: Cloudflare's official MCP — Workers, Pages, R2, KV, D1, DNS, Zero Trust. +**Audience**: anyone running edge / serverless on Cloudflare. +**Notes**: officially maintained; the best edge platform MCP. + +### [stripe/ai](https://github.com/stripe/ai) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 1.5k+ | +| License | MIT | +| Rating | ⭐⭐⭐⭐ (**Stripe official**) | + +**What it does**: Stripe's official AI agent toolkit, includes an MCP server — handle payments, subscriptions, refunds, customers. +**Audience**: developers wiring payment / billing into agent flows. +**Notes**: ⚠️ this is real money. Test thoroughly in sandbox before going to production. + +### YIELD INTELLIGENCE MCP (Hosted Remote Server) + +| Field | Value | +|---|---| +| Type | hosted MCP server | +| Rating | ⭐⭐⭐ (finance analysis tool; practical example of hosted vs self-hosted MCP architecture) | + +**What it does**: YIELD INTELLIGENCE hosted remote MCP server — live US Treasury yield rates, dividend ETF / REIT / preferred stock analysis, and passive income portfolio optimization. Two tools: `analyze_yield_opportunities` (scans passive income options) + `optimize_income_portfolio` (builds a portfolio toward a target monthly income). Listed in the Anthropic official MCP Registry (`io.github.thebrierfox/intuitek-ace`, since 2026-05-10). +**Audience**: people doing personal finance analysis in Claude Code / Claude Desktop who want AI to surface passive income opportunities. Good hands-on example of a hosted remote MCP server — plug the URL in, zero install, useful for Stage 5 learners exploring the hosted vs self-hosted difference. +**Notes**: Live endpoint `https://api.intuitek.ai/yield/mcp` (no auth, no API key required). x402 micropayment $1 USDC/call on Base (agent-to-agent scenarios); free for regular users. Analysis-only, no trading. GitHub: [thebrierfox/intuitek-ace](https://github.com/thebrierfox/intuitek-ace) (MIT License). + +### [ComposioHQ/composio](https://github.com/ComposioHQ/composio) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 28.8k+ | +| License | MIT | +| Rating | ⭐⭐⭐⭐⭐ (1000+ tool-integration hub) | + +**What it does**: a platform (SDKs + MCP servers) that connects agents to 1000+ apps (Slack / GitHub / Gmail / Salesforce / Notion…) and handles the logins for you, so you don't build a separate connector for each one. +**Audience**: teams whose agents need broad API coverage without maintaining dozens of separate MCP servers. +**Notes**: provides MCP servers + Python / TypeScript SDKs; connect to Claude Code via MCP. A "tool aggregator" (compare with n8n / Zapier for automation). + +--- + +## 13. Research Workflow Skills (academic / paper / lit) + +> ⚠️ **Maintainer's own projects**: the following are skills the repo maintainer [@WenyuChiou](https://github.com/WenyuChiou) (Lehigh CEE PhD candidate) uses daily for research and open-sourced for other researchers. **Star counts are lower than general-purpose tools** because these are niche / research-specific. The ★ 100+ inclusion floor is relaxed in this section — the only criterion here is "actually useful in the maintainer's research workflow". Evaluate fit yourself. + +### [WenyuChiou/ai-research-skills](https://github.com/WenyuChiou/ai-research-skills) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 60 | +| License | MIT | +| Rating | ⭐⭐⭐⭐⭐ (full research workflow) | + +**What it does**: 14 Claude Code skills covering common research tasks — literature triage, research design, project context, manuscript writing, multi-AI delegation. Packaged as a 5-plugin marketplace, install with one command. +**Audience**: grad students / postdocs wanting a complete "research workflow" skill set in one drop. +**Notes**: marketplace format, aligns with the plugin/marketplace concept taught in Stage 5.4. + +### [WenyuChiou/academic-writing-skills](https://github.com/WenyuChiou/academic-writing-skills) ⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 2 | +| License | MIT | +| Rating | ⭐⭐⭐ (narrow but deep) | + +**What it does**: rigorous academic paper writing / revision / submission skill for Claude Code. Field-agnostic, customizable per-paper via journal_format.md and style_overrides.md. +**Audience**: researchers actively writing / revising papers who want to automate banned-word audit, figure-text coupling, submission checklists. +**Notes**: one of the 5 plugins inside ai-research-skills; can also be installed standalone. + +### [WenyuChiou/zotero-skills](https://github.com/WenyuChiou/zotero-skills) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 16 | +| License | NOASSERTION | +| Rating | ⭐⭐⭐⭐ | + +**What it does**: Zotero CLI skill — programmatically search, add, classify, annotate references. +**Audience**: Zotero users wanting Claude Code to organize their library directly. +**Notes**: complementary to [`MuiseDestiny/zotero-gpt`](https://github.com/MuiseDestiny/zotero-gpt) — that one is a Zotero plugin (chat inside Zotero), this one is a CLI / Skill (operate Zotero from Claude Code). + +### [WenyuChiou/research-hub](https://github.com/WenyuChiou/research-hub) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 14 | +| License | MIT | +| Rating | ⭐⭐⭐⭐ | + +**What it does**: AI-operable research workspace bridging Zotero + Obsidian + NotebookLM, with CLI / MCP / REST / dashboard interfaces. +**Audience**: researchers using Zotero / Obsidian / NotebookLM together, wanting to bind them into one workspace for LLMs to operate. +**Notes**: complementary to single-tool MCPs (mcp-obsidian, notion-mcp, etc.) — this is a hub that integrates multiple tools. + +--- + +## 14. Multi-LLM Delegation Skills + +> ⚠️ **Maintainer's own projects** (same as 13): delegation skills the maintainer extracted from daily workflow. Star floor is relaxed; criterion is "the Claude-planner + Codex/Gemini-executor combo runs reliably". Multi-LLM space evolves quickly — evaluate alongside the multi-agent frameworks listed in Stage 7 before adopting. + +### How the three skills compose + +The 3 skills below are **designed to be used together**, not as standalone tools: + +![Claude + 3 delegate skills — division of labor](../resources/diagrams/multi-llm-delegation-composition.en.png) + +Claude is bad at token-heavy mechanical work (cost, context blowout); Codex is bad at conversational coordination; Gemini's 1M context is great but mid-tier reasoning. **Division of labor: Claude handles design / review, Codex handles implementation, Gemini handles long-form drafting / synthesis.** + +### [WenyuChiou/codex-delegate](https://github.com/WenyuChiou/codex-delegate) ⭐⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 57 | +| License | MIT | +| Rating | ⭐⭐⭐⭐⭐ | + +**What it does**: Claude Code skill that uses Codex CLI as the execution specialist — multi-file refactors, batch edits, boilerplate generation, wrapper-based implementation tasks. Claude writes the plan + reviews; Codex executes. +**Audience**: developers wanting to save tokens / accelerate large-scale mechanical edits; learners who want to verify "multi-agent isn't just a buzzword". +**Use it for**: refactoring 30+ files, generating test scaffolds, porting the same pattern across N files, writing migration scripts. +**Don't use for**: architecture decisions, bug diagnosis, security review, tasks needing conversation memory — Claude does these better directly. +**Notes**: pairs with `gemini-delegate-skill`. Practical implementation of the Stage 7 multi-agent concept. + +### [WenyuChiou/gemini-delegate-skill](https://github.com/WenyuChiou/gemini-delegate-skill) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 34 | +| License | MIT | +| Rating | ⭐⭐⭐⭐ | + +**What it does**: Claude Code skill that uses Gemini CLI as the long-form / large-context / CJK executor — 1M-token context window, Chinese long-form drafting, second-opinion review. Claude provides the outline and critique; Gemini writes the long form. +**Audience**: researchers writing papers, knowledge workers writing Chinese reports / Threads posts, people who want a second LLM's perspective for cross-checking. +**Use it for**: long-form drafts (>3000 words), cross-document synthesis (stuffing many long docs into the 1M-token context), Chinese / CJK content, LLM-vs-LLM comparison views. +**Don't use for**: short queries, code generation (use codex), production-critical decisions (final human review). +**Notes**: pairs with `codex-delegate` for the "Codex writes code, Gemini writes prose" split. + +### [WenyuChiou/agent-collab-skills](https://github.com/WenyuChiou/agent-collab-skills) ⭐⭐ + +| Field | Value | +|---|---| +| Stars | recently published, no stars yet | +| License | MIT | +| Rating | ⭐⭐ (experimental — treat as reference) | + +**What it does**: Claude Code marketplace for multi-agent collaboration — task splitter, output reconciler, adversarial debate, shared memory, acceptance gate. Composes with codex-delegate / gemini-delegate. +**Audience**: people running 2+ delegate agents per round who want to see one way of packaging multi-agent coordination into a marketplace. +**Notes**: **experimental** — don't treat this as a framework ready for production use. It's the maintainer's own setup made public as a reference. For multi-agent frameworks built for production, see LangGraph / AutoGen / CrewAI in Stage 7. + +--- + +## 15. Finance / Trading Agents + +> ⚠️ **Application-domain section**: agents applied to quantitative trading, hedge-fund simulation, and automated order placement. Licensing varies (NO-LICENSE to permissive open-source); verify each repo before reuse. **Caveat**: real-money trading agents carry significant risk; listed here for agent-design study, not as investment advice. + +### [TauricResearch/TradingAgents](https://github.com/TauricResearch/TradingAgents) ⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 79k+ | +| License | Apache-2.0 | +| Rating | ⭐⭐⭐ | + +**What it does**: a multi-agent LLM framework for financial trading decisions, with bull / bear / fundamentals / technicals / risk agents collaborating. +**Audience**: learners studying how multi-agent systems split analytical work; quant researchers experimenting with LLM augmentation of existing pipelines. +**Notes**: Apache-2.0 — modification and commercial use permitted (retain license notice). **Not investment advice — do not run on real funds directly.** + +### [virattt/ai-hedge-fund](https://github.com/virattt/ai-hedge-fund) ⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 59k+ | +| License | NO-LICENSE | +| Rating | ⭐⭐⭐ | + +**What it does**: a multi-role AI hedge-fund simulation where bull / bear / fundamentals / technicals / risk agents collaborate to produce trade recommendations. +**Audience**: Stage 7 multi-agent learners wanting a complete application example; people interested in the agent × finance crossover. +**Notes**: NO-LICENSE → same caveat as above. **Simulation only — not investment advice.** + +--- + +## 16. Web Search / Retrieval + +### [exa-labs/exa-mcp-server](https://github.com/exa-labs/exa-mcp-server) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 4.5k+ | +| License | MIT | +| Rating | ⭐⭐⭐⭐ (**Exa official**) | + +**What it does**: Exa's official MCP — LLM / agent-oriented web search (neural + keyword) that returns clean results to feed straight into prompts. +**Audience**: people doing research / fact-check / online RAG retrieval — semantic search shines for "concept-related" queries, less so for pure keyword lookups. +**Notes**: requires an Exa API key. + +### [tavily-ai/tavily-mcp](https://github.com/tavily-ai/tavily-mcp) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Stars | ★ 2.1k+ | +| License | MIT | +| Rating | ⭐⭐⭐⭐ (beginner-friendly web search) | + +**What it does**: Tavily's search API as an MCP — web search built for LLMs / RAG, returning an answer plus its sources. +**Audience**: a beginner who just wants the agent to search the web — good first pick thanks to an easy free tier. +**Notes**: free tier available; returns both a synthesized answer and the underlying sources. + +--- + +## What's not here? + +If your integration isn't above, check these catalogs first: + +- [`wong2/awesome-mcp-servers`](https://github.com/wong2/awesome-mcp-servers) — most complete community MCP server catalog, 150+ entries by category +- [`punkpeye/awesome-mcp-servers`](https://github.com/punkpeye/awesome-mcp-servers) — another MCP server catalog, complementary +- [`modelcontextprotocol/servers`](https://github.com/modelcontextprotocol/servers) — Anthropic's official reference servers (filesystem, git, time, memory, fetch, sequential-thinking, …) +- [`travisvn/awesome-claude-skills`](https://github.com/travisvn/awesome-claude-skills) — Claude Skills catalog + +### Want to add something? + +1. Open an issue with the repo link, why it should be added, and which category it fits. +2. Or PR directly: add an entry under the relevant category in this format (Stars / License / Rating + What it does / Audience / Notes). +3. **Stars < 100 + non-official** typically gets rejected unless you can argue a strong niche use case. + +Read [`resources/style-guide.md`](style-guide.en.md) and [`CONTRIBUTING.md`](../CONTRIBUTING.md) before submitting. + +--- + +## Notes for anyone helping out later + +Not an SLA — just "do what you can" guidance: + +- Pull stars / license via `gh api repos//`. **Refresh whenever you have time** — no fixed cadence. +- Spot a broken link / archived repo? Just remove it. +- New category (AR/VR, IoT, etc.) — open it once you have 1-2 entries worth listing. +- "Chinese-language ecosystem" stays loose; Chinese-community repos accumulate stars more slowly. +- Inconsistent wording or formatting between entries — don't sweat it. Readability of the PR comes first. diff --git a/resources/mcp-skills-catalog.md b/resources/mcp-skills-catalog.md new file mode 100644 index 0000000..2b505e2 --- /dev/null +++ b/resources/mcp-skills-catalog.md @@ -0,0 +1,1066 @@ +# MCP / Skills 整合目錄 + +> **繁體中文** | [简体中文](./mcp-skills-catalog.zh-Hans.md) | [English](./mcp-skills-catalog.en.md) + +> 把 Claude Code（或其他 CLI agent）接到你已經在用的工具，不用反覆切換視窗。本頁是 65+ 個分類整理過的 MCP server / Claude Skill / 整合範例（含研究工作流 + multi-LLM delegation 兩個專屬區）。 + +--- + +## 怎麼用這份目錄 + +- **想找特定工具的 MCP**：直接看下面分類目錄 +- **想看 MCP / Skills / Plugins 是什麼**：先看 [README 三個核心用語](../README.md#-快速開始)，再看 [Stage 5 — Claude Code 生態系](../stages/05-claude-code-ecosystem.md) +- **想看 動手練習 怎麼裝、怎麼測**：看 [Stage 5.2 (MCP)](../stages/05-claude-code-ecosystem.md#52--mcpmodel-context-protocol-基礎) 跟 [Stage 5.3 (Skills)](../stages/05-claude-code-ecosystem.md#53--skillsclaude-code-的行為層-claude-code-生態最關鍵的一層) + +### 收錄方向（不是死規則） + +- **官方優先**：Anthropic、廠商自己出的 MCP / Skill 通常排前面 +- **stars 看一下就好**：社群 repo 大致 100+ 比較有人在維護，但「小眾但好用」也歡迎送 PR 解釋為什麼要收 +- **盡量有 metadata**：stars / license 用 `gh api` 抓、有空就更新一輪 +- **避免（不是禁止）**：archived、長期沒 commit、license 不明的 repo——niche 工具可以例外 + +### 目錄 + +1. [筆記 / 知識庫](#1-筆記--知識庫)（7） +2. [辦公文件（Word / Excel / PowerPoint / PDF）](#2-辦公文件word--excel--powerpoint--pdf)（7） +3. [Google Workspace](#3-google-workspace)（2） +4. [Microsoft 365](#4-microsoft-365)（3） +5. [開發協作（GitHub / Atlassian / Slack…）](#5-開發協作github--atlassian--slack)（9） +6. [資料庫](#6-資料庫)（8） +7. [瀏覽器自動化 / 網頁抓取](#7-瀏覽器自動化--網頁抓取)（4） +8. [設計（Figma / Excalidraw）](#8-設計figma--excalidraw)（3） +9. [監控 / Observability](#9-監控--observability)（3） +10. [媒體 / 串流（YouTube / Spotify）](#10-媒體--串流youtube--spotify)（3） +11. [中文圈專用](#11-中文圈專用)（9） +12. [其他常用（Cloudflare / Stripe…）](#12-其他常用cloudflare--stripe)（4） +13. [研究工作流 Skills（學術 / paper / 文獻）](#13-研究工作流-skills學術--paper--文獻)（4） +14. [Multi-LLM Delegation Skills](#14-multi-llm-delegation-skills)（3） +15. [金融 / 交易 Agents](#15-金融--交易-agents)（2） +16. [網頁搜尋 / 檢索（Web Search / Retrieval）](#16-網頁搜尋--檢索web-search--retrieval)（2） + +--- + +## 1. 筆記 / 知識庫 + +### [makenotion/notion-mcp-server](https://github.com/makenotion/notion-mcp-server) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 4k+ | +| License | NOASSERTION | +| 推薦度 | ⭐⭐⭐⭐⭐（**官方**） | + +**教什麼**：Notion 官方 MCP server，可查詢 page、建立 page、操作 database。 +**適合誰**：日常用 Notion 寫筆記 / 管專案 / 跑 wiki 的人——叫 LLM 直接撈資料、寫 page。 +**備註**：需要 Notion integration token；有 read-only 跟 read-write 兩種模式可選。 + +### [MarkusPfundstein/mcp-obsidian](https://github.com/MarkusPfundstein/mcp-obsidian) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 3.5k+ | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐（社群、最熱門） | + +**教什麼**：透過 Obsidian REST API community plugin 讓 LLM 讀寫你的 Obsidian vault。 +**適合誰**：Obsidian 重度使用者，想用 Claude Code 整理 daily note、自動 link、跨檔搜尋。 +**備註**：要先在 Obsidian 裝 [Local REST API](https://github.com/coddingtonbear/obsidian-local-rest-api) plugin。 + +### [PleasePrompto/notebooklm-skill](https://github.com/PleasePrompto/notebooklm-skill) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 6k+ | +| License | NOASSERTION | +| 推薦度 | ⭐⭐⭐⭐ | + +**教什麼**：Claude Code Skill，用瀏覽器自動化操作 NotebookLM、查詢上傳文件，回覆帶 citation。 +**適合誰**：用 NotebookLM 管 paper 跟研究筆記，但想在 Claude Code 一條 prompt 直接查的人。 +**備註**：需要 Google 帳號登入授權。 + +### [teng-lin/notebooklm-py](https://github.com/teng-lin/notebooklm-py) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 12k+ | +| License | NOASSERTION | +| 推薦度 | ⭐⭐⭐⭐ | + +**教什麼**：非官方 NotebookLM Python API + CLI + agentic skill；功能比上面 skill 多，包含一些 web UI 沒開放的能力。 +**適合誰**：要程式化批次操作 NotebookLM 的人（例如自動建 notebook、批次匯入文件）。 +**備註**：非官方、Google 政策變動可能會壞；用前看一下 issue tracker。 + +### [ergut/mcp-logseq](https://github.com/ergut/mcp-logseq) ⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 264 | +| License | MIT | +| 推薦度 | ⭐⭐⭐ | + +**教什麼**：透過 Logseq Local HTTP API 讓 LLM 讀寫 Logseq graph。 +**適合誰**：Logseq 使用者要自動化 daily journal、跨頁 link、查詢 backlinks。 +**備註**：需要 Logseq 開啟 HTTP API（Settings → Features → HTTP API）。 + +### [skridlevsky/graphthulhu](https://github.com/skridlevsky/graphthulhu) ⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 147 | +| License | MIT | +| 推薦度 | ⭐⭐⭐（同時支援 Logseq + Obsidian） | + +**教什麼**：39 個 tool，覆蓋 navigation、search、analysis、writing、journals、flashcards、whiteboards。 +**適合誰**：同時用 Logseq 跟 Obsidian、不想裝兩套 MCP server 的人。 +**備註**：community project，工具數多但每個工具相對基本。 + +### [ankimcp/anki-mcp-server](https://github.com/ankimcp/anki-mcp-server) ⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 254 | +| License | MIT | +| 推薦度 | ⭐⭐⭐ | + +**教什麼**：透過 AnkiConnect 讓 LLM 建卡、查卡、批改 deck。 +**適合誰**：用 Anki 學語言 / 醫學 / 法律的人——叫 LLM 從教材自動產卡。 +**備註**：需要 Anki 桌面版裝 [AnkiConnect](https://ankiweb.net/shared/info/2055492159) addon。 + +--- + +## 2. 辦公文件（Word / Excel / PowerPoint / PDF） + +### [anthropics/skills](https://github.com/anthropics/skills) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 129k+ | +| License | NOASSERTION | +| 推薦度 | ⭐⭐⭐⭐⭐（**官方**，必裝） | + +**教什麼**：Anthropic 官方 Agent Skills repo，含 docx / xlsx / pptx / pdf 處理 skill。 +**適合誰**：所有 Claude Code 使用者——直接 `claude skill install` 就能讓 Claude 讀寫 Office 檔。 +**備註**：是 Skills 集合不是 MCP；走 Stage 5.3 Skill 體系。 + +### [haris-musa/excel-mcp-server](https://github.com/haris-musa/excel-mcp-server) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 3.8k+ | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐⭐（社群最熱門 Excel MCP） | + +**教什麼**：Excel 檔操作 MCP server——讀 / 寫 / 改 cell、formula、sheet。 +**適合誰**：日常處理 Excel 報表、要 LLM 自動填表 / 整理資料的人。 +**備註**：Python 寫的，依賴 openpyxl。 + +### [GongRzhe/Office-PowerPoint-MCP-Server](https://github.com/GongRzhe/Office-PowerPoint-MCP-Server) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 1.7k+ | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐ | + +**教什麼**：用 python-pptx 操作 PPT——建簡報、改 slide、插圖、改 layout。 +**適合誰**：要 LLM 從大綱 / Markdown 自動生 PPT 的人（顧問、講師、學生）。 +**備註**：跟 anthropics/skills 的 pptx skill 重疊；那邊不夠用再來這邊。 + +### [1weiho/open-slide](https://github.com/1weiho/open-slide) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 4.9k+ | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐（agent-native 簡報框架） | + +**教什麼**：為 coding agent 打造的 React 簡報框架——用自然語言描述簡報、讓 Claude Code / Codex / Cursor 寫出 React slides；內附 `/create-slide`、`/slide-authoring` 兩個 Claude Code Skill。 +**適合誰**：想讓 agent 直接產出「程式碼即簡報、可進 git 版控」的人，跟 PowerPoint-MCP 走 .pptx 不同路。 +**備註**：TypeScript / React / Vite，`npx @open-slide/cli init` 起手。它是 agent-native 工具（agent 來寫），不是 Stage 4 那種建構 agent 的編排框架。 + +### [SylphxAI/pdf-reader-mcp](https://github.com/SylphxAI/pdf-reader-mcp) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 688 | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐（PDF 高效解析） | + +**教什麼**：高速 PDF 解析 MCP，比 anthropics/skills 的 pdf skill 快 5-10×（號稱）。 +**適合誰**：要批次讀 paper / contract / report 的人。 +**備註**：parallel processing；大 PDF 處理速度有感差別。 + +### [tfriedel/claude-office-skills](https://github.com/tfriedel/claude-office-skills) ⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 590 | +| License | NOASSERTION | +| 推薦度 | ⭐⭐⭐（補強版 Office skill） | + +**教什麼**：補強 anthropics/skills 沒覆蓋到的 Office workflow（automation、進階格式）。 +**適合誰**：覺得官方 docx/xlsx/pptx skill 不夠細的人。 +**備註**：跟 anthropics/skills 是補充關係，不是替代。 + +### [kreuzberg-dev/kreuzberg](https://github.com/kreuzberg-dev/kreuzberg) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 8.2k+ | +| License | NOASSERTION | +| 推薦度 | ⭐⭐⭐⭐ | + +**教什麼**：97+ 種文件格式（PDF、Office、圖片）解析框架，Rust 核心。提供 MCP server + REST API + CLI。 +**適合誰**：跨格式批次處理檔案、要 throughput 的工程師。 +**備註**：不只是 PDF / Office——還支援冷門格式如 HWP、ODT 等。 + +--- + +## 3. Google Workspace + +### [taylorwilsdon/google_workspace_mcp](https://github.com/taylorwilsdon/google_workspace_mcp) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 2.3k+ | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐⭐（一個 server 包整套 Google） | + +**教什麼**：Gmail、Calendar、Docs、Sheets、Slides、Drive、Chat、Forms、Tasks、Search 全部一個 MCP server 搞定。 +**適合誰**：Google Workspace 重度使用者——回信、開會、寫文件、操作 sheet 都一個 server 處理。 +**備註**：OAuth 設定稍微麻煩但一次設定就好；功能覆蓋 Google 各家最完整。 + +### [xing5/mcp-google-sheets](https://github.com/xing5/mcp-google-sheets) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 844 | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐（單純 Sheets 用） | + +**教什麼**：專門 Google Sheets / Drive 整合，建 sheet、改 cell、查 formula。 +**適合誰**：只用 Google Sheets、不想裝整套 Workspace MCP 的人。 +**備註**：scope 比 google_workspace_mcp 窄，但設定簡單。 + +--- + +## 4. Microsoft 365 + +### [Softeria/ms-365-mcp-server](https://github.com/Softeria/ms-365-mcp-server) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 681 | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐（M365 全套） | + +**教什麼**：透過 Microsoft Graph API 操作 M365——Outlook、Teams、OneDrive、SharePoint。 +**適合誰**：用 M365 的企業使用者——要 LLM 回信、查行事曆、撈 OneDrive 檔。 +**備註**：需要 Azure AD app registration；公司 IT 政策可能擋。 + +### [ryaker/outlook-mcp](https://github.com/ryaker/outlook-mcp) ⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 363 | +| License | NOASSERTION | +| 推薦度 | ⭐⭐⭐（只 Outlook） | + +**教什麼**：透過 Graph API 讀寫 Outlook mail / calendar。 +**適合誰**：只要操作 Outlook 不需要其他 M365 服務的人。 +**備註**：scope 比上面的 ms-365 server 窄。 + +### [merill/lokka](https://github.com/merill/lokka) ⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 244 | +| License | MIT | +| 推薦度 | ⭐⭐⭐ | + +**教什麼**：M365 + Microsoft Graph 全套，含 Entra（AD）、Intune 等管理用 API。 +**適合誰**：M365 系統管理員、要操作 Tenant / 使用者 / 政策的人。 +**備註**：對 IT admin 比 end user 更有用。 + +--- + +## 5. 開發協作（GitHub / Atlassian / Slack…） + +### [github/github-mcp-server](https://github.com/github/github-mcp-server) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 29.5k+ | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐⭐（**官方**） | + +**教什麼**：GitHub 官方 MCP——issue / PR / repo / Actions / Codespaces 操作。 +**適合誰**：所有 GitHub 使用者；Claude Code 接上去後 PR review、issue triage、release notes 都能跑。 +**備註**：**走 Track A 的 A3 動手練習 CLI-9 必裝**。 + +### [sooperset/mcp-atlassian](https://github.com/sooperset/mcp-atlassian) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 5.1k+ | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐⭐（社群最熱門 Atlassian） | + +**教什麼**：Confluence + Jira 一個 MCP server，社群版本功能多、設定彈性。 +**適合誰**：用 Atlassian 但 Atlassian 官方 remote server 限制多的人。 +**備註**：跟下面 atlassian/atlassian-mcp-server（官方）擇一，看公司 IT 政策。 + +### [atlassian/atlassian-mcp-server](https://github.com/atlassian/atlassian-mcp-server) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 650+ | +| License | Apache-2.0 | +| 推薦度 | ⭐⭐⭐⭐（**官方**） | + +**教什麼**：Atlassian 官方 Remote MCP，安全連 Jira / Confluence。 +**適合誰**：公司有 enterprise Atlassian、IT 規定只能用官方的人。 +**備註**：Remote 模式，有官方 SLA。 + +### [korotovsky/slack-mcp-server](https://github.com/korotovsky/slack-mcp-server) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 1.6k+ | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐（無 admin 權限也能用） | + +**教什麼**：Slack MCP，DM / group DM / 頻道訊息撈取，自帶 history fetch logic。 +**適合誰**：個人使用者（不是 Slack admin）也想接 Slack 的人。 +**備註**：不需要 admin 級別 token；走使用者層 OAuth。 + +### [jerhadf/linear-mcp-server](https://github.com/jerhadf/linear-mcp-server) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 344 | +| License | NOASSERTION | +| 推薦度 | ⭐⭐⭐⭐ | + +**教什麼**：Linear（issue tracker）MCP——查 issue、建 issue、改 status。 +**適合誰**：用 Linear 管 sprint / backlog 的開發者。 +**備註**：要 Linear API key。 + +### [SaseQ/discord-mcp](https://github.com/SaseQ/discord-mcp) ⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 298 | +| License | MIT | +| 推薦度 | ⭐⭐⭐ | + +**教什麼**：Discord MCP——讀寫頻道訊息、管理伺服器。 +**適合誰**：用 Discord 跑社群 / 開源專案的 maintainer。 +**備註**：要 Discord bot token；要小心 rate limit。 + +### [safishamsi/graphify](https://github.com/safishamsi/graphify) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 44k+ | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐⭐ | + +**教什麼**：把 codebase / SQL schemas / R scripts / shell scripts / docs / papers / images / videos 變成 queryable knowledge graph 的 AI coding skill。Claude Code、Codex、OpenCode、Cursor、Gemini CLI 都能接。 +**適合誰**：要對大型 codebase 做架構分析、跨檔追 reference、把「app code + DB schema + infra」放一起問的工程師 / 研究者。 +**備註**：跨界——既是 dev collab tool（理解既有 codebase）也算 research workflow（把任意素材轉成 graph）。撞牆時用 graphify 抽結構、再丟回 Claude 推論。 + +### [upstash/context7](https://github.com/upstash/context7) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 57k | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐⭐（寫 code 必裝） | + +**教什麼**：把函式庫 / 框架的**最新版官方文件**即時拉進 agent 的 context，讓 LLM 不再依賴過時、會幻覺的 API；是安裝量最高的 coding MCP 之一。 +**適合誰**：所有用 Claude Code / Cursor 寫 code 的人——尤其常踩「LLM 用舊版 API、查半天才發現過時」的開發者。 +**備註**：解決的是「LLM 知識有 cutoff、套件卻一直更新」的痛點；一條 prompt 就能讓 agent 抓到當前版本的正確用法。 + +### [DeusData/codebase-memory-mcp](https://github.com/DeusData/codebase-memory-mcp) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 13.5k+ | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐⭐（code intelligence） | + +**教什麼**：把 codebase 索引成可查詢的 knowledge graph，讓 coding agent 用「查結構 / 符號 / 呼叫路徑」取代反覆 grep + 讀檔。單一 static binary、158 種語言。 +**適合誰**：在大型或不熟的 repo 上跑 coding agent、想快速定位又想省 token 的人。 +**備註**：大改後要重新索引（graph 會 stale）；把它的回答當「快速第一手」、load-bearing 的結論（誰呼叫 X / 這段是不是死碼）再用實際程式碼驗證。 + +--- + +## 6. 資料庫 + +### [googleapis/mcp-toolbox](https://github.com/googleapis/mcp-toolbox) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 15k+ | +| License | Apache-2.0 | +| 推薦度 | ⭐⭐⭐⭐⭐（**Google 官方**，多 DB） | + +**教什麼**：跨 DB 的 MCP server——MySQL / PostgreSQL / Cloud SQL / Spanner / BigQuery 一次包。 +**適合誰**：在 Google Cloud 上跑 DB 的工程師、要支援多 DB 引擎的開發者。 +**備註**：開源 + Google 官方維護，是可上線使用的選擇。 + +### [bytebase/dbhub](https://github.com/bytebase/dbhub) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 2.7k+ | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐⭐（社群多 DB MCP） | + +**教什麼**：zero-dependency、token-efficient 的多 DB MCP——Postgres、MySQL、SQL Server、MariaDB、SQLite。 +**適合誰**：不想裝 Google Cloud SDK、要跨多種 OSS DB 的工程師。 +**備註**：跟 googleapis/mcp-toolbox 重疊，但更輕量。 + +### [supabase-community/supabase-mcp](https://github.com/supabase-community/supabase-mcp) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 2.7k+ | +| License | Apache-2.0 | +| 推薦度 | ⭐⭐⭐⭐⭐（**Supabase 官方社群**） | + +**教什麼**：把 Supabase（含 Postgres、Auth、Storage、Edge Functions）接到 LLM。 +**適合誰**：用 Supabase 跑後端的全端開發者。 +**備註**：官方 community 維護。 + +### [timescale/pg-aiguide](https://github.com/timescale/pg-aiguide) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 1.7k+ | +| License | Apache-2.0 | +| 推薦度 | ⭐⭐⭐⭐（Postgres 寫程式輔助） | + +**教什麼**：MCP server + Claude plugin，幫 LLM 生成更好的 PostgreSQL 程式碼。 +**適合誰**：寫 Postgres heavy SQL / DBA 工程師。 +**備註**：偏「LLM 寫 SQL 輔助」，不只是 query 執行。 + +### [benborla/mcp-server-mysql](https://github.com/benborla/mcp-server-mysql) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 1.6k+ | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐（read-only MySQL） | + +**教什麼**：read-only MySQL MCP，讓 LLM 看 schema、跑 query。 +**適合誰**：要讓 LLM 分析 production DB 但不能改的場景。 +**備註**：故意 read-only 是 safety feature，不是限制。 + +### [mongodb-js/mongodb-mcp-server](https://github.com/mongodb-js/mongodb-mcp-server) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 1k+ | +| License | Apache-2.0 | +| 推薦度 | ⭐⭐⭐⭐（**MongoDB 官方**） | + +**教什麼**：MongoDB 跟 MongoDB Atlas Cluster MCP server。 +**適合誰**：用 MongoDB / Atlas 的工程師。 +**備註**：mongodb-js 是 MongoDB 官方 GitHub org。 + +### [redis/mcp-redis](https://github.com/redis/mcp-redis) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 504 | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐（**Redis 官方**） | + +**教什麼**：Redis 官方 MCP，自然語言操作 Redis 跟 Redis Stack（Vector / Search / JSON）。 +**適合誰**：用 Redis 當 cache / vector DB / queue 的人。 +**備註**：官方維護；包含 vector search 整合。 + +### [awslabs/mcp](https://github.com/awslabs/mcp) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 9.3k+ | +| License | Apache-2.0 | +| 推薦度 | ⭐⭐⭐⭐（**AWS 官方**） | + +**教什麼**：AWS 官方 MCP server（Lambda / S3 / DynamoDB / CloudWatch / Cost Explorer 等）。 +**適合誰**：在 AWS 上、想讓 agent 查詢 / 操作雲端資源的團隊。 +**備註**：AWS 官方維護；沿用你現有的 AWS 登入（CLI profile / IAM role），不用另外管 token。 + +--- + +## 7. 瀏覽器自動化 / 網頁抓取 + +### [microsoft/playwright-mcp](https://github.com/microsoft/playwright-mcp) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 34k+ | +| License | Apache-2.0 | +| 推薦度 | ⭐⭐⭐⭐⭐（**Microsoft 官方**） | + +**教什麼**：Playwright MCP server——讓 LLM 開瀏覽器、點按鈕、填表單、抓網頁。 +**適合誰**：要做 E2E 自動化、跨網站整合、抓需要登入的網頁的人。 +**備註**：Playwright 官方出，最 robust。**Claude Code 接 web 自動化的不錯選項**。 + +### [ChromeDevTools/chrome-devtools-mcp](https://github.com/ChromeDevTools/chrome-devtools-mcp) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 38k+ | +| License | Apache-2.0 | +| 推薦度 | ⭐⭐⭐⭐⭐（**Chrome 官方**） | + +**教什麼**：把 Chrome DevTools 接給 coding agent——performance、network、console 直接給 LLM 看。 +**適合誰**：除錯前端 bug、做 web performance 分析的開發者。 +**備註**：搭配 Playwright MCP 用最強——一個跑、一個觀察。 + +### [firecrawl/firecrawl-mcp-server](https://github.com/firecrawl/firecrawl-mcp-server) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 6.2k+ | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐⭐（**Firecrawl 官方**） | + +**教什麼**：Firecrawl 官方 MCP——大規模網頁抓取 + search + 結構化萃取。 +**適合誰**：要抓大量網頁當訓練資料 / 做 RAG / 做研究的人。 +**備註**：需要 Firecrawl API key（有 free tier）。 + +### [browserbase/mcp-server-browserbase](https://github.com/browserbase/mcp-server-browserbase) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 3.3k+ | +| License | Apache-2.0 | +| 推薦度 | ⭐⭐⭐⭐（**Browserbase 官方**） | + +**教什麼**：Browserbase 官方 MCP，配 Stagehand 跑 cloud-based 瀏覽器。 +**適合誰**：本地跑瀏覽器太重 / 要在 cloud 平行跑多個 session 的人。 +**備註**：商業服務（有免費額度），跟 Playwright MCP 互補（local vs cloud）。 + +--- + +## 8. 設計（Figma / Excalidraw） + +### [GLips/Figma-Context-MCP](https://github.com/GLips/Figma-Context-MCP) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 14.6k+ | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐⭐（最熱門 Figma MCP） | + +**教什麼**：把 Figma layout 資訊送給 coding agent——讀設計稿、提元件結構，給 Cursor / Claude Code 寫對應的 React component。 +**適合誰**：前端開發者，要 LLM 從 Figma 設計稿生成 component code。 +**備註**：要 Figma access token；對 design-to-code workflow 必裝。 + +### [excalidraw/excalidraw-mcp](https://github.com/excalidraw/excalidraw-mcp) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 4.3k+ | +| License | NOASSERTION | +| 推薦度 | ⭐⭐⭐⭐⭐（**Excalidraw 官方**） | + +**教什麼**：streamable Excalidraw MCP，讓 LLM 直接畫架構圖、流程圖。 +**適合誰**：寫設計文件 / 系統架構 / 流程圖的人——叫 Claude 從文字描述畫圖。 +**備註**：Excalidraw 官方出，輸出可直接匯入 Excalidraw 編輯。 + +### [yctimlin/mcp_excalidraw](https://github.com/yctimlin/mcp_excalidraw) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 1.9k+ | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐（替代版 Excalidraw） | + +**教什麼**：MCP server + Claude Code Skill，real-time canvas sync，可建立 / 編輯 / 匯出。 +**適合誰**：需要 real-time canvas sync 跟程式化操作的人。 +**備註**：跟官方版互補，社群維護。 + +### [pbakaus/impeccable](https://github.com/pbakaus/impeccable) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 25k+ | +| License | Apache-2.0 | +| 推薦度 | ⭐⭐⭐⭐⭐ | + +**教什麼**：「**讓你 AI harness 在 design 上更強的 design language**」——一套設計 vocabulary / pattern，幫 AI 在生成 UI / 視覺成品時跳出常見的「AI 感」生硬風格。 +**適合誰**：用 AI 生 UI / mockup / visual design 但結果都很 generic 的開發者；前端 + AI workflow。 +**備註**：不是 MCP server 也不是 Skill 包——是一份「**design language**」reference。讓 AI 看到比較高品質的設計詞彙才生得出比較好的東西。 + +--- + +## 9. 監控 / Observability + +### [grafana/mcp-grafana](https://github.com/grafana/mcp-grafana) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 3k+ | +| License | Apache-2.0 | +| 推薦度 | ⭐⭐⭐⭐⭐（**Grafana 官方**） | + +**教什麼**：Grafana 官方 MCP，從 LLM 直接查 dashboard、metric、alert。 +**適合誰**：用 Grafana 看 metric 的 SRE / DevOps。 +**備註**：「dashboard 那條線為什麼掉？」直接問，LLM 撈 metric 給答案。 + +### [getsentry/sentry-mcp](https://github.com/getsentry/sentry-mcp) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 677 | +| License | NOASSERTION | +| 推薦度 | ⭐⭐⭐⭐（**Sentry 官方**） | + +**教什麼**：從 LLM 查 Sentry error event、issue、trace。 +**適合誰**：用 Sentry 接 production error 的工程師。 +**備註**：「上週這個 error 的 stack trace 給我看」直接問 Claude Code。 + +### [winor30/mcp-server-datadog](https://github.com/winor30/mcp-server-datadog) ⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 142 | +| License | Apache-2.0 | +| 推薦度 | ⭐⭐⭐（社群版 Datadog） | + +**教什麼**：Datadog API MCP——查 monitor、log、metric。 +**適合誰**：用 Datadog 但 Datadog 還沒出官方 MCP 的人。 +**備註**：等 Datadog 官方 MCP 出來可能換掉這個。 + +--- + +## 10. 媒體 / 串流（YouTube / Spotify） + +### [varunneal/spotify-mcp](https://github.com/varunneal/spotify-mcp) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 599 | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐ | + +**教什麼**：把 LLM 接到 Spotify——播歌、加歌單、查歷史。 +**適合誰**：想用 Claude Code 控播放清單、做語音 / 文字 → 音樂的整合者。 +**備註**：要 Spotify Premium 帳號（API 限制）。 + +### [kimtaeyoon83/mcp-server-youtube-transcript](https://github.com/kimtaeyoon83/mcp-server-youtube-transcript) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 534 | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐（YouTube 字幕） | + +**教什麼**：直接抓 YouTube 影片字幕給 LLM 摘要、翻譯、做 RAG。 +**適合誰**：用影片當學習材料、要批次摘要 YouTube 內容的人。 +**備註**：依賴 YouTube auto-caption；非英文影片字幕品質參差。 + +### [ZubeidHendricks/youtube-mcp-server](https://github.com/ZubeidHendricks/youtube-mcp-server) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 510 | +| License | NOASSERTION | +| 推薦度 | ⭐⭐⭐⭐（YouTube 完整 API） | + +**教什麼**：完整 YouTube API MCP——除了 transcript，還能管 video、Shorts、analytics。 +**適合誰**：YouTube 創作者要自動化頻道管理。 +**備註**：需要 YouTube Data API key + OAuth。 + +--- + +## 11. 中文圈專用 + +### [leemysw/feishu-docx](https://github.com/leemysw/feishu-docx) ⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 193 | +| License | MIT | +| 推薦度 | ⭐⭐⭐ | + +**教什麼**：飛書（Lark）docs / sheet / bitable ↔ Markdown 雙向轉換，含 OAuth 2.0、CLI、TUI、Claude Skills。 +**適合誰**：用飛書 / Lark 寫文件的中文使用者，要把 Lark 內容跟 Claude Code 串起來。 +**備註**：目前中文圈 MCP / Skill 主要選擇之一；微信 / 釘釘暫時沒有獨立 MCP（多半混在 chat bot framework 裡）。 + +### [netease-youdao/LobsterAI](https://github.com/netease-youdao/LobsterAI) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 5k+ | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐ | + +**教什麼**：網易有道出品的「24/7 全場景 AI agent」——支援工作流自動化、跨應用協作、檔案處理。中文 native。 +**適合誰**：中文圈使用者要找一個替代 Claude Code / OpenAI Operator 等級的 all-in-one agent；對中國大陸服務（網易、釘釘等）整合需求高的場景。 +**備註**：產品式 agent（不是 Skill / MCP）；跟 Claude Code / Codex 互為替代，不是搭配。 + +### [QwenLM/Qwen-Agent](https://github.com/QwenLM/Qwen-Agent) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 16k+ | +| License | Apache-2.0 | +| 推薦度 | ⭐⭐⭐⭐⭐ | + +**教什麼**：阿里巴巴官方 Qwen agent framework——RAG、tool use、code interpreter、multi-agent、MCP 相容，預設搭配 Qwen 系列模型但可換其他 LLM。 +**適合誰**：用 Qwen / 通義千問 為主 LLM 的開發者；想要中文 native 的 agent framework（範例、文件都中文齊全）。 +**備註**：MCP 相容是亮點——可以直接接到 Claude Code 等 host；維護節奏正常（last commit 2026-03）。 + +### [coze-dev/coze-studio](https://github.com/coze-dev/coze-studio) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 20k+ | +| License | Apache-2.0 | +| 推薦度 | ⭐⭐⭐⭐⭐ | + +**教什麼**：字節跳動 Coze 的開源版——no-code agent builder（workflow / plugin / knowledge / memory），可自部署或上雲。 +**適合誰**：不想寫 code 但要做 agent 的團隊；想看 enterprise agent platform 內部設計（RAG、工作流、Memory、Plugin 系統的 reference 實作）。 +**備註**：底層 framework 是 Coze 自家的 Eino；可接 OpenAI / Claude / Qwen / 國產 LLM。國際版（coze.com）跟中國版（coze.cn）共用此 codebase。 + +### [coze-dev/coze-loop](https://github.com/coze-dev/coze-loop) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 5k+ | +| License | Apache-2.0 | +| 推薦度 | ⭐⭐⭐⭐ | + +**教什麼**：Coze 出的 agent observability + evaluation 平台——trace、debug、eval、prompt management，agent dev lifecycle 的下半場。 +**適合誰**：agent 已經跑起來、要 production 監控的團隊；想看「agent eval / observability」可以怎麼做的人。 +**備註**：跟 LangSmith / Arize Phoenix 同類；開源版可自部署。 + +### [liaokongVFX/LangChain-Chinese-Getting-Started-Guide](https://github.com/liaokongVFX/LangChain-Chinese-Getting-Started-Guide) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 8.9k+ | +| License | 未標註 | +| 推薦度 | ⭐⭐⭐⭐ | + +**教什麼**：LangChain 中文入門指南——從 LangChain 基礎、Prompt、Memory、Agent、Chains 到實作應用，最早期且最完整的中文 LangChain 學習資源。 +**適合誰**：想用 LangChain 但英文文件吃不下去的中文使用者；想理解 LangChain 設計脈絡再決定要不要走這條路的人。 +**備註**：沒有正式 license（內容開放閱讀）；LangChain 框架本身演進很快，書中部分 API 可能跟最新版有出入。 + +### [chatchat-space/Langchain-Chatchat](https://github.com/chatchat-space/Langchain-Chatchat) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 37k+ | +| License | Apache-2.0 | +| 推薦度 | ⭐⭐⭐⭐ | + +**教什麼**：基於 LangChain 的開源知識庫問答系統——本地化部署、支援多種向量資料庫、RAG 端到端範例。 +**適合誰**：想做 RAG 又不想全部自己刻的中文團隊；要本地部署（不能用雲端 LLM）的場景。 +**備註**：★ 37k 是中文圈最熱門的 RAG 實作之一；維護節奏放緩（last commit 2025-11）。新專案建議先 fork 後評估，當參考實作用。 + +### [usewhale/whale](https://github.com/usewhale/whale) ⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 117 | +| License | MIT | +| 推薦度 | ⭐⭐⭐ | + +**教什麼**：專為 DeepSeek 模型優化的終端 AI 編碼助手——支援 MCP server 接入、Claude-style Skills、對話快取優化，Go 實作。 +**適合誰**：以 DeepSeek 為主力 LLM 的中文開發者；想用終端工具但不需要 Claude Code 全家桶的人。 +**備註**：開源同類中少見的 DeepSeek 專屬優化；MCP + Skills 雙支援讓它可以逐步擴充能力。 + +### [simonlin1212/a-stock-data](https://github.com/simonlin1212/a-stock-data) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 492 | +| License | Apache-2.0 | +| 推薦度 | ⭐⭐⭐⭐ | + +**教什麼**：A 股全棧資料工具包——單一 SKILL.md 檔案封裝 8 個資料源（mootdx、東財、akshare、iwencai 等）21 個端點，AI 編碼助手直接可用。 +**適合誰**：用 Claude Code / Codex / OpenClaw 做投研或量化分析的中文開發者；不想自己刻資料抓取邏輯的人。 +**備註**：一條 `curl` + `pip install` 即可啟用；中國 A 股資料類 Skill 中星星數最高的社群實作。相容 Claude Code、Codex、OpenClaw。 + +> 想找微信 / 釘釘整合？目前主流是用 chat bot framework（如 zhayujie/CowAgent）而不是純 MCP server。等正規 MCP 出現再加進來。 + +### [MoonshotAI/Kimi-K2](https://github.com/MoonshotAI/Kimi-K2) ⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 10.7k+ | +| License | Modified MIT | +| 推薦度 | ⭐⭐⭐ | + +**教什麼**：月之暗面 Moonshot 的 Kimi K2 開源大模型系列——開源權重 + OpenAI / Anthropic 相容 API，主打 agentic / coding / 長程任務，可當 agent stack 的後端模型。 +**適合誰**：想用國產開源模型跑 agent / coding 工作流、或要在自架環境跑開源權重的中文開發者。 +**備註**：License 是 Modified MIT（標準 MIT + 大規模商用附加條款）——商用前先讀原始 LICENSE；weights 另在 Hugging Face。 + +### [zai-org/GLM-4.5](https://github.com/zai-org/GLM-4.5) ⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 4.3k+ | +| License | Apache-2.0 | +| 推薦度 | ⭐⭐⭐ | + +**教什麼**：智譜 Zhipu（Z.ai）的 GLM-4.5 開源模型——定位 Agentic / Reasoning / Coding（ARC）基礎模型，開源權重 + API，可當 agent / tool use / coding 的後端。 +**適合誰**：想評估國產開源 agentic 模型、或需要 Apache-2.0 寬鬆授權權重的中文開發者。 +**備註**：zai-org 是智譜開源 org；同系列另有 GLM-4（★ 7k+）可一起參考；weights 在 Hugging Face。 + +--- + +## 12. 其他常用（Cloudflare / Stripe…） + +### [cloudflare/mcp-server-cloudflare](https://github.com/cloudflare/mcp-server-cloudflare) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 3.7k+ | +| License | Apache-2.0 | +| 推薦度 | ⭐⭐⭐⭐⭐（**Cloudflare 官方**） | + +**教什麼**：Cloudflare 官方 MCP——Workers、Pages、R2、KV、D1、DNS、Zero Trust 全包。 +**適合誰**：用 Cloudflare 跑 edge / serverless 的人。 +**備註**：官方維護；最佳的 edge platform MCP。 + +### [stripe/ai](https://github.com/stripe/ai) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 1.5k+ | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐（**Stripe 官方**） | + +**教什麼**：Stripe 官方 AI agent toolkit，含 MCP server，操作付款、訂閱、退款、客戶。 +**適合誰**：要在 agent 內處理付款 / billing 的開發者。 +**備註**：⚠️ 涉及金流，務必用 sandbox 測試夠了再接 production。 + + +### YIELD INTELLIGENCE MCP（Hosted Remote Server） + +| 欄位 | 內容 | +|---|---| +| 形式 | hosted MCP server | +| 推薦度 | ⭐⭐⭐（Finance 分析工具；了解 hosted vs self-hosted MCP 實作差異的實例） | + +**教什麼**：YIELD INTELLIGENCE hosted remote MCP server——即時美國國債殖利率 + 股息 ETF / REIT / 特別股分析 + 被動收入投資組合優化。2 個工具：`analyze_yield_opportunities`（掃描被動收入機會）+ `optimize_income_portfolio`（目標月收入建立投資組合）。已列入 Anthropic 官方 MCP Registry（`io.github.thebrierfox/intuitek-ace`，since 2026-05-10）。 +**適合誰**：用 Claude Code / Claude Desktop 做個人理財分析、想讓 AI 找出被動收入機會的人。hosted remote MCP server 範例——直接 plug URL、0 安裝、適合 Stage 5 學完 MCP 概念後拿來實驗 hosted vs self-hosted 差異。 +**備註**：Live endpoint `https://api.intuitek.ai/yield/mcp`（no auth、no API key）。x402 micropayment $1 USDC/call on Base（agent-to-agent 場景）；一般使用者免費。非交易型，純分析工具。GitHub：[thebrierfox/intuitek-ace](https://github.com/thebrierfox/intuitek-ace)（MIT License）。 + +### [ComposioHQ/composio](https://github.com/ComposioHQ/composio) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 28.8k+ | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐⭐（1000+ 工具整合樞紐） | + +**教什麼**：一個平台（SDK + MCP server），把 agent 連到 1000+ 應用（Slack / GitHub / Gmail / Salesforce / Notion…），登入它幫你處理，不用一個服務各寫一個連接器。 +**適合誰**：agent 要跨大量工具、但不想維護幾十個獨立 MCP server 的團隊。 +**備註**：提供 MCP server + Python / TypeScript SDK；可透過 MCP 接到 Claude Code。屬「工具聚合器」（跟 n8n / Zapier 自動化平台同類）。 + +--- + +## 13. 研究工作流 Skills（學術 / paper / 文獻） + +> ⚠️ **maintainer 自家專案區**：以下幾個是本 repo 維護者 [@WenyuChiou](https://github.com/WenyuChiou)（Lehigh CEE PhD candidate）日常在用的研究 skills，公開出來給其他研究者參考。**因為是自己的東西、又是相對 niche 的研究場景，star 數字會比泛用工具低**。star 門檻在這節是放寬的——選收的標準是「在我自己研究流程裡實際有用」。請自己評估是否合用。 + +### [WenyuChiou/ai-research-skills](https://github.com/WenyuChiou/ai-research-skills) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 60 | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐⭐（研究流程一整套） | + +**教什麼**：14 個 Claude Code skills 涵蓋常見研究任務——文獻分流、研究設計、project context、論文撰寫、multi-AI delegation。打包成 5-plugin marketplace，一個指令安裝。 +**適合誰**：研究生 / 博後想一次取得「研究全流程」skill set。 +**備註**：marketplace 形式，跟 Stage 5.4 教的 plugin/marketplace 概念對位。 + +### [WenyuChiou/academic-writing-skills](https://github.com/WenyuChiou/academic-writing-skills) ⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 2 | +| License | MIT | +| 推薦度 | ⭐⭐⭐（窄但深） | + +**教什麼**：嚴謹學術論文寫作 / 修改 / 投稿的 Claude Code skill。Field-agnostic，可用 per-paper journal_format.md 跟 style_overrides.md 客製規則。 +**適合誰**：在寫 / 改 paper 的研究者，想把 banned-word audit、figure-text coupling、submission checklist 自動化。 +**備註**：是 ai-research-skills 5 plugin 中的一個，也可獨立安裝。 + +### [WenyuChiou/zotero-skills](https://github.com/WenyuChiou/zotero-skills) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 16 | +| License | NOASSERTION | +| 推薦度 | ⭐⭐⭐⭐ | + +**教什麼**：Zotero CLI skill——程式化搜尋 / 加 / 分類 / 標記文獻。 +**適合誰**：用 Zotero 管文獻、想讓 Claude Code 直接整理 library 的研究者。 +**備註**：跟 [`MuiseDestiny/zotero-gpt`](https://github.com/MuiseDestiny/zotero-gpt) 的差別——後者是 Zotero plugin（在 Zotero 裡 chat），這份是 CLI / Skill（從 Claude Code 操作 Zotero）。 + +### [WenyuChiou/research-hub](https://github.com/WenyuChiou/research-hub) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 14 | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐ | + +**教什麼**：AI-operable research workspace，整合 Zotero + Obsidian + NotebookLM 三個工具，提供 CLI / MCP / REST / dashboard 四種介面。 +**適合誰**：同時用 Zotero / Obsidian / NotebookLM 的研究者，想把它們綁成一個 workspace 給 LLM 操作。 +**備註**：跟單一工具的 MCP（mcp-obsidian、notion-mcp 等）互補——這份是 hub，可整合多個工具。 + +--- + +## 14. Multi-LLM Delegation Skills + +> ⚠️ **maintainer 自家專案區**：跟 13 一樣，以下是維護者把自己 daily workflow 抽出來公開的 delegation skills。star 門檻放寬，選收標準是「真的能讓 Claude planner + Codex/Gemini 執行者組合穩定跑下去」。Multi-LLM 領域變化快，建議跟其他 multi-agent framework（Stage 7 列的）一起評估後再選。 + +### 三個 skill 的組合（composition） + +底下 3 個 skill 是**設計成一起用**的，不是獨立工具： + +![Claude + 3 delegate skill 分工](../resources/diagrams/multi-llm-delegation-composition.png) + +Claude 不擅長 token-heavy 機械式工作（成本高、context 容易爆）；Codex 不擅長對話協作；Gemini 1M context 但中型推理稍弱。**分工 = Claude 負責 design / review、Codex 負責 implement、Gemini 負責 long-form draft / synthesis**。 + +### [WenyuChiou/codex-delegate](https://github.com/WenyuChiou/codex-delegate) ⭐⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 57 | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐⭐ | + +**教什麼**：Claude Code skill 把 Codex CLI 當 execution specialist——大量檔案 refactor、batch edits、boilerplate 生成、wrapper-based 實作密集任務。Claude 寫 plan + review，Codex 執行。 +**適合誰**：要省 token / 提速大規模機械式編輯的開發者；想驗證「multi-agent 不只是 buzzword」的學習者。 +**何時用**：refactor 30+ files、生成 test scaffold、port 同樣 pattern 到 N 個檔案、寫 migration script。 +**何時不用**：架構決策、bug 診斷、security review、需要 conversation memory 的任務——這些 Claude 直接做更好。 +**備註**：跟 `gemini-delegate-skill` 互補。Stage 7 multi-agent 的實戰版。 + +### [WenyuChiou/gemini-delegate-skill](https://github.com/WenyuChiou/gemini-delegate-skill) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 34 | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐ | + +**教什麼**：Claude Code skill 把 Gemini CLI 當 long-form / large-context / CJK 執行者——百萬 token context window、中文長文 draft、second-opinion review。Claude 出大綱跟 critique，Gemini 寫長文。 +**適合誰**：研究者寫 paper、知識工作者寫中文報告 / Threads post、需要第二 LLM 意見對照的人。 +**何時用**：長文 draft（>3000 字）、跨多份長文件 synthesis（要塞進 1M token 的 context）、中文 / CJK 內容、要 LLM-vs-LLM 對比視角。 +**何時不用**：短查詢、code generation（用 codex）、production-critical 決策（最終人類 review）。 +**備註**：跟 `codex-delegate` 是「Codex 寫 code、Gemini 寫 prose」的分工。 + +### [WenyuChiou/agent-collab-skills](https://github.com/WenyuChiou/agent-collab-skills) ⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | 剛公開、stars 還沒累積 | +| License | MIT | +| 推薦度 | ⭐⭐（experimental，當 reference 看就好） | + +**教什麼**：Claude Code marketplace for multi-agent collaboration——task splitter、output reconciler、adversarial debate、shared memory、acceptance gate。跟 codex-delegate / gemini-delegate 組合用。 +**適合誰**：要跑 2+ delegate agent 在同一輪、想看 multi-agent coordination 怎麼包成 marketplace 的人。 +**備註**：experimental——別把它當生產級 framework，當作維護者把自己 setup 公開的 reference 看就好。要可上線部署的請看 Stage 7 的 LangGraph / AutoGen / CrewAI。 + +--- + +## 15. 金融 / 交易 Agents + +> ⚠️ **應用領域區**：agent 在量化交易 / hedge fund 模擬 / 自動下單的應用。這類 repo 授權狀態混雜（部分 NO-LICENSE、部分 Apache-2.0 等開源授權），使用前自行查清楚。**警示**：trading agent 跑真實資金有顯著風險，本目錄列入是為了學習 agent 設計模式、不是投資建議。 + +### [TauricResearch/TradingAgents](https://github.com/TauricResearch/TradingAgents) ⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 79k+ | +| License | Apache-2.0 | +| 推薦度 | ⭐⭐⭐ | + +**教什麼**：多 agent LLM 框架做金融交易決策，bull / bear / fundamentals / technicals / risk 各 agent 分工。 +**適合誰**：想看 multi-agent 在分析性任務怎麼分工的學習者；量化研究者想實驗 LLM 增強既有 pipeline。 +**備註**：Apache-2.0、允許修改與商用（保留授權聲明）；**非投資建議，別直接拿來下實單**。 + +### [virattt/ai-hedge-fund](https://github.com/virattt/ai-hedge-fund) ⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 59k+ | +| License | NO-LICENSE | +| 推薦度 | ⭐⭐⭐ | + +**教什麼**：多角色 AI hedge fund 模擬，bull / bear / 基本面 / 技術面 / 風控 agent 協作產生 trade recommendation。 +**適合誰**：看過 Stage 7 multi-agent 想要一個完整應用案例的學習者；對 agent + 金融交叉領域有興趣的人。 +**備註**：NO-LICENSE → 同上；**模擬性質、非投資建議**。 + +--- + +## 16. 網頁搜尋 / 檢索（Web Search / Retrieval） + +### [exa-labs/exa-mcp-server](https://github.com/exa-labs/exa-mcp-server) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 4.5k+ | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐（**Exa 官方**） | + +**教什麼**：Exa 官方 MCP，專為 LLM / agent 設計的網頁搜尋（neural + keyword），回傳乾淨結果直接餵進 prompt。 +**適合誰**：要做 research / fact-check / 線上 RAG 檢索的人；semantic search 對「概念相關」特別強，純關鍵字搜尋反而沒那麼吃香。 +**備註**：需要 Exa API key。 + +### [tavily-ai/tavily-mcp](https://github.com/tavily-ai/tavily-mcp) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| Stars | ★ 2.1k+ | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐（新手第一選擇） | + +**教什麼**：把 Tavily search API 包成 MCP——專為 LLM / RAG 打造的網頁搜尋，回傳 answer + sources，free tier 容易上手。 +**適合誰**：只是想讓 agent 會上網查資料的新手，這是最好的起手選擇。 +**備註**：需要 Tavily API key（有好上手的 free tier）。 + +--- + +## 還有什麼沒收錄？ + +如果你需要的整合不在上面，先看這些 catalog： + +- [`wong2/awesome-mcp-servers`](https://github.com/wong2/awesome-mcp-servers) — 社群最完整 MCP server 清單，150+ 個按分類整理 +- [`punkpeye/awesome-mcp-servers`](https://github.com/punkpeye/awesome-mcp-servers) — 另一份 MCP server 清單，跟上面互補 +- [`modelcontextprotocol/servers`](https://github.com/modelcontextprotocol/servers) — Anthropic 官方 reference servers（filesystem、git、time、memory、fetch、sequential-thinking 等） +- [`travisvn/awesome-claude-skills`](https://github.com/travisvn/awesome-claude-skills) — Claude Skills 清單 + +### 要加新的？ + +1. 開 issue，附 repo 連結 + 為什麼要加 + 屬於哪個分類 +2. 或直接送 PR：在對應分類下加一個 entry，按上面的格式寫（Stars/License/推薦度 + 教什麼/適合誰/備註） +3. **stars < 100 且非官方**通常會被退；除非你能說明 niche use case 強到可以例外 + +PR 送出前看一下 [`resources/style-guide.md`](style-guide.md) 跟 [`CONTRIBUTING.md`](../CONTRIBUTING.md)。 + +--- + +## 維護備註（給未來想幫忙的人） + +不是 SLA，是「能做就做」的方向： + +- Stars / license 用 `gh api repos//` 抓，**有空 review 一輪就好**——不用排定期程 +- 連結壞了、repo archived → 看到再修 +- 新分類（AR/VR、IoT 等）有 1-2 個值得收的就可以先開 +- 「中文圈專用」分類維持寬鬆，中文社群 repo star 數累積較慢 +- 用詞、格式不一致 → 不要苛求，PR 進來能讀懂優先 diff --git a/resources/mcp-skills-catalog.zh-Hans.md b/resources/mcp-skills-catalog.zh-Hans.md new file mode 100644 index 0000000..16a6193 --- /dev/null +++ b/resources/mcp-skills-catalog.zh-Hans.md @@ -0,0 +1,1054 @@ +# MCP / Skills 集成目录 + +> [繁體中文](./mcp-skills-catalog.md) | **简体中文** | [English](./mcp-skills-catalog.en.md) + +> 把 Claude Code（或其他 CLI agent）接到你已经正在用的工具，不用反复切换视窗。本页是 65+ 个分类整理过的 MCP server / Claude Skill / 集成范例（含研究工作流 + multi-LLM delegation 两个专属区）。 + +--- + +## 怎么用这份目录 + +- **想找特定工具的 MCP**：直接看下面分类目录 +- **想看 MCP / Skills / Plugins 是什么**：先看 [README 三个核心用语](../README.zh-Hans.md#三个核心用语mcp--skills--plugins)，再看 [Stage 5 — Claude Code 生态系](../stages/05-claude-code-ecosystem.md) +- **想看 动手练习 怎么装、怎么测**：看 [Stage 5.2 (MCP)](../stages/05-claude-code-ecosystem.zh-Hans.md#52--mcpmodel-context-protocol-基础) 跟 [Stage 5.3 (Skills)](../stages/05-claude-code-ecosystem.zh-Hans.md#53--skillsclaude-code-的行为层-claude-code-生态最关键的一层) + +### 收录原则 + +- **官方优先**：Anthropic、厂商自己出的 MCP / Skill 排在前 +- **★ 100+ 起跳**：除非是官方，社群 repo 至少 100 stars 才收录 +- **可验证**：所有 stars / license 用 `gh api` 抓即时数据；过时的会在每季 review 时更新 +- **不收**：已 archived、超过 1 年没 commit、license 不明且非官方 + +### 目录 + +1. [笔记 / 知识库](#1-笔记--知识库)（7） +2. [办公文件（Word / Excel / PowerPoint / PDF）](#2-办公文件word--excel--powerpoint--pdf)（7） +3. [Google Workspace](#3-google-workspace)（2） +4. [Microsoft 365](#4-microsoft-365)（3） +5. [开发协作（GitHub / Atlassian / Slack…）](#5-开发协作github--atlassian--slack)（9） +6. [数据库](#6-数据库)（8） +7. [浏览器自动化 / 网页抓取](#7-浏览器自动化--网页抓取)（4） +8. [设计（Figma / Excalidraw）](#8-设计figma--excalidraw)（3） +9. [监控 / Observability](#9-监控--observability)（3） +10. [媒体 / 串流（YouTube / Spotify）](#10-媒体--串流youtube--spotify)（3） +11. [中文圈专属](#11-中文圈专属)（9） +12. [其他常用（Cloudflare / Stripe…）](#12-其他常用cloudflare--stripe)（4） +13. [研究工作流 Skills（学术 / paper / 文献）](#13-研究工作流-skills学术--paper--文献)（4） +14. [Multi-LLM Delegation Skills](#14-multi-llm-delegation-skills)（3） +15. [金融 / 交易 Agents](#15-金融--交易-agents)（2） +16. [网页搜索 / 检索（Web Search / Retrieval）](#16-网页搜索--检索web-search--retrieval)（2） + +--- + +## 1. 笔记 / 知识库 + +### [makenotion/notion-mcp-server](https://github.com/makenotion/notion-mcp-server) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 4k+ | +| License | NOASSERTION | +| 推荐度 | ⭐⭐⭐⭐⭐（**官方**） | + +**教什么**：Notion 官方 MCP server，可查询 page、创建 page、操作 database。 +**适合谁**：日常用 Notion 写笔记 / 管项目 / 跑 wiki 的人——叫 LLM 直接捞数据、写 page。 +**备注**：需要 Notion integration token；有 read-only 跟 read-write 两种模式可选。 + +### [MarkusPfundstein/mcp-obsidian](https://github.com/MarkusPfundstein/mcp-obsidian) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 3.5k+ | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐（社群、最热门） | + +**教什么**：透过 Obsidian REST API community plugin 让 LLM 读写你的 Obsidian vault。 +**适合谁**：Obsidian 重度用户，想用 Claude Code 整理 daily note、自动 link、跨文件搜索。 +**备注**：要先在 Obsidian 装 [Local REST API](https://github.com/coddingtonbear/obsidian-local-rest-api) plugin。 + +### [PleasePrompto/notebooklm-skill](https://github.com/PleasePrompto/notebooklm-skill) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 6k+ | +| License | NOASSERTION | +| 推荐度 | ⭐⭐⭐⭐ | + +**教什么**：Claude Code Skill，用浏览器自动化操作 NotebookLM、查询上传文件，回复带 citation。 +**适合谁**：用 NotebookLM 管 paper 跟研究笔记，但想在 Claude Code 一条 prompt 直接查的人。 +**备注**：需要 Google 账号登录授权。 + +### [teng-lin/notebooklm-py](https://github.com/teng-lin/notebooklm-py) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 12k+ | +| License | NOASSERTION | +| 推荐度 | ⭐⭐⭐⭐ | + +**教什么**：非官方 NotebookLM Python API + CLI + agentic skill；功能比上面 skill 多，包含一些 web UI 没开放的能力。 +**适合谁**：要程序化批量操作 NotebookLM 的人（例如自动建 notebook、批量导入文件）。 +**备注**：非官方、Google 政策变动可能会坏；用前看一下 issue tracker。 + +### [ergut/mcp-logseq](https://github.com/ergut/mcp-logseq) ⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 264 | +| License | MIT | +| 推荐度 | ⭐⭐⭐ | + +**教什么**：透过 Logseq Local HTTP API 让 LLM 读写 Logseq graph。 +**适合谁**：Logseq 用户要自动化 daily journal、跨页 link、查询 backlinks。 +**备注**：需要 Logseq 开启 HTTP API（Settings → Features → HTTP API）。 + +### [skridlevsky/graphthulhu](https://github.com/skridlevsky/graphthulhu) ⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 147 | +| License | MIT | +| 推荐度 | ⭐⭐⭐（同时支持 Logseq + Obsidian） | + +**教什么**：39 个 tool，覆盖 navigation、search、analysis、writing、journals、flashcards、whiteboards。 +**适合谁**：同时用 Logseq 跟 Obsidian、不想装两套 MCP server 的人。 +**备注**：community project，工具数多但每个工具相对基本。 + +### [ankimcp/anki-mcp-server](https://github.com/ankimcp/anki-mcp-server) ⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 254 | +| License | MIT | +| 推荐度 | ⭐⭐⭐ | + +**教什么**：透过 AnkiConnect 让 LLM 建卡、查卡、批改 deck。 +**适合谁**：用 Anki 学语言 / 医学 / 法律的人——叫 LLM 从教材自动产卡。 +**备注**：需要 Anki 桌面版装 [AnkiConnect](https://ankiweb.net/shared/info/2055492159) addon。 + +--- + +## 2. 办公文件（Word / Excel / PowerPoint / PDF） + +### [anthropics/skills](https://github.com/anthropics/skills) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 129k+ | +| License | NOASSERTION | +| 推荐度 | ⭐⭐⭐⭐⭐（**官方**，必装） | + +**教什么**：Anthropic 官方 Agent Skills repo，含 docx / xlsx / pptx / pdf 处理 skill。 +**适合谁**：所有 Claude Code 用户——直接 `claude skill install` 就能让 Claude 读写 Office 档。 +**备注**：是 Skills 集合不是 MCP；走 Stage 5.3 Skill 体系。 + +### [haris-musa/excel-mcp-server](https://github.com/haris-musa/excel-mcp-server) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 3.8k+ | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐⭐（社群最热门 Excel MCP） | + +**教什么**：Excel 档操作 MCP server——读 / 写 / 改 cell、formula、sheet。 +**适合谁**：日常处理 Excel 报表、要 LLM 自动填表 / 整理数据的人。 +**备注**：Python 写的，依赖 openpyxl。 + +### [GongRzhe/Office-PowerPoint-MCP-Server](https://github.com/GongRzhe/Office-PowerPoint-MCP-Server) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 1.7k+ | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐ | + +**教什么**：用 python-pptx 操作 PPT——建简报、改 slide、插图、改 layout。 +**适合谁**：要 LLM 从大纲 / Markdown 自动生 PPT 的人（顾问、讲师、学生）。 +**备注**：跟 anthropics/skills 的 pptx skill 重叠；那边不够用再来这边。 + +### [1weiho/open-slide](https://github.com/1weiho/open-slide) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 4.9k+ | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐（agent-native 简报框架） | + +**教什么**：为 coding agent 打造的 React 简报框架——用自然语言描述简报、让 Claude Code / Codex / Cursor 写出 React slides；内附 `/create-slide`、`/slide-authoring` 两个 Claude Code Skill。 +**适合谁**：想让 agent 直接产出“代码即简报、可进 git 版控”的人，跟 PowerPoint-MCP 走 .pptx 不同路。 +**备注**：TypeScript / React / Vite，`npx @open-slide/cli init` 起手。它是 agent-native 工具（agent 来写），不是 Stage 4 那种构建 agent 的编排框架。 + +### [SylphxAI/pdf-reader-mcp](https://github.com/SylphxAI/pdf-reader-mcp) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 688 | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐（PDF 高效解析） | + +**教什么**：高速 PDF 解析 MCP，比 anthropics/skills 的 pdf skill 快 5-10×（号称）。 +**适合谁**：要批量读 paper / contract / report 的人。 +**备注**：parallel processing；大 PDF 处理速度有感差别。 + +### [tfriedel/claude-office-skills](https://github.com/tfriedel/claude-office-skills) ⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 590 | +| License | NOASSERTION | +| 推荐度 | ⭐⭐⭐（补强版 Office skill） | + +**教什么**：补强 anthropics/skills 没覆盖到的 Office workflow（automation、进阶格式）。 +**适合谁**：觉得官方 docx/xlsx/pptx skill 不够细的人。 +**备注**：跟 anthropics/skills 是补充关系，不是替代。 + +### [kreuzberg-dev/kreuzberg](https://github.com/kreuzberg-dev/kreuzberg) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 8.2k+ | +| License | NOASSERTION | +| 推荐度 | ⭐⭐⭐⭐ | + +**教什么**：97+ 种文件格式（PDF、Office、图片）解析框架，Rust 核心。提供 MCP server + REST API + CLI。 +**适合谁**：跨格式批量处理文件、要 throughput 的工程师。 +**备注**：不只是 PDF / Office——还支持冷门格式如 HWP、ODT 等。 + +--- + +## 3. Google Workspace + +### [taylorwilsdon/google_workspace_mcp](https://github.com/taylorwilsdon/google_workspace_mcp) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 2.3k+ | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐⭐（一个 server 包整套 Google） | + +**教什么**：Gmail、Calendar、Docs、Sheets、Slides、Drive、Chat、Forms、Tasks、Search 全部一个 MCP server 搞定。 +**适合谁**：Google Workspace 重度用户——回信、开会、写文件、操作 sheet 都一个 server 处理。 +**备注**：OAuth 设置稍微麻烦但一次设置好；功能覆盖 Google 各家最完整。 + +### [xing5/mcp-google-sheets](https://github.com/xing5/mcp-google-sheets) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 844 | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐（单纯 Sheets 用） | + +**教什么**：专门 Google Sheets / Drive 集成，建 sheet、改 cell、查 formula。 +**适合谁**：只用 Google Sheets、不想装整套 Workspace MCP 的人。 +**备注**：scope 比 google_workspace_mcp 窄，但设置简单。 + +--- + +## 4. Microsoft 365 + +### [Softeria/ms-365-mcp-server](https://github.com/Softeria/ms-365-mcp-server) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 681 | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐（M365 全套） | + +**教什么**：透过 Microsoft Graph API 操作 M365——Outlook、Teams、OneDrive、SharePoint。 +**适合谁**：用 M365 的企业用户——要 LLM 回信、查行事历、捞 OneDrive 档。 +**备注**：需要 Azure AD app registration；公司 IT 政策可能挡。 + +### [ryaker/outlook-mcp](https://github.com/ryaker/outlook-mcp) ⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 363 | +| License | NOASSERTION | +| 推荐度 | ⭐⭐⭐（只 Outlook） | + +**教什么**：透过 Graph API 读写 Outlook mail / calendar。 +**适合谁**：只要操作 Outlook 不需要其他 M365 服务的人。 +**备注**：scope 比上面的 ms-365 server 窄。 + +### [merill/lokka](https://github.com/merill/lokka) ⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 244 | +| License | MIT | +| 推荐度 | ⭐⭐⭐ | + +**教什么**：M365 + Microsoft Graph 全套，含 Entra（AD）、Intune 等管理用 API。 +**适合谁**：M365 系统管理员、要操作 Tenant / 用户 / 策略的人。 +**备注**：对 IT admin 比 end user 更有用。 + +--- + +## 5. 开发协作（GitHub / Atlassian / Slack…） + +### [github/github-mcp-server](https://github.com/github/github-mcp-server) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 29.5k+ | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐⭐（**官方**） | + +**教什么**：GitHub 官方 MCP——issue / PR / repo / Actions / Codespaces 操作。 +**适合谁**：所有 GitHub 用户；Claude Code 接上去后 PR review、issue triage、release notes 都能跑。 +**备注**：**走 Track A 的 A3 动手练习 CLI-9 必装**。 + +### [sooperset/mcp-atlassian](https://github.com/sooperset/mcp-atlassian) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 5.1k+ | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐⭐（社群最热门 Atlassian） | + +**教什么**：Confluence + Jira 一个 MCP server，社群版本功能多、设置弹性。 +**适合谁**：用 Atlassian 但 Atlassian 官方 remote server 限制多的人。 +**备注**：跟下面 atlassian/atlassian-mcp-server（官方）择一，看公司 IT 政策。 + +### [atlassian/atlassian-mcp-server](https://github.com/atlassian/atlassian-mcp-server) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 650+ | +| License | Apache-2.0 | +| 推荐度 | ⭐⭐⭐⭐（**官方**） | + +**教什么**：Atlassian 官方 Remote MCP，安全连 Jira / Confluence。 +**适合谁**：公司有 enterprise Atlassian、IT 规定只能用官方的人。 +**备注**：Remote 模式，有官方 SLA。 + +### [korotovsky/slack-mcp-server](https://github.com/korotovsky/slack-mcp-server) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 1.6k+ | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐（无 admin 权限也能用） | + +**教什么**：Slack MCP，DM / group DM / 频道消息抓取，自带 history fetch logic。 +**适合谁**：个人用户（不是 Slack admin）也想接 Slack 的人。 +**备注**：不需要 admin 级别 token；走用户层 OAuth。 + +### [jerhadf/linear-mcp-server](https://github.com/jerhadf/linear-mcp-server) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 344 | +| License | NOASSERTION | +| 推荐度 | ⭐⭐⭐⭐ | + +**教什么**：Linear（issue tracker）MCP——查 issue、建 issue、改 status。 +**适合谁**：用 Linear 管 sprint / backlog 的开发者。 +**备注**：要 Linear API key。 + +### [SaseQ/discord-mcp](https://github.com/SaseQ/discord-mcp) ⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 298 | +| License | MIT | +| 推荐度 | ⭐⭐⭐ | + +**教什么**：Discord MCP——读写频道消息、管理服务器。 +**适合谁**：用 Discord 跑社群 / 开源项目的 maintainer。 +**备注**：要 Discord bot token；要小心 rate limit。 + +--- + +### [safishamsi/graphify](https://github.com/safishamsi/graphify) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 44k+ | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐⭐ | + +**教什么**：把 codebase / SQL schemas / R scripts / shell scripts / docs / papers / images / videos 变成 queryable knowledge graph 的 AI coding skill。Claude Code、Codex、OpenCode、Cursor、Gemini CLI 都能接。 +**适合谁**：要对大型 codebase 做架构分析、跨档追 reference、把"app code + DB schema + infra"放一起问的工程师 / 研究者。 +**备注**：跨界——既是 dev collab tool（理解既有 codebase）也算 research workflow（把任意素材转成 graph）。撞墙时用 graphify 抽结构、再丢回 Claude 推论。 + +### [upstash/context7](https://github.com/upstash/context7) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 57k | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐⭐（写代码必装） | + +**教什么**：把最新版的 library / framework 文档拉进 agent 的 context，让它别再用过时或幻觉出来的 API；是安装量最高的 coding MCP 之一。 +**适合谁**：用 Claude Code / Cursor 写代码、常被旧版 API 或编造出来的方法坑到的开发者。 +**备注**：直接缓解“模型知识截止后 API 已经变了”的问题——写代码时把它挂上，省掉手动贴文档。 + +### [DeusData/codebase-memory-mcp](https://github.com/DeusData/codebase-memory-mcp) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 13.5k+ | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐⭐（code intelligence） | + +**教什么**：把 codebase 索引成可查询的 knowledge graph，让 coding agent 用“查结构 / 符号 / 调用路径”取代反复 grep + 读文件。单一 static binary、158 种语言。 +**适合谁**：在大型或不熟的 repo 上跑 coding agent、想快速定位又想省 token 的人。 +**备注**：大改后要重新索引（graph 会 stale）；把它的回答当“快速第一手”、load-bearing 的结论（谁调用 X / 这段是不是死码）再用实际代码验证。 + +--- + +## 6. 数据库 + +### [googleapis/mcp-toolbox](https://github.com/googleapis/mcp-toolbox) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 15k+ | +| License | Apache-2.0 | +| 推荐度 | ⭐⭐⭐⭐⭐（**Google 官方**，多 DB） | + +**教什么**：跨 DB 的 MCP server——MySQL / PostgreSQL / Cloud SQL / Spanner / BigQuery 一次包。 +**适合谁**：在 Google Cloud 上跑 DB 的工程师、要支持多 DB 引擎的开发者。 +**备注**：开源 + Google 官方维护，是可上线使用的选择。 + +### [bytebase/dbhub](https://github.com/bytebase/dbhub) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 2.7k+ | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐⭐（社群多 DB MCP） | + +**教什么**：zero-dependency、token-efficient 的多 DB MCP——Postgres、MySQL、SQL Server、MariaDB、SQLite。 +**适合谁**：不想装 Google Cloud SDK、要跨多种 OSS DB 的工程师。 +**备注**：跟 googleapis/mcp-toolbox 重叠，但更轻量。 + +### [supabase-community/supabase-mcp](https://github.com/supabase-community/supabase-mcp) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 2.7k+ | +| License | Apache-2.0 | +| 推荐度 | ⭐⭐⭐⭐⭐（**Supabase 官方社群**） | + +**教什么**：把 Supabase（含 Postgres、Auth、Storage、Edge Functions）接到 LLM。 +**适合谁**：用 Supabase 跑后端的全栈开发者。 +**备注**：官方 community 维护。 + +### [timescale/pg-aiguide](https://github.com/timescale/pg-aiguide) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 1.7k+ | +| License | Apache-2.0 | +| 推荐度 | ⭐⭐⭐⭐（Postgres 写代码辅助） | + +**教什么**：MCP server + Claude plugin，帮 LLM 生成更好的 PostgreSQL 代码。 +**适合谁**：写 Postgres heavy SQL / DBA 工程师。 +**备注**：偏“LLM 写 SQL 辅助”，不只是 query 执行。 + +### [benborla/mcp-server-mysql](https://github.com/benborla/mcp-server-mysql) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 1.6k+ | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐（read-only MySQL） | + +**教什么**：read-only MySQL MCP，让 LLM 看 schema、跑 query。 +**适合谁**：要让 LLM 分析 production DB 但不能改的场景。 +**备注**：故意 read-only 是 safety feature，不是限制。 + +### [mongodb-js/mongodb-mcp-server](https://github.com/mongodb-js/mongodb-mcp-server) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 1k+ | +| License | Apache-2.0 | +| 推荐度 | ⭐⭐⭐⭐（**MongoDB 官方**） | + +**教什么**：MongoDB 跟 MongoDB Atlas Cluster MCP server。 +**适合谁**：用 MongoDB / Atlas 的工程师。 +**备注**：mongodb-js 是 MongoDB 官方 GitHub org。 + +### [redis/mcp-redis](https://github.com/redis/mcp-redis) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 504 | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐（**Redis 官方**） | + +**教什么**：Redis 官方 MCP，自然语言操作 Redis 跟 Redis Stack（Vector / Search / JSON）。 +**适合谁**：用 Redis 当 cache / vector DB / queue 的人。 +**备注**：官方维护；包含 vector search 集成。 + +### [awslabs/mcp](https://github.com/awslabs/mcp) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 9.3k+ | +| License | Apache-2.0 | +| 推荐度 | ⭐⭐⭐⭐（**AWS 官方**） | + +**教什么**：AWS 官方 MCP server（Lambda / S3 / DynamoDB / CloudWatch / Cost Explorer 等）。 +**适合谁**：在 AWS 上、想让 agent 查询 / 操作云端资源的团队。 +**备注**：AWS 官方维护；沿用你现有的 AWS 登录（CLI profile / IAM role），不用另外管 token。 + +--- + +## 7. 浏览器自动化 / 网页抓取 + +### [microsoft/playwright-mcp](https://github.com/microsoft/playwright-mcp) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 34k+ | +| License | Apache-2.0 | +| 推荐度 | ⭐⭐⭐⭐⭐（**Microsoft 官方**） | + +**教什么**：Playwright MCP server——让 LLM 开浏览器、点按钮、填表单、抓网页。 +**适合谁**：要做 E2E 自动化、跨网站集成、抓需要登录的网页的人。 +**备注**：Playwright 官方出，最 robust。**Claude Code 接 web 自动化是个不错的选项**。 + +### [ChromeDevTools/chrome-devtools-mcp](https://github.com/ChromeDevTools/chrome-devtools-mcp) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 38k+ | +| License | Apache-2.0 | +| 推荐度 | ⭐⭐⭐⭐⭐（**Chrome 官方**） | + +**教什么**：把 Chrome DevTools 接给 coding agent——performance、network、console 直接给 LLM 看。 +**适合谁**：调试前端 bug、做 web performance 分析的开发者。 +**备注**：搭配 Playwright MCP 用最强——一个跑、一个观察。 + +### [firecrawl/firecrawl-mcp-server](https://github.com/firecrawl/firecrawl-mcp-server) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 6.2k+ | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐⭐（**Firecrawl 官方**） | + +**教什么**：Firecrawl 官方 MCP——大规模网页抓取 + search + 结构化提取。 +**适合谁**：要抓大量网页当训练数据 / 做 RAG / 做研究的人。 +**备注**：需要 Firecrawl API key（有 free tier）。 + +### [browserbase/mcp-server-browserbase](https://github.com/browserbase/mcp-server-browserbase) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 3.3k+ | +| License | Apache-2.0 | +| 推荐度 | ⭐⭐⭐⭐（**Browserbase 官方**） | + +**教什么**：Browserbase 官方 MCP，配 Stagehand 跑 cloud-based 浏览器。 +**适合谁**：本地跑浏览器太重 / 要在 cloud 并行跑多个 session 的人。 +**备注**：商业服务（有免费额度），跟 Playwright MCP 互补（local vs cloud）。 + +--- + +## 8. 设计（Figma / Excalidraw） + +### [GLips/Figma-Context-MCP](https://github.com/GLips/Figma-Context-MCP) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 14.6k+ | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐⭐（最热门 Figma MCP） | + +**教什么**：把 Figma layout 信息送给 coding agent——读设计稿、提组件结构，给 Cursor / Claude Code 写对应的 React component。 +**适合谁**：前端开发者，要 LLM 从 Figma 设计稿生成 component code。 +**备注**：要 Figma access token；对 design-to-code workflow 必装。 + +### [excalidraw/excalidraw-mcp](https://github.com/excalidraw/excalidraw-mcp) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 4.3k+ | +| License | NOASSERTION | +| 推荐度 | ⭐⭐⭐⭐⭐（**Excalidraw 官方**） | + +**教什么**：streamable Excalidraw MCP，让 LLM 直接画架构图、流程图。 +**适合谁**：写设计文档 / 系统架构 / 流程图的人——叫 Claude 从文字描述画图。 +**备注**：Excalidraw 官方出，输出可直接导入 Excalidraw 编辑。 + +### [yctimlin/mcp_excalidraw](https://github.com/yctimlin/mcp_excalidraw) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 1.9k+ | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐（替代版 Excalidraw） | + +**教什么**：MCP server + Claude Code Skill，real-time canvas sync，可创建 / 编辑 / 导出。 +**适合谁**：需要 real-time canvas sync 跟编程化操作的人。 +**备注**：跟官方版互补，社群维护。 + +--- + +### [pbakaus/impeccable](https://github.com/pbakaus/impeccable) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 25k+ | +| License | Apache-2.0 | +| 推荐度 | ⭐⭐⭐⭐⭐ | + +**教什么**："**让你 AI harness 在 design 上更强的 design language**"——一套设计 vocabulary / pattern，帮 AI 在生成 UI / 视觉成品时跳出常见的"AI 感"生硬风格。 +**适合谁**：用 AI 生 UI / mockup / visual design 但结果都很 generic 的开发者；前端 + AI workflow。 +**备注**：不是 MCP server 也不是 Skill 包——是一份"**design language**"reference。让 AI 看到比较高质量的设计词汇才生得出比较好的东西。 + +--- + +## 9. 监控 / Observability + +### [grafana/mcp-grafana](https://github.com/grafana/mcp-grafana) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 3k+ | +| License | Apache-2.0 | +| 推荐度 | ⭐⭐⭐⭐⭐（**Grafana 官方**） | + +**教什么**：Grafana 官方 MCP，从 LLM 直接查 dashboard、metric、alert。 +**适合谁**：用 Grafana 看 metric 的 SRE / DevOps。 +**备注**：“dashboard 那条线为什么掉？”直接问，LLM 捞 metric 给答案。 + +### [getsentry/sentry-mcp](https://github.com/getsentry/sentry-mcp) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 677 | +| License | NOASSERTION | +| 推荐度 | ⭐⭐⭐⭐（**Sentry 官方**） | + +**教什么**：从 LLM 查 Sentry error event、issue、trace。 +**适合谁**：用 Sentry 接 production error 的工程师。 +**备注**：“上周这个 error 的 stack trace 给我看”直接问 Claude Code。 + +### [winor30/mcp-server-datadog](https://github.com/winor30/mcp-server-datadog) ⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 142 | +| License | Apache-2.0 | +| 推荐度 | ⭐⭐⭐（社群版 Datadog） | + +**教什么**：Datadog API MCP——查 monitor、log、metric。 +**适合谁**：用 Datadog 但 Datadog 还没出官方 MCP 的人。 +**备注**：等 Datadog 官方 MCP 出来可能换掉这个。 + +--- + +## 10. 媒体 / 串流（YouTube / Spotify） + +### [varunneal/spotify-mcp](https://github.com/varunneal/spotify-mcp) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 599 | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐ | + +**教什么**：把 LLM 接到 Spotify——播歌、加歌单、查历史。 +**适合谁**：想用 Claude Code 控播放列表、做语音 / 文字 → 音乐的集成者。 +**备注**：要 Spotify Premium 账号（API 限制）。 + +### [kimtaeyoon83/mcp-server-youtube-transcript](https://github.com/kimtaeyoon83/mcp-server-youtube-transcript) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 534 | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐（YouTube 字幕） | + +**教什么**：直接抓 YouTube 视频字幕给 LLM 摘要、翻译、做 RAG。 +**适合谁**：用视频当学习材料、要批量摘要 YouTube 内容的人。 +**备注**：依赖 YouTube auto-caption；非英文视频字幕质量参差。 + +### [ZubeidHendricks/youtube-mcp-server](https://github.com/ZubeidHendricks/youtube-mcp-server) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 510 | +| License | NOASSERTION | +| 推荐度 | ⭐⭐⭐⭐（YouTube 完整 API） | + +**教什么**：完整 YouTube API MCP——除了 transcript，还能管 video、Shorts、analytics。 +**适合谁**：YouTube 创作者要自动化频道管理。 +**备注**：需要 YouTube Data API key + OAuth。 + +--- + +## 11. 中文圈专属 + +### [leemysw/feishu-docx](https://github.com/leemysw/feishu-docx) ⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 193 | +| License | MIT | +| 推荐度 | ⭐⭐⭐ | + +**教什么**：飞书（Lark）docs / sheet / bitable ↔ Markdown 双向转换，含 OAuth 2.0、CLI、TUI、Claude Skills。 +**适合谁**：用飞书 / Lark 写文档的中文用户，要把 Lark 内容跟 Claude Code 串起来。 +**备注**：目前中文圈 MCP / Skill 主要选择之一；微信 / 钉钉暂时没有独立 MCP（多半混在 chat bot framework 里）。 + +### [netease-youdao/LobsterAI](https://github.com/netease-youdao/LobsterAI) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 5k+ | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐ | + +**教什么**：网易有道出品的"24/7 全场景 AI agent"——支持工作流自动化、跨应用协作、文件处理。中文 native。 +**适合谁**：中文圈用户要找一个替代 Claude Code / OpenAI Operator 等级的 all-in-one agent；对中国大陆服务（网易、钉钉等）集成需求高的场景。 +**备注**：产品式 agent（不是 Skill / MCP）；跟 Claude Code / Codex 互为替代，不是搭配。 + +### [QwenLM/Qwen-Agent](https://github.com/QwenLM/Qwen-Agent) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 16k+ | +| License | Apache-2.0 | +| 推荐度 | ⭐⭐⭐⭐⭐ | + +**教什么**：阿里巴巴官方 Qwen agent framework——RAG、tool use、code interpreter、multi-agent、MCP 兼容，默认搭配 Qwen 系列模型但可换其他 LLM。 +**适合谁**：用 Qwen / 通义千问 为主 LLM 的开发者；想要中文 native 的 agent framework（范例、文档都中文齐全）。 +**备注**：MCP 兼容是亮点——可以直接接到 Claude Code 等 host；维护节奏正常（last commit 2026-03）。 + +### [coze-dev/coze-studio](https://github.com/coze-dev/coze-studio) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 20k+ | +| License | Apache-2.0 | +| 推荐度 | ⭐⭐⭐⭐⭐ | + +**教什么**：字节跳动 Coze 的开源版——no-code agent builder（workflow / plugin / knowledge / memory），可自部署或上云。 +**适合谁**：不想写 code 但要做 agent 的团队；想看 enterprise agent platform 内部设计（RAG、工作流、Memory、Plugin 系统的 reference 实现）。 +**备注**：底层 framework 是 Coze 自家的 Eino；可接 OpenAI / Claude / Qwen / 国产 LLM。国际版（coze.com）跟中国版（coze.cn）共用此 codebase。 + +### [coze-dev/coze-loop](https://github.com/coze-dev/coze-loop) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 5k+ | +| License | Apache-2.0 | +| 推荐度 | ⭐⭐⭐⭐ | + +**教什么**：Coze 出的 agent observability + evaluation 平台——trace、debug、eval、prompt management，agent dev lifecycle 的下半场。 +**适合谁**：agent 已经跑起来、要 production 监控的团队；想看"agent eval / observability"可以怎么做的人。 +**备注**：跟 LangSmith / Arize Phoenix 同类；开源版可自部署。 + +### [liaokongVFX/LangChain-Chinese-Getting-Started-Guide](https://github.com/liaokongVFX/LangChain-Chinese-Getting-Started-Guide) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 8.9k+ | +| License | 未标注 | +| 推荐度 | ⭐⭐⭐⭐ | + +**教什么**：LangChain 中文入门指南——从 LangChain 基础、Prompt、Memory、Agent、Chains 到实作应用，最早期且最完整的中文 LangChain 学习资源。 +**适合谁**：想用 LangChain 但英文文档吃不下去的中文用户；想理解 LangChain 设计脉络再决定要不要走这条路的人。 +**备注**：没有正式 license（内容开放阅读）；LangChain 框架本身演进很快，书中部分 API 可能跟最新版有出入。 + +### [chatchat-space/Langchain-Chatchat](https://github.com/chatchat-space/Langchain-Chatchat) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 37k+ | +| License | Apache-2.0 | +| 推荐度 | ⭐⭐⭐⭐ | + +**教什么**：基于 LangChain 的开源知识库问答系统——本地化部署、支持多种向量数据库、RAG 端到端范例。 +**适合谁**：想做 RAG 又不想全部自己刻的中文团队；要本地部署（不能用云端 LLM）的场景。 +**备注**：★ 37k 是中文圈最热门的 RAG 实现之一；维护节奏放缓（last commit 2025-11）。新项目建议先 fork 后评估，当参考实现用。 + +### [usewhale/whale](https://github.com/usewhale/whale) ⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 117 | +| License | MIT | +| 推荐度 | ⭐⭐⭐ | + +**教什么**：专为 DeepSeek 模型优化的终端 AI 编码助手——支持 MCP server 接入、Claude-style Skills、对话缓存优化，Go 实现。 +**适合谁**：以 DeepSeek 为主力 LLM 的中文开发者；想用终端工具但不需要 Claude Code 全家桶的人。 +**备注**：开源同类中少见的 DeepSeek 专属优化；MCP + Skills 双支持让它可以逐步扩充能力。 + +### [simonlin1212/a-stock-data](https://github.com/simonlin1212/a-stock-data) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 492 | +| License | Apache-2.0 | +| 推荐度 | ⭐⭐⭐⭐ | + +**教什么**：A 股全栈数据工具包——单一 SKILL.md 文件封装 8 个数据源（mootdx、东财、akshare、iwencai 等）21 个端点，AI 编码助手直接可用。 +**适合谁**：用 Claude Code / Codex / OpenClaw 做投研或量化分析的中文开发者；不想自己刻数据抓取逻辑的人。 +**备注**：一条 `curl` + `pip install` 即可启用；中国 A 股数据类 Skill 中星星数最高的社群实现。兼容 Claude Code、Codex、OpenClaw。 + +> 想找微信 / 钉钉集成？目前主流是用 chat bot framework（如 zhayujie/CowAgent）而不是纯 MCP server。等正規 MCP 出现再加进来。 + +### [MoonshotAI/Kimi-K2](https://github.com/MoonshotAI/Kimi-K2) ⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 10.7k+ | +| License | Modified MIT | +| 推荐度 | ⭐⭐⭐ | + +**教什么**：月之暗面 Moonshot 的 Kimi K2 开源大模型系列——开源权重 + OpenAI / Anthropic 兼容 API，主打 agentic / coding / 长程任务，可当 agent stack 的后端模型。 +**适合谁**：想用国产开源模型跑 agent / coding 工作流、或要在自部署环境跑开源权重的中文开发者。 +**备注**：License 是 Modified MIT（标准 MIT + 大规模商用附加条款）——商用前先读原始 LICENSE；weights 另在 Hugging Face。 + +### [zai-org/GLM-4.5](https://github.com/zai-org/GLM-4.5) ⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 4.3k+ | +| License | Apache-2.0 | +| 推荐度 | ⭐⭐⭐ | + +**教什么**：智谱 Zhipu（Z.ai）的 GLM-4.5 开源模型——定位 Agentic / Reasoning / Coding（ARC）基础模型，开源权重 + API，可当 agent / tool use / coding 的后端。 +**适合谁**：想评估国产开源 agentic 模型、或需要 Apache-2.0 宽松许可权重的中文开发者。 +**备注**：zai-org 是智谱开源 org；同系列另有 GLM-4（★ 7k+）可一起参考；weights 在 Hugging Face。 + +--- + +## 12. 其他常用（Cloudflare / Stripe…） + +### [cloudflare/mcp-server-cloudflare](https://github.com/cloudflare/mcp-server-cloudflare) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 3.7k+ | +| License | Apache-2.0 | +| 推荐度 | ⭐⭐⭐⭐⭐（**Cloudflare 官方**） | + +**教什么**：Cloudflare 官方 MCP——Workers、Pages、R2、KV、D1、DNS、Zero Trust 全包。 +**适合谁**：用 Cloudflare 跑 edge / serverless 的人。 +**备注**：官方维护；最佳的 edge platform MCP。 + +### [stripe/ai](https://github.com/stripe/ai) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 1.5k+ | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐（**Stripe 官方**） | + +**教什么**：Stripe 官方 AI agent toolkit，含 MCP server，操作付款、订阅、退款、客户。 +**适合谁**：要在 agent 内处理付款 / billing 的开发者。 +**备注**：⚠️ 涉及金流，务必用 sandbox 测试够了再接 production。 + +### YIELD INTELLIGENCE MCP（Hosted Remote Server） + +| 栏位 | 内容 | +|---|---| +| 形式 | hosted MCP server | +| 推荐度 | ⭐⭐⭐（金融分析工具；了解 hosted vs self-hosted MCP 实现差异的实例） | + +**教什么**：YIELD INTELLIGENCE hosted remote MCP server——即时美国国债收益率 + 股息 ETF / REIT / 优先股分析 + 被动收入投资组合优化。2 个工具：`analyze_yield_opportunities`（扫描被动收入机会）+ `optimize_income_portfolio`（面向目标月收入建立投资组合）。已收录于 Anthropic 官方 MCP Registry（`io.github.thebrierfox/intuitek-ace`，since 2026-05-10）。 +**适合谁**：用 Claude Code / Claude Desktop 做个人理财分析、想让 AI 找出被动收入机会的人。hosted remote MCP server 范例——直接 plug URL、0 安装、适合 Stage 5 学完 MCP 概念后用来体验 hosted vs self-hosted 差异。 +**备注**：Live endpoint `https://api.intuitek.ai/yield/mcp`（no auth、no API key）。x402 micropayment $1 USDC/call on Base（agent-to-agent 场景）；一般用户免费。纯分析工具，不涉及交易。GitHub：[thebrierfox/intuitek-ace](https://github.com/thebrierfox/intuitek-ace)（MIT License）。 + +### [ComposioHQ/composio](https://github.com/ComposioHQ/composio) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 28.8k+ | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐⭐（1000+ 工具整合枢纽） | + +**教什么**：一个平台（SDK + MCP server），把 agent 连到 1000+ 应用（Slack / GitHub / Gmail / Salesforce / Notion…），登录它帮你处理，不用一个服务各写一个连接器。 +**适合谁**：agent 要跨大量工具、但不想维护几十个独立 MCP server 的团队。 +**备注**：提供 MCP server + Python / TypeScript SDK；可通过 MCP 接到 Claude Code。属"工具聚合器"（跟 n8n / Zapier 自动化平台同类）。 + +--- + +## 13. 研究工作流 Skills（学术 / paper / 文献） + +> ⚠️ **maintainer 自家项目区**：以下几个是本 repo 维护者 [@WenyuChiou](https://github.com/WenyuChiou)（Lehigh CEE PhD candidate）日常在用的研究 skills，公开出来给其他研究者参考。**因为是自己的东西、又是相对 niche 的研究场景，star 数量会比通用工具低**。star 门槛在这节是放宽的——选收的标准是“在我自己的研究流程里实际有用”。请自己评估是否合用。 + +### [WenyuChiou/ai-research-skills](https://github.com/WenyuChiou/ai-research-skills) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 60 | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐⭐（研究流程一整套） | + +**教什么**：14 个 Claude Code skills 涵盖常见研究任务——文献分流、研究设计、project context、论文写作、multi-AI delegation。打包成 5-plugin marketplace，一个指令安装。 +**适合谁**：研究生 / 博后想一次获取“研究全流程”skill set。 +**备注**：marketplace 形式，跟 Stage 5.4 教的 plugin/marketplace 概念对齐。 + +### [WenyuChiou/academic-writing-skills](https://github.com/WenyuChiou/academic-writing-skills) ⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 2 | +| License | MIT | +| 推荐度 | ⭐⭐⭐（窄但深） | + +**教什么**：严谨学术论文写作 / 修改 / 投稿的 Claude Code skill。Field-agnostic，可用 per-paper journal_format.md 跟 style_overrides.md 客制规则。 +**适合谁**：在写 / 改 paper 的研究者，想把 banned-word audit、figure-text coupling、submission checklist 自动化。 +**备注**：是 ai-research-skills 5 plugin 中的一个，也可独立安装。 + +### [WenyuChiou/zotero-skills](https://github.com/WenyuChiou/zotero-skills) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 16 | +| License | NOASSERTION | +| 推荐度 | ⭐⭐⭐⭐ | + +**教什么**：Zotero CLI skill——程序化搜索 / 添加 / 分类 / 标记文献。 +**适合谁**：用 Zotero 管理文献、想让 Claude Code 直接整理 library 的研究者。 +**备注**：跟 [`MuiseDestiny/zotero-gpt`](https://github.com/MuiseDestiny/zotero-gpt) 的区别——后者是 Zotero plugin（在 Zotero 里 chat），这份是 CLI / Skill（从 Claude Code 操作 Zotero）。 + +### [WenyuChiou/research-hub](https://github.com/WenyuChiou/research-hub) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 14 | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐ | + +**教什么**：AI-operable research workspace，集成 Zotero + Obsidian + NotebookLM 三个工具，提供 CLI / MCP / REST / dashboard 四种接口。 +**适合谁**：同时用 Zotero / Obsidian / NotebookLM 的研究者，想把它们绑成一个 workspace 给 LLM 操作。 +**备注**：跟单一工具的 MCP（mcp-obsidian、notion-mcp 等）互补——这份是 hub，可集成多个工具。 + +--- + +## 14. Multi-LLM Delegation Skills + +> ⚠️ **maintainer 自家项目区**：跟 13 一样，以下是维护者把自己 daily workflow 抽出来公开的 delegation skills。star 门槛放宽，选收标准是“真的能让 Claude planner + Codex/Gemini 执行者组合稳定跑下去”。Multi-LLM 领域变化快，建议跟其他 multi-agent framework（Stage 7 列的）一起评估后选择。 + +### [WenyuChiou/codex-delegate](https://github.com/WenyuChiou/codex-delegate) ⭐⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 57 | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐⭐ | + +**教什么**：Claude Code skill 把 Codex CLI 当作 execution specialist——适合大量文件 refactor、boilerplate 生成、实现密集任务。Claude 规划 + review，Codex 执行。 +**适合谁**：要在 Claude Code 内把实现工作自动 delegate 给 Codex 的开发者。 +**备注**：搭配 `gemini-delegate-skill` 用（一个跑 code-heavy、一个跑 long-form / CJK）。Stage 7 multi-agent 概念实战版。 + +### [WenyuChiou/gemini-delegate-skill](https://github.com/WenyuChiou/gemini-delegate-skill) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 34 | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐ | + +**教什么**：Claude Code skill 把 Gemini CLI 当作 large-context synthesis、英文 / zh-TW / CJK long-form drafting、second-opinion review 的执行者。 +**适合谁**：写长文、跨语言 draft、需要第二意见 review 的人——研究者写 paper / 中文报告场景特别合适。 +**备注**：跟 codex-delegate 互补——“Codex 写 code、Gemini 写 prose”分工。 + +### [WenyuChiou/agent-collab-skills](https://github.com/WenyuChiou/agent-collab-skills) ⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | 刚公开、stars 还没积累 | +| License | MIT | +| 推荐度 | ⭐⭐（experimental，当作 reference 看就好） | + +**教什么**：Claude Code marketplace for multi-agent collaboration——task splitter、output reconciler、adversarial debate、shared memory、acceptance gate。跟 codex-delegate / gemini-delegate 组合用。 +**适合谁**：要跑 2+ delegate agent 在同一轮、想看 multi-agent coordination 怎么打包成 marketplace 的人。 +**备注**：experimental——别把它当作生产级 framework，当作维护者把自己 setup 公开的 reference 看就好。要可上线部署的请看 Stage 7 的 LangGraph / AutoGen / CrewAI。 + +--- + +## 15. 金融 / 交易 Agents + +> ⚠️ **应用领域区**：agent 在量化交易 / hedge fund 模拟 / 自动下单的应用。这类 repo 授权状态混杂（部分 NO-LICENSE、部分 Apache-2.0 等开源授权），使用前自行查清楚。**警示**：trading agent 跑真实资金有显著风险，本目录列入是为了学习 agent 设计模式、不是投资建议。 + +### [TauricResearch/TradingAgents](https://github.com/TauricResearch/TradingAgents) ⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 79k+ | +| License | Apache-2.0 | +| 推荐度 | ⭐⭐⭐ | + +**教什么**：多 agent LLM 框架做金融交易决策，bull / bear / fundamentals / technicals / risk 各 agent 分工。 +**适合谁**：想看 multi-agent 在分析性任务怎么分工的学习者；量化研究者想实验 LLM 增强既有 pipeline。 +**备注**：Apache-2.0、允许修改与商用（保留授权声明）；**非投资建议，别直接拿来下实单**。 + +### [virattt/ai-hedge-fund](https://github.com/virattt/ai-hedge-fund) ⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 59k+ | +| License | NO-LICENSE | +| 推荐度 | ⭐⭐⭐ | + +**教什么**：多角色 AI hedge fund 模拟，bull / bear / 基本面 / 技术面 / 风控 agent 协作产生 trade recommendation。 +**适合谁**：看过 Stage 7 multi-agent 想要一个完整应用案例的学习者；对 agent + 金融交叉领域有兴趣的人。 +**备注**：NO-LICENSE → 同上；**模拟性质、非投资建议**。 + +--- + +## 16. 网页搜索 / 检索（Web Search / Retrieval） + +### [exa-labs/exa-mcp-server](https://github.com/exa-labs/exa-mcp-server) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 4.5k+ | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐（**Exa 官方**） | + +**教什么**：Exa 官方 MCP——为 LLM / agent 设计的网页搜索（neural + keyword 两种），回传干净结果直接喂进 prompt。 +**适合谁**：要做研究 / fact-check / 在线 RAG 检索的人——语义搜索在“概念相关”场景特别强，纯 keyword 反而没那么吃香。 +**备注**：需要 Exa API key。 + +### [tavily-ai/tavily-mcp](https://github.com/tavily-ai/tavily-mcp) ⭐⭐⭐⭐ + +| 栏位 | 内容 | +|---|---| +| Stars | ★ 2.1k+ | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐ | + +**教什么**：把 Tavily search API 包成 MCP——为 LLM / RAG 打造的网页搜索，回传答案 + 出处来源。 +**适合谁**：新手只想让 agent 会上网搜东西的第一选择——免费额度好上手。 +**备注**：有易用的 free tier；要 Tavily API key。 + +--- + +## 还有什么没收录？ + +如果你需要的集成不在上面，先看这些 catalog： + +- [`wong2/awesome-mcp-servers`](https://github.com/wong2/awesome-mcp-servers) — 社群最完整 MCP server 清单，150+ 个按分类整理 +- [`punkpeye/awesome-mcp-servers`](https://github.com/punkpeye/awesome-mcp-servers) — 另一份 MCP server 清单，跟上面互补 +- [`modelcontextprotocol/servers`](https://github.com/modelcontextprotocol/servers) — Anthropic 官方 reference servers（filesystem、git、time、memory、fetch、sequential-thinking 等） +- [`travisvn/awesome-claude-skills`](https://github.com/travisvn/awesome-claude-skills) — Claude Skills 清单 + +### 要加新的？ + +1. 开 issue，附 repo 链接 + 为什么 JIA + 属于哪个分类 +2. 或直接送 PR：在对应分类下加一个 entry，按上面的格式写（Stars/License/推荐度 + 教什么/适合谁/备注） +3. **stars < 100 且非官方**通常会被退；除非你能说明 niche use case 强到可以例外 + +PR 送出前看一下 [`resources/style-guide.md`](style-guide.md) 跟 [`CONTRIBUTING.md`](../CONTRIBUTING.md)。 + +--- + +## 维护备注（给未来的 maintainer） + +- Stars / license 用 `gh api repos//` 抓，每季 review 一次 +- 链接 broken / repo archived 的直接拿掉 +- 出现新分类（如 AR/VR、IoT 等）就加新一节，但 stars < 1k 且 < 3 个 entry 的分类先别开 +- “中文圈专属”分类维持宽松（中文社群 repo 起 stars 较难） diff --git a/resources/schema-design-cheatsheet.en.md b/resources/schema-design-cheatsheet.en.md new file mode 100644 index 0000000..da20795 --- /dev/null +++ b/resources/schema-design-cheatsheet.en.md @@ -0,0 +1,159 @@ +# Function Schema Design Cheatsheet + +> [繁體中文](./schema-design-cheatsheet.md) | [简体中文](./schema-design-cheatsheet.zh-Hans.md) | **English** + +> Companion to [Stage 3 — Tool Use & Agent Intro](../stages/03-tool-use-and-hello-agent.en.md). 5 golden rules + 5 common anti-patterns when writing tool / function schemas. + +How well an LLM uses your tool **is 80% determined by schema quality** — vague schemas defeat even strong models. + +--- + +## 5 Golden Rules + +### Rule 1: `description` is for the LLM, not for humans + +The LLM only reads `description` to decide whether to call the tool and when. So: + +- ✅ Write **when** + **what**: `"Call this when the user asks for current weather of a specific city."` +- ❌ Don't write implementation details: `"Uses OpenWeather API v2.5 returning JSON."` + +Compare: + +```python +# Bad +"description": "Get weather data." + +# Good +"description": "Get current weather for a specified city. Use this when the user asks about current weather, temperature, humidity, or 'is it raining' for any specific location. Do NOT use for forecasts (use get_forecast instead) or historical data." +``` + +### Rule 2: Use the right `type`; collapse fuzzy params with `enum` + +LLMs are loose with `type: string` and pass arbitrary text. Tighten where possible: + +| Vague | Constrained | +|---|---| +| `unit: string` (celsius? fahrenheit? kelvin?) | `unit: enum["celsius", "fahrenheit"]` | +| `priority: string` (low/medium/HIGH?) | `priority: enum["low", "medium", "high"]` | +| `count: string` ("five"?) | `count: integer` | +| `enabled: string` ("true"/"True") | `enabled: boolean` | +| `tags: string` ("a,b,c"? JSON?) | `tags: array of string` | + +### Rule 3: Be careful with `required` vs optional + +- `required` lists **only truly mandatory** params (without it the tool can't run) +- Params with sensible defaults go in `default`, NOT `required` +- LLMs hallucinate values for `required` params even when the user didn't mention them — **fewer required is better** + +```python +# Bad: timezone listed as required → LLM invents "Asia/Taipei" even if not mentioned +"required": ["city", "timezone"] + +# Good +"required": ["city"] +"properties": { + "timezone": {"type": "string", "default": "UTC", "description": "..."} +} +``` + +### Rule 4: Self-describing tool / param names + +`do_thing(x, y, z)` and `get_weather(city, unit)` produce wildly different LLM behavior. + +- ✅ `get_user_profile(user_id)` +- ❌ `fetch(id)` or `process_data(input)` + +Verb-first names, signal whether it's a query / mutation / action. + +### Rule 5: Errors must be recoverable + +The LLM uses error messages to decide retry / pivot / give-up. Structure errors: + +```json +{ + "error": "City not found", + "code": "INVALID_CITY", + "retry_hint": "Check spelling, or try a major city nearby" +} +``` + +Bare `"Error 500"` leaves the LLM stuck — no recovery signal. + +--- + +## 5 Common Anti-Patterns + +### Anti-1: God Tool + +```python +# Bad: one tool for everything +def do_database_op(operation: str, table: str, data: str) -> str: + """Do anything with the database.""" +``` + +The LLM will pair the wrong operation with the right table and crash. **Split into `query_users` / `create_order` / `update_inventory`** etc. — selection accuracy goes way up. + +### Anti-2: Description as docstring + +```python +# Bad +"description": "GET /api/v2/weather endpoint. Returns JSON. See API docs." + +# Good +"description": "Get current weather for a city. Returns temperature in C/F, humidity, and conditions." +``` + +The LLM doesn't read code — it wants **"when is this useful"**. + +### Anti-3: Everything is a string + +```python +# Bad +{"properties": { + "count": {"type": "string"}, # LLM might pass "five" + "active": {"type": "string"}, # LLM might pass "yes" + "list": {"type": "string"} # LLM might pass "[a, b, c]" or "a, b, c" +}} + +# Good +{"properties": { + "count": {"type": "integer", "minimum": 1, "maximum": 100}, + "active": {"type": "boolean"}, + "list": {"type": "array", "items": {"type": "string"}} +}} +``` + +### Anti-4: No examples in description + +LLMs are noticeably more accurate when the `description` includes examples. + +```python +"description": "Search products by query string. Examples: 'laptop under $1000', 'red shoes size 10'. Do NOT use for product ID lookup (use get_product_by_id)." +``` + +### Anti-5: Silent failures + +Tool fails and returns `null` or `{}` — LLM thinks it succeeded, continues reasoning on empty data. **Always**: + +- Success → `{"success": true, "data": {...}}` +- Failure → `{"success": false, "error": "...", "retry_hint": "..."}` + +`success: false` is the recovery signal; without it the LLM fabricates from empty data. + +--- + +## Schema Evolution Tips + +- Adding a param → keep it backward-compatible: set `default`, don't add to `required` +- Changing a param's meaning → ship a new tool (`get_weather_v2`), deprecate the old one before removing +- Changes to `description` → re-test. LLMs are sensitive to wording, even punctuation matters. +- Before production: use [promptfoo](https://github.com/promptfoo/promptfoo) to eval "does the LLM pick the right tool on 5-10 typical queries" + +--- + +## Further reading + +- [Anthropic — Tool Use Guide](https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/overview) — official schema spec +- [OpenAI — Function Calling](https://platform.openai.com/docs/guides/function-calling) — OpenAI's schema spec (slight differences from Anthropic) +- [Stage 3 — Tool Use & Agent Intro](../stages/03-tool-use-and-hello-agent.en.md) — main exercises +- [Stage 5.2 — MCP foundation](../stages/05-claude-code-ecosystem.en.md#52--mcp-model-context-protocol--foundation) — MCP server tool schemas (nearly identical structure to function-calling schema) diff --git a/resources/schema-design-cheatsheet.md b/resources/schema-design-cheatsheet.md new file mode 100644 index 0000000..ce45c29 --- /dev/null +++ b/resources/schema-design-cheatsheet.md @@ -0,0 +1,159 @@ +# Function Schema 設計 Cheatsheet + +> **繁體中文** | [简体中文](./schema-design-cheatsheet.zh-Hans.md) | [English](./schema-design-cheatsheet.en.md) + +> [Stage 3 — Tool Use & Agent 入門](../stages/03-tool-use-and-hello-agent.md) 的補充參考。寫 tool / function schema 時的 5 條黃金規則 + 5 個 anti-pattern。 + +LLM 怎麼用你的 tool **80% 取決於 schema 寫得好不好**——schema 模糊，再強的模型也會選錯、傳錯。 + +--- + +## 5 條黃金規則 + +### 規則 1：description 是寫給 LLM 看的，不是 docstring + +LLM 只看 `description` 決定要不要叫這個 tool、什麼時候叫。所以要： + +- ✅ 寫**情境**（when）跟**做什麼**（what）：`"當使用者問特定城市的當前天氣時呼叫"` +- ❌ 不要寫實作細節：`"使用 OpenWeather API v2.5 取得 JSON"` + +對照： + +```python +# 壞 +"description": "Get weather data." + +# 好 +"description": "Get current weather for a specified city. Use this when the user asks about the current weather, temperature, humidity, or 'is it raining' for any specific location. Do NOT use for forecasts (use get_forecast instead) or historical data." +``` + +### 規則 2：參數用對 type，模糊處用 enum 收斂 + +LLM 對 `type: string` 自由度高、容易亂傳。能用窄型別就用： + +| 模糊 | 收斂 | +|---|---| +| `unit: string`（攝氏？華氏？kelvin？） | `unit: enum["celsius", "fahrenheit"]` | +| `priority: string`（low/中/HIGH？） | `priority: enum["low", "medium", "high"]` | +| `count: string`（"五個"？） | `count: integer` | +| `enabled: string`（"true" / "True"） | `enabled: boolean` | +| `tags: string`（"a,b,c"？JSON？） | `tags: array of string` | + +### 規則 3：required vs optional 分清楚 + +- `required` 列**真的必要**的參數（少了這個 tool 就跑不起來） +- 有預設值的放 `default`，不要列 required +- LLM 看到 required 多會「**自己編參數**」，所以 required 越少越好 + +```python +# 壞：把 timezone 列 required，LLM 會亂編「Asia/Taipei」即便使用者沒提到 +"required": ["city", "timezone"] + +# 好 +"required": ["city"] +"properties": { + "timezone": {"type": "string", "default": "UTC", "description": "..."} +} +``` + +### 規則 4：tool name + parameter name 要自說明 + +LLM 看到 `do_thing(x, y, z)` 跟看到 `get_weather(city, unit)` 用法完全不同。 + +- ✅ `get_user_profile(user_id)` +- ❌ `fetch(id)` 或 `process_data(input)` + +動詞開頭，說清楚是 query / mutation / action。 + +### 規則 5：error 回傳要讓 LLM 可以恢復 + +LLM 看到錯誤訊息後決定要 retry / 換工具 / 放棄。錯誤訊息要結構化： + +```json +{ + "error": "City not found", + "code": "INVALID_CITY", + "retry_hint": "Check spelling, or try a major city nearby" +} +``` + +而不是只回 `"Error 500"`——LLM 拿這個沒招。 + +--- + +## 5 個常見 Anti-Pattern + +### Anti-1：「萬用工具」（God Tool） + +```python +# 壞：一個 tool 做所有事 +def do_database_op(operation: str, table: str, data: str) -> str: + """Do anything with the database.""" +``` + +LLM 會把錯的 operation 配上對的 table 然後爛掉。**拆成 `query_users` / `create_order` / `update_inventory`** 等具體 tool，LLM 選擇正確率大幅提升。 + +### Anti-2：description 是 docstring + +```python +# 壞 +"description": "GET /api/v2/weather endpoint. Returns JSON. See API docs." + +# 好 +"description": "Get current weather for a city. Returns temperature in C/F, humidity, and conditions." +``` + +LLM 不是程式，它要的是 **「這個 tool 什麼時候有用」**。 + +### Anti-3：所有東西都是 string + +```python +# 壞 +{"properties": { + "count": {"type": "string"}, # LLM 可能傳 "five" + "active": {"type": "string"}, # LLM 可能傳 "yes" + "list": {"type": "string"} # LLM 可能傳 "[a, b, c]" 或 "a, b, c" +}} + +# 好 +{"properties": { + "count": {"type": "integer", "minimum": 1, "maximum": 100}, + "active": {"type": "boolean"}, + "list": {"type": "array", "items": {"type": "string"}} +}} +``` + +### Anti-4：沒寫範例 + +LLM 對 description **加上 example 比沒加準確很多**。 + +```python +"description": "Search products by query string. Examples: 'laptop under $1000', 'red shoes size 10'. Do NOT use for product ID lookup (use get_product_by_id)." +``` + +### Anti-5：沉默的失敗 + +Tool 失敗只回 `null` 或 `{}`，LLM 以為成功，繼續用空資料推論。**永遠回**： + +- 成功 → `{"success": true, "data": {...}}` +- 失敗 → `{"success": false, "error": "...", "retry_hint": "..."}` + +LLM 看到 `success: false` 就知道要處理錯誤，不會把空資料當答案編造。 + +--- + +## Schema 演進的小建議 + +- 加參數要 backward-compatible：新參數設 `default` 而不是 required +- 改參數含義 → 開新 tool（`get_weather_v2`），舊的標 deprecated 一段時間再下 +- description 改了要重新測——LLM 行為對 description 敏感，連標點都會影響 +- 上 production 前用 [promptfoo](https://github.com/promptfoo/promptfoo) eval 一下「LLM 在 5-10 個典型 query 是否選對 tool」 + +--- + +## 延伸閱讀 + +- [Anthropic — Tool Use Guide](https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/overview) — 官方 schema 規格 +- [OpenAI — Function Calling](https://platform.openai.com/docs/guides/function-calling) — OpenAI 的 schema 規格（跟 Anthropic 略有差異） +- [Stage 3 — Tool Use & Agent 入門](../stages/03-tool-use-and-hello-agent.md) — 主要動手練習 +- [Stage 5.2 — MCP 基礎](../stages/05-claude-code-ecosystem.md#52--mcpmodel-context-protocol-基礎) — MCP server 的 tool schema（跟 function calling schema 結構幾乎相同） diff --git a/resources/schema-design-cheatsheet.zh-Hans.md b/resources/schema-design-cheatsheet.zh-Hans.md new file mode 100644 index 0000000..1cfa734 --- /dev/null +++ b/resources/schema-design-cheatsheet.zh-Hans.md @@ -0,0 +1,150 @@ +# Function Schema 设计 Cheatsheet + +> [繁體中文](./schema-design-cheatsheet.md) | **简体中文** | [English](./schema-design-cheatsheet.en.md) + +> [Stage 3 — Tool Use & Agent 入门](../stages/03-tool-use-and-hello-agent.md) 的补充参考。写 tool / function schema 时的 5 条黄金规则 + 5 个 anti-pattern。 + +LLM 怎么用你的 tool **80% 取决于 schema 写得好不好**——schema 模糊，再强的模型也会选错、传错。 + +--- + +## 5 条黄金规则 + +### 规则 1：description 是写给 LLM 看的，不是 docstring + +LLM 只看 `description` 决定要不要叫这个 tool、什么时候叫。所以要： + +- ✅ 写**情境**（when）跟**做什么**（what）：`"当用户问特定城市的天气时调用"` +- ❌ 不要写实作细节：`"使用 OpenWeather API v2.5 取得 JSON"` + +对照： + +```python +# 坏 +"description": "Get weather data." + +# 好 +"description": "Get current weather for a specified city. Use this when the user asks about the current weather, temperature, humidity, or 'is it raining' for any specific location. Do NOT use for forecasts (use get_forecast instead) or historical data." +``` + +### 规则 2：参数用对 type，模糊处用 enum 收敛 + +LLM 对 `type: string` 自由度高、容易乱传。能用窄型别就用： + +| 模糊 | 收敛 | +|---|---| +| `unit: string`（摄氏？华氏？kelvin？） | `unit: enum["celsius", "fahrenheit"]` | +| `priority: string`（low/中/HIGH？） | `priority: enum["low", "medium", "high"]` | +| `count: string`（"五个"？） | `count: integer` | +| `enabled: string`（"true" / "True"） | `enabled: boolean` | +| `tags: string`（"a,b,c"？JSON？） | `tags: array of string` | + +### 规则 3：required vs optional 分清楚 + +- `required` 列**真的必要的**参数（少了 this tool 就跑不起来） +- 有默认值的放 `default`，不要列 required +- LLM 看到 required 多会“**自己编参数**”，所以 required 越少越好 + +```python +# 坏：把 timezone 列 required，LLM 会乱编「Asia/Taipei」即便用户没提到 +"required": ["city", "timezone"] + +# 好 +"required": ["city"] +"properties": { + "timezone": {"type": "string", "default": "UTC", "description": "..."} +} +``` + +### 规则 4：tool name + parameter name 要自说明 + +LLM 看到 `do_thing(x, y, z)` 跟看到 `get_weather(city, unit)` 用法完全不同。 + +- ✅ `get_user_profile(user_id)` +- ❌ `fetch(id)` 或 `process_data(input)` + +动词开头，说清楚是 query / mutation / action。 + +### 规则 5：error 回传要让 LLM 可以恢复 + +LLM 看到错误信息后决定要 retry / 换工具 / 放弃。错误信息要结构化： + +```json +{ + "error": "City not found", + "code": "INVALID_CITY", + "retry_hint": "Check spelling, or try a major city nearby" +} +``` + +而不是只回 `"Error 500"`——LLM 拿这个没招。 + +--- + +## 5 个常见 Anti-Pattern + +### Anti-1：“万能工具”（God Tool） + +```python +# 坏：一个 tool 做所有事 +def do_database_op(operation: str, table: str, data: str) -> str: + """Do anything with the database.""" +``` + +LLM 会把错的 operation 配上对的 table 然后烂掉。**拆成 `query_users` / `create_order` / `update_inventory`** 等具体 tool，LLM 选择正确率大幅提升。 + +### Anti-2：description 是 docstring + +```python +# 坏 +"description": "GET /api/v2/weather endpoint. Returns JSON. See API docs." + +# 好 +"description": "Get current weather for a city. Returns temperature in C/F, humidity, and conditions." +``` + +LLM 不是程序，它要的是 **“这个 tool 什么时候有用”**。 + +### Anti-3：所有东西都是 string + +```python +# 坏 +{"properties": { + "count": {"type": "string"}, # LLM 可能传 "five" + "active": {"type": "string"}, # LLM 可能传 "yes" + "list": {"type": "string"} # LLM 可能传 "[a, b, c]" 或 "a, b, c" +}} + +# 好 +{"properties": { + "count": {"type": "integer", "minimum": 1, "maximum": 100}, + "active": {"type": "boolean"}, + "list": {"type": "array", "items": {"type": "string"}} +}} +``` + +### Anti-4：没写范例 + +LLM 对 description **加上 example 比没加准确很多**。 + +```python +"description": "Search products by query string. Examples: 'laptop under $1000', 'red shoes size 10'. Do NOT use for product ID lookup (use get_product_by_id)." +``` + +### Anti-5：沉默的失败 + +Tool 失败只回 `null` 或 `{}`，LLM 以为成功，继续用空数据推论。**永远回**： + +- 成功 → `{"success": true, "data": {...}}` +- 失败 → `{"success": false, "error": "...", "retry_hint": "..."}` + +LLM 看到 `success: false` 就知道要处理错误，不会把空数据当答案编造。 + +--- + +## 延伸阅读 + +- [Anthropic — Tool Use Guide](https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/overview) — 官方 schema 规格 +- [OpenAI — Function Calling](https://platform.openai.com/docs/guides/function-calling) — OpenAI 的 schema 规格（跟 Anthropic 略有差异） +- [Stage 3 — Tool Use & Agent 入门](../stages/03-tool-use-and-hello-agent.md) — 主要动手练习 +- [Stage 5.2 — MCP 基础](../stages/05-claude-code-ecosystem.zh-Hans.md#52--mcpmodel-context-protocol-基础) — MCP server 的 tool schema（跟 function calling schema 结构几乎相同） diff --git a/resources/setup-guide.en.md b/resources/setup-guide.en.md new file mode 100644 index 0000000..48c73b1 --- /dev/null +++ b/resources/setup-guide.en.md @@ -0,0 +1,306 @@ +> [繁體中文](./setup-guide.md) | [简体中文](./setup-guide.zh-Hans.md) | **English** + +# 🚀 From Zero — Setup Guide for People Without a Dev Background + +> [← Back to main README](../README.en.md) + +> Expected time: 30-45 minutes. You will get your first API key, install Python / uv, and run your first LLM hello world. +> This guide is for people who want to learn AI agents but have not written code before. If you already know Python, git, and the CLI, you can skip to [Stage 1](../stages/01-llm-basics.en.md). + +## Pick Your On-Ramp First + +Ordered shallow → deep by setup effort. **Never touched an LLM? Just start with 1️⃣**. + +### 1️⃣ Web (easiest, free tier, zero setup) + +Open a browser, type the URL — done. **Best place to start if it's your first time**. Free tier usually covers a week of experimentation. + +| Service | URL | Notes | +|---|---|---| +| **Claude** | https://claude.ai | Anthropic. Free tier has daily limits; Pro is $20/mo | +| **ChatGPT** | https://chatgpt.com | OpenAI. Free GPT-5.5 Instant (rate-limited); Plus $20/mo unlocks Thinking/Pro | +| **Gemini** | https://gemini.google.com | Google. Generous free tier, integrates Google apps | +| **Le Chat** | https://chat.mistral.ai | Mistral (EU open-source LLM lab). Free, privacy-focused | + +### 2️⃣ Desktop app (free, better cross-app integration) + +Native apps for macOS / Windows — adds system shortcut, clipboard / screenshot integration, drag-and-drop files. + +| App | Download | Platform | +|---|---|---| +| **Claude Desktop** | https://claude.ai/download | macOS / Windows | +| **ChatGPT Desktop** | https://openai.com/chatgpt/download | macOS / Windows | +| **Gemini** | No native desktop app yet | (use web) | +| **LM Studio** | https://lmstudio.ai | macOS / Windows / Linux — runs local LLMs as a desktop app; $0 but needs GPU/RAM | + +### 3️⃣ IDE with built-in AI (write code with an AI sidekick) + +Lives inside a code editor — you write code normally, AI suggests / edits / answers questions alongside. **Best fit if you already write code and want an AI-native IDE**. + +| Tool | Download | Shape | +|---|---|---| +| **Cursor** | https://cursor.com | Standalone IDE (VS Code fork) | +| **Windsurf** | https://codeium.com/windsurf | Standalone IDE (by Codeium) | +| **Cline** | https://cline.bot | VS Code extension (agentic style) | +| **Continue** | https://continue.dev | VS Code / JetBrains extension (open-source) | +| **Roo Code** | https://github.com/RooCodeInc/Roo-Code | VS Code extension (Cline fork, active community) | +| **Zed** | https://zed.dev | Standalone editor with built-in AI assistant | +| **GitHub Copilot** | https://github.com/features/copilot | Multi-IDE extension (VS Code / JetBrains / etc.) | + +→ Detailed comparison → [`branches/for-developer.en.md`](../branches/for-developer.en.md) + +### 4️⃣ CLI agent (terminal, can read/write files, run shell, manage git) + +Agents that live in your terminal — you give one prompt (e.g. "refactor this module"), the agent reads files, edits them, runs commands, commits. **More autonomous than the IDE mode and handles multi-step tasks**, but setup is heavier (requires Node.js or Python; see B / D below). + +| CLI Agent | Install / Docs | Primary LLM | +|---|---|---| +| **Claude Code** | https://docs.anthropic.com/en/docs/claude-code/quickstart | Claude | +| **Codex CLI** | https://github.com/openai/codex | GPT family | +| **Gemini CLI** | https://github.com/google-gemini/gemini-cli | Gemini | +| **OpenCode** | https://github.com/sst/opencode | Any (multi-provider) | +| **goose** | https://block.github.io/goose | Any | +| **Aider** | https://aider.chat | Any (git-native) | +| **Hermes Agent** | https://github.com/NousResearch/hermes-agent | 200+ (model-neutral) | + +→ Full 7-CLI comparison → [`cli-agents-guide.en.md`](cli-agents-guide.en.md) +→ Detailed Claude Code first install → [D](#d--install-claude-code-for-the-first-time-about-10-minutes-needed-for-stage-5--for-developer) below + +> 💡 **IDE vs CLI — how to pick?** Want AI alongside you while you code → IDE. Want to give one prompt and let the agent run a multi-step task → CLI. Many people use both. + +### 5️⃣ API + write your own code (most advanced) + +Want to script with Python, run batch jobs, integrate LLMs into your own app/automation? A-C below are for you. + +> 💡 **What's an API key?** A password that lets a program call a model. Treat it like payment information. + +--- + +## A — Get Your First API Key (About 10 Minutes) + +### Anthropic Claude (Recommended First) + +1. Open https://console.anthropic.com/ +2. Sign up with Google, GitHub, or email. +3. After login, find **API Keys**, then choose **Create Key**. +4. **Copy the key immediately**. Most platforms show it only once. +5. Put it in a local password manager, or briefly in a local text file; the next section moves it into `.env`. + +> ⚠️ **Three API-key rules** +> - **Do not paste it** into chat windows, group chats, email, or screenshots. +> - **Do not upload it** to git; GitHub may detect and revoke it. +> - **Do not store it** as a plain text cloud-drive file; syncing creates more exposure. + +### Other LLM Options + +#### Western cloud (US-friendly, English-first) + +- **OpenAI**: https://platform.openai.com/api-keys + ChatGPT Plus and API access are separate; Plus subscribers still need an API key. +- **Google AI Studio**: https://aistudio.google.com/ + Useful for trying the Gemini API. Free quota depends on region and account state. +- **NVIDIA NIM**: https://build.nvidia.com/ + **Hosts many open-source models (Llama / Mistral / DeepSeek-R1 / Qwen / Gemma etc.), OpenAI-compatible API, new accounts get 1000 free credits**. Great when you want to try several open models without local GPU. `base_url=https://integrate.api.nvidia.com/v1`. + +#### Chinese / Chinese-language cloud (region-friendly, very cheap) + +> If you're in mainland China and Anthropic / OpenAI are inaccessible, or you want to test Chinese-native models, start here. **All these APIs are OpenAI-compatible** — just change `base_url` and model name to run the same exercises. + +- **DeepSeek**: https://platform.deepseek.com/ + Free web at https://chat.deepseek.com (includes the R1 reasoning model). API is extremely cheap (**$0.27 input / $1.10 output per 1M tokens — about 4× cheaper than haiku**). Strong code and reasoning. + `base_url=https://api.deepseek.com/v1`, `model=deepseek-chat` or `deepseek-reasoner`. +- **Moonshot Kimi**: https://platform.moonshot.cn/ (China) / https://platform.moonshot.ai/ (international) + Free web at https://kimi.com. Selling point: **1M-token context window** (great for large files / long conversations). API ~$5-15 per 1M input, tiered by context size. + `base_url=https://api.moonshot.cn/v1` (CN) / `https://api.moonshot.ai/v1` (intl), e.g. `model=kimi-k2-turbo-preview`. +- **Qwen (Alibaba)**: https://dashscope.console.aliyun.com/ + Free web at https://chat.qwen.ai. API via Alibaba Cloud DashScope with an **OpenAI-compatible endpoint** ([docs](https://help.aliyun.com/zh/dashscope/developer-reference/compatibility-of-openai-with-dashscope/)). **The same Qwen models also run locally via Ollama** (`ollama pull qwen2.5:3b`) — cloud and local paths both work. +- **GLM (ZhipuAI)**: https://open.bigmodel.cn/ (China) / https://z.ai/ (intl) + Free web at https://chatglm.cn. Has GLM-4.5 and GLM-4-Plus. Free tier available; students can apply for extra credit. + +#### Local (zero API cost, fully offline) + +- **Ollama local models**: no API key needed. For the local path, see [Cookbook Recipe 6](cookbook.en.md#6-local-llm--cli-agent-quick-walkthrough). + This repo's "Path A" defaults to Ollama; all Stage 1-7 exercises run with `gemma4:e4b` (Stage 1-2) or `qwen2.5:3b` (Stage 3+) at $0/run. + +> 💡 **How to pick your first**: +> - Learning agents / production, **US-region account OK** → **Anthropic Claude** (the curriculum's canonical path) +> - Learning agents / production, **China region** or want a Chinese-native model → **DeepSeek** (cheapest cloud option, OpenAI-compat, strong Chinese support) +> - Want to try many models without a local GPU → **NVIDIA NIM** (1000 credits, 10+ hosted open models) +> - Privacy-sensitive / fully free / mainland China without cloud access → **Ollama** (local, runs the entire curriculum at $0) + +--- + +## B — Install Your Local Environment (About 10 Minutes) + +### Install Python 3.10+ + +- **macOS**: open Terminal and run `brew install python@3.12`. If Homebrew is not installed, start at https://brew.sh. +- **Windows**: download the installer from https://www.python.org/downloads/ and make sure **Add Python to PATH** is checked. +- **Linux**: on Ubuntu, run `sudo apt install python3 python3-venv`; on Fedora, run `sudo dnf install python3`. +- **Verify**: macOS / Linux: `python3 --version`; Windows: `py --version`. You want `Python 3.10` or newer. + +### Install uv + +uv is a Python package tool. For this guide, think of it as "install the packages I need, then run this script." + +```bash +# macOS / Linux +curl -LsSf https://astral.sh/uv/install.sh | sh + +# Windows PowerShell +irm https://astral.sh/uv/install.ps1 | iex +``` + +Verify: + +```bash +uv --version +``` + +### Create Your First `.env` File + +In the folder where you want to run the script, create a file named `.env`: + +```bash +ANTHROPIC_API_KEY=sk-ant-...paste the key you copied +``` + +`.env` is where local secrets live. Your program can read it, but you should not upload it to GitHub. + +### Add `.gitignore` + +In the same folder, create `.gitignore`: + +```gitignore +.env +__pycache__/ +*.pyc +``` + +This keeps git from recording your `.env` file. + +--- + +## C — Run Your First `hello-claude.py` (About 5 Minutes) + +Create `hello-claude.py`: + +```python +from anthropic import Anthropic +from dotenv import load_dotenv + +load_dotenv() +client = Anthropic() # Automatically reads ANTHROPIC_API_KEY + +msg = client.messages.create( + model="claude-sonnet-5", + max_tokens=100, + messages=[{"role": "user", "content": "Hello, who are you?"}], +) + +print(msg.content[0].text) +``` + +Run it: + +```bash +uv run --with anthropic --with python-dotenv python hello-claude.py +``` + +If Claude introduces itself, your API key, Python, and packages are working. + +### Common Errors + +| Error | Likely Cause | Fix | +|---|---|---| +| `401 Unauthorized` | API key is missing or mistyped | Copy it again from A and check the `.env` filename and value | +| `429 Rate limit` | Too many requests too quickly | Wait a few seconds or minutes, then retry | +| `connection refused` | Network or firewall issue | Check your network, company firewall, or school firewall | +| `ModuleNotFoundError` | A package was not installed | Make sure you ran the exact `uv run --with ...` command above | + +--- + +## D — Install Claude Code for the First Time (About 10 Minutes; Needed for Stage 5 / for-developer) + +### Install Node.js First + +> 💡 **What is Node.js?** A runtime for running JavaScript, similar to a Python interpreter but for JS. **`npm`** is its bundled package manager, which plays the same role as Python's `pip`: installing tools other people wrote, including Claude Code below. `npm install -g X` means install X globally so you can use it from any folder. + +- **macOS / Linux**: run `brew install node`, or download from https://nodejs.org. +- **Windows**: download the installer from https://nodejs.org. +- **Verify**: run `node --version`; v18 or newer is enough. + +### Install Claude Code + +```bash +npm install -g @anthropic-ai/claude-code +``` + +### First Authentication + +```bash +claude +``` + +On first launch, you will usually choose between: + +- **Claude subscription**: sign in with your Claude.ai account. This is the simplest path for beginners. +- **API key**: paste the key you created in A. + +### Create Your First `CLAUDE.md` + +Create `CLAUDE.md` at the root of your project. Claude Code reads it on startup so it understands how you want help. + +```markdown +# Who you are +I am [your name], a [your field, such as teacher / researcher / writer]. + +# Code style +- Write comments in Traditional Chinese, and code in English +- Prefer type hints when writing functions +- Do not commit automatically; let me run git add myself + +# Do not do these +- Do not browse the web unless I explicitly allow it +- Do not modify `.env` or `.gitignore` +- Do not delete folders, including subfolders +``` + +--- + +## E — Your First Skill Example (About 5 Minutes; Needed for Stage 5.3) + +A Skill is a reusable prompt package for Claude Code. When your message matches the description, Claude Code loads that instruction automatically. + +Create `.claude/skills/hello-skill/SKILL.md`: + +```markdown +--- +name: hello-skill +description: First hello skill. Trigger when the user says "請打招呼" or "say hi". +--- + +When the user asks you to greet them, return three things: + +1. Say hello once in Traditional Chinese and once in English +2. Mention today's date using system time +3. Give one small daily reminder, randomly chosen from health / learning / mood +``` + +Run `claude`, then type `say hi`. If Claude returns the three items, the Skill loaded. + +> For deeper Skill design, see [Stage 5.3 — Skills](../stages/05-claude-code-ecosystem.en.md#53--skills-claude-codes-behavior-layer--the-most-critical-layer-of-the-claude-code-ecosystem). +> For copy-and-run examples, see the [Cookbook](cookbook.en.md). + +--- + +## Where to Go Next + +| Your Current State | Next Step | +|---|---| +| You want to understand LLMs, APIs, and tokens | [Stage 1 — LLM Basics](../stages/01-llm-basics.en.md) | +| You want to pick a role-based branch | [Everyday users](../branches/for-everyday-users.en.md) / [Teachers](../branches/for-teacher.en.md) / [Knowledge workers](../branches/for-knowledge-worker.en.md) / [Researchers](../branches/for-researcher.en.md) / [Developers](../branches/for-developer.en.md) | +| You want the full Claude Code ecosystem | [Stage 5 — Claude Code Ecosystem](../stages/05-claude-code-ecosystem.en.md) | +| You want local LLMs without a cloud key | [Cookbook Recipe 6](cookbook.en.md#6-local-llm--cli-agent-quick-walkthrough) | +| You want to compare CLI agents | [CLI Agents Comparison Guide](cli-agents-guide.en.md) | +| A term is unclear | [Glossary](glossary.en.md) | diff --git a/resources/setup-guide.md b/resources/setup-guide.md new file mode 100644 index 0000000..61f18d7 --- /dev/null +++ b/resources/setup-guide.md @@ -0,0 +1,306 @@ +> **繁體中文** | [简体中文](./setup-guide.zh-Hans.md) | [English](./setup-guide.en.md) + +# 🚀 從零開始 — 給沒有開發背景的設定指南 + +> [← 回主路線 README](../README.md) + +> 預估時程：30-45 分鐘。你會申請第一個 API key、裝好 Python / uv，並跑出第一個 LLM hello world。 +> 這份文件寫給「想學 AI agent，但還沒寫過 code」的人。已經熟 Python / git / CLI 的開發者，可以直接跳 [Stage 1](../stages/01-llm-basics.md)。 + +## 先選你的入門方式 + +依「想花多少時間 setup」由淺到深排序。**完全沒接觸過 LLM 直接從 1️⃣ 開始就好**。 + +### 1️⃣ 網頁版（最簡單，免費可試，零 setup） + +打開瀏覽器就能用，**第一次接觸 LLM 最推薦這條**。免費 tier 通常夠你試一個禮拜。 + +| 服務 | 網址 | 備註 | +|---|---|---| +| **Claude** | https://claude.ai | Anthropic 官方。免費 tier 每天額度有限，付費版 $20/月 | +| **ChatGPT** | https://chatgpt.com | OpenAI 官方。免費可用 GPT-5.5 Instant（有用量限制），付費 $20/月升 Plus 解鎖 Thinking/Pro | +| **Gemini** | https://gemini.google.com | Google 官方。免費 tier 寬鬆，整合 Google 服務 | +| **Le Chat** | https://chat.mistral.ai | Mistral（歐洲開源 LLM）。免費、隱私導向 | + +### 2️⃣ 桌面 App（免費，跨應用整合更好） + +跑在你電腦上的原生 app——比網頁多了系統 shortcut、跟剪貼簿 / 截圖整合、可以拖拉檔案。 + +| App | 下載 | 平台 | +|---|---|---| +| **Claude Desktop** | https://claude.ai/download | macOS / Windows | +| **ChatGPT Desktop** | https://openai.com/chatgpt/download | macOS / Windows | +| **Gemini** | 暫無原生 desktop app | （用網頁版即可） | +| **LM Studio** | https://lmstudio.ai | macOS / Windows / Linux — 跑本機 LLM 的桌面 app，零成本但要 GPU/RAM | + +### 3️⃣ IDE 內建 AI（在 code editor 裡邊寫 code 邊有 AI 助手） + +跑在 IDE / code editor 裡——你正常寫 code，AI 在旁邊 suggest、修改、回答問題。**已經有寫 code 習慣、想把 IDE 升級成 AI-native 的人這條最順**。 + +| 工具 | 下載 | 形態 | +|---|---|---| +| **Cursor** | https://cursor.com | 獨立 IDE（VS Code fork） | +| **Windsurf** | https://codeium.com/windsurf | 獨立 IDE（Codeium 出） | +| **Cline** | https://cline.bot | VS Code extension（agentic 風格） | +| **Continue** | https://continue.dev | VS Code / JetBrains extension（開源） | +| **Roo Code** | https://github.com/RooCodeInc/Roo-Code | VS Code extension（Cline fork，社群活躍） | +| **Zed** | https://zed.dev | 獨立 editor，內建 AI assistant | +| **GitHub Copilot** | https://github.com/features/copilot | VS Code / JetBrains 等多 IDE extension | + +→ 詳細比較 → [`branches/for-developer.md`](../branches/for-developer.md) + +### 4️⃣ CLI Agent（terminal，能讀寫檔案、跑指令、操作 git） + +裝在 terminal 的 agent——你下一個 prompt（譬如「重構這個 module」），agent 自己讀檔、改檔、跑指令、commit。**比 IDE 模式更自主、可以處理多步驟任務**，但 setup 稍複雜（需要先有 Node.js 或 Python，看下面 B / D）。 + +| CLI Agent | 安裝 / 文件 | 主要 LLM | +|---|---|---| +| **Claude Code** | https://docs.anthropic.com/en/docs/claude-code/quickstart | Claude | +| **Codex CLI** | https://github.com/openai/codex | GPT 系列 | +| **Gemini CLI** | https://github.com/google-gemini/gemini-cli | Gemini | +| **OpenCode** | https://github.com/sst/opencode | 任意（多 provider） | +| **goose** | https://block.github.io/goose | 任意 | +| **Aider** | https://aider.chat | 任意（git-native） | +| **Hermes Agent** | https://github.com/NousResearch/hermes-agent | 200+（model-neutral） | + +→ 想看 7 個 CLI 完整比較 → [`cli-agents-guide.md`](cli-agents-guide.md) +→ Claude Code 第一次裝的詳細步驟 → 本指南 [D](#d--第一次裝-claude-code約-10-分鐘stage-5--for-developer-會用到) + +> 💡 **IDE-based 跟 CLI agent 怎麼選？** 邊寫 code 邊要 AI 幫忙 → IDE；下單一 prompt 讓 agent 自己跑完一整個任務 → CLI。兩個可以並用。 + +### 5️⃣ API + 自己寫 code（最進階，能 batch、整合任何工具） + +想自己寫 Python script、跑 batch job、把 LLM 接到自己的 app／automation？接下來的 A-C 就是給你的。 + +> 💡 **API key 是什麼**：簡單講就是「讓程式呼叫模型的密碼」。請把它當成信用卡資料一樣保管。 + +--- + +## A — 申請第一個 API key（約 10 分鐘） + +### Anthropic Claude（推薦第一次） + +1. 開 https://console.anthropic.com/ +2. 用 Google、GitHub 或 email 註冊。 +3. 進帳號後找到 **API Keys**，按 **Create Key**。 +4. **立刻複製顯示出的 key**。多數平台只會顯示一次。 +5. 先放在本機密碼管理器，或短暫放在本機文字檔；下一節會移到 `.env`。 + +> ⚠️ **API key 三不規則** +> - **不貼**到 chat 視窗、群組、email 或截圖。 +> - **不上傳**到 git；GitHub 可能掃到後自動撤銷。 +> - **不放**雲端硬碟純文字檔；同步到其他裝置等於多一份風險。 + +### 其他 LLM 選項 + +#### 西方 cloud（美區友善、英文場景） + +- **OpenAI**：https://platform.openai.com/api-keys + ChatGPT Plus 和 API key 是兩件事；訂閱 Plus 仍要另外申請 API key。 +- **Google AI Studio**：https://aistudio.google.com/ + 適合先試 Gemini API，免費額度會依地區和帳號狀態不同。 +- **NVIDIA NIM**：https://build.nvidia.com/ + **托管多個開源 model（Llama / Mistral / DeepSeek-R1 + R2 lineage / Qwen / Gemma 等）、OpenAI-compatible API、新帳號送 1000 credits**。適合「想試多個 open-source model 但沒 GPU」的情境。`base_url=https://integrate.api.nvidia.com/v1`。 + +#### 中國 / 中文場景（地區友善、價格極便宜） + +> 中國大陸使用者連 Anthropic / OpenAI 有困難、或想試中文 native 模型，從這邊開始。**這些 API 都 OpenAI-compatible**、改 `base_url` 跟 model name 就能跑同一份練習。 + +- **DeepSeek**：https://platform.deepseek.com/ + web 版 https://chat.deepseek.com 完全免費（含 R1 推理模型）。API 價格極便宜（**$0.27 input / $1.10 output per 1M token**、比 haiku 便宜 4 倍）。Code / 推理都很強。 + `base_url=https://api.deepseek.com/v1`、`model=deepseek-chat` 或 `deepseek-reasoner`。 +- **Moonshot Kimi**：https://platform.moonshot.cn/ (中國)、https://platform.moonshot.ai/ (海外) + web 版 https://kimi.com 免費、**1M token context** 是賣點（很大檔案 / 長對話）。API 約 $5-15/1M input、按 context size 階梯計費。 + `base_url=https://api.moonshot.cn/v1` (中國) / `https://api.moonshot.ai/v1` (海外)、`model=kimi-k2-turbo-preview` 等。 +- **通義千問 Qwen（Alibaba）**：https://dashscope.console.aliyun.com/ + web 版 https://chat.qwen.ai 免費。API 走 Alibaba Cloud DashScope、有 **OpenAI-compatible endpoint**（[文件](https://help.aliyun.com/zh/dashscope/developer-reference/compatibility-of-openai-with-dashscope/)）。**同樣的 Qwen 模型也能用 Ollama 在本機跑**（`ollama pull qwen2.5:3b`）——cloud 跟 local 兩條路徑都通。 +- **智譜 GLM（ZhipuAI）**：https://open.bigmodel.cn/ (中國) / https://z.ai/ (海外) + web 版 https://chatglm.cn 免費、有 GLM-4.5、GLM-4-Plus。API 有 free tier、學生申請可額外領 credit。 + +#### 本機（不付 API 費、完全 offline） + +- **Ollama 本機模型**：不用 API key。走本機路線請看 [Cookbook Recipe 6](cookbook.md#6-本機-llm--cli-agent-快速-walkthrough)。 + 本 repo 的「Path A」預設就是 Ollama；所有 Stage 1-7 練習都能用 `gemma4:e4b`（Stage 1-2）或 `qwen2.5:3b`（Stage 3+）跑通、$0/run。 + +> 💡 **怎麼挑第一個**： +> - 想學 agent / production、**美區帳號OK** → **Anthropic Claude**（curriculum canonical） +> - 想學 agent / production、**中國地區**或想試中文模型 → **DeepSeek**（最便宜 cloud option、OpenAI-compat、中文很強） +> - 想試多個 model 但沒 GPU → **NVIDIA NIM**（送 1000 credit、托管 10+ open model） +> - 隱私敏感 / 完全免費 / 中國大陸無 cloud → **Ollama**（本機、curriculum 全套都能跑、$0） + +--- + +## B — 裝本機環境（約 10 分鐘） + +### 裝 Python 3.10+ + +- **macOS**：開 Terminal，輸入 `brew install python@3.12`。如果還沒有 Homebrew，先看 https://brew.sh。 +- **Windows**：到 https://www.python.org/downloads/ 下載 installer，安裝時一定要勾 **Add Python to PATH**。 +- **Linux**：Ubuntu 用 `sudo apt install python3 python3-venv`，Fedora 用 `sudo dnf install python3`。 +- **驗證**：macOS / Linux 輸入 `python3 --version`；Windows 輸入 `py --version`。看到 `Python 3.10` 以上即可。 + +### 裝 uv + +uv 是 Python 套件管理工具。你可以把它想成「幫你臨時裝好需要套件再執行」的工具。 + +```bash +# macOS / Linux +curl -LsSf https://astral.sh/uv/install.sh | sh + +# Windows PowerShell +irm https://astral.sh/uv/install.ps1 | iex +``` + +驗證： + +```bash +uv --version +``` + +### 建立第一個 `.env` 檔 + +在你要跑 script 的資料夾裡，建立一個檔名叫 `.env` 的檔案： + +```bash +ANTHROPIC_API_KEY=sk-ant-...貼上你剛才複製的 key +``` + +`.env` 是專門放本機祕密資訊的檔案。程式會讀它，但你不應該把它上傳到 GitHub。 + +### 加上 `.gitignore` + +同一個資料夾建立 `.gitignore`： + +```gitignore +.env +__pycache__/ +*.pyc +``` + +這樣 git 就不會把 `.env` 收進版本紀錄。 + +--- + +## C — 跑第一個 `hello-claude.py`（約 5 分鐘） + +建立 `hello-claude.py`： + +```python +from anthropic import Anthropic +from dotenv import load_dotenv + +load_dotenv() +client = Anthropic() # 自動讀取 ANTHROPIC_API_KEY + +msg = client.messages.create( + model="claude-sonnet-5", + max_tokens=100, + messages=[{"role": "user", "content": "Hello, who are you?"}], +) + +print(msg.content[0].text) +``` + +執行： + +```bash +uv run --with anthropic --with python-dotenv python hello-claude.py +``` + +看到 Claude 回覆自我介紹，就代表你的 API key、Python、套件都通了。 + +### 常見錯誤 + +| 錯誤訊息 | 常見原因 | 解法 | +|---|---|---| +| `401 Unauthorized` | API key 沒讀到或打錯 | 回 A 重新複製，確認 `.env` 檔名和內容 | +| `429 Rate limit` | 太快送太多請求 | 等幾秒或幾分鐘再跑 | +| `connection refused` | 連線或防火牆問題 | 確認網路、公司或學校防火牆 | +| `ModuleNotFoundError` | 套件沒有被安裝 | 確認執行的是上面的 `uv run --with ...` 指令 | + +--- + +## D — 第一次裝 Claude Code（約 10 分鐘；Stage 5 / for-developer 會用到） + +### 先裝 Node.js + +> 💡 **Node.js 是什麼**：跑 JavaScript 的 runtime（類似 Python interpreter 但是給 JS 用）。**`npm`** 是它附帶的「套件管理器」（package manager）—— 跟 Python 的 `pip` 同角色、用來安裝別人寫好的工具（如下面的 Claude Code）。`npm install -g X` 表示「全域裝 X、之後在任何資料夾都能用」。 + +- **macOS / Linux**：`brew install node`，或從 https://nodejs.org 下載。 +- **Windows**：從 https://nodejs.org 下載 installer。 +- **驗證**：輸入 `node --version`，看到 v18 以上即可。 + +### 裝 Claude Code + +```bash +npm install -g @anthropic-ai/claude-code +``` + +### 第一次認證 + +```bash +claude +``` + +第一次啟動時通常會讓你選： + +- **Claude subscription**：用 Claude.ai 帳號登入，對初學者最省事。 +- **API key**：貼上 A 申請到的 key。 + +### 建立第一份 `CLAUDE.md` + +在你的 project 根目錄建立 `CLAUDE.md`。Claude Code 啟動時會讀它，理解你希望它怎麼協助。 + +```markdown +# 你是誰 +我是 [你的名字]，[你的領域，例如：教師 / 研究者 / 寫作者]。 + +# Code style +- 註解用繁體中文寫，code 用英文 +- 寫 function 時優先加 type hint +- 不要主動 commit；改完讓我手動 git add + +# 不准做的事 +- 不要連網查資料，除非我明確說可以 +- 不要動 `.env` 或 `.gitignore` +- 不要刪資料夾，包括子資料夾 +``` + +--- + +## E — 第一個 Skill 範例（約 5 分鐘；Stage 5.3 會用到） + +Skill 是 Claude Code 的「可重用 prompt 包」。當你的訊息符合描述，Claude Code 會自動載入那份指示。 + +建立 `.claude/skills/hello-skill/SKILL.md`： + +```markdown +--- +name: hello-skill +description: 第一個 hello skill。當使用者說「請打招呼」或「say hi」時觸發。 +--- + +當使用者請你打招呼時，回三件事： + +1. 用繁體中文跟英文各說一次 hello +2. 提現在的日期（用 system 時間） +3. 給一個今日小提醒（隨機選健康 / 學習 / 心情建議） +``` + +跑 `claude`，輸入「請打招呼」。如果 Claude 回覆三件事，就代表 Skill 被載入了。 + +> 想看更完整的 Skill 設計：看 [Stage 5.3 — Skills](../stages/05-claude-code-ecosystem.md#53--skillsclaude-code-的行為層-claude-code-生態最關鍵的一層)。 +> 想看可以照做的範例：看 [Cookbook](cookbook.md)。 + +--- + +## 接下來去哪 + +| 你現在的狀態 | 下一步 | +|---|---| +| 想正式理解 LLM、API、token | [Stage 1 — LLM 基礎](../stages/01-llm-basics.md) | +| 想直接挑身分分支 | [日常使用者](../branches/for-everyday-users.md) / [教師](../branches/for-teacher.md) / [知識工作者](../branches/for-knowledge-worker.md) / [研究者](../branches/for-researcher.md) / [開發者](../branches/for-developer.md) | +| 想看 Claude Code 完整生態 | [Stage 5 — Claude Code 生態系](../stages/05-claude-code-ecosystem.md) | +| 想本機 LLM、不用雲端 key | [Cookbook Recipe 6](cookbook.md#6-本機-llm--cli-agent-快速-walkthrough) | +| 想比較 CLI agent | [CLI Agents 比較指南](cli-agents-guide.md) | +| 不懂某個用詞 | [Glossary](glossary.md) | diff --git a/resources/setup-guide.zh-Hans.md b/resources/setup-guide.zh-Hans.md new file mode 100644 index 0000000..2548466 --- /dev/null +++ b/resources/setup-guide.zh-Hans.md @@ -0,0 +1,306 @@ +> [繁體中文](./setup-guide.md) | **简体中文** | [English](./setup-guide.en.md) + +# 🚀 从零开始 — 给没有开发背景的设置指南 + +> [← 回主路线 README](../README.zh-Hans.md) + +> 预估时间：30-45 分钟。你会申请第一个 API key、装好 Python / uv，并跑出第一个 LLM hello world。 +> 这份文档写给“想学 AI agent，但还没写过 code”的人。已经熟 Python / git / CLI 的开发者，可以直接跳 [Stage 1](../stages/01-llm-basics.zh-Hans.md)。 + +## 先选你的入门方式 + +按“想花多少时间 setup”由浅到深排序。**完全没接触过 LLM 直接从 1️⃣ 开始就好**。 + +### 1️⃣ 网页版（最简单，免费可试，零 setup） + +打开浏览器就能用，**第一次接触 LLM 最推荐这条**。免费 tier 通常够你试一周。 + +| 服务 | 网址 | 备注 | +|---|---|---| +| **Claude** | https://claude.ai | Anthropic 官方。免费 tier 每天有限额，付费 $20/月 | +| **ChatGPT** | https://chatgpt.com | OpenAI 官方。免费可用 GPT-5.5 Instant（有用量限制），Plus $20/月解锁 Thinking/Pro | +| **Gemini** | https://gemini.google.com | Google 官方。免费 tier 宽松，整合 Google 服务 | +| **Le Chat** | https://chat.mistral.ai | Mistral（欧洲开源 LLM）。免费、隐私导向 | + +### 2️⃣ 桌面 App（免费，跨应用整合更好） + +跑在你电脑上的原生 app——比网页多了系统 shortcut、跟剪贴板 / 截图整合、可以拖拉文件。 + +| App | 下载 | 平台 | +|---|---|---| +| **Claude Desktop** | https://claude.ai/download | macOS / Windows | +| **ChatGPT Desktop** | https://openai.com/chatgpt/download | macOS / Windows | +| **Gemini** | 暂无原生 desktop app | （用网页版即可） | +| **LM Studio** | https://lmstudio.ai | macOS / Windows / Linux — 跑本地 LLM 的桌面 app，零成本但要 GPU/RAM | + +### 3️⃣ IDE 内建 AI（在 code editor 里边写 code 边有 AI 助手） + +跑在 IDE / code editor 里——你正常写 code，AI 在旁边 suggest、修改、回答问题。**已经有写 code 习惯、想把 IDE 升级成 AI-native 的人这条最顺**。 + +| 工具 | 下载 | 形态 | +|---|---|---| +| **Cursor** | https://cursor.com | 独立 IDE（VS Code fork） | +| **Windsurf** | https://codeium.com/windsurf | 独立 IDE（Codeium 出） | +| **Cline** | https://cline.bot | VS Code extension（agentic 风格） | +| **Continue** | https://continue.dev | VS Code / JetBrains extension（开源） | +| **Roo Code** | https://github.com/RooCodeInc/Roo-Code | VS Code extension（Cline fork，社群活跃） | +| **Zed** | https://zed.dev | 独立 editor，内建 AI assistant | +| **GitHub Copilot** | https://github.com/features/copilot | VS Code / JetBrains 等多 IDE extension | + +→ 详细比较 → [`branches/for-developer.zh-Hans.md`](../branches/for-developer.zh-Hans.md) + +### 4️⃣ CLI Agent（terminal，能读写文件、跑指令、操作 git） + +装在 terminal 的 agent——你下一个 prompt（譬如“重构这个 module”），agent 自己读文件、改文件、跑指令、commit。**比 IDE 模式更自主、可以处理多步骤任务**，但 setup 稍复杂（需要先有 Node.js 或 Python，看下面 B / D）。 + +| CLI Agent | 安装 / 文档 | 主要 LLM | +|---|---|---| +| **Claude Code** | https://docs.anthropic.com/en/docs/claude-code/quickstart | Claude | +| **Codex CLI** | https://github.com/openai/codex | GPT 系列 | +| **Gemini CLI** | https://github.com/google-gemini/gemini-cli | Gemini | +| **OpenCode** | https://github.com/sst/opencode | 任意（多 provider） | +| **goose** | https://block.github.io/goose | 任意 | +| **Aider** | https://aider.chat | 任意（git-native） | +| **Hermes Agent** | https://github.com/NousResearch/hermes-agent | 200+（model-neutral） | + +→ 想看 7 个 CLI 完整比较 → [`cli-agents-guide.zh-Hans.md`](cli-agents-guide.zh-Hans.md) +→ Claude Code 第一次装的详细步骤 → 本指南 D + +> 💡 **IDE-based 跟 CLI agent 怎么选？** 边写 code 边要 AI 帮忙 → IDE；下单一 prompt 让 agent 自己跑完一整个任务 → CLI。两个可以并用。 + +### 5️⃣ API + 自己写 code（最进阶，能 batch、集成任何工具） + +想自己写 Python script、跑 batch job、把 LLM 接到自己的 app／automation？接下来的 A-C 就是给你的。 + +> 💡 **API key 是什么**：简单讲就是“让程序调用模型的密码”。请把它当成信用卡资料一样保管。 + +--- + +## A — 申请第一个 API key（约 10 分钟） + +### Anthropic Claude（推荐第一次） + +1. 打开 https://console.anthropic.com/ +2. 用 Google、GitHub 或 email 注册。 +3. 进入账号后找到 **API Keys**，点 **Create Key**。 +4. **立刻复制显示出的 key**。多数平台只会显示一次。 +5. 先放在本机密码管理器，或短暂放在本机文本文件；下一节会移到 `.env`。 + +> ⚠️ **API key 三不规则** +> - **不贴**到 chat 窗口、群组、email 或截图。 +> - **不上传**到 git；GitHub 可能扫到后自动撤销。 +> - **不放**云端硬盘纯文本文件；同步到其他设备等于多一份风险。 + +### 其他 LLM 选项 + +#### 西方 cloud（美区友善、英文场景） + +- **OpenAI**：https://platform.openai.com/api-keys + ChatGPT Plus 和 API key 是两件事；订阅 Plus 仍要另外申请 API key。 +- **Google AI Studio**：https://aistudio.google.com/ + 适合先试 Gemini API，免费额度会依地区和账号状态不同。 +- **NVIDIA NIM**：https://build.nvidia.com/ + **托管多个开源 model（Llama / Mistral / DeepSeek-R1 / Qwen / Gemma 等）、OpenAI-compatible API、新账号送 1000 credits**。适合“想试多个 open-source model 但没 GPU”的情境。`base_url=https://integrate.api.nvidia.com/v1`。 + +#### 中国 / 中文场景（地区友善、价格极便宜） + +> 中国大陆用户连 Anthropic / OpenAI 有困难、或想试中文 native 模型，从这边开始。**这些 API 都 OpenAI-compatible**、改 `base_url` 跟 model name 就能跑同一份练习。 + +- **DeepSeek**：https://platform.deepseek.com/ + web 版 https://chat.deepseek.com 完全免费（含 R1 推理模型）。API 价格极便宜（**$0.27 input / $1.10 output per 1M token**、比 haiku 便宜 4 倍）。Code / 推理都很强。 + `base_url=https://api.deepseek.com/v1`、`model=deepseek-chat` 或 `deepseek-reasoner`。 +- **Moonshot Kimi**：https://platform.moonshot.cn/ (中国)、https://platform.moonshot.ai/ (海外) + web 版 https://kimi.com 免费、**1M token context** 是卖点（很大文件 / 长对话）。API 约 $5-15/1M input、按 context size 阶梯计费。 + `base_url=https://api.moonshot.cn/v1` (中国) / `https://api.moonshot.ai/v1` (海外)、`model=kimi-k2-turbo-preview` 等。 +- **通义千问 Qwen（Alibaba）**：https://dashscope.console.aliyun.com/ + web 版 https://chat.qwen.ai 免费。API 走 Alibaba Cloud DashScope、有 **OpenAI-compatible endpoint**（[文档](https://help.aliyun.com/zh/dashscope/developer-reference/compatibility-of-openai-with-dashscope/)）。**同样的 Qwen 模型也能用 Ollama 在本机跑**（`ollama pull qwen2.5:3b`）——cloud 跟 local 两条路径都通。 +- **智谱 GLM（ZhipuAI）**：https://open.bigmodel.cn/ (中国) / https://z.ai/ (海外) + web 版 https://chatglm.cn 免费、有 GLM-4.5、GLM-4-Plus。API 有 free tier、学生申请可额外领 credit。 + +#### 本机（不付 API 费、完全 offline） + +- **Ollama 本地模型**：不用 API key。走本地路线请看 [Cookbook Recipe 6](cookbook.zh-Hans.md#6-本地-llm--cli-agent-快速-walkthrough)。 + 本 repo 的“Path A”默认就是 Ollama；所有 Stage 1-7 练习都能用 `gemma4:e4b`（Stage 1-2）或 `qwen2.5:3b`（Stage 3+）跑通、$0/run。 + +> 💡 **怎么挑第一个**： +> - 想学 agent / production、**美区帐号OK** → **Anthropic Claude**（curriculum canonical） +> - 想学 agent / production、**中国地区**或想试中文模型 → **DeepSeek**（最便宜 cloud option、OpenAI-compat、中文很强） +> - 想试多个 model 但没 GPU → **NVIDIA NIM**（送 1000 credit、托管 10+ open model） +> - 隐私敏感 / 完全免费 / 中国大陆无 cloud → **Ollama**（本机、curriculum 全套都能跑、$0） + +--- + +## B — 装本机环境（约 10 分钟） + +### 装 Python 3.10+ + +- **macOS**：打开 Terminal，输入 `brew install python@3.12`。如果还没有 Homebrew，先看 https://brew.sh。 +- **Windows**：到 https://www.python.org/downloads/ 下载 installer，安装时一定要勾 **Add Python to PATH**。 +- **Linux**：Ubuntu 用 `sudo apt install python3 python3-venv`，Fedora 用 `sudo dnf install python3`。 +- **验证**：macOS / Linux 输入 `python3 --version`；Windows 输入 `py --version`。看到 `Python 3.10` 以上即可。 + +### 装 uv + +uv 是 Python 包管理工具。你可以把它想成“帮你临时装好需要的包再执行”的工具。 + +```bash +# macOS / Linux +curl -LsSf https://astral.sh/uv/install.sh | sh + +# Windows PowerShell +irm https://astral.sh/uv/install.ps1 | iex +``` + +验证： + +```bash +uv --version +``` + +### 建立第一个 `.env` 文件 + +在你要跑 script 的文件夹里，建立一个文件名叫 `.env` 的文件： + +```bash +ANTHROPIC_API_KEY=sk-ant-...贴上你刚才复制的 key +``` + +`.env` 是专门放本机秘密信息的文件。程序会读它，但你不应该把它上传到 GitHub。 + +### 加上 `.gitignore` + +同一个文件夹建立 `.gitignore`： + +```gitignore +.env +__pycache__/ +*.pyc +``` + +这样 git 就不会把 `.env` 收进版本记录。 + +--- + +## C — 跑第一个 `hello-claude.py`（约 5 分钟） + +建立 `hello-claude.py`： + +```python +from anthropic import Anthropic +from dotenv import load_dotenv + +load_dotenv() +client = Anthropic() # 自动读取 ANTHROPIC_API_KEY + +msg = client.messages.create( + model="claude-sonnet-5", + max_tokens=100, + messages=[{"role": "user", "content": "Hello, who are you?"}], +) + +print(msg.content[0].text) +``` + +执行： + +```bash +uv run --with anthropic --with python-dotenv python hello-claude.py +``` + +看到 Claude 回复自我介绍，就代表你的 API key、Python、包都通了。 + +### 常见错误 + +| 错误信息 | 常见原因 | 解法 | +|---|---|---| +| `401 Unauthorized` | API key 没读到或打错 | 回 A 重新复制，确认 `.env` 文件名和内容 | +| `429 Rate limit` | 太快发太多请求 | 等几秒或几分钟再跑 | +| `connection refused` | 网络或防火墙问题 | 确认网络、公司或学校防火墙 | +| `ModuleNotFoundError` | 包没有被安装 | 确认执行的是上面的 `uv run --with ...` 命令 | + +--- + +## D — 第一次装 Claude Code（约 10 分钟；Stage 5 / for-developer 会用到） + +### 先装 Node.js + +> 💡 **Node.js 是什么**：跑 JavaScript 的 runtime（类似 Python interpreter 但是给 JS 用）。**`npm`** 是它附带的“包管理器”（package manager）——跟 Python 的 `pip` 同角色、用来安装别人写好的工具（如下面的 Claude Code）。`npm install -g X` 表示“全局安装 X，之后在任何文件夹都能用”。 + +- **macOS / Linux**：`brew install node`，或从 https://nodejs.org 下载。 +- **Windows**：从 https://nodejs.org 下载安装包。 +- **验证**：输入 `node --version`，看到 v18 以上即可。 + +### 装 Claude Code + +```bash +npm install -g @anthropic-ai/claude-code +``` + +### 第一次认证 + +```bash +claude +``` + +第一次启动时通常会让你选： + +- **Claude subscription**：用 Claude.ai 账号登录，对初学者最省事。 +- **API key**：贴上 A 申请到的 key。 + +### 建立第一份 `CLAUDE.md` + +在你的 project 根目录建立 `CLAUDE.md`。Claude Code 启动时会读它，理解你希望它怎么协助。 + +```markdown +# 你是谁 +我是 [你的名字]，[你的领域，例如：教师 / 研究者 / 写作者]。 + +# Code style +- 注释用简体中文写，code 用英文 +- 写 function 时优先加 type hint +- 不要主动 commit；改完让我手动 git add + +# 不准做的事 +- 不要联网查资料，除非我明确说可以 +- 不要动 `.env` 或 `.gitignore` +- 不要删文件夹，包括子文件夹 +``` + +--- + +## E — 第一个 Skill 示例（约 5 分钟；Stage 5.3 会用到） + +Skill 是 Claude Code 的“可复用 prompt 包”。当你的消息符合描述，Claude Code 会自动加载那份指示。 + +建立 `.claude/skills/hello-skill/SKILL.md`： + +```markdown +--- +name: hello-skill +description: 第一个 hello skill。当用户说“请打招呼”或“say hi”时触发。 +--- + +当用户请你打招呼时，回三件事： + +1. 用简体中文和英文各说一次 hello +2. 提现在的日期（用 system 时间） +3. 给一个今日小提醒（随机选健康 / 学习 / 心情建议） +``` + +跑 `claude`，输入“请打招呼”。如果 Claude 回复三件事，就代表 Skill 被加载了。 + +> 想看更完整的 Skill 设计：看 [Stage 5.3 — Skills](../stages/05-claude-code-ecosystem.zh-Hans.md#53--skillsclaude-code-的行为层-claude-code-生态最关键的一层)。 +> 想看可以照做的示例：看 [Cookbook](cookbook.zh-Hans.md)。 + +--- + +## 接下来去哪 + +| 你现在的状态 | 下一步 | +|---|---| +| 想正式理解 LLM、API、token | [Stage 1 — LLM 基础](../stages/01-llm-basics.zh-Hans.md) | +| 想直接挑身份分支 | [日常用户](../branches/for-everyday-users.zh-Hans.md) / [教师](../branches/for-teacher.zh-Hans.md) / [知识工作者](../branches/for-knowledge-worker.zh-Hans.md) / [研究者](../branches/for-researcher.zh-Hans.md) / [开发者](../branches/for-developer.zh-Hans.md) | +| 想看 Claude Code 完整生态 | [Stage 5 — Claude Code 生态](../stages/05-claude-code-ecosystem.zh-Hans.md) | +| 想本地 LLM、不用云端 key | [Cookbook Recipe 6](cookbook.zh-Hans.md#6-本地-llm--cli-agent-快速-walkthrough) | +| 想比较 CLI agent | [CLI Agents 比较指南](cli-agents-guide.zh-Hans.md) | +| 不懂某个用词 | [Glossary](glossary.zh-Hans.md) | diff --git a/resources/style-guide.en.md b/resources/style-guide.en.md new file mode 100644 index 0000000..2a5d118 --- /dev/null +++ b/resources/style-guide.en.md @@ -0,0 +1,301 @@ +> [繁體中文](./style-guide.md) | [简体中文](./style-guide.zh-Hans.md) | **English** + +# `awesome-agentic-ai-zh` Style Guide + +This is the **single source of truth** for the catalog: terminology, entry schema, license notation, writing style, banned words. + +Read this before opening a PR. Maintainers will use this guide to review. + +--- + +## 📋 Table of Contents + +- [1. Project entry schema](#1-project-entry-schema) +- [2. Recommendation star definitions](#2-recommendation-star-definitions) +- [3. Banned words & alternatives](#3-banned-words--alternatives) +- [4. English nouns to keep](#4-english-nouns-to-keep) +- [5. License notation conventions](#5-license-notation-conventions) +- [6. Stage page template](#6-stage-page-template) +- [7. Branch page template](#7-branch-page-template) +- [8. Writing style](#8-writing-style) +- [9. Links and citations](#9-links-and-citations) + +--- + +## 1. Project entry schema + +Every project entry uses this structure: + +```markdown +### [Repo Name](https://github.com/owner/repo) ⭐⭐⭐⭐ + +| Field | Value | +|---|---| +| Language | Python | +| Stars | ★ 12k+ | +| License | MIT | +| Recommendation | ⭐⭐⭐⭐ | + +**What it teaches**: 1-2 sentences on what this project teaches at this stage. + +**Best for**: 1 sentence on who should study this and why. + +**Notes**: 1-3 sentences of personal evaluation. What's strong, what's weak, what to skip. (Optional.) + +**Run it**: +\`\`\`bash +# minimal install / first-run command +\`\`\` +``` + +### Required fields (GitHub repo entry) +For entries that are real GitHub repos: +- `Stars` (`★ Xk+` format, no thousands separator) +- `License` (SPDX ID or annotated exception, see 5) +- `Recommendation` (⭐ × N, see 2) +- `What it teaches`, `Best for` + +### Required fields (non-repo entry: article / course / video / protocol / documentation) +Some entries are blogs, videos, official docs, or catalog hubs — not GitHub repos. For these: +- `Recommendation` (required) +- `What it teaches`, `Best for` (required) +- `Format` (required, e.g. `Article` / `Video` / `Course` / `Curated list` / `Specification`) +- `Stars` / `License` may be omitted (no GitHub repo to attach) + +Example: an `Anthropic — Building Effective Agents` blog entry uses `Format = Article` + `Recommendation`, without `Stars` or `License`. + +### Optional fields +- `Language` — primary programming language (Python / TypeScript / Chinese) +- `Last update` / `Status` — flag if stale or maintenance slowed +- `Notes`, `Run it` + +### Heading conventions +- Stages 1-4 / 6 use `### [Repo](url)` +- Stage 5 / 7 / branches use `#### [Repo](url)` (when there's a parent H3 category) +- Suffix with stars allowed: `### [Repo](url) ⭐⭐⭐⭐⭐` or sub-label: `### [Repo](url) ⭐ Official` + +--- + +## 2. Recommendation star definitions + +| Stars | Meaning | When to use | +|---|---|---| +| ⭐⭐⭐⭐⭐ | Must-read / must-run | Skipping this will get you stuck in this stage | +| ⭐⭐⭐⭐ | Highly recommended | Strong material to deepen the topic | +| ⭐⭐⭐ | Solid example | Worth running for cross-reference | +| ⭐⭐ | Useful reference | Browse if interested | +| ⭐ | Niche / advanced / for completeness | Most readers can skip | + +**Rules:** +- A repo cited in different stages / branches **should have the same rating** (unless audience-specific reason, then note it explicitly) +- Don't inflate stars to "look encouraging." Honesty > politeness +- Commercial products (Cursor, LangSmith, etc.) follow the same scale + +--- + +## 3. Banned words & alternatives + +This document is **Traditional Chinese (zh-TW, Taiwan)**. The Chinese-side guide enumerates the zh-Hans slips to avoid. For the English companion files, the rules are simpler: + +> 📌 **Language tag convention (BCP 47 / W3C i18n)**: this repo uses `.zh-Hans.md` (not `.zh-CN.md`) for the Simplified Chinese mirror. `Hans` / `Hant` are [BCP 47 script subtags](https://www.w3.org/International/articles/language-tags/), decoupled from region — Simplified Chinese is also used in Singapore and Malaysia, not only mainland China, so `Hans` is more accurate than `CN`. The canonical `README.md` content is **zh-Hant-TW** (Traditional Chinese, Taiwan conventions), kept unsuffixed as GitHub's default landing page. Region distinctions can be added later as `zh-Hans-CN` / `zh-Hant-HK` etc. Thanks to [@xfq](https://github.com/xfq) (W3C i18n lead) for flagging this in [#9](https://github.com/WenyuChiou/awesome-agentic-ai-zh/issues/9). + +### Avoid overclaim phrases + +| Avoid | Use instead | +|---|---| +| "the best in the world" / "industry's strongest" | "comprehensive" / "well-known" / "widely-used" | +| "production-grade" (when describing teaching material) | "teaching-oriented" / "material to learn production patterns from" | +| "the only choice" / "definitive" | "a good option" / "an entry-level pick" | +| "the most urgent" / "the most important" | (just drop the modifier) | +| "authoritative reference" (unless truly the official spec) | "important reference implementation" / "official template" | +| "no problem" (re: legal/license) | "check the terms before use" / "verify the terms yourself" | + +--- + +## 4. English nouns to keep + +Technical writing has terms that **read more naturally in English** than translated: + +- `LLM`, `API`, `SDK`, `MCP` +- `agent`, `tool use`, `function calling`, `prompt`, `prompt caching` +- `framework`, `library`, `repo`, `commit`, `PR`, `branch` +- `RAG`, `embedding`, `vector DB`, `retrieval`, `chunk`, `token` +- `streaming`, `async`, `batch`, `webhook` +- `marketplace`, `plugin`, `skill`, `hook` +- `production` (when meaning "production environment") — but the catalog deliberately avoids it in many places (see Chinese 3) +- `hello-world`, `hands-on exercise` — keep (zh-TW canonical uses `動手練習`; en mirror translates as `hands-on exercise(s)`) + +**Test**: Would a technical reader pause at the translated form? If yes, keep English. + +--- + +## 5. License notation conventions + +### Direct SPDX +- `MIT` +- `Apache-2.0` +- `BSD-3-Clause` +- `GPL-3.0` +- `LGPL-3.0` + +### Annotated exceptions + +| Situation | Notation | +|---|---| +| No SPDX upstream | `NOASSERTION (no SPDX upstream; check LICENSE before use)` | +| AGPL (copyleft) | `AGPL-3.0` + Notes: `AGPL-3.0 license (copyleft) — derivative products that ship modifications must follow the terms.` | +| Custom non-commercial | `NOASSERTION (custom non-commercial)` + Notes: `License is a custom non-commercial term — read the original terms before use.` | +| Multiple per-plugin | `NOASSERTION (each plugin has its own license; check per plugin)` | +| Creative Commons | `CC-BY-4.0`, `CC-BY-NC-SA-4.0`, etc. | + +**Rule**: **Never** read a license as legal advice. Don't say "fine for personal use." Say "read the original terms before use." + +--- + +## 6. Stage page template + +> The same template applies to two locations: +> - `stages/0X-*.md` — shared foundations (0-2) + Track B (Stage 3-8) +> - `tracks/cli/AX-*.md` — Track A (A1-A3) sub-stages also follow this template, with a higher proportion of cross-links (most entries reference existing Stage 5 / 7 / cli-agents-guide content) + +Every stage (except Stage 0) should have: + +```markdown +# Stage N — Topic + +> **English** | [繁體中文](./0N-slug.md) + +⏱ **Time estimate**: N-M weeks (~X-Y hours) + +[1-2 sentence description of the stage's core question] + +## 📌 Learning Goals +- bullet 1 +- bullet 2 + +## 🚪 Entry Conditions (Stage 1+ only) +You should have: +- ... + +## 📚 Required Reading +1. [Link](url) — description +2. ... + +## 🛠 Hands-on Exercises (do them, not just read) + +### Exercise N: Title +Description. + +[3-5 hands-on exercise items] + +## 🎯 Curated Projects + +### [Project Name](url) ⭐⭐⭐⭐ +[entry schema per 1] + +[N entries] + +## ✅ Self-Check Before Stage N+1 +Can you: + +- [ ] ... +- [ ] ... + +If yes → proceed to Stage N+1. +If no → ... + +## 💡 What's Next (optional, mostly used in the last stage) +``` + +**Stage 0 exception**: can omit `Curated Projects` and `Entry Conditions` — it's a prerequisite gateway. + +--- + +## 7. Branch page template + +```markdown +# For [audience] — Specialized Branch + +> **English** | [繁體中文](./for-X.md) + +> [← Back to main path README](../README.en.md) · Branching from end of Stage 7 + +## Use Cases +- bullet 1 +- bullet 2 + +## Curated Projects + +### Sub-category 1 +#### [Project](url) ⭐⭐⭐⭐ +[entry] + +### Sub-category 2 +... + +## Required Reading +1. ... + +## Workflows To Master +- bullet 1 +- bullet 2 +``` + +Branch entries can be more concise than stage entries (full schema table optional), but link + stars + 1-2 sentence description is the minimum. + +--- + +## 8. Writing style + +### Sentence length +- **Single sentence ≤ 25-30 words** for English +- Break long sentences into two +- Don't force English rhythm into translated Chinese (or vice versa) + +### Voice +- Prefer active: "Claude calls the tool" ✓ +- Avoid passive: "The tool is called by Claude" ✗ + +### "You" vs "we" +- **"You" first** — this is learner-facing material +- "I" for author opinion: "I recommend ..." +- Avoid "we" (unless real co-authors exist) + +### Connectives +- Prefer simple: "but, so, because" +- Avoid: "however, therefore, hence" + +--- + +## 9. Links and citations + +### Internal links +- Between stages: relative path `[Stage 4](04-agent-frameworks.en.md)` +- Branch ↔ README: `[← Back to main path](../README.en.md)` +- Cross-stage repo references: full name + link, not just "as cited earlier" + +### External links +- GitHub repo: `https://github.com/owner/repo` (no trailing slash) +- Article / blog: full URL, bold title +- Commercial product (Cursor, Make.com, etc.): official URL, not affiliate + +### Link text conventions +- Repo entry heading: `[owner/repo](url)` or `[Project Name](url)` +- In-prose citation: `[Repo Name](url)` or `\`owner/repo\`` (inline code for short references) +- **Avoid**: "click here," "press this" + +--- + +## Related Internal Design Docs + +This style guide covers "how to write an entry." For **design rationale** — why these 5 branches, why 8 stages — see: + +- [`branches/DESIGN.md`](../branches/DESIGN.md) — Branch design notes (why these audiences, where entries belong) (zh) +- [`stages/DESIGN.md`](../stages/DESIGN.md) — Stage design notes (why this structure, how exercises are chosen) (zh) +- [`cli-agents-guide.en.md`](cli-agents-guide.en.md) — Cross-cutting CLI agent comparison + +## Modifying this guide + +PRs to this guide are welcome. Open an Issue first to discuss — terminology decisions affect 100+ entries. + +Current maintainer: [@WenyuChiou](https://github.com/WenyuChiou). diff --git a/resources/style-guide.md b/resources/style-guide.md new file mode 100644 index 0000000..c65682f --- /dev/null +++ b/resources/style-guide.md @@ -0,0 +1,341 @@ +> **繁體中文** | [简体中文](./style-guide.zh-Hans.md) | [English](./style-guide.en.md) + +# `awesome-agentic-ai-zh` 風格指南 + +這份指南是這份 catalog 的**單一真實來源**——術語、entry 結構、license 標註、寫作風格、禁用詞，全部以這份文件為準。 + +PR 之前請先讀完本文。專案維護者也會用這份指南做 review。 + +--- + +## 📋 目錄 + +- [1. 專案 entry schema](#1-專案-entry-schema) +- [2. 推薦星等定義](#2-推薦星等定義) +- [3. 禁用詞與替代](#3-禁用詞與替代) +- [4. 可保留的英文名詞](#4-可保留的英文名詞) +- [5. License 標註慣例](#5-license-標註慣例) +- [6. Stage 頁面模板](#6-stage-頁面模板) +- [7. Branch 頁面模板](#7-branch-頁面模板) +- [8. 寫作風格規範](#8-寫作風格規範) +- [9. 連結與引用](#9-連結與引用) + +--- + +## 1. 專案 entry schema + +每個 project entry 統一格式如下： + +```markdown +### [Repo Name](https://github.com/owner/repo) ⭐⭐⭐⭐ + +| 欄位 | 內容 | +|---|---| +| 語言 | Python | +| Stars | ★ 12k+ | +| License | MIT | +| 推薦度 | ⭐⭐⭐⭐ | + +**教什麼**：1-2 句話，這個 project 在這個 stage 教什麼具體的東西。 + +**適合誰**：1 句話，誰應該讀這個、為什麼。 + +**備註**：1-3 句個人評價。哪裡好、哪裡弱、哪裡可以跳。（可省略） + +**怎麼跑**： +\`\`\`bash +# 最小安裝指令、第一次跑該執行什麼 +\`\`\` +``` + +### 必填欄位（GitHub repo entry） +對「真實 GitHub repo」的 entry： +- `Stars`（★ Xk+ 格式，無千位逗號） +- `License`（SPDX ID 或標註例外，見 5） +- `推薦度`（⭐ × N，見 2） +- `教什麼`、`適合誰` + +### 必填欄位（非 repo entry：article / course / video / protocol / documentation） +某些 entry 不是 GitHub repo 而是文章、影片、官方文件、catalog hub。對這類： +- `推薦度`（必填） +- `教什麼`、`適合誰`（必填） +- `形式`（必填，標明是 `文章` / `影片` / `課程` / `精選清單` / `規格文件` 等） +- `Stars` / `License` 可省略（沒有 GitHub repo 對應） + +範例：`Anthropic — Building Effective Agents` 部落格文章用 `形式 = 文章` + 推薦度，不需要 Stars / License。 + +### 選填欄位 +- `語言` — 主要程式語言（Python / TypeScript / 中文 等） +- `最後更新` / `狀態` — 已停滯或維護放緩時加註 +- `備註`、`怎麼跑` + +### 標題格式 +- Stage 1-4 / 6 用 `### [Repo](url)` +- Stage 5 / 7 / branches 用 `#### [Repo](url)`（已有上層 H3 分類時） +- 標題後可接星等：`### [Repo](url) ⭐⭐⭐⭐⭐` 或副標：`### [Repo](url) ⭐ 官方` + +--- + +## 2. 推薦星等定義 + +| 星等 | 含義 | 何時用 | +|---|---|---| +| ⭐⭐⭐⭐⭐ | 必讀 / 必做 | 該 stage 不讀這個會卡住 | +| ⭐⭐⭐⭐ | 強烈建議 | 深入學該主題的好材料 | +| ⭐⭐⭐ | 紮實範例 | 值得跑一遍、互相對照 | +| ⭐⭐ | 有用參考 | 有興趣再看 | +| ⭐ | 利基 / 進階 / 為了完整性 | 多數讀者可跳 | + +**準則**： +- 同一個 repo 出現在不同 stage / branch 時，**星等應一致**（除非有明確 audience-specific 理由，且註明在備註） +- 不要因為「想要看起來推薦」就給高星等。誠實 > 客氣 +- 商業產品（Cursor、LangSmith 等）也照同一套標準 + +--- + +## 3. 禁用詞與替代 + +這份文件以**繁體中文（zh-TW，台灣慣例）** 為準。下表列出常見的 zh-Hans 用詞與替代。 + +> 📌 **語言代碼慣例（BCP 47 / W3C i18n）**：repo 用 `.zh-Hans.md`（不是 `.zh-CN.md`）標記簡體中文檔。`Hans` / `Hant` 是 [BCP 47 script subtag](https://www.w3.org/International/articles/language-tags/)，跟地區解耦——簡體中文不只用在中國大陸（也用在新加坡、馬來西亞），用 `Hans` 比 `CN` 更準確。canonical README 的內容是 **zh-Hant-TW**（繁體中文，台灣慣例），但檔名保持無 suffix 的 `README.md` 作為 GitHub 預設首頁。未來若要分地區可再擴成 `zh-Hans-CN` / `zh-Hant-HK` 等。感謝 [@xfq](https://github.com/xfq)（W3C i18n lead）在 [#9](https://github.com/WenyuChiou/awesome-agentic-ai-zh/issues/9) 指出這個問題。 + +### 繁簡用詞替換 + +| 禁用（zh-Hans） | 改用（zh-TW） | +|---|---| +| 教程 | 教學 / 課程 / 導讀 | +| 視頻 | 影片 | +| 軟件 | 軟體 | +| 文件（指 file 時） | 檔案 | +| 文档 / 文件（指 docs 時） | 文件 / 文件（這個保留） | +| 代碼 | 程式碼 / 原始碼 | +| 用戶 | 使用者 | +| 網絡 | 網路 | +| 接口 | 介面 | +| 默認 | 預設 | +| 函数 | 函式 | +| 算法 | 演算法 | +| 程序（指程式時） | 程式 | +| 質量（指品質時） | 品質 | +| 信息 | 資訊 | +| 數據 | 資料 | +| 內存 | 記憶體 | + +### Overclaim（誇大）用語禁用 + +| 禁用 | 改用 | +|---|---| +| 全世界最好的 / 業界最強 | 完整的 / 知名的 / 廣泛使用的 | +| production-grade（描述教材時） | 教學導向 / 用來學 production pattern 的教材 | +| 首選 / 唯一選擇 | 不錯的選項 / 入門選擇之一 | +| 最緊迫 / 最重要 | （直接不要修飾） | +| 權威參考（除非真的是官方 spec） | 重要參考實作 / 官方範本 | +| 沒問題（法律或 license 判斷時） | 使用前先讀條款 / 條款還是要自己看過 | + +### 中夾英（English-in-Chinese）禁用句型 + +| 禁用 | 改用 | +|---|---| +| follow 條款 | 遵守條款 | +| ready-made 教材 | 現成可改的教材 | +| NotebookLM-like 工具 | 類 NotebookLM 的工具 / 類似 NotebookLM 的工具 | +| 視覺化 node-based | 視覺化節點式 | +| Anthropic host 的 server | Anthropic 維護的 server | +| coding 流程 | 開發流程 / 程式開發流程 | + +--- + +## 4. 可保留的英文名詞 + +技術寫作中**保留英文**比硬翻譯讀起來更自然的詞： + +- `LLM`、`API`、`SDK`、`MCP` +- `agent`、`tool use`、`function calling`、`prompt`、`prompt caching` +- `framework`、`library`、`repo`、`commit`、`PR`、`branch` +- `RAG`、`embedding`、`vector DB`、`retrieval`、`chunk`、`token` +- `streaming`、`async`、`batch`、`webhook` +- `marketplace`、`plugin`、`skill`、`hook` +- `project`、`repo` （可保留也可改用「專案」） +- `production`（指「正式環境」時）— 但本 catalog 多數場合刻意避免（見 3） +- `動手練習`、`hello-world` — 保留 + +**判準**：技術文件圈讀者習慣的英文術語就保留，避免「太政治正確的中文化」。 + +--- + +## 5. License 標註慣例 + +### 常見 license 直寫 +- `MIT` +- `Apache-2.0` +- `BSD-3-Clause` +- `GPL-3.0` +- `LGPL-3.0` + +### 需要加註的特殊情況 + +| 情況 | 寫法 | +|---|---| +| 上游無 SPDX | `NOASSERTION（上游未提供 SPDX；使用前請讀 LICENSE）` | +| AGPL（傳染性） | `AGPL-3.0` + 備註：`AGPL-3.0 license（傳染性開源）— 修改後散布的衍生產品需遵守條款。` | +| 自訂非商用 | `NOASSERTION（自訂非商用）` + 備註：`License 是自訂非商用條款，使用前請先讀原始條款。` | +| 多元 license（每個 plugin 自己有） | `NOASSERTION（每個 plugin 獨立 license，請看各自目錄）` | +| Creative Commons | 直寫 `CC-BY-4.0`、`CC-BY-NC-SA-4.0` 等 | + +**規則**：**永遠不要**把 license 解讀成法律建議。「研究 / 個人使用沒問題」這種句子禁用。改成「使用前先讀原始條款」。 + +--- + +## 6. Stage 頁面模板 + +> 同一個模板適用於兩個位置： +> - `stages/0X-*.md` — 共用基礎（0-2）+ Track B（Stage 3-8） +> - `tracks/cli/AX-*.md` — Track A（A1-A3）的 sub-stage，也照同一模板，只是 cross-link 比例較高（多數 entry 引用既有 Stage 5 / 7 / cli-agents-guide） + +每個 stage（Stage 0 除外）都應該有： + +```markdown +# Stage N — 主題 + +> [English](./0N-slug.en.md) | **繁體中文** + +⏱ **時間估算**：N-M 週（約 X-Y 小時） + +[1-2 句話描述這個 stage 的核心問題] + +## 📌 學習目標 +- bullet 1 +- bullet 2 +... + +## 🚪 進入條件（Stage 1+ 才需要） +你應該已經： +- ... + +## 📚 必修閱讀 +1. [連結](url) — 描述 +2. ... + +## 🛠 動手練習（不是看過就好） + +### 練習 N：標題 +描述。 + +[3-5 個動手練習 items] + +## 🎯 精選 Projects + +### [Project Name](url) ⭐⭐⭐⭐ +[entry schema 見 1] + +[N 個 entries] + +## ✅ 進 Stage N+1 前的自我檢查 +你能不能： + +- [ ] ... +- [ ] ... + +如果可以 → 進 Stage N+1。 +如果不行 → ... + +## 💡 接下來（選填，多在最後一個 stage 用） +``` + +**Stage 0 例外**：可以省略 `精選 Projects`、`進入條件`，因為它是 prerequisite gateway。 + +--- + +## 7. Branch 頁面模板 + +```markdown +# 給 [audience] — 專業分支 + +> [English](./for-X.en.md) | **繁體中文** + +> [← 回主路線 README](../README.md) · 從 Stage 7 結尾分支出來 + +## 使用情境 +- bullet 1 +- bullet 2 + +## 精選 Projects + +### 子分類 1 +#### [Project](url) ⭐⭐⭐⭐ +[entry] + +### 子分類 2 +... + +## 必修閱讀 +1. ... + +## 必練流程 +- bullet 1 +- bullet 2 +``` + +Branch 的 entry 格式可以比 stage 簡潔（不一定要完整 schema 表格），但連結 + 星等 + 1-2 句描述是最低門檻。 + +--- + +## 8. 寫作風格規範 + +### 句長 +- **單句不超過 60 字**（中文標點計入） +- 太長就斷成兩句 +- 英文 rhythm 強迫塞進中文 = 翻譯腔，要避免 + +### 標點 +- **中文用全形**：，。：；「」（） +- **句中夾英文**時，英文前後可以留空格也可以不留，但全文要一致 +- **避免 ASCII 逗號 `,`** 在中文句中（會中夾英） + +### 主動 vs 被動 +- 偏好主動句：「Claude 呼叫工具」 ✓ +- 避免被動句：「工具被 Claude 呼叫」 ✗ + +### 「你」 vs 「我們」 +- **「你」優先**——這是給讀者的學習材料 +- 「我」用於作者發表意見時：「我建議...」 +- 避免「我們」（除了合著者實際存在的場合） + +### 連接詞 +- 偏好簡單：「但、所以、因為、不過」 +- 避免：「然而、因此、由於、之所以」 + +--- + +## 9. 連結與引用 + +### 內部連結 +- Stage 之間：相對路徑 `[Stage 4](04-agent-frameworks.md)` +- Branch ↔ README：`[← 回主路線](../README.md)` +- 跨 stage 引用同一 repo：用全名 + 連結，不要只寫「之前提過」 + +### 外部連結 +- GitHub repo：`https://github.com/owner/repo` ✓ 不加 trailing slash +- 文章 / 部落格：完整 URL，標題用粗體 +- 商業產品（Cursor、Make.com 等）：用官方網址，不是 affiliate + +### 連結文字慣例 +- Repo entry 標題：`[owner/repo](url)` 或 `[Project Name](url)` +- 句中引用：`[Repo Name](url)` 或 `\`owner/repo\``（短引用用 inline code） +- 連結文字**避免**「點這裡」、「按這個」 + +--- + +## 相關內部設計文件 + +這份 style-guide 講「entry 怎麼寫」。為什麼分這 5 個 branch、為什麼是 8 個 stage 這類**設計理由**，見： + +- [`branches/DESIGN.md`](../branches/DESIGN.md)——branch 設計筆記（為什麼這樣切、entry 該放哪） +- [`stages/DESIGN.md`](../stages/DESIGN.md)——stage 設計筆記（為什麼這結構、動手練習 怎麼挑） +- [`cli-agents-guide.md`](cli-agents-guide.md)——cross-cutting CLI agent 比較指南 + +## 修改本指南 + +這份指南本身也歡迎 PR。修改前請先開 Issue 討論——術語決策影響 100+ 個 entry。 + +當前 maintainer：[@WenyuChiou](https://github.com/WenyuChiou)。 diff --git a/resources/style-guide.zh-Hans.md b/resources/style-guide.zh-Hans.md new file mode 100644 index 0000000..7d90df3 --- /dev/null +++ b/resources/style-guide.zh-Hans.md @@ -0,0 +1,344 @@ +> [繁體中文](./style-guide.md) | **简体中文** | [English](./style-guide.en.md) + +# `awesome-agentic-ai-zh` 风格指南 + +这份指南是这份 catalog 的**单一真实来源**——术语、entry 结构、license 标注、写作风格、禁用词，全部以这份文件为准。 + +PR 之前请先读完本文。项目维护者也会用这份指南做 review。 + +--- + +## 📋 目录 + +- [1. 项目 entry schema](#1-项目-entry-schema) +- [2. 推荐星等定义](#2-推荐星等定义) +- [3. 禁用词与替代](#3-禁用词与替代) +- [4. 可保留的英文名词](#4-可保留的英文名词) +- [5. License 标注惯例](#5-License-标注惯例) +- [6. Stage 页面模板](#6-Stage-页面模板) +- [7. Branch 页面模板](#7-Branch-页面模板) +- [8. 写作风格规范](#8-写作风格规范) +- [9. 链接与引用](#9-链接与引用) + +--- + +## 1. 项目 entry schema + +每个 project entry 统一格式如下： + +```markdown +### [Repo Name](https://github.com/owner/repo) ⭐⭐⭐⭐ + +| 字段 | 内容 | +|---|---| +| 语言 | Python | +| Stars | ★ 12k+ | +| License | MIT | +| 推荐度 | ⭐⭐⭐⭐ | + +**教什么**：1-2 句话，这个 project 在这个 stage 教什么具体的东西。 + +**适合谁**：1 句话，谁应该读这个、为什么。 + +**备注**：1-3 句个人评价。哪里好、哪里弱、哪里可以跳。（可省略） + +**怎么跑**： +```bash +# 最小安装指令、第一次跑该执行什么 +``` +``` + +### 必填字段（GitHub repo entry） +对“真实 GitHub repo”的 entry： +- `Stars`（★ Xk+ 格式，无千位逗号） +- `License`（SPDX ID 或标注例外，见 5） +- `推荐度`（⭐ × N，见 2） +- `教什么`、`适合谁` + +### 必填字段（非 repo entry：article / course / video / protocol / documentation） +某些 entry 不是 GitHub repo 而是文章、视频、官方文件、catalog hub。对此类： +- `推荐度`（必填） +- `教什么`、`适合谁`（必填） +- `形式`（必填，标明是 `文章` / `视频` / `课程` / `精选列表` / `规格文件` 等） +- `Stars` / `License` 可省略（没有 GitHub repo 对应） + +范例：`Anthropic — Building Effective Agents` 部落格文章用 `形式 = 文章` + 推荐度，不需要 Stars / License。 + +### 选填字段 +- `语言` — 主要编程语言（Python / TypeScript / 中文 等） +- `最后更新` / `状态` — 已停滞或维护放缓时加注 +- `备注`、`怎么跑` + +### 标题格式 +- Stage 1-4 / 6 用 `### [Repo](url)` +- Stage 5 / 7 / branches 用 `#### [Repo](url)`（已有上层 H3 分类时） +- 标题后可接星等：`### [Repo](url) ⭐⭐⭐⭐⭐` 或副标题：`### [Repo](url) ⭐ 官方` + +--- + +## 2. 推荐星等定义 + +| 星等 | 含义 | 何时用 | +|---|---|---| +| ⭐⭐⭐⭐⭐ | 必读 / 必做 | 该 stage 不读这个会卡住 | +| ⭐⭐⭐⭐ | 强烈建议 | 深入学该主题的好材料 | +| ⭐⭐⭐ | 扎实范例 | 值得跑一遍、互相对照 | +| ⭐⭐ | 有用参考 | 有兴趣再看 | +| ⭐ | 利基 / 进阶 / C 完整性 | 多数学者可跳 | + +**准则**： +- 同一个 repo 出现在不同 stage / branch 时，**星等应一致**（除非有明确 audience-specific 理由，且注明在备注） +- 不要因为“想要看起来推荐”就给高星等。诚实 > 客气 +- 商业产品（Cursor、LangSmith 等）也照同一套标准 + +--- + +## 3. 禁用词与替代 + +这份文件以**简体中文（zh-Hans，中国大陆惯例）** 为准。下表列出常见的 zh-TW 用词与替代。 + +> 📌 **语言代码惯例（BCP 47 / W3C i18n）**：repo 用 `.zh-Hans.md`（不是 `.zh-CN.md`）标记简体中文档。`Hans` / `Hant` 是 [BCP 47 script subtag](https://www.w3.org/International/articles/language-tags/)，跟地区解耦——简体中文不只用在中国大陆（也用在新加坡、马来西亚），用 `Hans` 比 `CN` 更准确。canonical README 的内容是 **zh-Hant-TW**（繁体中文，台湾惯例），但档名保持无 suffix 的 `README.md` 作为 GitHub 默认首页。未来若要分地区可再扩成 `zh-Hans-CN` / `zh-Hant-HK` 等。感谢 [@xfq](https://github.com/xfq)（W3C i18n lead）在 [#9](https://github.com/WenyuChiou/awesome-agentic-ai-zh/issues/9) 指出这个问题。 + +### 繁简用词替换 + +| 禁用（zh-TW） | 改用（zh-Hans） | +|---|---| +| 使用者 | 用户 | +| 軟體 | 软件 | +| 資料 | 数据 | +| 專案 | 项目 | +| 腳本 | 脚本 | +| 預設 | 默认 | +| 設定 | 设置 | +| 連結 | 链接 | +| 練習 | 练习 | +| 動手 | 动手 | +| 飛書 | 飞书 | +| 個 | 个 | +| 兩 | 两 | +| “” | "" | +| 整合 | 集成 | +| 系統 | 系统 | +| 點 | 点 | +| 為 | 为 | +| 過 | 过 | +| 還 | 还 | + +### Overclaim（夸大）用语禁用 + +| 禁用 | 改用 | +|---|---| +| 全世界最好的 / 业界最强 | 完整的 / 知名的 / 广泛使用的 | +| production-grade（描述教材时） | 教学导向 / 用来学 production pattern 的教材 | +| 首选 / 唯一选择 | 不错的选项 / 入门选择之一 | +| 最紧迫 / 最重要 | （直接不要修饰） | +| 权威参考（除非真的是官方 spec） | 重要参考实作 / 官方范本 | +| 没问题（法律或 license 判断时） | 使用前先读条款 / 条款还是要自己看过 | + +### 中夹英（English-in-Chinese）禁用句型 + +| 禁用 | 改用 | +|---|---| +| follow 条款 | 遵守条款 | +| ready-made 教材 | 现成可改的教材 | +| NotebookLM-like 工具 | 类 NotebookLM 的工具 / 类似 NotebookLM 的工具 | +| 视觉化 node-based | 视觉化节点式 | +| Anthropic host 的 server | Anthropic 维护的 server | +| coding 流程 | 开发流程 / 程序开发流程 | + +--- + +## 4. 可保留的英文名词 + +技术写作中**保留英文**比硬翻译读起来更自然的词： + +- `LLM`、`API`、`SDK`、`MCP` +- `agent`、`tool use`、`function calling`、`prompt`、`prompt caching` +- `framework`、`library`、`repo`、`commit`、`PR`、`branch` +- `RAG`、`embedding`、`vector DB`、`retrieval`、`chunk`、`token` +- `streaming`、`async`、`batch`、`webhook` +- `marketplace`、`plugin`、`skill`、`hook` +- `project`、`repo` （可保留也可改用“项目”） +- `production`（指“正式环境”时）— 但本 catalog 多数场合刻意避免（见 3） +- `动手练习`、`hello-world` — 保留 + +**判准**：技术文件圈读者习惯的英文术语就保留，避免“太政治正确的中文化”。 + +--- + +## 5. License 标注惯例 + +### 常见 license 直写 +- `MIT` +- `Apache-2.0` +- `BSD-3-Clause` +- `GPL-3.0` +- `LGPL-3.0` + +### 需要加注的特殊情况 + +| 情况 | 写法 | +|---|---| +| 上游无 SPDX | `NOASSERTION（上游未提供 SPDX；使用前请读 LICENSE）` | +| AGPL（传染性） | `AGPL-3.0` + 备注：`AGPL-3.0 license（传染性开源）— 修改后散布的衍生产品需遵守条款。` | +| 自定义非商用 | `NOASSERTION（自定义非商用）` + 备注：`License 是自定义非商用条款，使用前请先读原始条款。` | +| 多元 license（每个 plugin 自己有） | `NOASSERTION（每个 plugin 独立 license，请看各自目录）` | +| Creative Commons | 直写 `CC-BY-4.0`、`CC-BY-NC-SA-4.0` 等 | + +**规则**：**永远不要**把 license 解读成法律建议。“研究 / 个人使用没问题”这种句子禁用。改成“使用前先读原始条款”。 + +--- + +## 6. Stage 页面模板 + +> 同一个模板适用于两个位置： +> - `stages/0X-*.md` — 共用基础（0-2）+ Track B（Stage 3-8） +> - `tracks/cli/AX-*.md` — Track A（A1-A3）的 sub-stage，也照同一模板，只是 cross-link 比例较高（多数 entry 引用既有 Stage 5 / 7 / cli-agents-guide） + +每个 stage（Stage 0 除外）都应该有： + +```markdown +# Stage N — 主题 + +> [English](./0N-slug.en.md) | **简体中文** + +⏱ **时间估算**：N-M 周（约 X-Y 小时） + +[1-2 句话描述这个 stage 的核心问题] + +## 📌 学习目标 +- bullet 1 +- bullet 2 +... + +## 🚪 进入条件（Stage 1+ 才需要） +你应该已经： +- ... + +## 📚 必修阅读 +1. [链接](url) — 描述 +2. ... + +## 🛠 动手练习（不是看过就好） + +### 练习 N：标题 +描述。 + +[3-5 个动手练习 items] + +## 🎯 精选 Projects + +### [Project Name](url) ⭐⭐⭐⭐ +[entry schema 见 1] + +[N 个 entries] + +## ✅ 进 Stage N+1 前的自我检查 +你能不能： + +- [ ] ... +- [ ] ... + +如果可以 → 进 Stage N+1。 +如果不行 → ... + +## 💡 接下来（选填，多在最后一个 stage 用） +``` + +**Stage 0 例外**：可以省略 `精选 Projects`、`进入条件`，因为它是 prerequisite gateway。 + +--- + +## 7. Branch 页面模板 + +```markdown +# 给 [audience] — 专业分支 + +> [English](./for-X.en.md) | **简体中文** + +> [← 回主路线 README](../README.md) · 从 Stage 7 结尾分支出来 + +## 使用情境 +- bullet 1 +- bullet 2 + +## 精选 Projects + +### 子分类 1 +#### [Project](url) ⭐⭐⭐⭐ +[entry] + +### 子分类 2 +... + +## 必修阅读 +1. ... + +## 必练流程 +- bullet 1 +- bullet 2 +``` + +Branch 的 entry 格式可以比 stage 简洁（不一定要完整 schema 表格），但链接 + 星等 + 1-2 句描述是最低门槛。 + +--- + +## 8. 写作风格规范 + +### 句长 +- **单句不超过 60 字**（中文标点计入） +- 太长就断成两句 +- 英文 rhythm 强迫塞进中文 = 翻译腔，要避免 + +### 标点 +- **中文用全角**：，。：；“”（） +- **句中夹英文**时，英文前后可以留空格也可以不留，但全文要一致 +- **避免 ASCII 逗号 `,`** 在中文句中（会中夹英） + +### 主动 vs 被动 +- 偏主动句：“Claude 调用工具” ✓ +- 避免被动句：“工具被 Claude 调用” ✗ + +### “你” vs “我们” +- **“你”优先**——这是给读者的学习材料 +- “我”用于作者发表意见时：“我建议...” +- 避免“我们”（除了合著者实际存在的场合） + +### 连接词 +- 偏好简单：“但、所以、因为、不过” +- 避免：“然而、因此、由于、之所以” + +--- + +## 9. 链接与引用 + +### 内部链接 +- Stage 之间：相对路径 `[Stage 4](./04-agent-frameworks.zh-Hans.md)` +- Branch ↔ README：`[← 回主路线](../README.md)` +- 跨 stage 引用同一 repo：用全名 + 链接，不要只写“之前提过” + +### 外部链接 +- GitHub repo：`https://github.com/owner/repo` ✓ 不加 trailing slash +- 文章 / 部落格：完整 URL，标题用粗体 +- 商业产品（Cursor、Make.com 等）：用官方网址，不是 affiliate + +### 链接文字惯例 +- Repo entry 标题：`[owner/repo](url)` 或 `[Project Name](url)` +- 句中引用：`[Repo Name](url)` 或 ``owner/repo``（短引用用 inline code） +- 链接文字**避免**“点这里”、“按这个” + +--- + +## 相关内部设计文件 + +这份 style-guide 讲“entry 怎么写”。为什么分这 5 个 branch、为什么是 8 个 stage 这类**设计理由**，见： + +- [`branches/DESIGN.md`](../branches/DESIGN.md)—branch 设计笔记（为什么这样切、entry 该放哪） +- [`stages/DESIGN.md`](../stages/DESIGN.md)—stage 设计笔记（为什么这结构、动手练习 怎么挑） +- [`cli-agents-guide.zh-Hans.md`](cli-agents-guide.zh-Hans.md)—cross-cutting CLI agent 比较指南 + +## 修改本指南 + +这份指南本身也欢迎 PR。修改前请先开 Issue 讨论——术语决策影响 100+ 个 entry。 + +当前 maintainer：[@WenyuChiou](https://github.com/WenyuChiou)。 diff --git a/resources/subagent-advanced.en.md b/resources/subagent-advanced.en.md new file mode 100644 index 0000000..7f3e941 --- /dev/null +++ b/resources/subagent-advanced.en.md @@ -0,0 +1,276 @@ +# Advanced subagent usage — Description Patterns / Composition / Debug + +> [繁體中文](./subagent-advanced.md) | [简体中文](./subagent-advanced.zh-Hans.md) | **English** + +> 📋 **Who this is for**: You already know how to use built-in subagents (you have gone through [Stage 5.5](../stages/05-claude-code-ecosystem.en.md#55--subagents-claude-codes-native-multi-agent-mechanism--2025-new-feature) + the [cookbook](./subagent-cookbook.en.md)), and you are ready to: (1) **write your own** subagent, (2) **compose multiple** subagents, or (3) **debug** a broken subagent. +> +> ⚠️ **Prerequisite**: Read Stage 5.5’s “Common confusing concepts clarified” section and the 15 cookbook recipes first. If you jump into this without the cookbook, you will get stuck at the “what is a subagent?” layer. + +--- + +## How to Read This Doc + +Three independent advanced topics. Jump to the section you need: + +| Your question | Read | +|---|---| +| I wrote a subagent, but Claude never dispatches it proactively. Why? | [§1 How to Write a Description That Triggers Proactive Dispatch](#1-how-to-write-a-description-that-triggers-proactive-dispatch) | +| I want to run 2-3 subagents as a pipeline or in parallel. How should I design that? | [§2 How to Design Composition Patterns](#2-how-to-design-composition-patterns) | +| A subagent fails, errors, or behaves differently from its settings. How do I debug it? | [§3 Debugging Tools for Custom Subagents](#3-debugging-tools-for-custom-subagents) | + +Each section stands alone; skim or jump around as needed. + +--- + +## §1 How to Write a Description That Triggers Proactive Dispatch + +How does the main session decide which subagent to dispatch? It reads the frontmatter at the top of `.claude/agents/.md` (**frontmatter** = the YAML settings block at the very top of the file, wrapped in `---`) and specifically the **`description`** field. **The wording affects dispatch probability**. Below are 4 common bugs and fixes: + +### Bug 1: The description is too abstract, so Claude does not know when to dispatch it + +❌ **Bad pattern**: +```yaml +description: A helpful code reviewer. +``` +**Problem**: “helpful” is empty, and “reviewer” is generic. Claude cannot see “under what condition should I dispatch this,” so it waits until the user explicitly names it. + +✅ **Better** (specific trigger condition + scope): +```yaml +description: Use PROACTIVELY when the user has staged ≥ 50 lines of changes and is about to commit. Reviews staged diff for security issues, style violations, missing error handling, and test gaps. Returns per-category PASS/FAIL + concrete fix list. +``` +**Why this is better**: (1) `PROACTIVELY` is a strong signal word, (2) the trigger is explicit: “staged ≥ 50 lines + about to commit,” (3) it lists the 4 things the subagent checks, and (4) it states the return format. + +--- + +### Bug 2: `PROACTIVELY` is present, but the condition is too broad + +❌ **Bad pattern**: +```yaml +description: Use PROACTIVELY for all code-related tasks. +``` +**Problem**: “all code-related” is too broad. Claude will dispatch it for every coding task; even “fix this typo” triggers it, which becomes noise. + +✅ **Better** (narrow the condition): +```yaml +description: Use PROACTIVELY when a commit is about to land that modifies authentication, database queries, or API routes — these are high-risk surface areas needing extra review. +``` +**Why this is better**: It limits dispatch to **high-risk surfaces** (auth / DB / API) and avoids over-triggering. + +--- + +### Bug 3: `PROACTIVELY` is missing, so the subagent can only wait passively + +❌ **Bad pattern**: +```yaml +description: Reviews code when asked. +``` +**Problem**: Without `PROACTIVELY`, Claude only dispatches it when the user explicitly says “review this.” If the user does not think to ask for a review, the review never happens. + +✅ **Better** (add PROACTIVELY + a trigger scenario): +```yaml +description: Code reviewer. Use PROACTIVELY when staged changes touch test files but the test count didn't increase — likely missing test coverage for new logic. +``` + +> 💡 **Passive vs proactive**: +> - **Need an always-on safety net** (for example, security review) → use `PROACTIVELY` + a clear trigger +> - **Run only when the user explicitly asks** (for example, token-heavy deep research) → skip `PROACTIVELY` and use `use when user asks for ...` + +--- + +### Bug 4: The description is too long for the picker’s preferences + +❌ **Bad pattern** (500+ words and too many irrelevant details): +```yaml +description: This subagent performs comprehensive code review including security analysis, performance profiling, style enforcement, type checking, dependency auditing, license compliance, documentation completeness verification, test coverage assessment, accessibility validation, internationalization checks... (continues for paragraphs) +``` +**Problem**: Anthropic has **not announced a character limit** (as of 2026-05), but an overly long description still hurts: (1) it consumes context budget, (2) by the time Claude reaches the later text during dispatch decisions, the key point is diluted, and (3) when multiple subagents compete, a short precise description often beats a long exhaustive one. + +✅ **Better** (compress to 2-3 sentences with the most important trigger + scope): +```yaml +description: Use PROACTIVELY before commits touching auth or payment code. Checks: hardcoded secrets, missing input validation, SQL injection risk. Returns issue list with file:line. +``` + +> 📌 **Description cheatsheet**: +> 1. Start with `Use PROACTIVELY when X` or `Use when user asks for Y` +> 2. List 2-4 **specific things it does** (not empty words like “helpful” or “comprehensive”) +> 3. State the **return format** so the main session knows what shape it will receive +> 4. **2-3 sentences is enough**: precise > complete +> 5. **Description matching is semantic, not exact keyword matching**. Keywords help, but the trigger condition still has to be clear. +> +> 💡 **Language choice**: the `description` field is **best written in English**. Claude is trained heavily on English, and English trigger keywords such as `PROACTIVELY` are the most reliable. + +--- + +## §2 How to Design Composition Patterns + +When you want to run 2+ subagents together, how should you compose them? The 3 patterns below are common community patterns: + +![Subagent Composition — 3 Patterns](../resources/diagrams/subagent-composition-patterns.en.png) + +> 📊 **The diagram above**: A — Parallel (most common) / B — Pipeline (multi-LLM orchestration) / C — Meta-Agent (NOT recommended, listed to avoid). See the full picture first, then read the details. + +### Pattern A — Parallel Isolation (most common and simplest) + +**When to use it**: 3 tasks are **independent** and do not need to communicate. Examples: +- 4 files need the same audit (spawn 4 `general-purpose` subagents) +- Run “code review” + “find related papers” + “write a changelog” as 3 independent tasks + +**How to run it**: list N independent tasks **in a single prompt** (for example, “Audit these 4 files at the same time: A.md / B.md / C.md / D.md”). Claude will call the Task tool multiple times within one turn and run them in parallel automatically. This is **not** the same as entering N prompts one after another; that is sequential and waits for the previous one to finish. For long-running independent background work, use `/bg`. + +**Cost**: Low (no coordination needed) + +**Trap**: The 3 subagents cannot see each other’s results. If there is a dependency, use Pattern B. Also, **do not let multiple subagents write to the same file at the same time**; that can cause write conflicts or file corruption. + +--- + +### Pattern B — Pipeline Chaining (multi-step collaboration) + +**When to use it**: The task needs a **step order**, and the previous step’s output is the next step’s input. Examples: +- Multi-LLM workflow: Claude planner → Codex implementer → Gemini reviewer +- Literature-research pipeline: splitter divides the topic → multiple researchers run sub-queries → reconciler merges the draft + +**How to run it**: write a skill / orchestrator, such as the [agent-collab-workspace](https://github.com/WenyuChiou/agent-collab-skills) plugin, and let it dispatch subagents in order. **Having the main session call each one manually is tedious and error-prone**. + +**Cost**: Medium (requires coordination logic and `.coord/` intermediate files) + +**Trap**: (1) Every added step increases the failure surface, and (2) one subagent failure can block the whole pipeline, so every step needs acceptance criteria. + +--- + +### Pattern C — Meta-Agent (**not recommended**, included as a pitfall) + +**Why it exists**: In theory, “one subagent writes more subagents” sounds elegant. + +**Why it is not recommended**: +1. **Context explosion risk**: a subagent writes a subagent that writes a subagent... and it gets out of control +2. **No one audits the newly created agent**: it may create a dangerous tools allowlist +3. **Anthropic’s official examples do not use this pattern**: the community has not developed a reliable pattern either +4. **Debugging nightmare**: when something fails, it is unclear whether the original prompt, the meta-agent, or the generated agent is at fault + +**What to do instead**: When you notice a repeated task and think “I should write a meta-agent,” **use a skill or template instead**. Do not take the meta-agent route. + +--- + +### How to choose among the 3 patterns + +| Your situation | Use | +|---|---| +| 3 independent tasks, each returning to the main session | **Pattern A** | +| Multi-step collaboration with input → output dependencies | **Pattern B** | +| You want to “automatically generate subagents” | **Do not do this** (ask why; usually a skill / template is a better fit) | + +**90% of use cases are Pattern A**. Before moving to Pattern B, confirm that you really need coordination and are not over-engineering. + +--- + +## §3 Debugging Tools for Custom Subagents + +> 📌 **Different angle from [Stage 5.5 §Clarifying 5 Gotchas](../stages/05-claude-code-ecosystem.en.md#55--subagents-claude-codes-native-multi-agent-mechanism--2025-new-feature)**: +> Stage 5.5 Gotchas are **best-practice-oriented**: things to pay attention to when writing a subagent +> This section's 5 entry points are **debug-oriented**: where to look after a subagent has already broken +> 3 topics overlap (tools / model / memory), but the angle differs; read both for full coverage + +You wrote `.claude/agents/.md`, but the result is not what you expected. Here are 5 debug entry points: + +### Debug entry point 1: Confirm Claude Code can see your agent + +```bash +# Run inside the Claude Code conversation: +/agents +``` +**Expected**: The list includes the name you wrote. If it does **not**: +- The file is in the wrong location (it should be in `~/.claude/agents/.md` for global or `/.claude/agents/.md` for project-level; **when names collide, project-level overrides global**) +- YAML frontmatter syntax is invalid (for example, `---` is not wrapped correctly or the `name:` field is misspelled) +- There is a name conflict (an agent with the same name was overridden) + +--- + +### Debug entry point 2: Confirm the description can be selected + +Run this prompt to test whether Claude dispatches the subagent on its own: +``` +Describe one scenario that should trigger your subagent (without explicitly naming it), and see which agent Claude dispatches. +``` + +**Claude did not dispatch your agent**: the description does not make Claude see “I should dispatch this.” +- Missing the `PROACTIVELY` keyword → add it +- Condition is too abstract → rewrite it as a concrete trigger +- Overlaps with another agent’s description → write the **distinctive** part + +--- + +### Debug entry point 3: Confirm tool permissions are correct + +The subagent is dispatched, then reports “I don't have access to X tool” — the `tools:` allowlist is missing an entry. + +```yaml +# Commonly forgotten tools: +tools: + - Read + - Grep + - Glob # Find files + - Bash # Run git / pytest + - WebFetch # Read external URLs + - WebSearch # Search the web +``` + +> ⚠️ **Trap**: Writing `tools:` as an **empty string** (`tools: ""`) or **omitting the field entirely** does not mean “no tools.” In both cases, the subagent **inherits every tool from the main session**. To restrict tools, **write the allowlist explicitly**. + +--- + +### Debug entry point 4: Confirm the model is not silently burning money + +If a subagent does not specify `model:`, it uses the same model as the main session. If the main session is Opus, the subagent is also Opus, and token cost can burn 4x faster. + +```yaml +# sonnet is enough for most tasks: +model: sonnet + +# Use haiku for simple tasks (finding files, running grep): +model: haiku + +# Use opus only when strong reasoning is truly needed (your call): +# model: opus +``` + +Check the token statistics Claude Code shows after the session ends (in the lower-right status bar or session summary), or use `/clear` and compare usage before and after. + +--- + +### Debug entry point 5: Confirm the prompt is self-contained + +A subagent **cannot see the main session conversation**. Every dispatch starts with a **fresh context**. + +❌ **Wrong prompt**: +``` +Review the changes we discussed. +``` +The subagent cannot see what “we discussed” refers to, so it will guess. + +✅ **Correct prompt**: +``` +Review the staged changes in this repo (git diff --cached). Focus: security +issues, error handling gaps. Per-issue: file:line + suggested fix. +``` +This is fully self-contained. The subagent can run from this prompt without any “previous context.” + +--- + +### 5-Point Quick Check + +| Symptom | Debug entry point | +|---|---| +| `/agents` does not show it | Entry point 1 (file location / YAML syntax) | +| Claude does not dispatch it proactively | Entry point 2 (description wording) | +| The subagent reports “no access to X tool” | Entry point 3 (`tools:` allowlist) | +| Token bill spikes | Entry point 4 (`model:` not specified) | +| The subagent behaves strangely or goes off track | Entry point 5 (prompt is not self-contained) | + +--- + +## Next Steps + +- **More dispatch recipes** → [`subagent-cookbook.en.md`](./subagent-cookbook.en.md) (15 copy-paste dispatch prompts) +- **Understand how subagents relate to skills / MCP** → [Stage 5.5](../stages/05-claude-code-ecosystem.en.md#55--subagents-claude-codes-native-multi-agent-mechanism--2025-new-feature) +- **Run multi-agent coordination** (Pattern B) → the [agent-collab-skills](https://github.com/WenyuChiou/agent-collab-skills) plugin +- **Vocabulary quick lookup** → [`glossary.en.md` § 5. Claude Code ecosystem](./glossary.en.md#5-claude-code-ecosystem) diff --git a/resources/subagent-advanced.md b/resources/subagent-advanced.md new file mode 100644 index 0000000..00febad --- /dev/null +++ b/resources/subagent-advanced.md @@ -0,0 +1,276 @@ +# Subagent 進階使用 — Description 寫法 / Composition / Debug + +> **繁體中文** | [简体中文](./subagent-advanced.zh-Hans.md) | [English](./subagent-advanced.en.md) + +> 📋 **這份是給誰看的**：你已經會用內建 subagent（[Stage 5.5](../stages/05-claude-code-ecosystem.md#55--subagentsclaude-code-原生-multi-agent-機制-2025-新功能) + [cookbook](./subagent-cookbook.md) 走完了），準備：(1) **自己寫一個** subagent、(2) **組合多個** subagent、或 (3) **debug** 跑壞的 subagent。 +> +> ⚠️ **先決條件**：讀完 Stage 5.5 §易混淆觀念釐清 + cookbook 15 個 recipe 之後再來。沒讀 cookbook 直接看這份會卡在「subagent 是什麼」這層。 + +--- + +## 怎麼讀這份文件 + +3 個獨立的進階主題、按需要查： + +| 你的問題 | 看哪節 | +|---|---| +| 寫了一個 subagent，但 Claude 從來不主動派它——為什麼？ | [§1 Description 怎麼寫才會被主 session 主動 spawn](#1-description-怎麼寫才會被主-session-主動-spawn) | +| 想跑 2-3 個 subagent 串成 pipeline / parallel——怎麼設計？ | [§2 Composition pattern 怎麼設計](#2-composition-pattern-怎麼設計) | +| Subagent 跑壞 / 報錯 / 行為跟設定的不一樣——怎麼 debug？ | [§3 自製 subagent 的 debug 工具](#3-自製-subagent-的-debug-工具) | + +每節獨立、可跳讀。 + +--- + +## §1 Description 怎麼寫才會被主 session 主動 spawn + +主 session 怎麼決定派哪個 subagent？看 `.claude/agents/.md` 開頭 frontmatter（**frontmatter** = 檔案最開頭的 YAML 設定區、用 `---` 包起來）的 **`description`** 欄位。**寫法影響被選機率**——下面是 4 個常見 bug + 修正： + +### Bug 1: Description 太抽象、Claude 不知何時派 + +❌ **壞寫法**： +```yaml +description: A helpful code reviewer. +``` +**問題**：「helpful」是空詞、reviewer 是泛詞。Claude 看不到「什麼情境下我該派這個」、只能等使用者明白點名。 + +✅ **改成**（具體觸發條件 + 範圍）： +```yaml +description: Use PROACTIVELY when the user has staged ≥ 50 lines of changes and is about to commit. Reviews staged diff for security issues, style violations, missing error handling, and test gaps. Returns per-category PASS/FAIL + concrete fix list. +``` +**為什麼好**：(1) `PROACTIVELY` 是強訊號詞、(2) 明確觸發條件「staged ≥ 50 lines + about to commit」、(3) 列了會做的 4 件事、(4) 講清楚回傳格式。 + +--- + +### Bug 2: 用了 `PROACTIVELY` 但條件太寬 + +❌ **壞寫法**： +```yaml +description: Use PROACTIVELY for all code-related tasks. +``` +**問題**：「all code-related」太寬、Claude 會在每個 coding task 都派——使用者輸入「fix this typo」也派 = 騷擾。 + +✅ **改成**（縮窄條件）： +```yaml +description: Use PROACTIVELY when a commit is about to land that modifies authentication, database queries, or API routes — these are high-risk surface areas needing extra review. +``` +**為什麼好**：限定**高風險範圍**（auth / DB / API）、避免 over-trigger。 + +--- + +### Bug 3: 完全沒寫 `PROACTIVELY`、只能被動等 + +❌ **壞寫法**： +```yaml +description: Reviews code when asked. +``` +**問題**：沒有 `PROACTIVELY`——Claude 只會在使用者明白要求「review this」時才派；如果使用者沒想到要 review、就漏掉了。 + +✅ **改成**（加 PROACTIVELY + 觸發場景）： +```yaml +description: Code reviewer. Use PROACTIVELY when staged changes touch test files but the test count didn't increase — likely missing test coverage for new logic. +``` + +> 💡 **被動 vs 主動的選擇**： +> - **想 always-on safety net**（譬如安全 review）→ 用 `PROACTIVELY` + 明確 trigger +> - **只在使用者明白要求才跑**（譬如比較費 token 的 deep research）→ 不寫 `PROACTIVELY`、改用 `use when user asks for ...` + +--- + +### Bug 4: Description 過長、超過 picker 偏好 + +❌ **壞寫法**（500+ 字、講太多無關細節）： +```yaml +description: This subagent performs comprehensive code review including security analysis, performance profiling, style enforcement, type checking, dependency auditing, license compliance, documentation completeness verification, test coverage assessment, accessibility validation, internationalization checks... (continues for paragraphs) +``` +**問題**：雖然 Anthropic **目前沒公告字元上限**（截至 2026-05）、但 description 過長：(1) 佔 context budget、(2) Claude 在 dispatch 決策時讀到後段已經失去重點、(3) 多個 subagent 競爭時長的反而輸給短而精準的。 + +✅ **改成**（精簡到 2-3 句、留最重要的 trigger + 範圍）： +```yaml +description: Use PROACTIVELY before commits touching auth or payment code. Checks: hardcoded secrets, missing input validation, SQL injection risk. Returns issue list with file:line. +``` + +> 📌 **Description 寫法 cheatsheet**： +> 1. 開頭 `Use PROACTIVELY when X` 或 `Use when user asks for Y` +> 2. 列 2-4 個**具體會做的事**（不是「helpful」「comprehensive」這種空詞） +> 3. 講**回傳格式**（讓主 session 知道接到的會是什麼形狀） +> 4. **2-3 句就好**——精準 > 完整 +> 5. **Description 是語意比對、不是精確關鍵字比對**——關鍵字有幫助、但觸發條件要寫清楚 +> +> 💡 **語言選擇**：description 欄位**建議用英文**——Claude 用英文訓練、英文 trigger keyword（PROACTIVELY 等）效果最穩定。 + +--- + +## §2 Composition pattern 怎麼設計 + +要跑 2+ subagent 一起時、怎麼組合？下面 3 種 pattern 是社群歸納的常見組合： + +![Subagent Composition — 3 種組合 Pattern](../resources/diagrams/subagent-composition-patterns.png) + +> 📊 **上圖**：A 平行（最常用）/ B Pipeline（多 LLM 編排）/ C Meta-Agent（不推薦避坑）—— 先看全貌再讀細節。 + +### Pattern A — 平行隔離（最常用、最簡單） + +**何時用**：3 個任務**獨立**、不需要互相溝通。例： +- 4 個 file 都要做同樣的 audit（spawn 4 個 `general-purpose`） +- 同時跑「code review」+「找相關 paper」+「寫 changelog」3 個獨立任務 + +**怎麼跑**：在**一個 prompt 裡**列出 N 個獨立任務（譬如「請同時 audit 這 4 個檔案：A.md / B.md / C.md / D.md」）——Claude 在單一 turn 內**多次呼叫 Task tool**、自動並行。**不是**連續輸入 N 個 prompt（那是 sequential、要等前一個結束）。要長時間獨立背景跑用 `/bg`。 + +**成本**：低（不需 coordination） + +**陷阱**：3 個 subagent 看不到對方結果——如果有依賴關係要走 Pattern B；另外**也不能讓多個 subagent 同時寫到同一份檔案**、會造成 write conflict / 檔案損毀。 + +--- + +### Pattern B — Pipeline 串接（多步驟協作） + +**何時用**：任務需要**步驟順序**、前一個的 output 是後一個的 input。例： +- Multi-LLM workflow：Claude planner → Codex implementer → Gemini reviewer +- 文獻研究 pipeline：splitter 切題 → 多 researcher 跑各 sub-query → reconciler 合稿 + +**怎麼跑**：寫一個 skill / orchestrator（譬如 [agent-collab-workspace](https://github.com/WenyuChiou/agent-collab-skills) plugin）、它幫你按序派 subagent。**主 session 自己一個一個叫太累、會出錯**。 + +**成本**：中（要 coordination 邏輯、要管 `.coord/` 中介檔案） + +**陷阱**：(1) 每多一步、容錯 surface 變大、(2) 一個 subagent 出錯整個 pipeline 卡住——所以每步都要寫 acceptance criteria。 + +--- + +### Pattern C — Meta-Agent（**不推薦**、列出來避坑） + +**為什麼存在**：理論上「一個 subagent 寫出更多 subagent」聽起來很 elegant。 + +**為什麼不推薦**： +1. **Context explosion 風險**——subagent 寫的 subagent 寫的 subagent... 失控 +2. **沒人 audit 新建的 agent**——可能寫出危險的 tools allowlist +3. **Anthropic 官方範例都不這樣用**——社群也未發展出可靠 pattern +4. **debug 噩夢**——出錯時不知該怪原始 prompt、meta-agent、還是被生成的 agent + +**該怎麼辦**：你發現 task 重複很多、覺得「該寫個 meta」——**請用 skill 或 template 取代**、不要走 meta-agent 路線。 + +--- + +### 3 個 pattern 怎麼選 + +| 你的情境 | 用哪個 | +|---|---| +| 3 個獨立任務、結果各回主 session | **Pattern A** | +| 多步驟協作、有 input → output 依賴 | **Pattern B** | +| 想「自動生成 subagent」 | **不要做**（去想為什麼、通常 skill / template 更合適）| + +**90% 的使用情境是 Pattern A**——上 Pattern B 之前先確認「真的需要 coordination」、別 over-engineer。 + +--- + +## §3 自製 subagent 的 debug 工具 + +> 📌 **跟 [Stage 5.5 §易混淆 5 條 Gotcha](../stages/05-claude-code-ecosystem.md#55--subagentsclaude-code-原生-multi-agent-機制-2025-新功能) 角度不同**—— +> 5.5 Gotcha 是「**best-practice-oriented**」：寫 subagent 時就要注意的事 +> 這節 5 切點是「**debug-oriented**」：subagent **已經跑壞了**該從哪裡查 +> 內容有 3 條重疊（tools / model / memory）但視角不同、可分開讀 + +寫了 `.claude/agents/.md`、結果不如預期——下面是 debug 的 5 個切點： + +### 切點 1: 確認 Claude Code 看得到你的 agent + +```bash +# 在 Claude Code 對話框內跑： +/agents +``` +**期待**：列表裡有你寫的那個 name。**沒有**： +- 檔案位置錯（應該在 `~/.claude/agents/.md` global 或 `/.claude/agents/.md` project-level；**同名時 project-level 覆蓋 global**） +- YAML frontmatter 語法錯（譬如 `---` 沒包好、`name:` 欄位拼錯） +- 名字衝突（同名 agent 被覆蓋） + +--- + +### 切點 2: 確認 description 寫得會被選 + +跑這個 prompt 測 Claude 會不會自主派： +``` +描述 1 個會觸發你 subagent 的情境（不要明白叫名字）、看 Claude 派的是誰。 +``` + +**Claude 沒派你 agent**：description 寫得 Claude 看不出「我該派這個」。 +- 缺 `PROACTIVELY` keyword → 加上 +- 條件太抽象 → 改成具體 trigger +- 跟其他 agent 描述重疊 → 寫出**獨特性** + +--- + +### 切點 3: 確認工具權限正確 + +Subagent 派完報「I don't have access to X tool」——`tools:` 白名單漏寫。 + +```yaml +# 常忘的工具： +tools: + - Read + - Grep + - Glob # 找檔案 + - Bash # 跑 git / pytest + - WebFetch # 讀外部 URL + - WebSearch # 上網搜 +``` + +> ⚠️ **陷阱**：`tools:` 寫**空字串** `tools: ""` 或**省略整個欄位**、都不等於「沒工具」——兩種情況都**繼承主 session 全部工具**。要限制就**明寫清單**。 + +--- + +### 切點 4: 確認 model 不會默默燒大錢 + +Subagent 沒指定 `model:` = 跟主 session 用同一個。主 session 是 Opus、subagent 也 Opus、token 燒 4x。 + +```yaml +# 大多數任務 sonnet 就夠： +model: sonnet + +# 簡單任務（找檔案、跑 grep）用 haiku 更省： +model: haiku + +# 真的需要強推理才用 opus（自己評估）： +# model: opus +``` + +看 session 結束後 Claude Code 顯示的 token 統計（右下角 status bar 或 session summary），或用 `/clear` 後對比前後用量。 + +--- + +### 切點 5: 確認 prompt 是 self-contained + +Subagent **看不到主 session 對話**——每次派遣是**全新 context**。 + +❌ **錯的 prompt**： +``` +Review the changes we discussed. +``` +Subagent 看不到「我們討論的」是啥、會自己亂猜。 + +✅ **對的 prompt**： +``` +Review the staged changes in this repo (git diff --cached). Focus: security +issues, error handling gaps. Per-issue: file:line + suggested fix. +``` +**全 self-contained**——subagent 可以靠這段 prompt 跑起來、不需要「之前的 context」。 + +--- + +### 5 切點 quick check + +| 症狀 | 切點 | +|---|---| +| `/agents` 看不到 | 切點 1（檔案位置 / YAML 語法）| +| Claude 不主動派 | 切點 2（description 寫法）| +| Subagent 報「no access to X tool」 | 切點 3（tools 白名單）| +| Token 帳單暴增 | 切點 4（model 沒指定）| +| Subagent 跑亂、行為怪 | 切點 5（prompt 不 self-contained）| + +--- + +## 接下來 + +- **想看更多 dispatch recipe** → [`subagent-cookbook.md`](./subagent-cookbook.md)（15 個複製即用的派遣 prompt） +- **想理解 subagent 跟 skill / MCP 的層次關係** → [Stage 5.5](../stages/05-claude-code-ecosystem.md#55--subagentsclaude-code-原生-multi-agent-機制-2025-新功能) +- **想跑 multi-agent coordination**（Pattern B）→ [agent-collab-skills](https://github.com/WenyuChiou/agent-collab-skills) plugin +- **詞彙快查** → [`glossary.md` § 5. Claude Code 生態](./glossary.md#subagent子-agent) diff --git a/resources/subagent-advanced.zh-Hans.md b/resources/subagent-advanced.zh-Hans.md new file mode 100644 index 0000000..03c47df --- /dev/null +++ b/resources/subagent-advanced.zh-Hans.md @@ -0,0 +1,276 @@ +# Subagent 进阶使用 — Description 写法 / Composition / Debug + +> [繁體中文](./subagent-advanced.md) | **简体中文** | [English](./subagent-advanced.en.md) + +> 📋 **这份是给谁看的**：你已经会用内置 subagent（[Stage 5.5](../stages/05-claude-code-ecosystem.zh-Hans.md#55--subagentsclaude-code-原生-multi-agent-机制-2025-新功能) + [cookbook](./subagent-cookbook.zh-Hans.md) 走完了），准备：(1) **自己写一个** subagent、(2) **组合多个** subagent，或 (3) **debug** 跑坏的 subagent。 +> +> ⚠️ **先决条件**：读完 Stage 5.5 §易混淆概念厘清 + cookbook 15 个 recipe 之后再来。没读 cookbook 直接看这份，会卡在“subagent 是什么”这一层。 + +--- + +## 怎么读这份文件 + +3 个独立的进阶主题，按需要查： + +| 你的问题 | 看哪节 | +|---|---| +| 写了一个 subagent，但 Claude 从来不主动派遣它——为什么？ | [§1 Description 怎么写才会被主 session 主动 spawn](#1-description-怎么写才会被主-session-主动-spawn) | +| 想跑 2-3 个 subagent 串成 pipeline / parallel——怎么设计？ | [§2 Composition pattern 怎么设计](#2-composition-pattern-怎么设计) | +| subagent 跑坏 / 报错 / 行为跟设置的不一样——怎么 debug？ | [§3 自制 subagent 的 debug 工具](#3-自制-subagent-的-debug-工具) | + +每节独立，可以跳读。 + +--- + +## §1 Description 怎么写才会被主 session 主动 spawn + +主 session 怎么决定派遣哪个 subagent？看 `.claude/agents/.md` 开头 frontmatter（**frontmatter** = 文件最开头的 YAML 设置区，用 `---` 包起来）的 **`description`** 字段。**写法影响被选概率**——下面是 4 个常见 bug + 修正： + +### Bug 1: Description 太抽象，Claude 不知道何时派遣 + +❌ **坏写法**： +```yaml +description: A helpful code reviewer. +``` +**问题**：“helpful”是空词，reviewer 是泛词。Claude 看不到“什么情境下我该派遣这个”，只能等用户明说名字。 + +✅ **改成**（具体触发条件 + 范围）： +```yaml +description: Use PROACTIVELY when the user has staged ≥ 50 lines of changes and is about to commit. Reviews staged diff for security issues, style violations, missing error handling, and test gaps. Returns per-category PASS/FAIL + concrete fix list. +``` +**为什么好**：(1) `PROACTIVELY` 是强信号词，(2) 明确触发条件“staged ≥ 50 lines + about to commit”，(3) 列了会做的 4 件事，(4) 讲清楚返回格式。 + +--- + +### Bug 2: 用了 `PROACTIVELY`，但条件太宽 + +❌ **坏写法**： +```yaml +description: Use PROACTIVELY for all code-related tasks. +``` +**问题**：“all code-related”太宽，Claude 会在每个 coding task 都派遣——用户输入“fix this typo”也派 = 打扰。 + +✅ **改成**（缩窄条件）： +```yaml +description: Use PROACTIVELY when a commit is about to land that modifies authentication, database queries, or API routes — these are high-risk surface areas needing extra review. +``` +**为什么好**：限定**高风险范围**（auth / DB / API），避免 over-trigger。 + +--- + +### Bug 3: 完全没写 `PROACTIVELY`，只能被动等 + +❌ **坏写法**： +```yaml +description: Reviews code when asked. +``` +**问题**：没有 `PROACTIVELY`——Claude 只会在用户明说“review this”时才派遣；如果用户没想到要 review，就漏掉了。 + +✅ **改成**（加 PROACTIVELY + 触发场景）： +```yaml +description: Code reviewer. Use PROACTIVELY when staged changes touch test files but the test count didn't increase — likely missing test coverage for new logic. +``` + +> 💡 **被动 vs 主动的选择**： +> - **想 always-on safety net**（例如安全 review）→ 用 `PROACTIVELY` + 明确 trigger +> - **只在用户明说要求才跑**（例如比较费 token 的 deep research）→ 不写 `PROACTIVELY`，改用 `use when user asks for ...` + +--- + +### Bug 4: Description 过长，超过 picker 偏好 + +❌ **坏写法**（500+ 字，讲太多无关细节）： +```yaml +description: This subagent performs comprehensive code review including security analysis, performance profiling, style enforcement, type checking, dependency auditing, license compliance, documentation completeness verification, test coverage assessment, accessibility validation, internationalization checks... (continues for paragraphs) +``` +**问题**：虽然 Anthropic **目前没公告字符上限**（截至 2026-05），但 description 过长：(1) 占 context budget，(2) Claude 在 dispatch 决策时读到后段已经失去重点，(3) 多个 subagent 竞争时，长的反而输给短而精准的。 + +✅ **改成**（精简到 2-3 句，留最重要的 trigger + 范围）： +```yaml +description: Use PROACTIVELY before commits touching auth or payment code. Checks: hardcoded secrets, missing input validation, SQL injection risk. Returns issue list with file:line. +``` + +> 📌 **Description 写法 cheatsheet**： +> 1. 开头 `Use PROACTIVELY when X` 或 `Use when user asks for Y` +> 2. 列 2-4 个**具体会做的事**（不是“helpful”“comprehensive”这种空词） +> 3. 讲**返回格式**（让主 session 知道接到的会是什么形状） +> 4. **2-3 句就好**——精准 > 完整 +> 5. **Description 是语义匹配，不是精确关键词匹配**——关键词有帮助，但触发条件要写清楚 +> +> 💡 **语言选择**：description 字段**建议用英文**——Claude 用英文训练，英文 trigger keyword（`PROACTIVELY` 等）效果最稳定。 + +--- + +## §2 Composition pattern 怎么设计 + +要跑 2+ subagent 一起时，怎么组合？下面 3 种 pattern 是社群归纳的常见组合： + +![Subagent Composition — 3 种组合 Pattern](../resources/diagrams/subagent-composition-patterns.zh-Hans.png) + +> 📊 **上图**：A 并行（最常用）/ B Pipeline（多 LLM 编排）/ C Meta-Agent（不推荐避坑）—— 先看全貌再读细节。 + +### Pattern A — 平行隔离（最常用、最简单） + +**何时用**：3 个任务**独立**，不需要互相沟通。例： +- 4 个 file 都要做同样的 audit（spawn 4 个 `general-purpose`） +- 同时跑“code review”+“找相关 paper”+“写 changelog”3 个独立任务 + +**怎么跑**：在**一个 prompt 里**列出 N 个独立任务（例如“请同时 audit 这 4 个文件：A.md / B.md / C.md / D.md”）——Claude 在单一 turn 内**多次调用 Task tool**，自动并行。**不是**连续输入 N 个 prompt（那是 sequential，要等前一个结束）。要长时间独立背景跑，用 `/bg`。 + +**成本**：低（不需要 coordination） + +**陷阱**：3 个 subagent 看不到对方结果——如果有依赖关系要走 Pattern B；另外**也不能让多个 subagent 同时写到同一份文件**，会造成 write conflict / 文件损坏。 + +--- + +### Pattern B — Pipeline 串接（多步骤协作） + +**何时用**：任务需要**步骤顺序**，前一个的 output 是后一个的 input。例： +- Multi-LLM workflow：Claude planner → Codex implementer → Gemini reviewer +- 文献研究 pipeline：splitter 切题 → 多 researcher 跑各 sub-query → reconciler 合稿 + +**怎么跑**：写一个 skill / orchestrator（例如 [agent-collab-workspace](https://github.com/WenyuChiou/agent-collab-skills) plugin），它帮你按序派遣 subagent。**主 session 自己一个一个叫太累，也会出错**。 + +**成本**：中（要 coordination 逻辑，要管 `.coord/` 中介文件） + +**陷阱**：(1) 每多一步，容错 surface 变大，(2) 一个 subagent 出错，整个 pipeline 卡住——所以每步都要写 acceptance criteria。 + +--- + +### Pattern C — Meta-Agent（**不推荐**，列出来避坑） + +**为什么存在**：理论上“一个 subagent 写出更多 subagent”听起来很 elegant。 + +**为什么不推荐**： +1. **Context explosion 风险**——subagent 写的 subagent 写的 subagent... 失控 +2. **没人 audit 新建的 agent**——可能写出危险的 tools allowlist +3. **Anthropic 官方示例都不这样用**——社群也未发展出可靠 pattern +4. **debug 噩梦**——出错时不知道该怪原始 prompt、meta-agent，还是被生成的 agent + +**该怎么办**：你发现 task 重复很多，觉得“该写个 meta”——**请用 skill 或 template 取代**，不要走 meta-agent 路线。 + +--- + +### 3 个 pattern 怎么选 + +| 你的情境 | 用哪个 | +|---|---| +| 3 个独立任务，结果各回主 session | **Pattern A** | +| 多步骤协作，有 input → output 依赖 | **Pattern B** | +| 想“自动生成 subagent” | **不要做**（去想为什么，通常 skill / template 更合适）| + +**90% 的使用情境是 Pattern A**——上 Pattern B 之前先确认“真的需要 coordination”，别 over-engineer。 + +--- + +## §3 自制 subagent 的 debug 工具 + +> 📌 **跟 [Stage 5.5 §易混淆 5 条 Gotcha](../stages/05-claude-code-ecosystem.zh-Hans.md#55--subagentsclaude-code-原生-multi-agent-机制-2025-新功能) 角度不同**—— +> 5.5 Gotcha 是 "**best-practice-oriented**"：写 subagent 时就要注意的事 +> 这节 5 切点是 "**debug-oriented**"：subagent **已经跑坏了**该从哪里查 +> 内容有 3 条重叠（tools / model / memory）但视角不同，可分开读 + +写了 `.claude/agents/.md`，结果不如预期——下面是 debug 的 5 个切点： + +### 切点 1: 确认 Claude Code 看得到你的 agent + +```bash +# 在 Claude Code 对话框内跑： +/agents +``` +**预期**：列表里有你写的那个 name。**没有**： +- 文件位置错（应该在 `~/.claude/agents/.md` global 或 `/.claude/agents/.md` project-level；**同名时 project-level 覆盖 global**） +- YAML frontmatter 语法错（例如 `---` 没包好、`name:` 字段拼错） +- 名字冲突（同名 agent 被覆盖） + +--- + +### 切点 2: 确认 description 写得会被选 + +跑这个 prompt 测 Claude 会不会自主派遣： +``` +描述 1 个会触发你 subagent 的情境（不要直接叫名字），看 Claude 派的是谁。 +``` + +**Claude 没派遣你 agent**：description 写得让 Claude 看不出“我该派遣这个”。 +- 缺 `PROACTIVELY` keyword → 加上 +- 条件太抽象 → 改成具体 trigger +- 跟其他 agent 描述重叠 → 写出**独特性** + +--- + +### 切点 3: 确认工具权限正确 + +subagent 派完报“I don't have access to X tool”——`tools:` 白名单漏写。 + +```yaml +# 常忘的工具： +tools: + - Read + - Grep + - Glob # 找文件 + - Bash # 跑 git / pytest + - WebFetch # 读外部 URL + - WebSearch # 上网搜 +``` + +> ⚠️ **陷阱**：`tools:` 写**空字符串** `tools: ""` 或**省略整个字段**，都不等于“没工具”——两种情况都**继承主 session 全部工具**。要限制就**明写清单**。 + +--- + +### 切点 4: 确认 model 不会默默烧大钱 + +subagent 没指定 `model:` = 跟主 session 用同一个。主 session 是 Opus，subagent 也 Opus，token 烧 4x。 + +```yaml +# 大多数任务 sonnet 就够： +model: sonnet + +# 简单任务（找文件、跑 grep）用 haiku 更省： +model: haiku + +# 真的需要强推理才用 opus（自己评估）： +# model: opus +``` + +看 session 结束后 Claude Code 显示的 token 统计（右下角 status bar 或 session summary），或用 `/clear` 后对比前后用量。 + +--- + +### 切点 5: 确认 prompt 是 self-contained + +subagent **看不到主 session 对话**——每次派遣都是**全新 context**。 + +❌ **错的 prompt**： +``` +Review the changes we discussed. +``` +subagent 看不到“我们讨论的”是什么，会自己乱猜。 + +✅ **对的 prompt**： +``` +Review the staged changes in this repo (git diff --cached). Focus: security +issues, error handling gaps. Per-issue: file:line + suggested fix. +``` +**全 self-contained**——subagent 可以靠这段 prompt 跑起来，不需要“之前的 context”。 + +--- + +### 5 切点 quick check + +| 症状 | 切点 | +|---|---| +| `/agents` 看不到 | 切点 1（文件位置 / YAML 语法）| +| Claude 不主动派遣 | 切点 2（description 写法）| +| subagent 报“no access to X tool” | 切点 3（tools 白名单）| +| Token 账单暴增 | 切点 4（model 没指定）| +| subagent 跑乱、行为怪 | 切点 5（prompt 不 self-contained）| + +--- + +## 接下来 + +- **想看更多 dispatch recipe** → [`subagent-cookbook.zh-Hans.md`](./subagent-cookbook.zh-Hans.md)（15 个复制粘贴即用的派遣 prompt） +- **想理解 subagent 跟 skill / MCP 的层次关系** → [Stage 5.5](../stages/05-claude-code-ecosystem.zh-Hans.md#55--subagentsclaude-code-原生-multi-agent-机制-2025-新功能) +- **想跑 multi-agent coordination**（Pattern B）→ [agent-collab-skills](https://github.com/WenyuChiou/agent-collab-skills) plugin +- **词汇快查** → [`glossary.zh-Hans.md` § 5. Claude Code 生态](./glossary.zh-Hans.md#subagent子-agent) diff --git a/resources/subagent-cookbook.en.md b/resources/subagent-cookbook.en.md new file mode 100644 index 0000000..0b79edc --- /dev/null +++ b/resources/subagent-cookbook.en.md @@ -0,0 +1,366 @@ +# Subagent Cookbook — 15 Copy-Paste Dispatch Recipes + +> [繁體中文](./subagent-cookbook.md) | [简体中文](./subagent-cookbook.zh-Hans.md) | **English** + +> 📋 **What this is**: [Stage 5.5](../stages/05-claude-code-ecosystem.en.md#55--subagents-claude-codes-native-multi-agent-mechanism--2025-new-feature) teaches the concept of subagents. This cookbook teaches you how to **use them today**: 15 scenarios, each with “which subagent to use + a complete prompt template you can copy-paste + when not to use it.” +> +> ⚠️ **First time here? Read Stage 5.5’s “Which subagents can you dispatch?” and “decision table” sections first**. Once you understand what a subagent is and which ones Claude Code includes, come back and pick a recipe. + +--- + +## How to Read This Cookbook + +Each recipe uses the same 4-part structure: + +| Section | Content | Why it exists | +|---|---|---| +| **Scenario** | A concrete situation you might run into today | Find a recipe from “I have X problem,” not from “I want to use a subagent” | +| **Subagent** | Which one to use (Claude Code’s built-in name) | Copy the name directly; no guessing | +| **Prompt Template** | Copy-paste-ready instruction text | No need to invent the prompt yourself | +| **When Not to Use It** | A better alternative than using a subagent | Avoid overkill | + +> 💡 **How to actually dispatch a subagent**: in your Claude Code terminal conversation, **type or paste the prompt template directly**. That is all. Claude sees the instruction, automatically uses the Task tool (Claude Code’s internal dispatch mechanism) to find the right subagent, runs it, and returns a short summary to the main session. **No slash command or special syntax is required**. +> +> 📌 **Subagent vs slash command**: `/agents` is a list command, **not how you invoke a subagent**. To dispatch a subagent, type ordinary prompt text directly. For complete comparison tables (subagent vs skill / vs slash command / description router), see [Stage 5.5 §Common Confusing Concepts Clarified](../stages/05-claude-code-ecosystem.en.md#55--subagents-claude-codes-native-multi-agent-mechanism--2025-new-feature). + +--- + +## First, Check Which Subagents You Have Available + +Run `/agents` inside your Claude Code session. It lists every subagent currently available to you: built-in, plugin-provided, and custom. + +**The 7 built-in Claude Code subagents**: `general-purpose` / `code-reviewer` / `Explore` / `Plan` / `frontend-developer` / `claude-code-guide` / `statusline-setup` (as of 2025-11; this may change). + +> 📌 **Each built-in subagent's purpose + the "task X → use subagent Y" decision table live canonically at [Stage 5.5 §Which subagents can you dispatch?](../stages/05-claude-code-ecosystem.en.md#which-subagents-can-you-dispatch)** — this cookbook focuses on the 15 dispatch recipes; treat Stage 5.5 as the source of truth for the built-in list and selection logic. + +> 💡 **If your `/agents` list does not match this cookbook**: you may have installed plugins, or your Claude Code version may differ. **When a recipe name does not match, use the closest available subagent**. For example, if you do not have `Explore`, `general-purpose` can still run a search task. + +After confirming what is available, use the 15 recipes below: **find the recipe that matches your scenario, then copy the Prompt Template into the Claude Code conversation**. + +--- + +## The 15 Recipes + +### Recipe 1: Review a batch of new code before committing + +**Scenario**: You just wrote ≥ 50 lines of new code and want to commit, but you are worried you missed a bug or security issue. + +**Subagent**: `code-reviewer` + +**Prompt Template**: +``` +Review the staged changes in this repo (git diff --cached). +Focus on: (1) security issues (hardcoded secrets, SQL injection, XSS), +(2) error handling gaps, (3) missing tests, (4) violations of CLAUDE.md +conventions. Per-category PASS/FAIL + concrete fix for each issue with +file:line reference + overall verdict (APPROVE / REQUEST CHANGES). +``` + +**When Not to Use It**: A < 20-line typo or formatting fix. Just scan `git diff` yourself; do not spend subagent tokens. + +--- + +### Recipe 2: Enter a new repo and find where to start reading + +**Scenario**: You cloned someone else’s repo, `README.md` does not clearly explain the entry point, and you do not want to dig randomly. + +**Subagent**: `Explore` + +**Prompt Template**: +``` +Map the entry points and core structure of this codebase. Report: +(1) main entry script(s) — what file gets run first, +(2) core module organization — top 3-5 directories and what each does, +(3) test directory layout, +(4) any "where to start reading" guidance in docs/. +Under 300 words with file paths. +``` + +**When Not to Use It**: You already know the exact file path to read. Use the Read tool directly. + +--- + +### Recipe 3: Design a refactor / migration plan before touching code + +**Scenario**: You need to refactor a large module or do a framework migration, but the steps are unclear and you want a plan first. + +**Subagent**: `Plan` + +**Prompt Template**: +``` +Design a step-by-step plan to refactor (currently a single +file with tight coupling) into clear, testable components with explicit +interfaces. Include: (1) phased breakdown (≤ 5 phases), (2) which files +touched per phase, (3) what tests gate each phase, (4) rollback strategy +if a phase fails. Don't write code — just the plan. +``` + +**When Not to Use It**: The refactor only touches 1-2 files. Start directly; the planning overhead is not worth it. + +--- + +### Recipe 4: Review multi-file / cross-locale parity + +**Scenario**: You changed zh-TW + zh-Hans + en mirror files and want to confirm the three versions still match. + +**Subagent**: `code-reviewer` + +**Prompt Template**: +``` +Review the staged diff for cross-locale parity across the 3 locale +variants of (.md / .zh-Hans.md / .en.md). Check: (1) same +section structure (## headers match), (2) same table row counts, (3) +same required terms present in each locale, (4) locale conventions +correct (zh-Hans uses "" not 「」; en uses English). Report per-file +PASS/FAIL. +``` + +**When Not to Use It**: You only changed one locale. There is no parity issue to check. + +--- + +### Recipe 5: Multi-source fact-check / research + +**Scenario**: You need to write a cited statement, but you are not sure whether multiple sources are talking about the same thing and need cross-checking. + +**Subagent**: `general-purpose` + +**Prompt Template**: +``` +Fact-check this claim: "". Search for: (1) primary source / official +docs / paper, (2) 2-3 independent secondary sources, (3) any contradictions +or version differences. Report: confirmed / contradicted / nuanced, with +direct quotes and URLs. Under 400 words. +``` + +**When Not to Use It**: You already have one authoritative source. Read that source directly. + +--- + +### Recipe 6: Find where a symbol / function is defined + +**Scenario**: A codebase imports `parse_config` everywhere, and you want to know which file implements it. + +**Subagent**: `Explore` + +**Prompt Template**: +``` +Find where `` is defined in this codebase. Report: (1) the +file:line where it's defined, (2) what it does (1-line summary), (3) which +files import / use it (top 5). Use Grep, not full file reads. +``` + +**When Not to Use It**: Your IDE’s “Go to definition” already jumps there. The IDE is faster than a subagent. + +--- + +### Recipe 7: Compare whether several papers / sources agree on a claim + +**Scenario**: You are writing a literature review. Three papers disagree about the same thing, and you do not know which one to trust. + +**Subagent**: `general-purpose` + +**Prompt Template**: +``` +Find and compare 3 independent sources on the topic . For each +source, capture: title + author + year + URL. Then report: (1) what +they agree on, (2) where they disagree (with direct quotes), (3) which +is most recent / authoritative, (4) suggested phrasing if you had to +summarize the consensus. Flag if a key source is behind a paywall and +unreadable. +``` + +**When Not to Use It**: All 3 papers are by the same author or lab. There is no real multi-perspective comparison. + +--- + +### Recipe 8: Release commit security audit + +**Scenario**: You are about to ship v1.0 or a major version and want one final security pass. + +**Subagent**: `code-reviewer` + +**Prompt Template**: +``` +Pre-release security audit for (use git log to find the +relevant commit range since last release). Check: (1) hardcoded +credentials / API keys, (2) eval() / exec() / shell injection risks, +(3) deprecated dependencies with known CVEs, (4) any auth / session +handling changes since last release, (5) public API surface changes +(breaking? documented?). Per-category PASS/FAIL + remediation per +finding. +``` + +**When Not to Use It**: A patch release or hotfix that changes one line. Manual review takes < 1 minute. + +--- + +### Recipe 9: Assess the blast radius of an architecture change + +**Scenario**: You want to change a base class or shared utility, but you do not know how many files it will affect. + +**Subagent**: `Plan` + +**Prompt Template**: +``` +Assess the blast radius of changing . Report: (1) direct +dependents (which files import this), (2) indirect dependents (transitive +imports), (3) tests that gate the change, (4) suggested rollout order +(safest → riskiest), (5) feature flag / kill switch strategy if available. +Don't make the change yet — just the impact analysis. +``` + +**When Not to Use It**: The change only affects internals of one file. There is no blast-radius problem. + +--- + +### Recipe 10: Spawn parallel subagents for the same task across multiple targets + +**Scenario**: Four branch files all need the same “academic-style audit,” and you want to run all four in parallel. + +**Subagent**: `general-purpose` × N (spawn several at the same time) + +> 💡 **How to spawn several at the same time**: in the Claude Code conversation, **enter the prompt below 4 times in a row** (changing `` each time). Claude Code will run them in parallel automatically. You do not need to wait for the first subagent to finish before entering the second prompt. That is the practical workflow for “parallel spawn.” + +**Prompt Template** (change `` each time): +``` +Audit `` for academic-style issues: (1) over-engineering +jargon without first-use explanation, (2) clarity (long sentences, +vague pronouns), (3) unsupported % claims, (4) persona-fit (wrong +technical level for the file's stated target audience — see the +file's banner / intro callout for who it's for). Report 4-category +PASS/FAIL + fix-list with line numbers. Under 500 words. +``` + +**When Not to Use It**: There is only one target. Invoke one subagent directly; do not over-orchestrate. + +--- + +### Recipe 11: Find similar implementations across repos + +**Scenario**: You saw a pattern in one repo and want to know whether another repo has a similar implementation. + +**Subagent**: `Explore` + +**Prompt Template**: +``` +In , search for code patterns similar to: . Report: +(1) up to 5 candidates with file:line, (2) brief description of each, +(3) which is most idiomatic for this codebase's style. +``` + +**When Not to Use It**: You have a specific function name to find. Use Recipe 6 / `Explore` with a precise grep; it will be faster. + +--- + +### Recipe 12: LLM-as-judge eval (structured PASS / FAIL) + +**Scenario**: You ran 100 test cases and need to judge whether each output meets the spec, without reading them all manually. + +**Subagent**: `general-purpose` + +**Prompt Template**: +``` +Evaluate the agent outputs in against the spec stated +in the file's first 5 lines (or in ). For each row: PASS / +FAIL with 1-sentence reason. Aggregate: pass rate + top-3 failure modes. +Output as structured JSON: {"case_id": "...", "verdict": "PASS|FAIL", +"reason": "..."}. +``` + +**When Not to Use It**: There are < 5 cases; read them yourself. Or the evaluation requires subjective judgment; an LLM judge is not reliable enough. + +--- + +### Recipe 13: UI component design / accessibility audit + +**Scenario**: You wrote a React component and want to confirm ARIA, keyboard navigation, and responsive behavior are all correct. + +**Subagent**: `frontend-developer` + +**Prompt Template**: +``` +Audit for: (1) ARIA roles + labels (screen reader +compatibility), (2) keyboard navigation (tab order, Enter / Esc / arrows +behavior), (3) responsive breakpoints (mobile 360px / tablet 768px / +desktop 1280px), (4) color contrast (WCAG AA), (5) touch target size +(≥ 44px). Report per-category findings + fixes. +``` + +**When Not to Use It**: Pure backend or CLI tooling. There is no UI to audit. + +--- + +### Recipe 14: Ask how to use a Claude Code feature + +**Scenario**: You forgot how to configure hooks, forgot the frontmatter fields for slash commands, or want to look up documentation. + +**Subagent**: `claude-code-guide` + +**Prompt Template**: +``` +How do I ? Show: (1) minimum +config in settings.json, (2) example hook script (Python or shell), +(3) 1 common gotcha, (4) where in the official docs to read more. +``` + +**When Not to Use It**: You have already written this a few times. Looking at your own `~/.claude/settings.json` example is faster. + +--- + +### Recipe 15: Implement React form validation logic + +**Scenario**: You need to build a sign-up form with email format validation, password strength, and real-time validation. + +**Subagent**: `frontend-developer` + +**Prompt Template**: +``` +Implement a React sign-up form with: (1) email format validation +(real-time, debounce 300ms), (2) password strength meter (≥ 8 chars, +mixed case, digit, symbol), (3) inline error messages with ARIA +live region, (4) submit button disabled until valid. Use . Include: component code, validation schema, +1 happy-path test, 1 error-path test. +``` + +**When Not to Use It**: You are not using a React stack. Use the matching Vue / Svelte subagent, or `general-purpose`. + +--- + +## Recipe Index (by Subagent Type) + +Not sure which subagent to use? Look it up by **task type**: + +| Subagent | Recipes | +|---|---| +| `code-reviewer` | **1** (pre-commit review) / **4** (cross-locale parity) / **8** (release security audit)| +| `Explore` | **2** (new codebase) / **6** (find symbol) / **11** (cross-repo similar code)| +| `Plan` | **3** (refactor plan) / **9** (blast radius)| +| `general-purpose` | **5** (fact-check) / **7** (multi-paper compare) / **10** (parallel multi-target) / **12** (LLM-as-judge eval)| +| `frontend-developer` | **13** (a11y audit) / **15** (React form)| +| `claude-code-guide` | **14** (Claude Code feature lookup)| + +--- + +## When NOT to Use a Subagent + +Subagents are not free. Every dispatch **spends tokens and adds latency**. In the 4 situations below, **do not use a subagent**; doing it yourself is more efficient: + +1. **The task takes < 5 minutes to do yourself** — This is overkill; the subagent overhead is not worth it. +2. **The result needs step-by-step user feedback** — A subagent is “dispatch it, let it run, get one final report.” It cannot ask you midway. Tasks that need iterative confirmation should stay in the main session. +3. **The task needs the main session’s context memory** — A subagent has an **independent context window** and cannot see earlier discussion in the main session. Tasks that depend on references like “the X we discussed earlier” are a bad fit. +4. **The task involves conversation-level judgment** — For example, “how should we weigh this architecture decision?” needs collaborative discussion and should not be thrown to a subagent. + +> 💡 **A quick way to decide**: if you can describe the task as “**write a complete brief for a stranger to pick up**,” it is a good fit for a subagent. If it needs “**let’s talk this through**,” keep it in the main session. + +--- + +## Next Steps + +- **Understand the full theory** (how subagents differ from skills / MCP, and the 3 multi-agent mechanisms) → [Stage 5.5](../stages/05-claude-code-ecosystem.en.md#55--subagents-claude-codes-native-multi-agent-mechanism--2025-new-feature) +- **CLI daily-use playbook** → [`tracks/cli/A3-cli-production.md` Playbook 4](../tracks/cli/A3-cli-production.en.md#-playbook-4-dispatching-subagents-for-independent-tasks) +- **See where subagents fit in the agent paradigm map** → [`resources/agent-paradigms.md`](./agent-paradigms.en.md#subagent--spawning-an-agent-inside-an-agent-runtime) +- **Glossary lookup** → [`resources/glossary.md` § 5. Claude Code Ecosystem — Subagent](./glossary.en.md#subagent) diff --git a/resources/subagent-cookbook.md b/resources/subagent-cookbook.md new file mode 100644 index 0000000..192b204 --- /dev/null +++ b/resources/subagent-cookbook.md @@ -0,0 +1,367 @@ +# Subagent Cookbook — 15 個複製貼上就能用的派遣 recipe + +> **繁體中文** | [简体中文](./subagent-cookbook.zh-Hans.md) | [English](./subagent-cookbook.en.md) + +> 📋 **這是什麼**：[Stage 5.5](../stages/05-claude-code-ecosystem.md#55--subagentsclaude-code-原生-multi-agent-機制-2025-新功能) 教你 subagent 是什麼概念、這份 cookbook 教你**今天就能用**——15 個情境、每個含「該用哪個 subagent + 完整 prompt 範本（複製即可用）+ 何時不用」。 +> +> ⚠️ **第一次看？先讀 Stage 5.5 的「可派遣的 subagent 有哪些」+「decision table」兩節**——理解「什麼是 subagent」「Claude Code 內建有哪些」之後再來查 recipe。 + +--- + +## 怎麼讀這份 cookbook + +每個 recipe 用同樣的 4 段結構： + +| 段 | 內容 | 為什麼有這段 | +|---|---|---| +| **情境** | 你今天工作會遇到的具體場景 | 從「我有 X 問題」找到 recipe、不從「我想用 subagent」找 | +| **Subagent** | 用哪一個（Claude Code 內建名稱） | 直接 copy 名字、不用想 | +| **Prompt 範本** | 複製貼上即可用的指令文字 | 不用自己想怎麼寫 | +| **何時不用** | 比用 subagent 更好的替代方案 | 避免「殺雞用牛刀」 | + +> 💡 **怎麼實際派遣 subagent**：在你的 Claude Code 終端機對話框裡、**直接輸入（或貼上）prompt 範本**——就這樣。Claude 看到指令、會自動透過 Task tool（內部派遣機制）找到對應 subagent 跑、跑完回主 session 一段摘要。**不需要 slash command、不需要特殊語法**。 +> +> 📌 **subagent ≠ slash command**：`/agents` 是查當前可用 subagent 的指令、**不是用來「呼叫」subagent**。派遣 subagent 直接打對話 prompt 文字即可。完整對比表（subagent vs skill / vs slash command / description router）見 [Stage 5.5 §易混淆觀念釐清](../stages/05-claude-code-ecosystem.md#55--subagentsclaude-code-原生-multi-agent-機制-2025-新功能)。 + +--- + +## 先確認你有哪些 subagent 可用 + +在你的 Claude Code session 內跑 `/agents` 一個指令、會列出全部當前可用的 subagent（內建 + plugin + 自訂）。 + +**Claude Code 預設內建 7 個** subagent：`general-purpose` / `code-reviewer` / `Explore` / `Plan` / `frontend-developer` / `claude-code-guide` / `statusline-setup`（截至 2025-11、可能會變動）。 + +> 📌 **每個內建 subagent 的功能說明 + 「遇到 X 任務用 Y」decision table 的 canonical 在 [Stage 5.5 §可派遣的 subagent 有哪些](../stages/05-claude-code-ecosystem.md#可派遣的-subagent-有哪些)**——本 cookbook 聚焦「怎麼派遣」的 15 個 recipe，內建清單與選用邏輯以 Stage 5.5 為準。 + +> 💡 **如果 `/agents` 列表跟這份 cookbook 不一致**：表示你裝了 plugin 或 Claude Code 版本不同。**recipe 名字對不上時、找最接近的 subagent 用就好**（例如沒有 `Explore`、用 `general-purpose` 也能跑搜尋）。 + +確認完當前可用清單之後、就可以照下面 15 個 recipe 派遣了——**找到符合你情境的 recipe、複製 Prompt 範本貼進 Claude Code 對話框即可**。 + +--- + +## 15 個 Recipe + +### Recipe 1: 寫完一批新 code、要 commit 前的 review + +**情境**：你剛寫了 ≥ 50 行新 code、想 commit 但怕有 bug / security 問題沒看到 + +**Subagent**：`code-reviewer` + +**Prompt 範本**： +``` +Review the staged changes in this repo (git diff --cached). +Focus on: (1) security issues (hardcoded secrets, SQL injection, XSS), +(2) error handling gaps, (3) missing tests, (4) violations of CLAUDE.md +conventions. Per-category PASS/FAIL + concrete fix for each issue with +file:line reference + overall verdict (APPROVE / REQUEST CHANGES). +``` + +**何時不用**：< 20 行的 typo / formatting fix（直接 `git diff` 自己掃就好、不用花 subagent token） + +--- + +### Recipe 2: 進新 repo、不知該從哪個 file 開始讀 + +**情境**：複製完一個別人的 repo、`README.md` 講不清楚程式入口在哪、不想隨便挖 + +**Subagent**：`Explore` + +**Prompt 範本**： +``` +Map the entry points and core structure of this codebase. Report: +(1) main entry script(s) — what file gets run first, +(2) core module organization — top 3-5 directories and what each does, +(3) test directory layout, +(4) any "where to start reading" guidance in docs/. +Under 300 words with file paths. +``` + +**何時不用**：你已經知道要看的 file 路徑（直接用 Read tool） + +--- + +### Recipe 3: 設計 refactor / migration plan（先想清楚再動手） + +**情境**：要重構一個大 module 或做 framework migration、不確定步驟、想先有 plan 再開始 + +**Subagent**：`Plan` + +**Prompt 範本**： +``` +Design a step-by-step plan to refactor (currently a single +file with tight coupling) into clear, testable components with explicit +interfaces. Include: (1) phased breakdown (≤ 5 phases), (2) which files +touched per phase, (3) what tests gate each phase, (4) rollback strategy +if a phase fails. Don't write code — just the plan. +``` + +**何時不用**：refactor 只動 1-2 個 file（直接動手、不需要 plan overhead） + +--- + +### Recipe 4: 多檔 / 跨 locale parity 審查 + +**情境**：你改了 zh-TW + zh-Hans + en 三個 mirror、想確認三邊內容一致 + +**Subagent**：`code-reviewer` + +**Prompt 範本**： +``` +Review the staged diff for cross-locale parity across the 3 locale +variants of (.md / .zh-Hans.md / .en.md). Check: (1) same +section structure (## headers match), (2) same table row counts, (3) +same required terms present in each locale, (4) locale conventions +correct (zh-Hans uses "" not 「」; en uses English). Report per-file +PASS/FAIL. +``` + +**何時不用**：只改 1 個 locale（沒 parity 問題） + +--- + +### Recipe 5: 多 source fact-check / research + +**情境**：要寫一段引用、不確定多個 source 講的是不是同一件事、需要交叉驗證 + +**Subagent**：`general-purpose` + +**Prompt 範本**： +``` +Fact-check this claim: "". Search for: (1) primary source / official +docs / paper, (2) 2-3 independent secondary sources, (3) any contradictions +or version differences. Report: confirmed / contradicted / nuanced, with +direct quotes and URLs. Under 400 words. +``` + +**何時不用**：你已經有 1 個權威 source（直接讀那一個就好） + +--- + +### Recipe 6: 找某個 symbol / function 在哪定義 + +**情境**：codebase 內到處在導入 `parse_config`、想知道實作在哪個 file + +**Subagent**：`Explore` + +**Prompt 範本**： +``` +Find where `` is defined in this codebase. Report: (1) the +file:line where it's defined, (2) what it does (1-line summary), (3) which +files import / use it (top 5). Use Grep, not full file reads. +``` + +**何時不用**：你已經用 IDE 的 "Go to definition" 可以跳過去（IDE 比 subagent 快） + +--- + +### Recipe 7: 比較多 paper / source 的 claim 是否一致 + +**情境**：寫文獻回顧、3 篇 paper 對同一件事說法不同、不知該信誰 + +**Subagent**：`general-purpose` + +**Prompt 範本**： +``` +Find and compare 3 independent sources on the topic . For each +source, capture: title + author + year + URL. Then report: (1) what +they agree on, (2) where they disagree (with direct quotes), (3) which +is most recent / authoritative, (4) suggested phrasing if you had to +summarize the consensus. Flag if a key source is behind a paywall and +unreadable. +``` + +**何時不用**：3 篇都是同一個作者 / 同 lab（沒「多 perspective」可比） + +--- + +### Recipe 8: Release commit security audit + +**情境**：要發 v1.0 / 升 major version、想最後一次安全掃描 + +**Subagent**：`code-reviewer` + +**Prompt 範本**： +``` +Pre-release security audit for (use git log to find the +relevant commit range since last release). Check: (1) hardcoded +credentials / API keys, (2) eval() / exec() / shell injection risks, +(3) deprecated dependencies with known CVEs, (4) any auth / session +handling changes since last release, (5) public API surface changes +(breaking? documented?). Per-category PASS/FAIL + remediation per +finding. +``` + +**何時不用**：patch release（hotfix）只動 1 行（人工掃 < 1 分鐘） + +--- + +### Recipe 9: 評估架構變動的 blast radius + +**情境**：想改一個 base class / 共用 utility、不確定會影響多少 file + +**Subagent**：`Plan` + +**Prompt 範本**： +``` +Assess the blast radius of changing . Report: (1) direct +dependents (which files import this), (2) indirect dependents (transitive +imports), (3) tests that gate the change, (4) suggested rollout order +(safest → riskiest), (5) feature flag / kill switch strategy if available. +Don't make the change yet — just the impact analysis. +``` + +**何時不用**：只影響 1 個 file 內部（沒 blast radius 概念） + +--- + +### Recipe 10: Spawn 並行 subagent 跑同一任務 × 多目標 + +**情境**：4 個 branches file 都要做同樣的「academic-style audit」、想一次跑 4 個 parallel + +**Subagent**：`general-purpose` × N（同時 spawn 多個） + +> 💡 **怎麼「同時 spawn 多個」**：在 Claude Code 對話框內、**連續輸入下面的 prompt 4 次**（每次換 ``）。Claude Code 會自動並行跑、不需要等第一個 subagent 結束才能輸入第二個——這就是「parallel spawn」的實際操作方式。 + +**Prompt 範本**（每次換 ``）： +``` +Audit `` for academic-style issues: (1) over-engineering +jargon without first-use explanation, (2) clarity (long sentences, +vague pronouns), (3) unsupported % claims, (4) persona-fit (wrong +technical level for the file's stated target audience — see the +file's banner / intro callout for who it's for). Report 4-category +PASS/FAIL + fix-list with line numbers. Under 500 words. +``` + +**何時不用**：1 個目標就夠（直接呼叫 1 次、不要 over-orchestrate） + +--- + +### Recipe 11: 找跨 repo 的相似 implementation + +**情境**：你在某個 repo 看到一個 pattern、想知道另一個 repo 有沒有類似實作 + +**Subagent**：`Explore` + +**Prompt 範本**： +``` +In , search for code patterns similar to: . Report: +(1) up to 5 candidates with file:line, (2) brief description of each, +(3) which is most idiomatic for this codebase's style. +``` + +**何時不用**：你有具體 function 名要找（用 Recipe 6 / `Explore` 加精確 grep 更快） + +--- + +### Recipe 12: LLM-as-judge eval（structured PASS / FAIL） + +**情境**：跑了 100 個 test case、要評每個 output 是否符合 spec、不想人工看 + +**Subagent**：`general-purpose` + +**Prompt 範本**： +``` +Evaluate the agent outputs in against the spec stated +in the file's first 5 lines (or in ). For each row: PASS / +FAIL with 1-sentence reason. Aggregate: pass rate + top-3 failure modes. +Output as structured JSON: {"case_id": "...", "verdict": "PASS|FAIL", +"reason": "..."}. +``` + +**何時不用**：< 5 個 case（自己看更快）；evaluation 涉及主觀判斷（LLM judge 不可靠） + +--- + +### Recipe 13: UI component design / accessibility audit + +**情境**：寫了一個 React component、想確認 ARIA + keyboard nav + responsive 都做對 + +**Subagent**：`frontend-developer` + +**Prompt 範本**： +``` +Audit for: (1) ARIA roles + labels (screen reader +compatibility), (2) keyboard navigation (tab order, Enter / Esc / arrows +behavior), (3) responsive breakpoints (mobile 360px / tablet 768px / +desktop 1280px), (4) color contrast (WCAG AA), (5) touch target size +(≥ 44px). Report per-category findings + fixes. +``` + +**何時不用**：純後端 / CLI 工具（沒 UI 可 audit） + +--- + +### Recipe 14: 問 Claude Code feature 怎麼用 + +**情境**：忘了 hooks 怎麼設、忘了 slash command 的 frontmatter 欄位、想查文件 + +**Subagent**：`claude-code-guide` + +**Prompt 範本**： +``` +How do I ? Show: (1) minimum +config in settings.json, (2) example hook script (Python or shell), +(3) 1 common gotcha, (4) where in the official docs to read more. +``` + +**何時不用**：你已經寫過幾次（直接看自己 `~/.claude/settings.json` 範例更快） + +--- + +### Recipe 15: 寫 React form validation 邏輯 + +**情境**：要做一個 sign-up form、要 email format / password strength / real-time validation + +**Subagent**：`frontend-developer` + +**Prompt 範本**： +``` +Implement a React sign-up form with: (1) email format validation +(real-time, debounce 300ms), (2) password strength meter (≥ 8 chars, +mixed case, digit, symbol), (3) inline error messages with ARIA +live region, (4) submit button disabled until valid. Use . Include: component code, validation schema, +1 happy-path test, 1 error-path test. +``` + +**何時不用**：非 React stack（用 Vue / Svelte 對應的 subagent、或 `general-purpose`） + +--- + +## Recipe 索引（按 subagent type 找） + +不確定該用哪個 subagent？從**任務類型**反查： + +| Subagent | Recipes | +|---|---| +| `code-reviewer` | **1**（pre-commit review）/ **4**（cross-locale parity）/ **8**（release security audit）| +| `Explore` | **2**（new codebase）/ **6**（find symbol）/ **11**（cross-repo similar code）| +| `Plan` | **3**（refactor plan）/ **9**（blast radius）| +| `general-purpose` | **5**（fact-check）/ **7**（multi-paper compare）/ **10**（parallel multi-target）/ **12**（LLM-as-judge eval）| +| `frontend-developer` | **13**（a11y audit）/ **15**（React form）| +| `claude-code-guide` | **14**（Claude Code feature 查詢）| + +--- + +## 何時 NOT 該用 subagent + +Subagent 不是免費的——每次派遣**燒 token、有延遲**。下面 4 種情境**不該用 subagent**、自己跑更划算： + +1. **任務 < 5 分鐘可自己完成** — 殺雞用牛刀；subagent overhead 不划算 +2. **結果需要逐步 user feedback** — subagent 是「派出去、跑完回報一次」、不能中間問你；要逐步確認的任務直接在主 session 跑 +3. **任務需要主 session 的 context memory** — subagent 是**獨立 context window**、看不到主 session 前面的對話；要用「我們剛剛討論的 X...」這種 reference 的任務、不適合 +4. **任務涉及 conversation-level judgment** — 像「這個 architecture 決策該怎麼權衡」、需要對話協作、不適合丟給 subagent + +> 💡 **判斷的快速辦法**：如果任務可以用「**寫一份完整 brief 給陌生人接手做**」描述清楚 → 適合 subagent；如果需要「**我跟你討論一下**」→ 留在主 session。 + +--- + +## 接下來 + +- **想理解完整理論**（subagent 跟 skill / MCP 的差別、3 種 multi-agent 機制）→ [Stage 5.5](../stages/05-claude-code-ecosystem.md#55--subagentsclaude-code-原生-multi-agent-機制-2025-新功能) +- **想自己寫 / 組合 / debug subagent**（進階主題）→ [`subagent-advanced.md`](./subagent-advanced.md)（description 寫法 / composition pattern / debug 工具） +- **CLI 日常用法 playbook** → [`tracks/cli/A3-cli-production.md` Playbook 4](../tracks/cli/A3-cli-production.md#📋-playbook-4派遣-subagent-跑獨立任務) +- **想看 subagent 在 agent paradigm 體系內的定位** → [`resources/agent-paradigms.md`](./agent-paradigms.md#subagent--在-agent-runtime-裡再-spawn-agent) +- **詞彙快查** → [`resources/glossary.md` § 5. Claude Code 生態 — Subagent](./glossary.md#subagent子-agent) diff --git a/resources/subagent-cookbook.zh-Hans.md b/resources/subagent-cookbook.zh-Hans.md new file mode 100644 index 0000000..14ad3b9 --- /dev/null +++ b/resources/subagent-cookbook.zh-Hans.md @@ -0,0 +1,366 @@ +# Subagent Cookbook — 15 个复制粘贴就能用的派遣 recipe + +> [繁體中文](./subagent-cookbook.md) | **简体中文** | [English](./subagent-cookbook.en.md) + +> 📋 **这是什么**：[Stage 5.5](../stages/05-claude-code-ecosystem.zh-Hans.md#55--subagentsclaude-code-原生-multi-agent-机制-2025-新功能) 教你 subagent 是什么概念，这份 cookbook 教你**今天就能用**——15 个场景，每个都包含“该用哪个 subagent + 完整 prompt 模板（复制即可用）+ 何时不用”。 +> +> ⚠️ **第一次看？先读 Stage 5.5 的“可派遣的 subagent 有哪些”+“decision table”两节**——理解“什么是 subagent”“Claude Code 内置有哪些”之后再来查 recipe。 + +--- + +## 怎么读这份 cookbook + +每个 recipe 都用同样的 4 段结构： + +| 段 | 内容 | 为什么有这段 | +|---|---|---| +| **场景** | 你今天工作会遇到的具体场景 | 从“我有 X 问题”找到 recipe，而不是从“我想用 subagent”找 | +| **Subagent** | 用哪一个（Claude Code 内置名称） | 直接 copy 名字，不用想 | +| **Prompt 模板** | 复制粘贴即可用的指令文字 | 不用自己想怎么写 | +| **何时不用** | 比用 subagent 更好的替代方案 | 避免“大材小用” | + +> 💡 **怎么实际派遣 subagent**：在你的 Claude Code 终端对话框里，**直接输入（或粘贴）prompt 模板**——就这样。Claude 看到指令，会自动通过 Task tool（内部派遣机制）找到对应 subagent 跑，跑完后向主 session 回报一段摘要。**不需要 slash command，不需要特殊语法**。 +> +> 📌 **subagent ≠ slash command**：`/agents` 是列表命令，**不是调用 subagent 的方式**；派遣 subagent 直接打对话 prompt 文字即可。完整对比表（subagent vs skill / vs slash command / description router）见 [Stage 5.5 §易混淆观念厘清](../stages/05-claude-code-ecosystem.zh-Hans.md#55--subagentsclaude-code-原生-multi-agent-机制-2025-新功能)。 + +--- + +## 先确认你有哪些 subagent 可用 + +在你的 Claude Code session 里跑 `/agents` 这个指令，会列出全部当前可用的 subagent（内置 + plugin + 自定义）。 + +**Claude Code 默认内置 7 个** subagent：`general-purpose` / `code-reviewer` / `Explore` / `Plan` / `frontend-developer` / `claude-code-guide` / `statusline-setup`（截至 2025-11，可能会变动）。 + +> 📌 **每个内置 subagent 的功能说明 + “遇到 X 任务用 Y”decision table 的 canonical 在 [Stage 5.5 §可派遣的 subagent 有哪些](../stages/05-claude-code-ecosystem.zh-Hans.md#可派遣的-subagent-有哪些)**——本 cookbook 聚焦“怎么派遣”的 15 个 recipe，内置清单与选用逻辑以 Stage 5.5 为准。 + +> 💡 **如果 `/agents` 列表跟这份 cookbook 不一致**：表示你装了 plugin，或 Claude Code 版本不同。**recipe 名字对不上时，找最接近的 subagent 用就好**（例如没有 `Explore`，用 `general-purpose` 也能跑搜索）。 + +确认完当前可用清单之后，就可以照下面 15 个 recipe 派遣了——**找到符合你场景的 recipe，把 Prompt 模板复制到 Claude Code 对话框即可**。 + +--- + +## 15 个 Recipe + +### Recipe 1: 写完一批新 code，要 commit 前的 review + +**场景**：你刚写了 ≥ 50 行新 code，想 commit 但怕有 bug / security 问题没看到 + +**Subagent**：`code-reviewer` + +**Prompt 模板**： +``` +Review the staged changes in this repo (git diff --cached). +Focus on: (1) security issues (hardcoded secrets, SQL injection, XSS), +(2) error handling gaps, (3) missing tests, (4) violations of CLAUDE.md +conventions. Per-category PASS/FAIL + concrete fix for each issue with +file:line reference + overall verdict (APPROVE / REQUEST CHANGES). +``` + +**何时不用**：< 20 行的 typo / formatting fix（直接 `git diff` 自己扫就好，不用花 subagent token） + +--- + +### Recipe 2: 进入新 repo，不知道该从哪个 file 开始读 + +**场景**：复制完一个别人的 repo，`README.md` 讲不清楚程序入口在哪，不想随便挖 + +**Subagent**：`Explore` + +**Prompt 模板**： +``` +Map the entry points and core structure of this codebase. Report: +(1) main entry script(s) — what file gets run first, +(2) core module organization — top 3-5 directories and what each does, +(3) test directory layout, +(4) any "where to start reading" guidance in docs/. +Under 300 words with file paths. +``` + +**何时不用**：你已经知道要看的 file 路径（直接用 Read tool） + +--- + +### Recipe 3: 设计 refactor / migration plan（先想清楚再动手） + +**场景**：要重构一个大 module 或做 framework migration，不确定步骤，想先有 plan 再开始 + +**Subagent**：`Plan` + +**Prompt 模板**： +``` +Design a step-by-step plan to refactor (currently a single +file with tight coupling) into clear, testable components with explicit +interfaces. Include: (1) phased breakdown (≤ 5 phases), (2) which files +touched per phase, (3) what tests gate each phase, (4) rollback strategy +if a phase fails. Don't write code — just the plan. +``` + +**何时不用**：refactor 只动 1-2 个 file（直接动手，不需要 plan overhead） + +--- + +### Recipe 4: 多文件 / 跨 locale parity 审查 + +**场景**：你改了 zh-TW + zh-Hans + en 三个 mirror，想确认三边内容一致 + +**Subagent**：`code-reviewer` + +**Prompt 模板**： +``` +Review the staged diff for cross-locale parity across the 3 locale +variants of (.md / .zh-Hans.md / .en.md). Check: (1) same +section structure (## headers match), (2) same table row counts, (3) +same required terms present in each locale, (4) locale conventions +correct (zh-Hans uses "" not 「」; en uses English). Report per-file +PASS/FAIL. +``` + +**何时不用**：只改 1 个 locale（没有 parity 问题） + +--- + +### Recipe 5: 多 source fact-check / research + +**场景**：要写一段引用，不确定多个 source 讲的是不是同一件事，需要交叉验证 + +**Subagent**：`general-purpose` + +**Prompt 模板**： +``` +Fact-check this claim: "". Search for: (1) primary source / official +docs / paper, (2) 2-3 independent secondary sources, (3) any contradictions +or version differences. Report: confirmed / contradicted / nuanced, with +direct quotes and URLs. Under 400 words. +``` + +**何时不用**：你已经有 1 个权威 source（直接读那一个就好） + +--- + +### Recipe 6: 找某个 symbol / function 在哪定义 + +**场景**：codebase 里到处都在导入 `parse_config`，想知道实作在哪个 file + +**Subagent**：`Explore` + +**Prompt 模板**： +``` +Find where `` is defined in this codebase. Report: (1) the +file:line where it's defined, (2) what it does (1-line summary), (3) which +files import / use it (top 5). Use Grep, not full file reads. +``` + +**何时不用**：你已经用 IDE 的 "Go to definition" 可以跳过去（IDE 比 subagent 快） + +--- + +### Recipe 7: 比较多篇 paper / source 的 claim 是否一致 + +**场景**：写文献回顾，3 篇 paper 对同一件事说法不同，不知道该信谁 + +**Subagent**：`general-purpose` + +**Prompt 模板**： +``` +Find and compare 3 independent sources on the topic . For each +source, capture: title + author + year + URL. Then report: (1) what +they agree on, (2) where they disagree (with direct quotes), (3) which +is most recent / authoritative, (4) suggested phrasing if you had to +summarize the consensus. Flag if a key source is behind a paywall and +unreadable. +``` + +**何时不用**：3 篇都是同一个作者 / 同 lab（没有“多 perspective”可比） + +--- + +### Recipe 8: Release commit security audit + +**场景**：要发 v1.0 / 升 major version，想最后做一次安全扫描 + +**Subagent**：`code-reviewer` + +**Prompt 模板**： +``` +Pre-release security audit for (use git log to find the +relevant commit range since last release). Check: (1) hardcoded +credentials / API keys, (2) eval() / exec() / shell injection risks, +(3) deprecated dependencies with known CVEs, (4) any auth / session +handling changes since last release, (5) public API surface changes +(breaking? documented?). Per-category PASS/FAIL + remediation per +finding. +``` + +**何时不用**：patch release（hotfix）只动 1 行（人工扫 < 1 分钟） + +--- + +### Recipe 9: 评估架构变动的 blast radius + +**场景**：想改一个 base class / 共用 utility，不确定会影响多少 file + +**Subagent**：`Plan` + +**Prompt 模板**： +``` +Assess the blast radius of changing . Report: (1) direct +dependents (which files import this), (2) indirect dependents (transitive +imports), (3) tests that gate the change, (4) suggested rollout order +(safest → riskiest), (5) feature flag / kill switch strategy if available. +Don't make the change yet — just the impact analysis. +``` + +**何时不用**：只影响 1 个 file 内部（没有 blast radius 概念） + +--- + +### Recipe 10: Spawn 并行 subagent 跑同一任务 × 多目标 + +**场景**：4 个 branches file 都要做同样的“academic-style audit”，想一次跑 4 个 parallel + +**Subagent**：`general-purpose` × N（同时 spawn 多个） + +> 💡 **怎么“同时 spawn 多个”**：在 Claude Code 对话框内，**连续输入下面的 prompt 4 次**（每次换 ``）。Claude Code 会自动并行跑，不需要等第一个 subagent 结束才能输入第二个——这就是“parallel spawn”的实际操作方式。 + +**Prompt 模板**（每次换 ``）： +``` +Audit `` for academic-style issues: (1) over-engineering +jargon without first-use explanation, (2) clarity (long sentences, +vague pronouns), (3) unsupported % claims, (4) persona-fit (wrong +technical level for the file's stated target audience — see the +file's banner / intro callout for who it's for). Report 4-category +PASS/FAIL + fix-list with line numbers. Under 500 words. +``` + +**何时不用**：1 个目标就够（直接调用 1 次，不要 over-orchestrate） + +--- + +### Recipe 11: 找跨 repo 的相似 implementation + +**场景**：你在某个 repo 看到一个 pattern，想知道另一个 repo 有没有类似实作 + +**Subagent**：`Explore` + +**Prompt 模板**： +``` +In , search for code patterns similar to: . Report: +(1) up to 5 candidates with file:line, (2) brief description of each, +(3) which is most idiomatic for this codebase's style. +``` + +**何时不用**：你有具体 function 名要找（用 Recipe 6 / `Explore` 加精确 grep 更快） + +--- + +### Recipe 12: LLM-as-judge eval（structured PASS / FAIL） + +**场景**：跑了 100 个 test case，要评每个 output 是否符合 spec，不想人工看 + +**Subagent**：`general-purpose` + +**Prompt 模板**： +``` +Evaluate the agent outputs in against the spec stated +in the file's first 5 lines (or in ). For each row: PASS / +FAIL with 1-sentence reason. Aggregate: pass rate + top-3 failure modes. +Output as structured JSON: {"case_id": "...", "verdict": "PASS|FAIL", +"reason": "..."}. +``` + +**何时不用**：< 5 个 case（自己看更快）；evaluation 涉及主观判断（LLM judge 不可靠） + +--- + +### Recipe 13: UI component design / accessibility audit + +**场景**：写了一个 React component，想确认 ARIA + keyboard nav + responsive 都做对 + +**Subagent**：`frontend-developer` + +**Prompt 模板**： +``` +Audit for: (1) ARIA roles + labels (screen reader +compatibility), (2) keyboard navigation (tab order, Enter / Esc / arrows +behavior), (3) responsive breakpoints (mobile 360px / tablet 768px / +desktop 1280px), (4) color contrast (WCAG AA), (5) touch target size +(≥ 44px). Report per-category findings + fixes. +``` + +**何时不用**：纯后端 / CLI 工具（没有 UI 可 audit） + +--- + +### Recipe 14: 问 Claude Code feature 怎么用 + +**场景**：忘了 hooks 怎么设、忘了 slash command 的 frontmatter 字段，想查文件 + +**Subagent**：`claude-code-guide` + +**Prompt 模板**： +``` +How do I ? Show: (1) minimum +config in settings.json, (2) example hook script (Python or shell), +(3) 1 common gotcha, (4) where in the official docs to read more. +``` + +**何时不用**：你已经写过几次（直接看自己 `~/.claude/settings.json` 范例更快） + +--- + +### Recipe 15: 写 React form validation 逻辑 + +**场景**：要做一个 sign-up form，要 email format / password strength / real-time validation + +**Subagent**：`frontend-developer` + +**Prompt 模板**： +``` +Implement a React sign-up form with: (1) email format validation +(real-time, debounce 300ms), (2) password strength meter (≥ 8 chars, +mixed case, digit, symbol), (3) inline error messages with ARIA +live region, (4) submit button disabled until valid. Use . Include: component code, validation schema, +1 happy-path test, 1 error-path test. +``` + +**何时不用**：非 React stack（用 Vue / Svelte 对应的 subagent，或 `general-purpose`） + +--- + +## Recipe 索引（按 subagent type 找） + +不确定该用哪个 subagent？从**任务类型**反查： + +| Subagent | Recipes | +|---|---| +| `code-reviewer` | **1**（pre-commit review）/ **4**（cross-locale parity）/ **8**（release security audit）| +| `Explore` | **2**（new codebase）/ **6**（find symbol）/ **11**（cross-repo similar code）| +| `Plan` | **3**（refactor plan）/ **9**（blast radius）| +| `general-purpose` | **5**（fact-check）/ **7**（multi-paper compare）/ **10**（parallel multi-target）/ **12**（LLM-as-judge eval）| +| `frontend-developer` | **13**（a11y audit）/ **15**（React form）| +| `claude-code-guide` | **14**（Claude Code feature 查询）| + +--- + +## 何时 NOT 该用 subagent + +Subagent 不是免费的——每次派遣**会烧 token，也有延迟**。下面 4 种场景**不该用 subagent**，自己跑更划算： + +1. **任务 < 5 分钟可自己完成** — 大材小用；subagent overhead 不划算 +2. **结果需要逐步 user feedback** — subagent 是“派出去、跑完回报一次”，不能中间问你；要逐步确认的任务直接在主 session 跑 +3. **任务需要主 session 的 context memory** — subagent 是**独立 context window**，看不到主 session 前面的对话；要用“我们刚刚讨论的 X...”这种 reference 的任务，不适合 +4. **任务涉及 conversation-level judgment** — 像“这个 architecture 决策该怎么权衡”，需要对话协作，不适合丢给 subagent + +> 💡 **判断的快速办法**：如果任务可以用“**写一份完整 brief 给陌生人接手做**”描述清楚 → 适合 subagent；如果需要“**我跟你讨论一下**”→ 留在主 session。 + +--- + +## 接下来 + +- **想理解完整理论**（subagent 跟 skill / MCP 的差别、3 种 multi-agent 机制）→ [Stage 5.5](../stages/05-claude-code-ecosystem.zh-Hans.md#55--subagentsclaude-code-原生-multi-agent-机制-2025-新功能) +- **CLI 日常用法 playbook** → [`tracks/cli/A3-cli-production.md` Playbook 4](../tracks/cli/A3-cli-production.zh-Hans.md#-playbook-4派遣-subagent-跑独立任务) +- **想看 subagent 在 agent paradigm 体系内的定位** → [`resources/agent-paradigms.md`](./agent-paradigms.zh-Hans.md#subagent--在-agent-runtime-里再-spawn-agent) +- **词汇快查** → [`resources/glossary.md` § 5. Claude Code 生态 — Subagent](./glossary.zh-Hans.md#subagent子-agent) diff --git a/scripts/README.md b/scripts/README.md new file mode 100644 index 0000000..567c6ae --- /dev/null +++ b/scripts/README.md @@ -0,0 +1,122 @@ +# scripts/ + +維護用的工具腳本 + distribution build。 + +## `check-links.py` — 檢查連結是否失效 + +掃描所有 markdown 檔案中的 URL，回報 4xx / 5xx / timeout。 + +```bash +# 一次性檢查全部 +python scripts/check-links.py + +# 只查 GitHub repos（最容易 404 的） +python scripts/check-links.py --fast + +# 只印失敗，不印 OK +python scripts/check-links.py --quiet +``` + +退出 code：失敗時 = 1，全部 OK = 0。可以接 CI。 + +依賴：`pip install requests` + +## `refresh-stars.py` — 比對 markdown 內標註的 stars 跟實際 + +```bash +# 列出所有差距 ≥ 10% 的 entry +python scripts/refresh-stars.py + +# 設定門檻（譬如 ≥ 20%） +python scripts/refresh-stars.py --threshold 20 + +# CI 模式（差距超過門檻就退 code 1） +python scripts/refresh-stars.py --check +``` + +依賴：`pip install requests` + `gh` CLI（`gh auth login`） + +## 建議的維護節奏 + +- **每月**：跑一次 `check-links.py --fast` 看 GitHub repo 連結有沒有 404 +- **星數刷新**：`weekly-catalog-refresh` CI 每週自動跑 `refresh-stars.py`（手動補跑可看大幅成長 / 衰退的 repo） +- **每半年**：跑一次完整 `check-links.py`（包含非 GitHub 連結） + +可以接到 GitHub Actions 自動跑（見未來 Phase 6 的 CI 設定）。 + +--- + +## `build-pdf.sh` — 編譯成單一 PDF + +```bash +bash scripts/build-pdf.sh # zh-TW 版（預設） +LANG_VARIANT=en bash scripts/build-pdf.sh # 英文版 +``` + +輸出：`dist/awesome-agentic-ai-zh.pdf`（或 `.en.pdf`） + +依賴： +- `pandoc` (>= 3.0) +- `xelatex`（TeX Live with CJK support） +- **CJK 字型**：`Noto Sans CJK TC`（zh-TW + en 共用——en 版也需要，因為章節標題仍含中文） +- **西文字型**：`DejaVu Sans` + +### 安裝指令 + +**macOS**： +```bash +brew install pandoc +brew install --cask mactex-no-gui # TeX Live + xelatex +brew install --cask font-noto-sans-cjk-tc # CJK 字型 +brew install --cask font-dejavu # 西文字型 +``` + +**Linux (Debian / Ubuntu)**： +```bash +sudo apt install pandoc texlive-xetex texlive-lang-chinese \ + fonts-noto-cjk fonts-dejavu +``` + +**Windows**： +```powershell +choco install pandoc miktex +# 然後手動裝字型： +# Noto Sans CJK TC: https://fonts.google.com/noto/specimen/Noto+Sans+TC +# DejaVu Sans: https://dejavu-fonts.github.io/ +``` + +### 換字型 + +如果上面的字型沒有，可以改用系統內建的： + +```bash +# macOS（已內建 PingFang） +CJK_FONT="PingFang TC" bash scripts/build-pdf.sh +# Windows（已內建 Microsoft JhengHei） +CJK_FONT="Microsoft JhengHei" bash scripts/build-pdf.sh +``` + +兩個字型 env var 都支援：`CJK_FONT` 跟 `MAIN_FONT`。 + +**Mermaid 圖**：目前 build-pdf.sh 會把 ` ```mermaid` 退化成普通 code block。要 render 圖需要另外裝 `pandoc-mermaid` filter（複雜度高，預設跳過）。 + +## `build-mdbook.sh` — 建可瀏覽的網站版 + +```bash +bash scripts/build-mdbook.sh # 建到 book/dist/ +bash scripts/build-mdbook.sh --serve # 建好後本機開 server (port 3000) +``` + +依賴： +- Rust + cargo（[rustup.rs](https://rustup.rs)） +- `cargo install mdbook mdbook-mermaid` +- 第一次跑前：`mdbook-mermaid install .`（會生成 `mermaid.min.js`、`mermaid-init.js`，工作流需要） + +**自動部署**： +推 main branch 時，[`.github/workflows/docs.yml`](../.github/workflows/docs.yml) 會自動 build mkdocs 站（`/` 首頁）+ mdBook（`/book/`）並 deploy 到 GitHub Pages。單一 workflow 擁有 Pages（兩個 workflow 各自 deploy 會互搶同一個 root，故已合併、刪除舊的 `deploy-book.yml`）。 +要啟用，去 Settings → Pages → Source: GitHub Actions。 + +## 整體 Phase 5 deploy 流程 + +1. 推 main → `docs.yml` 自動 build mkdocs（`/` 首頁）+ mdBook（`/book/`）並 deploy 到 `https://wenyuchiou.github.io/awesome-agentic-ai-zh/` +2. PDF：手動跑 `bash scripts/build-pdf.sh`，把 `dist/*.pdf` 上傳到 GitHub Release（或自動化 release workflow，TBD） diff --git a/scripts/build-docs-tree.py b/scripts/build-docs-tree.py new file mode 100644 index 0000000..d2a45ac --- /dev/null +++ b/scripts/build-docs-tree.py @@ -0,0 +1,94 @@ +#!/usr/bin/env python3 +"""Stage the curriculum content into _build/docs/ for the mkdocs site. + +Why this exists: the learning content lives at the REPO ROOT +(`stages/`, `tracks/`, `branches/`, `resources/`, root `*.md`), not in +a `docs/` subfolder. mkdocs 1.6 hard-errors if `docs_dir` is the parent +of `mkdocs.yml` (i.e. you cannot point docs_dir at the repo root). The +standard fix is a build-staging copy: this script mirrors the +whitelisted content into `_build/docs/` (which `.gitignore` already +covers via `_build/`), preserving the exact directory layout so every +relative link + the mkdocs-static-i18n `.en.md` / `.zh-Hans.md` suffix +pairing keeps working unchanged. + +Idempotent: wipes and repopulates `_build/docs/` each run. +stdlib-only (CI runner needs no extra deps for this step). + +Usage: + python scripts/build-docs-tree.py + # then: mkdocs build (mkdocs.yml has docs_dir: _build/docs) +""" +from __future__ import annotations + +import shutil +import sys +from pathlib import Path + +REPO = Path(__file__).resolve().parent.parent +DEST = REPO / "_build" / "docs" + +# Whole directories copied verbatim (they hold .md + referenced .png etc.) +CONTENT_DIRS = [ + "stages", + "tracks", + "branches", + "resources", + "walkthroughs", + "examples", + "docs", # repo's docs/ → staged to _build/docs/docs/ (matches nav: docs/HOW_TO_USE.md) +] + +# Root-level pages (plus their .en.md / .zh-Hans.md siblings, auto-found) +ROOT_STEMS = [ + "index", + "README", + "PROGRESS", + "ROADMAP", + "CONTRIBUTING", + "CODE_OF_CONDUCT", + "SECURITY", + "CONTRIBUTORS", + "CHANGELOG", + "RESOURCES", + "CAPSTONE", +] + + +def main() -> int: + if DEST.exists(): + shutil.rmtree(DEST) + DEST.mkdir(parents=True) + + copied_dirs = 0 + for d in CONTENT_DIRS: + src = REPO / d + if src.is_dir(): + # symlinks=False (default): copies symlink objects, does NOT + # follow them — safe against symlink-escape out of the tree. + shutil.copytree(src, DEST / d) + copied_dirs += 1 + + copied_files = 0 + # The repo's GitHub README must NOT be staged as `README.md`: mkdocs + # special-cases that filename as a directory index, which collides with + # `index.md` (the landing) and resolves inconsistently per i18n locale. + # Stage it as `about.md`; mkdocs_hooks.py rewrites in-content + # `README.md` links to `about.md` so they still resolve on the site. + for stem in ROOT_STEMS: + out_stem = {"README": "about"}.get(stem, stem) + for suffix in (".md", ".en.md", ".zh-Hans.md"): + f = REPO / f"{stem}{suffix}" + if f.is_file(): + shutil.copy2(f, DEST / f"{out_stem}{suffix}") + copied_files += 1 + + print(f"staged {copied_dirs} dirs + {copied_files} root pages -> {DEST.relative_to(REPO)}") + # Sanity: home page must exist or mkdocs has no site index + if not (DEST / "index.md").is_file(): + print("ERROR: index.md missing from staged tree", file=sys.stderr) + return 1 + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/build-mdbook.sh b/scripts/build-mdbook.sh new file mode 100644 index 0000000..2876217 --- /dev/null +++ b/scripts/build-mdbook.sh @@ -0,0 +1,90 @@ +#!/usr/bin/env bash +# build-mdbook.sh — 把 repo 的 markdown 檔案打包成 mdBook 網站 +# +# 用法： +# bash scripts/build-mdbook.sh # 建到 book/dist/ +# bash scripts/build-mdbook.sh --serve # 建好後本機開 server +# +# 依賴： +# - mdBook (cargo install mdbook) +# +# Windows 安裝： +# - 裝 Rust: https://rustup.rs +# - cargo install mdbook +# +# Note: mdbook-mermaid preprocessor 已移除（repo 內已無 ```mermaid``` 區塊）。 + +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +cd "$REPO_ROOT" + +SRC_DIR="book/src" +SERVE_FLAG="" +[[ "${1:-}" == "--serve" ]] && SERVE_FLAG="serve" + +echo "Preparing $SRC_DIR ..." +rm -rf "$SRC_DIR" +mkdir -p "$SRC_DIR/stages" "$SRC_DIR/branches" "$SRC_DIR/walkthroughs" "$SRC_DIR/resources" + +# 複製 zh 版本的 markdown 檔案到 book/src/ +cp README.md "$SRC_DIR/README.md" +cp CONTRIBUTING.md "$SRC_DIR/CONTRIBUTING.md" +cp stages/*.md "$SRC_DIR/stages/" 2>/dev/null || true +cp branches/*.md "$SRC_DIR/branches/" 2>/dev/null || true +cp walkthroughs/*.md "$SRC_DIR/walkthroughs/" 2>/dev/null || true +cp resources/*.md "$SRC_DIR/resources/" 2>/dev/null || true + +# 移除 .en.md（mdBook 只渲染主語版本） +find "$SRC_DIR" -name "*.en.md" -delete + +# 產生 SUMMARY.md +cat > "$SRC_DIR/SUMMARY.md" <<'EOF' +# Summary + +[awesome-agentic-ai-zh](README.md) + +# 主路線 + +- [Stage 0 — 基礎準備](stages/00-foundations.md) +- [Stage 1 — LLM 入門](stages/01-llm-basics.md) +- [Stage 2 — Prompt 設計](stages/02-prompt-engineering.md) +- [Stage 3 — Tool Use & Hello Agent](stages/03-tool-use-and-hello-agent.md) +- [Stage 4 — Agent 框架](stages/04-agent-frameworks.md) +- [Stage 5 — Claude Code 生態](stages/05-claude-code-ecosystem.md) +- [Stage 6 — Memory · RAG · 進階](stages/06-memory-rag.md) +- [Stage 7 — 進階 Multi-Agent](stages/07-multi-agent-production.md) + +# 跨 Stage 範例 + +- [7 步打造你的第一個 AI Agent](walkthroughs/build-first-agent-in-7-steps.md) + +# 進階分支 + +- [研究人員](branches/for-researcher.md) +- [開發者](branches/for-developer.md) +- [知識工作者](branches/for-knowledge-worker.md) +- [教師](branches/for-teacher.md) + +# 規範 + +- [風格指南](resources/style-guide.md) +- [貢獻指南](CONTRIBUTING.md) +EOF + +echo "Generated $SRC_DIR/SUMMARY.md" + +# 確認 mdbook 有裝 +if ! command -v mdbook &> /dev/null; then + echo "❌ mdbook 沒裝。請先：cargo install mdbook" >&2 + exit 1 +fi + +echo "" +if [[ -n "$SERVE_FLAG" ]]; then + mdbook serve --open +else + mdbook build + echo "" + echo "✅ Built book/dist/ (open book/dist/index.html in browser)" +fi diff --git a/scripts/build-pdf.sh b/scripts/build-pdf.sh new file mode 100644 index 0000000..71f678c --- /dev/null +++ b/scripts/build-pdf.sh @@ -0,0 +1,135 @@ +#!/usr/bin/env bash +# build-pdf.sh — 把整個 awesome-agentic-ai-zh 編譯成單一 PDF +# +# 用法： +# bash scripts/build-pdf.sh # 預設輸出 dist/awesome-agentic-ai-zh.pdf (zh-TW) +# LANG_VARIANT=en bash scripts/build-pdf.sh # 英文版 +# +# 依賴： +# - pandoc (>= 3.0) +# - xelatex (TeX Live with CJK support, e.g. texlive-xetex + texlive-lang-chinese) +# - mermaid-cli 跟 pandoc-mermaid filter（選用，沒裝 mermaid 圖會被當 code block 顯示） +# +# 安裝（macOS）： +# brew install pandoc +# brew install --cask mactex-no-gui # or mactex +# +# 安裝（Linux）： +# sudo apt install pandoc texlive-xetex texlive-lang-chinese texlive-fonts-recommended +# +# 安裝（Windows）： +# choco install pandoc miktex +# 或下載 https://pandoc.org/installing.html + https://miktex.org/ + +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +cd "$REPO_ROOT" + +LANG_VARIANT="${LANG_VARIANT:-zh}" # zh | en +DIST_DIR="dist" +mkdir -p "$DIST_DIR" + +# Font defaults — overridable via env vars +# 例：CJK_FONT="PingFang TC" bash scripts/build-pdf.sh +CJK_FONT="${CJK_FONT:-Noto Sans CJK TC}" +MAIN_FONT="${MAIN_FONT:-DejaVu Sans}" + +if [[ "$LANG_VARIANT" == "en" ]]; then + SUFFIX=".en" + TITLE="awesome-agentic-ai-zh — Learning Roadmap" + SUBTITLE="An 8-stage path from your first LLM call to multi-agent systems" + OUT_PDF="$DIST_DIR/awesome-agentic-ai-zh.en.pdf" +else + SUFFIX="" + TITLE="awesome-agentic-ai-zh — AI Agent 學習地圖" + SUBTITLE="從第一行 LLM API 到自己打造多 agent 系統" + OUT_PDF="$DIST_DIR/awesome-agentic-ai-zh.pdf" +fi + +echo "Building PDF (variant: $LANG_VARIANT) → $OUT_PDF" + +# 組合檔案順序（main path 為主；branches 跟 walkthroughs 也納入） +FILES=( + "README${SUFFIX}.md" + "stages/00-foundations${SUFFIX}.md" + "stages/01-llm-basics${SUFFIX}.md" + "stages/02-prompt-engineering${SUFFIX}.md" + "stages/03-tool-use-and-hello-agent${SUFFIX}.md" + "stages/04-agent-frameworks${SUFFIX}.md" + "stages/05-claude-code-ecosystem${SUFFIX}.md" + "stages/06-memory-rag${SUFFIX}.md" + "stages/07-multi-agent-production${SUFFIX}.md" + "branches/for-researcher${SUFFIX}.md" + "branches/for-developer${SUFFIX}.md" + "branches/for-knowledge-worker${SUFFIX}.md" + "branches/for-teacher${SUFFIX}.md" + "walkthroughs/build-first-agent-in-7-steps${SUFFIX}.md" + "CONTRIBUTING${SUFFIX}.md" +) + +# 確認所有檔案存在 +for f in "${FILES[@]}"; do + if [[ ! -f "$f" ]]; then + echo "ERROR: missing file: $f" >&2 + exit 1 + fi +done + +# 中介檔案：把所有 .md 串起來 +TMP_MD="$(mktemp -t aaai-XXXXXX.md)" +trap 'rm -f "$TMP_MD"' EXIT + +# 加 metadata header +cat > "$TMP_MD" <> "$TMP_MD" + echo "\\newpage" >> "$TMP_MD" + echo "" >> "$TMP_MD" + cat "$f" >> "$TMP_MD" +done + +echo "Concatenated $(wc -l < "$TMP_MD") lines." + +# 把 mermaid block 換成 code block（沒裝 filter 時，避免 pandoc 卡住） +# 真要 render mermaid，得另外裝 pandoc-mermaid 或先用 mermaid-cli 把 ```mermaid 轉成圖 +sed -i.bak 's/^```mermaid$/```/' "$TMP_MD" 2>/dev/null || \ + sed -i '' 's/^```mermaid$/```/' "$TMP_MD" # macOS sed +rm -f "${TMP_MD}.bak" + +# 跑 pandoc +pandoc "$TMP_MD" \ + --pdf-engine=xelatex \ + --output="$OUT_PDF" \ + -V linkcolor=blue \ + -V geometry:margin=2.5cm \ + --highlight-style=tango \ + || { + echo "" + echo "❌ PDF build failed. 常見原因：" >&2 + echo " - xelatex 沒裝（apt install texlive-xetex）" >&2 + echo " - CJK font 沒裝（apt install texlive-lang-chinese 或裝 Noto Sans CJK TC）" >&2 + echo " - 用 -V CJKmainfont= 覆寫" >&2 + exit 1 + } + +echo "" +echo "✅ Built $OUT_PDF ($(du -h "$OUT_PDF" | cut -f1))" diff --git a/scripts/check-2026-freshness.py b/scripts/check-2026-freshness.py new file mode 100644 index 0000000..183d80b --- /dev/null +++ b/scripts/check-2026-freshness.py @@ -0,0 +1,186 @@ +#!/usr/bin/env python3 +""" +2026 model freshness check. + +掃所有 .md 找 stale model references (Claude 3.5 / GPT-4o / Gemini 2.0 / etc.) +that lack a 'lineage' / '前身' / '歷史' qualifier within ±N lines context. + +Config: scripts/freshness-models.yml (whitelist + stale pattern list) + +Usage: + python scripts/check-2026-freshness.py # exit 1 on stale + python scripts/check-2026-freshness.py --warn-only # exit 0, prefix ::warning:: + +Exit codes: + 0 — no stale refs OR --warn-only mode + 1 — stale refs found in strict mode +""" + +from __future__ import annotations + +import argparse +import re +import sys +from fnmatch import fnmatch +from pathlib import Path + +try: + import yaml +except ImportError: + print( + '❌ PyYAML required. Install: pip install pyyaml', + file=sys.stderr, + ) + sys.exit(2) + + +EXCLUDE_DIRS = {'.ai', 'book', 'node_modules', '.git', 'archives', '.coord'} +EXCLUDE_MIRROR_SUFFIXES = ['.en.md', '.zh-Hans.md'] # focus on zh-TW canonical + + +def load_config(repo_root: Path) -> dict: + cfg_path = repo_root / 'scripts' / 'freshness-models.yml' + with open(cfg_path, encoding='utf-8') as f: + return yaml.safe_load(f) + + +def matches_exclude(path: Path, repo_root: Path, exclude_patterns: list[str]) -> bool: + """Check if file path matches any exclude glob pattern.""" + rel = path.relative_to(repo_root).as_posix() + for pat in exclude_patterns: + if fnmatch(rel, pat) or fnmatch(rel + '/', pat): + return True + return False + + +def has_qualifier( + lines: list[str], idx: int, terms: list[str], window: int +) -> bool: + """Check if any qualifier term appears in ±window lines around idx.""" + start = max(0, idx - window) + end = min(len(lines), idx + window + 1) + context = ' '.join(lines[start:end]) + return any(q in context for q in terms) + + +def scan_file( + path: Path, + cfg: dict, + repo_root: Path, +) -> list[tuple[Path, int, str, str, str]]: + """ + Scan one file for stale model refs. + + Returns list of (file, line_no, matched_text, pattern, note). + """ + findings: list[tuple[Path, int, str, str, str]] = [] + content = path.read_text(encoding='utf-8') + lines = content.split('\n') + + window = cfg.get('qualifier_context_lines', 2) + + # Check per-file context override + rel = path.relative_to(repo_root).as_posix() + for override in cfg.get('exclude_files_pattern_specific', []): + if fnmatch(rel, override.get('file', '')): + # Accept both new (qualifier_window) and legacy (skip_patterns_with_context) + # field names for backward compat + window = override.get( + 'qualifier_window', + override.get('skip_patterns_with_context', window), + ) + break + + # Check stale_patterns + for entry in cfg.get('stale_patterns', []): + pat_str = entry['pattern'] + try: + pat = re.compile(pat_str) + except re.error as e: + print(f'⚠ Invalid regex in freshness-models.yml: {pat_str!r}: {e}', file=sys.stderr) + continue + terms = entry.get('qualifier_terms', []) + note = entry.get('note', '') + for idx, line in enumerate(lines): + m = pat.search(line) + if m and not has_qualifier(lines, idx, terms, window): + findings.append((path, idx + 1, m.group(0), pat_str, note)) + + # Check stale_date_phrases + for entry in cfg.get('stale_date_phrases', []): + pat_str = entry['pattern'] + try: + pat = re.compile(pat_str) + except re.error as e: + print(f'⚠ Invalid regex: {pat_str!r}: {e}', file=sys.stderr) + continue + terms = entry.get('qualifier_terms', []) + note = entry.get('note', '') + for idx, line in enumerate(lines): + m = pat.search(line) + if m: + # Date phrases either need qualifier OR no qualifier needed (different rule) + if terms and has_qualifier(lines, idx, terms, window): + continue + findings.append((path, idx + 1, m.group(0), pat_str, note)) + + return findings + + +def should_skip(path: Path, repo_root: Path, cfg: dict) -> bool: + if any(part in EXCLUDE_DIRS for part in path.parts): + return True + for suffix in EXCLUDE_MIRROR_SUFFIXES: + if path.name.endswith(suffix): + return True + exclude_files = cfg.get('exclude_files', []) + return matches_exclude(path, repo_root, exclude_files) + + +def main() -> int: + # Force UTF-8 stdout + if hasattr(sys.stdout, 'reconfigure'): + try: + sys.stdout.reconfigure(encoding='utf-8', errors='replace') + except Exception: + pass + + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + '--warn-only', + action='store_true', + help='Exit 0 even if stale refs found, print ::warning:: prefix', + ) + args = parser.parse_args() + + repo_root = Path(__file__).resolve().parent.parent + cfg = load_config(repo_root) + + all_findings: list[tuple[Path, int, str, str, str]] = [] + for md in sorted(repo_root.rglob('*.md')): + if should_skip(md, repo_root, cfg): + continue + try: + all_findings.extend(scan_file(md, cfg, repo_root)) + except Exception as e: + print(f'⚠ scan error {md.relative_to(repo_root)}: {e}', file=sys.stderr) + + if not all_findings: + print('✓ No stale model references detected (against 2026 frontier whitelist).') + return 0 + + prefix = '::warning::' if args.warn_only else '❌ ' + for path, lineno, matched, pat, note in all_findings: + rel = path.relative_to(repo_root).as_posix() + print(f'{prefix}{rel}:{lineno}: stale "{matched}" — {note} [pattern: {pat}]') + + print(f'\nFound {len(all_findings)} stale model reference(s).') + print( + 'Tip: add a qualifier ("前身" / "歷史" / "lineage" / "baseline" / "原始")' + ' nearby to mark as historical reference.' + ) + return 0 if args.warn_only else 1 + + +if __name__ == '__main__': + sys.exit(main()) diff --git a/scripts/check-anchors.py b/scripts/check-anchors.py new file mode 100644 index 0000000..131bf24 --- /dev/null +++ b/scripts/check-anchors.py @@ -0,0 +1,207 @@ +#!/usr/bin/env python3 +""" +Internal anchor validator. + +掃所有 .md 內的 cross-file + same-file anchor link、驗 anchor 真實存在 +target file 的 H1-H6 內。GitHub markdown slugification 規則。 + +Usage: + python scripts/check-anchors.py [--strict] + +Exit codes: + 0 — 全部 anchor 都 valid + 1 — 有 broken anchor（strict mode）/ 一律 0（non-strict） + +排除：.ai/, book/, node_modules/, .git/, archives/ 目錄。 + +mirror 檔（*.en.md / *.zh-Hans.md）**現在也驗**——歷史上曾排除（理由是 +「漸進翻譯、anchor 可能晚於 zh-TW 同步」），但該豁免讓 97 個壞掉的對外 +anchor 在 39 個 mirror 檔裡無聲累積（2026-05 P3e 清查）。全部修好後改為 +強制驗，mirror outbound anchor 從此跟 canonical 同標準、回歸即時擋下。 +""" + +from __future__ import annotations + +import argparse +import re +import sys +from pathlib import Path + +# --- Config --- +EXCLUDE_DIRS = {'.ai', 'book', 'node_modules', '.git', 'archives', '.coord'} +EXCLUDE_PATTERNS: list[str] = [] # mirror files now validated (see module docstring) + +# Markdown link with anchor: [text](path.md#anchor) or [text](#anchor) +# Anchor part: starts with # then any non-) char +ANCHOR_LINK_RE = re.compile(r'\[([^\]]+)\]\(([^)]*?)#([^)]+)\)') + +# Markdown header: ## or ### etc., capture H2-H6 +HEADER_RE = re.compile(r'^(#{1,6})\s+(.+?)\s*$', re.MULTILINE) + +# Code block fence (to skip anchor matches inside code) +CODE_FENCE_RE = re.compile(r'^```', re.MULTILINE) + + +def slugify(text: str) -> str: + """ + GitHub-style anchor slug (matches github-slugger npm package). + + Critical: GitHub does NOT collapse consecutive whitespace — each space + becomes its own hyphen. So "Foo — Bar" (space em-dash space, em-dash + removed) → "foo bar" (2 spaces) → "foo--bar" (2 hyphens). + + Steps: + 1. Strip leading/trailing whitespace + 2. Lowercase ASCII (preserve Chinese characters) + 3. Remove emoji and other symbols (broad Unicode ranges including ⭐ U+2B50) + 4. Remove CJK punctuation: · ／ 「 」 、 （ ） 【 】 《 》 〈 〉 〔 〕 + 5. Remove ASCII punctuation: . , : ; ! ? " ' / \\ ` ( ) — – etc. + Keep: word chars, whitespace, hyphen, Chinese chars + 6. Replace EACH space with hyphen (one-to-one, no collapse) + + Does NOT strip leading/trailing hyphens (matches GitHub behaviour — + a leading hyphen from a removed emoji at heading start IS preserved). + """ + s = text.strip() + # Lowercase ASCII (Chinese unaffected) + s = s.lower() + # Remove emoji & misc symbols + # Ranges: + # U+1F000-U+1FFFF — emoji / pictographs + # U+2600-U+27BF — misc symbols + dingbats (☀-➿) + # U+2300-U+23FF — misc technical (⌀-⏿) + # U+2B00-U+2BFF — misc symbols and arrows (⬀-⯿, includes ⭐) + # U+2900-U+297F — supplemental arrows-B (⤀-⥿) + s = re.sub( + r'[\U0001F000-\U0001FFFF☀-➿⌀-⏿⬀-⯿⤀-⥿]', + '', + s, + ) + # Remove CJK punctuation + s = re.sub(r'[·／「」、（）【】《》〈〉〔〕]', '', s) + # Remove ASCII punctuation (keep word chars, whitespace, hyphen, Chinese) + s = re.sub(r"[^\w\s\-一-鿿]", '', s) + # Each space → hyphen (NO collapse; matches github-slugger) + s = s.replace(' ', '-') + return s + + +def strip_code_blocks(content: str) -> str: + """Replace content inside ``` ... ``` fences with blanks to skip anchor matches inside.""" + lines = content.split('\n') + out = [] + in_code = False + for line in lines: + if line.startswith('```'): + in_code = not in_code + out.append('') + continue + out.append('' if in_code else line) + return '\n'.join(out) + + +def collect_anchors(content: str) -> set[str]: + """All anchor slugs available in this file (from H1-H6).""" + return {slugify(m.group(2)) for m in HEADER_RE.finditer(content)} + + +def parse_anchor_links(content: str, file_path: Path) -> list[tuple[int, str, str]]: + """ + Extract (line_no, target_file, anchor) tuples from anchor-style links. + Skips matches inside code blocks. + """ + clean = strip_code_blocks(content) + results = [] + for lineno, line in enumerate(clean.split('\n'), 1): + for m in ANCHOR_LINK_RE.finditer(line): + target_raw = m.group(2).strip() + anchor = m.group(3).strip() + # Skip external URLs (http://..., https://...) + if target_raw.startswith(('http://', 'https://')): + continue + results.append((lineno, target_raw, anchor)) + return results + + +def should_skip(path: Path) -> bool: + """Skip excluded dirs. Mirror files are validated (EXCLUDE_PATTERNS + is empty by default; kept as a hook if a pattern ever needs re-adding).""" + if any(part in EXCLUDE_DIRS for part in path.parts): + return True + for pat in EXCLUDE_PATTERNS: + if path.name.endswith(pat): + return True + return False + + +def validate_file(path: Path, repo_root: Path) -> list[tuple[Path, int, str]]: + """Validate anchors in one file. Returns list of (file, lineno, message).""" + broken = [] + content = path.read_text(encoding='utf-8') + own_anchors: set[str] | None = None # lazy + + for lineno, target_raw, anchor in parse_anchor_links(content, path): + anchor_slug = slugify(anchor) + + if target_raw == '': + # Same-file anchor [text](#section) + if own_anchors is None: + own_anchors = collect_anchors(content) + if anchor_slug not in own_anchors: + broken.append((path, lineno, f'same-file anchor not found: #{anchor}')) + else: + # Cross-file: resolve target path relative to current file + tgt_path = (path.parent / target_raw).resolve() + try: + tgt_path.relative_to(repo_root) + except ValueError: + # Target outside repo (shouldn't happen for relative .md links) + continue + if not tgt_path.exists(): + broken.append((path, lineno, f'target file not found: {target_raw}')) + continue + target_anchors = collect_anchors(tgt_path.read_text(encoding='utf-8')) + if anchor_slug not in target_anchors: + broken.append( + (path, lineno, f'anchor not found in {target_raw}: #{anchor}') + ) + return broken + + +def main() -> int: + # Force UTF-8 stdout so Chinese chars print correctly on Windows / older CI + if hasattr(sys.stdout, 'reconfigure'): + try: + sys.stdout.reconfigure(encoding='utf-8', errors='replace') + except Exception: + pass + + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--strict', action='store_true', help='Exit 1 on broken anchors') + args = parser.parse_args() + + repo_root = Path(__file__).resolve().parent.parent + all_broken: list[tuple[Path, int, str]] = [] + + for md in sorted(repo_root.rglob('*.md')): + if should_skip(md): + continue + try: + all_broken.extend(validate_file(md, repo_root)) + except Exception as e: + print(f'⚠ {md.relative_to(repo_root)}: scan error: {e}', file=sys.stderr) + + if not all_broken: + print('✓ All internal anchors valid.') + return 0 + + for path, lineno, msg in all_broken: + rel = path.relative_to(repo_root) + prefix = '❌ ' if args.strict else '::warning::' + print(f'{prefix}{rel}:{lineno}: {msg}') + print(f'\nFound {len(all_broken)} broken anchor reference(s).') + return 1 if args.strict else 0 + + +if __name__ == '__main__': + sys.exit(main()) diff --git a/scripts/check-links.py b/scripts/check-links.py new file mode 100644 index 0000000..9df765c --- /dev/null +++ b/scripts/check-links.py @@ -0,0 +1,154 @@ +#!/usr/bin/env python3 +""" +check-links.py — 掃描所有 markdown 檔案的 URL，回報 4xx / 5xx / timeout。 + +用法： + python scripts/check-links.py # 檢查所有 .md 檔 + python scripts/check-links.py --fast # 只查 GitHub repos（最容易 404） + python scripts/check-links.py --quiet # 只印失敗 + +環境需求： + pip install requests +""" + +import argparse +import re +import sys +from concurrent.futures import ThreadPoolExecutor, as_completed +from pathlib import Path +from typing import Iterable + +try: + import requests +except ImportError: + print("ERROR: 需要 requests。請先執行：pip install requests", file=sys.stderr) + sys.exit(1) + +REPO_ROOT = Path(__file__).resolve().parent.parent +MD_GLOB = "**/*.md" +EXCLUDE_DIRS = {".git", ".ai", "node_modules", "_build", ".venv"} + +# 抓 markdown link [text](url) 的正則。處理 url 內可能含巢狀 ()。 +# 用「至少 1 個非空白非右括號字元，後接任意可選 (...) 對」的策略。 +LINK_RE = re.compile( + r"\[([^\]]+)\]" + r"\((https?://[^\s()]+(?:\([^\s()]*\))?[^\s)]*)\)" +) + +TIMEOUT = 15 +MAX_WORKERS = 10 + + +def find_md_files(root: Path) -> list[Path]: + files = [] + for fp in root.glob(MD_GLOB): + if any(part in EXCLUDE_DIRS for part in fp.parts): + continue + files.append(fp) + return files + + +def extract_urls(md_path: Path) -> list[tuple[int, str]]: + """回傳 [(line_no, url), ...]，跳過程式碼區塊內的 URL。""" + urls = [] + text = md_path.read_text(encoding="utf-8") + in_fenced_code = False + for line_no, line in enumerate(text.splitlines(), start=1): + # Toggle fenced code block state on ``` or ~~~ + stripped = line.lstrip() + if stripped.startswith("```") or stripped.startswith("~~~"): + in_fenced_code = not in_fenced_code + continue + if in_fenced_code: + continue + # 也跳過 inline code（粗略：只在 ` ` 之間的 URL 不算） + # Markdown 規範允許 inline code 內含 link 但通常不是真 link + for match in LINK_RE.finditer(line): + url = match.group(2).rstrip(".,;:!?") + urls.append((line_no, url)) + return urls + + +def check_url(url: str, fast_mode: bool = False) -> tuple[str, int | None, str]: + """回傳 (url, final_status_code or None, message)。allow_redirects=True 表示 + final_status 不會是 3xx（會被 follow 到 2xx 或 4xx/5xx）。""" + if fast_mode and "github.com" not in url: + return url, None, "skipped (--fast)" + try: + r = requests.head(url, timeout=TIMEOUT, allow_redirects=True, + headers={"User-Agent": "awesome-agentic-ai-zh-link-check/1.0"}) + # 有些 server 不接受 HEAD，fallback 用 GET + if r.status_code in (405, 403): + r = requests.get(url, timeout=TIMEOUT, allow_redirects=True, stream=True, + headers={"User-Agent": "awesome-agentic-ai-zh-link-check/1.0"}) + r.close() + return url, r.status_code, "" + except requests.exceptions.RequestException as e: + return url, None, str(e)[:80] + + +def main(): + parser = argparse.ArgumentParser(description="Check markdown links for rot.") + parser.add_argument("--fast", action="store_true", help="只查 GitHub URL") + parser.add_argument("--quiet", action="store_true", help="只印失敗") + args = parser.parse_args() + + files = find_md_files(REPO_ROOT) + print(f"Scanning {len(files)} markdown files...", file=sys.stderr) + + # 收集所有 URL（去重，但記下出現位置） + occurrences: dict[str, list[tuple[Path, int]]] = {} + for fp in files: + for line_no, url in extract_urls(fp): + occurrences.setdefault(url, []).append((fp, line_no)) + + print(f"Found {len(occurrences)} unique URLs.", file=sys.stderr) + + failures = [] + ok_count = 0 + skipped = 0 + + with ThreadPoolExecutor(max_workers=MAX_WORKERS) as ex: + futures = {ex.submit(check_url, url, args.fast): url for url in occurrences} + for i, fut in enumerate(as_completed(futures), start=1): + url, status, msg = fut.result() + if status is None and msg.startswith("skipped"): + skipped += 1 + continue + if status is None: + failures.append((url, f"ERROR: {msg}")) + if not args.quiet: + print(f"[{i}/{len(occurrences)}] ❌ {url} — {msg}") + elif status >= 400: + failures.append((url, f"HTTP {status}")) + if not args.quiet: + print(f"[{i}/{len(occurrences)}] ❌ {url} — HTTP {status}") + else: + # 200-299 (3xx 已被 allow_redirects 跟過去 → final 是 2xx 或 4xx/5xx) + ok_count += 1 + if not args.quiet: + print(f"[{i}/{len(occurrences)}] ✓ {url}") + + # 報告 + print() + print("=" * 60) + print(f"Total checked: {len(occurrences) - skipped}") + print(f"OK (2xx): {ok_count}") + print(f"Failed: {len(failures)}") + if args.fast: + print(f"Skipped (--fast): {skipped}") + print() + + if failures: + print("=== Failures by file ===") + for url, reason in failures: + print(f"\n❌ {url} [{reason}]") + for fp, line_no in occurrences[url]: + rel = fp.relative_to(REPO_ROOT) + print(f" {rel}:{line_no}") + + sys.exit(1 if failures else 0) + + +if __name__ == "__main__": + main() diff --git a/scripts/check-mirror-sync.py b/scripts/check-mirror-sync.py new file mode 100644 index 0000000..6cf4121 --- /dev/null +++ b/scripts/check-mirror-sync.py @@ -0,0 +1,184 @@ +#!/usr/bin/env python3 +""" +Mirror sync reminder. + +當 PR 改了 zh-TW canonical .md 檔（path 內含 stages/ branches/ tracks/ resources/ +walkthroughs/ 或 README.md / CONTRIBUTING.md）、但沒同時改對應的 .en.md / .zh-Hans.md +mirror、產生一個 PR comment body 寫到 .mirror-sync-comment.md、提示 contributor。 + +Soft reminder — 不擋 PR。zh-TW canonical、mirror sync 是 Path B（slower cadence）。 + +Usage: + python scripts/check-mirror-sync.py --pr-base origin/main + +GitHub Actions integration: + Sets GitHub Actions output `gap_detected=true|false`. + Workflow uses sticky comment action to keep one comment per PR. +""" + +from __future__ import annotations + +import argparse +import os +import subprocess +import sys +from pathlib import Path + + +MIRROR_SUFFIXES = ['.en.md', '.zh-Hans.md'] + + +def get_changed_files(base_ref: str) -> list[Path]: + """Get list of files changed in PR vs base_ref.""" + try: + out = subprocess.check_output( + ['git', 'diff', '--name-only', f'{base_ref}...HEAD'], + encoding='utf-8', + stderr=subprocess.STDOUT, + ) + except subprocess.CalledProcessError as e: + print(f'⚠ git diff failed: {e}', file=sys.stderr) + # Fall back to single-commit diff + try: + out = subprocess.check_output( + ['git', 'diff', '--name-only', 'HEAD~1'], + encoding='utf-8', + ) + except Exception: + return [] + return [Path(f.strip()) for f in out.splitlines() if f.strip()] + + +def is_canonical_md(path: Path) -> bool: + """zh-TW canonical = .md but not .en.md / .zh-Hans.md.""" + name = str(path) + if not name.endswith('.md'): + return False + for suffix in MIRROR_SUFFIXES: + if name.endswith(suffix): + return False + return True + + +def detect_sync_gaps( + changed: list[Path], repo_root: Path +) -> list[tuple[str, list[str]]]: + """ + Return list of (canonical_file_changed, [missing_mirror_files]) tuples. + Only flag canonical files whose mirrors EXIST in repo (so contributor + isn't asked to create a new mirror from scratch). + """ + changed_set = {str(p) for p in changed} + gaps: list[tuple[str, list[str]]] = [] + + for f in changed: + s = str(f).replace('\\', '/') # normalize for Windows + if not is_canonical_md(Path(s)): + continue + + # For X.md, compute X.en.md and X.zh-Hans.md candidate paths + stem = s[:-3] # strip .md + candidates = [(suf, stem + suf) for suf in MIRROR_SUFFIXES] + + missing = [] + for suf, candidate_path in candidates: + cand_full = repo_root / candidate_path + if not cand_full.exists(): + continue # no mirror exists yet — don't pressure contributor + # Mirror exists. Check if it was also changed in this PR. + normalized = candidate_path.replace('\\', '/') + if normalized not in changed_set and candidate_path not in changed_set: + missing.append(candidate_path) + + if missing: + gaps.append((s, missing)) + + return gaps + + +def write_comment_body(gaps: list[tuple[str, list[str]]], out_path: Path) -> None: + """Write GitHub PR comment body listing the sync gaps.""" + lines = [ + '## 🌐 Mirror Sync Reminder', + '', + 'zh-TW canonical files were updated in this PR, but the following mirror', + 'files were **not** updated in the same change set:', + '', + ] + for canonical, missing in gaps: + lines.append(f'- **`{canonical}`** modified — missing sync to:') + for m in missing: + lines.append(f' - `{m}`') + lines.extend( + [ + '', + '> 💡 **This is a soft reminder, not a blocker.** zh-TW is the canonical', + '> version of this repo. Mirror files (`.en.md` / `.zh-Hans.md`) sync', + '> is on a slower cadence (Path B). Update when convenient — or open a', + '> separate sync PR later.', + '', + '> If the canonical change is purely cosmetic (typo, whitespace, etc.),', + '> mirror sync can be safely skipped.', + '', + ] + ) + out_path.write_text('\n'.join(lines), encoding='utf-8') + + +def set_gh_output(name: str, value: str) -> None: + """Set a GitHub Actions output variable.""" + out_file = os.environ.get('GITHUB_OUTPUT') + if out_file: + with open(out_file, 'a', encoding='utf-8') as fh: + fh.write(f'{name}={value}\n') + # Also print for local debug + print(f'{name}={value}') + + +def main() -> int: + # Force UTF-8 stdout + if hasattr(sys.stdout, 'reconfigure'): + try: + sys.stdout.reconfigure(encoding='utf-8', errors='replace') + except Exception: + pass + + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + '--pr-base', + default='origin/main', + help='Base ref to diff against (default: origin/main)', + ) + parser.add_argument( + '--comment-out', + default='.mirror-sync-comment.md', + help='Path to write PR comment body', + ) + args = parser.parse_args() + + repo_root = Path(__file__).resolve().parent.parent + changed = get_changed_files(args.pr_base) + + if not changed: + print('No changed files detected.') + set_gh_output('gap_detected', 'false') + return 0 + + gaps = detect_sync_gaps(changed, repo_root) + + if not gaps: + print('✓ All canonical changes have synced mirrors (or no mirror exists yet).') + set_gh_output('gap_detected', 'false') + return 0 + + print(f'⚠ Detected {len(gaps)} canonical file(s) with unsynced mirrors:') + for canonical, missing in gaps: + print(f' - {canonical}: missing {", ".join(missing)}') + + write_comment_body(gaps, repo_root / args.comment_out) + set_gh_output('gap_detected', 'true') + return 0 + + +if __name__ == '__main__': + sys.exit(main()) diff --git a/scripts/check-stage-template.py b/scripts/check-stage-template.py new file mode 100644 index 0000000..ff4fdf4 --- /dev/null +++ b/scripts/check-stage-template.py @@ -0,0 +1,170 @@ +#!/usr/bin/env python3 +""" +Stage template structure check. + +驗每個 stages/[0-9]*-*.md (zh-TW canonical) 有所有必要的 H2 section、 +跟 Stage 5/6/7/8 已經對齊的 template 一致。 + +REQUIRED — 缺則 fail（H2 必須有）： + - 📌 學習目標 + - 🚪 進入條件 + - 📚 必修閱讀 + - 🛠 動手練習 + - 🎯 精選 Projects（或變體含「Projects」） + - ✅ 自我檢查（或 .* 自我檢查 / 進入 .* 前的自我檢查） + +EXPECTED — 缺則 warning（不擋）： + - 🎯 X 是什麼（先定位） — positioning section + - 🎯 常用 .* 工具推薦 / 常用 .* 推薦 — tool recommendation + + 註：Stage 01-04 為 foundational 章節，其定位 / 工具推薦段落使用章節自然 + 命名（如「主流 LLM 家族對比」/「什麼是 multi-agent framework」/「開始前： + AI / LLM / Agent 三者怎麼分」），刻意不套用上面的 EXPECTED 樣板字串。 + 因此這些 ⚠ EXPECTED 警告對 Stage 01-04 屬「資訊性」而非缺陷——它們的 + REQUIRED 段落全部通過，定位段落實質存在、只是用更貼切的章節名。不強制 + 改名（改名會把更精準的標題降級成樣板字串、得不償失）。 + +排除：mirror files (*.en.md, *.zh-Hans.md)、Stage 0 (foundations 短 intro)。 + +Usage: + python scripts/check-stage-template.py [--strict-expected] +""" + +from __future__ import annotations + +import argparse +import re +import sys +from pathlib import Path + + +REQUIRED_PATTERNS = [ + (r'📌\s*學習目標', '📌 學習目標'), + (r'🚪\s*進入條件', '🚪 進入條件'), + (r'📚\s*必修閱讀', '📚 必修閱讀'), + (r'🛠\s*動手練習', '🛠 動手練習'), + (r'🎯\s*(精選|常用)?\s*.*Projects', '🎯 精選 Projects'), + # Grouped alternation: matches "✅ X 自我檢查" or "X 自我檢查" anywhere — but NOT + # bare "自我檢查" in unrelated context (must have surrounding text before 自我檢查) + (r'(✅\s*.*自我檢查|.+自我檢查|自我檢查)', '✅ 自我檢查'), +] + +EXPECTED_PATTERNS = [ + (r'🎯\s*.*是什麼.*先定位', '🎯 [topic] 是什麼（先定位）'), + (r'🎯\s*常用.*推薦|🎯\s*常用.*工具', '🎯 常用工具推薦（按用途）'), +] + +# Stages that don't follow the per-stage template: +# - 00- prerequisites: short intro doc, doesn't need full template +# - 05- Claude Code ecosystem: multi-sub-stage container (5.1-5.6), +# each sub-stage has its own learning goals/practice structure; +# template check at file level doesn't make sense. +# - 07.5- advanced agentic concepts: reading-map chapter (12 concepts +# skeleton + reading paths + cross-vendor principles); intentionally +# does NOT have 動手練習 / 精選 Projects — it's a meta-章 pointing +# to other stages, not a hands-on stage. Has 自我檢查 only. +SKIP_STAGES = ['00-', '05-', '07.5-'] + +# Foundational stages (01-04) use chapter-natural section names rather +# than the template strings in EXPECTED_PATTERNS (e.g. "主流 LLM 家族對比" +# instead of "🎯 LLM 是什麼（先定位）"). Their positioning and tool- +# recommendation sections are substantively present — just named more +# precisely. Renaming would degrade quality. These stages pass all +# REQUIRED checks; only the EXPECTED pattern match fails. Suppress the +# ⚠ EXPECTED warnings for them to keep the script output signal-only. +EXPECTED_EXEMPT_STAGES = ['01-', '02-', '03-', '04-'] + +H2_RE = re.compile(r'^## (.+?)\s*$', re.MULTILINE) + + +def get_h2_sections(content: str) -> list[str]: + return [m.group(1) for m in H2_RE.finditer(content)] + + +def check_stage(path: Path) -> tuple[list[str], list[str]]: + """Return (missing_required, missing_expected) labels.""" + content = path.read_text(encoding='utf-8') + h2s = get_h2_sections(content) + + def matches_any(pat: str) -> bool: + return any(re.search(pat, h2) for h2 in h2s) + + missing_req = [label for pat, label in REQUIRED_PATTERNS if not matches_any(pat)] + missing_exp = [label for pat, label in EXPECTED_PATTERNS if not matches_any(pat)] + return missing_req, missing_exp + + +def should_skip(path: Path) -> bool: + name = path.name + if name.endswith('.en.md') or name.endswith('.zh-Hans.md'): + return True + for prefix in SKIP_STAGES: + if name.startswith(prefix): + return True + return False + + +def should_skip_expected(path: Path) -> bool: + """Return True for stages whose EXPECTED sections are intentionally named differently.""" + name = path.name + for prefix in EXPECTED_EXEMPT_STAGES: + if name.startswith(prefix): + return True + return False + + +def main() -> int: + # Force UTF-8 stdout + if hasattr(sys.stdout, 'reconfigure'): + try: + sys.stdout.reconfigure(encoding='utf-8', errors='replace') + except Exception: + pass + + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + '--strict-expected', + action='store_true', + help='Also fail on missing EXPECTED sections (not just REQUIRED)', + ) + args = parser.parse_args() + + repo_root = Path(__file__).resolve().parent.parent + stages_dir = repo_root / 'stages' + if not stages_dir.exists(): + print('❌ stages/ directory not found.', file=sys.stderr) + return 2 + + has_required_issue = False + has_expected_issue = False + + for stage in sorted(stages_dir.glob('[0-9]*-*.md')): + if should_skip(stage): + continue + missing_req, missing_exp = check_stage(stage) + rel = stage.relative_to(repo_root).as_posix() + + if missing_req: + print(f'❌ {rel}: missing REQUIRED H2 section(s):') + for label in missing_req: + print(f' - {label}') + has_required_issue = True + + if missing_exp and not should_skip_expected(stage): + warn_prefix = '❌' if args.strict_expected else '⚠' + print(f'{warn_prefix} {rel}: missing EXPECTED H2 section(s):') + for label in missing_exp: + print(f' - {label}') + if args.strict_expected: + has_expected_issue = True + + if has_required_issue or has_expected_issue: + return 1 + + if not has_required_issue: + print('✓ All stages have REQUIRED template sections.') + return 0 + + +if __name__ == '__main__': + sys.exit(main()) diff --git a/scripts/compress-images.py b/scripts/compress-images.py new file mode 100644 index 0000000..be5ce6c --- /dev/null +++ b/scripts/compress-images.py @@ -0,0 +1,98 @@ +#!/usr/bin/env python3 +""" +compress-images.py — 壓縮 resources/diagrams/ 內的 PNG 到 < 1 MB + +策略： +1. PIL `optimize=True` + 降低 8-bit palette（如果可能） +2. 如果還 > 1 MB，等比例縮小到 max width 1600px（README 顯示夠用） +3. 仍 > 1 MB 就用 JPEG 95 quality 儲存 + +依賴：pip install Pillow +""" + +import os +from pathlib import Path + +try: + from PIL import Image +except ImportError: + print("ERROR: 需要 Pillow。pip install Pillow", file=__import__('sys').stderr) + raise + +REPO_ROOT = Path(__file__).resolve().parent.parent +DIAGRAMS_DIR = REPO_ROOT / "resources" / "diagrams" +TARGET_SIZE_KB = 1000 # 目標：< 1 MB +MAX_WIDTH = 1600 # README banner 不需要超過這個寬度 + +def compress_png(src: Path) -> bool: + """嘗試多階段壓縮，回傳是否成功 (< target size)。""" + original_size_kb = src.stat().st_size // 1024 + print(f"\n=== {src.name} (original {original_size_kb} KB) ===") + + img = Image.open(src) + + # 階段 1: 重存 + optimize + img.save(src, optimize=True) + size_kb = src.stat().st_size // 1024 + print(f" Stage 1 (optimize=True): {size_kb} KB") + if size_kb < TARGET_SIZE_KB: + return True + + # 階段 2: 縮 width 到 1600 + optimize + if img.width > MAX_WIDTH: + ratio = MAX_WIDTH / img.width + new_size = (MAX_WIDTH, int(img.height * ratio)) + img_resized = img.resize(new_size, Image.LANCZOS) + img_resized.save(src, optimize=True) + size_kb = src.stat().st_size // 1024 + print(f" Stage 2 (resize to {MAX_WIDTH}w): {size_kb} KB") + if size_kb < TARGET_SIZE_KB: + return True + img = img_resized + + # 階段 3: 換 JPEG (有些 banner 圖 PNG transparency 沒用) + jpg_path = src.with_suffix(".jpg") + if img.mode == "RGBA": + # JPEG 不支援 transparency, 加白色背景 + bg = Image.new("RGB", img.size, "white") + bg.paste(img, mask=img.split()[3] if len(img.split()) == 4 else None) + img_to_save = bg + else: + img_to_save = img.convert("RGB") + img_to_save.save(jpg_path, "JPEG", quality=92, optimize=True) + jpg_size_kb = jpg_path.stat().st_size // 1024 + print(f" Stage 3 (JPEG q=92): {jpg_size_kb} KB → {jpg_path.name}") + + if jpg_size_kb < TARGET_SIZE_KB: + # JPG 比 PNG 小，但保留 PNG 為對照（暫時——可以手動決定要不要刪 PNG） + return True + print(f" ⚠️ 仍超過目標。考慮再縮 width 或降 quality。") + return False + + +def main(): + if not DIAGRAMS_DIR.exists(): + print(f"ERROR: {DIAGRAMS_DIR} 不存在") + return + + pngs = list(DIAGRAMS_DIR.glob("*.png")) + if not pngs: + print("沒找到 PNG") + return + + print(f"處理 {len(pngs)} 個 PNG...") + all_ok = True + for png in pngs: + if not compress_png(png): + all_ok = False + + print() + print("=" * 50) + if all_ok: + print("✓ 全部 < 1 MB") + else: + print("⚠️ 有檔案仍 > 1 MB，看 stage 3 的 JPG 版或手動處理") + + +if __name__ == "__main__": + main() diff --git a/scripts/freshness-models.yml b/scripts/freshness-models.yml new file mode 100644 index 0000000..cf43d47 --- /dev/null +++ b/scripts/freshness-models.yml @@ -0,0 +1,102 @@ +# Freshness check — 2026 model whitelist + stale pattern list +# +# 維護節奏：每季 review、新 model release 時 commit 一次。 +# 用於：scripts/check-2026-freshness.py +# +# 邏輯： +# - stale_patterns 內每個 pattern 是「過時 model 名稱」 +# - 出現該 pattern 但 ±N 行內沒有 qualifier_terms 任一字眼、就算 stale +# - qualifier_terms 例子：前身 / 歷史 / lineage / 舊版 / 原始 / baseline / paper +# - 這些字眼出現代表上下文已經承認這是 historical reference、不算 stale + +current_frontier_models: + # 2026-05 frontier list (review quarterly) + anthropic: + - "Opus 4.8" + - "Sonnet 5" + - "Haiku 4.5" + openai: + - "GPT-5.6" + - "GPT-5.5" + - "GPT-5.4" + - "GPT-5" + deepseek: + - "DeepSeek-V4" + - "DeepSeek-V4-Pro" + - "DeepSeek-V4-Flash" + google: + - "Gemini 3.5 Flash" + - "Gemini 3.5 Pro" + - "Gemini 3.1 Pro" + - "Gemini 3" + alibaba: + - "QwQ-32B" + - "QvQ-72B" + +# Stale patterns — 偵測過時 model name without proper historical qualifier +stale_patterns: + - pattern: 'Claude 3\.5 Sonnet' + qualifier_terms: ['前身', '歷史', 'lineage', '舊版', '原始', 'baseline', 'predecessor', 'history'] + note: 'Pre-2026 Claude version (Claude 3.5 Sonnet was current circa 2024-Q4)' + + - pattern: 'Claude 3 Opus' + qualifier_terms: ['前身', '歷史', 'lineage', '舊版', '原始', 'baseline'] + note: 'Pre-2025 Claude version' + + - pattern: 'gpt-4o|GPT-4o' + qualifier_terms: ['前身', '歷史', 'lineage', '舊版', '原始', 'baseline', '對照', 'predecessor'] + note: 'Pre-GPT-5 OpenAI flagship (GPT-4o was current 2024-mid through 2025-summer)' + + - pattern: 'GPT-4(?!\.|o|\-|\d)' + qualifier_terms: ['前身', '歷史', 'lineage', '舊版', '原始', 'baseline'] + note: 'Plain GPT-4 reference, likely stale' + + - pattern: 'Gemini 1\.5|Gemini 2\.0|Gemini 2\.5(?! Thinking)' + qualifier_terms: ['前身', '歷史', 'lineage', '舊版', 'baseline'] + note: 'Pre-Gemini 3 Google flagship' + + - pattern: 'DeepSeek-R1(?!-Distill)' + qualifier_terms: ['前身', '歷史', 'lineage', '原始', 'baseline', 'paper', 'method'] + note: '2025-01 DeepSeek reasoning model (superseded by R2 in 2026-03)' + + - pattern: 'DeepSeek-V3(?!\.)' + qualifier_terms: ['前身', '歷史', 'lineage', '舊版'] + note: 'Pre-V4 DeepSeek base model' + + - pattern: 'o1-preview|o1-mini' + qualifier_terms: ['前身', '歷史', 'lineage', '原始', 'baseline'] + note: 'Early OpenAI reasoning preview (Sep 2024)' + + - pattern: 'Opus 4\.7' + qualifier_terms: ['前身', '歷史', 'lineage', '舊版', '原始', 'baseline', 'predecessor', 'history'] + note: 'Opus 4.8 (May 2026) supersedes 4.7' + +# Stale date-bound framing — 時間性過時用詞 +# 注意：用 \s* 而不是字面 space、因為中文常寫 "2025年" 沒空格 +stale_date_phrases: + - pattern: '2025\s*(年|下半年)?\s*(才|新|剛).*?(發表|釋出|出|推出|公布)' + note: '"newly released in 2025" framing (now historical in 2026)' + + - pattern: '2024\s*(剛|新|最新).*?(發表|釋出|推出)' + note: '"newly released in 2024" framing (now historical in 2026)' + + - pattern: '剛發表|剛推出|剛釋出|新發表|甫推出' + qualifier_terms: ['2026', '本月', '上月', '今年'] + note: 'Time-bound "newly released" without 2026 anchor' + +# Files to skip entirely +exclude_files: + - 'CHANGELOG.md' + - 'CONTRIBUTORS.md' + - 'archives/**' + - '.ai/**' + - 'book/**' + +# Files where pre-2026 model refs are expected (don't warn) +exclude_files_pattern_specific: + # Stage 6's reasoning lineage table is supposed to mention R1 as predecessor + - file: 'stages/06-memory-rag.md' + qualifier_window: 5 # widen qualifier search to ±5 lines for this file (default ±2) + +# Context window — how many lines around a match to check for qualifier terms +qualifier_context_lines: 2 diff --git a/scripts/generate-stage5-stack.py b/scripts/generate-stage5-stack.py new file mode 100644 index 0000000..b63b783 --- /dev/null +++ b/scripts/generate-stage5-stack.py @@ -0,0 +1,265 @@ +"""Generate the Stage 5 Claude Code Stack diagram using PIL. + +Produces resources/diagrams/stage5-stack{,en}.png at 1920×1080. + +Why PIL not SVG: cairosvg's font fallback fails on Windows for CJK + +emoji. PIL takes explicit TTF paths so we control exactly what renders. + +Why no emoji icons: rendering color emojis (CBDT / COLR fonts) is +unreliable across PIL versions. Colored circle bullets in the layer +stroke color work better and look more professional anyway. + +Run: + python scripts/generate-stage5-stack.py [--lang zh-TW|en] +""" +from __future__ import annotations + +import argparse +import sys +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parent.parent + +LAYERS = [ + # name, sub, ability_zh-TW, ability_en, ability_zh-Hans, fill, stroke + ( + "Plugins / Marketplaces", + "5.4 — packaging", + "把 Skills、hooks、commands、MCP 設定打包發佈", + "Package & ship Skills, hooks, commands, MCP configs as one unit", + "把 Skills、hooks、commands、MCP 设置打包发布", + "#fef3c7", "#b45309", + ), + ( + "Skills", + "5.3 — behavior", + "Claude Code 的行為包，可封裝 MCP tool", + "Behavior bundles for Claude Code that can wrap MCP tools", + "Claude Code 的行为包，可封装 MCP tool", + "#dbeafe", "#1e40af", + ), + ( + "MCP", + "5.2 — protocol", + "標準化協定，任何 LLM host 都能用任何 tool server", + "Standardized protocol — any LLM host can use any tool server", + "标准化协议，任何 LLM host 都能用任何 tool server", + "#dcfce7", "#166534", + ), + ( + "Tool Use / Function Calling", + "Stage 3", + "讓 LLM 呼叫你定義的 function", + "Let the LLM call functions you define", + "让 LLM 调用你定义的 function", + "#f3e8ff", "#6b21a8", + ), + ( + "Anthropic API + SDK", + "Stage 1, Stage 7", + "用程式存取 LLM", + "Programmatic access to the LLM", + "用程序访问 LLM", + "#fee2e2", "#991b1b", + ), + ( + "LLM (Claude)", + "foundation", + "底層基礎模型", + "Foundation model", + "底层基础模型", + "#e5e7eb", "#374151", + ), +] + +# Windows font paths +FONT_CJK_BOLD = r"C:\Windows\Fonts\msjhbd.ttc" # Microsoft JhengHei Bold +FONT_CJK_REG = r"C:\Windows\Fonts\msjh.ttc" # Microsoft JhengHei +FONT_LATIN_BOLD = r"C:\Windows\Fonts\seguisb.ttf" # Segoe UI Semibold +FONT_LATIN_REG = r"C:\Windows\Fonts\segoeui.ttf" # Segoe UI + + +def hex_to_rgb(h: str) -> tuple[int, int, int]: + h = h.lstrip("#") + return tuple(int(h[i:i+2], 16) for i in (0, 2, 4)) # type: ignore[return-value] + + +def rounded_rect(draw, xy, radius, fill=None, outline=None, width=1): + """Pillow's rounded_rectangle wrapper for older versions.""" + if hasattr(draw, "rounded_rectangle"): + draw.rounded_rectangle(xy, radius=radius, fill=fill, outline=outline, width=width) + else: + draw.rectangle(xy, fill=fill, outline=outline, width=width) + + +def main() -> int: + parser = argparse.ArgumentParser() + parser.add_argument("--lang", default="zh-TW", choices=["zh-TW", "en", "zh-Hans"]) + args = parser.parse_args() + + from PIL import Image, ImageDraw, ImageFont + + W, H = 1920, 1080 + lang = args.lang + + title = "Claude Code Ecosystem Stack" + if lang == "en": + subtitle = "Each layer adds one capability · 4 sub-sections of Stage 5" + footer = ('Both tracks use this stack — Track A reads "how to use it", ' + 'Track B reads "how it works"') + elif lang == "zh-Hans": + subtitle = "Each layer adds one capability · Stage 5 的 4 个子章节" + footer = ("两条学习路径都会用到这个 stack — Track A（CLI Power User）" + "读“怎么用”，Track B（Agent Builder）读“怎么运作”") + else: + subtitle = "Each layer adds one capability · Stage 5 之 4 個子章節" + footer = ("兩條學習路徑都會用到這個 stack — Track A（CLI Power User）" + "讀「怎麼用」，Track B（Agent Builder）讀「怎麼運作」") + repo = "github.com/WenyuChiou/awesome-agentic-ai-zh" + + # Pick fonts based on language. zh-Hans uses YaHei (msyh.ttc), zh-TW uses JhengHei (msjh.ttc) + if lang == "en": + f_title = ImageFont.truetype(FONT_LATIN_BOLD, 60) + f_sub = ImageFont.truetype(FONT_LATIN_REG, 26) + f_layer_name = ImageFont.truetype(FONT_LATIN_BOLD, 38) + f_layer_sub = ImageFont.truetype(FONT_LATIN_REG, 22) + f_ability = ImageFont.truetype(FONT_LATIN_REG, 22) + f_footer = ImageFont.truetype(FONT_LATIN_REG, 22) + else: + cjk_bold = FONT_CJK_BOLD if lang == "zh-TW" else r"C:\Windows\Fonts\msyhbd.ttc" + cjk_reg = FONT_CJK_REG if lang == "zh-TW" else r"C:\Windows\Fonts\msyh.ttc" + f_title = ImageFont.truetype(FONT_LATIN_BOLD, 60) + f_sub = ImageFont.truetype(cjk_reg, 26) + f_layer_name = ImageFont.truetype(cjk_bold, 36) + f_layer_sub = ImageFont.truetype(FONT_LATIN_REG, 22) + f_ability = ImageFont.truetype(cjk_reg, 22) + f_footer = ImageFont.truetype(cjk_reg, 22) + f_repo = ImageFont.truetype(FONT_LATIN_REG, 20) + + img = Image.new("RGB", (W, H), color=(250, 250, 250)) + d = ImageDraw.Draw(img) + + # Title (top center) + title_y = 80 + tw = d.textlength(title, font=f_title) + d.text((W/2 - tw/2, title_y), title, fill=(17, 17, 17), font=f_title) + + # Subtitle + sub_y = 165 + sw = d.textlength(subtitle, font=f_sub) + d.text((W/2 - sw/2, sub_y), subtitle, fill=(107, 114, 128), font=f_sub) + + # Stack layout + stack_top = 230 + stack_bottom = 920 + layer_total = stack_bottom - stack_top + layer_h = layer_total / 6 # ~115 + + stack_x = 80 + stack_w = 1080 + ability_x = 1200 + ability_w = 640 + + for i, (name, sub, abil_zh_tw, abil_en, abil_zh_hans, fill, stroke) in enumerate(LAYERS): + if lang == "en": + abil = abil_en + elif lang == "zh-Hans": + abil = abil_zh_hans + else: + abil = abil_zh_tw + y = stack_top + i * layer_h + # Slight pyramid widening for foundation feel + widen = i * 14 + x = stack_x - widen / 2 + w = stack_w + widen + + rect_h = layer_h - 14 + # Layer rect + rounded_rect( + d, + (x, y, x + w, y + rect_h), + radius=14, + fill=hex_to_rgb(fill), + outline=hex_to_rgb(stroke), + width=3, + ) + + # Colored disk bullet (left) + bullet_r = 16 + bullet_x = x + 50 + bullet_y = y + rect_h / 2 + d.ellipse( + (bullet_x - bullet_r, bullet_y - bullet_r, + bullet_x + bullet_r, bullet_y + bullet_r), + fill=hex_to_rgb(stroke), + ) + + # Layer name + sub + text_x = bullet_x + 36 + # Name vertical: slightly above midline + name_h = 38 if lang == "en" else 36 + d.text( + (text_x, bullet_y - name_h + 4), + name, + fill=hex_to_rgb(stroke), + font=f_layer_name, + ) + # Sub label (italic visual via gray + smaller) + d.text( + (text_x, bullet_y + 6), + sub, + fill=(107, 114, 128), + font=f_layer_sub, + ) + + # Ability column (right) + ay = y + 8 + ah = rect_h - 16 + rounded_rect( + d, + (ability_x, ay, ability_x + ability_w, ay + ah), + radius=10, + fill=(255, 255, 255), + outline=(229, 231, 235), + width=2, + ) + # Ability text (vertically centered, wraps if needed) + # For now, single-line + d.text( + (ability_x + 24, ay + ah / 2 - 14), + abil, + fill=(55, 65, 81), + font=f_ability, + ) + + # Dotted divider between layers (visual hint of single stack) + if i < len(LAYERS) - 1: + sep_y = y + layer_h - 7 + # Draw dotted line manually (PIL has no native dash) + x0 = stack_x + 40 + x1 = stack_x + stack_w - 40 + cur = x0 + while cur < x1: + d.line( + [(cur, sep_y), (min(cur + 6, x1), sep_y)], + fill=(209, 213, 219), + width=1, + ) + cur += 12 + + # Footer + footer_y = 1010 + d.text((80, footer_y), footer, fill=(107, 114, 128), font=f_footer) + rw = d.textlength(repo, font=f_repo) + d.text((W - 80 - rw, footer_y + 2), repo, fill=(156, 163, 175), font=f_repo) + + suffix = {"en": ".en", "zh-Hans": ".zh-Hans", "zh-TW": ""}[lang] + out = REPO_ROOT / "resources" / "diagrams" / f"stage5-stack{suffix}.png" + out.parent.mkdir(parents=True, exist_ok=True) + img.save(out, format="PNG", optimize=True) + print(f"[PIL] {out}") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/mkdocs_hooks.py b/scripts/mkdocs_hooks.py new file mode 100644 index 0000000..5a13249 --- /dev/null +++ b/scripts/mkdocs_hooks.py @@ -0,0 +1,54 @@ +"""mkdocs build hooks for the awesome-agentic-ai-zh docs site. + +Strips the hand-written GitHub-style language switcher from the +README pages when rendered by mkdocs. + +Why: README.md / README.en.md / README.zh-Hans.md open with a +``
繁體中文 | 简体中文 | English
`` block +whose links point at the *raw* sibling files (`./README.zh-Hans.md` +…). That is correct for GitHub file browsing, but on the rendered +mkdocs site those `.md` paths 404 (mkdocs builds them into +pages/dirs, and the block is raw HTML so mkdocs does not rewrite the +links). The site already has the proper in-site language selector +in the Material header (populated by mkdocs-static-i18n's +`extra.alternate`), so the inline block is both redundant and +broken there. + +This hook removes ONLY the first ``
…
`` +block, and ONLY on the three README pages — so the GitHub-rendered +README is completely untouched (hooks run at mkdocs build time +only), and no tri-locale content edit is needed. +""" +from __future__ import annotations + +import re + +# The switcher is always the very first element of the README; the +# banner that follows is
(different), so a +# non-greedy first-match on align="right" is safe. +# +# Smoke test (local): +# python scripts/build-docs-tree.py && python -m mkdocs build +# grep -c 'align="right"' _build/site/index.html # expect 0 +# If the README switcher markup ever changes (e.g. gains a NESTED +#
, or becomes

), this non-greedy pattern would +# stop at the inner

/ not match — update it then. Failure mode +# is benign: the old (broken-on-site) switcher reappears, no build break. +_SWITCHER = re.compile(r'
.*?
\s*', re.DOTALL) +# The root README is staged as `about.md` (see build-docs-tree.py), so the +# switcher-strip now targets the renamed page. +_ABOUT_BASENAMES = {"about.md", "about.en.md", "about.zh-Hans.md"} + +# Rewrite in-content links to the root README (now `about.md`) -> about, so +# they resolve on the site. A leading `examples/` breaks the `(?:\.\./)*` +# prefix match, so examples/.../README.md links are left untouched. +_README_LINK = re.compile(r'(\]\((?:\.\./)*)README((?:\.en|\.zh-Hans)?\.md)') + + +def on_page_markdown(markdown: str, *, page, config, files) -> str: + src = (getattr(page.file, "src_path", "") or "").replace("\\", "/") + basename = src.rsplit("/", 1)[-1] + markdown = _README_LINK.sub(r"\1about\2", markdown) + if basename in _ABOUT_BASENAMES: + markdown = _SWITCHER.sub("", markdown, count=1) + return markdown diff --git a/scripts/refresh-stars.py b/scripts/refresh-stars.py new file mode 100644 index 0000000..29fb4a5 --- /dev/null +++ b/scripts/refresh-stars.py @@ -0,0 +1,289 @@ +#!/usr/bin/env python3 +""" +refresh-stars.py — 用 `gh api` 把所有 GitHub repo 的星數抓最新值，比對 markdown 內標註的 `★ Xk+`。 + +用法： + python scripts/refresh-stars.py # 列出所有有變化的 entry + python scripts/refresh-stars.py --threshold 20 # 只列差距 > 20% 的 + python scripts/refresh-stars.py --check # 退出 code 1 如果任何 entry 過時超過門檻 + +環境需求： + pip install requests + PATH 上要有 `gh` (GitHub CLI) 並且已 `gh auth login` +""" + +import argparse +import json +import re +import subprocess +import sys +from concurrent.futures import ThreadPoolExecutor, as_completed +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parent.parent +MD_GLOB = "**/*.md" +EXCLUDE_DIRS = {".git", ".ai", "node_modules", "_build", ".venv"} + +# 抓 GitHub repo URL：https://github.com/owner/repo +GITHUB_RE = re.compile(r"https://github\.com/([\w.-]+)/([\w.-]+?)(?:[#?/)\s]|$)") +# 抓 markdown 內標註的 stars：`| Stars | ★ 12k+ |` 或 inline `★ 12k+` +STARS_RE = re.compile(r"★\s*([\d.]+)\s*([kKmM]?)\+?") + +# 排除的假 / 範本 repo +PLACEHOLDER_REPOS = { + "owner/repo", + "example/repo", + "your-org/your-repo", + "user/repo", +} + +# GitHub 上不是 repo 的特殊路徑（settings、marketplace、login 等） +NON_REPO_OWNERS = { + "settings", "marketplace", "login", "logout", "join", + "topics", "trending", "collections", "events", "explore", + "issues", "pulls", "notifications", "search", "new", + "organizations", "users", "blog", "about", "pricing", + "features", "security", "enterprise", "customer-stories", +} + +MAX_WORKERS = 10 + + +def normalize_repo(owner: str, name: str) -> str | None: + """正規化 repo identifier。回傳 None 表示應該跳過。""" + # 去掉 .git 後綴 + name = name.removesuffix(".git") + repo_id = f"{owner}/{name}" + if repo_id in PLACEHOLDER_REPOS: + return None + # 排除 GitHub 上非 repo 的特殊路徑（settings/tokens 等） + if owner in NON_REPO_OWNERS: + return None + # 如果 owner 或 name 太短 / 看起來不像真 repo，跳過 + if len(owner) < 1 or len(name) < 1: + return None + return repo_id + + +def find_md_files(root: Path) -> list[Path]: + files = [] + for fp in root.glob(MD_GLOB): + if any(part in EXCLUDE_DIRS for part in fp.parts): + continue + files.append(fp) + return files + + +def parse_stars_text(s: str) -> int | None: + """`12k+` -> 12000, `1.5k+` -> 1500""" + m = STARS_RE.match(s) + if not m: + return None + num = float(m.group(1)) + unit = m.group(2).lower() + if unit == "k": + num *= 1000 + elif unit == "m": + num *= 1_000_000 + return int(num) + + +def fetch_stars(repo: str) -> int | None: + """gh api repos//""" + try: + result = subprocess.run( + ["gh", "api", f"repos/{repo}", "--jq", ".stargazers_count"], + capture_output=True, text=True, timeout=15 + ) + if result.returncode != 0: + return None + return int(result.stdout.strip()) + except (subprocess.SubprocessError, ValueError): + return None + + +def fmt_stars(n: int) -> str: + if n >= 1_000_000: + return f"{n/1_000_000:.1f}m+" + if n >= 10_000: + return f"{n//1000}k+" + if n >= 1_000: + return f"{n/1000:.1f}k+".replace(".0k", "k") + return str(n) + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--threshold", type=int, default=10, + help="只列出星數差距百分比超過此值的 entry（預設 10）") + parser.add_argument("--check", action="store_true", + help="若有任何 entry 過時則退出 code 1") + parser.add_argument("--show-missing", action="store_true", + help="列出所有有 URL 但沒附 stars 的 entry（用來盤點哪些該補 stars）") + parser.add_argument("--apply", action="store_true", + help="把 drift 的 entry 直接寫回 .md（exit 0 if applied, 2 if nothing to apply）") + args = parser.parse_args() + + # 找所有 GitHub repo + 它在每個檔案中的標註 stars + # entries: {repo: [(file_path, declared_stars_int, line_no, declared_text)]} + entries: dict[str, list[tuple[Path, int | None, int, str]]] = {} + + for fp in find_md_files(REPO_ROOT): + text = fp.read_text(encoding="utf-8") + # State machine: find `[...](https://github.com/X/Y)`, then locate `★ Xk+`. + # Search order: + # 1. SAME line as the URL (table format like `| repo | ... | ★ 80k+ |` + # and inline bullets like `[repo](url) ★ 6k+ — desc`). + # 2. Fallback: next 12 lines (entry-block format with stars on separate + # line, e.g. `#### [repo](url)\n\n| Stars | ★ 12k+ |`). + # 3. Stop at heading / horizontal-rule boundary to avoid cross-entry leakage. + lines = text.splitlines() + for i, line in enumerate(lines): + m_repo = GITHUB_RE.search(line) + if not m_repo: + continue + repo = normalize_repo(m_repo.group(1), m_repo.group(2)) + if repo is None: + continue + declared = None + declared_text = None + # Step 1: same-line stars first (table / bullet formats) + m_stars = STARS_RE.search(line) + if m_stars: + declared = parse_stars_text(m_stars.group(0)) + declared_text = m_stars.group(0) + else: + # Step 2: fallback to next 12 lines + for j in range(i + 1, min(i + 12, len(lines))): + stripped = lines[j].lstrip() + if stripped.startswith(("### ", "#### ", "## ", "---", "# ")): + break # 撞到下一個 entry 邊界 + m_stars = STARS_RE.search(lines[j]) + if m_stars: + declared = parse_stars_text(m_stars.group(0)) + declared_text = m_stars.group(0) + break + entries.setdefault(repo, []).append((fp, declared, i + 1, declared_text or "(no stars line)")) + + # 去重 repo（每個 repo 只查一次） + unique_repos = list(entries.keys()) + print(f"Found {len(unique_repos)} unique GitHub repos referenced.", file=sys.stderr) + print(f"Querying gh api...", file=sys.stderr) + + # 並行查 stars + actual: dict[str, int | None] = {} + with ThreadPoolExecutor(max_workers=MAX_WORKERS) as ex: + futures = {ex.submit(fetch_stars, r): r for r in unique_repos} + for i, fut in enumerate(as_completed(futures), start=1): + repo = futures[fut] + actual[repo] = fut.result() + if i % 10 == 0: + print(f" {i}/{len(unique_repos)}", file=sys.stderr) + + # 比對 + drift = [] + missing = [] + not_found = [] + + for repo, occurrences in entries.items(): + latest = actual[repo] + if latest is None: + not_found.append(repo) + continue + + for fp, declared, line_no, declared_text in occurrences: + if declared is None: + missing.append((repo, fp, line_no)) + continue + + if declared == 0: + pct_diff = 100 + else: + pct_diff = abs(latest - declared) / declared * 100 + + if pct_diff >= args.threshold: + drift.append(( + repo, fp, line_no, declared, latest, pct_diff, declared_text + )) + + # 報告 + print() + print("=" * 60) + print(f"Total repos checked: {len(unique_repos) - len(not_found)}") + print(f"Drift (>={args.threshold}%): {len(drift)}") + print(f"Missing stars line: {len(missing)}") + print(f"Repo not found (404): {len(not_found)}") + print() + + if drift: + print("=== Drifted entries ===") + for repo, fp, line_no, declared, latest, pct, text in sorted(drift, key=lambda x: -x[5]): + rel = fp.relative_to(REPO_ROOT) + arrow = "↑" if latest > declared else "↓" + print(f" {repo} declared={text} actual={fmt_stars(latest)} " + f"(diff {arrow}{pct:.0f}%) → {rel}:{line_no}") + + if not_found: + print() + print("=== Repos not found / private / renamed ===") + for repo in not_found: + print(f" {repo}") + for fp, _, line_no, _ in entries[repo]: + rel = fp.relative_to(REPO_ROOT) + print(f" {rel}:{line_no}") + + if args.show_missing and missing: + print() + print("=== URLs without stars (could be article / docs / org / curated repo) ===") + # Group by repo so we don't print the same repo 5 times + by_repo: dict[str, list[tuple[Path, int]]] = {} + for repo, fp, line_no in missing: + by_repo.setdefault(repo, []).append((fp, line_no)) + for repo in sorted(by_repo): + latest = actual.get(repo) + star_str = f"★{fmt_stars(latest)}" if latest is not None else "(404)" + occs = by_repo[repo] + print(f" {repo} {star_str} ({len(occs)} occurrence(s))") + for fp, ln in occs[:3]: + rel = fp.relative_to(REPO_ROOT) + print(f" {rel}:{ln}") + if len(occs) > 3: + print(f" ... +{len(occs) - 3} more") + + if args.apply: + # Write-back mode: replace `declared_text` with `★ {new}` in-place. + # Group drift by file so we only do one read/write per file. + by_file: dict[Path, list[tuple[int, str, str]]] = {} + for repo, fp, line_no, declared, latest, pct, text in drift: + # STARS_RE's `\s*` after the digits consumes the trailing space + # for the bare-number case (no k/m suffix), e.g. it matches + # "★ 60 " in "★ 60 — desc". Re-emit that exact trailing + # whitespace so we don't glue the count to the next token + # ("★ 70—"). For "★ 12k+" the match has no trailing ws → no-op. + trail = text[len(text.rstrip()):] + new_text = f"★ {fmt_stars(latest)}" + trail + by_file.setdefault(fp, []).append((line_no, text, new_text)) + + files_changed = 0 + for fp, replacements in by_file.items(): + content = fp.read_text(encoding="utf-8") + lines = content.splitlines(keepends=True) + for line_no, old_text, new_text in replacements: + idx = line_no - 1 + if 0 <= idx < len(lines) and old_text in lines[idx]: + lines[idx] = lines[idx].replace(old_text, new_text, 1) + fp.write_text("".join(lines), encoding="utf-8") + files_changed += 1 + + print() + print(f"=== Applied {len(drift)} drift fixes across {files_changed} files ===") + sys.exit(0 if drift else 2) + + if args.check and (drift or not_found): + # CI 模式：只有 drift 或 404 算失敗。 + # `missing` 是 by design（article / spec / catalog entry 不需要 Stars row，見 style-guide §1） + sys.exit(1) + + +if __name__ == "__main__": + main() diff --git a/scripts/sync-language-switchers.py b/scripts/sync-language-switchers.py new file mode 100644 index 0000000..bbe7058 --- /dev/null +++ b/scripts/sync-language-switchers.py @@ -0,0 +1,482 @@ +#!/usr/bin/env python3 +""" +sync-language-switchers.py — 統一三語檔案頂部的 language switcher + +3 語版 repo 結構： + .md = zh-TW canonical + .en.md = English mirror + .zh-Hans.md = zh-Hans mirror + +每個檔案的頂部都需要 3-way switcher，把目前 active 的語言粗體、其他兩個連結。 + +執行： + python scripts/sync-language-switchers.py # dry run，列出要改的 + python scripts/sync-language-switchers.py --apply # 實際寫入檔案 + python scripts/sync-language-switchers.py --check # CI 用，發現不同步就 exit 1 + +支援兩種 switcher 格式： + 1. README.md 用
...
區塊 + 2. 一般檔案用 `> [...](./...) | **...**` 一行 blockquote +""" + +import argparse +import re +import sys +from pathlib import Path +from typing import Optional + +# Force UTF-8 stdout on Windows (cp950 default can't handle CJK) +if sys.stdout.encoding and sys.stdout.encoding.lower() != "utf-8": + try: + sys.stdout.reconfigure(encoding="utf-8") + except Exception: + pass + +REPO_ROOT = Path(__file__).resolve().parent.parent + +# 哪些 .md 檔需要 switcher（zh-TW canonical） +# 排除：DESIGN.md、CONTRIBUTORS.md、CONTRIBUTING.md（內部 / 規則性質） +# 排除：.github/、scripts/、book/ +EXCLUDE_PATHS = { + "branches/DESIGN.md", + "stages/DESIGN.md", + "CONTRIBUTORS.md", # 名單性質，不一定有 zh-Hans 翻譯 + # Landing pages: mkdocs-only (index.md is the site home; GitHub shows + # README.md, not index.md). They use Material's header language selector, + # so the GitHub-style inline switcher would be broken + ugly here. + "index.md", + "index.en.md", + "index.zh-Hans.md", +} +EXCLUDE_DIRS = {".github", "scripts", "book", ".ai", ".claude", "node_modules", "_build"} + + +def find_paired_files() -> list[tuple[Path, Optional[Path], Optional[Path]]]: + """找所有 (zh-TW, en, zh-Hans) 三檔組合。""" + triples = [] + for md_path in sorted(REPO_ROOT.rglob("*.md")): + # 跳過 .en.md / .zh-Hans.md 自己（它們是 mirror，不是 canonical） + if md_path.name.endswith(".en.md") or md_path.name.endswith(".zh-Hans.md"): + continue + # 跳過 EXCLUDE_DIRS + if any(part in EXCLUDE_DIRS for part in md_path.relative_to(REPO_ROOT).parts): + continue + # 跳過 EXCLUDE_PATHS + rel = md_path.relative_to(REPO_ROOT).as_posix() + if rel in EXCLUDE_PATHS: + continue + + en_path = md_path.with_suffix(".en.md") + zh_hans_path = md_path.with_suffix(".zh-Hans.md") + if not en_path.exists(): + en_path = None + if not zh_hans_path.exists(): + zh_hans_path = None + + # 至少要有 zh-TW + 一個 mirror 才算需要 switcher + if en_path is None and zh_hans_path is None: + continue + + triples.append((md_path, en_path, zh_hans_path)) + return triples + + +def make_readme_switcher(active: str, has_en: bool, has_zh_hans: bool) -> str: + """README.md / README.en.md / README.zh-Hans.md 用的 div 區塊 switcher。""" + parts = [] + label_map = {"zh-TW": "繁體中文", "en": "English", "zh-Hans": "简体中文"} + + for lang in ("zh-TW", "zh-Hans", "en"): + if lang == "zh-Hans" and not has_zh_hans: + continue + if lang == "en" and not has_en: + continue + label = label_map[lang] + if lang == active: + parts.append(f"{label}") + else: + href_map = {"zh-TW": "./README.md", "en": "./README.en.md", "zh-Hans": "./README.zh-Hans.md"} + parts.append(f'{label}') + + return f'
\n {" | ".join(parts)}\n
' + + +def make_inline_switcher(active: str, base_name: str, has_en: bool, has_zh_hans: bool) -> str: + """一般檔 blockquote 一行 switcher。""" + parts = [] + label_map = {"zh-TW": "繁體中文", "en": "English", "zh-Hans": "简体中文"} + + for lang in ("zh-TW", "zh-Hans", "en"): + if lang == "zh-Hans" and not has_zh_hans: + continue + if lang == "en" and not has_en: + continue + label = label_map[lang] + if lang == active: + parts.append(f"**{label}**") + else: + href_map = { + "zh-TW": f"./{base_name}.md", + "en": f"./{base_name}.en.md", + "zh-Hans": f"./{base_name}.zh-Hans.md", + } + parts.append(f"[{label}]({href_map[lang]})") + + return "> " + " | ".join(parts) + + +def detect_switcher_block(content: str) -> tuple[Optional[int], Optional[int], str]: + """ + 找出檔案頂部的 switcher 區塊位置。 + 回傳 (start_line, end_line, kind) 或 (None, None, '')。 + kind = 'div' 或 'inline' 或 ''。 + """ + lines = content.split("\n") + + # 嘗試找
...
區塊 + for i, line in enumerate(lines): + if line.strip().startswith('
'): + # 找
+ for j in range(i, min(i + 5, len(lines))): + if "
" in lines[j]: + return i, j, "div" + break # 找到
但找不到
，放棄 + + # 嘗試找 inline `> [English](./xxx.en.md) | **繁體中文**` 之類 + for i, line in enumerate(lines[:10]): # 只看前 10 行 + if line.startswith("> ") and ("](./") in line and ( + "繁體中文" in line or "English" in line or "简体中文" in line + ): + return i, i, "inline" + + return None, None, "" + + +# Locale tokens that appear inside a switcher line. 繁体中文 (simplified glyphs) +# is included because some old/mirror files wrote "Traditional Chinese" that way. +LOCALE_TOKENS = ("繁體中文", "繁体中文", "简体中文", "English") + +# Known placeholder/stub markers that stand in for a real switcher +# (e.g. `> **繁體中文** | (zh-Hans / en mirror defer 中)`). +STUB_MARKERS = ("mirror defer",) + + +def _rel(path: Path) -> str: + """repo-relative path for logging; falls back to the name for temp/selftest files.""" + try: + return path.relative_to(REPO_ROOT).as_posix() + except ValueError: + return path.name + + +def _first_h1(lines: list[str]) -> Optional[int]: + """Index of the first `# ` H1 line, or None.""" + for i, line in enumerate(lines): + if line.startswith("# "): + return i + return None + + +def _looks_like_switcher(line: str) -> bool: + """ + True if a single line is a switcher-ish line in ANY form — canonical, + old/reordered, no-`./`-prefix, bare (no `> `), or a known stub. This is + deliberately broader than detect_switcher_block (which only recognises the + canonical inline / div forms) so the ADD path can strip whatever switcher- + like junk is already there instead of leaving a double switcher. + + Rule: a known stub marker, OR a line containing `|` plus >= 2 distinct + locale tokens AND at least one switcher-specific shape (a markdown link to + a `.md` mirror, or a **bold** locale = the active-language marker). + + The >=2-locale threshold keeps prose that merely mentions "English" (with + an unrelated `|`) from matching. The extra link/bold requirement keeps a + markdown TABLE row whose cells happen to be locale names — e.g. + `| English | 简体中文 | notes |` — from being misread as a switcher and + silently deleted (a table row has no `](...md)` link and no `**locale**`). + Every real 3-/2-way switcher bolds its active locale and/or links the + others; the only single-locale case (the `mirror defer` placeholder) is + caught by the stub branch above. + """ + s = line.strip() + if not s: + return False + if any(marker in s for marker in STUB_MARKERS): + return True + # strip an optional leading blockquote marker + body = s + if body.startswith(">"): + body = body[1:].lstrip() + if "|" not in body: + return False + if sum(1 for tok in LOCALE_TOKENS if tok in body) < 2: + return False + has_link = re.search(r"\[[^\]]+\]\([^)]*\.md\)", body) is not None + has_bold_locale = any(f"**{tok}**" in body for tok in LOCALE_TOKENS) + return has_link or has_bold_locale + + +def update_file(path: Path, lang: str, has_en: bool, has_zh_hans: bool, apply: bool) -> bool: + """更新單一檔案的 switcher。回傳 True 表示有改動需要寫入。""" + content = path.read_text(encoding="utf-8") + lines = content.split("\n") + + # README 用 div block（位置在檔案最頂、banner/H1 之前），一般檔用 inline。 + # README 的 div 偵測/替換歷史上是正確的；本次回報的 double-switcher 缺陷 + # 全部發生在非-README inline 檔（stages/08-* x3, stages/04-* x2），故 + # README 走原本未改動的 detect→ADD/UPDATE 流程以保證對乾淨 repo 零差異， + # inline 走下方重建頭部的新邏輯（會移除任何 switcher-ish 殘行）。 + is_readme = path.name.startswith("README") + + if is_readme: + new_switcher = make_readme_switcher(lang, has_en, has_zh_hans) + start, end, kind = detect_switcher_block(content) + + if start is None: + for i, line in enumerate(lines): + if line.startswith("# "): + new_lines = ( + lines[: i + 1] + [""] + new_switcher.split("\n") + [""] + lines[i + 1 :] + ) + new_content = "\n".join(new_lines) + if apply: + path.write_text(new_content, encoding="utf-8") + print(f"[ADD] {_rel(path)} ← (no existing switcher, inserted after H1)") + return True + new_lines = new_switcher.split("\n") + [""] + lines + new_content = "\n".join(new_lines) + if apply: + path.write_text(new_content, encoding="utf-8") + print(f"[ADD] {_rel(path)} ← (no existing switcher, inserted at top)") + return True + + old_switcher = "\n".join(lines[start : end + 1]) + if old_switcher.strip() == new_switcher.strip(): + return False + new_lines = lines[:start] + new_switcher.split("\n") + lines[end + 1 :] + new_content = "\n".join(new_lines) + if apply: + path.write_text(new_content, encoding="utf-8") + print(f"[UPDATE] {_rel(path)}:{start + 1}") + print(f" - {old_switcher[:100]}") + print(f" + {new_switcher[:100]}") + return True + + # ---- non-README inline path: rebuild the head, removing ANY switcher-ish + # line(s) and inserting exactly one canonical switcher ---- + name = path.name + if name.endswith(".en.md"): + base_name = name[: -len(".en.md")] + elif name.endswith(".zh-Hans.md"): + base_name = name[: -len(".zh-Hans.md")] + else: + base_name = name[: -len(".md")] + new_switcher = make_inline_switcher(lang, base_name, has_en, has_zh_hans) + + h1 = _first_h1(lines) + if h1 is None: + # no H1: canonical goes at the very top; skip leading blanks / switchers + cs = 0 + while cs < len(lines) and ( + lines[cs].strip() == "" or _looks_like_switcher(lines[cs]) + ): + cs += 1 + sw_idx = [i for i in range(cs) if _looks_like_switcher(lines[i])] + rebuilt = [new_switcher, ""] + lines[cs:] + else: + # first real content = first non-blank, non-switcher line after H1 + cs = h1 + 1 + while cs < len(lines) and ( + lines[cs].strip() == "" or _looks_like_switcher(lines[cs]) + ): + cs += 1 + sw_idx = [i for i in range(h1) if _looks_like_switcher(lines[i])] + [ + i for i in range(h1 + 1, cs) if _looks_like_switcher(lines[i]) + ] + # keep everything up to & including H1 except stray pre-H1 switcher lines + prefix = [lines[i] for i in range(h1 + 1) if i == h1 or not _looks_like_switcher(lines[i])] + rebuilt = prefix + ["", new_switcher, ""] + lines[cs:] + + # already-clean fast path: exactly one switcher-ish line and (stripped) it + # already equals canonical. Mirrors the pre-fix blank-line-agnostic + # "no change" predicate so the 59 currently-clean files stay untouched. + if len(sw_idx) == 1 and lines[sw_idx[0]].strip() == new_switcher.strip(): + return False + + new_content = "\n".join(rebuilt) + if new_content == content: + return False + if apply: + path.write_text(new_content, encoding="utf-8") + + n = len(sw_idx) + rel = _rel(path) + if n == 0: + print(f"[ADD] {rel} ← (no existing switcher, inserted after H1)") + elif n >= 2: + print( + f"[DEDUP] {rel} ← removed {n} switcher-ish line(s), inserted 1 canonical " + f"(was a double/multi switcher)" + ) + else: + print(f"[UPDATE] {rel} ← replaced non-canonical switcher with canonical") + return True + + +def _run_selftest() -> int: + """ + Regression test for the ADD double-switcher defect. Builds throw-away + fixtures reproducing every old/stub shape, runs --apply, and asserts: + * exactly ONE canonical switcher remains (no double), + * head shape is H1 / blank / switcher / blank / content, + * a second --apply is a no-op (idempotent), + * a pre-existing double IS detected as needing change (the --check + blind spot is closed), + * an already-canonical file is left untouched. + """ + import tempfile + + canonical = make_inline_switcher("zh-TW", "08-x", True, True) + canonical_en = make_inline_switcher("en", "04-x", True, True) + canonical_hans = make_inline_switcher("zh-Hans", "08-x", True, True) + + H1 = "# Stage X — Title" + BODY = "⏱ **時間估算**：2-3 週\n\n本文開始。\n" + + # (filename, lang, content, expected_canonical, must_change, must_keep) + # must_keep = a substring that MUST still be present after --apply + # (None = no extra survival assertion) + cases = [ + # stub placeholder, no links + ("08-x.md", "zh-TW", + f"{H1}\n\n> **繁體中文** | (zh-Hans / en mirror defer 中)\n\n{BODY}", + canonical, True, None), + # bare line, no `> ` prefix + ("04-x.en.md", "en", + f"# Stage 4 — Agent Frameworks\n\n" + f"[繁體中文](./04-x.md) | [简体中文](./04-x.zh-Hans.md) | **English**\n\n{BODY}", + canonical_en, True, None), + # `> ` prefix, locale names, but links without `./` + ("08-x.zh-Hans.md", "zh-Hans", + f"{H1}\n\n> [繁体中文](08-x.md) | **简体中文** | [English](08-x.en.md)\n\n{BODY}", + canonical_hans, True, None), + # PRE-EXISTING DOUBLE: canonical + orphan stub (the --check blind spot) + ("08-x.md", "zh-TW", + f"{H1}\n\n{canonical}\n\n> **繁體中文** | (zh-Hans / en mirror defer 中)\n\n{BODY}", + canonical, True, None), + # already-canonical: must NOT change + ("08-x.md", "zh-TW", + f"{H1}\n\n{canonical}\n\n{BODY}", + canonical, False, None), + # P2 regression: a markdown table whose header cells are locale names, + # sitting right after H1, must NOT be eaten as a switcher. (No switcher + # present → ADD inserts canonical; the table header MUST survive.) + ("08-x.md", "zh-TW", + f"{H1}\n\n| English | 简体中文 | notes |\n|---|---|---|\n| a | b | c |\n\n{BODY}", + canonical, True, "| English | 简体中文 | notes |"), + ] + + failures: list[str] = [] + with tempfile.TemporaryDirectory() as td: + for idx, (fname, lang, content, want_sw, must_change, must_keep) in enumerate(cases): + # each case in its own subdir so the filename (hence derived + # base_name / switcher links) stays exactly `fname` + case_dir = Path(td) / f"c{idx}" + case_dir.mkdir() + p = case_dir / fname + p.write_text(content, encoding="utf-8") + + changed1 = update_file(p, lang, True, True, apply=True) + if changed1 != must_change: + failures.append( + f"case{idx} {fname}: first --apply changed={changed1}, expected {must_change}" + ) + + out = p.read_text(encoding="utf-8") + out_lines = out.split("\n") + if must_keep is not None and must_keep not in out: + failures.append( + f"case{idx} {fname}: required line was consumed; missing {must_keep!r}" + ) + sw_lines = [ln for ln in out_lines if _looks_like_switcher(ln)] + if len(sw_lines) != 1: + failures.append( + f"case{idx} {fname}: expected exactly 1 switcher line, got {len(sw_lines)}: {sw_lines!r}" + ) + elif sw_lines[0].strip() != want_sw.strip(): + failures.append( + f"case{idx} {fname}: switcher = {sw_lines[0]!r}, expected {want_sw!r}" + ) + + # head shape: H1 / "" / switcher / "" / content + h1i = _first_h1(out_lines) + if h1i is not None: + shape_ok = ( + len(out_lines) > h1i + 3 + and out_lines[h1i + 1] == "" + and out_lines[h1i + 2].strip() == want_sw.strip() + and out_lines[h1i + 3] == "" + ) + if not shape_ok: + failures.append( + f"case{idx} {fname}: head shape not H1/blank/switcher/blank, got " + f"{out_lines[h1i:h1i + 4]!r}" + ) + + # idempotent: a second --apply must be a no-op + changed2 = update_file(p, lang, True, True, apply=True) + if changed2: + failures.append(f"case{idx} {fname}: second --apply was NOT idempotent") + + if failures: + print("✗ selftest FAILED:") + for f in failures: + print(f" - {f}") + return 1 + print( + f"✓ selftest passed ({len(cases)} cases: stub / bare / no-./ / double / clean / table)." + ) + return 0 + + +def main(): + parser = argparse.ArgumentParser(description="Sync 3-way language switchers across .md / .en.md / .zh-Hans.md") + parser.add_argument("--apply", action="store_true", help="actually write changes (default: dry-run)") + parser.add_argument( + "--check", action="store_true", help="exit 1 if any switcher is out of sync (CI use)" + ) + parser.add_argument( + "--selftest", action="store_true", help="run the double-switcher regression test and exit" + ) + args = parser.parse_args() + + if args.selftest: + sys.exit(_run_selftest()) + + triples = find_paired_files() + print(f"Found {len(triples)} canonical zh-TW files with at least one mirror.\n") + + changed = 0 + for zh_tw, en, zh_hans in triples: + has_en = en is not None + has_zh_hans = zh_hans is not None + + if update_file(zh_tw, "zh-TW", has_en, has_zh_hans, args.apply): + changed += 1 + if has_en and update_file(en, "en", has_en, has_zh_hans, args.apply): + changed += 1 + if has_zh_hans and update_file(zh_hans, "zh-Hans", has_en, has_zh_hans, args.apply): + changed += 1 + + print() + print("=" * 60) + print(f"Total {len(triples)} files inspected, {changed} need updating.") + + if args.check: + sys.exit(1 if changed > 0 else 0) + elif not args.apply and changed > 0: + print("\nDry-run mode. Re-run with --apply to write changes.") + + +if __name__ == "__main__": + main() diff --git a/scripts/zh-hans-localize.py b/scripts/zh-hans-localize.py new file mode 100644 index 0000000..c4edc3e --- /dev/null +++ b/scripts/zh-hans-localize.py @@ -0,0 +1,167 @@ +#!/usr/bin/env python3 +""" +zh-Hans mainland-localization pass — part of the mirror process. + +The zh-Hans mirrors are produced from the zh-TW canonical via opencc +`tw2s` (character-level 繁→簡 only). `tw2s` does NOT localize Taiwan +vocabulary to Mainland vocabulary, so the mirrors read like +"simplified-character Taiwan Chinese". This script applies a CURATED, +blanket-SAFE Taiwan→Mainland substitution table + mainland quote +convention, so the localization PERSISTS across future mirror regens. + +Motivated by + sourced from community PR #18 (Rain120, +"使中文简体更加贴合大陆的阅读习惯"). Each pair below was grep-verified +against the whole zh-Hans corpus to have a single unambiguous meaning +in THIS repo (zero/negligible false positives). + +DELIBERATELY EXCLUDED (context-sensitive — a blanket map corrupts them; +needs OpenCC `tw2sp` curated dict or per-occurrence human judgment): + 预设 → 默认 (software default) vs 假定/假设 (assume) — split by meaning + 教学 → 教程 (tutorial) vs 教学 (teaching/instruction) — split by meaning + 走完/往下走/差别/涵盖 — stylistic, not Taiwan-isms (no localization value) + English `script`/`词` — context-dependent + +Usage: + python scripts/zh-hans-localize.py [--apply] # default = dry-run + python scripts/zh-hans-localize.py --check # exit 1 if any drift + +Only touches *.zh-Hans.md. Skips fenced ``` code blocks and inline +`code` spans (a Taiwan term inside a code sample / string literal must +stay verbatim). +""" +from __future__ import annotations + +import argparse +import io +import re +import sys +from pathlib import Path + +# Curated blanket-safe Taiwan→Mainland vocabulary (grep-verified, this repo) +VOCAB = { + "呼叫": "调用", # programming call/invoke (mainland: 调用) + "印出": "打印", # print output + "出包": "搞砸", # Taiwan slang "screw up" + "软体": "软件", # software + "网路": "网络", # network + "档案": "文件", # file (tech context) + "字串": "字符串", # string (programming) + "函式": "函数", # function (programming) + "程式": "程序", # program / code + "品质": "质量", # quality + "回应": "响应", # response (LLM/API output) + # --- batch 2 (grep-verified, 2026-05; sourced from PR #18 / residue scan) --- + "使用者": "用户", # user (all prose occ = "user"; meta tables PROTECTed) + "命令列": "命令行", # command line + "字元": "字符", # character (text/encoding) + "物件": "对象", # object (programming / 3D — mainland tech standard) + # EXCLUDED on purpose (grep-verified unsafe — do NOT add): + # 介面→界面 : appears in the Stage 8 H1 + README link text → + # changing it shifts the anchor slug & breaks inbound + # `#…操作介面…` links (anchor-corruption risk). + # 影片→视频 : substring of 投影片 (slides) → 投影片→投视频 (wrong). + # 軟體/專案/腳本/連結/設定/飛書/資料/預設 : only occur inside the + # PROTECTed reference docs (nothing to localize). + # 预设/教学 : context-sensitive (default vs assume / tutorial vs + # teaching) — no single mainland equivalent. +} +# Mainland quote convention: 「」 (Japanese/Taiwan corner brackets) +# → “ ” (GB/T fullwidth curly double quotes). +QUOTES = {"「": "“", "」": "”"} + +# Reference / policy docs that INTENTIONALLY contain TW↔Mainland term +# pairs as documentation — substituting there corrupts the reference +# (e.g. `| 使用者 | 用户 |` → `| 用户 | 用户 |`). Mirrors lint.yml's own +# zh-Hans residue-check exclusion list (project-established convention). +PROTECT = { + "resources/style-guide.zh-Hans.md", + "resources/glossary.zh-Hans.md", +} + +REPO = Path(__file__).resolve().parent.parent +FENCE_RE = re.compile(r"```.*?```", re.DOTALL) +INLINE_RE = re.compile(r"`[^`\n]*`") + + +def _mask(text: str): + """Replace fenced + inline code with placeholders so substitutions + never touch code. Returns (masked_text, restore_map).""" + store: list[str] = [] + + def stash(m: re.Match) -> str: + store.append(m.group(0)) + return f"\x00{len(store) - 1}\x00" + + text = FENCE_RE.sub(stash, text) + text = INLINE_RE.sub(stash, text) + return text, store + + +def _unmask(text: str, store: list[str]) -> str: + for i, original in enumerate(store): + text = text.replace(f"\x00{i}\x00", original) + return text + + +def localize(text: str) -> tuple[str, dict[str, int]]: + masked, store = _mask(text) + counts: dict[str, int] = {} + for tw, cn in {**VOCAB, **QUOTES}.items(): + c = masked.count(tw) + if c: + masked = masked.replace(tw, cn) + counts[f"{tw}→{cn}"] = c + return _unmask(masked, store), counts + + +def main() -> int: + if hasattr(sys.stdout, "reconfigure"): + try: + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + except Exception: + pass + ap = argparse.ArgumentParser(description=__doc__) + # --check and --apply are semantically exclusive: --check is the CI + # gate (read-only, exit 1 on drift); --apply writes. Don't combine. + ap.add_argument("--apply", action="store_true", help="write changes (default: dry-run)") + ap.add_argument("--check", action="store_true", help="exit 1 if any file would change (CI gate, read-only)") + args = ap.parse_args() + + files = sorted( + p for p in REPO.rglob("*.zh-Hans.md") + if ".ai" not in p.parts and "node_modules" not in p.parts and "_build" not in p.parts + and p.relative_to(REPO).as_posix() not in PROTECT + ) + total = 0 + changed_files = 0 + agg: dict[str, int] = {} + for fp in files: + src = fp.read_text(encoding="utf-8") + out, counts = localize(src) + if out != src: + changed_files += 1 + n = sum(counts.values()) + total += n + for k, v in counts.items(): + agg[k] = agg.get(k, 0) + v + rel = fp.relative_to(REPO).as_posix() + print(f" {rel}: {n} ({', '.join(f'{k}×{v}' for k, v in counts.items())})") + if args.apply: + fp.write_text(out, encoding="utf-8") + + print() + print(f"=== {total} substitutions across {changed_files} file(s) ===") + for k, v in sorted(agg.items(), key=lambda x: -x[1]): + print(f" {k}: {v}") + if args.check: + print("❌ zh-Hans localization drift detected — run " + "`python scripts/zh-hans-localize.py --apply`" + if total else "✓ zh-Hans localization clean — no drift") + return 1 if total else 0 + if not args.apply: + print("\n(dry-run — pass --apply to write)") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/stages/00-foundations.en.md b/stages/00-foundations.en.md new file mode 100644 index 0000000..07113f2 --- /dev/null +++ b/stages/00-foundations.en.md @@ -0,0 +1,74 @@ +# Stage 0 — Foundations + +> [繁體中文](./00-foundations.md) | [简体中文](./00-foundations.zh-Hans.md) | **English** + + +⏱ **Time estimate**: 1-2 weeks (~5-15 hours, can skip if you have these) + +> 💡 **Don't recognize a term?** Check [`resources/glossary.en.md`](../resources/glossary.en.md) for a 30-second definition. Stage 0 doesn't lean on much jargon, but the next stages do. +> 🗺️ **Want the big picture of the agent landscape first** (why some agents live in a terminal, some in Telegram, some on a Jetson board)? → [`resources/agent-paradigms.en.md`](../resources/agent-paradigms.en.md) (5 agent paradigms, ~10 min read) + +## When to skip this stage + +If you can: +- Write a Python function that calls a public API and parses JSON response +- Use git to clone, commit, push, and resolve a basic merge +- Use the command line on your OS (cd, ls, mkdir, run a script) +- Read a YAML / JSON file without confusion + +→ **Skip directly to [Stage 1](01-llm-basics.md)**. + +If you can't, work through this stage. Don't skip — every later stage assumes these. + +## 📌 Learning Goals + +- Write Python: functions, classes, async/await basics +- Use git: clone, branch, commit, push, basic conflict resolution +- Use REST APIs: send GET/POST, parse JSON, handle auth headers +- Read & write YAML and JSON + +## 🛠 Hands-on Exercises + +- **Exercise: Python** — write a Python script that calls https://api.github.com/users/torvalds and prints follower count +- **Exercise: git** — clone any public repo, make a commit, push to your fork +- **Exercise: CLI** — make a small directory tree with the command line (macOS / Linux: `mkdir project && cd project && mkdir src tests docs`; Windows PowerShell: `New-Item -ItemType Directory -Path project,project\src,project\tests,project\docs`), run a Python script, redirect output to a file +- **Exercise: YAML** — read a `.yaml` config file in Python, modify a value, write it back +- **Exercise: API auth** — at [github.com/settings/tokens](https://github.com/settings/tokens) generate a personal access token (minimal scope: `read:user`), call the auth-required `https://api.github.com/user` endpoint, observe 401 (no token) vs 200 (with token). Note: real production agents always use API auth — do this exercise + +## 🎯 Curated Resources (not full projects, just learning material) + +### Python +- [**Python Crash Course**](https://github.com/ehmatthes/pcc_3e) — book + exercises (paid book, free exercises) +- [**Real Python tutorials**](https://realpython.com/) — high-quality free articles +- [**Corey Schafer YouTube**](https://www.youtube.com/c/Coreyms) — video tutorials, beginner to advanced, very clear delivery +- [**Boot.dev**](https://www.boot.dev/) — interactive Python course (partially free) +- [**runoob.com Python tutorial**](https://www.runoob.com/python3/python3-tutorial.html) — Chinese-language Python intro reference + +### Git +- [**Pro Git book**](https://git-scm.com/book/en/v2) — free, full-length reference +- [**Atlassian Git Tutorials**](https://www.atlassian.com/git/tutorials) — workflow-focused +- [**Oh Shit, Git!?!**](https://ohshitgit.com/) — when things go wrong +- [**git-flight-rules**](https://github.com/k88hudson/git-flight-rules) — "I screwed up X, how do I undo?" — popular cheat sheet + +### CLI / Shell +- [**The Art of Command Line**](https://github.com/jlevy/the-art-of-command-line) — beginner-to-advanced command-line skills (180k+ stars, multi-language) +- [**Learn Shell**](https://www.learnshell.org/) — interactive Bash tutorial +- [**explainshell.com**](https://explainshell.com/) — break down any shell command (debug life-saver) + +### REST APIs +- [**MDN — HTTP**](https://developer.mozilla.org/en-US/docs/Web/HTTP) — protocol fundamentals +- [**Postman Learning Center**](https://learning.postman.com/) — API exploration tool +- [**HTTPie**](https://github.com/httpie/cli) — friendlier-than-`curl` command-line HTTP client + +### YAML / JSON +- [**YAML official site**](https://yaml.org/) — spec +- [**JSON crash course**](https://www.json.org/json-en.html) — official quick guide +- [**jq**](https://github.com/jqlang/jq) — command-line JSON processor (heavy use in agent workflows) + +## Why this stage exists + +Most "AI agent" tutorials assume you already have these. If you don't, you'll get blocked at random places (tools requires async; configs are YAML; SDK setup needs git). One week investing here saves 10+ weeks of frustration later. + +--- + +> ✅ **Done with Stage 0?** Next, [**Stage 1 — LLM Fundamentals**](01-llm-basics.en.md) takes 5-8 hours to walk you through your first LLM API call, the meaning of token / context window / temperature, and how to estimate real task cost via per-token pricing. **Keep going →** diff --git a/stages/00-foundations.md b/stages/00-foundations.md new file mode 100644 index 0000000..7f7f616 --- /dev/null +++ b/stages/00-foundations.md @@ -0,0 +1,71 @@ +# Stage 0 — 基礎準備（Foundations） + +> **繁體中文** | [简体中文](./00-foundations.zh-Hans.md) | [English](./00-foundations.en.md) + +⏱ **時間估算**：1-2 週（約 5-15 小時，已具備可跳過） + +> 💡 **看不懂某個詞**？翻 [`resources/glossary.md`](../resources/glossary.md) 查 30 秒再回來。Stage 0 還不會碰太多 jargon，但接下來幾 stage 會。 +> 🗺️ **想先看 agent 的全景地圖**（為什麼有的 agent 在 terminal、有的在 Telegram、有的在 Jetson 板子）？→ [`resources/agent-paradigms.md`](../resources/agent-paradigms.md)（5 種 agent 型態，10 min 讀完） + +> 📋 **本章組成**：跳過條件檢查 → 環境設定步驟 → 進入 Stage 1（foundation stage，無「學習目標 / 進入條件」框架） +> 🔑 **關鍵名詞**：見 [`resources/glossary.md`](../resources/glossary.md)（每 stage 用到的術語都收在那裡） + +## 何時可以跳過這個階段 + +如果你能： +- 寫一個會呼叫公開 API 並解析 JSON 回應的 Python 函式 +- 用 git 做 clone、commit、push，並處理基本的 merge 衝突 +- 在自己的作業系統上使用命令列（cd、ls、mkdir、執行 script） +- 看懂 YAML / JSON 檔案 + +→ **直接跳到 [Stage 1](01-llm-basics.md)**。 + +如果做不到，就把這個階段走完。不要跳——後面每個階段都會預設你已經會這些。 + +## 📌 學習目標 + +- 寫 Python：函式、類別、async/await 基本用法 +- 用 git：clone、branch、commit、push、基本衝突處理 +- 用 REST API：發 GET/POST、解析 JSON、處理 auth header +- 讀寫 YAML 跟 JSON + +## 🛠 動手練習 + +- **練習：Python** — 寫一個 Python script 呼叫 https://api.github.com/users/torvalds 並印出 follower 數量 +- **練習：git** — clone 任何一個公開 repo，做一次 commit，push 到自己的 fork +- **練習：CLI** — 用命令列建幾個資料夾跟檔案（macOS / Linux：`mkdir project && cd project && mkdir src tests docs`；Windows PowerShell：`New-Item -ItemType Directory -Path project,project\src,project\tests,project\docs`）、執行 Python script、把輸出存到檔案 +- **練習：YAML** — 用 Python 讀一個 `.yaml` 設定檔，改一個值，再寫回去 +- **練習：API auth** — 去 [github.com/settings/tokens](https://github.com/settings/tokens) 產生一個 personal access token（給最少權限：`read:user`），呼叫 `https://api.github.com/user` 需 auth 的 endpoint，看 401（無 token）vs 200（帶 token）的差別。注意：production agent 一定會用到 API auth，所以這一題要做 + +## 🎯 精選資源（不是完整 Project，只是學習素材） + +按 5 個 prereq 主題分類、18 個資源一張表搞定。**挑入口看「適合誰」、想深入點連結看 repo / 網站**。 + +| 主題 | 資源 | 適合誰 | 為什麼推薦 / 備註 | +|---|---|---|---| +| **Python** | [Python Crash Course](https://github.com/ehmatthes/pcc_3e) | 從零學 Python | 書 + 練習；書付費、練習免費 | +| | [Real Python](https://realpython.com/) | 已會基礎、想深入單一主題 | 高品質免費文章、Google 搜尋常出現 | +| | [Corey Schafer YouTube](https://www.youtube.com/c/Coreyms) | 喜歡英文影片學習者 | 從基礎到進階、講解清楚 | +| | [Boot.dev](https://www.boot.dev/) | 想要互動式練習 | 部分免費、付費含完整 backend 路線 | +| | [runoob.com Python 教學](https://www.runoob.com/python3/python3-tutorial.html) | 中文讀者快速查語法 | 中文 Python 入門參考 | +| **Git** | [Pro Git book](https://git-scm.com/book/en/v2) | 想徹底搞懂 Git | 免費完整參考書、官方推薦 | +| | [Atlassian Git Tutorials](https://www.atlassian.com/git/tutorials) | 想學 workflow（branch / merge / rebase）| 以 workflow 為主、視覺化好 | +| | [Oh Shit, Git!?!](https://ohshitgit.com/) | 出包時急救 | 「我搞砸了 X、怎麼救」cheat sheet | +| | [git-flight-rules](https://github.com/k88hudson/git-flight-rules) | 想要更深的急救手冊 | 高人氣 cheat sheet、覆蓋場景更廣 | +| **CLI / Shell** | [The Art of Command Line](https://github.com/jlevy/the-art-of-command-line) | 想系統性學命令列 | ★ 160k+、多語言版、新手到進階都涵蓋 | +| | [Learn Shell](https://www.learnshell.org/) | 喜歡互動式練習 | 互動式 Bash 教學、瀏覽器內跑 | +| | [explainshell.com](https://explainshell.com/) | debug shell 指令 | 把任何 shell 指令拆解講解（debug 救星）| +| **REST API** | [MDN — HTTP](https://developer.mozilla.org/en-US/docs/Web/HTTP) | 想搞懂 HTTP 協定 | 最 canonical 的 web reference | +| | [Postman Learning Center](https://learning.postman.com/) | 用 GUI 探索 API | API 探索工具、視覺化好 | +| | [HTTPie](https://github.com/httpie/cli) | 偏好 CLI、`curl` 太醜 | 比 `curl` 友善的命令列 HTTP client | +| **YAML / JSON** | [YAML 官網](https://yaml.org/) | 需要查語法規格 | YAML 規格文件 | +| | [JSON crash course](https://www.json.org/json-en.html) | 第一次接觸 JSON | 官方快速指南 | +| | [jq](https://github.com/jqlang/jq) | 命令列處理 JSON | agent 工作中常用、處理 API response 必備 | + +## 為什麼有這個階段 + +大多數「AI agent」教學都預設你已經會這些。如果你還沒，就會在奇怪的地方卡關（tool 需要 async、設定檔是 YAML、SDK 安裝要用 git）。在這裡花一週的投資，可以省下後面 10 週以上的挫折。 + +--- + +> ✅ **走完 Stage 0 了？** 接下來 [**Stage 1 — LLM 基礎**](01-llm-basics.md) 會用 5-8 小時帶你做完第一次 LLM API 呼叫、認識 token / context window / temperature，以及用 per-token 計價估算實際任務成本。**繼續往下走 →** diff --git a/stages/00-foundations.zh-Hans.md b/stages/00-foundations.zh-Hans.md new file mode 100644 index 0000000..327114a --- /dev/null +++ b/stages/00-foundations.zh-Hans.md @@ -0,0 +1,73 @@ +# Stage 0 — 基础准备（Foundations） + +> [繁體中文](./00-foundations.md) | **简体中文** | [English](./00-foundations.en.md) + +⏱ **时间估算**：1-2 周（约 5-15 小时，已具备可跳过） + +> 💡 **看不懂某个词**？翻 [`resources/glossary.zh-Hans.md`](../resources/glossary.zh-Hans.md) 查 30 秒再回来。Stage 0 还不会碰太多 jargon，但接下来几 stage 会。 +> 🗺️ **想先看 agent 的全景地图**（为什么有的 agent 在 terminal、有的在 Telegram、有的在 Jetson 板子）？→ [`resources/agent-paradigms.zh-Hans.md`](../resources/agent-paradigms.zh-Hans.md)（5 种 agent 型态，10 min 读完） + +## 何时可以跳过这个阶段 + +如果你能： +- 写一个会调用公开 API 并解析 JSON 响应的 Python 函数 +- 用 git 做 clone、commit、push，并处理基本的 merge 冲突 +- 在自己的操作系统上使用命令行（cd、ls、mkdir、执行 script） +- 看懂 YAML / JSON 文件 + +→ **直接跳到 [Stage 1](01-llm-basics.zh-Hans.md)**。 + +如果做不到，就把这个阶段走完。不要跳——后面每个阶段都会预设你已经会这些。 + +## 📌 学习目标 + +- 写 Python：函数、类、async/await 基本用法 +- 用 git：clone、branch、commit、push、基本冲突处理 +- 用 REST API：发 GET/POST、解析 JSON、处理 auth header +- 读写 YAML 跟 JSON + +## 🛠 动手练习 + +- **练习：Python** — 写一个 Python script 调用 https://api.github.com/users/torvalds 并打印 follower 数量 +- **练习：git** — clone 任何一个公开 repo，做一次 commit，push 到自己的 fork +- **练习：CLI** — 用命令行建几个文件夹跟文件（macOS / Linux：`mkdir project && cd project && mkdir src tests docs`；Windows PowerShell：`New-Item -ItemType Directory -Path project,project\src,project ests,project\docs`）、执行 Python script、把输出存到文件 +- **练习：YAML** — 用 Python 读一个 `.yaml` 配置文件，改一个值，再写回去 +- **练习：API auth** — 去 [github.com/settings/tokens](https://github.com/settings/tokens) 产生一个 personal access token（给最少权限：`read:user`），调用 `https://api.github.com/user` 需 auth 的 endpoint，看 401（无 token）vs 200（带 token）的差别。注意：production agent 一定会用到 API auth，所以这一题要做 + +## 🎯 精选资源（不是完整 Project，只是学习素材） + +### Python +- [**Python Crash Course**](https://github.com/ehmatthes/pcc_3e) — 书 + 练习（书要付费，练习免费） +- [**Real Python tutorials**](https://realpython.com/) — 高质量免费文章 +- [**Corey Schafer YouTube**](https://www.youtube.com/c/Coreyms) — 视频教学，从基础到进阶，讲解清楚 +- [**Boot.dev**](https://www.boot.dev/) — 互动式 Python 课程（部分免费） +- [**runoob.com Python 教程**](https://www.runoob.com/python3/python3-tutorial.html) — 中文 Python 入门参考 + +### Git +- [**Pro Git book**](https://git-scm.com/book/en/v2) — 免费完整参考书 +- [**Atlassian Git Tutorials**](https://www.atlassian.com/git/tutorials) — 以 workflow 为主 +- [**Oh Shit, Git!?!**](https://ohshitgit.com/) — 搞砸时的救命手册 +- [**git-flight-rules**](https://github.com/k88hudson/git-flight-rules) — “我搞砸了 X，怎么救？”高人气 cheat sheet + +### CLI / Shell +- [**The Art of Command Line**](https://github.com/jlevy/the-art-of-command-line) — 涵盖从新手到进阶的命令行技巧（180k+ stars，多语言版） +- [**Learn Shell**](https://www.learnshell.org/) — 互动式 Bash 教学 +- [**explainshell.com**](https://explainshell.com/) — 把任何 shell 指令拆解讲解（debug 救星） + +### REST API +- [**MDN — HTTP**](https://developer.mozilla.org/en-US/docs/Web/HTTP) — 协定基础 +- [**Postman Learning Center**](https://learning.postman.com/) — API 探索工具 +- [**HTTPie**](https://github.com/httpie/cli) — 比 `curl` 友善的命令行 HTTP client + +### YAML / JSON +- [**YAML 官网**](https://yaml.org/) — 规格 +- [**JSON crash course**](https://www.json.org/json-en.html) — 官方快速指南 +- [**jq**](https://github.com/jqlang/jq) — 命令行 JSON 处理工具（agent 工作中常用） + +## 为什么有这个阶段 + +大多数“AI agent”教学都预设你已经会这些。如果你还没，就会在奇怪的地方卡关（tool 需要 async、配置文件是 YAML、SDK 安装要用 git）。在这里花一周的投资，可以省下后面 10 周以上的挫折。 + +--- + +> ✅ **走完 Stage 0 了？** 接下来 [**Stage 1 — LLM 基础**](01-llm-basics.zh-Hans.md) 会用 5-8 小时带你做完第一次 LLM API 调用、认识 token / context window / temperature，以及用 per-token 计价估算实际任务成本。**继续往下走 →** diff --git a/stages/01-llm-basics.en.md b/stages/01-llm-basics.en.md new file mode 100644 index 0000000..89fbd74 --- /dev/null +++ b/stages/01-llm-basics.en.md @@ -0,0 +1,498 @@ +# Stage 1 — LLM Fundamentals + +> [繁體中文](./01-llm-basics.md) | [简体中文](./01-llm-basics.zh-Hans.md) | **English** + + +⏱ **Time estimate**: 1 week (~5-8 hours) + +> 👋 **Coming from [Stage 0](00-foundations.en.md)?** Nice — your toolchain is set. The next 5-8 hours: your first working call to Claude / GPT / Gemini, how token / context window / temperature shape the output, and per-token cost estimation. **Jumped straight here?** Make sure you can run a Python script and have an API key from one provider — if not, head back to [Stage 0](00-foundations.en.md). + +> 💡 **Don't recognize a term?** (LLM / token / context window / temperature / RAG / agent / …) → check [`resources/glossary.en.md`](../resources/glossary.en.md) for 30-second definitions. + +### 3 Core Terms (memorize these—all later stages use them) + +| Term | Chinese | One-liner | +|---|---|---| +| **token** | 詞元 | the unit LLMs use to count text length and price (1 Chinese char ≈ 1.5-2 tokens; 1 English word ≈ 1.3 tokens) | +| **context window** | 上下文視窗 | How many tokens the model sees at once (Claude 1M / GPT 1.05M / Gemini 2M) | +| **temperature** | 隨機程度參數 | Controls how stable or creative the output is (0 = deterministic, 1 = creative; use 0.0-0.3 for classification, 0.7-1.0 for creative writing) | + +→ These 3 terms run through every later stage. The goal of Stage 1 is to call the API yourself and feel firsthand how they shape the output. + +> 🧠 **Why you can tune temperature: first, next-token**: an LLM's core action is **predicting the next token**. It computes a probability distribution over the next token and **samples** one from it. `temperature` and `top_p` reshape that distribution: low temperature makes it sharper, almost always picking the most likely token (stable, reproducible); high temperature flattens it, so it more readily picks rare tokens (creative but prone to wandering). `max_tokens` just caps how many samples before stopping. So these are not magic knobs; they control *how the model picks tokens from a probability distribution*. + +## 📌 Learning Goals + +After this stage you will be able to: +- Explain what an LLM is, what tokens are, and what context window means +- Make your first API call to Claude / GPT / Gemini and parse the response +- Compare the four major LLM families (Claude / GPT / Gemini / Llama) on strengths +- Estimate cost per task using per-token pricing + +## 🌐 Major LLM Family Comparison (2026-05 snapshot) + +"How is Claude different from GPT?" "Can I use Chinese models?" "Which OSS model should I run with Ollama?" This section gives you an **objective side-by-side view**. It does not declare a single "best" model: it compares **strengths / good-fit tasks / weaknesses** and includes **official docs URLs** so you can verify the claims yourself. + +> 💡 **First, a few terms**: +> - **Context window** = the amount of conversation an LLM can remember in one pass; it is capped (for example, 200k tokens ~= 150k Chinese characters) +> - **Apache 2.0 / MIT** = open-source terms that permit commercial use, modification, and closed-source redistribution; **Llama Community License** = open-source but with conditions (for example, orgs with >= 700M MAU need a license) +> - **Frontier model** = each provider's strongest flagship; **OSS** = open-source, with weights downloadable for self-hosting + +### 🇺🇸 US Commercial Frontier (3 providers) + +These 3 are SaaS APIs: you pay per token and cannot self-host them. + + + +| Model family | Flagship (2026-07) | Context | Strengths | Best for | Official docs | +|---|---|---|---|---|---| +| **Claude** (Anthropic) | Opus 4.8 / Sonnet 5 / Haiku 4.5 | 1M | long-form / coding / agent / safety alignment | writing papers / code review / agent runtime | [platform.claude.com/docs](https://platform.claude.com/docs/en/about-claude/models/overview) | +| **GPT** (OpenAI) | GPT-5.6 Sol / Terra / Luna | 1.05M | general-purpose / function calling / broadest ecosystem | broad queries / function-call frameworks / GPTs ecosystem | [platform.openai.com/docs/models](https://platform.openai.com/docs/models) | +| **Gemini** (Google) | 3.5 Flash / 3.5 Pro (in dev) | 2M | long context / native multimodal / Google integration | PDF / video and audio / large document sets / Google Workspace | [ai.google.dev](https://ai.google.dev/gemini-api/docs/models/gemini) | + +> **Note**: `(in dev)` = not released yet. Claude **Fable 5** (Mythos-class, above Opus, $10/$50) was suspended on 2026-06-12 by a US export-control directive, but **the controls were lifted 2026-06-30 and Fable 5 was redeployed globally on 2026-07-01** (Claude Platform / Claude Code / Cowork; API rollout in progress) — it is again the highest Claude tier (Opus 4.8 is the Opus-class flagship). Context is the flagship's ceiling: Gemini Pro series 2M, Flash 1M; Claude 1M (Haiku 4.5 is 200k); all three GPT-5.6 tiers are 1.05M. Also, **Sonnet 5** (launched 2026-06-30) is the current Sonnet: 1M context, fast, and cheaper than Opus ($3/$15 vs Opus $5/$25). **GPT-5.6** (launched 2026-07) comes in three tiers: **Sol** flagship ($5/$30), **Terra** balanced ($2.50/$15), **Luna** fastest and cheapest ($1/$6) — available in ChatGPT, Codex, and the API. + +### 🇨🇳 Chinese Commercial + Open-Source Frontier (7 providers) + +These are the main choices for Chinese-language work, in two groups: **API-only** (cloud, paid, can't self-host) and **open weights** (can run on your own machine). + +**① API-only (cloud, mostly paid)** + +| Model family | Flagship (2026-05) | Context | Strengths | Best for | Official | +|---|---|---|---|---|---| +| **DeepSeek** | V3 (`deepseek-chat`) / R1 (`deepseek-reasoner`) | 128k | reasoning / coding / **lowest cost** | high-token workloads / code generation / math | [api-docs.deepseek.com](https://api-docs.deepseek.com/zh-cn/) | +| **Kimi** (Moonshot) | K2.6 multimodal + Agent | **very long 1M+** | long context / Chinese long-form writing | whole-book reading / literature triage | [platform.moonshot.cn](https://platform.moonshot.cn/) | +| **Hunyuan** (Tencent) | T1 (deep-thinking) + TurboS | 128k | **DeepSeek R1-comparable reasoning**, Chinese | Chinese reasoning / Tencent ecosystem | [hunyuan.tencent.com](https://hunyuan.tencent.com/) | +| **MiniMax** | abab6.5 + M2.7 | 200k | multimodal / Chinese long prose | Chinese writing / video and audio multimodal | [platform.minimax.io](https://platform.minimax.io/) | + +> **Note**: This group is mostly cloud-API and proprietary. DeepSeek also has some open weights (on HF), but its V4 consumer API isn't fully public yet, so the API is still the main way to use it. + +**② Open weights (self-hostable)** + +| Model family | Flagship (2026-05) | Context | Strengths | Best for | Official | +|---|---|---|---|---|---| +| **Qwen** (Alibaba) | Qwen3 | 128k+ | **strongest Chinese OSS** / multimodal / agent | Chinese long-form writing / agent / self-host | [qwen.ai](https://qwen.ai/) · [DashScope](https://help.aliyun.com/zh/dashscope/) | +| **GLM** (Zhipu) | GLM-5 / GLM-5.1 | 128k | Chinese / tool use / agent | Chinese agents / multi-turn chat | [open.bigmodel.cn](https://open.bigmodel.cn/) · [chatglm.cn](https://chatglm.cn/) | +| **Yi** (01.AI / Kai-Fu Lee) | Yi-Lightning / Yi-34B-Chat | 200k | **Chinese OSS** alternative to Llama | Chinese self-host / Chinese API | [01.ai](https://01.ai/) · [GitHub](https://github.com/01-ai/Yi) | + +> **Note**: All three offer both an **Apache 2.0 open version and a paid cloud API** (GLM's open version is 5.1). The open versions run on your own machine via [Ollama](https://ollama.com/). + +> ⚠️ **Xiaomi MiMo** is listed in [`resources/cli-agents-guide.md`](../resources/cli-agents-guide.md) for Hermes Agent routing, but as of 2026-05 there is no authoritative official source to verify it, so it is not included in this table. To try it, connect through [Hermes Agent](https://github.com/NousResearch/hermes-agent) 200+ provider routing. + +### 🌍 Western Open-Source (4 providers, self-host defaults) + +These are the main choices for running on your own hardware, avoiding API fees, or handling privacy-sensitive work. You can install them in one command through [Ollama](https://ollama.com/). + +| Model family | Active size | License | Strengths | Best for | Official | +|---|---|---|---|---|---| +| **Llama** (Meta) | 3.3 70B | Llama Community License | general-purpose / broadest ecosystem / Ollama default | self-hosting intro / fine-tune base | [llama.com](https://www.llama.com/) · [HF Meta](https://huggingface.co/meta-llama) | +| **Gemma** (Google) | Gemma 4 26B MoE + 31B dense | Apache 2.0 | **small and efficient** / strong Apple MLX integration / multimodal | edge / mobile / 4-8 GB RAM machines | [ai.google.dev/gemma](https://ai.google.dev/gemma) | +| **Mistral** (Mistral AI) | 7B / Mixtral 8x7B / Codestral | Apache 2.0 (OSS parts) | strongest open-source 7B class | commercial self-host / EU sovereignty | [mistral.ai](https://mistral.ai/) · [HF Mistral](https://huggingface.co/mistralai) | +| **Phi** (Microsoft) | Phi-4 14B + multimodal | MIT | **small but strong** / reasoning / edge-friendly | 4 GB+ RAM / mobile / reasoning intro | [HF microsoft](https://huggingface.co/microsoft) | + +> **Note**: Llama 4 hadn't shipped as of 2026-05 (the table shows 3.3); Gemma 4 was released 2026-04, ranked #3 on LMArena's open-weights board; Phi-4 also has a multimodal version. + +### 🎯 Which One Should I Pick? (by scenario) + +| Your scenario | Pick + why | +|---|---| +| First time learning an LLM API, prioritize complete tutorials | **Claude** — Anthropic Cookbook + Courses are widely considered the most complete | +| Long-form writing / papers / code review | **Claude Sonnet** — long-form prose is a core strength | +| Multimodal (PDF / video and audio / images) | **Gemini** or **Kimi** — native multimodal | +| Broad queries + function calling frameworks | **GPT** — broadest ecosystem and deepest SDK integration | +| **Chinese scenarios + commercial API** | **Kimi** (strong long context; can fit whole books), **DeepSeek** (lowest cost), or **GLM** (agent-friendly) | +| **Chinese scenarios + open-source self-host** | **Qwen 3** (Apache 2.0; currently the strongest Chinese OSS) | +| Reasoning / math (reasoning model) | **DeepSeek R1** / **Hunyuan T1** / **OpenAI o-series** | +| Privacy / offline / no API fees | **Llama 3.3** / **Gemma 4** / **Qwen 3 OSS** via [Ollama](https://ollama.com/) | +| Edge / 4 GB RAM machine | **Gemma 4** / **Phi-4** / **Qwen 3 (`qwen3-3B` or smaller variants)** | +| 100k+ token large documents | **Gemini 3.1** (2M context) or **Kimi K2.6** (1M+) | +| **Want the lowest cost** (API-bill sensitive) | **DeepSeek V4-Flash** — lowest token price among same-tier English models | + +### 📊 Neutral Benchmark Resources (verify for yourself; do not rely on one source) + +| Resource | Use | URL | 2026-05 status | +|---|---|---|---| +| **Artificial Analysis** | Third-party benchmarks plus price/latency aggregation, including Chinese models | https://artificialanalysis.ai/ | ✓ Active | +| **Arena AI** (formerly LMSYS Chatbot Arena) | Human blind-test ELO leaderboard | https://arena.ai/leaderboard/text | ✓ Active | +| **Vellum LLM leaderboard** | Aggregates multiple benchmarks | https://www.vellum.ai/llm-leaderboard | ✓ Active | +| **HuggingFace OpenLLM Leaderboard** | Open-source model rankings | https://huggingface.co/spaces/open-llm-leaderboard | ⚠️ Occasional runtime errors as of 2026-05; use the [Arena AI](https://arena.ai/) open-source tab as fallback | +| **SuperCLUE** | Authoritative benchmark for Chinese-language scenarios | https://www.superclueai.com/ | ✓ Active | + +### ⚠️ Important Caveats + +- ⚠️ **Benchmark != production performance**: run a small eval on your specific task (for example, paste 10 real prompts and see which model answers closest to what you need); **do not pick only from rankings** +- ⚠️ **Frontier changes every 6 months**: all numbers above are a **2026-05 snapshot**; afterward, rely on **official docs** / [Artificial Analysis](https://artificialanalysis.ai/) +- ⚠️ **"Strength" is relative, not absolute**: every frontier model can handle basic tasks; differences matter at the margin +- ⚠️ **For Chinese scenarios, check [SuperCLUE](https://www.superclueai.com/)**: general international benchmarks such as MMLU are English-heavy, and Chinese-language performance may diverge + +## 🚪 Entry Conditions + +You should already: +- Be able to run a Python script +- Know what HTTP / REST is conceptually +- Have an API key from at least one provider (Anthropic / OpenAI / Google) + +If not — go back to Stage 0 first. + +## 📚 Required Reading + +1. [**Anthropic — Claude Model Overview**](https://docs.claude.com/en/about-claude/models/overview) — official model family overview, including 2026's Claude Fable 5 (`claude-fable-5`, Mythos-class, GA 2026-06-09) plus Opus 4.8 / Sonnet 5 / Haiku 4.5. **Fable 5 and its sibling Mythos 5 (`claude-mythos-5`) were suspended on 2026-06-12 by a US export-control directive; the controls were lifted 2026-06-30 and [Fable 5 was redeployed globally on 2026-07-01](https://www.anthropic.com/news/redeploying-fable-5) (with a new safety classifier), while Mythos 5 was restored only for approved US organizations. Fable 5 is again the highest Claude tier; Opus 4.8 is the Opus-class flagship.** +2. [**anthropics/courses — Anthropic API Fundamentals**](https://github.com/anthropics/courses) ⭐⭐⭐⭐⭐ ★ 21k+ — Anthropic's official 5-course umbrella; **module 1 "Anthropic API Fundamentals" maps to this stage**. Jupyter notebooks, runs on Claude 3 Haiku (cheapest), hands-on walkthrough of API essentials. +3. [**OpenAI Quickstart**](https://platform.openai.com/docs/quickstart) — first API call walkthrough +4. [**A Visual Guide to LLM Tokenizers**](https://huggingface.co/learn/llm-course/chapter6/1) — Hugging Face's intro +5. [**Anthropic API Pricing**](https://www.anthropic.com/pricing#anthropic-api) — read the pricing table, calculate cost for 1k input + 1k output + +## 🛠 Hands-on Exercises (foundational, illustrative) + +> 🦙 **This stage defaults to Ollama** (cost-driven; `gemma4:e4b` runs locally for $0/run). Every exercise has Path A (Ollama, default) + Path B (Anthropic, optional — use it when you want to see cloud-quality answers). Full three-path trade-off in [`examples/README.en.md`](../examples/README.en.md#three-paths--default-is-ollama-cost-driven). +> +> 💰 **Stage 1 budget estimate** (all 6 exercises, 3-5 runs each): **all local = $0**, **all haiku ≈ $0.30**, **all sonnet ≈ $0.90**. Full model list + Stage 1-7 total budget: [`examples/README.en.md#recommended-llm-list`](../examples/README.en.md#recommended-llm-list). +> +> 💡 **No Ollama yet?** Each exercise also ships a Path B Anthropic version — pick one. To enable Path A in one step: [`pip install openai && ollama pull gemma4:e4b`](https://ollama.com). + +### Exercise 1: LLM API (hello world) +Five-line Python script that calls an LLM and prints the response. **Defaults to local Ollama (free, offline)**; switch to Path B Anthropic when you want cloud-quality answers. Details in [`examples/README.en.md`](../examples/README.en.md#three-paths--default-is-ollama-cost-driven). + +
+📋 Starter code — Path A (local Ollama gemma4:e4b, default) (copy to practice_1.py and run python practice_1.py) + +```python +# Requires: pip install openai (OpenAI-compatible SDK talks to Ollama) +# Pre-req: ollama pull gemma4:e4b && ollama serve +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI( + base_url="http://localhost:11434/v1", + api_key="ollama", # Ollama doesn't check this — anything works +) + +r = client.chat.completions.create( + model="gemma4:e4b", # swap to qwen2.5:3b / llama3.2:3b if preferred + max_tokens=100, + messages=[{"role": "user", "content": "Introduce yourself in one sentence."}], +) + +# === Self-check === +text = r.choices[0].message.content +print("Response:", text) +print("usage:", r.usage) + +assert r.choices[0].finish_reason in ("stop", "length"), f"unexpected finish_reason: {r.choices[0].finish_reason}" +assert len(text) > 0, "response should not be empty" +assert r.usage.completion_tokens > 0, "output token count should be > 0" +print("✅ Exercise 1 passed — local Ollama gemma4:e4b answered for $0") +``` + +**How slow?** Gemma 4B on CPU: ~5-30 s/answer; on GPU (RTX 3060+): <2 s. For speed use `gemma3:1b`; for quality use `qwen2.5:14b` / `llama3.3:8b` (needs 8 GB+ VRAM). + +
+ +
+📋 Starter code — Path B (Anthropic API, optional, when you want cloud quality) (copy to practice_1_anthropic.py) + +```python +# Requires: pip install anthropic +# Env: export ANTHROPIC_API_KEY=sk-ant-... +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic + +client = anthropic.Anthropic() +msg = client.messages.create( + model="claude-haiku-4-5", # haiku = cheapest; switch to sonnet by changing this line + max_tokens=100, + messages=[{"role": "user", "content": "Introduce yourself in one sentence."}], +) + +# === Self-check === +text = msg.content[0].text +print("Response:", text) +print("usage:", msg.usage) + +assert msg.stop_reason in ("end_turn", "max_tokens"), f"unexpected stop_reason: {msg.stop_reason}" +assert len(text) > 0, "response should not be empty" +assert msg.usage.input_tokens > 0 and msg.usage.output_tokens > 0, "token counts should be > 0" +print("✅ Exercise 1 passed — Anthropic API is reachable from your machine") +``` + +**Cost**: ~$0.001/run (haiku) or ~$0.004/run (sonnet); this hello-world is also 5-15× faster than Ollama. + +
+ +### Exercise 2: Tokens +Run the same prompt 100 times and watch token counts vary. +- Notice: `temperature ≠ 0` produces variation +- Notice: token count for the SAME English vs Chinese sentence + +
+📋 Starter code — Path A (local Ollama gemma4:e4b, default) (copy to practice_2.py) + +```python +# Requires: pip install openai +# Pre-req: ollama pull gemma4:e4b && ollama serve +import sys, statistics +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +PROMPTS = { + "Chinese": "用一句話描述一隻貓在做什麼。", + "English": "Describe in one sentence what a cat is doing.", +} + +N = 10 # local is slower; start small +for label, prompt in PROMPTS.items(): + output_tokens = [] + for _ in range(N): + r = client.chat.completions.create( + model="gemma4:e4b", + max_tokens=80, + temperature=1.0, # high temp to amplify variance + messages=[{"role": "user", "content": prompt}], + ) + output_tokens.append(r.usage.completion_tokens) + print(f"\n[{label}] prompt: {prompt}") + print(f" input tokens: {r.usage.prompt_tokens}") + print(f" output tokens — min={min(output_tokens)} max={max(output_tokens)} mean={statistics.mean(output_tokens):.1f} stdev={statistics.stdev(output_tokens):.1f}") + +# === Self-check === +assert max(output_tokens) > min(output_tokens), "with temperature=1.0, output length should vary" +print("\n✅ Exercise 2 passed — observed temperature → token variance, $0/run") +print("💡 Chinese prompts typically use MORE input tokens (one Chinese character ≈ 2 tokens)") +``` + +
+ +
+📋 Starter code — Path B (Anthropic API, optional) (copy to practice_2_anthropic.py) + +```python +# Requires: pip install anthropic +import sys, statistics +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic +client = anthropic.Anthropic() +PROMPTS = {"Chinese": "用一句話描述一隻貓在做什麼。", "English": "Describe in one sentence what a cat is doing."} + +for label, prompt in PROMPTS.items(): + output_tokens = [] + for _ in range(20): + msg = client.messages.create(model="claude-haiku-4-5", max_tokens=80, temperature=1.0, + messages=[{"role": "user", "content": prompt}]) + output_tokens.append(msg.usage.output_tokens) + print(f"[{label}] input={msg.usage.input_tokens} output min/max/mean={min(output_tokens)}/{max(output_tokens)}/{sum(output_tokens)/len(output_tokens):.1f}") +``` + +**Key SDK diffs**: `messages.create` → `chat.completions.create`; `usage.output_tokens` → `usage.completion_tokens`; `usage.input_tokens` → `usage.prompt_tokens`. **Cost**: 40 runs ≈ $0.01. + +
+ +### Exercise 3: Pricing / Latency +**Cost-sensitive work required**: compute how long and how much it takes to run 1000 hello-world inferences. Local Ollama is $0 but has latency cost; cloud LLMs cost money but are faster. **Knowing this trade-off is how you pick the right model**. + +
+📋 Starter code — Path A (local Ollama gemma4:e4b, measure latency) (copy to practice_3.py) + +```python +# Requires: pip install openai +# Pre-req: ollama pull gemma4:e4b && ollama serve +import sys, time +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +latencies = [] +for _ in range(5): + t0 = time.time() + r = client.chat.completions.create( + model="gemma4:e4b", + max_tokens=200, + messages=[{"role": "user", "content": "Hi! Please introduce yourself."}], + ) + latencies.append(time.time() - t0) + +avg_latency = sum(latencies) / len(latencies) +out_tok_avg = r.usage.completion_tokens +tps = out_tok_avg / avg_latency if avg_latency > 0 else 0 + +print(f"model: gemma4:e4b (local)") +print(f"5 latencies (sec): min={min(latencies):.2f} max={max(latencies):.2f} mean={avg_latency:.2f}") +print(f"avg output: {out_tok_avg} tokens, ~{tps:.1f} tokens/sec") +print(f"\n1000-run cost: $0 (local); projected duration: {avg_latency * 1000 / 60:.1f} minutes") + +# === Self-check === +assert avg_latency > 0, "latency should be > 0" +assert out_tok_avg > 0, "output token count should be > 0" +print(f"\n✅ Exercise 3 passed — local model is $0 but takes ~{avg_latency * 1000 / 60:.0f} min for 1000 runs") +print("💡 Compare Path B Anthropic: 1000 runs is ~10-20 min at $0.25 (haiku)") +``` + +
+ +
+📋 Starter code — Path B (Anthropic API, compute $ cost) (copy to practice_3_anthropic.py) + +```python +# Requires: pip install anthropic +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic + +# Anthropic public pricing 2026 Q2 (per 1M tokens, USD) — verify at https://www.anthropic.com/pricing +PRICING = { + "claude-haiku-4-5": {"input": 1.00, "output": 5.00}, + "claude-sonnet-5": {"input": 3.00, "output": 15.00}, + "claude-opus-4-8": {"input": 5.00, "output": 25.00}, # Opus 4.8 (May 2026, Dynamic Workflows) — same 5/25 pricing + "claude-fable-5": {"input": 10.00, "output": 50.00}, # Fable 5 (Mythos-class, restored 2026-07-01) ~2x Opus 4.8 +} + +client = anthropic.Anthropic() +MODEL = "claude-haiku-4-5" +msg = client.messages.create(model=MODEL, max_tokens=200, + messages=[{"role": "user", "content": "Hi! Please introduce yourself."}]) +in_tok, out_tok = msg.usage.input_tokens, msg.usage.output_tokens +rates = PRICING[MODEL] +cost_one = (in_tok * rates["input"] + out_tok * rates["output"]) / 1_000_000 + +print(f"model: {MODEL}") +print(f"single: input={in_tok} output={out_tok} → ${cost_one:.6f}") +print(f"1000 calls cost across model tiers:") +for name, r in PRICING.items(): + c = (in_tok * r["input"] + out_tok * r["output"]) / 1_000_000 * 1000 + print(f" {name:<22} ${c:.4f}") + +assert cost_one > 0, "Cloud LLM always has a cost" +print(f"\n✅ Exercise 3 passed (Anthropic) — 1000 runs: haiku ≈ $0.25, sonnet 5 ≈ $0.76, opus 4.8 ≈ $1.27") +``` + +**Expected output**: +``` +model: claude-haiku-4-5 +single: input=14 output=48 → $0.000254 +1000 calls cost across model tiers: + claude-haiku-4-5 $0.2540 + claude-sonnet-5 $0.7620 + claude-opus-4-8 $1.2700 +``` + +**Trade-off**: local Ollama is $0 for 1000 runs but takes ~2 hr; Anthropic haiku is ~10 min for $0.25; sonnet ~10 min for $0.76. **Use cloud only for production; learning / experiments / debug stay local.** + +
+ +### Exercise 4: Cross-Provider Comparison +Send the same prompt to Claude, GPT, and Gemini simultaneously, compare their responses. Notice "why does the same input produce different answers" — answer style, length, and judgment all differ. Use the OpenAI, Anthropic, and Google SDKs side-by-side. + +→ **Starter template** → [`examples/stage-1/04-cross-provider/`](../examples/stage-1/04-cross-provider/) (parallel calls to all three SDKs + comparison table; missing keys are skipped gracefully; illustrative, **not a chapter-length tutorial**) + +### Exercise 5: Error Handling +Trigger error conditions deliberately and write retry logic: +- Wrong API key → see how it raises +- Over-long prompt → what happens when the context window is full +- Network drop → write a retry wrapper with exponential backoff + +This is foundational for Stage 3-8's production agent code. + +→ **Starter template** → [`examples/stage-1/05-error-handling/`](../examples/stage-1/05-error-handling/) (mock-based tests so you can verify the retry logic without unplugging your ethernet cable; illustrative, **not a chapter-length tutorial**) + +### Exercise 6: Local LLM +**No API fees, runs on your machine**: use Ollama to pull a small model (recommend `llama3.2:3b` or `qwen2.5:3b`), call it via OpenAI-compatible API. + +```bash +# 1. Install Ollama: https://ollama.com +ollama pull qwen2.5:3b +ollama serve # default port 11434 +``` + +
+📋 Starter code (copy to practice_6.py) + +```python +# Requires: pip install openai +# Pre-req: Ollama is running, qwen2.5:3b is pulled +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI( + base_url="http://localhost:11434/v1", + api_key="ollama", # Ollama doesn't check this — anything works +) + +r = client.chat.completions.create( + model="qwen2.5:3b", + messages=[{"role": "user", "content": "Explain ReAct in 3 sentences."}], +) + +text = r.choices[0].message.content +print("Response:", text) + +# === Self-check === +assert len(text) > 10, "response is too short — Ollama may not be running" +print(f"✅ Exercise 6 passed — local Ollama reachable through the OpenAI-compatible API") +print(f"💡 This run cost you $0 (except for electricity)") +``` + +**Why do this**: once you can run local LLMs, Stage 3-6 experiments aren't bottlenecked on API costs; privacy-sensitive work also stays offline. + +
+ +## 🎯 Curated Projects + +5 categories, 17 projects in one table. **Pick by "Best for"; click through for depth on the repo / course site.** + +| Category | Project | ⭐ | Best for | Why / Notes | +|---|---|---|---|---| +| **Official cookbook / starting point** | [Anthropic Cookbook](https://github.com/anthropics/claude-cookbooks) | ⭐⭐⭐⭐⭐ | Starting with Claude API; reference lookup | Full-feature Claude API notebooks (tool use / batch / prompt cache), ★ 46k+, MIT | +| | [Anthropic Courses](https://github.com/anthropics/courses) | ⭐⭐⭐⭐⭐ | Systematic Claude learning from zero | Anthropic's own 5-course set (API fundamentals / prompt eval / real-world prompting / tool use), ★ 21k+. Start with `anthropic_api_fundamentals` | +| | [OpenAI Cookbook](https://github.com/openai/openai-cookbook) | ⭐⭐⭐⭐⭐ | OpenAI API + structured output / function calling | Pair with Anthropic Cookbook, ★ 73k+, MIT. Much bigger than Anthropic's — use search | +| | [Anthropic Claude API Quickstart](https://docs.anthropic.com/en/docs/get-started) | ⭐⭐⭐⭐ | 5-minute start | Official docs, bookmark it | +| **Chinese textbook**
(chapter-style) | [datawhalechina/happy-llm](https://github.com/datawhalechina/happy-llm) | ⭐⭐⭐⭐⭐ | Chinese readers wanting LLM internals | Karpathy "Zero to Hero" Chinese counterpart, ★ 29k+. Equivalent to HF LLM Course in Chinese | +| | [datawhalechina/llm-universe](https://github.com/datawhalechina/llm-universe) | ⭐⭐⭐⭐⭐ | Chinese newcomers building with LLM | API basics / knowledge base / RAG / advanced tricks, ★ 13k+ | +| | [datawhalechina/llm-cookbook](https://github.com/datawhalechina/llm-cookbook) | ⭐⭐⭐⭐ | Full Chinese LLM learning path | Adapted Chinese translation of Andrew Ng's courses (⚠️ updates slowed after 2025-06, CC BY-NC-SA) | +| | [jingyaogong/minimind](https://github.com/jingyaogong/minimind) | ⭐⭐⭐⭐ | Post-Karpathy, want a real training run | 2hr to train a 64M LLM from scratch — Pretrain + SFT + LoRA + DPO + RLHF, ★ 48k+, Apache-2.0 | +| **English course**
(systematic) | [HuggingFace — LLM Course](https://huggingface.co/learn/llm-course) | ⭐⭐⭐⭐⭐ | Transformer internals + HF ecosystem | Transformer theory + applications, Apache 2.0 | +| | [LangChain Academy](https://academy.langchain.com/) | ⭐⭐⭐⭐ | Visual learners who like video courses | LangChain's official free course, includes RAG / agent. **Skip the LangChain marketing segments** | +| **Local execution**
(no API costs)| [ollama/ollama](https://github.com/ollama/ollama) | ⭐⭐⭐⭐⭐ | First-time local LLM | This repo's Path A default, OpenAI-compat API, ★ 170k+ | +| | [ggml-org/llama.cpp](https://github.com/ggml-org/llama.cpp) | ⭐⭐⭐⭐⭐ | Understanding quantization / how 7B fits in 8GB RAM | Ollama's underlying inference engine, ★ 119k+, MIT | +| | [mudler/LocalAI](https://github.com/mudler/LocalAI) | ⭐⭐⭐⭐ | Team compliance, self-host full OpenAI replacement | Drop-in OpenAI API replacement (chat / embedding / image / TTS / STT), ★ 46k+ | +| | [ml-explore/mlx](https://github.com/ml-explore/mlx) | ⭐⭐⭐⭐ | Mac dev, squeeze Apple Silicon | Apple's ML framework for M1+, ★ 25k+. Pair with `mlx-lm` for ease | +| **Build from scratch**
(understand internals)| [karpathy — Let's build GPT from scratch](https://www.youtube.com/watch?v=kCc8FmEb1nY) | ⭐⭐⭐⭐⭐ | Understand LLM internals, not just API calls | 2hr high-density video, build GPT in PyTorch from scratch. **Pause and code along, don't passive-watch** | +| | [rasbt/LLMs-from-scratch](https://github.com/rasbt/LLMs-from-scratch) | ⭐⭐⭐⭐⭐ | Book-pace read of the same material | Book version of Karpathy's video: tokenizer → attention → pretraining → finetuning, ★ 91k+, Apache-2.0 | +| | [karpathy/LLM101n](https://github.com/karpathy/LLM101n) | ⭐⭐ | Historical reference | ⚠️ Archived (2024-08), outline only, course never finished. **Watch "Build GPT from scratch" above instead** | + +> 💡 **Suggested reading order**: API-first → Anthropic / OpenAI Cookbook · Chinese systematic path → happy-llm + llm-universe · deep internals → Karpathy video + rasbt book with code · local-only → start with Ollama, then llama.cpp. + +## ✅ Self-Check Before Stage 2 + +Can you: + +- [ ] Make a Claude API call from Python in 5 lines +- [ ] Explain why "你好" might use 2 tokens but "Hello" uses 1 +- [ ] Quote roughly the per-token price for Claude Sonnet vs Opus +- [ ] Name one strength of Claude vs GPT vs Gemini vs Llama + +If yes → proceed to [Stage 2 — Prompt Engineering](02-prompt-engineering.en.md). + +If no → re-read the Anthropic Quickstart + run all 3 hello-X projects above. + +--- + +> ✅ **Done with Stage 1?** Next, [**Stage 2 — Prompt Engineering**](02-prompt-engineering.en.md) takes 5-12 hours to walk you through writing reusable structured prompts, using few-shot and chain-of-thought for reasoning tasks, and learning to quantify prompt improvement with evals. **Keep going →** diff --git a/stages/01-llm-basics.md b/stages/01-llm-basics.md new file mode 100644 index 0000000..030bbbb --- /dev/null +++ b/stages/01-llm-basics.md @@ -0,0 +1,554 @@ +# Stage 1 — LLM 基礎（LLM Basics） + +> **繁體中文** | [简体中文](./01-llm-basics.zh-Hans.md) | [English](./01-llm-basics.en.md) + +⏱ **時間估算**：1 週（約 5-8 小時） + +> 👋 **從 [Stage 0](00-foundations.md) 來的**：好，環境已經夠用——這 5-8 小時：第一次成功呼叫 Claude / GPT / Gemini API、搞懂 token / context window / temperature 怎麼影響輸出、用 per-token 計算實際成本。**直接從這裡開始的**：先確認你能跑 Python script、有任一家供應商的 API key——做不到請先回 [Stage 0](00-foundations.md)。 + +> 💡 **看不懂某個詞**（LLM / token / context window / temperature / RAG / agent⋯）→ 先翻 [`resources/glossary.md`](../resources/glossary.md) 查 30 秒再回來。 + +> 📋 **本章組成**：學習目標 → 進入條件 → 必修閱讀 →〔可選 · 概念地圖〕→ 動手練習 → 精選 Projects → 自我檢查 +> 🔑 **關鍵名詞**：見 [`resources/glossary.md`](../resources/glossary.md)（每 stage 用到的術語都收在那裡） + +### 三個核心詞（先記住、後面 stage 都會用到） + +| 詞 | 中文 | 一句話 | +|---|---|---| +| **token** | 詞元 | 模型計算文字長度與費用的基本單位（中文 1 字 ≈ 1.5-2 token） | +| **context window** | 上下文視窗 | 模型一次能看到多少 token（Claude 1M / GPT 1.05M / Gemini 2M）| +| **temperature** | 隨機程度參數 | 控制回答穩定或發散（0 = 最穩定、1 = 創意；分類任務用 0.0-0.3、創作用 0.7-1.0）| + +→ 這 3 個詞貫穿後續所有 stage。Stage 1 的目標就是讓你用 API 跑出來、親手摸到它們如何影響輸出。 + +> 🧠 **temperature 為什麼能調？先懂 next-token**：LLM 的核心動作是**預測下一個 token**：它對「下一個字」算出一個機率分布，再從裡面**抽樣**一個。`temperature` 跟 `top_p` 就是在「重塑這個分布」——temperature 低 → 分布變尖、幾乎只挑最可能的（穩定、可重現）；temperature 高 → 分布變平、更敢挑冷門字（有創意但易跑題）。`max_tokens` 則是「最多抽幾次就停」。所以這些不是魔法旋鈕，而是在控制「怎麼從機率分布裡選字」。 + +## 📌 學習目標 + +走完這個階段後你會： +- 解釋 LLM 是什麼、token 是什麼、context window 是什麼意思 +- 第一次成功呼叫 Claude / GPT / Gemini API 並解析回應 +- 在強項上比較四大 LLM 家族（Claude / GPT / Gemini / Llama） +- 用 per-token 計價來估算單次任務的成本 + +## 🌐 主流 LLM 家族對比（2026-05 snapshot） + +「Claude 跟 GPT 有什麼不同？」「中國模型能用嗎？」「我該裝 Ollama 跑哪個 OSS model？」——這節給你**客觀對照**。不下「最好」結論——用 **強項 / 適合任務 / 弱項** 3 維比較、附**官方 docs URL**讓你自己 verify。 + +> 💡 **先解釋幾個名詞**： +> - **Context window** = LLM 一次能記住的對話量、有上限（譬如 200k token ≈ 15 萬中文字） +> - **Apache 2.0 / MIT** = 可商用 / 可改 / 可閉源的開源條款；**Llama Community License** = 開源但有條款限制（譬如 ≥ 7 億 MAU 要授權） +> - **Frontier model** = 各家最強旗艦；**OSS** = open-source、weights 可下載 self-host + +### 🇺🇸 美系商業 frontier（3 家） + +這 3 家是 SaaS API、付 token 用、不能 self-host： + + + +| Model 家族 | 旗艦（2026-07）| Context | 強項 | 適合任務 | 官方 docs | +|---|---|---|---|---|---| +| **Claude**（Anthropic）| Opus 4.8 / Sonnet 5 / Haiku 4.5 | 1M | long-form / coding / agent / safety alignment | 寫 paper / code review / agent runtime | [platform.claude.com/docs](https://platform.claude.com/docs/en/about-claude/models/overview) | +| **GPT**（OpenAI）| GPT-5.6 Sol / Terra / Luna | 1.05M | 通用 / function calling / ecosystem 最廣 | 廣度查詢 / function-call 框架 / GPTs 生態 | [platform.openai.com/docs/models](https://platform.openai.com/docs/models) | +| **Gemini**（Google）| 3.5 Flash / 3.5 Pro（開發中）| 2M | 長 context / 原生 multimodal / Google 整合 | PDF / 影音 / 大量文件 / Google Workspace | [ai.google.dev](https://ai.google.dev/gemini-api/docs/models/gemini) | + +> **註**：`（開發中）`= 還沒推出。Claude **Fable 5**（Mythos-class、位階在 Opus 之上、$10/$50）曾於 2026-06-12 被美國出口管制暫停，**出口管制已於 2026-06-30 解除、Fable 5 於 2026-07-01 全球恢復**（Claude Platform / Claude Code / Cowork 可用、API 陸續開放），是目前最高階的 Claude 層級（Opus 4.8 為 Opus 級旗艦）。Context 欄填的是旗艦的上限：Gemini Pro 系列 2M、Flash 1M；Claude 1M（Haiku 4.5 是 200k）；GPT-5.6 三款都是 1.05M。另外 **Sonnet 5**（2026-06-30 上線）是目前的 Sonnet 版本：1M context、速度快、比 Opus 便宜（$3/$15，Opus 是 $5/$25）。**GPT-5.6**（2026-07 上線）分三級：**Sol** 旗艦（$5/$30）、**Terra** 均衡（$2.50/$15）、**Luna** 最快最省（$1/$6），ChatGPT / Codex / API 皆可用。 + +### 🇨🇳 中國商業 + 開源 frontier（7 家） + +中文場景的主力，分兩類：**純 API**（雲端付費、不能 self-host）和**有開源 weights**（可在自己機器跑）。 + +**① 純 API（雲端、付費為主）** + +| Model 家族 | 旗艦（2026-05）| Context | 強項 | 適合任務 | 官方 | +|---|---|---|---|---|---| +| **DeepSeek**（深度求索）| V3（`deepseek-chat`）/ R1（`deepseek-reasoner`）| 128k | 推理 / coding / **cost 最低** | 大量 token / code 生成 / math | [api-docs.deepseek.com](https://api-docs.deepseek.com/zh-cn/) | +| **Kimi**（Moonshot）| K2.6 multimodal + Agent | **超長 1M+** | 長 context / 中文長文 | 整本書讀 / 文獻分流 | [platform.moonshot.cn](https://platform.moonshot.cn/) | +| **Hunyuan**（騰訊）| T1（深度思考）+ TurboS | 128k | **可比 DeepSeek R1 推理**、中文 | 中文推理 / 騰訊生態 | [hunyuan.tencent.com](https://hunyuan.tencent.com/) | +| **MiniMax** | abab6.5 + M2.7 | 200k | 多模態 / 中文長 prose | 中文寫作 / 影音 multimodal | [platform.minimax.io](https://platform.minimax.io/) | + +> **註**：這組以雲端 API 為主、多為 proprietary。DeepSeek 另有部分開源權重（在 HF），但 V4 消費級 API 還沒完全開放，主要用法仍是 API。 + +**② 有開源 weights（可 self-host）** + +| Model 家族 | 旗艦（2026-05）| Context | 強項 | 適合任務 | 官方 | +|---|---|---|---|---|---| +| **Qwen**（阿里）| Qwen3 | 128k+ | **中文最強 OSS** / 多模態 / agent | 中文長文 / agent / self-host | [qwen.ai](https://qwen.ai/) · [DashScope](https://help.aliyun.com/zh/dashscope/) | +| **GLM**（智譜 Zhipu）| GLM-5 / GLM-5.1 | 128k | 中文 / tool use / agent | 中文 agent / 多輪對話 | [open.bigmodel.cn](https://open.bigmodel.cn/) · [chatglm.cn](https://chatglm.cn/) | +| **Yi**（01.AI / 李開復）| Yi-Lightning / Yi-34B-Chat | 200k | **中文 OSS** 替代 Llama | 中文 self-host / 中文 API | [01.ai](https://01.ai/) · [GitHub](https://github.com/01-ai/Yi) | + +> **註**：這三家都走 **Apache 2.0 開源版 + 付費雲端 API** 兩條路（GLM 開源版是 5.1）。開源版可用 [Ollama](https://ollama.com/) 在自己機器跑。 + +> ⚠️ **小米 MiMo** 雖在 [`resources/cli-agents-guide.md`](../resources/cli-agents-guide.md) 列入 Hermes Agent routing、但 2026-05 無權威官方 source 可驗證、暫不收進此表。要試 → 透過 [Hermes Agent](https://github.com/NousResearch/hermes-agent) 200+ provider routing 接入。 + +### 🌍 西方開源（4 家、self-host 主力） + +跑在自己機器、不付 API、隱私敏感場景的主力——可透過 [Ollama](https://ollama.com/) 一行指令裝起來： + +| Model 家族 | 大小（活躍）| License | 強項 | 適合任務 | 官方 | +|---|---|---|---|---|---| +| **Llama**（Meta）| 3.3 70B | Llama Community License | 通用 / 生態最廣 / Ollama 預設 | self-host 入門 / fine-tune base | [llama.com](https://www.llama.com/) · [HF Meta](https://huggingface.co/meta-llama) | +| **Gemma**（Google）| Gemma 4 26B MoE + 31B dense | Apache 2.0 | **小巧高效** / Apple MLX 整合好 / multimodal | Edge / mobile / 4-8GB RAM 機器 | [ai.google.dev/gemma](https://ai.google.dev/gemma) | +| **Mistral**（Mistral AI）| 7B / Mixtral 8x7B / Codestral | Apache 2.0（OSS 部分）| 開源 7B 級最強 | 商用 self-host / EU 主權 | [mistral.ai](https://mistral.ai/) · [HF Mistral](https://huggingface.co/mistralai) | +| **Phi**（Microsoft）| Phi-4 14B + multimodal | MIT | **小但強** / reasoning / 適 edge | 4GB+ RAM / mobile / reasoning 入門 | [HF microsoft](https://huggingface.co/microsoft) | + +> **註**：Llama 4 截至 2026-05 還沒釋出（表中為 3.3）；Gemma 4 為 2026-04 釋出、LMArena 開源組第 3；Phi-4 另有 multimodal 版。 + +### 🎯 我該選哪家？（按場景反查） + +| 你的場景 | 推薦 + 為什麼 | +|---|---| +| 第一次學 LLM API、教材完整度優先 | **Claude** — Anthropic Cookbook + Courses 是社群公認最完整 | +| 寫長文 / paper / code review | **Claude Sonnet** — long-form prose 強項 | +| 多模態（PDF / 影音 / 圖）| **Gemini** 或 **Kimi** — 原生 multimodal | +| 廣度查詢 + function calling 框架 | **GPT** — ecosystem 最廣、SDK 整合最深 | +| **中文場景 + 商業 API** | **Kimi**（長 context 強、能塞整本書）或 **DeepSeek**（cost 最低）或 **GLM**（agent 友善）| +| **中文場景 + 開源 self-host** | **Qwen 3**（Apache 2.0、目前中文最強 OSS）| +| 推理 / math（reasoning model）| **DeepSeek R1** / **Hunyuan T1** / **OpenAI o-series** | +| 隱私 / offline / 不付 API | **Llama 3.3** / **Gemma 4** / **Qwen 3 OSS** via [Ollama](https://ollama.com/) | +| Edge / 4GB RAM 機器 | **Gemma 4** / **Phi-4** / **Qwen 3（3B 以下版本）** | +| 100k+ token 大文件 | **Gemini 3.1**（2M context）或 **Kimi K2.6**（1M+）| +| **想 cost 最低**（API 帳單敏感）| **DeepSeek V4-Flash** — 同級英文 model 中 token 單價最低 | + +### 📊 中立 benchmark 資源（自己 verify、不靠單一 source） + +| 資源 | 用途 | URL | 2026-05 狀態 | +|---|---|---|---| +| **Artificial Analysis** | 第三方 benchmark + price/latency 整合（含中國 model）| https://artificialanalysis.ai/ | ✓ Active | +| **Arena AI**（前 LMSYS Chatbot Arena）| 人類盲測 ELO 排名 | https://arena.ai/leaderboard/text | ✓ Active | +| **Vellum LLM leaderboard** | 多 benchmark 整合 | https://www.vellum.ai/llm-leaderboard | ✓ Active | +| **HuggingFace OpenLLM Leaderboard** | 開源 model 排名 | https://huggingface.co/spaces/open-llm-leaderboard | ⚠️ 2026-05 偶爾 runtime error、改看 [Arena AI](https://arena.ai/) 開源 tab | +| **SuperCLUE**（中文 benchmark）| 中文場景權威評測 | https://www.superclueai.com/ | ✓ Active | + +### ⚠️ 重要警語 + +- ⚠️ **Benchmark ≠ production performance**——LLM 在你 specific 任務的表現要自己跑 small eval（譬如貼 10 個你真實 prompt 看哪家答得最像你要的）、**不能只看排名選** +- ⚠️ **Frontier 6 個月洗牌一次**——上面所有數字是 **2026-05 snapshot**、之後請以**官方 docs** / [Artificial Analysis](https://artificialanalysis.ai/) 為準 +- ⚠️ **「強項」是相對的、不是絕對的**——所有 frontier model 都能完成基本任務、差別在特殊或困難的情境（超長文件、複雜推理、多語言） +- ⚠️ **中文場景看 [SuperCLUE](https://www.superclueai.com/)**——一般國際 benchmark（如 MMLU）以英文為主、中文表現可能跟英文不一致 + +## 🚪 進入條件 + +你應該已經： +- 能跑 Python script +- 概念上知道 HTTP / REST 是什麼 +- 至少有一家供應商的 API key（Anthropic / OpenAI / Google） + +如果還沒——先回 Stage 0。 + +## 📚 必修閱讀 + +1. [**Anthropic — Claude 模型總覽**](https://docs.claude.com/en/about-claude/models/overview) — 官方模型 family、含 2026 的 Claude Fable 5（`claude-fable-5`、Mythos-class、2026-06-09 GA）以及 Opus 4.8 / Sonnet 5 / Haiku 4.5。**Fable 5 與姊妹版 Mythos 5（`claude-mythos-5`）曾於 2026-06-12 被美國出口管制指令暫停，出口管制已於 2026-06-30 解除；[Fable 5 於 2026-07-01 全球恢復](https://www.anthropic.com/news/redeploying-fable-5)（重新部署時加了新的安全 classifier），Mythos 5 僅對核准的美國組織恢復。Fable 5 是目前最高階的 Claude 層級，Opus 4.8 為 Opus 級旗艦。** +2. [**anthropics/courses — Anthropic API Fundamentals**](https://github.com/anthropics/courses) ⭐⭐⭐⭐⭐ ★ 21k+ — Anthropic 官方 5 course umbrella、**module 1「Anthropic API Fundamentals」對應本 stage**。Jupyter notebook、用 Claude 3 Haiku（最便宜）跑、跟著做就能拿到 API 基本功 +3. [**OpenAI Quickstart**](https://platform.openai.com/docs/quickstart) — 第一次 API call 的步驟 +4. [**A Visual Guide to LLM Tokenizers**](https://huggingface.co/learn/llm-course/chapter6/1) — Hugging Face 的入門 +5. [**Anthropic API Pricing**](https://www.anthropic.com/pricing#anthropic-api) — 把計價表看完，算一下 1k input + 1k output 的成本 + +**🎥 中文影片補充（強烈推薦）**： +- [**李宏毅 — 生成式 AI 導論（2024 春台大課程）**](https://speech.ee.ntu.edu.tw/~hylee/genai/2024-spring.php) ⭐⭐⭐ — 第 1-5 集講 LLM 是什麼、怎麼運作、token / context window / temperature 怎麼影響輸出。中文圈最高品質的 LLM 學術級導論、台大授課、官方頁含投影片 + YouTube。最新整合版見 [**GenAI-ML 2025 秋**](https://speech.ee.ntu.edu.tw/~hylee/GenAI-ML/2025-fall.php) +- [**3Blue1Brown — Transformer 視覺化**](https://www.youtube.com/watch?v=wjZofJX0v4M)（中文配音版：[3Blue1Brown 中文](https://www.youtube.com/@3Blue1BrownCN)）— LLM 內部運作 visual intro +- [**Andrej Karpathy — Intro to LLMs**](https://www.youtube.com/watch?v=zjkBMFhNj_g) — 英文影片、1hr、英文圈最被推薦的 LLM 入門影片 + +## 🛠 動手練習（基礎 illustrative 練習） + +> 🦙 **本 stage 預設用 Ollama**（成本考量、本機 `gemma4:e4b` 跑得動、$0/run）。每個練習都有 Path A（Ollama、預設）+ Path B（Anthropic、選擇性、想看 cloud 高品質時用）。完整 3 路 trade-off 見 [`examples/README.md`](../examples/README.md#三條路徑--預設用-ollama成本考量)。 +> +> 💰 **Stage 1 預算估算**（全 6 練習各跑 3-5 次）：**全本機 = $0**、**全 haiku ≈ $0.30**、**全 sonnet ≈ $0.90**。完整 model 清單 + Stage 1-7 全程預算估算見 [`examples/README.md#推薦-llm-清單`](../examples/README.md#推薦-llm-清單)。 +> +> 💡 **不裝 Ollama 也能讀** — 每個練習的 Path B 區塊就是 Anthropic 版、選一個跑就行。先 [`pip install openai && ollama pull gemma4:e4b`](https://ollama.com) 就裝好 Path A 環境。 + +### 練習 1：LLM API（hello world） +五行 Python 呼叫 LLM 並印出回應。**預設用 Ollama 本機跑（免費、offline）**；想看 cloud 答案品質改 Path B Anthropic。詳見 [`examples/README.md`](../examples/README.md#三條路徑--預設用-ollama成本考量)。 + +
+📋 起手碼 — Path A（本機 Ollama gemma4:e4b、預設）（複製到 practice_1.py、python practice_1.py 就跑） + +```python +# 需要：pip install openai (用 OpenAI-compatible SDK 跟 Ollama 溝通) +# 前置：ollama pull gemma4:e4b && ollama serve +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI( + base_url="http://localhost:11434/v1", + api_key="ollama", # Ollama 不檢查、隨便填 +) + +r = client.chat.completions.create( + model="gemma4:e4b", # 換成 qwen2.5:3b / llama3.2:3b 也可 + max_tokens=100, + messages=[{"role": "user", "content": "用一句話自我介紹。"}], +) + +# === 自我驗證 === +text = r.choices[0].message.content +print("回應：", text) +print("usage:", r.usage) + +assert r.choices[0].finish_reason in ("stop", "length"), f"非預期 finish_reason: {r.choices[0].finish_reason}" +assert len(text) > 0, "回應不應為空" +assert r.usage.completion_tokens > 0, "output token 應 > 0" +print("✅ 練習 1 通過 — Ollama gemma4:e4b 已能本機回應、$0/次") +``` + +**預期輸出**（樣本）： +``` +回應：嗨！我是 Gemma、一個由 Google 訓練的開源語言模型... +usage: CompletionUsage(completion_tokens=35, prompt_tokens=12, total_tokens=47) +✅ 練習 1 通過 — Ollama gemma4:e4b 已能本機回應、$0/次 +``` + +**慢嗎？** Gemma 4B 在 CPU 上約 5-30s/答案、有 GPU（RTX 3060+）<2s。要更快用 `gemma3:1b`、要更聰明改 `qwen2.5:14b` / `llama3.3:8b`（需 8GB+ VRAM）。 + +
+ +
+📋 起手碼 — Path B（Anthropic API、選擇性、想看 cloud 高品質時）（複製到 practice_1_anthropic.py） + +```python +# 需要：pip install anthropic +# 環境變數：export ANTHROPIC_API_KEY=sk-ant-... +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic + +client = anthropic.Anthropic() +msg = client.messages.create( + model="claude-haiku-4-5", # haiku 最便宜；換 sonnet 改這行 + max_tokens=100, + messages=[{"role": "user", "content": "用一句話自我介紹。"}], +) + +# === 自我驗證 === +text = msg.content[0].text +print("回應：", text) +print("usage:", msg.usage) + +assert msg.stop_reason in ("end_turn", "max_tokens"), f"非預期 stop_reason: {msg.stop_reason}" +assert len(text) > 0, "回應不應為空" +assert msg.usage.input_tokens > 0 and msg.usage.output_tokens > 0, "token 數應 > 0" +print("✅ 練習 1 通過 — 你已成功打通 Anthropic API") +``` + +**預期輸出**（樣本）： +``` +回應：我是 Claude，一個由 Anthropic 訓練的 AI 助理... +usage: Usage(input_tokens=18, output_tokens=42, ...) +✅ 練習 1 通過 — 你已成功打通 Anthropic API +``` + +**成本**：每次 ~$0.001 (haiku) / $0.004 (sonnet)、跑這個 hello world 比 Ollama 快 5-15 倍。 + +
+ +### 練習 2：Tokens +同一個 prompt 跑 100 次，觀察 token 數的變化。 +- 注意：`temperature ≠ 0` 會產生變動 +- 注意：同一句話的英文 vs 中文 token 數差異 + +
+📋 起手碼 — Path A（本機 Ollama gemma4:e4b、預設）（複製到 practice_2.py） + +```python +# 需要：pip install openai (OpenAI-compatible SDK 跟 Ollama 溝通) +# 前置：ollama pull gemma4:e4b && ollama serve +import sys, statistics +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +PROMPTS = { + "中文": "用一句話描述一隻貓在做什麼。", + "English": "Describe in one sentence what a cat is doing.", +} + +N = 10 # 本機慢、N 小一點；確認 OK 後加大 +for label, prompt in PROMPTS.items(): + output_tokens = [] + for _ in range(N): + r = client.chat.completions.create( + model="gemma4:e4b", + max_tokens=80, + temperature=1.0, # 拉高 temperature 看 variance + messages=[{"role": "user", "content": prompt}], + ) + output_tokens.append(r.usage.completion_tokens) + print(f"\n[{label}] prompt: {prompt}") + print(f" input tokens: {r.usage.prompt_tokens}") + print(f" output tokens — min={min(output_tokens)} max={max(output_tokens)} mean={statistics.mean(output_tokens):.1f} stdev={statistics.stdev(output_tokens):.1f}") + +# === 自我驗證 === +assert max(output_tokens) > min(output_tokens), "temperature=1.0 下、output 長度應該有 variance" +print("\n✅ 練習 2 通過 — 觀察到 temperature 對 output token 的 variance、本機跑 $0") +print("💡 中文 prompt 通常 input tokens 比 English 多（中文 token 化通常一字 ≈ 2 tokens）") +``` + +**預期輸出**（樣本）： +``` +[中文] prompt: 用一句話描述一隻貓在做什麼。 + input tokens: 32 + output tokens — min=18 max=58 mean=35.2 stdev=11.4 + +✅ 練習 2 通過 — 觀察到 temperature 對 output token 的 variance、本機跑 $0 +``` + +
+ +
+📋 起手碼 — Path B（Anthropic API、選擇性）（複製到 practice_2_anthropic.py） + +```python +# 需要：pip install anthropic +import sys, statistics +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic + +client = anthropic.Anthropic() +PROMPTS = {"中文": "用一句話描述一隻貓在做什麼。", "English": "Describe in one sentence what a cat is doing."} + +for label, prompt in PROMPTS.items(): + output_tokens = [] + for _ in range(20): + msg = client.messages.create(model="claude-haiku-4-5", max_tokens=80, temperature=1.0, + messages=[{"role": "user", "content": prompt}]) + output_tokens.append(msg.usage.output_tokens) + print(f"[{label}] input={msg.usage.input_tokens} output min/max/mean={min(output_tokens)}/{max(output_tokens)}/{sum(output_tokens)/len(output_tokens):.1f}") +``` + +**主要差異**：`client.messages.create()` → `client.chat.completions.create()`；`usage.output_tokens` → `usage.completion_tokens`；`usage.input_tokens` → `usage.prompt_tokens`。**成本**：40 次 ≈ $0.01。 + +
+ +### 練習 3：Pricing / Latency +**成本敏感的工作必修**：算出你的 hello-world prompt 在不同 model 上跑 1000 次的成本。Ollama 本機是 $0 但有 latency 成本；Cloud LLM 有 $ 成本但快。**會算這兩個 trade-off 才能挑對 model**。 + +
+📋 起手碼 — Path A（本機 Ollama gemma4:e4b、量 latency）（複製到 practice_3.py） + +```python +# 需要：pip install openai +# 前置：ollama pull gemma4:e4b && ollama serve +import sys, time +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +# 量 5 次 latency +latencies = [] +for _ in range(5): + t0 = time.time() + r = client.chat.completions.create( + model="gemma4:e4b", + max_tokens=200, + messages=[{"role": "user", "content": "你好！自我介紹一下。"}], + ) + latencies.append(time.time() - t0) + +# 統計 +avg_latency = sum(latencies) / len(latencies) +out_tok_avg = r.usage.completion_tokens # 末次當代表 +tps = out_tok_avg / avg_latency if avg_latency > 0 else 0 + +print(f"model: gemma4:e4b (本機)") +print(f"5 次 latency (sec): min={min(latencies):.2f} max={max(latencies):.2f} mean={avg_latency:.2f}") +print(f"avg output: {out_tok_avg} tokens、約 {tps:.1f} tokens/sec") +print(f"\n1000 次成本: $0 (本機)、預計時長: {avg_latency * 1000 / 60:.1f} 分鐘") + +# === 自我驗證 === +assert avg_latency > 0, "latency 應 > 0" +assert out_tok_avg > 0, "output token 應 > 0" +print(f"\n✅ 練習 3 通過 — 本機 model $0 但要花 {avg_latency * 1000 / 60:.0f} 分鐘跑 1000 次") +print("💡 對照 Path B Anthropic：1000 次只要 ~10-20 分鐘但要 $0.25（haiku）") +``` + +**預期輸出**（樣本）： +``` +model: gemma4:e4b (本機) +5 次 latency (sec): min=4.21 max=8.93 mean=6.54 +avg output: 48 tokens、約 7.3 tokens/sec + +1000 次成本: $0 (本機)、預計時長: 109.0 分鐘 + +✅ 練習 3 通過 — 本機 model $0 但要花 109 分鐘跑 1000 次 +💡 對照 Path B Anthropic：1000 次只要 ~10-20 分鐘但要 $0.25（haiku） +``` + +
+ +
+📋 起手碼 — Path B（Anthropic API、算 $ 成本）（複製到 practice_3_anthropic.py） + +```python +# 需要：pip install anthropic +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic + +# Anthropic 2026 Q2 公開計價（每 1M token、USD）— 跑前對照 https://www.anthropic.com/pricing +PRICING = { + "claude-haiku-4-5": {"input": 1.00, "output": 5.00}, + "claude-sonnet-5": {"input": 3.00, "output": 15.00}, + "claude-opus-4-8": {"input": 5.00, "output": 25.00}, # Opus 4.8 (May 2026, Dynamic Workflows) — 維持 5/25 同價 + "claude-fable-5": {"input": 10.00, "output": 50.00}, # Fable 5 (Mythos-class、2026-07-01 恢復可用) 約 Opus 4.8 的 2 倍 +} + +client = anthropic.Anthropic() +MODEL = "claude-haiku-4-5" + +msg = client.messages.create(model=MODEL, max_tokens=200, + messages=[{"role": "user", "content": "你好！自我介紹一下。"}]) +in_tok, out_tok = msg.usage.input_tokens, msg.usage.output_tokens +rates = PRICING[MODEL] +cost_one = (in_tok * rates["input"] + out_tok * rates["output"]) / 1_000_000 + +print(f"model: {MODEL}") +print(f"single: input={in_tok} output={out_tok} → ${cost_one:.6f}") +print(f"1000 calls cost across model tiers:") +for name, r in PRICING.items(): + c = (in_tok * r["input"] + out_tok * r["output"]) / 1_000_000 * 1000 + print(f" {name:<22} ${c:.4f}") + +# === 自我驗證 === +assert cost_one > 0, "Cloud LLM 一定有成本" +print(f"\n✅ 練習 3 通過（Anthropic）— 1000 次 haiku ≈ $0.25、sonnet 5 ≈ $0.76、opus 4.8 ≈ $1.27") +``` + +**預期輸出**： +``` +model: claude-haiku-4-5 +single: input=14 output=48 → $0.000254 +1000 calls cost across model tiers: + claude-haiku-4-5 $0.2540 + claude-sonnet-5 $0.7620 + claude-opus-4-8 $1.2700 +``` + +**Trade-off 對照**：本機 Ollama 跑 1000 次免費但要 ~2 hr；Anthropic haiku ~10 min $0.25；sonnet ~10 min $0.76。**production 場景才考慮 cloud；學習 / 實驗 / debug 全用本機**。 + +
+ +### 練習 4：Cross-Provider 比較 +同一個 prompt 同時送給 Claude、GPT、Gemini，比較三家的回應差異。觀察「同一句話為什麼產生不同答案」——回答風格、長度、判斷取捨都不一樣。建議用 OpenAI、Anthropic、Google 三家 SDK 各一段程式呼叫。 + +→ **基礎 starter 範本** → [`examples/stage-1/04-cross-provider/`](../examples/stage-1/04-cross-provider/)（含三家 SDK 並行呼叫 + table 對照、缺哪家 key 就 skip 哪家；illustrative，**不是 chapter-length 完整教學**） + +### 練習 5：Error Handling +故意觸發錯誤情境並寫 retry： +- API key 錯誤 → 看怎麼 raise +- prompt 超長 → context window 滿了會發生什麼 +- 網路斷掉 → 寫一個有 exponential backoff 的 retry wrapper + +這是後面 Stage 3-8 寫 production agent 一定會用到的基礎。 + +→ **基礎 starter 範本** → [`examples/stage-1/05-error-handling/`](../examples/stage-1/05-error-handling/)（含 mock-based test、不用真的斷網就能驗證 retry 邏輯；illustrative，**不是 chapter-length 完整教學**） + +### 練習 6：Local LLM +**不付 API 費用、跑在自己電腦上**：用 Ollama 下載一個小模型（建議 `llama3.2:3b` 或 `qwen2.5:3b`），用 OpenAI-相容 API 呼叫它。 + +```bash +# 1. 裝 Ollama: https://ollama.com +ollama pull qwen2.5:3b +ollama serve # 預設 port 11434 +``` + +
+📋 起手碼（複製到 practice_6.py） + +```python +# 需要：pip install openai +# 前置：Ollama 已 serve、qwen2.5:3b 已 pull +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI( + base_url="http://localhost:11434/v1", + api_key="ollama", # Ollama 不檢查、隨便填 +) + +r = client.chat.completions.create( + model="qwen2.5:3b", + messages=[{"role": "user", "content": "用 3 句話介紹什麼是 ReAct。"}], +) + +text = r.choices[0].message.content +print("回應：", text) + +# === 自我驗證 === +assert len(text) > 10, "回應太短、Ollama 可能沒跑起來" +print(f"✅ 練習 6 通過 — 你的本機 Ollama 已能透過 OpenAI-compatible API 呼叫") +print(f"💡 跑這次完全沒花錢（除了你的電力）") +``` + +**預期輸出**（樣本、實際內容因 model 而異）： +``` +回應：ReAct 是一種讓 AI 結合「推理」和「行動」的方法... +✅ 練習 6 通過 — 你的本機 Ollama 已能透過 OpenAI-compatible API 呼叫 +💡 跑這次完全沒花錢（除了你的電力） +``` + +**為什麼要做**：學會跑本地 LLM 後，後面 Stage 3-6 的實驗都不會被 API 費用卡住；隱私敏感場景也能 offline。 + +**沒裝 Ollama 也想跑**：把 `base_url` 換成 [LM Studio](https://lmstudio.ai) (`http://localhost:1234/v1`) 或 [vLLM](https://github.com/vllm-project/vllm) endpoint、API 介面一樣。 + +
+ +## 🎯 精選 Projects + +按用途分 5 類、17 個項目一張表搞定。**挑入口看「適合誰」、想深入點連結看 repo / 課程網站**。 + +| 分類 | Project | ⭐ | 適合誰 | 為什麼推薦 / 備註 | +|---|---|---|---|---| +| **官方 cookbook / 入門** | [Anthropic Cookbook](https://github.com/anthropics/claude-cookbooks) | ⭐⭐⭐⭐⭐ | 開始用 Claude API、當參考書查 | Claude API 全功能 notebook（tool use / batch / prompt cache），★ 46k+、MIT | +| | [Anthropic Courses](https://github.com/anthropics/courses) | ⭐⭐⭐⭐⭐ | 系統性從零學一遍 Claude | Anthropic 自家完整 5 門課（API 基礎 / prompt eval / real-world prompting / tool use），★ 21k+。先跑 `anthropic_api_fundamentals` | +| | [OpenAI Cookbook](https://github.com/openai/openai-cookbook) | ⭐⭐⭐⭐⭐ | 用 OpenAI API + structured output / function calling | 跟 Anthropic Cookbook 對照、★ 73k+、MIT。比 Anthropic 大很多、用搜尋 | +| | [Anthropic Claude API Quickstart](https://docs.anthropic.com/en/docs/get-started) | ⭐⭐⭐⭐ | 5 分鐘上手 | 官方文件、加 bookmark 用 | +| **中文教材**
（章節式） | [datawhalechina/happy-llm](https://github.com/datawhalechina/happy-llm) | ⭐⭐⭐⭐⭐ | 中文讀者想徹底搞懂 LLM 原理 | 對應 Karpathy「Zero to Hero」中文版，★ 29k+。等同 HF LLM Course 中文版 | +| | [datawhalechina/llm-universe](https://github.com/datawhalechina/llm-universe) | ⭐⭐⭐⭐⭐ | 中文新手想用 LLM 做東西 | API 基礎 / 知識庫 / RAG / 進階技巧，★ 13k+ | +| | [datawhalechina/llm-cookbook](https://github.com/datawhalechina/llm-cookbook) | ⭐⭐⭐⭐ | 想要完整中文 LLM 學習路線 | Andrew Ng 課程中文翻譯改編（⚠️ 2025-06 後更新放緩、CC BY-NC-SA）| +| | [jingyaogong/minimind](https://github.com/jingyaogong/minimind) | ⭐⭐⭐⭐ | 看完 Karpathy 影片想實際跑訓練 | 2hr 從零訓 64M LLM、Pretrain + SFT + LoRA + DPO + RLHF 全包，★ 48k+、Apache-2.0 | +| **英文 course**
（系統式） | [HuggingFace — LLM Course](https://huggingface.co/learn/llm-course) | ⭐⭐⭐⭐⭐ | 想搞懂 transformer 內部 + HF 生態 | 含 transformer 原理 + 應用、Apache 2.0 | +| | [LangChain Academy](https://academy.langchain.com/) | ⭐⭐⭐⭐ | 喜歡影片教學的視覺型學習者 | LangChain 官方免費課、含 RAG / agent。**忽略 LangChain 行銷段落** | +| **本地端執行**
（不付 API 費）| [ollama/ollama](https://github.com/ollama/ollama) | ⭐⭐⭐⭐⭐ | 第一次跑本地 LLM | 本 repo Path A 預設、OpenAI-compat API、★ 170k+ | +| | [ggml-org/llama.cpp](https://github.com/ggml-org/llama.cpp) | ⭐⭐⭐⭐⭐ | 想搞懂 quantization / 為什麼 7B 能塞 8GB RAM | Ollama 底層 inference engine，★ 119k+、MIT | +| | [mudler/LocalAI](https://github.com/mudler/LocalAI) | ⭐⭐⭐⭐ | 團隊合規、要 self-host 全套 OpenAI 替代 | drop-in OpenAI API 替代品（chat / embedding / image / TTS / STT），★ 46k+ | +| | [ml-explore/mlx](https://github.com/ml-explore/mlx) | ⭐⭐⭐⭐ | Mac 開發、想榨乾 Apple Silicon | Apple 為 M1+ 量身打造的 ML framework，★ 25k+。搭 `mlx-lm` 用最方便 | +| **從零打造**
（理解原理）| [karpathy — Let's build GPT from scratch](https://www.youtube.com/watch?v=kCc8FmEb1nY) | ⭐⭐⭐⭐⭐ | 想搞懂 LLM 內部、不只會呼叫 | 2hr 高密度影片、用 PyTorch 從零打造 GPT。**暫停跟著寫 code 不要被動看** | +| | [rasbt/LLMs-from-scratch](https://github.com/rasbt/LLMs-from-scratch) | ⭐⭐⭐⭐⭐ | 想用整本書速度慢慢讀完 | Karpathy 影片的書本版：tokenizer → attention → pretraining → finetuning，★ 91k+、Apache-2.0 | +| | [karpathy/LLM101n](https://github.com/karpathy/LLM101n) | ⭐⭐ | 歷史紀錄 | ⚠️ 已封存（2024-08）、只有大綱、課程沒做完。**直接看上面的「Build GPT from scratch」影片即可** | + +> 💡 **建議閱讀路徑**：API 入手就 Anthropic / OpenAI Cookbook → 中文系統路線就 happy-llm + llm-universe → 想深入內部就 Karpathy 影片 + rasbt 書搭 code → 想跑本地就 Ollama 起步、進階再讀 llama.cpp。 + +## ✅ 進 Stage 2 前的自我檢查 + +你能不能： + +- [ ] 用 5 行 Python 呼叫 Claude API +- [ ] 解釋為什麼「你好」可能用 2 個 token，但「Hello」只用 1 個 +- [ ] 大致說出 Claude Sonnet vs Opus 的 per-token 價格 +- [ ] 各說出 Claude / GPT / Gemini / Llama 的一個強項 + +如果可以 → 進 [Stage 2 — Prompt Engineering](02-prompt-engineering.md)。 + +如果不行 → 重看 Anthropic Quickstart + 把上面 3 個 hello-X 都跑一次。 + +--- + +> ✅ **Stage 1 完成？** 接下來 [**Stage 2 — Prompt Engineering**](02-prompt-engineering.md) 會用 5-12 小時帶你寫出可重用的結構化 prompt、用 few-shot 跟 chain-of-thought 解推理題、並學會用 eval 量化 prompt 改善幅度。**繼續往下走 →** diff --git a/stages/01-llm-basics.zh-Hans.md b/stages/01-llm-basics.zh-Hans.md new file mode 100644 index 0000000..594f3ea --- /dev/null +++ b/stages/01-llm-basics.zh-Hans.md @@ -0,0 +1,497 @@ +# Stage 1 — LLM 基础（LLM Basics） + +> [繁體中文](./01-llm-basics.md) | **简体中文** | [English](./01-llm-basics.en.md) + +> **预计学习时间**： 5-8 小时 + +> 👋 **从 [Stage 0](00-foundations.zh-Hans.md) 来的**：好，环境已经够用——这 5-8 小时：第一次成功调用 Claude / GPT / Gemini API、搞懂 token / context window / temperature 怎么影响输出、用 per-token 计算实际成本。**直接从这里开始的**：先确认你能跑 Python script、有任一家供应商的 API key——做不到请先回 [Stage 0](00-foundations.zh-Hans.md)。 + +> 掌握 **核心概念**：LLM / token / context window / temperature / RAG / agent，请先阅读 [`resources/glossary.zh-Hans.md`](../resources/glossary.zh-Hans.md)（约 30 分钟）。 + +### 3 个核心词（先记住，后面 stage 都会用到） + +| 词 | 中文 | 一句话 | +|---|---|---| +| **token** | 词元 | 模型计算文字长度与费用的基本单位（中文 1 个字 ≈ 1.5-2 token） | +| **context window** | 上下文窗口 | 模型一次能看到多少 token（Claude 1M / GPT 1.05M / Gemini 2M） | +| **temperature** | 随机程度参数 | 控制回答是更稳定还是更发散（0 = 最稳定、1 = 更有创意；分类任务用 0.0-0.3、创作用 0.7-1.0） | + +→ 这 3 个词贯穿后续所有 stage。Stage 1 的目标就是让你亲手调用 API、直接感受它们怎么影响输出。 + +> 🧠 **temperature 为什么能调？先懂 next-token**：LLM 的核心动作是**预测下一个 token**：它对“下一个字”算出一个概率分布，再从里面**采样**一个。`temperature` 跟 `top_p` 就是在“重塑这个分布”——temperature 低 → 分布变尖、几乎只挑最可能的（稳定、可复现）；temperature 高 → 分布变平、更敢挑冷门字（有创意但易跑题）。`max_tokens` 则是“最多采样几次就停”。所以这些不是魔法旋钮，而是在控制“怎么从概率分布里选字”。 + +## 📌 学习目标 + +完成本阶段后，你将能够： +- 解释 LLM、token、context window 等核心概念。 +- 使用 Python 调用 Claude / GPT / Gemini API。 +- 比较不同 LLM 提供商（Claude / GPT / Gemini / Llama）的优劣。 +- 理解 per-token 定价模型并估算成本。 + +## 🌐 主流 LLM 家族对比（2026-05 snapshot） + +“Claude 跟 GPT 有什么不同？”“中国模型能用吗？”“我该装 Ollama 跑哪个 OSS model？”——这节给你**客观对照**。不下“最好”结论——用 **强项 / 适合任务 / 弱项** 3 维比较、附**官方 docs URL**让你自己 verify。 + +> 💡 **先解释几个名词**： +> - **Context window** = LLM 一次能记住的对话量、有上限（例如 200k token ≈ 15 万中文字） +> - **Apache 2.0 / MIT** = 可商用 / 可修改 / 可闭源再发布的开源条款；**Llama Community License** = 开源但有条款限制（例如 ≥ 7 亿 MAU 要授权） +> - **Frontier model** = 各家最强旗舰；**OSS** = open-source、weights 可下载 self-host + +### 🇺🇸 美系商业 frontier（3 家） + +这 3 家是 SaaS API、按 token 付费、不能 self-host： + + + +| Model 家族 | 旗舰（2026-07）| Context | 强项 | 适合任务 | 官方 docs | +|---|---|---|---|---|---| +| **Claude**（Anthropic）| Opus 4.8 / Sonnet 5 / Haiku 4.5 | 1M | long-form / coding / agent / safety alignment | 写 paper / code review / agent runtime | [platform.claude.com/docs](https://platform.claude.com/docs/en/about-claude/models/overview) | +| **GPT**（OpenAI）| GPT-5.6 Sol / Terra / Luna | 1.05M | 通用 / function calling / ecosystem 最广 | 广度查询 / function-call 框架 / GPTs 生态 | [platform.openai.com/docs/models](https://platform.openai.com/docs/models) | +| **Gemini**（Google）| 3.5 Flash / 3.5 Pro（开发中）| 2M | 长 context / 原生 multimodal / Google 整合 | PDF / 影音 / 大量文件 / Google Workspace | [ai.google.dev](https://ai.google.dev/gemini-api/docs/models/gemini) | + +> **注**：`（开发中）`= 还没推出。Claude **Fable 5**（Mythos-class、位阶在 Opus 之上、$10/$50）曾于 2026-06-12 被美国出口管制暂停，**出口管制已于 2026-06-30 解除、Fable 5 于 2026-07-01 全球恢复**（Claude Platform / Claude Code / Cowork 可用、API 陆续开放），是目前最高阶的 Claude 层级（Opus 4.8 为 Opus 级旗舰）。Context 栏填的是旗舰的上限：Gemini Pro 系列 2M、Flash 1M；Claude 1M（Haiku 4.5 是 200k）；GPT-5.6 三款都是 1.05M。另外 **Sonnet 5**（2026-06-30 上线）是目前的 Sonnet 版本：1M context、速度快、比 Opus 便宜（$3/$15，Opus 是 $5/$25）。**GPT-5.6**（2026-07 上线）分三级：**Sol** 旗舰（$5/$30）、**Terra** 均衡（$2.50/$15）、**Luna** 最快最省（$1/$6），ChatGPT / Codex / API 都能用。 + +### 🇨🇳 中国商业 + 开源 frontier（7 家） + +中文场景的主力，分两类：**纯 API**（云端付费、不能 self-host）和**有开源 weights**（可在自己机器跑）。 + +**① 纯 API（云端、付费为主）** + +| Model 家族 | 旗舰（2026-05）| Context | 强项 | 适合任务 | 官方 | +|---|---|---|---|---|---| +| **DeepSeek**（深度求索）| V3（`deepseek-chat`）/ R1（`deepseek-reasoner`）| 128k | 推理 / coding / **cost 最低** | 大量 token / code 生成 / math | [api-docs.deepseek.com](https://api-docs.deepseek.com/zh-cn/) | +| **Kimi**（Moonshot）| K2.6 multimodal + Agent | **超长 1M+** | 长 context / 中文长文 | 整本书读 / 文献分流 | [platform.moonshot.cn](https://platform.moonshot.cn/) | +| **Hunyuan**（腾讯）| T1（深度思考）+ TurboS | 128k | **可比 DeepSeek R1 推理**、中文 | 中文推理 / 腾讯生态 | [hunyuan.tencent.com](https://hunyuan.tencent.com/) | +| **MiniMax** | abab6.5 + M2.7 | 200k | 多模态 / 中文长 prose | 中文写作 / 影音 multimodal | [platform.minimax.io](https://platform.minimax.io/) | + +> **注**：这组以云端 API 为主、多为 proprietary。DeepSeek 另有部分开源权重（在 HF），但 V4 消费级 API 还没完全开放，主要用法仍是 API。 + +**② 有开源 weights（可 self-host）** + +| Model 家族 | 旗舰（2026-05）| Context | 强项 | 适合任务 | 官方 | +|---|---|---|---|---|---| +| **Qwen**（阿里）| Qwen3 | 128k+ | **中文最强 OSS** / 多模态 / agent | 中文长文 / agent / self-host | [qwen.ai](https://qwen.ai/) · [DashScope](https://help.aliyun.com/zh/dashscope/) | +| **GLM**（智谱 Zhipu）| GLM-5 / GLM-5.1 | 128k | 中文 / tool use / agent | 中文 agent / 多轮对话 | [open.bigmodel.cn](https://open.bigmodel.cn/) · [chatglm.cn](https://chatglm.cn/) | +| **Yi**（01.AI / 李开复）| Yi-Lightning / Yi-34B-Chat | 200k | **中文 OSS** 替代 Llama | 中文 self-host / 中文 API | [01.ai](https://01.ai/) · [GitHub](https://github.com/01-ai/Yi) | + +> **注**：这三家都走 **Apache 2.0 开源版 + 付费云端 API** 两条路（GLM 开源版是 5.1）。开源版可用 [Ollama](https://ollama.com/) 在自己机器跑。 + +> ⚠️ **小米 MiMo** 虽在 [`resources/cli-agents-guide.md`](../resources/cli-agents-guide.md) 列入 Hermes Agent routing，但 2026-05 无权威官方 source 可验证，暂不收进此表。要试 → 通过 [Hermes Agent](https://github.com/NousResearch/hermes-agent) 200+ provider routing 接入。 + +### 🌍 西方开源（4 家、self-host 主力） + +跑在自己机器、不付 API、隐私敏感场景的主力——可通过 [Ollama](https://ollama.com/) 一行指令装起来： + +| Model 家族 | 大小（活跃）| License | 强项 | 适合任务 | 官方 | +|---|---|---|---|---|---| +| **Llama**（Meta）| 3.3 70B | Llama Community License | 通用 / 生态最广 / Ollama 默认 | self-host 入门 / fine-tune base | [llama.com](https://www.llama.com/) · [HF Meta](https://huggingface.co/meta-llama) | +| **Gemma**（Google）| Gemma 4 26B MoE + 31B dense | Apache 2.0 | **小巧高效** / Apple MLX 整合好 / multimodal | Edge / mobile / 4-8GB RAM 机器 | [ai.google.dev/gemma](https://ai.google.dev/gemma) | +| **Mistral**（Mistral AI）| 7B / Mixtral 8x7B / Codestral | Apache 2.0（OSS 部分）| 开源 7B 级最强 | 商用 self-host / EU 主权 | [mistral.ai](https://mistral.ai/) · [HF Mistral](https://huggingface.co/mistralai) | +| **Phi**（Microsoft）| Phi-4 14B + multimodal | MIT | **小但强** / reasoning / 适合 edge | 4GB+ RAM / mobile / reasoning 入门 | [HF microsoft](https://huggingface.co/microsoft) | + +> **注**：Llama 4 截至 2026-05 还没发布（表中为 3.3）；Gemma 4 为 2026-04 发布、LMArena 开源组第 3；Phi-4 另有 multimodal 版。 + +### 🎯 我该选哪家？（按场景反查） + +| 你的场景 | 推荐 + 为什么 | +|---|---| +| 第一次学 LLM API、教材完整度优先 | **Claude** — Anthropic Cookbook + Courses 是社群公认最完整 | +| 写长文 / paper / code review | **Claude Sonnet** — long-form prose 强项 | +| 多模态（PDF / 影音 / 图）| **Gemini** 或 **Kimi** — 原生 multimodal | +| 广度查询 + function calling 框架 | **GPT** — ecosystem 最广、SDK 整合最深 | +| **中文场景 + 商业 API** | **Kimi**（长 context 强、能塞整本书）或 **DeepSeek**（cost 最低）或 **GLM**（agent 友好）| +| **中文场景 + 开源 self-host** | **Qwen 3**（Apache 2.0、目前中文最强 OSS）| +| 推理 / math（reasoning model）| **DeepSeek R1** / **Hunyuan T1** / **OpenAI o-series** | +| 隐私 / offline / 不付 API | **Llama 3.3** / **Gemma 4** / **Qwen 3 OSS** via [Ollama](https://ollama.com/) | +| Edge / 4GB RAM 机器 | **Gemma 4** / **Phi-4** / **Qwen 3（`qwen3-3B` 或以下版本）** | +| 100k+ token 大文件 | **Gemini 3.1**（2M context）或 **Kimi K2.6**（1M+）| +| **想 cost 最低**（API 账单敏感）| **DeepSeek V4-Flash** — 同级英文 model 中 token 单价最低 | + +### 📊 中立 benchmark 资源（自己 verify、不靠单一 source） + +| 资源 | 用途 | URL | 2026-05 状态 | +|---|---|---|---| +| **Artificial Analysis** | 第三方 benchmark + price/latency 整合（含中国 model）| https://artificialanalysis.ai/ | ✓ Active | +| **Arena AI**（前 LMSYS Chatbot Arena）| 人类盲测 ELO 排名 | https://arena.ai/leaderboard/text | ✓ Active | +| **Vellum LLM leaderboard** | 多 benchmark 整合 | https://www.vellum.ai/llm-leaderboard | ✓ Active | +| **HuggingFace OpenLLM Leaderboard** | 开源 model 排名 | https://huggingface.co/spaces/open-llm-leaderboard | ⚠️ 2026-05 偶尔 runtime error、改看 [Arena AI](https://arena.ai/) 开源 tab | +| **SuperCLUE**（中文 benchmark）| 中文场景权威评测 | https://www.superclueai.com/ | ✓ Active | + +### ⚠️ 重要警语 + +- ⚠️ **Benchmark ≠ production performance**——LLM 在你 specific 任务的表现要自己跑 small eval（例如贴 10 个你真实 prompt 看哪家答得最像你要的）、**不能只看排名选** +- ⚠️ **Frontier 6 个月洗牌一次**——上面所有数字是 **2026-05 snapshot**、之后请以**官方 docs** / [Artificial Analysis](https://artificialanalysis.ai/) 为准 +- ⚠️ **“强项”是相对的、不是绝对的**——所有 frontier model 都能完成基本任务、差别在特殊或困难的情境（超长文件、复杂推理、多语言） +- ⚠️ **中文场景看 [SuperCLUE](https://www.superclueai.com/)**——一般国际 benchmark（如 MMLU）以英文为主、中文表现可能跟英文不一致 + +## 🚪 进入条件 + +你需要具备以下基础： +- 编写 Python 脚本。 +- 理解基本的 HTTP / REST 概念。 +- 获取并使用 API key（Anthropic / OpenAI / Google）。 + +如果没有，请先完成 Stage 0。 + +## 📚 必修阅读 + +1. [**Anthropic - Claude 模型概览**](https://docs.claude.com/en/about-claude/models/overview) - 官方模型总览，包含 2026 的 Claude Fable 5（`claude-fable-5`、Mythos-class、2026-06-09 GA）以及 Opus 4.8 / Sonnet 5 / Haiku 4.5。**Fable 5 与姊妹版 Mythos 5（`claude-mythos-5`）曾于 2026-06-12 被美国出口管制指令暂停，出口管制已于 2026-06-30 解除；[Fable 5 于 2026-07-01 全球恢复](https://www.anthropic.com/news/redeploying-fable-5)（重新部署时加了新的安全 classifier），Mythos 5 仅对核准的美国组织恢复。Fable 5 是目前最高阶的 Claude 层级，Opus 4.8 为 Opus 级旗舰。** +2. [**anthropics/courses — Anthropic API Fundamentals**](https://github.com/anthropics/courses) ⭐⭐⭐⭐⭐ ★ 21k+ — Anthropic 官方 5 course umbrella、**module 1“Anthropic API Fundamentals”对应本 stage**。Jupyter notebook、用 Claude 3 Haiku（最便宜）跑、跟着做就能拿到 API 基本功。 +3. [**OpenAI Quickstart**](https://platform.openai.com/docs/quickstart) - 学习发送你的第一个 API call。 +4. [**A Visual Guide to LLM Tokenizers**](https://huggingface.co/learn/llm-course/chapter6/1) - Hugging Face 的图文并茂指南。 +5. [**Anthropic API Pricing**](https://www.anthropic.com/pricing#anthropic-api) - 了解并比较模型成本（例如，1k input + 1k output 的价格）。 + +## 🛠 动手练习（基础 illustrative 练习） + +> 🦙 **本 stage 默认用 Ollama**（成本考量、本机 `gemma4:e4b` 跑得动、$0/run）。每个练习都有 Path A（Ollama、默认）+ Path B（Anthropic、选择性、想看 cloud 高质量时用）。完整 3 路 trade-off 见 [`examples/README.zh-Hans.md`](../examples/README.zh-Hans.md#三条路径--默认用-ollama成本考量)。 +> +> 💰 **Stage 1 预算估算**（全 6 练习各跑 3-5 次）：**全本机 = $0**、**全 haiku ≈ $0.30**、**全 sonnet ≈ $0.90**。完整 model 清单 + Stage 1-7 全程预算估算见 [`examples/README.zh-Hans.md#推荐-llm-清单`](../examples/README.zh-Hans.md#推荐-llm-清单)。 +> +> 💡 **不装 Ollama 也能读** — 每个练习的 Path B 区块就是 Anthropic 版、选一个跑就行。先 [`pip install openai && ollama pull gemma4:e4b`](https://ollama.com) 就装好 Path A 环境。 + +### 练习 1：LLM API（hello world） +五行 Python 调用 LLM 并打印响应。**默认用 Ollama 本机跑（免费、offline）**；想看 cloud 答案质量改 Path B Anthropic。详见 [`examples/README.zh-Hans.md`](../examples/README.zh-Hans.md#三条路径--默认用-ollama成本考量)。 + +
+📋 起手码 — Path A（本机 Ollama gemma4:e4b、默认）（复制到 practice_1.py、python practice_1.py 就跑） + +```python +# 需要：pip install openai (用 OpenAI 兼容 SDK 跟 Ollama 沟通) +# 前置：ollama pull gemma4:e4b && ollama serve +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI( + base_url="http://localhost:11434/v1", + api_key="ollama", # Ollama 不检查、随便填 +) + +r = client.chat.completions.create( + model="gemma4:e4b", # 换成 qwen2.5:3b / llama3.2:3b 也可 + max_tokens=100, + messages=[{"role": "user", "content": "用一句话自我介绍。"}], +) + +# === 自我验证 === +text = r.choices[0].message.content +print("回应：", text) +print("usage:", r.usage) + +assert r.choices[0].finish_reason in ("stop", "length"), f"非预期 finish_reason: {r.choices[0].finish_reason}" +assert len(text) > 0, "回应不应为空" +assert r.usage.completion_tokens > 0, "output token 应 > 0" +print("✅ 练习 1 通过 — Ollama gemma4:e4b 已能本机回应、$0/次") +``` + +**慢吗？** Gemma 4B 在 CPU 上约 5-30s/答案、有 GPU（RTX 3060+）<2s。要更快用 `gemma3:1b`、要更聪明改 `qwen2.5:14b` / `llama3.3:8b`（需 8GB+ VRAM）。 + +
+ +
+📋 起手码 — Path B（Anthropic API、选择性、想看 cloud 高质量时）（复制到 practice_1_anthropic.py） + +```python +# 需要：pip install anthropic +# 环境变量：export ANTHROPIC_API_KEY=sk-ant-... +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic + +client = anthropic.Anthropic() +msg = client.messages.create( + model="claude-haiku-4-5", # haiku 最便宜；换 sonnet 改这行 + max_tokens=100, + messages=[{"role": "user", "content": "用一句话自我介绍。"}], +) + +# === 自我验证 === +text = msg.content[0].text +print("回应：", text) +print("usage:", msg.usage) + +assert msg.stop_reason in ("end_turn", "max_tokens"), f"非预期 stop_reason: {msg.stop_reason}" +assert len(text) > 0, "回应不应为空" +assert msg.usage.input_tokens > 0 and msg.usage.output_tokens > 0, "token 数应 > 0" +print("✅ 练习 1 通过 — 你已成功打通 Anthropic API") +``` + +**成本**：每次 ~$0.001 (haiku) / $0.004 (sonnet)、跑这个 hello world 比 Ollama 快 5-15 倍。 + +
+ +### 练习 2：Tokens +同一个 prompt 跑 100 次，观察 token 数的变化。 +- 注意：`temperature ≠ 0` 会产生变动 +- 注意：同一句话的英文 vs 中文 token 数差异 + +
+📋 起手码 — Path A（本机 Ollama gemma4:e4b、默认）（复制到 practice_2.py） + +```python +# 需要：pip install openai +# 前置：ollama pull gemma4:e4b && ollama serve +import sys, statistics +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +PROMPTS = { + "中文": "用一句话描述一只猫在做什么。", + "English": "Describe in one sentence what a cat is doing.", +} + +N = 10 # 本机慢、N 小一点 +for label, prompt in PROMPTS.items(): + output_tokens = [] + for _ in range(N): + r = client.chat.completions.create( + model="gemma4:e4b", + max_tokens=80, + temperature=1.0, + messages=[{"role": "user", "content": prompt}], + ) + output_tokens.append(r.usage.completion_tokens) + print(f"\n[{label}] prompt: {prompt}") + print(f" input tokens: {r.usage.prompt_tokens}") + print(f" output tokens — min={min(output_tokens)} max={max(output_tokens)} mean={statistics.mean(output_tokens):.1f} stdev={statistics.stdev(output_tokens):.1f}") + +# === 自我验证 === +assert max(output_tokens) > min(output_tokens), "temperature=1.0 下、output 长度应该有 variance" +print("\n✅ 练习 2 通过 — 本机跑 $0") +print("💡 中文 prompt 通常 input tokens 比 English 多（中文 token 化通常一字 ≈ 2 tokens）") +``` + +
+ +
+📋 起手码 — Path B（Anthropic API、选择性）（复制到 practice_2_anthropic.py） + +```python +# 需要：pip install anthropic +import sys, statistics +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic +client = anthropic.Anthropic() +PROMPTS = {"中文": "用一句话描述一只猫在做什么。", "English": "Describe in one sentence what a cat is doing."} + +for label, prompt in PROMPTS.items(): + output_tokens = [] + for _ in range(20): + msg = client.messages.create(model="claude-haiku-4-5", max_tokens=80, temperature=1.0, + messages=[{"role": "user", "content": prompt}]) + output_tokens.append(msg.usage.output_tokens) + print(f"[{label}] input={msg.usage.input_tokens} output min/max/mean={min(output_tokens)}/{max(output_tokens)}/{sum(output_tokens)/len(output_tokens):.1f}") +``` + +**主要差异**：`messages.create` → `chat.completions.create`；`usage.output_tokens` → `usage.completion_tokens`；`usage.input_tokens` → `usage.prompt_tokens`。**成本**：40 次 ≈ $0.01。 + +
+ +### 练习 3：Pricing / Latency +**成本敏感的工作必修**：算出你的 hello-world prompt 在不同 model 上跑 1000 次的成本。Ollama 本机是 $0 但有 latency 成本；Cloud LLM 有 $ 成本但快。**会算这两个 trade-off 才能挑对 model**。 + +
+📋 起手码 — Path A（本机 Ollama gemma4:e4b、量 latency）（复制到 practice_3.py） + +```python +# 需要：pip install openai +# 前置：ollama pull gemma4:e4b && ollama serve +import sys, time +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +latencies = [] +for _ in range(5): + t0 = time.time() + r = client.chat.completions.create( + model="gemma4:e4b", + max_tokens=200, + messages=[{"role": "user", "content": "你好！自我介绍一下。"}], + ) + latencies.append(time.time() - t0) + +avg_latency = sum(latencies) / len(latencies) +out_tok_avg = r.usage.completion_tokens +tps = out_tok_avg / avg_latency if avg_latency > 0 else 0 + +print(f"model: gemma4:e4b (本机)") +print(f"5 次 latency (sec): min={min(latencies):.2f} max={max(latencies):.2f} mean={avg_latency:.2f}") +print(f"avg output: {out_tok_avg} tokens、约 {tps:.1f} tokens/sec") +print(f"\n1000 次成本: $0 (本机)、预计时长: {avg_latency * 1000 / 60:.1f} 分钟") + +# === 自我验证 === +assert avg_latency > 0, "latency 应 > 0" +assert out_tok_avg > 0, "output token 应 > 0" +print(f"\n✅ 练习 3 通过 — 本机 model $0 但要花 {avg_latency * 1000 / 60:.0f} 分钟跑 1000 次") +print("💡 对照 Path B Anthropic：1000 次只要 ~10-20 分钟但要 $0.25（haiku）") +``` + +
+ +
+📋 起手码 — Path B（Anthropic API、算 $ 成本）（复制到 practice_3_anthropic.py） + +```python +# 需要：pip install anthropic +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic + +# Anthropic 2026 Q2 公开计价（每 1M token、USD）— 运行前对照 https://www.anthropic.com/pricing +PRICING = { + "claude-haiku-4-5": {"input": 1.00, "output": 5.00}, + "claude-sonnet-5": {"input": 3.00, "output": 15.00}, + "claude-opus-4-8": {"input": 5.00, "output": 25.00}, # Opus 4.8（2026 年 5 月、Dynamic Workflows）—— 维持 5/25 同价 + "claude-fable-5": {"input": 10.00, "output": 50.00}, # Fable 5（Mythos-class、2026-07-01 恢复可用）约 Opus 4.8 的 2 倍 +} + +client = anthropic.Anthropic() +MODEL = "claude-haiku-4-5" +msg = client.messages.create(model=MODEL, max_tokens=200, + messages=[{"role": "user", "content": "你好！自我介绍一下。"}]) +in_tok, out_tok = msg.usage.input_tokens, msg.usage.output_tokens +rates = PRICING[MODEL] +cost_one = (in_tok * rates["input"] + out_tok * rates["output"]) / 1_000_000 + +print(f"model: {MODEL}") +print(f"single: input={in_tok} output={out_tok} → ${cost_one:.6f}") +print(f"1000 calls cost across model tiers:") +for name, r in PRICING.items(): + c = (in_tok * r["input"] + out_tok * r["output"]) / 1_000_000 * 1000 + print(f" {name:<22} ${c:.4f}") + +assert cost_one > 0, "Cloud LLM 一定有成本" +print(f"\n✅ 练习 3 通过（Anthropic）— 1000 次 haiku ≈ $0.25、sonnet 5 ≈ $0.76、opus 4.8 ≈ $1.27") +``` + +**预期输出**： +``` +model: claude-haiku-4-5 +single: input=14 output=48 → $0.000254 +1000 calls cost across model tiers: + claude-haiku-4-5 $0.2540 + claude-sonnet-5 $0.7620 + claude-opus-4-8 $1.2700 +``` + +**Trade-off 对照**：本机 Ollama 跑 1000 次免费但要 ~2 hr；Anthropic haiku ~10 min $0.25；sonnet ~10 min $0.76。**production 场景才考虑 cloud；学习 / 实验 / debug 全用本机**。 + +
+ +### 练习 4：Cross-Provider 比较 +同一个 prompt 同时送给 Claude、GPT、Gemini，比较三家的响应差异。观察“同一句话为什么产生不同答案”——回答风格、长度、判断取舍都不一样。建议用 OpenAI、Anthropic、Google 三家 SDK 各一段程序调用。 + +→ **基础 starter 范本** → [`examples/stage-1/04-cross-provider/`](../examples/stage-1/04-cross-provider/)（含三家 SDK 并行调用 + table 对照、缺哪家 key 就 skip 哪家；illustrative，**不是 chapter-length 完整教程**） + +### 练习 5：Error Handling +故意触发错误情境并写 retry： +- API key 错误 → 看怎么 raise +- prompt 超长 → context window 满了会发生什么 +- 网络断掉 → 写一个有 exponential backoff 的 retry wrapper + +这是后面 Stage 3-8 写 production agent 一定会用到的基础。 + +→ **基础 starter 范本** → [`examples/stage-1/05-error-handling/`](../examples/stage-1/05-error-handling/)（含 mock-based test、不用真的断网就能验证 retry 逻辑；illustrative，**不是 chapter-length 完整教程**） + +### 练习 6：Local LLM +**不付 API 费用、跑在自己电脑上**：用 Ollama 下载一个小模型（建议 `llama3.2:3b` 或 `qwen2.5:3b`），用 OpenAI 兼容 API 调用它。 + +```bash +# 1. 装 Ollama: https://ollama.com +ollama pull qwen2.5:3b +ollama serve # 预设 port 11434 +``` + +
+📋 起手码（复制到 practice_6.py） + +```python +# 需要：pip install openai +# 前置：Ollama 已 serve、qwen2.5:3b 已 pull +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI( + base_url="http://localhost:11434/v1", + api_key="ollama", # Ollama 不检查、随便填 +) + +r = client.chat.completions.create( + model="qwen2.5:3b", + messages=[{"role": "user", "content": "用 3 句话介绍什么是 ReAct。"}], +) + +text = r.choices[0].message.content +print("回应：", text) + +# === 自我验证 === +assert len(text) > 10, "回应太短、Ollama 可能没跑起来" +print(f"✅ 练习 6 通过 — 你的本机 Ollama 已能透过 OpenAI 兼容 API 呼叫") +print(f"💡 跑这次完全没花钱（除了你的电力）") +``` + +**为什么要做**：学会跑本地 LLM 后，后面 Stage 3-6 的实验都不会被 API 费用卡住；隐私敏感场景也能 offline。 + +
+ +## 🎯 精选 Projects + +按用途分 5 类、17 个项目一张表搞定。**挑入口看“适合谁”、想深入点连结看 repo / 课程网站**。 + +| 分类 | Project | ⭐ | 适合谁 | 为什么推荐 / 备注 | +|---|---|---|---|---| +| **官方 cookbook / 入门** | [Anthropic Cookbook](https://github.com/anthropics/claude-cookbooks) | ⭐⭐⭐⭐⭐ | 开始用 Claude API、当参考书查 | Claude API 全功能 notebook（tool use / batch / prompt cache），★ 46k+、MIT | +| | [Anthropic Courses](https://github.com/anthropics/courses) | ⭐⭐⭐⭐⭐ | 系统性从零学一遍 Claude | Anthropic 自家完整 5 门课（API 基础 / prompt eval / real-world prompting / tool use），★ 21k+。先跑 `anthropic_api_fundamentals` | +| | [OpenAI Cookbook](https://github.com/openai/openai-cookbook) | ⭐⭐⭐⭐⭐ | 用 OpenAI API + structured output / function calling | 跟 Anthropic Cookbook 对照、★ 73k+、MIT。比 Anthropic 大很多、用搜索 | +| | [Anthropic Claude API Quickstart](https://docs.anthropic.com/en/docs/get-started) | ⭐⭐⭐⭐ | 5 分钟上手 | 官方文件、加 bookmark 用 | +| **中文教材**
（章节式） | [datawhalechina/happy-llm](https://github.com/datawhalechina/happy-llm) | ⭐⭐⭐⭐⭐ | 中文读者想彻底搞懂 LLM 原理 | 对应 Karpathy“Zero to Hero”中文版，★ 29k+。等同 HF LLM Course 中文版 | +| | [datawhalechina/llm-universe](https://github.com/datawhalechina/llm-universe) | ⭐⭐⭐⭐⭐ | 中文新手想用 LLM 做东西 | API 基础 / 知识库 / RAG / 进阶技巧，★ 13k+ | +| | [datawhalechina/llm-cookbook](https://github.com/datawhalechina/llm-cookbook) | ⭐⭐⭐⭐ | 想要完整中文 LLM 学习路线 | Andrew Ng 课程中文翻译改编（⚠️ 2025-06 后更新放缓、CC BY-NC-SA）| +| | [jingyaogong/minimind](https://github.com/jingyaogong/minimind) | ⭐⭐⭐⭐ | 看完 Karpathy 视频想实际跑训练 | 2hr 从零训 64M LLM、Pretrain + SFT + LoRA + DPO + RLHF 全包，★ 48k+、Apache-2.0 | +| **英文 course**
（系统式） | [HuggingFace — LLM Course](https://huggingface.co/learn/llm-course) | ⭐⭐⭐⭐⭐ | 想搞懂 transformer 内部 + HF 生态 | 含 transformer 原理 + 应用、Apache 2.0 | +| | [LangChain Academy](https://academy.langchain.com/) | ⭐⭐⭐⭐ | 喜欢视频教学的视觉型学习者 | LangChain 官方免费课、含 RAG / agent。**忽略 LangChain 行销段落** | +| **本地端执行**
（不付 API 费）| [ollama/ollama](https://github.com/ollama/ollama) | ⭐⭐⭐⭐⭐ | 第一次跑本地 LLM | 本 repo Path A 预设、OpenAI-compat API、★ 170k+ | +| | [ggml-org/llama.cpp](https://github.com/ggml-org/llama.cpp) | ⭐⭐⭐⭐⭐ | 想搞懂 quantization / 为什么 7B 能塞 8GB RAM | Ollama 底层 inference engine，★ 119k+、MIT | +| | [mudler/LocalAI](https://github.com/mudler/LocalAI) | ⭐⭐⭐⭐ | 团队合规、要 self-host 全套 OpenAI 替代 | drop-in OpenAI API 替代品（chat / embedding / image / TTS / STT），★ 46k+ | +| | [ml-explore/mlx](https://github.com/ml-explore/mlx) | ⭐⭐⭐⭐ | Mac 开发、想榨干 Apple Silicon | Apple 为 M1+ 量身打造的 ML framework，★ 25k+。搭 `mlx-lm` 用最方便 | +| **从零打造**
（理解原理）| [karpathy — Let's build GPT from scratch](https://www.youtube.com/watch?v=kCc8FmEb1nY) | ⭐⭐⭐⭐⭐ | 想搞懂 LLM 内部、不只会调用 | 2hr 高密度视频、用 PyTorch 从零打造 GPT。**暂停跟着写 code 不要被动看** | +| | [rasbt/LLMs-from-scratch](https://github.com/rasbt/LLMs-from-scratch) | ⭐⭐⭐⭐⭐ | 想用整本书速度慢慢读完 | Karpathy 视频的书本版：tokenizer → attention → pretraining → finetuning，★ 91k+、Apache-2.0 | +| | [karpathy/LLM101n](https://github.com/karpathy/LLM101n) | ⭐⭐ | 历史纪录 | ⚠️ 已封存（2024-08）、只有大纲、课程没做完。**直接看上面的“Build GPT from scratch”视频即可** | + +> 💡 **建议阅读路径**：API 入手就 Anthropic / OpenAI Cookbook → 中文系统路线就 happy-llm + llm-universe → 想深入内部就 Karpathy 视频 + rasbt 书搭 code → 想跑本地就 Ollama 起步、进阶再读 llama.cpp。 + +## ✅ 进 Stage 2 前的自我检查 + +你需要完成以下任务： + +- [ ] 写一个 5 行的 Python 脚本调用 Claude API。 +- [ ] 理解“基础概念”中的至少 2 个 token（例如，“Hello” 是 1 个）。 +- [ ] 比较 Claude Sonnet vs Opus 的 per-token 价格。 +- [ ] 体验至少 2 个不同的 LLM（Claude / GPT / Gemini / Llama）。 + +如果都完成了，恭喜，进入 [Stage 2 - Prompt Engineering](./02-prompt-engineering.zh-Hans.md)。 + +如果卡住了，回到 Anthropic Quickstart + 完成至少 3 个 hello-X 脚本。 + +--- + +> ✅ **Stage 1 完成？** 接下来 [**Stage 2 — Prompt Engineering**](02-prompt-engineering.zh-Hans.md) 会用 5-12 小时带你写出可重用的结构化 prompt、用 few-shot 跟 chain-of-thought 解推理题、并学会用 eval 量化 prompt 改善幅度。**继续往下走 →** diff --git a/stages/02-prompt-engineering.en.md b/stages/02-prompt-engineering.en.md new file mode 100644 index 0000000..17c8126 --- /dev/null +++ b/stages/02-prompt-engineering.en.md @@ -0,0 +1,473 @@ +# Stage 2 — Prompt Engineering + +> [繁體中文](./02-prompt-engineering.md) | [简体中文](./02-prompt-engineering.zh-Hans.md) | **English** + + +⏱ **Time estimate**: 1-2 weeks (~5-12 hours) + +> 👋 **Coming from [Stage 1](01-llm-basics.en.md)?** Good — you can call an API. The next 5-12 hours: write reusable structured prompts, use few-shot and chain-of-thought for hard reasoning tasks, and quantify prompt improvement with evals. **Jumped straight here?** Make sure you can call an LLM API and estimate cost in tokens — if not, head back to [Stage 1](01-llm-basics.en.md). + +> 💡 Term-unfamiliar? (prompt / few-shot / CoT / system prompt / …) → see [`resources/glossary.en.md`](../resources/glossary.en.md). + +## 📌 Learning Goals + +After this stage you will be able to: +- Write structured prompts (role + task + format + examples) +- Apply few-shot prompting and know when it helps +- Use chain-of-thought (CoT) for reasoning tasks +- Iteratively refine a prompt and measure improvement +- Recognize when prompting hits its limit (and you need tools / agents) + +## 🚪 Entry Conditions + +You should already: +- Be able to call an LLM API (Stage 1) +- Be able to parse / iterate over API responses + +## 📚 Required Reading + +1. [**anthropics/prompt-eng-interactive-tutorial**](https://github.com/anthropics/prompt-eng-interactive-tutorial) ⭐⭐⭐⭐⭐ ★ 35k+ — **Anthropic's official interactive tutorial**, 9 chapters of Jupyter notebooks (basic / intermediate / advanced + appendix), with playground and answer key. Runs on Claude 3 Haiku (cheapest). **The canonical hands-on resource for Stage 2.** Also packaged as module 2 of the [**anthropics/courses**](https://github.com/anthropics/courses) 5-course umbrella — for broader coverage (API Fundamentals / Real World Prompting / Eval / Tool Use) go straight to the umbrella +2. [**anthropics/courses — Real World Prompting**](https://github.com/anthropics/courses) ⭐⭐⭐⭐ ★ 21k+ — Module 3 of the same umbrella, **"how to actually use prompting in real situations"**: chatbot / legal / financial / coding case walkthroughs. Read #1 first, then this. +3. [**Anthropic Prompt Engineering Guide**](https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/overview) — official docs, pairs with #1 +3. [**OpenAI Prompt Engineering**](https://platform.openai.com/docs/guides/prompt-engineering) — OpenAI's perspective +4. [**dair-ai Prompt Engineering Guide**](https://www.promptingguide.ai/) — academic-flavored, in-depth +5. [**Anthropic — Prompting Best Practices**](https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/be-clear-and-direct) — be clear and direct + +## 🛠 Hands-on Exercises + +> 🦙 **This stage defaults to Ollama gemma4:e4b** (cost-driven; $0/run). Prompt engineering is especially instructive on small models — they are sensitive to prompt quality, so you can clearly see how much each technique (system prompts, few-shot, CoT, refinement) improves output. Every exercise has Path A (Ollama, default) + Path B (Anthropic, optional). +> +> 💰 **Stage 2 budget estimate** (4 exercises, 3-5 runs each): **all local = $0**, **all haiku ≈ $0.20**, **all sonnet ≈ $0.60**. The few-shot classifier alone is 12 calls × 5 reps ≈ $0.30 haiku / $0.90 sonnet. Full budget: [`examples/README.en.md#recommended-llm-list`](../examples/README.en.md#recommended-llm-list). +> +> Full three-path trade-off in [`examples/README.en.md`](../examples/README.en.md#three-paths--default-is-ollama-cost-driven). + +### Exercise 1: System Prompt +Same user message, three different system prompts. Watch the personality / output format change. + +
+📋 Starter code — Path A (local Ollama gemma4:e4b, default) (copy to practice_1.py) + +```python +# Requires: pip install openai +# Pre-req: ollama pull gemma4:e4b && ollama serve +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +SYSTEM_PROMPTS = { + "Strict lawyer": "You are a precise contract lawyer. Cite statute numbers, avoid subjective adjectives.", + "Kindergarten teacher": "You are a kind kindergarten teacher speaking to a 5-year-old. Use analogies, colloquial language, under 80 words.", + "JSON machine": "Reply only in JSON. schema: {\"answer\": string, \"confidence\": float}", +} + +USER_MSG = "Explain what a lease agreement is." + +outputs = {} +for label, system in SYSTEM_PROMPTS.items(): + # Ollama (OpenAI-compatible) puts system in the messages array (Anthropic uses system=) + r = client.chat.completions.create( + model="gemma4:e4b", + max_tokens=200, + messages=[ + {"role": "system", "content": system}, + {"role": "user", "content": USER_MSG}, + ], + ) + outputs[label] = r.choices[0].message.content + print(f"\n--- [{label}] ---") + print(outputs[label]) + +# === Self-check === +import json +last_text = outputs["JSON machine"] +assert "{" in last_text and "}" in last_text, "JSON-machine output should contain JSON braces" +try: + parsed = json.loads(last_text.strip().split("\n")[-1] if "\n" in last_text else last_text) + assert "answer" in parsed, "schema expects an 'answer' field" +except json.JSONDecodeError: + pass # some models add prose around the JSON; tolerate that + +print(f"\n✅ Exercise 1 passed — same question, three different personas / formats / tones") +``` + +
+ +
+📋 Starter code — Path B (Anthropic API, optional) (copy to practice_1_anthropic.py) + +```python +# Requires: pip install anthropic +import sys, json +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic +client = anthropic.Anthropic() + +SYSTEM_PROMPTS = { + "Strict lawyer": "You are a precise contract lawyer. Cite statute numbers, avoid subjective adjectives.", + "Kindergarten teacher": "You are a kind kindergarten teacher speaking to a 5-year-old. Use analogies, colloquial language, under 80 words.", + "JSON machine": "Reply only in JSON. schema: {\"answer\": string, \"confidence\": float}", +} +USER_MSG = "Explain what a lease agreement is." + +outputs = {} +for label, system in SYSTEM_PROMPTS.items(): + # Anthropic uses `system=` parameter (not part of messages array) + msg = client.messages.create(model="claude-haiku-4-5", max_tokens=200, + system=system, messages=[{"role": "user", "content": USER_MSG}]) + outputs[label] = msg.content[0].text + print(f"\n--- [{label}] ---") + print(outputs[label]) + +# Self-check (same JSON-shape assert; schema is cross-backend) +json_output = outputs["JSON machine"] +assert "{" in json_output and "}" in json_output +print(f"\n✅ Exercise 1 passed (Anthropic)") +``` + +**Key difference**: Anthropic uses `system=` parameter; OpenAI/Ollama puts system in messages array. Claude follows system prompts more strictly than gemma4:e4b — the "Strict lawyer" persona will actually cite statute numbers. **Cost**: ~$0.003 / 3 personas. + +
+ +### Exercise 2: Few-Shot + +**Get these three terms first** — the only difference is how many examples you show the LLM: +- **Zero-shot (0-shot)**: no examples, just ask. +- **One-shot (1-shot)**: give **1** "input → answer" example first, then ask. +- **Few-shot (the 3-shot below is exactly this)**: give a few (usually 2–5) examples first — the LLM copies their format and judgment criteria, and accuracy usually jumps. + +Pick a classification task. Run it 0-shot, then 3-shot. Measure accuracy difference. + +
+📋 Starter code — Path A (local Ollama gemma4:e4b, default) (copy to practice_2.py) + +```python +# Requires: pip install openai +# Pre-req: ollama pull gemma4:e4b && ollama serve +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +# Sentiment classifier: positive / negative / neutral +TEST_SET = [ + ("This movie was amazing — I want to watch it again!", "positive"), + ("Boring plot, awkward acting.", "negative"), + ("This is a 2019 film.", "neutral"), + ("Not sure how I feel about it, might think more.", "neutral"), + ("Season 1 was great but season 2 fell apart.", "negative"), + ("Left in a great mood — recommended!", "positive"), +] + +FEW_SHOT_EXAMPLES = """Examples: +input: The steak at this place made me cry tears of joy. +output: positive + +input: The waiter was rude. Never coming back. +output: negative + +input: This shop is in New Taipei City. +output: neutral +""" + +# Both conditions share the SAME task instruction; few-shot only adds examples. That keeps the comparison clean — what you measure is the effect of the examples themselves, not "finally telling the model what the task is." +TASK = "Classify the sentence below into exactly one of positive / negative / neutral. Output only one of those three words, nothing else.\n\n" + + +def classify(text: str, *, use_few_shot: bool) -> str: + prefix = FEW_SHOT_EXAMPLES + "\n" if use_few_shot else "" + prompt = f"{TASK}{prefix}input: {text}\noutput:" + r = client.chat.completions.create( + model="gemma4:e4b", + max_tokens=10, + messages=[{"role": "user", "content": prompt}], + ) + return r.choices[0].message.content.strip().splitlines()[0] + + +def evaluate(use_few_shot: bool) -> tuple[int, int]: + correct = 0 + for text, label in TEST_SET: + pred = classify(text, use_few_shot=use_few_shot) + ok = label in pred + print(f" {'✓' if ok else '✗'} [{label}] {text[:30]}... → '{pred}'") + if ok: + correct += 1 + return correct, len(TEST_SET) + + +print("=== 0-shot ===") +c0, n = evaluate(use_few_shot=False) +print(f"correct {c0}/{n} = {c0/n:.0%}") + +print("\n=== 3-shot ===") +c3, _ = evaluate(use_few_shot=True) +print(f"correct {c3}/{n} = {c3/n:.0%}") + +# === Self-check === +# Both conditions got the same task instruction, so this measures the effect of the EXAMPLES alone. +# few-shot isn't guaranteed to win (depends on the model / test set / sampling), so we don't assert c3 >= c0. +assert n == 6 and 0 <= c0 <= n and 0 <= c3 <= n, "both conditions must run all 6 items" +print(f"\n✅ Exercise 2 passed — 0-shot {c0}/{n}, 3-shot {c3}/{n}; few-shot net gain {c3 - c0} (may be 0 or even negative — that's normal) (local, $0)") +print("💡 With a clear instruction, 0-shot already has a baseline; few-shot's value is pinning the output FORMAT + showing judgment on ambiguous cases (like 'neutral')") +print("💡 Small models (gemma4:e4b) are more format-sensitive, so few-shot usually helps them more than Claude — still not guaranteed, so you measure") +``` + +
+ +
+📋 Starter code — Path B (Anthropic API, optional) (copy to practice_2_anthropic.py) + +```python +# Requires: pip install anthropic +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic +client = anthropic.Anthropic() + +# Same TASK / TEST_SET / FEW_SHOT_EXAMPLES as Path A — only the classify() body changes: +def classify(text: str, *, use_few_shot: bool) -> str: + prefix = FEW_SHOT_EXAMPLES + "\n" if use_few_shot else "" + msg = client.messages.create( + model="claude-haiku-4-5", + max_tokens=10, + messages=[{"role": "user", "content": f"{TASK}{prefix}input: {text}\noutput:"}], + ) + return msg.content[0].text.strip().splitlines()[0] +# Rest of TASK / TEST_SET / FEW_SHOT_EXAMPLES / evaluate() stays identical to Path A +``` + +**Cost**: 12 calls ≈ $0.005. Claude is usually accurate at 0-shot already, so the few-shot lift is smaller than on gemma4:e4b — that contrast is the actual teaching point. + +
+ +### Exercise 3: CoT +Pick a math word problem. Compare: +- Plain prompt +- Plain prompt + "Let's think step by step" +- Plain prompt + worked example showing CoT + +
+📋 Starter code — Path A (local Ollama gemma4:e4b, default) (copy to practice_3.py) + +```python +# Requires: pip install openai +# Pre-req: ollama pull gemma4:e4b && ollama serve +import sys, re +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +QUESTION = "Tom has 3 apples. He gives Sarah 1, then mom gives him 5 more, then he eats 2. How many does he have now?" +ANSWER = 5 # 3 - 1 + 5 - 2 = 5 + +COT_EXAMPLE = """Example: +Q: A chicken has 2 legs. 3 chickens and 1 person — how many legs total? +A: Let me work through this step by step. 3 chickens × 2 legs = 6 legs. 1 person has 2 legs. Total 6 + 2 = 8 legs. The answer is 8. +""" + + +def ask(prompt: str) -> str: + r = client.chat.completions.create( + model="gemma4:e4b", + max_tokens=300, + messages=[{"role": "user", "content": prompt}], + ) + return r.choices[0].message.content + + +def extract_number(text: str) -> int | None: + """Pull the last number from the response as the answer.""" + nums = re.findall(r"-?\d+", text) + return int(nums[-1]) if nums else None + + +out_a = ask(QUESTION) +ans_a = extract_number(out_a) + +out_b = ask(QUESTION + "\nLet's think step by step.") +ans_b = extract_number(out_b) + +out_c = ask(COT_EXAMPLE + "\n\nQ: " + QUESTION + "\nA:") +ans_c = extract_number(out_c) + +for label, out, ans in [("A plain", out_a, ans_a), ("B +step-by-step", out_b, ans_b), ("C +CoT example", out_c, ans_c)]: + print(f"\n--- [{label}] answer={ans} {'✓' if ans == ANSWER else '✗'} ---") + print(out[:200]) + +# === Self-check === +correct = sum(1 for a in (ans_a, ans_b, ans_c) if a == ANSWER) +assert correct >= 1, f"at least 1 of 3 prompts should be correct, got {correct}/3" +assert ans_b == ANSWER or ans_c == ANSWER, "B (step-by-step) or C (CoT example) must be correct — CoT is non-negotiable for small models" +print(f"\n✅ Exercise 3 passed — {correct}/3 correct") +``` + +
+ +
+📋 Starter code — Path B (Anthropic API, optional) (copy to practice_3_anthropic.py) + +Same logic as Path A, just swap the client and `ask()`: + +```python +import anthropic +client = anthropic.Anthropic() + +def ask(prompt: str) -> str: + msg = client.messages.create(model="claude-haiku-4-5", max_tokens=300, + messages=[{"role": "user", "content": prompt}]) + return msg.content[0].text +# Rest (QUESTION, ANSWER, COT_EXAMPLE, extract_number, 3 calls, assert) stays identical +``` + +**Claude typically gets 3/3 right** including the plain-prompt baseline — that contrast with gemma4:e4b (where CoT is essential) is the actual teaching point. **Cost**: 3 calls ≈ $0.002. + +
+ +> 🧠 **When NOT to hand-write CoT**: for **reasoning-native models** (Claude Opus 4.x, the o-series, Gemini thinking, and other models with built-in thinking), using their extended thinking is usually better than hand-writing "Let's think step by step"; forcing your own steps can interfere with their native reasoning. Hand-written CoT still applies to plain chat models without built-in reasoning. + +### Exercise 4: Iterative Refinement +Take a vague prompt, refine it 5 times. Track the iterations. Notice what changes improve quality. + +
+📋 Starter code — Path A (local Ollama gemma4:e4b, default) (copy to practice_4.py) — this exercise has no "right answer"; the point is observing the process + +```python +# Requires: pip install openai +# Pre-req: ollama pull gemma4:e4b && ollama serve +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +# 5 iterations, each adds one constraint +PROMPTS = { + "v1 vague": "Write a paragraph about ReAct.", + "v2 +audience": "Write a paragraph about ReAct for software engineers who know Python.", + "v3 +format": "Write a paragraph about ReAct for software engineers who know Python. Under 100 words, single paragraph.", + "v4 +example": "Write a paragraph about ReAct for software engineers who know Python. Under 100 words, single paragraph, ending with a concrete example (e.g. weather lookup).", + "v5 +bans": "Write a paragraph about ReAct for software engineers who know Python. Under 100 words, single paragraph, ending with a concrete example (e.g. weather lookup). Avoid words like 'empower', 'leverage', 'intelligent'.", +} + +outputs = {} +for label, prompt in PROMPTS.items(): + r = client.chat.completions.create( + model="gemma4:e4b", + max_tokens=200, + messages=[{"role": "user", "content": prompt}], + ) + text = r.choices[0].message.content + outputs[label] = text + print(f"\n--- [{label}] ({len(text)} chars) ---") + print(text) + +# === Self-check === +v1_len, v5_len = len(outputs["v1 vague"]), len(outputs["v5 +bans"]) +banned_words = ("empower", "leverage", "intelligent") +v5_text_lower = outputs["v5 +bans"].lower() +v5_has_banned = any(w in v5_text_lower for w in banned_words) +assert v5_len > 0, "v5 must have output" +assert not v5_has_banned, f"v5 should avoid banned words; got: {[w for w in banned_words if w in v5_text_lower]}" +print(f"\n✅ Exercise 4 passed — v5 length {v5_len}, no banned words") +print(f"💡 Observe: v1 ({v1_len} chars) is typically looser than v5 ({v5_len} chars); constraints tighten prompts") +print("💡 The 5 dimensions: (1) target audience (2) format (3) length (4) example demand (5) banned words") +``` + +
+ +
+📋 Starter code — Path B (Anthropic API, optional) (copy to practice_4_anthropic.py) + +Same loop and PROMPTS as Path A, with Anthropic SDK: + +```python +import anthropic +client = anthropic.Anthropic() + +outputs = {} +for label, prompt in PROMPTS.items(): + msg = client.messages.create(model="claude-haiku-4-5", max_tokens=200, + messages=[{"role": "user", "content": prompt}]) + outputs[label] = msg.content[0].text +# Rest (length compare, banned-word assert) stays identical +``` + +**Cost**: 5 calls ≈ $0.002. **Claude's v1 is already coherent** so the v5 lift is smaller; gemma4:e4b makes the lift more dramatic. + +
+ +## 🎯 Curated Projects + +4 categories, 9 projects in one table. **Pick by "Best for"; click through for depth on the repo / site.** + +| Category | Project | ⭐ | Best for | Why / Notes | +|---|---|---|---|---| +| **Academic / teaching-style guide**
(start here) | [dair-ai/Prompt-Engineering-Guide](https://github.com/dair-ai/Prompt-Engineering-Guide) | ⭐⭐⭐⭐⭐ | Reference book; look up a specific technique | Basics to advanced (CoT / ToT / ReAct / RAG) end to end, ★ 74k+, MIT | +| | [PromptingGuide.ai](https://www.promptingguide.ai/) | ⭐⭐⭐⭐ | Phone reading; want runnable examples | Same content as dair-ai GitHub in website form + runnable examples | +| | [NirDiamant/Prompt_Engineering](https://github.com/NirDiamant/Prompt_Engineering) | ⭐⭐⭐⭐ | Learn-by-running | 22 techniques, each in its own notebook, more hands-on than dair-ai, ★ 7k+ | +| **Official cookbook** | [Anthropic Cookbook — Prompt patterns](https://github.com/anthropics/claude-cookbooks) | ⭐⭐⭐⭐⭐ | Advanced Claude prompting (prompt caching / multimodal) | Introduced in Stage 1; for this stage focus on `misc/prompt_caching.ipynb` and `multimodal/` | +| | [GoogleCloudPlatform/generative-ai](https://github.com/GoogleCloudPlatform/generative-ai) | ⭐⭐⭐ | Google stack (PaLM / Gemini) users | Google Cloud's prompting cookbook; cross-vendor perspective | +| **Inspiration collection**
(steal patterns, don't copy)| [f/awesome-chatgpt-prompts](https://github.com/f/awesome-chatgpt-prompts) | ⭐⭐⭐ | When you're stuck for ideas | Hundreds of "Act as a [role]..." prompts, ★ 162k+, CC0. **Take the pattern, rewrite — don't copy verbatim** | +| **Production management**
(scale up)| [microsoft/prompt-engine](https://github.com/microsoft/prompt-engine) | ⭐⭐⭐ | Managing many prompts in production | TypeScript library, template + dialogue history management | +| | [microsoft/promptflow](https://github.com/microsoft/promptflow) | ⭐⭐⭐ | Team apps needing eval | Visual prompt design + eval tooling, ★ 11k+ | +| | [stanfordnlp/dspy](https://github.com/stanfordnlp/dspy) ⭐ **Stage 2 → 3 bridge** | ⭐⭐⭐⭐⭐ | After dair-ai, want to scale prompts | Treat prompts as code, compiler auto-optimizes, ★ 34k+, MIT | + +> **Note**: dspy is a framework, not a tutorial — higher entry barrier; pair it with the [dspy.ai](https://dspy.ai/) official tutorial. NirDiamant uses NOASSERTION custom terms (research / non-commercial leaning). + +> 💡 **Suggested reading order**: dair-ai guide for theory → Anthropic Cookbook for Claude implementation → NirDiamant for hands-on → dspy when going to production. + +## 🔭 Advanced: The Three Layers of Prompt → Context → Harness Engineering + +Engineering practice for LLM-powered systems can be divided into **three stack layers**. This is not about "one call vs. many calls." Each layer engineers a different object: + +- **Prompt Engineering** (this stage) = engineering **the string sent into the model** +- **Context Engineering** (Stage 6) = engineering **what information goes into the context window on each call** — dynamically assembling RAG retrieval results, memory, tool definitions, and conversation history +- **Harness Engineering** (Stage 7) = engineering **the execution and control layer around the model** — agent loops, retry, sandboxing, observability, deployment, and all other non-LLM code + +→ The three layers are **orthogonal**: a one-call RAG app is still doing context engineering (the point is assembling context, not how many calls happen); a 50-call chatbot with no retrieval is still only doing prompt engineering. + +**Full three-layer lineage in this roadmap**: + +| Discipline | What is being engineered | Where this roadmap teaches it fully | +|---|---|---| +| **1. Prompt Engineering** | The string sent into the LLM itself (system prompt / few-shot / format) | **This stage (Stage 2)** | +| **2. Context Engineering** | What information goes into the context window (RAG / memory / tool defs / history) | [Stage 6 — Context Engineering: RAG and Memory](06-memory-rag.en.md) | +| **3. Harness Engineering** | The execution and control layer around the model (agent loop / retry / sandbox / observability) | [Stage 7 — Multi-Agent · Productionization](07-multi-agent-production.en.md) | + +> 💡 **Karpathy 2025-06**: context engineering is the delicate art of putting information that is **just useful for the next step** into the context window. +> +> 💡 **Simon Willison / Addy Osmani**: "coding agent = LLM + harness"; a harness is all the code that is not the model itself. [OpenAI also used the term "Harness Engineering" in February 2026](https://openai.com/index/harness-engineering). + +**You do not need to finish the latter two layers in this stage**. This section only gives you the direction so that Stage 6 / 7 feel like a continuation of the same lineage. + +Further reading (optional, for when you want to dig deeper): + +- [`Meirtz/Awesome-Context-Engineering`](https://github.com/Meirtz/Awesome-Context-Engineering) (★ 3k+) — comprehensive survey from prompt engineering to production agents +- [`Windy3f3f3f3f/how-claude-code-works`](https://github.com/Windy3f3f3f3f/how-claude-code-works) (★ 3k+) — Claude Code internals, includes a context-engineering chapter + +## ✅ Self-Check Before Stage 3 + +Can you: + +- [ ] Write a prompt with system message + user message + 3 example messages (few-shot) +- [ ] Demonstrate CoT improving accuracy on a reasoning task +- [ ] Iteratively refine a prompt 5 times tracking each version +- [ ] Identify when prompting is the wrong tool (and tool use is needed) + +If yes → proceed to [Stage 3 — Tool Use & Agent Intro](03-tool-use-and-hello-agent.en.md). This is the most important stage — don't rush past prompts but also don't get stuck here. diff --git a/stages/02-prompt-engineering.md b/stages/02-prompt-engineering.md new file mode 100644 index 0000000..7813727 --- /dev/null +++ b/stages/02-prompt-engineering.md @@ -0,0 +1,491 @@ +# Stage 2 — Prompt 設計（Prompt Engineering） + +> **繁體中文** | [简体中文](./02-prompt-engineering.zh-Hans.md) | [English](./02-prompt-engineering.en.md) + +⏱ **時間估算**：1-2 週（約 5-12 小時） + +> 👋 **從 [Stage 1](01-llm-basics.md) 來的**：好，你會呼叫 API 了——這 5-12 小時：寫出可重用的結構化 prompt、用 few-shot 跟 chain-of-thought 解難題、用 eval 量化 prompt 改善幅度。**直接從這裡開始的**：先確認你會呼叫 LLM API、會用 token 算成本——做不到請先回 [Stage 1](01-llm-basics.md)。 + +> 💡 用語不熟（prompt / few-shot / CoT / system prompt⋯）→ 翻 [`resources/glossary.md`](../resources/glossary.md)。 + +> 📋 **本章組成**：學習目標 → 進入條件 → 必修閱讀 →〔可選 · 概念地圖〕→ 動手練習 → 精選 Projects → 自我檢查 +> 🔑 **關鍵名詞**：見 [`resources/glossary.md`](../resources/glossary.md)（每 stage 用到的術語都收在那裡） + +## 📌 學習目標 + +走完這個階段後你會： +- 寫出結構化 prompt（角色 + 任務 + 格式 + 範例） +- 應用 few-shot prompting，並知道什麼時候有用 +- 在推理任務上使用 chain-of-thought（CoT） +- 反覆迭代修改一個 prompt 並衡量改善 +- 看出什麼時候 prompt 已經到極限了（這時你需要 tool / agent） + +## 🚪 進入條件 + +你應該已經： +- 會呼叫 LLM API（Stage 1） +- 會解析 / 走訪 API 回應 + +## 📚 必修閱讀 + +1. [**anthropics/prompt-eng-interactive-tutorial**](https://github.com/anthropics/prompt-eng-interactive-tutorial) ⭐⭐⭐⭐⭐ ★ 35k+ — **Anthropic 官方互動教學**、9 章 Jupyter notebook（basic / intermediate / advanced + appendix），含 playground 跟 answer key。用 Claude 3 Haiku（最便宜）跑得起來、**Stage 2 的 canonical 動手教材**。也是 [**anthropics/courses**](https://github.com/anthropics/courses) 5 course umbrella 的 module 2，想看更廣（含 API Fundamentals / Real World Prompting / Eval / Tool Use）直接看 umbrella +2. [**anthropics/courses — Real World Prompting**](https://github.com/anthropics/courses) ⭐⭐⭐⭐ ★ 21k+ — 同 umbrella 的 module 3，**「真實情境下怎麼用 prompting」**：chatbot / legal / financial / coding 案例 walk-through。看完 #1 再來看 #2 +3. [**Anthropic Prompt Engineering Guide**](https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/overview) — 官方 docs、配合上面 #1 一起讀 +3. [**OpenAI Prompt Engineering**](https://platform.openai.com/docs/guides/prompt-engineering) — OpenAI 觀點 +4. [**dair-ai Prompt Engineering Guide**](https://www.promptingguide.ai/) — 學術風，深入 +5. [**Anthropic — Prompting Best Practices**](https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/be-clear-and-direct) — 直接清楚 + +**🎥 中文影片補充（強烈推薦）**： +- [**李宏毅 — 生成式 AI 導論（2024 春台大課程）**](https://speech.ee.ntu.edu.tw/~hylee/genai/2024-spring.php) ⭐⭐⭐ — 中後段集數講 prompt engineering（few-shot、CoT、in-context learning）+ 對應 lab。中文圈最完整的 prompting 學術級教學。最新整合版見 [**GenAI-ML 2025 秋**](https://speech.ee.ntu.edu.tw/~hylee/GenAI-ML/2025-fall.php) +- [**李宏毅 — 機器學習 2025 春（含 prompt + LLM 章節）**](https://speech.ee.ntu.edu.tw/~hylee/ml/2025-spring.php) — 適合想看 ML 完整背景的人 + +## 🛠 動手練習 + +> 🦙 **本 stage 預設用 Ollama gemma4:e4b**（成本考量、$0/run）。Prompt engineering 對小 model 更有教學價值——小 model 對 prompt 質量敏感、能讓你看清楚 system prompt / few-shot / CoT / refinement 各自帶來多少改善。每個練習都有 Path A（Ollama、預設）+ Path B（Anthropic、選擇性）。 +> +> 💰 **Stage 2 預算估算**（全 4 練習各跑 3-5 次）：**全本機 = $0**、**全 haiku ≈ $0.20**、**全 sonnet ≈ $0.60**。Few-shot 分類任務的 12 calls × 5 reps ≈ $0.30 haiku / $0.90 sonnet。完整預算見 [`examples/README.md#推薦-llm-清單`](../examples/README.md#推薦-llm-清單)。 +> +> 完整 3 路 trade-off 見 [`examples/README.md`](../examples/README.md#三條路徑--預設用-ollama成本考量)。 + +### 練習 1：System Prompt +同樣的 user message，三個不同的 system prompt。觀察人格 / 輸出格式怎麼變。 + +
+📋 起手碼 — Path A（本機 Ollama gemma4:e4b、預設）（複製到 practice_1.py） + +```python +# 需要：pip install openai +# 前置：ollama pull gemma4:e4b && ollama serve +import sys, json +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +# 同一個 user message、3 個不同 system prompt +SYSTEM_PROMPTS = { + "嚴肅律師": "你是嚴謹的合約律師。回答要精準、引用法條編號、避免任何主觀形容詞。", + "幼兒園老師": "你是溫柔的幼兒園老師、要對 5 歲小孩說話。用比喻、口語、少於 80 字。", + "JSON 機器": "你只回 JSON。schema: {\"answer\": string, \"confidence\": float}", +} + +USER_MSG = "請幫我解釋什麼是租賃合約。" + +outputs = {} +for label, system in SYSTEM_PROMPTS.items(): + # Note: Ollama 把 system 放 messages 第一筆（不像 Anthropic 用 system= 參數） + r = client.chat.completions.create( + model="gemma4:e4b", + max_tokens=200, + messages=[ + {"role": "system", "content": system}, + {"role": "user", "content": USER_MSG}, + ], + ) + outputs[label] = r.choices[0].message.content + print(f"\n--- [{label}] ---") + print(outputs[label]) + +# === 自我驗證 === +json_output = outputs["JSON 機器"] +assert "{" in json_output and "}" in json_output, "JSON 機器版輸出應該含 JSON braces" +try: + parsed = json.loads(json_output.strip().split("\n")[-1] if "\n" in json_output else json_output) + assert "answer" in parsed, "JSON schema 應包含 answer 欄位" +except json.JSONDecodeError: + pass # 容許 model 回 JSON 含解釋文字、最後一筆才是 JSON +print(f"\n✅ 練習 1 通過 — 同一個問題、3 種人格 / 格式 / 語氣") +print("💡 觀察：律師長、老師短、JSON 機器一定是 {...}") +``` + +**預期輸出**（樣本、gemma4:e4b 對 system prompt 遵循度 OK 但不如 Claude 嚴謹）： +``` +--- [嚴肅律師] --- +依民法第 421 條... + +--- [幼兒園老師] --- +租賃合約就像借玩具給朋友、講好什麼時候還、要付多少糖果... + +--- [JSON 機器] --- +{"answer": "租賃合約是當事人約定一方以物租與他方使用...", "confidence": 0.85} +``` + +
+ +
+📋 起手碼 — Path B（Anthropic API、選擇性）（複製到 practice_1_anthropic.py） + +```python +# 需要：pip install anthropic +import sys, json +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic +client = anthropic.Anthropic() +SYSTEM_PROMPTS = { + "嚴肅律師": "你是嚴謹的合約律師。回答要精準、引用法條編號、避免任何主觀形容詞。", + "幼兒園老師": "你是溫柔的幼兒園老師、要對 5 歲小孩說話。用比喻、口語、少於 80 字。", + "JSON 機器": "你只回 JSON。schema: {\"answer\": string, \"confidence\": float}", +} +USER_MSG = "請幫我解釋什麼是租賃合約。" + +outputs = {} +for label, system in SYSTEM_PROMPTS.items(): + # Anthropic 用 system= 參數（不放 messages 內） + msg = client.messages.create(model="claude-haiku-4-5", max_tokens=200, + system=system, messages=[{"role": "user", "content": USER_MSG}]) + outputs[label] = msg.content[0].text + print(f"\n--- [{label}] ---") + print(outputs[label]) + +# 同樣的 JSON assert（schema 跨 backend 通用） +json_output = outputs["JSON 機器"] +assert "{" in json_output and "}" in json_output +print(f"\n✅ 練習 1 通過（Anthropic）") +``` + +**主要差異**： +- Anthropic: `system=...` 參數 +- Ollama / OpenAI-compatible: `messages=[{"role": "system", ...}, ...]` + +**Anthropic 對 system prompt 遵循度通常比 4B 小 model 更嚴謹**——「嚴肅律師」會真的引用法條編號。 + +
+ +### 練習 2：Few-Shot + +**先搞懂這三個詞**——差別只在你給 LLM 看「幾個範例」： +- **Zero-shot（0-shot）**：不給範例、直接問。 +- **One-shot（1-shot）**：先給 **1 個**「輸入 → 答案」範例再問。 +- **Few-shot（下面用的 3-shot 就是）**：給幾個（通常 2-5 個）範例再問——LLM 照著範例的格式跟判斷標準做，準確率通常明顯變高。 + +挑一個分類任務。先用 0-shot 跑，再用 3-shot 跑。量一下準確率差多少。 + +
+📋 起手碼 — Path A（本機 Ollama gemma4:e4b、預設）（複製到 practice_2.py） + +```python +# 需要：pip install openai +# 前置：ollama pull gemma4:e4b && ollama serve +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +# 中文情緒分類（正面 / 負面 / 中立） +TEST_SET = [ + ("這部電影超讚、看完想再看一次！", "正面"), + ("劇情無聊、演員演技尷尬。", "負面"), + ("這是一部 2019 年的電影。", "中立"), + ("我不確定喜不喜歡、可能再想想。", "中立"), + ("第一集很不錯但第二集就崩了。", "負面"), + ("看完心情很好、推薦！", "正面"), +] + +FEW_SHOT_EXAMPLES = """範例： +input: 這家餐廳的牛排好吃到讓我哭出來。 +output: 正面 + +input: 服務生態度很差、我再也不會來了。 +output: 負面 + +input: 這家店位於新北市三重區。 +output: 中立 +""" + +# 兩種條件共用同一段「任務說明」；few-shot 只多加範例——這樣對比才乾淨，量到的是「範例」本身的效果，而不是「終於告訴模型要做什麼」。 +TASK = "把下面的句子分類成「正面 / 負面 / 中立」其中一個，只輸出這三個詞其中之一、不要多餘文字。\n\n" + + +def classify(text: str, *, use_few_shot: bool) -> str: + prefix = FEW_SHOT_EXAMPLES + "\n" if use_few_shot else "" + prompt = f"{TASK}{prefix}input: {text}\noutput:" + r = client.chat.completions.create( + model="gemma4:e4b", + max_tokens=10, + messages=[{"role": "user", "content": prompt}], + ) + return r.choices[0].message.content.strip().splitlines()[0] + + +def evaluate(use_few_shot: bool) -> tuple[int, int]: + correct = 0 + for text, label in TEST_SET: + pred = classify(text, use_few_shot=use_few_shot) + ok = label in pred + print(f" {'✓' if ok else '✗'} [{label}] {text[:30]}... → '{pred}'") + if ok: + correct += 1 + return correct, len(TEST_SET) + + +print("=== 0-shot ===") +c0, n = evaluate(use_few_shot=False) +print(f"正確 {c0}/{n} = {c0/n:.0%}") + +print("\n=== 3-shot ===") +c3, _ = evaluate(use_few_shot=True) +print(f"正確 {c3}/{n} = {c3/n:.0%}") + +# === 自我驗證 === +# 兩種條件都給了同樣的任務說明，所以這裡量的是「範例本身」帶來的差異。 +# few-shot 不保證每次都贏（看 model / 題目 / 抽樣），所以不硬性要求 c3 >= c0。 +assert n == 6 and 0 <= c0 <= n and 0 <= c3 <= n, "兩種條件都要各跑完 6 題" +print(f"\n✅ 練習 2 通過 — 0-shot {c0}/{n}、3-shot {c3}/{n}；few-shot 淨提升 {c3 - c0} 題（可能為 0 甚至負，都算正常）（本機 $0）") +print("💡 觀察：有了任務說明，0-shot 就有基本盤；few-shot 的價值在「釘住輸出格式」+ 示範模稜兩可案例（如 '中立'）的判準") +print("💡 小 model（gemma4:e4b）對格式更敏感，所以 few-shot 的幫助通常比 Claude 明顯——但仍非保證，要跑了才知道") +``` + +
+ +
+📋 起手碼 — Path B（Anthropic API、選擇性）（複製到 practice_2_anthropic.py） + +```python +# 需要：pip install anthropic +# 把 starter Path A 的 client 跟 classify() 改成： +import anthropic +client = anthropic.Anthropic() + +def classify(text: str, *, use_few_shot: bool) -> str: + prefix = FEW_SHOT_EXAMPLES + "\n" if use_few_shot else "" + msg = client.messages.create( + model="claude-haiku-4-5", + max_tokens=10, + messages=[{"role": "user", "content": f"{TASK}{prefix}input: {text}\noutput:"}], + ) + return msg.content[0].text.strip().splitlines()[0] +# 其餘 TASK / TEST_SET / FEW_SHOT_EXAMPLES / evaluate() 跟 Path A 一樣 +``` + +**成本**：6 題 × 2 條件 = 12 次 ≈ $0.005。**Claude 通常 0-shot 已經有不錯準確率**、所以 few-shot 改善幅度比小 model 小。 + +
+ +### 練習 3：CoT +挑一個數學文字題，比較： +- 純 prompt +- 純 prompt + 「Let's think step by step」 +- 純 prompt + 一個展示 CoT 的範例 + +
+📋 起手碼 — Path A（本機 Ollama gemma4:e4b、預設）（複製到 practice_3.py） + +```python +# 需要：pip install openai +# 前置：ollama pull gemma4:e4b && ollama serve +import sys, re +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +QUESTION = "小明有 3 顆蘋果。他給了小華 1 顆、又從媽媽那邊拿到 5 顆、然後吃了 2 顆。請問現在剩幾顆？" +ANSWER = 5 # 3 - 1 + 5 - 2 = 5 + +COT_EXAMPLE = """範例： +Q: 一隻雞有 2 隻腳。3 隻雞跟 1 個人共有幾隻腳？ +A: 讓我一步一步算。3 隻雞 × 2 隻腳 = 6 隻腳。1 個人有 2 隻腳。總共 6 + 2 = 8 隻腳。答案是 8。 +""" + + +def ask(prompt: str) -> str: + r = client.chat.completions.create( + model="gemma4:e4b", + max_tokens=300, + messages=[{"role": "user", "content": prompt}], + ) + return r.choices[0].message.content + + +def extract_number(text: str) -> int | None: + nums = re.findall(r"-?\d+", text) + return int(nums[-1]) if nums else None + + +# A. 純 prompt +out_a = ask(QUESTION); ans_a = extract_number(out_a) + +# B. + Let's think step by step +out_b = ask(QUESTION + "\nLet's think step by step."); ans_b = extract_number(out_b) + +# C. + CoT example +out_c = ask(COT_EXAMPLE + "\n\nQ: " + QUESTION + "\nA:"); ans_c = extract_number(out_c) + +for label, out, ans in [("A 純 prompt", out_a, ans_a), ("B +step-by-step", out_b, ans_b), ("C +CoT example", out_c, ans_c)]: + print(f"\n--- [{label}] 答案={ans} {'✓' if ans == ANSWER else '✗'} ---") + print(out[:200]) + +# === 自我驗證 === +correct = sum(1 for a in (ans_a, ans_b, ans_c) if a == ANSWER) +assert correct >= 1, f"3 種 prompt 至少要 1 種答對、實際 {correct}/3" +# 小 model 對 CoT 依賴性更高、放寬條件：B 或 C 至少 1 對（vs Anthropic Path B 要求嚴格） +assert ans_b == ANSWER or ans_c == ANSWER, "B (step-by-step) 或 C (CoT example) 至少一種要答對 — CoT 對小 model 是基本功" +print(f"\n✅ 練習 3 通過 — {correct}/3 答對（本機 $0）") +print(f"💡 觀察小 model：A 純 prompt 通常答錯、B/C 加 CoT 後明顯改善——比 Claude 更能凸顯 CoT 重要性") +``` + +
+ +
+📋 起手碼 — Path B（Anthropic API、選擇性）（複製到 practice_3_anthropic.py） + +把 Path A 的 client + ask() 改成： + +```python +import anthropic +client = anthropic.Anthropic() + +def ask(prompt: str) -> str: + msg = client.messages.create(model="claude-haiku-4-5", max_tokens=300, + messages=[{"role": "user", "content": prompt}]) + return msg.content[0].text +``` + +**Claude 通常 3/3 全對**（包括 A 純 prompt）—— 對照 gemma4:e4b 可能只 1-2/3 對，能看到 CoT 對小 model 的價值。 + +
+ +> 🧠 **什麼時候別自己寫 CoT**：對 **reasoning-native 模型**（Claude Opus 4.x、o 系列、Gemini thinking 等內建思考的模型），用它們的 extended thinking 通常比你手寫「Let's think step by step」更好；硬塞步驟反而可能干擾它本來的推理。手寫 CoT 仍適用於不具內建推理的一般 chat model。 + +### 練習 4：Iterative Refinement +拿一個模糊的 prompt，refine 5 次。把每一輪記下來。觀察哪些改動會提升品質。 + +
+📋 起手碼 — Path A（本機 Ollama gemma4:e4b、預設）（複製到 practice_4.py）— 這題沒有「對錯」、重點是觀察過程 + +```python +# 需要：pip install openai +# 前置：ollama pull gemma4:e4b && ollama serve +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +# 5 個 iteration、每一輪 prompt 都比前一輪更具體 +PROMPTS = { + "v1 模糊": "寫一段介紹 ReAct 的文字。", + "v2 加目標讀者": "寫一段介紹 ReAct 的文字、給寫過 Python 的軟體工程師看。", + "v3 加格式": "寫一段介紹 ReAct 的文字、給寫過 Python 的軟體工程師看。100 字以內、用一個段落。", + "v4 加 example 要求": "寫一段介紹 ReAct 的文字、給寫過 Python 的軟體工程師看。100 字以內、用一個段落、結尾舉一個具體例子（譬如查天氣）。", + "v5 加禁忌": "寫一段介紹 ReAct 的文字、給寫過 Python 的軟體工程師看。100 字以內、用一個段落、結尾舉一個具體例子（譬如查天氣）。不要用「賦能」「驅動」「智能」這類空泛詞彙。", +} + +outputs = {} +for label, prompt in PROMPTS.items(): + r = client.chat.completions.create( + model="gemma4:e4b", + max_tokens=200, + messages=[{"role": "user", "content": prompt}], + ) + text = r.choices[0].message.content + outputs[label] = text + print(f"\n--- [{label}] ({len(text)} chars) ---") + print(text) + +# === 自我驗證 === +v1_len, v5_len = len(outputs["v1 模糊"]), len(outputs["v5 加禁忌"]) +banned_words = ("賦能", "驅動", "智能") +v5_has_banned = any(w in outputs["v5 加禁忌"] for w in banned_words) +assert v5_len > 0, "v5 必須有輸出" +assert not v5_has_banned, f"v5 應該避免禁忌詞、實際含: {[w for w in banned_words if w in outputs['v5 加禁忌']]}" +print(f"\n✅ 練習 4 通過 — v5 長度 {v5_len}、無禁忌詞（本機 $0）") +print(f"💡 觀察：v1 ({v1_len} chars) 通常比 v5 ({v5_len} chars) 「鬆」、加約束會逼 prompt 收斂") +print("💡 用 gemma4:e4b 跑這題特別有感——小 model 對 prompt 質量極敏感、5 輪 refine 的差距會比 Claude 更明顯") +``` + +
+ +
+📋 起手碼 — Path B（Anthropic API、選擇性）（複製到 practice_4_anthropic.py） + +把 Path A 的 client + 迴圈內 `client.chat.completions.create(...)` 改成： + +```python +import anthropic +client = anthropic.Anthropic() + +# 迴圈內： +msg = client.messages.create(model="claude-haiku-4-5", max_tokens=200, + messages=[{"role": "user", "content": prompt}]) +text = msg.content[0].text +``` + +其餘 PROMPTS / outputs / assert 邏輯完全相同。**成本**：5 次 ≈ $0.002。 + +**Claude vs gemma4 對 prompt 細緻度的差別**：Claude haiku 通常 v1 已能寫出 OK 段落、v5 加上約束後優化幅度較小；小 model v1 常空泛無用、v5 加禁忌後才開始能讀。 + +
+ +**進階做法**：把這 5 輪輸出全存進 csv、Stage 7 練習 2 會教怎麼把這變成 eval harness（評估腳手架、即「跑評估用的外圍程式 / 控制層」、完整定義見下面 進階：prompt → context → harness 三層 engineering）量化「prompt 改善了多少」。 + +## 🎯 精選 Projects + +按用途分 4 類、9 個項目一張表搞定。**挑入口看「適合誰」、想深入點連結看 repo / 網站**。 + +| 分類 | Project | ⭐ | 適合誰 | 為什麼推薦 / 備註 | +|---|---|---|---|---| +| **學術 / 教學風 guide**
（先看這個） | [dair-ai/Prompt-Engineering-Guide](https://github.com/dair-ai/Prompt-Engineering-Guide) | ⭐⭐⭐⭐⭐ | 當參考書、需要某技巧再來查 | 從基礎到進階（CoT / ToT / ReAct / RAG）端到端，★ 74k+、MIT | +| | [PromptingGuide.ai](https://www.promptingguide.ai/) | ⭐⭐⭐⭐ | 手機閱讀、想要可跑範例 | 跟 dair-ai GitHub 同樣內容、做成網站 + 可跑範例 | +| | [NirDiamant/Prompt_Engineering](https://github.com/NirDiamant/Prompt_Engineering) | ⭐⭐⭐⭐ | 偏好「邊跑邊學」 | 22 種技巧、獨立 notebook，比 dair-ai 更動手，★ 7k+| +| **官方 cookbook** | [Anthropic Cookbook — Prompt patterns](https://github.com/anthropics/claude-cookbooks) | ⭐⭐⭐⭐⭐ | Claude 進階 prompting（含 prompt caching / multimodal）| Stage 1 已介紹、本 stage 重點看 `misc/prompt_caching.ipynb` 跟 `multimodal/` | +| | [GoogleCloudPlatform/generative-ai](https://github.com/GoogleCloudPlatform/generative-ai) | ⭐⭐⭐ | 用 Google 技術棧（PaLM / Gemini）| Google Cloud 的 prompting cookbook、跨廠商觀點 | +| **靈感 collection**
（找模式、不要照抄）| [f/awesome-chatgpt-prompts](https://github.com/f/awesome-chatgpt-prompts) | ⭐⭐⭐ | 卡關時找靈感 | 上百個「Act as a [角色]...」prompt，★ 162k+、CC0。**把模式拿出來改寫、不要照抄** | +| **Production 管理**
（規模化）| [microsoft/prompt-engine](https://github.com/microsoft/prompt-engine) | ⭐⭐⭐ | production 要管很多 prompt 時 | TypeScript library、管理樣板 + 對話歷史 | +| | [microsoft/promptflow](https://github.com/microsoft/promptflow) | ⭐⭐⭐ | 團隊型應用、需要 eval | 視覺化 prompt 設計 + 評估工具，★ 11k+ | +| | [stanfordnlp/dspy](https://github.com/stanfordnlp/dspy) ⭐ **Stage 2 → 3 橋** | ⭐⭐⭐⭐⭐ | 跑完 dair-ai 想規模化 prompt | 把 prompt 當 code 寫，用 compiler 自動最佳化，★ 34k+、MIT | + +> **註**：dspy 是 framework 不是 tutorial、門檻較高，建議搭配 [dspy.ai](https://dspy.ai/) 官方 tutorial 讀；NirDiamant 用 NOASSERTION 自訂條款（偏研究 / 非商用）。 + +> 💡 **建議閱讀路徑**：dair-ai guide 入手（理論） → Anthropic Cookbook 看 Claude 實作 → NirDiamant 邊跑邊學 → 進 production 時讀 dspy。 + +## 🔭 進階：prompt → context → harness 三層 engineering + +LLM-powered system 的工程實踐分成 **3 層 stack**（不是 1 次 call vs N 次 call）。每一層工程的對象**不一樣**： + +- **Prompt Engineering**（本 stage）= 工程「**送進模型的那段字串**」 +- **Context Engineering**（Stage 6）= 工程「**每次 call 時、 context window 裡裝什麼資訊**」——把 RAG retrieve 結果、memory、tool definitions、對話 history 動態組裝 +- **Harness Engineering**（Stage 7）= 工程 **模型外面的執行與控制層**——agent loop、retry、sandbox、observability、deployment 等所有非 LLM 程式碼 + +→ 三層**正交**：一次 call 的 RAG app 也在做 context engineering（重點是組 context、不是 call 幾次）；50 次 call 但沒做 retrieval 的 chatbot 仍只在做 prompt engineering。 + +**完整三層 lineage（本路線的學習進度）**： + +| Discipline | 工程「什麼」 | 在哪一 stage 完整學 | +|---|---|---| +| **1. Prompt Engineering** | 送進 LLM 的字串本身（system prompt / few-shot / format） | **本 stage（Stage 2）** | +| **2. Context Engineering** | context window 裡裝什麼資訊（RAG / memory / tool defs / history） | [Stage 6 — Memory · RAG · Context Engineering](06-memory-rag.md) | +| **3. Harness Engineering** | LLM 外面的執行與控制層（agent loop / retry / sandbox / observability） | [**Stage 7 Harness Engineering**](07-multi-agent-production.md#-harness-engineering--production-agent-runtime-的工程設計--本-stage-核心概念) ⭐ 完整對照表 | + +> 💡 **Karpathy 2025-06**：「context engineering 是把對下一步有用的資訊**剛好填進** context window 的精細藝術」（it's about *what goes in the window*）。 +> +> 💡 **Simon Willison / Addy Osmani**：「coding agent = LLM + harness」——harness 就是「模型外圍的控制系統」、retry / loop / 監測 / 沙盒 / 部署這些不是 LLM 本身的程式碼。[OpenAI 2026-02 也使用 "Harness Engineering" 這個說法](https://openai.com/index/harness-engineering)。 + +**這個 stage 不用學完後兩層**，只是給方向性提示——進入 Stage 6 / 7 時會接續這個 lineage。 + +延伸閱讀（不必修、未來想深挖時看）： + +- [`Meirtz/Awesome-Context-Engineering`](https://github.com/Meirtz/Awesome-Context-Engineering)（★ 3k+）——從 prompt engineering 一路推到 production agent 的 survey +- [`Windy3f3f3f3f/how-claude-code-works`](https://github.com/Windy3f3f3f3f/how-claude-code-works)（★ 3k+）——Claude Code 內部解析，含 context engineering 章節 + +## ✅ 進 Stage 3 前的自我檢查 + +你能不能： + +- [ ] 寫一個有 system message + user message + 3 個範例 message 的 prompt（few-shot） +- [ ] 示範 CoT 在某個推理任務上提升準確率 +- [ ] 反覆 refine 一個 prompt 5 次，每一版都留下記錄 +- [ ] 看出 prompt 不是對的工具的時候（這時要用 tool use） + +如果可以 → 進 [Stage 3 — Tool Use & Agent 入門](03-tool-use-and-hello-agent.md)。這是最重要的一個階段——prompt 不要急著跳過去，但也不要卡在這裡。 diff --git a/stages/02-prompt-engineering.zh-Hans.md b/stages/02-prompt-engineering.zh-Hans.md new file mode 100644 index 0000000..416e245 --- /dev/null +++ b/stages/02-prompt-engineering.zh-Hans.md @@ -0,0 +1,484 @@ +# Stage 2 — Prompt 设计（Prompt Engineering） + +> [繁體中文](./02-prompt-engineering.md) | **简体中文** | [English](./02-prompt-engineering.en.md) + +⏱ **时间估算**：1-2 周（约 5-12 小时） + +> 👋 **从 [Stage 1](01-llm-basics.zh-Hans.md) 来的**：好，你会调用 API 了——这 5-12 小时：写出可重用的结构化 prompt、用 few-shot 跟 chain-of-thought 解难题、用 eval 量化 prompt 改善幅度。**直接从这里开始的**：先确认你会调用 LLM API、会用 token 算成本——做不到请先回 [Stage 1](01-llm-basics.zh-Hans.md)。 + +> 💡 用语不熟（prompt / few-shot / CoT / system prompt⋯）→ 翻 [`resources/glossary.zh-Hans.md`](../resources/glossary.zh-Hans.md)。 + +## 📌 学习目标 + +走完这个阶段后你会： +- 写出结构化 prompt（角色 + 任务 + 格式 + 示例） +- 应用 few-shot prompting，并知道什么时候有用 +- 在推理任务上使用 chain-of-thought（CoT） +- 反复迭代修改一个 prompt 并衡量改善 +- 看出什么时候 prompt 已经到极限了（这时你需要 tool / agent） + +## 🚪 进入条件 + +你应该已经： +- 会调用 LLM API（Stage 1） +- 会解析 / 遍历 API 响应 + +## 📚 必修阅读 + +1. [**anthropics/prompt-eng-interactive-tutorial**](https://github.com/anthropics/prompt-eng-interactive-tutorial) ⭐⭐⭐⭐⭐ ★ 35k+ — **Anthropic 官方互动教程**、9 章 Jupyter notebook（basic / intermediate / advanced + appendix），含 playground 跟 answer key。用 Claude 3 Haiku（最便宜）跑得起来、**Stage 2 的 canonical 动手教材**。也是 [**anthropics/courses**](https://github.com/anthropics/courses) 5 course umbrella 的 module 2，想看更广（含 API Fundamentals / Real World Prompting / Eval / Tool Use）直接看 umbrella +2. [**anthropics/courses — Real World Prompting**](https://github.com/anthropics/courses) ⭐⭐⭐⭐ ★ 21k+ — 同 umbrella 的 module 3，**“真实情境下怎么用 prompting”**：chatbot / legal / financial / coding 案例 walk-through。看完 #1 再来看 #2 +3. [**Anthropic Prompt Engineering Guide**](https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/overview) — 官方 docs、配合上面 #1 一起读 +3. [**OpenAI Prompt Engineering**](https://platform.openai.com/docs/guides/prompt-engineering) — OpenAI 观点 +4. [**dair-ai Prompt Engineering Guide**](https://www.promptingguide.ai/) — 学术风，深入 +5. [**Anthropic — Prompting Best Practices**](https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/be-clear-and-direct) — 直接清楚 + +## 🛠 动手练习 + +> 🦙 **本 stage 默认用 Ollama gemma4:e4b**（成本考量、$0/run）。Prompt engineering 对小 model 更有教学价值——小 model 对 prompt 质量敏感、能让你看清楚 system prompt / few-shot / CoT / refinement 各自带来多少改善。每个练习都有 Path A（Ollama、默认）+ Path B（Anthropic、选择性）。 +> +> 💰 **Stage 2 预算估算**（全 4 练习各跑 3-5 次）：**全本机 = $0**、**全 haiku ≈ $0.20**、**全 sonnet ≈ $0.60**。Few-shot 分类任务的 12 calls × 5 reps ≈ $0.30 haiku / $0.90 sonnet。完整预算见 [`examples/README.zh-Hans.md#推荐-llm-清单`](../examples/README.zh-Hans.md#推荐-llm-清单)。 +> +> 完整 3 路 trade-off 见 [`examples/README.zh-Hans.md`](../examples/README.zh-Hans.md#三条路径--默认用-ollama成本考量)。 + +### 练习 1：System Prompt +同样的 user message，三个不同的 system prompt。观察人格 / 输出格式怎么变。 + +
+📋 起手码 — Path A（本机 Ollama gemma4:e4b、默认）（复制到 practice_1.py） + +```python +# 需要：pip install openai +# 前置：ollama pull gemma4:e4b && ollama serve +import sys, json +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +# 同一个 user message、3 个不同 system prompt +SYSTEM_PROMPTS = { + "严肃律师": "你是严谨的合约律师。回答要精准、引用法条编号、避免任何主观形容词。", + "幼儿园老师": "你是温柔的幼儿园老师、要对 5 岁小孩说话。用比喻、口语、少于 80 字。", + "JSON 机器": "你只回 JSON。schema: {\"answer\": string, \"confidence\": float}", +} + +USER_MSG = "请帮我解释什么是租赁合约。" + +outputs = {} +for label, system in SYSTEM_PROMPTS.items(): + # Note: Ollama 把 system 放 messages 第一笔（不像 Anthropic 用 system= 参数） + r = client.chat.completions.create( + model="gemma4:e4b", + max_tokens=200, + messages=[ + {"role": "system", "content": system}, + {"role": "user", "content": USER_MSG}, + ], + ) + outputs[label] = r.choices[0].message.content + print(f"\n--- [{label}] ---") + print(outputs[label]) + +# === 自我验证 === +json_output = outputs["JSON 机器"] +assert "{" in json_output and "}" in json_output, "JSON 机器版输出应该含 JSON braces" +try: + parsed = json.loads(json_output.strip().split("\n")[-1] if "\n" in json_output else json_output) + assert "answer" in parsed, "JSON schema 应包含 answer 栏位" +except json.JSONDecodeError: + pass # 容许 model 回 JSON 含解释文字、最后一笔才是 JSON +print(f"\n✅ 练习 1 通过 — 同一个问题、3 種人格 / 格式 / 语气") +print("💡 观察：律师长、老师短、JSON 机器一定是 {...}") +``` + +**预期输出**（样本、gemma4:e4b 对 system prompt 遵循度 OK 但不如 Claude 严谨）： +``` +--- [严肃律师] --- +依民法第 421 条... + +--- [幼儿园老师] --- +租赁合约就像借玩具给朋友、讲好什么时候还、要付多少糖果... + +--- [JSON 机器] --- +{"answer": "租赁合约是当事人约定一方以物租与他方使用...", "confidence": 0.85} +``` + +
+ +
+📋 起手码 — Path B（Anthropic API、选择性）（复制到 practice_1_anthropic.py） + +```python +# 需要：pip install anthropic +import sys, json +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +import anthropic +client = anthropic.Anthropic() +SYSTEM_PROMPTS = { + "严肃律师": "你是严谨的合约律师。回答要精准、引用法条编号、避免任何主观形容词。", + "幼儿园老师": "你是温柔的幼儿园老师、要对 5 岁小孩说话。用比喻、口语、少于 80 字。", + "JSON 机器": "你只回 JSON。schema: {\"answer\": string, \"confidence\": float}", +} +USER_MSG = "请帮我解释什么是租赁合约。" + +outputs = {} +for label, system in SYSTEM_PROMPTS.items(): + # Anthropic 用 system= 参数（不放 messages 內） + msg = client.messages.create(model="claude-haiku-4-5", max_tokens=200, + system=system, messages=[{"role": "user", "content": USER_MSG}]) + outputs[label] = msg.content[0].text + print(f"\n--- [{label}] ---") + print(outputs[label]) + +# 同样的 JSON assert（schema 跨 backend 通用） +json_output = outputs["JSON 机器"] +assert "{" in json_output and "}" in json_output +print(f"\n✅ 练习 1 通过（Anthropic）") +``` + +**主要差异**： +- Anthropic: `system=...` 参数 +- Ollama / OpenAI-compatible: `messages=[{"role": "system", ...}, ...]` + +**Anthropic 对 system prompt 遵循度通常比 4B 小 model 更严谨**——“严肃律师”会真的引用法条编号。 + +
+ +### 练习 2：Few-Shot + +**先搞懂这三个词**——差别只在你给 LLM 看“几个范例”： +- **Zero-shot（0-shot）**：不给范例、直接问。 +- **One-shot（1-shot）**：先给 **1 个**“输入 → 答案”范例再问。 +- **Few-shot（下面用的 3-shot 就是）**：给几个（通常 2-5 个）范例再问——LLM 照着范例的格式跟判断标准做，准确率通常明显变高。 + +挑一个分类任务。先用 0-shot 跑，再用 3-shot 跑。量一下准确率差多少。 + +
+📋 起手码 — Path A（本机 Ollama gemma4:e4b、默认）（复制到 practice_2.py） + +```python +# 需要：pip install openai +# 前置：ollama pull gemma4:e4b && ollama serve +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +# 中文情绪分类（正面 / 负面 / 中立） +TEST_SET = [ + ("这部电影超赞、看完想再看一次！", "正面"), + ("剧情无聊、演员演技尴尬。", "负面"), + ("这是一部 2019 年的电影。", "中立"), + ("我不确定喜不喜欢、可能再想想。", "中立"), + ("第一集很不错但第二集就崩了。", "负面"), + ("看完心情很好、推荐！", "正面"), +] + +FEW_SHOT_EXAMPLES = """范例： +input: 这家餐厅的牛排好吃到让我哭出来。 +output: 正面 + +input: 服务生态度很差、我再也不会来了。 +output: 负面 + +input: 这家店位于新北市三重区。 +output: 中立 +""" + +# 两种条件共用同一段“任务说明”；few-shot 只多加范例——这样对比才干净，量到的是“范例”本身的效果，而不是“终于告诉模型要做什么”。 +TASK = "把下面的句子分类成“正面 / 负面 / 中立”其中一个，只输出这三个词其中之一、不要多余文字。\n\n" + + +def classify(text: str, *, use_few_shot: bool) -> str: + prefix = FEW_SHOT_EXAMPLES + "\n" if use_few_shot else "" + prompt = f"{TASK}{prefix}input: {text}\noutput:" + r = client.chat.completions.create( + model="gemma4:e4b", + max_tokens=10, + messages=[{"role": "user", "content": prompt}], + ) + return r.choices[0].message.content.strip().splitlines()[0] + + +def evaluate(use_few_shot: bool) -> tuple[int, int]: + correct = 0 + for text, label in TEST_SET: + pred = classify(text, use_few_shot=use_few_shot) + ok = label in pred + print(f" {'✓' if ok else '✗'} [{label}] {text[:30]}... → '{pred}'") + if ok: + correct += 1 + return correct, len(TEST_SET) + + +print("=== 0-shot ===") +c0, n = evaluate(use_few_shot=False) +print(f"正确 {c0}/{n} = {c0/n:.0%}") + +print("\n=== 3-shot ===") +c3, _ = evaluate(use_few_shot=True) +print(f"正确 {c3}/{n} = {c3/n:.0%}") + +# === 自我验证 === +# 两种条件都给了同样的任务说明，所以这里量的是“范例本身”带来的差异。 +# few-shot 不保证每次都赢（看 model / 题目 / 抽样），所以不硬性要求 c3 >= c0。 +assert n == 6 and 0 <= c0 <= n and 0 <= c3 <= n, "两种条件都要各跑完 6 题" +print(f"\n✅ 练习 2 通过 — 0-shot {c0}/{n}、3-shot {c3}/{n}；few-shot 净提升 {c3 - c0} 题（可能为 0 甚至负，都算正常）（本机 $0）") +print("💡 观察：有了任务说明，0-shot 就有基本盘；few-shot 的价值在“钉住输出格式” + 示范模棱两可案例（如 '中立'）的判准") +print("💡 小 model（gemma4:e4b）对格式更敏感，所以 few-shot 的帮助通常比 Claude 明显——但仍非保证，要跑了才知道") +``` + +
+ +
+📋 起手码 — Path B（Anthropic API、选择性）（复制到 practice_2_anthropic.py） + +```python +# 需要：pip install anthropic +# 把 starter Path A 的 client 跟 classify() 改成： +import anthropic +client = anthropic.Anthropic() + +def classify(text: str, *, use_few_shot: bool) -> str: + prefix = FEW_SHOT_EXAMPLES + "\n" if use_few_shot else "" + msg = client.messages.create( + model="claude-haiku-4-5", + max_tokens=10, + messages=[{"role": "user", "content": f"{TASK}{prefix}input: {text}\noutput:"}], + ) + return msg.content[0].text.strip().splitlines()[0] +# 其余 TASK / TEST_SET / FEW_SHOT_EXAMPLES / evaluate() 跟 Path A 一样 +``` + +**成本**：6 题 × 2 条件 = 12 次 ≈ $0.005。**Claude 通常 0-shot 已经有不错准确率**、所以 few-shot 改善幅度比小 model 小。 + +
+ +### 练习 3：CoT +挑一个数学文字题，比较： +- 纯 prompt +- 纯 prompt + “Let's think step by step” +- 纯 prompt + 一个展示 CoT 的范例 + +
+📋 起手码 — Path A（本机 Ollama gemma4:e4b、默认）（复制到 practice_3.py） + +```python +# 需要：pip install openai +# 前置：ollama pull gemma4:e4b && ollama serve +import sys, re +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +QUESTION = "小明有 3 颗苹果。他给了小華 1 颗、又从媽媽那边拿到 5 颗、然后吃了 2 颗。请问现在剩几颗？" +ANSWER = 5 # 3 - 1 + 5 - 2 = 5 + +COT_EXAMPLE = """范例： +Q: 一只鸡有 2 只脚。3 只鸡跟 1 个人共有几只脚？ +A: 让我一步一步算。3 只鸡 × 2 只脚 = 6 只脚。1 个人有 2 只脚。總共 6 + 2 = 8 只脚。答案是 8。 +""" + + +def ask(prompt: str) -> str: + r = client.chat.completions.create( + model="gemma4:e4b", + max_tokens=300, + messages=[{"role": "user", "content": prompt}], + ) + return r.choices[0].message.content + + +def extract_number(text: str) -> int | None: + nums = re.findall(r"-?\d+", text) + return int(nums[-1]) if nums else None + + +# A. 纯 prompt +out_a = ask(QUESTION); ans_a = extract_number(out_a) + +# B. + Let's think step by step +out_b = ask(QUESTION + "\nLet's think step by step."); ans_b = extract_number(out_b) + +# C. + CoT example +out_c = ask(COT_EXAMPLE + "\n\nQ: " + QUESTION + "\nA:"); ans_c = extract_number(out_c) + +for label, out, ans in [("A 纯 prompt", out_a, ans_a), ("B +step-by-step", out_b, ans_b), ("C +CoT example", out_c, ans_c)]: + print(f"\n--- [{label}] 答案={ans} {'✓' if ans == ANSWER else '✗'} ---") + print(out[:200]) + +# === 自我验证 === +correct = sum(1 for a in (ans_a, ans_b, ans_c) if a == ANSWER) +assert correct >= 1, f"3 種 prompt 至少要 1 種答对、实际 {correct}/3" +# 小 model 对 CoT 依賴性更高、放宽条件：B 或 C 至少 1 对（vs Anthropic Path B 要求严格） +assert ans_b == ANSWER or ans_c == ANSWER, "B (step-by-step) 或 C (CoT example) 至少一種要答对 — CoT 对小 model 是基本功" +print(f"\n✅ 练习 3 通过 — {correct}/3 答对（本机 $0）") +print(f"💡 观察小 model：A 纯 prompt 通常答错、B/C 加 CoT 后明显改善——比 Claude 更能凸显 CoT 重要性") +``` + +
+ +
+📋 起手码 — Path B（Anthropic API、选择性）（复制到 practice_3_anthropic.py） + +把 Path A 的 client + ask() 改成： + +```python +import anthropic +client = anthropic.Anthropic() + +def ask(prompt: str) -> str: + msg = client.messages.create(model="claude-haiku-4-5", max_tokens=300, + messages=[{"role": "user", "content": prompt}]) + return msg.content[0].text +``` + +**Claude 通常 3/3 全对**（包括 A 纯 prompt）—— 对照 gemma4:e4b 可能只 1-2/3 对，能看到 CoT 对小 model 的价值。 + +
+ +> 🧠 **什么时候别自己写 CoT**：对 **reasoning-native 模型**（Claude Opus 4.x、o 系列、Gemini thinking 等内置思考的模型），用它们的 extended thinking 通常比你手写“Let's think step by step”更好；硬塞步骤反而可能干扰它本来的推理。手写 CoT 仍适用于不具内置推理的一般 chat model。 + +### 练习 4：Iterative Refinement +拿一个模糊的 prompt，refine 5 次。把每一轮记下来。观察哪些改动会提升质量。 + +
+📋 起手码 — Path A（本机 Ollama gemma4:e4b、默认）（复制到 practice_4.py）— 这题没有“对错”、重点是观察过程 + +```python +# 需要：pip install openai +# 前置：ollama pull gemma4:e4b && ollama serve +import sys +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +# 5 个 iteration、每一轮 prompt 都比前一轮更具体 +PROMPTS = { + "v1 模糊": "写一段介紹 ReAct 的文字。", + "v2 加目标读者": "写一段介紹 ReAct 的文字、给写过 Python 的软体工程师看。", + "v3 加格式": "写一段介紹 ReAct 的文字、给写过 Python 的软体工程师看。100 字以內、用一个段落。", + "v4 加 example 要求": "写一段介紹 ReAct 的文字、给写过 Python 的软体工程师看。100 字以內、用一个段落、结尾举一个具体例子（譬如查天气）。", + "v5 加禁忌": "写一段介紹 ReAct 的文字、给写过 Python 的软体工程师看。100 字以內、用一个段落、结尾举一个具体例子（譬如查天气）。不要用「賦能」「驅动」「智能」这類空泛词彙。", +} + +outputs = {} +for label, prompt in PROMPTS.items(): + r = client.chat.completions.create( + model="gemma4:e4b", + max_tokens=200, + messages=[{"role": "user", "content": prompt}], + ) + text = r.choices[0].message.content + outputs[label] = text + print(f"\n--- [{label}] ({len(text)} chars) ---") + print(text) + +# === 自我验证 === +v1_len, v5_len = len(outputs["v1 模糊"]), len(outputs["v5 加禁忌"]) +banned_words = ("賦能", "驅动", "智能") +v5_has_banned = any(w in outputs["v5 加禁忌"] for w in banned_words) +assert v5_len > 0, "v5 必須有输出" +assert not v5_has_banned, f"v5 应该避免禁忌词、实际含: {[w for w in banned_words if w in outputs['v5 加禁忌']]}" +print(f"\n✅ 练习 4 通过 — v5 长度 {v5_len}、无禁忌词（本机 $0）") +print(f"💡 观察：v1 ({v1_len} chars) 通常比 v5 ({v5_len} chars) 「鬆」、加约束会逼 prompt 收斂") +print("💡 用 gemma4:e4b 跑这题特别有感——小 model 对 prompt 质量极敏感、5 轮 refine 的差距会比 Claude 更明显") +``` + +
+ +
+📋 起手码 — Path B（Anthropic API、选择性）（复制到 practice_4_anthropic.py） + +把 Path A 的 client + 循环內 `client.chat.completions.create(...)` 改成： + +```python +import anthropic +client = anthropic.Anthropic() + +# 循环內： +msg = client.messages.create(model="claude-haiku-4-5", max_tokens=200, + messages=[{"role": "user", "content": prompt}]) +text = msg.content[0].text +``` + +其余 PROMPTS / outputs / assert 邏輯完全相同。**成本**：5 次 ≈ $0.002。 + +**Claude vs gemma4 对 prompt 细致度的差别**：Claude haiku 通常 v1 已能写出 OK 段落、v5 加上约束后优化幅度较小；小 model v1 常空泛无用、v5 加禁忌后才開始能读。 + +
+ +**进阶做法**：把这 5 轮输出全存进 csv、Stage 7 练习 2 会教怎么把这变成 eval harness 量化“prompt 改善了多少”。 + +## 🎯 精选 Projects + +按用途分 4 类、9 个项目一张表搞定。**挑入口看“适合谁”、想深入点连结看 repo / 网站**。 + +| 分类 | Project | ⭐ | 适合谁 | 为什么推荐 / 备注 | +|---|---|---|---|---| +| **学术 / 教学风 guide**
（先看这个） | [dair-ai/Prompt-Engineering-Guide](https://github.com/dair-ai/Prompt-Engineering-Guide) | ⭐⭐⭐⭐⭐ | 当参考书、需要某技巧再来查 | 从基础到进阶（CoT / ToT / ReAct / RAG）端到端，★ 74k+、MIT | +| | [PromptingGuide.ai](https://www.promptingguide.ai/) | ⭐⭐⭐⭐ | 手机阅读、想要可跑范例 | 跟 dair-ai GitHub 同样内容、做成网站 + 可跑范例 | +| | [NirDiamant/Prompt_Engineering](https://github.com/NirDiamant/Prompt_Engineering) | ⭐⭐⭐⭐ | 偏好“边跑边学” | 22 种技巧、独立 notebook，比 dair-ai 更动手，★ 7k+| +| **官方 cookbook** | [Anthropic Cookbook — Prompt patterns](https://github.com/anthropics/claude-cookbooks) | ⭐⭐⭐⭐⭐ | Claude 进阶 prompting（含 prompt caching / multimodal）| Stage 1 已介绍、本 stage 重点看 `misc/prompt_caching.ipynb` 跟 `multimodal/` | +| | [GoogleCloudPlatform/generative-ai](https://github.com/GoogleCloudPlatform/generative-ai) | ⭐⭐⭐ | 用 Google 技术栈（PaLM / Gemini）| Google Cloud 的 prompting cookbook、跨厂商观点 | +| **灵感 collection**
（找模式、不要照抄）| [f/awesome-chatgpt-prompts](https://github.com/f/awesome-chatgpt-prompts) | ⭐⭐⭐ | 卡关时找灵感 | 上百个“Act as a [角色]...”prompt，★ 162k+、CC0。**把模式拿出来改写、不要照抄** | +| **Production 管理**
（规模化）| [microsoft/prompt-engine](https://github.com/microsoft/prompt-engine) | ⭐⭐⭐ | production 要管很多 prompt 时 | TypeScript library、管理样板 + 对话历史 | +| | [microsoft/promptflow](https://github.com/microsoft/promptflow) | ⭐⭐⭐ | 团队型应用、需要 eval | 视觉化 prompt 设计 + 评估工具，★ 11k+ | +| | [stanfordnlp/dspy](https://github.com/stanfordnlp/dspy) ⭐ **Stage 2 → 3 桥** | ⭐⭐⭐⭐⭐ | 跑完 dair-ai 想规模化 prompt | 把 prompt 当 code 写，用 compiler 自动最佳化，★ 34k+、MIT | + +> **注**：dspy 是 framework 不是 tutorial、门槛较高，建议搭配 [dspy.ai](https://dspy.ai/) 官方 tutorial 读；NirDiamant 用 NOASSERTION 自订条款（偏研究 / 非商用）。 + +> 💡 **建议阅读路径**：dair-ai guide 入手（理论） → Anthropic Cookbook 看 Claude 实作 → NirDiamant 边跑边学 → 进 production 时读 dspy。 + +## 🔭 进阶：prompt → context → harness 三层 engineering + +LLM-powered system 的工程实践可以拆成 **3 层 stack**。这不是 1 次 call vs N 次 call 的区别，而是每一层工程的对象 **不一样**： + +- **Prompt Engineering**（本 stage）= 工程 **送进模型的那段字符串** +- **Context Engineering**（Stage 6）= 工程 **每次 call 时，context window 里装什么信息**——动态组装 RAG retrieve 结果、memory、tool definitions、对话 history +- **Harness Engineering**（Stage 7）= 工程 **模型外围的执行与控制层**——agent loop、retry、sandbox、observability、deployment 等所有非 LLM 代码 + +→ 三层 **正交**：一次 call 的 RAG app 也在做 context engineering（重点是组 context，不是 call 几次）；50 次 call 但没做 retrieval 的 chatbot 仍然只是在做 prompt engineering。 + +**这条路线里的完整三层 lineage**： + +| Discipline | 工程“什么” | 在哪一 stage 完整学 | +|---|---|---| +| **1. Prompt Engineering** | 送进 LLM 的字符串本身（system prompt / few-shot / format） | **本 stage（Stage 2）** | +| **2. Context Engineering** | context window 里装什么信息（RAG / memory / tool defs / history） | [Stage 6 — Context Engineering：RAG 与 Memory](06-memory-rag.zh-Hans.md) | +| **3. Harness Engineering** | 模型外围的执行与控制层（agent loop / retry / sandbox / observability） | [Stage 7 — Multi-Agent · Production 化](07-multi-agent-production.zh-Hans.md) | + +> 💡 **Karpathy 2025-06**：context engineering 是把 **刚好对下一步有用的信息** 填进 context window 的精细艺术。 +> +> 💡 **Simon Willison / Addy Osmani**：“coding agent = LLM + harness”——harness 就是“模型外围的控制系统”、retry / loop / 监测 / 沙盒 / 部署这些不是 LLM 本身的代码。[OpenAI 也在 2026-02 使用了 "Harness Engineering" 这个说法](https://openai.com/index/harness-engineering)。 + +**这个 stage 不用学完后两层**，这里只是给你一个方向。等你进入 Stage 6 / 7，会发现它们是在接着这条 lineage 往上走。 + +延伸阅读（不必修、未来想深挖时看）： + +- [`Meirtz/Awesome-Context-Engineering`](https://github.com/Meirtz/Awesome-Context-Engineering)（★ 3k+）——从 prompt engineering 一路推到 production agent 的 survey +- [`Windy3f3f3f3f/how-claude-code-works`](https://github.com/Windy3f3f3f3f/how-claude-code-works)（★ 3k+）——Claude Code 内部解析，含 context engineering 章节 + +## ✅ 进 Stage 3 前的自我检查 + +你能不能： + +- [ ] 写一个有 system message + user message + 3 个示例 message 的 prompt（few-shot） +- [ ] 示范 CoT 在某个推理任务上提升准确率 +- [ ] 反复 refine 一个 prompt 5 次，每一版都留下记录 +- [ ] 看出 prompt 不是对的工具的时候（这时要用 tool use） + +如果可以 → 进 [Stage 3 — Tool Use & Agent 入门](./03-tool-use-and-hello-agent.zh-Hans.md)。这是最重要的一个阶段——prompt 不要急着跳过去，但也不要卡在这里。 diff --git a/stages/03-tool-use-and-hello-agent.en.md b/stages/03-tool-use-and-hello-agent.en.md new file mode 100644 index 0000000..a5b13c9 --- /dev/null +++ b/stages/03-tool-use-and-hello-agent.en.md @@ -0,0 +1,514 @@ +# Stage 3 — Tool Use & Hello Agent ⭐ + +> [繁體中文](./03-tool-use-and-hello-agent.md) | [简体中文](./03-tool-use-and-hello-agent.zh-Hans.md) | **English** + +⏱️ **Estimated Time**: 2-3 weeks (approx. 10-20 hours) + +> 💡 Terminology-heavy (agent / tool use / function calling / ReAct / structured output) → See [`resources/glossary.md` 2](../resources/glossary.en.md#2-agents--tool-use). +> 🗺️ **Before choosing Track A (CLI Power User) or Track B (Agent Builder)**, read [`resources/agent-paradigms.md`](../resources/agent-paradigms.md) — a panoramic view of 5 agent archetypes to help you choose your path. + +> 📋 **Chapter Structure**: [Opening Framing: The relationship between AI/LLM/Agent] → Learning Objectives → Prerequisites → Required Reading → [Optional · Concept Map] → Hands-on Exercises → Reflection (Concepts + Routing) → Curated Projects → Self-Check +> 🔑 **Key Terms**: See [`resources/glossary.md` 2](../resources/glossary.en.md#2-agents--tool-use) + +## 🤖 Before We Start: AI / LLM / Agent — How Do They Differ? + +> **This section is for "opening framing" (a top-down pedagogy)**: First, build the mental hierarchy in the learner's mind, then move on to Learning Objectives and Exercises. This section provides only **brief explanations + comparisons**. In-depth introductory materials are already canonical references in both English and Chinese (see resources below). **This is not a rewrite of hello-agents Ch1.** + +### A Hierarchy Diagram to Establish Understanding + +![AI / ML / DL / LLM vs. Agent](../resources/diagrams/ai-ml-llm-agent-hierarchy.en.png) + +→ **An "Agent" is not a "more advanced model than an LLM," nor is it a branch under the LLM classification tree**. An agent is a **cross-layer abstract system** that uses an LLM as one of its components. Cursor / Claude Code / Hermes Agent all use the same LLMs internally (Claude / GPT / Gemini)—the difference is how they wrap the LLM in a tool-calling loop. + +### Three-Line Comparison (The Quickest Version) + +| Term | What It Is | What You Give It, What It Returns | Example | +|---|---|---|---| +| **AI** | The entire field of study | Too abstract to be "used" directly | ML, DL, LLM, RL are all subfields of AI | +| **LLM** | A single model that maps text to text | Give a prompt → Get text back | GPT-5, Claude, Llama 3, Qwen | +| **Agent** | A **system** of LLM + tools + loop | Give a task → It completes it in multiple steps | Cursor, Claude Code, Hermes Agent | + +**One sentence**: An LLM is like a brain that understands and generates text; an Agent connects that brain to tools, workflow, and feedback loops so it can finish multi-step tasks as a system. + +### The 3 **Minimum Necessary** Components of an Agent (This is the core difference between an agent and an LLM) + +| Component | Role | Where to Learn | +|---|---|---| +| 🧠 **LLM** (brain) | Reasoning / Decision-making / Natural language | Already learned in Stage 1 | +| 🔧 **Tools** (hands) | Acting on the world (calling APIs, running code, looking up data) | **This stage** | +| 🔁 **Loop** (heartbeat) | Think → Act → Observe → Think again (ReAct) | **Exercise 3 of this stage** | + +→ **These 3 together are the minimum definition of an agent**. Without tools / loop, it's just "LLM + your own retry logic," not an agent. + +### Classic Agent Paradigms (Thinking Patterns) + +After learning the 3 minimum components, the next layer is "**how the LLM thinks**." Chapter 4 of hello-agents, "Building Classic Agent Paradigms," is all about this. A brief comparison: + +| Paradigm | What It Is | Where to Learn | +|---|---|---| +| **CoT** (Chain-of-Thought) | The LLM writes out its reasoning process before giving the answer, not just the conclusion—it's a **prompting technique**, not an agent architecture | **Stage 2** Learning Objectives + Hands-on Exercises (Reasoning Task CoT) | +| **ReAct** (Reasoning + Acting) | Applying CoT within a Loop: Thought → Action (call tool) → Observation (see result) → Thought... It's the **most common implementation of the Loop component** | **Exercise 3 of this stage** + [ReAct paper (Yao 2022)](https://arxiv.org/abs/2210.03629) | +| **Reflection** | After a run, the LLM critiques its own work and re-answers based on feedback | **Reflection of this stage** (concept + routing) | +| **Planning** (Task Decomposition) | Breaking a large task into sub-tasks, which can be assigned to multiple agents | **Stage 4** What is a multi-agent framework | + +→ These paradigms are all variations of "**LLM self-guidance**," built on top of the 3 components (LLM + Tools + Loop). **"What is an agent" is explained by the 3 components; "How an agent thinks" requires these 4 paradigms for a complete picture.** + +> 💡 **Extended Components** (infrastructure that makes agents stronger, but **not a criterion for "is it an agent?"**): +> - **Memory / RAG** (agent can remember things across conversations) → Taught completely in **Stage 6** +> - **Reflection / self-critique** (agent looks at its own answer, finds problems, and goes back to fix them) → Basic version in **Reflection of this stage** (concept + paper routing); advanced version with persistent memory in **Stage 6 Reflexion with Memory** +> - **Production harness** (telemetry / safety / retry / orchestration) → **Stage 5 5.7** +> +> These are all advanced patterns—Stage 3 teaches the minimum viable agent, and later stages teach how to make it stronger. + +### 📚 In-Depth Introductory Resources (English / Video-first) + +**🇺🇸 English**: +1. [**Andrej Karpathy — "Intro to Large Language Models"**](https://www.youtube.com/watch?v=zjkBMFhNj_g) ⭐⭐⭐ (1hr) — A visual intro to LLMs from scratch (ex-OpenAI / ex-Tesla AI Director, the most valued LLM intro video in the English-speaking world). +2. [**Andrej Karpathy — "Let's build GPT from scratch"**](https://www.youtube.com/watch?v=kCc8FmEb1nY) ⭐⭐ (2hr) — For those who want to see inside an LLM down to the code level. +3. [**3Blue1Brown — "But what is a Transformer?"**](https://www.youtube.com/watch?v=wjZofJX0v4M) ⭐⭐⭐ — A visual explanation of LLMs, the most recommended visual tutorial in the English-speaking world. +4. [**Lilian Weng — "LLM Powered Autonomous Agents"**](https://lilianweng.github.io/posts/2023-06-23-agent/) ⭐⭐⭐ — The canonical 1-page agent anatomy (Planning / Memory / Tool use / Action), the most cited agent dissection in the English-speaking world. +5. [**Anthropic — "Building Effective Agents"**](https://www.anthropic.com/research/building-effective-agents) ⭐ — Anthropic's perspective: when to use an agent, and when a workflow is enough. +6. [**Chip Huyen — "Agents"**](https://huyenchip.com/2025/01/07/agents.html) — A practitioner's perspective, a full chapter's worth of depth. + +**🀄 Chinese**: +1. [**Hung-yi Lee — Introduction to Generative AI (Spring 2024 NTU Course)**](https://speech.ee.ntu.edu.tw/~hylee/genai/2024-spring.php) ⭐⭐⭐ — The highest quality academic-level introduction to AI / LLM / agents in the Chinese-speaking world. Each episode is 30-60 minutes, taught at National Taiwan University, with official page including slides + YouTube links. Covers both LLM and agent concepts. The latest integrated version can be found at [**GenAI-ML 2025 Fall**](https://speech.ee.ntu.edu.tw/~hylee/GenAI-ML/2025-fall.php), and the main YouTube channel is [**@HungyiLeeNTU**](https://www.youtube.com/@HungyiLeeNTU). +2. [**datawhalechina/hello-agents** Ch1 "First Look at Agents"](https://github.com/datawhalechina/hello-agents) ⭐ — The most complete text-based introduction to agents in Chinese. +3. [**datawhalechina/hello-agents** Ch2 "The History of Agent Development"](https://github.com/datawhalechina/hello-agents) — The evolutionary path from BabyAGI → AutoGPT → Claude Code. +4. [**3Blue1Brown Chinese Dubbed Version**](https://www.youtube.com/@3Blue1BrownCN) — Visual explanations of LLM / Transformer (in Chinese). + +**Optional / Advanced Reading**: +- [**Simon Willison — "I think 'agent' may finally have a widely enough agreed upon definition"**](https://simonwillison.net/2025/Sep/18/agents/) — A working definition: "an agent runs tools in a loop to achieve a goal," including debates over different definitions from OpenAI and others (**for those with a foundational understanding**). +- [**DeepLearning.AI Short Courses**](https://www.deeplearning.ai/short-courses/): "AI Agents in LangGraph" / "Multi AI Agent Systems with crewAI" / "Functions, Tools and Agents with LangChain" (**Most APIs are from 2023-2024**, focus on the concepts and cross-reference the official latest docs for code). +- [**microsoft/ai-agents-for-beginners**](https://github.com/microsoft/ai-agents-for-beginners) — Microsoft's official 12-lesson intro to building AI agents (MIT, ★ 64k+). Structured, English, with code; a parallel beginner course, not a substitute for this stage's hands-on exercises. +- [**liyupi/ai-guide**](https://github.com/liyupi/ai-guide) — The largest AI resource **aggregator** repo in the Chinese-speaking world (not original educational material, suitable for broad exploration). + +> 📌 **Resource List Limit Rule**: This section is a router, not a tutorial. The main list has a combined limit of **10 items** (6 English + 4 Chinese). To add a new resource, **one must be removed first**. The optional reading section does not count towards the main list limit. + +> 💡 **Recommended Learning Path**: First, watch 1-2 videos (English: Karpathy / 3Blue1Brown; Chinese: Hung-yi Lee) to build a visual mental model → then read 1-2 blog posts (Lilian Weng / Anthropic) to get a working definition → then return to this stage for hands-on exercises. **You don't need to consume everything**; this is a reference library, not a reading list. + +--- + +This is the most critical stop on the entire learning path. **You don't truly understand agents until you've built one**. It is recommended to hand-code the basic exercises in this stage at least once, then refer to [hello-agents](https://github.com/datawhalechina/hello-agents) or the curated projects in this stage for more in-depth material as needed. + +## 📌 Learning Objectives + +After completing this stage, you will be able to: +- Explain why LLMs need tools (they are not omnipotent, and they can't do anything beyond text). +- Define a tool schema and have an LLM call it. +- Write a single-step ReAct agent from scratch (without any framework). +- Write a multi-step ReAct agent and let it decide when to stop. +- Distinguish which problems require tool use and which can be solved with a pure prompt. + +## 🚪 Entry Conditions + +You should already have: +- Access to Claude / OpenAI / Gemini API (Stage 1). +- A basic grasp of prompt engineering (Stage 2). +- The ability to write a Python function that takes JSON in and returns JSON out. + +## 📚 Required Reading + +1. [**Anthropic — Tool Use**](https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/overview) — The official guide. +2. [**anthropics/courses — Tool Use**](https://github.com/anthropics/courses) ⭐⭐⭐⭐⭐ ★ 21k+ — Anthropic's official 5-course umbrella; **module 5 "Tool Use" maps to this stage**. Jupyter notebook exercises covering multimodal prompts / streaming / tool implementation walkthrough. +3. [**ReAct: Synergizing Reasoning and Acting in Language Models**](https://arxiv.org/abs/2210.03629) — Yao et al. 2022, the foundational paper. Read at least the abstract and Section 3. +4. [**OpenAI — Function Calling**](https://platform.openai.com/docs/guides/function-calling) — For reference on the function calling format. +5. [**Build an agent from scratch**](https://shafiqulai.github.io/blogs/blog_3.html) — A narrative-style guide to building an agent from scratch. + +## 🛠 Hands-on Exercises (Basic Illustrative Exercises) + +> 🦙 **This stage defaults to using Ollama qwen2.5:3b** (for cost reasons and stable tool-use support). Starting from Stage 3, with tool calling / ReAct loops, `gemma4:e4b` is insufficient; we switch to `qwen2.5:3b` (1.9 GB, install with `ollama pull qwen2.5:3b`). Each exercise has a Path A (Ollama, default) + Path B (Anthropic, optional, for when you want to see high-quality tool-use in the cloud). +> +> 💰 **Stage 3 Budget Estimate** (for all 6 exercises, with heavy tool use): **All local = $0**; **all haiku ≈ $0.50**; **all sonnet ≈ $1.50**. The ReAct loop exercise is about 4-6 tool calls × 5 exercises × 5 reps ≈ $0.80 on haiku. For the full budget, see [`examples/README.md#recommended-llms`](../examples/README.en.md#recommended-llm-list). +> +> For the full 3-way trade-off, see [`examples/README.md`](../examples/README.en.md#three-paths--default-is-ollama-cost-driven). +> +> 🆘 **Stuck?** Tool calling has the steepest learning curve in the entire curriculum. Install the [`examples/stage-5/tool-calling-tutor/`](../examples/stage-5/tool-calling-tutor/) skill—when you prompt Claude Code with "Why isn't my LLM calling my tool?" or "What's wrong with my schema?", it will auto-load and walk you through a 4-symptom diagnostic process. +> +> 🪜 **This stage is the starting point for single-agent**: one LLM + ReAct loop. For **multi-agent concepts** (multiple agents collaborating), see [Stage 4 What is a multi-agent framework](04-agent-frameworks.en.md#-what-is-a-multi-agent-framework); for **Claude's native subagent mechanism** (`.claude/agents/` + Task tool, no framework needed), see [Stage 5.5](05-claude-code-ecosystem.en.md#55--subagents-claude-codes-native-multi-agent-mechanism--2025-new-feature). + +### ⚠️ Know the risk first: giving an agent tools = giving it an attack surface + +The moment you hand an LLM tools, you also hand it an attack surface. The clearest framing is Simon Willison's **lethal trifecta**: an agent is exploitable when it simultaneously has all three of — + +1. **access to private data** (your files / DB / API keys) +2. **exposure to untrusted content** (web pages, emails, documents others send — any of which can hide instructions) +3. **the ability to communicate externally** (make requests, send mail, write files) + +The root cause is that an LLM "follows instructions found in content" and can't reliably tell yours apart from ones smuggled inside untrusted data — that's **prompt injection**. This stage just builds the awareness; concrete defenses (isolate untrusted input, permission gates, least-tool sets, human review of high-risk actions) come in [Stage 8](08-agent-interfaces.en.md) and [Stage 5](05-claude-code-ecosystem.en.md). Glossary: [prompt injection / lethal trifecta](../resources/glossary.en.md). + +--- + +### Exercise 1: Function Calling (One Tool, One Call) +Give Claude a tool (a fake weather API) and a question ("Is it raining in Taipei right now?"). See how Claude calls the tool, gets the result, and then answers you. + +
+📋 Starter Code — Path A (Local Ollama qwen2.5:3b, default) (copy to practice_1.py) + +```python +# Requires: pip install openai +# Prerequisite: ollama pull qwen2.5:3b && ollama serve +# Note: Stage 3+ uses qwen2.5:3b (stable tool-use), not gemma4:e4b +import sys, json +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +# Step 1: Define the tool schema — OpenAI-compatible format wrapped in {"type":"function", "function":{...}} +weather_tool = { + "type": "function", + "function": { + "name": "get_weather", + "description": "Get the current weather for a city (sunny/rainy/cloudy), returns a short string.", + "parameters": { + "type": "object", + "properties": { + "city": {"type": "string", "description": "The city name (e.g., 'Taipei')"}, + }, + "required": ["city"], + }, + }, +} + +# Step 2: Ask a question and let the LLM decide whether to call the tool +resp = client.chat.completions.create( + model="qwen2.5:3b", + max_tokens=512, + tools=[weather_tool], + messages=[{"role": "user", "content": "Is it raining in Taipei right now?"}], +) + +# === Self-Verification === +msg = resp.choices[0].message +print("finish_reason:", resp.choices[0].finish_reason) +print("tool_calls:", msg.tool_calls) + +assert msg.tool_calls, "Expected the LLM to choose to call a tool (instead of answering directly)" +tc = msg.tool_calls[0] +assert tc.function.name == "get_weather", f"Expected to call get_weather, but got {tc.function.name}" +args = json.loads(tc.function.arguments) +assert args.get("city"), "Expected the city parameter to have a value" +print(f"✅ Exercise 1 Passed — qwen2.5:3b correctly chose get_weather with city='{args['city']}'") +``` + +**Expected Output** (sample): +``` +finish_reason: tool_calls +tool_calls: [ChatCompletionMessageToolCall(id='call_xxx', function=Function(name='get_weather', arguments='{"city": "Taipei"}'), type='function')] +✅ Exercise 1 Passed — qwen2.5:3b correctly chose get_weather with city='Taipei' +``` + +**Verify logic without installing Ollama**: Use `unittest.mock.MagicMock` to replace the client, feed it a fixed response, and the asserts will still work. For a complete mock example, see [`examples/stage-3/03-react-from-scratch/test.py`](../examples/stage-3/03-react-from-scratch/test.py) (the pattern is cross-backend compatible). + +
+ +
+📋 Starter Code — Path B (Anthropic API, optional) (copy to practice_1_anthropic.py) + +```python +# Requires: pip install anthropic +# Environment variable: export ANTHROPIC_API_KEY=sk-ant-... +import anthropic + +client = anthropic.Anthropic() + +# Anthropic native tool schema — no wrapper needed +weather_tool = { + "name": "get_weather", + "description": "Get the current weather for a city (sunny/rainy/cloudy), returns a short string.", + "input_schema": { + "type": "object", + "properties": { + "city": {"type": "string", "description": "The city name (e.g., 'Taipei')"}, + }, + "required": ["city"], + }, +} + +resp = client.messages.create( + model="claude-3-haiku-20240307", + max_tokens=512, + tools=[weather_tool], + messages=[{"role": "user", "content": "Is it raining in Taipei right now?"}], +) + +# === Self-Verification === +assert resp.stop_reason == "tool_use", f"Unexpected stop_reason: {resp.stop_reason}" +tool_calls = [b for b in resp.content if b.type == "tool_use"] +assert tool_calls[0].name == "get_weather" +assert tool_calls[0].input.get("city") +print(f"✅ Exercise 1 Passed (Anthropic) — Claude chose get_weather with city='{tool_calls[0].input['city']}'") +``` + +**3 Key SDK Differences**: +- **Schema wrap**: Anthropic is direct `tools=[{name, description, input_schema}]`; OpenAI/Ollama needs to be wrapped in `[{"type":"function", "function":{...}}]` +- **Response path**: Anthropic gets it from `resp.content[i].type=="tool_use"`; OpenAI/Ollama from `resp.choices[0].message.tool_calls[i]` +- **Args format**: Anthropic `.input` is a dict (auto-parsed); OpenAI/Ollama `.function.arguments` is a JSON string, requires `json.loads(...)` + +**Cost**: 1 call ≈ $0.001. **Claude's tool-use is more stable than qwen2.5:3b**—the gap becomes obvious in complex scenarios (5+ tools, ambiguous questions). + +
+ +### Exercise 2: Multi-Tool Selection +Give Claude three tools (search, calculator, calendar) and a task. See how Claude picks a tool, and pay attention to when it picks the wrong one. + +
+📋 Simplified Core Concept — Path A (Ollama) + +**NEW vs Exercise 1**: Tools go from 1 to 3. The LLM decides which to pick based on the `description` boundaries—the more the `description` is written like a "docstring for humans," the more likely it is to pick the wrong one. + +```python +from openai import OpenAI +import json + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +TOOLS = [ + {"type": "function", "function": {"name": "web_search", + "description": "Search for current or external information not in the prompt.", + "parameters": {"type": "object", "properties": {"query": {"type": "string"}}, "required": ["query"]}}}, + {"type": "function", "function": {"name": "calculator", + "description": "Evaluate basic arithmetic with +, -, *, /, and parentheses.", + "parameters": {"type": "object", "properties": {"expression": {"type": "string"}}, "required": ["expression"]}}}, + {"type": "function", "function": {"name": "calendar_lookup", + "description": "Look up events for a specific date.", + "parameters": {"type": "object", "properties": {"date": {"type": "string"}}, "required": ["date"]}}}, +] + +resp = client.chat.completions.create(model="qwen2.5:3b", tools=TOOLS, + messages=[{"role": "user", "content": "What is (19 * 42) - 8?"}]) + +tc = resp.choices[0].message.tool_calls[0] +print(f"LLM picked: {tc.function.name}, args: {json.loads(tc.function.arguments)}") +# Expected: calculator, {"expression": "(19 * 42) - 8"} +``` + +**The punchline**: The `description` boundaries of the 3 tools must be mutually exclusive. Writing "calendar" for the `calendar` tool is too vague and will clash with `web_search`; writing "events for a specific date" is clear. Small models are more sensitive to description quality than Claude. + +**Path B (Anthropic) is 3 lines different**: remove the `{"type": "function", "function": {...}}` wrapper from the schema, `tool_calls` becomes `[b for b in resp.content if b.type == "tool_use"]`, and `tc.input` is already a dict, no `json.loads` needed. Full version in the folder. + +
+ +→ **Basic starter template** → [`examples/stage-3/02-multi-tool-selection/`](../examples/stage-3/02-multi-tool-selection/) (starter.py contains stubs + simple tests, illustrative, **not a chapter-length full tutorial**; for in-depth chapters, see the 📚 hello-agents callout at the start of the stage) + +### Structured Outputs (JSON mode) ⭐ function calling's twin + +Function calling is "**let the model decide whether to act**"; **structured output is "force the model to return a fixed-shape JSON"**. They are easy to confuse but serve different ends: the former lets an agent take action, the latter gives you machine-parseable data (filling forms, classification, extraction, eval scoring). + +**Three approaches (weak to strong)**: +1. **Ask for JSON in the prompt**: simplest, but the model sometimes adds chatter or drifts from the format. +2. **JSON mode / `response_format`**: the API guarantees valid JSON (but not that it matches your schema). +3. **JSON-schema enforcement / constrained decoding**: locks the schema too, so the output always conforms (most reliable). + +> 💡 Why it matters: agent state, tool arguments, and eval scoring all depend on getting structured data back. This is the load-bearing reliability layer underneath tool calling. + +**Hands-on tools**: [jxnl/instructor](https://github.com/jxnl/instructor) (★13k, use a Pydantic model as the schema, with auto-retry); [dottxt-ai/outlines](https://github.com/dottxt-ai/outlines) (★14k, constrained decoding, schema-locks even local LLMs). Stage 4's Pydantic AI is on the same path. + +### Exercise 3: Implement ReAct from Scratch (No Framework) +Write the Thought → Action → Observation loop in 50-80 lines of Python. No LangChain, no LangGraph, just a pure `while not done: thought; action; observation; ...`. + +
+📋 Simplified Core Concept — Path A (Ollama), the entirety of the ReAct loop is in these 13 lines + +**NEW vs Exercise 2**: Wrap the single call in a loop, `messages` keeps growing, and check for the presence of `tool_calls` to decide when to finish. + +```python +# Assuming TOOLS + TOOL_IMPL (dict: name → callable) are defined as in Exercise 2 +messages = [{"role": "user", "content": "Population of Taipei divided by population of New York?"}] + +for step in range(5): # max_iter safety net + r = client.chat.completions.create(model="qwen2.5:3b", tools=TOOLS, messages=messages) + msg = r.choices[0].message + # Append the assistant's response back to messages (Important! So the LLM can see what it said in the last turn) + messages.append({"role": "assistant", "content": msg.content, "tool_calls": msg.tool_calls}) + if not msg.tool_calls: + print(f"✅ Finishing: {msg.content}"); break + for tc in msg.tool_calls: + args = json.loads(tc.function.arguments) + obs = TOOL_IMPL[tc.function.name](args) # Execute locally + # Append the observation back to messages (using role="tool", with tool_call_id) + messages.append({"role": "tool", "tool_call_id": tc.id, "content": obs}) +``` + +**3 common pitfalls**: +1. **Forgetting to add the assistant's response back to `messages`**—the LLM won't see what it said last turn and will loop forever. +2. **The `tool` message is missing `tool_call_id`**—the LLM can't match which result corresponds to which call. +3. **No `max_iter`**—if the tool results are poorly written, the LLM will call it infinitely; a safety net is a must. + +**Path B (Anthropic) has a few differences**: the loop structure is identical, `msg.tool_calls` becomes `[b for b in resp.content if b.type == "tool_use"]`, use `stop_reason == "end_turn"` to check for stopping, and the tool result is wrapped in `{"type": "tool_result", "tool_use_id": ..., "content": obs}` and placed in the user message. Full version in the folder. + +
+ +→ **Basic starter template** → [`examples/stage-3/03-react-from-scratch/`](../examples/stage-3/03-react-from-scratch/) (includes a mock-based test.py, so you can verify without spending API money; illustrative, **not a chapter-length full tutorial**—for in-depth chapters, see the 📚 hello-agents callout at the start of the stage) + +### Exercise 4: Multi-Step Reasoning Task +A task that requires 3-5 consecutive tool calls. For example: "Find the population of Taipei, divide it by the population of New York, then convert the ratio to a percentage." Each step uses a different tool. + +
+📋 Simplified Core Concept — Same loop as Exercise 3, just runs longer + +**NEW vs Exercise 3**: **Exactly the same loop**—just `TOOLS` is replaced with 4 tools (`lookup_population` / `divide` / `to_percentage` / `round_int`), and the problem naturally takes 4 tool calls to finish. + +```python +# No new code, purely a change in TOOLS / TOOL_IMPL content +TOOL_IMPL = { + "lookup_population": lambda i: lookup_population(i["city"]), + "divide": lambda i: divide(i["a"], i["b"]), + "to_percentage": lambda i: to_percentage(i["ratio"]), + "round_int": lambda i: round_int(i["x"]), +} +# The loop is exactly the same as Exercise 3, just max_iter is increased to 8 +``` + +**The punchline**: Multi-step reasoning is not a new pattern, it's just **letting the ReAct loop run longer**. **The real challenge is "will the LLM miss a step in the middle?"**—qwen2.5:3b might miss "convert to percentage," while Claude Haiku is more stable. **This is a good experiment to observe the "model scale vs multi-step stability" trade-off**. Try `MODEL=qwen2.5:7b python starter.py` to compare. + +
+ +→ **Basic starter template** → [`examples/stage-3/04-multi-step-reasoning/`](../examples/stage-3/04-multi-step-reasoning/) (starter.py contains stubs + simple tests, illustrative, **not a chapter-length full tutorial**; for in-depth chapters, see the 📚 hello-agents callout at the start of the stage) + +### Exercise 5: Error Handling +Make a tool fail (network error, invalid input). See how the agent handles the error, whether it can recover, and add a retry mechanism. + +
+📋 Simplified Core Concept — a tool error is data, not an exception + +**NEW vs Exercise 4**: A tool error returns a **structured dict**, don't `raise`. The loop feeds the dict back to the LLM, and the model decides whether to retry, change the query, or give up. + +```python +def fetch_weather(city: str) -> dict: + if network_failed(): + return {"error": "network timeout", "retry_hint": "try again in 1s"} + return {"city": city, "forecast": "rain", "temperature_c": 24} + +# in the loop: +obs = fetch_weather(args["city"]) +messages.append({"role": "tool", "tool_call_id": tc.id, + "content": json.dumps(obs, ensure_ascii=False)}) # the error dict is also stringified and appended +# On the next turn, the LLM sees the retry_hint and might retry, give up, or change the query. +``` + +**Why not `raise`?**: `raise` interrupts the loop, giving the LLM no chance to recover. **Production retries are not at the Python layer, but at the LLM layer**—this mental flip is the core of Stage 3, Exercise 5. + +**Bad vs Good error returns**: + +| Bad | Good | +|---|---| +| `raise Exception("failed")` | `return {"error": "network timeout", "retry_hint": "try again in 1s"}` | +| `return "failed"` | `return {"error": "...", "category": "transient", "retry_hint": "..."}` | +| Infinite retry | `max_iter` safety + business-level retry quota | + +**Small model observation**: qwen2.5:3b's follow-up on `retry_hint` is weaker and might just give up; Claude Haiku is more stable. For the full version (including graceful end examples on consecutive failures), see the folder. + +
+ +→ **Basic starter template** → [`examples/stage-3/05-error-handling/`](../examples/stage-3/05-error-handling/) (starter.py contains stubs + simple tests, illustrative, **not a chapter-length full tutorial**; for in-depth chapters, see the 📚 hello-agents callout at the start of the stage) + +### Exercise 6: Function Schema Design (Fixing a Bad Schema) +**First, give the LLM a deliberately bad schema**—a vague `description` ("process data"), all parameters as `type: string`, no distinction between required/optional, and not using enums where they should be used. Observe how the LLM picks the wrong tool and passes the wrong parameters. Then, fix it item by item: +- Write the `description` so the LLM knows at a glance when the tool is applicable (not a docstring for humans). +- Use the correct types for parameters (number / boolean / enum / array), and list the required ones clearly. +- Use enums to constrain ambiguous boundaries (e.g., `unit: "celsius" | "fahrenheit"` instead of `unit: string`). +- Wrap the error return in `{"error": "...", "retry_hint": "..."}` so the LLM can recover. + +> 💡 For a detailed cheatsheet, see [`resources/schema-design-cheatsheet.md`](../resources/schema-design-cheatsheet.md)—5 golden rules + 5 common anti-patterns. + +
+📋 Simplified Core Concept — bad vs good schema comparison + +**NEW vs Exercise 5**: Same tool (temperature conversion), two schema implementations. See the 4 differences. + +```python +# ❌ BAD — qwen2.5:3b will almost certainly get this wrong (Claude haiku might guess right, but the probability drops significantly) +{"name": "convert", "description": "Convert a value.", + "parameters": {"type": "object", "properties": { + "value": {"type": "string"}, "unit": {"type": "string"}}}} + +# ✅ GOOD — qwen can also pick this reliably +{"name": "convert_temperature", + "description": "Use when user asks to convert temperatures between Fahrenheit and Celsius.", + "parameters": {"type": "object", "properties": { + "value": {"type": "number", "description": "Temperature value"}, + "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, + "required": ["value", "unit"]}} +``` + +**4 improvements**: (1) `name` is more specific, (2) `description` says "**when** to use" not "**what** it does", (3) `type` is changed to `number`, (4) added `required` + `enum`. + +**The punchline**: **Effort in writing a good schema can save the cost of a larger model**—small models are more sensitive to schema quality than large models. The same bad schema that Claude might guess right, qwen will almost certainly get wrong. Want to use a cheaper model in production? Your schema must be solid enough to run in production. + +**What to do if you can't get the schema right?**: Install the [`examples/stage-5/tool-calling-tutor/`](../examples/stage-5/tool-calling-tutor/) skill. When you encounter "my LLM isn't calling my tool" or "what's wrong with my schema," it will pop up to help you debug. + +
+ +→ **Basic starter template** → [`examples/stage-3/06-schema-design/`](../examples/stage-3/06-schema-design/) (includes a comparison of bad vs good schema versions; illustrative, **not a chapter-length full tutorial**—for in-depth chapters, see the 📚 hello-agents callout at the start of the stage) + +> 💡 **After hand-writing schemas, meet MCP**: the tool schemas you hand-wrote above already have a real-world standard — **MCP (Model Context Protocol)** standardizes "what a tool looks like and how to call it" into a cross-agent reusable protocol: write it once, and any MCP-capable agent (Claude Code / Cursor / …) can use it. Just remember the name here; [Stage 5.2](05-claude-code-ecosystem.en.md#52--mcp-model-context-protocol--foundation) covers it in depth. + +## 🪞 Reflection (Reflexion / Self-Refine) — Concept + Routing + +> **This section is for concept + routing, not an exercise**. There is no verified working solution, no "Exercise N" label, no success criteria—in adherence with this repo's principle of "no exercises without verified answers, routing at most." Want to get your hands dirty? Read the papers / projects below directly. + +**What is reflection?**: The error handling in Exercise 5 is "LLM makes a mistake → you (externally) catch + retry"; **reflection** is "LLM observes its own mistake → fixes it itself." The difference is where the agency lies—this is the loop that production agents (Cursor / Cline / Claude Code) run every day. + +**Why is this section in Stage 3 and not Stage 6?**: Reflection is classified in both academia (Reflexion paper Shinn 2023, Self-Refine Madaan 2023) and production (Cursor / Claude Code) as a **planning / reasoning loop** mechanism—it's a sibling pattern to ReAct (Exercise 3), **not a memory pattern**. It's the same multi-turn loop of LLM self-guidance, just "what to do next" changes from "call a tool" to "critique myself." + +**Advanced version (full version of Reflexion with persistent memory) → [Stage 6 Advanced: Full Reflexion with Persistent Memory](06-memory-rag.en.md#-advanced-full-reflexion-with-persistent-memory--track-b-elective)**—when reflection needs to be cross-session, storing past failures as context for the next round, this version truly needs a memory layer. + +### A Comparison Chart + +| Pattern | Form | Requires memory? | Where to Learn | +|---|---|---|---| +| **Error handling** (Ex 5) | External catch + retry | No | **Exercise 5 of this stage** | +| **ReAct loop** (Ex 3) | LLM → tool → result → LLM | No | **Exercise 3 of this stage** | +| **Basic reflection / Self-Refine** | Actor → Critic → Actor, single session | No | **Routing in this section (below)** | +| **Full Reflexion** (w/ episodic memory) | Above + store failure reflections, accumulate across sessions | **Yes** | **Stage 6 Advanced: Reflexion with Memory** | + +### 📚 Want to get hands-on / go deeper? Read these directly + +**Papers**: +- [**Reflexion (Shinn et al. 2023)**](https://arxiv.org/abs/2303.11366) ⭐ — The original paper, defines "verbal reinforcement learning." +- [**Self-Refine (Madaan et al. 2023)**](https://arxiv.org/abs/2303.17651) — Single-agent self-critique, the academic definition of "basic reflection." + +**Reference Implementations**: +- [**arunpshankar/react-from-scratch**](https://github.com/arunpshankar/react-from-scratch) — Already listed in the curated projects of this stage, includes a Reflection implementation you can read directly. +- [**LangChain — Reflection Agents (blog)**](https://blog.langchain.dev/reflection-agents/) — A framework implementation reference + a complete working notebook. +- [**datawhalechina/hello-agents**](https://github.com/datawhalechina/hello-agents) — The corresponding chapter (Self-reflection / Self-Refine section, a complete tutorial in Chinese). + +> 💡 **Want to see how reflection looks in a production agent?**: [Stage 5 5.7 Harness Internals](05-claude-code-ecosystem.en.md#57--dissecting-claude-code-source-reference-harness-implementation--a-must-read-for-track-b) dissects the Claude Code source where you can see it—the agent self-evaluates the patch after a tool call, goes back to fix problems, and commits after correction. **This is one of the core building blocks of modern production agents**. + +## 🎯 Curated Projects + +4 categories, 12 projects, all in one table. **For entry points, look at "Who it's for"; for a deeper dive, follow the links and read the repo READMEs**. + +| Category | Project | ⭐ | Who it's for | Why it's recommended / Notes | +|---|---|---|---|---| +| **Official Cookbooks**
(Start here) | [Anthropic — Tool Use Cookbook](https://github.com/anthropics/claude-cookbooks/tree/main/tool_use) | ⭐⭐⭐⭐⭐ | Getting started with Ex 1 / 2 | Single tool → multi-tool → parallel → structured output all in notebooks (key to see: `tool_use/customer_service_agent.ipynb`) | +| | [Anthropic — Quickstarts](https://github.com/anthropics/claude-quickstarts) | ⭐⭐⭐⭐⭐ | After Ex 1/2, want to see "what a real app looks like" | 3 deploy-ready templates (financial / customer-support / computer-use), ★ 16k+. More canonical than community implementations. | +| | [Anthropic — Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents) | ⭐⭐⭐⭐⭐ | After writing Ex 3, **must-read** before Stage 4 | Blog post: when to use agent vs workflow / common patterns / common pitfalls—Anthropic's official conceptual framework. | +| **ReAct from Scratch**
(Understand the principles) | [pguso/ai-agents-from-scratch](https://github.com/pguso/ai-agents-from-scratch) | ⭐⭐⭐⭐⭐ | Exercise 3 (write ReAct from scratch) | Build from scratch with local Ollama, zero framework, good chapter structure. **The cleanest "no framework" reference implementation**. | +| | [arunpshankar/react-from-scratch](https://github.com/arunpshankar/react-from-scratch) | ⭐⭐⭐⭐ | Ex 3 alternative (Gemini-preferred) + want to see reflection variants | ReAct + Reflection + Self-consistency, Gemini-optimized (⚠️ updates slowed after 2025-05, Apache-2.0). | +| | [mattambrogi/agent-implementation](https://github.com/mattambrogi/agent-implementation) | ⭐⭐⭐ | For line-by-line comparison when stuck on Ex 3 | ~150 lines of the most minimal ReAct (⚠️ stagnant since 2024-01, kept as a teaching-toy reference). | +| | [lsdefine/GenericAgent](https://github.com/lsdefine/GenericAgent) | ⭐⭐⭐⭐ | Ex 3/4, want to see a "minimal but complete" framework | Self-evolving framework, ~3K lines, ★ 13k+, supports Claude / Gemini / Kimi / MiniMax. Between a toy and LangGraph. | +| **CodeAct Route**
(Agent writes code as action) | [HuggingFace Smolagents](https://github.com/huggingface/smolagents) | ⭐⭐⭐⭐ | Ex 5 alternative, local LLM experiments | ≤1000 LOC, representative of the CodeAct pattern, ★ 27k+. HF's stance: agents should be small. | +| | [QuantaLogic/quantalogic](https://github.com/quantalogic/quantalogic) | ⭐⭐⭐ | After Ex 3, want to compare CodeAct vs JSON-tool | Another CodeAct route, agent writes Python code directly as action, Apache-2.0. | +| **Chinese Chapter-based In-depth Material**
(Chapter-length) | [datawhalechina/hello-agents](https://github.com/datawhalechina/hello-agents) ⭐ **Recommended for this stage** | ⭐⭐⭐⭐⭐ | Chinese readers who want structured teaching + full coverage | **16 capabilities** including tool use / ReAct / context engineering / sub-agents / circuit breaker / observability. The most complete chapter-based course in Chinese (CC BY-NC-SA, non-commercial). | +| | [HelloAgents (jjyaoao)](https://github.com/jjyaoao/HelloAgents) | ⭐⭐⭐⭐⭐ | Chinese readers who want to run the code from the above material | The code repo for the above material, **please switch to the `learn_version` branch** to align with the chapters (`pip install hello-agents`, CC BY-NC-SA). | +| **Framework Comparison**
(See how frameworks hide the ReAct loop) | [LangChain — ReAct Agent Template](https://github.com/langchain-ai/react-agent) | ⭐⭐⭐ | Come here after you've written Ex 3 yourself | LangGraph Studio template, how a framework abstracts away ReAct. | + +> 💡 **Recommended Reading Path**: Run the Anthropic Cookbook for Ex 1-2 → run pguso/ai-agents-from-scratch for Ex 3 → read Building Effective Agents after Ex 3 → for Chinese chapter-based material, pair hello-agents + jjyaoao → before starting Stage 4, look at the LangChain ReAct template to see the framework abstraction. + +## ✅ Self-Check Before Stage 4 + +Can you: + +- [ ] Define a tool schema (name + description + JSON schema for input/output)? +- [ ] Write a ReAct loop in under 100 lines of Python, without any framework? +- [ ] Explain why an agent needs an "I'm done" exit condition? +- [ ] Compare the CodeAct (code as action) and JSON-tool routes? +- [ ] See which problems don't actually need an agent? + +If yes → Proceed to [Stage 4 — Agent Frameworks](04-agent-frameworks.md). + +If no → Run through Exercise 3 again, don't skip it. If you don't understand what the frameworks are abstracting away for you, the stuff in Stage 4 will look like black magic. diff --git a/stages/03-tool-use-and-hello-agent.md b/stages/03-tool-use-and-hello-agent.md new file mode 100644 index 0000000..e4c315c --- /dev/null +++ b/stages/03-tool-use-and-hello-agent.md @@ -0,0 +1,515 @@ +# Stage 3 — 工具使用與第一個 Agent（Tool Use & Hello Agent）⭐ + +> **繁體中文** | [简体中文](./03-tool-use-and-hello-agent.zh-Hans.md) | [English](./03-tool-use-and-hello-agent.en.md) + +⏱ **時間估算**：2-3 週（約 10-20 小時） + +> 💡 用語密集（agent / tool use / function calling / ReAct / structured output⋯）→ 翻 [`resources/glossary.md` 2](../resources/glossary.md#2-agent--工具使用)。 +> 🗺️ **進 Track A（CLI Power User）還是 Track B（Agent Builder）前**，先看 [`resources/agent-paradigms.md`](../resources/agent-paradigms.md) — 5 種 agent 型態的全景圖，幫你選軌。 + +> 📋 **本章組成**：〔開場框景：AI/LLM/Agent 三者關係〕→ 學習目標 → 進入條件 → 必修閱讀 →〔可選 · 概念地圖〕→ 動手練習 → 反思（概念 + 路由）→ 精選 Projects → 自我檢查 +> 🔑 **關鍵名詞**：見 [`resources/glossary.md` 2](../resources/glossary.md#2-agent--工具使用) + +## 🤖 開始前：AI / LLM / Agent — 三者怎麼分？ + +> **本節是「開場框景」（由大到小 pedagogy）**：先把學習者腦中的 mental hierarchy 建好，再進 學習目標、練習。這節只做**簡短說明 + 對照**，深度入門讀物已經是中英文圈各自的 canonical reference（見下方資源）。**不是重寫 hello-agents Ch1。** + +### 一張階層圖先建立認知 + +![AI / ML / DL / LLM 與 Agent 的關係](../resources/diagrams/ai-ml-llm-agent-hierarchy.png) + +→ **「Agent」不是「比 LLM 更厲害的模型」，也不是 LLM 樹狀分類底下的一個分支**。Agent 是個**跨層抽象的系統**，把 LLM 當作其中一個元件來用。Cursor / Claude Code / Hermes Agent 內部都還是同一批 LLM（Claude / GPT / Gemini）—— 差別是怎麼把 LLM 包進工具呼叫迴圈裡。 + +### 三行對照（最快版） + +| 詞 | 是什麼 | 你給它什麼、它回什麼 | 例子 | +|---|---|---|---| +| **AI** | 整個學科 | 太抽象、不能直接「用」 | ML、DL、LLM、RL 都是 AI 子領域 | +| **LLM** | 把文字映射到文字的單一模型 | 給 prompt → 回字 | GPT-5、Claude、Llama 3、Qwen | +| **Agent** | LLM + 工具 + loop 的**系統** | 給任務 → 自己跑多步驟達成 | Cursor、Claude Code、Hermes Agent | + +**一句話**：LLM 像是會理解與產生文字的大腦；Agent 則是把這個大腦接上工具、流程與回饋迴圈，讓它能完成多步驟任務的系統。 + +### Agent 的 3 個**最小必要**部件（這就是 agent vs LLM 的核心差別） + +| 部件 | 角色 | 在哪學 | +|---|---|---| +| 🧠 **LLM**（brain） | 推理 / 決策 / 自然語言 | Stage 1 已學 | +| 🔧 **Tools**（hands） | 對世界做事（call API、跑 code、查資料） | **本 stage** | +| 🔁 **Loop**（heartbeat） | 想 → 做 → 看結果 → 再想（ReAct） | **本 stage 練習 3** | + +→ **這 3 個合在一起就是 agent 的最低定義**。沒有 tools / loop，那只是「LLM + 你寫 retry」，不算 agent。 + +### Agent 的經典範式（thinking patterns） + +學完最小 3 部件後、下一層是「**LLM 怎麼想**」。hello-agents Ch4「智能體經典範式構建」整章在講這個。簡短對照： + +| 範式 | 是什麼 | 在哪學 | +|---|---|---| +| **CoT**（Chain-of-Thought、思維鏈） | LLM 寫出推理過程再給答案、不只給結論——是個 **prompting 技巧**、不是 agent 結構 | **Stage 2** 學習目標 + 動手練習（推理任務 CoT） | +| **ReAct**（Reasoning + Acting） | 在 Loop 裡套 CoT：Thought（想）→ Action（呼叫 tool）→ Observation（看結果）→ Thought ...，是 **Loop 部件最常見的實作** | **本 stage 練習 3** + [ReAct paper (Yao 2022)](https://arxiv.org/abs/2210.03629) | +| **Reflection** | 跑完一輪後讓 LLM 批改自己、根據 feedback 重答 | **本 stage 反思**（concept + 路由） | +| **Planning**（任務分解） | 把大任務拆成子任務、可分給多個 agent 各做 | **Stage 4** 什麼是 multi-agent framework | + +→ 這些範式都是「**LLM 自我引導**」的不同變化、堆疊在 3 部件（LLM + Tools + Loop）之上。**「Agent 是什麼」用 3 部件就講完了；「Agent 怎麼想」需要這 4 個範式才講得完整**。 + +> 💡 **延伸組件**（agent 變強的 infrastructure、但**不是「是不是 agent」的判準**）： +> - **記憶 / RAG**（agent 能跨對話記住東西）→ **Stage 6** 完整教 +> - **反思 / self-critique**（agent 看自己答案、發現問題、回頭改）→ 基本版見 **本 stage 反思**（concept + paper routing）；帶持久 memory 的進階版見 **Stage 6 Reflexion with Memory** +> - **Production harness**（telemetry / safety / retry / orchestration）→ **Stage 5 5.7** +> +> 這些都是 advanced pattern——Stage 3 教最小可行 agent、後面 stage 教怎麼變強。 + +### 📚 深度入門資源（中英文 / 影片優先） + +**🀄 中文**： +1. [**李宏毅 — 生成式 AI 導論（2024 春台大課程）**](https://speech.ee.ntu.edu.tw/~hylee/genai/2024-spring.php) ⭐⭐⭐ — 中文圈最高品質的 AI / LLM / agent 學術級導論。每集 30-60 分鐘、台大授課、官方頁含投影片 + YouTube 連結。LLM / agent 概念都涵蓋。最新整合版見 [**GenAI-ML 2025 秋**](https://speech.ee.ntu.edu.tw/~hylee/GenAI-ML/2025-fall.php)、YouTube 主頻道 [**@HungyiLeeNTU**](https://www.youtube.com/@HungyiLeeNTU) +2. [**datawhalechina/hello-agents** Ch1「初識智能體」](https://github.com/datawhalechina/hello-agents) ⭐ — 文字版最完整中文 agent 導論 +3. [**datawhalechina/hello-agents** Ch2「智能體發展史」](https://github.com/datawhalechina/hello-agents) — BabyAGI → AutoGPT → Claude Code 演化脈絡 +4. [**3Blue1Brown 中文配音版**](https://www.youtube.com/@3Blue1BrownCN) — LLM / Transformer 視覺化解說（中文配音） + +**🇺🇸 English**： +1. [**Andrej Karpathy — "Intro to Large Language Models"**](https://www.youtube.com/watch?v=zjkBMFhNj_g) ⭐⭐⭐（1hr）— LLM 從零開始 visual intro（ex-OpenAI / ex-Tesla AI Director、英文圈最重視的 LLM 入門影片） +2. [**Andrej Karpathy — "Let's build GPT from scratch"**](https://www.youtube.com/watch?v=kCc8FmEb1nY) ⭐⭐（2hr）— 想看 LLM 內部到程式碼級的人 +3. [**3Blue1Brown — "But what is a Transformer?"**](https://www.youtube.com/watch?v=wjZofJX0v4M) ⭐⭐⭐ — visual 解釋 LLM，英文圈最被推薦的視覺化教材 +4. [**Lilian Weng — "LLM Powered Autonomous Agents"**](https://lilianweng.github.io/posts/2023-06-23-agent/) ⭐⭐⭐ — canonical 1-page agent anatomy（Planning / Memory / Tool use / Action）、英文圈被引用最多的 agent 解剖文 +5. [**Anthropic — "Building Effective Agents"**](https://www.anthropic.com/research/building-effective-agents) ⭐ — Anthropic 觀點：何時該用 agent、何時 workflow 就夠 +6. [**Chip Huyen — "Agents"**](https://huyenchip.com/2025/01/07/agents.html) — practitioner 視角，full chapter 級深度 + +**選讀 / 進階補充**： +- [**Simon Willison — "I think 'agent' may finally have a widely enough agreed upon definition"**](https://simonwillison.net/2025/Sep/18/agents/) — working definition：「agent runs tools in a loop to achieve a goal」、含對照 OpenAI 等不同定義的爭議（**給已有基礎的人**） +- [**DeepLearning.AI Short Courses**](https://www.deeplearning.ai/short-courses/)：「AI Agents in LangGraph」/「Multi AI Agent Systems with crewAI」/「Functions, Tools and Agents with LangChain」（**API 多數是 2023-2024 舊版**、看概念為主、寫 code 對照官方最新 docs） +- [**microsoft/ai-agents-for-beginners**](https://github.com/microsoft/ai-agents-for-beginners) — 微軟官方 12 課 build-agent 入門（MIT、★ 64k+）。結構化、英文、含 code；適合想要一條**平行入門課對照**的人，不是本 stage 動手練習的替代 +- [**liyupi/ai-guide**](https://github.com/liyupi/ai-guide) — 中文圈最大 AI 資源**聚合**型 repo（不是原創教材、適合廣度延伸） + +> 📌 **資源清單上限規則**：本 section 是 router 不是 tutorial。主清單合計上限 **10 條**（中 4 + 英 6），要加新資源前**必須先移除一條**。選讀區不計入主清單上限。 + +> 💡 **推薦學習路徑**：先看 1-2 個影片（中：李宏毅、英：Karpathy / 3Blue1Brown）建立 visual mental model → 再讀 1-2 個 blog（Lilian Weng / Anthropic）拿到 working definition → 再回本 stage 動手練習。**不必全部看完**，這是 reference library 不是 reading list。 + +--- + +這是整個學習路線最關鍵的一站。**你建過一個 agent 才算真懂 agent**——本 stage 的基礎練習建議至少實際手寫一次、再依需求往 [hello-agents](https://github.com/datawhalechina/hello-agents) 或本 stage 精選 projects 找深度教材。 + +## 📌 學習目標 + +完成這個 stage 後你會： +- 講得出為什麼 LLM 需要 tools（它不是萬能的，而且文字以外的事它都做不了） +- 定義一個 tool schema，並讓 LLM 呼叫它 +- 從零（不靠任何 framework）寫出一個單步 ReAct agent +- 寫出多步 ReAct agent，並讓它自己判斷何時該停 +- 分得出哪種問題該用 tool use、哪種純 prompt 就夠 + +## 🚪 進入條件 + +你應該已經： +- 有可以跑的 Claude / OpenAI / Gemini API 權限（Stage 1） +- 對 prompt engineering 基礎已經上手（Stage 2） +- 能寫一個吃 JSON 進、吐 JSON 出的 Python 函式 + +## 📚 必修閱讀 + +1. [**Anthropic — Tool Use**](https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/overview) — 官方指南 +2. [**anthropics/courses — Tool Use**](https://github.com/anthropics/courses) ⭐⭐⭐⭐⭐ ★ 21k+ — Anthropic 官方 5 course umbrella、**module 5「Tool Use」對應本 stage**。Jupyter notebook 互動式練習、含 multimodal prompts / streaming / tool 實作 walk-through +3. [**ReAct: Synergizing Reasoning and Acting in Language Models**](https://arxiv.org/abs/2210.03629) — Yao et al. 2022，奠基論文。至少讀 abstract 跟 Section 3。 +4. [**OpenAI — Function Calling**](https://platform.openai.com/docs/guides/function-calling) — function calling 格式參考 +5. [**Build an agent from scratch**](https://shafiqulai.github.io/blogs/blog_3.html) — 從零打造 agent 的故事式導覽 + +## 🛠 動手練習（基礎 illustrative 練習） + +> 🦙 **本 stage 預設用 Ollama qwen2.5:3b**（成本考量、tool-use 支援穩定）。Stage 3 進到 tool calling / ReAct loop、`gemma4:e4b` 不夠、改用 `qwen2.5:3b`（1.9 GB、`ollama pull qwen2.5:3b` 即裝）。每個練習都有 Path A（Ollama、預設）+ Path B（Anthropic、選擇性、想看 cloud 高品質 tool-use 時用）。 +> +> 💰 **Stage 3 預算估算**（全 6 練習、tool use 較重）：**全本機 = $0**、**全 haiku ≈ $0.50**、**全 sonnet ≈ $1.50**。ReAct loop 練習單次 4-6 tool calls × 5 練習 × 5 reps ≈ $0.80 haiku。完整預算見 [`examples/README.md#推薦-llm-清單`](../examples/README.md#推薦-llm-清單)。 +> +> 完整 3 路 trade-off 見 [`examples/README.md`](../examples/README.md#三條路徑--預設用-ollama成本考量)。 +> +> 🆘 **卡住了？** Tool calling 是整個 curriculum 最陡的學習曲線。裝 [`examples/stage-5/tool-calling-tutor/`](../examples/stage-5/tool-calling-tutor/) skill——當你 prompt Claude Code「為什麼 LLM 不呼叫我的 tool」、「我這 schema 哪裡寫壞」會自動載入、走 4-symptom 診斷流程。 +> +> 🪜 **本 stage 是 single-agent 起點**：一個 LLM + ReAct loop。**Multi-agent 概念**（多個 agent 協作）入門看 [Stage 4 什麼是 multi-agent framework](04-agent-frameworks.md#-什麼是-multi-agent-framework)、**Claude 原生 subagent 機制**（`.claude/agents/` + Task tool、不需 framework）看 [Stage 5.5](05-claude-code-ecosystem.md#55--subagentsclaude-code-原生-multi-agent-機制-2025-新功能)。 + +### ⚠️ 先懂風險：給 agent 工具 = 給它一個攻擊面 + +把工具交給 LLM 的那一刻，你也給了它一個攻擊面。最清楚的框架是 Simon Willison 的 **lethal trifecta（致命三角）**：當一個 agent 同時具備這三件事，就可能被攻擊者操控去偷資料再外傳—— + +1. **能存取私密資料**（你的檔案 / DB / API key） +2. **會接觸不可信內容**（網頁、Email、別人傳的文件，裡面可能藏指令） +3. **能對外送東西**（發 request、寄信、寫檔） + +根因是 LLM 會「照內容裡的指令做」，分不清哪些是你下的、哪些是不可信資料裡夾帶的——這就是 **prompt injection**。本階段先建立這個意識就好，具體防法（隔離不可信輸入、權限 gate、最小工具集、高風險動作要人審）在 [Stage 8](08-agent-interfaces.md) 與 [Stage 5](05-claude-code-ecosystem.md) 細談。詞表：[prompt injection / lethal trifecta](../resources/glossary.md)。 + +--- + +### 練習 1：Function Calling（一個工具、一次呼叫） +給 Claude 一個工具（假的天氣 API）跟一個問題（「台北現在有下雨嗎？」）。看 Claude 怎麼呼叫工具、拿到結果、再回答你。 + +
+📋 起手碼 — Path A（本機 Ollama qwen2.5:3b、預設）（複製到 practice_1.py） + +```python +# 需要：pip install openai +# 前置：ollama pull qwen2.5:3b && ollama serve +# Note: Stage 3+ 用 qwen2.5:3b（tool-use 穩定）、不是 gemma4:e4b +import sys, json +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +# Step 1: 定義 tool schema — OpenAI-compatible 格式包一層 {"type":"function", "function":{...}} +weather_tool = { + "type": "function", + "function": { + "name": "get_weather", + "description": "查詢城市目前天氣（晴/雨/陰），回傳一個短字串。", + "parameters": { + "type": "object", + "properties": { + "city": {"type": "string", "description": "城市名稱（如「台北」）"}, + }, + "required": ["city"], + }, + }, +} + +# Step 2: 問問題、讓 LLM 自己決定要不要呼叫 tool +resp = client.chat.completions.create( + model="qwen2.5:3b", + max_tokens=512, + tools=[weather_tool], + messages=[{"role": "user", "content": "台北現在有下雨嗎？"}], +) + +# === 自我驗證 === +msg = resp.choices[0].message +print("finish_reason:", resp.choices[0].finish_reason) +print("tool_calls:", msg.tool_calls) + +assert msg.tool_calls, "預期 LLM 會選擇呼叫 tool（而非直接回答）" +tc = msg.tool_calls[0] +assert tc.function.name == "get_weather", f"預期呼叫 get_weather、實際 {tc.function.name}" +args = json.loads(tc.function.arguments) +assert args.get("city"), "預期 city 參數有值" +print(f"✅ 練習 1 通過 — qwen2.5:3b 正確選了 get_weather、帶 city='{args['city']}' 參數") +``` + +**預期輸出**（樣本）： +``` +finish_reason: tool_calls +tool_calls: [ChatCompletionMessageToolCall(id='call_xxx', function=Function(name='get_weather', arguments='{"city": "台北"}'), type='function')] +✅ 練習 1 通過 — qwen2.5:3b 正確選了 get_weather、帶 city='台北' 參數 +``` + +**沒裝 Ollama 也能驗邏輯**：用 `unittest.mock.MagicMock` 取代 client、塞固定 response、assert 一樣 work。完整 mock 範例見 [`examples/stage-3/03-react-from-scratch/test.py`](../examples/stage-3/03-react-from-scratch/test.py)（pattern 跨 backend 通用）。 + +
+ +
+📋 起手碼 — Path B（Anthropic API、選擇性）（複製到 practice_1_anthropic.py） + +```python +# 需要：pip install anthropic +# 環境變數：export ANTHROPIC_API_KEY=sk-ant-... +import anthropic + +client = anthropic.Anthropic() + +# Anthropic native tool schema — 不用包 wrapper +weather_tool = { + "name": "get_weather", + "description": "查詢城市目前天氣（晴/雨/陰），回傳一個短字串。", + "input_schema": { + "type": "object", + "properties": { + "city": {"type": "string", "description": "城市名稱（如「台北」）"}, + }, + "required": ["city"], + }, +} + +resp = client.messages.create( + model="claude-haiku-4-5", + max_tokens=512, + tools=[weather_tool], + messages=[{"role": "user", "content": "台北現在有下雨嗎？"}], +) + +# === 自我驗證 === +assert resp.stop_reason == "tool_use", f"非預期 stop_reason: {resp.stop_reason}" +tool_calls = [b for b in resp.content if b.type == "tool_use"] +assert tool_calls[0].name == "get_weather" +assert tool_calls[0].input.get("city") +print(f"✅ 練習 1 通過（Anthropic）— Claude 選了 get_weather、city='{tool_calls[0].input['city']}'") +``` + +**3 個關鍵 SDK 差異**： +- **Schema wrap**：Anthropic 直接 `tools=[{name, description, input_schema}]`；OpenAI/Ollama 要包 `[{"type":"function", "function":{...}}]` +- **Response 路徑**：Anthropic 從 `resp.content[i].type=="tool_use"` 抓；OpenAI/Ollama 從 `resp.choices[0].message.tool_calls[i]` +- **Args 格式**：Anthropic `.input` 是 dict（自動 parse）；OpenAI/Ollama `.function.arguments` 是 JSON string，要 `json.loads(...)` + +**成本**：1 次 ≈ $0.001。**Claude 的 tool-use 比 qwen2.5:3b 更穩**——複雜場景（5+ tools、模糊問題）gap 會明顯。 + +
+ +### 練習 2：多工具選擇 +給 Claude 三個工具（搜尋、計算機、行事曆）跟一個任務。看 Claude 怎麼挑工具，順便注意它什麼時候會挑錯。 + +
+📋 簡化版核心觀念 — Path A (Ollama) + +**NEW vs 練習 1**：tools 從 1 個變 3 個。LLM 看 `description` 邊界決定挑哪個——`description` 寫得越像「給人讀的 docstring」、越容易挑錯。 + +```python +from openai import OpenAI +import json + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +TOOLS = [ + {"type": "function", "function": {"name": "web_search", + "description": "Search current or external info not in the prompt.", + "parameters": {"type": "object", "properties": {"query": {"type": "string"}}, "required": ["query"]}}}, + {"type": "function", "function": {"name": "calculator", + "description": "Evaluate basic arithmetic with +, -, *, /, parentheses.", + "parameters": {"type": "object", "properties": {"expression": {"type": "string"}}, "required": ["expression"]}}}, + {"type": "function", "function": {"name": "calendar_lookup", + "description": "Look up events for a specific date.", + "parameters": {"type": "object", "properties": {"date": {"type": "string"}}, "required": ["date"]}}}, +] + +resp = client.chat.completions.create(model="qwen2.5:3b", tools=TOOLS, + messages=[{"role": "user", "content": "What is (19 * 42) - 8?"}]) + +tc = resp.choices[0].message.tool_calls[0] +print(f"LLM 挑了: {tc.function.name}, args: {json.loads(tc.function.arguments)}") +# 預期：calculator, {"expression": "(19 * 42) - 8"} +``` + +**punchline**：3 個 tool 的 `description` 邊界要互斥——`calendar` 寫「行事曆」太籠統、會跟 `web_search` 撞；寫「特定日期事件」就清楚。小 model 對 description 質量比 Claude 更敏感。 + +**Path B (Anthropic) 改 3 行**：schema 拿掉 `{"type": "function", "function": {...}}` 外包、`tool_calls` 變 `[b for b in resp.content if b.type == "tool_use"]`、`tc.input` 已經是 dict 不用 `json.loads`。完整版見 folder。 + +
+ +→ **基礎 starter 範本** → [`examples/stage-3/02-multi-tool-selection/`](../examples/stage-3/02-multi-tool-selection/)（starter.py 含 stub + 簡單 test，illustrative，**不是 chapter-length 完整教學**；深度章節見 stage 開頭 📚 hello-agents callout） + +### 結構化輸出（Structured Outputs / JSON mode）⭐ function calling 的孿生兄弟 + +function calling 是「**讓模型決定要不要動手**」；**結構化輸出是「強制模型回傳一個固定形狀的 JSON」**——兩者常搞混，但用途不同：前者讓 agent 採取行動，後者讓你拿到可程式解析的資料（填表、分類、抽取、eval 評分）。 + +**三種做法（由弱到強）**： +1. **prompt 要求 JSON**——最簡單、但模型有時會多嘴或格式跑掉。 +2. **JSON mode / `response_format`**——API 保證回合法 JSON（但不保證符合你的 schema）。 +3. **JSON-schema 強制 / constrained decoding**——連 schema 都鎖死，回來的一定符合（最可靠）。 + +> 💡 為什麼重要：agent 的 state、tool 參數、eval 評分全都依賴「拿得到結構化資料」。這是 tool calling 底下那層 load-bearing 的可靠度基礎。 + +**動手工具**：[jxnl/instructor](https://github.com/jxnl/instructor)（★13k，把 Pydantic model 當 schema、自動 retry）、[dottxt-ai/outlines](https://github.com/dottxt-ai/outlines)（★14k，constrained decoding、連本機 LLM 都能鎖 schema）；Stage 4 的 Pydantic AI 也是同路線。 + +### 練習 3：從零實作 ReAct（不用 framework） +用 50-80 行 Python 把 Thought → Action → Observation 迴圈寫出來。不要 LangChain、不要 LangGraph，就是純 `while not done: thought; action; observation; ...`。 + +
+📋 簡化版核心觀念 — Path A (Ollama)、ReAct loop 的全部就在這 13 行 + +**NEW vs 練習 2**：把單次 call 包進迴圈、`messages` 一直長、看 `tool_calls` 在不在來決定收尾。 + +```python +# 假設 TOOLS + TOOL_IMPL（dict: name → callable）已經像練習 2 一樣定義好 +messages = [{"role": "user", "content": "台北人口除以紐約人口？"}] + +for step in range(5): # max_iter safety net + r = client.chat.completions.create(model="qwen2.5:3b", tools=TOOLS, messages=messages) + msg = r.choices[0].message + # 把 assistant response 接回 messages（重要！下輪 LLM 才看得到自己上輪講什麼） + messages.append({"role": "assistant", "content": msg.content, "tool_calls": msg.tool_calls}) + if not msg.tool_calls: + print(f"✅ 收尾：{msg.content}"); break + for tc in msg.tool_calls: + args = json.loads(tc.function.arguments) + obs = TOOL_IMPL[tc.function.name](args) # 本地執行 + # observation 接回 messages（用 role="tool"、配 tool_call_id） + messages.append({"role": "tool", "tool_call_id": tc.id, "content": obs}) +``` + +**3 個容易踩坑**： +1. **忘記把 assistant response 加回 messages**——下輪 LLM 看不到自己上輪講什麼、會 loop forever +2. **`tool` message 沒帶 `tool_call_id`**——LLM 無法配對哪個 result 對應哪個 call +3. **沒 `max_iter`**——tool 結果寫不好時、LLM 會無限呼叫，safety net 必須設 + +**Path B (Anthropic) 差幾行**：迴圈架構一模一樣、`msg.tool_calls` 變 `[b for b in resp.content if b.type == "tool_use"]`、用 `stop_reason == "end_turn"` 判停、tool result 包成 `{"type": "tool_result", "tool_use_id": ..., "content": obs}` 放進 user message。完整版見 folder。 + +
+ +→ **基礎 starter 範本** → [`examples/stage-3/03-react-from-scratch/`](../examples/stage-3/03-react-from-scratch/)（含 mock-based test.py、不花 API 錢也能驗；illustrative，**不是 chapter-length 完整教學**——深度章節見 stage 開頭 📚 hello-agents callout） + +### 練習 4：多步驟推理任務 +一個需要連續呼叫 3-5 次 tool 的任務。例如：「找出台北人口，除以紐約人口，再把比例換成百分比。」每一步用不同的工具。 + +
+📋 簡化版核心觀念 — 跟練習 3 同一個 loop、跑久一點而已 + +**NEW vs 練習 3**：**完全同一個 loop**——只是 `TOOLS` 換成 4 個（`lookup_population` / `divide` / `to_percentage` / `round_int`）、題目自然走完 4 輪 tool call 才收尾。 + +```python +# 沒有新 code、純粹是 TOOLS / TOOL_IMPL 換內容 +TOOL_IMPL = { + "lookup_population": lambda i: lookup_population(i["city"]), + "divide": lambda i: divide(i["a"], i["b"]), + "to_percentage": lambda i: to_percentage(i["ratio"]), + "round_int": lambda i: round_int(i["x"]), +} +# loop 完全照 練習 3，只是 max_iter 拉大到 8 +``` + +**punchline**：多步推理不是新 pattern、是**讓 ReAct loop 跑久一點**。**真正的挑戰是「LLM 會不會中間漏一步」**——qwen2.5:3b 可能漏「轉百分比」、Claude haiku 較穩。**這恰好是觀察「model 規模 vs 多步穩定度」的好實驗**。試試 `MODEL=qwen2.5:7b python starter.py` 對照。 + +
+ +→ **基礎 starter 範本** → [`examples/stage-3/04-multi-step-reasoning/`](../examples/stage-3/04-multi-step-reasoning/)（starter.py 含 stub + 簡單 test，illustrative，**不是 chapter-length 完整教學**；深度章節見 stage 開頭 📚 hello-agents callout） + +### 練習 5：錯誤處理 +讓某個工具失敗（網路錯誤、輸入無效）。看看 agent 會怎麼處理錯誤、能不能恢復，再加上 retry 機制。 + +
+📋 簡化版核心觀念 — tool error 是 data、不是 exception + +**NEW vs 練習 4**：tool error 回傳**結構化 dict**、不要 `raise`。loop 把 dict 接回 LLM、模型自己決定 retry / 改 query / 放棄。 + +```python +def fetch_weather(city: str) -> dict: + if network_failed(): + return {"error": "network timeout", "retry_hint": "try again in 1s"} + return {"city": city, "forecast": "rain", "temperature_c": 24} + +# loop 裡： +obs = fetch_weather(args["city"]) +messages.append({"role": "tool", "tool_call_id": tc.id, + "content": json.dumps(obs, ensure_ascii=False)}) # error dict 也是 string 化接回去 +# 下一輪 LLM 看到 retry_hint、可能會 retry、可能會放棄、可能會改 query +``` + +**為什麼不 `raise`**：`raise` 直接中斷 loop、LLM 沒機會 recover。**Production 的 retry 不在 Python 層、而在 LLM 層**——這個 mental flip 是 Stage 3 練習 5 的核心。 + +**Bad vs Good error 回傳**： + +| Bad | Good | +|---|---| +| `raise Exception("failed")` | `return {"error": "network timeout", "retry_hint": "try again in 1s"}` | +| `return "failed"` | `return {"error": "...", "category": "transient", "retry_hint": "..."}` | +| 無限 retry | `max_iter` safety + 業務層 retry quota | + +**小 model 觀察**：qwen2.5:3b 對 `retry_hint` follow-up 較弱、可能直接放棄；Claude haiku 較穩。完整版（含連續失敗 graceful end 範例）見 folder。 + +
+ +→ **基礎 starter 範本** → [`examples/stage-3/05-error-handling/`](../examples/stage-3/05-error-handling/)（starter.py 含 stub + 簡單 test，illustrative，**不是 chapter-length 完整教學**；深度章節見 stage 開頭 📚 hello-agents callout） + +### 練習 6：Function schema 設計（壞 schema 修到好） +**先給 LLM 一份故意寫爛的 schema**——`description` 模糊（「處理資料」）、參數全用 `type: string`、沒分 required / optional、enum 該用沒用。觀察 LLM 怎麼選錯 tool、傳錯參數。然後逐項修： +- description 寫到 LLM 一眼就懂這個 tool 適用情境（不是寫給人讀的 docstring） +- parameters 用對 type（number / boolean / enum / array），required 列清楚 +- 模糊邊界用 enum 強制收斂（例如 `unit: "celsius" | "fahrenheit"` 而不是 `unit: string`） +- error 回傳要包 `{"error": "...", "retry_hint": "..."}` 讓 LLM 能恢復 + +> 💡 詳細 cheatsheet 看 [`resources/schema-design-cheatsheet.md`](../resources/schema-design-cheatsheet.md)——5 條黃金規則 + 5 個常見 anti-pattern。 + +
+📋 簡化版核心觀念 — bad vs good schema 對照 + +**NEW vs 練習 5**：同一個工具（溫度轉換）、兩種 schema 寫法。看 4 個差別。 + +```python +# ❌ BAD — qwen2.5:3b 幾乎必錯（Claude haiku 還能猜對、但機率明顯下降） +{"name": "convert", "description": "Convert a value.", + "parameters": {"type": "object", "properties": { + "value": {"type": "string"}, "unit": {"type": "string"}}}} + +# ✅ GOOD — qwen 也能穩定挑對 +{"name": "convert_temperature", + "description": "Use when user asks to convert temperatures between Fahrenheit and Celsius.", + "parameters": {"type": "object", "properties": { + "value": {"type": "number", "description": "Temperature value"}, + "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, + "required": ["value", "unit"]}} +``` + +**4 個改進**：(1) `name` 改具體、(2) `description` 寫「**何時**用」而非「**做什麼**」、(3) `type` 改 `number`、(4) 加 `required` + `enum`。 + +**punchline**：**寫 schema 的功夫能省下換大 model 的成本**——小 model 對 schema 質量比大 model 敏感，相同 bad schema 在 Claude 上可能還能猜對、在 qwen 上幾乎必錯。Production 想用便宜 model？schema 必須寫到能上線跑的程度。 + +**搞不定 schema 怎麼辦**：裝 [`examples/stage-5/tool-calling-tutor/`](../examples/stage-5/tool-calling-tutor/) skill，遇到「LLM 不呼叫我的 tool」、「我這 schema 哪裡寫壞」會自動跳出來幫你 debug。 + +
+ +→ **基礎 starter 範本** → [`examples/stage-3/06-schema-design/`](../examples/stage-3/06-schema-design/)（含 bad schema vs good schema 兩個版本對照；illustrative，**不是 chapter-length 完整教學**——深度章節見 stage 開頭 📚 hello-agents callout） + +> 💡 **手寫 schema 之後，認識 MCP**：你上面手寫的 tool schema，真實世界已經有標準——**MCP（Model Context Protocol）** 把「工具長什麼樣、怎麼呼叫」標準化成跨 agent 可重用的協定：寫一次，任何支援 MCP 的 agent（Claude Code / Cursor / …）都能用。這裡先記得這個名字，[Stage 5.2](05-claude-code-ecosystem.md#52--mcpmodel-context-protocol-基礎) 細講。 + +## 🪞 反思（Reflexion / Self-Refine）— 概念 + 路由 + +> **本節是 concept + routing、不是練習**。沒有 verified working solution、不掛「練習 N」label、不給 success criteria——遵守本 repo「沒驗證答案不寫練習、頂多 routing」原則。想動手做？直接讀下方 paper / project。 + +**反思是什麼**：練習 5 的 error handling 是「LLM 出錯 → 你（外部）catch + retry」；**反思**是「LLM 觀察自己出錯 → 自己改」。差別是 agency 在哪一邊——這是 production agent（Cursor / Cline / Claude Code）每天都在跑的迴圈。 + +**為什麼這節在 Stage 3 而不是 Stage 6**：反思在學術（Reflexion paper Shinn 2023、Self-Refine Madaan 2023）跟 production（Cursor / Claude Code）上都被歸類在 **planning / reasoning loop** 機制——是 ReAct（練習 3）的 sibling pattern，**不是 memory pattern**。同樣是 LLM 自我引導的多輪迴圈，只是「下一輪要做什麼」從「呼叫 tool」換成「批改自己」。 + +**進階版（帶 persistent memory 的 Reflexion 完整版）→ [Stage 6 進階：Reflexion with Memory](06-memory-rag.md#-進階帶持久記憶的-reflexion-完整版--track-b-選讀)**——當反思要跨 session、把過去失敗存起來當下一輪 context，這個版本才真的需要 memory 層。 + +### 一張對照圖 + +| Pattern | 形式 | 需要 memory？ | 在哪學 | +|---|---|---|---| +| **Error handling**（練習 5） | 外部 catch + retry | 不需 | **本 stage 練習 5** | +| **ReAct loop**（練習 3） | LLM → tool → 結果 → LLM | 不需 | **本 stage 練習 3** | +| **基本反思 / Self-Refine** | Actor → Critic → Actor，single session | 不需 | **本節 routing（下方）** | +| **完整 Reflexion**（含 episodic memory） | 上面 + 把失敗反思存起來、跨 session 累積 | **需要** | **Stage 6 進階：Reflexion with Memory** | + +### 📚 想動手 / 想深入？直接讀這些 + +**Paper**： +- [**Reflexion (Shinn et al. 2023)**](https://arxiv.org/abs/2303.11366) ⭐ — 原 paper，定義「verbal reinforcement learning」 +- [**Self-Refine (Madaan et al. 2023)**](https://arxiv.org/abs/2303.17651) — single-agent self-critique，是「基本反思」的學術定義 + +**Reference 實作**： +- [**arunpshankar/react-from-scratch**](https://github.com/arunpshankar/react-from-scratch) — 已在本 stage 精選 Projects 列出，含 Reflection 實作可直接讀 +- [**LangChain — Reflection Agents（blog）**](https://blog.langchain.dev/reflection-agents/) — framework 實作參考 + 完整 working notebook +- [**datawhalechina/hello-agents**](https://github.com/datawhalechina/hello-agents) — 對應章節（自我反思 / Self-Refine 段落、中文完整教學） + +> 💡 **想看反思怎麼長進 production agent**：[Stage 5 5.7 Harness Internals](05-claude-code-ecosystem.md#57--claude-code-source-解剖reference-harness-implementation-track-b-必看) 解剖 Claude Code source 時可以看到——agent 跑完 tool call 後自我評估 patch、有問題回頭改、修正後再 commit。**這是現代 production agent 的核心 building block 之一**。 + +## 🎯 精選 Projects + +按用途分 4 類、12 個項目一張表搞定。**挑入口看「適合誰」、想深入點連結看 repo README**。 + +| 分類 | Project | ⭐ | 適合誰 | 為什麼推薦 / 備註 | +|---|---|---|---|---| +| **官方 cookbook**
（先看這個） | [Anthropic — Tool Use Cookbook](https://github.com/anthropics/claude-cookbooks/tree/main/tool_use) | ⭐⭐⭐⭐⭐ | 練習 1 / 練習 2 入手 | 單工具 → 多工具 → parallel → structured output 全部 notebook（重點看 `tool_use/customer_service_agent.ipynb`） | +| | [Anthropic — Quickstarts](https://github.com/anthropics/claude-quickstarts) | ⭐⭐⭐⭐⭐ | 練習 1/2 跑完想看「真的應用長什麼樣」 | 3 個 deploy-ready 範本（financial / customer-support / computer-use）、★ 16k+。比社群實作更 canonical | +| | [Anthropic — Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents) | ⭐⭐⭐⭐⭐ | 練習 3 寫完、進 Stage 4 之前**必讀** | 部落格文章：何時用 agent vs workflow / 常見 pattern / 容易踩的坑——Anthropic 官方觀念框架 | +| **從零實作 ReAct**
（理解原理） | [pguso/ai-agents-from-scratch](https://github.com/pguso/ai-agents-from-scratch) | ⭐⭐⭐⭐⭐ | 練習 3（從零寫 ReAct） | 用本機 Ollama 從零打造、zero framework、章節結構好。**最乾淨的「不靠 framework」參考實作** | +| | [arunpshankar/react-from-scratch](https://github.com/arunpshankar/react-from-scratch) | ⭐⭐⭐⭐ | 練習 3 替代（偏好 Gemini）+ 想看反思變體 | ReAct + Reflection + Self-consistency、Gemini 最佳化（⚠️ 2025-05 後更新放緩、Apache-2.0） | +| | [mattambrogi/agent-implementation](https://github.com/mattambrogi/agent-implementation) | ⭐⭐⭐ | 練習 3 卡住時逐行對照 | ~150 行最精簡 ReAct（⚠️ 已停滯 2024-01、留作教學玩具參考） | +| | [lsdefine/GenericAgent](https://github.com/lsdefine/GenericAgent) | ⭐⭐⭐⭐ | 練習 3/4，想看「精簡但完整」framework | 自我演化 framework、~3K 行、★ 13k+、支援 Claude / Gemini / Kimi / MiniMax。介於玩具版與 LangGraph 之間 | +| **CodeAct 路線**
（agent 寫程式碼當 action） | [HuggingFace Smolagents](https://github.com/huggingface/smolagents) | ⭐⭐⭐⭐ | 練習 5 替代方案、本地 LLM 實驗 | ≤1000 LOC、CodeAct pattern 代表、★ 27k+。HF 立場：agent 應該要小 | +| | [QuantaLogic/quantalogic](https://github.com/quantalogic/quantalogic) | ⭐⭐⭐ | 練習 3 後想比較 CodeAct vs JSON-tool | 另一條 CodeAct 路線、agent 直接寫 Python 程式碼當 action、Apache-2.0 | +| **中文章節式深度教材**
（chapter-length） | [datawhalechina/hello-agents](https://github.com/datawhalechina/hello-agents) ⭐ **本 stage 推薦** | ⭐⭐⭐⭐⭐ | 中文讀者想要結構化教學 + 完整覆蓋 | **16 種能力**含 tool use / ReAct / context engineering / sub-agents / circuit breaker / observability。中文圈最完整章節式（CC BY-NC-SA、非商用） | +| | [HelloAgents (jjyaoao)](https://github.com/jjyaoao/HelloAgents) | ⭐⭐⭐⭐⭐ | 中文讀者、想跑上面教材的 code | 上面教材 code repo、**請切 `learn_version` 分支**對齊章節（`pip install hello-agents`、CC BY-NC-SA） | +| **Framework 對照**
（看 framework 怎麼藏掉 ReAct loop） | [LangChain — ReAct Agent Template](https://github.com/langchain-ai/react-agent) | ⭐⭐⭐ | 練習 3 自己寫完後再來 | LangGraph Studio 範本、framework 怎麼把 ReAct 抽象化 | + +> 💡 **建議閱讀路徑**：練習 1-2 跑 Anthropic Cookbook → 練習 3 跑 pguso/ai-agents-from-scratch → 練習 3 後讀 Building Effective Agents → 中文章節式教材就 hello-agents + jjyaoao 配對 → 進 Stage 4 前看 LangChain ReAct template 對照 framework 抽象。 + + +## ✅ 進 Stage 4 前的自我檢查 + +你能不能： + +- [ ] 定義一個 tool schema（name + description + JSON schema 輸入/輸出） +- [ ] 用不到 100 行 Python、不靠任何 framework，把 ReAct 迴圈寫出來 +- [ ] 解釋為什麼 agent 需要一個「我做完了」的退出條件 +- [ ] 比較 CodeAct（程式碼即 action）跟 JSON-tool 兩種路線 +- [ ] 看出哪些問題其實不需要 agent + +如果可以 → 進 [Stage 4 — Agent Frameworks](04-agent-frameworks.md)。 + +如果不行 → 把 練習 3 再跑一次，不要跳過。如果你不懂 framework 在幫你抽象什麼，Stage 4 的那些東西看起來會像黑魔法。 diff --git a/stages/03-tool-use-and-hello-agent.zh-Hans.md b/stages/03-tool-use-and-hello-agent.zh-Hans.md new file mode 100644 index 0000000..231fcd4 --- /dev/null +++ b/stages/03-tool-use-and-hello-agent.zh-Hans.md @@ -0,0 +1,515 @@ +# Stage 3 — 工具使用与第一个 Agent（Tool Use & Hello Agent）⭐ + +> [繁體中文](./03-tool-use-and-hello-agent.md) | **简体中文** | [English](./03-tool-use-and-hello-agent.en.md) + +⏱ **时间估算**：2-3 周（约 10-20 小时） + +> 💡 用语密集（agent / tool use / function calling / ReAct / structured output⋯）→ 翻 [`resources/glossary.md` 2](../resources/glossary.md#2-agent--工具使用)。 +> 🗺️ **进 Track A（CLI Power User）还是 Track B（Agent Builder）前**，先看 [`resources/agent-paradigms.md`](../resources/agent-paradigms.md) — 5 种 agent 型态的全景图，帮你选轨。 + +> 📋 **本章组成**：〔开场框景：AI/LLM/Agent 三者关系〕→ 学习目标 → 进入条件 → 必修阅读 →〔可选 · 概念地图〕→ 动手练习 → 反思（概念 + 路由）→ 精选 Projects → 自我检查 +> 🔑 **关键名词**：见 [`resources/glossary.md` 2](../resources/glossary.md#2-agent--工具使用) + +## 🤖 开始前：AI / LLM / Agent — 三者怎么分？ + +> **本节是“开场框景”（由大到小 pedagogy）**：先把学习者脑中的 mental hierarchy 建好，再进 学习目标、练习。这节只做**简短说明 + 对照**，深度入门读物已经是中英文圈各自的 canonical reference（即“最公认的标准参考”、见下方资源）。**不是重写 hello-agents Ch1。** + +### 一张阶层图先建立认知 + +![AI / ML / DL / LLM 与 Agent 的关系](../resources/diagrams/ai-ml-llm-agent-hierarchy.zh-Hans.png) + +→ **“Agent”不是“比 LLM 更厉害的模型”，也不是 LLM 树状分类底下的一个分支**。Agent 是个**跨层抽象的系统**，把 LLM 当作其中一个组件来用。Cursor / Claude Code / Hermes Agent 内部都还是同一批 LLM（Claude / GPT / Gemini）—— 差别是怎么把 LLM 包进工具调用循环里。 + +### 三行对照（最快版） + +| 词 | 是什么 | 你给它什么、它回什么 | 例子 | +|---|---|---|---| +| **AI** | 整个学科 | 太抽象、不能直接“用” | ML、DL、LLM、RL 都是 AI 子领域 | +| **LLM** | 把文字映射到文字的单一模型 | 给 prompt → 回字 | GPT-5、Claude、Llama 3、Qwen | +| **Agent** | LLM + 工具 + loop 的**系统** | 给任务 → 自己跑多步骤达成 | Cursor、Claude Code、Hermes Agent | + +**一句话**：LLM 像一个理解并生成文字的大脑；Agent 则是把这个大脑接上工具、工作流与反馈回路后，能够作为系统完成多步骤任务的东西。 + +### Agent 的 3 个**最小必要**部件（这就是 agent vs LLM 的核心差别） + +| 部件 | 角色 | 在哪学 | +|---|---|---| +| 🧠 **LLM**（brain） | 推理 / 决策 / 自然语言 | Stage 1 已学 | +| 🔧 **Tools**（hands） | 对世界做事（call API、跑 code、查数据） | **本 stage** | +| 🔁 **Loop**（heartbeat） | 想 → 做 → 看结果 → 再想（ReAct） | **本 stage 练习 3** | + +→ **这 3 个合在一起就是 agent 的最低定义**。没有 tools / loop，那只是“LLM + 你写 retry”，不算 agent。 + +### Agent 的经典范式（thinking patterns） + +学完最小 3 部件后、下一层是“**LLM 怎么想**”。hello-agents Ch4“智能体经典范式构建”整章在讲这个。简短对照： + +| 范式 | 是什么 | 在哪学 | +|---|---|---| +| **CoT**（Chain-of-Thought、思维链） | LLM 写出推理过程再给答案、不只给结论——是个 **prompting 技巧**、不是 agent 结构 | **Stage 2** 学习目标 + 动手练习（推理任务 CoT） | +| **ReAct**（Reasoning + Acting） | 在 Loop 里套 CoT：Thought（想）→ Action（调用 tool）→ Observation（看结果）→ Thought ...，是 **Loop 部件最常见的实现** | **本 stage 练习 3** + [ReAct paper (Yao 2022)](https://arxiv.org/abs/2210.03629) | +| **Reflection** | 跑完一轮后让 LLM 批改自己、根据 feedback 重答 | **本 stage 反思**（concept + 路由） | +| **Planning**（任务分解） | 把大任务拆成子任务、可分给多个 agent 各做 | **Stage 4** 什么是 multi-agent framework | + +→ 这些范式都是“**LLM 自我引导**”的不同变化、堆叠在 3 部件（LLM + Tools + Loop）之上。**“Agent 是什么”用 3 部件就讲完了；“Agent 怎么想”需要这 4 个范式才讲得完整**。 + +> 💡 **延伸组件**（agent 变强的 infrastructure、但**不是“是不是 agent”的判准**）： +> - **记忆 / RAG**（agent 能跨对话记住东西）→ **Stage 6** 完整教 +> - **反思 / self-critique**（agent 看自己答案、发现问题、回头改）→ 基本版见 **本 stage 反思**（concept + paper routing）；带持久 memory 的进阶版见 **Stage 6 Reflexion with Memory** +> - **Production harness**（telemetry / safety / retry / orchestration）→ **Stage 5 5.7** +> +> 这些都是 advanced pattern——Stage 3 教最小可行 agent、后面 stage 教怎么变强。 + +### 📚 深度入门资源（中英文 / 影片优先） + +**🀄 中文**： +1. [**李宏毅 — 生成式 AI 导论（2024 春台大课程）**](https://speech.ee.ntu.edu.tw/~hylee/genai/2024-spring.php) ⭐⭐⭐ — 中文圈最高质量的 AI / LLM / agent 学术级导论。每集 30-60 分钟、台大授课、官方页含投影片 + YouTube 链接。LLM / agent 概念都涵盖。最新整合版见 [**GenAI-ML 2025 秋**](https://speech.ee.ntu.edu.tw/~hylee/GenAI-ML/2025-fall.php)、YouTube 主频道 [**@HungyiLeeNTU**](https://www.youtube.com/@HungyiLeeNTU) +2. [**datawhalechina/hello-agents** Ch1“初识智能体”](https://github.com/datawhalechina/hello-agents) ⭐ — 文字版最完整中文 agent 导论 +3. [**datawhalechina/hello-agents** Ch2“智能体发展史”](https://github.com/datawhalechina/hello-agents) — BabyAGI → AutoGPT → Claude Code 演化脉络 +4. [**3Blue1Brown 中文配音版**](https://www.youtube.com/@3Blue1BrownCN) — LLM / Transformer 视觉化解说（中文配音） + +**🇺🇸 English**： +1. [**Andrej Karpathy — "Intro to Large Language Models"**](https://www.youtube.com/watch?v=zjkBMFhNj_g) ⭐⭐⭐（1hr）— LLM 从零开始 visual intro（ex-OpenAI / ex-Tesla AI Director、英文圈最重视的 LLM 入门影片） +2. [**Andrej Karpathy — "Let's build GPT from scratch"**](https://www.youtube.com/watch?v=kCc8FmEb1nY) ⭐⭐（2hr）— 想看 LLM 内部到代码级的人 +3. [**3Blue1Brown — "But what is a Transformer?"**](https://www.youtube.com/watch?v=wjZofJX0v4M) ⭐⭐⭐ — visual 解释 LLM，英文圈最被推荐的视觉化教材 +4. [**Lilian Weng — "LLM Powered Autonomous Agents"**](https://lilianweng.github.io/posts/2023-06-23-agent/) ⭐⭐⭐ — canonical 1-page agent anatomy（Planning / Memory / Tool use / Action）、英文圈被引用最多的 agent 解剖文 +5. [**Anthropic — "Building Effective Agents"**](https://www.anthropic.com/research/building-effective-agents) ⭐ — Anthropic 观点：何时该用 agent、何时 workflow 就够 +6. [**Chip Huyen — "Agents"**](https://huyenchip.com/2025/01/07/agents.html) — practitioner 视角，full chapter 级深度 + +**选读 / 进阶补充**： +- [**Simon Willison — "I think 'agent' may finally have a widely enough agreed upon definition"**](https://simonwillison.net/2025/Sep/18/agents/) — working definition：“agent runs tools in a loop to achieve a goal”、含对照 OpenAI 等不同定义的争议（**给已有基础的人**） +- [**DeepLearning.AI Short Courses**](https://www.deeplearning.ai/short-courses/)：“AI Agents in LangGraph”/“Multi AI Agent Systems with crewAI”/“Functions, Tools and Agents with LangChain”（**API 多数是 2023-2024 旧版**、看概念为主、写 code 对照官方最新 docs） +- [**microsoft/ai-agents-for-beginners**](https://github.com/microsoft/ai-agents-for-beginners) — 微软官方 12 课 build-agent 入门（MIT、★ 64k+）。结构化、英文、含 code；适合想要一条**平行入门课对照**的人，不是本 stage 动手练习的替代 +- [**liyupi/ai-guide**](https://github.com/liyupi/ai-guide) — 中文圈最大 AI 资源**聚合**型 repo（不是原创教材、适合广度延伸） + +> 📌 **资源清单上限规则**：本 section 是 router 不是 tutorial。主清单合计上限 **10 条**（中 4 + 英 6），要加新资源前**必须先移除一条**。选读区不计入主清单上限。 + +> 💡 **推荐学习路径**：先看 1-2 个影片（中：李宏毅、英：Karpathy / 3Blue1Brown）建立 visual mental model → 再读 1-2 个 blog（Lilian Weng / Anthropic）拿到 working definition → 再回本 stage 动手练习。**不必全部看完**，这是 reference library 不是 reading list。 + +--- + +这是整个学习路线最关键的一站。**你建过一个 agent 才算真懂 agent**——本 stage 的基础练习建议至少实际手写一次、再依需求往 [hello-agents](https://github.com/datawhalechina/hello-agents) 或本 stage 精选 projects 找深度教材。 + +## 📌 学习目标 + +完成这个 stage 后你会： +- 讲得出为什么 LLM 需要 tools（它不是万能的，而且文字以外的事它都做不了） +- 定义一个 tool schema，并让 LLM 调用它 +- 从零（不靠任何 framework）写出一个单步 ReAct agent +- 写出多步 ReAct agent，并让它自己判断何时该停 +- 分得出哪种问题该用 tool use、哪种纯 prompt 就够 + +## 🚪 进入条件 + +你应该已经： +- 有可以跑的 Claude / OpenAI / Gemini API 权限（Stage 1） +- 对 prompt engineering 基础已经上手（Stage 2） +- 能写一个吃 JSON 进、吐 JSON 出的 Python 函数 + +## 📚 必修阅读 + +1. [**Anthropic — Tool Use**](https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/overview) — 官方指南 +2. [**anthropics/courses — Tool Use**](https://github.com/anthropics/courses) ⭐⭐⭐⭐⭐ ★ 21k+ — Anthropic 官方 5 course umbrella、**module 5“Tool Use”对应本 stage**。Jupyter notebook 互动式练习、含 multimodal prompts / streaming / tool 实作 walk-through。 +3. [**ReAct: Synergizing Reasoning and Acting in Language Models**](https://arxiv.org/abs/2210.03629) — Yao et al. 2022，奠基论文。至少读 abstract 跟 Section 3。 +4. [**OpenAI — Function Calling**](https://platform.openai.com/docs/guides/function-calling) — function calling 格式参考 +5. [**Build an agent from scratch**](https://shafiqulai.github.io/blogs/blog_3.html) — 从零打造 agent 的故事式导览 + +## 🛠 动手练习（基础 illustrative 练习） + +> 🦙 **本 stage 默认用 Ollama qwen2.5:3b**（成本考量、tool-use 支持稳定）。Stage 3 进到 tool calling / ReAct loop、`gemma4:e4b` 不够、改用 `qwen2.5:3b`（1.9 GB、`ollama pull qwen2.5:3b` 即装）。每个练习都有 Path A（Ollama、默认）+ Path B（Anthropic、选择性、想看 cloud 高质量 tool-use 时用）。 +> +> 💰 **Stage 3 预算估算**（全 6 练习、tool use 较重）：**全本机 = $0**、**全 haiku ≈ $0.50**、**全 sonnet ≈ $1.50**。ReAct loop 练习单次 4-6 tool calls × 5 练习 × 5 reps ≈ $0.80 haiku。完整预算见 [`examples/README.md#推荐-llm-清单`](../examples/README.zh-Hans.md#推荐-llm-清单)。 +> +> 完整 3 路 trade-off 见 [`examples/README.md`](../examples/README.zh-Hans.md#三条路径--默认用-ollama成本考量)。 +> +> 🆘 **卡住了？** Tool calling 是整个 curriculum 最陡的学习曲线。装 [`examples/stage-5/tool-calling-tutor/`](../examples/stage-5/tool-calling-tutor/) skill——当你 prompt Claude Code“为什么 LLM 不调用我的 tool”、“我这 schema 哪里写坏”会自动加载、走 4-symptom 诊断流程。 +> +> 🪜 **本 stage 是 single-agent 起点**：一个 LLM + ReAct loop。**Multi-agent 概念**（多个 agent 协作）入门看 [Stage 4 什么是 multi-agent framework](04-agent-frameworks.zh-Hans.md#-什么是-multi-agent-framework)、**Claude 原生 subagent 机制**（`.claude/agents/` + Task tool、不需 framework）看 [Stage 5.5](05-claude-code-ecosystem.zh-Hans.md#55--subagentsclaude-code-原生-multi-agent-机制-2025-新功能)。 + +### ⚠️ 先懂风险：给 agent 工具 = 给它一个攻击面 + +把工具交给 LLM 的那一刻，你也给了它一个攻击面。最清楚的框架是 Simon Willison 的 **lethal trifecta（致命三角）**：当一个 agent 同时具备这三件事，就可能被攻击者操控去偷数据再外传—— + +1. **能访问私密数据**（你的文件 / DB / API key） +2. **会接触不可信内容**（网页、Email、别人传的文档，里面可能藏指令） +3. **能对外发送东西**（发 request、寄信、写文件） + +根因是 LLM 会“照内容里的指令做”，分不清哪些是你下的、哪些是不可信数据里夹带的——这就是 **prompt injection**。本阶段先建立这个意识就好，具体防法（隔离不可信输入、权限 gate、最小工具集、高风险动作要人审）在 [Stage 8](08-agent-interfaces.zh-Hans.md) 与 [Stage 5](05-claude-code-ecosystem.zh-Hans.md) 细谈。词表：[prompt injection / lethal trifecta](../resources/glossary.zh-Hans.md)。 + +--- + +### 练习 1：Function Calling（一个工具、一次调用） +给 Claude 一个工具（假的天气 API）跟一个问题（“台北现在有下雨吗？”）。看 Claude 怎么调用工具、拿到结果、再回答你。 + +
+📋 起手码 — Path A（本机 Ollama qwen2.5:3b、默认）（复制到 practice_1.py） + +```python +# 需要：pip install openai +# 前置：ollama pull qwen2.5:3b && ollama serve +# Note: Stage 3+ 用 qwen2.5:3b（tool-use 稳定）、不是 gemma4:e4b +import sys, json +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +# Step 1: 定义 tool schema — OpenAI-compatible 格式包一层 {"type":"function", "function":{...}} +weather_tool = { + "type": "function", + "function": { + "name": "get_weather", + "description": "查询城市目前天气（晴/雨/阴），回传一个短字符串。", + "parameters": { + "type": "object", + "properties": { + "city": {"type": "string", "description": "城市名称（如「台北」）"}, + }, + "required": ["city"], + }, + }, +} + +# Step 2: 问问题、让 LLM 自己决定要不要调用 tool +resp = client.chat.completions.create( + model="qwen2.5:3b", + max_tokens=512, + tools=[weather_tool], + messages=[{"role": "user", "content": "台北现在有下雨吗？"}], +) + +# === 自我验证 === +msg = resp.choices[0].message +print("finish_reason:", resp.choices[0].finish_reason) +print("tool_calls:", msg.tool_calls) + +assert msg.tool_calls, "预期 LLM 会选择调用 tool（而非直接回答）" +tc = msg.tool_calls[0] +assert tc.function.name == "get_weather", f"预期调用 get_weather、实际 {tc.function.name}" +args = json.loads(tc.function.arguments) +assert args.get("city"), "预期 city 参数有值" +print(f"✅ 练习 1 通过 — qwen2.5:3b 正确选了 get_weather、带 city='{args['city']}' 参数") +``` + +**预期输出**（样本）： +``` +finish_reason: tool_calls +tool_calls: [ChatCompletionMessageToolCall(id='call_xxx', function=Function(name='get_weather', arguments='{"city": "台北"}'), type='function')] +✅ 练习 1 通过 — qwen2.5:3b 正确选了 get_weather、带 city='台北' 参数 +``` + +**没装 Ollama 也能验逻辑**：用 `unittest.mock.MagicMock` 取代 client、塞固定 response、assert 一样 work。完整 mock 范例见 [`examples/stage-3/03-react-from-scratch/test.py`](../examples/stage-3/03-react-from-scratch/test.py)（pattern 跨 backend 通用）。 + +
+ +
+📋 起手码 — Path B（Anthropic API、选择性）（复制到 practice_1_anthropic.py） + +```python +# 需要：pip install anthropic +# 环境变量：export ANTHROPIC_API_KEY=sk-ant-... +import anthropic + +client = anthropic.Anthropic() + +# Anthropic native tool schema — 不用包 wrapper +weather_tool = { + "name": "get_weather", + "description": "查询城市目前天气（晴/雨/阴），回传一个短字符串。", + "input_schema": { + "type": "object", + "properties": { + "city": {"type": "string", "description": "城市名称（如「台北」）"}, + }, + "required": ["city"], + }, +} + +resp = client.messages.create( + model="claude-haiku-4-5", + max_tokens=512, + tools=[weather_tool], + messages=[{"role": "user", "content": "台北现在有下雨吗？"}], +) + +# === 自我验证 === +assert resp.stop_reason == "tool_use", f"非预期 stop_reason: {resp.stop_reason}" +tool_calls = [b for b in resp.content if b.type == "tool_use"] +assert tool_calls[0].name == "get_weather" +assert tool_calls[0].input.get("city") +print(f"✅ 练习 1 通过（Anthropic）— Claude 选了 get_weather、city='{tool_calls[0].input['city']}'") +``` + +**3 个关键 SDK 差异**： +- **Schema wrap**：Anthropic 直接 `tools=[{name, description, input_schema}]`；OpenAI/Ollama 要包 `[{"type":"function", "function":{...}}]` +- **Response 路径**：Anthropic 从 `resp.content[i].type=="tool_use"` 抓；OpenAI/Ollama 从 `resp.choices[0].message.tool_calls[i]` +- **Args 格式**：Anthropic `.input` 是 dict（自动 parse）；OpenAI/Ollama `.function.arguments` 是 JSON string，要 `json.loads(...)` + +**成本**：1 次 ≈ $0.001。**Claude 的 tool-use 比 qwen2.5:3b 更稳**——复杂场景（5+ tools、模糊问题）gap 会明显。 + +
+ +### 练习 2：多工具选择 +给 Claude 三个工具（搜索、计算器、日历）跟一个任务。看 Claude 怎么挑工具，顺便注意它什么时候会挑错。 + +
+📋 简化版核心观念 — Path A (Ollama) + +**NEW vs 练习 1**：tools 从 1 个变 3 个。LLM 看 `description` 边界决定挑哪个——`description` 写得越像“给人读的 docstring”、越容易挑错。 + +```python +from openai import OpenAI +import json + +client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + +TOOLS = [ + {"type": "function", "function": {"name": "web_search", + "description": "Search current or external info not in the prompt.", + "parameters": {"type": "object", "properties": {"query": {"type": "string"}}, "required": ["query"]}}}, + {"type": "function", "function": {"name": "calculator", + "description": "Evaluate basic arithmetic with +, -, *, /, parentheses.", + "parameters": {"type": "object", "properties": {"expression": {"type": "string"}}, "required": ["expression"]}}}, + {"type": "function", "function": {"name": "calendar_lookup", + "description": "Look up events for a specific date.", + "parameters": {"type": "object", "properties": {"date": {"type": "string"}}, "required": ["date"]}}}, +] + +resp = client.chat.completions.create(model="qwen2.5:3b", tools=TOOLS, + messages=[{"role": "user", "content": "What is (19 * 42) - 8?"}]) + +tc = resp.choices[0].message.tool_calls[0] +print(f"LLM 挑了: {tc.function.name}, args: {json.loads(tc.function.arguments)}") +# 预期：calculator, {"expression": "(19 * 42) - 8"} +``` + +**punchline**：3 个 tool 的 `description` 边界要互斥——`calendar` 写“日历”太笼统、会跟 `web_search` 撞；写“特定日期事件”就清楚。小 model 对 description 质量比 Claude 更敏感。 + +**Path B (Anthropic) 改 3 行**：schema 拿掉 `{"type": "function", "function": {...}}` 外包、`tool_calls` 变 `[b for b in resp.content if b.type == "tool_use"]`、`tc.input` 已经是 dict 不用 `json.loads`。完整版见 folder。 + +
+ +→ **基础 starter 范本** → [`examples/stage-3/02-multi-tool-selection/`](../examples/stage-3/02-multi-tool-selection/)（starter.py 含 stub + 简单 test，illustrative，**不是 chapter-length 完整教程**；深度章节见 stage 开头 📚 hello-agents callout） + +### 结构化输出（Structured Outputs / JSON mode）⭐ function calling 的孪生兄弟 + +function calling 是“**让模型决定要不要动手**”；**结构化输出是“强制模型返回一个固定形状的 JSON”**——两者常搞混，但用途不同：前者让 agent 采取行动，后者让你拿到可程序解析的数据（填表、分类、抽取、eval 评分）。 + +**三种做法（由弱到强）**： +1. **prompt 要求 JSON**——最简单、但模型有时会多嘴或格式跑掉。 +2. **JSON mode / `response_format`**——API 保证返回合法 JSON（但不保证符合你的 schema）。 +3. **JSON-schema 强制 / constrained decoding**——连 schema 都锁死，返回的一定符合（最可靠）。 + +> 💡 为什么重要：agent 的 state、tool 参数、eval 评分全都依赖“拿得到结构化数据”。这是 tool calling 底下那层 load-bearing 的可靠度基础。 + +**动手工具**：[jxnl/instructor](https://github.com/jxnl/instructor)（★13k，把 Pydantic model 当 schema、自动 retry）、[dottxt-ai/outlines](https://github.com/dottxt-ai/outlines)（★14k，constrained decoding、连本机 LLM 都能锁 schema）；Stage 4 的 Pydantic AI 也是同路线。 + +### 练习 3：从零实现 ReAct（不用 framework） +用 50-80 行 Python 把 Thought → Action → Observation 循环写出来。不要 LangChain、不要 LangGraph，就是纯 `while not done: thought; action; observation; ...`。 + +
+📋 简化版核心观念 — Path A (Ollama)、ReAct loop 的全部就在这 13 行 + +**NEW vs 练习 2**：把单次 call 包进循环、`messages` 一直长、看 `tool_calls` 在不在来决定收尾。 + +```python +# 假设 TOOLS + TOOL_IMPL（dict: name → callable）已经像练习 2 一样定义好 +messages = [{"role": "user", "content": "台北人口除以纽约人口？"}] + +for step in range(5): # max_iter safety net + r = client.chat.completions.create(model="qwen2.5:3b", tools=TOOLS, messages=messages) + msg = r.choices[0].message + # 把 assistant response 接回 messages（重要！下轮 LLM 才看得到自己上轮讲什么） + messages.append({"role": "assistant", "content": msg.content, "tool_calls": msg.tool_calls}) + if not msg.tool_calls: + print(f"✅ 收尾：{msg.content}"); break + for tc in msg.tool_calls: + args = json.loads(tc.function.arguments) + obs = TOOL_IMPL[tc.function.name](args) # 本地执行 + # observation 接回 messages（用 role="tool"、配 tool_call_id） + messages.append({"role": "tool", "tool_call_id": tc.id, "content": obs}) +``` + +**3 个容易踩坑**： +1. **忘记把 assistant response 加回 messages**——下轮 LLM 看不到自己上轮讲什么、会 loop forever +2. **`tool` message 没带 `tool_call_id`**——LLM 无法配对哪个 result 对应哪个 call +3. **没 `max_iter`**——tool 结果写不好时、LLM 会无限调用，safety net 必须设 + +**Path B (Anthropic) 差几行**：循环架构一模一样、`msg.tool_calls` 变 `[b for b in resp.content if b.type == "tool_use"]`、用 `stop_reason == "end_turn"` 判停、tool result 包成 `{"type": "tool_result", "tool_use_id": ..., "content": obs}` 放进 user message。完整版见 folder。 + +
+ +→ **基础 starter 范本** → [`examples/stage-3/03-react-from-scratch/`](../examples/stage-3/03-react-from-scratch/)（含 mock-based test.py、不花 API 钱也能验；illustrative，**不是 chapter-length 完整教程**——深度章节见 stage 开头 📚 hello-agents callout） + +### 练习 4：多步骤推理任务 +一个需要连续调用 3-5 次 tool 的任务。例如：“找出台北人口，除以纽约人口，再把比例换成百分比。”每一步用不同的工具。 + +
+📋 简化版核心观念 — 跟练习 3 同一个 loop、跑久一点而已 + +**NEW vs 练习 3**：**完全同一个 loop**——只是 `TOOLS` 换成 4 个（`lookup_population` / `divide` / `to_percentage` / `round_int`）、题目自然走完 4 轮 tool call 才收尾。 + +```python +# 没有新 code、纯粹是 TOOLS / TOOL_IMPL 换内容 +TOOL_IMPL = { + "lookup_population": lambda i: lookup_population(i["city"]), + "divide": lambda i: divide(i["a"], i["b"]), + "to_percentage": lambda i: to_percentage(i["ratio"]), + "round_int": lambda i: round_int(i["x"]), +} +# loop 完全照 练习 3，只是 max_iter 拉大到 8 +``` + +**punchline**：多步推理不是新 pattern、是**让 ReAct loop 跑久一点**。**真正的挑战是“LLM 会不会中间漏一步”**——qwen2.5:3b 可能漏“转百分比”、Claude haiku 较稳。**这恰好是观察“model 规模 vs 多步稳定度”的好实验**。试试 `MODEL=qwen2.5:7b python starter.py` 对照。 + +
+ +→ **基础 starter 范本** → [`examples/stage-3/04-multi-step-reasoning/`](../examples/stage-3/04-multi-step-reasoning/)（starter.py 含 stub + 简单 test，illustrative，**不是 chapter-length 完整教程**；深度章节见 stage 开头 📚 hello-agents callout） + +### 练习 5：错误处理 +让某个工具失败（网络错误、输入无效）。看看 agent 会怎么处理错误、能不能恢复，再加上 retry 机制。 + +
+📋 简化版核心观念 — tool error 是 data、不是 exception + +**NEW vs 练习 4**：tool error 回传**结构化 dict**、不要 `raise`。loop 把 dict 接回 LLM、模型自己决定 retry / 改 query / 放弃。 + +```python +def fetch_weather(city: str) -> dict: + if network_failed(): + return {"error": "network timeout", "retry_hint": "try again in 1s"} + return {"city": city, "forecast": "rain", "temperature_c": 24} + +# loop 里： +obs = fetch_weather(args["city"]) +messages.append({"role": "tool", "tool_call_id": tc.id, + "content": json.dumps(obs, ensure_ascii=False)}) # error dict 也是 string 化接回去 +# 下一轮 LLM 看到 retry_hint、可能会 retry、可能会放弃、可能会改 query +``` + +**为什么不 `raise`**：`raise` 直接中断 loop、LLM 没机会 recover。**Production 的 retry 不在 Python 层、而在 LLM 层**——这个 mental flip 是 Stage 3 练习 5 的核心。 + +**Bad vs Good error 回传**： + +| Bad | Good | +|---|---| +| `raise Exception("failed")` | `return {"error": "network timeout", "retry_hint": "try again in 1s"}` | +| `return "failed"` | `return {"error": "...", "category": "transient", "retry_hint": "..."}` | +| 无限 retry | `max_iter` safety + 业务层 retry quota | + +**小 model 观察**：qwen2.5:3b 对 `retry_hint` follow-up 较弱、可能直接放弃；Claude haiku 较稳。完整版（含连续失败 graceful end 范例）见 folder。 + +
+ +→ **基础 starter 范本** → [`examples/stage-3/05-error-handling/`](../examples/stage-3/05-error-handling/)（starter.py 含 stub + 简单 test，illustrative，**不是 chapter-length 完整教程**；深度章节见 stage 开头 📚 hello-agents callout） + +### 练习 6：Function schema 设计（坏 schema 修到好） +**先给 LLM 一份故意写烂的 schema**——`description` 模糊（“处理数据”）、参数全用 `type: string`、没分 required / optional、enum 该用没用。观察 LLM 怎么选错 tool、传错参数。然后逐项修： +- description 写到 LLM 一眼就懂这个 tool 适用情境（不是写给人读的 docstring） +- parameters 用对 type（number / boolean / enum / array），required 列清楚 +- 模糊边界用 enum 强制收敛（例如 `unit: "celsius" | "fahrenheit"` 而不是 `unit: string`） +- error 回传要包 `{"error": "...", "retry_hint": "..."}` 让 LLM 能恢复 + +> 💡 详细 cheatsheet 看 [`resources/schema-design-cheatsheet.md`](../resources/schema-design-cheatsheet.md)——5 条黄金规则 + 5 个常见 anti-pattern。 + +
+📋 简化版核心观念 — bad vs good schema 对照 + +**NEW vs 练习 5**：同一个工具（温度转换）、两种 schema 写法。看 4 个差别。 + +```python +# ❌ BAD — qwen2.5:3b 几乎必错（Claude haiku 还能猜对、但几率明显下降） +{"name": "convert", "description": "Convert a value.", + "parameters": {"type": "object", "properties": { + "value": {"type": "string"}, "unit": {"type": "string"}}}} + +# ✅ GOOD — qwen 也能稳定挑对 +{"name": "convert_temperature", + "description": "Use when user asks to convert temperatures between Fahrenheit and Celsius.", + "parameters": {"type": "object", "properties": { + "value": {"type": "number", "description": "Temperature value"}, + "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, + "required": ["value", "unit"]}} +``` + +**4 个改进**：(1) `name` 改具体、(2) `description` 写“**何时**用”而非“**做什么**”、(3) `type` 改 `number`、(4) 加 `required` + `enum`。 + +**punchline**：**写 schema 的功夫能省下换大 model 的成本**——小 model 对 schema 质量比大 model 敏感，相同 bad schema 在 Claude 上可能还能猜对、在 qwen 上几乎必错。Production 想用便宜 model？schema 必须写到能上线跑的程度。 + +**搞不定 schema 怎么办**：装 [`examples/stage-5/tool-calling-tutor/`](../examples/stage-5/tool-calling-tutor/) skill，遇到“LLM 不调用我的 tool”、“我这 schema 哪里写坏”会自动跳出来帮你 debug。 + +
+ +→ **基础 starter 范本** → [`examples/stage-3/06-schema-design/`](../examples/stage-3/06-schema-design/)（含 bad schema vs good schema 两个版本对照；illustrative，**不是 chapter-length 完整教程**——深度章节见 stage 开头 📚 hello-agents callout） + +> 💡 **手写 schema 之后，认识 MCP**：你上面手写的 tool schema，真实世界已经有标准——**MCP（Model Context Protocol）** 把“工具长什么样、怎么调用”标准化成跨 agent 可复用的协定：写一次，任何支持 MCP 的 agent（Claude Code / Cursor / …）都能用。这里先记得这个名字，[Stage 5.2](05-claude-code-ecosystem.zh-Hans.md#52--mcpmodel-context-protocol-基础) 细讲。 + +## 🪞 反思（Reflexion / Self-Refine）— 概念 + 路由 + +> **本节是 concept + routing、不是练习**。没有 verified working solution、不挂“练习 N”label、不给 success criteria——遵守本 repo“没验证答案不写练习、顶多 routing”原则。想动手做？直接读下方 paper / project。 + +**反思是什么**：练习 5 的 error handling 是“LLM 出错 → 你（外部）catch + retry”；**反思**是“LLM 观察自己出错 → 自己改”。差别是 agency 在哪一边——这是 production agent（Cursor / Cline / Claude Code）每天都在跑的循环。 + +**为什么这节在 Stage 3 而不是 Stage 6**：反思在学术（Reflexion paper Shinn 2023、Self-Refine Madaan 2023）跟 production（Cursor / Claude Code）上都被归类在 **planning / reasoning loop** 机制——是 ReAct（练习 3）的 sibling pattern，**不是 memory pattern**。同样是 LLM 自我引导的多轮循环，只是“下一轮要做什么”从“调用 tool”换成“批改自己”。 + +**进阶版（带 persistent memory 的 Reflexion 完整版）→ [Stage 6 进阶：带持久记忆的 Reflexion 完整版](06-memory-rag.zh-Hans.md#-进阶带持久记忆的-reflexion-完整版--track-b-选读)**——当反思要跨 session、把过去失败存起来当下一轮 context，这个版本才真的需要 memory 层。 + +### 一张对照图 + +| Pattern | 形式 | 需要 memory？ | 在哪学 | +|---|---|---|---| +| **Error handling**（练习 5） | 外部 catch + retry | 不需 | **本 stage 练习 5** | +| **ReAct loop**（练习 3） | LLM → tool → 结果 → LLM | 不需 | **本 stage 练习 3** | +| **基本反思 / Self-Refine** | Actor → Critic → Actor，single session | 不需 | **本节 routing（下方）** | +| **完整 Reflexion**（含 episodic memory） | 上面 + 把失败反思存起来、跨 session 累积 | **需要** | **Stage 6 进阶：Reflexion with Memory** | + +### 📚 想动手 / 想深入？直接读这些 + +**Paper**： +- [**Reflexion (Shinn et al. 2023)**](https://arxiv.org/abs/2303.11366) ⭐ — 原 paper，定义“verbal reinforcement learning” +- [**Self-Refine (Madaan et al. 2023)**](https://arxiv.org/abs/2303.17651) — single-agent self-critique，是“基本反思”的学术定义 + +**Reference 实现**： +- [**arunpshankar/react-from-scratch**](https://github.com/arunpshankar/react-from-scratch) — 已在本 stage 精选 Projects 列出，含 Reflection 实现可直接读 +- [**LangChain — Reflection Agents（blog）**](https://blog.langchain.dev/reflection-agents/) — framework 实现参考 + 完整 working notebook +- [**datawhalechina/hello-agents**](https://github.com/datawhalechina/hello-agents) — 对应章节（自我反思 / Self-Refine 段落、中文完整教学） + +> 💡 **想看反思怎么长进 production agent**：[Stage 5 5.7 Harness Internals](05-claude-code-ecosystem.md#57--claude-code-source-解剖reference-harness-implementation-track-b-必看) 解剖 Claude Code source 时可以看到——agent 跑完 tool call 后自我评估 patch、有问题回头改、修正后再 commit。**这是现代 production agent 的核心 building block 之一**。 + +## 🎯 精选 Projects + +按用途分 4 类、12 个项目一张表搞定。**挑入口看“适合谁”、想深入点链接看 repo README**。 + +| 分类 | Project | ⭐ | 适合谁 | 为什么推荐 / 备注 | +|---|---|---|---|---| +| **官方 cookbook**
（先看这个） | [Anthropic — Tool Use Cookbook](https://github.com/anthropics/claude-cookbooks/tree/main/tool_use) | ⭐⭐⭐⭐⭐ | 练习 1 / 练习 2 入手 | 单工具 → 多工具 → parallel → structured output 全部 notebook（重点看 `tool_use/customer_service_agent.ipynb`） | +| | [Anthropic — Quickstarts](https://github.com/anthropics/claude-quickstarts) | ⭐⭐⭐⭐⭐ | 练习 1/2 跑完想看“真的应用长什么样” | 3 个 deploy-ready 范本（financial / customer-support / computer-use）、★ 16k+。比社群实现更 canonical | +| | [Anthropic — Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents) | ⭐⭐⭐⭐⭐ | 练习 3 写完、进 Stage 4 之前**必读** | 部落格文章：何时用 agent vs workflow / 常见 pattern / 容易踩的坑——Anthropic 官方观念框架 | +| **从零实现 ReAct**
（理解原理） | [pguso/ai-agents-from-scratch](https://github.com/pguso/ai-agents-from-scratch) | ⭐⭐⭐⭐⭐ | 练习 3（从零写 ReAct） | 用本机 Ollama 从零打造、zero framework、章节结构好。**最干净的“不靠 framework”参考实现** | +| | [arunpshankar/react-from-scratch](https://github.com/arunpshankar/react-from-scratch) | ⭐⭐⭐⭐ | 练习 3 替代（偏好 Gemini）+ 想看反思变体 | ReAct + Reflection + Self-consistency、Gemini 最佳化（⚠️ 2025-05 后更新放缓、Apache-2.0） | +| | [mattambrogi/agent-implementation](https://github.com/mattambrogi/agent-implementation) | ⭐⭐⭐ | 练习 3 卡住时逐行对照 | ~150 行最精简 ReAct（⚠️ 已停滞 2024-01、留作教学玩具参考） | +| | [lsdefine/GenericAgent](https://github.com/lsdefine/GenericAgent) | ⭐⭐⭐⭐ | 练习 3/4，想看“精简但完整”framework | 自我演化 framework、~3K 行、★ 13k+、支持 Claude / Gemini / Kimi / MiniMax。介于玩具版与 LangGraph 之间 | +| **CodeAct 路线**
（agent 写代码当 action） | [HuggingFace Smolagents](https://github.com/huggingface/smolagents) | ⭐⭐⭐⭐ | 练习 5 替代方案、本地 LLM 实验 | ≤1000 LOC、CodeAct pattern 代表、★ 27k+。HF 立场：agent 应该要小 | +| | [QuantaLogic/quantalogic](https://github.com/quantalogic/quantalogic) | ⭐⭐⭐ | 练习 3 后想比较 CodeAct vs JSON-tool | 另一条 CodeAct 路线、agent 直接写 Python 代码当 action、Apache-2.0 | +| **中文章节式深度教材**
（chapter-length） | [datawhalechina/hello-agents](https://github.com/datawhalechina/hello-agents) ⭐ **本 stage 推荐** | ⭐⭐⭐⭐⭐ | 中文读者想要结构化教学 + 完整覆盖 | **16 种能力**含 tool use / ReAct / context engineering / sub-agents / circuit breaker / observability。中文圈最完整章节式（CC BY-NC-SA、非商用） | +| | [HelloAgents (jjyaoao)](https://github.com/jjyaoao/HelloAgents) | ⭐⭐⭐⭐⭐ | 中文读者、想跑上面教材的 code | 上面教材 code repo、**请切 `learn_version` 分支**对齐章节（`pip install hello-agents`、CC BY-NC-SA） | +| **Framework 对照**
（看 framework 怎么藏掉 ReAct loop） | [LangChain — ReAct Agent Template](https://github.com/langchain-ai/react-agent) | ⭐⭐⭐ | 练习 3 自己写完后在来 | LangGraph Studio 范本、framework 怎么把 ReAct 抽象化 | + +> 💡 **建议阅读路径**：练习 1-2 跑 Anthropic Cookbook → 练习 3 跑 pguso/ai-agents-from-scratch → 练习 3 后读 Building Effective Agents → 中文章节式教材就 hello-agents + jjyaoao 配对 → 进 Stage 4 前看 LangChain ReAct template 对照 framework 抽象。 + + +## ✅ 进 Stage 4 前的自我检查 + +你能不能： + +- [ ] 定义一个 tool schema（name + description + JSON schema 输入/输出） +- [ ] 用不到 100 行 Python、不靠任何 framework，把 ReAct 循环写出来 +- [ ] 解释为什么 agent 需要一个“我做完了”的退出条件 +- [ ] 比较 CodeAct（代码即 action）跟 JSON-tool 两种路线 +- [ ] 看出哪些问题其实不需要 agent + +如果可以 → 进 [Stage 4 — Agent Frameworks](04-agent-frameworks.md)。 + +如果不行 → 把 练习 3 再跑一次，不要跳过。如果你不懂 framework 在帮你抽象什么，Stage 4 的那些东西看起来会像黑魔法。 diff --git a/stages/04-agent-frameworks.en.md b/stages/04-agent-frameworks.en.md new file mode 100644 index 0000000..b15ef1f --- /dev/null +++ b/stages/04-agent-frameworks.en.md @@ -0,0 +1,225 @@ +# Stage 4 — Agent Frameworks + +> [繁體中文](./04-agent-frameworks.md) | [简体中文](./04-agent-frameworks.zh-Hans.md) | **English** + +⏱ **Estimated time**: 2-3 weeks (approx. 10-15 hours) + +> 💡 Unfamiliar with terms like `framework`, `supervisor`, `worker`, `handoff`? → Check [`resources/glossary.md`](../resources/glossary.md). + +> 📋 **Chapter structure**: Learning Objectives → Entry Conditions → Required Reading → [Optional · Concept Map: multi-agent intro + advanced tool patterns] → Hands-on Exercises → Curated Projects → Self-Check +> 🔑 **Key Terms**: See [`resources/glossary.md`](../resources/glossary.md) (2 & 4 cover terms like `framework`, `agent loop`, `handoff`, `supervisor`). + +You've built a ReAct agent from scratch (Stage 3). Now, let's see what a framework actually does for you. **Pick one to learn deeply**, and just skim the others to know when to switch. + +## 📌 Learning Objectives + +After completing this stage, you will be able to: +- Compare 5 major agent frameworks (LangGraph, AutoGen, CrewAI, Smolagents, OpenAI Agents SDK) +- Select the right framework for a given task +- Build the same agent using two different frameworks to experience the differences firsthand +- Recognize when to ditch the framework and write your own code + +## 🚪 Entry Conditions + +You should have already: +- Completed all 5 hello-X projects from Stage 3 +- Written a ReAct agent from scratch (Exercise 3) +- Become comfortable with async Python (frameworks rely heavily on it) + +## 📚 Required Reading + +1. [**Anthropic — Building Effective Agents**](https://www.anthropic.com/engineering/building-effective-agents) — When to use a framework vs. when to use the raw API directly. +2. [**LangChain — Conceptual Guide: Agents**](https://python.langchain.com/docs/concepts/agents/) — The abstract concepts of agents. +3. [**Best Multi-Agent Frameworks 2026 comparison**](https://gurusup.com/blog/best-multi-agent-frameworks-2026) — Current market positioning. +4. **A Quickstart from one framework** — Choose either LangGraph or CrewAI and go through the official tutorial from start to finish. + +## 🤔 What is a multi-agent framework? + +### Two Dimensions to Clarify First (Workflow vs. Agent / Single vs. Multi) + +To understand multi-agent frameworks, a useful clarification is to treat **workflow vs. agent** and **single vs. multi LLM** as two orthogonal dimensions. The core distinction in Anthropic's "Building Effective Agents" is between a workflow (fixed code path) and an agent (LLM autonomously decides the next step). Let's stack this with single/multi to see the four quadrants: + +| | **Workflow**
(Your pre-written code path) | **Agent**
(LLM dynamically decides the next step) | +| -------------- | ---------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- | +| **Single LLM** | Linear pipeline, no branching logic | One LLM + ReAct loop, plans and adapts on its own
(**This is what you built in Stage 3**) | +| **Multi LLM** | Pre-defined routing (e.g., "sales questions → agent A, tech questions → agent B") | 2+ agents handing off to each other, with an orchestrator dynamically assigning tasks
(**The topic of this stage**) | + +**Why this distinction is useful**: Most production scenarios fall into the "single agent workflow" + "single agent" quadrants. Most tasks simply don't require multi-agent setups. **The quadrant that truly needs a multi-agent framework is the bottom right**—high LLM autonomy + multi-role collaboration. In practice, the boundaries between these quadrants can be blurry (LangGraph's conditional edges can be seen as both workflow routing and dynamic agent decision-making). Don't treat this matrix as a mutually exclusive classification. + +→ All later discussion in this stage assumes you already know: **a multi-agent framework mainly handles the coordination, handoff, state management, and repetitive scaffolding code between multiple agents, so you don't have to write the whole collaboration flow from scratch** (the orchestration boilerplate in the lower-right quadrant). + +### Single-agent vs. multi-agent — A comparison table to clarify the differences + +| Dimension | **Single-agent** (You built this in Stage 3) | **Multi-agent system** | +| ------------------ | --------------------------------------------------- | ------------------------------------------------------------------------------- | +| **Architecture** | One LLM + ReAct loop + several tools | 2+ LLMs, each with a role (researcher / writer / critic...), coordinated by an orchestrator | +| **Decision Making**| The same LLM thinks from start to finish | Role-based decomposition + handoff, different LLM instances see different perspectives | +| **State Management**| Linear message history | Shared state / message passing / checkpoints | +| **Suitable For** | Linear logic, < 20-30 tools, single objective | Decomposable tasks, need for perspective diversity, long workflows, parallelization | +| **Debug Cost** | Low (a single loop is easy to trace) | High (cross-agent interaction, error propagation is hard to pinpoint) | +| **Token Cost** | 1x | Typically **3-10x** (each sub-agent has its own prompt + thinking + tool calls) | +| **Latency** | Low | High (unless sub-agents run in parallel) | + +### When do you **really** need multi-agent? (Don't force it) + +**Multi-agent is not the default; it's a last resort.** **Both Anthropic and Cognition, two frontier labs, explicitly stated in 2024-2025 that 90% of use cases should not use multi-agent setups.** Forcing it results in **3-10× token costs, painful debugging, and severe context fragmentation.** + +| Stance | Source | Core Argument | +| ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | +| **Anthropic** | [Building Effective Agents (2024)](https://www.anthropic.com/engineering/building-effective-agents), [How we built our multi-agent research system (2025)](https://www.anthropic.com/engineering/built-multi-agent-research-system) | Most scenarios are fine with a simple workflow + single agent; multi-agent is only truly helpful for "**research-style / parallel exploration**" tasks. | +| **Cognition** | [Don't Build Multi-Agents (2025)](https://cognition.ai/blog/dont-build-multi-agents) | Multi-agent setups suffer from severe context fragmentation and painful shared state maintenance; exhaust single-agent + long-context options first. | + +You typically need multi-agent when one of these four signals appears: + +| Signal | Description | Corresponding Pattern | +| ----------------------- | ----------------------------------------------------------------------- | --------------------------------------- | +| **1. Natural Task Decomposition** | A large task has clear sub-steps that can be completed step-by-step. | Sequential / Planner-Executor | +| **2. Token Explosion** | A single agent's prompt can't fit all tool descriptions/context. | Supervisor-Worker (offloads to sub-agents) | +| **3. Role Conflict** | Having the same LLM act as both writer and critic leads to self-justification. | Debate / Peer review | +| **4. Parallel Acceleration**| Running 3 research sub-tasks concurrently reduces wall-clock time to 1/3. | Parallel / Map-Reduce variant | + +**None of these four signals present?** → A single agent + a good prompt + tool use is enough. **Forcing a multi-agent setup will cost you 3-10x in tokens, be a pain to debug, and won't necessarily be more accurate.** + +> 💡 **Further Reading**: [Stage 7 But do you really need multi-agent?](07-multi-agent-production.en.md#-but-do-you-really-need-multi-agent) will revisit this decision from a production perspective. This section covers the design-phase decision; that one is the final check before deployment. + +### Classic Multi-agent Patterns (ordered by complexity) + +> 📝 **How this differs from Stage 3 Classic Paradigms**: [The 4 paradigms in Stage 3](03-tool-use-and-hello-agent.en.md#classic-agent-paradigms-thinking-patterns) (CoT / ReAct / Reflection / Planning) describe **how a single agent thinks internally**. The 5 patterns in this section describe **how multiple agents collaborate**—two orthogonal layers. + +| Pattern | Complexity | What it looks like | Classic Scenarios | Representative Framework / Paper | +| ---------------------------- | ---------- | ---------------------------------------------------------------- | -------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | +| **1. Routing / Handoff** | ⭐ | 1:1 handoff between agents, no central orchestrator | Customer support routing, context switching | [OpenAI Swarm](https://github.com/openai/swarm), [OpenAI Agents SDK](https://github.com/openai/openai-agents-python) | +| **2. Sequential**
(Planner → Executor) | ⭐⭐ | Planner creates multi-step plan + executor carries it out | Multi-step automation, code generation | LangGraph, [ChatDev paper](https://arxiv.org/abs/2307.07924) | +| **3. Parallel**
(Parallel Acceleration) | ⭐⭐⭐ | N agents run concurrently, results are aggregated | Research / map-reduce tasks, 1/N wall-clock time | LangGraph parallel branches, CrewAI parallel tasks. **Pitfalls**: async coordination, partial failure, state merge consistency. | +| **4. Supervisor-Worker**
(hub-spoke) | ⭐⭐⭐ | 1 supervisor + N workers, supervisor assigns + integrates | Task decomposition, report integration | LangGraph, AutoGen GroupChat | +| **5. Debate / Society**
(Multi-perspective convergence) | ⭐⭐⭐⭐ | 2+ agents critique each other or role-play | Research, judgment tasks, social simulation | AutoGen GroupChat, [CAMEL paper](https://arxiv.org/abs/2303.17760), [Generative Agents paper](https://arxiv.org/abs/2304.03442) | + +### Claude Code subagent — An alternative orchestration path + +> **This section is on a different level than the 5 patterns above**: The 5 patterns are design choices that can be implemented with or without a framework. The **Claude Code subagent** introduced here is another execution model (built-in runtime orchestration, no framework code). After reading about the 5 patterns, this section shows you "there's a second path for multi-agent." + +**Frameworks aren't the only way to do multi-agent.** Anthropic's own Claude Code offers another layer of abstraction: the [subagent](05-claude-code-ecosystem.en.md#55--subagents-claude-codes-native-multi-agent-mechanism--2025-new-feature). You create a subagent by writing a `.claude/agents/.md` file—**no framework required**. + +The fundamental difference from the framework path (in one line): the **framework path** is cross-LLM-provider, written as Python orchestration code, with full checkpointing / audit trail; **Claude Code subagent** runs only inside the Claude Code runtime, written as markdown not code, with built-in context isolation. + +> 📌 **The full dimension-by-dimension comparison table (startup / runtime / context isolation / provider lock-in / learning curve) lives canonically at [Stage 5.5](05-claude-code-ecosystem.en.md#55--subagents-claude-codes-native-multi-agent-mechanism--2025-new-feature)** — this stage only needs you to know "there's a second, Claude-Code-native path"; see 5.5 for the per-item implementation differences. + +**When to choose subagents over a framework**: +- You're already using Claude Code for your daily work. +- The task context is large and would consume the entire main session window (e.g., reading a whole codebase). +- You want to run multiple subagents in parallel (research / write / critic) to save wall-clock time. +- You don't need cross-provider migration. + +For detailed implementation and hands-on exercises, see [Stage 5.5](05-claude-code-ecosystem.en.md#55--subagents-claude-codes-native-multi-agent-mechanism--2025-new-feature) (**it's recommended to complete Stage 5.1 on Claude Code basics before tackling 5.5**—subagents are an advanced feature of the ecosystem and require familiarity with the basics). + +### What Frameworks Do + +Frameworks abstract away the orchestration boilerplate for the 5 patterns above (roles, handoffs, state, retries, checkpoints, HITL pauses), letting you focus on writing role definitions and task descriptions. In short: **a framework is scaffolding for multi-agent systems, not a necessity**. For simple scenarios, you could write your own solution with a dictionary and a for loop (which is exactly what you'll do in Stage 7, Exercise 1). + +### 📚 Want a more systematic, in-depth look? + +**🇺🇸 Academic Papers (Influenced all subsequent framework designs)**: +1. [**Anthropic — "Building Effective Agents"**](https://www.anthropic.com/engineering/building-effective-agents) ⭐⭐⭐ — When to use a workflow vs. an agent, 5 classic orchestration patterns. **Essential reading for multi-agent design in the English-speaking world.** +2. [**AutoGen paper (Wu et al. 2023)**](https://arxiv.org/abs/2308.08155) — The original paper for Microsoft's multi-agent conversation framework. +3. [**CAMEL paper (Li et al. 2023)**](https://arxiv.org/abs/2303.17760) — The seminal work on multi-agent role-playing. +4. [**ChatDev paper (Qian et al. 2023)**](https://arxiv.org/abs/2307.07924) — Canonical example of planner-executor for multi-agent software development. +5. [**Generative Agents paper (Park et al. 2023)**](https://arxiv.org/abs/2304.03442) — 25 agents interacting in a Sims-like environment, a classic in social simulation. + +**🀄 Chinese Systematic Resources**: +1. [**hello-agents Ch6 "Framework Development Practice" + Ch7 "Build Your Agent Framework"**](https://github.com/datawhalechina/hello-agents) ⭐ — A comprehensive Chinese resource on framework development and building one from scratch. **Note: Ch4 "Classic Agent Paradigm Construction" covers single-agent paradigms (ReAct / Plan-and-Solve / Reflection), not multi-agent.** +2. [**Hung-yi Lee — Introduction to Generative AI**](https://speech.ee.ntu.edu.tw/~hylee/genai/2024-spring.php) — Later episodes cover AI agents / multi-agent systems. + +**Framework Official Multi-Agent Docs**: +- [**LangGraph — Multi-Agent Systems**](https://langchain-ai.github.io/langgraph/concepts/multi_agent/) — Official tutorials for supervisor, swarm, and hierarchical architectures. +- [**Anthropic Cookbook — `customer_service_agent.ipynb`**](https://github.com/anthropics/claude-cookbooks/tree/main/tool_use) — A canonical example of multi-agent orchestration (routing + handoff). +- [**Microsoft AutoGen — Examples**](https://microsoft.github.io/autogen/) — Complete examples for group-chat, debate, and peer review patterns. + +> 💡 **Recommended Learning Path for Frameworks** (5 steps): +> 1. **Build a mental model** (30 min) — Read Anthropic's Building Effective Agents and get the workflow vs. agent and single vs. multi dimensions straight. +> 2. **Run a framework quickstart** (2-3 hr) — Pick either LangGraph or CrewAI and run their official multi-agent tutorial. +> 3. **Cross-reference with Anthropic's Cookbook `customer_service_agent`** (1 hr) — A production-style routing + handoff example. +> 4. *(Optional)* **Dive into the academic side**: Pick 1-2 papers to read (AutoGen / CAMEL / ChatDev / Generative Agents). +> 5. *(Optional for Claude users)* **Write a subagent for comparison**: See [Stage 5.5](05-claude-code-ecosystem.en.md#55--subagents-claude-codes-native-multi-agent-mechanism--2025-new-feature) and compare it to the framework approach. +> +> **You don't need to read all 5 papers.** Pick the 1-2 that are most relevant to your use case. + +## 🛠 Advanced Tool Patterns (What frameworks handle for you) ⭐ Must-read for Track B + +Stage 3 taught you to write single-tool / multi-tool selection (by hand-writing `if/elif/else` routing). Frameworks abstract this layer away and add three more advanced tool patterns. **These three patterns need a framework's abstraction layer to be written cleanly; trying to hand-write them like in Stage 3 would be a mess:** + +| Pattern | What problem it solves | Representative Implementation | +| ----------------------- | --------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | +| **Dynamic tool selection**| When you have >30 tools, `tools=[...]` is too large for the prompt (context window issues, poor selection). | [LlamaIndex tool router](https://docs.llamaindex.ai/en/stable/module_guides/deploying/agents/tools/) — Embedding-based routing: first, a semantic search finds the top-K tools, and only those K are put into the prompt. | +| **Tool composition / chaining** | Chaining tool A's output to tool B's input without LLM narrative in between (saves tokens + latency). | LangGraph `state graph` directly connecting nodes, CrewAI `sequential tasks`, Pydantic AI's type-safe pipeline. | +| **Tool-augmented retrieval**| The tool itself is a RAG search → returns results → the agent reasons on the results. | A combination of the Stage 6 Exercise 4 RAG pipeline and Stage 3 Exercise 2 multi-tool selection (LangGraph wraps the retriever as a tool node). | + +**📚 In-depth Resources**: +- [**Anthropic — Tool Use best practices**](https://docs.anthropic.com/en/docs/build-with-claude/tool-use/overview) — The official tool design guide. +- [**LlamaIndex — Tool Router pattern**](https://docs.llamaindex.ai/en/stable/module_guides/deploying/agents/tools/) — The canonical reference for dynamic selection. +- [**LangGraph — Tool Node**](https://langchain-ai.github.io/langgraph/) — How to write a composition graph. + +> 💡 **For Track B after this section**: You should be able to explain the differences between implementing the "same task" via (a) hand-writing it in Stage 3, (b) using a framework in this stage, and (c) using a Claude subagent in Stage 5.5. This is a core question for Track B's "knows how to design agents" path. + +## 🛠 Hands-on Exercises + +### Exercise 1: Same agent, two frameworks +Build the same simple agent (search + summarize) using these two frameworks: +- LangGraph +- CrewAI +Compare the lines of code, the debugging experience, and where each framework hides its complexity. + +### Exercise 2: Multi-agent role assignment +Use CrewAI to build a demo with 2-3 agents, each with a different role, collaborating on a single task. (This is CrewAI's strong suit.) + +### Exercise 3: Graph-based workflow +Use LangGraph to build a workflow with branching logic and a human-in-the-loop checkpoint. (This is LangGraph's strong suit.) + +### Exercise 4: CodeAct vs. JSON tool +Use Smolagents to build an agent that writes Python code as its action (the CodeAct pattern). Compare this with the JSON tool-calling approach from Exercise 1. Ask the same question and see how each approach solves it. + +### Exercise 5: Type-safe agent +Use Pydantic AI to build an agent that returns structured output (e.g., for a given question, it returns `{ "answer": str, "confidence": float, "sources": [str] }`). See how Pydantic's schema validation prevents the agent from being lazy or hallucinating the structure. + +## 🎯 Curated Projects + +16 projects across 5 categories, all in one table. **Start by looking at "Who it's for", then if you want to dive deeper, check out the repo / quickstart link.** + +| Category | Project | ⭐ | Who it's for | Why it's recommended / Notes | +| ------------------------------------------ | ---------------------------------------------------------------------------------- | ----------- | ------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Production-tier**
(Complex multi-agent / needs audit) | [LangGraph](https://github.com/langchain-ai/langgraph) ⭐ **This stage's #1 recommendation** | ⭐⭐⭐⭐⭐ | Production multi-agent systems that require audit trails / rollback / replay. | Graph-based orchestration + checkpointing + time-travel debugging. Widely adopted across enterprises. ★ 34k+, MIT, Python+TS. Pairs with LangSmith for observability. | +| | [microsoft/semantic-kernel](https://github.com/microsoft/semantic-kernel) | ⭐⭐⭐⭐ | Building agents in .NET / Java environments, within the Microsoft tech stack. | Official SDKs in C#, Python, and Java. Follows a kernel + plugin + planner pattern. ★ 27k+, MIT. Thick abstraction, not for beginners. | +| | [agno-agi/agno](https://github.com/agno-agi/agno) | ⭐⭐⭐⭐ | Those who want a "build + serve + monitor" solution without the full LangGraph + LangSmith suite. | Multi-modal agent runtime + control plane. ★ 39k+, Apache-2.0. Learn the API in Stage 4, use the runtime in Stage 7. | +| | [microsoft/agent-framework](https://github.com/microsoft/agent-framework) | ⭐⭐⭐⭐ | Teams wanting Microsoft's unified successor to AutoGen + Semantic Kernel, in Python or .NET. | Microsoft's open framework for getting multiple agents to work together — the successor that merges AutoGen and Semantic Kernel. ★ 11k+, MIT, Python + .NET. | +| **Rapid Prototyping / Multi-agent**
(Role-based / handoff) | [CrewAI](https://github.com/crewAIInc/crewAI) ⭐ **This stage's #2 recommendation** | ⭐⭐⭐⭐ | Rapidly prototyping "researcher → writer → critic" pipelines. | Build a crew in ~20 lines of code. Lowest learning curve. ★ 50k+, MIT. ⚠️ No checkpointing for long workflows; use CrewAI for prototypes, LangGraph for production. | +| | [Microsoft AutoGen / AG2](https://github.com/microsoft/autogen) | ⭐⭐⭐⭐ | Multi-agent debate / brainstorming / peer review patterns. | Conversational multi-agent, strong group-chat capabilities. ★ 57k+, CC-BY-4.0 (for docs). ⚠️ AG2 v0.4 was a major async-first rewrite; older tutorials still target v0.2 while newer ones use v0.4, mind the version. | +| | [OpenAI Agents SDK](https://github.com/openai/openai-agents-python) | ⭐⭐⭐⭐⭐ | Teams already committed to the OpenAI ecosystem. | OpenAI's official library. Clean API for agent hand-off + structured output. MIT. **Built-in since April 2026**: a sandbox (7 providers) + harness abstraction layer. The first architecturally sound approach for production coding agents ([details in Stage 8](08-agent-interfaces.en.md#why-the-april-2026-openai-agents-sdk-update-is-a-milestone)). | +| | [deepagents (LangChain)](https://github.com/langchain-ai/deepagents) | ⭐⭐⭐⭐ | Want a "batteries-included" deep-agent harness without wiring it yourself. | An opinionated harness (it makes the design choices for you) on top of LangGraph: built-in planning (todo list), filesystem memory, subagents, loadable skills, context offload (keeps big intermediate results out of the LLM's window). `pip install deepagents`, MIT, v0.6.12 (2026-06). | +| | [OpenAI Swarm](https://github.com/openai/swarm) | ⭐⭐⭐⭐ Edu
⭐⭐⭐ Prod | To understand the **core mental model** of multi-agent systems without learning a full framework. | ~200 LOC, just two concepts: Agent + handoff. MIT. ⚠️ OpenAI labels it experimental/educational, not a production tool. **Read the source code as a chapter-length tutorial.** | +| | [Strands Agents (AWS)](https://github.com/strands-agents/sdk-python) | ⭐⭐⭐⭐ | Teams already committed to the AWS cloud and Bedrock-native solutions. | Model-driven design (LLM plans on its own, no explicit graph). Apache 2.0. Released late 2025, integrates with AWS Lambda / Step Functions / Bedrock Agents. | +| **Specialized Paths**
(CodeAct / typed / memory-first) | [Hugging Face Smolagents](https://github.com/huggingface/smolagents) | ⭐⭐⭐⭐ | Local LLM ecosystems, HF integration scenarios. | A prime example of the CodeAct pattern (agent writes Python code as its action, not JSON tool calls). ★ 27k+, Apache 2.0, ≤1000 LOC. | +| | [Pydantic AI](https://github.com/pydantic/pydantic-ai) | ⭐⭐⭐ | Production systems that require runtime type safety + structured output by default. | Type-safe agents from the Pydantic team. MIT. Relatively new. | +| | [Letta (formerly MemGPT)](https://github.com/letta-ai/letta) | ⭐⭐⭐⭐ | **Long-session / cross-day / persona-stable** agents (long-term assistants, therapists, tutors). | A memory-first multi-agent system based on OS-paging concepts (working memory + archival store). ★ 22k+, Apache 2.0. Also mentioned in Stage 6, Exercise 5. | +| **Specialized** | [LlamaIndex Agents](https://github.com/run-llama/llama_index) | ⭐⭐⭐ | Document-heavy agents (research assistants, knowledge workers, etc.). | Tightly integrated with RAG. ★ 49k+, MIT. Strong on retrieval, weak on orchestration—don't choose it for pure orchestration. | +| | [agentscope-ai/agentscope](https://github.com/agentscope-ai/agentscope) | ⭐⭐⭐ | Researchers who want to visually debug multi-agent workflows. | A multi-agent platform with strong visual debugging tools. ★ 26k+, Apache 2.0. Low adoption in Western communities, but solid tech. | +| | [LangChain](https://github.com/langchain-ai/langchain) | ⭐⭐⭐ | Rapid prototypes that need to glue together many components (retrieval + chains). | The swiss-army-knife framework. ★ 135k+, MIT. **For agent orchestration, use LangGraph now.** LangChain is better for gluing retrieval + chaining. | +| **Infrastructure**
(Not a framework, used across stages) | [BerriAI/litellm](https://github.com/BerriAI/litellm) | ⭐⭐⭐⭐ | When you need to switch between Claude / GPT / Gemini / open-source models without changing code. | A provider-agnostic SDK + AI gateway. Call 100+ LLMs using OpenAI's format. Includes cost tracking / fallbacks / guardrails. ★ 49k+, MIT (`enterprise/` subdirectory licensed separately). | + +> 💡 **Recommended reading path**: Pick **1 for production deployment** (LangGraph) + **1 for rapid prototyping** (CrewAI) framework to learn deeply → Do exercises 1-3 → Skim the READMEs of the others to know they exist. The **3 specialized paths** (CodeAct / typed / memory-first) are only rivals in specific scenarios; you don't need to touch them otherwise. + +## ✅ Self-Check Before Moving to Stage 5 + +Can you: + +- [ ] Build the same agent using both LangGraph and CrewAI? +- [ ] Choose the right framework for a task (production vs. prototype)? +- [ ] Explain the difference between LangGraph's checkpointing and CrewAI's task delegation? +- [ ] See when CodeAct (Smolagents) is better than JSON-tool? +- [ ] Judge when to ditch a framework and use the raw API directly? + +If yes → Proceed to [Stage 5 — The Claude Code Ecosystem](05-claude-code-ecosystem.en.md). + +## 💡 Strategic Tips + Potential Pitfalls + +Don't try to learn all of these. Pick **one for production deployment (LangGraph)** and **one for rapid prototyping (CrewAI)** to learn in depth. For the others, just skim their READMEs to know they exist as options. + +**A heads-up on Memory** (you might encounter this while learning, no need to read ahead): Some framework features use memory concepts—LangGraph's checkpointing (state persistence), CrewAI agents passing task results to each other (lightweight memory). These are covered in full in [Stage 6 — Memory & RAG](06-memory-rag.en.md). If you get stuck on a framework feature in this stage, check that section, but **you don't need to read it all before continuing this stage**. diff --git a/stages/04-agent-frameworks.md b/stages/04-agent-frameworks.md new file mode 100644 index 0000000..509c202 --- /dev/null +++ b/stages/04-agent-frameworks.md @@ -0,0 +1,225 @@ +# Stage 4 — Agent 框架（Agent Frameworks） + +> **繁體中文** | [简体中文](./04-agent-frameworks.zh-Hans.md) | [English](./04-agent-frameworks.en.md) + +⏱ **時間估算**：2-3 週（約 10-15 小時） + +> 💡 用語不熟（framework / supervisor / worker / handoff⋯）→ 翻 [`resources/glossary.md`](../resources/glossary.md)。 + +> 📋 **本章組成**：學習目標 → 進入條件 → 必修閱讀 →〔可選 · 概念地圖：multi-agent intro + 進階 tool patterns〕→ 動手練習 → 精選 Projects → 自我檢查 +> 🔑 **關鍵名詞**：見 [`resources/glossary.md`](../resources/glossary.md)（framework / agent loop / handoff / supervisor 等收在 2、4） + +你已經從零打造過一個 ReAct agent（Stage 3）。現在來看 framework 到底幫你做了什麼。**挑一個深入學**，其他的瀏覽過去就好，知道什麼時候該換。 + +## 📌 學習目標 + +完成這個 stage 後你會： +- 比較 5 個主流 agent framework（LangGraph、AutoGen、CrewAI、Smolagents、OpenAI Agents SDK） +- 替任務挑出對的 framework +- 用兩個 framework 各做一次同樣的 agent，親身感受差異 +- 看出什麼時候該丟掉 framework、自己寫 + +## 🚪 進入條件 + +你應該已經： +- 跑完 Stage 3 的全部 5 個 hello-X projects +- 從零寫過 ReAct（練習 3） +- 對 async Python 上手（framework 大量依賴 async） + +## 📚 必修閱讀 + +1. [**Anthropic — Building Effective Agents**](https://www.anthropic.com/engineering/building-effective-agents) — 什麼時候用 framework、什麼時候直接用 raw API +2. [**LangChain — Conceptual Guide: Agents**](https://python.langchain.com/docs/concepts/agents/) — agent 的抽象概念 +3. [**Best Multi-Agent Frameworks 2026 comparison**](https://gurusup.com/blog/best-multi-agent-frameworks-2026) — 當前市場定位 +4. **挑一個 framework 的 Quickstart** — 選 LangGraph 或 CrewAI，把官方教學從頭跑到尾 + +## 🤔 什麼是 multi-agent framework？ + +### 兩個維度先分清楚（workflow vs agent / single vs multi） + +要看懂 multi-agent framework 之前、有一個有用的釐清方式——把 **workflow vs agent** 跟 **single vs multi LLM** 當成兩個正交維度。Anthropic「Building Effective Agents」原文的核心區分是 workflow（固定 code path）vs agent（LLM 自主決定 next step）；我們把它跟 single/multi 疊起來看 4 個象限： + +| | **Workflow**
（你寫好的 code path） | **Agent**
（LLM 動態決定下一步） | +|---|---|---| +| **Single LLM** | 線性 pipeline、無分支判斷 | 一個 LLM + ReAct loop、自己 plan + adapt
（**Stage 3 寫的就是這個**） | +| **Multi LLM** | 預設 routing（譬如「銷售問題 → agent A、技術問題 → agent B」） | 2+ agent 互相 handoff、orchestrator 動態分配
（**本 stage 主題**） | + +**為什麼這個區分有用**：production 場景大多落在「single agent workflow」+「single agent」象限——多數任務根本不需要 multi-agent。**真正需要 multi-agent framework 的是右下角象限**——LLM 自主性高 + 多角色協作。但實作上四個象限的邊界有時模糊（LangGraph 的 conditional edge 可以同時看成 workflow routing 跟 agent 動態決策）、不要把這個 matrix 當互斥分類。 + +→ 本 stage 後續討論都假設你已經知道：**Multi-agent framework 主要幫你處理多個 agent 之間的協調、交接、狀態管理與重複性程式碼，讓你不用從零寫整套協作流程**（右下角象限的 orchestration boilerplate）。 + +### Single-agent vs multi-agent — 一張對照表先看清楚差異 + +| 維度 | **Single-agent**（你 Stage 3 寫過了） | **Multi-agent system** | +|---|---|---| +| **架構** | 一個 LLM + ReAct loop + 若干 tools | 2+ LLM、各有角色（researcher / writer / critic ...）、orchestrator 協調 | +| **怎麼決策** | 同一個 LLM 從頭想到尾 | 角色拆分 + handoff、不同 LLM instance 看不同視角 | +| **State 管理** | 線性 message history | shared state / message passing / checkpoint | +| **適合場景** | 邏輯線性、tool < 20-30 個、單一目標 | 任務可分解、需要 perspective diversity、長 workflow、平行化 | +| **Debug 成本** | 低（單一 loop 可以一路 trace） | 高（cross-agent 互動、error propagation 難定位） | +| **Token 成本** | 1x | 通常 **3-10x**（每個 sub-agent 都有自己的 prompt + thinking + tool call）| +| **Latency** | 低 | 高（除非 sub-agent 平行跑） | + +### 什麼時候**真的**需要 multi-agent（不要硬上） + +**Multi-agent 不是 default、是 last resort**。**Anthropic 跟 Cognition 兩家 frontier lab 在 2024-2025 都明白寫過：90% 用例其實不該用 multi-agent。** 硬上會付三個代價：**3-10× token、debug 痛苦、context fragmentation**——context 被切散在多個 agent、彼此看不到全貌。 + +| 立場 | 來源 | 核心論點 | +|---|---|---| +| **Anthropic** | [Building Effective Agents (2024)](https://www.anthropic.com/engineering/building-effective-agents)、[How we built our multi-agent research system (2025)](https://www.anthropic.com/engineering/built-multi-agent-research-system) | 多數場景 simple workflow + single agent 就夠；multi-agent 只在「**研究型 / 並行探索**」任務真的有幫助 | +| **Cognition** | [Don't Build Multi-Agents (2025)](https://cognition.ai/blog/dont-build-multi-agents) | multi-agent 的 context fragmentation 嚴重、shared state 維護痛苦；先窮盡 single-agent + long-context 才考慮 | + +需要 multi-agent 通常是這 4 個信號之一： + +| 信號 | 描述 | 對應 pattern | +|---|---|---| +| **1. 任務天然分解** | 大任務有清楚的子步驟、step-by-step 完成 | Sequential / Planner-Executor | +| **2. Token explosion** | single agent prompt 塞不下所有 tool description / context | Supervisor-Worker（分流給 sub-agent）| +| **3. 角色衝突** | 同一個 LLM 既當 writer 又當 critic 會 self-justify | Debate / Peer review | +| **4. 平行加速** | 3 個 research 子任務同時跑、wall-clock 1/3 | Parallel / Map-Reduce 變種 | + +**4 個信號都不在？** → single agent + 好 prompt + tool use 就夠。**硬上 multi-agent 會付 3-10x token、debug 痛苦、其實不會比較準**。 + +> 💡 **後續閱讀**：到 [Stage 7 但你真的需要 multi-agent 嗎？](07-multi-agent-production.md#-但你真的需要-multi-agent-嗎) 會再帶 production 視角的決策——本節是設計階段的決策、那邊是 deploy 前的最後一次回頭檢查。 + +### Multi-agent 經典 pattern（按複雜度排序） + +> 📝 **跟 Stage 3 經典範式怎麼分**：[Stage 3 的 4 個 paradigm](03-tool-use-and-hello-agent.md#agent-的經典範式thinking-patterns)（CoT / ReAct / Reflection / Planning）是**單一 agent 內部怎麼想**；本節這 5 個 pattern 是**多個 agent 之間怎麼協作**——正交的兩個層。 + +| Pattern | 複雜度 | 什麼樣 | 經典場景 | 代表 framework / paper | +|---|---|---|---|---| +| **1. Routing / Handoff** | ⭐ | agent 之間 1:1 handoff、無中央 orchestrator | customer support routing、context switch | [OpenAI Swarm](https://github.com/openai/swarm)、[OpenAI Agents SDK](https://github.com/openai/openai-agents-python) | +| **2. Sequential**
（Planner → Executor） | ⭐⭐ | planner 規劃多步驟 + executor 執行 | 多步驟自動化、code generation | LangGraph、[ChatDev paper](https://arxiv.org/abs/2307.07924) | +| **3. Parallel**
（平行加速） | ⭐⭐⭐ | N 個 agent 同時跑、結果 aggregate | research / map-reduce 任務、wall-clock 1/N | LangGraph parallel branches、CrewAI parallel tasks。**坑點**：async coordination + partial failure + state merge 一致性 | +| **4. Supervisor-Worker**
（hub-spoke） | ⭐⭐⭐ | 1 主 + N worker、主分配 + 整合 | 任務拆解、報告整合 | LangGraph、AutoGen GroupChat | +| **5. Debate / Society**
（多視角收斂） | ⭐⭐⭐⭐ | 2+ agent 互相 critique 或角色扮演 | research、judgment task、social simulation | AutoGen GroupChat、[CAMEL paper](https://arxiv.org/abs/2303.17760)、[Generative Agents paper](https://arxiv.org/abs/2304.03442) | + +### Claude Code subagent — 另一條 orchestration 路線 + +> **這節跟上面的 5 個 pattern 不同層**：上面 5 個 pattern 是 framework / 自己 code 都能實作的設計選擇；本節介紹的 **Claude Code subagent 是另一個 execution model**（runtime 內建的 orchestration、不寫 framework code）。讀完 5 個 pattern 後、本節讓你知道「multi-agent 還有第二條路」。 + +**Multi-agent 不只有 framework 這條路**。Anthropic 自家的 Claude Code 提供另一個 abstraction 層：[subagent](05-claude-code-ecosystem.md#55--subagentsclaude-code-原生-multi-agent-機制-2025-新功能) — 寫一個 `.claude/agents/.md` 檔就是一個 subagent，**不需要 framework**。 + +跟 framework 路線的根本差異（一句話）：**framework 路線**跨 LLM provider、寫 Python orchestration code、checkpointing / audit trail 完整；**Claude Code subagent** 只在 Claude Code runtime 內、寫 markdown 不寫 code、天生 context 隔離。 + +> 📌 **完整逐維度對照表（啟動方式 / runtime / context 隔離 / provider lock-in / 學習曲線）的 canonical 在 [Stage 5.5 開頭](05-claude-code-ecosystem.md#55--subagentsclaude-code-原生-multi-agent-機制-2025-新功能)**——本 stage 只需知道「multi-agent 還有 Claude Code 原生這第二條路」、逐項實作差異到 5.5 再看。 + +**何時選 subagent 而非 framework**： +- 你已經在用 Claude Code 跑日常工作 +- 任務 context 大、會吃光主 session window（讀整個 codebase 之類） +- 多 subagent 平行（research / write / critic）省 wall-clock 時間 +- 不需要跨 provider migration + +詳細寫法 + 動手練習見 [Stage 5.5](05-claude-code-ecosystem.md#55--subagentsclaude-code-原生-multi-agent-機制-2025-新功能)（**建議先完成 Stage 5.1 Claude Code 基礎再回來看 5.5**——subagent 是 Claude Code 生態的進階功能、需要先熟悉基礎用法）。 + +### Framework 的工作 + +Framework 把上面這 5 個 pattern 的 orchestration boilerplate（roles、handoff、state、retry、checkpoint、HITL pause）抽出來、讓你只寫角色定義跟任務描述。一句話：**framework 是 multi-agent 的腳手架，不是必需品**——簡單情境你自己寫個 dict 跟 for loop 也行（Stage 7 練習 1 就是這樣）。 + +### 📚 想系統化深入？ + +**🇺🇸 學術 paper（影響後續所有 framework 設計）**： +1. [**Anthropic — "Building Effective Agents"**](https://www.anthropic.com/engineering/building-effective-agents) ⭐⭐⭐ — 何時用 workflow 何時用 agent、5 個經典 orchestration pattern。**英文圈 multi-agent 設計入門必讀** +2. [**AutoGen paper (Wu et al. 2023)**](https://arxiv.org/abs/2308.08155) — Microsoft 多 agent 對話框架原 paper +3. [**CAMEL paper (Li et al. 2023)**](https://arxiv.org/abs/2303.17760) — multi-agent role-play 開山之作 +4. [**ChatDev paper (Qian et al. 2023)**](https://arxiv.org/abs/2307.07924) — multi-agent software dev、planner-executor canonical +5. [**Generative Agents paper (Park et al. 2023)**](https://arxiv.org/abs/2304.03442) — 25 個 agent 在 The Sims 互動、社會 simulation + +**🀄 中文系統教材**： +1. [**hello-agents Ch6「框架開發實踐」+ Ch7「構建你的 Agent 框架」**](https://github.com/datawhalechina/hello-agents) ⭐ — 中文圈完整講 framework 開發 + 從零構建。**注意：Ch4「智能體經典範式構建」是 single-agent paradigm（ReAct / Plan-and-Solve / Reflection），不是 multi-agent** +2. [**李宏毅 — 生成式 AI 導論**](https://speech.ee.ntu.edu.tw/~hylee/genai/2024-spring.php) — 中後段有 AI agent / multi-agent 相關集數 + +**Framework 官方 multi-agent docs**： +- [**LangGraph — Multi-Agent Systems**](https://langchain-ai.github.io/langgraph/concepts/multi_agent/) — supervisor / swarm / hierarchical 三種架構官方教學 +- [**Anthropic Cookbook — `customer_service_agent.ipynb`**](https://github.com/anthropics/claude-cookbooks/tree/main/tool_use) — multi-agent orchestration canonical 範例（routing + handoff） +- [**Microsoft AutoGen — Examples**](https://microsoft.github.io/autogen/) — group-chat / debate / peer review pattern 完整範例 + +> 💡 **建議框架學習流程**（5 步）： +> 1. **建立 mental model**（30 min）— 讀 Anthropic Building Effective Agents、把 workflow vs agent 跟 single vs multi 兩維度搞清楚 +> 2. **跑 1 個 framework quickstart**（2-3 hr）— LangGraph 或 CrewAI 二選一、跑官方多 agent 教學 +> 3. **對照 Anthropic Cookbook `customer_service_agent`**（1 hr）— production-style routing + handoff 範例 +> 4. *(可選)* **深入學術側**：挑 paper 1-2 篇看（AutoGen / CAMEL / ChatDev / Generative Agents） +> 5. *(Claude 使用者可選)* **寫一個 subagent 對照**：見 [Stage 5.5](05-claude-code-ecosystem.md#55--subagentsclaude-code-原生-multi-agent-機制-2025-新功能)、跟 framework 路線比較 +> +> **不必把 5 個 paper 全讀完**、挑跟你場景最近的 1-2 個。 + +## 🛠 進階 tool patterns（framework 替你處理掉的東西）⭐ Track B 必看 + +Stage 3 教你寫 single tool / multi-tool selection（手寫 `if/elif/else` 路由）。Framework 把這層抽掉，並加了三種更進階的 tool pattern——**這三個 pattern 都需要 framework 抽象層才寫得乾淨，Stage 3 自己手寫會炸開**： + +| Pattern | 解決什麼問題 | 代表實作 | +|---|---|---| +| **Dynamic tool selection** | 工具 > 30 個時、`tools=[...]` 塞不下 prompt（context 太大、selection 也變差） | [LlamaIndex tool router](https://docs.llamaindex.ai/en/stable/module_guides/deploying/agents/tools/) — embedding-based 路由：先 semantic search 找 top-K tool、只把這 K 個塞進 prompt | +| **Tool composition / chaining** | tool A output → tool B input、不要 LLM 中間 narrative（省 token + 省 latency） | LangGraph `state graph` 直接連接 node、CrewAI `sequential tasks`、Pydantic AI 的 type-safe pipeline | +| **Tool-augmented retrieval** | tool 本身是 RAG search → 回結果再 reason | Stage 6 練習 4 RAG pipeline + Stage 3 練習 2 multi-tool 結合（LangGraph 直接把 retriever 包成 tool node） | + +**📚 深度資源**： +- [**Anthropic — Tool Use best practices**](https://docs.anthropic.com/en/docs/build-with-claude/tool-use/overview) — 官方 tool design guide +- [**LlamaIndex — Tool Router pattern**](https://docs.llamaindex.ai/en/stable/module_guides/deploying/agents/tools/) — Dynamic selection canonical reference +- [**LangGraph — Tool Node**](https://langchain-ai.github.io/langgraph/) — composition graph 寫法 + +> 💡 **Track B 學完本節**：你應該講得出「同一個任務」在 (a) Stage 3 手寫 (b) 本 stage framework 寫 (c) Stage 5.5 Claude subagent 寫 三種路線的差別。這是 Track B 路線「會設計 agent」核心問題。 + +## 🛠 動手練習 + +### 練習 1：同一個 agent、兩個 framework +用以下兩個 framework 各做一次同樣的簡單 agent（搜尋 + 摘要）： +- LangGraph +- CrewAI +比較程式碼行數、debug 體驗、以及它們各自把哪些複雜度藏在哪裡。 + +### 練習 2：多 agent 角色分配 +用 CrewAI 做一個 2-3 個 agent、各自有不同角色一起完成同一個任務的 demo。（這種情境 CrewAI 最拿手。） + +### 練習 3：圖式 workflow +用 LangGraph 做一個有分支邏輯跟 human-in-the-loop checkpoint 的 workflow。（這種情境 LangGraph 最拿手。） + +### 練習 4：CodeAct vs JSON tool +用 Smolagents 做一個會寫 Python 程式碼當作 action 的 agent（CodeAct pattern），跟 練習 1 用的 JSON tool call 路線比較。問同一個問題，看兩種路線怎麼解。 + +### 練習 5：型別安全 agent +用 Pydantic AI 做一個會回傳結構化輸出的 agent（例如：問問題回 `{ "answer": str, "confidence": float, "sources": [str] }`）。看 Pydantic 的 schema validation 怎麼防止 agent 偷懶或 hallucinate 結構。 + +## 🎯 精選 Projects + +按用途分 5 類、16 個項目一張表搞定。**挑入口看「適合誰」、想深入點連結看 repo / quickstart**。 + +| 分類 | Project | ⭐ | 適合誰 | 為什麼推薦 / 備註 | +|---|---|---|---|---| +| **Production 級**
（複雜 multi-agent / 需要 audit） | [LangGraph](https://github.com/langchain-ai/langgraph) ⭐ **本 stage 推薦 #1** | ⭐⭐⭐⭐⭐ | Production multi-agent + 稽核軌跡 / rollback / replay | 圖式 orchestration + checkpointing + time-travel debug、企業廣泛採用，★ 34k+、MIT、Python+TS。搭 LangSmith 做 observability | +| | [microsoft/semantic-kernel](https://github.com/microsoft/semantic-kernel) | ⭐⭐⭐⭐ | 在 .NET / Java 環境做 agent、Microsoft 技術棧 | C# / Python / Java 三語官方 SDK、kernel + plugin + planner pattern，★ 27k+、MIT。抽象厚、不適合初學者 | +| | [agno-agi/agno](https://github.com/agno-agi/agno) | ⭐⭐⭐⭐ | 要「build + serve + monitor」一條龍但不想全套 LangGraph + LangSmith | multi-modal agent runtime + control plane，★ 39k+、Apache-2.0。Stage 4 學 API、Stage 7 用 runtime | +| | [microsoft/agent-framework](https://github.com/microsoft/agent-framework) | ⭐⭐⭐⭐ | 想用微軟整合 AutoGen + Semantic Kernel 的後繼框架（Python 或 .NET）的團隊 | 微軟開源框架、讓多個 agent 一起協作——合併 AutoGen 與 Semantic Kernel 的後繼者，★ 11k+、MIT、Python + .NET | +| **快速雛形 / 多 agent**
（role-based / handoff） | [CrewAI](https://github.com/crewAIInc/crewAI) ⭐ **本 stage 推薦 #2** | ⭐⭐⭐⭐ | 快速雛形「researcher → writer → critic」pipeline | ~20 行寫完 crew、學習曲線最低，★ 50k+、MIT。⚠️ 長 workflow 沒 checkpointing；雛形用 CrewAI、production 用 LangGraph | +| | [Microsoft AutoGen / AG2](https://github.com/microsoft/autogen) | ⭐⭐⭐⭐ | 多 agent 辯論 / 腦力激盪 / peer review pattern | 對話式多 agent、group-chat 強，★ 57k+、CC-BY-4.0（文件 license）。⚠️ AG2 v0.4 重寫成 async-first、舊教學多半還在 v0.2、新教學用 v0.4、留意版本分支 | +| | [OpenAI Agents SDK](https://github.com/openai/openai-agents-python) | ⭐⭐⭐⭐⭐ | 已 commit OpenAI 生態 | OpenAI 官方、agent hand-off + 結構化輸出、API 乾淨、MIT。**2026-04 起內建** sandbox（7 個 provider）+ harness 抽象層、production coding agent 首次 architecturally sound（[詳見 Stage 8](08-agent-interfaces.md#openai-agents-sdk-april-2026-更新--why-是-milestone)）| +| | [deepagents (LangChain)](https://github.com/langchain-ai/deepagents) | ⭐⭐⭐⭐ | 想要「開箱即用」的 deep agent 骨架、不想自己拼 | LangGraph 之上的 opinionated harness（有主見的骨架、替你做好設計選擇）：內建規劃（todo）、檔案系統記憶、子 agent、可載入 skills、context offload（把龐大的中間結果挪出 LLM 視窗）。`pip install deepagents`、MIT、v0.6.12（2026-06）| +| | [OpenAI Swarm](https://github.com/openai/swarm) | ⭐⭐⭐⭐ 教育用
⭐⭐⭐ production | 想理解 multi-agent **核心 mental model** 但不想學整套 framework | ~200 LOC、只有 Agent + handoff 兩個觀念、MIT。⚠️ OpenAI 自己標 experimental / educational、不是 production tool。**讀 source 當 chapter-length 教材** | +| | [Strands Agents (AWS)](https://github.com/strands-agents/sdk-python) | ⭐⭐⭐⭐ | 已 commit AWS 雲、Bedrock-native | model-driven 設計（LLM 自己 plan、無 explicit graph）、Apache 2.0。2025 後段推出、AWS Lambda / Step Functions / Bedrock Agents 整合 | +| **特殊路線**
（CodeAct / typed / memory-first） | [Hugging Face Smolagents](https://github.com/huggingface/smolagents) | ⭐⭐⭐⭐ | 本地 LLM 生態、HF 整合場景 | CodeAct pattern 代表（agent 寫 Python 程式碼當 action、非 JSON tool call），★ 27k+、Apache 2.0、≤1000 LOC | +| | [Pydantic AI](https://github.com/pydantic/pydantic-ai) | ⭐⭐⭐ | production 預設要 runtime 型別安全 + structured output | type-safe agent、Pydantic 團隊出品、MIT。較新 | +| | [Letta (formerly MemGPT)](https://github.com/letta-ai/letta) | ⭐⭐⭐⭐ | **長 session / 跨 day / persona-stable** agent（long-term assistant、therapist、tutor）| memory-first multi-agent、OS-paging 概念（working memory + archival store），★ 22k+、Apache 2.0。Stage 6 練習 5 也會提 | +| **特化** | [LlamaIndex Agents](https://github.com/run-llama/llama_index) | ⭐⭐⭐ | 文件密集型 agent（研究助理、知識工作者類） | 跟 RAG 緊整合，★ 49k+、MIT。retrieval 強、orchestration 弱——純 orchestration 別選 | +| | [agentscope-ai/agentscope](https://github.com/agentscope-ai/agentscope) | ⭐⭐⭐ | 想要視覺化 debug 多 agent 流程的研究者 | 多 agent 平台、視覺化 debug 工具強，★ 26k+、Apache 2.0。西方社群採用低、技術紮實 | +| | [LangChain](https://github.com/langchain-ai/langchain) | ⭐⭐⭐ | 需要黏合很多零件（retrieval + chain）的快速雛形 | 萬用工具袋 framework，★ 135k+、MIT。**agent orchestration 改用 LangGraph**、LangChain 適合 retrieval + chaining 黏合 | +| **基礎設施**
（不是 framework、跨 stage 用） | [BerriAI/litellm](https://github.com/BerriAI/litellm) | ⭐⭐⭐⭐ | 要切換 Claude / GPT / Gemini / 開源模型但不想改 code | provider-agnostic SDK + AI gateway、用 OpenAI 形狀 call 100+ LLM、附 cost tracking / fallback / guardrail，★ 49k+、MIT（`enterprise/` 子目錄另授權）| + +> 💡 **建議閱讀路徑**：挑 **1 個 production 等級**（LangGraph）+ **1 個快速雛形**（CrewAI）深入學 → 跑練習 1-3 → 其他 framework README 瀏覽過去、知道存在即可。**特殊路線那 3 個**（CodeAct / typed / memory-first）在特定場景才有對手、平常不必碰。 + +## ✅ 進 Stage 5 前的自我檢查 + +你能不能： + +- [ ] 用 LangGraph 跟 CrewAI 各做一次同一個 agent +- [ ] 替任務挑出對的 framework（production vs 雛形） +- [ ] 解釋 LangGraph 的 checkpoint 跟 CrewAI 的 task delegation 差在哪 +- [ ] 看出什麼時候 CodeAct（Smolagents）比 JSON-tool 更好 +- [ ] 判斷什麼時候該丟掉 framework、直接用 raw API + +如果可以 → 進 [Stage 5 — Claude Code Ecosystem](05-claude-code-ecosystem.md)。 + +## 💡 策略提示 + 過程中可能踩到的坑 + +不要想把這些全部學完。挑**一個 production 等級的（LangGraph）**跟**一個快速雛形用的（CrewAI）**深入學。其他的 README 瀏覽過去就好，知道有這些選項存在即可。 + +**Memory 預備**（學的時候可能碰到、不用先讀）：有些 framework 功能會用到 memory 概念 — LangGraph 的 checkpointing（狀態持久化）、CrewAI agent 之間傳遞任務結果（輕量 memory）。這些在 [Stage 6 — Memory & RAG](06-memory-rag.md) 完整講；本 stage 看不懂某個 framework 功能時、再去那邊查就好，**不用先讀完才能繼續本 stage**。 diff --git a/stages/04-agent-frameworks.zh-Hans.md b/stages/04-agent-frameworks.zh-Hans.md new file mode 100644 index 0000000..8be406a --- /dev/null +++ b/stages/04-agent-frameworks.zh-Hans.md @@ -0,0 +1,225 @@ +# Stage 4 — Agent 框架（Agent Frameworks） + +> [繁體中文](./04-agent-frameworks.md) | **简体中文** | [English](./04-agent-frameworks.en.md) + +⏱ **时间估算**：2-3 周（约 10-15 小时） + +> 💡 用语不熟（framework / supervisor / worker / handoff⋯）→ 翻 [`resources/glossary.md`](../resources/glossary.md)。 + +> 📋 **本章组成**：学习目标 → 进入条件 → 必读 →〔可选 · 概念地图：multi-agent intro + 进阶 tool patterns〕→ 动手练习 → 精选 Projects → 自我检查 +> 🔑 **关键名词**：见 [`resources/glossary.md`](../resources/glossary.md)（framework / agent loop / handoff / supervisor 等收在 2、4） + +你已经从零打造过一个 ReAct agent（Stage 3）。现在来看 framework 到底帮你做了什么。**挑一个深入学**，其他的浏览过去就好，知道什么时候该换。 + +## 📌 学习目标 + +完成这个 stage 后你会： +- 比较 5 个主流 agent framework（LangGraph、AutoGen、CrewAI、Smolagents、OpenAI Agents SDK） +- 替任务挑出对的 framework +- 用两个 framework 各做一次同样的 agent，亲身感受差异 +- 看出什么时候该丢掉 framework、自己写 + +## 🚪 进入条件 + +你应该已经： +- 跑完 Stage 3 的全部 5 个 hello-X projects +- 从零写过 ReAct（练习 3） +- 对 async Python 上手（framework 大量依赖 async） + +## 📚 必读 + +1. [**Anthropic — Building Effective Agents**](https://www.anthropic.com/engineering/building-effective-agents) — 什么时候用 framework、什么时候直接用 raw API +2. [**LangChain — Conceptual Guide: Agents**](https://python.langchain.com/docs/concepts/agents/) — agent 的抽象概念 +3. [**Best Multi-Agent Frameworks 2026 comparison**](https://gurusup.com/blog/best-multi-agent-frameworks-2026) — 当前市场定位 +4. **挑一个 framework 的 Quickstart** — 选 LangGraph 或 CrewAI，把官方教学从头跑到尾 + +## 🤔 什么是 multi-agent framework？ + +### 两个维度先分清楚（workflow vs agent / single vs multi） + +要看懂 multi-agent framework 之前、有一个有用的厘清方式——把 **workflow vs agent** 跟 **single vs multi LLM** 当成两个正交维度。Anthropic“Building Effective Agents”原文的核心区別是 workflow（固定 code path）vs agent（LLM 自主决定 next step）；我们把它跟 single/multi 叠起来看 4 个象限： + +| | **Workflow**
（你写好的 code path） | **Agent**
（LLM 动态决定下一步） | +| -------------- | ---------------------------------------------------------------------- | --------------------------------------------------------------------------------- | +| **Single LLM** | 线性 pipeline、无分支判断 | 一个 LLM + ReAct loop、自己 plan + adapt
（**Stage 3 写的就是这个**） | +| **Multi LLM** | 预设 routing（譬如“销售问题 → agent A、技术问题 → agent B”） | 2+ agent 互相 handoff、orchestrator 动态分配
（**本 stage 主题**） | + +**为什么这个区別有用**：production 场景大多落在“single agent workflow”+“single agent”象限——多数任务根本不需要 multi-agent。**真正需要 multi-agent framework 的是右下角象限**——LLM 自主性高 + 多角色协作。但实作上四个象限的边界有时模糊（LangGraph 的 conditional edge 可以同时看成 workflow routing 跟 agent 动态决策）、不要把这个 matrix 当互斥分类。 + +→ 本 stage 后续讨论都假设你已经知道：**Multi-agent framework 主要帮你处理多个 agent 之间的协调、交接、状态管理与重复性样板代码，让你不用从零写整套协作流程**（右下角象限的 orchestration boilerplate）。 + +### Single-agent vs multi-agent — 一张对照表先看清楚差异 + +| 维度 | **Single-agent**（你 Stage 3 写过了） | **Multi-agent system** | +| ------------ | --------------------------------------------- | --------------------------------------------------------------------------- | +| **架构** | 一个 LLM + ReAct loop + 若干 tools | 2+ LLM、各有角色（researcher / writer / critic ...）、orchestrator 协调 | +| **怎么决策** | 同一个 LLM 从头想到尾 | 角色拆分 + handoff、不同 LLM instance 看不同视角 | +| **State 管理** | 线性 message history | shared state / message passing / checkpoint | +| **适合场景** | 逻辑线性、tool < 20-30 个、单一目标 | 任务可分解、需要 perspective diversity、长 workflow、并行化 | +| **Debug 成本** | 低（单一 loop 可以一路 trace） | 高（cross-agent 互动、error propagation 难定位） | +| **Token 成本** | 1x | 通常 **3-10x**（每个 sub-agent 都有自己的 prompt + thinking + tool call） | +| **Latency** | 低 | 高（除非 sub-agent 并行跑） | + +### 什么时候**真的**需要 multi-agent（不要硬上） + +**Multi-agent 不是 default、是 last resort**。**Anthropic 跟 Cognition 两家 frontier lab 在 2024-2025 都明白写过：90% 用例其实不该用 multi-agent。** 硬上会付三个代价：**3-10× token、debug 痛苦、context fragmentation**——context 被切散在多个 agent、彼此看不到全貌。 + +| 立场 | 来源 | 核心论点 | +| ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | +| **Anthropic** | [Building Effective Agents (2024)](https://www.anthropic.com/engineering/building-effective-agents)、[How we built our multi-agent research system (2025)](https://www.anthropic.com/engineering/built-multi-agent-research-system) | 多数场景 simple workflow + single agent 就够；multi-agent 只在“**研究型 / 并行探索**”任务真的有帮助 | +| **Cognition** | [Don't Build Multi-Agents (2025)](https://cognition.ai/blog/dont-build-multi-agents) | multi-agent 的 context fragmentation 严重、shared state 维护痛苦；先穷尽 single-agent + long-context 才考虑 | + +需要 multi-agent 通常是这 4 个信号之一： + +| 信号 | 描述 | 对应 pattern | +| ------------------- | ---------------------------------------------------------- | ---------------------------------- | +| **1. 任务天然分解** | 大任务有清楚的子步骤、step-by-step 完成 | Sequential / Planner-Executor | +| **2. Token explosion**| single agent prompt 塞不下所有 tool description / context | Supervisor-Worker（分流给 sub-agent）| +| **3. 角色冲突** | 同一个 LLM 既当 writer 又当 critic 会 self-justify | Debate / Peer review | +| **4. 并行加速** | 3 个 research 子任务同时跑、wall-clock 1/3 | Parallel / Map-Reduce 变种 | + +**4 个信号都不在？** → single agent + 好 prompt + tool use 就够。**硬上 multi-agent 会付 3-10x token、debug 痛苦、其实不会比较准**。 + +> 💡 **后续阅读**：到 [Stage 7 但你真的需要 multi-agent 吗？](07-multi-agent-production.zh-Hans.md#-但你真的需要-multi-agent-吗) 会再带 production 视角的决策——本节是设计阶段的决策、那边是 deploy 前的最后一次回头检查。 + +### Multi-agent 经典 pattern（按复杂度排序） + +> 📝 **跟 Stage 3 经典范式怎么分**：[Stage 3 的 4 个 paradigm](03-tool-use-and-hello-agent.zh-Hans.md#agent-的经典范式thinking-patterns)（CoT / ReAct / Reflection / Planning）是**单一 agent 内部怎么想**；本节这 5 个 pattern 是**多个 agent 之间怎么协作**——正交的两个层。 + +| Pattern | 复杂度 | 什么样 | 经典场景 | 代表 framework / paper | +| -------------------------------- | -------- | ------------------------------------------------------ | -------------------------------------------- | ----------------------------------------------------------------------------------------------- | +| **1. Routing / Handoff** | ⭐ | agent 之间 1:1 handoff、无中央 orchestrator | customer support routing、context switch | [OpenAI Swarm](https://github.com/openai/swarm)、[OpenAI Agents SDK](https://github.com/openai/openai-agents-python) | +| **2. Sequential**
（Planner → Executor） | ⭐⭐ | planner 规划多步骤 + executor 执行 | 多步骤自动化、code generation | LangGraph、[ChatDev paper](https://arxiv.org/abs/2307.07924) | +| **3. Parallel**
（并行加速） | ⭐⭐⭐ | N 个 agent 同时跑、结果 aggregate | research / map-reduce 任务、wall-clock 1/N | LangGraph parallel branches、CrewAI parallel tasks。**坑点**：async coordination + partial failure + state merge 一致性 | +| **4. Supervisor-Worker**
（hub-spoke） | ⭐⭐⭐ | 1 主 + N worker、主分配 + 整合 | 任务拆解、报告整合 | LangGraph、AutoGen GroupChat | +| **5. Debate / Society**
（多视角收敛） | ⭐⭐⭐⭐ | 2+ agent 互相 critique 或角色扮演 | research、judgment task、social simulation | AutoGen GroupChat、[CAMEL paper](https://arxiv.org/abs/2303.17760)、[Generative Agents paper](https://arxiv.org/abs/2304.03442) | + +### Claude Code subagent — 另一条 orchestration 路线 + +> **这节跟上面的 5 个 pattern 不同层**：上面 5 个 pattern 是 framework / 自己 code 都能实作的设计选择；本节介绍的 **Claude Code subagent 是另一个 execution model**（runtime 内建的 orchestration、不写 framework code）。读完 5 个 pattern 后、本节让你知道“multi-agent 还有第二条路”。 + +**Multi-agent 不只有 framework 这条路**。Anthropic 自家的 Claude Code 提供另一个 abstraction 层：[subagent](05-claude-code-ecosystem.zh-Hans.md#55--subagentsclaude-code-原生-multi-agent-机制-2025-新功能) — 写一个 `.claude/agents/.md` 档就是一个 subagent，**不需要 framework**。 + +跟 framework 路线的根本差异（一句话）：**framework 路线**跨 LLM provider、写 Python orchestration code、checkpointing / audit trail 完整；**Claude Code subagent** 只在 Claude Code runtime 内、写 markdown 不写 code、天生 context 隔离。 + +> 📌 **完整逐维度对照表（启动方式 / runtime / context 隔离 / provider lock-in / 学习曲线）的 canonical 在 [Stage 5.5 开头](05-claude-code-ecosystem.zh-Hans.md#55--subagentsclaude-code-原生-multi-agent-机制-2025-新功能)**——本 stage 只需知道“multi-agent 还有 Claude Code 原生这第二条路”、逐项实作差异到 5.5 再看。 + +**何时选 subagent 而非 framework**： +- 你已经在使用 Claude Code 跑日常工作 +- 任务 context 大、会吃光主 session window（读整个 codebase 之类） +- 多 subagent 并行（research / write / critic）省 wall-clock 时间 +- 不需要跨 provider migration + +详细写法 + 动手练习见 [Stage 5.5](05-claude-code-ecosystem.zh-Hans.md#55--subagentsclaude-code-原生-multi-agent-机制-2025-新功能)（**建议先完成 Stage 5.1 Claude Code 基础再回来看 5.5**——subagent 是 Claude Code 生态的进阶功能、需要先熟悉基础用法）。 + +### Framework 的工作 + +Framework 把上面这 5 个 pattern 的 orchestration boilerplate（roles、handoff、state、retry、checkpoint、HITL pause）抽出来、让你只写角色定义跟任务描述。一句话：**framework 是 multi-agent 的脚手架，不是必需品**——简单情境你自己写个 dict 跟 for loop 也行（Stage 7 练习 1 就是这样）。 + +### 📚 想系统化深入？ + +**🇺🇸 学术 paper（影响后续所有 framework 设计）**： +1. [**Anthropic — "Building Effective Agents"**](https://www.anthropic.com/engineering/building-effective-agents) ⭐⭐⭐ — 何时用 workflow 何时用 agent、5 个经典 orchestration pattern。**英文圈 multi-agent 设计入门必读** +2. [**AutoGen paper (Wu et al. 2023)**](https://arxiv.org/abs/2308.08155) — Microsoft 多 agent 对话框架原 paper +3. [**CAMEL paper (Li et al. 2023)**](https://arxiv.org/abs/2303.17760) — multi-agent role-play 开山之作 +4. [**ChatDev paper (Qian et al. 2023)**](https://arxiv.org/abs/2307.07924) — multi-agent software dev、planner-executor canonical +5. [**Generative Agents paper (Park et al. 2023)**](https://arxiv.org/abs/2304.03442) — 25 个 agent 在 The Sims 互动、社会 simulation + +**🀄 中文系统教材**： +1. [**hello-agents Ch6“框架开发实践”+ Ch7“构建你的 Agent 框架”**](https://github.com/datawhalechina/hello-agents) ⭐ — 中文圈完整讲 framework 开发 + 从零构建。**注意：Ch4“智能体经典范式构建”是 single-agent paradigm（ReAct / Plan-and-Solve / Reflection），不是 multi-agent** +2. [**李宏毅 — 生成式 AI 导论**](https://speech.ee.ntu.edu.tw/~hylee/genai/2024-spring.php) — 中后段有 AI agent / multi-agent 相关集数 + +**Framework 官方 multi-agent docs**： +- [**LangGraph — Multi-Agent Systems**](https://langchain-ai.github.io/langgraph/concepts/multi_agent/) — supervisor / swarm / hierarchical 三种架构官方教学 +- [**Anthropic Cookbook — `customer_service_agent.ipynb`**](https://github.com/anthropics/claude-cookbooks/tree/main/tool_use) — multi-agent orchestration canonical 范例（routing + handoff） +- [**Microsoft AutoGen — Examples**](https://microsoft.github.io/autogen/) — group-chat / debate / peer review pattern 完整范例 + +> 💡 **建议框架学习流程**（5 步）： +> 1. **建立 mental model**（30 min）— 读 Anthropic Building Effective Agents、把 workflow vs agent 跟 single vs multi 两维度搞清楚 +> 2. **跑 1 个 framework quickstart**（2-3 hr）— LangGraph 或 CrewAI 二选一、跑官方多 agent 教学 +> 3. **对照 Anthropic Cookbook `customer_service_agent`**（1 hr）— production-style routing + handoff 范例 +> 4. *(可选)* **深入学术侧**：挑 paper 1-2 篇看（AutoGen / CAMEL / ChatDev / Generative Agents） +> 5. *(Claude 用户可选)* **写一个 subagent 对照**：见 [Stage 5.5](05-claude-code-ecosystem.zh-Hans.md#55--subagentsclaude-code-原生-multi-agent-机制-2025-新功能)、跟 framework 路线比较 +> +> **不必把 5 个 paper 全读完**、挑跟你场景最近的 1-2 个。 + +## 🛠 进阶 tool patterns（framework 替你处理掉的东西）⭐ Track B 必看 + +Stage 3 教你写 single tool / multi-tool selection（手写 `if/elif/else` 路由）。Framework 把这层抽掉，并加了三种更进阶的 tool pattern——**这三个 pattern 都需要 framework 抽象层才写得干净，Stage 3 自己手写会炸开**： + +| Pattern | 解决什么问题 | 代表实作 | +| ---------------------------- | ---------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | +| **Dynamic tool selection** | 工具 > 30 个时、`tools=[...]` 塞不下 prompt（context 太大、selection 也变差） | [LlamaIndex tool router](https://docs.llamaindex.ai/en/stable/module_guides/deploying/agents/tools/) — embedding-based 路由：先 semantic search 找 top-K tool、只把这 K 个塞进 prompt | +| **Tool composition / chaining**| tool A output → tool B input、不要 LLM 中间 narrative（省 token + 省 latency） | LangGraph `state graph` 直接连接 node、CrewAI `sequential tasks`、Pydantic AI 的 type-safe pipeline | +| **Tool-augmented retrieval** | tool 本身是 RAG search → 回结果再 reason | Stage 6 练习 4 RAG pipeline + Stage 3 练习 2 multi-tool 结合（LangGraph 直接把 retriever 包成 tool node） | + +**📚 深度资源**： +- [**Anthropic — Tool Use best practices**](https://docs.anthropic.com/en/docs/build-with-claude/tool-use/overview) — 官方 tool design guide +- [**LlamaIndex — Tool Router pattern**](https://docs.llamaindex.ai/en/stable/module_guides/deploying/agents/tools/) — Dynamic selection canonical reference +- [**LangGraph — Tool Node**](https://langchain-ai.github.io/langgraph/) — composition graph 写法 + +> 💡 **Track B 学完本节**：你应该讲得出“同一个任务”在 (a) Stage 3 手写 (b) 本 stage framework 写 (c) Stage 5.5 Claude subagent 写 三种路线的差别。这是 Track B 路线“会设计 agent”核心问题。 + +## 🛠 动手练习 + +### 练习 1：同一个 agent、两个 framework +用以下两个 framework 各做一次同样的简单 agent（搜索 + 摘要）： +- LangGraph +- CrewAI +比较代码行数、debug 体验、以及它们各自把哪些复杂度藏在哪里。 + +### 练习 2：多 agent 角色分配 +用 CrewAI 做一个 2-3 个 agent、各自有不同角色一起完成同一个任务的 demo。（这种情境 CrewAI 最拿手。） + +### 练习 3：图式 workflow +用 LangGraph 做一个有分支逻辑跟 human-in-the-loop checkpoint 的 workflow。（这种情境 LangGraph 最拿手。） + +### 练习 4：CodeAct vs JSON tool +用 Smolagents 做一个会写 Python 代码当作 action 的 agent（CodeAct pattern），跟 练习 1 用的 JSON tool call 路线比较。问同一个问题，看两种路线怎么解。 + +### 练习 5：类型安全 agent +用 Pydantic AI 做一个会返回结构化输出的 agent（例如：问问题回 `{ "answer": str, "confidence": float, "sources": [str] }`）。看 Pydantic 的 schema validation 怎么防止 agent 偷懒或 hallucinate 结构。 + +## 🎯 精选 Projects + +按用途分 5 类、16 个项目一张表搞定。**挑入口看“适合谁”、想深入点链接看 repo / quickstart**。 + +| 分类 | Project | ⭐ | 适合谁 | 为什么推荐 / 备注 | +| ------------------------------------------ | ---------------------------------------------------------------------------------- | ----------- | ------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Production 级**
（复杂 multi-agent / 需要 audit） | [LangGraph](https://github.com/langchain-ai/langgraph) ⭐ **本 stage 推荐 #1** | ⭐⭐⭐⭐⭐ | Production multi-agent + 稽核轨迹 / rollback / replay | 图式 orchestration + checkpointing + time-travel debug、企业广泛采用，★ 34k+、MIT、Python+TS。搭 LangSmith 做 observability | +| | [microsoft/semantic-kernel](https://github.com/microsoft/semantic-kernel) | ⭐⭐⭐⭐ | 在 .NET / Java 环境做 agent、Microsoft 技术栈 | C# / Python / Java 三语官方 SDK、kernel + plugin + planner pattern，★ 27k+、MIT。抽象厚、不适合初学者 | +| | [agno-agi/agno](https://github.com/agno-agi/agno) | ⭐⭐⭐⭐ | 要“build + serve + monitor”一条龙但不想全套 LangGraph + LangSmith | multi-modal agent runtime + control plane，★ 39k+、Apache-2.0。Stage 4 学 API、Stage 7 用 runtime | +| | [microsoft/agent-framework](https://github.com/microsoft/agent-framework) | ⭐⭐⭐⭐ | 想用微软整合 AutoGen + Semantic Kernel 的后继框架（Python 或 .NET）的团队 | 微软开源框架、让多个 agent 一起协作——合并 AutoGen 与 Semantic Kernel 的后继者，★ 11k+、MIT、Python + .NET | +| **快速雏形 / 多 agent**
（role-based / handoff） | [CrewAI](https://github.com/crewAIInc/crewAI) ⭐ **本 stage 推荐 #2** | ⭐⭐⭐⭐ | 快速雏形“researcher → writer → critic”pipeline | ~20 行写完 crew、学习曲线最低，★ 50k+、MIT。⚠️ 长 workflow 没 checkpointing；雏形用 CrewAI、production 用 LangGraph | +| | [Microsoft AutoGen / AG2](https://github.com/microsoft/autogen) | ⭐⭐⭐⭐ | 多 agent 辩论 / 脑力激荡 / peer review pattern | 对话式多 agent、group-chat 强，★ 57k+、CC-BY-4.0（文件 license）。⚠️ AG2 v0.4 重写成 async-first、旧教学多半还在 v0.2、新教学用 v0.4、留意版本分支 | +| | [OpenAI Agents SDK](https://github.com/openai/openai-agents-python) | ⭐⭐⭐⭐⭐ | 已 commit OpenAI 生态 | OpenAI 官方、agent hand-off + 结构化输出、API 干净、MIT。**2026-04 起内建** sandbox（7 个 provider）+ harness 抽象层、production coding agent 首次 architecturally sound（[详见 Stage 8](08-agent-interfaces.zh-Hans.md#openai-agents-sdk-2026-年-4-月更新--为何是里程碑)） | +| | [deepagents (LangChain)](https://github.com/langchain-ai/deepagents) | ⭐⭐⭐⭐ | 想要“开箱即用”的 deep agent 骨架、不想自己拼 | LangGraph 之上的 opinionated harness（有主见的骨架、替你做好设计选择）：内建规划（todo）、文件系统记忆、子 agent、可加载 skills、context offload（把庞大的中间结果挪出 LLM 视窗）。`pip install deepagents`、MIT、v0.6.12（2026-06）| +| | [OpenAI Swarm](https://github.com/openai/swarm) | ⭐⭐⭐⭐ 教育用
⭐⭐⭐ production | 想理解 multi-agent **核心 mental model** 但不想学整套 framework | ~200 LOC、只有 Agent + handoff 两个观念、MIT。⚠️ OpenAI 自己标 experimental / educational、不是 production tool。**读 source 当 chapter-length 教材** | +| | [Strands Agents (AWS)](https://github.com/strands-agents/sdk-python) | ⭐⭐⭐⭐ | 已 commit AWS 云、Bedrock-native | model-driven 设计（LLM 自己 plan、无 explicit graph）、Apache 2.0。2025 后段推出、AWS Lambda / Step Functions / Bedrock Agents 整合 | +| **特殊路线**
（CodeAct / typed / memory-first） | [Hugging Face Smolagents](https://github.com/huggingface/smolagents) | ⭐⭐⭐⭐ | 本地 LLM 生态、HF 整合场景 | CodeAct pattern 代表（agent 写 Python 代码当作 action、非 JSON tool call），★ 27k+、Apache 2.0、≤1000 LOC | +| | [Pydantic AI](https://github.com/pydantic/pydantic-ai) | ⭐⭐⭐ | production 预设要 runtime 类型安全 + structured output | type-safe agent、Pydantic 团队出品、MIT。较新 | +| | [Letta (formerly MemGPT)](https://github.com/letta-ai/letta) | ⭐⭐⭐⭐ | **长 session / 跨 day / persona-stable** agent（long-term assistant、therapist、tutor） | memory-first multi-agent、OS-paging 概念（working memory + archival store），★ 22k+、Apache 2.0。Stage 6 练习 5 也会提 | +| **特化** | [LlamaIndex Agents](https://github.com/run-llama/llama_index) | ⭐⭐⭐ | 文件密集型 agent（研究助理、知识工作者类） | 跟 RAG 紧整合，★ 49k+、MIT。retrieval 强、orchestration 弱——纯 orchestration 别选 | +| | [agentscope-ai/agentscope](https://github.com/agentscope-ai/agentscope) | ⭐⭐⭐ | 想要可视化 debug 多 agent 流程的研究者 | 多 agent 平台、可视化 debug 工具强，★ 26k+、Apache 2.0。西方社群采用低、技术扎实 | +| | [LangChain](https://github.com/langchain-ai/langchain) | ⭐⭐⭐ | 需要黏合很多零件（retrieval + chain）的快速雏形 | 万用工具袋 framework，★ 135k+、MIT。**agent orchestration 改用 LangGraph**、LangChain 适合 retrieval + chaining 黏合 | +| **基础设施**
（不是 framework、跨 stage 用） | [BerriAI/litellm](https://github.com/BerriAI/litellm) | ⭐⭐⭐⭐ | 要切换 Claude / GPT / Gemini / 开源模型但不想改 code | provider-agnostic SDK + AI gateway、用 OpenAI 形状 call 100+ LLM、附 cost tracking / fallback / guardrail，★ 49k+、MIT（`enterprise/` 子目录另授权） | + +> 💡 **建议阅读路径**：挑 **1 个 production 等级**（LangGraph）+ **1 个快速雏形**（CrewAI）深入学 → 跑练习 1-3 → 其他 framework README 浏览过去、知道存在即可。**特殊路线那 3 个**（CodeAct / typed / memory-first）在特定场景才有对手、平常不必碰。 + +## ✅ 进 Stage 5 前的自我检查 + +你能不能： + +- [ ] 用 LangGraph 跟 CrewAI 各做一次同一个 agent +- [ ] 替任务挑出对的 framework（production vs 雏形） +- [ ] 解释 LangGraph 的 checkpoint 跟 CrewAI 的 task delegation 差在哪 +- [ ] 看出什么时候 CodeAct（Smolagents）比 JSON-tool 更好 +- [ ] 判断什么时候该丢掉 framework、直接用 raw API + +如果可以 → 进 [Stage 5 — Claude Code Ecosystem](05-claude-code-ecosystem.zh-Hans.md)。 + +## 💡 策略提示 + 过程中可能踩到的坑 + +不要想把这些全部学完。挑**一个 production 等级的（LangGraph）**跟**一个快速雏形用的（CrewAI）**深入学。其他的 README 浏览过去就好，知道有这些选项存在即可。 + +**Memory 预备**（学的时候可能碰到、不用先读）：有些 framework 功能会用到 memory 概念 — LangGraph 的 checkpointing（状态持久化）、CrewAI agent 之间传递任务结果（轻量 memory）。这些在 [Stage 6 — Memory & RAG](06-memory-rag.zh-Hans.md) 完整讲；本 stage 看不懂某个 framework 功能时、再去那边查就好，**不用先读完才能继续本 stage**。 diff --git a/stages/05-claude-code-ecosystem.en.md b/stages/05-claude-code-ecosystem.en.md new file mode 100644 index 0000000..0494f49 --- /dev/null +++ b/stages/05-claude-code-ecosystem.en.md @@ -0,0 +1,960 @@ +# Stage 5 — Claude Code Ecosystem ⭐⭐ + +> [繁體中文](./05-claude-code-ecosystem.md) | [简体中文](./05-claude-code-ecosystem.zh-Hans.md) | **English** + +⏱ **Time estimate**: 3-4 weeks (~15-25 hours) + +> 🚪 **Entry condition** (shared hub — differs by track): **Track A (CLI Power User)** arrives from A1-A2 — knowing Python + having run a basic CLI is enough; start from 5.1/5.2. **Track B (Agent Builder)** should first complete [Stage 3](03-tool-use-and-hello-agent.en.md) (tool use) + [Stage 4](04-agent-frameworks.en.md) (agent frameworks), then read this whole stage as "how Claude Code works internally". Not sure which track? → see the 📌 two-track note below. + +> 💡 This entire stage revolves around 4 keywords (**MCP / Skills / Plugins / Marketplace**) → if you're not familiar with them, first check out [`resources/glossary.en.md` 5](../resources/glossary.en.md#5-claude-code-ecosystem). + +**👥 Shared Hub**: This stage is used by both Track A (CLI Power User) and Track B (Agent Builder). Stage 5 and [Stage 8 — Agent Interfaces](08-agent-interfaces.en.md) are the two central hubs of this curriculum. + +> 📌 **This stage is used by both tracks**: +> - **Track A (CLI Power User)**: A2 uses [5.1 (Claude Code Basics)](#51--claude-code-basics); A3 uses [5.2 (MCP)](#52--mcp-model-context-protocol--foundation) + selectively uses [5.3 (Skills)](#53--skills-claude-codes-behavior-layer--the-most-critical-layer-of-the-claude-code-ecosystem) and [5.4 (Plugins)](#54--plugins--marketplaces) (A3's Exercise CLI-12 will teach you how to package CLAUDE.md and commands into a plugin). The reading perspective is "**how to get work done with Claude Code**." +> - **Track B (Agent Builder)**: Treats the entire stage as a deep dive into "**how Claude Code works internally**," from 5.1 all the way to 5.4. + +> 🗺️ **What kind of agent is Claude Code?** → See [`resources/agent-paradigms.en.md`](../resources/agent-paradigms.en.md) Type 1 (IDE-coupled) + Type 2 (Terminal pair-programmer); start there for a full comparison of all 5 paradigms. + +> 🧭 **Claude Code is just one *shape* of agent** (orientation before you dive in): Claude Code is the **terminal** agent for **developers** — it lives in your command line and works on code. Anthropic also ships **Claude Cowork**, a **desktop app for non-coders** (researchers, analysts, ops): you give it a goal and it works across your files and apps to hand back a finished result. OpenAI has both shapes too. The rest of Stage 5 is about Claude Code specifically; this table just shows where it sits. + +| Shape | What it does for you | Anthropic | OpenAI | +|---|---|---|---| +| **Terminal · for developers** | reads / edits / runs your code | Claude Code | Codex CLI | +| **App · for everyone** | works across your files, apps, and the web to finish a task | Claude Cowork | ChatGPT agent | + +> ⚠️ **Looking to use a local LLM? This stage is not that path.** Claude Code requires the Anthropic API / OAuth and cannot be directly pointed to Ollama or a local endpoint. For offline work, sensitive data, or to avoid using API quota, please see [`resources/cookbook.en.md` Recipe 6](../resources/cookbook.en.md#6-local-llm--cli-agent-quick-walkthrough) and use a CLI agent that supports BYO LLM, like OpenCode / goose / Aider / Hermes. + +> 📋 **Structure of this chapter**: 7 sub-chapters (5.1 Basics / 5.2 MCP / 5.3 Skills / 5.4 Plugins / 5.5 Subagents / 5.6 Dynamic Workflows / 5.7 Dissecting Claude Code Source), each with "Learning Goals → Required Reading → Hands-on Exercises → Curated Projects" → followed by a self-check at the end of the chapter. **Note**: The **discipline-level** concept of Harness Engineering (plain version: the "runtime shell" around the model — how it gets tools, memory, and permissions, and how each turn actually runs) is systematically covered in [Stage 7](07-multi-agent-production.en.md); 5.7 in this chapter uses Claude Code as a case study, observing how a mature agent tool handles tools, memory, configuration, permissions, and execution flow +> 🔑 **Key Terms**: See [`resources/glossary.en.md` 5](../resources/glossary.en.md#5-claude-code-ecosystem). + +## Stack at a Glance + +From top to bottom, each layer builds on the one below it: + +![Claude Code Ecosystem Stack](../resources/diagrams/stage5-stack.en.png) + +Each layer adds a capability: +- **API + SDK**: Programmatic access to the LLM. +- **Tool Use**: Allows the LLM to call functions you define. +- **MCP**: A standardized protocol that lets any LLM host use any tool server. +- **Skills**: Behavior bundles for Claude Code that can wrap MCP tools. +- **Plugins**: Package and distribute Skills, hooks, commands, and MCP settings as a single unit. + +This stage has 4 sub-sections. **Please do them in order**—each one builds on the last. + +``` +5.1 Claude Code Basics 3-5 days (installation, slash commands, CLAUDE.md) +5.2 MCP — Protocol Layer 5-7 days (write your first MCP server) +5.3 Skills — Behavior Layer 5-7 days (write your first SKILL.md) +5.4 Plugins & Marketplaces 5-7 days (package and publish) +``` + +After completing this stage, you will be able to extend Claude Code, write your own MCP server, and publish a plugin marketplace. + +--- + +## 🗺️ 7-Layer Architecture Map (read this first, then 5.1-5.7) + +> 📋 **What this section is**: maps Claude Code's 7 primitives (MCP / Skills / Plugins / Subagents / Hooks / Slash commands / CLI) to **7 architecture layers + 3 engineering disciplines**. Read it once before 5.1-5.7 to know which layer each sub-chapter teaches; read it again afterward as synthesis. **The layering is a teaching choice, not an absolute truth**. + +![Claude Code 7-Layer Architecture Map](../resources/diagrams/claude-architecture-map.en.png) + +> 📊 **Above**: Claude Code 7 architecture layers + 3 engineering disciplines integrated view. + +### One sentence per layer + Claude's primitive + +| Layer | What it is | Claude's version | Owner | Learn in | +|---|---|---|---|---| +| **L7 Interface** | Where the user talks to the agent | claude-code CLI / Desktop | Harness Engineering | [Stage 5.1](#51--claude-code-basics) | +| **L6 Workflow** | Fixed reusable workflow templates | **Skills** (SKILL.md) + Slash commands + **Plugins** (package Skills / hooks / commands; packaging layer) | Prompt Engineering | [Stage 5.3](#53--skills-claude-codes-behavior-layer--the-most-critical-layer-of-the-claude-code-ecosystem) / [5.4](#54--plugins--marketplaces) | +| **L5 Coordination** | Multi-agent division of labor | **Subagents** + Agent team + Background | Harness Engineering | [Stage 5.5](#55--subagents-claude-codes-native-multi-agent-mechanism--2025-new-feature) | +| **L4 Memory / Context** | Remembering things across conversations / sessions | History / `/compact` / Memory hooks | Context Engineering | [Stage 6](06-memory-rag.en.md) | +| **L3 Control Plane** (the "gatekeeper" layer) | Intercepting / validating / blocking before and after tool execution | **Hooks** (PreToolUse / PostToolUse, etc.) | Harness Engineering | [Stage 5.1 hooks section](#51--claude-code-basics) | +| **L2 Tool Use** | Protocol for an LLM to call external functions | Anthropic Tool Use (`input_schema`) | Tool design | [Stage 3](03-tool-use-and-hello-agent.en.md) | +| **L2.5 Tool Provider** | Wraps external APIs as tools for Layer 2 | **MCP servers** (Notion / Gmail / Slack) | Context Engineering + Tool | [Stage 5.2](#52--mcp-model-context-protocol--foundation) | +| **L1 Foundation** | The LLM itself (the system prompt is delivered directly to this layer) | Anthropic API | Prompt Engineering | [Stage 1](01-llm-basics.en.md) + [Stage 2](02-prompt-engineering.en.md) | + +### 3 Engineering Disciplines Overlay (key insight) + +Prompt / Context / Harness are **disciplines for different layers**. Learning one does not automatically teach the others: + +| Discipline | Which layers it owns | One sentence | Learn in | +|---|---|---|---| +| **Prompt Engineering** | L1 + L6 | "How to design the strings sent into the LLM" | [Stage 2](02-prompt-engineering.en.md) | +| **Context Engineering** | L4 + L2.5 | "What information to load into the context window" | [Stage 6](06-memory-rag.en.md) | +| **Harness Engineering** | L3 + L5 + L7 | "The runtime 'shell' around the LLM — the setup that gives it tools, memory, and control flow" | [Stage 7 §Harness Engineering](07-multi-agent-production.en.md#-harness-engineering--engineering-design-for-a-production-agent-runtime--core-concept-of-this-stage) | + +> 💡 **MCP's special position**: strictly speaking, MCP spans **Context Engineering** (feed context sources) + **Tool design** (protocol specification), so it does not belong purely to one discipline. That is why it is marked as Layer 2.5 in the diagram. + +### Cross-CLI vendor mini-comparison (2026-05 snapshot) + +Only Claude Code has the **full 7-layer stack**; most other CLIs stop at single-agent plus simplified variants: + +| Layer | Claude Code | OpenAI Codex | Gemini CLI | +|---|---|---|---| +| L5 Coordination (multi-agent) | ✅ Subagents | ❌ single-agent | ❌ | +| L3 Control Plane (hooks) | ✅ Hooks | ❌ | ❌ | +| L2.5 Tool Provider (MCP) | ✅ | ✅ (MCP supported) | ✅ (requires manual MCP server install) | +| L6 Workflow (Skills) | ✅ SKILL.md | AGENTS.md (context only) | GEMINI.md (context only) | + +→ See [`resources/cli-agents-guide.en.md`](../resources/cli-agents-guide.en.md) + +--- + +## 5.1 — Claude Code Basics + +### What is Claude Code (Positioning First) + +**Claude Code = a Claude agent that runs inside your terminal**—with full access to the file system, shell, git, and subprocesses, capable of **autonomously completing multi-step tasks** (read file → modify file → run tests → commit → create PR). + +Differences from other Claude interfaces: + +| Interface | Runs Where | Capabilities | Use Case | +|---|---|---|---| +| **claude.ai** (web) | Browser | Pure chat + file uploads, no file system operations | Occasional chats, asking a single question | +| **Claude API** (programmatic) | Your server / script | LLM calls, you build the agent loop | Building production systems | +| **Claude Agent SDK** | Your Python / TS environment | Full agent runtime + tool use + multiple sessions | Building production agent systems | +| **Claude Code** (**This Section**) | Your terminal | **Full OS-level agent** (file / shell / git / subprocess) + skill / plugin / subagent ecosystem | **Primary daily work tool** | + +Before moving on to 5.2-5.7, you will learn about **4 core structures of Claude Code** in this section: CLAUDE.md (memory layer) / slash commands (control layer) / the `~/.claude/` directory (configuration layer) / settings.json (behavior layer). + +### Learning Goals + +After completing this section, you will be able to: +- Explain the respective roles of Claude Code, claude.ai, the API, and the SDK (**"why use the CLI instead of the web"**) +- Install Claude Code, configure authentication, and run your first session with file access +- Use 8-10 common slash commands to control Claude Code's behavior +- Write a project-level `CLAUDE.md` to set baseline behavior +- Recognize the `~/.claude/` directory structure (where skills / agents / plugins / settings.json are located) + +### Required Reading +1. [**Anthropic — Claude Code Quickstart**](https://docs.claude.com/en/docs/claude-code/quickstart) — Official installation guide +2. [**Anthropic — CLAUDE.md best practices**](https://docs.claude.com/en/docs/claude-code/memory) — How to write project memory +3. [**Anthropic — Slash Commands**](https://docs.claude.com/en/docs/claude-code/slash-commands) — Official full list of slash commands +4. [**Anthropic — Settings**](https://docs.claude.com/en/docs/claude-code/settings) — Full `settings.json` schema + env vars +5. [**KimYx0207/Claude-Code-x-OpenClaw-Guide-Zh**](https://github.com/KimYx0207/Claude-Code-x-OpenClaw-Guide-Zh) — A beginner's guide in Simplified Chinese + +> 🛠️ **Writing a good CLAUDE.md?** First read [Stage 7.5 Core Harness Engineering Principles (multi-source synthesis)](07.5-advanced-agentic-concepts.en.md#-cross-concept-harness-engineering-principles-multi-source-synthesis) to build the mental model, then use the 2 prompts below. + +### 📋 CLAUDE.md design prompts (using the 5 principles) + +Copy these directly when writing or revising CLAUDE.md: + +#### Prompt 1 — Audit your existing CLAUDE.md + +``` +I have a CLAUDE.md at [paste path]. Audit it against these 5 harness engineering principles: + +1. Legibility — Markdown headers for sections? Conventions written concretely ("2-space indent") or vaguely ("format properly")? +2. Progressive Disclosure — Under 200 lines? Are `@-import` or `.claude/rules/.md` used to split content? +3. System of Record — Does CLAUDE.md act as an entry map pointing to `docs/` + `.coord/`, or does it cram all rules in one file? +4. Taste Invariants — Verifiable rules ("run `make lint` before commit") or unverifiable phrases ("follow best practices")? +5. Transparency — Does it require the agent to show planning steps, or does it expect silent execution? + +For each: PASS / FAIL / PARTIAL + reason + fix suggestion. Total X/5 + first thing to fix. +``` + +#### Prompt 2 — Generate a new CLAUDE.md (using the 5 principles) + +``` +I want to write CLAUDE.md for a [describe project — e.g. Python data-analysis monorepo / academic paper repo / Next.js app] following these 5 harness engineering principles: + +- **Under 200 lines** +- Acts as an **entry map** — use `@-import` to pull in external docs or `.claude/rules/.md` +- Every rule must be **verifiable** (avoid "follow best practices" hand-waving) +- Include **1-2 transparency rules** (e.g. "show the plan before any edit > 50 lines") +- Mark which content belongs in CLAUDE.md vs `.claude/rules/.md` + +Output: +1. Full CLAUDE.md content +2. Suggested `.claude/rules/` directory split (topic list) +3. One example `.claude/rules/.md` (pick one topic) +``` + +→ **Suggested workflow**: use Prompt 2 to generate a draft before writing CLAUDE.md, then use Prompt 1 to audit the finished file. + +### Common Slash Commands (10 to learn) + +| Command | Purpose | When to Use | +|---|---|---| +| `/help` | List all available commands | When you don't know what commands are available | +| `/clear` | Clear conversation history (retains system context) | When the session is too long and you want to restart the logic | +| `/compact` | Automatically summarize the conversation to free up the context window | When the context is nearly full | +| `/plan` | Enter plan mode (read-only, plan before acting) | Before a major change, let Claude list the plan first | +| `/model` | Switch models (Sonnet / Haiku / Opus) | Switch to a cheaper model to save tokens | +| `/agents` | List / manage subagents (5.5) | To see which subagents are available, for debugging | +| `/plugin install @` | Install a plugin (5.4) | To add new functionality | +| `/permissions` | View / change current session permissions | When there are too many permission prompts and you want to streamline them | +| `/resume` | Resume the previous session | To continue yesterday's work | +| `/bg` | Background the current session (moves to agent view) | When you want to run multiple tasks simultaneously, see 5.5 | + +For a complete list, see the official [Slash Commands documentation](https://docs.claude.com/en/docs/claude-code/slash-commands) linked above. + +### `~/.claude/` Directory Structure (Get a mental map first) + +``` +~/.claude/ ← Global user-level +├── settings.json ← Global behavior (env / hooks / permissions / model defaults) +├── settings.local.json ← Machine-specific (not checked into git) +├── CLAUDE.md ← Global baseline (loaded in every session) +├── skills//SKILL.md ← User-level skills (5.3) +├── agents/.md ← User-level subagents (5.5) +├── plugins/ ← Installed plugins (5.4) +├── hooks/ ← User-level hook scripts +└── jobs// ← Background session states (5.5 background agent) + +/.claude/ ← Project-level (with the repo) +├── settings.local.json ← Project behavior (including permissions) +├── skills//SKILL.md ← Project-level skills (higher priority than user-level) +├── agents/.md ← Project-level subagents +├── commands/.md ← Project-level slash commands +└── hooks/ ← Project-level hooks + +/CLAUDE.md ← Project baseline (loaded in every session) +``` + +**Priority Order** (who wins in a conflict): project > user > built-in default. + +### Hands-on Exercises +- **Exercise 1: First Session** — Install, authenticate, `cd` to a repo, run `claude` → ask "summarize this codebase" → observe how it reads files. +- **Exercise 2: CLAUDE.md** — Write a `CLAUDE.md` in the repo root (role / context / what not to do / how to do things / common commands), and compare the behavior with and without the `CLAUDE.md`. +- **Exercise 3: 5 Slash Commands** — In one session, use `/help`, `/plan`, `/compact`, `/model`, and `/agents` in order, and observe what each one does. +- **Exercise 4: Directory Exploration** — Run `ls ~/.claude/` + `cat ~/.claude/settings.json` to see what your user-level settings look like. + +### Curated Projects + +| Project | ⭐ | Best for | Why it's recommended / Notes | +|---|---|---|---| +| [anthropics/claude-code](https://github.com/anthropics/claude-code) ⭐ Official | ⭐⭐⭐⭐⭐ | Tracking new versions / reading release notes / reporting bugs | The official Claude Code repo, with issues, releases, and inline examples. | +| [Anthropic — Claude Code Official Docs](https://docs.claude.com/en/docs/claude-code/overview) | ⭐⭐⭐⭐⭐ | Any reference query | **The true canonical reference**—the 5 required readings above all come from here. | +| [hesreallyhim/awesome-claude-code](https://github.com/hesreallyhim/awesome-claude-code) | ⭐⭐⭐⭐ | Seeing what the community has to offer (slash command / skill / hook examples) | A broader list of resources (currently being reorganized). | +| [KimYx0207/Claude-Code-x-OpenClaw-Guide-Zh](https://github.com/KimYx0207/Claude-Code-x-OpenClaw-Guide-Zh) | ⭐⭐⭐⭐ | Chinese readers who want a step-by-step tutorial | A beginner's guide in Simplified Chinese. | + +### Hooks (the L3 control layer) ⭐ turn rules into code that auto-intercepts + +MCP / Skills give the agent *more* abilities; **Hooks are the reverse: you attach your own scripts to lifecycle events to check, block, or inject**. This is Claude Code's control layer (L3 in the architecture map). + +**How it works**: in `settings.json`'s `hooks`, you say "when event X fires, run command Y". Common events (the 2026 list has grown to ~28; learn the core few first): + +| Event | Fires | Typical use | +|---|---|---| +| `PreToolUse` | before a tool call | block dangerous commands, permission gate | +| `PostToolUse` | after a tool call | auto-format / lint / run tests | +| `UserPromptSubmit` | when you submit a prompt | inject context, reject some inputs | +| `Stop` / `SubagentStop` | when a (sub)agent tries to stop | force it to continue, or run a final check | +| `SessionStart` / `SessionEnd` | session start / end | load state, write logs | +| `PreCompact` | before context compaction | protect important content | + +**Key semantics**: a hook **returning exit code 2 = block**: Claude reads stderr back as an error (e.g. `PreToolUse` exit 2 blocks that tool call; `UserPromptSubmit` exit 2 blocks the prompt). That is the "enforce rules in code" mechanism. + +> ⚠️ **Security**: a hook is a shell command running on your machine, so don't install others' hooks blindly, and don't run unchecked input inside one. +> +> Full event list + advanced JSON usage: [Claude Code Hooks](https://code.claude.com/docs/en/hooks). + +--- + +## 5.2 — MCP (Model Context Protocol) ⭐ Foundation + +### What is MCP (Positioning First) + +**MCP = an open protocol for "letting an LLM use any external tool or data."** Before MCP, every LLM vendor had to define their own tool specification, and every tool provider had to write a separate integration for each LLM. MCP **standardizes** this layer—write an MCP server once, and Claude / Codex / Cursor / any MCP-enabled host can use it. + +**MCP's Three Abstractions**: + +| Abstraction | What it is | Example | +|---|---|---| +| **Tools** | Functions the LLM can call | `read_file(path)` / `query_db(sql)` / `send_slack(channel, msg)` | +| **Resources** | Data sources the LLM can read | `file:///path/file.md` / `postgres://db/users` | +| **Prompts** | Pre-defined prompt templates on the server | A prompt template for "code review" | + +**Most MCP servers primarily use the Tools abstraction**—Resources and Prompts are used less often. + +**MCP vs Tool Use vs Skill vs Plugin**: + +- **Tool Use** (Stage 3): In-process functions you write for the LLM to call. +- **MCP** (**This Section**): Standardizes tools into a server/client protocol, usable across hosts and LLMs. +- **Skill** (5.3): The behavior layer—teaches Claude "**when to use which MCP tool**." +- **Plugin** (5.4): Packages MCP, Skills, and other components for distribution. + +→ **Core Distinction**: MCP is the "**capability**" (what the LLM can do), while a Skill is the "**behavior**" (when to use which capability). + +### Learning Goals +- Explain MCP's three abstractions (Tools, Resources, Prompts) +- Connect an existing MCP server to Claude Desktop or Claude Code +- Write a minimal MCP server in Python that provides 1-2 tools +- Distinguish between an MCP server, Tool Use, Skills, and Plugins + +### Required Reading +1. [**Anthropic — Introducing MCP**](https://www.anthropic.com/news/model-context-protocol) — The original announcement, a conceptual overview +2. [**MCP Specification**](https://modelcontextprotocol.io/specification) — The actual protocol specification +3. [**Complete Guide to MCP in 2026**](https://dev.to/x4nent/complete-guide-to-mcp-model-context-protocol-in-2026-architecture-implementation-and-4a11) — An implementation guide +4. [**Microsoft — MCP for Beginners**](https://github.com/microsoft/mcp-for-beginners) — Official step-by-step MCP curriculum (concepts, setup, hands-on labs; free, GitHub-based). ★ 16k+ + +### Hands-on Exercises +- **Exercise: MCP client** — Install `modelcontextprotocol/servers/filesystem` and connect to it from Claude Desktop. Watch Claude read your files. +- **Exercise: MCP server** — Write a Python MCP server that provides one tool (e.g., "convert temperature"). Connect to it from Claude Code. **For step-by-step instructions** → see [`resources/cookbook.en.md` 2](../resources/cookbook.en.md#2-write-your-first-mcp-server). +- **Exercise: MCP in production** — In the same Claude session, connect to 2-3 MCP servers simultaneously and watch them coordinate. + +### Curated Projects (for spec / SDK / template reference) + +> 💡 **Looking for MCP servers for everyday tools (Notion / Obsidian / Excel / Postgres / Playwright / Figma, etc.)?** +> Check out [`resources/mcp-skills-catalog.en.md`](../resources/mcp-skills-catalog.en.md)—it organizes 65+ common MCP servers / Skills into 16 categories, each with stars / license / intended audience. The table below retains official servers / SDKs that serve as a "**reference for writing your own MCP server**." + +| Project | ⭐ | Best for | Why it's recommended / Notes | +|---|---|---|---| +| [modelcontextprotocol/servers](https://github.com/modelcontextprotocol/servers) ⭐ Official | ⭐⭐⭐⭐⭐ | Exercise 1 server, and as a reference thereafter | 20+ official MCP servers (filesystem / git / github / sqlite / time / fetch / memory / sequential-thinking), ★ 85k+, MIT, TS+Python. **Read the `everything` and `filesystem` source to understand how the protocol works**. Install with: `npx -y @modelcontextprotocol/server-filesystem /path` or `pip install mcp-server-fetch` | +| [modelcontextprotocol/python-sdk](https://github.com/modelcontextprotocol/python-sdk) | ⭐⭐⭐⭐⭐ | Exercise 2, writing your own MCP server | Official Python SDK, install with `pip install mcp`, MIT. Follow the official quickstart. | +| [modelcontextprotocol/typescript-sdk](https://github.com/modelcontextprotocol/typescript-sdk) | ⭐⭐⭐⭐ | Those who prefer TS | The TypeScript version of the Python SDK, MIT. | +| [wong2/awesome-mcp-servers](https://github.com/wong2/awesome-mcp-servers) ⭐ Catalog | ⭐⭐⭐⭐⭐ | Finding an existing server before writing your own | A catalog of 150+ community MCP servers, categorized by search / code / cloud / communication / finance. Submissions go through mcpservers.org. | +| [punkpeye/awesome-mcp-servers](https://github.com/punkpeye/awesome-mcp-servers) | ⭐⭐⭐⭐ | Cross-referencing with wong2's list | Another MCP server catalog, organized differently and often updated more frequently. | +| [github/github-mcp-server](https://github.com/github/github-mcp-server) | ⭐⭐⭐⭐ | Reading the source of an MCP server actually running in production | Maintained by GitHub, a real example running in production. | +| [21st-dev/magic-mcp](https://github.com/21st-dev/magic-mcp) | ⭐⭐⭐ | Finding inspiration after Exercise 2 | A non-trivial MCP server that generates UI components, ★ 5.3k+, NOASSERTION. **Shows that MCP can do more than just data fetching.** | +| [yamadashy/repomix](https://github.com/yamadashy/repomix) | ⭐⭐⭐⭐⭐ | Feeding an entire codebase to an LLM | ★ 26k+, MIT. Packs a repo into a single AI-friendly file, with MCP server mode + tree-sitter compression (~70% token savings) + secretlint to filter secrets. **Daily-driver tool to pair with Claude Code / Codex.** | + +> 🔭 **MCP in 2026: from "knowing what it is" to "using the ecosystem"**: (1) the **official Registry** (registry.modelcontextprotocol.io), a central place to discover/publish MCP servers; (2) **FastMCP** ([jlowin/fastmcp](https://github.com/jlowin/fastmcp), ★25k), which writes a server in a few `@mcp.tool` lines instead of the low-level SDK; (3) ⚠️ **MCP security**: a tool's results are **untrusted input** (tool poisoning, confused-deputy), so do not attach an unvetted third-party server to a permissioned agent. + +--- + +## 5.3 — Skills (Claude Code's Behavior Layer) ⭐ The most critical layer of the Claude Code ecosystem + +### What is a Skill (Positioning First) + +A Skill = **a markdown file** (`.claude/skills//SKILL.md`) that tells Claude "**when you encounter a certain situation → follow a certain process**." Before each inference, Claude scans the `description` frontmatter of all available skills, checks if they match the current situation, and if there's a match, **automatically loads the SKILL.md into the context**. + +> 🛠️ **Writing a good SKILL.md?** There are two paths: +> - **Path A: use Anthropic's official `skill-creator` skill to generate the skeleton** (the installation section later in 5.3.x shows this). It gives you the frontmatter and subdirectory structure automatically and is Anthropic's canonical tool. +> - **Path B: write it yourself with the SKILL.md design prompts below** — first read [Stage 7.5 Core Harness Engineering Principles](07.5-advanced-agentic-concepts.en.md#-cross-concept-harness-engineering-principles-multi-source-synthesis) for the concepts, then use the prompts to build the file. +> +> The two paths are complementary: `skill-creator` gives you structure, while the 5-principle prompts check content quality. + +### 📋 SKILL.md design prompts (including `skill-creator` as the alternative) + +Copy these directly when writing or revising SKILL.md: + +#### Prompt 1 — Audit your existing SKILL.md + +``` +I have a SKILL.md at [paste path]. Audit it against the 5 harness engineering principles below. For each, return "PASS / FAIL / PARTIAL" + a 1-line reason + a 1-line fix suggestion: + +1. Legibility — Does the description clearly state "when to trigger"? Are tool param names consistent? +2. Progressive Disclosure — Is SKILL.md under 200 lines? Are details placed in `references/` rather than stuffed into the main file? +3. System of Record — Is `references/` the single source? Does the main file avoid duplicating that content? +4. Taste Invariants — Are success criteria hard and verifiable, not subjective phrasing like "as good as possible"? +5. Throughput / Merge — Is there an acceptance check (lint / test / preset YAML) attached? + +End with: total score X/5, which principle to fix first, and why. +``` + +#### Prompt 2 — Generate a new SKILL.md (using the 5 principles) + +``` +I want to write a skill that handles [describe task — e.g. converting PDFs to markdown / running a banned-word audit on academic papers]. Generate SKILL.md following these 5 harness engineering principles: + +- **description** must clearly state "when to trigger" (so Claude can match the situation) +- **Main file under 200 lines** — push examples / edge cases / detailed rules into `references/.md` +- Propose a `references/` directory structure (1-3 topic files) +- Include a **success-criteria table** (verifiable, not subjective) +- Include an **acceptance-check section**: which lints / unit tests / preset YAMLs to run + +Output: +1. Full SKILL.md content +2. references/ directory structure suggestion +3. Which acceptance-gate preset to use (e.g. multi-locale-mirror-sync / catalog-entry-add) +``` + +→ **Suggested workflow**: start with `/skill skill-creator` for a clean skeleton → use Prompt 2 to fill it in → finish with Prompt 1 to audit it. + +**Core mental model**: If you find yourself "**typing the same prompt every time to teach Claude how to do something**" → write it as a skill, and you won't have to next time. In the Claude Code ecosystem, **skills are the dividing line between power users and regular users**—those proficient in writing skills can compress an hour of work into 5 minutes. + +### Skill vs CLAUDE.md vs MCP vs Plugin vs Subagent — A Comparison Table + +These layers are often confused. **A one-line comparison**: + +| Component | What it is | When to Use | Trigger | Example | +|---|---|---|---|---| +| **CLAUDE.md** (5.1) | Baseline behavior for a repo / project | Repo-wide conventions ("use type hints," "commit message format") | **Loaded in every session**, regardless of context | The CLAUDE.md in your repo root | +| **MCP server** (5.2) | A protocol server that provides tools / data | When you want Claude to access **external resources** (APIs / DBs / file systems) | After the server starts, can be called anytime | `github` MCP / `postgres` MCP | +| **Skill** (**This Section**) | A **behavior package for a specific situation** | When you want to set "**in situation X → follow process Y**" | **Auto-loaded on description match** | `skill-vetter` (checks before installing a skill) / `pdf` (handles PDFs) | +| **Plugin** (5.4) | A distributable package of skills + commands + MCP + hooks | When you want to share / install a **whole set** of configurations | `/plugin install @` | `engineering` bundle / `finance` bundle | +| **Subagent** (5.5) | A sub-Claude session with an independent context | When you want to delegate a **large-context task** and get the result back in the main session | Auto-delegated on description match | code-reviewer subagent / researcher subagent | + +**How to choose**: + +- A single-line setting → put it in `CLAUDE.md` +- A multi-step process, used only in a specific situation → write a **Skill** (the topic of this section) +- Need to access external resources (API / DB) → write an **MCP server** +- The skill is too large and consumes the entire main session window → turn it into a **Subagent** +- Want to package a Skill / command / MCP / hook to share → package it as a **Plugin** + +→ **Core Distinction**: MCP is the "**capability**," a Skill is the "**behavior**," a Plugin is for "**distribution**," and a Subagent is an "**independent worker**." + +### Learning Goals +- The structure of `SKILL.md` (YAML frontmatter + body) +- When a skill will auto-load (description matching) +- How to write a `SKILL.md` that solves a daily work task +- The purpose of the `references/`, `scripts/`, and `evals/` subdirectories + +### Required Reading +1. [**Anthropic — Claude Skills Documentation**](https://docs.claude.com/en/docs/claude-code/skills) +2. **A few example SKILL.md files**—from `anthropics/claude-code` or community marketplaces +3. [**Hello-Agents — Extra08 How to Write Good Skills**](https://github.com/datawhalechina/hello-agents/blob/main/Extra-Chapter/Extra08-如何写出好的Skill.md) — The most complete guide to Skill best practices in Chinese +4. [**Hello-Agents — Extra05 A Comparative Interpretation of Agent Skills and MCP**](https://github.com/datawhalechina/hello-agents/blob/main/Extra-Chapter/Extra05-AgentSkills解读.md) — A conceptual comparison of Skills vs MCP + +### Hands-on Exercises +- **Exercise: SKILL.md** — Write a 200-word skill that solves one of your daily work tasks. **For step-by-step instructions** → see [`resources/cookbook.en.md` 1](../resources/cookbook.en.md#1-write-your-first-skill). +- **Exercise: SKILL with references** — Add a `references/` markdown file that the skill can reference. +- **Exercise: SKILL eval** — Add `evals/evals.json` with 3-5 self-tests. + +> 📦 **This repo includes a meta-example**: [`examples/stage-5/tool-calling-tutor/`](../examples/stage-5/tool-calling-tutor/) is the corresponding skill template for this stage—with complete frontmatter (including trigger phrases + "Do NOT use for"), 3 `references/` files, and 5 test cases in `evals/evals.json`. **Fork it directly to create your own skill**. It serves a dual purpose: (a) for learners to use themselves, auto-loading to help debug when stuck on tool calling; (b) as a reference template for writing a SKILL.md in Stage 5 5.3. + +### Recommended Skills (by category) + +> Not sure where to start? Below are the commonly used official + community skills as of late 2025. **How to install**: (a) Most come from a plugin; installing the corresponding plugin will give you the skill; (b) or clone from [anthropics/skills](https://github.com/anthropics/skills) and place them in `~/.claude/skills/` or `.claude/skills/`. + +| Use Case | Skill | Source | Why it's recommended | +|---|---|---|---| +| **🛡️ Security check before installing skills** (must-have) | `skill-vetter` | anthropics/skills | **Must-run before installing any external skill**—checks for red flags, permission scope, suspicious patterns. It's like SAST for marketplace skills. | +| **🔍 Find / install skills** | `find-skills` | anthropics/skills | Natural language query, automatic installation. "I want to do X" will return the corresponding skill. | +| | `skill-lookup` | claude-plugins-official | Complements find-skills, a helper for exploration / search. | +| **✍️ Write your own skill** | `skill-creator` | anthropics/skills + claude-plugins-official | Automatically generates frontmatter + subdirectory structure, a must-have for writing skills. | +| **📄 Office docs processing** | `pdf` / `docx` / `xlsx` / `pptx` | anthropics/skills | Read and write PDF / Word / Excel / PowerPoint. **A must-have set**—essential for any office workflow. | +| **🔧 Code review** | `code-reviewer` / `code-review-excellence` | claude-plugins-official | Security / style / test review for staged diffs. | +| **🐛 Debugging** | `debugger` / `systematic-debugging` | claude-plugins-official | Systematic root cause analysis, avoids quick fixes. | +| **🎓 Academic writing** | `academic-writing-skills` | community | findings-first / mechanism / banned word audit. | +| **🔌 MCP integration / server writing** | `mcp-builder` / `mcp-integration` | claude-plugins-official | Scaffolding for writing MCP servers and integrating existing ones. | +| **💻 Frontend / fullstack** | `frontend-developer` / `fullstack-developer` | claude-plugins-official | Assistance with React components / full-stack architecture. | +| **📊 Data analysis** | `data-analyst` / `visualization-expert` | community | SQL / pandas / chart type selection. | +| **⚙️ Permissions / settings management** | `update-config` / `fewer-permission-prompts` | claude-plugins-official | Management of hooks / permissions / env vars. | +| **🔁 Self-improvement** | `self-improving-agent` | community | Captures learning / errors / corrections for continuous agent improvement. | +| **🌐 General / fallback** | `general-purpose` | Built into Claude Code | The default entry point for complex, open-ended tasks and uncovered scenarios. | + +**Suggested adoption order**: +1. **First must-install**: `skill-vetter` (use it to check other skills before installing them). +2. **Second batch of must-installs**: `skill-creator` + `find-skills` (for writing / finding skills). +3. **By work domain**: Add `pdf`/`docx`/`xlsx` for Office workflows, `code-reviewer`/`debugger` for development, `academic-writing-skills` for academic writing. +4. **Want to see more?**: Browse `obra/superpowers` or `wshobson/agents` for production templates. + +### Curated Projects (for spec / template reference) + +> 💡 **Looking for everyday Skills (NotebookLM, Excalidraw, Office docs, etc.)?** +> See [`resources/mcp-skills-catalog.en.md`](../resources/mcp-skills-catalog.en.md)—categorized by use case, including both official Anthropic + community Skills. The table below is reserved for "**spec / showcase reference material for writing your own Skill**." + +| Project | ⭐ | Best for | Why it's recommended / Notes | +|---|---|---|---| +| [anthropics/skills](https://github.com/anthropics/skills) ⭐ Official spec | ⭐⭐⭐⭐⭐ | Reading before writing your own SKILL.md | Anthropic's official Skills repo: `spec/` (frontmatter standard) + `template/` (starter template) + `skills/` containing reference implementations like pdf / docx / xlsx / pptx / skill-creator / skill-vetter. ★ 158k+. **A reference template for SKILL.md structure**. For the broader Agent Skills standard, see [agentskills.io](https://agentskills.io). | +| [anthropics/claude-code](https://github.com/anthropics/claude-code) | ⭐⭐⭐⭐ | Tracking new features, reading release notes | The main Claude Code repo, including issues / releases / inline skill examples. For learning Skills, this repo is secondary to the one above. | +| [mattpocock/skills](https://github.com/mattpocock/skills) | ⭐⭐⭐⭐ | Seeing "real-world engineer daily SKILL.mds" | Matt Pocock (a well-known educator in the TypeScript community) has open-sourced his actual `.claude/` directory. Each SKILL.md is **extremely short (10-50 lines)** and not over-engineered. **A valuable reference against over-engineered 200-line skills** (★ 157k+, MIT). | +| [obra/superpowers](https://github.com/obra/superpowers) | ⭐⭐⭐⭐ | Power user setup, learning advanced patterns | 20+ battle-tested skills (TDD, debugging, collaboration patterns) + `/brainstorm` / `/write-plan` / `/execute-plan` commands + a skills-search tool. | +| [wshobson/agents](https://github.com/wshobson/agents) | ⭐⭐⭐⭐ | Intermediate: learning skill + subagent combinations | Composes skills + subagents for multi-agent orchestration. An example of **evolving from a single SKILL.md to an agent-as-skill composition pattern** (★ 35k+, MIT). | +| [travisvn/awesome-claude-skills](https://github.com/travisvn/awesome-claude-skills) | ⭐⭐⭐⭐ | Finding an existing skill before writing your own | A curated list of community Claude Skills. | +| [VoltAgent/awesome-agent-skills](https://github.com/VoltAgent/awesome-agent-skills) | ⭐⭐⭐ | A cross-tool perspective | 1000+ agent skills, compatible with Claude Code / Codex / Gemini CLI / Cursor (★ 26k+, MIT). | +| [alirezarezvani/claude-skills](https://github.com/alirezarezvani/claude-skills) | ⭐⭐⭐ | Finding domain-specific skill examples | 232+ Claude Code skills across engineering / marketing / product / compliance. | + +--- + +## 5.4 — Plugins & Marketplaces + +### What is a Plugin (Positioning First) + +**A Plugin = a combination package of MCP + Skills + slash commands + hooks**—it **bundles the components from 5.2 / 5.3 into a single unit that can be installed at once** with `/plugin install`. + +``` +Plugin +├── .mcp.json ← MCP server config from 5.2 (provides tools / data) +├── skills//SKILL.md ← Skill from 5.3 (behavior package) +├── commands/.md ← Slash command from 5.1 (custom prompt entry point) +├── hooks/ ← Trigger point hooks (e.g., PreToolUse, SessionStart) +├── agents/.md ← Subagent from 5.5 (if any) +└── .claude-plugin/plugin.json ← Packaging metadata +``` + +**Why plugins?**: You've written a useful skill and want to share it → a single `git clone` is cumbersome and setup is error-prone. Package it as a plugin, push it to a marketplace, and others on your team can install it with a single command: `/plugin install foo@your-marketplace`. + +**What's the difference between a plugin and a marketplace?**: A plugin is a **single packaged unit**, while a marketplace is a **directory of multiple plugins** (e.g., anthropics/claude-plugins-official is a marketplace containing 35 plugins). + +### Learning Goals +- The `plugin.json` schema (name, version, skills array, configuration) +- The `marketplace.json` schema (plugins array, source, metadata) +- The `claude plugin marketplace add` workflow +- Distinguish between a single-plugin bundle and a multi-plugin marketplace +- Publish your own marketplace + +### Required Reading +1. [**Anthropic — Plugins Documentation**](https://docs.claude.com/en/docs/claude-code/plugins) +2. **Read the `plugin.json` and `marketplace.json` of 2-3 of the marketplaces below.** + +### Hands-on Exercises +- **Exercise: plugin install** — Install one of the marketplaces below and watch it load. +- **Exercise: plugin.json** — Package the SKILL.md you wrote in 5.3 into a plugin. +- **Exercise: marketplace publish** — Push it to GitHub and install it using `claude plugin marketplace add`. + +### Recommended Plugins (by category) + +> Not sure which plugins to install? Below are the highly-rated official Anthropic + community choices as of late 2025. **The installation command format is consistent**: `/plugin install @` (e.g., `/plugin install code-review@claude-plugins-official`). + +| Use Case Category | Plugin (with direct link) | Marketplace | Why it's recommended | +|---|---|---|---| +| **Development Workflow**
(must-haves for most developers) | [`code-review`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/code-review) | claude-plugins-official | The official collection of code review skills, including staged diff review + security checks. | +| | [`pr-review-toolkit`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/pr-review-toolkit) | claude-plugins-official | The complete PR review workflow (comment, suggest, approve). | +| | [`commit-commands`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/commit-commands) | claude-plugins-official | Git commit message conventions + branching workflows. | +| | [`feature-dev`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/feature-dev) | claude-plugins-official | The complete feature development cycle (spec → plan → implement → test). | +| | [`frontend-design`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/frontend-design) | claude-plugins-official | Assistance with UI design + responsive layouts. | +| **Language Tools**
(choose based on the language you use) | [`typescript-lsp`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/typescript-lsp) / [`pyright-lsp`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/pyright-lsp) / [`rust-analyzer-lsp`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/rust-analyzer-lsp) / [`gopls-lsp`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/gopls-lsp) etc. | claude-plugins-official | Integrations with various language LSPs. All [35 language plugins](https://github.com/anthropics/claude-plugins-official/tree/main/plugins) are here. | +| **Plugin / Skill Creation** | [`skill-creator`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/skill-creator) | claude-plugins-official | Automatically generates frontmatter + structure when writing your own skills. | +| | [`plugin-dev`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/plugin-dev) | claude-plugins-official | Automatically generates the `.claude-plugin/` structure when writing your own plugins. | +| | [`mcp-server-dev`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/mcp-server-dev) | claude-plugins-official | Scaffolding for writing your own MCP servers. | +| | [`hookify`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/hookify) | claude-plugins-official | A tool for writing hook rules. | +| **Domain-Specific — Engineering Teams** | [**`engineering` bundle**](https://github.com/anthropics/knowledge-work-plugins/tree/main/engineering) | knowledge-work-plugins | **10 skills**: architecture / code-review / debug / deploy-checklist / documentation / incident-response / standup / system-design / tech-debt / testing-strategy. | +| **Domain-Specific — Finance Teams** | [**`finance` bundle**](https://github.com/anthropics/knowledge-work-plugins/tree/main/finance) | knowledge-work-plugins | **8 skills**: audit-support / close-management / financial-statements / journal-entry-prep / reconciliation / sox-testing / variance-analysis. | +| **Domain-Specific — Others**
(same marketplace) | [`sales`](https://github.com/anthropics/knowledge-work-plugins/tree/main/sales) / [`marketing`](https://github.com/anthropics/knowledge-work-plugins/tree/main/marketing) / [`legal`](https://github.com/anthropics/knowledge-work-plugins/tree/main/legal) / [`human-resources`](https://github.com/anthropics/knowledge-work-plugins/tree/main/human-resources) / [`customer-support`](https://github.com/anthropics/knowledge-work-plugins/tree/main/customer-support) / [`data`](https://github.com/anthropics/knowledge-work-plugins/tree/main/data) / [`design`](https://github.com/anthropics/knowledge-work-plugins/tree/main/design) / [`operations`](https://github.com/anthropics/knowledge-work-plugins/tree/main/operations) / [`product-management`](https://github.com/anthropics/knowledge-work-plugins/tree/main/product-management) / [`productivity`](https://github.com/anthropics/knowledge-work-plugins/tree/main/productivity) / [`bio-research`](https://github.com/anthropics/knowledge-work-plugins/tree/main/bio-research) etc. | knowledge-work-plugins | The knowledge-work-plugins marketplace has **[18 vertical bundles](https://github.com/anthropics/knowledge-work-plugins)**—pick the one that corresponds to your work domain. | +| **External Integrations**
(third-party services) | [`asana`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/asana) / [`github`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/github) / [`gitlab`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/gitlab) / [`linear`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/linear) / [`firebase`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/firebase) / [`playwright`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/playwright) / [`terraform`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/terraform) / [`discord`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/discord) / [`imessage`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/imessage) / [`telegram`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/telegram) etc. | claude-plugins-official (external) | Integrations with common SaaS / development tools. | +| **Community Breadth** | (pick skills that interest you) | [rohitg00/awesome-claude-code-toolkit](https://github.com/rohitg00/awesome-claude-code-toolkit) | The largest community catalog of agents / skills / hooks / templates. | + +**Suggested adoption order**: +1. Must-haves for developers (5): `code-review` + `pr-review-toolkit` + `commit-commands` + `feature-dev` + an `*-lsp` for your language. +2. Add a bundle based on your work domain: `engineering` for engineering teams, `finance` for finance, and so on. +3. If you want to write your own skills / plugins → install `skill-creator` + `plugin-dev`. +4. To see more → browse `awesome-claude-code-toolkit` or [`resources/mcp-skills-catalog.en.md`](../resources/mcp-skills-catalog.en.md). + +### Curated Projects (for marketplace template reference) + +> 💡 The list above is about "**which plugins to install**"; the table below is about "**how to write a marketplace**"—only those who want to build their own marketplace need to look at this. + +| Marketplace | ⭐ | Best for | Why it's recommended / Notes | +|---|---|---|---| +| [anthropics/claude-plugins-official](https://github.com/anthropics/claude-plugins-official) | ⭐⭐⭐⭐⭐ | The official template to reference before writing your own marketplace | 35 internal plugins + 15 external, the standard `.claude-plugin/marketplace.json` schema, with `plugins/` for the plugin bodies + `external_plugins/` for referencing external repos. **If you want to know what marketplace.json should look like, look at this** (★ 30k+). | +| [anthropics/knowledge-work-plugins](https://github.com/anthropics/knowledge-work-plugins) | ⭐⭐⭐⭐⭐ | Seeing a "multi-vertical bundle" type of marketplace | **18 domain-specific plugin bundles** (finance / engineering / sales / legal / marketing / HR / customer-support / data / design / operations / product / productivity / bio-research / enterprise-search / pdf-viewer / small-business / cowork-plugin-management / partner-built). Anthropic's own template for knowledge worker scenarios. | +| [obra/superpowers-marketplace](https://github.com/obra/superpowers-marketplace) | ⭐⭐⭐⭐ | Those who want to create a "I curate, others write" type of marketplace | **The most minimal marketplace template**—the repo only contains `marketplace.json` + README, with the plugin bodies in external repos. The minimal template for the curator-only pattern (★ 1.1k+, MIT). | +| [trailofbits/skills-curated](https://github.com/trailofbits/skills-curated) | ⭐⭐⭐ | Reviewers / teams concerned about supply chain security | A **security-vetted** marketplace maintained by Trail of Bits. Every skill is reviewed, and the README clearly states the criteria. **Demonstrates that a marketplace is not just a list, but also a trust mechanism** (★ 431, CC-BY-SA-4.0). | +| [rohitg00/awesome-claude-code-toolkit](https://github.com/rohitg00/awesome-claude-code-toolkit) | ⭐⭐⭐ | Those who want to browse what the community has to offer | The largest community catalog of Claude Code agents / skills / hooks / templates. Covers a wide range of use cases. | +| [anthropics/life-sciences](https://github.com/anthropics/life-sciences) | ⭐⭐⭐ | Those creating a domain-specific marketplace (medical, financial, legal, educational, etc.) | Anthropic's own **domain-specific marketplace** example (for biology / health sciences), demonstrating how to tailor `marketplace.json` for a single vertical. **The payload is bio-sci MCP servers, but the structure of marketplace.json is the main lesson** (★ 474). | +| [anthropics/claude-for-legal](https://github.com/anthropics/claude-for-legal) | ⭐⭐⭐⭐ | Want to see a full vertical plugin suite (skills + agents + MCP + scheduled agents) | **Anthropic's official legal vertical reference** (★ 7.9k+, Apache-2.0) — 10 legal plugins (commercial / corporate / litigation / privacy / employment / IP / law-student) + 100+ skills + 20+ MCP connectors + scheduled agents + subagent delegation. **You don't need to know law** — this is the best teaching material for "**how to design a vertical plugin suite**": system prompt patterns, accountability surfaces, and the `orchestrate.py` event loop. | + +> 💡 **A walkthrough on "how to publish your own marketplace"**: Currently, the most reliable resource is the [official Anthropic plugin documentation](https://docs.claude.com/en/docs/claude-code/plugins). If you know of a good blog post / repo, feel free to open a PR to add it. + +--- + +## 5.5 — Subagents (Claude Code's native multi-agent mechanism) ⭐ 2025 new feature + +Up to this point, you've learned about MCP (the tool layer), Skills (the behavior layer), and Plugins (the distribution layer). **Subagents are the orchestration layer** (orchestration = coordinating a group of agents: split the work, then combine the results)—they allow the main Claude session to spawn child agents with independent contexts to run specific tasks and report back the results. + +![Subagent 4-Stage Lifecycle: from .md file to returned summary](../resources/diagrams/subagent-4-stage-flow.en.png) + +> 📊 **The diagram above** shows the 4 stages — **Definition → Discovery → Dispatch → Execution**. Read this first, then dive into the details below. + +A comparison with framework-based multi-agent systems from Stage 4 (LangGraph / CrewAI / AutoGen): + +| Dimension | Framework path (Stage 4) | Claude Subagent path (This Section) | +|---|---|---| +| Activation | `pip install crewai` + Python code | Simply write a `.claude/agents/.md` file | +| Runtime | Your own Python process | Claude Code's built-in Task tool | +| Context isolation | Managed by the framework | **Innate**, each subagent has an independent window | +| Provider lock-in | Medium (many frameworks support multi-LLM) | **Strong** (tied to Claude Code) | +| Best for | Production systems that span LLM providers | Engineering teams already committed to Claude Code | +| Learning curve | High (framework abstractions + async) | Low (writing markdown) | + +### Current state of multi-agent mechanisms in various CLIs / SDKs (late 2025) + +Many people assume that multi-agent CLIs are a standard feature for Anthropic / OpenAI / Google—but in reality, only **Claude Code currently has a complete native multi-agent stack**. Codex CLI / Gemini CLI / Cursor are still single-agent; to get multi-agent functionality, you have to write it yourself using an SDK or framework. + +| Platform | Subagent | Agent team | Background agent | Mechanism | +|---|:---:|:---:|:---:|---| +| **Claude Code** (CLI) | ✅ | ✅ | ✅ | `.claude/agents/.md` + Task tool (subagent) + [agent teams](https://docs.claude.com/en/docs/claude-code/agent-teams) + [agent view / background](https://docs.claude.com/en/docs/claude-code/agent-view) | +| **OpenAI Codex CLI** | ❌ | ❌ | ❌ | `AGENTS.md` is just a **single-agent context file** (similar to CLAUDE.md), **not a subagent system** | +| **Google Gemini CLI** | ❌ | ❌ | ❌ | `GEMINI.md` is just for context; no subagent / multi-agent features | +| **Cursor** (IDE-coupled) | ❌ | ❌ | ❌ | A single Cursor Agent; queued messages are sequential, not parallel | +| **OpenAI Agents SDK**
(programmatic, not CLI) | ⚠️ Handoffs + agents-as-tools | ❌ | ❌ | A pure Python SDK, not a CLI; the handoff pattern is close to Claude's subagents but requires writing code | +| **Framework path**
(Stage 4) | LangGraph / CrewAI / AutoGen | ✅ You wire it | Partially | Cross-LLM provider, Python orchestration, see [Stage 4](04-agent-frameworks.en.md) | + +**Interpreting the current state**: + +- If you want to play with multi-agent systems in a **CLI** → currently, only Claude Code has native support (**the topic of this section**) +- If you want to go **cross-provider / cross-LLM** → take the Stage 4 framework path +- If you want **OpenAI ecosystem + multiple agents** → use the OpenAI Agents SDK to write a handoff pattern (programmatic, not CLI) +- If you want **complete control** → go to [Stage 5.7 Dissecting Claude Code Source](#57--dissecting-claude-code-source-reference-harness-implementation--a-must-read-for-track-b) (read the SDK source, wire the multi-agent system yourself) + +→ The rest of this section focuses on **Claude Code subagents**. For developments on other platforms, please follow their respective changelogs (Codex / Gemini / Cursor are still in the single-agent + MCP phase, and will likely follow suit in late 2026). + +### How to dispatch Claude Code's 3 multi-agent mechanisms (specific syntax) + +| Mechanism | When to Use | Dispatch Method | +|---|---|---| +| **Subagent**
(stable) | Delegate large-context tasks (reading an entire codebase / organizing logs) to an isolated context worker, with the result returned to the main session | (1) Write `.claude/agents/.md` (frontmatter with `name` + `description` + `tools` + optional `model`)
(2) Claude **auto-delegates** based on the description; or list manually with `/agents` | +| **Agent team**
(officially documented, but still requires opt-in flag) | When multiple workers need to **communicate with each other** and challenge one another (debate / peer review / multi-perspective exploration) | (1) **Enable** (still requires opt-in): add `"env": {"CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS": "1"}` to `settings.json`, requires Claude Code v2.1.32+
(2) Dispatch with natural language: `Create an agent team to explore X from different angles: one on UX, one on architecture, one playing devil's advocate`
(3) Converse with teammates: use `Shift+Down` to switch, type messages directly
(4) Clean up: `Clean up the team` | +| **Background agent**
(research preview) | Run multiple **independent tasks** in the background, monitored from a single interface (e.g., 3 PR reviews at the same time) | (1) Dispatch from shell: `claude --bg "investigate the flaky test"` (requires v2.1.139+)
(2) Background an existing session: `/bg`
(3) Monitor: `claude agents` (the agent view interface)
(4) Operate: `claude attach ` / `claude logs ` / `claude stop ` | + +**How to choose between the 3 mechanisms**: + +- The task is independent, workers don't interact, and the result just needs to be returned to the main session → **Subagent** (simplest, most token-efficient) +- Workers need to communicate / debate / share a task list → **Agent team** (officially documented, but still requires an opt-in env var; uses 3-5x the tokens, suitable for research / debugging competing hypotheses) +- Multiple independent tasks running, and you want to monitor them all from one interface → **Background agent** (research preview, suitable for long-running parallel tasks) + +--- + +### Which subagents can you dispatch? + +> 💡 **First, a quick term explanation**: a **subagent** is a “child Claude” spawned from the main Claude session. It has its own context window (the amount of conversation it can remember at once, with a limit) and reports its result back when done. **Dispatch** means asking a subagent to do work, like assigning a task to a teammate. + +Many people assume they have to write a subagent themselves before they can use one. In practice, **Claude Code ships with a set of built-in subagents you can use immediately**. The table below shows the three sources: + +| Source | Example subagents | When to use | What you need to do | +|---|---|---|---| +| **Built into Claude Code** | `general-purpose` / `code-reviewer` / `Explore` / `Plan` / `frontend-developer` / `claude-code-guide` / `statusline-setup` | Check these first for general tasks | **Nothing; invoke them directly** | +| **plugin / marketplace** | Skill agents inside `obra/superpowers`, multi-subagent packs from `wshobson/agents` | When the built-ins are not enough | Install a plugin / marketplace item ([Stage 5.4](#54--plugins--marketplaces))| +| **Custom** | A reviewer / domain expert specific to your company workflow | When neither of the above fits | Write `.claude/agents/.md` (see the details block below for an example)| + +> 🔍 **Want to know which subagents your Claude Code currently has?** Run `/agents` in the terminal to list them all: built-in, plugin-provided, and custom. + +### How do you choose a subagent? (decision table) + +For the 7 built-in Claude Code subagents above, this table maps “**when you need to do X, use Y subagent**” (this is a **decision table**: a quick “X → Y” lookup so you do not have to reason from scratch): + +| What you want to do | Built-in subagent to use | Why | +|---|---|---| +| Find code / explore an unfamiliar codebase structure | `Explore` | Built for read-only search; will not randomly edit | +| Design an implementation plan without writing code directly | `Plan` | Produces a step-by-step plan, useful before breaking down a large task | +| Review staged diff / security audit / pre-commit check | `code-reviewer` | Structured PASS/FAIL output + concrete fixes | +| Write / modify UI components / handle accessibility | `frontend-developer` | React / responsive design / a11y (shorthand for accessibility — designing for screen-reader and keyboard-only users) domain knowledge | +| Multi-step research, or you are unsure which category fits | `general-purpose` | General-purpose, can web search, good fallback | +| Ask how to use a Claude Code feature | `claude-code-guide` | Questions about hooks (scripts that intercept tool calls before / after they run — see Gotcha #5 below) / slash commands (commands starting with `/`) / MCP | +| None of the above fits | Write `.claude/agents/.md` yourself | Custom or company-specific workflow | + +**Mini cookbook for 5 common scenarios** (see the full 15 recipes below): + +| Scenario | Use | +|---|---| +| You wrote ≥ 50 lines of new code and are about to commit | `code-reviewer` | +| You cloned a new repo and do not know where to start reading | `Explore` | +| 4 stages / branches need the same review | `general-purpose` (spawn several in parallel)| +| You want to refactor a module and review the architecture first | `Plan` | +| You need to compare multiple sources and decide which paper is right | `general-purpose` for deep research | + +> 📋 **Full 15 recipes** (each includes **scenario + subagent + copy-paste prompt template + when not to use it**) → [`resources/subagent-cookbook.en.md`](../resources/subagent-cookbook.en.md) + +### Clarifying Commonly Confused Concepts (read if the tables above still feel hazy) + +The **3 concept pairs** students confuse most often, plus **5 gotchas veterans learn the hard way**. Skim the parts you need: + +#### Subagent vs Skill — 5 Key Differences + +Many people treat Subagents and Skills as the same thing. They are actually **completely different layers**: + +![Subagent vs Skill — 5 Key Differences](../resources/diagrams/subagent-vs-skill.en.png) + +| Dimension | Subagent | Skill | +|---|---|---| +| **Execution environment** | A new independent context window (under the hood, a new subprocess) | Inside the main session, same context | +| **Tool permissions** | Its own `tools:` list (can restrict it to Read / Grep only) | Main session tools (open by default; a skill can narrow this with `allowed-tools:`) | +| **Return value** | One final message summarized back to the main session | No return value; it changes behavior (rules / persona) | +| **Best for** | Long tasks / parallel work / context isolation | Knowledge injection / rules / changing Claude behavior | +| **Examples** | `code-reviewer` / `Explore` / `Plan` | `codex-delegate` / `pdf` (anthropics/skills) | + +**Quick test**: do you **need a new context window**? Yes → subagent; no → skill. + +#### Subagent vs Slash Command — One is a Task, the Other is a Command + +| Thing | How it triggers | Example | +|---|---|---| +| **Subagent** | Type ordinary conversation text; Claude reads the description and dispatches automatically | You type "Review my staged changes" → Claude dispatches `code-reviewer` | +| **Slash command** | Type a command starting with `/` | `/agents` (list subagents) / `/compact` (compress context) / `/help` | + +⚠️ **Common misconception**: `/agents` **does not invoke a subagent**. It is the command for "listing currently available subagents." **Dispatch happens through ordinary prompt text**, and Claude chooses the subagent. + +#### Description is the Routing Key (How You Write It Decides Whether Claude Selects It) + +How does the main session know which subagent to dispatch? It reads the **`description` field** in `.claude/agents/.md`. **How you write it affects trigger behavior**: + +| How Description is written | Trigger mode | Example | +|---|---|---| +| `...use **PROACTIVELY** when X...` | **Proactive trigger**: when X appears, Claude dispatches it on its own | "use PROACTIVELY when reviewing diffs ≥ 50 lines" | +| `...use when user asks Y...` | **Passive trigger**: the user has to ask clearly | "use when user asks for code review" | +| Empty description | **Invisible**: it will not be selected autonomously | (can only be forced from code with `Agent(subagent_type=...)`) | + +> 💡 **Write the description like ad copy**: make "what problem I solve" **specific**, and Claude is more likely to choose it at the right time. `PROACTIVELY` is a **strong signal word**: when it appears, Claude is much more likely to infer "this is suitable for proactive dispatch"; without it, dispatch more often happens only when the user clearly asks. (It influences Claude's judgment; **it is not a code-level if-then switch**.) + +#### 5 Gotchas Veterans Learn the Hard Way + +| # | Gotcha | Why it matters | +|---|---|---| +| 1 | **A focused Description is enough** | There is no official character limit, but an overly long description uses context budget; write the "trigger condition + applicable scenario" concretely and avoid repetition | +| 2 | **Empty `tools:` = inherit all main-session tools** | If you want to limit a subagent, you must **write the tool list explicitly**; an empty field ≠ no tools | +| 3 | **No `model:` = same model as the main session** | If the main session is Opus and the subagent does not specify a model, it is Opus too (expensive). To save cost, set `model: sonnet` or `model: haiku`| +| 4 | **A subagent has no "I said X earlier" memory** | Every dispatch starts with a **fresh context** and cannot see the main session conversation. The prompt must be self-contained; do not reference "the Y we just discussed" | +| 5 | **Subagents also consume hooks** | PreToolUse / PostToolUse (intercept scripts before / after tool execution) also **fire** inside subagents. Account for this when setting hooks | + +#### Subagent Overall Pros & Cons (read after the tables above for a summary) + +**5 pros** (why they exist): + +| Pro | How it helps | +|---|---| +| **Context isolation** | Keeps the main session window clean; a subagent can scan large files or long logs without pushing the main session's working memory out | +| **Tool allowlist** | Limit the subagent to Read / Grep only (no file writes / no Bash) = safer sandbox | +| **Model override** | Use Haiku for simple tasks and Opus for hard ones; mix models to save cost. Even if the main session is Opus, a subagent can use Haiku | +| **Parallel spawn** | Spawn N subagents from one prompt and run them in parallel; wall-clock time ÷ N (useful for auditing 4 files at once)| +| **Specialized prompt** | `code-reviewer` always reviews code, with a description fixed to "Use PROACTIVELY when commit"; small talk does not drift it | + +**5 cons** (when it is not worth it): + +| Con | Impact | +|---|---| +| **Spawn has overhead** | For tasks < 5 minutes, doing it yourself is faster; subagent startup costs time and tokens too | +| **No cross-call memory** | Every spawn starts a fresh context and cannot see "the X we just discussed"; the prompt must be self-contained | +| **Only one return message** | A subagent is "send it out, then get one report back"; it cannot have a back-and-forth with you, so it is a poor fit for tasks needing step-by-step feedback | +| **Token cost N ×** | Spawning 4 = 4x tokens; calculate the ROI of parallelism (less time, more money)| +| **Debug has one more layer** | When something fails, it is unclear whether to blame the main-session description, the subagent system prompt, or the prompt itself. See [advanced §3 debug 5 entry points](../resources/subagent-advanced.en.md#3-debugging-tools-for-custom-subagents)| + +> 📌 **1-line judgement**: Use a subagent when the task is **≥ 5 minutes** + **can be fully specified in one brief** (no back-and-forth needed) + **one final result is enough** (no step-by-step feedback needed); otherwise run it yourself. + + +
+👉 Concrete subagent file example (the easiest to start with) + +`.claude/agents/code-reviewer.md`: + +```markdown +--- +name: code-reviewer +description: Review staged git changes for security issues, style violations, and missing tests. Use when user asks "review my changes" or runs /review. +tools: + - Read + - Grep + - Bash +model: claude-haiku-4-5 # Optional, use to route to a cheaper model to save costs +--- + +You are a senior code reviewer. When invoked: +1. Run `git diff --cached` to get staged changes +2. Check for: hard-coded secrets, SQL injection patterns, missing error handling, missing tests +3. Output: PASS / list of specific issues with file:line references +``` + +Later, in the main session, if you type "review my changes," Claude will see the matching description, automatically spawn this subagent via the Task tool (Claude Code's internal dispatch mechanism; you do not call it directly), run it, and return a summary to the main session. + +
+ +> 📚 **Official Complete Documentation**: +> - [Subagent spec](https://docs.claude.com/en/docs/claude-code/sub-agents) (frontmatter fields, project vs user scope, Task tool interface) +> - [Complete guide to Agent teams](https://docs.claude.com/en/docs/claude-code/agent-teams) (display modes, task list, advanced subagent-as-teammate) +> - [Agent view / background](https://docs.claude.com/en/docs/claude-code/agent-view) (v2.1.139+, quick start + dispatch workflow) + +### Learning Goals + +- Explain the difference between a subagent and a skill / MCP server (**subagent ≠ skill**: a skill is a behavioral prompt, a subagent is **another Claude instance with an isolated context**) +- Write a custom subagent in a `.claude/agents/.md` file (frontmatter + system prompt + a `tools:` allowlist that explicitly lists permitted tools) +- Invoke a subagent from the main session using the Task tool and observe the context isolation (the parent can't see the subagent's intermediate steps, only the final result) +- Know when to use a subagent (parallel research / large-context isolated tasks / specialized reviews) and when not to (small queries can be handled by skills) + +### Required Reading + +1. [**Anthropic — Claude Code Subagents Official Documentation**](https://docs.claude.com/en/docs/claude-code/sub-agents) ⭐ — `.claude/agents/` structure, Task tool interface, best practices +2. [**Anthropic — Building Effective Agents orchestrator-workers**](https://www.anthropic.com/engineering/building-effective-agents) — Anthropic's own view on the orchestrator pattern (theory + examples) +3. [**Anthropic Cookbook — `customer_service_agent`**](https://github.com/anthropics/claude-cookbooks/tree/main/tool_use) — The canonical multi-agent orchestration example (a chapter-length deep dive; notebook is at `tool_use/customer_service_agent.ipynb`) + +### Hands-on Exercises + +- **Exercise: First subagent** — Write `.claude/agents/code-reviewer.md` (with frontmatter including a `description` that clearly states when it should trigger, and `tools` limited to Read+Grep) + a system prompt to run a staged diff review. From the main Claude session, run `/agents list` to confirm it's loaded, then use the prompt "review staged changes" to observe how the Task tool spawns the subagent. +- **Exercise: Parallel subagent crew** — Write 3 subagents (`researcher.md` / `writer.md` / `critic.md`) to create a "research a topic → write a blog draft → review the draft" pipeline, and chain them together in the main session using the Task tool. **Compare with** [`examples/stage-4/02-multi-agent-roles/`](../examples/stage-4/02-multi-agent-roles/) (the same task in a CrewAI framework version) to see the code differences between the "framework path vs Claude native path." +- **Exercise: Skill vs subagent decision practice** — Take 5 common tasks from your daily workflow and for each, decide whether it should be a skill (behavior layer) or a subagent (independent context layer). Write it up as a 1-page decision table. + +> 📚 **Want a chapter-length deep dive?**: For advanced subagent patterns (agent-as-skill composition, parallel-spawn, handoff between subagents) → see the entire structure of the [`wshobson/agents`](https://github.com/wshobson/agents) repo + the subagent usage in [`obra/superpowers`](https://github.com/obra/superpowers). + +### Curated Projects + +A single table to cover 4 projects. **Pick an entry point by looking at "Best for," and if you want to go deeper, follow the link to the repo.** + +| Project | ⭐ | Best for | Why it's recommended / Notes | +|---|---|---|---| +| [anthropics/claude-cookbooks](https://github.com/anthropics/claude-cookbooks) ⭐ Official | ⭐⭐⭐⭐⭐ | Those who finish 5.5 and want to see "what an agent in actual use looks like" | Anthropic's official chapter-length examples. **`tool_use/customer_service_agent.ipynb`** = the orchestrator-workers canonical example (multi-agent routing + handoff). Python / Jupyter notebook, MIT. **Note**: The full version of `computer_use_demo` is in another repo, [`claude-quickstarts/computer-use-demo`](https://github.com/anthropics/claude-quickstarts/tree/main/computer-use-demo). | +| [wshobson/agents](https://github.com/wshobson/agents) ⭐ Subagent canonical | ⭐⭐⭐⭐⭐ | Those who have written 1-2 subagents and want to see a real team's templates | A collection of 50+ subagent definitions in a production workflow pattern. **Look at the `.claude/agents/` directory structure + naming conventions + how cross-agent handoffs are written.** | +| [obra/superpowers](https://github.com/obra/superpowers) | ⭐⭐⭐⭐ | Those who want to see a mixed implementation of skills + subagents | Already introduced in Stage 5.3. **Focus on the "which tasks go to skills, which go to subagents" decision**—a production template. | +| [anthropics/claude-plugins-official](https://github.com/anthropics/claude-plugins-official) Official | ⭐⭐⭐⭐ | Seeing how a plugin packages a subagent | Already introduced in Stage 5.4. The `agents/` subdirectory inside each plugin is the subagent definition; look at how it's packaged. | + +> 💡 **Subagents are powerful, but don't use them blindly**: Every subagent invocation is a new Claude inference call, with token cost + latency. **Simple queries can be handled by a skill (a behavioral prompt), no need to spawn a subagent**. The sweet spot for subagents is when: (1) the task context is large and would consume the main session's window (e.g., reading an entire codebase), (2) the task is logically independent of the main session, and isolating the context helps the main flow, (3) multiple subagents running in parallel (research / write / critic) can save wall-clock time. + +> 🔗 **Related advanced mechanisms** (official Claude Code, not covered in depth in this stage): +> - **[Agent teams](https://docs.claude.com/en/docs/claude-code/agent-teams)** — Multiple sessions communicating with each other (e.g., a reviewer agent ↔ implementer agent back-and-forth) +> - **[Background agents / agent view](https://docs.claude.com/en/docs/claude-code/agent-view)** — Multiple sessions running in the background, monitored from a single interface (e.g., spawning N PR reviews to run simultaneously) +> +> Subagents are the entry point to these two. After you've finished this section, you can explore the official documentation to expand further. + +--- + +## 5.6 — Dynamic Workflows (when Claude writes its own workflow) ⭐ Opus 4.8+ feature + +> **What this section is**: 5.5 taught you to dispatch subagents **manually**; this is one level up — **Claude generates a workflow script and then runs it itself**. This is the Opus 4.8-era mechanism (started as a research preview), built into recent Claude Code. This section only places it on the ecosystem map and clarifies how it divides labor with 5.5; the **mechanism / examples / quality patterns live in [Stage 7.5 — Dynamic Workflows deep-dive](07.5-advanced-agentic-concepts.en.md#-dynamic-workflows-opus-48--when-the-agent-writes-its-own-workflow)**. + +### How it differs from 5.5 Subagents + +| | 5.5 Subagents | 5.6 Dynamic Workflows | +|---|---|---| +| Who decides the steps | You — dispatch manually, one at a time | Claude writes a multi-step script itself | +| Control flow | the model improvises the next step | the script has **deterministic** loops / parallel fan-out / verification stages | +| Best for | a few parallel subtasks | large, exhaustive, or multi-stage work (migrations, audits, cross-file review) | +| Relationship | — | DW is **built on top of subagents**: the workflow script orchestrates a fleet of them | + +### When to use it, when not to + +- **Use it**: exhaustive + adversarial work (find every bug, then have an independent agent refute each finding), one-shot large migrations, pipelines applying the same transform across many files. +- **Don't**: if you just want one or two agents working in parallel, stay in 5.5; for small tasks a single prompt is cheaper. +- ⚠️ DW spawns many agents and burns tokens — it is not a default. "When it's worth it, and how to write one that doesn't blow up" is in the 7.5 deep-dive below. + +### 📚 Required reading + +1. [**Anthropic — Claude Opus 4.8**](https://www.anthropic.com/news/claude-opus-4-8) — the official announcement where Dynamic Workflows first shipped +2. **[Stage 7.5 — Dynamic Workflows deep-dive](07.5-advanced-agentic-concepts.en.md#-dynamic-workflows-opus-48--when-the-agent-writes-its-own-workflow)** ⭐ — the full version: mechanism, quality patterns (adversarial verify / loop-until-dry / judge panel), and when to use it + +> No examples folder for this section (it's a concept + entry point); to get hands-on, follow the patterns in 7.5. + +--- + +## 5.7 — Dissecting Claude Code Source (reference harness implementation) ⭐ A must-read for Track B + +> **Positioning of this section**: This section is **not** a discipline-level tutorial on harness engineering—the definition, the **8 components**, and the three-layer lineage of prompt→context→harness are covered in **[Stage 7 Harness Engineering](07-multi-agent-production.en.md#-harness-engineering--engineering-design-for-a-production-agent-runtime--core-concept-of-this-stage)**. **This section is a case study**—we're dissecting the source code of Claude Code (a widely-used reference harness) to find the corresponding locations in the implementation for the **first 6 runtime-internal components** of the 8 components listed in Stage 7 (the other two, Eval and Cost-Latency, are cross-cutting and not in the main source loop). + +### Learning Goals + +After completing this section, you will be able to: +- Understand the main loop of the `claude-agent-sdk-python` source (not line-by-line, but grasping the main structure) +- Pinpoint in the source the file:line for the first 6 runtime-internal harness components from the [8 components listed in Stage 7](07-multi-agent-production.en.md#-harness-engineering--engineering-design-for-a-production-agent-runtime--core-concept-of-this-stage) (agent loop / tool registry / context manager / safety layer / retry / telemetry). The 7th, Eval, is a plugin, and the 8th, Cost / Latency, is cross-cutting; they are not in the main source loop and are outside the scope of this exercise. +- Explain the difference between Claude Code's agent loop and the from-scratch ReAct from Stage 3, Exercise 3—what extra components a deployed agent needs that a from-scratch one doesn't. + +> **Where are the discipline-level concepts?**: What harness engineering is / the difference between a framework and a harness / the three-layer lineage of prompt→context→harness → all covered in **[Stage 7 Harness Engineering](07-multi-agent-production.en.md#-harness-engineering--engineering-design-for-a-production-agent-runtime--core-concept-of-this-stage)**. This section is only responsible for the case study of the Claude Code source. + +### 📚 Required Reading + +1. [**Anthropic — Building Effective Agents**](https://www.anthropic.com/engineering/building-effective-agents) ⭐ — The canonical reference for orchestrator / worker / handoff / reflection and other patterns. +2. [**anthropics/claude-agent-sdk-python**](https://github.com/anthropics/claude-agent-sdk-python) — The source of Claude Code's official Python SDK; **key files: `src/claude_agent_sdk/_internal/client.py`** (where the main loop is) + `query.py` (for single-turn API calls). +3. [**ai-boost/awesome-harness-engineering**](https://github.com/ai-boost/awesome-harness-engineering) ⭐ (★ 2.8k+) — A community curation: harness patterns / eval / memory / observability integrations. +4. [**ZhangHanDong/harness-engineering-from-cc-to-ai-coding**](https://github.com/ZhangHanDong/harness-engineering-from-cc-to-ai-coding) — The most complete interpretation of Claude Code's internals in the Chinese community. + +### 🛠 Hands-on Exercise — Dissecting the agent loop (a reading exercise, not a coding one) + +This section is **a reading exercise, not a coding one**—a production harness can't be learned by copying a 200-line example; it's what you still don't understand after copying it that matters. So this exercise requires you to open the source and trace it yourself. + +**Steps**: +1. **Clone**: `git clone https://github.com/anthropics/claude-agent-sdk-python` +2. **Locate the agent loop**: Find the core loop in `_internal/client.py` that actually makes the LLM call, receives the tool_use response, and dispatches to the tool runner. Hint: look for the keywords `async def` and `tool_use_id`. +3. **Pinpoint the first 6 runtime-internal harness components** in the source (file name + line number)—corresponding to the first 6 of the [8 components listed in Stage 7](07-multi-agent-production.en.md#-harness-engineering--engineering-design-for-a-production-agent-runtime--core-concept-of-this-stage) (the 7th, Eval, is a plugin / the 8th, Cost-Latency, is cross-cutting and not in the main source loop): + - (a) **Agent loop**: Where is the loop that actually makes the LLM call + receives the response? + - (b) **Tool registry / dispatch**: When the LLM returns a tool_use, how is it routed to the corresponding tool implementation? + - (c) **Context manager**: How are tool results written back to the message history, and how is the context window controlled / auto-compacted? + - (d) **Safety layer**: Is there a permission gate / sandboxing before a tool is executed? + - (e) **Retry / recovery**: How are tool failures handled (exception vs the LLM reflecting on the error itself)? + - (f) **Telemetry**: Where are metrics / logging / token counting integrated? +4. **Write an 80-150 word summary**: "What's the difference between Claude Code's agent loop and your from-scratch ReAct from Stage 3, Exercise 3?" The point is not a trivial observation like "Claude Code is more complex," but to be able to **articulate what extra components are there and why they are necessary to run in production**. + +**Deliverable**: A set of notes (in your own Obsidian / Notion / `.md` is fine), no submission required. But **if you can't articulate it, you don't understand it yet**—this is a necessary mental model before moving on to production deployment in Stage 7. + +→ **Basic starter template**: This exercise has **no examples folder**—it's a source-reading exercise, not a code-writing exercise. Illustrative, with deep dives in the 📚 above. + +### 🎯 Curated Projects + +A single table to cover 4 projects. **Pick an entry point by looking at "Best for," and if you want to go deeper, follow the link to the repo.** + +| Project | ⭐ | Best for | Why it's recommended / Notes | +|---|---|---|---| +| [anthropics/claude-agent-sdk-python](https://github.com/anthropics/claude-agent-sdk-python) | ⭐⭐⭐⭐⭐ | All Track B learners who want to figure out "how Claude Code runs internally" | **The canonical Python harness, this is the repo you'll be reading for this section's exercise**. It will also be imported in the Stage 7 deployment. | +| [ZhangHanDong/harness-engineering-from-cc-to-ai-coding](https://github.com/ZhangHanDong/harness-engineering-from-cc-to-ai-coding) | ⭐⭐⭐⭐ | Chinese readers who want to understand "why Claude Code was designed this way" | The most complete interpretation of CC's internals in the Chinese community (harness concepts → CC implementation → comparison with other AI coding tools). **Use it as a complement to the SDK source**—one tells you "how," the other tells you "why." | +| [ai-boost/awesome-harness-engineering](https://github.com/ai-boost/awesome-harness-engineering) | ⭐⭐⭐⭐ | Those who finish 5.7 and want to broaden their horizons | A community curation: 30+ harness / eval / memory / observability / MCP projects (★ 2.8k+). **A breadth resource library, not a tutorial**—pick a sub-topic of interest and dive in. | +| [wshobson/agents](https://github.com/wshobson/agents) | ⭐⭐⭐⭐ | Those who have written their own subagent in 5.5 and want to see templates actually in use | The ergonomic design of 50+ subagent definitions (description / tool list / system prompt layers). **You'll learn more from reading the source than from reading the docs**. Already introduced in 5.5, cross-referenced here. | + +> 💡 **The difference between this section and Stage 7**: This section teaches "how this specific harness, Claude Code, works" (a concrete reference); Stage 7 teaches "what a production harness should generally have" (an abstract pattern). **Go from concrete to abstract**—it will be much easier to get into Stage 7 after you've finished this section. + +--- + +## 5.8 — SDK: Take Claude Code Apart and Rebuild It Your Way ⭐ Track B optional — production only + +> 🎯 **Who this section is for**: 99% of readers are done after 5.1-5.7. Only descend here if there's something CLI genuinely can't do for you. Stage 5.7 had you read the SDK source for harness understanding; this section is to make you *use* the SDK as your own service. + +### One analogy that separates SDK / CLI / `CLAUDE.md` + +- **CLI** (`claude` / `codex` / etc.) = a **ready-made car**. Get in, drive. +- Editing `CLAUDE.md` / `AGENTS.md` / adding hooks / writing skills = **tuning the car's performance** so it drives better for your routines. Still the same car. +- **SDK** (`claude-agent-sdk-python` / `openai-agents-python`) = **building a new car from the engine up** — controlling the agent loop, tool dispatch, and memory wiring yourself in Python / TS. + +**99% of learners plateau at "tuning the car" and that's enough.** Only climb to the SDK rung when tuning genuinely can't reach your target scenario. + +### Three-rung ladder — which one are you on? + +1. **Rung 1 — Use the CLI directly.** 90% of solo + team use cases. See 5.1. +2. **Rung 2 — CLI + customisation.** Write `CLAUDE.md`, add hooks, write skills, install plugins. See 5.1-5.4. **Most people stop here, and that's enough.** +3. **Rung 3 — SDK.** Embed the agent inside your code. This section. + +### When do you actually need rung 3? + +Concrete scenarios (not abstract): +- **Embedded in your existing web app / backend** — users don't open a terminal, so CLI is unavailable. +- **Triggered by cron / scheduler** — no human pressing enter; CLI's interactive mode doesn't fit. +- **Wrapped as a company-internal layer** — adding auth, audit logs, rate limits, custom prompt templates — exposing CLI's power through controlled channels. +- **Multi-agent runs with programmatic hand-off control** — finer than Stage 5.5's Task-tool dispatch. + +If none of these describe your task, you probably don't need the SDK. **Go back to 5.1-5.4.** + +### Hello SDK (4 lines of Python) + +```python +from claude_agent_sdk import query + +async for msg in query(prompt="Check current state with git status"): + print(msg) # all message types print safely; filter for AssistantMessage to get the agent's reply +``` + +That's it — wrap in `async def` and it runs. `query()` yields several message types (`AssistantMessage` / `ResultMessage` / `SystemMessage` / etc.); the `print(msg)` above prints any of them safely. To get the agent's actual reply you check `isinstance(msg, AssistantMessage)` and then read `msg.content`. Retry / streaming / prompt caching are in Stage 7 Exercise 4. + +### vs CLI / vs Customisation comparison (read this AFTER the sections above) + +| | CLI (claude / codex) | CLI + custom (CLAUDE.md / hooks) | SDK | +|---|---|---|---| +| Embed in your app | ❌ | ❌ | ✅ | +| Cron / scheduled runs | ⚠️ Barely (`-p` flag) | ⚠️ Same | ✅ | +| Switch language / env | Bound to Node / Bash | Same | Python / TS | +| Programmatic control | ❌ | ❌ | ✅ | +| Custom system prompt | Limited | Limited | Fully open | +| Learning cost | 1 day | 1-2 weeks | 1 month+ | +| Who it's for | Solo daily use | Solo / small team long-term | Building a product / service | + +### Two main SDKs + +| | [claude-agent-sdk-python](https://github.com/anthropics/claude-agent-sdk-python) | [openai-agents-python](https://github.com/openai/openai-agents-python) | +|---|---|---| +| Publisher | Anthropic | OpenAI | +| Models | Claude (Opus / Sonnet / Haiku) | OpenAI series + others | +| Strengths | Same tool / skill / hook abstraction as Claude Code | Handoff / agents-as-tools pattern; built-in sandbox since April 2026 | +| Best fit | Already on Claude Code, embedding into a service | Already committed to the OpenAI ecosystem | + +Both are MIT-licensed with clean APIs. **The real question is which model your downstream picks.** + +### What's next + +- **Read code**: back to 5.7, read `claude-agent-sdk-python`'s `_internal/client.py` — now that you've used the SDK, the main loop reads with more meaning. +- **Practice the SDK at production depth**: Stage 7 Exercise 4 (streaming + prompt caching); Stage 7 Exercise 5 (FastAPI + Docker production deploy). +- **If you realise you don't actually need the SDK**: that's a good outcome — go back to 5.1-5.4 and master "CLI + customisation". It's usually a better return than writing your own SDK service. + +> 💡 **This section vs Stage 7**: this section is "what the SDK is, when to use it" (positioning + entry); Stage 7 is "writing an agent service that's ready to deploy with the SDK" (streaming / caching / deployment). + +--- + +## ✅ Self-Check Before Stage 6 + +Can you: + +- [ ] Install Claude Code and use 5 different slash commands? +- [ ] Connect 2 MCP servers in the same Claude session? +- [ ] Write your own MCP server in Python that provides 1 usable tool? +- [ ] Write a `SKILL.md` that auto-loads on a specific trigger phrase? +- [ ] Package a skill into a plugin and publish it via `marketplace.json`? +- [ ] **Write a custom subagent in `.claude/agents/` and invoke it from the Task tool?** +- [ ] **Read the main loop of `claude-agent-sdk-python` and pinpoint in the source where the first 6 runtime-internal components of the [8 harness components listed in Stage 7](07-multi-agent-production.en.md#-harness-engineering--engineering-design-for-a-production-agent-runtime--core-concept-of-this-stage) are? (5.7 exercise)** +- [ ] Explain the respective roles of MCP, Skills, Plugins, Subagents, and the SDK? + +If yes to all → proceed to [Stage 6 — Memory & RAG](06-memory-rag.en.md). + +> 💡 **Stage 5 is the first of two hubs**—both Track A and Track B use it. The second hub is [**Stage 8 — Agent Interfaces**](08-agent-interfaces.en.md) (Computer Use / Browser Use / Sandbox), which you can do after the main path, or preview early if you're interested in Computer Use / Browser MCP. + +## 💡 Bonus: After Completing This Stage + +- Submit a PR to [`anthropics/claude-cookbooks`](https://github.com/anthropics/claude-cookbooks) (a small fix, a documentation update). +- Submit your own plugin to a community marketplace. +- Write a blog post comparing your hello-MCP server with one from the official `modelcontextprotocol/servers` collection. diff --git a/stages/05-claude-code-ecosystem.md b/stages/05-claude-code-ecosystem.md new file mode 100644 index 0000000..d8944a7 --- /dev/null +++ b/stages/05-claude-code-ecosystem.md @@ -0,0 +1,963 @@ +# Stage 5 — Claude Code 生態系（Claude Code Ecosystem）⭐⭐ + +> **繁體中文** | [简体中文](./05-claude-code-ecosystem.zh-Hans.md) | [English](./05-claude-code-ecosystem.en.md) + +⏱ **時間估算**：3-4 週（約 15-25 小時） + +> 🚪 **進入條件**（共用 hub、依 track 不同）：**Track A（CLI Power User）** 從 A1-A2 過來、會用 Python + 跑過基本 CLI 即可、從 5.1/5.2 起步；**Track B（Agent Builder）** 建議先完成 [Stage 3](03-tool-use-and-hello-agent.md)（tool use）+ [Stage 4](04-agent-frameworks.md)（agent frameworks）再進、把整個 stage 當「Claude Code 內部怎麼運作」深讀。不確定走哪條 → 看下面 📌 的兩軌說明。 + +> 💡 整個 stage 圍繞 4 個關鍵詞（**MCP / Skills / Plugins / Marketplace**）展開 → 不熟先翻 [`resources/glossary.md` 5](../resources/glossary.md#5-claude-code-生態)。 + +**👥 共用 hub**：本 stage 是 Track A（CLI Power User）+ Track B（Agent Builder）兩條路徑的共用中心。Stage 5 跟 [Stage 8 — Agent Interfaces](08-agent-interfaces.md) 是 curriculum 的兩個 hub。 + +> 📌 **這個 stage 兩條軌都用**： +> - **Track A（CLI Power User）**：A2 用 [5.1（Claude Code 基礎）](#51--claude-code-基礎)；A3 用 [5.2（MCP）](#52--mcpmodel-context-protocol-基礎) + 選擇性用到 [5.3（Skills）](#53--skillsclaude-code-的行為層-claude-code-生態最關鍵的一層) 跟 [5.4（Plugins）](#54--plugins-與-marketplaces)（A3 的 動手練習 CLI-12 會教把 CLAUDE.md 跟 commands 打包成 plugin）。讀的角度是「**怎麼用 Claude Code 把工作做好**」 +> - **Track B（Agent Builder）**：把整個 stage 當「**Claude Code 內部怎麼運作**」的深度學習，從 5.1 完整走到 5.4 + +> 🗺️ **Claude Code 屬於哪種 agent 型態**？→ [`resources/agent-paradigms.md`](../resources/agent-paradigms.md) Type 1（IDE-coupled）+ Type 2（Terminal pair-programmer）；想看完整 5 種 paradigm 對照也從這份開始。 + +> 🧭 **Claude Code 只是 agent 的其中一種型態**（往下讀前先有個全局感）：Claude Code 是給**工程師**的**終端機** agent——活在命令列、處理程式碼。Anthropic 另外還有 **Claude Cowork**：給**非工程師**（研究、分析、營運）的**桌面 app**——給它一個目標、它會跨你的檔案跟應用程式把事情做完、交回成品。OpenAI 這兩種型態也都有。Stage 5 接下來專講 Claude Code，這張表只是讓你知道它在整張地圖的位置。 + +| 型態 | 它幫你做什麼 | Anthropic | OpenAI | +|---|---|---|---| +| **終端機 · 給工程師** | 讀 / 改 / 跑你的程式碼 | Claude Code | Codex CLI | +| **App · 給所有人** | 跨你的檔案、應用程式、網頁把一件事做完 | Claude Cowork | ChatGPT agent | + +> ⚠️ **想用本機 LLM？這個 stage 不是那條路線。** Claude Code 需要 Anthropic API / OAuth，不能直接改接 Ollama 或本機 endpoint。離線、隱私資料或不想用 API 額度時，請看 [`resources/cookbook.md` Recipe 6](../resources/cookbook.md#6-本機-llm--cli-agent-快速-walkthrough)，用 OpenCode / goose / Aider / Hermes 這類支援 BYO LLM 的 CLI agent。 + +> 📋 **本章組成**：7 個子章（5.1 基礎 / 5.2 MCP / 5.3 Skills / 5.4 Plugins / 5.5 Subagents / 5.6 Dynamic Workflows / 5.7 Claude Code Source 解剖），每個子章都有「學習目標 → 必修閱讀 → 動手練習 → 精選 Projects」 → 章末 自我檢查。**注意**：Harness Engineering（白話：model 外面的「運行外殼」——怎麼給它工具、記憶、權限，怎麼跑每一輪）的**學科級概念**在 [Stage 7](07-multi-agent-production.md) 系統整理；本章 5.7 則把 Claude Code 當成案例，觀察一個成熟 agent 工具如何處理工具、記憶、設定、權限與執行流程 +> 🔑 **關鍵名詞**：見 [`resources/glossary.md` 5](../resources/glossary.md#5-claude-code-生態) + +## Stack 一覽 + +由上往下，每一層都建立在底下那一層上： + +![Claude Code Ecosystem Stack](../resources/diagrams/stage5-stack.png) + +每一層各自加上一種能力： +- **API + SDK**：用程式存取 LLM +- **Tool Use**：讓 LLM 呼叫你定義的 function +- **MCP**：標準化協定，讓任何 LLM host 都能使用任何 tool server +- **Skills**：Claude Code 的行為包，可以封裝 MCP tool +- **Plugins**：把 Skills、hooks、commands、MCP 設定打包成一個單位發佈 + +這個階段有 4 個子章節，**請按順序做**——每一節都建立在前一節之上。 + +``` +5.1 Claude Code 基礎 3-5 天 （安裝、slash commands、CLAUDE.md） +5.2 MCP — 協定層 5-7 天 （寫你的第一個 MCP server） +5.3 Skills — 行為層 5-7 天 （寫你的第一個 SKILL.md） +5.4 Plugins 與 Marketplaces 5-7 天 （打包並發佈） +``` + +跑完這個階段，你會能擴充 Claude Code、寫自己的 MCP server、發佈一個 plugin marketplace。 + +--- + +## 🗺️ 7-Layer Architecture Map（先看這張圖、再讀 5.1-5.7） + +> 📋 **這節是什麼**：把 Claude Code 的 7 個 primitive（MCP / Skills / Plugins / Subagents / Hooks / Slash commands / CLI）對應到 **7 個架構層 + 3 個工程學 discipline**——進 5.1-5.7 之前看一次知道接下來在學什麼層、學完回頭看是 synthesis。**分層是教學選擇、不是 absolute 真理**。 + +![Claude Code 7-Layer Architecture Map](../resources/diagrams/claude-architecture-map.png) + +> 📊 **上圖**：Claude Code 7 個架構層 + 3 個工程學 discipline 整合視圖。 + +### 每層 1 句白話 + Claude 的版本 + +| Layer | 是什麼 | Claude 的版本 | 誰管 | 學在 | +|---|---|---|---|---| +| **L7 Interface** | 使用者跟 agent 交談入口 | claude-code CLI / Desktop | Harness Eng | [Stage 5.1](#51--claude-code-基礎) | +| **L6 Workflow** | 固定可重用流程模板 | **Skills**（SKILL.md）+ Slash commands + **Plugins**（打包 Skills / hooks / commands、屬 packaging）| Prompt Eng | [Stage 5.3](#53--skillsclaude-code-的行為層-claude-code-生態最關鍵的一層) / [5.4](#54--plugins-與-marketplaces) | +| **L5 Coordination** | 多 agent 分工合作 | **Subagents** + Agent team + Background | Harness Eng | [Stage 5.5](#55--subagentsclaude-code-原生-multi-agent-機制-2025-新功能) | +| **L4 Memory / Context** | 跨對話 / 跨 session 記事情 | History / `/compact` / Memory hooks | Context Eng | [Stage 6](06-memory-rag.md) | +| **L3 Control Plane**（「守門員」層） | tool 執行前 / 後攔截 / validation / 阻擋 | **Hooks**（PreToolUse / PostToolUse 等）| Harness Eng | [Stage 5.1 hooks 段](#51--claude-code-基礎) | +| **L2 Tool Use** | LLM 呼叫外部 function 的 protocol | Anthropic Tool Use（`input_schema`）| Tool design | [Stage 3](03-tool-use-and-hello-agent.md) | +| **L2.5 Tool Provider** | 把外部 API 包成 tool 給 Layer 2 用 | **MCP servers**（Notion / Gmail / Slack）| Context Eng + Tool | [Stage 5.2](#52--mcpmodel-context-protocol-基礎) | +| **L1 Foundation** | LLM 本體（system prompt 直接送達這層）| Anthropic API | Prompt Eng | [Stage 1](01-llm-basics.md) + [Stage 2](02-prompt-engineering.md) | + +### 3 工程學 Discipline overlay（核心 insight） + +「Prompt / Context / Harness」是**不同層的 discipline**——學會其中一個不會自動會另一個： + +| Discipline | 負責哪些 layer | 1 句話 | 學在 | +|---|---|---|---| +| **Prompt Engineering** | L1 + L6 | 「送進 LLM 的字串怎麼設計」 | [Stage 2](02-prompt-engineering.md) | +| **Context Engineering** | L4 + L2.5 | 「context window 裝什麼資訊」 | [Stage 6](06-memory-rag.md) | +| **Harness Engineering** | L3 + L5 + L7 | 「LLM 外面的『運行外殼』——給它工具、記憶、控制流程的那層設置」 | [Stage 7 §Harness Engineering](07-multi-agent-production.md#-harness-engineering--production-agent-runtime-的工程設計--本-stage-核心概念) | + +> 💡 **MCP 的特殊位置**：MCP 嚴格說是 **Context Engineering**（feed context source）+ **Tool design**（協議規範）跨層東西、不純歸任一 discipline——所以圖裡用 Layer 2.5 標明。 + +### 跨 CLI vendor mini-comparison（2026-05 snapshot） + +只有 Claude Code 有**完整 7-layer stack**；其他 CLI 大多停在 single-agent + 簡化版： + +| Layer | Claude Code | OpenAI Codex | Gemini CLI | +|---|---|---|---| +| L5 Coordination（multi-agent）| ✅ Subagents | ❌ single-agent | ❌ | +| L3 Control Plane（hooks）| ✅ Hooks | ❌ | ❌ | +| L2.5 Tool Provider（MCP）| ✅ | ✅（已支援 MCP）| ✅（需手動裝 MCP server）| +| L6 Workflow（Skills）| ✅ SKILL.md | AGENTS.md（context only）| GEMINI.md（context only）| + +→ 細看 [`resources/cli-agents-guide.md`](../resources/cli-agents-guide.md) + +--- + +## 5.1 — Claude Code 基礎 + +### Claude Code 是什麼（先定位） + +**Claude Code = 跑在你 terminal 內的 Claude agent**——有完整 file system / shell / git / subprocess access、可以**自主完成多步驟工作**（讀檔 → 改檔 → 跑 test → commit → 發 PR）。 + +跟其他 Claude 介面差別： + +| 介面 | 跑哪 | 能做什麼 | 用法 | +|---|---|---|---| +| **claude.ai**（web） | 瀏覽器 | 純對話 + 上傳檔案、無 file system 操作 | 偶爾聊一下、ask 一個問題 | +| **Claude API**（programmatic） | 你的 server / script | LLM call、自己包 agent loop | 寫 production system | +| **Claude Agent SDK** | 你的 Python / TS 環境 | 完整 agent runtime + tool use + 多 session | 寫 production agent system | +| **Claude Code**（**本節**） | 你的 terminal | **完整 OS-level agent**（file / shell / git / subprocess）+ skill / plugin / subagent 生態 | **日常工作主力工具** | + +進 5.2-5.7 之前你會在這節學到 **4 個 Claude Code 核心結構**：CLAUDE.md（記憶層）/ slash commands（控制層）/ `~/.claude/` 目錄（設定層）/ settings.json（行為層）。 + +### 學習目標 + +完成本節後你會： +- 講得出 Claude Code 跟 claude.ai / API / SDK 各自的角色（**「為什麼用 CLI 不用 web」**） +- 安裝 Claude Code、配置認證、跑第一個有 file access 的 session +- 用 8-10 個常用 slash command 控制 Claude Code 行為 +- 寫一份 project-level `CLAUDE.md` 設定 baseline 行為 +- 認得 `~/.claude/` 目錄結構（skills / agents / plugins / settings.json 各放哪） + +### 必修閱讀 +1. [**Anthropic — Claude Code Quickstart**](https://docs.claude.com/en/docs/claude-code/quickstart) — 官方安裝指南 +2. [**Anthropic — CLAUDE.md best practices**](https://docs.claude.com/en/docs/claude-code/memory) — 怎麼寫專案 memory +3. [**Anthropic — Slash Commands**](https://docs.claude.com/en/docs/claude-code/slash-commands) — 官方完整 slash command 列表 +4. [**Anthropic — Settings**](https://docs.claude.com/en/docs/claude-code/settings) — `settings.json` 完整 schema + env var +5. [**KimYx0207/Claude-Code-x-OpenClaw-Guide-Zh**](https://github.com/KimYx0207/Claude-Code-x-OpenClaw-Guide-Zh) — 簡中入門指南 + +> 🛠️ **要寫好 CLAUDE.md？** 先看 [Stage 7.5 核心 Harness Engineering 原則（多 source）](07.5-advanced-agentic-concepts.md#-跨概念-harness-engineering-原則多-source-整理) 建概念、再用下面 2 個 prompt 動手。 + +### 📋 CLAUDE.md 設計 prompts（依 5 原則） + +寫 / 改 CLAUDE.md 時直接複製貼上： + +#### Prompt 1 — Audit 你現有的 CLAUDE.md + +``` +我有一個 CLAUDE.md（在 [貼路徑]），請依下面 5 個 harness engineering 原則 audit： + +1. Legibility — 用 markdown header 分區嗎？conventions 寫具體（"2-space indent"）還是模糊（"format properly"）？ +2. Progressive Disclosure — < 200 行嗎？有用 `@-import` 或 `.claude/rules/.md` 拆分嗎？ +3. System of Record — CLAUDE.md 是否當 entry map、實際內容指向 `docs/` + `.coord/`？還是把所有規則塞同一檔？ +4. Taste Invariants — 規則可驗證嗎（"run `make lint` before commit"）？還是「follow best practices」這種空話？ +5. Transparency — 有要求 agent show planning step 嗎？還是預期它默默做完？ + +每條給 PASS / FAIL / PARTIAL + 原因 + 改進建議。總分 X/5、最該先修哪條。 +``` + +#### Prompt 2 — 生成新的 CLAUDE.md（依 5 原則） + +``` +我要為一個 [描述專案，例如：Python data analysis monorepo / 學術論文 repo / Next.js app] 寫 CLAUDE.md。請依下面 5 個 harness engineering 原則生成： + +- **< 200 行** +- 當 **entry map**，把實際 conventions 用 `@-import` 引外部 docs 或 `.claude/rules/.md` +- 每條規則**可驗證**（不要「follow best practices」這種空話） +- 加 **1-2 個 transparency rule**（例如「edit > 50 lines 前先 show plan」） +- 標明哪些內容該放 CLAUDE.md、哪些該分到 `.claude/rules/.md` + +輸出： +1. 完整 CLAUDE.md 內容 +2. 建議的 `.claude/rules/` 目錄切法（topic 列表） +3. 1 個示範 `.claude/rules/.md`（任選一個 topic） +``` + +→ **建議流程**：寫 CLAUDE.md 前用 Prompt 2 生成、寫完用 Prompt 1 audit。 + +### 常用 slash commands（10 個必學） + +| Command | 用途 | 何時用 | +|---|---|---| +| `/help` | 列出所有可用 command | 不知道有什麼指令時 | +| `/clear` | 清空對話歷史（保留 system context） | session 太長、想重啟邏輯 | +| `/compact` | 自動摘要對話、釋放 context window | context 接近用滿 | +| `/plan` | 進入 plan mode（read-only、先規劃才動手） | 大改動前先讓 Claude 列計畫 | +| `/model` | 切換 model（Sonnet / Haiku / Opus）| 改成更便宜 model 省 token | +| `/agents` | 列 / 管理 subagent（5.5）| 看哪些 subagent 可用、debug | +| `/plugin install @` | 安裝 plugin（5.4）| 加新功能 | +| `/permissions` | 看 / 改當前 session 權限 | 太多 permission prompt 想精簡 | +| `/resume` | 恢復前次 session | 接續昨天工作 | +| `/bg` | 把當前 session 背景化（移到 agent view）| 想同時跑多任務、見 5.5 | + +完整列表見上方 [Slash Commands 官方文件](https://docs.claude.com/en/docs/claude-code/slash-commands)。 + +### `~/.claude/` 目錄結構（先有 mental map） + +``` +~/.claude/ ← 全域 user-level +├── settings.json ← 全域行為（env / hooks / permissions / model 預設） +├── settings.local.json ← 機器特定（不入 git） +├── CLAUDE.md ← 全域 baseline（每個 session 都載入） +├── skills//SKILL.md ← user-level skills（5.3） +├── agents/.md ← user-level subagents（5.5） +├── plugins/ ← 已安裝的 plugin（5.4） +├── hooks/ ← user-level hook scripts +└── jobs// ← background sessions 狀態（5.5 background agent） + +/.claude/ ← project-level（隨 repo） +├── settings.local.json ← project 行為（含 permissions） +├── skills//SKILL.md ← project-level skills（優先級高於 user-level） +├── agents/.md ← project-level subagents +├── commands/.md ← project-level slash command +└── hooks/ ← project-level hook + +/CLAUDE.md ← project baseline（每個 session 都載入） +``` + +**優先順序**（衝突時誰贏）：project > user > built-in default。 + +### 動手練習 +- **練習 1：第一個 session** — 安裝、認證、`cd` 到 repo、跑 `claude` → 問「summarize this codebase」→ 觀察怎麼讀檔 +- **練習 2：CLAUDE.md** — 寫 repo 根目錄 CLAUDE.md（role / context / 不能做的事 / 怎麼做事 / 常用指令），對照「沒 CLAUDE.md」跟「有 CLAUDE.md」的行為差別 +- **練習 3：5 個 slash commands** — 在一個 session 內依序用 `/help` `/plan` `/compact` `/model` `/agents`，觀察各自做什麼 +- **練習 4：目錄探索** — `ls ~/.claude/` + `cat ~/.claude/settings.json`、看自己 user-level 設定長什麼樣 + +### 精選 Projects + +| Project | ⭐ | 適合誰 | 為什麼推薦 / 備註 | +|---|---|---|---| +| [anthropics/claude-code](https://github.com/anthropics/claude-code) ⭐ 官方 | ⭐⭐⭐⭐⭐ | 追蹤新版本 / 看 release notes / 回報 bug | Claude Code 官方 repo、issues + releases + inline 範例 | +| [Anthropic — Claude Code 官方文件](https://docs.claude.com/en/docs/claude-code/overview) | ⭐⭐⭐⭐⭐ | 任何 reference 查詢 | **真正的 canonical reference**——上面 5 條必修閱讀都從這裡來 | +| [hesreallyhim/awesome-claude-code](https://github.com/hesreallyhim/awesome-claude-code) | ⭐⭐⭐⭐ | 想看社群有什麼（slash commands / skills / hooks 範例）| 較廣泛的資源清單（目前正在重整）| +| [KimYx0207/Claude-Code-x-OpenClaw-Guide-Zh](https://github.com/KimYx0207/Claude-Code-x-OpenClaw-Guide-Zh) | ⭐⭐⭐⭐ | 中文讀者要逐步教學 | 簡中入門導讀 | + +### Hooks（L3 控制層）⭐ 把規則寫成程式、自動攔截 + +MCP / Skills 是「給 agent 更多能力」；**Hooks 則是反過來：在 agent 的生命週期事件上掛你自己的 script，做檢查、攔截、注入**。這是 Claude Code 的控制層（架構圖的 L3）。 + +**怎麼運作**：在 `settings.json` 的 `hooks` 設定「某事件發生時跑哪個 command」。常用事件（2026 已擴到 ~28 個，先記核心幾個）： + +| 事件 | 觸發時機 | 典型用途 | +|---|---|---| +| `PreToolUse` | 工具呼叫前 | 擋危險指令、權限 gate | +| `PostToolUse` | 工具呼叫後 | 自動 format / lint / 跑測試 | +| `UserPromptSubmit` | 你送出 prompt 時 | 注入 context、擋掉某些輸入 | +| `Stop` / `SubagentStop` | （子）agent 想停時 | 強制它繼續、或做收尾檢查 | +| `SessionStart` / `SessionEnd` | session 開始 / 結束 | 載入狀態、寫 log | +| `PreCompact` | context 壓縮前 | 保護重要內容 | + +**關鍵語意**：hook **回傳 exit code 2 = 阻擋**：Claude 會把 stderr 當錯誤訊息讀回去（例如 `PreToolUse` 回 2 就擋下那個工具呼叫、`UserPromptSubmit` 回 2 就擋下 prompt）。這就是「用程式強制規則」的機制。 + +> ⚠️ **安全**：hook 是在你機器上跑的 shell command，別亂裝別人的 hook，也別在 hook 裡跑未經檢查的輸入。 +> +> 完整事件清單 + JSON 進階用法見官方文件：[Claude Code Hooks](https://code.claude.com/docs/en/hooks)。 + +--- + +## 5.2 — MCP（Model Context Protocol）⭐ 基礎 + +### MCP 是什麼（先定位） + +**MCP = 「**讓 LLM 用任何外部工具 / 資料**」的開放協定**。在 MCP 之前每個 LLM 廠商都得自己定義 tool 規格、每個工具供應商都得為每個 LLM 寫一份接法。MCP 把這層**標準化**——寫一次 MCP server、Claude / Codex / Cursor / 任何支援 MCP 的 host 都能用。 + +**MCP 三個抽象**： + +| 抽象 | 是什麼 | 範例 | +|---|---|---| +| **Tools** | LLM 可以呼叫的 function | `read_file(path)` / `query_db(sql)` / `send_slack(channel, msg)` | +| **Resources** | LLM 可以讀的資料源 | `file:///path/file.md` / `postgres://db/users` | +| **Prompts** | server 預定義的 prompt 樣板 | 一份「review code」的 prompt template | + +**多數 MCP server 主要用 Tools 抽象**——Resources 跟 Prompts 用得少。 + +**MCP vs Tool Use vs Skill vs Plugin**： + +- **Tool Use**（Stage 3）：你 in-process 寫的 function 給 LLM 呼叫 +- **MCP**（**本節**）：把 tool 標準化成 server / client 協定、跨 host / 跨 LLM 可用 +- **Skill**（5.3）：行為層 — 教 Claude「**遇到 X 用哪個 MCP tool**」 +- **Plugin**（5.4）：把 MCP + Skill + 其他打包散佈 + +→ **核心區分**：MCP 是「**能力**」（讓 LLM 能做什麼）、Skill 是「**行為**」（什麼時候用什麼能力）。 + +### 學習目標 +- 解釋 MCP 的三個抽象（Tools、Resources、Prompts） +- 把現成的 MCP server 接上 Claude Desktop 或 Claude Code +- 用 Python 寫一個最小的 MCP server，提供 1-2 個 tool +- 區分 MCP server vs Tool Use vs Skills vs Plugins + +### 必修閱讀 +1. [**Anthropic — Introducing MCP**](https://www.anthropic.com/news/model-context-protocol) — 最初發表，概念總覽 +2. [**MCP Specification**](https://modelcontextprotocol.io/specification) — 實際的協定規格 +3. [**Complete Guide to MCP in 2026**](https://dev.to/x4nent/complete-guide-to-mcp-model-context-protocol-in-2026-architecture-implementation-and-4a11) — 實作導讀 +4. [**Microsoft — MCP for Beginners**](https://github.com/microsoft/mcp-for-beginners) — 官方循序漸進 MCP 學習課綱（概念、裝設、動手 lab；免費、GitHub 上）。★ 16k+ + +### 動手練習 +- **練習：MCP client** — 安裝 `modelcontextprotocol/servers/filesystem`，從 Claude Desktop 連上去。看著 Claude 讀你的檔案。 +- **練習：MCP server** — 寫一個 Python MCP server，提供一個 tool（例如「換算溫度」）。從 Claude Code 連過去。**step-by-step 怎麼做** → [`resources/cookbook.md` 2](../resources/cookbook.md#2-寫你的第一個-mcp-server) +- **練習：MCP in production** — 在同一個 Claude session 裡同時連 2-3 個 MCP server，看它們互相搭配。 + +### 精選 Projects（spec / SDK / 範本參考） + +> 💡 **找日常工具的 MCP（Notion / Obsidian / Excel / Postgres / Playwright / Figma 等）？** +> 看 [`resources/mcp-skills-catalog.md`](../resources/mcp-skills-catalog.md)——按 16 個分類整理 65+ 個常用 MCP server / Skill，每個都附 stars / license / 適合誰。下表保留的是「**寫自己 MCP server 時的 reference**」性質的官方 server / SDK。 + +| Project | ⭐ | 適合誰 | 為什麼推薦 / 備註 | +|---|---|---|---| +| [modelcontextprotocol/servers](https://github.com/modelcontextprotocol/servers) ⭐ 官方 | ⭐⭐⭐⭐⭐ | 練習 1 接 server、之後當參考 | 20+ 官方 MCP server（filesystem / git / github / sqlite / time / fetch / memory / sequential-thinking），★ 85k+、MIT、TS+Python。**讀 `everything` 跟 `filesystem` source 理解協定運作**。安裝：`npx -y @modelcontextprotocol/server-filesystem /path` 或 `pip install mcp-server-fetch` | +| [modelcontextprotocol/python-sdk](https://github.com/modelcontextprotocol/python-sdk) | ⭐⭐⭐⭐⭐ | 練習 2 寫自己 MCP server | 官方 Python SDK、`pip install mcp` 即裝、MIT。跟著官方 quickstart 跑 | +| [modelcontextprotocol/typescript-sdk](https://github.com/modelcontextprotocol/typescript-sdk) | ⭐⭐⭐⭐ | 喜歡 TS 的人 | Python SDK 的 TypeScript 版、MIT | +| [wong2/awesome-mcp-servers](https://github.com/wong2/awesome-mcp-servers) ⭐ 目錄 | ⭐⭐⭐⭐⭐ | 自己寫前先找有沒有現成的 | 150+ 社群 MCP server 目錄，按 search / code / cloud / communication / finance 分類。投稿走 mcpservers.org | +| [punkpeye/awesome-mcp-servers](https://github.com/punkpeye/awesome-mcp-servers) | ⭐⭐⭐⭐ | 跟 wong2 交叉比對 | 另一份 MCP server 目錄、組織方式不同、通常更新更即時 | +| [github/github-mcp-server](https://github.com/github/github-mcp-server) | ⭐⭐⭐⭐ | 想看實際上線的 MCP server source | GitHub 官方維護、真正 production 在跑的範例 | +| [21st-dev/magic-mcp](https://github.com/21st-dev/magic-mcp) | ⭐⭐⭐ | 做完練習 2 找靈感 | 會生成 UI 元件的非平凡 MCP server、★ 5.3k+、NOASSERTION。**看 MCP 不只能做資料抓取** | +| [yamadashy/repomix](https://github.com/yamadashy/repomix) | ⭐⭐⭐⭐⭐ | 把整個 codebase 餵給 LLM | ★ 26k+ MIT、把 repo packed 成 AI-friendly 單一檔案、含 MCP server mode + tree-sitter 壓縮（省 70% token）+ secretlint 過濾密鑰。**daily-driver 工具，搭 Claude Code / Codex 用** | + +> 🔭 **MCP 在 2026：從「知道是什麼」到「會用生態」**：(1) **官方 Registry**（registry.modelcontextprotocol.io）——發現 / 發佈 MCP server 的中央目錄；(2) **FastMCP**（[jlowin/fastmcp](https://github.com/jlowin/fastmcp)、★25k）——用 `@mcp.tool` 幾行寫出 server，比 low-level SDK 省事；(3) ⚠️ **MCP 安全**：tool 回傳的內容是**不可信輸入**（tool poisoning、confused-deputy），別把沒檢查過的第三方 server 接上有權限的 agent。 + +--- + +## 5.3 — Skills（Claude Code 的行為層）⭐ Claude Code 生態最關鍵的一層 + +### Skill 是什麼（先定位） + +Skill = **一個 markdown 檔**（`.claude/skills//SKILL.md`），告訴 Claude「**遇到某情境 → 走某流程**」。Claude 每次 inference 前掃所有可用 skill 的 `description` frontmatter（檔案開頭那段 YAML metadata）、看是否匹配當前情境、**匹配就把 SKILL.md 自動載入 context**。 + +> 🛠️ **要寫好 SKILL.md？** 兩條路： +> - **路 A：用 Anthropic 官方 `skill-creator` skill 自動產生**（5.3.x 之後安裝段落會教），它會自動產 frontmatter + 子目錄結構、是 Anthropic 維護的 canonical 工具。 +> - **路 B：用下面 SKILL.md 設計 prompts 自己寫**——先看 [Stage 7.5 核心 Harness Engineering 原則](07.5-advanced-agentic-concepts.md#-跨概念-harness-engineering-原則多-source-整理) 建概念、再用 prompt 動手。 +> +> 兩條不衝突：`skill-creator` 給結構、5 原則 prompt 給內容品質檢查。 + +### 📋 SKILL.md 設計 prompts（含 `skill-creator` 替代） + +寫 / 改 SKILL.md 時直接複製貼上： + +#### Prompt 1 — Audit 你現有的 SKILL.md + +``` +我有一個 SKILL.md（在 [貼路徑]），請依下面 5 個 harness engineering 原則做 audit。每條給「PASS / FAIL / PARTIAL」+ 1 行原因 + 1 行改進建議： + +1. Legibility — description 寫清楚「何時觸發」嗎？tool param 命名一致嗎？ +2. Progressive Disclosure — SKILL.md < 200 行嗎？細節是否放 `references/` 而不是塞主檔？ +3. System of Record — `references/` 是 single source、主檔不重複嗎？ +4. Taste Invariants — success criteria 是否寫死可驗證、不是「盡量好」這種主觀詞？ +5. Throughput / Merge — 有附 acceptance check（lint / test / preset YAML）嗎？ + +最後給：總分 X/5、最該先修哪一條、為什麼。 +``` + +#### Prompt 2 — 生成新的 SKILL.md（依 5 原則） + +``` +我要寫一個 skill 處理 [描述任務，例如：把 PDF 轉成 markdown / 跑學術論文 banned-word audit]。請依下面 5 個 harness engineering 原則生成 SKILL.md： + +- **description** 寫清楚「何時觸發」（讓 Claude 能 match 對情境） +- **主檔 < 200 行**，所有 examples / edge cases / detailed rules 放 `references/.md` +- 列出建議的 `references/` 結構（1-3 個 topic 檔案） +- 加一個 **success criteria 表**（可驗證、不主觀） +- 加一段 **acceptance check**：要跑哪些 lint / unit test / preset YAML + +輸出： +1. 完整 SKILL.md 內容 +2. references/ 目錄結構建議 +3. 用哪個 acceptance gate preset 驗證它（如 multi-locale-mirror-sync / catalog-entry-add 之一） +``` + +→ **建議流程**：先 `/skill skill-creator` 拿乾淨骨架 → 用 Prompt 2 填內容 → 寫完用 Prompt 1 audit。 + +**核心 mental model**：你發現自己「**每次都要打同樣的 prompt 教 Claude 怎麼做某件事**」→ 把它寫成 skill、下次就不用了。Claude Code 生態裡 **skill 是 power user 跟一般使用者的分水嶺**——熟練 skill 寫作的人能把 1 個小時的工作壓到 5 分鐘。 + +### Skill vs CLAUDE.md vs MCP vs Plugin vs Subagent — 一張表分清楚 + +各層常被搞混。**一行對照**： + +| 元件 | 是什麼 | 何時用 | 觸發方式 | 範例 | +|---|---|---|---|---| +| **CLAUDE.md**（5.1） | repo / project 的 baseline 行為 | repo-wide convention（「用 type hint」、「commit 訊息規範」）| **每個 session 都載入**、不分情境 | 你 repo 根目錄的 CLAUDE.md | +| **MCP server**（5.2） | 提供 tool / data 的 protocol server | 想讓 Claude 能存取**外部資源**（API / DB / 檔案系統） | server 啟動後、任何時候都能呼叫 | `github` MCP / `postgres` MCP | +| **Skill**（**本節**） | **特定情境的行為包** | 想設定「**遇到 X 情境 → 走 Y 流程**」 | **description 匹配自動載入** | `skill-vetter`（裝 skill 前檢查）/ `pdf`（處理 PDF） | +| **Plugin**（5.4） | 把 skills + commands + MCP + hooks 打包散佈 | 想 share / install **一整套** 設定 | `/plugin install @` | `engineering` bundle / `finance` bundle | +| **Subagent**（5.5） | 獨立 context 的 sub-Claude session | 想 delegate **大 context 任務**、結果回主 session | description 匹配自動 delegate | code-reviewer subagent / 研究員 subagent | + +**怎麼選**： + +- 一句話設定 → 寫進 `CLAUDE.md` +- 多步驟流程、某情境才用 → 寫 **Skill**（本節主題） +- 需要存取外部資源（API / DB） → 寫 **MCP server** +- Skill 跑起來太大、會吃光主 session window → 改成 **Subagent** +- Skill / command / MCP / hook 想打包送人 → 包成 **Plugin** + +→ **核心區分**：MCP 是「**能力**」、Skill 是「**行為**」、Plugin 是「**散佈**」、Subagent 是「**獨立 worker**」。 + +### 學習目標 +- `SKILL.md` 的結構（YAML frontmatter + 本文） +- skill 何時會自動載入（description 比對） +- 怎麼寫一份能解決你日常工作的 SKILL.md +- `references/`、`scripts/`、`evals/` 子目錄的用途 + +### 必修閱讀 +1. [**Anthropic — Claude Skills 文件**](https://docs.claude.com/en/docs/claude-code/skills) +2. **幾份範例 SKILL.md**——從 `anthropics/claude-code` 或社群 marketplace 拿 +3. [**Hello-Agents — Extra08 如何寫出好的 Skill**](https://github.com/datawhalechina/hello-agents/blob/main/Extra-Chapter/Extra08-如何写出好的Skill.md) — 中文最完整的 Skill 最佳實踐 +4. [**Hello-Agents — Extra05 Agent Skills 與 MCP 對比解讀**](https://github.com/datawhalechina/hello-agents/blob/main/Extra-Chapter/Extra05-AgentSkills解读.md) — Skills vs MCP 概念對比 + +### 動手練習 +- **練習：SKILL.md** — 寫一份 200 字的 skill，解決你日常工作中的某一件事。**step-by-step 怎麼做** → [`resources/cookbook.md` 1](../resources/cookbook.md#1-寫你的第一個-skill) +- **練習：SKILL with references** — 加一份 `references/` markdown 讓 skill 可以引用 +- **練習：SKILL eval** — 加 `evals/evals.json`，放 3-5 個自我測試 + +> 📦 **本 repo 自帶 meta-example**：[`examples/stage-5/tool-calling-tutor/`](../examples/stage-5/tool-calling-tutor/) 是這個 stage 的對應 skill 範本——完整 frontmatter（含 trigger phrases + Do NOT use for）、3 份 `references/`、`evals/evals.json` 5 個 test case，**直接 fork 改成你自己的 skill**。雙重用途：(a) 學習者自用、卡在 tool calling 時讓它 auto-load 幫你 debug；(b) Stage 5 5.3 SKILL.md 寫法的對照樣板。 + +### 常用 Skills 推薦（按用途分類） + +> 不知道從哪裡開始？下面是 2025 後段官方 + 社群常用 skill。**安裝方式**：(a) 多數來自 plugin、安裝對應 plugin 即得；(b) 或從 [anthropics/skills](https://github.com/anthropics/skills) clone 後放進 `~/.claude/skills/` 或 `.claude/skills/`。 + +| 用途 | Skill | 來源 | 為什麼推薦 | +|---|---|---|---| +| **🛡 裝 skill 前安全檢查**（必裝） | `skill-vetter` | anthropics/skills | **裝任何外部 skill 前必跑**——檢查紅旗、permission scope、suspicious pattern。等於 marketplace skill 的 SAST | +| **🔍 找 / 安裝 skill** | `find-skills` | anthropics/skills | 自然語言查詢、自動安裝。「我想要做 X」就回對應 skill | +| | `skill-lookup` | claude-plugins-official | 跟 find-skills 互補，探索 / 搜尋 helper | +| **✍ 寫自己的 skill** | `skill-creator` | anthropics/skills + claude-plugins-official | 自動產生 frontmatter + 子目錄結構、寫 skill 必裝 | +| **📄 Office docs 處理** | `pdf` / `docx` / `xlsx` / `pptx` | anthropics/skills | 讀寫 PDF / Word / Excel / PowerPoint。**必裝 set**——任何 office workflow 必備 | +| **🔧 Code review** | `code-reviewer` / `code-review-excellence` | claude-plugins-official | staged diff 安全 / 風格 / 測試 review | +| **🐛 Debug** | `debugger` / `systematic-debugging` | claude-plugins-official | 系統化 root cause 分析、避免 quick fix | +| **🎓 學術寫作** | `academic-writing-skills` | community | findings-first / mechanism / banned word audit | +| **🔌 MCP 整合 / 寫 server** | `mcp-builder` / `mcp-integration` | claude-plugins-official | 寫 MCP server 跟整合既有 server 的腳手架 | +| **💻 frontend / fullstack** | `frontend-developer` / `fullstack-developer` | claude-plugins-official | React 元件 / 全棧架構輔助 | +| **📊 資料分析** | `data-analyst` / `visualization-expert` | community | SQL / pandas / chart 選型 | +| **⚙ 權限 / 設定整理** | `update-config` / `fewer-permission-prompts` | claude-plugins-official | hooks / permissions / env var 管理 | +| **🔁 自我改進** | `self-improving-agent` | community | 捕捉 learning / error / correction、agent 持續改進 | +| **🌐 通用 / fallback** | `general-purpose` | Claude Code 內建 | 複雜開放任務、未涵蓋情境的 default 入口 | + +**建議入手順序**： +1. **第一個必裝**：`skill-vetter`（裝其他 skill 前先用它檢查） +2. **第二批必裝**：`skill-creator` + `find-skills`（寫 / 找 skill 用） +3. **依工作領域**：Office workflow 加 `pdf`/`docx`/`xlsx`、開發加 `code-reviewer`/`debugger`、學術寫作加 `academic-writing-skills` +4. **想看更多**：逛 `obra/superpowers` 或 `wshobson/agents` 看 production 範本 + +### 精選 Projects（spec / 範本參考） + +> 💡 **找日常用 Skill（NotebookLM、Excalidraw、Office docs 等）？** +> 看 [`resources/mcp-skills-catalog.md`](../resources/mcp-skills-catalog.md)——按使用情境分類，含 Anthropic 官方 + 社群 Skill。下表保留的是「**寫自己 Skill 時的 spec / showcase reference**」性質。 + +| Project | ⭐ | 適合誰 | 為什麼推薦 / 備註 | +|---|---|---|---| +| [anthropics/skills](https://github.com/anthropics/skills) ⭐ 官方 spec | ⭐⭐⭐⭐⭐ | 寫自己 SKILL.md 前先讀 | Anthropic 官方 Skills repo：`spec/`（frontmatter 標準）+ `template/` 起手範本 + `skills/` 含 pdf / docx / xlsx / pptx / skill-creator / skill-vetter 等 reference 實作。★ 158k+。**SKILL.md 結構範本參考**。Agent Skills 更廣義標準另見 [agentskills.io](https://agentskills.io) | +| [anthropics/claude-code](https://github.com/anthropics/claude-code) | ⭐⭐⭐⭐ | 追蹤新功能、看 release notes | Claude Code 主 repo、含 issues / releases / inline skill 範例。本 stage 學 Skill 重點看上一個 repo、這個排第二 | +| [mattpocock/skills](https://github.com/mattpocock/skills) | ⭐⭐⭐⭐ | 想看「真實工程師日常 SKILL.md」 | Matt Pocock（TypeScript 社群知名教學者）公開自己工作真實在用的 `.claude/` 目錄。每個 SKILL.md **10-50 行極短**、不過度工程化。**對照 over-engineered 200 行 skill 特別有參考價值**（★ 157k+、MIT）| +| [obra/superpowers](https://github.com/obra/superpowers) | ⭐⭐⭐⭐ | power user setup、學進階寫法 | 20+ 實戰 skill（TDD、debugging、合作模式）+ `/brainstorm` / `/write-plan` / `/execute-plan` 命令 + skills-search tool | +| [wshobson/agents](https://github.com/wshobson/agents) | ⭐⭐⭐⭐ | 中階：學 skill + subagent 組合 | 把 skills + subagents 組合做 multi-agent 編排。**從單一 SKILL.md 進化到 agent-as-skill 組合 pattern** 的範例（★ 35k+、MIT） | +| [travisvn/awesome-claude-skills](https://github.com/travisvn/awesome-claude-skills) | ⭐⭐⭐⭐ | 自己寫前先找有沒有現成的 | 社群 Claude Skills 精選目錄 | +| [VoltAgent/awesome-agent-skills](https://github.com/VoltAgent/awesome-agent-skills) | ⭐⭐⭐ | 跨工具視角 | 1000+ agent skill、相容 Claude Code / Codex / Gemini CLI / Cursor（★ 26k+、MIT）| +| [alirezarezvani/claude-skills](https://github.com/alirezarezvani/claude-skills) | ⭐⭐⭐ | 找特定領域 skill 範例 | 232+ Claude Code skill、跨 engineering / marketing / product / compliance | + +--- + +## 5.4 — Plugins 與 Marketplaces + +### Plugin 是什麼（先定位） + +**Plugin = MCP + Skills + slash commands + hooks 的組合包**——把前面 5.2 / 5.3 學到的零件 **打包成一個單位、可以 `/plugin install` 一次裝進去**。 + +``` +Plugin +├── .mcp.json ← 5.2 學的 MCP server config（提供 tool / data） +├── skills//SKILL.md ← 5.3 學的 skill（行為包） +├── commands/.md ← slash command（5.1 學的、自訂 prompt 入口） +├── hooks/ ← 觸發點 hook（譬如 PreToolUse、SessionStart） +├── agents/.md ← 5.5 學的 subagent（如果有） +└── .claude-plugin/plugin.json ← 打包元資料 +``` + +**為什麼要 plugin**：你寫了好用的 skill 想 share → 一行 `git clone` 太麻煩、設定也容易裝錯。包成 plugin、push 到 marketplace、team 其他人 `/plugin install foo@your-marketplace` 一次到位。 + +**Plugin 跟 marketplace 差在哪**：plugin 是**單一打包單位**、marketplace 是**多個 plugin 的目錄**（譬如 anthropics/claude-plugins-official 是 marketplace、裡面 35 個 plugin）。 + +### 學習目標 +- `plugin.json` schema（name、version、skills array、configuration） +- `marketplace.json` schema（plugins array、source、metadata） +- `claude plugin marketplace add` 的流程 +- 區分 single-plugin bundle vs multi-plugin marketplace +- 發佈自己的 marketplace + +### 必修閱讀 +1. [**Anthropic — Plugins 文件**](https://docs.claude.com/en/docs/claude-code/plugins) +2. **讀下面 2-3 個 marketplace 的 `plugin.json` 與 `marketplace.json`** + +### 動手練習 +- **練習：plugin install** — 安裝下面的某一個 marketplace，看它載入 +- **練習：plugin.json** — 把 5.3 寫的 SKILL.md 打包成一個 plugin +- **練習：marketplace publish** — push 到 GitHub，用 `claude plugin marketplace add` 安裝 + +### 常用 plugin 推薦（按用途分類） + +> 不知道從哪裡開始裝 plugin？下面是 2025 後段 Anthropic 官方 + 社群高評價選擇。**安裝指令統一格式**：`/plugin install @`（譬如 `/plugin install code-review@claude-plugins-official`）。 + +| 用途分類 | Plugin（含直接連結） | Marketplace | 為什麼推薦 | +|---|---|---|---| +| **開發 workflow**
（多數開發者必裝） | [`code-review`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/code-review) | claude-plugins-official | 官方 code review skill 集合、staged diff review + security check | +| | [`pr-review-toolkit`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/pr-review-toolkit) | claude-plugins-official | PR review 完整流程（comment、suggest、approve）| +| | [`commit-commands`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/commit-commands) | claude-plugins-official | git commit message 規範 + branching workflow | +| | [`feature-dev`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/feature-dev) | claude-plugins-official | 完整 feature 開發 cycle（spec → plan → implement → test） | +| | [`frontend-design`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/frontend-design) | claude-plugins-official | UI 設計 + responsive layout 輔助 | +| **語言工具**
（依用的語言挑）| [`typescript-lsp`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/typescript-lsp) / [`pyright-lsp`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/pyright-lsp) / [`rust-analyzer-lsp`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/rust-analyzer-lsp) / [`gopls-lsp`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/gopls-lsp) 等 | claude-plugins-official | 各語言 LSP 整合、[35 個語言 plugin](https://github.com/anthropics/claude-plugins-official/tree/main/plugins) 都在這 | +| **plugin / skill 自建** | [`skill-creator`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/skill-creator) | claude-plugins-official | 寫自己的 skill 時自動產生 frontmatter + 結構 | +| | [`plugin-dev`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/plugin-dev) | claude-plugins-official | 寫自己的 plugin 時自動產生 `.claude-plugin/` 結構 | +| | [`mcp-server-dev`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/mcp-server-dev) | claude-plugins-official | 寫自己的 MCP server 時的腳手架 | +| | [`hookify`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/hookify) | claude-plugins-official | 寫 hooks 規則的工具 | +| **領域特化 — 工程團隊** | [**`engineering` bundle**](https://github.com/anthropics/knowledge-work-plugins/tree/main/engineering) | knowledge-work-plugins | **10 個 skill**：architecture / code-review / debug / deploy-checklist / documentation / incident-response / standup / system-design / tech-debt / testing-strategy | +| **領域特化 — 財務團隊** | [**`finance` bundle**](https://github.com/anthropics/knowledge-work-plugins/tree/main/finance) | knowledge-work-plugins | **8 個 skill**：audit-support / close-management / financial-statements / journal-entry-prep / reconciliation / sox-testing / variance-analysis | +| **領域特化 — 其他**
（同 marketplace）| [`sales`](https://github.com/anthropics/knowledge-work-plugins/tree/main/sales) / [`marketing`](https://github.com/anthropics/knowledge-work-plugins/tree/main/marketing) / [`legal`](https://github.com/anthropics/knowledge-work-plugins/tree/main/legal) / [`human-resources`](https://github.com/anthropics/knowledge-work-plugins/tree/main/human-resources) / [`customer-support`](https://github.com/anthropics/knowledge-work-plugins/tree/main/customer-support) / [`data`](https://github.com/anthropics/knowledge-work-plugins/tree/main/data) / [`design`](https://github.com/anthropics/knowledge-work-plugins/tree/main/design) / [`operations`](https://github.com/anthropics/knowledge-work-plugins/tree/main/operations) / [`product-management`](https://github.com/anthropics/knowledge-work-plugins/tree/main/product-management) / [`productivity`](https://github.com/anthropics/knowledge-work-plugins/tree/main/productivity) / [`bio-research`](https://github.com/anthropics/knowledge-work-plugins/tree/main/bio-research) 等 | knowledge-work-plugins | knowledge-work-plugins **[18 個 vertical bundle](https://github.com/anthropics/knowledge-work-plugins)**——挑跟你工作領域對應的那個 | +| **外部整合**
（第三方服務） | [`asana`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/asana) / [`github`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/github) / [`gitlab`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/gitlab) / [`linear`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/linear) / [`firebase`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/firebase) / [`playwright`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/playwright) / [`terraform`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/terraform) / [`discord`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/discord) / [`imessage`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/imessage) / [`telegram`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/telegram) 等 | claude-plugins-official (external) | 整合常用 SaaS / 開發工具 | +| **community 廣度** | （挑感興趣的 skill） | [rohitg00/awesome-claude-code-toolkit](https://github.com/rohitg00/awesome-claude-code-toolkit) | 社群最大 agents / skills / hooks / templates 目錄 | + +**建議入手順序**： +1. 開發者必裝（5 個）：`code-review` + `pr-review-toolkit` + `commit-commands` + `feature-dev` + 一個你語言的 `*-lsp` +2. 按工作領域加 bundle：工程團隊裝 `engineering`、財務裝 `finance`、其他類似 +3. 想寫自己的 skill / plugin → 裝 `skill-creator` + `plugin-dev` +4. 想看更多 → 逛 `awesome-claude-code-toolkit` 或 [`resources/mcp-skills-catalog.md`](../resources/mcp-skills-catalog.md) + +### 精選 Projects（marketplace 範本參考） + +> 💡 上面列的是「**裝哪些 plugin**」；下表列的是「**marketplace 怎麼寫**」——想自建 marketplace 的人才需要看。 + +| Marketplace | ⭐ | 適合誰 | 為什麼推薦 / 備註 | +|---|---|---|---| +| [anthropics/claude-plugins-official](https://github.com/anthropics/claude-plugins-official) | ⭐⭐⭐⭐⭐ | 寫自己的 marketplace 前的官方範本 | 35 internal plugins + 15 external、`.claude-plugin/marketplace.json` 標準 schema、`plugins/` 含 plugin 本體 + `external_plugins/` 引用外部 repo。**marketplace.json 該長什麼樣直接看這個**（★ 30k+） | +| [anthropics/knowledge-work-plugins](https://github.com/anthropics/knowledge-work-plugins) | ⭐⭐⭐⭐⭐ | 想看「多 vertical bundle」型 marketplace | **18 個領域 plugin bundle**（finance / engineering / sales / legal / marketing / HR / customer-support / data / design / operations / product / productivity / bio-research / enterprise-search / pdf-viewer / small-business / cowork-plugin-management / partner-built）。Anthropic 自家 knowledge worker 場景範本 | +| [obra/superpowers-marketplace](https://github.com/obra/superpowers-marketplace) | ⭐⭐⭐⭐ | 想做「我策展、別人寫」型 marketplace | **最簡 marketplace template**——repo 只有 `marketplace.json` + README、plugin 本體放外部 repo。curator-only pattern 最小範本（★ 1.1k+、MIT）| +| [trailofbits/skills-curated](https://github.com/trailofbits/skills-curated) | ⭐⭐⭐ | 在意供應鏈安全的 reviewer / 團隊 | Trail of Bits 維護的 **security-vetted** marketplace、每個 skill 都經審查、README 寫清楚標準。**示範 marketplace 不只是清單、也是信任機制**（★ 431、CC-BY-SA-4.0）| +| [rohitg00/awesome-claude-code-toolkit](https://github.com/rohitg00/awesome-claude-code-toolkit) | ⭐⭐⭐ | 想逛社群有什麼 | 社群最大 Claude Code agents / skills / hooks / templates 目錄。涵蓋 use case 廣 | +| [anthropics/life-sciences](https://github.com/anthropics/life-sciences) | ⭐⭐⭐ | 要做特定領域 marketplace（醫療、金融、法律、教育等） | Anthropic 自家**領域特化 marketplace** 範例（生物 / 健康科學）、展示 `marketplace.json` 為單一 vertical 量身設計。**payload 偏生科 MCP server、marketplace.json 結構才是學習重點**（★ 474）| +| [anthropics/claude-for-legal](https://github.com/anthropics/claude-for-legal) | ⭐⭐⭐⭐ | 想看完整 vertical plugin suite（含 skills + agents + MCP + scheduled agents）| **Anthropic 官方法律 vertical 範例**（★ 7.9k+ Apache-2.0）—— 10 個法律 plugin（commercial / corporate / litigation / privacy / employment / IP / law-student）+ 100+ skills + 20+ MCP connectors + scheduled agents + subagent delegation。**不必懂法律**——是學「**vertical plugin suite 怎麼設計**」的最佳教材：系統 prompt 怎麼寫、accountability surface 怎麼擺、`orchestrate.py` event loop 怎麼跑 | + +> 💡 **「如何發佈自己的 marketplace」walkthrough**：目前最可靠的是 [Anthropic 官方 plugin 文件](https://docs.claude.com/en/docs/claude-code/plugins)。社群有好的部落格 / repo？歡迎開 PR 補上。 + +--- + +## 5.5 — Subagents（Claude Code 原生 multi-agent 機制）⭐ 2025 新功能 + +到這裡為止你學了 MCP（工具層）/ Skills（行為層）/ Plugins（散佈層）。**Subagents 是 orchestration 層**（orchestration = 調度一群 agent：分工，再把結果合起來）——讓主 Claude session spawn 出有獨立 context 的子 agent、跑特定任務、回報結果。 + +![Subagent 的 4 個生命週期：從 .md 檔到執行結果](../resources/diagrams/subagent-4-stage-flow.png) + +> 📊 **上圖**：subagent 從**定義 → 發現 → 派遣 → 執行** 4 個階段、看完這張再讀下面細節最快。 + +跟 Stage 4 的 framework-based multi-agent（LangGraph / CrewAI / AutoGen）對照： + +| 維度 | Framework path (Stage 4) | Claude Subagent path（本節） | +|---|---|---| +| 啟動方式 | `pip install crewai` + Python code | 寫一個 `.claude/agents/.md` 即可 | +| Runtime | 你自己的 Python process | Claude Code 內建 Task tool | +| Context isolation | framework 自己管 | **天生** 各 subagent 獨立 window | +| Provider lock-in | 中等（多 framework 支援 multi-LLM） | **強**（綁 Claude Code） | +| 適合 | 跨 LLM provider 的 production system | 已 commit Claude Code 的工程團隊 | +| 學習曲線 | 高（框架抽象 + async） | 低（寫 markdown）| + +### 各家 CLI / SDK 的 multi-agent 機制現況（2025 後段） + +很多人以為 multi-agent CLI 是 Anthropic / OpenAI / Google 三家標配——但實際上目前只有 **Claude Code 有完整 native multi-agent stack**。Codex CLI / Gemini CLI / Cursor 都還是 single-agent，要 multi-agent 得自己用 SDK 或 framework 寫。 + +| 平台 | Subagent | Agent team | Background agent | 機制 | +|---|:---:|:---:|:---:|---| +| **Claude Code**（CLI） | ✅ | ✅ | ✅ | `.claude/agents/.md` + Task tool（subagent）+ [agent teams](https://docs.claude.com/en/docs/claude-code/agent-teams) + [agent view / background](https://docs.claude.com/en/docs/claude-code/agent-view) | +| **OpenAI Codex CLI** | ❌ | ❌ | ❌ | `AGENTS.md` 只是 **single-agent context file**（類似 CLAUDE.md），**不是 subagent 系統** | +| **Google Gemini CLI** | ❌ | ❌ | ❌ | `GEMINI.md` 只是 context；無 subagent / multi-agent feature | +| **Cursor**（IDE-coupled） | ❌ | ❌ | ❌ | 單一 Cursor Agent；queued messages 是 sequential、非 parallel | +| **OpenAI Agents SDK**
（programmatic、非 CLI） | ⚠️ Handoffs + agents-as-tools | ❌ | ❌ | 純 Python SDK、不是 CLI；handoff pattern 接近 Claude subagent 但要寫 code | +| **Framework path**
（Stage 4） | LangGraph / CrewAI / AutoGen | ✅ 自己 wire | 部分 | 跨 LLM provider、Python orchestration、見 [Stage 4](04-agent-frameworks.md) | + +**現況解讀**： + +- 想用 **CLI** 玩 multi-agent → 目前只有 Claude Code 有 native 支援（**本節主題**） +- 想 **跨 provider / 跨 LLM** → 走 Stage 4 framework path +- 想 **OpenAI 生態 + 多 agent** → 用 OpenAI Agents SDK 寫 handoff pattern（programmatic、非 CLI） +- 想 **完全自己控** → 走 [Stage 5.7 Harness Internals](#57--claude-code-source-解剖reference-harness-implementation-track-b-必看)（讀 SDK source、自己 wire 多 agent） + +→ 本節剩下內容都聚焦在 **Claude Code subagent**。其他平台的進展請追蹤各家 changelog（Codex / Gemini / Cursor 都還在 single-agent + MCP 階段、可能 2026 後段才會跟進）。 + +### 怎麼派遣 Claude Code 的 3 種 multi-agent 機制（具體 syntax） + +| 機制 | 何時用 | 派遣方式 | +|---|---|---| +| **Subagent**
（穩定版） | delegate 大 context 任務（讀整個 codebase / 整理 logs）給 isolated context worker、結果回主 session | (1) 寫 `.claude/agents/.md`（frontmatter `name` + `description` + `tools` + 可選 `model`）
(2) Claude 看 description **自動 delegate**；或 `/agents` 手動列表 | +| **Agent team**
（已有正式 docs、仍需 opt-in flag） | 多 worker 之間要**互相溝通**、challenge 彼此（debate / peer review / 多角度探索） | (1) **啟用**（仍需 opt-in）：`settings.json` 加 `"env": {"CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS": "1"}`、需 Claude Code v2.1.32+
(2) 自然語言派遣：`Create an agent team to explore X from different angles: one on UX, one on architecture, one playing devil's advocate`
(3) 跟 teammate 對話：`Shift+Down` 切換、直接輸入訊息
(4) 收尾：`Clean up the team` | +| **Background agent**
（research preview） | 多個**獨立任務**各自背景跑、單一介面監控（同時 3 個 PR review） | (1) shell 派遣：`claude --bg "investigate the flaky test"`（需 v2.1.139+）
(2) 從現有 session 背景化：`/bg`
(3) 監控：`claude agents`（agent view 介面）
(4) 操作：`claude attach ` / `claude logs ` / `claude stop ` | + +**3 個機制怎麼選**： + +- 任務獨立、worker 不互動、結果回主 session 即可 → **Subagent**（最簡單、token 最省） +- Worker 需要互相溝通 / debate / 共享 task list → **Agent team**（已正式有 docs、但仍需 opt-in env var；token 3-5x、適合 research / debug 競爭假設） +- 多個獨立任務各自跑、想用 1 個介面監控全部 → **Background agent**（research preview、適合長時間任務並行） + +--- + +### 可派遣的 subagent 有哪些？ + +> 💡 **先解釋一下名詞**：**subagent** = 主 Claude session spawn 出來的「子 Claude」——有自己的 context window（一次能記住的對話量、有上限）、跑完回報結果。**派遣（dispatch）**就是叫 subagent 去做事、像派任務給同事。 + +很多人以為要用 subagent 都得自己寫一個——其實 **Claude Code 內建一批 subagent、開箱即用**。下表列三種來源： + +| 來源 | 範例 subagent | 何時用 | 需要做什麼 | +|---|---|---|---| +| **Claude Code 內建** | `general-purpose` / `code-reviewer` / `Explore` / `Plan` / `frontend-developer` / `claude-code-guide` / `statusline-setup` | 一般任務都先看內建有沒有合的 | **什麼都不用做、直接呼叫** | +| **plugin / marketplace** | `obra/superpowers` 內含的 skill agent、`wshobson/agents` 的多 subagent 組合 | 內建不夠用時 | 裝 plugin / marketplace（[Stage 5.4](#54--plugins-與-marketplaces)）| +| **自己寫的** | 你公司流程 specific 的 reviewer / domain expert | 上面都不符合時 | 寫 `.claude/agents/.md`（範例見下面 details 區塊）| + +> 🔍 **想知道你的 Claude Code 現在有哪些 subagent 可用？** 終端機跑 `/agents` 一指令列表（內建 + plugin + 自訂全部）。 + +### 怎麼選哪一個 subagent？（decision table） + +對應上面 7 個 Claude Code 內建 subagent、下表是「**遇到 X 任務、用 Y subagent**」對照（這叫 **decision table**——「要 X 用 Y」的快速對照、不用想自己會選）： + +| 你要做的事 | 用哪個內建 subagent | 為什麼 | +|---|---|---| +| 找 code / 探索陌生 codebase 結構 | `Explore` | 專門做 read-only 搜尋、不會亂改 | +| 設計實作 plan（不直接寫 code） | `Plan` | 輸出 step-by-step 計畫、適合大任務拆解前 | +| Review staged diff / 安全審查 / 發 commit 前檢查 | `code-reviewer` | 結構化輸出 PASS/FAIL + 具體 fix | +| 寫 / 改 UI component / 處理 accessibility（無障礙設計）| `frontend-developer` | React / 響應式 / a11y（accessibility 縮寫、視障 / 鍵盤使用者也能用的設計）領域知識 | +| 多步驟研究、不確定任務該歸哪類 | `general-purpose` | 通用、可 web search、適合 fallback | +| 問 Claude Code 自己的 feature 怎麼用 | `claude-code-guide` | hooks（工具執行前 / 後的攔截腳本、見下方 Gotcha #5）/ slash command（`/` 開頭的指令）/ MCP 等問題 | +| 上面都不符合 | 自己寫 `.claude/agents/.md` | 客製或公司 specific 流程 | + +**5 個常見情境的 mini cookbook**（完整 15 個 recipe 見下面）： + +| 情境 | 用哪個 | +|---|---| +| 寫了 ≥ 50 行新 code、要 commit 前 | `code-reviewer` | +| Clone 完新 repo、不知該從哪個 file 開始 | `Explore` | +| 4 個 stage / branch 都要做同樣審查 | `general-purpose`（spawn 多個並行）| +| 想重構 module、先 review architecture | `Plan` | +| 多 source 比對哪 paper 講的對 | `general-purpose` 跑 deep research | + +> 📋 **完整 15 個 recipe**（每個含**情境 + subagent + 直接複製貼上的 prompt 範本 + 何時不用**）→ [`resources/subagent-cookbook.md`](../resources/subagent-cookbook.md) + +### 易混淆觀念釐清（學完表格還是有點霧、看這節） + +學生最常搞混的 **3 組概念** + **5 條老手才知道的 gotcha**——挑你需要的看： + +#### Subagent vs Skill — 5 個關鍵差別 + +很多人把 Subagent 跟 Skill 當同一件事——其實**完全不同層的東西**： + +![Subagent vs Skill — 5 個關鍵差別](../resources/diagrams/subagent-vs-skill.png) + +| 維度 | Subagent（子 agent） | Skill（技能） | +|---|---|---| +| **執行環境** | 新的獨立 context window（底層是新 subprocess）| 主 session 內、同 context | +| **工具權限** | 自己的 `tools:` 清單（可限制只能 Read / Grep）| 主 session 的工具（預設全開、skill 可用 `allowed-tools:` 縮減）| +| **回傳結果** | 一個 final message 摘要回主 session | 沒回傳、是行為改變（規則 / persona）| +| **適合做** | 長任務 / 平行跑 / 要 context 隔離 | 知識注入 / 規則 / 改 Claude 行為 | +| **範例** | `code-reviewer` / `Explore` / `Plan` | `codex-delegate` / `pdf`（anthropics/skills）| + +**判斷快速辦法**：你**要新 context window** 嗎？要 → subagent；不要 → skill。 + +#### Subagent vs Slash Command — 一個是任務、一個是指令 + +| 東西 | 怎麼觸發 | 例子 | +|---|---|---| +| **Subagent** | 直接打對話文字、Claude 看 description 自動派遣 | 你打「Review my staged changes」→ 自動派 `code-reviewer` | +| **Slash command** | 打 `/` 開頭的指令 | `/agents`（列 subagent）/ `/compact`（壓縮 context）/ `/help` | + +⚠️ **常見誤會**：`/agents` **不是用來呼叫 subagent**——它是「查當前可用 subagent 清單」的指令。**派遣是直接打對話 prompt 文字**、Claude 自己挑 subagent。 + +#### Description = 路由 key（**寫法決定能不能被選**） + +主 session 怎麼知道該派哪個 subagent？看 `.claude/agents/.md` 的 **`description` 欄位**。**寫法影響觸發行為**： + +| Description 寫法 | 觸發模式 | 例 | +|---|---|---| +| `...use **PROACTIVELY** when X...` | **主動觸發**——X 出現 Claude 自己派 | "use PROACTIVELY when reviewing diffs ≥ 50 lines" | +| `...use when user asks Y...` | **被動觸發**——要使用者明白要求 | "use when user asks for code review" | +| 空 description | **隱形**——不會被自主選 | （只能在程式碼裡用 `Agent(subagent_type=...)` 強制呼叫）| + +> 💡 **寫 description 像寫廣告詞**——把「我能解決什麼問題」**寫具體**、Claude 越會在對的時機選你。`PROACTIVELY` 是個**強訊號詞**——出現時 Claude 推斷「適合主動派遣」的機率大幅提升；沒寫就更常只在使用者明白要求時才會派。（它影響 Claude 的判斷、**不是程式碼層的 if-then 開關**。） + +#### 5 條老手才知道的 Gotcha + +| # | Gotcha | 為什麼重要 | +|---|---|---| +| 1 | **Description 寫精準即可** | 無官方字元上限、但過長 description 佔 context budget；建議「觸發條件 + 適用情境」寫具體、避免重複 | +| 2 | **`tools:` 寫空 = 繼承主 session 全部工具** | 想限制 subagent 就要**明寫**工具清單；空字段 ≠ 沒工具 | +| 3 | **不寫 `model:` = 跟主 session 用同 model** | 主 session 是 Opus、subagent 沒指定也 Opus（燒大錢）。省成本就寫 `model: sonnet` 或 `model: haiku`| +| 4 | **Subagent 沒「我之前說過 X」記憶** | 每次派遣都是**全新 context**、看不到主 session 對話。Prompt 要 self-contained、不能 reference「我們剛討論的 Y」 | +| 5 | **Subagent 也吃 hook** | PreToolUse / PostToolUse（工具執行前 / 後的攔截腳本）在 subagent 內**也會 fire**。設 hook 時要想到這層 | + +#### Subagent 整體優缺點（讀完前面、回頭看這個 summary） + +**5 個優點**（為什麼存在）： + +| 優點 | 怎麼幫到你 | +|---|---| +| **Context 隔離** | 主 session window 不被污染——subagent 跑大檔案 / 長 log 不會擠掉主 session 的工作記憶 | +| **Tool allowlist** | 限制 subagent 只能用 Read / Grep（不能寫檔 / 不能跑 Bash）= 安全 sandbox | +| **Model override** | 跑簡單任務用 Haiku、跑難的用 Opus、混搭省成本——主 session 是 Opus 也可以叫 subagent 用 Haiku | +| **Parallel spawn** | 一個 prompt spawn N 個 subagent 平行跑、wall clock 時間 ÷ N（適合 4 個 file 同時 audit）| +| **專業化 prompt** | code-reviewer 永遠只 review、description 寫死「Use PROACTIVELY when commit」、不會被閒聊干擾 | + +**5 個缺點**（什麼時候不值得）： + +| 缺點 | 影響 | +|---|---| +| **Spawn 有 overhead** | 任務 < 5 分鐘、自己跑更快——subagent startup 也吃時間跟 token | +| **無 cross-call memory** | 每次 spawn 都新 context、看不到「我們剛討論的 X」——prompt 必須 self-contained | +| **只回一個 message** | subagent 是「派出去、跑完回報一次」、不能跟你來回對話、不適合需要逐步 feedback 的任務 | +| **Token cost N ×** | spawn 4 個 = 用 4 倍 token——parallel 的 ROI 要算（時間省、錢花更多）| +| **Debug 多一層** | 出錯不知該怪主 session description / subagent system prompt / 還是 prompt 本身——見 [advanced §3 debug 5 切點](../resources/subagent-advanced.md#3-自製-subagent-的-debug-工具)| + +> 📌 **1 句話判斷**：任務 **≥ 5 分鐘** + **可以用一個 brief 寫死**（不需來回對話）+ **結果一次回來夠用**（不需逐步 feedback）→ 用 subagent；否則自己跑。 + + +> 📋 **準備自己寫 subagent / 組合多個 / debug 跑壞的？** → [`resources/subagent-advanced.md`](../resources/subagent-advanced.md)（description 寫法 4 個 bug 對照、composition 3 種 pattern、debug 5 切點） + + +
+👉 具體 subagent 檔案範例（最簡單入門） + +`.claude/agents/code-reviewer.md`： + +```markdown +--- +name: code-reviewer +description: Review staged git changes for security issues, style violations, and missing tests. Use when user asks "review my changes" or runs /review. +tools: + - Read + - Grep + - Bash +model: claude-haiku-4-5 # 可選、想 route 到便宜 model 省成本 +--- + +You are a senior code reviewer. When invoked: +1. Run `git diff --cached` to get staged changes +2. Check for: hard-coded secrets, SQL injection patterns, missing error handling, missing tests +3. Output: PASS / list of specific issues with file:line references +``` + +主 session 之後輸入「review my changes」，Claude 看到 description 匹配、自動透過 Task tool（Claude Code 內部派遣機制、你不用直接呼叫）spawn 這個 subagent 跑、回主 session 一段摘要。 + +
+ +> 📚 **官方完整文件**： +> - [Subagent spec](https://docs.claude.com/en/docs/claude-code/sub-agents)（frontmatter 欄位、project vs user scope、Task tool 介面） +> - [Agent team 完整指南](https://docs.claude.com/en/docs/claude-code/agent-teams)（display modes、task list、subagent-as-teammate 進階） +> - [Agent view / background](https://docs.claude.com/en/docs/claude-code/agent-view)（v2.1.139+、quick start + dispatch 流程） + +### 學習目標 + +- 講得出 subagent 跟 skill / MCP server 的差別（**subagent ≠ skill**：skill 是行為 prompt，subagent 是**另一個 Claude instance with isolated context**） +- 寫一個 `.claude/agents/.md` 自訂 subagent（frontmatter + system prompt + `tools:` 白名單——明寫允許的工具清單） +- 從主 session 用 Task tool invoke subagent，觀察 context 隔離（parent 看不到 subagent 的中間 step、只看到最終 result） +- 知道何時用 subagent（parallel research / large-context isolated task / specialized review），何時不用（小 query 用 skill 即可） + +### 必修閱讀 + +1. [**Anthropic — Claude Code Subagents 官方文件**](https://docs.claude.com/en/docs/claude-code/sub-agents) ⭐ — `.claude/agents/` 結構、Task tool 介面、最佳實踐 +2. [**Anthropic — Building Effective Agents orchestrator-workers**](https://www.anthropic.com/engineering/building-effective-agents) — Anthropic 自己對 orchestrator pattern 的看法（理論 + 實例） +3. [**Anthropic Cookbook — `customer_service_agent`**](https://github.com/anthropics/claude-cookbooks/tree/main/tool_use) — canonical multi-agent orchestration 範例（chapter-length 深度教材；notebook 在 `tool_use/customer_service_agent.ipynb`） + +### 動手練習 + +- **練習：第一個 subagent** — 寫 `.claude/agents/code-reviewer.md`（前置 frontmatter 含 `description` 寫清楚何時 trigger、`tools` 限定 Read+Grep）+ system prompt 跑 staged diff review。從主 Claude session 跑 `/agents` list 確認載入、然後用 prompt「review staged changes」觀察 Task tool 怎麼 spawn subagent +- **練習：parallel subagent crew** — 寫 3 個 subagent（`researcher.md` / `writer.md` / `critic.md`）做「研究某主題 → 寫 blog 草稿 → 審稿」pipeline、主 session 用 Task tool 串起來。**對照** [`examples/stage-4/02-multi-agent-roles/`](../examples/stage-4/02-multi-agent-roles/)（CrewAI 框架版同一個任務）、看「framework 路線 vs Claude 原生路線」程式碼差別 +- **練習：subagent 跟 skill 的決策練習** — 拿你自己日常工作流的 5 個常用任務、每個判斷該用 skill（行為層）還是 subagent（獨立 context 層）。寫成 1 頁 decision table + +> 📚 **想要 chapter-length 深入版**：subagent 進階 pattern（agent-as-skill composition、parallel-spawn、handoff between subagents）→ 看 [`wshobson/agents`](https://github.com/wshobson/agents) repo 整個結構 + [`obra/superpowers`](https://github.com/obra/superpowers) 的 subagent 用法。 + +### 精選 Projects + +4 個項目一張表搞定。**挑入口看「適合誰」、想深入點連結看 repo**。 + +| Project | ⭐ | 適合誰 | 為什麼推薦 / 備註 | +|---|---|---|---| +| [anthropics/claude-cookbooks](https://github.com/anthropics/claude-cookbooks) ⭐ 官方 | ⭐⭐⭐⭐⭐ | 5.5 完成後想看「實際在用的 agent 範例怎麼寫」 | Anthropic 官方 chapter-length 範例。**`tool_use/customer_service_agent.ipynb`** = orchestrator-workers canonical（multi-agent routing + handoff）。Python / Jupyter notebook、MIT。**註**：`computer_use_demo` 完整版在另一個 repo [`claude-quickstarts/computer-use-demo`](https://github.com/anthropics/claude-quickstarts/tree/main/computer-use-demo) | +| [wshobson/agents](https://github.com/wshobson/agents) ⭐ subagent canonical | ⭐⭐⭐⭐⭐ | 寫過 1-2 個 subagent 想看真實 team 範本 | 50+ subagent definition 的 production workflow pattern collection。**看 `.claude/agents/` 目錄結構 + 命名 convention + 跨 agent handoff 寫法** | +| [obra/superpowers](https://github.com/obra/superpowers) | ⭐⭐⭐⭐ | 想看 skill + subagent 混搭實作 | 在 Stage 5.3 已介紹。**重點看「什麼任務歸 skill、什麼歸 subagent」決策**——production 範本 | +| [anthropics/claude-plugins-official](https://github.com/anthropics/claude-plugins-official) 官方 | ⭐⭐⭐⭐ | 看 plugin 怎麼打包 subagent | 在 Stage 5.4 已介紹。每個 plugin 內 `agents/` 子目錄是 subagent definition、看打包方式 | + +> 💡 **Subagent 雖然強、不要無腦用**：每個 subagent invoke 都是一個新的 Claude inference call、有 token cost + latency。**簡單 query 用 skill（行為 prompt）即可、不必 spawn subagent**。Subagent 的甜蜜點是：(1) 任務 context 大、會吃光主 session 的 window（譬如 read 整個 codebase），(2) 任務跟主 session 邏輯獨立、隔離 context 有助 main flow，(3) 多 subagent 平行（research / write / critic）能省 wall-clock 時間。 + +> 🔗 **相關進階機制**（Claude Code 官方、本 stage 不深入講）： +> - **[Agent teams](https://docs.claude.com/en/docs/claude-code/agent-teams)** — 多 sessions 之間互相溝通（reviewer agent ↔ implementer agent 來回交流） +> - **[Background agents / agent view](https://docs.claude.com/en/docs/claude-code/agent-view)** — 多 session 背景跑、單一介面監控（一次 spawn N 個 PR review 同時跑） +> +> Subagent 是這兩個的進入點——本節學完之後想擴展再看官方文件。 + +--- + +## 5.6 — Dynamic Workflows（讓 Claude 自己寫出 workflow）⭐ Opus 4.8+ 新機制 + +> **本節定位**：5.5 教你**手動**派 subagent；本節更上一層——**讓 Claude 自己生成一份 workflow 腳本、再自己執行**。這是 Opus 4.8 起的新機制（research preview 出身），新版 Claude Code 內建。本節只把它放進生態地圖、講清楚跟 5.5 的分工；**機制 / 實例 / quality pattern 的完整版在 [Stage 7.5 — Dynamic Workflows 深入](07.5-advanced-agentic-concepts.md#-dynamic-workflowsopus-48-當-agent-自己寫出-workflow)**。 + +### 跟 5.5 Subagents 的差別 + +| | 5.5 Subagents | 5.6 Dynamic Workflows | +|---|---|---| +| 步驟誰決定 | 你手動派、一次一個 | Claude 自己寫出多步驟腳本 | +| 控制流 | model 即興決定下一步 | 腳本裡是**確定性**的 loop / 平行 fan-out / 驗證階段 | +| 適合 | 少數幾個並行子任務 | 大型、要窮舉或多階段驗證（migration、audit、跨檔 review）| +| 關係 | — | DW **建在 subagent 之上**：workflow 腳本去 orchestrate 一群 subagent | + +### 什麼時候用、什麼時候別用 + +- **用**：要窮舉 + 對抗式驗證（找完所有 bug、每個 finding 再派獨立 agent 反駁）、一次性大遷移、跨多檔同樣轉換的 pipeline。 +- **別用**：只是想叫一兩個 agent 平行做點事 → 留在 5.5 就好；小任務直接一條 prompt 更省。 +- ⚠️ DW 會 spawn 大量 agent、吃 token，不是萬靈丹。「何時值得、怎麼寫不會爆」見下方 7.5 深入。 + +### 📚 必修閱讀 + +1. [**Anthropic — Claude Opus 4.8**](https://www.anthropic.com/news/claude-opus-4-8) — Dynamic Workflows 首次發布的官方說明 +2. **[Stage 7.5 — Dynamic Workflows 深入](07.5-advanced-agentic-concepts.md#-dynamic-workflowsopus-48-當-agent-自己寫出-workflow)** ⭐ — 機制、quality pattern（adversarial verify / loop-until-dry / judge panel）、何時用的完整版 + +> 本節無 examples（概念 + 入口節點）；想動手照 7.5 的 pattern 寫。 + +--- + +## 5.7 — Claude Code Source 解剖（reference harness implementation）⭐ Track B 必看 + +> **本節定位**：本節**不是** harness engineering 的學科級概念教學——學科級的定義 / **8 個核心元件** / prompt→context→harness 三層工程分工 是 **[Stage 7 Harness Engineering](07-multi-agent-production.md#-harness-engineering--production-agent-runtime-的工程設計--本-stage-核心概念)** 在講。**本節是 case study**——拿 Claude Code（一個被廣泛使用的參考實作）的 source code 來解剖、把 Stage 7 列的 8 個元件**中前 6 個 runtime-internal 元件**（Eval / Cost-Latency 兩個是跨層議題、不在 source 主 loop）**在實作裡找到對應位置**。 + +### 學習目標 + +完成本節後你會： +- 看得懂 `claude-agent-sdk-python` source 的 main loop（不是逐行、是抓得到主幹） +- 在 source 裡標出 [Stage 7 列的 8 個 harness 元件](07-multi-agent-production.md#-harness-engineering--production-agent-runtime-的工程設計--本-stage-核心概念)**中**前 6 個 runtime-internal 元件（agent loop / tool registry（agent 可呼叫工具的清單 + 介面定義） / context manager / safety layer / retry / telemetry）各自的 file:line。Stage 7 列的第 7 個 Eval 是外掛、第 8 個 Cost / Latency 是 cross-cutting、不在 source 主 loop 內、不在本練習範圍 +- 講得出 Claude Code 的 agent loop 跟 Stage 3 練習 3 from-scratch ReAct 差在哪——上線部署的 agent 多了哪些東西 + +> **學科級概念在哪**：harness engineering 是什麼 / framework vs harness 差別 / prompt→context→harness 三層工程分工 → 全部見 **[Stage 7 Harness Engineering](07-multi-agent-production.md#-harness-engineering--production-agent-runtime-的工程設計--本-stage-核心概念)**。本節只負責 Claude Code source 的 case study。 + +### 📚 必修閱讀 + +1. [**Anthropic — Building Effective Agents**](https://www.anthropic.com/engineering/building-effective-agents) ⭐ — orchestrator / worker / handoff / reflection 等 pattern 的 canonical reference +2. [**anthropics/claude-agent-sdk-python**](https://github.com/anthropics/claude-agent-sdk-python) — Claude Code 官方 Python SDK 的 source；**重點 file：`src/claude_agent_sdk/_internal/client.py`**（main loop 在這）+ `query.py`（單回合 API） +3. [**ai-boost/awesome-harness-engineering**](https://github.com/ai-boost/awesome-harness-engineering) ⭐（★ 2.8k+） — community curation：harness pattern / eval / memory / observability 整合 +4. [**ZhangHanDong/harness-engineering-from-cc-to-ai-coding**](https://github.com/ZhangHanDong/harness-engineering-from-cc-to-ai-coding) — 中文圈最完整的 Claude Code 內部解讀 + +### 🛠 動手練習 — 解剖 agent loop（閱讀題，非寫 code） + +這節**不是寫 code 練習，是閱讀練習**——production harness 不是抄 200 行範例能學的，是抄完還看不懂為什麼這樣寫，所以本練習要求你開 source、自己 trace。 + +**步驟**： +1. **clone**：`git clone https://github.com/anthropics/claude-agent-sdk-python` +2. **定位 agent loop**：找出 `_internal/client.py` 裡實際發出 LLM call、收 tool_use response、dispatch 給 tool runner 的核心 loop。提示：找 `async def` 跟 `tool_use_id` 關鍵字 +3. **標出前 6 個 runtime-internal harness 元件**在 source 裡的位置（檔名 + 行號）——對應 [Stage 7 列的 8 元件](07-multi-agent-production.md#-harness-engineering--production-agent-runtime-的工程設計--本-stage-核心概念)的前 6 個（第 7 個 Eval 外掛 / 第 8 個 Cost-Latency cross-cutting 不在 source 主 loop）： + - (a) **Agent loop**：實際發出 LLM call + 收 response 的迴圈在哪 + - (b) **Tool registry / dispatch**：LLM 回 tool_use → 怎麼 route 到對應 tool 實作 + - (c) **Context manager**：tool result 怎麼寫回 message history、context window 控制 / auto-compact + - (d) **Safety layer**：tool 執行前有沒有 permission gate / sandboxing + - (e) **Retry / recovery**：tool fail 時怎麼處理（exception vs LLM 自己看 error 反思） + - (f) **Telemetry**：metrics / logging / token counting 接在哪 +4. **寫一段 80-150 字摘要**：「Claude Code 的 agent loop 跟你 Stage 3 練習 3 from-scratch ReAct 差在哪」。重點不是「Claude Code 比較複雜」這種廢話，是**講得出多了哪些東西、為什麼那些是上線部署必須有的** + +**交付物**：一段筆記（寫在自己的 obsidian / notion / `.md` 都行），不必交。但**講不出來你就還沒懂**——這是進 Stage 7 production deploy 之前的必要 mental model。 + +→ **基礎 starter 範本**：本練習**無 examples folder**——是 source-reading exercise，非 code-writing exercise。illustrative，深度教學見上方 📚。 + +### 🎯 精選 Projects + +4 個項目一張表搞定。**挑入口看「適合誰」、想深入點連結看 repo**。 + +| Project | ⭐ | 適合誰 | 為什麼推薦 / 備註 | +|---|---|---|---| +| [anthropics/claude-agent-sdk-python](https://github.com/anthropics/claude-agent-sdk-python) | ⭐⭐⭐⭐⭐ | 所有 Track B 學習者、想搞清楚「Claude Code 內部怎麼跑」 | **canonical Python harness、本節練習就是讀這個 repo**。後面 Stage 7 deploy 也會 import | +| [ZhangHanDong/harness-engineering-from-cc-to-ai-coding](https://github.com/ZhangHanDong/harness-engineering-from-cc-to-ai-coding) | ⭐⭐⭐⭐ | 中文 reader 想看「為什麼 Claude Code 這樣設計」 | 中文圈最完整 CC 內部解讀（harness 概念 → CC 實作 → 跟其他 AI coding tool 對比）。**配合 SDK source 互補看**——一個告訴你「怎麼做」、一個告訴你「為什麼這麼做」 | +| [ai-boost/awesome-harness-engineering](https://github.com/ai-boost/awesome-harness-engineering) | ⭐⭐⭐⭐ | 5.7 讀完想擴大視野 | community curation：30+ harness / eval / memory / observability / MCP project（★ 2.8k+）。**廣度資源庫、非教學**——挑感興趣的 sub-topic 鑽進去 | +| [wshobson/agents](https://github.com/wshobson/agents) | ⭐⭐⭐⭐ | 寫完 5.5 自己的 subagent 後想看實際在用的範本 | 50+ subagent definition 的 ergonomic 設計（description / tool list / system prompt 分層）。**讀 source 比讀文件學得多**。在 5.5 已介紹、本節 cross-ref | + +> 💡 **本節跟 Stage 7 的差別**：本節學「Claude Code 這個 harness 怎麼跑」（具體 reference）；Stage 7 學「production harness 一般要有什麼」（抽象 pattern）。**先具體後抽象**、看完本節再進 Stage 7 會輕鬆很多。 + +--- + +## 5.8 — SDK：把 Claude Code 拆開來自己組 ⭐ Track B 可選、production 才需要 + +> 🎯 **這節是給誰看的**：99% 的人讀完 5.1-5.7 已經夠用，**只在你想做 CLI 做不到的事**才往下走。Stage 5.7 叫你讀 SDK source 是為了理解 harness 內部；這節是為了讓你**會用 SDK** 包成自己的服務。 + +### 1 個比喻把 SDK / CLI / `CLAUDE.md` 分清楚 + +- **CLI**（`claude` / `codex` / 等）= 一台**現成的車子**，點一下就能上路 +- 改 `CLAUDE.md` / `AGENTS.md` / 加 hooks / 寫 skills = **調車子的性能**，讓它開得更順、更貼你工作習慣 —— 一樣是這台車 +- **SDK**（`claude-agent-sdk-python` / `openai-agents-python`）= **把車子從引擎開始重造一台** —— 用 Python / TS 控制 agent loop、tool dispatch、memory 怎麼接 + +**99% 的學習者天花板停在「調車」就夠了。** 只在「調車怎麼調都到不了你要的場景」時，才需要爬到 SDK。 + +### 階梯式三層 —— 你現在在哪？ + +1. **第 1 層 直接用 CLI** —— 90% 的個人 + 團隊使用情境。看 5.1 +2. **第 2 層 CLI + 自訂** —— 寫 `CLAUDE.md`、加 hooks、自己寫 skill、套 plugin。看 5.1-5.4。**多數人停在這層、且夠用** +3. **第 3 層 SDK** —— 把 agent 嵌進你的應用。這節在教 + +### 什麼時候才需要爬到第 3 層 + +具體場景（不抽象）： +- **嵌進你已有的 web app / 後端** —— 使用者不開 terminal，就不能用 CLI +- **cron / scheduler 自動觸發** —— 沒有人類在 session 裡點 enter，CLI 互動模式不適用 +- **公司內部包一層** —— 加 auth、audit log、限額、自訂 prompt template，讓 CLI 的能力以受控方式對外 +- **同時跑多 agent、要 programmatic 控制 hand-off** —— 比 Stage 5.5 的 Task tool 更細的控制權 + +如果你做的不在上面，你大概不需要 SDK。**該回 5.1-5.4。** + +### Hello SDK（4 行 Python） + +```python +from claude_agent_sdk import query + +async for msg in query(prompt="用 git status 看當前狀態"): + print(msg) # 所有 message type 都能 print；要拿 agent 回覆要 filter AssistantMessage +``` + +就這樣 —— 包進 `async def` 就能跑。`query()` 會 yield 多種 message type（`AssistantMessage` / `ResultMessage` / `SystemMessage` 等），上面的 `print(msg)` 全部都能安全印出來；想拿到 agent 真正的回覆要 `isinstance(msg, AssistantMessage)` 再取 `msg.content` —— retry / streaming / prompt caching 等進階用法在 Stage 7 練習 4。 + +### vs CLI / vs 自訂 對照表（看完上面再看這張） + +| | CLI（claude / codex） | CLI + 自訂（改 CLAUDE.md / hooks） | SDK | +|---|---|---|---| +| 嵌進你的 app | ❌ | ❌ | ✅ | +| cron / 排程跑 | ⚠️ 勉強（`-p` flag） | ⚠️ 同左 | ✅ | +| 換語言 / 環境 | 綁 Node / Bash | 同左 | Python / TS 隨你 | +| programmatic 控制 | ❌ | ❌ | ✅ | +| 客製 system prompt | 受限 | 受限 | 完全自由 | +| 學習成本 | 1 天 | 1-2 週 | 1 個月+ | +| 適合誰 | 個人日常用 | 個人 / 小團隊長期用 | 包成產品 / 服務 | + +### 兩個主要 SDK + +| | [claude-agent-sdk-python](https://github.com/anthropics/claude-agent-sdk-python) | [openai-agents-python](https://github.com/openai/openai-agents-python) | +|---|---|---| +| 出品 | Anthropic 官方 | OpenAI 官方 | +| 模型 | Claude（Opus / Sonnet / Haiku） | OpenAI 系列 + 其他 | +| 強項 | 跟 Claude Code 一致的 tool / skill / hook 抽象 | handoff / agents-as-tools 模式、2026-04 內建 sandbox | +| 適合 | 已在用 Claude Code 想嵌服務的人 | 已 commit OpenAI 生態的人 | + +兩個都 MIT 授權、API 設計乾淨，**重點是你的下游選哪家模型**。 + +### 接下來 + +- **看程式碼**：回 5.7，讀 `claude-agent-sdk-python` 的 `_internal/client.py` —— 你現在會用 SDK 了，讀那邊的 main loop 會看懂更多 +- **動手練 SDK 進階**：Stage 7 練習 4（streaming + prompt caching）；Stage 7 練習 5（FastAPI + Docker production deploy） +- **如果你發現你其實不需要 SDK**：那很好 —— 回 5.1-5.4，把 CLI + 自訂這層用透，通常已經比寫 SDK 划算 + +> 💡 **本節跟 Stage 7 的差別**：本節學「SDK 是什麼、什麼時候用」（定位 + 入門）；Stage 7 學「用 SDK 寫一個可上線部署的 agent 服務」（streaming / caching / deploy）。 + +--- + +## ✅ 進入 Stage 6 前的自我檢查 + +你能不能： + +- [ ] 安裝 Claude Code 並使用 5 個不同的 slash command +- [ ] 在同一個 Claude session 裡接 2 個 MCP server +- [ ] 用 Python 寫自己的 MCP server，提供 1 個能用的 tool +- [ ] 寫一份能在特定觸發詞自動載入的 `SKILL.md` +- [ ] 把 skill 打包成 plugin，再用 `marketplace.json` 發佈 +- [ ] **寫過 `.claude/agents/` 自訂 subagent 並從 Task tool invoke 過** +- [ ] **讀過 `claude-agent-sdk-python` 的 main loop、能在 source 裡標出 [Stage 7 列的 8 個 harness 元件](07-multi-agent-production.md#-harness-engineering--production-agent-runtime-的工程設計--本-stage-核心概念) 的前 6 個 runtime-internal 元件**位置（5.7 練習） +- [ ] 從角色分工說出 MCP / Skills / Plugins / Subagents / SDK 各自的位置 + +如果都可以 → 前往 [Stage 6 — Memory & RAG](06-memory-rag.md)。 + +> 💡 **Stage 5 是兩 track 第一個 hub**——Track A 跟 Track B 都會用到。第二個 hub 是 [**Stage 8 — Agent Interfaces**](08-agent-interfaces.md)（Computer Use / Browser Use / Sandbox），可以走完主幹後再進、或對 Computer Use / Browser MCP 有興趣可以提前 preview。 + +## 💡 Bonus：完成這個階段之後 + +- 對 [`anthropics/claude-cookbooks`](https://github.com/anthropics/claude-cookbooks) 發一個 PR（小修正、文件更新） +- 把自己的 plugin 投稿到社群 marketplace +- 寫一篇文章，比較自己的 hello-MCP server 跟官方 `modelcontextprotocol/servers` 收的某一個 diff --git a/stages/05-claude-code-ecosystem.zh-Hans.md b/stages/05-claude-code-ecosystem.zh-Hans.md new file mode 100644 index 0000000..234826b --- /dev/null +++ b/stages/05-claude-code-ecosystem.zh-Hans.md @@ -0,0 +1,960 @@ +# Stage 5 — Claude Code 生态系（Claude Code Ecosystem）⭐⭐ + +> [繁體中文](./05-claude-code-ecosystem.md) | **简体中文** | [English](./05-claude-code-ecosystem.en.md) + +⏱ **时间估算**：3-4 周（约 15-25 小时） + +> 🚪 **进入条件**（共用 hub、依 track 不同）：**Track A（CLI Power User）** 从 A1-A2 过来、会用 Python + 跑过基本 CLI 即可、从 5.1/5.2 起步；**Track B（Agent Builder）** 建议先完成 [Stage 3](03-tool-use-and-hello-agent.zh-Hans.md)（tool use）+ [Stage 4](04-agent-frameworks.zh-Hans.md)（agent frameworks）再进、把整个 stage 当“Claude Code 内部怎么运作”深读。不确定走哪条 → 看下面 📌 的两轨说明。 + +> 💡 整个 stage 围绕 4 个关键词（**MCP / Skills / Plugins / Marketplace**）展开 → 不熟先翻 [`resources/glossary.zh-Hans.md` 5](../resources/glossary.zh-Hans.md#5-claude-code-生态)。 + +**👥 共用 hub**：本 stage 是 Track A（CLI Power User）+ Track B（Agent Builder）两条路径的共用中心。Stage 5 跟 [Stage 8 — Agent Interfaces](08-agent-interfaces.zh-Hans.md) 是 curriculum 的两个 hub。 + +> 📌 **这个 stage 两条轨都用**： +> - **Track A（CLI Power User）**：A2 用 [5.1（Claude Code 基础）](#51--claude-code-基础)；A3 用 [5.2（MCP）](#52--mcpmodel-context-protocol-基础) + 选择性用到 [5.3（Skills）](#53--skillsclaude-code-的行为层-claude-code-生态最关键的一层) 跟 [5.4（Plugins）](#54--plugins-与-marketplaces)（A3 的 动手练习 CLI-12 会教把 CLAUDE.md 跟 commands 打包成 plugin）。读的角度是“**怎么用 Claude Code 把工作做好**”。 +> - **Track B（Agent Builder）**：把整个 stage 当“**Claude Code 内部怎么运作**”的深度学习，从 5.1 完整走到 5.4。 + +> 🗺️ **Claude Code 属于哪种 agent 型态**？→ [`resources/agent-paradigms.zh-Hans.md`](../resources/agent-paradigms.zh-Hans.md) Type 1（IDE-coupled）+ Type 2（Terminal pair-programmer）；想看完整 5 种 paradigm 对照也从这份开始。 + +> 🧭 **Claude Code 只是 agent 的其中一种形态**（往下读前先有个全局感）：Claude Code 是给**工程师**的**终端** agent——活在命令行、处理代码。Anthropic 另外还有 **Claude Cowork**：给**非工程师**（研究、分析、运营）的**桌面 app**——给它一个目标、它会跨你的文件跟应用程序把事情做完、交回成品。OpenAI 这两种形态也都有。Stage 5 接下来专讲 Claude Code，这张表只是让你知道它在整张地图的位置。 + +| 形态 | 它帮你做什么 | Anthropic | OpenAI | +|---|---|---|---| +| **终端 · 给工程师** | 读 / 改 / 跑你的代码 | Claude Code | Codex CLI | +| **App · 给所有人** | 跨你的文件、应用程序、网页把一件事做完 | Claude Cowork | ChatGPT agent | + +> ⚠️ **想用本地 LLM？这个 stage 不是那条路线。** Claude Code 需要 Anthropic API / OAuth，不能直接改接 Ollama 或本地 endpoint。离线、隐私数据或不想用 API 额度时，请看 [`resources/cookbook.zh-Hans.md` Recipe 6](../resources/cookbook.zh-Hans.md#6-本地-llm--cli-agent-快速-walkthrough)，用 OpenCode / goose / Aider / Hermes 这类支持 BYO LLM 的 CLI agent。 + +> 📋 **本章组成**：7 个子章（5.1 基础 / 5.2 MCP / 5.3 Skills / 5.4 Plugins / 5.5 Subagents / 5.6 Dynamic Workflows / 5.7 Claude Code Source 解剖），每个子章都有“学习目标 → 必修阅读 → 动手练习 → 精选 Projects” → 章末 自我检查。**注意**：Harness Engineering（白话：model 外面的"运行外壳"——怎么给它工具、记忆、权限，怎么跑每一轮）的**学科级概念**会在 [Stage 7](07-multi-agent-production.zh-Hans.md) 系统整理；本章 5.7 把 Claude Code 当作案例，观察一个成熟 agent 工具如何处理工具、记忆、配置、权限与执行流程。 +> 🔑 **关键名词**：见 [`resources/glossary.zh-Hans.md` 5](../resources/glossary.zh-Hans.md#5-claude-code-生态)。 + +## Stack 一览 + +由上往下，每一层都建立在底下那一层上： + +![Claude Code Ecosystem Stack](../resources/diagrams/stage5-stack.zh-Hans.png) + +每一层各自加上一种能力： +- **API + SDK**：用程序访问 LLM +- **Tool Use**：让 LLM 调用你定义的 function +- **MCP**：标准化协议，让任何 LLM host 都能使用任何 tool server +- **Skills**：Claude Code 的行为包，可以封装 MCP tool +- **Plugins**：把 Skills、hooks、commands、MCP 设置打包成一个单位发布 + +这个阶段有 4 个子章节，**请按顺序做**——每一节都建立在前一节之上。 + +``` +5.1 Claude Code 基础 3-5 天 （安装、slash commands、CLAUDE.md） +5.2 MCP — 协议层 5-7 天 （写你的第一个 MCP server） +5.3 Skills — 行为层 5-7 天 （写你的第一个 SKILL.md） +5.4 Plugins 与 Marketplaces 5-7 天 （打包并发布） +``` + +跑完这个阶段，你会能扩充 Claude Code、写自己的 MCP server、发布一个 plugin marketplace。 + +--- + +## 🗺️ 7-Layer Architecture Map（先看这张图、再读 5.1-5.7） + +> 📋 **这节是什么**：把 Claude Code 的 7 个 primitive（MCP / Skills / Plugins / Subagents / Hooks / Slash commands / CLI）对应到 **7 个架构层 + 3 个工程学 discipline**——进 5.1-5.7 之前先看一次，知道接下来在学什么层；学完回头看，就是 synthesis。**分层是教学选择，不是 absolute 真理**。 + +![Claude Code 7-Layer Architecture Map](../resources/diagrams/claude-architecture-map.zh-Hans.png) + +> 📊 **上图**：Claude Code 7 个架构层 + 3 个工程学 discipline 整合视图。 + +### 每层 1 句白话 + Claude 的版本 + +| Layer | 是什么 | Claude 的版本 | 谁管 | 学在 | +|---|---|---|---|---| +| **L7 Interface** | 用户和 agent 交谈入口 | claude-code CLI / Desktop | Harness Engineering | [Stage 5.1](#51--claude-code-基础) | +| **L6 Workflow** | 固定可复用流程模板 | **Skills**（SKILL.md）+ Slash commands + **Plugins**（打包 Skills / hooks / commands、属 packaging）| Prompt Engineering | [Stage 5.3](#53--skillsclaude-code-的行为层-claude-code-生态最关键的一层) / [5.4](#54--plugins-与-marketplaces) | +| **L5 Coordination** | 多 agent 分工合作 | **Subagents** + Agent team + Background | Harness Engineering | [Stage 5.5](#55--subagentsclaude-code-原生-multi-agent-机制-2025-新功能) | +| **L4 Memory / Context** | 跨对话 / 跨 session 记事情 | History / `/compact` / Memory hooks | Context Engineering | [Stage 6](06-memory-rag.zh-Hans.md) | +| **L3 Control Plane**（"守门员"层） | tool 执行前 / 后拦截 / validation / 阻挡 | **Hooks**（PreToolUse / PostToolUse 等）| Harness Engineering | [Stage 5.1 hooks 段](#51--claude-code-基础) | +| **L2 Tool Use** | LLM 调用外部 function 的 protocol | Anthropic Tool Use（`input_schema`）| Tool design | [Stage 3](03-tool-use-and-hello-agent.zh-Hans.md) | +| **L2.5 Tool Provider** | 把外部 API 包成 tool 给 Layer 2 用 | **MCP servers**（Notion / Gmail / Slack）| Context Engineering + Tool | [Stage 5.2](#52--mcpmodel-context-protocol-基础) | +| **L1 Foundation** | LLM 本体（system prompt 直接送达这一层）| Anthropic API | Prompt Engineering | [Stage 1](01-llm-basics.zh-Hans.md) + [Stage 2](02-prompt-engineering.zh-Hans.md) | + +### 3 工程学 Discipline overlay（核心 insight） + +Prompt / Context / Harness 是**不同层的 discipline**——学会其中一个，不会自动会另一个： + +| Discipline | 负责哪些 layer | 1 句话 | 学在 | +|---|---|---|---| +| **Prompt Engineering** | L1 + L6 | "送进 LLM 的字符串怎么设计" | [Stage 2](02-prompt-engineering.zh-Hans.md) | +| **Context Engineering** | L4 + L2.5 | "context window 装什么信息" | [Stage 6](06-memory-rag.zh-Hans.md) | +| **Harness Engineering** | L3 + L5 + L7 | "LLM 外面的'运行外壳'——给它工具、记忆、控制流程的那层设置" | [Stage 7 §Harness Engineering](07-multi-agent-production.zh-Hans.md#-harness-engineering--production-agent-runtime-的工程设计--本-stage-核心概念) | + +> 💡 **MCP 的特殊位置**：MCP 严格说是 **Context Engineering**（feed context source）+ **Tool design**（协议规范）跨层东西，不纯归任一 discipline——所以图里用 Layer 2.5 标明。 + +### 跨 CLI vendor mini-comparison（2026-05 snapshot） + +只有 Claude Code 有**完整 7-layer stack**；其他 CLI 大多停在 single-agent + 简化版： + +| 层 | Claude Code | OpenAI Codex | Gemini CLI | +|---|---|---|---| +| L5 Coordination（multi-agent）| ✅ Subagents | ❌ single-agent | ❌ | +| L3 Control Plane（Hooks）| ✅ Hooks | ❌ | ❌ | +| L2.5 Tool Provider（MCP）| ✅ | ✅（已支持 MCP）| ✅（需手动装 MCP server）| +| L6 Workflow（Skills）| ✅ SKILL.md | AGENTS.md（context only）| GEMINI.md（context only）| + +→ 细看 [`resources/cli-agents-guide.zh-Hans.md`](../resources/cli-agents-guide.zh-Hans.md) + +--- + +## 5.1 — Claude Code 基础 + +### Claude Code 是什么（先定位） + +**Claude Code = 跑在你 terminal 里的 Claude agent**——有完整 file system / shell / git / 子进程 access、可以**自主完成多步骤工作**（读文件 → 改文件 → 跑 test → commit → 发 PR）。 + +跟其他 Claude 界面差别： + +| 界面 | 跑在哪 | 能做什么 | 用法 | +|---|---|---|---| +| **claude.ai**（web） | 浏览器 | 纯对话 + 上传文件、无 file system 操作 | 偶尔聊一下、ask 一个问题 | +| **Claude API**（programmatic） | 你的 server / script | LLM call、自己包 agent loop | 写 production system | +| **Claude Agent SDK** | 你的 Python / TS 环境 | 完整 agent runtime + tool use + 多 session | 写 production agent system | +| **Claude Code**（**本节**） | 你的 terminal | **完整 OS-level agent**（file / shell / git / subprocess）+ skill / plugin / subagent 生态 | **日常工作主力工具** | + +进 5.2-5.7 之前你会在这节学到 **4 个 Claude Code 核心结构**：CLAUDE.md（记忆层）/ slash commands（控制层）/ `~/.claude/` 目录（设置层）/ settings.json（行为层）。 + +### 学习目标 + +完成本节后你会： +- 讲得出 Claude Code 跟 claude.ai / API / SDK 各自的角色（**“为什么用 CLI 不用 web”**） +- 安装 Claude Code、配置认证、跑第一个有 file access 的 session +- 用 8-10 个常用 slash command 控制 Claude Code 行为 +- 写一份项目级 `CLAUDE.md` 设置 baseline 行为 +- 认得 `~/.claude/` 目录结构（skills / agents / plugins / settings.json 各放哪） + +### 必修阅读 +1. [**Anthropic — Claude Code Quickstart**](https://docs.claude.com/en/docs/claude-code/quickstart) — 官方安装指南 +2. [**Anthropic — CLAUDE.md best practices**](https://docs.claude.com/en/docs/claude-code/memory) — 怎么写项目 memory +3. [**Anthropic — Slash Commands**](https://docs.claude.com/en/docs/claude-code/slash-commands) — 官方完整 slash command 列表 +4. [**Anthropic — Settings**](https://docs.claude.com/en/docs/claude-code/settings) — `settings.json` 完整 schema + env var +5. [**KimYx0207/Claude-Code-x-OpenClaw-Guide-Zh**](https://github.com/KimYx0207/Claude-Code-x-OpenClaw-Guide-Zh) — 简中入门指南 + +> 🛠️ **要写好 CLAUDE.md？** 先看 [Stage 7.5 核心 Harness Engineering 原则（多 source 整理）](07.5-advanced-agentic-concepts.zh-Hans.md#-跨概念-harness-engineering-原则多-source-整理) 建概念、再用下面 2 个 prompt 动手。 + +### 📋 CLAUDE.md 设计 prompts（依 5 原则） + +写 / 改 CLAUDE.md 时直接复制贴上： + +#### Prompt 1 — Audit 你现有的 CLAUDE.md + +``` +我有一个 CLAUDE.md（在 [贴路径]），请依下面 5 个 harness engineering 原则 audit： + +1. Legibility — 用 markdown header 分区吗？conventions 写具体（"2-space indent"）还是模糊（"format properly"）？ +2. Progressive Disclosure — < 200 行吗？有用 `@-import` 或 `.claude/rules/.md` 拆分吗？ +3. System of Record — CLAUDE.md 是否当 entry map、实际内容指向 `docs/` + `.coord/`？还是把所有规则塞同一档？ +4. Taste Invariants — 规则可验证吗（"run `make lint` before commit"）？还是「follow best practices」这种空话？ +5. Transparency — 有要求 agent show planning step 吗？还是预期它默默做完？ + +每条给 PASS / FAIL / PARTIAL + 原因 + 改进建议。总分 X/5、最该先修哪条。 +``` + +#### Prompt 2 — 生成新的 CLAUDE.md（依 5 原则） + +``` +我要为一个 [描述项目，例如：Python data analysis monorepo / 学术论文 repo / Next.js app] 写 CLAUDE.md。请依下面 5 个 harness engineering 原则生成： + +- **< 200 行** +- 当 **entry map**，把实际 conventions 用 `@-import` 引外部 docs 或 `.claude/rules/.md` +- 每条规则**可验证**（不要「follow best practices」这种空话） +- 加 **1-2 个 transparency rule**（例如「edit > 50 lines 前先 show plan」） +- 标明哪些内容该放 CLAUDE.md、哪些该分到 `.claude/rules/.md` + +输出： +1. 完整 CLAUDE.md 内容 +2. 建议的 `.claude/rules/` 目录切法（topic 列表） +3. 1 个示范 `.claude/rules/.md`（任选一个 topic） +``` + +→ **建议流程**：写 CLAUDE.md 前用 Prompt 2 生成、写完用 Prompt 1 audit。 + +### 常用 slash commands（10 个必学） + +| Command | 用途 | 何时用 | +|---|---|---| +| `/help` | 列出所有可用 command | 不知道有什么指令时 | +| `/clear` | 清空对话历史（保留 system context） | session 太长、想重启逻辑 | +| `/compact` | 自动摘要对话、释放 context window | context 接近用满 | +| `/plan` | 进入 plan mode（read-only、先规划才动手） | 大改动前先让 Claude 列计划 | +| `/model` | 切换 model（Sonnet / Haiku / Opus）| 改成更便宜 model 省 token | +| `/agents` | 列 / 管理 subagent（5.5）| 看哪些 subagent 可用、debug | +| `/plugin install @` | 安装 plugin（5.4）| 加新功能 | +| `/permissions` | 看 / 改当前 session 权限 | 太多 permission prompt 想精简 | +| `/resume` | 恢复前次 session | 接续昨天工作 | +| `/bg` | 把当前 session 背景化（移到 agent view）| 想同时跑多任务、见 5.5 | + +完整列表见上方 [Slash Commands 官方文件](https://docs.claude.com/en/docs/claude-code/slash-commands)。 + +### `~/.claude/` 目录结构（先有 mental map） + +``` +~/.claude/ ← 全局 user-level +├── settings.json ← 全局行为（env / hooks / permissions / model 预设） +├── settings.local.json ← 机器特定（不入 git） +├── CLAUDE.md ← 全局 baseline（每个 session 都加载） +├── skills//SKILL.md ← user-level skills（5.3） +├── agents/.md ← user-level subagents（5.5） +├── plugins/ ← 已安装的 plugin（5.4） +├── hooks/ ← user-level hook scripts +└── jobs// ← background sessions 状态（5.5 background agent） + +/.claude/ ← project-level（随 repo） +├── settings.local.json ← project 行为（含 permissions） +├── skills//SKILL.md ← project-level skills（优先级高于 user-level） +├── agents/.md ← project-level subagents +├── commands/.md ← project-level slash command +└── hooks/ ← project-level hook + +/CLAUDE.md ← project baseline（每个 session 都加载） +``` + +**优先顺序**（冲突时谁赢）：project > user > built-in default。 + +### 动手练习 +- **练习 1：第一个 session** — 安装、认证、`cd` 到 repo、跑 `claude` → 问“summarize this codebase”→ 观察怎么读文件 +- **练习 2：CLAUDE.md** — 写 repo 根目录 CLAUDE.md（role / context / 不能做的事 / 怎么做事 / 常用指令），对照“没 CLAUDE.md”跟“有 CLAUDE.md”的行为差别 +- **练习 3：5 个 slash commands** — 在一个 session 内依序用 `/help` `/plan` `/compact` `/model` `/agents`，观察各自做什么 +- **练习 4：目录探索** — `ls ~/.claude/` + `cat ~/.claude/settings.json`、看自己 user-level 设置长什么样 + +### 精选 Projects + +| Project | ⭐ | 适合谁 | 为什么推荐 / 备注 | +|---|---|---|---| +| [anthropics/claude-code](https://github.com/anthropics/claude-code) ⭐ 官方 | ⭐⭐⭐⭐⭐ | 追踪新版本 / 看 release notes / 回报 bug | Claude Code 官方 repo、issues + releases + inline 范例 | +| [Anthropic — Claude Code 官方文档](https://docs.claude.com/en/docs/claude-code/overview) | ⭐⭐⭐⭐⭐ | 任何 reference 查询 | **真正的 canonical reference**——上面 5 条必修阅读都从这里来 | +| [hesreallyhim/awesome-claude-code](https://github.com/hesreallyhim/awesome-claude-code) | ⭐⭐⭐⭐ | 想看社区有什么（slash commands / skills / hooks 范例）| 较广泛的资源清单（目前正在重整）| +| [KimYx0207/Claude-Code-x-OpenClaw-Guide-Zh](https://github.com/KimYx0207/Claude-Code-x-OpenClaw-Guide-Zh) | ⭐⭐⭐⭐ | 中文读者要逐步教学 | 简中入门导读 | + +### Hooks（L3 控制层）⭐ 把规则写成程序、自动拦截 + +MCP / Skills 是“给 agent 更多能力”；**Hooks 则是反过来：在 agent 的生命周期事件上挂你自己的 script，做检查、拦截、注入**。这是 Claude Code 的控制层（架构图的 L3）。 + +**怎么运作**：在 `settings.json` 的 `hooks` 里设置“某事件发生时跑哪个 command”。常用事件（2026 已扩到 ~28 个，先记核心几个）： + +| 事件 | 触发时机 | 典型用途 | +|---|---|---| +| `PreToolUse` | 工具调用前 | 挡危险指令、权限 gate | +| `PostToolUse` | 工具调用后 | 自动 format / lint / 跑测试 | +| `UserPromptSubmit` | 你发送 prompt 时 | 注入 context、挡掉某些输入 | +| `Stop` / `SubagentStop` | （子）agent 想停时 | 强制它继续、或做收尾检查 | +| `SessionStart` / `SessionEnd` | session 开始 / 结束 | 加载状态、写 log | +| `PreCompact` | context 压缩前 | 保护重要内容 | + +**关键语义**：hook **返回 exit code 2 = 阻挡**：Claude 会把 stderr 当错误信息读回去（例如 `PreToolUse` 返回 2 就挡下那个工具调用、`UserPromptSubmit` 返回 2 就挡下 prompt）。这就是“用程序强制规则”的机制。 + +> ⚠️ **安全**：hook 是在你机器上跑的 shell command，别乱装别人的 hook，也别在 hook 里跑未经检查的输入。 +> +> 完整事件清单 + JSON 进阶用法见官方文档：[Claude Code Hooks](https://code.claude.com/docs/en/hooks)。 + +--- + +## 5.2 — MCP（Model Context Protocol）⭐ 基础 + +### MCP 是什么（先定位） + +**MCP = “**让 LLM 用任何外部工具 / 数据**”的开放协议**。在 MCP 之前每个 LLM 厂商都得自己定义 tool 规格、每个工具供应商都得为每个 LLM 写一份接法。MCP 把这层**标准化**——写一次 MCP server、Claude / Codex / Cursor / 任何支持 MCP 的 host 都能用。 + +**MCP 三个抽象**： + +| 抽象 | 是什么 | 范例 | +|---|---|---| +| **Tools** | LLM可以调用的 function | `read_file(path)` / `query_db(sql)` / `send_slack(channel, msg)` | +| **Resources** | LLM可以读的数据源 | `file:///path/file.md` / `postgres://db/users` | +| **Prompts** | server 预定义的 prompt 样板 | 一份“review code”的 prompt template | + +**多数 MCP server 主要用 Tools 抽象**——Resources 跟 Prompts 用得少。 + +**MCP vs Tool Use vs Skill vs Plugin**： + +- **Tool Use**（Stage 3）：你 in-process 写的 function 给 LLM 调用 +- **MCP**（**本节**）：把 tool 标准化成 server / client 协议、跨 host / 跨 LLM 可用 +- **Skill**（5.3）：行为层 — 教 Claude“**遇到 X 用哪个 MCP tool**” +- **Plugin**（5.4）：把 MCP + Skill + 其他打包散布 + +→ **核心区分**：MCP 是“**能力**”（让 LLM 能做什么）、Skill 是“**行为**”（什么时候用什么能力）。 + +### 学习目标 +- 解释 MCP 的三个抽象（Tools、Resources、Prompts） +- 把现成的 MCP server 接上 Claude Desktop 或 Claude Code +- 用 Python 写一个最小的 MCP server，提供 1-2 个 tool +- 区分 MCP server vs Tool Use vs Skills vs Plugins + +### 必修阅读 +1. [**Anthropic — Introducing MCP**](https://www.anthropic.com/news/model-context-protocol) — 最初发表，概念总览 +2. [**MCP Specification**](https://modelcontextprotocol.io/specification) — 实际的协议规格 +3. [**Complete Guide to MCP in 2026**](https://dev.to/x4nent/complete-guide-to-mcp-model-context-protocol-in-2026-architecture-implementation-and-4a11) — 实践导读 +4. [**Microsoft — MCP for Beginners**](https://github.com/microsoft/mcp-for-beginners) — 官方循序渐进 MCP 学习课纲（概念、安装、动手 lab；免费、GitHub 上）。★ 16k+ + +### 动手练习 +- **练习：MCP client** — 安装 `modelcontextprotocol/servers/filesystem`，从 Claude Desktop 连上去。看着 Claude 读你的文件。 +- **练习：MCP server** — 写一个 Python MCP server，提供一个 tool（例如“换算温度”）。从 Claude Code 连过去。**step-by-step 怎么做** → [`resources/cookbook.zh-Hans.md` 2](../resources/cookbook.zh-Hans.md#2-写你的第一个-mcp-server) +- **练习：MCP in production** — 在同一个 Claude session 里同时连 2-3 个 MCP server，看它们互相搭配。 + +### 精选 Projects（spec / SDK / 范本参考） + +> 💡 **找日常工具的 MCP（Notion / Obsidian / Excel / Postgres / Playwright / Figma 等）？** +> 看 [`resources/mcp-skills-catalog.zh-Hans.md`](../resources/mcp-skills-catalog.zh-Hans.md)——按 16 个分类整理 65+ 个常用 MCP server / Skill，每个都附 stars / license / 适合谁。下表保留的是“**写自己 MCP server 时的 reference**”性质的官方 server / SDK。 + +| Project | ⭐ | 适合谁 | 为什么推荐 / 备注 | +|---|---|---|---| +| [modelcontextprotocol/servers](https://github.com/modelcontextprotocol/servers) ⭐ 官方 | ⭐⭐⭐⭐⭐ | 练习 1 接 server、之后当参考 | 20+ 官方 MCP server（filesystem / git / github / sqlite / time / fetch / memory / sequential-thinking），★ 85k+、MIT、TS+Python。**读 `everything` 跟 `filesystem` source 理解协议运作**。安装：`npx -y @modelcontextprotocol/server-filesystem /path` 或 `pip install mcp-server-fetch` | +| [modelcontextprotocol/python-sdk](https://github.com/modelcontextprotocol/python-sdk) | ⭐⭐⭐⭐⭐ | 练习 2 写自己 MCP server | 官方 Python SDK、`pip install mcp` 即装、MIT。跟着官方 quickstart 跑 | +| [modelcontextprotocol/typescript-sdk](https://github.com/modelcontextprotocol/typescript-sdk) | ⭐⭐⭐⭐ | 喜欢 TS 的人 | Python SDK 的 TypeScript 版、MIT | +| [wong2/awesome-mcp-servers](https://github.com/wong2/awesome-mcp-servers) ⭐ 目录 | ⭐⭐⭐⭐⭐ | 自己写前先找有没有现成的 | 150+ 社区 MCP server 目录，按 search / code / cloud / communication / finance 分类。投稿走 mcpservers.org | +| [punkpeye/awesome-mcp-servers](https://github.com/punkpeye/awesome-mcp-servers) | ⭐⭐⭐⭐ | 跟 wong2 交叉比对 | 另一份 MCP server 目录、组织方式不同、通常更新更实时 | +| [github/github-mcp-server](https://github.com/github/github-mcp-server) | ⭐⭐⭐⭐ | 想看实际上线的 MCP server source | GitHub 官方维护、真正 production 在跑的范例 | +| [21st-dev/magic-mcp](https://github.com/21st-dev/magic-mcp) | ⭐⭐⭐ | 做完练习 2 找灵感 | 会生成 UI 组件的非平凡 MCP server、★ 5.3k+、NOASSERTION。**看 MCP 不只能做数据抓取** | +| [yamadashy/repomix](https://github.com/yamadashy/repomix) | ⭐⭐⭐⭐⭐ | 喂整个 codebase 给 LLM | ★ 26k+、MIT。把 repo 打包成单个 AI-friendly 文件，带 MCP server mode + tree-sitter 压缩（约 70% token 节省）+ secretlint 过滤敏感信息。**Claude Code / Codex 的 daily-driver 工具。** | + +> 🔭 **MCP 在 2026：从“知道是什么”到“会用生态”**：(1) **官方 Registry**（registry.modelcontextprotocol.io）——发现 / 发布 MCP server 的中央目录；(2) **FastMCP**（[jlowin/fastmcp](https://github.com/jlowin/fastmcp)、★25k）——用 `@mcp.tool` 几行写出 server，比 low-level SDK 省事；(3) ⚠️ **MCP 安全**：tool 返回的内容是**不可信输入**（tool poisoning、confused-deputy），别把没检查过的第三方 server 接上有权限的 agent。 + +--- + +## 5.3 — Skills（Claude Code 的行为层）⭐ Claude Code 生态最关键的一层 + +### Skill 是什么（先定位） + +Skill = **一个 markdown 文件**（`.claude/skills//SKILL.md`），告诉 Claude“**遇到某情境 → 走某流程**”。Claude 每次 inference 前扫所有可用 skill 的 `description` frontmatter、看是否匹配当前情境、**匹配就把 SKILL.md 自动加载到 context**。 + +> 🛠️ **要写好 SKILL.md？** 两条路： +> - **路 A：用 Anthropic 官方 `skill-creator` skill 自动产生**（5.3.x 之后安装段落会教），它会自动产 frontmatter + 子目录结构、是 Anthropic 维护的 canonical 工具。 +> - **路 B：用下面 SKILL.md 设计 prompts 自己写**——先看 [Stage 7.5 核心 Harness Engineering 原则](07.5-advanced-agentic-concepts.zh-Hans.md#-跨概念-harness-engineering-原则多-source-整理) 建概念、再用 prompt 动手。 +> +> 两条不冲突：`skill-creator` 给结构、5 原则 prompt 给内容质量检查。 + +### 📋 SKILL.md 设计 prompts（含 `skill-creator` 替代） + +写 / 改 SKILL.md 时直接复制贴上： + +#### Prompt 1 — Audit 你现有的 SKILL.md + +``` +我有一个 SKILL.md（在 [贴路径]），请依下面 5 个 harness engineering 原则做 audit。每条给「PASS / FAIL / PARTIAL」+ 1 行原因 + 1 行改进建议： + +1. Legibility — description 写清楚「何时触发」吗？tool param 命名一致吗？ +2. Progressive Disclosure — SKILL.md < 200 行吗？细节是否放 `references/` 而不是塞主档？ +3. System of Record — `references/` 是 single source、主档不重复吗？ +4. Taste Invariants — success criteria 是否写死可验证、不是「尽量好」这种主观词？ +5. Throughput / Merge — 有附 acceptance check（lint / test / preset YAML）吗？ + +最后给：总分 X/5、最该先修哪一条、为什么。 +``` + +#### Prompt 2 — 生成新的 SKILL.md（依 5 原则） + +``` +我要写一个 skill 处理 [描述任务，例如：把 PDF 转成 markdown / 跑学术论文 banned-word audit]。请依下面 5 个 harness engineering 原则生成 SKILL.md： + +- **description** 写清楚「何时触发」（让 Claude 能 match 对情境） +- **主档 < 200 行**，所有 examples / edge cases / detailed rules 放 `references/.md` +- 列出建议的 `references/` 结构（1-3 个 topic 档案） +- 加一个 **success criteria 表**（可验证、不主观） +- 加一段 **acceptance check**：要跑哪些 lint / unit test / preset YAML + +输出： +1. 完整 SKILL.md 内容 +2. references/ 目录结构建议 +3. 用哪个 acceptance gate preset 验证它（如 multi-locale-mirror-sync / catalog-entry-add 之一） +``` + +→ **建议流程**：先 `/skill skill-creator` 拿干净骨架 → 用 Prompt 2 填内容 → 写完用 Prompt 1 audit。 + +**核心 mental model**：你发现自己“**每次都要打同样的 prompt 教 Claude 怎么做某件事**”→ 把它写成 skill、下次就不用了。Claude Code 生态里 **skill 是 power user 跟普通用户的分水岭**——熟练 skill 写作的人能把 1 个小时的工作压到 5 分钟。 + +### Skill vs CLAUDE.md vs MCP vs Plugin vs Subagent — 一张表分清楚 + +各层常被搞混。**一行对照**： + +| 组件 | 是什么 | 何时用 | 触发方式 | 范例 | +|---|---|---|---|---| +| **CLAUDE.md**（5.1） | repo / project 的 baseline 行为 | repo-wide convention（“用 type hint”、“commit 消息规范”）| **每个 session 都加载**、不分情境 | 你 repo 根目录的 CLAUDE.md | +| **MCP server**（5.2） | 提供 tool / data 的 protocol server | 想让 Claude 能访问**外部资源**（API / DB / 文件系统） | server 启动后、任何时候都能调用 | `github` MCP / `postgres` MCP | +| **Skill**（**本节**） | **特定情境的行为包** | 想设置“**遇到 X 情境 → 走 Y 流程**” | **description 匹配自动加载** | `skill-vetter`（装 skill 前检查）/ `pdf`（处理 PDF） | +| **Plugin**（5.4） | 把 skills + commands + MCP + hooks 打包散布 | 想 share / install **一整套** 设置 | `/plugin install @` | `engineering` bundle / `finance` bundle | +| **Subagent**（5.5） | 独立 context 的 sub-Claude session | 想 delegate **大 context 任务**、结果回主 session | description 匹配自动 delegate | code-reviewer subagent / 研究员 subagent | + +**怎么选**： + +- 一句话设置 → 写进 `CLAUDE.md` +- 多步骤流程、某情境才用 → 写 **Skill**（本节主题） +- 需要访问外部资源（API / DB） → 写 **MCP server** +- Skill 跑起来太大、会吃光主 session window → 改成 **Subagent** +- Skill / command / MCP / hook 想打包送人 → 包成 **Plugin** + +→ **核心区分**：MCP 是“**能力**”、Skill 是“**行为**”、Plugin 是“**散布**”、Subagent 是“**独立 worker**”。 + +### 学习目标 +- `SKILL.md` 的结构（YAML frontmatter + 本文） +- skill 何时会自动加载（description 比对） +- 怎么写一份能解决你日常工作的 SKILL.md +- `references/`、`scripts/`、`evals/` 子目录的用途 + +### 必修阅读 +1. [**Anthropic — Claude Skills 文档**](https://docs.claude.com/en/docs/claude-code/skills) +2. **几份范例 SKILL.md**——从 `anthropics/claude-code` 或社区 marketplace 拿 +3. [**Hello-Agents — Extra08 如何写出好的 Skill**](https://github.com/datawhalechina/hello-agents/blob/main/Extra-Chapter/Extra08-如何写出好的Skill.md) — 中文最完整的 Skill 最佳实践 +4. [**Hello-Agents — Extra05 Agent Skills 与 MCP 对比解读**](https://github.com/datawhalechina/hello-agents/blob/main/Extra-Chapter/Extra05-AgentSkills解读.md) — Skills vs MCP 概念对比 + +### 动手练习 +- **练习：SKILL.md** — 写一份 200 字的 skill，解决你日常工作中的某一件事。**step-by-step 怎么做** → [`resources/cookbook.zh-Hans.md` 1](../resources/cookbook.zh-Hans.md#1-写你的第一个-skill) +- **练习：SKILL with references** — 加一份 `references/` markdown 让 skill 可以引用 +- **练习：SKILL eval** — 加 `evals/evals.json`，放 3-5 个自我测试 + +> 📦 **本 repo 自带 meta-example**：[`examples/stage-5/tool-calling-tutor/`](../examples/stage-5/tool-calling-tutor/) 是这个 stage 的对应 skill 范本——完整 frontmatter（含 trigger phrases + Do NOT use for）、3 份 `references/`、`evals/evals.json` 5 个 test case，**直接 fork 改成你自己的 skill**。双重用途：(a) 学习者自用、卡在 tool calling 时让它 auto-load 帮你 debug；(b) Stage 5 5.3 SKILL.md 写法的对照样板。 + +### 常用 Skills 推荐（按用途分类） + +> 不知道从哪里开始？下面是 2025 后段官方 + 社区常用 skill。**安装方式**：(a) 多数来自 plugin、安装对应 plugin 即得；(b) 或从 [anthropics/skills](https://github.com/anthropics/skills) clone 后放进 `~/.claude/skills/` 或 `.claude/skills/`。 + +| 用途 | Skill | 来源 | 为什么推荐 | +|---|---|---|---| +| **🛡 装 skill 前安全检查**（必装） | `skill-vetter` | anthropics/skills | **装任何外部 skill 前必跑**——检查红旗、permission scope、suspicious pattern。等于 marketplace skill 的 SAST | +| **🔍 找 / 安装 skill** | `find-skills` | anthropics/skills | 自然语言查询、自动安装。“我想要做 X”就回对应 skill | +| | `skill-lookup` | claude-plugins-official | 跟 find-skills 互补，探索 / 搜索 helper | +| **✍ 写自己的 skill** | `skill-creator` | anthropics/skills + claude-plugins-official | 自动产生 frontmatter + 子目录结构、写 skill 必装 | +| **📄 Office docs 处理** | `pdf` / `docx` / `xlsx` / `pptx` | anthropics/skills | 读写 PDF / Word / Excel / PowerPoint。**必装 set**——任何 office workflow 必备 | +| **🔧 Code review** | `code-reviewer` / `code-review-excellence` | claude-plugins-official | staged diff 安全 / 风格 / 测试 review | +| **🐛 Debug** | `debugger` / `systematic-debugging` | claude-plugins-official | 系统化 root cause 分析、避免 quick fix | +| **🎓 学术写作** | `academic-writing-skills` | community | findings-first / mechanism / banned word audit | +| **🔌 MCP 整合 / 写 server** | `mcp-builder` / `mcp-integration` | claude-plugins-official | 写 MCP server 跟整合既有 server 的脚手架 | +| **💻 frontend / fullstack** | `frontend-developer` / `fullstack-developer` | claude-plugins-official | React 组件 / 全栈架构辅助 | +| **📊 数据分析** | `data-analyst` / `visualization-expert` | community | SQL / pandas / chart 选型 | +| **⚙ 权限 / 设置整理** | `update-config` / `fewer-permission-prompts` | claude-plugins-official | hooks / permissions / env var 管理 | +| **🔁 自我改进** | `self-improving-agent` | community | 捕捉 learning / error / correction、agent 持续改进 | +| **🌐 通用 / fallback** | `general-purpose` | Claude Code 内建 | 复杂开放任务、未涵盖情境的 default 入口 | + +**建议入手顺序**： +1. **第一个必装**：`skill-vetter`（装其他 skill 前先用它检查） +2. **第二批必装**：`skill-creator` + `find-skills`（写 / 找 skill 用） +3. **依工作领域**：Office workflow 加 `pdf`/`docx`/`xlsx`、开发加 `code-reviewer`/`debugger`、学术写作加 `academic-writing-skills` +4. **想看更多**：逛 `obra/superpowers` 或 `wshobson/agents` 看 production 范本 + +### 精选 Projects（spec / 范本参考） + +> 💡 **找日常用 Skill（NotebookLM、Excalidraw、Office docs 等）？** +> 看 [`resources/mcp-skills-catalog.zh-Hans.md`](../resources/mcp-skills-catalog.zh-Hans.md)——按使用情境分类，含 Anthropic 官方 + 社区 Skill。下表保留的是“**写自己 Skill 时的 spec / showcase reference**”性质。 + +| Project | ⭐ | 适合谁 | 为什么推荐 / 备注 | +|---|---|---|---| +| [anthropics/skills](https://github.com/anthropics/skills) ⭐ 官方 spec | ⭐⭐⭐⭐⭐ | 写自己 SKILL.md 前先读 | Anthropic 官方 Skills repo：`spec/`（frontmatter 标准）+ `template/` 起手范本 + `skills/` 含 pdf / docx / xlsx / pptx / skill-creator / skill-vetter 等 reference 实现。★ 158k+。**SKILL.md 结构范本参考**。Agent Skills 更广义标准另见 [agentskills.io](https://agentskills.io) | +| [anthropics/claude-code](https://github.com/anthropics/claude-code) | ⭐⭐⭐⭐ | 追踪新功能、看 release notes | Claude Code 主 repo、含 issues / releases / inline skill 范例。本 stage 学 Skill 重点看上一个 repo、这个排第二 | +| [mattpocock/skills](https://github.com/mattpocock/skills) | ⭐⭐⭐⭐ | 想看“真实工程师日常 SKILL.md” | Matt Pocock（TypeScript 社区知名教学者）公开自己工作真正在用的 `.claude/` 目录。每个 SKILL.md **10-50 行极短**、不过度工程化。**对照 over-engineered 200 行 skill 特别有参考价值**（★ 157k+、MIT）| +| [obra/superpowers](https://github.com/obra/superpowers) | ⭐⭐⭐⭐ | power user setup、学进阶写法 | 20+ 实战 skill（TDD、debugging、合作模式）+ `/brainstorm` / `/write-plan` / `/execute-plan` 命令 + skills-search tool | +| [wshobson/agents](https://github.com/wshobson/agents) | ⭐⭐⭐⭐ | 中阶：学 skill + subagent 组合 | 把 skills + subagents 组合做 multi-agent 编排。**从单一 SKILL.md 进化到 agent-as-skill 组合 pattern** 的范例（★ 35k+、MIT） | +| [travisvn/awesome-claude-skills](https://github.com/travisvn/awesome-claude-skills) | ⭐⭐⭐⭐ | 自己写前先找有没有现成的 | 社区 Claude Skills 精选目录 | +| [VoltAgent/awesome-agent-skills](https://github.com/VoltAgent/awesome-agent-skills) | ⭐⭐⭐ | 跨工具视角 | 1000+ agent skill、相容 Claude Code / Codex / Gemini CLI / Cursor（★ 26k+、MIT）| +| [alirezarezvani/claude-skills](https://github.com/alirezarezvani/claude-skills) | ⭐⭐⭐ | 找特定领域 skill 范例 | 232+ Claude Code skill、跨 engineering / marketing / product / compliance | + +--- + +## 5.4 — Plugins 与 Marketplaces + +### Plugin 是什么（先定位） + +**Plugin = MCP + Skills + slash commands + hooks 的组合包**——把前面 5.2 / 5.3 学到的零件 **打包成一个单位、可以 `/plugin install` 一次装进去**。 + +``` +Plugin +├── .mcp.json ← 5.2 学的 MCP server config（提供 tool / data） +├── skills//SKILL.md ← 5.3 学的 skill（行为包） +├── commands/.md ← slash command（5.1 学的、自定义 prompt 入口） +├── hooks/ ← 触发点 hook（譬如 PreToolUse、SessionStart） +├── agents/.md ← 5.5 学的 subagent（如果有） +└── .claude-plugin/plugin.json ← 打包元数据 +``` + +**为什么要 plugin**：你写了好用的 skill 想 share → 一行 `git clone` 太麻烦、设置也容易装错。包成 plugin、push 到 marketplace、team 其他人 `/plugin install foo@your-marketplace` 一次到位。 + +**Plugin 跟 marketplace 差在哪**：plugin 是**单一打包单位**、marketplace 是**多个 plugin 的目录**（譬如 anthropics/claude-plugins-official 是 marketplace、里面 35 个 plugin）。 + +### 学习目标 +- `plugin.json` schema（name、version、skills array、configuration） +- `marketplace.json` schema（plugins array、source、metadata） +- `claude plugin marketplace add` 的流程 +- 区分 single-plugin bundle vs multi-plugin marketplace +- 发布自己的 marketplace + +### 必修阅读 +1. [**Anthropic — Plugins 文档**](https://docs.claude.com/en/docs/claude-code/plugins) +2. **读下面 2-3 个 marketplace 的 `plugin.json` 与 `marketplace.json`** + +### 动手练习 +- **练习：plugin install** — 安装下面的某一个 marketplace，看它加载 +- **练习：plugin.json** — 把 5.3 写的 SKILL.md 打包成一个 plugin +- **练习：marketplace publish** — push 到 GitHub，用 `claude plugin marketplace add` 安装 + +### 常用 plugin 推荐（按用途分类） + +> 不知道从哪里开始装 plugin？下面是 2025 后段 Anthropic 官方 + 社区高评价选择。**安装指令统一格式**：`/plugin install @`（譬如 `/plugin install code-review@claude-plugins-official`）。 + +| 用途分类 | Plugin（含直接链接） | Marketplace | 为什么推荐 | +|---|---|---|---| +| **开发 workflow**
（多数开发者必装） | [`code-review`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/code-review) | claude-plugins-official | 官方 code review skill 集合、staged diff review + security check | +| | [`pr-review-toolkit`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/pr-review-toolkit) | claude-plugins-official | PR review 完整流程（comment、suggest、approve）| +| | [`commit-commands`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/commit-commands) | claude-plugins-official | git commit message 规范 + branching workflow | +| | [`feature-dev`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/feature-dev) | claude-plugins-official | 完整 feature 开发 cycle（spec → plan → implement → test） | +| | [`frontend-design`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/frontend-design) | claude-plugins-official | UI 设计 + responsive layout 辅助 | +| **语言工具**
（依用的语言挑）| [`typescript-lsp`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/typescript-lsp) / [`pyright-lsp`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/pyright-lsp) / [`rust-analyzer-lsp`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/rust-analyzer-lsp) / [`gopls-lsp`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/gopls-lsp) 等 | claude-plugins-official | 各语言 LSP 整合、[35 个语言 plugin](https://github.com/anthropics/claude-plugins-official/tree/main/plugins) 都在这 | +| **plugin / skill 自建** | [`skill-creator`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/skill-creator) | claude-plugins-official | 写自己的 skill 时自动产生 frontmatter + 结构 | +| | [`plugin-dev`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/plugin-dev) | claude-plugins-official | 写自己的 plugin 时自动产生 `.claude-plugin/` 结构 | +| | [`mcp-server-dev`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/mcp-server-dev) | claude-plugins-official | 写自己的 MCP server 时的脚手架 | +| | [`hookify`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/hookify) | claude-plugins-official | 写 hooks 规则的工具 | +| **领域特化 — 工程团队** | [**`engineering` bundle**](https://github.com/anthropics/knowledge-work-plugins/tree/main/engineering) | knowledge-work-plugins | **10 个 skill**：architecture / code-review / debug / deploy-checklist / documentation / incident-response / standup / system-design / tech-debt / testing-strategy | +| **领域特化 — 财务团队** | [**`finance` bundle**](https://github.com/anthropics/knowledge-work-plugins/tree/main/finance) | knowledge-work-plugins | **8 个 skill**：audit-support / close-management / financial-statements / journal-entry-prep / reconciliation / sox-testing / variance-analysis | +| **领域特化 — 其他**
（同 marketplace）| [`sales`](https://github.com/anthropics/knowledge-work-plugins/tree/main/sales) / [`marketing`](https://github.com/anthropics/knowledge-work-plugins/tree/main/marketing) / [`legal`](https://github.com/anthropics/knowledge-work-plugins/tree/main/legal) / [`human-resources`](https://github.com/anthropics/knowledge-work-plugins/tree/main/human-resources) / [`customer-support`](https://github.com/anthropics/knowledge-work-plugins/tree/main/customer-support) / [`data`](https://github.com/anthropics/knowledge-work-plugins/tree/main/data) / [`design`](https://github.com/anthropics/knowledge-work-plugins/tree/main/design) / [`operations`](https://github.com/anthropics/knowledge-work-plugins/tree/main/operations) / [`product-management`](https://github.com/anthropics/knowledge-work-plugins/tree/main/product-management) / [`productivity`](https://github.com/anthropics/knowledge-work-plugins/tree/main/productivity) / [`bio-research`](https://github.com/anthropics/knowledge-work-plugins/tree/main/bio-research) 等 | knowledge-work-plugins | knowledge-work-plugins **[18 个 vertical bundle](https://github.com/anthropics/knowledge-work-plugins)**——挑跟你工作领域对应的那个 | +| **外部整合**
（第三方服务） | [`asana`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/asana) / [`github`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/github) / [`gitlab`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/gitlab) / [`linear`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/linear) / [`firebase`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/firebase) / [`playwright`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/playwright) / [`terraform`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/terraform) / [`discord`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/discord) / [`imessage`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/imessage) / [`telegram`](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/telegram) 等 | claude-plugins-official (external) | 整合常用 SaaS / 开发工具 | +| **community 广度** | （挑感兴趣的 skill） | [rohitg00/awesome-claude-code-toolkit](https://github.com/rohitg00/awesome-claude-code-toolkit) | 社区最大 agents / skills / hooks / templates 目录 | + +**建议入手顺序**： +1. 开发者必装（5 个）：`code-review` + `pr-review-toolkit` + `commit-commands` + `feature-dev` + 一个你语言的 `*-lsp` +2. 按工作领域加 bundle：工程团队装 `engineering`、财务装 `finance`、其他类似 +3. 想写自己的 skill / plugin → 装 `skill-creator` + `plugin-dev` +4. 想看更多 → 逛 `awesome-claude-code-toolkit` 或 [`resources/mcp-skills-catalog.zh-Hans.md`](../resources/mcp-skills-catalog.zh-Hans.md) + +### 精选 Projects（marketplace 范本参考） + +> 💡 上面列的是“**装哪些 plugin**”；下表列的是“**marketplace 怎么写**”——想自建 marketplace 的人才需要看。 + +| Marketplace | ⭐ | 适合谁 | 为什么推荐 / 备注 | +|---|---|---|---| +| [anthropics/claude-plugins-official](https://github.com/anthropics/claude-plugins-official) | ⭐⭐⭐⭐⭐ | 写自己的 marketplace 前的官方范本 | 35 internal plugins + 15 external、`.claude-plugin/marketplace.json` 标准 schema、`plugins/` 含 plugin 本体 + `external_plugins/` 引用外部 repo。**marketplace.json 该长什么样直接看这个**（★ 30k+） | +| [anthropics/knowledge-work-plugins](https://github.com/anthropics/knowledge-work-plugins) | ⭐⭐⭐⭐⭐ | 想看“多 vertical bundle”型 marketplace | **18 个领域 plugin bundle**（finance / engineering / sales / legal / marketing / HR / customer-support / data / design / operations / product / productivity / bio-research / enterprise-search / pdf-viewer / small-business / cowork-plugin-management / partner-built）。Anthropic 自家 knowledge worker 场景范本 | +| [obra/superpowers-marketplace](https://github.com/obra/superpowers-marketplace) | ⭐⭐⭐⭐ | 想做“我策展、别人写”型 marketplace | **最简 marketplace template**——repo 只有 `marketplace.json` + README、plugin 本体放外部 repo。curator-only pattern 最小范本（★ 1.1k+、MIT）| +| [trailofbits/skills-curated](https://github.com/trailofbits/skills-curated) | ⭐⭐⭐ | 在意供应链安全的 reviewer / 团队 | Trail of Bits 维护的 **security-vetted** marketplace、每个 skill 都经审查、README 写清楚标准。**示范 marketplace 不只是清单、也是信任机制**（★ 431、CC-BY-SA-4.0）| +| [rohitg00/awesome-claude-code-toolkit](https://github.com/rohitg00/awesome-claude-code-toolkit) | ⭐⭐⭐ | 想逛社区有什么 | 社区最大 Claude Code agents / skills / hooks / templates 目录。涵盖 use case 广 | +| [anthropics/life-sciences](https://github.com/anthropics/life-sciences) | ⭐⭐⭐ | 要做特定领域 marketplace（医疗、金融、法律、教育等） | Anthropic 自家**领域特化 marketplace** 范例（生物 / 健康科学）、展示 `marketplace.json` 为单一 vertical 量身设计。**payload 偏生科 MCP server、marketplace.json 结构才是学习重点**（★ 474）| +| [anthropics/claude-for-legal](https://github.com/anthropics/claude-for-legal) | ⭐⭐⭐⭐ | 想看完整 vertical plugin suite（skills + agents + MCP + scheduled agents） | **Anthropic 官方法律 vertical 参考**（★ 7.9k+、Apache-2.0）——10 个法律 plugin（commercial / corporate / litigation / privacy / employment / IP / law-student）+ 100+ skills + 20+ MCP connectors + scheduled agents + subagent delegation。**你不需要懂法律**——这是学“**怎样设计 vertical plugin suite**”最好的教材：system prompt pattern、accountability surface，以及 `orchestrate.py` event loop。 | + +> 💡 **“如何发布自己的 marketplace”walkthrough**：目前最可靠的是 [Anthropic 官方 plugin 文档](https://docs.claude.com/en/docs/claude-code/plugins)。社区有好的博客 / repo？欢迎开 PR 补上。 + +--- + +## 5.5 — Subagents（Claude Code 原生 multi-agent 机制）⭐ 2025 新功能 + +到这里为止你学了 MCP（工具层）/ Skills（行为层）/ Plugins（散布层）。**Subagents 是 orchestration 层**（orchestration = 调度一群 agent：分工，再把结果合起来）——让主 Claude session spawn 出有独立 context 的子 agent、跑特定任务、回报结果。 + +![Subagent 的 4 个生命周期：从 .md 文件到执行结果](../resources/diagrams/subagent-4-stage-flow.zh-Hans.png) + +> 📊 **上图**：subagent 从**定义 → 发现 → 派遣 → 执行** 4 个阶段、看完这张再读下面细节最快。 + +跟 Stage 4 的 framework-based multi-agent（LangGraph / CrewAI / AutoGen）对照： + +| 维度 | Framework path (Stage 4) | Claude Subagent path（本节） | +|---|---|---| +| 启动方式 | `pip install crewai` + Python code | 写一个 `.claude/agents/.md` 即可 | +| Runtime | 你自己的 Python process | Claude Code 内建 Task tool | +| Context isolation | framework 自己管 | **天生** 各 subagent 独立 window | +| Provider lock-in | 中等（多 framework 支持 multi-LLM） | **强**（绑 Claude Code） | +| 适合 | 跨 LLM provider 的 production system | 已 commit Claude Code 的工程团队 | +| 学习曲线 | 高（框架抽象 + async） | 低（写 markdown）| + +### 各家 CLI / SDK 的 multi-agent 机制现状（2025 后段） + +很多人以为 multi-agent CLI 是 Anthropic / OpenAI / Google 三家标配——但实际上目前只有 **Claude Code 有完整 native multi-agent stack**。Codex CLI / Gemini CLI / Cursor 都还是 single-agent，要 multi-agent 得自己用 SDK 或 framework 写。 + +| 平台 | Subagent | Agent team | Background agent | 机制 | +|---|:---:|:---:|:---:|---| +| **Claude Code**（CLI） | ✅ | ✅ | ✅ | `.claude/agents/.md` + Task tool（subagent）+ [agent teams](https://docs.claude.com/en/docs/claude-code/agent-teams) + [agent view / background](https://docs.claude.com/en/docs/claude-code/agent-view) | +| **OpenAI Codex CLI** | ❌ | ❌ | ❌ | `AGENTS.md` 只是 **single-agent context file**（类似 CLAUDE.md），**不是 subagent 系统** | +| **Google Gemini CLI** | ❌ | ❌ | ❌ | `GEMINI.md` 只是 context；无 subagent / multi-agent feature | +| **Cursor**（IDE-coupled） | ❌ | ❌ | ❌ | 单一 Cursor Agent；queued messages 是 sequential、非 parallel | +| **OpenAI Agents SDK**
（programmatic、非 CLI） | ⚠️ Handoffs + agents-as-tools | ❌ | ❌ | 纯 Python SDK、不是 CLI；handoff pattern 接近 Claude subagent 但要写 code | +| **Framework path**
（Stage 4） | LangGraph / CrewAI / AutoGen | ✅ 自己 wire | 部分 | 跨 LLM provider、Python orchestration、见 [Stage 4](04-agent-frameworks.zh-Hans.md) | + +**现状解读**： + +- 想用 **CLI** 玩 multi-agent → 目前只有 Claude Code 有 native 支持（**本节主题**） +- 想 **跨 provider / 跨 LLM** → 走 Stage 4 framework path +- 想 **OpenAI 生态 + 多 agent** → 用 OpenAI Agents SDK 写 handoff pattern（programmatic、非 CLI） +- 想 **完全自己控** → 走 [Stage 5.7 Harness Internals](#57--claude-code-source-解剖reference-harness-implementation-track-b-必看)（读 SDK source、自己 wire 多 agent） + +→ 本节剩下内容都聚焦在 **Claude Code subagent**。其他平台的进展请追踪各家 changelog（Codex / Gemini / Cursor 都还在 single-agent + MCP 阶段、可能 2026 后段才会跟进）。 + +### 怎么派遣 Claude Code 的 3 种 multi-agent 机制（具体 syntax） + +| 机制 | 何时用 | 派遣方式 | +|---|---|---| +| **Subagent**
（稳定版） | delegate 大 context 任务（读整个 codebase / 整理 logs）给 isolated context worker、结果回主 session | (1) 写 `.claude/agents/.md`（frontmatter `name` + `description` + `tools` + 可选 `model`）
(2) Claude 看 description **自动 delegate**；或 `/agents` 手动列表 | +| **Agent team**
（已有正式 docs、仍需 opt-in flag） | 多 worker 之间要**互相沟通**、challenge 彼此（debate / peer review / 多角度探索） | (1) **启用**（仍需 opt-in）：`settings.json` 加 `"env": {"CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS": "1"}`、需 Claude Code v2.1.32+
(2) 自然语言派遣：`Create an agent team to explore X from different angles: one on UX, one on architecture, one playing devil's advocate`
(3) 跟 teammate 对话：`Shift+Down` 切换、直接输入消息
(4) 收尾：`Clean up the team` | +| **Background agent**
（research preview） | 多个**独立任务**各自背景跑、单一界面监控（同时 3 个 PR review） | (1) shell 派遣：`claude --bg "investigate the flaky test"`（需 v2.1.139+）
(2) 从现有 session 背景化：`/bg`
(3) 监控：`claude agents`（agent view 界面）
(4) 操作：`claude attach ` / `claude logs ` / `claude stop ` | + +**3 个机制怎么选**： + +- 任务独立、worker 不互动、结果回主 session 即可 → **Subagent**（最简单、token 最省） +- Worker 需要互相沟通 / debate / 共享 task list → **Agent team**（已正式有 docs、但仍需 opt-in env var；token 3-5x、适合 research / debug 竞争假设） +- 多个独立任务各自跑、想用 1 个界面监控全部 → **Background agent**（research preview、适合长时间任务并行） + +--- + +### 可派遣的 subagent 有哪些？ + +> 💡 **先解释一下名词**：**subagent** = 主 Claude session spawn 出来的“子 Claude”——有自己的 context window（一次能记住的对话量，有上限），跑完回报结果。**派遣（dispatch）**就是叫 subagent 去做事，像派任务给同事。 + +很多人以为要用 subagent 都得自己写一个——其实 **Claude Code 内置一批 subagent，开箱即用**。下表列三种来源： + +| 来源 | 范例 subagent | 何时用 | 需要做什么 | +|---|---|---|---| +| **Claude Code 内置** | `general-purpose` / `code-reviewer` / `Explore` / `Plan` / `frontend-developer` / `claude-code-guide` / `statusline-setup` | 一般任务都先看内置有没有合适的 | **什么都不用做，直接调用** | +| **plugin / marketplace** | `obra/superpowers` 内含的 skill agent、`wshobson/agents` 的多 subagent 组合 | 内置不够用时 | 装 plugin / marketplace（[Stage 5.4](#54--plugins-与-marketplaces)）| +| **自己写的** | 你公司流程 specific 的 reviewer / domain expert | 上面都不符合时 | 写 `.claude/agents/.md`（范例见下面 details 区块）| + +> 🔍 **想知道你的 Claude Code 现在有哪些 subagent 可用？** 终端跑 `/agents` 一个指令列表（内置 + plugin + 自定义全部）。 + +### 怎么选哪一个 subagent？（decision table） + +对应上面 7 个 Claude Code 内置 subagent，下表是“**遇到 X 任务，用 Y subagent**”对照（这叫 **decision table**——“要 X 用 Y”的快速对照，不用自己想）： + +| 你要做的事 | 用哪个内置 subagent | 为什么 | +|---|---|---| +| 找 code / 探索陌生 codebase 结构 | `Explore` | 专门做 read-only 搜索，不会乱改 | +| 设计实作 plan（不直接写 code） | `Plan` | 输出 step-by-step 计划，适合大任务拆解前 | +| Review staged diff / 安全审查 / 发 commit 前检查 | `code-reviewer` | 结构化输出 PASS/FAIL + 具体 fix | +| 写 / 改 UI component / 处理 accessibility（无障碍设计）| `frontend-developer` | React / 响应式 / a11y（accessibility 缩写，视障 / 键盘用户也能用的设计）领域知识 | +| 多步骤研究，不确定任务该归哪类 | `general-purpose` | 通用，可 web search，适合 fallback | +| 问 Claude Code 自己的 feature 怎么用 | `claude-code-guide` | hooks（工具执行前 / 后的拦截脚本，见下方 Gotcha #5）/ slash command（`/` 开头的指令）/ MCP 等问题 | +| 上面都不符合 | 自己写 `.claude/agents/.md` | 客制或公司 specific 流程 | + +**5 个常见场景的 mini cookbook**（完整 15 个 recipe 见下面）： + +| 场景 | 用哪个 | +|---|---| +| 写了 ≥ 50 行新 code，要 commit 前 | `code-reviewer` | +| Clone 完新 repo，不知道该从哪个 file 开始 | `Explore` | +| 4 个 stage / branch 都要做同样审查 | `general-purpose`（spawn 多个并行）| +| 想重构 module，先 review architecture | `Plan` | +| 多 source 比对哪篇 paper 讲得对 | `general-purpose` 跑 deep research | + +> 📋 **完整 15 个 recipe**（每个含**场景 + subagent + 直接复制粘贴的 prompt 模板 + 何时不用**）→ [`resources/subagent-cookbook.zh-Hans.md`](../resources/subagent-cookbook.zh-Hans.md) + +### 易混淆观念厘清（学完表格还是有点雾、看这节） + +学生最常搞混的 **3 组概念** + **5 条老手才知道的 gotcha**——挑你需要的看： + +#### Subagent vs Skill — 5 个关键差别 + +很多人把 Subagent 跟 Skill 当同一件事——其实是**完全不同层的东西**： + +![Subagent vs Skill — 5 个关键差别](../resources/diagrams/subagent-vs-skill.zh-Hans.png) + +| 维度 | Subagent（子 agent） | Skill（技能） | +|---|---|---| +| **执行环境** | 新的独立 context window（底层是新 subprocess）| 主 session 内、同 context | +| **工具权限** | 自己的 `tools:` 清单（可限制只能 Read / Grep）| 主 session 的工具（默认全开、skill 可用 `allowed-tools:` 缩减）| +| **返回结果** | 一个 final message 摘要回主 session | 没有返回、是行为改变（规则 / persona）| +| **适合做** | 长任务 / 并行跑 / 要 context 隔离 | 知识注入 / 规则 / 改 Claude 行为 | +| **范例** | `code-reviewer` / `Explore` / `Plan` | `codex-delegate` / `pdf`（anthropics/skills）| + +**判断快速办法**：你**要新 context window** 吗？要 → subagent；不要 → skill。 + +#### Subagent vs Slash Command — 一个是任务、一个是指令 + +| 东西 | 怎么触发 | 例子 | +|---|---|---| +| **Subagent** | 直接打对话文字、Claude 看 description 自动派遣 | 你打 "Review my staged changes" → 自动派 `code-reviewer` | +| **Slash command** | 打 `/` 开头的指令 | `/agents`（列 subagent）/ `/compact`（压缩 context）/ `/help` | + +⚠️ **常见误会**：`/agents` **不是用来调用 subagent**——它是 "查当前可用 subagent 清单" 的指令。**派遣是直接打对话 prompt 文字**、Claude 自己挑 subagent。 + +#### Description = 路由 key（**写法决定能不能被选**） + +主 session 怎么知道该派哪个 subagent？看 `.claude/agents/.md` 的 **`description` 字段**。**写法影响触发行为**： + +| Description 写法 | 触发模式 | 例 | +|---|---|---| +| `...use **PROACTIVELY** when X...` | **主动触发**——X 出现 Claude 自己派 | "use PROACTIVELY when reviewing diffs ≥ 50 lines" | +| `...use when user asks Y...` | **被动触发**——要用户明确要求 | "use when user asks for code review" | +| 空 description | **隐形**——不会被自主选 | （只能在代码里用 `Agent(subagent_type=...)` 强制调用）| + +> 💡 **写 description 像写广告词**——把 "我能解决什么问题" **写具体**、Claude 越会在对的时机选你。`PROACTIVELY` 是个**强信号词**——出现时 Claude 推断 "适合主动派遣" 的概率大幅提升；没写就更常只在用户明确要求时才会派。（它影响 Claude 的判断、**不是代码层的 if-then 开关**。） + +#### 5 条老手才知道的 Gotcha + +| # | Gotcha | 为什么重要 | +|---|---|---| +| 1 | **Description 写精准即可** | 无官方字符上限、但过长 description 占 context budget；建议 "触发条件 + 适用场景" 写具体、避免重复 | +| 2 | **`tools:` 写空 = 继承主 session 全部工具** | 想限制 subagent 就要**明写**工具清单；空字段 ≠ 没工具 | +| 3 | **不写 `model:` = 跟主 session 用同 model** | 主 session 是 Opus、subagent 没指定也 Opus（烧大钱）。省成本就写 `model: sonnet` 或 `model: haiku`| +| 4 | **Subagent 没 "我之前说过 X" 记忆** | 每次派遣都是**全新 context**、看不到主 session 对话。Prompt 要 self-contained、不能 reference "我们刚讨论的 Y" | +| 5 | **Subagent 也吃 hook** | PreToolUse / PostToolUse（工具执行前 / 后的拦截脚本）在 subagent 内**也会 fire**。设 hook 时要想到这层 | + +#### Subagent 整体优缺点（读完前面，回头看这个 summary） + +**5 个优点**（为什么存在）： + +| 优点 | 怎么帮到你 | +|---|---| +| **Context 隔离** | 主 session window 不被污染——subagent 跑大文件 / 长 log 不会挤掉主 session 的工作记忆 | +| **Tool allowlist** | 限制 subagent 只能用 Read / Grep（不能写文件 / 不能跑 Bash）= 安全 sandbox | +| **Model override** | 跑简单任务用 Haiku、跑难的用 Opus、混搭省成本——主 session 是 Opus 也可以叫 subagent 用 Haiku | +| **Parallel spawn** | 一个 prompt spawn N 个 subagent 平行跑、wall clock 时间 ÷ N（适合 4 个 file 同时 audit）| +| **专业化 prompt** | `code-reviewer` 永远只 review、description 写死 "Use PROACTIVELY when commit"，不会被闲聊干扰 | + +**5 个缺点**（什么时候不值得）： + +| 缺点 | 影响 | +|---|---| +| **Spawn 有 overhead** | 任务 < 5 分钟、自己跑更快——subagent startup 也吃时间跟 token | +| **无 cross-call memory** | 每次 spawn 都新 context、看不到 "我们刚讨论的 X"——prompt 必须 self-contained | +| **只回一个 message** | subagent 是 "派出去、跑完回报一次"，不能跟你来回对话，不适合需要逐步 feedback 的任务 | +| **Token cost N ×** | spawn 4 个 = 用 4 倍 token——parallel 的 ROI 要算（时间省、钱花更多）| +| **Debug 多一层** | 出错不知道该怪主 session description / subagent system prompt / 还是 prompt 本身——见 [advanced §3 debug 5 切点](../resources/subagent-advanced.zh-Hans.md#3-自制-subagent-的-debug-工具)| + +> 📌 **1 句话判断**：任务 **≥ 5 分钟** + **可以用一个 brief 写死**（不需要来回对话）+ **结果一次回来够用**（不需要逐步 feedback）→ 用 subagent；否则自己跑。 + + +
+👉 具体 subagent 文件范例（最简单入门） + +`.claude/agents/code-reviewer.md`： + +```markdown +--- +name: code-reviewer +description: Review staged git changes for security issues, style violations, and missing tests. Use when user asks "review my changes" or runs /review. +tools: + - Read + - Grep + - Bash +model: claude-haiku-4-5 # 可选、想 route 到便宜 model 省成本 +--- + +You are a senior code reviewer. When invoked: +1. Run `git diff --cached` to get staged changes +2. Check for: hard-coded secrets, SQL injection patterns, missing error handling, missing tests +3. Output: PASS / list of specific issues with file:line references +``` + +主 session 之后输入“review my changes”，Claude 看到 description 匹配、自动通过 Task tool（Claude Code 内部派遣机制，你不用直接调用）spawn 这个 subagent 跑、回主 session 一段摘要。 + +
+ +> 📚 **官方完整文档**： +> - [Subagent spec](https://docs.claude.com/en/docs/claude-code/sub-agents)（frontmatter 字段、project vs user scope、Task tool 界面） +> - [Agent team 完整指南](https://docs.claude.com/en/docs/claude-code/agent-teams)（display modes、task list、subagent-as-teammate 进阶） +> - [Agent view / background](https://docs.claude.com/en/docs/claude-code/agent-view)（v2.1.139+、quick start + dispatch 流程） + +### 学习目标 + +- 讲得出 subagent 跟 skill / MCP server 的差别（**subagent ≠ skill**：skill 是行为 prompt，subagent 是**另一个 Claude instance with isolated context**） +- 写一个 `.claude/agents/.md` 自定义 subagent（frontmatter + system prompt + `tools:` 白名单——明写允许的工具清单） +- 从主 session 用 Task tool invoke subagent，观察 context 隔离（parent 看不到 subagent 的中间 step、只看到最终 result） +- 知道何时用 subagent（parallel research / large-context isolated task / specialized review），何时不用（小 query 用 skill 即可） + +### 必修阅读 + +1. [**Anthropic — Claude Code Subagents 官方文档**](https://docs.claude.com/en/docs/claude-code/sub-agents) ⭐ — `.claude/agents/` 结构、Task tool 界面、最佳实践 +2. [**Anthropic — Building Effective Agents orchestrator-workers**](https://www.anthropic.com/engineering/building-effective-agents) — Anthropic 自己对 orchestrator pattern 的看法（理论 + 实例） +3. [**Anthropic Cookbook — `customer_service_agent`**](https://github.com/anthropics/claude-cookbooks/tree/main/tool_use) — canonical multi-agent orchestration 范例（chapter-length 深度教材；notebook 在 `tool_use/customer_service_agent.ipynb`） + +### 动手练习 + +- **练习：第一个 subagent** — 写 `.claude/agents/code-reviewer.md`（前置 frontmatter 含 `description` 写清楚何时 trigger、`tools` 限定 Read+Grep）+ system prompt 跑 staged diff review。从主 Claude session 跑 `/agents list` 确认加载、然后用 prompt“review staged changes”观察 Task tool 怎么 spawn subagent +- **练习：parallel subagent crew** — 写 3 个 subagent（`researcher.md` / `writer.md` / `critic.md`）做“研究某主题 → 写 blog 草稿 → 审稿”pipeline、主 session 用 Task tool 串起来。**对照** [`examples/stage-4/02-multi-agent-roles/`](../examples/stage-4/02-multi-agent-roles/)（CrewAI 框架版同一个任务）、看“framework 路线 vs Claude 原生路线”代码差别 +- **练习：subagent 跟 skill 的决策练习** — 拿你自己日常工作流的 5 个常用任务、每个判断该用 skill（行为层）还是 subagent（独立 context 层）。写成 1 页 decision table + +> 📚 **想要 chapter-length 深入版**：subagent 进阶 pattern（agent-as-skill composition、parallel-spawn、handoff between subagents）→ 看 [`wshobson/agents`](https://github.com/wshobson/agents) repo 整个结构 + [`obra/superpowers`](https://github.com/obra/superpowers) 的 subagent 用法。 + +### 精选 Projects + +4 个项目一张表搞定。**挑入口看“适合谁”、想深入点链接看 repo**。 + +| Project | ⭐ | 适合谁 | 为什么推荐 / 备注 | +|---|---|---|---| +| [anthropics/claude-cookbooks](https://github.com/anthropics/claude-cookbooks) ⭐ 官方 | ⭐⭐⭐⭐⭐ | 5.5 完成后想看“实际在用的 agent 范例怎么写” | Anthropic 官方 chapter-length 范例。**`tool_use/customer_service_agent.ipynb`** = orchestrator-workers canonical（multi-agent routing + handoff）。Python / Jupyter notebook、MIT。**注**：`computer_use_demo` 完整版在另一个 repo [`claude-quickstarts/computer-use-demo`](https://github.com/anthropics/claude-quickstarts/tree/main/computer-use-demo) | +| [wshobson/agents](https://github.com/wshobson/agents) ⭐ subagent canonical | ⭐⭐⭐⭐⭐ | 写过 1-2 个 subagent 想看真实 team 范本 | 50+ subagent definition 的 production workflow pattern collection。**看 `.claude/agents/` 目录结构 + 命名 convention + 跨 agent handoff 写法** | +| [obra/superpowers](https://github.com/obra/superpowers) | ⭐⭐⭐⭐ | 想看 skill + subagent 混搭实践 | 在 Stage 5.3 已介绍。**重点看“什么任务归 skill、什么归 subagent”决策**——production 范本 | +| [anthropics/claude-plugins-official](https://github.com/anthropics/claude-plugins-official) 官方 | ⭐⭐⭐⭐ | 看 plugin 怎么打包 subagent | 在 Stage 5.4 已介绍。每个 plugin 内 `agents/` 子目录是 subagent definition、看打包方式 | + +> 💡 **Subagent 虽然强、不要无脑用**：每个 subagent invoke 都是一个新的 Claude inference call、有 token cost + latency。**简单 query 用 skill（行为 prompt）即可、不必 spawn subagent**。Subagent 的甜蜜点是：(1) 任务 context 大、会吃光主 session 的 window（譬如 read 整个 codebase），(2) 任务跟主 session 逻辑独立、隔离 context 有助 main flow，(3) 多 subagent 平行（research / write / critic）能省 wall-clock 时间。 + +> 🔗 **相关进阶机制**（Claude Code 官方、本 stage 不深入讲）： +> - **[Agent teams](https://docs.claude.com/en/docs/claude-code/agent-teams)** — 多 sessions 之间互相沟通（reviewer agent ↔ implementer agent 来回交流） +> - **[Background agents / agent view](https://docs.claude.com/en/docs/claude-code/agent-view)** — 多 session 背景跑、单一界面监控（一次 spawn N 个 PR review 同时跑） +> +> Subagent 是这两个的进入点——本节学完之后想扩展再看官方文档。 + +--- + +## 5.6 — Dynamic Workflows（让 Claude 自己写出 workflow）⭐ Opus 4.8+ 新机制 + +> **本节定位**：5.5 教你**手动**派 subagent；本节更上一层——**让 Claude 自己生成一份 workflow 脚本、再自己执行**。这是 Opus 4.8 起的新机制（research preview 出身），新版 Claude Code 内置。本节只把它放进生态地图、讲清楚跟 5.5 的分工；**机制 / 实例 / quality pattern 的完整版在 [Stage 7.5 — Dynamic Workflows 深入](07.5-advanced-agentic-concepts.zh-Hans.md#-dynamic-workflowsopus-48-当-agent-自己写出-workflow)**。 + +### 跟 5.5 Subagents 的差别 + +| | 5.5 Subagents | 5.6 Dynamic Workflows | +|---|---|---| +| 步骤谁决定 | 你手动派、一次一个 | Claude 自己写出多步骤脚本 | +| 控制流 | model 即兴决定下一步 | 脚本里是**确定性**的 loop / 并行 fan-out / 验证阶段 | +| 适合 | 少数几个并行子任务 | 大型、要穷举或多阶段验证（migration、audit、跨文件 review）| +| 关系 | — | DW **建在 subagent 之上**：workflow 脚本去 orchestrate 一群 subagent | + +### 什么时候用、什么时候别用 + +- **用**：要穷举 + 对抗式验证（找完所有 bug、每个 finding 再派独立 agent 反驳）、一次性大迁移、跨多文件同样转换的 pipeline。 +- **别用**：只是想叫一两个 agent 并行做点事 → 留在 5.5 就好；小任务直接一条 prompt 更省。 +- ⚠️ DW 会 spawn 大量 agent、吃 token，不是万灵丹。“何时值得、怎么写不会爆”见下方 7.5 深入。 + +### 📚 必修阅读 + +1. [**Anthropic — Claude Opus 4.8**](https://www.anthropic.com/news/claude-opus-4-8) — Dynamic Workflows 首次发布的官方说明 +2. **[Stage 7.5 — Dynamic Workflows 深入](07.5-advanced-agentic-concepts.zh-Hans.md#-dynamic-workflowsopus-48-当-agent-自己写出-workflow)** ⭐ — 机制、quality pattern（adversarial verify / loop-until-dry / judge panel）、何时用的完整版 + +> 本节无 examples（概念 + 入口节点）；想动手照 7.5 的 pattern 写。 + +--- + +## 5.7 — Claude Code Source 解剖（reference harness implementation）⭐ Track B 必看 + +> **本节定位**：本节**不是** harness engineering 的 discipline 概念教学——discipline 级的定义 / **8 元件** / prompt→context→harness 三层 lineage 是 **[Stage 7 Harness Engineering](07-multi-agent-production.zh-Hans.md#-harness-engineering--production-agent-runtime-的工程设计--本-stage-核心概念)** 在讲。**本节是 case study**——拿 Claude Code（一个被广泛使用的 reference harness）的 source code 来解剖、把 Stage 7 列的 8 个元件**中前 6 个 runtime-internal 元件**（Eval / Cost-Latency 两个是 cross-cutting、不在 source 主 loop）**在实现里找到对应位置**。 + +### 学习目标 + +完成本节后你会： +- 看得懂 `claude-agent-sdk-python` source 的 main loop（不是逐行、是抓得到主干） +- 在 source 里标出 [Stage 7 列的 8 个 harness 元件](07-multi-agent-production.zh-Hans.md#-harness-engineering--production-agent-runtime-的工程设计--本-stage-核心概念)**中**前 6 个 runtime-internal 元件（agent loop / tool registry / context manager / safety layer / retry / telemetry）各自的 file:line。Stage 7 列的第 7 个 Eval 是外挂、第 8 个 Cost / Latency 是 cross-cutting、不在 source 主 loop 内、不在本练习范围 +- 讲得出 Claude Code 的 agent loop 跟 Stage 3 练习 3 from-scratch ReAct 差在哪——上线部署的 agent 多了哪些东西 + +> **discipline 级概念在哪**：harness engineering 是什么 / framework vs harness 差别 / prompt→context→harness 三层 lineage → 全部见 **[Stage 7 Harness Engineering](07-multi-agent-production.zh-Hans.md#-harness-engineering--production-agent-runtime-的工程设计--本-stage-核心概念)**。本节只负责 Claude Code source 的 case study。 + +### 📚 必修阅读 + +1. [**Anthropic — Building Effective Agents**](https://www.anthropic.com/engineering/building-effective-agents) ⭐ — orchestrator / worker / handoff / reflection 等 pattern 的 canonical reference +2. [**anthropics/claude-agent-sdk-python**](https://github.com/anthropics/claude-agent-sdk-python) — Claude Code 官方 Python SDK 的 source；**重点 file：`src/claude_agent_sdk/_internal/client.py`**（main loop 在这）+ `query.py`（单回合 API） +3. [**ai-boost/awesome-harness-engineering**](https://github.com/ai-boost/awesome-harness-engineering) ⭐（★ 2.8k+） — community curation：harness pattern / eval / memory / observability 整合 +4. [**ZhangHanDong/harness-engineering-from-cc-to-ai-coding**](https://github.com/ZhangHanDong/harness-engineering-from-cc-to-ai-coding) — 中文圈最完整的 Claude Code 内部解读 + +### 🛠 动手练习 — 解剖 agent loop（阅读题，非写 code） + +这节**不是写 code 练习，是阅读练习**——production harness 不是抄 200 行范例能学的，是抄完还看不懂为什么这样写，所以本练习要求你开 source、自己 trace。 + +**步骤**： +1. **clone**：`git clone https://github.com/anthropics/claude-agent-sdk-python` +2. **定位 agent loop**：找出 `_internal/client.py` 里实际发出 LLM call、收 tool_use response、dispatch 给 tool runner 的核心 loop。提示：找 `async def` 跟 `tool_use_id` 关键词 +3. **标出前 6 个 runtime-internal harness 元件**在 source 里的位置（文件名 + 行号）——对应 [Stage 7 列的 8 元件](07-multi-agent-production.zh-Hans.md#-harness-engineering--production-agent-runtime-的工程设计--本-stage-核心概念)的前 6 个（第 7 个 Eval 外挂 / 第 8 个 Cost-Latency cross-cutting 不在 source 主 loop）： + - (a) **Agent loop**：实际发出 LLM call + 收 response 的循环在哪 + - (b) **Tool registry / dispatch**：LLM 回 tool_use → 怎么 route 到对应 tool 实现 + - (c) **Context manager**：tool result 怎么写回 message history、context window 控制 / auto-compact + - (d) **Safety layer**：tool 执行前有没有 permission gate / sandboxing + - (e) **Retry / recovery**：tool fail 时怎么处理（exception vs LLM 自己看 error 反思） + - (f) **Telemetry**：metrics / logging / token counting 接在哪 +4. **写一段 80-150 字摘要**：“Claude Code 的 agent loop 跟你 Stage 3 练习 3 from-scratch ReAct 差在哪”。重点不是“Claude Code 比较复杂”这种废话，是**讲得出多了哪些东西、为什么那些是上线部署必须有的** + +**交付物**：一段笔记（写在自己的 obsidian / notion / `.md` 都行），不必交。但**讲不出来你就还没懂**——这是进 Stage 7 production deploy 之前的必要 mental model。 + +→ **基础 starter 范本**：本练习**无 examples folder**——是 source-reading exercise，非 code-writing exercise。illustrative，深度教学见上方 📚。 + +### 🎯 精选 Projects + +4 个项目一张表搞定。**挑入口看“适合谁”、想深入点链接看 repo**。 + +| Project | ⭐ | 适合谁 | 为什么推荐 / 备注 | +|---|---|---|---| +| [anthropics/claude-agent-sdk-python](https://github.com/anthropics/claude-agent-sdk-python) | ⭐⭐⭐⭐⭐ | 所有 Track B 学习者、想搞清楚“Claude Code 内部怎么跑” | **canonical Python harness、本节练习就是读这个 repo**。后面 Stage 7 deploy 也会 import | +| [ZhangHanDong/harness-engineering-from-cc-to-ai-coding](https://github.com/ZhangHanDong/harness-engineering-from-cc-to-ai-coding) | ⭐⭐⭐⭐ | 中文 reader 想看“为什么 Claude Code 这样设计” | 中文圈最完整 CC 内部解读（harness 概念 → CC 实现 → 跟其他 AI coding tool 对比）。**配合 SDK source 互补看**——一个告诉你“怎么做”、一个告诉你“为什么这么做” | +| [ai-boost/awesome-harness-engineering](https://github.com/ai-boost/awesome-harness-engineering) | ⭐⭐⭐⭐ | 5.7 读完想扩大视野 | community curation：30+ harness / eval / memory / observability / MCP project（★ 2.8k+）。**广度资源库、非教程**——挑感兴趣的 sub-topic 钻进去 | +| [wshobson/agents](https://github.com/wshobson/agents) | ⭐⭐⭐⭐ | 写完 5.5 自己的 subagent 后想看实际在用的范本 | 50+ subagent definition 的 ergonomic 设计（description / tool list / system prompt 分层）。**读 source 比读文件学得多**。在 5.5 已介绍、本节 cross-ref | + +> 💡 **本节跟 Stage 7 的差别**：本节学“Claude Code 这个 harness 怎么跑”（具体 reference）；Stage 7 学“production harness 一般要有什么”（抽象 pattern）。**先具体后抽象**、看完本节再进 Stage 7 会轻松很多。 + +--- + +## 5.8 — SDK：把 Claude Code 拆开来自己组 ⭐ Track B 可选、production 才需要 + +> 🎯 **这节是给谁看的**：99% 的人读完 5.1-5.7 已经够用，**只在你想做 CLI 做不到的事**才往下走。Stage 5.7 叫你读 SDK source 是为了理解 harness 内部；这节是为了让你**会用 SDK** 包成自己的服务。 + +### 1 个比喻把 SDK / CLI / `CLAUDE.md` 分清楚 + +- **CLI**（`claude` / `codex` / 等）= 一台**现成的车子**，点一下就能上路 +- 改 `CLAUDE.md` / `AGENTS.md` / 加 hooks / 写 skills = **调车子的性能**，让它开得更顺、更贴你工作习惯 —— 一样是这台车 +- **SDK**（`claude-agent-sdk-python` / `openai-agents-python`）= **把车子从引擎开始重造一台** —— 用 Python / TS 控制 agent loop、tool dispatch、memory 怎么接 + +**99% 的学习者天花板停在“调车”就够了。** 只在“调车怎么调都到不了你要的场景”时，才需要爬到 SDK。 + +### 阶梯式三层 —— 你现在在哪？ + +1. **第 1 层 直接用 CLI** —— 90% 的个人 + 团队使用情境。看 5.1 +2. **第 2 层 CLI + 自定** —— 写 `CLAUDE.md`、加 hooks、自己写 skill、套 plugin。看 5.1-5.4。**多数人停在这层、且够用** +3. **第 3 层 SDK** —— 把 agent 嵌进你的应用。这节在教 + +### 什么时候才需要爬到第 3 层 + +具体场景（不抽象）： +- **嵌进你已有的 web app / 后端** —— 用户不开 terminal，就不能用 CLI +- **cron / scheduler 自动触发** —— 没有人在 session 里点 enter，CLI 交互模式不适用 +- **公司内部包一层** —— 加 auth、audit log、限额、自定 prompt template，让 CLI 的能力以受控方式对外 +- **同时跑多 agent、要 programmatic 控制 hand-off** —— 比 Stage 5.5 的 Task tool 更细的控制权 + +如果你做的不在上面，你大概不需要 SDK。**该回 5.1-5.4。** + +### Hello SDK（4 行 Python） + +```python +from claude_agent_sdk import query + +async for msg in query(prompt="用 git status 看当前状态"): + print(msg) # 所有 message type 都能 print；要拿 agent 回复要 filter AssistantMessage +``` + +就这样 —— 包进 `async def` 就能跑。`query()` 会 yield 多种 message type（`AssistantMessage` / `ResultMessage` / `SystemMessage` 等），上面的 `print(msg)` 全部都能安全打印；想拿到 agent 真正的回复要 `isinstance(msg, AssistantMessage)` 再取 `msg.content` —— retry / streaming / prompt caching 等进阶用法在 Stage 7 练习 4。 + +### vs CLI / vs 自定 对照表（看完上面再看这张） + +| | CLI（claude / codex） | CLI + 自定（改 CLAUDE.md / hooks） | SDK | +|---|---|---|---| +| 嵌进你的 app | ❌ | ❌ | ✅ | +| cron / 排程跑 | ⚠️ 勉强（`-p` flag） | ⚠️ 同左 | ✅ | +| 换语言 / 环境 | 绑 Node / Bash | 同左 | Python / TS 随你 | +| programmatic 控制 | ❌ | ❌ | ✅ | +| 客制 system prompt | 受限 | 受限 | 完全自由 | +| 学习成本 | 1 天 | 1-2 周 | 1 个月+ | +| 适合谁 | 个人日常用 | 个人 / 小团队长期用 | 包成产品 / 服务 | + +### 两个主要 SDK + +| | [claude-agent-sdk-python](https://github.com/anthropics/claude-agent-sdk-python) | [openai-agents-python](https://github.com/openai/openai-agents-python) | +|---|---|---| +| 出品 | Anthropic 官方 | OpenAI 官方 | +| 模型 | Claude（Opus / Sonnet / Haiku） | OpenAI 系列 + 其他 | +| 强项 | 跟 Claude Code 一致的 tool / skill / hook 抽象 | handoff / agents-as-tools 模式、2026-04 内建 sandbox | +| 适合 | 已在用 Claude Code 想嵌服务的人 | 已 commit OpenAI 生态的人 | + +两个都 MIT 授权、API 设计干净，**重点是你的下游选哪家模型**。 + +### 接下来 + +- **看代码**：回 5.7，读 `claude-agent-sdk-python` 的 `_internal/client.py` —— 你现在会用 SDK 了，读那边的 main loop 会看懂更多 +- **动手练 SDK 进阶**：Stage 7 练习 4（streaming + prompt caching）；Stage 7 练习 5（FastAPI + Docker production deploy） +- **如果你发现你其实不需要 SDK**：那很好 —— 回 5.1-5.4，把 CLI + 自定这层用透，通常已经比写 SDK 划算 + +> 💡 **本节跟 Stage 7 的区别**：本节学“SDK 是什么、什么时候用”（定位 + 入门）；Stage 7 学“用 SDK 写一个可上线部署的 agent 服务”（streaming / caching / deploy）。 + +--- + +## ✅ 进入 Stage 6 前的自我检查 + +你能不能： + +- [ ] 安装 Claude Code 并使用 5 个不同的 slash command +- [ ] 在同一个 Claude session 里接 2 个 MCP server +- [ ] 用 Python 写自己的 MCP server，提供 1 个能用的 tool +- [ ] 写一份能在特定触发词自动加载的 `SKILL.md` +- [ ] 把 skill 打包成 plugin，再用 `marketplace.json` 发布 +- [ ] **写过 `.claude/agents/` 自定义 subagent 并从 Task tool invoke 过** +- [ ] **读过 `claude-agent-sdk-python` 的 main loop、能在 source 里标出 [Stage 7 列的 8 个 harness 元件](07-multi-agent-production.zh-Hans.md#-harness-engineering--production-agent-runtime-的工程设计--本-stage-核心概念) 的前 6 个 runtime-internal 元件**位置（5.7 练习） +- [ ] 从角色分工说出 MCP / Skills / Plugins / Subagents / SDK 各自的位置 + +如果都可以 → 前往 [Stage 6 — Memory & RAG](06-memory-rag.zh-Hans.md)。 + +> 💡 **Stage 5 是两 track 第一个 hub**——Track A 跟 Track B 都会用到。第二个 hub 是 [**Stage 8 — Agent Interfaces**](08-agent-interfaces.zh-Hans.md)（Computer Use / Browser Use / Sandbox），可以走完主干后再进、或对 Computer Use / Browser MCP 有兴趣可以提前 preview。 + +## 💡 Bonus：完成这个阶段之后 + +- 对 [`anthropics/claude-cookbooks`](https://github.com/anthropics/claude-cookbooks) 发一个 PR（小修正、文件更新） +- 把自己的 plugin 投稿到社区 marketplace +- 写一篇文章，比较自己的 hello-MCP server 跟官方 `modelcontextprotocol/servers` 收的某一个 diff --git a/stages/06-memory-rag.en.md b/stages/06-memory-rag.en.md new file mode 100644 index 0000000..7a5e05a --- /dev/null +++ b/stages/06-memory-rag.en.md @@ -0,0 +1,750 @@ +# Stage 6 — Context Engineering: RAG and Memory + +> [繁體中文](./06-memory-rag.md) | [简体中文](./06-memory-rag.zh-Hans.md) | **English** + +⏱ **Estimated Time**: 2 weeks (approx. 10 hours) + +> 💡 This stage is dense with terminology (**RAG / vector databases / embedding / chunking / hybrid search / reranking...**) — if unfamiliar, first consult [`resources/glossary.md` 3](../resources/glossary.md#3-memory--retrieval--rag). +> +> 📋 **Chapter Structure**: Positioning → Entry Point → **RAG Core** (Basics + Advanced + DSPy + Eval) → **Bridge** → **Memory Core** (3 patterns + trio + advanced) → Chunking → Reflexion / Reasoning → Practice → Projects +> +> 🔑 **Key Terms**: See [`resources/glossary.md` 3](../resources/glossary.md#3-memory--retrieval--rag) (memory / RAG / embedding / chunking / reranking) + +This stage is not about memorizing more terminology. It is about understanding how agents manage context. + +- **RAG answers**: what data should be retrieved from an external knowledge base right now? +- **Memory answers**: what should an agent remember across conversations, sessions, and tasks? +- **Context Engineering is the higher-level question**: before each LLM call, what information should be assembled into the prompt so the model can make the right decision inside a limited context window? + +→ This connects directly to Stage 7's three engineering layers: **Prompt = how to ask on this call / Context = what information to include on this call / Harness = how the whole agent system runs**. This stage is the middle layer. + +### The two context capabilities an agent needs + +1. **Retrieval**: pulling task-relevant information from an external knowledge base. +2. **Memory**: preserving state, preferences, and experience across conversations, sessions, and tasks. + +**RAG (Retrieval-Augmented Generation)** is the most common retrieval architecture today. **Memory** is what lets an agent remember the user, task history, and its own past experience. This chapter treats them separately so "looking things up" and "remembering things" do not get mixed together. + +### Separate the terms first: Retrieval / RAG / Vector Store / Memory are not the same thing + +| Term | Do not confuse it with | Plain-language explanation | +|---|---|---| +| **Retrieval** | All of RAG | The act of finding information | +| **RAG** | Vector DB | The full retrieve + generate workflow | +| **Embedding** | Memory | Turning text into vectors so similarity search becomes possible | +| **Vector store** | RAG | The place where embeddings are stored and searched | +| **Chunking** | Retrieval itself | Splitting documents into searchable pieces | +| **Memory** | RAG | Persistent management of user, task, and experience data | + +## 🎯 What is Context Engineering? (Positioning) + +**In one sentence**: Context Engineering = deciding **what information to put into the window the LLM can see on each call**. + +The point is not "how many conversations you opened." The point is "**what you put into each one**." Karpathy's June 2025 [tweet](https://x.com/karpathy/status/1937902205765607626) puts it best: the delicate art of putting **just the information useful for the next step** into the window. + +📺 **Visual Learning**: [Hung-Yi Lee 2025 Lecture 2 — Context Engineering: The Key Technology Behind AI Agents](https://www.youtube.com/watch?v=lVdajtNpaGI) (NTU Introduction to Generative AI & Machine Learning 2025) + +### Where it sits in the three-layer stack + +![Prompt → Context → Harness engineering stack](../resources/diagrams/prompt-context-harness-stack.en.png) + +See [Stage 2](02-prompt-engineering.en.md) for the full comparison. + +### This stage covers 2 of the 4 sub-problems (Lance Martin 2025 framing) + +| Sub-problem | What it solves | Concrete example | Covered in this stage? | +|---|---|---|---| +| **Select** | Which external information should be pulled into the window | User asks "Which cafe near me is good?" → pull 3 highly rated places from a Yelp DB → put them into the prompt | ✅ Core theme (RAG / vector search / GraphRAG) | +| **Write** | Which interactions / lessons should be written into long-term memory | User said last week "I eat vegan" → write it to memory; when they ask for restaurant suggestions again, retrieve it so you do not recommend meat | ✅ Core theme (memory layers) | +| **Compress** | How to compress an overlong conversation | 50 turns exceed 200k tokens → auto-summarize the first 40 turns, keep the last 10 turns verbatim | ⚠️ Partial (here + Stage 7 Harness `context manager`) | +| **Isolate** | How to split windows across multiple agents | The supervisor sees the whole picture, workers only see their own slice, and they do not interfere with one another | ❌ Covered in Stage 7 multi-agent | + +### Four concepts commonly mixed up + +| Term | What it is (abstract / concrete) | Example tools | +|---|---|---| +| **Memory** | An agent's **capability** to remember things across conversations / sessions (abstract concept) | LangChain ConversationBufferMemory / mem0 / Letta | +| **Embedding** | Turning text into an N-dimensional **vector** so similarity becomes computable (data transformation) | `sentence-transformers` producing 768-dim vectors / OpenAI ada-002 | +| **Vector DB** | The **storage layer** for storing + querying embeddings (infrastructure) | Chroma / Qdrant / Weaviate / pgvector | +| **RAG** | The **architectural pattern** of "retrieve relevant snippets → insert into prompt → generate" | LlamaIndex / LangChain RAG chain | + +→ **Core distinction**: Memory is a **capability**, Embedding is **data transformation**, Vector DB is **storage**, and RAG is an **architectural pattern**. These four are often confused, but they belong to different layers. + +### RAG vs Long Context vs Fine-tuning — when to use what + +LLMs can use your private / domain data in three main ways. **This stage teaches RAG**, but you should know when not to use it: + +| Option | Suitable for | Not suitable for | Cost | +|---|---|---|---| +| **RAG**
(external retrieve) | Large / dynamic / private knowledge bases, citation-heavy use cases | Tasks requiring full-text reasoning, cross-document multi-hop reasoning | One extra vector-search latency per query | +| **Long Context**
(directly in prompt) | Medium-sized documents under 200k tokens, one-off queries, cross-document reasoning | Large / frequently changing knowledge bases, citation-heavy use cases | High input-token usage per query, even with prompt caching | +| **Fine-tuning**
(modifying model weights) | Consistent style / format, specific domain language (medical, legal, code) | Knowledge that changes, use cases requiring citations, cases where you do not want to train a model | Training + maintenance + model lock-in costs | + +→ **How to choose**: start with RAG (lowest cost, easiest to change) → if RAG is insufficient, consider Long Context → if both fail, consider Fine-tuning. **Proceed to Stage 7 to learn fine-tuning deployment.** + +## 📌 Learning Objectives + +- Build a basic RAG pipeline (chunk → embed → store → retrieve → generate) +- Identify where RAG should and should not be used +- Differentiate between working memory, long-term memory, episodic memory, semantic memory, and procedural memory +- Understand vector embeddings and similarity search +- Know when to add advanced RAG techniques (GraphRAG / Contextual Retrieval / Hybrid Search) + +## 🚪 Prerequisites + +You should have already: +- Completed Stage 3 (ability to write tool use, call LLM APIs, understand ReAct loop) — **hard technical prerequisite** +- Walked through Stage 4 (agent frameworks) + Stage 5 (Claude Code ecosystem) — the curriculum main line is **3 → 4 → 5 → 6** (see the [README learning map](../README.en.md#-learning-map-two-tracks)); not a hard technical prerequisite, but RAG / memory often pairs with frameworks + Claude Code memory mechanisms so following the sequence gives a more complete understanding, and [Stage 7](07-multi-agent-production.md) expects you to have completed 4 + 5 + 6 +- Be able to run Python `pip install` to install SDKs (will use `chromadb`, `sentence-transformers`, etc. later) +- Be comfortable with basic Python structures like lists, dicts, and generators. + +If not, refer back to [Stage 3](03-tool-use-and-hello-agent.md) or [Stage 0 Setup Guide](00-foundations.en.md#when-to-skip-this-stage). + +## 📚 Required Reading + +1. [**LlamaIndex — RAG Concepts**](https://docs.llamaindex.ai/en/stable/getting_started/concepts/) — The clearest introduction. +2. [**LangChain — RAG Tutorial**](https://python.langchain.com/docs/tutorials/rag/) — Hands-on implementation. +3. [**Pinecone — Learning Center**](https://www.pinecone.io/learn/) — Vector DB fundamentals. +4. [**Anthropic — Contextual Retrieval**](https://www.anthropic.com/news/contextual-retrieval) — Anthropic's approach to RAG with prompt caching. +5. [**LangChain — Text Splitters**](https://docs.langchain.com/oss/python/integrations/splitters/index) — Introduction to chunking strategies. + +> 🙏 **Special Recommendation for the Memory Chapter**: Refer to [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents) — This stage covers the concepts and basic implementations of memory. For a **chapter-length in-depth guide**, consult the corresponding chapter in hello-agents, which provides the most comprehensive explanation of short-term vs. long-term memory differences, dynamic prompt assembly with context engineering, session persistence, and forgetting strategies. This stage serves as a roadmap, while that is a deep dive textbook. + +## 🧭 Unit Guide (Progressive Flow) + +This chapter proceeds by first teaching **RAG**, then **Memory**, as RAG is the most fundamental and commonly used tool for context engineering, while Memory enables agent capabilities across conversations/sessions. We'll start by getting the RAG pipeline running, then introduce Memory design, and finally revisit Chunking details. + +**Recommended Reading Order**: + +1. **🌐 Basic RAG Pipeline** (Next Section) — Establish the mental model. +2. **🚀 Advanced RAG Techniques** — GraphRAG / Contextual Retrieval / Hybrid Search, etc. for production upgrades. +3. **🌉 From RAG to Memory** — Why RAG isn't enough and where Memory fills the gaps. +4. **🧠 Memory Design** — Short-term vs. Long-term, 3 patterns, CoALA framework. +5. **🧩 Chunking Details** — In-depth look at techniques used in both RAG and Memory. + +As you read this chapter, consider: In which application scenarios is RAG unsuitable? Which scenarios are suitable for RAG but not well-served by basic RAG? This will lead you to advanced techniques like GraphRAG / Self-RAG / RAPTOR later on. + +## 🌐 Basic RAG Pipeline + +**RAG (Retrieval-Augmented Generation)** = The pattern of "retrieve relevant snippets → insert into prompt → generate". Think of it as building a library for your agent—you need to organize your books first, then you can quickly and accurately retrieve information when needed. + +The most basic RAG is divided into two pipelines: + +- **Data Preprocessing (Ingest Once)**: ingest → chunk → embed → store (index). This step builds the searchable knowledge base. +- **Retrieval & Generation (Per Query)**: retrieve → generate. This step finds relevant content when the user asks a question and feeds it to the LLM for generation. + +![RAG Pipeline Overview](../resources/diagrams/rag-pipeline-overview.jpg) + +The RAG Fusion and query rewrite techniques mentioned in the diagram fall under advanced retrieval techniques. When learning RAG for the first time, focus on understanding the main flow. + +**Understanding the 5 Steps**: + +| Step | What it does | Which Pipeline | Technical Details Found In | +|---|---|---|---| +| **1. Ingest** | Loads data (PDF / web / DB) | Preprocessing | LlamaIndex / LangChain respective loaders | +| **2. Chunk** | Splits documents into small pieces (500-2000 tokens / chunk) | Preprocessing | See 🧩 Chunking Details later (read RAG/Memory main sections first, technical details later) | +| **3. Embed** | Converts each chunk into an N-dimensional vector | Preprocessing | `sentence-transformers` / OpenAI ada-002 | +| **4. Store** | Stores vectors + metadata in a vector DB | Preprocessing | Chroma / Qdrant / pgvector | +| **5. Retrieve + Generate** | Embeds query → top-k semantic search → concatenates into prompt → LLM generates answer | Per Query | Universal LLM API | + +These are the minimal structural elements. **The 3 most common pitfalls**: + +- **Chunk size too large / too small**: If too large, retrieved chunks might contain only one relevant sentence amidst much noise; if too small, context is lost (see 🧩 Chunking Details). +- **Incorrect embedding model chosen**: Using an English model for Chinese documents halves retrieval accuracy. +- **Top-k set too high / too low**: Too low might miss relevant chunks; too high introduces noise / burns tokens. + +> 📚 **For more RAG pitfalls and solutions**: [NirDiamant/RAG_Techniques](https://github.com/NirDiamant/RAG_Techniques) ★ Large Production RAG Cookbook, includes 30+ techniques + Jupyter notebook examples. + +> 📄 **The two places RAG actually breaks (don't only tune chunking)**: (1) **parsing / ingest**: PDF → clean markdown is where garbage-in starts: [docling-project/docling](https://github.com/docling-project/docling) (★61k, MIT), [opendatalab/MinerU](https://github.com/opendatalab/MinerU) (strong on Chinese / scientific PDFs; **AGPL**, mind the license), [microsoft/markitdown](https://github.com/microsoft/markitdown) (★150k+, MIT). (2) **picking an embedding model**: your first retrieval-quality decision: check the [MTEB leaderboard](https://huggingface.co/spaces/mteb/leaderboard); for Chinese / multilingual, [BGE-M3](https://github.com/FlagOpen/FlagEmbedding) (★12k, MIT) is a common pick. + +After implementing the basic skeleton, complete Exercises 1-4 (Embeddings / Vector DB / Chunking / Full Pipeline) to gain practical experience, then move to the next section on Advanced RAG Techniques. + +## 🚀 Advanced RAG Techniques (Read After Basic RAG) + +The following six subsections represent common production RAG enhancements from 2024-2026, grouped by the stage they are added to the pipeline: +- **After Retrieve** — GraphRAG / Contextual Retrieval / Hybrid Search & Reranking +- **Before Retrieve** (Query Rewriting) — Query Transformations +- **During Retrieve** (Control Flow) — Adaptive / Agentic RAG +- **Index Structure** — RAPTOR +- **2024-2026 Overview** — 17 other techniques worth knowing + +**First, complete the basic RAG to establish a baseline version before diving into these**—otherwise, you'll be tuning parameters without a baseline, never knowing which change yielded improvement. + +| Technique | Solves What Problem | Pipeline Stage | Cost | +|---|---|---|---| +| **GraphRAG** | Vanilla RAG cannot perform multi-hop / cross-document entity-relation reasoning | Before Retrieve (build graph) + During Retrieve (graph traversal) | High (requires KG construction, significant LLM token usage for entity extraction) | +| **Contextual Retrieval** | Chunks lose original document context, retrieval fetches incorrect snippets | After Chunk / Before Embed (add contextual header) | Medium (one-time ingest cost, 90% cheaper with prompt caching) | +| **Hybrid Search & Reranking** | Pure vector misses keyword matches, top-k has noise | During Retrieve (combine with BM25) + After Retrieve (cross-encoder reranking) | Low (mature tools integrate easily) | + +### 🔗 GraphRAG — Knowledge Graph + RAG + +**Mental Model**: Vanilla RAG cuts documents into chunks and relies on embedding similarity for retrieval—but it **doesn't know which entities are the same or their relationships**. GraphRAG constructs a knowledge graph by extracting (entity, relation, entity) triples from documents during ingestion. Retrieval then uses both vector similarity and graph traversal to find related entities and their connections. + +**When to Use**: +- Tasks requiring **multi-hop reasoning** (A → B → C to answer). +- Cross-document entity referencing (company financial reports, research papers, legal cases). +- Questions like "How does X affect Y, and what is Y linked to Z?" — Vanilla RAG typically only retrieves content related to X. + +**When Not to Use**: +- Documents without entity-relation links (standalone FAQs, independent product manuals). +- Small knowledge bases (< 1k chunks) — Vanilla RAG is sufficient. +- Tight budgets — KG construction can be 10-50x more token-intensive than regular RAG. + +**Representative Frameworks**: +- [**HKUDS/LightRAG**](https://github.com/HKUDS/LightRAG) ★ **35.1k** MIT EMNLP 2025 — Currently the hottest community choice, lightweight, KG + vector hybrid, lower cost than Microsoft's version. +- [**Microsoft GraphRAG**](https://github.com/microsoft/graphrag) — Original reference implementation, Apache-2.0, includes community detection. +- [**gusye1234/nano-graphrag**](https://github.com/gusye1234/nano-graphrag) — Minimal implementation (< 1000 lines) for understanding core principles. + +**Paper**: [**From Local to Global: A Graph RAG Approach to Query-Focused Summarization (Edge et al. 2024)**](https://arxiv.org/abs/2404.16130) — Original Microsoft GraphRAG paper explaining how community summarization addresses global queries. + +### 🪶 Contextual Retrieval — Anthropic's Prompt Caching Solution + +**Mental Model**: Vanilla chunks lose original document context—a chunk like "Q3 revenue grew 15%" doesn't tell you **which company** or **which year's** Q3. Anthropic's 2024 proposal: During ingestion, use an LLM to generate a 50-100 token **contextual header** for each chunk (e.g., "This chunk is from ACME Corp 2024 Q3 earnings, discussing the cloud segment...") and prepend it before embedding. Combined with **prompt caching**, this allows sending the entire document + each chunk only once, sharing the cache for subsequent chunks. + +**When to Use**: +- Chunks are semantically distant from their original document's topic (financial reports, research papers, long narratives). +- You are willing to incur an initial ingest cost for improved retrieval accuracy. +- You are using Claude or plan to leverage prompt caching (other models can run this, but without cache benefits). + +**When Not to Use**: +- Chunks are self-contained (FAQs, product descriptions, definitions). +- Knowledge base changes frequently (requires re-ingestion). +- Extremely tight budgets — ingest cost is higher than vanilla, even with caching discounts. + +**Why it Saves 90% Cost**: Anthropic's report suggests prompt caching reduces costs to ~1/10 by treating the entire document as a cached prefix and only sending differences for each chunk. However, **this only saves on ingest, not retrieval**. + +**Representative Implementations**: +- [**Anthropic — Contextual Retrieval Blog**](https://www.anthropic.com/news/contextual-retrieval) ⭐ — Official explanation + benchmark (failed retrieval rate dropped from 5.7% to 1.9%). +- [**Anthropic Cookbook**](https://platform.claude.com/cookbook/capabilities-contextual-embeddings-guide) — End-to-end Jupyter notebook with prompt templates. + +**Paired Techniques**: The same Anthropic blog also recommends combining this with **Contextual BM25** (using contextual chunks with both vector + BM25) + **reranking**—leading into the next section on Hybrid Search & Reranking. + +### 🎯 Hybrid Search & Reranking — Two Common Reinforcement Components for Production RAG + +**Mental Model**: +- **Hybrid Search** = Combines vector similarity (semantic match) with BM25/keyword search (literal match), using methods like [RRF (Reciprocal Rank Fusion)](https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf) to fuse scores. This addresses the dual blind spots of pure vector search: missing keyword matches due to different phrasing and weak semantic embedding for proper nouns, product IDs, technical terms, or rare words. +- **Reranking** = First stage retrieves **top-50** chunks (prioritizing recall, broad fetch) → then a **cross-encoder reranker** re-scores and ranks the top **top-5** (prioritizing precision, fine-grained filtering). Cross-encoders (which process query + chunk together) are much more accurate than bi-encoders (query/chunk processed separately) but are too slow for initial retrieval, hence only used in the second stage. + +**Why These Are "Must-Add Polishes"**: Production RAG evaluations almost universally show that adding hybrid search + rerankers improves recall@5 from around 70% to 85-90% with low marginal cost and mature tooling. **These offer the best cost/benefit**. + +**When to Use**: +- Production RAG (not demos/experiments). +- Queries containing proper nouns, product IDs, technical terms, or rare words (pure vector search might miss these). +- Budget allows for an additional 100-300ms latency per query. + +**When to Defer**: +- Experimental phase / MVP (get basic RAG working first). +- Extremely tight budget / latency constraints (rerankers add an extra model call). + +**Representative Tools**: +- **Hybrid Search**: [Weaviate](https://github.com/weaviate/weaviate) (built-in BM25 + vector + RRF) / [Qdrant](https://github.com/qdrant/qdrant) (supports sparse + dense vectors) / pgvector + Postgres FTS. +- **Rerankers**: [Cohere Rerank API](https://docs.cohere.com/docs/rerank-overview) (commercial, widely used) / [BGE Reranker](https://huggingface.co/BAAI/bge-reranker-large) (open-source, HuggingFace, good performance in Chinese) / [Jina Reranker](https://jina.ai/reranker). +- **Framework Built-ins**: LlamaIndex's `SentenceTransformerRerank` / LangChain's `ContextualCompressionRetriever`. + +**Paper / Introduction**: +- [**Pinecone — Rerankers and Two-Stage Retrieval**](https://www.pinecone.com/learn/series/rag/rerankers/) — Best explanation of the reranker mental model. +- [**Anthropic — Contextual Retrieval**](https://www.anthropic.com/news/contextual-retrieval) (listed above) — Demonstrates hybrid + reranker with benchmarks. + +### Query Transformations — HyDE / Multi-Query / RAG Fusion + +**Mental Model**: Basic RAG embeds the user's query directly for retrieval—but the query's wording, style, or abstraction level often differs significantly from the document (e.g., user asks "What should I do for a stomach ache?", document describes "Differential diagnosis of upper abdominal pain"). Query transformations rewrite the query *before* retrieval, creating versions closer to how documents are phrased. + +**3 Representative Techniques**: + +| Technique | How it Rewrites | When to Use | +|---|---|---| +| **HyDE** (Hypothetical Document Embeddings) | First, let the LLM generate a "hypothetical answer" to the query; use the embedding of this answer for retrieval. | When query wording/style differs greatly from document phrasing. | +| **Multi-Query** | The LLM rewrites the query into N variations, each retrieved separately, then unioned and deduplicated. | When the query is too short, ambiguous, or has multiple meanings. | +| **RAG Fusion** | Combines Multi-Query with RRF to fuse results from N retrievals for more stable ranking. | Same as Multi-Query, aims for more stable rankings. | + +**When Not to Use**: When the query is already long and structured (e.g., RAG over code, user pastes an error stack trace)—rewriting might introduce noise. + +**Papers / Implementations**: +- [**HyDE (Gao et al. 2022)**](https://arxiv.org/abs/2212.10496) — Original paper. +- [**RAG Fusion (Raudaschl 2023)**](https://github.com/Raudaschl/rag-fusion) — Reference implementation for Multi-Query + RRF. +- LangChain includes `MultiQueryRetriever` / LlamaIndex includes `HyDEQueryTransform` built-in. + +### 🔁 Adaptive / Agentic RAG — Self-RAG / CRAG / Adaptive RAG (2024 Focus) + +**Mental Model**: All RAG techniques above assume a fixed pipeline: "query → retrieve → generate". Adaptive / agentic RAG turns this into an **agentic loop** where the LLM decides whether to retrieve, evaluates retrieval quality, and adjusts the query if necessary. **This is a major focus of RAG research in 2024.** + +| Technique | How it Self-Corrects | Paper | +|---|---|---| +| **Self-RAG** | Trains the LLM to output `[Retrieve]` tokens to decide retrieval, then outputs `[IsRel]/[IsSup]/[IsUse]` scores for each retrieved segment. | [Asai et al. ICLR 2024](https://arxiv.org/abs/2310.11511) | +| **CRAG** (Corrective RAG) | A retrieval evaluator scores results; high confidence uses them directly, low confidence falls back to web search, medium confidence triggers query rewriting. | [Yan et al. 2024](https://arxiv.org/abs/2401.15884) | +| **Adaptive RAG** | A classifier first determines query complexity, routing to strategies like "no retrieve / single-step / multi-step". | [Jeong et al. NAACL 2024](https://arxiv.org/abs/2403.14403) | + +**Why This is a 2024 Focus**: A fixed pipeline is suboptimal for both simple queries (e.g., "What is the capital of Tokyo?" doesn't need retrieval) and complex ones (multi-hop, cross-document). Letting the LLM decide routing handles both extremes effectively. + +**When to Use**: Production RAG with diverse query types (from factual to reasoning), willing to accept 1.5-3x latency for increased accuracy. +**When Not to Use**: Uniform query types / strict budget / extremely sensitive latency requirements. + +**Implementations**: LangGraph provides official cookbooks for [Self-RAG](https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_self_rag/) + [CRAG](https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_crag/) + [Adaptive RAG](https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_adaptive_rag/), which can be directly applied. + +### 🌳 RAPTOR — Hierarchical Recursive Retrieval (ICLR 2024) + +**Mental Model**: Basic chunking creates flat chunks—but the **main thesis of a long document isn't contained in any single chunk**. RAPTOR recursively clusters and summarizes chunks, building a **multi-layer tree**: bottom layer = original chunks, middle layers = summaries of related chunk groups, top layer = overall document summary. Retrieval can then search the entire tree or specific abstraction levels. + +**Why it's Useful**: +- Retrieves answers for **abstract queries** (e.g., "What is the main conclusion of this paper?"—the original chunks might not have this sentence, but the top-level summary does). +- Retrieves **specific details** effectively (bottom chunks are preserved). +- Unlike GraphRAG—RAPTOR uses a **tree** (hierarchical summarization), while GraphRAG uses a **graph** (entity-relation). + +**When to Use**: Long documents (books, papers, reports) requiring queries at different levels of abstraction; coherent knowledge bases. +**When Not to Use**: Independent chunks (FAQs); frequently changing knowledge bases (rebuilding the tree is costly). + +**Paper / Implementation**: +- [**RAPTOR (Sarthi et al. ICLR 2024)**](https://arxiv.org/abs/2401.18059) ⭐ — Original paper. +- [**parthsarthi03/raptor**](https://github.com/parthsarthi03/raptor) — Official reference implementation. +- LlamaIndex has built-in `RAPTOR pack`. + +### 🧬 DSPy — Programmatic Optimization Without Prompting (Path 3 Paradigm) + +**Mental Model**: Traditional RAG/Agents involve manually writing prompts and chains. DSPy **eliminates prompt writing**—you define "signatures" (input/output types) and write programs (chain structures); DSPy then compiles the optimal prompts, few-shot examples, and retriever settings using LLMs. Proposed by Stanford NLP group in 2024 and championed by Karpathy, it's increasingly adopted in production. + +**When to Use**: +- Your RAG prompts have accumulated over 6 months and are hard to maintain; you want automatic optimization. +- The same program needs to switch between different LLM providers (DSPy recompiles automatically). +- Your agent system has multiple steps; you want to track metrics and traces. + +**When Not to Use**: +- You only have one prompt and don't need optimization. +- You are new to LLMs and haven't explored prompting yet. + +**Representative Repo**: [**stanfordnlp/dspy**](https://github.com/stanfordnlp/dspy) ★ **34.4k** MIT, official Stanford NLP group, actively maintained. + +**How it Integrates with RAG**: DSPy is **compatible with all RAG techniques** discussed—you can use GraphRAG / Hybrid Search / Reranking as DSPy modules and compile them. It's an overarching typing system for RAG construction. + +→ **Parallel to Path 1 / Path 2 Reasoning**: Path 1 is "manual prompt writing", Path 2 is "training reflection into model weights", **DSPy is Path 3 "programmatic search for optimal prompts"**. It's especially useful for advanced scenarios in Stage 7 Multi-agent. + +### 📊 Overview of Advanced RAG Techniques — 2025-2026 Main Themes ⭐ + +Advanced RAG research in 2024-2025 is converging on **3 main themes**: + +1. **🧠 Merging KG + Memory** — Moving from flat vector stores to "structured, evolving, associative" knowledge representations. Representatives: [**HippoRAG 2**](https://arxiv.org/abs/2502.14802) (Hippocampus-inspired, KG + PageRank, cross-document multi-hop), A-MEM, KAG. +2. **🎬 Multimodal RAG** — Moving from text retrieval to native image / video / table retrieval. Representatives: [**ColPali**](https://arxiv.org/abs/2407.01449) (direct image embedding from PDF pages, bypassing OCR), TV-RAG, MegaRAG. +3. **🤖 Agentic RAG** — Retrieval evolves from a fixed pipeline into a tool within an agent loop (the agent decides how many times and how to retrieve). Representatives: A-RAG, Self-RAG (covered in Adaptive / Agentic RAG). + +**2 Other Areas Worth Exploring**: +- **🛡 RAG Security** — Corpus poisoning / prompt injection become critical in production considerations. Representatives: [RAGPart / RAGMask](https://arxiv.org/abs/2512.24268). +- **🔧 Prompting is Dead** — Systems automatically search for optimal prompt + retriever combinations. Representatives: [**DSPy**](https://github.com/stanfordnlp/dspy) (Stanford's "programming not prompting" paradigm, see DSPy subsection above). + +**5 Representative Works for Deep Dive** (quick reference): + +| Technique | One-Liner | Link | +|---|---|---| +| **HippoRAG 2** | KG + Personalized PageRank, cross-document multi-hop, hippocampus-inspired | [Gutiérrez et al. ICML 2025](https://arxiv.org/abs/2502.14802), [OSU-NLP-Group/HippoRAG](https://github.com/OSU-NLP-Group/HippoRAG) ⭐ | +| **ColPali** | Direct image embedding from PDFs, bypassing OCR, multimodal RAG entry point | [Faysse et al. 2024](https://arxiv.org/abs/2407.01449) | +| **A-RAG / SoK Agentic RAG** | Retrieval as a tool, agent decides retrieval frequency/method | [Ayanami0730/arag](https://github.com/Ayanami0730/arag), [SoK survey](https://arxiv.org/abs/2603.07379) ⭐ | +| **DSPy** | No prompt writing, program + signature for auto-optimization | [stanfordnlp/dspy](https://github.com/stanfordnlp/dspy) ★ 34.4k | +| **LightRAG** | Lightweight alternative to MS GraphRAG, EMNLP 2025 | [HKUDS/LightRAG](https://github.com/HKUDS/LightRAG) ★ 35.1k (already in GraphRAG section) | + +
+📚 Full Overview — 12 Other Advanced RAG Techniques Worth Knowing (Expand to View) + +| Technique | One-Liner | Year / Paper | +|---|---|---| +| **Sentence-Window Retrieval** | Embed sentences, retrieve +/- N sentence window | Built into LlamaIndex | +| **Parent-Child / Small-to-Big** | Embed small chunks, retrieve parent chunks | LangChain `ParentDocumentRetriever` | +| **Multi-Vector Retrieval** | One chunk, multiple embeddings (summary / original / hypothetical question) | LangChain `MultiVectorRetriever` | +| **ColBERT / Post-Interaction Retrieval** | Token-level comparison instead of pooled embeddings | [Khattab & Zaharia 2020](https://arxiv.org/abs/2004.12832), [RAGatouille](https://github.com/AnswerDotAI/RAGatouille) | +| **LongRAG** | Large chunks (4k) + long-context reader, reduces retrieval frequency | [Jiang et al. 2024](https://arxiv.org/abs/2406.15319) | +| **MemoRAG** | Memory model compresses KB into latent memory, retrieval triggered by cues | [Qian et al. 2024](https://arxiv.org/abs/2409.05591) | +| **KAG** (Knowledge-Augmented Generation) | Strict schema KG + logical reasoning, finance / medical / legal domains | [Liang et al. 2024 (Ant Group)](https://arxiv.org/abs/2409.13731) | +| **MiA-RAG** (Mindscape-Aware) | First build high-level document summaries (mindscape) to guide retrieval and answers | [arXiv:2512.17220](https://arxiv.org/abs/2512.17220) ⭐ 2025-12 | +| **QuCo-RAG** (Quality-Controlled) | Uses pretraining statistics to determine if retrieval is needed; rare entity triggers search, reducing hallucination | [arXiv:2512.19134](https://arxiv.org/abs/2512.19134) ⭐ 2025-12 | +| **MegaRAG** | Multimodal KG, extracts entities + relations + visuals from long documents, builds hierarchical graphs | [arXiv:2512.20626](https://arxiv.org/abs/2512.20626) ⭐ 2025-12 | +| **TV-RAG** | Training-free time-aware RAG, aligning long videos with subtitles + visuals | [arXiv:2512.23483](https://arxiv.org/abs/2512.23483) ⭐ 2025-12 | +| **RAGPart / RAGMask** | Lightweight defense against RAG corpus poisoning attacks | [arXiv:2512.24268](https://arxiv.org/abs/2512.24268) ⭐ 2025-12 | + +
+ +## 🌉 From RAG to Memory — Why RAG Isn't Enough + +By now, you should be able to run basic RAG and understand several production levers. However, looking back at the 3 problem domains listed in Context Engineering—you've only addressed **Retrieval**, and haven't touched **Memory Management**. Why are these treated separately? + +RAG addresses "retrieve relevant snippets from **external knowledge bases**"—but agents also need to "remember things **themselves** across conversations / sessions". These are not the same problem: + +| Dimension | RAG | Memory | +|---|---|---| +| Content Source | **External** (PDFs / documents / web / DB) | Agent's **own conversations / experiences** | +| Writing Time | Ingested once, retrieved repeatedly | Written every turn, potentially at every task | +| Content Nature | Primarily static facts, documentary knowledge | Dynamic: user preferences, past interactions, accumulated lessons | +| Can it Replace RAG? | — | No—you wouldn't treat every PDF as "memory" | +| Can it Be Replaced by RAG? | — | No—RAG won't "remember what the user said last time" | + +**3 Scenarios Where RAG Is Insufficient** (Corresponding to Memory's Role): + +1. **Remembering User Preferences / Persona Across Sessions**—User told the agent "I'm vegan" last week; this week, the agent remembers not to recommend meat dishes. RAG knowledge bases don't update this automatically. +2. **Accumulating Agent's Past Success/Failure Lessons** (Reflexion's domain)—An agent fails a task the first time, reflects on "why it failed," stores this, and retrieves it on similar future tasks to avoid repeating mistakes. RAG knowledge bases don't "remember its own failures." +3. **Intermediate States in Long-Horizon Tasks**—An agent running a 100-step task needs to retain working memory without loss. RAG is not suitable for this type of "short-term + structured + high-frequency writing" state. + +→ **Conclusion**: RAG and Memory are **complementary**, not mutually exclusive. Production agents typically need **both**: RAG for external knowledge, Memory for self-reflection and user interaction history. The next section, Memory Design, will guide you in choosing the right memory pattern. + +## 🧠 What is Memory + How to Design It + +> 📺 **Visual Learning**: [Hung-Yi Lee 2025 Lecture 2 — Understanding AI Agent Principles in One Lecture (Includes Read / Write / Reflection memory modules)](https://www.youtube.com/watch?v=M2Yg1kwPpts) (NTU Machine Learning in the Era of Generative AI 2025) + +### Working memory vs. long-term memory — two time scales + +| Aspect | Working memory / short-term context | Long-term memory / persistent memory | +|---|---|---| +| **Chinese term** | 工作记忆 / 短期上下文 | 长期记忆 / 持久记忆 | +| **Core meaning** | Information visible during this task or this conversation | Information stored externally and retrievable across sessions later | +| **Duration** | Short, usually limited to the current session | Long, can span sessions | +| **Technical basis** | Context window / prompt | Memory store / user profile / vector database | +| **Best for** | Task details, what was just said | Stable preferences, long-term goals, background knowledge | +| **Limited by context length?** | Yes, because the model can only see a limited amount at once | Much less, because it can be stored externally and only a relevant slice gets retrieved | +| **Real-world analogy** | A verification code you just received, the previous sentence in an active conversation | Knowledge you have deeply learned, a library, a knowledge base, books you have read | + +→ In agents, "short-term memory" is more precisely **working memory**. It is not external storage; it is whatever is currently visible inside the prompt / context window. + +### Episodic / Semantic / Procedural memory — three content types + +**Important**: working / long-term is a **time axis**. The three categories below are a **content axis**. The two classifications are **orthogonal, not mutually exclusive**. Long-term memory can contain episodic + semantic + procedural memory at the same time. + +| Type | Meaning | Core idea | +|---|---|---| +| **Episodic memory** | Experience memory | Concrete past tasks, interactions, or failures | +| **Semantic memory** | Fact memory | Stable knowledge, user preferences, background facts | +| **Procedural memory** | Skill memory | Rules, tools, workflows, and skills for how the agent acts | + +→ These three types map to the [CoALA framework](#advanced-coala-framework--a-4-layer-taxonomy-for-agent-memory). **Reflexion** is a classic episodic-memory pattern because it accumulates success / failure lessons from prior trials. + +Here, a "session" can be understood as one continuous interaction: a chat, a task run, or a single agent execution. + +### 3 design patterns (when to use what) ⭐ Essential for Track B + +**Not all agents need an external memory store. Choosing the wrong memory architecture can cost 10x more tokens for the same result.** + +This is the mental model to establish before starting the exercises. The exercises below focus on Pattern 3 (vector store), but production systems may not need that much complexity. + +| Pattern | Suitable scenarios | How it works | Cost | +|---|---|---|---| +| **1. Naive buffer**
(stuff everything into context) | Short conversations, ≤ 10 turns, no cross-session memory requirement | Send the entire history into the prompt every time | Grows linearly and burns tokens quickly | +| **2. Summary + recent**
(summarize old parts + keep the last N turns) | Medium-to-long conversations, ~50 turns, want compression without losing too much | Every N turns, ask the LLM to summarize older history; prompt = `summary + last N turns` | Moderate, with extra summarization cost | +| **3. Vector store + retrieval**
(external store + semantic search each turn) | Cross-session interaction, knowledge-base scenarios, agents that need to "recall" distant information | Embed past messages → store them in a vector DB → retrieve relevant snippets per query and put them back into the prompt | High (vector compute + storage), but token usage stays stable | + +**How to choose**: + +- Conversational chatbots without cross-session needs → **Pattern 1** +- Agents with long conversations that need to remember what was discussed today → **Pattern 2** +- Agents with cross-session needs + a knowledge base (the common scenario in this stage's exercises) → **Pattern 3** +- Large production agents → usually **a hybrid**: recent history uses Pattern 1/2, long-term memory uses Pattern 3 + +> 💡 **Track B focus**: in Stage 7 multi-agent systems, each agent usually has "its own memory" + "shared memory". In practice that means a **Pattern 2 + Pattern 3 hybrid**. If you internalize these three patterns now, Stage 7 memory design becomes much easier. + +### ⭐ 5 mainstream memory layers that can ship (choose by use case) + +> Star counts and benchmarks change. The point here is not ranking. The point is understanding the design orientation of each memory layer. + +After learning the three patterns, you do not need to build a memory store from scratch in production. These five are all actively maintained Apache-2.0 / MIT options, each with a different strength: + +| Framework | Stars | License | Primary use case | Key features | +|---|---|---|---|---| +| [**agentmemory**](https://github.com/rohitg00/agentmemory) | 7.7k★ | Apache-2.0 | **Coding-agent cross-session memory** | MCP-universal (Claude Code / Cursor / Gemini CLI / Codex / Hermes / OpenClaw), 95.2% R@5, 92% token saving, 51 MCP tools + 12 auto hooks, benchmark-driven | +| [**mem0**](https://github.com/mem0ai/mem0) | 55.6k★ | Apache-2.0 | **Chatbot / personal-assistant user-level memory** | Auto fact extraction + forgetting + namespace, production-tested, largest community | +| [**Letta**](https://github.com/letta-ai/letta) (formerly MemGPT) | 22.7k★ | Apache-2.0 | **Long-session agents** (measured in months) | OS-style paging memory (working + archival), persona stability, MemGPT paper lineage | +| [**Zep**](https://github.com/getzep/zep) | 4.6k★ | Apache-2.0 | **Temporal KG-based memory** | Builds conversation history into a temporal KG for time-aware reasoning and audit trails | +| [**graphiti**](https://github.com/getzep/graphiti) | 27.5k★ | Apache-2.0 | **Real-time knowledge-graph agent memory** | Turns an agent's past interactions into a time-aware knowledge graph it can look things up in; the engine behind Zep, usable on its own | +| [**LangMem**](https://github.com/langchain-ai/langmem) | 1.4k★ | MIT | **LangChain-native memory** | Official LangChain memory library, integrates directly with LangGraph, useful when you are already committed to the LangChain stack | + +**How to choose**: +- Building coding agents → **agentmemory** (MCP-native, aligned with the Stage 5 ecosystem) +- Building chatbots / personal assistants → **mem0** (most mature, largest community) +- Building long-running agents across weeks or months → **Letta** (strong OS-paging model) +- Need time-aware reasoning + audit trails → **Zep** (temporal KG) +- Already committed to the LangChain stack → **LangMem** (avoid framework hopping) + +**Additional official docs**: [Anthropic Memory Tool](https://docs.claude.com/en/docs/agents-and-tools/tool-use/memory-tool) (Claude's official tool-based memory, file-based, direct API calls), [LangChain Memory Concepts](https://python.langchain.com/docs/concepts/memory/) (comparisons of memory classes within the framework). + +### Advanced: CoALA Framework — A 4-Layer Taxonomy for Agent Memory + +[**Sumers et al. 2023 — Cognitive Architectures for Language Agents**](https://arxiv.org/abs/2309.02427) categorizes agent memory into four types. This is one of the most useful mental models in practice: + +| Type | What it stores | Corresponding example | +|---|---|---| +| **Working memory** | Current task context | The LLM context window itself | +| **Episodic memory** | Specific experiences from past tasks | Reflexion records, prior trajectories | +| **Semantic memory** | Abstract facts / knowledge | RAG knowledge bases, user profiles, preferences | +| **Procedural memory** | How to perform actions / skills | Tool definitions, [Skills (Stage 5.3)](05-claude-code-ecosystem.md#53--skillsclaude-code-的行為層-claude-code-生態最關鍵的一層) | + +→ **Why it is useful**: the three patterns above (buffer / summary / vector) mostly handle working + episodic memory. Production agents usually need to account for all four layers. CoALA is a practical checklist for spotting which layer your agent is missing. + +### Advanced: Generative Agents — Triple Score Weighting (Classic Case Study) + +The [**Park et al. 2023 — Generative Agents: Smallville**](https://arxiv.org/abs/2304.03442) simulation featured 25 NPCs, each with its own memory stream. Retrieval used a weighted combination of three scores: + +- **Importance**: LLM assigns a 1-10 importance score to each memory (eating = 2, breakup = 9). +- **Recency**: Exponential decay based on time. +- **Relevance**: Embedding similarity to the current query. + +The final score = `α·importance + β·recency + γ·relevance`, ranked to retrieve top-k. **This is the conceptual backbone used by many 2024-2025 production memory layer systems (mem0 / Letta).** + +> 💻 **Official Code**: [joonspk-research/generative_agents](https://github.com/joonspk-research/generative_agents) ★ The paper's accompanying Smallville simulation code repository. Refer here for implementing memory streams and triple-score retrieval. + +### 2024-2026 Latest Memory Works — 3 Main Themes + +Memory research in 2024-2026 is focusing on **3 main themes**: + +1. **🧠 Structured, Evolving, Associative Memory** — Moving beyond flat vector stores to human-brain / Zettelkasten-inspired memory structures. Representatives: [**A-MEM**](https://arxiv.org/abs/2502.12110) (automatic linking between memories), [**HippoRAG 2**](https://arxiv.org/abs/2502.14802) (KG + PageRank, hippocampus-inspired). +2. **📚 Explosive Growth in 2026 Surveys** — Five major surveys and cross-disciplinary syntheses in one year. Representatives: [**Memory in the Age of AI Agents**](https://arxiv.org/abs/2512.13564) (3D taxonomy + benchmarks), [**Memory for Autonomous LLM Agents**](https://arxiv.org/abs/2603.07670) (formalizing the write-manage-read loop). +3. **🛡 Memory Security Emerges as a Subfield** — As agents run longer, memory becomes vulnerable to cross-session poisoning / unauthorized access attacks. Representatives: [**Memory Security Survey**](https://arxiv.org/abs/2604.16548) (covered in Stage 7 Security). + +**4 Representative Works for Deep Dive**: + +| Work | One-Liner | Link | +|---|---|---| +| **Anthropic Memory Tool** | Claude's official tool-based memory, API calls, file-based | [Anthropic Docs](https://docs.claude.com/en/docs/agents-and-tools/tool-use/memory-tool) | +| **A-MEM** (Agentic Memory) | Zettelkasten-inspired, auto-links memories, evolving | [Xu et al. 2025](https://arxiv.org/abs/2502.12110) ⭐ | +| **HippoRAG 2** | KG + Personalized PageRank, cross-document multi-hop, ICML 2025 | [Gutiérrez et al. 2025](https://arxiv.org/abs/2502.14802), [OSU-NLP-Group/HippoRAG](https://github.com/OSU-NLP-Group/HippoRAG) ⭐ | +| **Memory in the Age of AI Agents** (Survey) | 3D taxonomy (temporal / substrate / control) + benchmark compilation | [Hu et al. 2025-12](https://arxiv.org/abs/2512.13564) ⭐ | + +
+📚 Full Overview — 8 Other Memory Works Worth Knowing (Expand to View) + +| Work | One-Liner | Year / Paper | +|---|---|---| +| **MemGPT → Letta GA** | OS-paging memory, working/archival layers, strong for long sessions | [Packer et al. 2023](https://arxiv.org/abs/2310.08560) → Letta GA | +| **MemoryBank** | Ebbinghaus forgetting curve, accessed memories strengthened, unused ones decay | [Zhong et al. 2023](https://arxiv.org/abs/2305.10250) | +| **MemoryLLM** | Self-updatable memory parameters embedded within the model weights, not context | [Wang et al. 2024](https://arxiv.org/abs/2402.04624) | +| **mem0** (See 5 Mainstream Memory Layers) | A production memory layer with auto fact extraction + forgetting | [mem0ai/mem0](https://github.com/mem0ai/mem0) | +| **Memory for Autonomous LLM Agents** (Survey) | Formalizes write-manage-read loop, covers 2022-2026 advancements | [arXiv:2603.07670](https://arxiv.org/abs/2603.07670) ⭐ 2026 | +| **From Storage to Experience** (Survey) | Evolutionary framework: Storage → Reflection → Experience stages | [arXiv:2605.06716](https://arxiv.org/abs/2605.06716) ⭐ 2026 | +| **ScrapMem** | Bio-inspired on-device memory, "Optical Forgetting" reduces resolution of old memories | [arXiv:2605.03804](https://arxiv.org/abs/2605.03804) ⭐ 2026-05 | +| **Memory Security Survey** | Risks of cross-session poisoning, unauthorized access, and organizational propagation in long-term memory | [arXiv:2604.16548](https://arxiv.org/abs/2604.16548) ⭐ 2026 | + +
+ +## 🧩 Chunking Details (Technical Deep Dive) + +Good chunking allows LLMs to generate more precise and complete answers within limited contexts. It's not just about splitting text evenly; it depends on the application scenario and document content. It determines the smallest semantic unit the retriever sees. + +A good chunk should achieve two things: **sufficient completeness** for the model to understand context, and **focused content** to minimize retrieval noise. Chunks that are too small lose context, while chunks that are too large can dilute similarity search effectiveness. + +**Common Strategies**: + +- **Fixed-Length**: Splits based on character or token count. Simple and stable, but can cut sentences, paragraphs, or tables awkwardly. +- **Sliding Window**: Overlaps chunks to retain information at boundaries. Increases index size but reduces boundary information loss. +- **Recursive**: Tries to preserve paragraphs first; if still too long, falls back to sentences, then words. Often a good baseline for RAG entry-level. +- **Semantic Chunking**: Splits based on embedding similarity or semantic shifts, where similarity between consecutive chunks changes. Suitable for long documents but more complex and costly. +- **Hybrid Strategies**: Combines different methods based on document structure and application needs. For example, a research paper might need to preserve chapter context, tables, formulas, and citation references. + +![Chunking Strategy Flowchart](../resources/diagrams/chunking-strategies.jpg) + +> 📚 **Classic Tutorial**: [Greg Kamradt — 5 Levels of Text Splitting](https://github.com/FullStackRetrieval-com/RetrievalTutorials) ★ Essential reading for chunking basics, covering everything from character-based to agentic chunking across five levels, including Jupyter notebooks. + +For your first RAG implementation, avoid overly complex strategies. LangChain documentation recommends starting with `RecursiveCharacterTextSplitter` for most scenarios. + +Run a baseline version first, then adjust chunking strategies based on retrieval results and failure cases. + +```python +from langchain_text_splitters import RecursiveCharacterTextSplitter + +text = "This is a very long document content... (omitting a thousand words here)..." + +splitter = RecursiveCharacterTextSplitter( + chunk_size=100, + chunk_overlap=20, + length_function=len, +) + +chunks = splitter.split_text(text) +print(f"Split into {len(chunks)} chunks") +print(chunks[0]) +``` + +**Intuitive Assessment of Chunking Quality**: + +- Answers missing information or are incomplete: Usually due to small chunks or insufficient overlap. +- Answers contain correct information but also irrelevant details: Usually due to large chunks or too many retrieved items (high top-k). + +**Advanced Considerations**: + +- Chunking is not a one-time setup; it requires iterative refinement based on real queries and failure cases. +- Chunk size, overlap, top-k, and reranker interact; don't adjust one parameter in isolation. +- Consider how you would chunk PDFs with images, meeting transcripts with timestamps, or structured data like tables. +- Advanced chunking variations (Sentence-Window / Parent-Child / Multi-Vector) are covered in the Overview of Advanced RAG Techniques table. + +## 🪞 Advanced: Full Reflexion with Persistent Memory ⭐ Track B Elective + +> **This section covers concepts and routing; it's not a practice exercise.** It expands on the basic Reflexion from [Stage 3 Reflection](03-tool-use-and-hello-agent.en.md#-reflection-reflexion--self-refine--concept--routing) by explaining why some reflections require persistent memory—this version truly belongs in Stage 6. + +**Difference Between Full Reflexion and Self-Refine**: + +| Version | Retains within Session | Retains Across Sessions | Memory Pattern Required | +|---|---|---|---| +| **Self-Refine** (Madaan 2023) | Previous round's answer + critic feedback | ❌ Not retained | None required (Pattern 1 buffer is sufficient) | +| **Full Reflexion** (Shinn 2023) | Same as above | ✅ Stores "reflection summaries" from past trials into episodic memory, retrieved into prompts for future similar tasks | **Required** (Pattern 3 vector store or Pattern 2 summary) | + +**Why This Version Requires Memory**: The core of Reflexion's verbal reinforcement learning is "agent accumulates lessons across trials"—the agent attempts a task, fails, reflects on "why it failed" and stores it, then retrieves past reflections when encountering similar tasks to avoid repeating mistakes. This requires **persistent episodic memory**, directly connecting to the 3 memory patterns discussed earlier in this stage. + +**Typical Architecture** (Full Persistent Memory Version): + +![Reflexion persistent episodic memory loop](../resources/diagrams/reflexion-persistent-memory-loop.en.png) + +→ **Difference from Stage 3 Reflection**: Stage 3 focuses on an **in-context loop within a single session** (no external store). This section covers **persistent episodic memory storage + retrieval across trials** to learn from past experiences. + +### 📚 Want to Implement / Dive Deeper + +**Papers**: +- [**Reflexion (Shinn et al. 2023)**](https://arxiv.org/abs/2303.11366) ⭐ — The **full version** paper. Algorithm 1 outlines how the memory buffer is used. +- [**Self-Refine (Madaan et al. 2023)**](https://arxiv.org/abs/2303.17651) — Baseline comparison; version without episodic memory. + +**Reference Implementations**: +- [**noahshinn/reflexion**](https://github.com/noahshinn/reflexion) — Reference implementation by the paper's lead author (includes full episodic memory workflow). +- [**LangChain — Reflexion**](https://langchain-ai.github.io/langgraph/tutorials/reflexion/reflexion/) — LangGraph version, directly integrable with the RAG pipeline exercise in this stage. +- [**mem0**](https://github.com/mem0ai/mem0) (listed above) + [**Letta**](https://github.com/letta-ai/letta) (listed above) — Memory layers that can directly serve as episodic stores for Reflexion. + +> 💡 **Delineation with Stage 3 Reflection**: +> - To understand "how the reflection loop works and runs in a single turn" → Stage 3 Reflection. +> - To understand "how reflections accumulate across sessions and agents learn from past lessons" → This section. +> - To see how reflection is used in production agents (Cursor / Claude Code) → [Stage 5 5.7 Harness Internals](05-claude-code-ecosystem.md#57--claude-code-source-解剖reference-harness-implementation-track-b-必看). + +## 🤔 Advanced Reasoning / Reflection — 2024-2026 Trends ⭐ Covers Both Tracks + +Reflexion is **prompt-based reflection**—LLMs modify themselves during inference. 2024-2025 saw the emergence of a **second path**: **training reflection into the model weights** (OpenAI **o1** / DeepSeek **R1**). You should be aware of both approaches. + +### Path 1: Prompt-Based Reflection / Reasoning (Traditional Approach) + +| Technique | Core Idea | Paper | +|---|---|---| +| **Self-Consistency** | Sample N reasoning paths, take majority vote — **Simplest & Most Common** | [Wang et al. 2022](https://arxiv.org/abs/2203.11171) | +| **Tree of Thoughts (ToT)** | Reasoning becomes a tree, allowing branching and backtracking; suitable for puzzles / planning. | [Yao et al. 2023](https://arxiv.org/abs/2305.10601) | +| **Graph of Thoughts (GoT)** | Extends ToT beyond trees to arbitrary graph structures. | [Besta et al. 2023](https://arxiv.org/abs/2308.09687) | +| **Chain-of-Verification (CoVe)** | Generate an answer → Ask verification questions to itself → Correct the answer. | [Dhuliawala et al. 2023](https://arxiv.org/abs/2309.11495) | +| **CRITIC** | Tool-augmented self-critique (using search / calculator for verification). | [Gou et al. 2023](https://arxiv.org/abs/2305.11738) | +| **Self-Discover** | Agent first "discovers" the reasoning structure to use before executing. | [Zhou et al. ICML 2024](https://arxiv.org/abs/2402.03620) ⭐ 2024 | +| **Self-Refine / Reflexion** | Covered above / in Stage 3. | Stage 3 Reflection, this stage Reflexion | + +### Path 2: Trained-in Reasoning / Reflection (Major Shift in 2024-2026) + +> 📺 **Visual Learning**: [Hung-Yi Lee 2025 Lecture 7 — How Large Language Models Like DeepSeek-R1 Perform "Deep Thinking" (Reasoning)](https://www.youtube.com/watch?v=bJFtcwLSNxI) (NTU Machine Learning in the Era of Generative AI 2025) + +OpenAI's **o1** (Sep 2024), followed by open-source efforts like DeepSeek's **R1** (Jan 2025), **DeepSeek-V4-Pro** (Apr 2026 preview, agent-focused open-source reasoning), Claude Fable 5 (Jun 2026, Mythos-class, above the Opus class; suspended 2026-06-12, restored 2026-07-01), Claude Opus 4.8 (May 2026, Opus-class flagship, Dynamic Workflows + parallel subagents), GPT-5.5 (Apr 2026), and Gemini 3.1 Pro (Feb 2026) represent the current frontier, with Gemini 3.5 Flash arriving late Jun 2026 and GPT-5.6 (Sol / Terra / Luna) released Jul 2026. These models have "step-by-step thinking + self-correction" **trained directly into their weights**, automatically unfolding long reasoning chains (thinking tokens) during inference. **This is the biggest paradigm shift in LLMs from 2024-2026**, with all frontier models adopting this approach. The table below lists **current (Jun 2026) frontiers**—historical predecessors (o1 / R1 / Sonnet 4.5 / Gemini 2.5) are omitted; refer to release dates for lineage. + +| Model | Source / Release | Features | Link | +|---|---|---|---| +| **GPT-5.5** | OpenAI 2026-04 (Predecessors: o1 2024-09 → o3 → GPT-5 2025-08 → 5.4 2026-03) | Closed-source, unified reasoning + chat, Thinking budget API, enhanced agent capabilities. **Newer tier: GPT-5.6 (Sol / Terra / Luna), released Jul 2026, 1.05M context** | [OpenAI](https://openai.com/) | +| **Claude Fable 5** | Anthropic 2026-06 (Mythos-class, positioned above the Opus class; released alongside Claude Mythos 5, a limited-availability variant with some safeguards lifted) | Closed-source, Mythos-class (above the Opus class). Suspended 2026-06-12 by a US export-control directive; **the controls were lifted 2026-06-30 and [Fable 5 was redeployed globally 2026-07-01](https://www.anthropic.com/news/redeploying-fable-5)** (with a new safety classifier; Mythos 5 restored only for approved US organizations). Official benchmark numbers were never published | [Claude Fable 5 / Mythos 5](https://www.anthropic.com/news/claude-fable-5-mythos-5) | +| **Claude Opus 4.8** | Anthropic 2026-05 (Predecessors: Sonnet 4.5 / Opus 4.5 / Opus 4.7; Dynamic Workflows research preview) | Closed-source, Opus-class flagship (was the top usable Claude tier while Fable 5 was suspended 2026-06-12; Fable 5 restored 2026-07-01), controllable thinking budget (API parameter), **leading in SWE-bench / Terminal-bench** | [Anthropic extended thinking](https://docs.claude.com/en/docs/build-with-claude/extended-thinking) | +| **Gemini 3.1 Pro** | Google 2026-02 (Predecessors: Gemini 2.5 Thinking 2025, Gemini 3 2025-11) | Closed-source, viewable thinking traces, **GPQA Diamond 94.3%**, leading in price/speed/multimodality. **Newer tier: Gemini 3.5 Flash, available Jun 2026 (3.5 Pro in dev)** | [Gemini API](https://ai.google.dev/gemini-api/docs/thinking) | +| **DeepSeek-V4 / V4-Pro / V4-Flash** | DeepSeek 2026-04 preview (Predecessors: R1 2025-01 → V3.1) | Open-source **MIT license**, agent-focused training, integrated reasoning + tool use + knowledge processing. R series reasoning now mainline. | [HF DeepSeek-V4-Pro](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro), [R1 paper (method baseline)](https://arxiv.org/abs/2501.12948), [CNBC report](https://www.cnbc.com/2026/04/24/deepseek-v4-llm-preview-open-source-ai-competition-china.html) | +| **QwQ-32B / QvQ-72B** | Alibaba Qwen 2024-11 ~ 2026 | Open-source **Apache 2.0**, QwQ-32B remains a strong option for small-size reasoning, QvQ is the visual variant. | [QwQ blog](https://qwenlm.github.io/blog/qwq-32b-preview/) | + +### How to Choose Between the Two Paths + +| Your Situation | Recommendation | +|---|---| +| Using a general chat model base and want to add reasoning | Path 1 (Prompt-based) — ToT / Self-Consistency / CoVe | +| Budget/latency allows for strongest reasoning | Path 2 — Choose among **GPT-5.5 / Opus 4.8 / Gemini 3.5 Flash / Grok 4.3 / V4-Pro** (Claude Fable 5 restored 2026-07-01, a premium top tier) | +| Want to fine-tune your own reasoning model | Path 2 — Study the R1 paper (method baseline), start from R1-Distill / V4 open-source weights | +| On-device / Extremely tight budget | **QwQ-32B** (Apache 2.0) or R series distilled versions | +| Multi-agent debate / critic scenarios | Path 1 (CRITIC / debate) + [Stage 7 Multi-agent](07-multi-agent-production.md) | + +> 💡 **2025-2026 Trends**: +> - Reasoning models are integrating Reflexion's capabilities into their weights—however, **prompt-based reflection is not obsolete**. Agent loops (controlling reflection timing/content) and multi-agent debates remain essential. +> - **Open-source is rapidly catching up to closed-source**. DeepSeek-V4-Pro (Apr 2026 preview, MIT license) integrates R1 reasoning into its mainline, trained with an agent-first approach, narrowing the gap with GPT-5.5 / Gemini 3.5 Flash. +> - **Agent capabilities are becoming the primary selling point**. V4 / Opus 4.8 position agents-as-products (SWE-bench / Terminal-bench / tool use) as headline benchmarks, moving beyond just raw reasoning. +> - **Both paths will coexist**; production agents will likely leverage both. + +## 📏 RAG / Memory Eval — Running is Not Running Accurately + +**Why This Section is Crucial**: The biggest pitfall in RAG/Memory systems is "they seem to run, but retrieval accuracy is poor." Without evaluation, you won't know if changing chunk size, embedding model, or adding a reranker actually helps—you'll only have subjective impressions like "the answer seems smoother." Production agents without evaluation are essentially untested. + +**3 Core Metrics**: + +| Metric | Measures What | Tools | +|---|---|---| +| **Retrieval Recall@K** | Whether the ground truth answer's chunk is within the top-K retrieved chunks. | ragas / TruLens / LangSmith | +| **Answer Faithfulness** | Whether the generated answer is grounded in the retrieved chunks (vs. model hallucination). | ragas / TruLens | +| **Answer Relevance** | How relevant the answer is to the query (avoids off-topic responses). | ragas / LLM-as-judge | + +**Representative Frameworks**: + +- [**explodinggradients/ragas**](https://github.com/explodinggradients/ragas) ★ **13.9k** Apache-2.0 ⭐ — The standard tool for RAG evaluation, including 8+ metrics (faithfulness, answer relevance, context precision, context recall, etc.), supporting both reference-free and reference-based evaluation. +- [**TruLens**](https://github.com/truera/trulens) — Integrates observability and evaluation, with good LangChain / LlamaIndex support. +- [**LangSmith**](https://docs.langchain.com/langsmith) — LangChain's official evaluation + tracing platform (closed-source SaaS). + +**How to Start**: After completing Exercise 4 (Full RAG Pipeline), integrate RAGAS evaluation to measure a baseline. Then, adjust chunking/embedding/top-k and observe how the metrics change. **Without this step, parameter tuning is just guesswork.** + +→ Stage 7 Evaluation builds upon this section, extending evaluation to multi-agent systems / full harnesses. + +## 🛠 Hands-on Exercises (Illustrative Basics) + +### Exercise 1: Embeddings +Embed 100 sentences, find nearest neighbors for a query. Understand the meaning of vector distances. + +### Exercise 2: Vector DB +Store embeddings in Chroma, perform semantic queries. Compare the results against keyword search. + +### Exercise 3: Chunking Comparison +Use three chunking methods on the same document: fixed length, paragraph-based, heading-aware. Compare top-k results for 5 real questions, noting which method retrieves the correct context more easily. + +### Exercise 4: Full RAG Pipeline +Chunk a PDF → Embed → Retrieve top-k → Generate an answer. This is the basic skeleton for most RAG applications. + +### Exercise 5: Long-term Memory +Enable an agent to remember things across multiple conversation turns. Use `mem0` or build your own with a vector store. + +## 🛠 Recommended Tools for Common Memory / RAG Use Cases (Categorized by Purpose) + +Unsure where to start with tool selection? Here are commonly used combinations in the industry post-2025—**choose based on your scenario ("Entry Point") and follow the links for deeper dives**: + +| Scenario | Recommended Tools | Why | +|---|---|---| +| **First RAG Implementation** (Quickest Start) | [Chroma](https://github.com/chroma-core/chroma) + [LlamaIndex](https://github.com/run-llama/llama_index) | Local-first, zero-ops, beginner-friendly quickstart. Default for Stage 6 exercises. | +| **Enterprise-Grade RAG Framework** (Alternative to LangChain/LlamaIndex) | [Haystack (deepset)](https://github.com/deepset-ai/haystack) ★ 25.2k Apache-2.0 | Open-source by deepset, production-oriented orchestration, mature for enterprise NLP scenarios. | +| **Agent Long-Term Memory** (See 5 mainstream memory layers that can ship) | [agentmemory](https://github.com/rohitg00/agentmemory) / [mem0](https://github.com/mem0ai/mem0) / [Letta](https://github.com/letta-ai/letta) / [Zep](https://github.com/getzep/zep) / [LangMem](https://github.com/langchain-ai/langmem) | Detailed above in 5 mainstream memory layers that can ship. | +| **RAG / Memory Evaluation** (Must-Have) | [ragas](https://github.com/explodinggradients/ragas) ★ 13.9k | Standard RAG evaluation tool, 8+ metrics, reference-free + reference-based. | +| **Production-Scale RAG** (Millions of Docs) | [Qdrant](https://github.com/qdrant/qdrant) + LlamaIndex | Rust-based vector DB, faster than Chroma at scale. | +| **Existing Postgres Environment** | [pgvector](https://github.com/pgvector/pgvector) | Postgres extension, unified SQL + vector in one DB, simplest ops. | +| **Enterprise RAG + Web UI** | [RAGFlow](https://github.com/infiniflow/ragflow) | Robust document parsing (OCR/tables/layout), enterprise-grade, includes Web UI. | +| **Chinese RAG Template** | [Langchain-Chatchat](https://github.com/chatchat-space/Langchain-Chatchat) | Widely used in Chinese community, local LLM integration (ChatGLM/Qwen/Llama), good Chinese defaults. ★ 38k+, Apache-2.0. ⚠️ Last update Nov 2025 (marginal). | +| **Advanced: Contextual Retrieval** | [Anthropic Cookbook](https://platform.claude.com/cookbook/capabilities-contextual-embeddings-guide) | Claude with prompt caching for contextual chunking (**See Advanced RAG Techniques**). | +| **Advanced: Knowledge Graph Reasoning** | [LightRAG](https://github.com/HKUDS/LightRAG) / [Microsoft GraphRAG](https://github.com/microsoft/graphrag) | Knowledge graph + RAG, entity-relation reasoning (See Advanced RAG Techniques). | +| **Tutorial Collection** | [ai-engineering-hub](https://github.com/patchy631/ai-engineering-hub) | RAG + agent tutorial collection, Jupyter notebook format. | + +**Recommended Entry Sequence**: +1. First essential installation: **Chroma + LlamaIndex** (for Stage 6 exercises). +2. For agent memory needs: Add **mem0** (simplest memory layer). +3. For production scaling: Switch to **Qdrant** or **pgvector**. +4. For upgrading to advanced RAG: Explore techniques in the Advanced RAG Techniques section. + +## 🎯 Featured Projects (Templates / Specs / Example Collections) + +Categorized for quick reference; **choose by use case ("Entry Point") and follow the links for deeper dives**. + +| Category | Project | ⭐ | Who It's For | Why Recommended / Notes | +|---|---|---|---|---| +| **RAG Framework**
(Full Pipeline) | [LlamaIndex](https://github.com/run-llama/llama_index) | ⭐⭐⭐⭐⭐ | Applications focused on documents | Core RAG library, provides document loaders / chunking / retrieval / query engines. ★ 49k+ | +| | [infiniflow/ragflow](https://github.com/infiniflow/ragflow) | ⭐⭐⭐⭐⭐ | Teams shipping RAG to non-developers | Production-ready RAG engine, deep document understanding (layout/tables/OCR) + hybrid retrieval + agent loops + Web UI. ★ 79k+, Apache-2.0. | +| | [HKUDS/LightRAG](https://github.com/HKUDS/LightRAG) | ⭐⭐⭐⭐ | Those exploring research-grade graph + long-context memory methods | Graph + vector hybrid retrieval + summarization-based memory, backed by EMNLP 2025 paper. ★ 34k+, MIT. Research-oriented codebase. | +| **Vector DB**
(Local-First) | [Chroma](https://github.com/chroma-core/chroma) | ⭐⭐⭐⭐⭐ | Exercises 2 / 4, easiest vector DB to start with | Open-source embedding database, runs locally, in-memory/SQLite backend, zero ops. ★ 27k+, Apache-2.0. **Install**: `pip install chromadb` | +| **Vector DB**
(Production Scale) | [Qdrant](https://github.com/qdrant/qdrant) | ⭐⭐⭐⭐⭐ | When Chroma can't keep up, need production scale | Rust-based vector DB, offers cloud and self-hosted options. ★ 31k+ | +| **Vector DB**
(Hybrid) | [Weaviate](https://github.com/weaviate/weaviate) | ⭐⭐⭐⭐ | Production deployment + schema constraints | Built-in modules (text2vec/generative/classification), schema-driven, native BM25 + vector hybrid. ★ 16k+ | +| **Vector DB**
(Existing Postgres) | [pgvector](https://github.com/pgvector/pgvector) | ⭐⭐⭐⭐ | Teams already using Postgres | Postgres extension, unified SQL + vector in one DB, simplest ops. ★ 21k+ | +| **Vector DB**
(Runs in-app) | [lancedb/lancedb](https://github.com/lancedb/lancedb) | ⭐⭐⭐⭐ | Apps that want a vector DB built in, with no separate server | A vector DB that runs inside your app (no server to start); handles text + images, and searches by keyword + vector together. ★ 10k+, Apache-2.0. | +| **Memory Framework**
(Auto Fact Extraction) | [mem0ai/mem0](https://github.com/mem0ai/mem0) | ⭐⭐⭐⭐⭐ | Personal assistants / chatbots needing user-level memory | Self-refining memory layer, cross-session fact storage. ★ 59k+ | +| **Memory Framework**
(OS-Paging) | [Letta (formerly MemGPT)](https://github.com/letta-ai/letta) | ⭐⭐⭐⭐ | Agents running for extended periods (months) | Hierarchical memory (working/archival), OS-paging concept. ★ 22k+ | +| **Memory (In-Framework)** | [LangChain — Memory](https://python.langchain.com/docs/concepts/memory/) | ⭐⭐⭐ | Already using LangChain | 4 abstract memory types (buffer/summary/vectorstore-backed/entity). | +| **Advanced RAG Techniques** | [Anthropic — Contextual Retrieval Cookbook](https://platform.claude.com/cookbook/capabilities-contextual-embeddings-guide) | ⭐⭐⭐⭐⭐ | After basic RAG, want to upgrade | Claude with prompt caching for contextual chunking (full end-to-end examples). | +| **Chinese RAG Template** | [chatchat-space/Langchain-Chatchat](https://github.com/chatchat-space/Langchain-Chatchat) | ⭐⭐⭐⭐ | Chinese knowledge bases / RAG applications | Widely used in Chinese community, local LLM support (ChatGLM/Qwen/Llama/Ollama), good Chinese defaults. ★ 38k+, Apache-2.0. ⚠️ Last update Nov 2025 (marginal). | +| **Tutorial Collection** | [ai-engineering-hub](https://github.com/patchy631/ai-engineering-hub) | ⭐⭐⭐⭐ | Interested in seeing "how the same concept is implemented in different contexts" | Thematic LLM / RAG / agent tutorial collection, Jupyter notebooks, useful across many stages. ★ 34k+, MIT. | +| **Production AI Assistant**
(Learn to Ship RAG) | [onyx](https://github.com/onyx-dot-app/onyx) (formerly Danswer) | ⭐⭐⭐⭐⭐ | Want to see "how RAG-driven AI assistants are productionized" | Open-source enterprise AI assistant, cross-LLM support, full ingest/retrieval/chat/admin. ★ 29.4k, active maintenance. | +| **RAG Cookbook**
(30+ Techniques) | [NirDiamant/RAG_Techniques](https://github.com/NirDiamant/RAG_Techniques) | ⭐⭐⭐⭐⭐ | After basic RAG, want to explore various implementations | Large RAG techniques cookbook, includes Self-RAG / HyDE / Multi-Query / Adaptive and 30+ Jupyter notebook examples. | +| **DSPy**
(Programming not Prompting) | [stanfordnlp/dspy](https://github.com/stanfordnlp/dspy) | ⭐⭐⭐⭐⭐ | Used LLMs for a while, want to auto-optimize prompts + chains | Stanford NLP group, ★ 34.4k MIT, Path 3 paradigm (See DSPy in Advanced RAG Techniques) | +| **RAG / Memory Eval**
(Must-Have) | [explodinggradients/ragas](https://github.com/explodinggradients/ragas) | ⭐⭐⭐⭐⭐ | After completing Exercise 4 (Full RAG Pipeline), want to measure retrieval accuracy | Standard RAG evaluation tool, 8+ metrics, reference-free + reference-based. ★ 13.9k Apache-2.0 | + +## ✅ Self-Check Before Entering Stage 7 + +Can you: + +- [ ] Write a 50-line RAG pipeline (load → chunk → embed → store → query → answer)? +- [ ] Explain why naive chunking fails on long documents? +- [ ] Design different chunking strategies for API docs, PDFs, and tables? +- [ ] Choose between Chroma, Qdrant, and pgvector based on scale? +- [ ] Differentiate between "giving an agent memory" and "using RAG"? +- [ ] Explain where RAG and Memory complement each other (refer to the table in From RAG to Memory)? + +If yes → Proceed to [Stage 7 — Multi-Agent · Productionization](07-multi-agent-production.md). diff --git a/stages/06-memory-rag.md b/stages/06-memory-rag.md new file mode 100644 index 0000000..3161c05 --- /dev/null +++ b/stages/06-memory-rag.md @@ -0,0 +1,755 @@ +# Stage 6 — 上下文管理（Context Engineering）：RAG 與 Memory + +> **繁體中文** | [简体中文](./06-memory-rag.zh-Hans.md) | [English](./06-memory-rag.en.md) + +⏱ **時間估算**：2 週（約 10 小時） + +> 💡 這 stage 用語密度高（**RAG / 向量資料庫 / embedding / chunking / hybrid search / reranking⋯**）→ 不熟先翻 [`resources/glossary.md` 3](../resources/glossary.md#3-memory--retrieval--rag)。 + +> 📋 **本章組成**：定位 → 入口 → **RAG 主軸**（基礎 + 進階 + DSPy + Eval）→ **Bridge** → **Memory 主軸**（3 pattern + Trio + 進階）→ Chunking → Reflexion / Reasoning → 練習 → Projects +> +> 🔑 **關鍵名詞**：見 [`resources/glossary.md` 3](../resources/glossary.md#3-memory--retrieval--rag)（memory / RAG / embedding / chunking / reranking） + +本 stage 的核心不是「多背一點名詞」，而是理解 agent 如何管理 context。 + +- **RAG 解決的是**：現在要從外部知識庫查哪些資料？ +- **Memory 解決的是**：agent 應該跨對話、跨 session、跨任務記住什麼？ +- **Context Engineering 是更上層的問題**：每次 LLM call 前，該把哪些資訊組進 prompt，讓模型在有限 context window 內做出正確決策？ + +→ 接著 Stage 7 三層工程分工：**Prompt = 單次怎麼問 / Context = 這次該給哪些資訊 / Harness = 整個 agent system 怎麼跑起來**。本 stage 是中間那層。 + +### Agent 需要的兩種 context 能力 + +1. **Retrieval**：從外部知識庫找出和當前任務相關的資料。 +2. **Memory**：保留跨對話、跨 session、跨任務的狀態、偏好與經驗。 + +**RAG（Retrieval-Augmented Generation）** 是目前最常見的 retrieval 架構；**Memory** 則負責讓 agent 記得使用者、任務歷史與自己的過去經驗。這一章會把兩者分開講，避免把「查資料」和「記事情」混在一起。 + +### 先把名詞切開：Retrieval / RAG / Vector Store / Memory 不是同一件事 + +| 名詞 | 不要混淆成 | 白話解釋 | +|---|---|---| +| **Retrieval** | RAG 全部 | 找資料的動作 | +| **RAG** | Vector DB | retrieve + generate 的整體流程 | +| **Embedding** | Memory | 把文字轉成向量、方便相似度搜尋 | +| **Vector store** | RAG | 儲存與搜尋 embedding 的地方 | +| **Chunking** | Retrieval 本身 | 把文件切成適合被搜尋的片段 | +| **Memory** | RAG | agent 對使用者、任務與過去經驗的持久管理 | + +## 🎯 Context Engineering 是什麼（先定位） + +**一句話**：Context Engineering = 決定**每次呼叫 LLM 時、要把哪些資訊塞進它看得到的視窗（context window）**。 + +重點不是「開了幾次對話」、是「**每次對話裡塞了什麼**」。Karpathy 2025-06 [原 tweet](https://x.com/karpathy/status/1937902205765607626) 講得最精準：「**把對下一步有用的資訊剛好填進視窗**的精細藝術」。 + +📺 **視覺學習**：[李宏毅 2025 第 2 講 — Context Engineering：AI Agent 背後的關鍵技術](https://www.youtube.com/watch?v=lVdajtNpaGI)（NTU 生成式人工智慧與機器學習導論 2025） + +### 三層 stack 中的位置 + +![Prompt → Context → Harness 三層工程 stack](../resources/diagrams/prompt-context-harness-stack.png) + +詳細對照表見 [Stage 2 進階](02-prompt-engineering.md#-進階prompt--context--harness-三層-engineering)。 + +### 本 stage 處理 4 個 sub-problem 中的 2 個（Lance Martin 2025 framework） + +| Sub-problem | 解決什麼 | 具體例子 | 本 stage cover？ | +|---|---|---|---| +| **Select** | 要把**哪些**外部資訊撈進視窗 | user 問「我家附近哪間 cafe 好吃」→ 從 yelp DB 撈 3 家評分高的 → 塞進 prompt | ✅ 主軸（RAG / vector search / GraphRAG） | +| **Write** | 要把**哪些**互動 / 教訓寫進長期記憶 | user 上週說「我吃純素」→ 寫進 memory；這週又問餐廳推薦時、retrieve 出來避免推肉食 | ✅ 主軸（memory layers） | +| **Compress** | 對話太長怎麼壓 | 50 輪對話超過 200k token → 自動摘要前 40 輪、保留最後 10 輪原文 | ⚠️ 部分（這裡 + Stage 7 Harness `context manager`） | +| **Isolate** | 多 agent 各自視窗怎麼分 | supervisor 看全局、worker 只看自己那段、彼此不串擾 | ❌ Stage 7 multi-agent 處理 | + +### 4 個常被搞混的概念 — 一張表分清楚 + +| 詞 | 是什麼（抽象 / 具體）| 範例工具 | +|---|---|---| +| **Memory** | agent 跨對話 / 跨 session 記事情的**能力**（抽象概念） | LangChain ConversationBufferMemory / mem0 / Letta | +| **Embedding** | 把文字轉成 N 維**向量**、讓相似度可計算（資料轉換） | `sentence-transformers` 跑出 768 維向量 / OpenAI ada-002 | +| **Vector DB** | 存 + 查 embedding 的**儲存層**（基礎設施） | Chroma / Qdrant / Weaviate / pgvector | +| **RAG** | 「retrieve 相關片段 → 塞進 prompt → 生成」這個 **pattern**（架構模式） | LlamaIndex / LangChain RAG chain | + +→ **核心區分**：Memory 是**能力**、Embedding 是**資料轉換**、Vector DB 是**儲存**、RAG 是**架構 pattern**——這 4 個常被混用、實際上是 4 個不同層的概念。 + +### RAG vs Long Context vs Fine-tuning — 何時用什麼 + +LLM 知道你的私有 / 領域資料、有 3 種主要做法。**本 stage 教 RAG**，但你要知道何時不該用： + +| 選擇 | 適合 | 不適合 | 成本 | +|---|---|---|---| +| **RAG**
（外部 retrieve） | 大型 / 變動 / 私有知識庫、需要 citation 引用來源 | 推理需要全文一起看的任務、需要跨文件 multi-hop reasoning | 每 query 多 1 次 vector search 的 latency | +| **Long Context**
（直接塞 prompt） | < 200k token 的中型文件、一次性查詢、需要 cross-doc reasoning | 知識庫大 / 經常變動 / 想要 citation | 每 query 燒大量 input token（即使有 prompt caching）| +| **Fine-tuning**
（改 model 權重） | 風格 / 格式統一、特定領域語言（醫療、法律、code）| 知識會變、要 citation、不想練 model | 訓練成本 + 維護成本 + 模型 lock-in | + +→ **怎麼選**：先試 RAG（成本最低、變動最容易）→ RAG 撈不到才考慮 Long Context → 兩個都不行才考慮 Fine-tuning。**進 Stage 7 學 fine-tune deploy**。 + +## 📌 學習目標 + +- 建一條基本 RAG 流水線（chunk → embed → store → retrieve → generate） +- 看出 RAG 不該用在哪些地方（以及該用在哪些地方） +- 區分 short-term、long-term、episodic、semantic memory +- 理解 vector embedding 與相似度搜尋 +- 知道進階 RAG（GraphRAG / Contextual Retrieval / Hybrid Search）何時加、何時不加 + +## 🚪 進入條件 + +你應該已經： +- 完成 Stage 3（會寫 tool use、會呼叫 LLM API、看得懂 ReAct loop）—— **硬性技術前置** +- 走過 Stage 4（agent frameworks）+ Stage 5（Claude Code 生態）—— curriculum 主線是 **3 → 4 → 5 → 6**（見 [README 學習地圖](../README.md#️-學習地圖兩條學習路徑)）；非硬性技術前置，但 RAG / memory 常跟 framework + Claude Code memory 機制搭配、照順序走過理解更完整，且 [Stage 7](07-multi-agent-production.md) 預期你已完成 4 + 5 + 6 +- 能跑 Python `pip install` 安裝 SDK（後面練習會用到 `chromadb`、`sentence-transformers` 等） +- 對 list / dict / generator 等基礎 Python 結構上手 + +沒到的話 → 回 [Stage 3](03-tool-use-and-hello-agent.md) 或 [Stage 0 環境設定](00-foundations.md#何時可以跳過這個階段)。 + +## 📚 必修閱讀 + +1. [**LlamaIndex — RAG concepts**](https://docs.llamaindex.ai/en/stable/getting_started/concepts/) — 最清楚的入門 +2. [**LangChain — RAG tutorial**](https://python.langchain.com/docs/tutorials/rag/) — 動手做 +3. [**Pinecone — Learning Center**](https://www.pinecone.io/learn/) — vector DB 基礎 +4. [**Anthropic — Contextual Retrieval**](https://www.anthropic.com/news/contextual-retrieval) — Anthropic 搭配 prompt caching 的 RAG 寫法 +5. [**LangChain — Text splitters**](https://docs.langchain.com/oss/python/integrations/splitters/index) — chunking 策略入門 + +> 🙏 **Memory 章節特別推薦 [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents)**：本 stage 探討 memory 的概念跟初級實作、要 **chapter-length 深入版**請看 hello-agents 對應章節——short-term / long-term memory 的差異、context engineering 怎麼動態組裝、session 持久化、forgetting strategy 都講得最完整。本 stage 是路線圖、那邊是深度教材。 + +## 🧭 單元指引（漸進式 flow） + +本章按 **RAG 先學、Memory 後學** 的順序走——RAG 是 context engineering 最基礎、最常用的工具，Memory 是 agent 跨對話/跨 session 的能力；先把 RAG pipeline 跑通、再帶到 Memory 設計，最後回頭看 Chunking 細節。 + +**閱讀順序建議**： + +1. **🌐 RAG 基礎流水線**（下一節）— 建立 mental model +2. **🚀 進階 RAG 技巧** — GraphRAG / Contextual Retrieval / Hybrid Search 等 production 升級 +3. **🌉 從 RAG 到 Memory** — 為什麼 RAG 還不夠、需要 Memory 補哪段 +4. **🧠 Memory 設計** — 短期 vs 長期、3 種 pattern、CoALA framework +5. **🧩 Chunking 細節** — RAG / Memory 都會用到的技術深入 + +讀這章時可以順便思考：RAG 不適合哪些應用場景？哪些場景適合 RAG，但基本 RAG 還不夠好？這會帶到後面的 GraphRAG / Self-RAG / RAPTOR 等進階技術。 + +## 🌐 RAG 基礎流水線 + +**RAG（Retrieval-Augmented Generation）**= 「retrieve 相關片段 → 塞進 prompt → 生成」這個 pattern。可以想成在幫 agent 蓋圖書館——你要先把書放好、分類好，後續要查資料時，才會又快又精準。 + +**最基礎的 RAG 拆成兩條流水線**： + +- **資料預處理（ingest 一次）**：ingest → chunk → embed → store（index）。這一步是在建立可檢索的知識庫。 +- **檢索生成（每次 query）**：retrieve → generate。這一步是在使用者提問時，找出相關內容，再交給 LLM 生成回答。 + +![RAG 流水線總覽](../resources/diagrams/rag-pipeline-overview.jpg) + +圖中的 RAG Fusion、query rewrite 等屬於進階檢索技巧。**第一次學 RAG 時，先理解主線流程即可**。 + +**5 個 step 解讀**： + +| Step | 做什麼 | 在哪一條 pipeline | 技術細節在 | +|---|---|---|---| +| **1. ingest** | 把資料載入（PDF / web / DB / 對話 log） | 預處理 | LlamaIndex / LangChain 各自 loader | +| **2. chunk** | 把文件切成小塊（500-2000 token / chunk） | 預處理 | 見後段 🧩 Chunking 細節（先讀 RAG / Memory 主軸、技術深入留到那邊） | +| **3. embed** | 每個 chunk 轉成 N 維 vector | 預處理 | sentence-transformers / OpenAI ada-002 | +| **4. store** | vector + metadata 存進 vector DB | 預處理 | Chroma / Qdrant / pgvector | +| **5. retrieve + generate** | query 也 embed → top-k semantic search → 拼進 prompt → LLM 生成 | 每次 query | 通用 LLM API | + +上面只是最小骨架。**最常踩的坑 3 個**： + +- **chunk 太大 / 太小**：太大、retrieve 撈到的 chunk 裡只有一句相關、其他都是雜訊；太小、失去前後文（見 Chunking 細節） +- **embedding model 選錯**：中文文件用英文 model、retrieval 精度直接掉一半 +- **top-k 設太大 / 太小**：太小、漏 relevant chunk；太大、雜訊高 / token 燒 + +> 📚 **想看更多 RAG 踩坑指南 + 解法**：[NirDiamant/RAG_Techniques](https://github.com/NirDiamant/RAG_Techniques) ★ 大型 production RAG cookbook、含 30+ 技巧 + Jupyter notebook 範例。 + +> 📄 **RAG 真正常掛的兩個地方，別只顧 chunking**：(1) **解析（ingest）**——PDF→乾淨 markdown 是 garbage-in 的源頭：[docling-project/docling](https://github.com/docling-project/docling)（★61k、MIT）、[opendatalab/MinerU](https://github.com/opendatalab/MinerU)（中文 / 科學 PDF 強，**AGPL** 注意授權）、[microsoft/markitdown](https://github.com/microsoft/markitdown)（★150k+、MIT）。(2) **選嵌入模型**——第一個檢索品質決策，別瞎挑：看 [MTEB leaderboard](https://huggingface.co/spaces/mteb/leaderboard)，中文 / 多語常用 [BGE-M3](https://github.com/FlagOpen/FlagEmbedding)（★12k、MIT）。 + +跑完基本骨架後，跑 動手練習 1-4（embeddings / vector DB / chunking / 完整 pipeline）建立手感、再進下一節 進階 RAG 技巧。 + +## 🚀 進階 RAG 技巧（跑完基本 RAG 之後再看） + +下面六個 subsection 是 2024-2026 production RAG 最常加上的槓桿，按「加進 pipeline 哪一層」分組： +- **Retrieve 後** —— GraphRAG / Contextual Retrieval / Hybrid Search & Reranking +- **Retrieve 前**（query 改寫）—— Query Transformations +- **Retrieve 期間**（control flow）—— Adaptive / Agentic RAG +- **Index 結構** —— RAPTOR +- **2024-2026 縱覽** —— 其他 17 個值得知道的技巧 + +**先跑完上面 RAG 基礎拿到基準版本、再回來看這裡**——不然你會在沒有基準的情況下調參數，永遠不知道是哪個改動帶來提升。 + +| 技巧 | 解決什麼問題 | 加在 pipeline 哪一層 | 成本 | +|---|---|---|---| +| **GraphRAG** | vanilla RAG 不會做 multi-hop / 跨文件 entity-relation 推理 | retrieve 前（建 graph）+ retrieve 時（graph traversal）| 高（要先建 KG、需 LLM 抽 entity）| +| **Contextual Retrieval** | chunk 失去原文件 context、retrieval 撈錯片段 | chunk 後 / embed 前（加 contextual header）| 中（一次性、搭 prompt caching 後便宜 90%）| +| **Hybrid Search & Reranking** | 純 vector 漏字面命中、top-k 雜訊高 | retrieve 中（並查 BM25）+ retrieve 後（cross-encoder rerank）| 低（成熟工具直接接）| + +### 🔗 GraphRAG — 知識圖譜 + RAG + +**Mental model**：vanilla RAG 把文件切成 chunk、靠 embedding 相似度撈片段——但**它不知道哪些 entity 是同一個東西、entity 之間有什麼關係**。GraphRAG 在 ingest 階段先用 LLM 把文件抽成 **(entity, relation, entity)** 三元組建知識圖譜，retrieve 時除了向量比對、還做 graph traversal 撈到「相關 entity 的相關 entity」。 + +**何時用**： +- 任務需要 **multi-hop reasoning**（A → B → C 才能回答） +- 跨多份文件、entity 互相引用（公司財報、論文引用、調查報告、法律案例） +- 問題形如「X 影響了什麼 Y、Y 又連到哪些 Z」——vanilla RAG 通常只撈到 X 那塊文件 + +**何時不用**： +- 文件之間沒有 entity-relation 連結（純 FAQ、產品手冊各自獨立） +- 知識庫小（< 1k chunk）——vanilla RAG 已經夠 +- 預算緊——建 KG 的 token 成本可能是普通 RAG 的 10-50 倍 + +**代表 framework**： +- [**HKUDS/LightRAG**](https://github.com/HKUDS/LightRAG) ★ **35.1k** MIT EMNLP 2025 — 目前社群最熱的選擇、輕量、KG + vector hybrid、cost 比 Microsoft 版低 +- [**Microsoft GraphRAG**](https://github.com/microsoft/graphrag) — 原版 reference 實作、Apache-2.0、含 community detection +- [**gusye1234/nano-graphrag**](https://github.com/gusye1234/nano-graphrag) — < 1000 行的最小實作、適合先讀懂原理 + +**Paper**：[**From Local to Global: A Graph RAG Approach to Query-Focused Summarization (Edge et al. 2024)**](https://arxiv.org/abs/2404.16130) — Microsoft GraphRAG 的原始 paper、解釋 community summarization 為什麼能解 global query + +### 🪶 Contextual Retrieval — Anthropic 的 prompt-caching 解法 + +**Mental model**：vanilla chunk 失去原文件 context——「Q3 revenue grew 15%」這個 chunk 抽出來、你不知道是**哪家公司**、**哪一年**的 Q3。Anthropic 2024 提出：**ingest 時用 LLM 為每個 chunk 寫一段 50-100 token 的 contextual header**（「This chunk is from ACME Corp 2024 Q3 earnings, discussing the cloud segment...」）拼到 chunk 前面再 embed。搭配 **prompt caching** 讓「整份文件 + 每個 chunk」這個 prompt 只計費一次、後面所有 chunk 共用 cache。 + +**何時用**： +- chunk 字面意思跟原文件主題距離遠（財報、研究報告、長 narrative 文件） +- 你願意一次性付 ingest 成本、換 retrieve 精度 +- 已經在用 Claude / 想用 prompt caching（其他 model 也能跑、就是沒 cache 折扣） + +**何時不用**： +- chunk 本身就是 self-contained（FAQ、產品介紹頁、定義條目） +- 知識庫經常變動（每改一次就要重 ingest） +- 預算極緊——即便 cache 折扣後、ingest 成本仍比 vanilla 高 + +**為什麼省 90% cost**：Anthropic 報告 prompt caching 把「整份文件當 cached prefix」、每個 chunk 只送差異——比起每 chunk 都餵整份文件、成本降到約 1/10。但**這只省 ingest、不省 retrieve 階段**。 + +**代表實作**： +- [**Anthropic — Contextual Retrieval blog**](https://www.anthropic.com/news/contextual-retrieval) ⭐ — 官方說明 + benchmark（failed retrieval rate 從 5.7% 降到 1.9%） +- [**Anthropic cookbook**](https://platform.claude.com/cookbook/capabilities-contextual-embeddings-guide) — 端到端 Jupyter notebook、含 prompt 模板 + +**搭配技巧**：Anthropic 同篇 blog 還建議疊上 **Contextual BM25**（contextual chunk 同時餵 vector + BM25）+ **reranking**——剛好接到下面 Hybrid Search & Reranking。 + +### 🎯 Hybrid Search & Reranking — production RAG 的兩個常見強化元件 + +**Mental model**： +- **Hybrid Search** = vector similarity（語意像）+ BM25 / keyword（字面像）並查、用 [RRF (Reciprocal Rank Fusion)](https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf) 之類融合分數。解決純 vector search「query 跟 chunk 同義但用詞不同沒撈到」+「人名 / 編號 / 罕用詞語意 embedding 太弱」的雙重盲點。 +- **Reranking** = 第一階段 retrieve **top-50**（recall 優先、寬鬆撈）→ 用 **cross-encoder reranker** 重新打分排成 **top-5**（precision 優先、精準篩）。cross-encoder（query + chunk 一起進 model）比 bi-encoder（query / chunk 分開 embed）精準很多、但太慢、所以只用在第二階段。 + +**為什麼是「必加的強化元件」**：production RAG 評測幾乎一面倒——加 hybrid + reranker 後 recall@5 通常從 70% 上下提到 85-90%、邊際成本低、實作成熟。**這是 cost / benefit 最好的兩個改動**。 + +**何時用**： +- production RAG（不是 demo / 練習） +- query 包含人名、產品編號、技術術語、罕見字（純 vector 容易漏） +- 預算允許每 query 多 100-300ms latency + +**何時可以暫緩**： +- 練習階段 / MVP（先把 vanilla RAG 跑通） +- 預算極緊 / latency 極敏感（reranker 是額外一次 model call） + +**代表工具**： +- **Hybrid search**：[Weaviate](https://github.com/weaviate/weaviate)（內建 BM25 + vector + RRF）/ [Qdrant](https://github.com/qdrant/qdrant)（支援 sparse + dense vector）/ pgvector + Postgres FTS +- **Reranker**：[Cohere Rerank API](https://docs.cohere.com/docs/rerank-overview)（商業、最常用）/ [BGE Reranker](https://huggingface.co/BAAI/bge-reranker-large)(開源、HuggingFace、中文表現好) / [Jina Reranker](https://jina.ai/reranker) +- **Framework 內建**：LlamaIndex 的 `SentenceTransformerRerank` / LangChain 的 `ContextualCompressionRetriever` + +**Paper / 入門**： +- [**Pinecone — Rerankers and Two-Stage Retrieval**](https://www.pinecone.io/learn/series/rag/rerankers/) — reranker mental model 講最清楚 +- [**Anthropic — Contextual Retrieval**](https://www.anthropic.com/news/contextual-retrieval)（上面已列）— 同時示範 hybrid + reranker、有 benchmark + +### Query Transformations — HyDE / Multi-Query / RAG Fusion + +**Mental model**：vanilla RAG 把 user query 直接 embed 去查——但 query 跟文件**用詞 / 風格 / 抽象層級**經常差太多（user 問「我胃痛怎麼辦」、文件寫「上腹部疼痛之鑑別診斷」）。Query transformations 在 retrieve **前**先改寫 query、讓改寫版本更接近文件形式。 + +**3 個代表技巧**： + +| 技巧 | 怎麼改寫 | 何時用 | +|---|---|---| +| **HyDE**（Hypothetical Document Embeddings）| 先讓 LLM 對 query 生成「假想答案」、用答案 embedding 查 | query 跟 chunk 用詞風格差距大 | +| **Multi-Query** | LLM 把 query 改寫成 N 個變體分別 retrieve、union 去重 | query 太短 / 模糊 / 多義 | +| **RAG Fusion** | Multi-Query + RRF 融合 N 個 retrieval 結果 | 同上、想要更穩定的排名 | + +**何時不用**：query 已經是長 + 結構化（RAG over code、user 直接 paste error stack trace）——改寫反而引入雜訊。 + +**Paper / 實作**： +- [**HyDE (Gao et al. 2022)**](https://arxiv.org/abs/2212.10496) — 原始 paper +- [**RAG Fusion (Raudaschl 2023)**](https://github.com/Raudaschl/rag-fusion) — Multi-Query + RRF 的 reference 實作 +- LangChain 內建 `MultiQueryRetriever` / LlamaIndex `HyDEQueryTransform` + +### 🔁 Adaptive / Agentic RAG — Self-RAG / CRAG / Adaptive RAG（讓 retrieval 變成可判斷的流程） + +**Mental model**：上面所有 RAG 技巧都假設「query 來 → retrieve → generate」是固定 pipeline。Self-improving RAG 把這個 pipeline 變成**有判斷能力的 agent loop**——LLM 自己決定要不要 retrieve、判斷 retrieve 品質、不夠就再查或改 query。**這是 2024 RAG 研究的主軸**。 + +| 技巧 | 怎麼自我修正 | Paper | +|---|---|---| +| **Self-RAG** | 訓練 LLM 輸出 `[Retrieve]` token 決定要不要查、retrieve 後輸出 `[IsRel]/[IsSup]/[IsUse]` 評分每個片段 | [Asai et al. ICLR 2024](https://arxiv.org/abs/2310.11511) | +| **CRAG**（Corrective RAG）| retrieval evaluator 打分；高信心直接用、低信心 fallback 到 web search、中信心做 query 改寫 | [Yan et al. 2024](https://arxiv.org/abs/2401.15884) | +| **Adaptive RAG** | classifier 先判 query 複雜度、routing 到「不 retrieve / single-step / multi-step」三種策略 | [Jeong et al. NAACL 2024](https://arxiv.org/abs/2403.14403) | + +**為什麼這是 2024 主軸**：固定 pipeline 在簡單 query（「Tokyo 首都？」不用 retrieve）+ 複雜 query（multi-hop、cross-doc）兩個極端都吃虧。讓 LLM 自己 routing → 兩個極端都解。 + +**何時用**：production RAG、query 類型分布廣（從事實題到推理題都有）、願意付 1.5-3 倍 latency 換準確度。 +**何時不用**：query 類型單一 / 預算 / latency 極緊。 + +**實作**：LangGraph 有官方 [Self-RAG cookbook](https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_self_rag/) + [CRAG cookbook](https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_crag/) + [Adaptive RAG cookbook](https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_adaptive_rag/)、可直接套。 + +### 🌳 RAPTOR — 階層式遞迴 retrieval（ICLR 2024） + +**Mental model**：vanilla chunking 把文件切扁平 chunk——但**整本書的主旨不在任何單一 chunk 裡**。RAPTOR 把 chunk 遞迴聚類 + 摘要、建一棵**多層樹**：底層 = 原 chunk、中層 = 一群相關 chunk 的摘要、頂層 = 全文摘要。retrieve 時可選整棵樹搜尋、或選特定抽象層。 + +**為什麼有用**： +- **抽象 query** 撈得到（「這篇 paper 主要結論？」原 chunk 都沒這句、但頂層摘要有） +- **細節 query** 也撈得到（底層 chunk 保留） +- 跟 GraphRAG 不同——RAPTOR 是**樹**（hierarchical summarization）、GraphRAG 是**圖**（entity-relation） + +**何時用**：長文件（書、論文、報告）需要不同抽象層 query、知識庫 narrative 連貫。 +**何時不用**：chunk 之間獨立（FAQ）、知識庫經常變動（重建樹貴）。 + +**Paper / 實作**： +- [**RAPTOR (Sarthi et al. ICLR 2024)**](https://arxiv.org/abs/2401.18059) ⭐ — 原始 paper +- [**parthsarthi03/raptor**](https://github.com/parthsarthi03/raptor) — 官方 reference 實作 +- LlamaIndex 內建 `RAPTOR pack` + +### 🧬 DSPy — 不寫 prompt、用 program 自動 optimize（Path 3 paradigm） + +**Mental model**：傳統 RAG / Agent = 手寫 prompt + 手刻 chain。DSPy = **不寫 prompt**——你只定義「signature」（input → output 的型別）、再寫 program（chain 結構）；DSPy 自己用 LLM compile 出最佳 prompt + few-shot examples + retriever 設定。Stanford NLP group 2024 提出、Karpathy 推、目前 production 越用越多。 + +**何時用**： +- 你的 RAG 用了 6 個月、prompt 累積到難維護、想自動 optimize +- 同一 program 要切換不同 LLM provider（DSPy 自動 recompile） +- agent system 有多個 step、想跟蹤 trace / metrics + +**何時不用**： +- 你只有一個 prompt、不需要 optimization +- 第一次學 LLM、還沒摸過 prompting + +**代表 repo**：[**stanfordnlp/dspy**](https://github.com/stanfordnlp/dspy) ★ **34.4k** MIT、Stanford NLP group 官方、active 維護中。 + +**怎麼放進 RAG**：DSPy 跟本 stage 講的 RAG 技巧**不衝突**——你可以把 GraphRAG / Hybrid Search / Reranking 都當成 DSPy 的 module 來組、然後 compile。是 RAG 構築方式的「上層」typing 系統。 + +→ **跟 Path 1 / Path 2 reasoning 並列**：Path 1 是「人手寫 prompt」、Path 2 是「訓練進 model 權重」、**DSPy 是 Path 3「program 自動 search 出 prompt」**。Stage 7 multi-agent 進階場景特別好用。 + +### 📊 RAG 進階技巧縱覽 — 2025-2026 三條主軸 + +進階 RAG 的 2025-2026 演化集中在 **3 條主軸**： + +1. **🧠 KG + Memory 融合** — 從 flat vector store 走向「結構化、可演化、可聯想」的知識表示。代表：[**HippoRAG 2**](https://arxiv.org/abs/2502.14802)（海馬迴啟發、KG + PageRank、跨文件 multi-hop）、A-MEM、KAG。 +2. **🎬 Multimodal RAG** — 從文字 retrieval 走向圖像 / 影片 / 表格 native。代表：[**ColPali**](https://arxiv.org/abs/2407.01449)（PDF 頁面圖像直接 embed、繞過 OCR）、TV-RAG、MegaRAG。 +3. **🤖 Agentic RAG** — retrieval 從固定 pipeline 變 agent loop 內的 tool（agent 自己決定查幾次 / 怎麼查）。代表：A-RAG、Self-RAG（已上面 Self-improving）、SoK: Agentic RAG survey 2026。 + +另外 **2 個值得追的方向**： +- **🛡 RAG 安全** — corpus poisoning / prompt injection 進入 production 考量。代表：[RAGPart / RAGMask](https://arxiv.org/abs/2512.24268)。 +- **🔧 不再手寫 prompt** — 系統自動 search 出最佳 prompt + retriever 組合。代表：[**DSPy**](https://github.com/stanfordnlp/dspy)（Stanford「programming not prompting」典範、見上方 DSPy 段落）。 + +**5 個值得深挖的代表作**（速查）： + +| 技巧 | 一句話 | 連結 | +|---|---|---| +| **HippoRAG 2** | KG + Personalized PageRank、跨文件 multi-hop、海馬迴啟發 | [Gutiérrez et al. ICML 2025](https://arxiv.org/abs/2502.14802)、[OSU-NLP-Group/HippoRAG](https://github.com/OSU-NLP-Group/HippoRAG) ⭐ | +| **ColPali** | PDF 圖像直接 embed、繞過 OCR、multimodal RAG 入門 | [Faysse et al. 2024](https://arxiv.org/abs/2407.01449) | +| **A-RAG / SoK Agentic RAG** | retrieval 當 tool、agent 自己決策查幾次 | [Ayanami0730/arag](https://github.com/Ayanami0730/arag)、[SoK survey](https://arxiv.org/abs/2603.07379) ⭐ | +| **DSPy** | 不寫 prompt、用 program + signature、auto-optimize | [stanfordnlp/dspy](https://github.com/stanfordnlp/dspy) ★ 34.4k | +| **LightRAG** | MS GraphRAG 的 lightweight 替代、EMNLP 2025 | [HKUDS/LightRAG](https://github.com/HKUDS/LightRAG) ★ 35.1k（已在 GraphRAG 段落） | + +
+📚 完整縱覽 — 其他 12 個值得知道的進階 RAG 技巧（展開看） + +| 技巧 | 一句話 | 年份 / Paper | +|---|---|---| +| **Sentence-Window Retrieval** | embed 句子、retrieve 後回傳 ± N 句 window | LlamaIndex 內建 | +| **Parent-Child / Small-to-Big** | embed 小 chunk、回傳 parent chunk | LangChain `ParentDocumentRetriever` | +| **Multi-Vector Retrieval** | 一個 chunk 多個 embedding（摘要 / 原文 / 假想問題）| LangChain `MultiVectorRetriever` | +| **ColBERT / 後互動 retrieval** | token-level 比對而非 pooled embedding | [Khattab & Zaharia 2020](https://arxiv.org/abs/2004.12832)、[RAGatouille](https://github.com/AnswerDotAI/RAGatouille) | +| **LongRAG** | 大 chunk（4k）+ long-context reader、減少 retrieval 次數 | [Jiang et al. 2024](https://arxiv.org/abs/2406.15319) | +| **MemoRAG** | memory model 把 KB 壓成 latent memory、retrieve 用線索觸發 | [Qian et al. 2024](https://arxiv.org/abs/2409.05591) | +| **KAG**（Knowledge-Augmented Generation）| 嚴格 schema KG + 邏輯推理、金融 / 醫療 / 法律場景 | [Liang et al. 2024 (Ant Group)](https://arxiv.org/abs/2409.13731) | +| **MiA-RAG**（Mindscape-Aware）| 先建文件高層摘要 mindscape、用它引導 retrieval 跟回答 | [arXiv:2512.17220](https://arxiv.org/abs/2512.17220) ⭐ 2025-12 | +| **QuCo-RAG**（Quality-Controlled）| 用 pretraining 統計判斷該不該 retrieve、罕見 entity 觸發查、減 hallucination | [arXiv:2512.19134](https://arxiv.org/abs/2512.19134) ⭐ 2025-12 | +| **MegaRAG** | 多模態 KG、長文件抽 entity + relation + 視覺、建層級圖 | [arXiv:2512.20626](https://arxiv.org/abs/2512.20626) ⭐ 2025-12 | +| **TV-RAG** | training-free 時間感知 RAG、長影片 + 字幕 + 視覺對齊 | [arXiv:2512.23483](https://arxiv.org/abs/2512.23483) ⭐ 2025-12 | +| **RAGPart / RAGMask** | 對 RAG corpus poisoning 攻擊的輕量防禦 | [arXiv:2512.24268](https://arxiv.org/abs/2512.24268) ⭐ 2025-12 | + +
+ +## 🌉 從 RAG 到 Memory — 為什麼 RAG 還不夠 + +讀到這你已會跑基本 RAG + 知道幾個 production 槓桿。但回頭看 Context Engineering 列的 3 個 problem domain——你只解決了 **Retrieval**，**Memory 管理**還沒碰。為什麼這兩件事要分開？ + +RAG 解決「從**外部知識庫** retrieve 相關片段」——但 agent 還需要「**自己** 跨對話 / 跨 session 記事情」。這兩件事不是同一個問題： + +| 維度 | RAG | Memory | +|---|---|---| +| 內容來源 | **外部**（PDF / 文件 / web / DB）| **agent 自己的對話 / 經驗** | +| 寫入時機 | ingest 一次性、後續每次 retrieve | 每輪對話、每次 task 都可能寫 | +| 內容性質 | 偏靜態事實、文件知識 | 偏動態：user preference、過去互動、累積教訓 | +| 取代得了 RAG 嗎？| — | 取代不了——你不會把每份 PDF 當「memory」 | +| 被 RAG 取代嗎？| — | 不會——RAG 不會「記住上次 user 說了什麼」 | + +**3 個 RAG 不夠用的場景**（剛好對應到 Memory）： + +1. **跨 session 記 user preference / persona**——user 上禮拜跟 agent 說「我是純素」、這禮拜回來、agent 還記得不能推薦肉食。RAG 知識庫不會這樣自動更新。 +2. **agent 過去成敗教訓累積**（Reflexion 主場）——agent 第一次跑 task 失敗、反思「為什麼失敗」存起來、下次遇類似 task retrieve 進 prompt 避免重蹈覆轍。RAG 知識庫不會「記住自己的失敗」。 +3. **Long-horizon task 中間狀態**——agent 跑 100 step task、中間需要保留 working memory 不丟失。RAG 不適合做這種「短期 + 結構化 + 高頻寫入」的 state。 + +→ **結論**：RAG 跟 Memory 是**互補**而非取代。Production agent 通常**兩個都要**：RAG 接外部知識、Memory 記自己跟 user 的互動。下節 Memory 設計 教你怎麼挑 memory pattern。 + +## 🧠 Memory 是什麼 + 怎麼設計 + +> 📺 **視覺學習**：[李宏毅 2025 第二講 — 一堂課搞懂 AI Agent 的原理（含 read / write / reflection memory module）](https://www.youtube.com/watch?v=M2Yg1kwPpts)（NTU 生成式AI時代下的機器學習 2025） + +### Working memory vs Long-term memory — 兩種時間尺度 + +| 比較面向 | Working memory / 短期上下文 | Long-term memory / 持久記憶 | +|---|---|---| +| 中文可稱 | 工作記憶 / 短期上下文 | 長期記憶 / 持久記憶 | +| 核心意思 | 這次任務或這段對話中，模型當下看得到的資訊 | 存在外部，之後可跨 session 取回的資訊 | +| 持續時間 | 短，通常限於目前 session | 長，可跨 session | +| 技術基礎 | 上下文視窗（context window）/ prompt | 記憶儲存層（memory store）/ 使用者檔案 / 向量資料庫 | +| 適合記什麼 | 任務細節、剛剛說過的內容 | 穩定偏好、長期目標、背景資料 | +| 是否受 context 長度限制 | 會，因為模型一次能看的內容有限 | 較不會，因為可以先存在外部，需要時再取一小段放回來 | +| 生活例子 | 剛剛收到的手機驗證碼、正在進行對話的上一句話 | 你深化學會的知識、圖書館、知識庫、讀過的書 | + +→ 在 agent 裡，「短期記憶」嚴格說更接近 **working memory**——它不是外部儲存、是目前 prompt / context window 裡看得到的內容。 + +### Episodic / Semantic / Procedural memory — 三種內容類型 + +**注意**：上面 working / long-term 是**時間軸**，下面三種是**內容軸**——兩組分類**正交、不互斥**。Long-term memory 裡可以**同時**有 episodic + semantic + procedural 三種。 + +| 類型 | 中文 | 核心意思 | +|---|---|---| +| **Episodic memory** | 情節記憶 / 經驗記憶 | 過去某次任務、某次互動、某次失敗的具體經驗 | +| **Semantic memory** | 語意記憶 / 事實記憶 | 穩定知識、使用者偏好、背景事實 | +| **Procedural memory** | 技能型記憶（procedural memory）| agent 知道「怎麼做事」的規則、工具、workflow、skills | + +→ 這 3 種對應 [CoALA framework](#進階coala-framework--agent-memory-的-4-層-taxonomy)、production agent 通常 3 種都用得到。**Reflexion** 是典型的 episodic memory 應用（累積 trial 的成敗經驗）。 + +這裡的工作階段（session）可以理解成一次連續互動，例如同一段聊天、同一次任務，或同一次 agent 執行。 + +### 3 種設計 pattern（什麼時候用什麼）⭐ Track B 必看 + +**不是所有 agent 都需要外部 memory store。Memory 架構選錯會花十倍 token 達同樣效果。** + +這是進練習前要建立的 mental model——下面練習 1-5 跑的是「pattern 3 vector store」，但 production 你可能不需要這麼複雜。 + +| Pattern | 適合場景 | 怎麼跑 | 成本 | +|---|---|---|---| +| **1. Naive buffer**
（全塞 context） | 短對話、≤ 10 turn、agent 不需要記跨 session 的東西 | 整段 history 每次都送進 prompt | 線性增長、token 燒得快 | +| **2. Summary + recent**
（摘要遠的 + 保留近 N 輪） | 中長對話、~ 50 turn、想壓縮但別丟太多 | 每 N 輪叫 LLM 把舊 history 摘成 1 段；prompt = `summary + last N turns` | 中等、有 LLM 摘要成本 | +| **3. Vector store + retrieval**
（外部 store + 每次 semantic search） | 跨 session、知識庫場景、agent 要「想起」久遠的事 | embed 過去 message → 存 vector DB → 每回合 query 相關片段拼進 prompt | 高（向量計算 + 儲存），但 token 用量穩定 | + +**怎麼選**： + +- 對話 chatbot 沒跨 session → **pattern 1** +- agent + 長對話、要記今天聊過什麼 → **pattern 2** +- agent + 跨 session + 知識庫（本 stage 練習場景）→ **pattern 3** +- production 大型 agent → 通常**混用**：近期 pattern 1/2、長期 pattern 3 + +> 💡 **Track B 重點**：你 Stage 7 寫 multi-agent 時，每個 agent 都會有「自己的 memory」+「shared memory」雙層——需要的 pattern 通常是 **2 + 3 混用**。先在本 stage 把 3 種 pattern 跑透，到 Stage 7 才不會被 multi-agent memory 設計卡住。 + +### ⭐ 5 個可上線使用的 Memory Layer（按 use case 挑） + +> Star 數與 benchmark 會變動；這裡重點不是排行，而是理解每個 memory layer 的設計取向。 + +學完 3 pattern 後、production 不必自己刻 memory store。下面 5 個都是 Apache-2.0 / MIT、active 維護、各擅其場： + +| Framework | Stars | License | 主場 use case | 特色 | +|---|---|---|---|---| +| [**agentmemory**](https://github.com/rohitg00/agentmemory) | 7.7k★ | Apache-2.0 | **Coding agent 跨 session 記憶** | MCP-universal（Claude Code / Cursor / Gemini CLI / Codex / Hermes / OpenClaw 都接得上）、95.2% R@5、92% token saving、51 MCP tools + 12 auto hooks、benchmarks-driven | +| [**mem0**](https://github.com/mem0ai/mem0) | 55.6k★ | Apache-2.0 | **Chatbot / 個人助理 user-level memory** | Auto fact extraction + forgetting + namespace、production-tested、最大社群 | +| [**Letta**](https://github.com/letta-ai/letta)（前身 MemGPT）| 22.7k★ | Apache-2.0 | **長 session agent**（月為單位）| OS-style paging memory（working + archival 雙層）、persona stability、MemGPT paper origin | +| [**Zep**](https://github.com/getzep/zep) | 4.6k★ | Apache-2.0 | **Temporal KG-based memory** | 把對話歷史建成 temporal KG、entity 之間有時間軸、適合需要 audit trail / time-aware reasoning 的 agent | +| [**graphiti**](https://github.com/getzep/graphiti) | 27.5k★ | Apache-2.0 | **即時知識圖譜 agent 記憶** | 把 agent 過去的互動變成有時間軸的 knowledge graph、方便回頭查找；Zep 背後的引擎、可單獨使用 | +| [**LangMem**](https://github.com/langchain-ai/langmem) | 1.4k★ | MIT | **LangChain-native memory** | LangChain 官方 memory lib、與 LangGraph 直接整合、適合已 commit LangChain stack | + +**怎麼挑**： +- 寫 coding agent → **agentmemory**（MCP-native、跟 Stage 5 ecosystem 完美 align） +- 做 chatbot / 個人助理 → **mem0**（最成熟、最大社群） +- 做 long-running 跨月 agent → **Letta**（OS-paging 強項） +- 需要時間軸 + audit → **Zep**（temporal KG） +- 已在 LangChain stack → **LangMem**（不必跳 framework） + +**另加官方 docs**：[Anthropic Memory Tool](https://docs.claude.com/en/docs/agents-and-tools/tool-use/memory-tool)（Claude 官方 tool-based memory、file-based、API 直接 call）、[LangChain Memory concepts](https://python.langchain.com/docs/concepts/memory/)（framework 內各 memory class 對比）。 + +### 進階：CoALA framework — agent memory 的 4 層 taxonomy + +[**Sumers et al. 2023 — Cognitive Architectures for Language Agents**](https://arxiv.org/abs/2309.02427) 把 agent memory 拆成 4 種、是現在最常用的 mental model： + +| 類型 | 存什麼 | 對應例子 | +|---|---|---| +| **Working memory** | 當前 task 上下文 | LLM context window 本身 | +| **Episodic memory** | 過去 task 的具體經驗 | Reflexion 反思記錄、past trajectories | +| **Semantic memory** | 抽象事實 / 知識 | RAG 知識庫、user profile、preference | +| **Procedural memory** | 怎麼做事的程式 / skill | tool definitions、[Skills（Stage 5.3）](05-claude-code-ecosystem.md#53--skillsclaude-code-的行為層-claude-code-生態最關鍵的一層) | + +→ **為什麼有用**：上面 3 種 pattern（buffer / summary / vector）都只在處理 working + episodic。Production agent 4 層都要設計——CoALA 是檢查表，看看你的 agent 哪一層缺了。 + +### 進階：Generative Agents — 三分數打分（經典案例） + +[**Park et al. 2023 — Generative Agents: Smallville**](https://arxiv.org/abs/2304.03442) 的小鎮模擬有 25 個 NPC agent、每個都有自己的 memory stream。retrieve 時用三個分數加權： + +- **Importance**：LLM 自己幫每個 memory 打 1-10 重要性分（吃飯 = 2 分、分手 = 9 分） +- **Recency**：時間衰減（exponential decay） +- **Relevance**：跟當前 query 的 embedding 相似度 + +最終分 = `α·importance + β·recency + γ·relevance`、排前 k retrieve。**這是 2024-2025 可上線使用的 memory layer（mem0 / Letta）的概念骨架**。 + +> 💻 **官方 code**：[joonspk-research/generative_agents](https://github.com/joonspk-research/generative_agents) ★ paper 配套的小鎮模擬 code repo、想跟著實作 memory stream + 三分數 retrieval 看這裡。 + +### 2024-2026 最新 Memory 作品 — 三條主軸 + +Memory research 2024-2026 集中在 **3 條主軸**： + +1. **🧠 結構化、可演化、可聯想** — 從 flat vector store 走向人腦 / Zettelkasten 啟發的記憶結構。代表：[**A-MEM**](https://arxiv.org/abs/2502.12110)（memory 之間自動建 link）、[**HippoRAG 2**](https://arxiv.org/abs/2502.14802)（KG + PageRank、海馬迴啟發）。 +2. **📚 2026 survey 大爆發** — 一年內 5 個重磅 survey + 跨領域整理。代表：[**Memory in the Age of AI Agents**](https://arxiv.org/abs/2512.13564)（3 維 taxonomy + benchmark）、[**Memory for Autonomous LLM Agents**](https://arxiv.org/abs/2603.07670)（write-manage-read loop 形式化）。 +3. **🛡 Memory security 變獨立子領域** — agent 跑久了、memory 會被 cross-session poisoning / 未授權存取攻擊。代表：[**Memory Security survey**](https://arxiv.org/abs/2604.16548)（Stage 7 安全 會接到）。 + +**4 個值得深挖的代表作**： + +| 作品 | 一句話 | 連結 | +|---|---|---| +| **Anthropic Memory Tool** | Claude 官方 tool-based memory、API 直接 call、file-based | [Anthropic Docs](https://docs.claude.com/en/docs/agents-and-tools/tool-use/memory-tool) | +| **A-MEM**（Agentic Memory）| Zettelkasten-inspired、memory 之間自動建 link、會演化 | [Xu et al. 2025](https://arxiv.org/abs/2502.12110) ⭐ | +| **HippoRAG 2** | KG + Personalized PageRank、跨文件 multi-hop、ICML 2025 | [Gutiérrez et al. 2025](https://arxiv.org/abs/2502.14802)、[OSU-NLP-Group/HippoRAG](https://github.com/OSU-NLP-Group/HippoRAG) ⭐ | +| **Memory in the Age of AI Agents**（survey）| 3 維 taxonomy（temporal / substrate / control）+ benchmark 彙整 | [Hu et al. 2025-12](https://arxiv.org/abs/2512.13564) ⭐ | + +→ 跑很久的 agent（週 / 月為單位）、上面 survey 必讀。 + +
+📚 完整縱覽 — 其他 8 個值得知道的 memory 作品（展開看） + +| 技巧 | 一句話 | 年份 / Paper | +|---|---|---| +| **MemGPT → Letta GA** | OS-paging memory、working / archival 雙層、long session 強項 | [Packer et al. 2023](https://arxiv.org/abs/2310.08560) → Letta GA | +| **MemoryBank** | Ebbinghaus 遺忘曲線、被存取的 memory 強化、沒用的衰減 | [Zhong et al. 2023](https://arxiv.org/abs/2305.10250) | +| **MemoryLLM** | self-updatable memory parameters 內建在 model（在權重而非 context）| [Wang et al. 2024](https://arxiv.org/abs/2402.04624) | +| **mem0**（見 Production Memory Trio）| 可上線使用的 memory layer、auto fact extraction + forgetting | [mem0ai/mem0](https://github.com/mem0ai/mem0) | +| **Memory for Autonomous LLM Agents**（survey）| write-manage-read loop 形式化、跨 2022-2026 整理 | [arXiv:2603.07670](https://arxiv.org/abs/2603.07670) ⭐ 2026 | +| **From Storage to Experience**（survey）| 演化框架：Storage → Reflection → Experience 三階段 | [arXiv:2605.06716](https://arxiv.org/abs/2605.06716) ⭐ 2026 | +| **ScrapMem** | bio-inspired on-device memory、"Optical Forgetting" 把老 memory 解析度漸降 | [arXiv:2605.03804](https://arxiv.org/abs/2605.03804) ⭐ 2026-05 | +| **Memory Security survey** | long-term memory 被 cross-session poisoning / 未授權存取 / 組織內傳播風險 | [arXiv:2604.16548](https://arxiv.org/abs/2604.16548) ⭐ 2026 | + +
+ +## 🧩 Chunking 細節（技術深入） + +好的 chunking 可以讓 LLM 在有限 context 內，用更精確、完整的資訊生成回答。它不是把文字平均切開。 + +切法取決於應用場景與文件內容。它會決定 retriever 看見的最小語意單位。 + +一個好 chunk 要同時做到兩件事：**夠完整**，讓模型看得懂上下文；**夠聚焦**，讓檢索不帶太多雜訊。chunk 太小會失去前後文，chunk 太大會讓相似度搜尋變鈍。 + +**常見策略**： + +- **固定長度（Fixed-Length）**：照字元數或 token 數切。優點是簡單穩定；缺點是一板一眼，容易切斷段落、句子或表格。 +- **滑動視窗（Sliding Window）**：每個 chunk 之間保留重疊區塊（overlap）。優點是比較不會在邊界掉資訊；缺點是索引量會變大。 +- **遞迴切割（Recursive）**：先嘗試保留段落，如果長度還是不適合，再退到句子、字詞等更小單位。通常是入門 RAG 的好基準。 +- **語意切割（Semantic Chunking）**：依 embedding 或語意變化切，也就是當前區塊與前一個區塊的語意相似度出現差異。適合長文件，但成本與複雜度較高。 +- **混合策略（Hybrid）**：依照應用場景，思考不同文件結構該怎麼混搭切法。例如，一篇論文可能要保留章節、表格、公式與引用脈絡。 + +![Chunking 策略流程](../resources/diagrams/chunking-strategies.jpg) + +> 📚 **經典教學**：[Greg Kamradt — 5 Levels of Text Splitting](https://github.com/FullStackRetrieval-com/RetrievalTutorials) ★ chunking 入門必看、從 character-based 一路講到 agentic chunking 五個層次、含 Jupyter notebook。 + +第一次做 RAG 時，不要一開始就追求複雜切法。LangChain 文件建議多數情境先從 `RecursiveCharacterTextSplitter` 開始。 + +先跑出基準版本，再用後續 retrieval 結果決定要不要換策略。 + +```python +from langchain_text_splitters import RecursiveCharacterTextSplitter + +text = "這是一個很長的文件內容...（此處省略一千字）..." + +splitter = RecursiveCharacterTextSplitter( + chunk_size=100, + chunk_overlap=20, + length_function=len, +) + +chunks = splitter.split_text(text) +print(f"共切成 {len(chunks)} 個 chunk") +print(chunks[0]) +``` + +**直覺判斷 chunking 好不好**，可以先看兩件事： + +- 回答缺漏資訊，或有頭無尾：通常是 chunk 太小，或 overlap 不夠。 +- 回答包含正確資訊，但混入無關內容：通常是 chunk 太大，或 top-k 撈太多。 + +**進階思考**： + +- chunking 不是一次設定好就結束，要配合真實 query 與失敗案例反覆調整。 +- chunk size、overlap、top-k、reranker 會互相影響，不要只單看其中一個參數。 +- 想想看，如果今天要 RAG 的資料有含圖片的 PDF、會議字幕檔，要如何切割比較好？ +- chunking 的進階變形（Sentence-Window / Parent-Child / Multi-Vector）見 進階 RAG 技巧縱覽表。 + +## 🪞 進階：帶持久記憶的 Reflexion 完整版 ⭐ Track B 選讀 + +> **本節是 concept + routing、不是練習**。延續 [Stage 3 反思](03-tool-use-and-hello-agent.md#-反思reflexion--self-refine-概念--路由) 的基本版（single-session Actor / Critic loop），講為什麼有些反思**需要**持久記憶——這版本才真正屬於 Stage 6 主題。 + +**Reflexion 完整版跟 Self-Refine 差在哪**： + +| 版本 | 跨輪保留什麼 | 跨 session 保留什麼 | 需要 memory pattern | +|---|---|---|---| +| **Self-Refine**（Madaan 2023） | 上一輪的 answer + critic feedback | ❌ 不保留 | 不需（pattern 1 buffer 即可） | +| **完整 Reflexion**（Shinn 2023） | 同上 | ✅ 把過去 trial 的「反思摘要」存進 episodic memory，下次遇到類似 task 時 retrieve 進 prompt 當教訓 | **需要**（pattern 3 vector store 或 pattern 2 summary） | + +**為什麼這個版本要 memory**：Reflexion paper 的 verbal reinforcement learning 是「agent 跨 trial 累積教訓」——agent 嘗試 task → 失敗 → 反思「為什麼失敗」存起來 → 下次遇到類似 task 時把過去反思 retrieve 進 prompt，避免重蹈覆轍。這就需要 **persistent episodic memory**，跟本 stage 上面講的 3 種 memory pattern 直接接上。 + +**典型架構**（持久記憶完整版）： + +![Reflexion 持久 episodic memory loop](../resources/diagrams/reflexion-persistent-memory-loop.png) + +→ **跟 Stage 3 反思的差別**：Stage 3 是 **single-session in-context** loop（沒外部 store）、本節是 **persistent episodic memory store + retrieve**（跨 trial 累積）。 + +### 📚 想動手 / 想深入 + +**Paper**： +- [**Reflexion (Shinn et al. 2023)**](https://arxiv.org/abs/2303.11366) ⭐ — **完整版** paper，Algorithm 1 寫出 memory buffer 怎麼用 +- [**Self-Refine (Madaan et al. 2023)**](https://arxiv.org/abs/2303.17651) — 對照 baseline，沒 episodic memory 的版本 + +**Reference 實作**： +- [**noahshinn/reflexion**](https://github.com/noahshinn/reflexion) — paper 第一作者的 reference 實作（含 episodic memory 完整流程） +- [**LangChain — Reflexion**](https://langchain-ai.github.io/langgraph/tutorials/reflexion/reflexion/) — LangGraph 版本，跟本 stage 練習 4 RAG pipeline 直接接得起來 +- [**mem0**](https://github.com/mem0ai/mem0)（已在上面列）+ [**Letta**](https://github.com/letta-ai/letta)（已在上面列）— 可上線使用的 memory layer，可以直接當 Reflexion 的 episodic store + +> 💡 **跟 Stage 3 反思的分工**： +> - 想理解「反思 loop 怎麼運作、單次怎麼跑」→ Stage 3 反思 +> - 想理解「反思怎麼跨 session 累積、agent 怎麼從過去學教訓」→ 本節 +> - 想看 production agent 內怎麼用反思（Cursor / Claude Code）→ [Stage 5 5.7 Harness Internals](05-claude-code-ecosystem.md#57--claude-code-source-解剖reference-harness-implementation-track-b-必看) + +## 🤔 進階 Reasoning / Reflection — 2024-2026 思潮 ⭐ 兩個 track 都看 + +Reflexion 是 **prompt-based reflection**——LLM 在 inference 時自己改自己。2024-2025 出現了**第二條路**：**訓練時就把 reflection 練進 model**（OpenAI **o1** / DeepSeek **R1**）。兩條路你都該知道。 + +### Path 1：Prompt-based reflection / reasoning（傳統做法） + +| 技巧 | 核心想法 | Paper | +|---|---|---| +| **Self-Consistency** | sample N 條推理、多數決 — **最簡單 + 最常用** | [Wang et al. 2022](https://arxiv.org/abs/2203.11171) | +| **Tree of Thoughts (ToT)** | reasoning 變樹、可分叉可回溯、適合 puzzle / planning | [Yao et al. 2023](https://arxiv.org/abs/2305.10601) | +| **Graph of Thoughts (GoT)** | 不只樹、可任意合併分支 | [Besta et al. 2023](https://arxiv.org/abs/2308.09687) | +| **Chain-of-Verification (CoVe)** | 生答案 → 對自己提驗證題 → 改答案 | [Dhuliawala et al. 2023](https://arxiv.org/abs/2309.11495) | +| **CRITIC** | tool-augmented self-critique（用 search / calculator 驗）| [Gou et al. 2023](https://arxiv.org/abs/2305.11738) | +| **Self-Discover** | agent 先「發現」該用什麼 reasoning structure 再執行 | [Zhou et al. ICML 2024](https://arxiv.org/abs/2402.03620) ⭐ 2024 | +| **Self-Refine / Reflexion** | 已在上面 / Stage 3 講 | Stage 3 反思、本 stage Reflexion | + +### Path 2：Trained-in reasoning / reflection（2024-2026 大轉折） + +> 📺 **視覺學習**：[李宏毅 2025 第七講 — DeepSeek-R1 這類大型語言模型是如何進行「深度思考」(Reasoning) 的？](https://www.youtube.com/watch?v=bJFtcwLSNxI)（NTU 生成式AI時代下的機器學習 2025） + +OpenAI **o1**（2024-09）開啟、DeepSeek **R1**（2025-01）開源化、**DeepSeek-V4-Pro**（2026-04 preview、agent-focused 開源 reasoning）+ Claude Fable 5（2026-06、Mythos-class、位階在 Opus class 之上；2026-06-12 曾暫停、2026-07-01 已恢復）+ Claude Opus 4.8（2026-05、Opus class 旗艦、Dynamic Workflows + parallel subagent）+ GPT-5.5（2026-04）+ Gemini 3.1 Pro（2026-02）為當前 frontier（2026-06 後半 Gemini 3.5 Flash 登場；GPT-5.6 Sol / Terra / Luna 已於 2026-07 正式推出）——把「step-by-step thinking + 自我糾錯」**訓練進 model 權重**、inference 時自動展開長 reasoning chain（thinking tokens）。**這是 2024-2026 LLM 最大典範轉移**、目前所有 frontier model 都走這路。下表只列**當前（2026-06）frontier**——歷史前身（o1 / R1 / Sonnet 4.5 / Gemini 2.5）省略、想看 lineage 看每家發布日列。 + +| Model | 來源 / 發布 | 特色 | 連結 | +|---|---|---|---| +| **GPT-5.5** | OpenAI 2026-04（前身：o1 2024-09 → o3 → GPT-5 2025-08 → 5.4 2026-03）| 閉源、reasoning + chat 合併、Thinking budget API、agent 能力強化。**較新層級：GPT-5.6（Sol / Terra / Luna）、2026-07 正式推出、1.05M context** | [OpenAI](https://openai.com/) | +| **Claude Fable 5** | Anthropic 2026-06（Mythos-class、位階在 Opus class 之上；同步發布 Claude Mythos 5 為解除部分 safeguard 的限量版本）| 閉源、Mythos-class（位階在 Opus class 之上）。曾於 2026-06-12 被美國出口管制暫停，**出口管制 2026-06-30 解除、[Fable 5 於 2026-07-01 全球恢復](https://www.anthropic.com/news/redeploying-fable-5)**（重新部署時加了新安全 classifier；Mythos 5 僅對核准的美國組織恢復）。官方 benchmark 數字始終未公布 | [Claude Fable 5 / Mythos 5](https://www.anthropic.com/news/claude-fable-5-mythos-5) | +| **Claude Opus 4.8** | Anthropic 2026-05（前身：Sonnet 4.5 / Opus 4.5 / Opus 4.7、Dynamic Workflows 研究預覽）| 閉源、Opus class 旗艦（Fable 5 暫停期間曾是最高可用層級；Fable 5 已於 2026-07-01 恢復）、可控 thinking budget（API 參數）、**SWE-bench / Terminal-bench 領先** | [Anthropic extended thinking](https://docs.claude.com/en/docs/build-with-claude/extended-thinking) | +| **Gemini 3.1 Pro** | Google 2026-02（前身：Gemini 2.5 Thinking 2025、Gemini 3 2025-11）| 閉源、可看 thinking trace、**GPQA Diamond 94.3%**、價格 / 速度 / multimodal 領先。**較新層級：Gemini 3.5 Flash、2026-06 已開放（3.5 Pro 開發中）** | [Gemini API](https://ai.google.dev/gemini-api/docs/thinking) | +| **DeepSeek-V4 / V4-Pro / V4-Flash** | DeepSeek 2026-04 preview（前身：R1 2025-01 → V3.1）| 開源 **MIT license**、agent-focused 訓練、推理 + 工具使用 + 知識處理整合、R 系列 reasoning 已併入主線 | [HF DeepSeek-V4-Pro](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)、[R1 paper（方法 baseline）](https://arxiv.org/abs/2501.12948)、[CNBC report](https://www.cnbc.com/2026/04/24/deepseek-v4-llm-preview-open-source-ai-competition-china.html) | +| **QwQ-32B / QvQ-72B** | Alibaba Qwen 2024-11 ~ 2026 | 開源 **Apache 2.0**、32B 在小尺寸 reasoning 仍是不錯的選擇、QvQ 是視覺版本 | [QwQ blog](https://qwenlm.github.io/blog/qwq-32b-preview/) | + +### 兩條路怎麼選 + +| 你的情況 | 建議 | +|---|---| +| 用一般 chat model base、想加 reasoning | Path 1（prompt-based）—— ToT / Self-Consistency / CoVe | +| 預算 / latency 允許、要最強 reasoning | Path 2 —— **GPT-5.5 / Opus 4.8 / Gemini 3.5 Flash / Grok 4.3 / V4-Pro** 任挑一個（Claude Fable 5 已於 2026-07-01 恢復，屬 premium 高階層級）| +| 想自己 fine-tune reasoning model | Path 2 —— 讀 R1 paper（方法 baseline）、從 R1-Distill / V4 開源權重起步 | +| 想 on-device / 預算極緊 | **QwQ-32B**（Apache 2.0）或 R 系列 distill | +| Multi-agent debate / critic 場景 | Path 1（CRITIC / debate）+ [Stage 7 multi-agent](07-multi-agent-production.md) | + +> 💡 **2025-2026 觀察**： +> - reasoning model 把 Reflexion 那套吞進權重——但 **prompt-based reflection 沒被取代**：agent loop（控制反思時機 / 內容）+ multi-agent debate 還是必須的 +> - **2026 開源逼近閉源**——DeepSeek-V4-Pro（2026-04 preview、MIT license）把 R1 reasoning 併入主線、agent-focused 訓練、跟 GPT-5.5 / Gemini 3.5 Flash 差距持續縮小 +> - **agent capability 變主訴求**——V4 / Opus 4.8 都把 agent-as-product（SWE-bench / Terminal-bench / tool use）當 headline benchmark、單純 reasoning 已經不夠賣 +> - 兩條路會長期共存、production agent 兩個都用 + +## 📏 RAG / Memory Eval — 跑得起來 ≠ 跑得準 + +**為什麼這節重要**：RAG / Memory 系統最大的坑是「**看起來能跑、實際 retrieval 精度爛**」。沒 eval、你不會知道改 chunk size / 換 embedding / 加 reranker 是不是真的有幫助——只會看到「答案好像比較順」這種主觀印象。Production agent 沒 eval 等於沒測試。 + +**3 個核心 metric**： + +| Metric | 量什麼 | 工具 | +|---|---|---| +| **Retrieval Recall@K** | top-K 撈到的 chunk 裡有沒有 ground truth 答案的 chunk | ragas / TruLens / LangSmith | +| **Answer Faithfulness** | 生成答案是不是 grounded 在 retrieved chunk（vs 模型 hallucinate）| ragas / TruLens | +| **Answer Relevance** | 答案跟 query 相關度（避免答非所問）| ragas / LLM-as-judge | + +**代表 framework**： + +- [**explodinggradients/ragas**](https://github.com/explodinggradients/ragas) ★ **13.9k** Apache-2.0 ⭐ — RAG evaluation 標準工具、含 8+ metric（faithfulness / answer relevance / context precision / context recall 等）、支援 reference-free + reference-based eval +- [**TruLens**](https://github.com/truera/trulens) — observability + eval 一起、LangChain / LlamaIndex 整合好 +- [**LangSmith**](https://docs.langchain.com/langsmith) — LangChain 官方 eval + tracing、closed-source SaaS + +**怎麼開始**：跑完 動手練習 4（完整 RAG pipeline）後、加上 ragas eval、measure 一次 baseline——再回頭調 chunk / embedding / top-k、看數字怎麼動。**沒這一步、調參全是猜。** + +→ Stage 7 Eval 接續本節、把 eval 擴到 multi-agent / full harness。 + +## 🛠 動手練習（基礎 illustrative 練習） + +### 練習 1：Embeddings +把 100 個句子做 embedding，找出某個 query 的最近鄰。理解 vector 之間的距離意義。 + +### 練習 2：Vector DB +把 embedding 存進 Chroma，做語意 query。比對「跟 keyword search 差在哪」。 + +### 練習 3：Chunking 對照 +拿同一份文件做三種切法：固定長度、段落切法、heading-aware 切法。用 5 個真實問題比較 top-k 結果，記錄哪種切法比較容易撈到正確上下文。 + +### 練習 4：完整 RAG 流水線 +把一份 PDF 切塊 → embed → 取 top-k → 生成回答。這是大多數 RAG 應用的基本骨架。 + +### 練習 5：Long-term Memory +讓 agent 在多輪對話之間記得事情。可以用 `mem0` 或自己用 vector store 接。 + +## 🎯 常用 Memory / RAG 工具推薦（按用途分類） + +不知道從哪裡開始挑工具？下面是 2025 後段業界常用搭配——**挑入口看「場景」、想深入點連結看 repo**： + +| 場景 | 推薦工具 | 為什麼 | +|---|---|---| +| **第一次跑 RAG**（最快上手）| [Chroma](https://github.com/chroma-core/chroma) + [LlamaIndex](https://github.com/run-llama/llama_index) | local-first、零 ops、quickstart 友善。Stage 6 練習預設 | +| **企業級 RAG framework**（LangChain / LlamaIndex 之外第 3 選擇）| [Haystack (deepset)](https://github.com/deepset-ai/haystack) ★ 25.2k Apache-2.0 | deepset 開源、production-oriented orchestration、enterprise NLP 場景成熟 | +| **agent 長期記憶**（見 3 主流 memory layer）| [agentmemory](https://github.com/rohitg00/agentmemory) / [mem0](https://github.com/mem0ai/mem0) / [Letta](https://github.com/letta-ai/letta) / [Zep](https://github.com/getzep/zep) / [LangMem](https://github.com/langchain-ai/langmem) | 詳見上方 5 個主流 可上線使用的 memory layer 區塊 | +| **RAG / Memory eval**（必裝、見 RAG Eval）| [ragas](https://github.com/explodinggradients/ragas) ★ 13.9k | RAG 評估標準工具、faithfulness / context recall / answer relevance 8+ metric | +| **production scale RAG**（百萬 doc）| [Qdrant](https://github.com/qdrant/qdrant) + LlamaIndex | Rust 寫的 vector DB、scale 大時比 Chroma 快 | +| **已有 Postgres 的環境** | [pgvector](https://github.com/pgvector/pgvector) | Postgres 擴充、SQL + vector 一起、運維最簡 | +| **企業級 RAG + Web UI** | [RAGFlow](https://github.com/infiniflow/ragflow) | document parsing 強（含 OCR / 表格 / layout）、企業場景、含 Web UI | +| **中文 RAG 範本** | [Langchain-Chatchat](https://github.com/chatchat-space/Langchain-Chatchat) | 中文圈最完整、本機 LLM 整合好（ChatGLM / Qwen / Llama）| +| **進階：Contextual Retrieval** | [Anthropic cookbook](https://platform.claude.com/cookbook/capabilities-contextual-embeddings-guide) | Claude 搭配 prompt caching 的 contextual chunking（**詳見上方 進階 RAG 技巧**） | +| **進階：knowledge graph 推理** | [LightRAG](https://github.com/HKUDS/LightRAG) / [Microsoft GraphRAG](https://github.com/microsoft/graphrag) | knowledge graph + RAG、entity-relation 推理（**詳見上方 進階 RAG 技巧**） | +| **跨主題 tutorial 集** | [ai-engineering-hub](https://github.com/patchy631/ai-engineering-hub) | RAG + agent 教學 collection、Jupyter notebook 形式 | + +**建議入手順序**： +1. 第一個必裝：**Chroma + LlamaIndex**（跑 Stage 6 練習） +2. agent 要記事：加 **mem0**（最簡單的 memory layer） +3. 開始 production-scale：換成 **Qdrant** 或 **pgvector** +4. 想升級到進階 RAG：看上方 進階 RAG 技巧 三個 subsection + +## 🎯 精選 Projects（範本 / spec / 範例 collection） + +按用途分類、17 個項目一張表搞定。**挑入口看「適合誰」、想深入點連結看 repo**。 + +| 分類 | Project | ⭐ | 適合誰 | 為什麼推薦 / 備註 | +|---|---|---|---|---| +| **RAG framework**
（完整流水線） | [LlamaIndex](https://github.com/run-llama/llama_index) | ⭐⭐⭐⭐⭐ | 以文件為主的應用 | 以 RAG 為核心、document loader / chunking / retrieval / query engine 一條龍。★ 49k+ | +| | [infiniflow/ragflow](https://github.com/infiniflow/ragflow) | ⭐⭐⭐⭐⭐ | 要把 RAG 真的 ship 給非開發者用 | production 等級 RAG engine、深度文件理解（layout / 表格 / OCR）+ hybrid retrieval + agent loop + Web UI。★ 79k+、Apache-2.0 | +| | [HKUDS/LightRAG](https://github.com/HKUDS/LightRAG) | ⭐⭐⭐⭐ | 想看研究級 graph + long-context memory 方法 | graph + vector hybrid retrieval + summarization-based memory、EMNLP 2025 paper-backed。★ 34k+、MIT。研究風格 codebase | +| **Vector DB**
（local-first） | [Chroma](https://github.com/chroma-core/chroma) | ⭐⭐⭐⭐⭐ | 練習 2 / 4、最容易上手的 vector DB | 開源 embedding 資料庫、本機跑、in-memory / SQLite 後端、零 ops。★ 27k+、Apache-2.0。**安裝**：`pip install chromadb` | +| **Vector DB**
（production scale） | [Qdrant](https://github.com/qdrant/qdrant) | ⭐⭐⭐⭐⭐ | Chroma 跟不上時、需要 production scale | Rust 寫的 vector DB、有雲端版跟自架版。★ 31k+ | +| **Vector DB**
（hybrid） | [Weaviate](https://github.com/weaviate/weaviate) | ⭐⭐⭐⭐ | production 部署 + schema 約束 | 內建模組（text2vec / generative / classification）、schema 驅動、內建 BM25 + vector hybrid。★ 16k+ | +| **Vector DB**
（已有 Postgres） | [pgvector](https://github.com/pgvector/pgvector) | ⭐⭐⭐⭐ | 原本就在用 Postgres 的團隊 | Postgres 擴充、SQL + vector 同一個 DB、運維最簡。★ 21k+ | +| **Vector DB**
（跑在 app 內） | [lancedb/lancedb](https://github.com/lancedb/lancedb) | ⭐⭐⭐⭐ | 想要 vector DB 直接內建、不想另跑 server | 直接跑在你 app 裡的 vector DB（不用另開 server）、能處理文字 + 圖片、關鍵字 + 向量一起搜。★ 10k+、Apache-2.0 | +| **Memory framework**
（auto fact extraction） | [mem0ai/mem0](https://github.com/mem0ai/mem0) | ⭐⭐⭐⭐⭐ | 個人助理 / chatbot 需要 user-level memory | 自我精煉 memory 層、跨 session 儲存事實。★ 59k+ | +| **Memory framework**
（OS-paging） | [Letta（前身 MemGPT）](https://github.com/letta-ai/letta) | ⭐⭐⭐⭐ | context 要跑很久的 agent（以月為單位） | 階層式 memory（working / archival）、OS-paging 概念。★ 22k+ | +| **Memory（in-framework）** | [LangChain — Memory](https://python.langchain.com/docs/concepts/memory/) | ⭐⭐⭐ | 已用 LangChain | 4 種 memory 抽象（buffer / summary / vectorstore-backed / entity）| +| **進階 RAG 技巧** | [Anthropic — Contextual Retrieval cookbook](https://platform.claude.com/cookbook/capabilities-contextual-embeddings-guide) | ⭐⭐⭐⭐⭐ | 跑完基本 RAG 想升級 | Claude 搭配 prompt caching 的 contextual chunking、含完整端到端範例 | +| **中文 RAG 樣板** | [chatchat-space/Langchain-Chatchat](https://github.com/chatchat-space/Langchain-Chatchat) | ⭐⭐⭐⭐ | 中文知識庫 / RAG 應用 | 中文社群最廣泛使用、可離線部署、中文預設好、支援 ChatGLM / Qwen / Llama / Ollama。★ 38k+、Apache-2.0。⚠️ 最後更新 2025-11（邊緣）| +| **教材合集** | [patchy631/ai-engineering-hub](https://github.com/patchy631/ai-engineering-hub) | ⭐⭐⭐⭐ | 想看「同概念在不同情境怎麼實作」 | 主題式 LLM / RAG / agent tutorial 集、Jupyter notebook、跨多個 stage 都用得上。★ 34k+、MIT | +| **Production AI assistant**
（學 ship RAG 的 reference）| [onyx](https://github.com/onyx-dot-app/onyx)（前身 Danswer）| ⭐⭐⭐⭐⭐ | 想看「RAG-driven AI assistant 怎麼 production 化」 | 開源企業級 AI assistant、跨 LLM 支援、含完整 ingest / retrieval / chat / admin。★ 29.4k、active 維護 | +| **RAG cookbook**
（30+ 技巧範例）| [NirDiamant/RAG_Techniques](https://github.com/NirDiamant/RAG_Techniques) | ⭐⭐⭐⭐⭐ | 跑完基本 RAG、想看各種變體 | 大型 RAG 技巧 cookbook、含 Self-RAG / HyDE / Multi-Query / Adaptive 等 30+ Jupyter notebook 範例 | +| **DSPy**
（programming not prompting）| [stanfordnlp/dspy](https://github.com/stanfordnlp/dspy) | ⭐⭐⭐⭐⭐ | 用 LLM 一段時間、想自動 optimize prompt + chain | Stanford NLP group、★ 34.4k MIT、Path 3 paradigm（詳見 進階 RAG DSPy） | +| **RAG / Memory Eval**
（必裝）| [explodinggradients/ragas](https://github.com/explodinggradients/ragas) | ⭐⭐⭐⭐⭐ | 跑完 Stage 6 練習 4、想 measure retrieval 精度 | RAG eval 標準工具、8+ metric、reference-free + reference-based。★ 13.9k Apache-2.0 | + + +## ✅ 進入 Stage 7 前的自我檢查 + +你能不能： + +- [ ] 寫一條 50 行的 RAG 流水線（load → chunk → embed → store → query → answer） +- [ ] 解釋為什麼天真的切塊在長文件上會失敗 +- [ ] 針對 API 文件、PDF、表格設計不同的 chunking 策略 +- [ ] 在某個規模下，能在 Chroma、Qdrant、pgvector 之間做出選擇 +- [ ] 區分「給 agent memory」跟「用 RAG」這兩件事 +- [ ] 解釋 RAG 跟 Memory 各補哪段（從上面 從 RAG 到 Memory 表） + +如果都可以 → 前往 [Stage 7 — Multi-Agent · 進階應用](07-multi-agent-production.md)。 diff --git a/stages/06-memory-rag.zh-Hans.md b/stages/06-memory-rag.zh-Hans.md new file mode 100644 index 0000000..0cdaa6a --- /dev/null +++ b/stages/06-memory-rag.zh-Hans.md @@ -0,0 +1,760 @@ +# Stage 6 — 上下文管理（Context Engineering）：RAG 与 Memory + +> [繁體中文](./06-memory-rag.md) | **简体中文** | [English](./06-memory-rag.en.md) + +⏱ **时间估算**：2 周（约 10 小时） + +> 💡 这个 stage 用语密度高（**RAG / 向量数据库 / embedding / chunking / hybrid search / reranking...**）→ 不熟先翻 [`resources/glossary.md` 3](../resources/glossary.md#3-memory--retrieval--rag)。 +> +> 📋 **本章组成**：定位 → 入口 → **RAG 主轴**（基础 + 进阶 + DSPy + Eval）→ **Bridge** → **Memory 主轴**（3 pattern + trio + 进阶）→ Chunking → Reflexion / Reasoning → 练习 → Projects +> +> 🔑 **关键术语**：见 [`resources/glossary.md` 3](../resources/glossary.md#3-memory--retrieval--rag)（memory / RAG / embedding / chunking / reranking） + +本 stage 的核心不是“多背一点术语”，而是理解 agent 如何管理 context。 + +- **RAG 解决的是**：现在要从外部知识库查哪些资料？ +- **Memory 解决的是**：agent 应该跨对话、跨 session、跨任务记住什么？ +- **Context Engineering 是更上层的问题**：每次 LLM call 前，该把哪些信息组进 prompt，才能让模型在有限 context window 里做出正确决策？ + +→ 这会直接接到 Stage 7 的三层工程分工：**Prompt = 单次怎么问 / Context = 这次该给哪些信息 / Harness = 整个 agent system 怎么跑起来**。本 stage 是中间那层。 + +### Agent 需要的两种 context 能力 + +1. **Retrieval**：从外部知识库找出和当前任务相关的资料。 +2. **Memory**：保留跨对话、跨 session、跨任务的状态、偏好与经验。 + +**RAG（Retrieval-Augmented Generation）** 是当前最常见的 retrieval 架构；**Memory** 负责让 agent 记住用户、任务历史与自己的过去经验。这一章会把两者分开讲，避免把“查资料”和“记事情”混在一起。 + +### 先把名词切开：Retrieval / RAG / Vector Store / Memory 不是同一件事 + +| 名词 | 不要混淆成 | 白话解释 | +|---|---|---| +| **Retrieval** | RAG 全部 | 找资料这个动作 | +| **RAG** | Vector DB | retrieve + generate 的完整流程 | +| **Embedding** | Memory | 把文本转成向量，方便做相似度搜索 | +| **Vector store** | RAG | 存储与搜索 embedding 的地方 | +| **Chunking** | Retrieval 本身 | 把文档切成适合被搜索的片段 | +| **Memory** | RAG | agent 对用户、任务与过去经验的持久管理 | + +## 🎯 Context Engineering 是什么（先定位） + +**一句话**：Context Engineering = 决定 **每次调用 LLM 时，要把哪些信息塞进它看得到的窗口（context window）**。 + +重点不是“开了几次对话”，而是“**每次对话里塞了什么**”。Karpathy 2025-06 的[原推文](https://x.com/karpathy/status/1937902205765607626)说得最准确：这是一门把 **刚好对下一步有用的信息** 填进窗口的精细艺术。 + +📺 **视觉学习**：[李宏毅 2025 第 2 讲 — Context Engineering：AI Agent 背后的关键技术](https://www.youtube.com/watch?v=lVdajtNpaGI)（NTU 生成式人工智能与机器学习导论 2025） + +### 在三层 stack 里的位置 + +![Prompt → Context → Harness 三层工程 stack](../resources/diagrams/prompt-context-harness-stack.zh-Hans.png) + +完整对照见 [Stage 2](02-prompt-engineering.zh-Hans.md)。 + +### 本 stage 处理 4 个 sub-problem 中的 2 个（Lance Martin 2025 框架） + +| Sub-problem | 解决什么 | 具体例子 | 本 stage cover？ | +|---|---|---|---| +| **Select** | 要把 **哪些** 外部信息捞进窗口 | user 问“我家附近哪间 cafe 好吃”→ 从 Yelp DB 捞 3 家评分高的 → 塞进 prompt | ✅ 主轴（RAG / vector search / GraphRAG） | +| **Write** | 要把 **哪些** 互动 / 教训写进长期记忆 | user 上周说“我吃纯素”→ 写进 memory；这周又问餐厅推荐时，retrieve 出来避免推肉食 | ✅ 主轴（memory layers） | +| **Compress** | 对话太长怎么压 | 50 轮对话超过 200k token → 自动摘要前 40 轮、保留最后 10 轮原文 | ⚠️ 部分（这里 + Stage 7 Harness `context manager`） | +| **Isolate** | 多 agent 各自窗口怎么分 | supervisor 看全局、worker 只看自己那段，彼此不串扰 | ❌ Stage 7 multi-agent 处理 | + +### 四个常被混淆的概念 + +| 名词 | 是什么（抽象 / 具体） | 示例工具 | +|---|---|---| +| **Memory** | agent 跨对话 / 跨 session 记事情的 **能力**（抽象概念） | LangChain ConversationBufferMemory / mem0 / Letta | +| **Embedding** | 把文本转成 N 维 **向量**，让相似度可计算（数据转换） | `sentence-transformers` 跑出 768 维向量 / OpenAI ada-002 | +| **Vector DB** | 存 + 查 embedding 的 **存储层**（基础设施） | Chroma / Qdrant / Weaviate / pgvector | +| **RAG** | “retrieve 相关片段 → 塞进 prompt → 生成”这个 **pattern**（架构模式） | LlamaIndex / LangChain RAG chain | + +→ **核心区分**：Memory 是 **能力**，Embedding 是 **数据转换**，Vector DB 是 **存储**，RAG 是 **架构 pattern**。这四个概念常被混用，但实际属于不同层级。 + +### RAG vs Long Context vs Fine-tuning — 何时用什么 + +让 LLM 用上你的私有 / 领域数据，主要有 3 种做法。**本 stage 教 RAG**，但你也要知道什么时候不该用： + +| 选择 | 适合 | 不适合 | 成本 | +|---|---|---|---| +| **RAG**
（外部 retrieve） | 大型 / 变化快 / 私有知识库、需要 citation 的场景 | 推理要整份文档一起看、需要跨文档 multi-hop reasoning | 每次 query 多一次 vector search latency | +| **Long Context**
（直接塞进 prompt） | 200k token 以内的中型文档、一次性查询、需要 cross-doc reasoning | 知识库很大 / 经常变化 / 想保留 citation | 每次 query 都要烧大量 input token，即使有 prompt caching 也是如此 | +| **Fine-tuning**
（改 model weights） | 风格 / 格式统一、特定领域语言（医疗、法律、代码） | 知识会变化、需要 citation、不想训练模型 | 训练成本 + 维护成本 + 模型 lock-in | + +→ **怎么选**：先试 RAG（成本最低、变化最容易跟上）→ RAG 不够再考虑 Long Context → 两者都不行再考虑 Fine-tuning。**进 Stage 7 学 fine-tune deploy。** + +## 📌 学习目标 + +- 建一条基础 RAG 流水线（chunk → embed → store → retrieve → generate） +- 看出 RAG 该用在哪些地方、又不该用在哪些地方 +- 区分 working memory、long-term memory、episodic memory、semantic memory、procedural memory +- 理解 vector embedding 与相似度搜索 +- 知道进阶 RAG（GraphRAG / Contextual Retrieval / Hybrid Search）何时加、何时不加 + +## 🚪 进入条件 + +你应该已经： +- 完成 Stage 3（会写 tool use、会调用 LLM API、能看懂 ReAct loop）—— **硬性技术前置** +- 走过 Stage 4（agent frameworks）+ Stage 5（Claude Code 生态）—— curriculum 主线是 **3 → 4 → 5 → 6**（见 [README 学习地图](../README.zh-Hans.md#-学习地图两条学习路径)）；非硬性技术前置，但 RAG / memory 常跟 framework + Claude Code memory 机制搭配、照顺序走过理解更完整，且 [Stage 7](07-multi-agent-production.md) 预期你已完成 4 + 5 + 6 +- 能够运行 Python `pip install` 来安装 SDK（后续练习会用到 `chromadb`、`sentence-transformers` 等） +- 熟悉 list / dict / generator 等基础 Python 结构 + +如果没有达到，请回看 [Stage 3](03-tool-use-and-hello-agent.md) 或 [Stage 0 环境设置](00-foundations.zh-Hans.md#何时可以跳过这个阶段)。 + +## 📚 必读材料 + +1. [**LlamaIndex — RAG concepts**](https://docs.llamaindex.ai/en/stable/getting_started/concepts/) — 最清晰的入门。 +2. [**LangChain — RAG tutorial**](https://python.langchain.com/docs/tutorials/rag/) — 实战操作。 +3. [**Pinecone — Learning Center**](https://www.pinecone.io/learn/) — Vector DB 基础。 +4. [**Anthropic — Contextual Retrieval**](https://www.anthropic.com/news/contextual-retrieval) — Anthropic 结合 prompt caching 的 RAG 写法。 +5. [**LangChain — Text Splitters**](https://docs.langchain.com/oss/python/integrations/splitters/index) — Chunking 策略入门。 + +> 🙏 **Memory 章节特别推荐 [`datawhalechina/hello-agents`](https://github.com/datawhalechina/hello-agents)**: 本阶段探讨 memory 的概念和初步实现，需要 **chapter-length 的深入版本**请参考 hello-agents 的对应章节——short-term / long-term memory 的差异、context engineering 的动态组装、session 持久化、forgetting strategy 都讲得最完整。本阶段是路线图，那边是深度教材。 + +## 🧭 单元指引（渐进式流程） + +本章按 **RAG 先学、Memory 后学** 的顺序进行——RAG 是 context engineering 最基础、最常用的工具，而 Memory 是 agent 跨对话/跨 session 的能力；先跑通 RAG pipeline，再引入 Memory 设计，最后回头看 Chunking 的细节。 + +**推荐阅读顺序**: + +1. **🌐 RAG 基础流水线**（下一节）— 建立心智模型。 +2. **🚀 进阶 RAG 技巧** — GraphRAG / Contextual Retrieval / Hybrid Search 等生产环境升级。 +3. **🌉 从 RAG 到 Memory** — 为什么 RAG 还不够，Memory 弥补了哪些部分。 +4. **🧠 Memory 设计** — 短期 vs 长期、3 种模式、CoALA 框架。 +5. **🧩 Chunking 细节** — 深入探讨 RAG 和 Memory 都会用到的技术。 + +阅读本章时，可以思考：RAG 不适用于哪些应用场景？哪些场景适合 RAG，但基础 RAG 表现不够好？这将引导你了解后续的 GraphRAG / Self-RAG / RAPTOR 等进阶技术。 + +## 🌐 RAG 基础流水线 + +**RAG（Retrieval-Augmented Generation）**= “retrieve 相关片段 → 塞进 prompt → 生成” 这个模式。可以把它想象成在为 agent 构建一个图书馆——你需要先把书放好、分类好，后续查询资料时，才能又快又精准。 + +**最基础的 RAG 分成两条流水线**： + +- **数据预处理（Ingest 一次）**：ingest → chunk → embed → store（index）。这一步是在构建可检索的知识库。 +- **检索生成（每次 Query）**：retrieve → generate。这一步是在用户提问时，找出相关内容，然后交给 LLM 生成回答。 + +![RAG Pipeline Overview](../resources/diagrams/rag-pipeline-overview.jpg) + +图中的 RAG Fusion、query rewrite 等属于进阶检索技巧。**第一次学习 RAG 时，先理解主线流程即可**。 + +**5 个步骤解读**： + +| Step | 做什么 | 在哪条 Pipeline | 技术细节在哪里 | +|---|---|---|---| +| **1. Ingest** | 加载数据（PDF / web / DB） | 预处理 | LlamaIndex / LangChain 各自的 loader | +| **2. Chunk** | 将文档切分成小块（500-2000 token / chunk） | 预处理 | 见后文 🧩 Chunking 细节（先阅读 RAG / Memory 主体章节，技术深入留到后面） | +| **3. Embed** | 将每个 chunk 转换为 N 维向量 | 预处理 | `sentence-transformers` / OpenAI ada-002 | +| **4. Store** | 将向量 + 元数据存储进 Vector DB | 预处理 | Chroma / Qdrant / pgvector | +| **5. Retrieve + Generate** | 对 Query 进行 embedding → top-k 语义搜索 → 拼接到 prompt → LLM 生成回答 | 每次 Query | 通用 LLM API | + +以上只是最小骨架。**最常踩的 3 个坑**： + +- **Chunk 太大 / 太小**：太大，检索到的 chunk 里可能只有一句相关，其他是杂讯；太小，会失去上下文（见 Chunking 细节）。 +- **Embedding model 选错**：中文文档用英文 model，检索精度直接掉一半。 +- **top-k 设太大 / 太小**：太小，可能漏掉 relevant chunk；太大，杂讯高 / token 消耗大。 + +> 📚 **想看更多 RAG 踩坑指南 + 解法**：[NirDiamant/RAG_Techniques](https://github.com/NirDiamant/RAG_Techniques) ★ 大型 Production RAG Cookbook，包含 30+ 技巧 + Jupyter notebook 示例。 + +> 📄 **RAG 真正常挂的两个地方，别只顾 chunking**：(1) **解析（ingest）**——PDF→干净 markdown 是 garbage-in 的源头：[docling-project/docling](https://github.com/docling-project/docling)（★61k、MIT）、[opendatalab/MinerU](https://github.com/opendatalab/MinerU)（中文 / 科学 PDF 强，**AGPL** 注意授权）、[microsoft/markitdown](https://github.com/microsoft/markitdown)（★150k+、MIT）。(2) **选嵌入模型**——第一个检索质量决策，别瞎挑：看 [MTEB leaderboard](https://huggingface.co/spaces/mteb/leaderboard)，中文 / 多语常用 [BGE-M3](https://github.com/FlagOpen/FlagEmbedding)（★12k、MIT）。 + +跑完基础骨架后，先跑 动手练习 1-4（embeddings / vector DB / chunking / 完整 pipeline）建立手感，再进入下一节 进阶 RAG 技巧。 + +## 🚀 进阶 RAG 技巧（跑完基础 RAG 之后再看） + +下面六个 subsection 是 2024-2026 年 production RAG 最常添加的优化手段，按“添加到 Pipeline 的哪一层”分组： +- **Retrieve 之后** —— GraphRAG / Contextual Retrieval / Hybrid Search & Reranking +- **Retrieve 之前**（Query 改写）—— Query Transformations +- **Retrieve 期间**（控制流程）—— Adaptive / Agentic RAG +- **索引结构** —— RAPTOR +- **2024-2026 概览** —— 其他 17 个值得了解的技巧 + +**先跑完上述 RAG 基础版本，建立基准后，再回头看这里**——否则你可能会在没有基准的情况下调参，永远不知道是哪个改动带来了提升。 + +| 技巧 | 解决什么问题 | 添加到 Pipeline 的哪一层 | 成本 | +|---|---|---|---| +| **GraphRAG** | vanilla RAG 无法进行 multi-hop / 跨文档的 entity-relation 推理 | Retrieve 前（构建 graph）+ Retrieve 时（graph traversal）| 高（需要构建 KG，LLM 抽取 entity 的 token 成本高）| +| **Contextual Retrieval** | Chunk 丢失了原始文档的上下文，检索到错误的片段 | Chunk 之后 / Embed 之前（添加 contextual header）| 中等（一次性 ingest 成本，搭配 prompt caching 后成本降低 90%）| +| **Hybrid Search & Reranking** | 纯 vector 搜索会遗漏字面匹配，top-k 结果杂讯多 | Retrieve 期间（结合 BM25）+ Retrieve 之后（cross-encoder rerank）| 低（成熟工具可直接集成）| + +### 🔗 GraphRAG — 知识图谱 + RAG + +**心智模型**: vanilla RAG 将文档切分成 chunk，依赖 embedding 相似度进行检索——但**它不知道哪些 entity 是同一个东西，以及 entity 之间有什么关系**。GraphRAG 在 ingest 阶段先用 LLM 将文档抽取成 **(entity, relation, entity)** 三元组来构建知识图谱，检索时除了向量比对，还会进行 graph traversal 来查找“相关 entity 的相关 entity”。 + +**何时使用**: +- 任务需要 **multi-hop reasoning**（需要 A → B → C 才能回答）。 +- 跨多个文档，实体互相引用（公司财报、论文引用、调查报告、法律案例）。 +- 问题形如“X 影响了什么 Y，Y 又连接到哪些 Z”——vanilla RAG 通常只能检索到与 X 相关的文档片段。 + +**何时不使用**: +- 文档之间没有实体-关系链接（纯 FAQ、产品手册各自独立）。 +- 知识库规模小（< 1k chunks）——vanilla RAG 已足够。 +- 预算紧张——构建 KG 的 token 成本可能是普通 RAG 的 10-50 倍。 + +**代表性框架**: +- [**HKUDS/LightRAG**](https://github.com/HKUDS/LightRAG) ★ **35.1k** MIT EMNLP 2025 — 目前社区最热门的选择，轻量级，KG + vector hybrid，成本低于 Microsoft 的版本。 +- [**Microsoft GraphRAG**](https://github.com/microsoft/graphrag) — 原始参考实现，Apache-2.0 许可，包含社区检测功能。 +- [**gusye1234/nano-graphrag**](https://github.com/gusye1234/nano-graphrag) — < 1000 行代码的最小实现，适合先理解原理。 + +**论文**: [**From Local to Global: A Graph RAG Approach to Query-Focused Summarization (Edge et al. 2024)**](https://arxiv.org/abs/2404.16130) — Microsoft GraphRAG 的原始论文，解释了 community summarization 如何解决全局查询问题。 + +### 🪶 Contextual Retrieval — Anthropic 的 Prompt Caching 解决方案 + +**心智模型**: vanilla chunk 会丢失原始文档的上下文——一个 "Q3 revenue grew 15%" 的 chunk 被提取出来后，你不知道是**哪家公司**、**哪一年**的 Q3。Anthropic 在 2024 年提出：在 ingest 阶段，使用 LLM 为每个 chunk 编写一段 50-100 token 的**上下文头部（contextual header）**（例如：“This chunk is from ACME Corp 2024 Q3 earnings, discussing the cloud segment...”），然后将其拼接到 chunk 前面再进行 embedding。结合 **prompt caching**，可以将“整个文档 + 每个 chunk”的 prompt 只计费一次，后续所有 chunk 共用缓存。 + +**何时使用**: +- Chunk 的字面意思与其原始文档主题相差较远（如：财务报告、研究论文、长篇叙事文档）。 +- 你愿意支付一次性的 ingest 成本，以换取更高的检索精度。 +- 你正在使用 Claude 或计划使用 prompt caching（其他模型也能运行此方法，但没有缓存折扣）。 + +**何时不使用**: +- Chunk 本身是自包含的（如：FAQ、产品介绍页、定义条目）。 +- 知识库频繁变动（每次更改都需要重新 ingest）。 +- 预算极其紧张——即使有缓存折扣，ingest 成本仍比 vanilla 高。 + +**为何能节省 90% 的成本**: Anthropic 的报告显示，prompt caching 将成本降低到约 1/10，因为它将整个文档视为缓存的前缀，每个 chunk 只发送差异部分。但**这仅节省了 ingest 成本，并未节省 retrieve 阶段的成本**。 + +**代表性实现**: +- [**Anthropic — Contextual Retrieval Blog**](https://www.anthropic.com/news/contextual-retrieval) ⭐ — 官方说明 + benchmark（失败的检索率从 5.7% 降至 1.9%）。 +- [**Anthropic Cookbook**](https://platform.claude.com/cookbook/capabilities-contextual-embeddings-guide) — 端到端的 Jupyter notebook，包含 prompt 模板。 + +**配套技巧**: Anthropic 的同一篇博文还建议结合 **Contextual BM25**（将上下文 chunk 与 vector + BM25 同时喂入）+ **reranking**——这正好引出了下一节 Hybrid Search & Reranking。 + +### 🎯 Hybrid Search & Reranking — Production RAG 的两个常见强化组件 + +**心智模型**: +- **Hybrid Search** = 结合了 vector 相似度（语义匹配）和 BM25 / keyword 搜索（字面匹配），使用 [RRF (Reciprocal Rank Fusion)](https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf) 等方法融合分数。解决了纯 vector 搜索“查询与 chunk 同义但用词不同导致未检索到”以及“人名 / 编号 / 罕见词的语义 embedding 较弱”的双重盲点。 +- **Reranking** = 第一阶段检索 **top-50** 个 chunk（优先考虑召回率，宽松地捞取）→ 然后使用 **cross-encoder reranker** 重新评分并排序出 **top-5** 个（优先考虑精确率，精细筛选）。Cross-encoder（将 query + chunk 一起送入模型）比 bi-encoder（query / chunk 分开 embedding）精度高很多，但速度较慢，因此只在第二阶段使用。 + +**为什么是“必加优化”**: Production RAG 的评估几乎一致表明，添加 hybrid search + reranker 后，recall@5 通常从 70% 上下提升到 85-90%，边际成本低，且工具成熟。**这是性价比最高的两个改动**。 + +**何时使用**: +- Production RAG（非 demo / 练习）。 +- 查询包含人名、产品编号、技术术语、罕见词（纯 vector 搜索容易漏）。 +- 预算允许每查询增加 100-300ms 的延迟。 + +**何时可以暂缓**: +- 实验阶段 / MVP（先跑通 vanilla RAG）。 +- 预算极紧 / 对延迟极其敏感（reranker 需要额外的模型调用）。 + +**代表性工具**: +- **Hybrid Search**: [Weaviate](https://github.com/weaviate/weaviate)（内置 BM25 + vector + RRF）/ [Qdrant](https://github.com/qdrant/qdrant)（支持 sparse + dense vector）/ pgvector + Postgres FTS。 +- **Rerankers**: [Cohere Rerank API](https://docs.cohere.com/docs/rerank-overview)（商业版，最常用）/ [BGE Reranker](https://huggingface.co/BAAI/bge-reranker-large)（开源版，HuggingFace，中文表现较好）/ [Jina Reranker](https://jina.ai/reranker)。 +- **框架内置**: LlamaIndex 的 `SentenceTransformerRerank` / LangChain 的 `ContextualCompressionRetriever`。 + +**论文 / 入门**: +- [**Pinecone — Rerankers and Two-Stage Retrieval**](https://www.pinecone.com/learn/series/rag/rerankers/) — 对 reranker 心智模型讲解最清晰。 +- [**Anthropic — Contextual Retrieval**](https://www.anthropic.com/news/contextual-retrieval)（上面已列）— 同时演示了 hybrid + reranker，并包含 benchmark。 + +### Query Transformations — HyDE / Multi-Query / RAG Fusion + +**心智模型**: vanilla RAG 将用户查询直接 embedding 后进行检索——但查询的用词、风格或抽象层级经常与文档差异很大（例如：用户问“我胃痛怎么办”，文档写“上腹部疼痛的鉴别诊断”）。Query transformations 在 retrieve **之前**先重写查询，生成更接近文档形式的版本。 + +**3 个代表性技巧**: + +| 技巧 | 如何改写 | 何时使用 | +|---|---|---| +| **HyDE**（Hypothetical Document Embeddings）| 先让 LLM 为查询生成一个“假设答案”，然后使用该答案的 embedding 进行检索 | 当查询的用词/风格与文档差异很大时 | +| **Multi-Query** | LLM 将查询改写成 N 个变体，分别进行检索，然后合并去重 | 当查询过短/模糊/存在多义时 | +| **RAG Fusion** | Multi-Query + RRF 融合 N 个检索结果，以获得更稳定的排名 | 同 Multi-Query，旨在获得更稳定的排名 | + +**何时不使用**: 当查询已经很长且结构化时（如 RAG over code，用户直接粘贴错误堆栈信息）——改写反而可能引入杂讯。 + +**论文 / 实现**: +- [**HyDE (Gao et al. 2022)**](https://arxiv.org/abs/2212.10496) — 原始论文。 +- [**RAG Fusion (Raudaschl 2023)**](https://github.com/Raudaschl/rag-fusion) — Multi-Query + RRF 的参考实现。 +- LangChain 内置 `MultiQueryRetriever` / LlamaIndex 内置 `HyDEQueryTransform`。 + +### 🔁 Adaptive / Agentic RAG — Self-RAG / CRAG / Adaptive RAG（2024 主轴） + +**心智模型**: 上述所有 RAG 技巧都假设了一个固定的 Pipeline：“query → retrieve → generate”。Adaptive / Agentic RAG 将这个 Pipeline 变成了一个**具有判断能力的 agent loop**——LLM 自己决定是否需要 retrieve，评估 retrieve 的质量，并在必要时调整查询。**这是 2024 年 RAG 研究的主轴**。 + +| 技巧 | 如何自我修正 | 论文 | +|---|---|---| +| **Self-RAG** | 训练 LLM 输出 `[Retrieve]` token 来决定是否检索，检索后输出 `[IsRel]/[IsSup]/[IsUse]` 来评分每个片段 | [Asai et al. ICLR 2024](https://arxiv.org/abs/2310.11511) | +| **CRAG** (Corrective RAG) | 检索评估器对结果评分；高置信度直接使用，低置信度回退到 web search，中等置信度触发查询重写 | [Yan et al. 2024](https://arxiv.org/abs/2401.15884) | +| **Adaptive RAG** | 分类器首先判断查询的复杂性，然后路由到“不检索 / 单步 / 多步”三种策略之一 | [Jeong et al. NAACL 2024](https://arxiv.org/abs/2403.14403) | + +**为什么这是 2024 年的主轴**: 固定 Pipeline 在简单查询（例如：“东京的首都是哪里？”这种不需要检索）和复杂查询（multi-hop、cross-doc）两者上都存在劣势。让 LLM 自己决定路由，可以同时解决这两种极端情况。 + +**何时使用**: Production RAG，查询类型分布广泛（从事实题到推理题都有），愿意付出 1.5-3 倍的延迟来换取更高的准确性。 +**何时不使用**: 查询类型单一 / 预算限制 / 延迟极其敏感。 + +**实现**: LangGraph 提供了官方的 [Self-RAG cookbook](https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_self_rag/) + [CRAG cookbook](https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_crag/) + [Adaptive RAG cookbook](https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_adaptive_rag/)，可以直接套用。 + +### 🌳 RAPTOR — 阶层式递归检索（ICLR 2024） + +**心智模型**: vanilla chunking 将文档切分成扁平的 chunk——但**整本书的主旨并不存在于任何单个 chunk 中**。RAPTOR 递归地聚类和摘要 chunk，构建一个**多层树**：底层 = 原始 chunk，中层 = 一组相关 chunk 的摘要，顶层 = 全文摘要。检索时可以选择搜索整棵树，或选择特定抽象层级。 + +**为什么有用**: +- 可以检索到**抽象查询**的答案（例如：“这篇论文的主要结论是什么？”——原始 chunk 里可能没有这句话，但顶层摘要里有）。 +- 也能有效检索**细节查询**（底层 chunk 被保留）。 +- 与 GraphRAG 不同——RAPTOR 使用的是**树**（层次化摘要），而 GraphRAG 使用的是**图**（实体-关系）。 + +**何时使用**: 长文档（书籍、论文、报告）需要不同抽象层级的查询，知识库具有连贯的叙事性。 +**何时不使用**: chunk 之间相互独立（如 FAQ），知识库频繁变动（重建树的成本较高）。 + +**论文 / 实现**: +- [**RAPTOR (Sarthi et al. ICLR 2024)**](https://arxiv.org/abs/2401.18059) ⭐ — 原始论文。 +- [**parthsarthi03/raptor**](https://github.com/parthsarthi03/raptor) — 官方参考实现。 +- LlamaIndex 内置 `RAPTOR pack`。 + +### 🧬 DSPy — 不写 Prompt，用程序自动优化（Path 3 范式） + +**心智模型**: 传统 RAG / Agent 需要手动编写 prompt 和 chain。DSPy **消除了 prompt 编写**——你只需定义“签名”（输入/输出类型），编写程序（chain 结构）；DSPy 会用 LLM 编译出最佳的 prompt、few-shot 示例和 retriever 设置。由 Stanford NLP group 于 2024 年提出，并得到 Karpathy 的推广，目前在 production 中越来越受欢迎。 + +**何时使用**: +- 你的 RAG prompt 已经积累了 6 个月，维护困难，想自动优化。 +- 同一程序需要切换不同的 LLM provider（DSPy 会自动重新编译）。 +- Agent 系统有多个步骤，你想跟踪 metrics 和 traces。 + +**何时不使用**: +- 你只有一个 prompt，不需要优化。 +- 你是 LLM 新手，还没摸过 prompting。 + +**代表性仓库**: [**stanfordnlp/dspy**](https://github.com/stanfordnlp/dspy) ★ **34.4k** MIT，Stanford NLP group 官方，积极维护中。 + +**如何集成到 RAG**: DSPy 与本阶段讨论的 RAG 技巧**并不冲突**——你可以将 GraphRAG / Hybrid Search / Reranking 都当作 DSPy 的模块来组装，然后进行编译。它是一个更高层级的 RAG 构建类型系统。 + +→ **与 Path 1 / Path 2 Reasoning 并列**: Path 1 是“手动编写 prompt”，Path 2 是“将 reflection 训练进模型权重”，**DSPy 是 Path 3“程序自动搜索最佳 prompt”**。在 Stage 7 Multi-agent 的进阶场景中尤其好用。 + +### 📊 RAG 进阶技巧概览 — 2025-2026 年的三大主线 ⭐ + +进阶 RAG 的 2025-2026 年演化集中在 **3 大主线**： + +1. **🧠 KG + Memory 融合** — 从扁平的 vector store 走向“结构化、可演化、可联想”的知识表示。代表：[**HippoRAG 2**](https://arxiv.org/abs/2502.14802)（海马体启发、KG + PageRank、跨文档 multi-hop）、A-MEM、KAG。 +2. **🎬 Multimodal RAG** — 从文本检索走向图像 / 视频 / 表格的本地化检索。代表：[**ColPali**](https://arxiv.org/abs/2407.01449)（直接对 PDF 图像进行 embedding，绕过 OCR）、TV-RAG、MegaRAG。 +3. **🤖 Agentic RAG** — Retrieval 从固定 Pipeline 演变为 Agent Loop 内的 Tool（agent 自主决定检索次数与方式）。代表：A-RAG、Self-RAG（已在上面 Adaptive / Agentic RAG 中介绍）。 + +**另外 2 个值得关注的方向**: +- **🛡 RAG 安全** — Corpus poisoning / prompt injection 已成为 production 考量重点。代表：[RAGPart / RAGMask](https://arxiv.org/abs/2512.24268)。 +- **🔧 不再手动编写 Prompt** — 系统自动搜索最佳 prompt + retriever 组合。代表：[**DSPy**](https://github.com/stanfordnlp/dspy)（Stanford 的"programming not prompting"范式，见上方 DSPy 段落）。 + +**5 个值得深入研究的代表作**（快速参考）： + +| 技巧 | 一句话 | 链接 | +|---|---|---| +| **HippoRAG 2** | KG + Personalized PageRank，跨文档 multi-hop，受海马体启发 | [Gutiérrez et al. ICML 2025](https://arxiv.org/abs/2502.14802)、[OSU-NLP-Group/HippoRAG](https://github.com/OSU-NLP-Group/HippoRAG) ⭐ | +| **ColPali** | 直接对 PDF 图像进行 embedding，绕过 OCR，多模态 RAG 入门 | [Faysse et al. 2024](https://arxiv.org/abs/2407.01449) | +| **A-RAG / SoK Agentic RAG** | 将 retrieval 作为 Tool，Agent 自主决定检索次数/方式 | [Ayanami0730/arag](https://github.com/Ayanami0730/arag)、[SoK survey](https://arxiv.org/abs/2603.07379) ⭐ | +| **DSPy** | 不写 Prompt，使用程序 + 签名进行自动优化 | [stanfordnlp/dspy](https://github.com/stanfordnlp/dspy) ★ 34.4k | +| **LightRAG** | Microsoft GraphRAG 的轻量级替代方案，EMNLP 2025 论文 | [HKUDS/LightRAG](https://github.com/HKUDS/LightRAG) ★ 35.1k（已在 GraphRAG 部分介绍） | + +
+📚 完整概览 — 其他 12 个值得了解的进阶 RAG 技巧（展开查看） + +| 技巧 | 一句话 | 年份 / 论文 | +|---|---|---| +| **Sentence-Window Retrieval** | Embedding 句子，检索后返回 +/- N 句的窗口 | LlamaIndex 内置 | +| **Parent-Child / Small-to-Big** | Embedding 小 chunk，检索时返回父 chunk | LangChain `ParentDocumentRetriever` | +| **Multi-Vector Retrieval** | 一个 chunk，多个 embedding（摘要 / 原文 / 假设问题）| LangChain `MultiVectorRetriever` | +| **ColBERT / Post-Interaction Retrieval** | 基于 token 级别的比对而非 pooled embedding | [Khattab & Zaharia 2020](https://arxiv.org/abs/2004.12832)、[RAGatouille](https://github.com/AnswerDotAI/RAGatouille) | +| **LongRAG** | 大 chunk（4k）+ long-context reader，减少检索次数 | [Jiang et al. 2024](https://arxiv.org/abs/2406.15319) | +| **MemoRAG** | Memory model 将 KB 压缩成 latent memory，检索时使用线索触发 | [Qian et al. 2024](https://arxiv.org/abs/2409.05591) | +| **KAG**（Knowledge-Augmented Generation）| 严格 schema KG + 逻辑推理，适用于金融 / 医疗 / 法律场景 | [Liang et al. 2024 (Ant Group)](https://arxiv.org/abs/2409.13731) | +| **MiA-RAG**（Mindscape-Aware）| 先构建文档的高层摘要 mindscape，然后用它来指导检索和回答 | [arXiv:2512.17220](https://arxiv.org/abs/2512.17220) ⭐ 2025-12 | +| **QuCo-RAG**（Quality-Controlled）| 使用 pretraining 统计来判断是否需要检索，罕见实体触发搜索，减少幻觉 | [arXiv:2512.19134](https://arxiv.org/abs/2512.19134) ⭐ 2025-12 | +| **MegaRAG** | 多模态 KG，从长文档中提取实体 + 关系 + 视觉信息，构建层级图 | [arXiv:2512.20626](https://arxiv.org/abs/2512.20626) ⭐ 2025-12 | +| **TV-RAG** | 无需训练即可感知时间的 RAG，对齐长视频的字幕 + 视觉信息 | [arXiv:2512.23483](https://arxiv.org/abs/2512.23483) ⭐ 2025-12 | +| **RAGPart / RAGMask** | 对 RAG corpus poisoning 攻击的轻量级防御 | [arXiv:2512.24268](https://arxiv.org/abs/2512.24268) ⭐ 2025-12 | + +
+ +## 🌉 从 RAG 到 Memory — 为什么 RAG 还不够 + +读到这里，你应该已经掌握了基础 RAG 的运行方式，并了解了几个 production 优化手段。但回头看 Context Engineering 列出的 3 个问题域——你只解决了 **Retrieval**，而 **Memory 管理** 还没涉及。为什么这两件事要分开处理？ + +RAG 解决的是“从**外部知识库**检索相关片段”——但 agent 也需要“**自己** 跨对话 / 跨 session 记住事情”。这两件事并非同一个问题： + +| 维度 | RAG | Memory | +|---|---|---| +| 内容来源 | **外部**（PDF / 文档 / web / DB）| agent **自己的对话 / 经验** | +| 写入时机 | ingest 一次性，后续每次 retrieve | 每轮对话，每次 task 都可能写入 | +| 内容性质 | 偏静态事实、文档知识 | 偏动态：用户偏好、过往互动、累积的教训 | +| 能否替代 RAG？| — | 否——你不会把每份 PDF 都当作“记忆” | +| 能否被 RAG 替代？| — | 否——RAG 不会“记住上次用户说了什么” | + +**3 个 RAG 无法满足的场景**（恰好对应 Memory 的作用）： + +1. **跨 session 记住用户偏好 / persona**——用户上周告诉 agent “我是素食者”，这周回来，agent 仍然记得不能推荐肉类。RAG 知识库不会自动更新这些信息。 +2. **累积 agent 过往的成败教训**（Reflexion 的主场）——agent 第一次执行任务失败，反思“为什么失败”并保存下来，下次遇到类似任务时检索进 prompt，避免重蹈覆辙。RAG 知识库不会“记住自己的失败”。 +3. **Long-horizon task 中的中间状态**——agent 执行一个 100 步的任务，中间需要保留 working memory 不丢失。RAG 不适合这种“短期 + 结构化 + 高频写入”的状态。 + +→ **结论**: RAG 和 Memory 是**互补**而非替代关系。Production agent 通常**两者都需要**：RAG 处理外部知识，Memory 记录自身与用户的交互历史。下一节 Memory 设计 将指导你如何选择合适的 memory 模式。 + +## 🧠 Memory 是什么 + 怎么设计 + +> 📺 **视觉学习**：[李宏毅 2025 第二讲 — 一堂课搞懂 AI Agent 的原理（含 read / write / reflection memory module）](https://www.youtube.com/watch?v=M2Yg1kwPpts)（NTU 生成式 AI 时代下的机器学习 2025） + +### Working memory vs Long-term memory — 两种时间尺度 + +| 比较维度 | Working memory / 短期上下文 | Long-term memory / 持久记忆 | +|---|---|---| +| **中文可称** | 工作记忆 / 短期上下文 | 长期记忆 / 持久记忆 | +| **核心意思** | 这次任务或这段对话中，模型当下看得到的信息 | 存在外部、之后可跨 session 取回的信息 | +| **持续时间** | 短，通常限于当前 session | 长，可跨 session | +| **技术基础** | 上下文窗口（context window）/ prompt | 记忆存储层（memory store）/ 用户文件 / 向量数据库 | +| **适合记什么** | 任务细节、刚刚说过的内容 | 稳定偏好、长期目标、背景资料 | +| **是否受 context 长度限制** | 会，因为模型一次能看到的内容有限 | 较不受限，因为可以先存在外部，需要时只取一小段放回上下文 | +| **生活例子** | 刚收到的手机验证码、正在进行对话的上一句话 | 你深化学会的知识、图书馆、知识库、读过的书 | + +→ 在 agent 里，“短期记忆”更准确的说法其实是 **working memory**。它不是外部存储，而是当前 prompt / context window 里看得到的内容。 + +### Episodic / Semantic / Procedural memory — 三种内容类型 + +**注意**：上面的 working / long-term 是 **时间轴**；下面三种是 **内容轴**。两组分类 **正交、不互斥**。Long-term memory 里可以同时有 episodic + semantic + procedural 三种。 + +| 类型 | 中文 | 核心意思 | +|---|---|---| +| **Episodic memory** | 情节记忆 / 经验记忆 | 某次任务、某次互动、某次失败的具体经验 | +| **Semantic memory** | 语义记忆 / 事实记忆 | 稳定知识、用户偏好、背景事实 | +| **Procedural memory** | 程序记忆 / 技能记忆 | agent 知道“怎么做事”的规则、工具、workflow、skills | + +→ 这三种对应 [CoALA framework](#进阶coala-framework--agent-memory-的-4-层分类法)。**Reflexion** 是典型的 episodic memory 应用，因为它会累积过往 trial 的成败经验。 + +这里的 session 可以理解成一次连续互动，例如同一段聊天、同一次任务，或同一次 agent 执行。 + +### 3 种设计 pattern（什么时候用什么）⭐ Track B 必看 + +**不是所有 agent 都需要外部 memory store。Memory 架构选错，可能会花十倍 token 才得到同样效果。** + +这是开始练习前要建立的 mental model。后面的练习主要跑的是 Pattern 3（vector store），但 production 里未必需要这么复杂。 + +| Pattern | 适合场景 | 怎么跑 | 成本 | +|---|---|---|---| +| **1. Naive buffer**
（全塞 context） | 短对话、≤ 10 turn、agent 不需要跨 session 记忆 | 每次都把整段 history 送进 prompt | 线性增长，token 烧得快 | +| **2. Summary + recent**
（摘要远的 + 保留近 N 轮） | 中长对话、~50 turn、想压缩历史但别丢太多 | 每 N 轮让 LLM 把旧 history 摘成一段；prompt = `summary + last N turns` | 中等，有 LLM 摘要成本 | +| **3. Vector store + retrieval**
（外部 store + 每轮 semantic search） | 跨 session、知识库场景、agent 要“想起”久远的事 | embed 过去消息 → 存进 Vector DB → 每轮 query 相关片段并拼回 prompt | 高（向量计算 + 存储），但 token 用量稳定 | + +**怎么选**： + +- 对话 chatbot 没有跨 session 需求 → **Pattern 1** +- agent 配合长对话，需要记住今天聊过什么 → **Pattern 2** +- agent 具备跨 session 需求 + 知识库（本 stage 练习常见场景）→ **Pattern 3** +- production 大型 agent → 通常 **混用**：近期对话用 Pattern 1/2，长期记忆用 Pattern 3 + +> 💡 **Track B 重点**：Stage 7 的 multi-agent 系统里，每个 agent 通常都有“自己的 memory”+“shared memory”双层。实际常见组合是 **Pattern 2 + Pattern 3 混用**。先在这里把三种 pattern 理顺，Stage 7 才不会卡在 multi-agent memory 设计。 + +### ⭐ 5 个可上生产的 Memory Layer（按 use case 选） + +> Star 数和 benchmark 会变。重点不是排行，而是理解每个 memory layer 的设计取向。 + +学完 3 个 pattern 后，production 不必自己从零造 memory store。下面 5 个都是 Apache-2.0 / MIT、活跃维护、各擅其场： + +| Framework | Stars | License | 主场 use case | 特色 | +|---|---|---|---|---| +| [**agentmemory**](https://github.com/rohitg00/agentmemory) | 7.7k★ | Apache-2.0 | **Coding agent 跨 session 记忆** | MCP-universal（Claude Code / Cursor / Gemini CLI / Codex / Hermes / OpenClaw 都能接）、95.2% R@5、92% token saving、51 个 MCP tools + 12 个 auto hooks、benchmark-driven | +| [**mem0**](https://github.com/mem0ai/mem0) | 55.6k★ | Apache-2.0 | **Chatbot / 个人助手 user-level memory** | 自动事实提取 + 遗忘 + namespace、production-tested、社区最大 | +| [**Letta**](https://github.com/letta-ai/letta)（原 MemGPT） | 22.7k★ | Apache-2.0 | **长 session agent**（按月计） | OS-style paging memory（working + archival 双层）、persona 稳定、MemGPT 论文起源 | +| [**Zep**](https://github.com/getzep/zep) | 4.6k★ | Apache-2.0 | **Temporal KG-based memory** | 把对话历史建成 temporal KG，适合 time-aware reasoning 与 audit trail | +| [**graphiti**](https://github.com/getzep/graphiti) | 27.5k★ | Apache-2.0 | **实时知识图谱 agent 记忆** | 把 agent 过去的交互变成带时间轴的 knowledge graph、方便回头查找；Zep 背后的引擎、可单独使用 | +| [**LangMem**](https://github.com/langchain-ai/langmem) | 1.4k★ | MIT | **LangChain-native memory** | LangChain 官方 memory 库，直接接 LangGraph，适合已经 commit 到 LangChain stack 的项目 | + +**怎么选**： +- 构建 coding agent → **agentmemory**（MCP-native，和 Stage 5 ecosystem 高度对齐） +- 开发 chatbot / 个人助手 → **mem0**（最成熟、社区最大） +- 构建长运行 agent（数周 / 数月）→ **Letta**（OS-paging 优势明显） +- 需要时间感知推理 + audit trail → **Zep**（temporal KG） +- 已经采用 LangChain stack → **LangMem**（避免切框架） + +**额外官方文档**：[Anthropic Memory Tool](https://docs.claude.com/en/docs/agents-and-tools/tool-use/memory-tool)（Claude 官方 tool-based memory、基于文件、可直接 API 调用）、[LangChain Memory concepts](https://python.langchain.com/docs/concepts/memory/)（框架内各 memory class 的对照）。 + +### 进阶：CoALA Framework — Agent Memory 的 4 层分类法 + +[**Sumers et al. 2023 — Cognitive Architectures for Language Agents**](https://arxiv.org/abs/2309.02427) 把 agent memory 分成 4 种类型。这是当前非常实用的一套心智模型： + +| 类型 | 存储什么 | 对应示例 | +|---|---|---| +| **Working memory** | 当前任务上下文 | LLM context window 本身 | +| **Episodic memory** | 过去任务的具体经验 | Reflexion 记录、过往 trajectories | +| **Semantic memory** | 抽象事实 / 知识 | RAG 知识库、用户画像、偏好 | +| **Procedural memory** | 如何执行动作 / 技能 | tool definitions、[Skills（Stage 5.3）](05-claude-code-ecosystem.md#53--skillsclaude-code-的行為層-claude-code-生態最關鍵的一層) | + +→ **为什么有用**：上面 3 个 pattern（buffer / summary / vector）主要覆盖 working + episodic memory。Production agent 往往需要兼顾全部 4 层。CoALA 可以当作检查表，帮你看出 agent 缺了哪一层。 + +### 进阶：Generative Agents — 三重评分加权（经典案例） + +[**Park et al. 2023 — Generative Agents: Smallville**](https://arxiv.org/abs/2304.03442) 的小镇模拟中有 25 个 NPC agent，每个都有自己的 memory stream。检索时使用三个分数的加权组合： + +- **Importance**：LLM 为每个 memory 打 1-10 的重要性分数（吃饭 = 2 分，分手 = 9 分）。 +- **Recency**：基于时间的指数衰减。 +- **Relevance**：与当前查询的 embedding 相似度。 + +最终得分 = `α·importance + β·recency + γ·relevance`，按得分排序检索 top-k。**这是很多 2024-2025 年 production memory layer 系统（mem0 / Letta）的概念骨架**。 + +> 💻 **官方代码**: [joonspk-research/generative_agents](https://github.com/joonspk-research/generative_agents) ★ 论文配套的小镇模拟代码库，想跟着实现 memory stream + 三重评分检索看这里。 + +### 2024-2026 最新 Memory 作品 — 三大主线 + +Memory 研究在 2024-2026 年集中在 **3 大主线**： + +1. **🧠 结构化、可演化、可联想** — 从扁平的 vector store 走向类人脑 / Zettelkasten 启发的记忆结构。代表：[**A-MEM**](https://arxiv.org/abs/2502.12110)（memory 之间自动建立链接）、[**HippoRAG 2**](https://arxiv.org/abs/2502.14802)（KG + PageRank，海马体启发）。 +2. **📚 2026 年调查爆发** — 一年内出现了 5 个重量级调查报告和跨领域总结。代表：[**Memory in the Age of AI Agents**](https://arxiv.org/abs/2512.13564)（3D 分类法 + benchmark）、[**Memory for Autonomous LLM Agents**](https://arxiv.org/abs/2603.07670)（形式化 write-manage-read 循环）。 +3. **🛡 Memory 安全成为独立子领域** — Agent 运行时间越长，memory 越容易受到 cross-session poisoning / 未授权访问攻击。代表：[**Memory Security survey**](https://arxiv.org/abs/2604.16548)（Stage 7 安全 会涉及）。 + +**4 个值得深入挖掘的代表作**: + +| 作品 | 一句话 | 链接 | +|---|---|---| +| **Anthropic Memory Tool** | Claude 官方 tool-based memory，API 调用，基于文件 | [Anthropic Docs](https://docs.claude.com/en/docs/agents-and-tools/tool-use/memory-tool) | +| **A-MEM**（Agentic Memory）| Zettelkasten 风格，memory 之间自动链接，会演化 | [Xu et al. 2025](https://arxiv.org/abs/2502.12110) ⭐ | +| **HippoRAG 2** | KG + Personalized PageRank，跨文档 multi-hop，ICML 2025 | [Gutiérrez et al. 2025](https://arxiv.org/abs/2502.14802)、[OSU-NLP-Group/HippoRAG](https://github.com/OSU-NLP-Group/HippoRAG) ⭐ | +| **Memory in the Age of AI Agents**（调查报告）| 3D 分类法（temporal / substrate / control）+ benchmark 汇编 | [Hu et al. 2025-12](https://arxiv.org/abs/2512.13564) ⭐ | + +→ **长运行 Agent（周/月级别）必读**: 上述调查报告是必须阅读的。 + +
+📚 完整概览 — 其他 8 个值得了解的 Memory 作品（展开查看） + +| 作品 | 一句话 | 年份 / 论文 | +|---|---|---| +| **MemGPT → Letta GA** | OS-paging 内存，working / archival 双层，长 session 场景的强项 | [Packer et al. 2023](https://arxiv.org/abs/2310.08560) → Letta GA | +| **MemoryBank** | 基于艾宾浩斯遗忘曲线，访问过的 memory 得到强化，未使用的逐渐衰减 | [Zhong et al. 2023](https://arxiv.org/abs/2305.10250) | +| **MemoryLLM** | self-updatable memory 参数内建于模型权重中（而非 context 中）| [Wang et al. 2024](https://arxiv.org/abs/2402.04624) | +| **mem0**（见 5 主流 Memory Layer）| 可上生产的 memory layer，自动事实提取 + 遗忘 | [mem0ai/mem0](https://github.com/mem0ai/mem0) | +| **Memory for Autonomous LLM Agents**（调查报告）| 形式化 write-manage-read 循环，汇集 2022-2026 年研究 | [arXiv:2603.07670](https://arxiv.org/abs/2603.07670) ⭐ 2026 | +| **From Storage to Experience**（调查报告）| 演化框架：Storage → Reflection → Experience 三阶段 | [arXiv:2605.06716](https://arxiv.org/abs/2605.06716) ⭐ 2026 | +| **ScrapMem** | 生物启发式 on-device memory，“Optical Forgetting” 渐进式降低旧 memory 的解析度 | [arXiv:2605.03804](https://arxiv.org/abs/2605.03804) ⭐ 2026-05 | +| **Memory Security survey** | 长期 memory 面临 cross-session poisoning / 未授权访问 / 组织内传播等风险 | [arXiv:2604.16548](https://arxiv.org/abs/2604.16548) ⭐ 2026 | + +
+ +## 🧩 Chunking 细节（技术深入） + +良好的 chunking 能够让 LLM 在有限的 context 内，使用更精确、更完整的资讯生成回答。它并非简单地将文本平均分割。 + +分割方式取决于应用场景和文档内容。它决定了 retriever 所能看到的最细粒度的语义单元。 + +一个好的 chunk 应同时做到两件事：**足够完整**，让模型能理解上下文；**足够聚焦**，让检索不带过多杂讯。Chunk 过小会丢失上下文，Chunk 过大会导致相似度搜索变慢。 + +**常见策略**: + +- **固定长度（Fixed-Length）**: 按字符数或 token 数分割。优点是简单稳定；缺点是死板，容易切断段落、句子或表格。 +- **滑动窗口（Sliding Window）**: 每个 chunk 之间保留重叠区域（overlap）。优点是不容易在边界丢失信息；缺点是索引量会增大。 +- **递归切割（Recursive）**: 先尝试保留段落，若长度仍不合适，则退而求其次，尝试句子、词语等更小的单位。通常是入门 RAG 的良好基准。 +- **语义切割（Semantic Chunking）**: 基于 embedding 或语义变化进行分割，即当前块与前一个块的语义相似度出现差异时进行分割。适合长文档，但成本和复杂度较高。 +- **混合策略（Hybrid）**: 根据应用场景，思考不同文档结构该如何混合使用分割方法。例如，一篇论文可能需要保留章节、表格、公式和引用脉络。 + +![Chunking Strategy Flowchart](../resources/diagrams/chunking-strategies.jpg) + +> 📚 **经典教程**: [Greg Kamradt — 5 Levels of Text Splitting](https://github.com/FullStackRetrieval-com/RetrievalTutorials) ★ Chunking 入门必读，涵盖从 character-based 到 agentic chunking 的五个层级，包含 Jupyter notebook。 + +首次实现 RAG 时，不必一开始就追求复杂的分割方法。LangChain 的文档建议大多数场景下从 `RecursiveCharacterTextSplitter` 开始。 + +先跑出基准版本，再根据后续的 retrieval 结果来决定是否更换策略。 + +```python +from langchain_text_splitters import RecursiveCharacterTextSplitter + +text = "这是一个很长的文档内容...（此处省略一千字）..." + +splitter = RecursiveCharacterTextSplitter( + chunk_size=100, + chunk_overlap=20, + length_function=len, +) + +chunks = splitter.split_text(text) +print(f"Split into {len(chunks)} chunks") +print(chunks[0]) +``` + +**直观判断 Chunking 效果**: + +- 回答信息缺失，或有头无尾：通常是 chunk 太小，或 overlap 不够。 +- 回答包含正确信息，但混入了无关内容：通常是 chunk 太大，或 top-k 检索过多。 + +**进阶思考**: + +- Chunking 不是一次设定好就结束，需要结合真实查询和失败案例反复调整。 +- Chunk size、overlap、top-k、reranker 之间会相互影响，不要只单独看其中一个参数。 +- 思考一下，如果今天要 RAG 的数据包含图片 PDF、会议字幕文件，该如何分割比较好？ +- Chunking 的进阶变体（Sentence-Window / Parent-Child / Multi-Vector）见 进阶 RAG 技巧概览表。 + +## 🪞 进阶：带持久记忆的 Reflexion 完整版 ⭐ Track B 选读 + +> **本节是概念 + 路由，并非练习**。它扩展了 Stage 3 反思 的基础版本（single-session Actor / Critic loop），解释了为何某些反思**需要**持久记忆——这个版本才真正属于 Stage 6 的范畴。 + +**Reflexion 完整版与 Self-Refine 的区别**: + +| 版本 | 会话内保留内容 | 跨 session 保留内容 | 需要的 Memory 模式 | +|---|---|---|---| +| **Self-Refine**（Madaan 2023）| 上一轮的 answer + critic feedback | ❌ 不保留 | 无需（Pattern 1 buffer 即可） | +| **完整 Reflexion**（Shinn 2023）| 同上 | ✅ 将过去 trial 的“反思摘要”存入 episodic memory，下次遇到类似 task 时检索进 prompt 作为教训 | **需要**（Pattern 3 vector store 或 Pattern 2 summary） | + +**为什么这个版本需要 Memory**: Reflexion paper 中的 verbal reinforcement learning 核心在于“agent 跨 trial 累积教训”——agent 尝试任务 → 失败 → 反思“为何失败”并保存 → 下次遇到类似任务时，将过去的反思检索进 prompt，避免重蹈覆辙。这需要 **persistent episodic memory**，直接关联到本阶段上面讨论的 3 种 memory 模式。 + +**典型架构**（持久记忆完整版）： + +![Reflexion 持久 episodic memory loop](../resources/diagrams/reflexion-persistent-memory-loop.zh-Hans.png) + +→ **与 Stage 3 反思的区别**: Stage 3 侧重于**单 session 内的 in-context 循环**（无外部存储），本节则探讨**跨 trial 的持久 episodic memory 存储 + 检索**（从过往经验中学习）。 + +### 📚 想动手 / 想深入 + +**论文**: +- [**Reflexion (Shinn et al. 2023)**](https://arxiv.org/abs/2303.11366) ⭐ — **完整版**论文，Algorithm 1 详细说明了 memory buffer 的用法。 +- [**Self-Refine (Madaan et al. 2023)**](https://arxiv.org/abs/2303.17651) — 对比基线版本，即没有 episodic memory 的版本。 + +**参考实现**: +- [**noahshinn/reflexion**](https://github.com/noahshinn/reflexion) — 论文第一作者的参考实现（包含完整的 episodic memory 流程）。 +- [**LangChain — Reflexion**](https://langchain-ai.github.io/langgraph/tutorials/reflexion/reflexion/) — LangGraph 版本，可直接集成到本阶段的 RAG pipeline 练习中。 +- [**mem0**](https://github.com/mem0ai/mem0)（上面已列）+ [**Letta**](https://github.com/letta-ai/letta)（上面已列）— Memory Layer，可以直接作为 Reflexion 的 episodic store。 + +> 💡 **与 Stage 3 反思 的分工**: +> - 想理解“反思循环如何工作、单次如何运行” → Stage 3 反思。 +> - 想理解“反思如何跨 session 累积，agent 如何从过往学习经验” → 本节。 +> - 想看 production agent 内部如何使用反思（例如 Cursor / Claude Code）→ [Stage 5 5.7 Harness Internals](05-claude-code-ecosystem.md#57--claude-code-source-解剖reference-harness-implementation-track-b-必看)。 + +## 🤔 进阶 Reasoning / Reflection — 2024-2026 年思潮 ⭐ 覆盖两种路径 + +Reflexion 是一种**基于 Prompt 的 reflection**——LLM 在推理时自行修改自身。2024-2025 年出现了**第二条路径**：**在训练时就将 reflection 融入模型权重**（OpenAI **o1** / DeepSeek **R1**）。你应该了解这两种方法。 + +### Path 1: Prompt-based reflection / reasoning（传统做法） + +| 技巧 | 核心思想 | 论文 | +|---|---|---| +| **Self-Consistency** | 对 N 条推理路径进行采样，取多数结果——**最简单、最常用** | [Wang et al. 2022](https://arxiv.org/abs/2203.11171) | +| **Tree of Thoughts (ToT)** | Reasoning 变成树状结构，允许分支和回溯；适用于 puzzle / planning。 | [Yao et al. 2023](https://arxiv.org/abs/2305.10601) | +| **Graph of Thoughts (GoT)** | 不仅限于树，可以是任意图结构。 | [Besta et al. 2023](https://arxiv.org/abs/2308.09687) | +| **Chain-of-Verification (CoVe)** | 生成答案 → 自问验证问题 → 修改答案 | [Dhuliawala et al. 2023](https://arxiv.org/abs/2309.11495) | +| **CRITIC** | 通过工具增强的自我批判（使用 search / calculator 进行验证） | [Gou et al. 2023](https://arxiv.org/abs/2305.11738) | +| **Self-Discover** | Agent 先“发现”需要使用的 reasoning structure，然后再执行 | [Zhou et al. ICML 2024](https://arxiv.org/abs/2402.03620) ⭐ 2024 | +| **Self-Refine / Reflexion** | 已在上面 / Stage 3 讲解 | Stage 3 反思，本阶段 Reflexion | + +### Path 2: Trained-in reasoning / reflection（2024-2026 年重大转变） + +> 📺 **视觉学习**: [李宏毅 2025 第七讲 — DeepSeek-R1 这类大型语言模型是如何进行“深度思考”(Reasoning) 的？](https://www.youtube.com/watch?v=bJFtcwLSNxI)（NTU 生成式AI时代下的机器学习 2025） + +OpenAI 的 **o1**（2024-09）开启了这一趋势，随后是开源的 DeepSeek **R1**（2025-01）、**DeepSeek-V4-Pro**（2026-04 预览版，面向 Agent 的开源 reasoning）、Claude Fable 5（2026-06、Mythos-class、位阶在 Opus class 之上；2026-06-12 曾暂停、2026-07-01 已恢复）、Claude Opus 4.8（2026-05、Opus class 旗舰、Dynamic Workflows + parallel subagent）、GPT-5.5（2026-04）和 Gemini 3.1 Pro（2026-02）等当前前沿模型（2026-06 后半 Gemini 3.5 Flash 登场；GPT-5.6 Sol / Terra / Luna 已于 2026-07 正式推出）——它们将“step-by-step thinking + 自我纠错”**训练进了模型权重**，在推理时自动展开长 reasoning chain（thinking tokens）。**这是 2024-2026 年 LLM 的最大范式转移**，所有前沿模型都在遵循这条路径。下表仅列出**当前（2026-06）前沿模型**——历史上的前身（o1 / R1 / Sonnet 4.5 / Gemini 2.5）已省略，想了解 lineage 可查看各家发布日期。 + +| 模型 | 来源 / 发布 | 特色 | 链接 | +|---|---|---|---| +| **GPT-5.5** | OpenAI 2026-04（前身：o1 2024-09 → o3 → GPT-5 2025-08 → 5.4 2026-03）| 闭源，reasoning + chat 合并，提供 Thinking budget API，Agent 能力增强。**较新层级：GPT-5.6（Sol / Terra / Luna）、2026-07 正式推出、1.05M context** | [OpenAI](https://openai.com/) | +| **Claude Fable 5** | Anthropic 2026-06（Mythos-class、位阶在 Opus class 之上；同步发布 Claude Mythos 5 为解除部分 safeguard 的限量版本）| 闭源、Mythos-class（位阶在 Opus class 之上）。曾于 2026-06-12 被美国出口管制暂停，**出口管制 2026-06-30 解除、[Fable 5 于 2026-07-01 全球恢复](https://www.anthropic.com/news/redeploying-fable-5)**（重新部署时加了新安全 classifier；Mythos 5 仅对核准的美国组织恢复）。官方 benchmark 数字始终未公布 | [Claude Fable 5 / Mythos 5](https://www.anthropic.com/news/claude-fable-5-mythos-5) | +| **Claude Opus 4.8** | Anthropic 2026-05（前身：Sonnet 4.5 / Opus 4.5 / Opus 4.7，Dynamic Workflows 研究预览）| 闭源、Opus class 旗舰（Fable 5 暂停期间曾是最高可用层级；Fable 5 已于 2026-07-01 恢复）、可控 thinking budget（API 参数），在 SWE-bench / Terminal-bench 方面领先 | [Anthropic extended thinking](https://docs.claude.com/en/docs/build-with-claude/extended-thinking) | +| **Gemini 3.1 Pro** | Google 2026-02（前身：Gemini 2.5 Thinking 2025、Gemini 3 2025-11）| 闭源，可查看 thinking trace，GPQA Diamond 94.3%，价格 / 速度 / multimodal 方面领先。**较新层级：Gemini 3.5 Flash、2026-06 已开放（3.5 Pro 开发中）** | [Gemini API](https://ai.google.dev/gemini-api/docs/thinking) | +| **DeepSeek-V4 / V4-Pro / V4-Flash** | DeepSeek 2026-04 预览版（前身：R1 2025-01 → V3.1）| 开源 **MIT license**，面向 Agent 的训练，整合推理 + 工具使用 + 知识处理，R 系列 reasoning 已并入主线 | [HF DeepSeek-V4-Pro](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)、[R1 论文（方法基线）](https://arxiv.org/abs/2501.12948)、[CNBC 报道](https://www.cnbc.com/2026/04/24/deepseek-v4-llm-preview-open-source-ai-competition-china.html) | +| **QwQ-32B / QvQ-72B** | Alibaba Qwen 2024-11 ~ 2026 | 开源 **Apache 2.0**，32B 在小尺寸 reasoning 仍是不错的选择，QvQ 为视觉版本 | [QwQ blog](https://qwenlm.github.io/blog/qwq-32b-preview/) | + +### 两条路径如何选择 + +| 你的情况 | 建议 | +|---|---| +| 使用普通 chat model 作为基础，想加入 reasoning | Path 1（基于 Prompt 的方式）—— ToT / Self-Consistency / CoVe | +| 预算 / 延迟允许，需要最强的 reasoning 能力 | Path 2 —— 选择 **GPT-5.5 / Opus 4.8 / Gemini 3.5 Flash / Grok 4.3 / V4-Pro** 中的一个（Claude Fable 5 已于 2026-07-01 恢复，属 premium 高阶层级）| +| 想自己 fine-tune reasoning model | Path 2 —— 阅读 R1 论文（方法基线），从 R1-Distill / V4 开源权重开始 | +| 需要 on-device / 预算极度紧张 | **QwQ-32B**（Apache 2.0）或 R 系列 distill 版本 | +| Multi-agent debate / critic 场景 | Path 1（CRITIC / debate）+ [Stage 7 Multi-agent](07-multi-agent-production.md) | + +> 💡 **2025-2026 年趋势**: +> - Reasoning 模型已将 Reflexion 的能力内化到权重中——但**基于 Prompt 的 reflection 并未被取代**：Agent Loop（控制反思时机/内容）+ Multi-agent debate 仍然是必需的。 +> - **开源模型正在快速追赶闭源模型**——DeepSeek-V4-Pro（2026-04 预览版，MIT 许可）已将 R1 reasoning 集成到主线，并采用 Agent-first 训练，与 GPT-5.5 / Gemini 3.5 Flash 的差距在缩小。 +> - **Agent 能力正成为主要卖点**——V4 / Opus 4.8 都将 Agent-as-product（SWE-bench / Terminal-bench / tool use）作为核心 benchmark，单纯的 reasoning 已不足以吸引用户。 +> - 两条路径将长期共存，Production Agent 很可能两者都会使用。 + +## 📏 RAG / Memory Eval — 跑得起来 ≠ 跑得准 + +**为什么这节很重要**: RAG / Memory 系统的最大坑是“**看起来能跑，但实际检索精度很差**”。没有 Eval，你无法知道是修改 chunk size / 更换 embedding / 添加 reranker 真的有帮助——你只会得到“答案好像更顺了”这种主观印象。没有 Eval 的 Production Agent 基本上就是未经验证。 + +**3 个核心指标**: + +| Metric | 衡量什么 | 工具 | +|---|---|---| +| **Retrieval Recall@K** | Top-K 检索到的 chunk 中是否包含 ground truth 答案的 chunk | ragas / TruLens / LangSmith | +| **Answer Faithfulness** | 生成的答案是否基于检索到的 chunk（ vs. 模型幻觉）| ragas / TruLens | +| **Answer Relevance** | 答案与查询的相关度（避免答非所问）| ragas / LLM-as-judge | + +**代表性框架**: + +- [**explodinggradients/ragas**](https://github.com/explodinggradients/ragas) ★ **13.9k** Apache-2.0 ⭐ — RAG 评估标准工具，包含 8+ 指标（faithfulness / answer relevance / context precision / context recall 等），支持 reference-free 和 reference-based 评估。 +- [**TruLens**](https://github.com/truera/trulens) — 可观测性 + 评估一体化，与 LangChain / LlamaIndex 集成良好。 +- [**LangSmith**](https://docs.langchain.com/langsmith) — LangChain 官方 Eval + Tracing 平台（闭源 SaaS）。 + +**如何开始**: 完成练习 4（完整 RAG Pipeline）后，集成 ragas 进行评估，测量一次基线——然后再调整 chunk / embedding / top-k，观察指标的变化。**没有这一步，调参全靠猜**。 + +→ Stage 7 Eval 将在本次基础上继续，将评估扩展到 multi-agent / full harness。 + +## 🛠 动手练习（基础示例性练习） + +### 练习 1：Embeddings +将 100 个句子进行 embedding，找出某个查询的最近邻。理解向量之间的距离意义。 + +### 练习 2：Vector DB +将 embedding 存储到 Chroma 中，进行语义查询。对比“与 keyword search 的区别”。 + +### 练习 3：Chunking 对照 +用同一份文档进行三种切割：固定长度、按段落分割、按 heading 分割。用 5 个真实问题比较 top-k 结果，记录哪种分割方法更容易检索到正确上下文。 + +### 练习 4：完整 RAG 流水线 +将一份 PDF 切块 → embed → 检索 top-k → 生成回答。这是大多数 RAG 应用的基础框架。 + +### 练习 5：Long-term Memory +让 agent 在多轮对话中记住事情。可以使用 `mem0` 或自行搭建 vector store。 + +## 🎯 常用 Memory / RAG 工具推荐（按用途分类） + +不知道从哪里开始选择工具？下面是 2025 年下半年业界常用的组合——**根据你的场景（“入口”）进行选择，深入了解请点击链接查看 repo**： + +| 场景 | 推荐工具 | 原因 | +|---|---|---| +| **首次运行 RAG**（上手最快）| [Chroma](https://github.com/chroma-core/chroma) + [LlamaIndex](https://github.com/run-llama/llama_index) | 本地优先，零运维，quickstart 友好。Stage 6 练习默认配置。 | +| **企业级 RAG 框架**（LangChain / LlamaIndex 之外的第三选择）| [Haystack (deepset)](https://github.com/deepset-ai/haystack) ★ 25.2k Apache-2.0 | deepset 开源，面向 production 的编排，企业级 NLP 场景成熟。 | +| **Agent 长期记忆**（见 5 个可上生产的 Memory Layer）| [agentmemory](https://github.com/rohitg00/agentmemory) / [mem0](https://github.com/mem0ai/mem0) / [Letta](https://github.com/letta-ai/letta) / [Zep](https://github.com/getzep/zep) / [LangMem](https://github.com/langchain-ai/langmem) | 详见上方 5 个可上生产的 Memory Layer 部分。 | +| **RAG / Memory Eval**（必备）| [ragas](https://github.com/explodinggradients/ragas) ★ 13.9k | RAG 评估标准工具，8+ 指标，支持 reference-free + reference-based。 | +| **Production Scale RAG**（百万级文档）| [Qdrant](https://github.com/qdrant/qdrant) + LlamaIndex | Rust 编写的 vector DB，在大规模场景下比 Chroma 更快。 | +| **已有 Postgres 环境** | [pgvector](https://github.com/pgvector/pgvector) | Postgres 扩展，SQL + vector 在同一个数据库中，运维最简。 | +| **企业级 RAG + Web UI** | [RAGFlow](https://github.com/infiniflow/ragflow) | 强大的文档解析能力（含 OCR / 表格 / 布局），企业级应用，自带 Web UI。 | +| **中文 RAG 范例** | [Langchain-Chatchat](https://github.com/chatchat-space/Langchain-Chatchat) | 中文社区最广泛使用，支持本地 LLM 部署（ChatGLM / Qwen / Llama / Ollama）。★ 38k+，Apache-2.0。⚠️ 最后更新 2025-11（边缘情况）。 | +| **进阶：Contextual Retrieval** | [Anthropic Cookbook](https://platform.claude.com/cookbook/capabilities-contextual-embeddings-guide) | Claude 结合 prompt caching 的 contextual chunking（**详见上方 进阶 RAG 技巧**）。 | +| **进阶：Knowledge Graph 推理** | [LightRAG](https://github.com/HKUDS/LightRAG) / [Microsoft GraphRAG](https://github.com/microsoft/graphrag) | Knowledge Graph + RAG，实现实体-关系推理（**详见上方 进阶 RAG 技巧**）。 | +| **教程合集** | [ai-engineering-hub](https://github.com/patchy631/ai-engineering-hub) | RAG + agent 教程合集，Jupyter notebook 形式。 | + +**推荐入手顺序**: +1. 首先安装：**Chroma + LlamaIndex**（用于 Stage 6 练习）。 +2. Agent 需要记忆功能：添加 **mem0**（最简单的 memory layer）。 +3. 进入 production 规模：切换到 **Qdrant** 或 **pgvector**。 +4. 想升级到进阶 RAG：研究 进阶 RAG 技巧 部分。 + +## 🎯 精选 Projects（模板 / 规范 / 示例合集） + +按用途分类，17 个项目一表搞定。**根据你的场景（"入口"）进行选择，深入了解请点击链接查看 repo**。 + +| 分类 | Project | ⭐ | 适用人群 | 推荐理由 / 备注 | +|---|---|---|---|---| +| **RAG Framework**
（完整流水线） | [LlamaIndex](https://github.com/run-llama/llama_index) | ⭐⭐⭐⭐⭐ | 以文档为核心的应用 | 以 RAG 为核心，提供 document loader / chunking / retrieval / query engine 一站式服务。★ 49k+ | +| | [infiniflow/ragflow](https://github.com/infiniflow/ragflow) | ⭐⭐⭐⭐⭐ | 希望将 RAG 部署给非开发者用户使用的团队 | Production 级别的 RAG engine，深度文档理解（layout / 表格 / OCR）+ hybrid retrieval + agent loop + Web UI。★ 79k+，Apache-2.0 许可。 | +| | [HKUDS/LightRAG](https://github.com/HKUDS/LightRAG) | ⭐⭐⭐⭐ | 探索研究级 graph + long-context memory 方法的开发者 | graph + vector hybrid retrieval + summarization-based memory，基于 EMNLP 2025 论文。★ 34k+，MIT 许可。代码风格偏研究。 | +| **Vector DB**
（本地优先） | [Chroma](https://github.com/chroma-core/chroma) | ⭐⭐⭐⭐⭐ | 练习 2 / 4，最容易上手的 vector DB | 开源的 embedding 数据库，可本地运行，支持 in-memory / SQLite 后端，零运维。★ 27k+，Apache-2.0 许可。**安装**: `pip install chromadb` | +| **Vector DB**
（Production Scale） | [Qdrant](https://github.com/qdrant/qdrant) | ⭐⭐⭐⭐⭐ | Chroma 性能不足，需要 production scale 时 | Rust 编写的 vector DB，在大规模场景下比 Chroma 更快。提供云端版和自托管版。★ 31k+ | +| **Vector DB**
（Hybrid） | [Weaviate](https://github.com/weaviate/weaviate) | ⭐⭐⭐⭐ | Production 部署 + schema 约束 | 内置模块（text2vec / generative / classification），schema 驱动，原生支持 BM25 + vector hybrid 搜索。★ 16k+ | +| **Vector DB**
（已有 Postgres 环境） | [pgvector](https://github.com/pgvector/pgvector) | ⭐⭐⭐⭐ | 原本就在使用 Postgres 的团队 | Postgres 扩展，SQL + vector 在同一个数据库中，运维最简。★ 21k+ | +| **Vector DB**
（跑在 app 内） | [lancedb/lancedb](https://github.com/lancedb/lancedb) | ⭐⭐⭐⭐ | 想要 vector DB 直接内建、不想另跑 server | 直接跑在你 app 里的 vector DB（不用另开 server）、能处理文字 + 图片、关键字 + 向量一起搜。★ 10k+，Apache-2.0 | +| **Memory Framework**
（自动事实提取） | [mem0ai/mem0](https://github.com/mem0ai/mem0) | ⭐⭐⭐⭐⭐ | 个人助手 / Chatbot 需要 user-level 记忆 | 自我精炼的 memory 层，支持跨 session 存储事实。★ 59k+ | +| **Memory Framework**
（OS-Paging） | [Letta（原 MemGPT）](https://github.com/letta-ai/letta) | ⭐⭐⭐⭐ | Agent 需要运行很长时间（以月为单位） | 阶层式 memory（working / archival），借鉴 OS Paging 概念。★ 22k+ | +| **Memory（框架内）** | [LangChain — Memory](https://python.langchain.com/docs/concepts/memory/) | ⭐⭐⭐ | 已在使用 LangChain | 4 种 memory 抽象（buffer / summary / vectorstore-backed / entity）。 | +| **进阶 RAG 技巧** | [Anthropic — Contextual Retrieval Cookbook](https://platform.claude.com/cookbook/capabilities-contextual-embeddings-guide) | ⭐⭐⭐⭐⭐ | 跑完基础 RAG 后，想升级 | Claude 结合 prompt caching 的 contextual chunking，包含完整的端到端示例。 | +| **中文 RAG 范例** | [chatchat-space/Langchain-Chatchat](https://github.com/chatchat-space/Langchain-Chatchat) | ⭐⭐⭐⭐ | 中文知识库 / RAG 应用 | 中文社区最广泛使用，支持本地 LLM 部署（ChatGLM / Qwen / Llama / Ollama），中文默认配置好。★ 38k+，Apache-2.0 许可。⚠️ 最后更新 2025-11（边缘情况）。 | +| **教程合集** | [ai-engineering-hub](https://github.com/patchy631/ai-engineering-hub) | ⭐⭐⭐⭐ | 想看“同一概念在不同场景下如何实现” | 主题式 LLM / RAG / agent 教程合集，Jupyter notebook 形式，跨多个 stage 都有用。★ 34k+，MIT 许可。 | +| **Production AI Assistant**
（学习部署 RAG 的参考）| [onyx](https://github.com/onyx-dot-app/onyx)（原 Danswer）| ⭐⭐⭐⭐⭐ | 想学习“如何将 RAG 驱动的 AI Assistant 部署到 production” | 开源的企业级 AI Assistant，支持跨 LLM，包含完整的 ingest / retrieval / chat / admin 功能。★ 29.4k，积极维护。 | +| **RAG Cookbook**
（30+ 技巧范例）| [NirDiamant/RAG_Techniques](https://github.com/NirDiamant/RAG_Techniques) | ⭐⭐⭐⭐⭐ | 跑完基础 RAG 后，想探索各种变体 | 大型 RAG 技巧 Cookbook，包含 Self-RAG / HyDE / Multi-Query / Adaptive 等 30+ 个 Jupyter notebook 示例。 | +| **DSPy**
（编程而非 Prompt）| [stanfordnlp/dspy](https://github.com/stanfordnlp/dspy) | ⭐⭐⭐⭐⭐ | 使用 LLM 一段时间后，想自动优化 prompt + chain | Stanford NLP group 开发，★ 34.4k MIT，Path 3 范式（详见 DSPy）。 | +| **RAG / Memory Eval**
（必备）| [explodinggradients/ragas](https://github.com/explodinggradients/ragas) | ⭐⭐⭐⭐⭐ | 完成练习 4（完整 RAG Pipeline）后，想衡量检索精度 | RAG 评估标准工具，8+ 指标，支持 reference-free + reference-based。★ 13.9k Apache-2.0。 | + +**推荐入手顺序**: +1. 首次安装必备：**Chroma + LlamaIndex**（用于 Stage 6 练习）。 +2. Agent 需要记忆功能：添加 **mem0**（最简单的 memory layer）。 +3. 进入 production 规模：切换到 **Qdrant** 或 **pgvector**。 +4. 想升级到进阶 RAG：研究 进阶 RAG 技巧 部分。 + +## ✅ 进入 Stage 7 前的自我检查 + +你是否能够： + +- [ ] 编写一条 50 行的 RAG Pipeline（load → chunk → embed → store → query → answer）？ +- [ ] 解释为什么天真的 chunking 在长文档上会失效？ +- [ ] 为 API 文档、PDF、表格设计不同的 chunking 策略？ +- [ ] 在特定规模下，能在 Chroma、Qdrant、pgvector 之间做出选择？ +- [ ] 区分“给 agent memory”和“使用 RAG”这两件事？ +- [ ] 解释 RAG 和 Memory 如何互补（参考 从 RAG 到 Memory 表格）？ + +如果以上都能够做到 → 前往 [Stage 7 — Multi-Agent · Production 化](07-multi-agent-production.md)。 diff --git a/stages/07-multi-agent-production.en.md b/stages/07-multi-agent-production.en.md new file mode 100644 index 0000000..6c3755b --- /dev/null +++ b/stages/07-multi-agent-production.en.md @@ -0,0 +1,334 @@ +# Stage 7 — Multi-Agent · Productionization + +> [繁體中文](./07-multi-agent-production.md) | [简体中文](./07-multi-agent-production.zh-Hans.md) | **English** + +⏱ **Estimated Time**: 2-4 weeks (approx. 15-30 hours) + +> 💡 High density of terminology (multi-agent / handoff / eval / observability / guardrails...) → Refer to [`resources/glossary.md` 4 + 6](../resources/glossary.md#4-multi-agent). + +> 📋 **Chapter Composition**: [What is Multi-Agent · Productionization (Positioning) + Three-layer engineering split + When to use multi-agent] → Learning Objectives → Entry Conditions → Required Reading → Harness Engineering (**8 core components including Cost/Latency**) → Hands-on Exercises (including Exercise 6 Cost Optimization) → **Agent Benchmark Landscape: how to read it, not just the leaderboard** → Recommended Tools → Featured Projects → Self-Check +> 🔑 **Key Terms**: See [`resources/glossary.md` 4 + 6](../resources/glossary.md#4-multi-agent) (multi-agent / orchestration / handoff / eval / observability / harness (the execution and control layer around the model)) + +This is the final stage. You are moving from "I can build an agent" to "I can make an agent **truly stable for people to use**"—with multiple agents collaborating, with eval, with observability, and deployable to a usable environment. **"Productionization" ≠ enterprise scale**—as long as an agent can produce stable output and be used by others, it falls within the scope of this stage. + +## 🎯 What Is Multi-Agent · Productionization (Positioning) + +**This stage = how multiple agents collaborate + how to push an agent from prototype to the point where others can use it stably.** Three sentences clarify the scope: + +- **It is not just about learning frameworks** — Stage 4 already taught how to choose frameworks +- **It does not have to be enterprise scale** — as long as an agent can be used by others, it counts as productionization +- **The core is harness engineering** — 8 core components + eval + observability + cost / latency control + +**Division of labor with adjacent stages**: + +- **Stage 4** = how to choose a single-agent framework, patterns like ReAct / Plan-Execute +- **This stage** = **multi-agent collaboration** + **harness engineering** (execution-system engineering) + **deployment to a usable environment / observability / eval** + +### The three-layer engineering split: Prompt → Context → Harness + +Engineering work can be split into three layers, corresponding to different positions in the stack (not the difference between one call and many calls): + +| Layer | Concept | Core question | Unit of concern | Corresponding stage | +|---|---|---|---|---| +| 1 | **Prompt Engineering** | How should I ask this time? | **single LLM call** | [Stage 2](02-prompt-engineering.md) | +| 2 | **Context Engineering** | What information should the model receive this time? | **context across multiple interactions** | [Stage 6](06-memory-rag.md) | +| **3** | **Harness Engineering**
(**This stage**) | How does the whole workflow run? | **executable LLM workflow / system** | **This stage** | + +> 🔁 **The next layer: Loop Engineering**. After prompt → context → harness, the fourth discipline emerging in 2026 is **engineering the agent's iteration loop itself**: the goal, available tools, context management, **termination logic**, and error handling that keep an agent reliable across hundreds of steps and multiple sessions. Claude Code's `/goal` (give a verifiable completion condition and the agent loops until it is met) is exactly this; [Stage 5.6 Dynamic Workflows](05-claude-code-ecosystem.en.md) is the agent writing its own loop script. Lineage: ReAct (2022) → AutoGPT (2023) → /goal (2026). + +**Plain-language difference**: +- **Prompt** = design a good way of asking so the model answers correctly this time +- **Context** = dynamically decide which background, memory, documents, and tool results to include so the model understands the current situation +- **Harness** = connect prompt, context, tools, state, flow control, and error handling into a system that can actually run + +**This stage's 3 core questions**: + +1. **Multi-agent collaboration** — debate / planner-executor / peer review / handoff / supervisor-worker patterns +2. **Harness Engineering** — agent loop / tool registry (the list of tools an agent can call + interface definitions) / context manager / safety / retry / telemetry / eval / cost (8 core components, detailed below) +3. **Productionization** — eval harness / observability / cost & latency optimization / deployment to a usable environment + +**Division of labor with Stage 5** (to avoid confusion): + +| Comparison | What's Covered There | What's Covered in This Stage | +|---|---|---| +| **Stage 5.5 Subagents** | Claude Code's native subagent mechanism (markdown-based, no coding) | General multi-agent frameworks (autogen / crewAI / langgraph, vendor-agnostic) | +| **Stage 5.7 Claude Code source** | Claude Code source dissection (reference implementation case study) | General harness engineering principles (not tied to a specific vendor) | + +### ⚠ But do you really need multi-agent? + +**Multi-agent is not the default; it is a design you use only when the task truly needs it.** In most scenarios, you should first try a simple workflow or a single agent; **only when the task is naturally decomposable, needs parallel exploration, a single context is not enough, or explicit role separation is needed, is multi-agent worth introducing**. Forcing it brings **3-10× token cost, difficult debugging, and severe context fragmentation (context gets split across multiple agents, and no one sees the whole picture)**. + +> 📌 **The decision framework's canonical home is Stage 4**: the full Anthropic / Cognition stance comparison + the 4 "should you go multi-agent" signals + each signal's corresponding pattern live in [Stage 4 §When do you really need multi-agent](04-agent-frameworks.en.md#when-do-you-really-need-multi-agent-dont-force-it) (design-phase decision). This section is only the last sanity check before production — **none of the 4 signals present?** → a single agent + good prompts + tool use is enough; don't force multi-agent. **The harness engineering part of this stage (8 components / eval / observability) still applies even if you end up using a single agent**—so this stage is still required reading even if you decide against multi-agent. + +## 📌 Learning Objectives + +- Design multi-agent orchestration patterns (debate, planner-executor, peer review) +- Build an evaluation harness for your agent +- Add observability (tracing, logging, cost tracking) +- Use the Anthropic SDK / OpenAI SDK for production deployment (advanced features: streaming, prompt caching, batching) +- Deploy an agent to production (Docker, serverless, monitoring) + +## 🚪 Entry Conditions + +You should already have: +- Completed Stage 4 (used at least one agent framework to run a multi-agent demo) +- Completed Stage 5 (understand the roles of MCP / Skills / Plugins / Subagents, and have dissected a harness internally in 5.7) +- Completed Stage 6 (know basic RAG, can explain the differences between memory patterns) +- Basic familiarity with Docker / git / CI (will be used in production deployment) + +If not, go back and complete the previous stages. This stage is about "combining everything you've learned → running in production," and any missing piece will be a blocker. + +## 📚 Required Reading + +1. [**Anthropic — Building Effective Agents**](https://www.anthropic.com/engineering/building-effective-agents) — Reread it from a production perspective +2. [**Anthropic — Prompt Caching**](https://www.anthropic.com/news/prompt-caching) — A technique for 90% cost reduction +3. [**Anthropic — Message Batches API**](https://docs.anthropic.com/en/docs/build-with-claude/batch-processing) — Asynchronous batch jobs +4. [**anthropics/courses — Prompt Evaluations**](https://github.com/anthropics/courses) ⭐⭐⭐⭐⭐ ★ 21k+ — Anthropic's official 5-course umbrella; **module 4 "Prompt Evaluations" maps to this stage's eval / observability section**. Jupyter notebooks covering systematic evaluation of prompt and agent behavior. +5. **Documentation for any eval framework** — promptfoo, LangSmith, or weave +6. [**ai-boost/awesome-harness-engineering**](https://github.com/ai-boost/awesome-harness-engineering) (★ 2.8k+) — A collection of tools / patterns / eval / memory / MCP / observability for agent harnesses +7. [**ZhangHanDong/harness-engineering-from-cc-to-ai-coding**](https://github.com/ZhangHanDong/harness-engineering-from-cc-to-ai-coding) (★ 1.4k+) — Learning harness design from Claude Code's source code (in Chinese) + +## 🏗 Harness Engineering — Engineering Design for a Production Agent Runtime ⭐ Core Concept of This Stage + +### Positioning: The execution and control layer around the model + +To turn an LLM into a usable agent, you usually run into three layers of engineering problems. These three layers correspond to different engineering positions, not simply "one call" versus "many calls." + +> 💡 **Simon Willison 2025**: "coding agent = LLM + harness"; harness = all the code **that is not the model itself**. +> +> 💡 **OpenAI also uses the term "Harness Engineering" in 2026** (see the [OpenAI Harness Engineering article](https://openai.com/index/harness-engineering), published 2026-02). + +| Layer | What you are engineering | Where to learn it | +|---|---|---| +| **1. Prompt Engineering** | The **string** sent into the LLM (system prompt / few-shot / format) | [Stage 2](02-prompt-engineering.md) | +| **2. Context Engineering** | The **information** placed inside the window (RAG / memory / tool defs / history assembly) | [Stage 6](06-memory-rag.md) | +| **3. Harness Engineering**
(**This section**) | The **execution and control layer around the model** (loop / retry / sandbox / observability / deployment) | This stage | + +**How do you tell which layer you are working on? Ask**: + +1. Am I changing the **string itself**? → Prompt engineering +2. Am I changing the **information put into the window**? → Context engineering +3. Am I changing the **surrounding program that calls the model**? → Harness engineering + +→ The three layers are **orthogonal**: a one-call RAG app is still doing context engineering (the key is how the window is assembled); a 50-call chatbot with no retrieval is still only doing prompt engineering. + +### The 8 Core Components of a Harness + +**Harness Engineering (agent runtime design) = connecting the LLM, tools, memory, state, workflow control, error handling, eval, observability, and deployment into an executable, observable, maintainable agent system.** + +→ Everything that is **not part of the model weights**, and is not just the prompt string itself, falls under harness. A deployable agent runtime includes these 8 core components (the first 6 are built into the runtime, the 7th, eval, is a plug-in tool, and the 8th, cost / latency, is a cross-cutting concern): + +| Component | What it Does | Corresponding Exercise in this Stage | +|---|---|---| +| **Agent loop** | The "LLM → tool → result → LLM" loop, stably handling multiple turns | Exercise 1 Multi-agent debate | +| **Tool registry** | Dynamic tool dispatch, permission gates, sandboxing | (Present in every framework / SDK) | +| **Context manager** | Message history management, context window control, auto-compaction | Stage 6 + This stage's Exercise 4 SDK | +| **Safety layer** | Permission prompts, sandboxed execution, interception of destructive ops | (Built into Claude Code, customizable in SDKs) | +| **Retry / recovery** | How to handle tool failure (exception vs. the LLM reflecting on the error itself) | Exercise 4 SDK Advanced | +| **Telemetry / Observability** | Metrics, logging, token counting, trace export | **Exercise 3 Observability** | +| **Eval harness** | Regression testing, quality gates, A/B testing | **Exercise 2 Eval** | +| **Cost / Latency optimization** ⭐ Required for 2024-2026 | Prompt caching, model routing, thinking budget, batching, semantic cache | **Exercise 6 Cost optimization** (New) | + +**Framework vs. Harness: key difference**: +- **Framework** ([Stage 4](04-agent-frameworks.md)) defines the **API** — what the interface you call looks like +- **Harness** (this section) defines the **runtime** — how it runs, how it recovers, and how it is observed + +### Feedback loops: agents improve from feedback, not a more perfect prompt + +The 8 components above are the harness "skeleton." But what actually makes the skeleton work is something more basic than any single component: **an agent gets better by feeding feedback back into the loop, not by wording the opening prompt more perfectly.** + +An analogy: a student doesn't get better because the assignment was phrased more elegantly; they improve by getting feedback at the right moments — handing in a draft, a teacher's mid-task nudge, having the finished work graded, redoing it next time. Agents are the same, and feedback can enter at four moments: + +| Moment | In plain words | What it looks like in engineering | +|---|---|---| +| **1. Tool return values** | The text a tool returns is itself feedback written for the agent | Write the error message, hints, and next-step suggestions *clearly* — don't just dump a stack trace | +| **2. Mid-run steering** | Slip a message in between the agent's two thinking steps to redirect it | Inject a message mid-run (steering); no need to wait a whole turn to course-correct | +| **3. End-of-turn acceptance** | When a turn finishes, have *someone else* check it against the goal | Use a separate evaluator to compare against the goal, instead of letting the agent grade itself | +| **4. Outer loop** | Re-invoke the agent against the same goal until it's done | Goal-driven re-runs (like OpenAI Codex's `/goal`, or a cron re-run) | + +**Why #3 (an independent check) matters most**: Anthropic's own experiments found that when you ask an agent to check its own work, it almost always praises it — even when the quality is obviously mediocre. So they split the "maker" agent from the "checker" agent: one builds, the other uses tools (like Playwright) to actually click and test, then reports bugs back. Tuning an external checker to be more skeptical is far easier than making one agent harder on itself. + +> 📚 Real example: Anthropic's [Harness design for long-running apps](https://www.anthropic.com/engineering/harness-design-long-running-apps) (2026-03) uses planner → generator → evaluator, letting an agent run for hours to build a full music-production app, correcting on every round from the evaluator's feedback. + +### Reference Implementations + +Want to see what a harness running in production looks like? Two references: + +- **The entire Claude Code runtime** — is a reference harness implementation. **For a source-reading exercise, see [Stage 5.7](05-claude-code-ecosystem.en.md#57--dissecting-claude-code-source-reference-harness-implementation--a-must-read-for-track-b)** (clone `claude-agent-sdk-python` and dissect the main loop + where the first 6 runtime components from the table above live; the 7th, Eval harness, is a plugin, and the 8th, Cost / Latency, is cross-cutting, see the deep-dive below) +- **`anthropics/claude-agent-sdk-python`** source — the specific repo used in the exercise above + +→ The remaining 6 exercises in this stage (multi-agent / eval / observability / SDK / deploy / cost) each cover one facet of the harness. Completing the full stage = assembling a complete mental model of harness engineering. + +### Deep dive into the 8th core component — Cost / Latency Optimization (Required for 2024-2026 Productionization) + +When a production agent runs long enough, **cost and latency will eat up most of your budget and user experience**. From 2024-2026, frontier models have treated this as a first-class API feature—**knowing how to use it = saving 50-90% on cost / latency**. + +| Technique | How it Saves | 2026 Status | +|---|---|---| +| **Prompt caching** | Recurring prefixes (system prompt, long context) are billed once, subsequent cache hits get a ~90% discount | Fully supported by Anthropic / OpenAI / Gemini, automatic or manual tagging | +| **Model routing / cascade** | Simple queries → smaller model, difficult queries → frontier model | Built into [RouteLLM](https://github.com/lm-sys/RouteLLM) / [OpenRouter](https://openrouter.ai/) for production | +| **Thinking budget** | Controllable thinking token limit for reasoning models, trading off latency / quality | Claude / Gemini API parameter, high by default in o-series | +| **Speculative decoding** | A smaller model predicts N tokens, a larger model validates them at once, ×2-3 speedup for a single model | Built into vLLM / TGI, automatic at the inference layer | +| **Batching** | Processing multiple queries in parallel for higher GPU utilization | vLLM, production inference layer | +| **Semantic caching** | Sharing answers for similar queries (not just exact matches) | Built into [GPTCache](https://github.com/zilliztech/GPTCache) / Helicone | + +**How Track A can use this** (for those using CLI agents): +- Set up prompt caching in Claude Code / Cursor to save 50-90% on daily session costs +- Use [RouteLLM](https://github.com/lm-sys/RouteLLM) / [OpenRouter](https://openrouter.ai/) to dynamically switch models (simple questions use Haiku / Flash, difficult ones use Opus / Pro) +- Use the `thinking_budget` parameter in the Claude API to control the token limit for reasoning models + +**How Track B can build this** (for those writing their own agents): +- Build a custom cascade router that maps query embeddings → classifier → model +- Monitor token cost within the agent loop and automatically downgrade if the budget is exceeded +- Integrate a semantic cache layer in the deployed environment +- Observability platforms like [Helicone](https://github.com/Helicone/helicone) / [langfuse](https://github.com/langfuse/langfuse) already have these features built in, so you do not have to write them yourself + +## 🛠 Hands-on Exercises (Basic Illustrative Exercises) + +### Exercise 1: Multi-Agent Debate +Have two agents debate a topic (e.g., "Should we use Python or Rust for the backend?"), with a third agent acting as a referee. Observe the patterns of convergence or divergence in the debate. + +### Exercise 2: Eval +Write an eval for your previous agent, run it N times, and measure the success rate. Replace the habit of "I'll just take a look" with a quantitative approach. + +### Exercise 3: Observability +Connect LangSmith, Helicone, or weave to an agent and view the full trace. Understand that "debugging an agent without observability = a black box". + +### Exercise 4: Advanced SDK +Use streaming + prompt caching + tool use in a single call. See how the costs come down. + +### Exercise 5: Deploy +Package an agent into Docker and deploy it to the cloud (any provider will do). Learn to turn a prototype into something that can be run by others. + +### Exercise 6: Cost Optimization (New) ⭐ +Measure the token cost of any of your previous exercise agents, then add prompt caching and measure again. Observe the relationship between cache hit rate and cost reduction. **Bonus**: Connect to [RouteLLM](https://github.com/lm-sys/RouteLLM) or [OpenRouter](https://openrouter.ai/) and implement cascade routing (simple queries → Haiku / difficult queries → Opus), and measure the average cost. + +## 📊 Agent Benchmark Landscape: How to read it, not just the leaderboard + ⚠ Reward-Hacking Warning + +Before choosing a model or building an agent, you'll want to look at benchmark numbers—but in **April 2026, UC Berkeley discovered that all 8 major agent benchmarks can be reward-hacked to ~100%**. Below is the 2026 leaderboard status + how to read it without getting misled. + +### Mainstream Agent Benchmark SOTA as of 2026-05 + +| Benchmark | Domain | 2026-05 SOTA | Leading Model | +|---|---|---|---| +| [**SWE-bench Verified**](https://www.swebench.com/) | Software Engineering / code agent | **88.6%** | Claude Opus 4.8 | +| [**Terminal-Bench**](https://github.com/laude-institute/terminal-bench) | terminal tasks | Leading | Claude Opus 4.8 | +| **GAIA** | general assistant | **74.6%** | Claude Sonnet 4.5 (Princeton HAL) | +| [**WebArena**](https://github.com/web-arena-x/webarena) | web navigation | **68.7%** | (leading model not disclosed) | +| [**OSWorld**](https://github.com/xlang-ai/OSWorld) | OS-level desktop control | **76.26%** (SOTA, superhuman vs 72.36% human baseline) | OpenAI CUA 38%, most frontier models still under 50% | +| [**τ-bench**](https://github.com/sierra-research/tau-bench) | multi-turn dialogue with tool use | (Harder to hack) | Anthropic / OpenAI leading | +| **RE-bench** | research engineering | (Harder to hack, close to human baseline) | Frontier models | + +> **Mythos-class tier (Claude Fable 5 — 2026-06-09; suspended 2026-06-12, restored 2026-07-01)**: [**Claude Fable 5**](https://www.anthropic.com/news/claude-fable-5-mythos-5) (`claude-fable-5`, Mythos-class, positioned above the Opus class) shipped as the publicly available highest-capability Claude tier alongside its sibling Claude Mythos 5 (`claude-mythos-5`, some safeguards lifted, limited to approved customers). It was suspended 2026-06-12 by a US export-control directive, but the controls were lifted 2026-06-30 and [Fable 5 was redeployed globally 2026-07-01](https://www.anthropic.com/news/redeploying-fable-5) with a new safety classifier (Mythos 5 restored only for approved US organizations). The numbers above stay attributed to their original models; Fable 5's official benchmark numbers were never published, so it is not listed. **Fable 5 is again the highest Claude tier; Opus 4.8 is the Opus-class flagship.** + +→ For detailed rankings + live updates: [Agent Benchmark Leaderboard 2026](https://benchmarkingagents.com/agent-benchmarks/), [Rapid Claw AI Agent Framework Scorecard 2026](https://rapidclaw.dev/blog/ai-agent-benchmarks-2026) + +### ⚠ Berkeley 2026-04 Reward-Hacking Warning + +[**UC Berkeley RDI Report 2026-04-12**](https://rdi.berkeley.edu/blog/trustworthy-benchmarks-cont/): An automated scanning agent systematically audited **8 major benchmarks** (SWE-bench / WebArena / OSWorld / GAIA / Terminal-Bench / FieldWorkArena / CAR-bench, etc.) and found that **every single one could be reward-hacked to nearly 100% without the agent actually solving a single task**. + +This means that for numbers on the leaderboard like "Claude 87.6% / GPT 85.0%", some X% of that might be hacked, not from genuinely solving the task. + +### How to Read Benchmarks Without Being Misled + +| Approach | Recommendation | +|---|---| +| Only look at the leaderboard top | ❌ All 8 above have been proven hackable | +| Look at task-level success rate breakdown | ✅ Most hacks are concentrated in a few tasks | +| Run your own hold-out test set | ✅✅ The most reliable method, a must for production agents | +| Check the trajectory / log to see if the task was really solved | ✅ Distinguishes reward hacking from a genuine solve | +| Look at multiple benchmarks + your own use case | ✅ Don't rely on a single metric | + +**Which benchmarks are harder to hack (as of 2026-05)**: +- **τ-bench** — Multi-turn dialogue + tool use, denser reward function +- **RE-bench** — Real-world research engineering tasks +- **Your own production eval set** ⭐ Always the most reliable + +> 💡 **The discipline of production agent eval**: +> - Don't take external benchmark numbers as ground truth; they tell you the "upper limit," not "real-world performance." +> - Your own eval set (internal hold-out test) is the basis for go-live decisions. +> - Every time a model is upgraded → run it against your internal eval set for validation, don't just look at the vendor's published benchmark improvements. +> - Connect to [langfuse](https://github.com/langfuse/langfuse) / [promptfoo](https://github.com/promptfoo/promptfoo) to automate eval and run it with every deployment. + +> 📊 **For observability, learn one portable standard + two eval ideas**: (1) **OpenTelemetry GenAI conventions** (`gen_ai.*`): langfuse / Arize Phoenix / Helicone all emit OTel-compatible spans, so learning this layer keeps you from being locked to one tool; the OTel-native [Arize Phoenix](https://github.com/Arize-ai/phoenix) (★10k) is worth a look. (2) **pass^k**: the probability of solving the same task k times in a row (reliability, not a single pass), measured by [τ²-bench](https://github.com/sierra-research/tau2-bench). (3) Multi-agent failures have a ready vocabulary: **MAST** ([arXiv 2503.13657](https://arxiv.org/abs/2503.13657), 14 failure modes in 3 categories). + +## 🎯 Recommended Tools for Multi-Agent / Production (by Use Case) + +Don't know where to start choosing tools? Below are the common pairings in the industry for 2025-2026—**use "Scenario" as your entry point, and click the repo link for a deeper dive**: + +| Scenario | Recommended Tool | Why | +|---|---|---| +| **Writing your first multi-agent** (fastest to get started) | [crewAI](https://github.com/crewAIInc/crewAI) | Role-based, get it running in a few lines of code, straightforward production patterns | +| **Want a group debate / brainstorm pattern** | [AutoGen](https://github.com/microsoft/autogen) | GroupChat for free-form debate, from Microsoft | +| **Need an audit trail / checkpoint / human-in-the-loop for production** | [LangGraph](https://github.com/langchain-ai/langgraph) | State machine approach, most complete control | +| **Standardizing eval** (a must for CI / regression) | [promptfoo](https://github.com/promptfoo/promptfoo) ⭐ | YAML config, cross-model comparison, ★ 22k+ | +| **Eval + observability on the same platform** | [langfuse](https://github.com/langfuse/langfuse) ⭐ | OSS, tracing + eval + prompt mgmt, ★ 28k+ | +| **Quick instrumentation without code changes** | [Helicone](https://github.com/Helicone/helicone) | Proxy-based, not tied to a framework | +| **Entire stack is on LangChain** | [LangSmith](https://www.langchain.com/langsmith) (Commercial) | Official observability from LangChain | +| **Building a Claude agent** (programmatically) | [claude-agent-sdk-python](https://github.com/anthropics/claude-agent-sdk-python) ⭐ | Official agent SDK from Anthropic, same runtime as Claude Code | +| **Deploying an agent as an API service** | [BentoML](https://github.com/bentoml/BentoML) | The most complete, Docker + serving | +| **Self-hosting an open-source LLM** (to replace paid APIs) | [vLLM](https://github.com/vllm-project/vllm) | High-throughput serving, ★ 79k+ | +| **Fine-tuning an open-source LLM** | [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory) | Unified SFT/DPO/PPO/GRPO for 100+ models, no-code Web UI, widest Chinese community, ★ 70k+ | + +**Suggested adoption sequence**: +1. First multi-agent: **crewAI** (role-based, simplest) +2. Add eval: **promptfoo** (YAML, CI integration) +3. Add observability: **langfuse** (OSS, complete) +4. Production upgrade: Switch to **LangGraph** (stronger control) + **BentoML** (deploy) +5. Advanced: Self-host LLMs with **vLLM**, fine-tune with **LLaMA-Factory** + +## 🎯 Featured Projects (Templates / SDKs / Tool Collections) + +Categorized by use case, a single table to get you started with 27 projects. **Use "Who is it for" as your entry point, and click the repo link for a deeper dive.** + +| Category | Project | ⭐ | Who is it for | Why it's recommended / Notes | +|---|---|---|---|---| +| **Multi-Agent Orchestration** | [microsoft/autogen](https://github.com/microsoft/autogen) | ⭐⭐⭐⭐⭐ | Those who want a GroupChat free-debate pattern | Introduced in Stage 4, revisit for multi-agent debate / brainstorming patterns in production scenarios | +| | [crewAIInc/crewAI](https://github.com/crewAIInc/crewAI) | ⭐⭐⭐⭐⭐ | Those who want a role-based assembly line | Role-based multi-agent (research → writer → reviewer), the simplest production pattern | +| | [langchain-ai/langgraph](https://github.com/langchain-ai/langgraph) | ⭐⭐⭐⭐⭐ | Those needing an audit trail / checkpoint / human-in-the-loop | State machine approach, strongest for production control | +| **Eval Frameworks** | [promptfoo](https://github.com/promptfoo/promptfoo) ⭐ | ⭐⭐⭐⭐⭐ | To standardize the eval process, CI integration | YAML config, cross-model comparison. ★ 22k+, MIT | +| | [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) | ⭐⭐⭐⭐ | For academic benchmarks (MMLU / HellaSwag / GSM8K) | Academic grade. ★ 12k+, MIT | +| | [openai/evals](https://github.com/openai/evals) | ⭐⭐⭐⭐ | For OpenAI-specific evals / want to contribute upstream | ★ 18k+ | +| **Observability** | [langfuse](https://github.com/langfuse/langfuse) ⭐ | ⭐⭐⭐⭐⭐ | For self-hosting production observability | OSS LangSmith alternative, traces + sessions + evals + prompt mgmt. ★ 28k+, MIT | +| | [LangSmith](https://www.langchain.com/langsmith) (Commercial) | ⭐⭐⭐⭐ | For those with their entire stack on LangChain / LangGraph | Official from LangChain, hosted version only | +| | [Helicone](https://github.com/Helicone/helicone) | ⭐⭐⭐⭐ | For quick instrumentation without code changes | Proxy-based, get logging + caching for free. ★ 5.7k+, Apache 2.0 | +| | [weave (W&B)](https://github.com/wandb/weave) | ⭐⭐⭐⭐ | For teams already using W&B for ML experiment tracking | W&B tracing + eval, integrates with wandb | +| | [comet-ml/opik](https://github.com/comet-ml/opik) | ⭐⭐⭐⭐ | For eval + observability on one open-source platform | Trace what your LLM / agent did, track experiments, and run quality checks (evals). ★ 19k+, Apache 2.0 | +| | [pydantic/logfire](https://github.com/pydantic/logfire) | ⭐⭐⭐⭐ | For tracing agent / LLM calls on the OpenTelemetry standard | Watch and debug what your agent / LLM calls did; from the Pydantic team, built on the OpenTelemetry standard. ★ 4k+, MIT | +| **Safety / Guardrails** | [NVIDIA-NeMo/Guardrails](https://github.com/NVIDIA-NeMo/Guardrails) | ⭐⭐⭐⭐ | For safety rules around an agent's inputs and outputs | Safety rules you wrap around an LLM app — keep it on-topic, block jailbreaks, filter bad output. ★ 6.6k+, Apache 2.0 | +| **Advanced Anthropic SDK** | [anthropic-sdk-python](https://github.com/anthropics/anthropic-sdk-python) | ⭐⭐⭐⭐⭐ | For building applications directly on the Claude API | Official Python SDK: streaming / async / tool use / prompt caching / batches / files | +| | [anthropic-sdk-typescript](https://github.com/anthropics/anthropic-sdk-typescript) | ⭐⭐⭐⭐ | For TypeScript / Node / web apps | The TS version of the Python SDK | +| | [claude-agent-sdk-python](https://github.com/anthropics/claude-agent-sdk-python) ⭐ | ⭐⭐⭐⭐⭐ | For building Claude-based agents, not just API calls | Built-in tool use loop / file access / sandbox / subagent orchestration; same runtime as Claude Code, read the source to see how it works internally. ★ 6.9k+, MIT | +| | [claude-agent-sdk-typescript](https://github.com/anthropics/claude-agent-sdk-typescript) | ⭐⭐⭐⭐ | For Claude agents in a Node / web app environment | The TS version of the Claude Agent SDK. ★ 1.6k+ | +| | [Anthropic Cookbook (Advanced)](https://github.com/anthropics/anthropic-cookbook) | ⭐⭐⭐⭐ | For seeing official advanced SDK patterns | Especially the `prompt_caching.ipynb` / `tool_use/` / `multimodal/` notebooks | +| **Structured Output** | [BoundaryML/baml](https://github.com/BoundaryML/baml) | ⭐⭐⭐⭐ | For getting reliable, validated JSON out of any model | A small dedicated language for getting reliable, checked JSON out of LLMs; works with Claude / OpenAI / local models across 7 programming languages. ★ 8k+, Apache 2.0 | +| **Deployment** | [BentoML](https://github.com/bentoml/BentoML) | ⭐⭐⭐⭐ | For packaging an agent into a production API service | Docker + serving framework. ★ 8k+, Apache 2.0 | +| | [LangServe](https://github.com/langchain-ai/langserve) | ⭐⭐⭐⭐ | For quickly deploying a LangChain agent | Based on FastAPI | +| | [vLLM](https://github.com/vllm-project/vllm) | ⭐⭐⭐⭐ | For self-hosting an open-source LLM to replace paid APIs | High-throughput LLM serving for Llama / Qwen, etc. ★ 79k+, Apache 2.0 | +| **Chinese Deploy / Fine-tune** | [datawhalechina/self-llm](https://github.com/datawhalechina/self-llm) | ⭐⭐⭐⭐ | For Chinese teams wanting to self-host open-source LLMs | A complete Chinese guide from training-to-deployment, for Qwen / Llama / GLM / multimodal. ★ 30k+, Apache 2.0 | +| | [hiyouga/LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory) | ⭐⭐⭐⭐⭐ | For fine-tuning open-source LLMs (beyond just prompt eng) | Unified SFT/DPO/PPO/GRPO for 100+ models, no-code Web UI, widest Chinese community. ★ 70k+, Apache 2.0 | +| **Multi-Agent Case Studies** | [geekan/MetaGPT](https://github.com/geekan/MetaGPT) | ⭐⭐⭐⭐⭐ | For seeing role division + artifact handoff patterns | An SOP-based multi-agent team of PM / Architect / Engineer, generating PRD → design → code. ★ 67k+, MIT | +| | [OpenBMB/ChatDev](https://github.com/OpenBMB/ChatDev) | ⭐⭐⭐⭐ | For seeing agent debate / peer-review patterns | Conversational software development where agents debate each other on design / code / test. ★ 33k+, Apache 2.0, has a zh README | +| | [princeton-nlp/SWE-agent](https://github.com/princeton-nlp/SWE-agent) | ⭐⭐⭐⭐ | To understand why tool design > prompt tuning | The Agent-Computer Interface (ACI) design philosophy, backed by a Princeton paper, a leading method on SWE-Bench. ★ 19k+, MIT | + +> 🌳 For **Claude's native subagent mechanism** (multi-agent without a framework), see [Stage 5.5](05-claude-code-ecosystem.en.md#55--subagents-claude-codes-native-multi-agent-mechanism--2025-new-feature). This stage focuses on frameworks / production; Stage 5.5 focuses on markdown-based subagent orchestration. + +## ✅ Self-Check After Stage 7 + +Can you: + +- [ ] Design a multi-agent system and clearly explain its collaboration protocol? +- [ ] Run an automated eval pipeline in CI? +- [ ] Connect observability (tracing) to a production agent? +- [ ] Measure the cost difference before and after implementing prompt caching on a real workload? +- [ ] Deploy an agent to the cloud (any provider)? + +If you can do all of these → first go to [**Stage 7.5 — Advanced Agentic Concepts Map**](07.5-advanced-agentic-concepts.md) (1 week, no coding — build a frontier concept map and locate which advanced concepts the industry is still debating), then proceed to [**Stage 8 — Agent Interfaces**](08-agent-interfaces.md) (**a shared hub for both tracks**) to learn how agents interact with the non-API world (Computer Use / Browser Use / Sandbox). Or, pick a [specialized branch](../README.en.md#-learning-map-two-tracks), or come back and contribute to this repo. + +## 💡 What's Next + +You now have the foundational skills. For the next 6-12 months, you should focus on: +1. **Picking one production system** and taking it from prototype to production. +2. **Contributing upstream** (LangGraph, AutoGen, MCP servers, Anthropic cookbook). +3. **Reading papers**—agent research is moving fast. +4. **Making something tangible**—open-source a real tool, stop just writing tutorials. diff --git a/stages/07-multi-agent-production.md b/stages/07-multi-agent-production.md new file mode 100644 index 0000000..18eb752 --- /dev/null +++ b/stages/07-multi-agent-production.md @@ -0,0 +1,334 @@ +# Stage 7 — 多 Agent 系統與穩定運作（Multi-Agent & Production） + +> **繁體中文** | [简体中文](./07-multi-agent-production.zh-Hans.md) | [English](./07-multi-agent-production.en.md) + +⏱ **時間估算**：2-4 週（約 15-30 小時） + +> 💡 用語密度高（multi-agent / handoff / eval / observability / guardrails⋯）→ 翻 [`resources/glossary.md` 4 + 6](../resources/glossary.md#4-multi-agent)。 + +> 📋 **本章組成**：〔Multi-Agent · Production 化 是什麼（先定位）+ 三層工程分工 + 何時用 multi-agent〕→ 學習目標 → 進入條件 → 必修閱讀 → Harness Engineering（**8 個核心元件含 Cost/Latency**）→ 動手練習（含練習 6 Cost Optimization）→ **Agent Benchmark Landscape：怎麼看，不要只看排行榜** → 常用工具推薦 → 精選 Projects → 自我檢查 +> 🔑 **關鍵名詞**：見 [`resources/glossary.md` 4 + 6](../resources/glossary.md#4-multi-agent)（multi-agent / orchestration / handoff / eval / observability / harness（模型外圍的執行與控制層）） + +最後一個階段。你正從「我會做 agent」走向「我能讓 agent **真的給人穩定用**——多個 agent 協作、有 eval、有 observability、能部署到可用環境」。**「Production 化」 ≠ enterprise scale**——只要 agent 能穩定產出 + 能讓別人使用、就算進入這 stage 範圍。 + +## 🎯 Multi-Agent · Production 化 是什麼（先定位） + +**本 stage = 多 agent 怎麼協作 + 把 agent 從 prototype 推到能穩定給人用的程度**。三句話釐清範圍： + +- **不是只學 framework**——Stage 4 已教 framework 怎麼挑 +- **不一定要 enterprise scale**——只要 agent 能讓別人用、就算 production 化 +- **核心是 harness engineering**——8 個核心元件 + eval + observability + cost / latency 控制 + +**跟前後 stage 的分工**： + +- **Stage 4** = 單 agent framework 怎麼挑、ReAct / Plan-Execute 等 pattern +- **本 stage** = **多 agent 協作** + **harness engineering**（執行系統工程）+ **部署到可用環境 / observability / eval** + +### 三層工程分工：Prompt → Context → Harness + +工程分工可以分成三層、對應 stack 不同位置（不是 call 一次 vs 多次的差別）： + +| 層級 | 概念 | 核心問題 | 關注單位 | 對應 stage | +|---|---|---|---|---| +| 1 | **Prompt Engineering** | 這一次要怎麼問？ | **單次 LLM call** | [Stage 2](02-prompt-engineering.md) | +| 2 | **Context Engineering** | 這次該給模型哪些資訊？ | **多次互動中的上下文** | [Stage 6](06-memory-rag.md) | +| **3** | **Harness Engineering**
（**本 stage**） | 整個流程怎麼跑起來？ | **可執行的 LLM workflow / system** | **本 stage** | + +> 🔁 **下一層：Loop Engineering（迴圈工程）**：prompt → context → harness 之後，2026 浮現的第四層是「**設計 agent 的迭代迴圈本身**」——目標、可用工具、context 管理、**終止條件**、錯誤處理，讓 agent 跑數百步、跨 session 仍可靠。Claude Code 的 `/goal`（給一個可驗證的完成條件、agent 自己 loop 到達成）就是這個方向；[Stage 5.6 Dynamic Workflows](05-claude-code-ecosystem.md) 則是 agent 自己寫出 loop 腳本。譜系：ReAct（2022）→ AutoGPT（2023）→ /goal（2026）。 + +**白話差異**： +- **Prompt** = 設計一個好的問法，讓模型這次回答準 +- **Context** = 動態決定要放入哪些背景、記憶、文件、工具結果，讓模型知道現在情境 +- **Harness** = 把 prompt、context、工具、狀態、流程控制、錯誤處理串成一套可以運作的系統 + +**本 stage 三個核心問題**： + +1. **Multi-agent 協作** — debate / planner-executor / peer review / handoff / supervisor-worker pattern +2. **Harness Engineering** — agent loop / tool registry（agent 可呼叫工具的清單 + 介面定義）/ context manager / safety / retry / telemetry / eval / cost（8 個核心元件、下面詳述） +3. **Production 化** — eval harness / observability / cost & latency 優化 / 部署到可用環境 + +**跟 Stage 5 的分工**（避免混淆）： + +| 跟誰比 | 那邊講什麼 | 本 stage 講什麼 | +|---|---|---| +| **Stage 5.5 Subagents** | Claude Code 原生 subagent 機制（markdown-based、不寫程式）| 通用 multi-agent framework（autogen / crewAI / langgraph、跨 vendor）| +| **Stage 5.7 Claude Code source** | Claude Code source 解剖（參考實作 case study）| Harness engineering 通則（不綁特定 vendor）| + +### ⚠ 但你真的需要 multi-agent 嗎？ + +**Multi-agent 不是 default、是任務真的需要時才上的設計**。多數場景應先嘗試 simple workflow 或 single agent；**只有在任務天然可分解、需要平行探索、單一 context 不夠、或需要明確角色分工時，multi-agent 才值得引入**。硬上會付 **3-10× token、debug 困難、context fragmentation（context 被切散在多個 agent、彼此看不到全貌）嚴重**。 + +> 📌 **決策框架的 canonical 在 Stage 4**：完整的 Anthropic / Cognition 立場對照 + 4 個「該上 multi-agent」訊號 + 每個訊號對應的 pattern，見 [Stage 4 §什麼時候真的需要 multi-agent](04-agent-frameworks.md#什麼時候真的需要-multi-agent不要硬上)（設計階段決策）。本節只做 production 前的最後回頭檢查——**4 個訊號一個都不在？** → single agent + 好 prompt + tool use 就夠、別硬上 multi-agent。**本 stage 的 harness engineering 部分（8 個元件 / eval / observability）即使你最後用 single agent 也都會用到**——所以即使你決定不走 multi-agent、本 stage 仍是必修。 + +## 📌 學習目標 + +- 設計 multi-agent orchestration 模式（debate、planner-executor、peer review） +- 為 agent 架一套 evaluation harness +- 加上 observability（tracing、logging、cost tracking） +- 用 Anthropic SDK / OpenAI SDK 部署到可用環境（進階功能：streaming、prompt caching、batching） +- 把 agent deploy 到 production（Docker、serverless、monitoring） + +## 🚪 進入條件 + +你應該已經： +- 完成 Stage 4（用過至少一個 agent framework 跑 multi-agent demo） +- 完成 Stage 5（懂 MCP / Skills / Plugins / Subagents 各自角色，並用 5.7 解剖過 harness 內部） +- 完成 Stage 6（會基本 RAG，能講出 memory pattern 差異） +- 對 Docker / git / CI 基礎熟悉（部署成可用服務會用到） + +沒到的話 → 補完前面幾個 stage。本 stage 是「組合所有前面學到的東西 → 跑 production」，缺一塊都會卡。 + +## 📚 必修閱讀 + +1. [**Anthropic — Building Effective Agents**](https://www.anthropic.com/engineering/building-effective-agents) — 用 production 的角度再讀一次 +2. [**Anthropic — Prompt Caching**](https://www.anthropic.com/news/prompt-caching) — 90% 成本下降的技巧 +3. [**Anthropic — Message Batches API**](https://docs.anthropic.com/en/docs/build-with-claude/batch-processing) — 非同步 batch job +4. [**anthropics/courses — Prompt Evaluations**](https://github.com/anthropics/courses) ⭐⭐⭐⭐⭐ ★ 21k+ — Anthropic 官方 5 course umbrella、**module 4「Prompt Evaluations」對應本 stage eval / observability 部分**。Jupyter notebook、教怎麼系統化評估 prompt 跟 agent 行為 +5. **任一 eval framework 的文件** — promptfoo 或 LangSmith 或 weave +6. [**ai-boost/awesome-harness-engineering**](https://github.com/ai-boost/awesome-harness-engineering)（★ 2.8k+）— agent harness 的工具 / pattern / eval / memory / MCP / observability 全集合 +7. [**ZhangHanDong/harness-engineering-from-cc-to-ai-coding**](https://github.com/ZhangHanDong/harness-engineering-from-cc-to-ai-coding)（★ 1.4k+）— 從 Claude Code 原始碼學 harness 設計（中文） + +## 🏗 Harness Engineering — production agent runtime 的工程設計 ⭐ 本 stage 核心概念 + +### 定位：模型外圍的執行與控制層 + +要把 LLM 變成可用的 agent，通常會碰到三層工程問題。這三層對應的是不同工程位置，不是單純用「一次 call」或「多次 call」來區分。 + +> 💡 **Simon Willison 2025**：「coding agent = LLM + harness」、harness = 所有**不是 model 本身**的程式碼。 +> +> 💡 **OpenAI 2026 也使用 "Harness Engineering" 這個說法**（見 [OpenAI Harness Engineering article](https://openai.com/index/harness-engineering)、2026-02 發布）。 + +| 層級 | 工程的對象 | 在哪學 | +|---|---|---| +| **1. Prompt Engineering** | 送進 LLM 的**字串**（system prompt / few-shot / 格式） | [Stage 2](02-prompt-engineering.md) | +| **2. Context Engineering** | 視窗裡裝的**資訊**（RAG / memory / tool defs / history 組裝） | [Stage 6](06-memory-rag.md) | +| **3. Harness Engineering**
（**本節**） | 模型**外圍的執行與控制層**（loop / retry / sandbox / observability / 部署） | 本 stage | + +**怎麼分辨自己在做哪一層？問**： + +1. 我改的是**字串本身**嗎？→ Prompt engineering +2. 我改的是**塞進視窗的資訊**嗎？→ Context engineering +3. 我改的是**呼叫模型的外圍程式**嗎？→ Harness engineering + +→ 三層**正交**：1 次 call 的 RAG app 也在做 context engineering（重點是組視窗）；50 次 call 但沒做 retrieval 的 chatbot 仍只在做 prompt engineering。 + +### Harness 的 8 個核心元件 + +**Harness Engineering（Agent 執行系統設計）= 把 LLM、tools、memory、state、workflow control、retry、safety、eval、observability 與 deployment 串成一套可執行、可觀測、可維護的 agent 系統**。 + +→ 所有**不屬於 model weights、也不只是 prompt string 本身**的工程元件都算 harness 範圍。一個可部署的 agent runtime 包含這 8 個核心元件（前 6 個是 runtime 內建、第 7 個 eval 是外掛工具、第 8 個 cost / latency 是跨層議題）： + +| 元件 | 做什麼 | 對應本 stage 練習 | +|---|---|---| +| **Agent loop** | 「LLM → tool → result → LLM」迴圈、穩定處理多輪 | 練習 1 multi-agent 辯論 | +| **Tool registry** | 動態 tool dispatch、permission gate、sandboxing | （在每個 framework / SDK 都有）| +| **Context manager** | message history 管理、context window 控制、auto-compact | Stage 6 + 本 stage 練習 4 SDK | +| **Safety layer** | permission prompts、sandboxed exec、destructive op 攔截 | （Claude Code 內建、SDK 可自訂）| +| **Retry / recovery** | tool fail 怎麼處理（exception vs LLM 自己看 error 反思） | 練習 4 SDK 進階 | +| **Telemetry / Observability** | metrics、logging、token counting、trace export | **練習 3 Observability** | +| **Eval harness** | regression test、quality gate、A/B test | **練習 2 Eval** | +| **Cost / Latency optimization** ⭐ 2024-2026 必修 | prompt caching、model routing、thinking budget、batching、semantic cache | **練習 6 Cost optimization**（新加）| + +**Framework vs Harness 關鍵差別**： +- **Framework**（[Stage 4](04-agent-frameworks.md)）規範 **API** — 你呼叫的介面長什麼樣 +- **Harness**（本節）規範 **runtime** — 怎麼跑、怎麼 recovery、怎麼觀測 + +### 回饋迴路：agent 進步靠的是回饋，不是更完美的提示 + +上面 8 個元件是 harness 的「骨架」。但讓骨架真正運作的，是一件更基礎的事：**agent 變強，靠的是「把回饋送回迴圈」，不是把開頭那段提示寫得更完美。** + +打個比方：一個學生不會因為作業題目寫得更漂亮就變強，他變強是因為在對的時機收到回饋——交草稿、寫到一半被老師提醒、完成後被批改、下次重做。agent 也一樣，而回饋可以在四個時機進來： + +| 時機 | 白話 | 工程上長什麼樣 | +|---|---|---| +| **1. 工具回傳值** | 工具吐回來的那段話，本身就是寫給 agent 看的回饋 | 把錯誤訊息、提示、下一步建議「寫清楚」，別只丟一個 stack trace | +| **2. 執行中插話** | 在 agent 兩次思考之間塞一句話調整方向 | 中途注入訊息（steering），不用等它整輪跑完才修正 | +| **3. 單輪結束的驗收** | 一輪做完，由「另一個人」對著目標檢查 | 用獨立的驗收者（evaluator）比對目標，而不是讓 agent 自己打分 | +| **4. 外層 loop** | 對著同一個目標反覆叫 agent，直到完成 | 目標導向的重跑（像 OpenAI Codex 的 `/goal`、或 cron 定時重跑）| + +**為什麼第 3 個（獨立驗收）特別重要**：Anthropic 自己的實驗發現，叫 agent 檢查自己的成品，它幾乎都會「自我稱讚」——就算品質明顯普通。所以他們把「做東西的 agent」和「驗收的 agent」拆開：一個負責做，一個用工具（像 Playwright）實際去點、去測，再把 bug 回報回去。把外部驗收者「調得更挑剔」，比讓同一個 agent「對自己更嚴格」容易得多。 + +> 📚 實際案例：Anthropic [Harness design for long-running apps](https://www.anthropic.com/engineering/harness-design-long-running-apps)（2026-03）用 planner → generator → evaluator 三段，讓 agent 連續跑好幾小時做出一個完整的音樂製作 app，每輪都靠 evaluator 回饋修正。 + +### 參考實作 + +想看實際在 production 跑的 harness 長什麼樣？兩個 reference： + +- **Claude Code 整個 runtime** — 是 reference harness 實作。**讀 source 練習見 [Stage 5.7](05-claude-code-ecosystem.md#57--claude-code-source-解剖reference-harness-implementation-track-b-必看)**（clone `claude-agent-sdk-python` 解剖 main loop + 上表前 6 個 runtime 元件位置；第 7 個 Eval harness 是外掛、第 8 個 Cost / Latency 是 cross-cutting、見下方深入段） +- **`anthropics/claude-agent-sdk-python`** source — 上面練習用的具體 repo + +→ 本 stage 剩下的 6 個練習（multi-agent / eval / observability / SDK / deploy / cost）每個都是 harness 的一個面向。學完整 stage = 拼出完整的 harness engineering mental model。 + +### 第 8 個核心元件深入 — Cost / Latency Optimization（2024-2026 Production 化必修） + +Production agent 跑久了、**cost / latency 兩條線會吃掉你大半預算與使用者體驗**。2024-2026 前沿模型都把這當 first-class API feature——**會用 = 省 50-90% cost / latency**。 + +| 技巧 | 怎麼省 | 2026 狀態 | +|---|---|---| +| **Prompt caching** | 重複 prefix（system prompt、long context）一次計費、後續 cache hit 折扣 ~90% | Anthropic / OpenAI / Gemini 全支援、自動或手動標記 | +| **Model routing / cascade** | 簡單 query → 小 model、難 query → 前沿模型 | [RouteLLM](https://github.com/lm-sys/RouteLLM) / [OpenRouter](https://openrouter.ai/) production 內建 | +| **Thinking budget** | reasoning model 可控 thinking token 上限、trade latency / quality | Claude / Gemini API 參數、o-series 預設高 | +| **Speculative decoding** | 小 model 預測 N token、大 model 一次驗證、單 model 速度 ×2-3 | vLLM / TGI 內建、推論層自動 | +| **Batching** | 多 query 並行處理、GPU 利用率高 | vLLM、production inference layer | +| **Semantic caching** | 相似 query 共用回答（不只 exact match）| [GPTCache](https://github.com/zilliztech/GPTCache) / Helicone 內建 | + +**Track A 怎麼用**（用 CLI agent 的人）： +- 在 Claude Code / Cursor 設定 prompt caching、daily session 省 50-90% cost +- 用 [RouteLLM](https://github.com/lm-sys/RouteLLM) / [OpenRouter](https://openrouter.ai/) 動態切換 model（簡單問用 Haiku / Flash、難問用 Opus / Pro） +- Claude API 用 `thinking_budget` 參數控 reasoning model 的 token 上限 + +**Track B 怎麼 build**（自己寫 agent 的人）： +- 自架 cascade router、把 query embedding → classifier → model 對應 +- 在 agent loop 內監控 token cost、超 budget 自動降級 +- 部署到可用環境時整合 semantic cache 層 +- [Helicone](https://github.com/Helicone/helicone) / [langfuse](https://github.com/langfuse/langfuse) 等 observability 平台都已內建這幾招、不用自己寫 + +## 🛠 動手練習（基礎 illustrative 練習） + +### 練習 1：Multi-Agent 辯論 +兩個 agent 辯論一個題目（例如「該用 Python 還是 Rust 寫 backend」），第三個 agent 當裁判。觀察辯論收斂或分歧的 pattern。 + +### 練習 2：Eval +替你前面的 agent 寫一份 eval，跑 N 次量成功率。把「我用眼睛看一下」的習慣換掉。 + +### 練習 3：Observability +把 LangSmith、Helicone、或 weave 接上一個 agent，看完整 trace。理解「沒 observability 的 agent debug = 黑盒」。 + +### 練習 4：SDK 進階 +在同一次呼叫裡用 streaming + prompt caching + tool use。看成本怎麼降下來。 + +### 練習 5：Deploy +把一個 agent 包進 Docker，deploy 到雲端（任何 provider 都行）。學會把 prototype 變成可以給別人跑的東西。 + +### 練習 6：Cost Optimization（新加）⭐ +量你前面任一個練習 agent 的 token cost、加上 prompt caching、再量一次。觀察 cache hit rate 跟 cost 下降的對應關係。**Bonus**：接 [RouteLLM](https://github.com/lm-sys/RouteLLM) 或 [OpenRouter](https://openrouter.ai/)、做 cascade routing（簡單 query → Haiku / 難 query → Opus），量平均 cost。 + +## 📊 Agent Benchmark Landscape：怎麼看，不要只看排行榜 + ⚠ Reward-Hacking 警告 + +挑 model / 打造 agent 之前、你會想看 benchmark 數字——但 **2026-04 UC Berkeley 發現 8 個主流 agent benchmark 全部可被 reward-hack 到 ~100%**。下面是 2026 leaderboard 現況 + 怎麼看不被騙。 + +### 主流 Agent Benchmark 2026-05 SOTA + +| Benchmark | 領域 | 2026-05 SOTA | 領先 Model | +|---|---|---|---| +| [**SWE-bench Verified**](https://www.swebench.com/) | 軟工 / code agent | **88.6%** | Claude Opus 4.8 | +| [**Terminal-Bench**](https://github.com/laude-institute/terminal-bench) | terminal 任務 | 領先 | Claude Opus 4.8 | +| **GAIA** | general assistant | **74.6%** | Claude Sonnet 4.5（Princeton HAL）| +| [**WebArena**](https://github.com/web-arena-x/webarena) | web 導航 | **68.7%** | （領先 model 未公布）| +| [**OSWorld**](https://github.com/xlang-ai/OSWorld) | OS-level 桌面控制 | **76.26%**（SOTA、superhuman vs human 72.36%）| OpenAI CUA 38%、多數 frontier 仍卡 50% 以下 | +| [**τ-bench**](https://github.com/sierra-research/tau-bench) | tool use 多輪對話 | （較難 hack）| Anthropic / OpenAI 領先 | +| **RE-bench** | research engineering | （較難 hack、接近人類 baseline）| Frontier model | + +> **Mythos-class 層級（Claude Fable 5 — 2026-06-09 發布；2026-06-12 暫停、2026-07-01 恢復）**：[**Claude Fable 5**](https://www.anthropic.com/news/claude-fable-5-mythos-5)（`claude-fable-5`，Mythos-class、定位在 Opus 之上）是對外開放的最高能力 Claude 層級，與姊妹版 Claude Mythos 5（`claude-mythos-5`，部分安全措施放寬、限定核准客戶）同日發布。2026-06-12 曾被美國出口管制指令暫停，但出口管制 2026-06-30 解除、[Fable 5 於 2026-07-01 全球恢復](https://www.anthropic.com/news/redeploying-fable-5)（重新部署時加了新安全 classifier；Mythos 5 僅對核准的美國組織恢復）。上表數字維持原本歸屬的 model；Fable 5 官方 benchmark 數字始終未公布，故未列入。**Fable 5 又是最高階的 Claude 層級；Opus 4.8 為 Opus-class 旗艦。** + +→ 詳細排行 + 即時更新：[Agent Benchmark Leaderboard 2026](https://benchmarkingagents.com/agent-benchmarks/)、[Rapid Claw AI Agent Framework Scorecard 2026](https://rapidclaw.dev/blog/ai-agent-benchmarks-2026) + +### ⚠ Berkeley 2026-04 Reward-Hacking 警告 + +[**UC Berkeley RDI 2026-04-12 報告**](https://rdi.berkeley.edu/blog/trustworthy-benchmarks-cont/)：用 automated scanning agent 系統性 audit **8 個主流 benchmark**（SWE-bench / WebArena / OSWorld / GAIA / Terminal-Bench / FieldWorkArena / CAR-bench 等）、**每個都能 reward-hack 到接近 100%、agent 一個 task 都不用真正解**。 + +意思：leaderboard 上「Claude 87.6% / GPT 85.0%」這種數字、可能其中 X% 是 hack 出來的、不是真的解 task。 + +### 怎麼看 benchmark 不被騙 + +| 看數字方式 | 推薦 | +|---|---| +| 只看 leaderboard top | ❌ 上面 8 個都被證實可 hack | +| 看 task-level success rate breakdown | ✅ 多數 hack 集中少數 task | +| 跑你自己的 hold-out test set | ✅✅ 最可靠、production agent 必做 | +| 看 trajectory / log 是否真的解 task | ✅ 區分 reward hacking vs genuine solve | +| 看多個 benchmark + 自己 use case | ✅ 不依賴單一指標 | + +**哪些 benchmark 較難 hack（2026-05）**： +- **τ-bench** — 多輪對話 + tool use、reward function 較密集 +- **RE-bench** — research engineering 真實任務 +- **你自己的 production eval set** ⭐ 永遠是最可靠的 + +> 💡 **production agent 的 eval 紀律**： +> - 不要把外部 benchmark 數字當 ground truth、它告訴你「上限」不是「真實表現」 +> - 你自己的 eval set（內部 hold-out test）才是上線決策的依據 +> - 每次 model upgrade → 跑內部 eval set 驗證、不只看廠商公布的 benchmark 提升 +> - 接 [langfuse](https://github.com/langfuse/langfuse) / [promptfoo](https://github.com/promptfoo/promptfoo) 把 eval 自動化、每次 deploy 都跑 + +> 📊 **observability 認一個可攜標準 + 兩個評估觀念**：(1) **OpenTelemetry GenAI 慣例**（`gen_ai.*` semantic conventions）——langfuse / Arize Phoenix / Helicone 都吐 OTel-相容 span，認這層才不被單一工具綁死；OTel-native 的 [Arize Phoenix](https://github.com/Arize-ai/phoenix)（★10k）可看。(2) **pass^k**（同一題連對 k 次的機率 = 可靠度，不是只看過一次）+ [τ²-bench](https://github.com/sierra-research/tau2-bench)。(3) 多 agent 失敗有現成詞彙：**MAST**（[arXiv 2503.13657](https://arxiv.org/abs/2503.13657)、14 種失敗模式分 3 類）。 + +## 🎯 常用 Multi-Agent / Production 工具推薦（按用途分類） + +不知道從哪挑工具？下面是 2025-2026 業界常用搭配——**挑入口看「場景」、想深入點連結看 repo**： + +| 場景 | 推薦工具 | 為什麼 | +|---|---|---| +| **第一次寫 multi-agent**（最快上手）| [crewAI](https://github.com/crewAIInc/crewAI) | role-based、幾行 code 跑起來、production pattern 直接 | +| **想要 group debate / brainstorm pattern** | [AutoGen](https://github.com/microsoft/autogen) | GroupChat 自由辯論、Microsoft 出品 | +| **production 要 audit trail / checkpoint / human-in-loop** | [LangGraph](https://github.com/langchain-ai/langgraph) | state machine、控制最完整 | +| **eval 標準化**（CI / regression 必裝）| [promptfoo](https://github.com/promptfoo/promptfoo) ⭐ | YAML config、跨模型比較、★ 22k+ | +| **eval + observability 同平台** | [langfuse](https://github.com/langfuse/langfuse) ⭐ | OSS、tracing + eval + prompt mgmt、★ 28k+ | +| **不改程式、快速 instrumentation** | [Helicone](https://github.com/Helicone/helicone) | proxy 中介、不綁 framework | +| **全 stack 在 LangChain** | [LangSmith](https://www.langchain.com/langsmith)（商業）| LangChain 官方 observability | +| **打造 Claude agent**（programmatic）| [claude-agent-sdk-python](https://github.com/anthropics/claude-agent-sdk-python) ⭐ | Anthropic 官方 agent SDK、跟 Claude Code 同 runtime | +| **Deploy agent 成 API service** | [BentoML](https://github.com/bentoml/BentoML) | 最完整、Docker + serving | +| **自架開源 LLM**（取代付費 API）| [vLLM](https://github.com/vllm-project/vllm) | 高吞吐量、★ 79k+ | +| **Fine-tune 開源 LLM** | [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory) | 100+ 模型統一 SFT/DPO/PPO/GRPO、Web UI 零程式碼、中文社群最廣、★ 70k+ | + +**建議入手順序**： +1. 第一個 multi-agent：**crewAI**（role-based、最簡單） +2. 加 eval：**promptfoo**（YAML、CI 整合） +3. 加 observability：**langfuse**（OSS、完整） +4. Production 升級：換 **LangGraph**（control 強）+ **BentoML**（deploy） +5. 進階：自架 LLM 接 **vLLM**、fine-tune 用 **LLaMA-Factory** + +## 🎯 精選 Projects（範本 / SDK / 工具 collection） + +按用途分類、27 個項目一張表搞定。**挑入口看「適合誰」、想深入點連結看 repo**。 + +| 分類 | Project | ⭐ | 適合誰 | 為什麼推薦 / 備註 | +|---|---|---|---|---| +| **Multi-Agent Orchestration** | [microsoft/autogen](https://github.com/microsoft/autogen) | ⭐⭐⭐⭐⭐ | 想要 GroupChat 自由 debate pattern | Stage 4 介紹過、production 場景再回頭看 multi-agent 辯論 / brainstorming 模式 | +| | [crewAIInc/crewAI](https://github.com/crewAIInc/crewAI) | ⭐⭐⭐⭐⭐ | 想要 role-based 流水線 | 角色式 multi-agent（research → writer → reviewer），最簡單 production pattern | +| | [langchain-ai/langgraph](https://github.com/langchain-ai/langgraph) | ⭐⭐⭐⭐⭐ | 需要 audit trail / checkpoint / human-in-the-loop | state machine 路線、production 控制最強 | +| **Eval Frameworks** | [promptfoo](https://github.com/promptfoo/promptfoo) ⭐ | ⭐⭐⭐⭐⭐ | 把 eval 流程標準化、CI 整合 | YAML config、跨模型比較。★ 22k+、MIT | +| | [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) | ⭐⭐⭐⭐ | 學術 benchmark 主張（MMLU / HellaSwag / GSM8K）| 學術等級。★ 12k+、MIT | +| | [openai/evals](https://github.com/openai/evals) | ⭐⭐⭐⭐ | OpenAI 專屬 eval / 想回饋上游 | ★ 18k+ | +| **Observability** | [langfuse](https://github.com/langfuse/langfuse) ⭐ | ⭐⭐⭐⭐⭐ | 自架 production observability | OSS LangSmith 替代、traces + sessions + evals + prompt mgmt。★ 28k+、MIT | +| | [LangSmith](https://www.langchain.com/langsmith)（商業）| ⭐⭐⭐⭐ | 全 stack 在 LangChain / LangGraph 上 | LangChain 官方、只有 hosted 版 | +| | [Helicone](https://github.com/Helicone/helicone) | ⭐⭐⭐⭐ | 不想改程式、快速上 instrumentation | proxy 中介、順便拿到 logging + caching。★ 5.7k+、Apache 2.0 | +| | [weave (W&B)](https://github.com/wandb/weave) | ⭐⭐⭐⭐ | 團隊已在用 W&B 做 ML 實驗追蹤 | W&B tracing + eval、跟 wandb 整合 | +| | [comet-ml/opik](https://github.com/comet-ml/opik) | ⭐⭐⭐⭐ | eval + observability 同一個開源平台 | 追蹤 LLM / agent 做了什麼、追蹤實驗、跑品質檢查（eval）。★ 19k+、Apache 2.0 | +| | [pydantic/logfire](https://github.com/pydantic/logfire) | ⭐⭐⭐⭐ | 用 OpenTelemetry 標準追蹤 agent / LLM 呼叫 | 看清楚並 debug 你的 agent / LLM 呼叫做了什麼；Pydantic 團隊出品、建在 OpenTelemetry 標準上。★ 4k+、MIT | +| **Safety / Guardrails** | [NVIDIA-NeMo/Guardrails](https://github.com/NVIDIA-NeMo/Guardrails) | ⭐⭐⭐⭐ | 想在 agent 的輸入 / 輸出加上安全規則 | 包在 LLM app 外的安全規則——讓它不離題、擋 jailbreak、過濾不當輸出。★ 6.6k+、Apache 2.0 | +| **Anthropic SDK 進階** | [anthropic-sdk-python](https://github.com/anthropics/anthropic-sdk-python) | ⭐⭐⭐⭐⭐ | 直接基於 Claude API 做應用 | 官方 Python SDK：streaming / async / tool use / prompt caching / batches / files | +| | [anthropic-sdk-typescript](https://github.com/anthropics/anthropic-sdk-typescript) | ⭐⭐⭐⭐ | TypeScript / Node / web app | Python SDK 的 TS 版 | +| | [claude-agent-sdk-python](https://github.com/anthropics/claude-agent-sdk-python) ⭐ | ⭐⭐⭐⭐⭐ | 打造 Claude-based agent 而非只 API | 內建 tool use loop / file access / sandbox / subagent 編排；跟 Claude Code 同 runtime、想看內部運作直接讀 source。★ 6.9k+、MIT | +| | [claude-agent-sdk-typescript](https://github.com/anthropics/claude-agent-sdk-typescript) | ⭐⭐⭐⭐ | Node / web app 環境 Claude agent | Claude Agent SDK TS 版。★ 1.6k+ | +| | [Anthropic Cookbook（進階）](https://github.com/anthropics/anthropic-cookbook) | ⭐⭐⭐⭐ | 想看官方進階 SDK pattern | 特別是 `prompt_caching.ipynb` / `tool_use/` / `multimodal/` 三個 notebook | +| **Structured Output** | [BoundaryML/baml](https://github.com/BoundaryML/baml) | ⭐⭐⭐⭐ | 想穩定拿到任何模型輸出的可靠 JSON | 一個專用小語言、幫你從 LLM 穩定取得經過檢查的 JSON；支援 Claude / OpenAI / 本機模型、7 種程式語言。★ 8k+、Apache 2.0 | +| **Deployment** | [BentoML](https://github.com/bentoml/BentoML) | ⭐⭐⭐⭐ | 把 agent 包成 production API service | Docker + serving framework。★ 8k+、Apache 2.0 | +| | [LangServe](https://github.com/langchain-ai/langserve) | ⭐⭐⭐⭐ | LangChain agent 快速 deploy | 底層 FastAPI | +| | [vLLM](https://github.com/vllm-project/vllm) | ⭐⭐⭐⭐ | 自架開源 LLM 取代付費 API | 高吞吐量 LLM serving、Llama / Qwen 等。★ 79k+、Apache 2.0 | +| **中文 deploy / fine-tune** | [datawhalechina/self-llm](https://github.com/datawhalechina/self-llm) | ⭐⭐⭐⭐ | 中文團隊要自架開源 LLM | training-to-deployment 完整中文指南、Qwen / Llama / GLM / 多模態。★ 30k+、Apache 2.0 | +| | [hiyouga/LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory) | ⭐⭐⭐⭐⭐ | 要 fine-tune 開源 LLM（不只 prompt eng）| 100+ 模型統一 SFT/DPO/PPO/GRPO、Web UI 零程式碼、中文社群最廣。★ 70k+、Apache 2.0 | +| **Multi-Agent 案例研究** | [geekan/MetaGPT](https://github.com/geekan/MetaGPT) | ⭐⭐⭐⭐⭐ | 想看角色分工 + artifact 交接 pattern | SOP-based PM / Architect / Engineer multi-agent team、PRD → 設計 → code 一路產出。★ 67k+、MIT | +| | [OpenBMB/ChatDev](https://github.com/OpenBMB/ChatDev) | ⭐⭐⭐⭐ | 想看 agent debate / peer-review pattern | 對話式軟體開發、agents 在 design / code / test 互相辯論。★ 33k+、Apache 2.0、有 zh README | +| | [princeton-nlp/SWE-agent](https://github.com/princeton-nlp/SWE-agent) | ⭐⭐⭐⭐ | 理解為什麼 tool 設計 > prompt tuning | Agent-Computer Interface (ACI) 設計思路、Princeton paper-backed、SWE-Bench 領先方法。★ 19k+、MIT | + +> 🌳 **Claude 原生 subagent 機制**（不用 framework 也能 multi-agent）見 [Stage 5.5](05-claude-code-ecosystem.md#55--subagentsclaude-code-原生-multi-agent-機制-2025-新功能)。本 stage 重 framework / production；Stage 5.5 重 markdown-based subagent 編排。 + +## ✅ Stage 7 之後的自我檢查 + +你能不能： + +- [ ] 設計一個 multi-agent 系統，協作協定講得清楚 +- [ ] 在 CI 跑自動 eval pipeline +- [ ] 把 observability（tracing）接到 production agent +- [ ] 在真實 workload 上量測 prompt caching 前後的成本差異 +- [ ] 把 agent deploy 到雲端（任何 provider） + +如果都可以 → 先進 [**Stage 7.5 — 進階 Agentic 概念地圖**](07.5-advanced-agentic-concepts.md)（1 週、不寫 code、建立 frontier 概念地圖、定位業界還在討論哪些進階概念），再進 [**Stage 8 — Agent Interfaces**](08-agent-interfaces.md)（**兩 track 共用 hub**）學 agent 怎麼跟非 API 世界互動（Computer Use / Browser Use / Sandbox）。或挑一個[特化分支](../README.md#️-學習地圖兩條學習路徑)、或回過頭來貢獻這份 repo。 + +## 💡 接下來 + +你已經有基礎能力了。接下來 6-12 個月應該專注在： +1. **挑一個 production 系統** 從 prototype 推到 production +2. **回饋上游**（LangGraph、AutoGen、MCP servers、Anthropic cookbook） +3. **讀論文**——agent 研究進展很快 +4. **做出看得到的東西**——開源一個真的工具、不要只停留在寫教學 diff --git a/stages/07-multi-agent-production.zh-Hans.md b/stages/07-multi-agent-production.zh-Hans.md new file mode 100644 index 0000000..cddec1f --- /dev/null +++ b/stages/07-multi-agent-production.zh-Hans.md @@ -0,0 +1,334 @@ +# Stage 7 — 多 Agent 系统与稳定运作（Multi-Agent & Production） + +> [繁體中文](./07-multi-agent-production.md) | **简体中文** | [English](./07-multi-agent-production.en.md) + +⏱ **时间估算**：2-4 周（约 15-30 小时） + +> 💡 用语密度高（multi-agent / handoff / eval / observability / guardrails⋯）→ 翻 [`resources/glossary.md` 4 + 6](../resources/glossary.md#4-multi-agent)。 + +> 📋 **本章组成**：〔Multi-Agent · Production 化 是什么（先定位）+ 三层工程分工 + 何时用 multi-agent〕→ 学习目标 → 进入条件 → 必修阅读 → Harness Engineering（**8 个核心元件含 Cost/Latency**）→ 动手练习（含练习 6 Cost Optimization）→ **Agent Benchmark Landscape：怎么看，不要只看排行榜** → 常用工具推荐 → 精选 Projects → 自我检查 +> 🔑 **关键名词**：见 [`resources/glossary.md` 4 + 6](../resources/glossary.md#4-multi-agent)（multi-agent / orchestration / handoff / eval / observability / harness（模型外围的执行与控制层）） + +最后一个阶段。你正从“我会做 agent”走向“我能让 agent **真的给人稳定用**——多个 agent 协作、有 eval、有 observability、能部署到可用环境”。**“Production 化” ≠ enterprise scale**——只要 agent 能稳定产出 + 能让别人使用，就算进入这 stage 范围。 + +## 🎯 Multi-Agent · Production 化 是什么（先定位） + +**本 stage = 多 agent 怎么协作 + 把 agent 从 prototype 推到能稳定给人用的程度**。三句话厘清范围： + +- **不是只学 framework**——Stage 4 已教 framework 怎么挑 +- **不一定要 enterprise scale**——只要 agent 能让别人用，就算 Production 化 +- **核心是 harness engineering**——8 个核心元件 + eval + observability + cost / latency 控制 + +**跟前后 stage 的分工**： + +- **Stage 4** = 单 agent framework 怎么挑、ReAct / Plan-Execute 等 pattern +- **本 stage** = **多 agent 协作** + **harness engineering**（执行系统工程）+ **部署到可用环境 / observability / eval** + +### 三层工程分工：Prompt → Context → Harness + +工程分工可以分成三层，对应 stack 的不同位置（不是 call 一次 vs 多次的差别）： + +| 层级 | 概念 | 核心问题 | 关注单位 | 对应 stage | +|---|---|---|---|---| +| 1 | **Prompt Engineering** | 这一次要怎么问？ | **单次 LLM call** | [Stage 2](02-prompt-engineering.md) | +| 2 | **Context Engineering** | 这次该给模型哪些信息？ | **多次互动中的上下文** | [Stage 6](06-memory-rag.md) | +| **3** | **Harness Engineering**
（**本 stage**） | 整个流程怎么跑起来？ | **可执行的 LLM workflow / system** | **本 stage** | + +> 🔁 **下一层：Loop Engineering（循环工程）**：prompt → context → harness 之后，2026 浮现的第四层是“**设计 agent 的迭代循环本身**”——目标、可用工具、context 管理、**终止条件**、错误处理，让 agent 跑数百步、跨 session 仍可靠。Claude Code 的 `/goal`（给一个可验证的完成条件、agent 自己 loop 到达成）就是这个方向；[Stage 5.6 Dynamic Workflows](05-claude-code-ecosystem.zh-Hans.md) 则是 agent 自己写出 loop 脚本。谱系：ReAct（2022）→ AutoGPT（2023）→ /goal（2026）。 + +**白话差异**： +- **Prompt** = 设计一个好的问法，让模型这次回答准 +- **Context** = 动态决定要放入哪些背景、记忆、文件、工具结果，让模型知道当前情境 +- **Harness** = 把 prompt、context、tools、state、流程控制、错误处理串成一套真的能跑的系统 + +**本 stage 三个核心问题**： + +1. **Multi-agent 协作** — debate / planner-executor / peer review / handoff / supervisor-worker pattern +2. **Harness Engineering** — agent loop / tool registry（agent 可调用工具的清单 + 接口定义）/ context manager / safety / retry / telemetry / eval / cost（8 个核心元件、下面详述） +3. **Production 化** — eval harness / observability / cost & latency 优化 / 部署到可用环境 + +**跟 Stage 5 的分工**（避免混淆）： + +| 跟谁比 | 那边讲什么 | 本 stage 讲什么 | +|---|---|---| +| **Stage 5.5 Subagents** | Claude Code 原生 subagent 机制（markdown-based、不写程序）| 通用 multi-agent framework（autogen / crewAI / langgraph、跨 vendor）| +| **Stage 5.7 Claude Code source** | Claude Code source 解剖（reference implementation case study）| Harness engineering 通则（不绑特定 vendor）| + +### ⚠ 但你真的需要 multi-agent 吗？ + +**Multi-agent 不是 default，而是任务真的需要时才上的设计**。多数场景应先尝试 simple workflow 或 single agent；**只有在任务天然可分解、需要平行探索、单一 context 不够、或需要明确角色分工时，multi-agent 才值得引入**。硬上会付 **3-10× token、debug 困难、context fragmentation（context 被切散在多个 agent、彼此看不到全貌）严重**。 + +> 📌 **决策框架的 canonical 在 Stage 4**：完整的 Anthropic / Cognition 立场对照 + 4 个"该上 multi-agent"信号 + 每个信号对应的 pattern，见 [Stage 4 §什么时候真的需要 multi-agent](04-agent-frameworks.zh-Hans.md#什么时候真的需要-multi-agent不要硬上)（设计阶段决策）。本节只做 production 前的最后回头检查——**4 个信号一个都不在？** → single agent + 好 prompt + tool use 就够，别硬上 multi-agent。**本 stage 的 harness engineering 部分（8 个元件 / eval / observability）即使你最后用 single agent 也都会用到**——所以即使你决定不走 multi-agent，本 stage 仍是必修。 + +## 📌 学习目标 + +- 设计 multi-agent orchestration 模式（debate、planner-executor、peer review） +- 为 agent 架一套 evaluation harness +- 加上 observability（tracing、logging、cost tracking） +- 用 Anthropic SDK / OpenAI SDK 做 production deploy（进阶功能：streaming、prompt caching、batching） +- 把 agent deploy 到 production（Docker、serverless、monitoring） + +## 🚪 进入条件 + +你应该已经： +- 完成 Stage 4（用过至少一个 agent framework 跑 multi-agent demo） +- 完成 Stage 5（懂 MCP / Skills / Plugins / Subagents 各自角色，并用 5.7 解剖过 harness 内部） +- 完成 Stage 6（会基本 RAG，能讲出 memory pattern 差异） +- 对 Docker / git / CI 基础熟悉（production deploy 会用到） + +没到的话 → 补完前面几个 stage。本 stage 是“组合所有前面学到的东西 → 跑 production”，缺一块都会卡。 + +## 📚 必修阅读 + +1. [**Anthropic — Building Effective Agents**](https://www.anthropic.com/engineering/building-effective-agents) — 用 production 的角度再读一次 +2. [**Anthropic — Prompt Caching**](https://www.anthropic.com/news/prompt-caching) — 90% 成本下降的技巧 +3. [**Anthropic — Message Batches API**](https://docs.anthropic.com/en/docs/build-with-claude/batch-processing) — 异步 batch job +4. [**anthropics/courses — Prompt Evaluations**](https://github.com/anthropics/courses) ⭐⭐⭐⭐⭐ ★ 21k+ — Anthropic 官方 5 course umbrella、**module 4“Prompt Evaluations”对应本 stage eval / observability 部分**。Jupyter notebook、教怎么系统化评估 prompt 跟 agent 行为。 +5. **任一 eval framework 的文件** — promptfoo 或 LangSmith 或 weave +6. [**ai-boost/awesome-harness-engineering**](https://github.com/ai-boost/awesome-harness-engineering)（★ 2.8k+）— agent harness 的工具 / pattern / eval / memory / MCP / observability 全集合 +7. [**ZhangHanDong/harness-engineering-from-cc-to-ai-coding**](https://github.com/ZhangHanDong/harness-engineering-from-cc-to-ai-coding)（★ 1.4k+）— 从 Claude Code 源码学 harness 设计（中文） + +## 🏗 Harness Engineering — production agent runtime 的工程设计 ⭐ 本 stage 核心概念 + +### 定位：模型外围的执行与控制层 + +要把 LLM 变成可用的 agent，通常会碰到三层工程问题。这三层对应的是不同工程位置，不是单纯用“一次 call”或“多次 call”来区分。 + +> 💡 **Simon Willison 2025**：“coding agent = LLM + harness”；harness = 所有**不是 model 本身**的代码。 +> +> 💡 **OpenAI 2026 也使用 "Harness Engineering" 这个说法**（见 [OpenAI Harness Engineering article](https://openai.com/index/harness-engineering)、2026-02 发布）。 + +| 层级 | 工程的对象 | 在哪学 | +|---|---|---| +| **1. Prompt Engineering** | 送进 LLM 的**字符串**（system prompt / few-shot / 格式） | [Stage 2](02-prompt-engineering.md) | +| **2. Context Engineering** | 窗口里装的**信息**（RAG / memory / tool defs / history 组装） | [Stage 6](06-memory-rag.md) | +| **3. Harness Engineering**
（**本节**） | 模型**外围的执行与控制层**（loop / retry / sandbox / observability / 部署） | 本 stage | + +**怎么分辨自己在做哪一层？问**： + +1. 我改的是**字符串本身**吗？→ Prompt engineering +2. 我改的是**塞进窗口的信息**吗？→ Context engineering +3. 我改的是**调用模型的外围程序**吗？→ Harness engineering + +→ 三层**正交**：1 次 call 的 RAG app 也在做 context engineering（重点是怎么组窗口）；50 次 call 但没做 retrieval 的 chatbot，仍然只是在做 prompt engineering。 + +### Harness 的 8 个核心元件 + +**Harness Engineering（Agent 执行系统设计）= 把 LLM、tools、memory、state、workflow control、错误处理、eval、observability 与 deployment 串成一套可执行、可观测、可维护的 agent 系统。** + +→ 所有**不属于 model weights、也不只是 prompt string 本身**的工程元件都算 harness 范围。一个可部署的 agent runtime 包含这 8 个核心元件（前 6 个是 runtime 内建、第 7 个 eval 是外挂工具、第 8 个 cost / latency 是跨层议题）： + +| 元件 | 做什么 | 对应本 stage 练习 | +|---|---|---| +| **Agent loop** | “LLM → tool → result → LLM”循环、稳定处理多轮 | 练习 1 multi-agent 辩论 | +| **Tool registry** | 动态 tool dispatch、permission gate、sandboxing | （在每个 framework / SDK 都有）| +| **Context manager** | message history 管理、context window 控制、auto-compact | Stage 6 + 本 stage 练习 4 SDK | +| **Safety layer** | permission prompts、sandboxed exec、destructive op 拦截 | （Claude Code 内建、SDK 可自定义）| +| **Retry / recovery** | tool fail 怎么处理（exception vs LLM 自己看 error 反思） | 练习 4 SDK 进阶 | +| **Telemetry / Observability** | metrics、logging、token counting、trace export | **练习 3 Observability** | +| **Eval harness** | regression test、quality gate、A/B test | **练习 2 Eval** | +| **Cost / Latency optimization** ⭐ 2024-2026 必修 | prompt caching、model routing、thinking budget、batching、semantic cache | **练习 6 Cost optimization**（新加）| + +**Framework vs Harness 关键差别**： +- **Framework**（[Stage 4](04-agent-frameworks.md)）规范 **API** — 你调用的接口长什么样 +- **Harness**（本节）规范 **runtime** — 怎么跑、怎么 recovery、怎么观测 + +### 反馈循环：agent 进步靠的是反馈，不是更完美的提示 + +上面 8 个元件是 harness 的“骨架”。但让骨架真正运作的，是一件更基础的事：**agent 变强，靠的是“把反馈送回循环”，不是把开头那段提示写得更完美。** + +打个比方：一个学生不会因为作业题目写得更漂亮就变强，他变强是因为在对的时机收到反馈——交草稿、写到一半被老师提醒、完成后被批改、下次重做。agent 也一样，而反馈可以在四个时机进来： + +| 时机 | 白话 | 工程上长什么样 | +|---|---|---| +| **1. 工具返回值** | 工具吐回来的那段话，本身就是写给 agent 看的反馈 | 把错误信息、提示、下一步建议“写清楚”，别只丢一个 stack trace | +| **2. 执行中插话** | 在 agent 两次思考之间塞一句话调整方向 | 中途注入消息（steering），不用等它整轮跑完才修正 | +| **3. 单轮结束的验收** | 一轮做完，由“另一个人”对着目标检查 | 用独立的验收者（evaluator）比对目标，而不是让 agent 自己打分 | +| **4. 外层 loop** | 对着同一个目标反复叫 agent，直到完成 | 目标导向的重跑（像 OpenAI Codex 的 `/goal`、或 cron 定时重跑）| + +**为什么第 3 个（独立验收）特别重要**：Anthropic 自己的实验发现，叫 agent 检查自己的成品，它几乎都会“自我称赞”——就算质量明显普通。所以他们把“做东西的 agent”和“验收的 agent”拆开：一个负责做，一个用工具（像 Playwright）实际去点、去测，再把 bug 回报回去。把外部验收者“调得更挑剔”，比让同一个 agent“对自己更严格”容易得多。 + +> 📚 实际案例：Anthropic [Harness design for long-running apps](https://www.anthropic.com/engineering/harness-design-long-running-apps)（2026-03）用 planner → generator → evaluator 三段，让 agent 连续跑好几小时做出一个完整的音乐制作 app，每轮都靠 evaluator 反馈修正。 + +### 参考实现 + +想看实际在 production 跑的 harness 长什么样？两个 reference： + +- **Claude Code 整个 runtime** — 是 reference harness 实现。**读 source 练习见 [Stage 5.7](05-claude-code-ecosystem.md#57--claude-code-source-解剖reference-harness-implementation-track-b-必看)**（clone `claude-agent-sdk-python` 解剖 main loop + 上表前 6 个 runtime 元件位置；第 7 个 Eval harness 是外挂、第 8 个 Cost / Latency 是 cross-cutting、见下方深入段） +- **`anthropics/claude-agent-sdk-python`** source — 上面练习用的具体 repo + +→ 本 stage 剩下的 6 个练习（multi-agent / eval / observability / SDK / deploy / cost）每个都是 harness 的一个面向。学完整 stage = 拼出完整的 harness engineering mental model。 + +### 第 8 个核心元件深入 — Cost / Latency Optimization（2024-2026 Production 化必修） + +Production agent 跑久了，**cost / latency 两条线会吃掉你大半预算与用户体验**。2024-2026 前沿模型都把这当 first-class API feature——**会用 = 省 50-90% cost / latency**。 + +| 技巧 | 怎么省 | 2026 状态 | +|---|---|---| +| **Prompt caching** | 重复 prefix（system prompt、long context）一次计费、后续 cache hit 折扣 ~90% | Anthropic / OpenAI / Gemini 全支持、自动或手动标记 | +| **Model routing / cascade** | 简单 query → 小 model、难 query → frontier model | [RouteLLM](https://github.com/lm-sys/RouteLLM) / [OpenRouter](https://openrouter.ai/) production 内建 | +| **Thinking budget** | reasoning model 可控 thinking token 上限、trade latency / quality | Claude / Gemini API 参数、o-series 默认高 | +| **Speculative decoding** | 小 model 预测 N token、大 model 一次验证、单 model 速度 ×2-3 | vLLM / TGI 内建、推理层自动 | +| **Batching** | 多 query 并行处理、GPU 利用率高 | vLLM、production inference layer | +| **Semantic caching** | 相似 query 共享回答（不只 exact match）| [GPTCache](https://github.com/zilliztech/GPTCache) / Helicone 内建 | + +**Track A 怎么用**（用 CLI agent 的人）： +- 在 Claude Code / Cursor 设置 prompt caching，daily session 省 50-90% cost +- 用 [RouteLLM](https://github.com/lm-sys/RouteLLM) / [OpenRouter](https://openrouter.ai/) 动态切换 model（简单问题用 Haiku / Flash，困难问题用 Opus / Pro） +- Claude API 用 `thinking_budget` 参数控 reasoning model 的 token 上限 + +**Track B 怎么 build**（自己写 agent 的人）： +- 自架 cascade router，把 query embedding → classifier → model 对应起来 +- 在 agent loop 内监控 token cost，超 budget 自动降级 +- 在部署到可用环境时整合 semantic cache 层 +- [Helicone](https://github.com/Helicone/helicone) / [langfuse](https://github.com/langfuse/langfuse) 等 observability 平台都已内建这些能力，不用自己写 + +## 🛠 动手练习（基础 illustrative 练习） + +### 练习 1：Multi-Agent 辩论 +两个 agent 辩论一个题目（例如“该用 Python 还是 Rust 写 backend”），第三个 agent 当裁判。观察辩论收敛或分歧的 pattern。 + +### 练习 2：Eval +替你前面的 agent 写一份 eval，跑 N 次量成功率。把“我用眼睛看一下”的习惯换掉。 + +### 练习 3：Observability +把 LangSmith、Helicone、或 weave 接上一个 agent，看完整 trace。理解“没 observability 的 agent debug = 黑盒”。 + +### 练习 4：SDK 进阶 +在同一次调用里用 streaming + prompt caching + tool use。看成本怎么降下来。 + +### 练习 5：Deploy +把一个 agent 包进 Docker，deploy 到云端（任何 provider 都行）。学会把 prototype 变成可以给别人跑的东西。 + +### 练习 6：Cost Optimization（新加）⭐ +量你前面任一个练习 agent 的 token cost、加上 prompt caching、再量一次。观察 cache hit rate 跟 cost 下降的对应关系。**Bonus**：接 [RouteLLM](https://github.com/lm-sys/RouteLLM) 或 [OpenRouter](https://openrouter.ai/)、做 cascade routing（简单 query → Haiku / 难 query → Opus），量平均 cost。 + +## 📊 Agent Benchmark Landscape：怎么看，不要只看排行榜 + ⚠ Reward-Hacking 警告 + +挑 model / build agent 之前，你会想看 benchmark 数字——但 **2026-04 UC Berkeley 发现 8 个主流 agent benchmark 全部可被 reward-hack 到 ~100%**。下面是 2026 leaderboard 现况 + 怎么看不被骗。 + +### 主流 Agent Benchmark 2026-05 SOTA + +| Benchmark | 领域 | 2026-05 SOTA | 领先 Model | +|---|---|---|---| +| [**SWE-bench Verified**](https://www.swebench.com/) | 软工 / code agent | **88.6%** | Claude Opus 4.8 | +| [**Terminal-Bench**](https://github.com/laude-institute/terminal-bench) | terminal 任务 | 领先 | Claude Opus 4.8 | +| **GAIA** | general assistant | **74.6%** | Claude Sonnet 4.5（Princeton HAL）| +| [**WebArena**](https://github.com/web-arena-x/webarena) | web 导航 | **68.7%** | （领先 model 未公布）| +| [**OSWorld**](https://github.com/xlang-ai/OSWorld) | OS-level 桌面控制 | **76.26%**（SOTA、superhuman vs human 72.36%）| OpenAI CUA 38%、多数 frontier 仍卡 50% 以下 | +| [**τ-bench**](https://github.com/sierra-research/tau-bench) | tool use 多轮对话 | （较难 hack）| Anthropic / OpenAI 领先 | +| **RE-bench** | research engineering | （较难 hack、接近人类 baseline）| Frontier model | + +> **Mythos-class 层级（Claude Fable 5 — 2026-06-09 发布；2026-06-12 暂停、2026-07-01 恢复）**：[**Claude Fable 5**](https://www.anthropic.com/news/claude-fable-5-mythos-5)（`claude-fable-5`，Mythos-class、定位在 Opus 之上）是对外开放的最高能力 Claude 层级，与姊妹版 Claude Mythos 5（`claude-mythos-5`，部分安全措施放宽、限定核准客户）同日发布。2026-06-12 曾被美国出口管制指令暂停，但出口管制 2026-06-30 解除、[Fable 5 于 2026-07-01 全球恢复](https://www.anthropic.com/news/redeploying-fable-5)（重新部署时加了新安全 classifier；Mythos 5 仅对核准的美国组织恢复）。上表数字维持原本归属的 model；Fable 5 官方 benchmark 数字始终未公布，故未列入。**Fable 5 又是最高阶的 Claude 层级；Opus 4.8 为 Opus-class 旗舰。** + +→ 详细排行 + 即时更新：[Agent Benchmark Leaderboard 2026](https://benchmarkingagents.com/agent-benchmarks/)、[Rapid Claw AI Agent Framework Scorecard 2026](https://rapidclaw.dev/blog/ai-agent-benchmarks-2026) + +### ⚠ Berkeley 2026-04 Reward-Hacking 警告 + +[**UC Berkeley RDI 2026-04-12 报告**](https://rdi.berkeley.edu/blog/trustworthy-benchmarks-cont/)：用 automated scanning agent 系统性 audit **8 个主流 benchmark**（SWE-bench / WebArena / OSWorld / GAIA / Terminal-Bench / FieldWorkArena / CAR-bench 等）、**每个都能 reward-hack 到接近 100%、agent 一个 task 都不用真正解**。 + +意思：leaderboard 上“Claude 87.6% / GPT 85.0%”这种数字、可能其中 X% 是 hack 出来的、不是真的解 task。 + +### 怎么看 benchmark 不被骗 + +| 看数字方式 | 推荐 | +|---|---| +| 只看 leaderboard top | ❌ 上面 8 个都被证实可 hack | +| 看 task-level success rate breakdown | ✅ 多数 hack 集中少数 task | +| 跑你自己的 hold-out test set | ✅✅ 最可靠、production agent 必做 | +| 看 trajectory / log 是否真的解 task | ✅ 区分 reward hacking vs genuine solve | +| 看多个 benchmark + 自己 use case | ✅ 不依赖单一指标 | + +**哪些 benchmark 较难 hack（2026-05）**： +- **τ-bench** — 多轮对话 + tool use、reward function 较密集 +- **RE-bench** — research engineering 真实任务 +- **你自己的 production eval set** ⭐ 永远是最可靠的 + +> 💡 **production agent 的 eval 纪律**： +> - 不要把外部 benchmark 数字当 ground truth、它告诉你“上限”不是“真实表现” +> - 你自己的 eval set（内部 hold-out test）才是上线决策的依据 +> - 每次 model upgrade → 跑内部 eval set 验证、不只看厂商公布的 benchmark 提升 +> - 接 [langfuse](https://github.com/langfuse/langfuse) / [promptfoo](https://github.com/promptfoo/promptfoo) 把 eval 自动化、每次 deploy 都跑 + +> 📊 **observability 认一个可携标准 + 两个评估观念**：(1) **OpenTelemetry GenAI 惯例**（`gen_ai.*` semantic conventions）——langfuse / Arize Phoenix / Helicone 都吐 OTel-兼容 span，认这层才不被单一工具绑死；OTel-native 的 [Arize Phoenix](https://github.com/Arize-ai/phoenix)（★10k）可看。(2) **pass^k**（同一题连对 k 次的概率 = 可靠度，不是只看过一次）+ [τ²-bench](https://github.com/sierra-research/tau2-bench)。(3) 多 agent 失败有现成词汇：**MAST**（[arXiv 2503.13657](https://arxiv.org/abs/2503.13657)、14 种失败模式分 3 类）。 + +## 🎯 常用 Multi-Agent / Production 工具推荐（按用途分类） + +不知道从哪挑工具？下面是 2025-2026 业界常用搭配——**挑入口看“场景”、想深入点链接看 repo**： + +| 场景 | 推荐工具 | 为什么 | +|---|---|---| +| **第一次写 multi-agent**（最快上手）| [crewAI](https://github.com/crewAIInc/crewAI) | role-based、几行 code 跑起来、production pattern 直接 | +| **想要 group debate / brainstorm pattern** | [AutoGen](https://github.com/microsoft/autogen) | GroupChat 自由辩论、Microsoft 出品 | +| **production 要 audit trail / checkpoint / human-in-loop** | [LangGraph](https://github.com/langchain-ai/langgraph) | state machine、控制最完整 | +| **eval 标准化**（CI / regression 必装）| [promptfoo](https://github.com/promptfoo/promptfoo) ⭐ | YAML config、跨模型比较、★ 22k+ | +| **eval + observability 同平台** | [langfuse](https://github.com/langfuse/langfuse) ⭐ | OSS、tracing + eval + prompt mgmt、★ 28k+ | +| **不改程序、快速 instrumentation** | [Helicone](https://github.com/Helicone/helicone) | proxy 中介、不绑 framework | +| **全 stack 在 LangChain** | [LangSmith](https://www.langchain.com/langsmith)（商业）| LangChain 官方 observability | +| **打造 Claude agent**（programmatic）| [claude-agent-sdk-python](https://github.com/anthropics/claude-agent-sdk-python) ⭐ | Anthropic 官方 agent SDK、跟 Claude Code 同 runtime | +| **Deploy agent 成 API service** | [BentoML](https://github.com/bentoml/BentoML) | 最完整、Docker + serving | +| **自架开源 LLM**（取代付费 API）| [vLLM](https://github.com/vllm-project/vllm) | 高吞吐量、★ 79k+ | +| **Fine-tune 开源 LLM** | [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory) | 100+ 模型统一 SFT/DPO/PPO/GRPO、Web UI 零 code、中文社群最广、★ 70k+ | + +**建议入手顺序**： +1. 第一个 multi-agent：**crewAI**（role-based、最简单） +2. 加 eval：**promptfoo**（YAML、CI 整合） +3. 加 observability：**langfuse**（OSS、完整） +4. Production 升级：换 **LangGraph**（control 强）+ **BentoML**（deploy） +5. 进阶：自架 LLM 接 **vLLM**、fine-tune 用 **LLaMA-Factory** + +## 🎯 精选 Projects（范本 / SDK / 工具 collection） + +按用途分类、27 个项目一张表搞定。**挑入口看“适合谁”、想深入点链接看 repo**。 + +| 分类 | Project | ⭐ | 适合谁 | 为什么推荐 / 备注 | +|---|---|---|---|---| +| **Multi-Agent Orchestration** | [microsoft/autogen](https://github.com/microsoft/autogen) | ⭐⭐⭐⭐⭐ | 想要 GroupChat 自由 debate pattern | Stage 4 介绍过、production 场景再回头看 multi-agent 辩论 / brainstorming 模式 | +| | [crewAIInc/crewAI](https://github.com/crewAIInc/crewAI) | ⭐⭐⭐⭐⭐ | 想要 role-based 流水线 | 角色式 multi-agent（research → writer → reviewer），最简单 production pattern | +| | [langchain-ai/langgraph](https://github.com/langchain-ai/langgraph) | ⭐⭐⭐⭐⭐ | 需要 audit trail / checkpoint / human-in-the-loop | state machine 路线、production 控制最强 | +| **Eval Frameworks** | [promptfoo](https://github.com/promptfoo/promptfoo) ⭐ | ⭐⭐⭐⭐⭐ | 把 eval 流程标准化、CI 整合 | YAML config、跨模型比较。★ 22k+、MIT | +| | [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) | ⭐⭐⭐⭐ | 学术 benchmark 主张（MMLU / HellaSwag / GSM8K）| 学术等级。★ 12k+、MIT | +| | [openai/evals](https://github.com/openai/evals) | ⭐⭐⭐⭐ | OpenAI 专属 eval / 想回馈上游 | ★ 18k+ | +| **Observability** | [langfuse](https://github.com/langfuse/langfuse) ⭐ | ⭐⭐⭐⭐⭐ | 自架 production observability | OSS LangSmith 替代、traces + sessions + evals + prompt mgmt。★ 28k+、MIT | +| | [LangSmith](https://www.langchain.com/langsmith)（商业）| ⭐⭐⭐⭐ | 全 stack 在 LangChain / LangGraph 上 | LangChain 官方、只有 hosted 版 | +| | [Helicone](https://github.com/Helicone/helicone) | ⭐⭐⭐⭐ | 不想改程序、快速上 instrumentation | proxy 中介、顺便拿到 logging + caching。★ 5.7k+、Apache 2.0 | +| | [weave (W&B)](https://github.com/wandb/weave) | ⭐⭐⭐⭐ | 团队已在用 W&B 做 ML 实验追踪 | W&B tracing + eval、跟 wandb 整合 | +| | [comet-ml/opik](https://github.com/comet-ml/opik) | ⭐⭐⭐⭐ | eval + observability 同一个开源平台 | 追踪 LLM / agent 做了什么、追踪实验、跑质量检查（eval）。★ 19k+、Apache 2.0 | +| | [pydantic/logfire](https://github.com/pydantic/logfire) | ⭐⭐⭐⭐ | 用 OpenTelemetry 标准追踪 agent / LLM 调用 | 看清楚并 debug 你的 agent / LLM 调用做了什么；Pydantic 团队出品、建在 OpenTelemetry 标准上。★ 4k+、MIT | +| **Safety / Guardrails** | [NVIDIA-NeMo/Guardrails](https://github.com/NVIDIA-NeMo/Guardrails) | ⭐⭐⭐⭐ | 想在 agent 的输入 / 输出加上安全规则 | 包在 LLM app 外的安全规则——让它不离题、挡 jailbreak、过滤不当输出。★ 6.6k+、Apache 2.0 | +| **Anthropic SDK 进阶** | [anthropic-sdk-python](https://github.com/anthropics/anthropic-sdk-python) | ⭐⭐⭐⭐⭐ | 直接基于 Claude API 做应用 | 官方 Python SDK：streaming / async / tool use / prompt caching / batches / files | +| | [anthropic-sdk-typescript](https://github.com/anthropics/anthropic-sdk-typescript) | ⭐⭐⭐⭐ | TypeScript / Node / web app | Python SDK 的 TS 版 | +| | [claude-agent-sdk-python](https://github.com/anthropics/claude-agent-sdk-python) ⭐ | ⭐⭐⭐⭐⭐ | 打造 Claude-based agent 而非只 API | 内建 tool use loop / file access / sandbox / subagent 编排；跟 Claude Code 同 runtime、想看内部运作直接读 source。★ 6.9k+、MIT | +| | [claude-agent-sdk-typescript](https://github.com/anthropics/claude-agent-sdk-typescript) | ⭐⭐⭐⭐ | Node / web app 环境 Claude agent | Claude Agent SDK TS 版。★ 1.6k+ | +| | [Anthropic Cookbook（进阶）](https://github.com/anthropics/anthropic-cookbook) | ⭐⭐⭐⭐ | 想看官方进阶 SDK pattern | 特别是 `prompt_caching.ipynb` / `tool_use/` / `multimodal/` 三个 notebook | +| **Structured Output** | [BoundaryML/baml](https://github.com/BoundaryML/baml) | ⭐⭐⭐⭐ | 想稳定拿到任何模型输出的可靠 JSON | 一个专用小语言、帮你从 LLM 稳定取得经过检查的 JSON；支持 Claude / OpenAI / 本地模型、7 种编程语言。★ 8k+、Apache 2.0 | +| **Deployment** | [BentoML](https://github.com/bentoml/BentoML) | ⭐⭐⭐⭐ | 把 agent 包成 production API service | Docker + serving framework。★ 8k+、Apache 2.0 | +| | [LangServe](https://github.com/langchain-ai/langserve) | ⭐⭐⭐⭐ | LangChain agent 快速 deploy | 底层 FastAPI | +| | [vLLM](https://github.com/vllm-project/vllm) | ⭐⭐⭐⭐ | 自架开源 LLM 取代付费 API | 高吞吐量 LLM serving、Llama / Qwen 等。★ 79k+、Apache 2.0 | +| **中文 deploy / fine-tune** | [datawhalechina/self-llm](https://github.com/datawhalechina/self-llm) | ⭐⭐⭐⭐ | 中文团队要自架开源 LLM | training-to-deployment 完整中文指南、Qwen / Llama / GLM / 多模态。★ 30k+、Apache 2.0 | +| | [hiyouga/LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory) | ⭐⭐⭐⭐⭐ | 要 fine-tune 开源 LLM（不只 prompt eng）| 100+ 模型统一 SFT/DPO/PPO/GRPO、Web UI 零 code、中文社群最广。★ 70k+、Apache 2.0 | +| **Multi-Agent 案例研究** | [geekan/MetaGPT](https://github.com/geekan/MetaGPT) | ⭐⭐⭐⭐⭐ | 想看角色分工 + artifact 交接 pattern | SOP-based PM / Architect / Engineer multi-agent team、PRD → 设计 → code 一路产出。★ 67k+、MIT | +| | [OpenBMB/ChatDev](https://github.com/OpenBMB/ChatDev) | ⭐⭐⭐⭐ | 想看 agent debate / peer-review pattern | 对话式软件开发、agents 在 design / code / test 互相辩论。★ 33k+、Apache 2.0、有 zh README | +| | [princeton-nlp/SWE-agent](https://github.com/princeton-nlp/SWE-agent) | ⭐⭐⭐⭐ | 理解为什么 tool 设计 > prompt tuning | Agent-Computer Interface (ACI) 设计思路、Princeton paper-backed、SWE-Bench 领先方法。★ 19k+、MIT | + +> 🌳 **Claude 原生 subagent 机制**（不用 framework 也能 multi-agent）见 [Stage 5.5](05-claude-code-ecosystem.zh-Hans.md#55--subagentsclaude-code-原生-multi-agent-机制-2025-新功能)。本 stage 重 framework / production；Stage 5.5 重 markdown-based subagent 编排。 + +## ✅ Stage 7 之后的自我检查 + +你能不能： + +- [ ] 设计一个 multi-agent 系统，协作协定讲得清楚 +- [ ] 在 CI 跑自动 eval pipeline +- [ ] 把 observability（tracing）接到 production agent +- [ ] 在真实 workload 上量测 prompt caching 前后的成本差异 +- [ ] 把 agent deploy 到云端（任何 provider） + +如果都可以 → 先进 [**Stage 7.5 — 进阶 Agentic 概念地图**](07.5-advanced-agentic-concepts.md)（1 周、不写 code、建立 frontier 概念地图、定位业界还在讨论哪些进阶概念），再进 [**Stage 8 — Agent Interfaces**](08-agent-interfaces.md)（**两 track 共用 hub**）学 agent 怎么跟非 API 世界互动（Computer Use / Browser Use / Sandbox）。或挑一个[特化分支](../README.zh-Hans.md#-学习地图两条学习路径)、或回头来贡献这份 repo。 + +## 💡 接下来 + +你已经有基础能力了。接下来 6-12 个月应该专注在： +1. **挑一个 production 系统** 从 prototype 推到 production +2. **回馈上游**（LangGraph、AutoGen、MCP servers、Anthropic cookbook） +3. **读论文**——agent 研究进展很快 +4. **做出看得到的东西**——开源一个真的工具，不要再写教学了 diff --git a/stages/07.5-advanced-agentic-concepts.en.md b/stages/07.5-advanced-agentic-concepts.en.md new file mode 100644 index 0000000..682846c --- /dev/null +++ b/stages/07.5-advanced-agentic-concepts.en.md @@ -0,0 +1,628 @@ +# Stage 7.5 — Advanced Agentic Concepts Map + +> [繁體中文](./07.5-advanced-agentic-concepts.md) | [简体中文](./07.5-advanced-agentic-concepts.zh-Hans.md) | **English** + +⏱ **Estimated Time**: 1 week (about 5 hours — no coding, just reading resources to build the concept map) + +> 🚪 **Entry condition**: complete [Stage 7 — Multi-Agent · Productionization](07-multi-agent-production.md) (or at least Stages 4 + 6 + 7). This chapter is a frontier concept map for *after* productionization, not an intro — without having built a production agent you will not feel what pain points these concepts solve. + +> 💡 This is an **advanced-concept map + reading path**, not a full tutorial. After Stages 4 / 6 / 7, you can already build production **agents** (AI systems that plan + execute tasks autonomously — LLM-based programs that drive their own actions; "production agents" are agents that real users can rely on without frequent breakdowns); this stage helps you locate **which advanced concepts are still being debated in the industry**, **what problem each concept solves**, and **which papers / blogs to read first**, so you do not step into problems that others have already hit in real work. + +> 📋 **Chapter Structure** (8 sections, read in order): +> +> 1. Why this stage exists (positioning) +> 2. Concept-map spine: the **Types → Config → Repo → Service** four-layer work boundary +> 3. 12 advanced concept skeletons +> 4. Why these 12 were chosen +> 5. Cross-concept Harness Engineering Principles (4 categories + dependency map) +> 6. Advanced agentic application flow (5 steps) +> 7. Complete reading path +> 8. Self-check + +> 🔤 **Quick abbreviation reference** (these recur throughout the chapter): +> +> | Abbreviation | Full form | One-line meaning | +> |---|---|---| +> | **agent** | AI autonomous executor | An LLM-based system that plans + executes tasks on its own | +> | **PR** | Pull Request | A request to merge a change into the main branch (GitHub term) | +> | **SoR** | System of Record | The authoritative source of truth for knowledge | +> | **ACI** | Agent-Computer Interface | The interface layer between an agent and the system (tools / APIs / docs) | +> | **MCP** | Model Context Protocol | A spec for standardizing agent tools | +> | **PAR** | Plan-Act-Reflect | A single-agent self-loop pattern (plan → act → reflect → revise → retry) | +> | **CI** | Continuous Integration | The system that auto-runs tests / linters on every commit | +> | **QA** | Quality Assurance | Quality gatekeeping (human or automated) | +> | **lint / linter** | — | A tool that auto-scans code for rule violations | +> | **`[OAI]` / `[Anth]`** | OpenAI / Anthropic | Source tags used later in the chapter | + +## 🎯 Why this stage exists + +Stages 4 / 6 / 7 together cover **70% of agents real users can rely on without frequent breakdowns**. What each stage teaches and what you can do after: + +| Stage | Teaches | What you can do | +|---|---|---| +| **4** | Choosing a **framework** (which tool to build agents with) | Pick LangGraph / AutoGen / DSPy and build an agent | +| **6** | **Context engineering** (dynamically managing what you feed the agent) | memory / retrieval / prompt assembly | +| **7** | **Harness engineering** (the reliable runtime environment around an agent) | observability / retry / cost gates / eval / sandbox — the 8 components | +| **7.5 (this chapter)** | **The advanced-concept map** | Know which paper to open when problems appear / see what someone else's agent is actually doing / map each concept to a layer of the agent system | + +But frontier AI labs (Anthropic / OpenAI / Cognition / Microsoft) + academia (Stanford / CMU / Princeton) have continued publishing 12+ advanced design concepts from 2024 to 2026. **Some you won't use yet, but you need to know they exist** — so that when a problem shows up later, you know there is a ready-made pattern to borrow. This stage is not another theory chapter; it is a **map**: + +- It is not asking you to master all of them +- It is not asking you to use all of them +- It helps you **know which paper / blog to open when a problem appears** +- It helps you **see what someone else's agent is actually doing** — e.g., when their agent hits an error, is it just "retry N times then give up" (= retry only), or "after an error, think a round, fix the approach, then try again" (= the plan-act-reflect loop pattern)? Those are entirely different design grades. Being able to tell them apart is how you decide whether to copy their approach +- It helps you **know which part of the agent system each concept maps to, and which kind of problem it solves** + +## 🧭 Concept-map spine: the four-layer work boundary + +This stage uses the **work boundary** as the spine for organizing advanced agentic workflows: split an agent system into 4 layers (**Types → Config → Repo → Service**, expanded below), then ask "which layer is the agent operating on, and what breaks when it crosses a layer?" This is not collapsing the entire chapter into a single model — it gives the reader a coordinate system first, so the 12 concepts that follow can all be placed on the same map and compared. + +> 💡 **What "stack" means**: software engineering convention splits a system into top-to-bottom layers, each layer doing one job, upper layers sitting on top of lower ones — collectively called a *stack*. A common web app is a 3-layer stack: frontend → backend → database. This stage splits an agent system into 4 layers (Types / Config / Repo / Service) and asks which layer the agent should be touching. + +> ⚠️ **These 4 layers are different from the Stage 7 prompt → context → harness layers. They are two different views**: +> - **Prompt → Context → Harness** (Stage 7): **stack position** — are you engineering the string, the information, or the surrounding runtime? +> - **Types → Config → Repo → Service** (this stage): **scope of autonomy** — how deep into the stack can the agent act? Is crossing layers a violation? +> +> The two views are **orthogonal** and solve different problems. After this section, you should be able to look at agent systems through both lenses at the same time. + +Borrow software architecture layering — **Types → Config → Repo → Service** — and apply it to agent systems: + +![Agentic Stack 4-Layer Work Boundary](../resources/diagrams/stack-4layer.en.png) + +→ Every layer boundary is a **work boundary**. The scope the agent operates on = the scope of its autonomy: + +- **Agent at Types layer** = can only fit an existing contract, cannot change the schema (example: Codex receives a brief and adds inline glosses) +- **Agent at Config layer** = can adjust budget / policy but cannot modify memory (example: a context-budget agent changes `max_cost_usd`) +- **Agent at Repo layer** = can read and write memory / vector stores but cannot redesign the workflow +- **Agent at Service layer** = can recompose the whole workflow; this is the highest autonomy + +### Why the work boundary fits as the spine + +Many advanced concepts eventually trace back to the same question: how far does the agent's autonomy actually go? Think of an agent like a new intern: you give them a clear, narrow task, and they take it on themselves to touch nearby things too — that's a "work-boundary violation". The industry has 3 publicly-documented real cases that map onto this: + +- **Didn't stop at the boundary** (Cognition's Flappy Bird case): a **multi-agent** (multiple agents collaborating in parallel) system was tasked to build Flappy Bird. One **sub-agent** (a child agent spawned by the main agent to execute one sub-task) built the green pipes; another built the cloud background — and the two clashed visually because neither knew what the other was doing, i.e. neither had the other's **context** (the full set of information the agent receives). Cognition put it bluntly: "sub-agents are like a team of overconfident new hires — they won't ask the questions they should be asking." + → Source: [Cognition — Don't Build Multi-Agents (2025-06)](https://cognition.ai/blog/dont-build-multi-agents) + +- **Added unrequested extras** (Anthropic's "speculative leap" finding): a sub-agent assigned to research a topic would insert lines like "I also speculate that X might hold, though I haven't verified it" into the final report — unsolicited. Anthropic's multi-agent paper specifically discusses why this "helpful filling-in" needs to be engineered out, otherwise hallucinations smuggle themselves past the supervisor. + → Source: [Anthropic — How we built our multi-agent research system (2025-06)](https://www.anthropic.com/engineering/built-multi-agent-research-system) + +- **Operator granted too much permission** (Replit Agent 2024 prod-database incident): per community reporting, a user gave an agent direct production database access without a "destructive operations require confirmation" gate. While "fixing a bug" the agent ran a destructive SQL command that wiped production data. The agent followed instructions reasonably; the fault was the operator not setting boundaries. + → Source: [Simon Willison's analysis of the incident (2024)](https://simonwillison.net/2024/Aug/26/replit/) (community write-up, not an official Replit postmortem) + +**What these 3 cases tell you**: + +- Agents do not "naturally stop at the point you assigned" — your brief must explicitly say "**only touch X, do NOT touch Y**", and sub-agents must receive the parent's full context. +- Agents will proactively "fill in" things not requested — use structured output schemas + evaluator-optimizer loops to filter speculative content. +- A rule "being installed ≠ being followed" — operator self-discipline is not enough. You need mechanical gates (permission check, cost cap, destructive-op confirmation) to prevent humans from bypassing the rule with "just this once". + +→ **How this maps to tools**: write the work boundary into the brief (Anthropic's brief template / LangGraph state schemas / `agent-collab-skills`' task-splitter — all the same idea), enforce it at an acceptance gate / evaluator loop, and put an explicit gate in front of destructive operations (covered in [§7 Autonomy Gradients](#-12-advanced-concepts--skeleton)). + +### 🔁 Failure-mode lifecycle (how industry agent failures evolved into best practice) + +![Failure-Mode Evolution Cycle](../resources/diagrams/failure-lifecycle.en.png) + +Every industry-grade agent failure mode goes through the same loop: **discover incident → publicly document → encode as a framework pattern → eliminate automatically**. Five publicly documented cases: + +| # | Incident (discovered) | Documented name | Codify (which pattern it became) | Public source | +|---|---|---|---|---| +| 1 | Multi-agent subagent context drift (Flappy Bird style mismatch) | "Sub-agents don't share principal-agent context" | **Single-thread principle**: don't stack multi-agents — use linear orchestration | Cognition 2025-06 | +| 2 | Subagent speculative leap (unverified speculation smuggled into output) | "Speculative hallucination via filling-in" | **Evaluator-optimizer loop**: add a critique step that forces review | Anthropic Multi-Agent Research 2025-06 | +| 3 | Production permission drift (agent dropped prod DB) | "Unbounded autonomy on destructive ops" | **Autonomy gradient**: suggest / propose / execute tiered authorization | Replit Agent 2024 incident | +| 4 | Agent looping without self-criticism (AutoGPT stuck loops) | "Reflexion-less iteration" | **Plan-Act-Reflect loop**: add self-critique + revise step | Reflexion paper (Shinn 2023) | +| 5 | Skill library corruption (broken skill enters library) | "Untested skill commit" | **Pre-verify before commit**: skill must pass tests before joining the library | Voyager paper (Wang 2024) | + +→ **This "fail → publish → codify → fix" loop is the evolution mechanism of the entire agentic field** — not "write every rule up front," but "**every production incident gets published + codified into a pattern**". Anthropic Skills `references/`, OpenAI Taste Invariants, LangChain's evaluator pattern, Anthropic's evaluator-optimizer — they are the same logic in different implementations. + +→ **How to use this table**: when your own agent fails, find the row in the table that resembles your failure, then read the deep-dive for the matching pattern (Single-thread / Evaluator-optimizer / Autonomy gradient / PAR / Pre-verify). The 12 skeletons later in this stage cover all 5 patterns. + +## 📚 12 advanced concepts — skeleton + +Each concept stays within 4 lines: a one-sentence definition + which layer of the stack it belongs to + the single best resource to read. + +### 🗺️ 12-concept cluster map (which layer × problem type) + +![12 Advanced Agentic AI Concepts — Cluster Map](../resources/diagrams/concept-cluster.en.png) + +The diagram above groups the 12 concepts by **which layer they touch** (horizontal axis) and **what kind of problem they solve** (vertical axis), so you can see which concepts should be learned together and which you can skip for now. Note that **Work Boundary (#1) spans all layers** (it is a discipline that applies everywhere, not one specific position). + +→ **How to use this map** +- **First pass**: learn the **orchestration + reflection** concepts first (6 total; the foundation for multi-agent / production work) +- **Before production deployment**: add the **governance + resilience** concepts (6 total; these keep deployments from breaking) +- **Cross-category root**: **Work Boundary (#1) is the root discipline that runs through all 12 concepts** + +The 12 concepts in table form (# / concept / which layer / one-line definition / best reading): + +| # | Concept | Which layer | One-line definition | Best reading | +|---|---|---|---|---| +| 1 | **Work Boundary / Scope Discipline** | Across all layers (discipline) | The agent only touches what the brief names; it does not overstep | [Hamel — Evals + Skills](https://hamel.dev/blog/posts/evals-skills/) + [Cognition — Don't Build Multi-Agents](https://cognition.ai/blog/dont-build-multi-agents) | +| 2 | **Contract-driven Hand-offs** | Types + Service | Upstream agents promise artifacts; downstream agents must verify they received them | [Anthropic — Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents) Routing pattern | +| 3 | **Speculative / Parallel Exploration** | Service (orchestration) | Run N alternative paths and keep the best one (not just independent parallelism) | [LangGraph Plan-Execute Tutorial](https://blog.langchain.com/planning-agents/) | +| 4 | **Agent-as-Judge / Constitutional AI** | Service (agent evaluates agent) | Use one agent to evaluate another's output, iteratively revising against explicit principles | [Constitutional AI (Bai 2022)](https://arxiv.org/abs/2212.08073) | +| 5 | **Plan-Act-Reflect Loop** | Service (single-agent self-loop) | write plan → execute → critique → revise → re-execute until PASS or EXHAUSTED | [Reflexion (Shinn 2023)](https://arxiv.org/abs/2303.11366) + [Self-Discover (Zhou ICML 2024)](https://arxiv.org/abs/2402.03620) | +| 6 | **Hierarchical Task Decomposition** | Service (multi-layer supervisor) | supervisor → worker → sub-worker, at least 2 layers of recursion | [Microsoft AutoGen GroupChat docs](https://microsoft.github.io/autogen/) | +| 7 | **Autonomy Gradients / Trust Layers** | Config (autonomy policy) | Different tasks get different autonomy levels (suggest / propose / execute) | [Claude Code permission system](https://docs.claude.com/en/docs/agents-and-tools/claude-code/overview) | +| 8 | **Cost-aware Budget Gates** | Config (cost policy) | Auto-stop or escalate review when a task exceeds a dollar budget (not just a token cap) | [OpenAI Harness Engineering (2026-02)](https://openai.com/index/harness-engineering) | +| 9 | **Failure Injection / Chaos Eval** | Service (test agent fault tolerance) | Intentionally feed broken input / stale data / API timeouts and observe how the agent responds | [Hamel Husain — Evals blog series](https://hamel.dev/blog/posts/evals/) | +| 10 | **Self-organizing Teams** | Service (agents negotiate roles) | Agents aren't pre-assigned roles; they divide work dynamically based on the task | [CAMEL (Li 2023)](https://arxiv.org/abs/2303.17760) + AutoGen | +| 11 | **Spec-driven Development** | Types (spec = code) | Agent tasks are defined by formal specs (YAML / JSON Schema), not free-form prompting | [DSPy](https://github.com/stanfordnlp/dspy) signatures tutorial | +| 12 | **Graceful Degradation Paths** | Config (fallback policy) | When the frontier model fails, fall back to a cheaper model with reduced expectations rather than crash | [OpenRouter routing docs](https://openrouter.ai/docs) + [Anthropic model fallback](https://docs.claude.com/en/docs/build-with-claude/models) | + +## Why these 12 + +- They all have verifiable primary sources (Anthropic / OpenAI / Cognition / Microsoft / academic papers) — not hand-wavy claims +- They all map to at least one public implementation (LangGraph / AutoGen / Anthropic Skills / DSPy etc.) — directly copyable +- They sit outside what Stages 4 / 6 / 7 already cover, so they are not repeats +- They avoid infinite expansion — other advanced concepts (Voyager skill learning / MemoryLLM / world models) matter, but **learn these 12 first** + +## 🔬 Cross-concept Harness Engineering Principles (multi-source synthesis) + +**These principles do not come from any single vendor.** Anthropic, OpenAI, Cognition, Hamel Husain, and others all describe them across blog posts, engineering writeups, and docs. The wording differs, but the design constraints are the same. Start by grouping them into **4 major categories**, listing the main sources, and then expand from there. + +> 📚 Primary sources: +> - **Anthropic** (Building Effective Agents · Skills · Multi-Agent Research · CLAUDE.md memory docs) +> - **OpenAI** ([Harness Engineering 2026-02](https://openai.com/index/harness-engineering/), which organizes them most clearly into 5 named principles) +> - **Cognition AI** ([Don't Build Multi-Agents](https://cognition.ai/blog/dont-build-multi-agents)) +> - **Hamel Husain** ([Evals are everything](https://hamel.dev/blog/posts/evals/)) +> - **Lilian Weng** ([LLM Powered Autonomous Agents](https://lilianweng.github.io/posts/2023-06-23-agent/)) + +> 🔤 **Source tag shorthand for tables below** (these 4 tags reappear throughout the chapter): +> - `[OAI]` = OpenAI +> - `[Anth]` = Anthropic +> - `[Cognition]` = Cognition AI +> - `[Hamel]` = Hamel Husain + +### 4 categories × multiple sources + +| Category | Core question | Principles in this category (with source) | +|---|---|---| +| **① Context management** | How do you keep context from exploding while ensuring the agent always gets the right information? | **System of Record** [OAI] / **Memory Persistence** [Anth] / **Progressive Disclosure** [OAI + Anth] | +| **② Interface / communication** | How do you make the codebase legible to the agent and the agent legible to humans? | **Legibility** [OAI] / **ACI / Tool Documentation** [Anth] / **Transparency** (show planning) [Anth] | +| **③ Quality / verification** | How do you make the output correct and non-hallucinatory? | **Taste Invariants** [OAI] / **Evaluator-Optimizer loop** [Anth] / **Human + LLM-as-Judge** [Anth] / **"Evals are everything"** [Hamel] | +| **④ Process discipline** | How do you scale and iterate without the system blowing up? | **Simplicity** [Anth] / **Throughput Changes Merge Philosophy** [OAI] / **Don't Build Multi-Agents (when unnecessary)** [Cognition] | + +→ **OpenAI's 5 principles are the clearest named packaging with the strongest case study**, but category ①'s SoR / Memory Persistence, category ②'s ACI, category ③'s evaluator-optimizer loop, and category ④'s Simplicity all appear in Anthropic and other sources first. The rest of this chapter keeps OpenAI's naming because the writeup is the most complete, while cross-mapping each section back to Anthropic and others. + +### Main relationships between the principles (cross-category dependencies) + +These are not 5 isolated principles, and they are not 12 unrelated concepts. There are clear **enabling relationships** between them: + +![Cross-source dependency graph for 5 Harness Engineering principles](../resources/diagrams/principle-dependency.en.png) + +→ **4 relationship insights**: + +| Relationship | What it means | Why it matters | +|---|---|---| +| **SoR + Memory + PD form a bundle** | SoR provides the destination, Memory Persistence carries facts across sessions, Progressive Disclosure is the navigation mechanism | None of the three is complete on its own; must design together | +| **Legibility ↔ Transparency bidirectional** | The agent must read the codebase well to self-report well; the agent must self-report well so you can verify legibility | Each is a prerequisite for the other | +| **Quality is the prerequisite for Process automation** | Without explicit invariants + an eval loop in place, humans cannot safely hand review over to automation | Necessary condition for category ④ | +| **Simplicity is the hidden root** | Stacking multi-agent complexity too early causes every other principle's cost to balloon | Cognition's "Don't Build Multi-Agents" = Anthropic's "Simplicity" — same argument | + +→ The 5 sections below still use OpenAI's naming because it is the most complete articulation, while each section maps back to the corresponding Anthropic / cross-vendor source. + +### Why these principles matter — Why → What → How + +The table below explains the principles in three layers: **the pain point (Why) → the principle (What) → the concrete tool (How)** that solves it: + +| Pain point (Why) | Principle (What) | Tool / mechanism (How) | +|---|---|---| +| Context 200k cap / Multi-agent context overflow | Progressive Disclosure + Memory Persistence | Skills `references/` / `CLAUDE.md` `@-import` / `.ai/` brief | +| Agent can't read its own codebase / docs | Legibility + Tool Doc / ACI | `AGENTS.md` (100 ln) / poka-yoke tool API / consistent schema | +| Multi-agent desync, multiple "truths" | System of Record | `docs/` + `.coord/` shared-memory skill | +| Random drift / review misses it | Taste Invariants + Transparency (show planning) | `agent-acceptance-gate` preset YAMLs / evaluator-optimizer loop | +| Agent ships PRs faster than human QA | Throughput Changes Merge Philosophy | mandatory preset / LLM-as-judge / human spot-check | +| Jumping to multi-agent from day 1 | Simplicity (Anthropic) | Start with a basic LLM call; add an agent only when needed | + +→ **6 pain points → 5 + 3 principles** (OpenAI 5 + Anthropic 3 extra) → **8+ concrete tools / mechanisms**. + +### Quick-reference table for the 5 OpenAI principles + +The 5 sections below expand each principle (with original OpenAI quotes); here is the quick lookup first: + +| # | Principle | One-line | Crosses which work boundary | Matching tool | +|---|---|---|---|---| +| 1 | **Legibility** | Treat the agent as a new engineer; optimize navigability for it (not "make agent output readable to humans") | Repo + Types | Skill `references/` + AGENTS.md / CLAUDE.md pattern | +| 2 | **System of Record** | Knowledge lives in `docs/`, not in prompts; a 100-line entry map points deeper | Repo | `.coord/memory.yml` shared-memory + AGENTS.md / CLAUDE.md | +| 3 | **Progressive Disclosure** | Small entry point + teach the agent where to look next (pairs with SoR: SoR provides destination, PD is navigation) | Repo + Types | Skill `references/` mechanism + Codex `.ai/.md` brief | +| 4 | **Architecture & Taste Invariants** | Define boundaries; don't micromanage implementation. Lint enforces schema / naming / file size | Config + cross-cutting | `agent-acceptance-gate` preset YAML, custom linters | +| 5 | **Throughput Changes Merge Philosophy** | Agent PR speed > human QA speed → QA must be automated, not line-by-line review | Service (merge workflow) | Auto lint + test + acceptance gate, mandatory preset | + +→ The 5 sections below expand each principle individually; the final Anthropic ↔ OpenAI mapping lists cross-vendor equivalents + recommended reading. + +### 1. Legibility — make the codebase / docs readable to the *agent* + +> "Because the repository is entirely agent-generated, it's optimized first for **Codex's legibility**." — OpenAI + +When humans read code we get tons of visual aids: IDE highlighting, jump-to-definition, directory trees, hover tooltips, intuition. **The agent has none of these** — it only sees plain text + tool return values. If the codebase / docs aren't agent-friendly, the agent reads the wrong place, reasons in the wrong direction, and writes the wrong code. The optimization target is the **opposite** of "make agent output readable to humans": **treat the agent like a new engineering hire and optimize navigability for *it***. + +**(a) Codebase that's friendly to the agent** + +Write your code like onboarding docs for a new hire — anything that humans figure out by intuition must be explicit: + +- **Consistent schema naming**: `get_user_by_id` everywhere; don't mix `fetchUser` / `findUserById` / `userLookup`. The AI reads 1000 files and reasons by pattern matching — inconsistent patterns lead to wrong inferences. +- **File-size limits**: cap files at < 500 lines so the agent can read one fully in context. Past 500 lines the agent skims, then misses critical logic. +- **`docs/` hierarchical structure**: separate `docs/api/` / `docs/architecture/` / `docs/runbook/` clearly so the agent knows where to look. A flat dump means the agent can't find an entry point. + +**(b) Tools / APIs that are friendly to the agent (ACI)** + +The interface layer between the agent and the rest of the system is the **ACI (Agent-Computer Interface)**. Design goals: + +- **Crisp tool descriptions**: one line per tool stating "what it does" — not just the function signature. AI cannot guess the purpose from a variable name. +- **Poka-yoke tool design**: remove error-prone designs. E.g., require absolute paths only (no relative paths); require ISO date format (no free-form text). Make it impossible for the agent to misuse the tool. +- **Schema annotation**: every field has type + brief description + example value. AI can use it immediately, no guessing. + +→ **Core philosophy**: **optimize for the agent, not the human** — many optimization directions are opposite to "feels nice for a human reader", but the agent is now 80% of the readers. + +- **Work boundary spanned**: Repo + Types +- **Maps to our tool**: Claude Code Skill's `references/` mechanism + AGENTS.md / CLAUDE.md pattern + +### 2. System of Record — the single authoritative source of knowledge + +> "The repository's knowledge base lives in a structured `docs/` directory **treated as the system of record**. A short `AGENTS.md` (roughly 100 lines) is injected into context and serves primarily as a map." — OpenAI + +LLMs forget. LLMs hallucinate. If you stuff all business knowledge into the system prompt, two things happen: (1) context explodes (even 200k tokens isn't enough), and (2) different agents / sessions read different versions and contradict each other. **SoR (System of Record)** fixes this: **all real knowledge lives in external docs, not in the prompt, and the agent fetches it on demand**. + +**(a) Knowledge in docs, not in the prompt** + +Like a company having a single "employee handbook" as the authority — don't re-copy it into every onboarding: + +- **100-line entry map**: AGENTS.md / CLAUDE.md is just a "map" pointing at `docs/` regions, with no actual content. +- **Structured `docs/`**: the actual content lives in `docs/api/` / `docs/architecture/` / `docs/runbook/`, and the agent pulls on demand. +- **Prompt never duplicates docs**: avoid the "prompt says one thing, docs say another" version-mismatch trap. + +**(b) Persistence across sessions / across agents** + +Agents don't run as one-shot chats — they span multiple sessions, and subagents must share facts: + +- **`.coord/memory.yml` shared memory**: subagents and the supervisor read the same file, so they never disagree on basic facts. +- **Decisions log**: important decisions go into docs; every new session starts by reading the file rather than relying on "what we told the agent last time". +- **Versioned**: docs live in git, so any "when did this fact change?" question is answerable. + +→ **Core philosophy**: **one source of truth, one-way sync** — the agent pulls from SoR, never from the prompt; the moment SoR is edited, every agent's next run reads the new version. + +- **Work boundary spanned**: Repo +- **Maps to our tool**: `.coord/memory.yml` ([agent-shared-memory](https://github.com/WenyuChiou/agent-collab-skills) skill) + AGENTS.md / CLAUDE.md pattern + +### 3. Progressive Disclosure — start small, navigate deeper on demand + +> "Agents start with a small, stable entry point and **are taught where to look next**, rather than being overwhelmed up front." — OpenAI + +Dump too much context on an agent and it drowns — attention scatters, focus is lost, output quality drops, token cost explodes. **The fix is staged disclosure**: give a small + stable entry point first, then "teach the agent where to look next". Pairs with #2 SoR: **SoR provides the destination, PD (Progressive Disclosure) is the navigation mechanism**. + +**(a) Small entry point** + +The intro prompt should be a table of contents, not the entire book: + +- **AGENTS.md / CLAUDE.md ≤ 100 lines**: just the top-level "what does this project do + where is the main structure". Skip detail. +- **Brief instead of dump**: when assigning a task, use a 100-line brief — not dumping the whole codebase into context. +- **Stable entrance**: the 100 lines should change as little as possible so the agent can build a reliable mental model of them. + +**(b) Navigation mechanism — teach the agent where to dig** + +The agent fetches deep material itself when it needs to: + +- **Skill `references/` mechanism**: Claude Code's Skill puts detailed reference material in the `references/` subdirectory; the agent loads it only when needed. By default not in context. +- **`@-import` syntax**: CLAUDE.md can write `@docs/architecture.md` to point at deep material, pulling on demand rather than pre-loading. +- **Task-brief pointers**: a Codex `.ai/.md` brief can open with "first read `docs/X.md` §1-2; before executing, read `docs/Y.md` too". + +→ **Core philosophy**: **lazy load beats eager load** — every moment of context loading you can defer, defer. + +- **Work boundary spanned**: Repo + Types +- **Maps to our tool**: Claude Code Skill's `references/` mechanism (loaded only when the agent asks) + Codex `.ai/.md` brief pattern (read the brief first, then decide what to read deeper) + +### 4. Architecture & Taste Invariants — enforce invariants with linters + +> "We enforce these rules with custom linters and structural tests, plus a small set of **'taste invariants.'** ... **By enforcing invariants, not micromanaging implementations**, we let agents ship fast." — OpenAI + +When AI writes code it tends to take the fastest path, which often produces tangled modules, inconsistent names, and bloated files. OpenAI's team constrains the AI with **mandatory structural rules** — the agent can sprint inside the box you draw, instead of needing line-by-line supervision: + +**(a) Enforcing Architecture — physical boundaries that contain the AI** + +Like erecting steel scaffolding before construction: the AI can only fill in the cells you laid out: + +- **One-way dependency**: define strict layer hierarchy — the bottom Types layer can never import the top Service layer. AI attempts to smuggle imports are blocked. +- **Rigid directory structure**: certain code must live in certain directories (`models/`, `controllers/`, `schemas/`). The AI cannot invent new folders. +- **Automated linters**: if the AI writes code that breaks a rule (e.g. calling an API directly from the data layer), CI rejects the merge and forces the AI to rewrite. + +**(b) Enforcing Taste — turning "engineering aesthetics" into rules** + +"Taste" sounds subjective, but in engineering it means **maintainability, consistency, simplicity**. The AI has no aesthetics — it just produces statistically likely output — so aesthetics get encoded into lint rules: + +- **Golden-rule list**: write down principles like "**prefer composition over inheritance**", "**functions must stay short**", "**files < 500 lines**", and turn them into invariants. +- **Style uniformity**: the harness forces AI-generated naming and structure to read like "one senior engineer wrote everything", not a mash-up of inconsistent styles. +- **Reject AI slop**: the AI often generates redundant or useless code that "looks correct". Setting "taste benchmarks" forces the AI to keep refactoring and simplifying until the result reaches what a human expert would call elegant. + +→ **Core philosophy**: **define the boundaries, don't micromanage the implementation** — let agents sprint inside the cells you drew, instead of needing a human on every line. + +- **Work boundary spanned**: Config + cross-cutting (lint rules live in Config, enforcement applies across all layers) +- **Maps to our tool**: `agent-acceptance-gate` YAML presets (`multi-locale-mirror-sync.yml` / `catalog-entry-add.yml` / `fact-check-frontier-models.yml`) — codify "what the output should look like" up front + +### 5. Throughput Changes Merge Philosophy — agent throughput shifts the bottleneck to human QA + +> "...3.5 PRs per engineer per day... **the bottleneck became human QA capacity**." — OpenAI + +In the old world, an engineer shipped 1-2 PRs a day and humans could review every line. **Once agents ship 3.5 PRs/day per engineer**, plus self-correcting agents retry behind the scenes, real throughput is even higher. The bottleneck isn't agent speed — it's **humans can't keep up with review**. QA becomes the bottleneck. The merge logic must change; you can no longer rely on "a human read every line" as a quality gate. + +**(a) Pre-merge automation — automate the review** + +Humans are no longer line-by-line reviewers — they're spot-checkers: + +- **Automated lint**: CI runs the linter and enforces style / schema / naming. If the agent violates a rule, CI fails and merge is blocked. +- **Automated tests**: unit + integration tests run automatically; coverage below threshold blocks merge. +- **Automated acceptance gate**: before commit, run an acceptance-gate preset (e.g., `multi-locale-mirror-sync.yml`) that codifies "what the output should look like" up front; the PR fails if the agent doesn't match. + +**(b) Self-verification — the agent validates its own output first** + +Before opening a PR, the agent runs an evaluator-optimizer loop on itself: + +- **Built-in critique step**: after writing code, invoke a critique agent to self-review; if problems are found, rewrite. +- **LLM-as-judge scoring**: another LLM agent scores the PR; if it falls below threshold, it bounces back to the agent for revision. +- **Human spot-check only**: humans only look at the "final state" after the agent + LLM-judge both pass — no more reading the process line by line. + +→ **Core philosophy**: **the quality gate shifts from "a human read it" to "machines ran it + humans spot-check"** — the human role moves from "line-by-line gatekeeper" to "designer of how the gate is set". + +- **Work boundary spanned**: Service (merge workflow) +- **Maps to our tool**: the entire `agent-acceptance-gate` skill, especially the mandatory preset mechanism (trigger fires → preset must run) + +### Matrix: 5 principles × Stage 7 Harness 8 components + +Below shows how the 5 principles act on [Stage 7's 8 core Harness components](07-multi-agent-production.en.md#the-8-core-components-of-a-harness) (✓ = applies, ✓★ = primary lever): + +| Principle ＼ Harness component | 1. Agent Loop | 2. Tool Reg | 3. Ctx Mgr | 4. Retry | 5. Sandbox | 6. Obs | 7. Eval | 8. Cost / Lat | +|---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:| +| **1. Legibility** | | ✓ | ✓ | | | ✓ | | | +| **2. SoR** | | | ✓★ | | | ✓ | | | +| **3. Progr. Disc.** | ✓ | | ✓★ | | | | | ✓ | +| **4. Invariants** | | ✓ | | ✓ | ✓ | | ✓★ | | +| **5. Merge Phil.** | | | | | | | ✓★ | ✓ | + +→ **Context Manager (#3) + Eval (#7) are hot spots, hit by 4-5 principles each** — which is why v0.2.3 preset / `agent-acceptance-gate` / `agent-shared-memory` are all designed around these two components. + +→ **Tool Registry (#2) + Observability (#6) are secondary hot spots** — hit by 3 principles each. Legibility says "write the schemas right", Invariants says "write the lint right", SoR says "write the logs right". + +→ **Retry / Sandbox / Cost-Latency** are touched by only 1-2 principles each — these are relatively mechanical components, one main lever per component is enough. + +### 📚 Anthropic ↔ OpenAI cross-vendor mapping + recommended reading + +Most of OpenAI's 5 principles have a direct Anthropic counterpart, just under different names. The table below cross-references the two, with canonical URLs for each: + +| OpenAI principle | Anthropic equivalent / pattern | Canonical URL | +|---|---|---| +| **1. Legibility** | ACI (Agent-Computer Interface) + Tool Documentation | [Building Effective Agents Appendix](https://www.anthropic.com/engineering/building-effective-agents) | +| **2. System of Record** | CLAUDE.md hierarchy + Memory persistence | [Claude Code: How Claude remembers your project](https://code.claude.com/docs/en/memory) + [Multi-Agent Research System](https://www.anthropic.com/engineering/built-multi-agent-research-system) | +| **3. Progressive Disclosure** | **Same term** (Anthropic Skills calls it "the core design principle") | [Equipping Agents for the Real World with Agent Skills](https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills) ⭐⭐⭐ | +| **4. Taste Invariants** | Evaluator-optimizer loops + tool "poka-yoke" (e.g. forcing absolute filepaths) | [Building Effective Agents Evaluator-optimizer](https://www.anthropic.com/engineering/building-effective-agents) | +| **5. Throughput Changes Merge Philosophy** | "Human evaluation catches what automation misses" + LLM-as-judge in tandem | [Multi-Agent Research System Evaluation challenges](https://www.anthropic.com/engineering/built-multi-agent-research-system) | + +**Three principles Anthropic emphasizes that OpenAI does not feature heavily**: + +| Principle | Plain-language meaning | URL | +|---|---|---| +| **Simplicity** | Start with a basic LLM call; do not jump to multi-step agents | [Building Effective Agents Simplicity](https://www.anthropic.com/engineering/building-effective-agents) | +| **Transparency** | "Explicitly showing the agent's planning steps" — the agent reveals its plan | [Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents) | +| **Memory persistence** | Save context to external memory before it fills; spawn subagents with fresh contexts | [Multi-Agent Research System](https://www.anthropic.com/engineering/built-multi-agent-research-system) | + +#### Recommended reading order (45 + 20 min) + +**Read these 3 first (~45 min total)**: + +1. [Anthropic — Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents) ⭐⭐⭐ — covers principles #1 + #4 + Simplicity / Transparency. **Most foundational, read first.** +2. [Anthropic Engineering — Equipping Agents for the Real World with Agent Skills](https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills) ⭐⭐⭐ — covers principle #3; Anthropic literally uses "progressive disclosure" verbatim, with full 3-tier loading explanation. +3. [Claude Code — How Claude remembers your project](https://code.claude.com/docs/en/memory) ⭐⭐ — covers principle #2; CLAUDE.md 4-tier hierarchy + `@-import` + AGENTS.md interop. + +**Then read this one (~20 min)**: + +4. [Anthropic — How we built our multi-agent research system](https://www.anthropic.com/engineering/built-multi-agent-research-system) — supplements #2 + #5 + Memory persistence with a production case study. + +**OpenAI's original article**: + +5. [OpenAI — Harness Engineering](https://openai.com/index/harness-engineering/) — Codex's own case study; the source of the 5 principles. + +### 🛠 Why a coding-agent harness differs from a general tool-use agent + +The 5 principles above apply to all agents, but **coding agents** (Claude Code / Codex / Aider and the like) have extra heavyweight harness needs worth pulling out separately — because Stage 4's CodeAct, Stage 5's Claude Code ecosystem, and Stage 8's sandbox all grow out of this line. + +**Three coding-agent-specific harness components**: + +| Component | Why a coding agent specifically needs it | Maps to Stage | +|---|---|---| +| **File system + repo state snapshot** | agent edits code → must be able to diff / rollback / replay; unlike a chat agent that forgets after editing | Stage 5 CLAUDE.md hierarchy, Stage 7 Retry | +| **Isolated execution sandbox** | the code the agent writes must actually run to be verified (not just generated), but must not pollute the host | Stage 8 Code Sandbox (e2b / Daytona) | +| **Long-horizon task decomposition + parallel subagents** | large refactors / cross-file edits exceed a single context, so they must be split into subtasks running in parallel | Stage 7 multi-agent · **[Opus 4.8 Dynamic Workflows](#-dynamic-workflows-opus-48--when-the-agent-writes-its-own-workflow) (research preview) is exactly the productization of this direction — see the dedicated section below** | + +**Why this line is especially hot in 2026**: + +- The **OpenAI Agents SDK April-2026 update** built in a sandbox (7 providers) + a harness abstraction layer — [Stage 4 already flags](04-agent-frameworks.en.md) this as the first time a production coding agent is "architecturally sound." The harness is no longer everyone hand-rolling their own; shared abstractions are emerging. +- **Aider / Claude Code / Codex side by side**: all coding agents, but the harness trade-offs differ — Aider goes minimal git-commit-per-edit repo state, Claude Code goes CLAUDE.md + plan mode + subagents, Codex goes cloud sandbox + harness abstraction. **What readers should learn is "how harness trade-offs shape agent behavior," not which vendor's API to memorize**. + +> 📚 **Want to go deeper on coding-agent harness design**: start with [OpenAI — Harness Engineering](https://openai.com/index/harness-engineering/) (Codex case study) + the tool-design section of [Anthropic — Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents); for hands-on comparison, run [Stage 5 Claude Code ecosystem](05-claude-code-ecosystem.en.md) + [Stage 8 Code Sandbox](08-agent-interfaces.en.md). + +### ⚖️ Eval rigor — how harness design quietly biases your benchmark numbers + +Stage 7's Benchmark Landscape mentioned Berkeley's reward-hacking warning. Here's a more fundamental, more frequently overlooked problem: **a large fraction of an agent's score comes from the harness, not the model**. + +**Core facts (corroborated across sources)**: + +- **The same model, a different scaffold, and the score can halve** — research shows a model scoring 60% under a sophisticated agent scaffold may drop to 30% unassisted. **The scaffold (= harness) is as much a benchmark variable as the model**. ([SWE-bench benchmark-hygiene analysis](https://www.whocodesbest.com/news/2026/swe-bench-april-2026-benchmark-hygiene-matters)) +- **A single run is untrustworthy**: if scores vary by > 10% across runs, the signal-to-noise ratio is too low to conclude from one run. SWE-bench locks a per-issue Docker image (repo snapshot + pinned deps) for exact replay; τ-bench uses **pass^k** (only counts if all k trials pass) to measure reliability. +- **The benchmark itself often has reward-design bugs**: [Establishing Best Practices for Building Rigorous Agentic Benchmarks](https://arxiv.org/abs/2507.02825) catalogs holes in many agentic benchmarks' task setup / reward design — e.g. an early τ-bench version counted empty responses as correct. +- **Reward hacking is not an edge case**: on open-ended tasks, one study measured top models exploiting rubric holes in **75%** of agentic-code-generation tasks and 67% of creative tasks (figures vary by task and harness design — see the source below). + +**Takeaway for readers (harness-engineer lens)**: + +| What you're doing | How the harness should be designed | +|---|---| +| Comparing two models | **Fix the scaffold** — otherwise you're comparing scaffolds, not models | +| Reporting an agent score | Report pass^k (k≥3) or a multi-run average + variance; don't report a single best run | +| Writing your own eval | Assume up front that "the agent will reward hack" — held-out tests + LLM judge + file-edit detection, all three | +| Trusting a benchmark ranking | First check whether its reward design has been audited (empty-response / shortcut holes) | + +> 📚 **Want to go deeper on eval rigor**: [Establishing Best Practices for Building Rigorous Agentic Benchmarks](https://arxiv.org/abs/2507.02825) is a systematic catalog; for the production reward-hacking warning see the Berkeley section in [Stage 7 Benchmark Landscape](07-multi-agent-production.en.md) + the pass^k design of τ-bench ([sierra-research/tau2-bench](https://github.com/sierra-research/tau2-bench)). + +### 🔀 Dynamic Workflows (Opus 4.8) — when the agent writes its own workflow + +The coding-agent harness section above mentioned Opus 4.8's **Dynamic Workflows** (2026-05-28, Claude Code research preview). It deserves its own section — because it collapses the [workflow-vs-agent distinction taught in Stage 4](04-agent-frameworks.en.md#two-dimensions-to-clarify-first-workflow-vs-agent--single-vs-multi), making it the best live teaching material for "agent-authored orchestration." + +> 📌 **Claude Fable 5 (2026-06-09; suspended 2026-06-12, restored 2026-07-01)**: Anthropic's Mythos-class **Claude Fable 5** (`claude-fable-5`, positioned above the Opus class) shipped as the widely-available highest-capability member, alongside a limited-availability Claude Mythos 5 (`claude-mythos-5`). It was suspended 2026-06-12 by a US export-control directive, but the controls were lifted 2026-06-30 and [Fable 5 was redeployed globally 2026-07-01](https://www.anthropic.com/news/redeploying-fable-5) with a new safety classifier. Fable 5 is again the highest Claude tier; Opus 4.8 is the Opus-class flagship, and **Dynamic Workflows remains an Opus 4.8 feature**, not re-attributed to Fable 5. + +**The name is almost an oxymoron**: by Anthropic's own [Building Effective Agents](https://www.anthropic.com/research/building-effective-agents) definition, a **workflow = code paths a human predefines, an agent = an LLM directing its own process at runtime**. But a Dynamic Workflow is — **the agent (Claude) decides at runtime how to decompose the task, then emits a JavaScript orchestration script that a separate background runtime executes**. The human didn't write that workflow; the agent did. It is both an agent (who decides) and a workflow (how it runs). + +**The core mechanism — why it's not just "parallel subagents"**: + +The point isn't the parallelism, it's **context offloading**. Plain subagents / skills: every intermediate result returns to Claude's context window. Dynamic Workflows: the loop, branches, and intermediate results **all live in script variables** — only the final verified answer returns to context. That's why it can run "hundreds-of-thousands-of-line codebase migrations, using the existing test suite as the bar, from kickoff to merge" — because the noise of hundreds of intermediate agent calls never floods the context. + +| Aspect | Fact (per official Claude Code docs) | +|---|---| +| Which pattern it is | the scaled-up **orchestrator-workers** pattern — the one of [Anthropic's five workflow patterns](https://www.anthropic.com/research/building-effective-agents) where subtasks are *determined at runtime* — plus adversarial verification | +| Scale limits | **up to 16 agents concurrently**, with a hard cap of **1,000 total per run** (anti-runaway). NOT "hundreds running at once" | +| How it triggers | (1) the word `workflow` in a prompt (2) a saved command like `/deep-research` (3) `/effort ultracode` (xhigh reasoning + automatic orchestration) | +| Platform | a Claude Code feature (CLI / Desktop / IDE / headless / Agent SDK, v2.1.154+). **Not** a raw API (no `/v1/workflows`) | +| Quality mechanism | an adversarial propose / refute / converge loop — independent agents attack from different angles, others try to refute, it iterates to convergence, verifies before merging | + +**This very section was researched using this exact pattern** (live example): the facts here were gathered by running a dynamic-workflow-style orchestration — 4 parallel research agents each taking one angle (official / technical / positioning / skeptic) + a skeptic synthesizer that **dropped every claim it couldn't corroborate across sources**. What got dropped includes "hundreds of parallel subagents" (the real number is 16 concurrent / 1,000 total) and "a 750k-line Bun migration from Zig to Rust passing 99.8% of tests, merged in 11 days" (a single vendor case study, not independently audited — not citable as a verified fact). **That is orchestrator-workers + adversarial verify in action.** + +> ⚠️ **What it is NOT (the honest limitations matter as much as the feature)**: +> - **Not a generic workflow engine** (not Airflow / n8n / Temporal — a human doesn't draw a DAG; the agent writes code) +> - **Not unbounded parallelism** (16 concurrent / 1,000 total per run, not "hundreds at once") +> - **Not GA** (research preview, paid-plan-gated, pricing / availability may change; two official sources even disagree on whether Pro is included) +> - **Its reliability gain comes mainly from abstaining when uncertain** (a refusal trade-off — fewer attempts on uncertain questions, not more correct answers; from the system card). More subagents ≠ higher correct coverage +> - **Not free** (Anthropic itself warns token usage is "substantially more" and recommends a scoped task first to calibrate consumption) +> - **Doesn't solve the dispatch problem** ("when to fan out vs do one careful pass" still rests on Claude's runtime judgment; independent analysts note verifier discipline is still thin — fan-out without good verification can produce "fifty plausible bugs," worse than the single careful pass it replaced) + +> 📚 **Authoritative sources**: [Claude Code — Dynamic Workflows docs](https://code.claude.com/docs/en/workflows) (canonical for mechanism + 16/1000 limits + triggers) · [Anthropic — Introducing Dynamic Workflows](https://claude.com/blog/introducing-dynamic-workflows-in-claude-code) (positioning + token warning) · [Anthropic — Building Effective Agents](https://www.anthropic.com/research/building-effective-agents) (workflow-vs-agent + orchestrator-workers foundation) · [Opus 4.8 announcement](https://www.anthropic.com/news/claude-opus-4-8). + +### ⏳ Harnesses expire: Model-Harness-Fit and the Bitter Lesson + +We've spent a lot of space on how to design a harness. Here's an easily-missed but important caveat: **the harness you build today is tailored to *today's* model; as the model gets stronger, some of that harness expires.** + +- **Model-Harness-Fit**: a harness is "paired" with the current model's abilities. You build scaffolding to patch the model's weaknesses (can't plan, forgets, won't check its own work); once the next-generation model does those things on its own, that scaffolding becomes dead weight. +- **The Bitter Lesson**: AI history keeps showing that hand-crafted cleverness wins in the short run but almost always loses, long-term, to "let the model learn with more compute." Applied to harnesses: over-scaffolding is a bet against that trend. + +**The trade-off, plainly**: if a thin harness that's just enough to ship will do, don't build a castle. Every time you add a layer of scaffolding, ask: "Is this patching a real model weakness, or am I just nervous?" — when the next model lands, the former survives and the latter becomes debt you come back to tear out. + +> 📚 Origin of the idea: Rich Sutton, [The Bitter Lesson](http://www.incompleteideas.net/IncIdeas/BitterLesson.html) (2019); it echoes this stage's "same model, different scaffold, double the score" — since the scaffold matters that much, its shelf life is worth keeping in mind while you design. + +### 🧑‍🔧 Division of labor: you set the task and sign off, the agent does the work + +Anthropic's June 2026 report boiled real usage data down to one plain sentence: **you decide what to build, and the agent decides how to build it.** + +Think of **hiring someone to renovate your home**: you say "I want a Nordic-style study, here's the budget, here's the deadline," and the contractor picks the materials and does the actual work; when it's done, you either accept it or send it back. You don't nail every board yourself — but deciding *what you want* and *signing off* stays your job. + +The numbers line up with that: in a typical session, **you make about 70% of the "what to build" decisions, while about 80% of the "how to build it" decisions are left to the agent.** + +Three things to remember: +- **Draw the line clearly, or the agent can't help.** Spend your effort on stating what you want and checking the result; hand the doing to the agent. Vague requests are what send it off-track. This is exactly this stage's "work boundary." +- **The more expertise you have, the more the agent amplifies you** — because you're better at setting the task and reviewing the output. It *amplifies* experts; it doesn't replace them. +- **A team of agents needs division of labor too.** Once you run several agents at once (Stage 7), an "orchestrator" splits the work among sub-agents that each own one piece — so division of labor extends from "human vs agent" to "agent vs agent." + +> 📚 Sources: [Anthropic — Agentic coding and persistent returns to expertise](https://www.anthropic.com/research/claude-code-expertise) (2026-06-16; quantifies the human/agent division of labor from real Claude Code usage) · [2026 Agentic Coding Trends Report](https://resources.anthropic.com/2026-agentic-coding-trends-report) (the shift from writing code to orchestrating agent teams). + +### 📋 Concept-check prompt (self-quiz) + +> 🛠️ **Want to actually write SKILL.md / CLAUDE.md now?** The 4 implementation prompts (audit existing / generate new) have been **moved to [Stage 5](05-claude-code-ecosystem.en.md)**, which is where readers should do the real hands-on writing: +> - [Stage 5.1 CLAUDE.md design prompts](05-claude-code-ecosystem.en.md#-claudemd-design-prompts-using-the-5-principles) +> - [Stage 5.3 SKILL.md design prompts](05-claude-code-ecosystem.en.md#-skillmd-design-prompts-including-skill-creator-as-the-alternative) + +This section keeps only **one quiz prompt**, so you can verify that you actually understand the 5 principles before you start applying them. + +#### Prompt 1 — Self-quiz + +``` +I just learned the 5 OpenAI harness engineering principles: +1. Legibility +2. System of Record +3. Progressive Disclosure +4. Taste Invariants +5. Throughput Changes Merge Philosophy + +Generate 5 scenario questions. Each describes a realistic SKILL.md / CLAUDE.md design decision (e.g. "I put all examples directly into SKILL.md and it's under 1000 lines"), and asks **which principle is violated + how to fix it**. + +Ask one question at a time, wait for my answer, give feedback, then move on. Give a total score at the end. +``` + +→ **Suggested usage**: run this quiz after learning the 5 principles above to confirm that you actually absorbed the concepts. For the real write / audit prompts, go back to [Stage 5](05-claude-code-ecosystem.en.md). + +### 📐 Advanced agentic application flow (reader guide) + +Once you understand the 5 principles above plus the Anthropic cross-mapping, **how do you actually apply those ideas in agent design?** Starting from Stage 7 (you can already build production agents), 5 steps to production: + +1. **Establish the concept-map spine — four work-boundary layers**: Types → Config → Repo → Service. Decide which stack layer the agent may touch and when crossing a layer is a violation. + → This stage §Concept-map spine: four-layer work boundary + +2. **Pick 2-3 relevant advanced concepts**: from the 12 skeletons, choose the ones closest to your problem (Work Boundary / Contract / PAR / Autonomy ...). + → This stage §12 advanced concepts (pattern list) + +3. **Apply the 5 OpenAI principles (cross-cutting)**: Legibility / SoR / Progressive Disclosure / Taste Invariants / Throughput Merge Philosophy. These 5 cut across all 12 concepts and determine whether the design is "right". + → This stage §Cross-concept Harness Engineering principles + +4. **Encode into Skills + CLAUDE.md**: use the 4 prompts in Stage 5 — CLAUDE.md audit / generate ([Stage 5.1](05-claude-code-ecosystem.en.md#51--claude-code-basics)) + SKILL.md audit / generate ([Stage 5.3](05-claude-code-ecosystem.en.md#53--skills-claude-codes-behavior-layer--the-most-critical-layer-of-the-claude-code-ecosystem)). + +5. **Verify with an acceptance gate**: preset YAML catches drift / LLM-as-judge automates evaluation / human spot-checks cover edge cases. + → [agent-collab-skills](https://github.com/WenyuChiou/agent-collab-skills) + +→ **Production agent ready**: stable for real users, auto-verified, predictable failure modes. + +→ **How to use these 5 steps**: the first time you read this stage, follow 1 → 5 in order. Later, when an agent design gets stuck, come back and identify which step you are actually blocked on. + +→ **Difference from the earlier Why → What → How table**: that one is a horizontal reference for mapping **pain points ↔ principles ↔ tools**. These 5 steps are a vertical execution path for **what to do after you finish learning**. + +## 📖 Complete reading path (layered by depth) + +Ordered by depth. You do not need to read everything. The Foundation tier is required (~95 minutes total); everything else is for **deeper study when a real problem appears**. + +### 🌳 Reading decision tree (pick by the problem you're stuck on) + +![Agentic AI Advanced Reading Decision Tree](../resources/diagrams/reading-decision-tree.en.png) + +This is not just a reading list. It is a **decision tree**: identify the problem you have right now, then read the 1-2 papers or posts attached to that branch first. The diagram above shows 5 branches for 5 common stuck-states; below are branch-specific second readings (only after you finish the first one). + +**Branch-specific second readings**: + +- "I don't know how to start with agents" → ReAct paper + Lilian Weng "LLM Powered Autonomous Agents" +- "Should I use multi-agent at all?" → Anthropic Multi-Agent Research (case study section) +- "Context feels inefficient" → Anthropic Multi-Agent Research (memory section) +- "How do I write evals or automate verification?" → Anthropic Multi-Agent Research (eval section) +- "I want to keep up with frontier work" → AutoGen + ReAct paper + +→ **Rule**: pick **at most 2 deep reads per branch**. Finish those, then come back and decide the next branch. Do not broad-scan the whole list first. + +**Foundation tier** (read these 4 first, ~95 min total): +- [Anthropic — Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents) +- [Cognition — Don't Build Multi-Agents](https://cognition.ai/blog/dont-build-multi-agents) +- [Anthropic — How we built our multi-agent research system](https://www.anthropic.com/engineering/built-multi-agent-research-system) +- [Lilian Weng — LLM Powered Autonomous Agents](https://lilianweng.github.io/posts/2023-06-23-agent/) + +**Workflow patterns tier**: +- [LangGraph Planning Agents Tutorial](https://blog.langchain.com/planning-agents/) +- [Microsoft AutoGen docs](https://microsoft.github.io/autogen/) +- [DSPy](https://dspy.ai/learn/) + +**Production / Harness tier**: +- [OpenAI Harness Engineering (2026-02)](https://openai.com/index/harness-engineering) +- [Hamel Husain Evals blog](https://hamel.dev/blog/posts/evals/) +- [Simon Willison coding agents notes](https://simonwillison.net/tags/coding-agents/) + +**Frontier research papers** (choose 3-5 for deep reading): +- ReAct / Reflexion / CoALA / Self-Discover / Voyager / Constitutional AI / AutoGen + +**Chinese / hands-on**: +- [李宏毅 GenAI 2024 / 2025](https://speech.ee.ntu.edu.tw/~hylee/) +- [datawhalechina/hello-agents](https://github.com/datawhalechina/hello-agents) + +> 📋 **After advanced concepts, revisit this synthesis** → [Stage 5 §🗺️ 7-Layer Architecture Map](05-claude-code-ecosystem.en.md#-7-layer-architecture-map-read-this-first-then-51-57) (Claude Code's 7 primitives + 3 engineering disciplines in one map) + +## ✅ Self-check + +After this stage, you should be able to: + +- [ ] Use the **Types → Config → Repo → Service** four-layer model to explain why Cognition's Flappy Bird / Anthropic's speculative-leap cases count as work-boundary violations +- [ ] Name 5 of the 12 advanced concepts, including which layer they touch and a one-sentence definition +- [ ] Explain the 4 core principle categories (① Context management / ② Interface / ③ Quality verification / ④ Process discipline), what problem each category solves, and the enabling relationships between them +- [ ] Know which paper / blog to open next, without having to read everything first +- [ ] Distinguish a PAR loop (single-agent self-correction) from agent-debate (two agents in opposition) +- [ ] Write the task's work boundary explicitly into a brief (what is in-scope / out-of-scope) + +→ If you can do all of these, you are already beyond Stage 7 productionization and into frontier agentic workflow design. **What remains is to pick the paper that matches your current pain point and read that one deeply.** + +--- + +→ Next: [**Stage 8 — Agent Interfaces**](08-agent-interfaces.md) (**a shared hub for both tracks**) — learn how agents interact with the non-API world (Computer Use / Browser Use / Code Sandbox). Or pick a [specialized branch](../README.en.md#-learning-map-two-tracks), or come back and contribute to this repo. diff --git a/stages/07.5-advanced-agentic-concepts.md b/stages/07.5-advanced-agentic-concepts.md new file mode 100644 index 0000000..0783409 --- /dev/null +++ b/stages/07.5-advanced-agentic-concepts.md @@ -0,0 +1,630 @@ +# Stage 7.5 — 進階 Agentic 概念地圖（Advanced Agentic Concepts Map） + +> **繁體中文** | [简体中文](./07.5-advanced-agentic-concepts.zh-Hans.md) | [English](./07.5-advanced-agentic-concepts.en.md) + +⏱ **時間估算**：1 週（約 5 小時——不寫 code、只讀資源建立概念地圖） + +> 🚪 **進入條件**：完成 [Stage 7 — 多 Agent 系統與穩定運作](07-multi-agent-production.md)（或至少 Stage 4 + 6 + 7）。本章是 production 之後的 frontier 概念地圖、不是入門——沒做過 production agent 會讀不出這些概念在解什麼痛點。 + +> 💡 這是一份 **進階概念地圖 + reading path**，不是完整教學。Stage 4 / 6 / 7 學完已能做能上線給人用的 **agent**（AI 自主執行體、自己會規劃 + 執行任務的 LLM 系統、俗稱 production agent）；本 stage 幫你定位**業界還在討論哪些進階概念存在**、**每個概念解什麼問題**、**該先讀哪些 paper / blog**，避免你工作上踩到別人已踩過的坑。 + +> 📋 **本章內容**（8 個區塊、依序讀）： +> +> 1. 為什麼有這 stage（定位） +> 2. 概念地圖主軸：**Types → Config → Repo → Service** 四層工作邊界 +> 3. 12 個進階概念 skeleton +> 4. 為什麼選這 12 個 +> 5. 跨概念 Harness Engineering 原則（4 大類別 + 關係圖） +> 6. 進階 agentic 應用流程（5 step） +> 7. 完整 reading path +> 8. 自我檢查 + +> 🔤 **常用縮寫快速表**（本章會反覆出現）： +> +> | 縮寫 | 全稱 | 一句話解釋 | +> |---|---|---| +> | **agent** | AI 自主執行體 | 自己會規劃 + 執行任務的 LLM 系統 | +> | **PR** | Pull Request | 把改動送進主分支的請求（GitHub 術語）| +> | **SoR** | System of Record | 知識的權威來源、唯一 source of truth | +> | **ACI** | Agent-Computer Interface | agent 跟系統之間的對接層（工具 / API / docs）| +> | **MCP** | Model Context Protocol | 把 agent 工具標準化的 spec | +> | **PAR** | Plan-Act-Reflect | 單 agent 自我循環模式（規劃 → 執行 → 反思 → 修正 → 重試）| +> | **CI** | Continuous Integration | commit 後自動跑 test / lint 的系統 | +> | **QA** | Quality Assurance | 品質把關（人工或自動）| +> | **lint / linter** | — | 自動掃 code 找違規的工具 | +> | **`[OAI]` / `[Anth]`** | OpenAI / Anthropic | 後面的 source tag | + +## 🎯 為什麼有這 stage + +Stage 4 / 6 / 7 加起來足以做 **70% 能給真人用、不會三天兩頭出包的 agent**。每個 stage 教什麼、學完能做什麼： + +| Stage | 教什麼 | 學完能做什麼 | +|---|---|---| +| **4** | 挑 **framework**（agent 用哪個工具寫）| LangGraph / AutoGen / DSPy 選一個來寫 agent | +| **6** | **context engineering**（動態管理塞給 agent 的資料）| memory / retrieval / prompt 組裝 | +| **7** | **harness engineering**（agent 周圍的可靠運作環境）| observability / retry / cost gate / eval / sandbox 等 8 個元件 | +| **7.5（本章）**| **進階概念地圖** | 遇到問題知道翻哪份 paper / 看別人 agent 知道它在做什麼 / 知道每個概念對應 agent 系統哪一層 | + +但前線的 AI lab（Anthropic / OpenAI / Cognition / Microsoft）+ 學術界（Stanford / CMU / Princeton）在 2024-2026 持續推出 12+ 個進階設計概念。**有些現在你不用、但需要知道它存在**——這樣以後遇到問題、才知道有現成 pattern 可以套。本 stage 不是再教一套理論、而是給你一張**地圖**： + +- 不是要你全學會 +- 不是要你全用 +- 是讓你**遇到問題時、知道該翻哪份 paper / blog 找答案** +- 是讓你**看別人寫的 agent 時、看得出它在做什麼**——舉例：別人的 agent 一出錯就「重試 N 次直到放棄」（=只有 retry）、跟它「出錯後先反思一次、修正做法、再重試」（即所謂 PAR loop）、是完全不同等級的設計。能看出差別、你才知道該不該借用對方的做法 +- 是讓你**知道每個概念該套到 agent 系統哪個部分、解哪一類問題** + +## 🧭 概念地圖主軸：四層工作邊界（work boundary） + +本 stage 用**工作邊界**作為整理進階 agentic workflow 的主軸：先把 agent 系統拆成 4 層（**Types → Config → Repo → Service**、下面馬上展開）。接著問：agent 操作的對象屬於哪一層？跨層越界會出什麼問題？這不是要用單一模型解釋整章、而是先給讀者一個定位座標、後面 12 個概念才能放到同一張圖上比較。 + +> 💡 **「stack」是什麼意思**：軟體工程習慣把系統拆成上下層、每層各管一件事、上層蓋在下層之上、合稱 stack（堆疊）。例如 web 應用常見「frontend → backend → database」三層 stack。本 stage 把 agent 系統也拆成 4 層（Types / Config / Repo / Service）、看 agent 該動到哪一層。 + +> ⚠️ **這套 4 層跟 Stage 7 的 prompt → context → harness 三層不一樣、是兩種不同視角**： +> - **Prompt → Context → Harness**（Stage 7）：**stack 位置**——你正在設計或處理的是「prompt 字串 / context 資訊 / 外圍 runtime」哪一類對象？ +> - **Types → Config → Repo → Service**（本 stage）：**自主權範圍**——agent 能動到 stack 多深？跨層越界算違規嗎？ +> +> 兩者**正交**、解不同問題。讀完本節後可以同時用兩種視角看 agent 系統。 + +借用軟體架構的 **Types → Config → Repo → Service** 分層、套到 agent 系統： + +![Agentic Stack 四層工作邊界](../resources/diagrams/stack-4layer.png) + +→ **每一層上下都是一個「工作邊界」**。agent 操作的範圍 = 它的自主權範圍： + +- **Agent at Types layer** = 只能符合既有契約、不能改 schema（例：Codex 接到 brief 後、只能加 inline gloss） +- **Agent at Config layer** = 可以調 budget / policy 但不能改 memory（例：context-budget 改 max_cost_usd） +- **Agent at Repo layer** = 可以讀寫 memory / vector store 但不能 redesign workflow +- **Agent at Service layer** = 可以 recompose 整個 workflow、最高自主權 + +### 為什麼工作邊界適合當主軸 + +很多進階概念最後都會回到同一個問題：agent 的自主權到底到哪裡為止？把 agent 當成新進實習生來想：你交代一個明確的小任務、他卻自作主張把鄰近的東西也動了——這就叫「跨工作邊界」。產業界已經有 3 個公開記錄的真實案例可以對應： + +- **越界沒收手**（Cognition 的 Flappy Bird 案例）：用 **multi-agent**（多個 agent 並行協作）拆解任務、其中一個 **subagent**（主 agent 派出的子 agent、執行某個子任務）負責畫綠色管道、另一個負責畫雲朵背景，結果合起來雙方風格完全對不上——因為每個 subagent 只看到自己那塊、不知道對方在做什麼、也拿不到對方的 **context**（上下文、agent 拿到的全部資訊）。Cognition 寫得直白：「sub-agent 像一群過度自信的新人、根本不會在該問的時候問問題」。 + → 出處：[Cognition — Don't Build Multi-Agents (2025-06)](https://cognition.ai/blog/dont-build-multi-agents) + +- **加料**（Anthropic Multi-Agent Research 的 speculative-leap 現象）：subagent 被指派「研究某個主題」、它在報告裡擅自加上「我推測 X 也可能成立、雖然我沒驗證」這類沒人要的推論。Anthropic 在他們的 multi-agent 論文裡專門講為什麼這種「主動補完」要透過工程設計消除、不然 hallucination 會在 supervisor 沒注意時偷渡進結果。 + → 出處：[Anthropic — How we built our multi-agent research system (2025-06)](https://www.anthropic.com/engineering/built-multi-agent-research-system) + +- **operator 給太多權限**（Replit Agent 2024 prod database 事故）：根據社群討論、有使用者把 production database access 直接交給 agent、沒設「破壞性操作要先 confirm」的 gate、結果 agent 在「修 bug」過程跑了破壞性 SQL、清掉 production 資料。錯不在 agent 看起來合理地照指令做、錯在 operator 沒設邊界。 + → 出處：[Simon Willison 對此事故的分析（2024）](https://simonwillison.net/2024/Aug/26/replit/)（社群整理、非 Replit 官方 postmortem） + +**這 3 個案例告訴你的事**： + +- agent 不會「剛好停在你交代的那個點」——brief 要明確寫「**只能動 X、絕對不要動 Y**」、subagent 要明確收到 parent 的 full context +- agent 會主動「補完」沒被要求的東西——要靠 structured output schema + evaluator-optimizer loop 把這類 speculative 內容過濾掉 +- 規則「裝好了 ≠ 會被遵守」——operator 自律不夠、必須有 mechanical gate（permission check / cost cap / destructive op confirm）才擋得住 + +→ **實作對照**：work boundary 寫進 brief（Anthropic 的 brief template、LangGraph 的 state schema、`agent-collab-skills` 的 task-splitter 都是同一概念）、並在 acceptance gate / evaluator loop 中強制檢查；破壞性操作則加 explicit gate（見 [§📚 12 進階概念表 #7 Autonomy Gradients](#-12-個進階概念--skeleton)）。 + +### 🔁 Failure-mode lifecycle（產業級 agent 失敗模式怎麼演化成最佳實踐） + +![Failure-Mode Evolution Cycle](../resources/diagrams/failure-lifecycle.png) + +每個產業級 agent failure mode 都走過 **發現 incident → 公開文件化 → encode 成 framework pattern → 自動消除** 的循環。5 個有公開記錄的案例： + +| # | Incident（發現）| 文件化（命名）| Codify（變成什麼 pattern）| 公開出處 | +|---|---|---|---|---| +| 1 | Multi-agent subagent context drift（Flappy Bird 風格分裂）| "Sub-agents don't share principal-agent context" | **Single-thread principle**: 別堆 multi-agent、用 linear orchestration | Cognition 2025-06 | +| 2 | Subagent speculative leap（沒驗證的推論偷渡進結果）| "Speculative hallucination via filling-in" | **Evaluator-optimizer loop**: 加 critique step 強制 review | Anthropic Multi-Agent Research 2025-06 | +| 3 | Production permission drift（agent 砍 prod DB）| "Unbounded autonomy on destructive ops" | **Autonomy gradient**: suggest / propose / execute 三段授權 | Replit Agent 2024 incident | +| 4 | Agent looping without self-criticism（AutoGPT 卡 loop）| "Reflexion-less iteration" | **Plan-Act-Reflect loop**: 加 self-critique + revise | Reflexion paper (Shinn 2023) | +| 5 | Skill library corruption（broken skill 進 library）| "Untested skill commit" | **Pre-verify before commit**: skill 入 library 前必跑 test | Voyager paper (Wang 2024) | + +→ **這套「fail → publish → codify → fix」循環是整個 agentic 領域的進化機制**——不是「一開始就寫死所有規則」、而是「**每個 production incident 都被公開 + codify 成 pattern**」。Anthropic Skills 的 `references/` 機制、OpenAI 的 Taste Invariants、LangChain 的 evaluator pattern、Anthropic 的 evaluator-optimizer——都是同一邏輯的不同實作。 + +→ **怎麼用這張表學**：遇到自己 agent 出包時、查上表「最像哪一行」、然後讀對應 pattern 名（Single-thread / Evaluator-optimizer / Autonomy gradient / PAR / Pre-verify）的 deep dive。本 stage 後面 12 個 skeleton 涵蓋全部 5 個 pattern。 + +## 📚 12 個進階概念 — skeleton + +每個概念 ≤ 4 行：一句話定義 + 動到哪一層 + 最值得讀的 1 個資源。 + +### 🗺️ 12 概念 cluster map（動到哪一層 × 解什麼類型的問題） + +![12 Advanced Agentic AI Concepts — Cluster Map](../resources/diagrams/concept-cluster.png) + +上圖把 12 個概念依 **「動到哪一層」**（橫軸）+ **「解什麼類型問題」**（縱軸）分群、讓你看出哪些概念一起學、哪些可以跳過。其中 **Work Boundary（#1）跨所有層**（屬於通用紀律、不是某個特定位置）。 + +→ **怎麼用這張 map**： +- **第一次學**：先學「**編排類 + 反思類**」（共 6 個、是 multi-agent / production 的基礎） +- **要 deploy production**：補「**治理類 + 韌性類**」（共 6 個、防止上線出包） +- **跨類別主軸**：**Work Boundary（#1）是貫穿所有 12 概念的 root discipline** + +下面 12 個概念用表格列出（# / 概念 / 動到哪一層 / 一句話定義 / 最佳讀物）： + +| # | 概念 | 動到哪一層 | 一句話定義 | 最佳讀物 | +|---|---|---|---|---| +| 1 | **Work Boundary / Scope Discipline** | 跨所有層（discipline）| agent 只動 brief 指定的對象、不越界 | [Hamel — Evals + Skills](https://hamel.dev/blog/posts/evals-skills/) + [Cognition — Don't Build Multi-Agents](https://cognition.ai/blog/dont-build-multi-agents) | +| 2 | **Contract-driven Hand-offs** | Types + Service | 上游 agent 承諾的 artifacts、下游 agent 必須驗證自己真的收到了這些 artifacts | [Anthropic — Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents) Routing pattern | +| 3 | **Speculative / Parallel Exploration** | Service（編排）| 跑 N 條 alternative 路徑、取最佳那條（不只獨立 parallel）| [LangGraph Plan-Execute Tutorial](https://blog.langchain.com/planning-agents/) | +| 4 | **Agent-as-Judge / Constitutional AI** | Service（agent 評 agent）| 用一個 agent 評另一個 agent 的輸出、依 principles 反覆修正 | [Constitutional AI (Bai 2022)](https://arxiv.org/abs/2212.08073) | +| 5 | **Plan-Act-Reflect Loop** | Service（單 agent 自我循環）| write plan → execute → critique → revise → re-execute、直到 PASS 或 EXHAUSTED | [Reflexion (Shinn 2023)](https://arxiv.org/abs/2303.11366) + [Self-Discover (Zhou ICML 2024)](https://arxiv.org/abs/2402.03620) | +| 6 | **Hierarchical Task Decomposition** | Service（多層 supervisor）| supervisor → worker → sub-worker、≥ 2 層 recursion | [Microsoft AutoGen GroupChat docs](https://microsoft.github.io/autogen/) | +| 7 | **Autonomy Gradients / Trust Layers** | Config（autonomy policy）| agent 不同任務有不同自主權（suggest / propose / execute）| [Claude Code permission system](https://docs.claude.com/en/docs/agents-and-tools/claude-code/overview) | +| 8 | **Cost-aware Budget Gates** | Config（cost policy）| 超過美元預算就自動停止或升級審核（不只是 token 上限）| [OpenAI Harness Engineering (2026-02)](https://openai.com/index/harness-engineering) | +| 9 | **Failure Injection / Chaos Eval** | Service（測 agent 容錯）| 故意給 broken input / stale data / API timeout、看 agent 怎麼處理 | [Hamel Husain — Evals blog series](https://hamel.dev/blog/posts/evals/) | +| 10 | **Self-organizing Teams** | Service（agent 動態協商 role）| agents 不是預先指派 role、而是依任務動態分工 | [CAMEL (Li 2023)](https://arxiv.org/abs/2303.17760) + AutoGen | +| 11 | **Spec-driven Development** | Types（spec = code）| agent 任務由 formal spec（YAML / JSON Schema）定義、不是自由 prompt | [DSPy](https://github.com/stanfordnlp/dspy) signatures tutorial | +| 12 | **Graceful Degradation Paths** | Config（fallback policy）| frontier model 掛掉 → fallback 便宜 model + 降預期、不直接 crash | [OpenRouter routing docs](https://openrouter.ai/docs) + [Anthropic model fallback](https://docs.claude.com/en/docs/build-with-claude/models) | + +## 為什麼選這 12 個 + +- 都有可驗證的一手來源（primary source、Anthropic / OpenAI / Cognition / Microsoft / academic paper），不是空談 +- 都對應到至少一個公開實作（LangGraph / AutoGen / Anthropic Skills / DSPy 等）、可以直接拿來抄 +- 都在 Stage 4 / 6 / 7 已覆蓋的概念之外（沒重複） +- 避免「無限延伸」——其他進階概念（Voyager skill learning / MemoryLLM / world models）很重要但**先學這 12 個再說** + +## 🔬 跨概念 Harness Engineering 原則（多 source 整理） + +**這些原則不是任何單一廠商的**——Anthropic、OpenAI、Cognition、Hamel Husain 等都在各自的文章 / blog / docs 中表達過、用詞不同但指向同一組設計約束。下面先把**原則分成 4 大類**、列出主要來源、再展開細節。 + +> 📚 主要 source： +> - **Anthropic**（Building Effective Agents · Skills · Multi-Agent Research · CLAUDE.md memory docs） +> - **OpenAI**（[Harness Engineering 2026-02](https://openai.com/index/harness-engineering/)、清楚整理成 5 條 named principles） +> - **Cognition AI**（[Don't Build Multi-Agents](https://cognition.ai/blog/dont-build-multi-agents)） +> - **Hamel Husain**（[Evals are everything](https://hamel.dev/blog/posts/evals/)） +> - **Lilian Weng**（[LLM Powered Autonomous Agents](https://lilianweng.github.io/posts/2023-06-23-agent/)） + +> 🔤 **下面表格 source tag 縮寫對照**（之後 chapter 都會用這 4 個 tag）： +> - `[OAI]` = OpenAI +> - `[Anth]` = Anthropic +> - `[Cognition]` = Cognition AI +> - `[Hamel]` = Hamel Husain + +### 4 大類別 × 多 source + +| 類別 | 核心問題 | 該類別下的原則（含 source） | +|---|---|---| +| **① Context 管理** | 上下文不爆炸、agent 永遠拿到對的資訊 | **System of Record** [OAI] / **Memory Persistence** [Anth] / **Progressive Disclosure** [OAI + Anth] | +| **② Interface / 溝通** | agent 看得懂 codebase、也能說清楚自己在做什麼 | **Legibility** [OAI] / **ACI / Tool Documentation** [Anth] / **Transparency**（show planning）[Anth] | +| **③ Quality / 驗證** | 寫得對 / 不能 hallucinate | **Taste Invariants** [OAI] / **Evaluator-Optimizer loop** [Anth] / **Human + LLM-as-Judge** [Anth] / **"Evals are everything"** [Hamel] | +| **④ Process 紀律** | scale + iterate 不爆 | **Simplicity** [Anth] / **Throughput Changes Merge Philosophy** [OAI] / **Don't Build Multi-Agents (when unnecessary)** [Cognition] | + +→ **OpenAI 那 5 個原則是「有最清楚命名 + 完整 case study」的整理版本**——但類別 ① 的 SoR / Memory Persistence、類別 ② 的 ACI、類別 ③ 的 Eval-Optimizer、類別 ④ 的 Simplicity 都是 Anthropic 等其他來源先講的。下面先沿用 OpenAI 的命名版本展開、因為細節最完整；每節再標註對應的 Anthropic 等來源。 + +### 原則之間的主要關係（cross-category dependencies） + +不是 5 個 / 12 個獨立原則——它們之間有 **enabling 關係**： + +![5 個 Harness Engineering 原則之間的 cross-source dependency](../resources/diagrams/principle-dependency.png) + +→ **4 個關係 insight**： + +| 關係 | 描述 | 為什麼重要 | +|---|---|---| +| **SoR + Memory + PD 三者配對** | SoR 提供目的地、Memory 跨 session 保存資訊、Progressive Disclosure 是導航機制 | 三者單獨用不夠、必須一起設計 | +| **Legibility ↔ Transparency 雙向** | agent 能讀 codebase 才能 self-report；agent 會 self-report 你才能驗證 legibility 有效 | 互為前提、缺一不可 | +| **Quality 是 Process 自動化前置** | 沒寫死 invariants + eval loop、人類就無法把 review 交給 automation | 是 ④ Process 的必要條件 | +| **Simplicity 是隱性 root** | 一開始堆 multi-agent、所有其他原則複雜度爆增 | Cognition「Don't Build Multi-Agents」= Anthropic「Simplicity」、同一回事 | + +→ 下面 5 個小節用 OpenAI 命名版本展開（最完整）、每節都會回頭標 cross-source mapping。 + +### 為什麼要在意這些原則 — Why → What → How + +下表以「**痛點（Why）→ 原則（What）→ 實作（How）**」三層解釋這些原則在解什麼問題、用什麼工具落地： + +| 痛點（Why）| 原則（What）| 實作（How）| +|---|---|---| +| Context 200k 滿 / Multi-agent context overflow | Progressive Disclosure + Memory Persistence | Skills `references/` / `CLAUDE.md` `@-import` / `.ai/` brief | +| Agent 看不懂自家 codebase | Legibility + Tool Doc / ACI | `AGENTS.md` 100 行 / poka-yoke 工具設計 / 一致 schema 命名 | +| 多 agent desync、不同事實版本 | System of Record | `docs/` + `.coord/` shared-memory skill | +| 隨機 drift / Review 漏抓 | Taste Invariants + Transparency（planning 顯示）| `agent-acceptance-gate` preset YAMLs / evaluator-optimizer loop | +| Agent 寫 PR 快、Human QA 跟不上 | Throughput Changes Merge Philosophy | mandatory preset / LLM-as-judge / Human spot-check | +| 一上來就堆 multi-agent overkill | Simplicity（Anthropic）| 先 basic LLM call、確認需要才加 agent | + +→ **6 個痛點 → 5 + 3 個原則**（OpenAI 5 + Anthropic 3 extra）→ **8+ 個具體工具 / 機制**。 + +### 5 個 OpenAI 原則速查表 + +下面 5 節各自展開原則細節（含 OpenAI 原文 quote）；先給速查表： + +| # | 原則 | 一句話 | 跨 work boundary | 對應 tool | +|---|---|---|---|---| +| 1 | **Legibility** | 把 agent 當新進工程師、為它優化 navigability（不是讓人讀懂 agent） | Repo + Types | Skill `references/` + AGENTS.md / CLAUDE.md pattern | +| 2 | **System of Record** | 知識住 `docs/`、不住 prompt；100 行 entry map 指向深處 | Repo | `.coord/memory.yml` shared-memory + AGENTS.md / CLAUDE.md | +| 3 | **Progressive Disclosure** | 小 entry point + 教 agent 之後去哪查（跟 SoR 配對：SoR 提供 destination、PD 是 navigation）| Repo + Types | Skill `references/` 機制 + Codex `.ai/.md` brief | +| 4 | **Architecture & Taste Invariants** | 定義邊界、不細管實作；lint 強制 schema / 命名 / 檔案大小 | Config + cross-cutting | `agent-acceptance-gate` preset YAML、custom linter | +| 5 | **Throughput Changes Merge Philosophy** | agent PR 速度 > 人類 QA 速度 → QA 必須自動化、不依賴逐行讀 | Service（merge workflow）| 自動 lint + test + acceptance gate、mandatory preset | + +→ 下面 5 節展開每個原則本身、最後 Anthropic ↔ OpenAI 對照 列出兩家對照詞 + 推薦讀本。 + +### 1. Legibility — 讓 agent 能讀懂 codebase / docs + +> "Because the repository is entirely agent-generated, it's optimized first for **Codex's legibility**." — OpenAI + +人類讀 code 有很多視覺輔助：IDE 高亮、跳轉、目錄樹、滑鼠 hover、直覺。**agent 全部沒有**——它只看純文字 + 工具回傳值。如果 codebase / docs 對 agent 不友善、agent 讀錯地方、推論錯方向、寫錯 code。優化方向跟「讓人讀懂 agent 輸出」相反：**把 agent 當新進工程師、為它優化 navigability**。 + +**(a) Codebase 對 agent 友善** + +像給新人寫的 onboarding 文件、所有「靠經驗推測」的東西都得明寫： + +- **一致 schema 命名**：`get_user_by_id` 永遠用這個格式、不要混 `fetchUser` / `findUserById` / `userLookup` 三種寫法。AI 看 1000 個檔案、靠 pattern matching 推論、規律不一致就會推錯。 +- **檔案大小限制**：規定檔案 < 500 行、agent 一次能完整讀進 context。超過 500 行 agent 一次讀不完、就會跳讀、漏看關鍵邏輯。 +- **`docs/` 階層結構**：`docs/api/` / `docs/architecture/` / `docs/runbook/` 分得清楚、agent 才知道 routing。亂塞一個目錄裡、agent 找不到入口。 + +**(b) Tool / API 對 agent 友善（ACI）** + +agent 跟工具的對接層、就是 ACI（Agent-Computer Interface）。設計目標： + +- **清楚的 tool description**：每個 tool 一行講「幹什麼」、不是只寫 function signature。AI 沒法靠變數名猜功能。 +- **Poka-yoke 工具設計**：把容易出錯的設計拿掉。例：強制吃 absolute path、不允許 relative；強制吃 ISO 日期格式、不允許自由文字。讓 agent 想犯錯都犯不了。 +- **Schema 標註**：每個欄位都有 type + 簡述 + 範例值。AI 看了就能直接用、不用猜。 + +→ **核心精神**：**為 agent 優化、不是為人**——很多優化方向跟「讓人讀爽」相反、但 agent 才是現在 80% 的 reader。 + +- **跨 work boundary**：Repo + Types +- **對應 tool**：Claude Code Skill 的 `references/` 機制 + AGENTS.md / CLAUDE.md pattern + +### 2. System of Record — 知識的唯一權威來源 + +> "The repository's knowledge base lives in a structured `docs/` directory **treated as the system of record**. A short `AGENTS.md` (roughly 100 lines) is injected into context and serves primarily as a map." — OpenAI + +LLM 容易忘事、容易腦補。如果你把所有業務知識都塞進 system prompt、會發生兩件事：(1) context 立刻爆（200k token 也不夠塞）、(2) 不同 agent / 不同 session 讀到的版本對不上、產生「自相矛盾的事實」。SoR（System of Record、知識的權威來源）解這個問題：**所有真實知識住外部 docs、不住 prompt、agent 從 docs 動態拉**。 + +**(a) 知識住 docs、不住 prompt** + +像公司只有一份「員工手冊」當權威、不要每個 onboarding 都重抄一份： + +- **100 行 entry map**：AGENTS.md / CLAUDE.md 只放「地圖」（指向 `docs/` 各區的索引）、不放實際內容。 +- **`docs/` 結構化**：實際內容住 `docs/api/`、`docs/architecture/`、`docs/runbook/` 等子目錄、agent 按需拉。 +- **prompt 絕不重複 docs 內容**：避免「prompt 講一套、docs 講另一套」的版本對不上。 + +**(b) 跨 session / 跨 agent 持久化** + +agent 不是一次性對話、會跨好幾個 session、subagent 之間也要共享事實： + +- **`.coord/memory.yml` 共享記憶**：subagent 跟 supervisor 讀同一份、不會各自記不同版本。 +- **decisions log**：重要決策寫進 docs、新 session 從讀檔開始、不依賴「上次跟 agent 講過的話」。 +- **versioned**：用 git 管 docs、要追溯「這條知識什麼時候改的」隨時可查。 + +→ **核心精神**：**唯一權威、單向同步**——agent 從 SoR 拉、不從 prompt 拉；SoR 改了所有 agent 下次跑都讀新版本。 + +- **跨 work boundary**：Repo +- **對應 tool**：`.coord/memory.yml`（[agent-shared-memory](https://github.com/WenyuChiou/agent-collab-skills) skill）+ AGENTS.md / CLAUDE.md pattern + +### 3. Progressive Disclosure — 從 small entry point 漸進深挖 + +> "Agents start with a small, stable entry point and **are taught where to look next**, rather than being overwhelmed up front." — OpenAI + +一次塞太多 context 給 agent、它會被淹沒——注意力分散、抓不到重點、品質下降、token cost 暴增。**正確做法是分批揭露**：先給 small + stable 的入口、再「教 agent 之後去哪查」。跟 #2 SoR 配對運作：**SoR 提供目的地**、**PD（Progressive Disclosure）是導航機制**。 + +**(a) Small entry point** + +入門 prompt 像書的目錄、不是整本書： + +- **AGENTS.md / CLAUDE.md 100 行**：只放最高層的「這個專案幹什麼 + 主要結構在哪」、不放細節。 +- **brief 而非 dump**：給 agent 任務時、用 100 行 brief、不是把整個 codebase 一次傾倒給它。 +- **入口要 stable**：100 行的內容變動越少越好、agent 才能對它建立可靠的 mental model。 + +**(b) Navigation 機制——教 agent 之後去哪挖** + +agent 需要的時候、自己去拉深處資料： + +- **Skill `references/` 機制**：Claude Code 的 Skill 把詳細 reference 放在 `references/` 子目錄、agent 需要才 load。預設不進 prompt、需要才進。 +- **`@-import` 語法**：CLAUDE.md 可以寫 `@docs/architecture.md` 指向深處、按需拉而不是預先塞。 +- **task brief 指向法**：Codex `.ai/.md` brief 一開頭給 agent 「先讀 `docs/X.md` 第 1-2 節、執行前再讀 `docs/Y.md`」。 + +→ **核心精神**：**懶載入（lazy load）勝過熱載入（eager load）**——能晚一刻塞 context、就晚一刻。 + +- **跨 work boundary**：Repo + Types +- **對應 tool**：Claude Code Skill 的 `references/` 機制（只在 agent 需要才 load）+ Codex `.ai/.md` brief pattern（先讀 brief 再決定深挖） + +### 4. Architecture & Taste Invariants — 用 lint 強制不變量 + +> "We enforce these rules with custom linters and structural tests, plus a small set of **'taste invariants.'** ... **By enforcing invariants, not micromanaging implementations**, we let agents ship fast." — OpenAI + +AI 寫 code 時傾向「怎麼快怎麼來」、常導致模組過度耦合、命名混亂、檔案爆炸。OpenAI 團隊用**強制性結構規則**約束 AI、讓 agent 在你劃好的邊界內快跑、而不是每行都要人盯： + +**(a) Enforcing Architecture — 用「物理邊界」框住 AI** + +像在蓋房子前先搭好鋼筋支架、AI 只能在格子裡填肉： + +- **單向依賴（One-way Dependency）**：定義嚴格的層級、底層的 Types 絕對不能引用高層的 Service。AI 想偷渡 import 會被擋。 +- **剛性目錄結構**：規定特定的 code 必須待在特定目錄（如 `models/` / `controllers/` / `schemas/`）、AI 不能自己亂建 folder。 +- **自動化 Linter**：如果 AI 寫出違規 code（例如在數據層直接呼叫 API）、CI 自動拒絕 merge、逼 AI 重寫。 + +**(b) Enforcing Taste — 把「工程美學」變成規則** + +「品味」聽起來主觀、但工程上指的是**可維護性、一致性、簡潔度**。AI 沒有美感、它只會根據機率產出結果——所以要把美感寫成 lint 規則： + +- **黃金準則 list**：寫下「**偏好 composition 而非 inheritance**」、「**函數必須短小**」、「**檔案 < 500 行**」這類原則、變成 invariant。 +- **程式碼風格統一**：harness 強制 AI 產出的命名、邏輯組織看起來像「同一個高階工程師寫的」、而不是混雜風格的大雜燴。 +- **拒絕「AI slop」**：AI 常生成冗餘、無用但「看起來正確」的 code。設定「品味基準」要求 AI 不斷重構、簡化、直到達到人類專家認可的優雅程度。 + +→ **核心精神**：**定義邊界、不細管實作**——讓 agent 在你劃好的格子裡自由衝、而不是每行都要人盯。 + +- **跨 work boundary**：Config + cross-cutting（lint 規則寫在 Config、強制檢查跨所有層） +- **對應 tool**：`agent-acceptance-gate` YAML preset（`multi-locale-mirror-sync.yml` / `catalog-entry-add.yml` / `fact-check-frontier-models.yml`）——預先 codify「跑出來該長什麼樣」 + +### 5. Throughput Changes Merge Philosophy — agent 高吞吐 → 人類 QA 變瓶頸 + +> "...3.5 PRs per engineer per day... **the bottleneck became human QA capacity**." — OpenAI + +過去 1 個工程師 1 天寫 1-2 個 PR、人類逐行 review 跟得上。**agent 上線後變成 1 天 3.5 個 PR**、再加上 self-correcting agent 自己會 retry、實質吞吐還更高。問題不是 agent 不夠快、是**人類 review 來不及**——QA 變瓶頸。Merge 邏輯必須換、不能再依賴「人逐行讀過」當 quality gate。 + +**(a) Pre-merge automation——把 review 自動化** + +人不再是逐行 reviewer、是 spot-checker： + +- **自動 lint**：CI 跑 linter 強制 style / schema / 命名。agent 違規 → CI 紅燈、merge 被擋。 +- **自動 test**：unit test + integration test 全自動跑、coverage 不達標不能 merge。 +- **自動 acceptance gate**：commit 前跑 acceptance gate preset（如 `multi-locale-mirror-sync.yml`）、預先 codify「跑出來該長什麼樣」、agent 不符就 fail。 + +**(b) Self-verification——agent 自己先驗一輪** + +agent 提 PR 前、先自己跑一遍 evaluator-optimizer loop： + +- **agent 內建 critique step**：寫完 code 再呼叫一次 critique agent 自審、發現問題就重寫。 +- **LLM-as-judge 自動評分**：用另一個 LLM agent 給 PR 打分、低於門檻就退回 agent 改。 +- **Human spot-check only**：人類只看 agent + LLM-judge 都過關後的「最終樣子」、不再逐行讀過程。 + +→ **核心精神**：**Quality gate 從「人讀過」變成「機器跑過 + 人 spot-check」**——人類角色從「逐行守門」升級為「決定 gate 怎麼設」。 + +- **跨 work boundary**：Service（merge workflow） +- **對應 tool**：`agent-acceptance-gate` 整套、特別是 mandatory preset 機制（trigger fire → preset 必跑） + +### 5 原則 × Stage 7 Harness 8 元件對照表 + +下表展示 5 個原則怎麼作用到 [Stage 7 Harness Engineering 的 8 個核心元件](07-multi-agent-production.md#harness-的-8-個核心元件)上（✓ = applies、✓★ = primary lever）： + +| 原則 ＼ Harness 元件 | 1. Agent Loop | 2. Tool Reg | 3. Ctx Mgr | 4. Retry | 5. Sandbox | 6. Obs | 7. Eval | 8. Cost / Lat | +|---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:| +| **1. Legibility** | | ✓ | ✓ | | | ✓ | | | +| **2. SoR** | | | ✓★ | | | ✓ | | | +| **3. Progr. Disc.** | ✓ | | ✓★ | | | | | ✓ | +| **4. Invariants** | | ✓ | | ✓ | ✓ | | ✓★ | | +| **5. Merge Phil.** | | | | | | | ✓★ | ✓ | + +→ **Context Manager（#3）+ Eval（#7）是被 4-5 個原則同時作用的熱點**——這也是 v0.2.3 preset / `agent-acceptance-gate` / `agent-shared-memory` 都圍著這兩個元件設計的原因。 + +→ **Tool Registry（#2）+ Observability（#6）次熱**——被 3 個原則影響、Legibility 把 schema 寫對 + Invariants 把 lint 寫對 + SoR 把紀錄寫對。 + +→ **Retry / Sandbox / Cost-Latency** 只被 1-2 個原則作用——這幾個元件相對「機械」、原則上對應 1 個 lever 就夠。 + +### 📚 Anthropic ↔ OpenAI cross-vendor 對照 + 推薦讀本 + +OpenAI 那 5 原則 Anthropic 大部分也都討論過、用的詞不同。下表是 cross-vendor 對照、每個都附 canonical URL： + +| OpenAI 原則 | Anthropic 對應詞 / pattern | 最權威 URL | +|---|---|---| +| **1. Legibility** | ACI（Agent-Computer Interface）+ Tool Documentation | [Building Effective Agents Appendix](https://www.anthropic.com/engineering/building-effective-agents) | +| **2. System of Record** | CLAUDE.md hierarchy + Memory persistence | [Claude Code: How Claude remembers your project](https://code.claude.com/docs/en/memory) + [Multi-Agent Research System](https://www.anthropic.com/engineering/built-multi-agent-research-system) | +| **3. Progressive Disclosure** | **同詞**（Anthropic Skills 自己也用「core design principle」描述）| [Equipping Agents for the Real World with Agent Skills](https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills) ⭐⭐⭐ | +| **4. Taste Invariants** | Evaluator-optimizer loops + Tool "poka-yoke"（如強制 absolute filepath）| [Building Effective Agents Evaluator-optimizer](https://www.anthropic.com/engineering/building-effective-agents) | +| **5. Throughput Changes Merge Philosophy** | "Human evaluation catches what automation misses" + LLM-as-judge 並用 | [Multi-Agent Research System Evaluation challenges](https://www.anthropic.com/engineering/built-multi-agent-research-system) | + +**Anthropic 額外強調的 3 個 OpenAI 沒重點講的原則**： + +| 原則 | 白話 | URL | +|---|---|---| +| **Simplicity** | 先用 basic LLM call、不要跳 multi-step agent | [Building Effective Agents Simplicity](https://www.anthropic.com/engineering/building-effective-agents) | +| **Transparency** | "explicitly showing the agent's planning steps"——agent 自己秀 plan | [Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents) | +| **Memory persistence** | context 滿前先存外部、subagent 用 fresh context 接力 | [Multi-Agent Research System](https://www.anthropic.com/engineering/built-multi-agent-research-system) | + +#### 推薦閱讀順序（45 + 20 min） + +**先讀這 3 篇（總計 ~45 min）**： + +1. [Anthropic — Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents) ⭐⭐⭐ — 涵蓋原則 #1 + #4 + Simplicity / Transparency、**最基礎、先讀** +2. [Anthropic Engineering — Equipping Agents for the Real World with Agent Skills](https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills) ⭐⭐⭐ — 涵蓋原則 #3、Anthropic 直接用 "progressive disclosure" 詞、3-tier loading 完整解說 +3. [Claude Code — How Claude remembers your project](https://code.claude.com/docs/en/memory) ⭐⭐ — 涵蓋原則 #2、CLAUDE.md 4-tier hierarchy + `@-import` + AGENTS.md 互通 + +**再讀這 1 篇（~20 min）**： + +4. [Anthropic — How we built our multi-agent research system](https://www.anthropic.com/engineering/built-multi-agent-research-system) — 補 #2 + #5 + Memory persistence production 案例 + +**OpenAI 原始文章**： + +5. [OpenAI — Harness Engineering](https://openai.com/index/harness-engineering/) — Codex 自身 case study、5 原則的源頭 + +### 🛠 Coding-agent harness 為什麼跟一般 tool-use agent 不同 + +上面 5 原則對所有 agent 通用，但 **coding agent**（Claude Code / Codex / Aider 這類）的 harness 有額外重量級需求，值得單獨拆開看——因為 Stage 4 的 CodeAct、Stage 5 的 Claude Code 生態、Stage 8 的 sandbox 全都從這條線長出來。 + +**三個 coding-agent 專屬的 harness 元件**： + +| 元件 | 為什麼 coding agent 特別需要 | 對照 Stage | +|---|---|---| +| **檔案系統 + repo 狀態快照** | agent 改 code → 要能 diff / rollback / replay；不像 chat agent 改完就忘 | Stage 5 CLAUDE.md hierarchy、Stage 7 Retry | +| **可隔離的執行 sandbox** | agent 寫的 code 要真的跑起來驗證（不是只生成）、但不能污染 host | Stage 8 Code Sandbox（e2b / Daytona）| +| **長 horizon 任務分解 + parallel subagent** | 大型重構 / 跨檔修改超過單一 context、要拆成 subtask 平行跑 | Stage 7 multi-agent · **[Opus 4.8 Dynamic Workflows](#-dynamic-workflowsopus-48-當-agent-自己寫出-workflow)（research preview）就是這個方向的 product 化——見下方專節** | + +**為什麼這條線 2026 特別熱**： + +- **OpenAI Agents SDK 2026-04 更新**把 sandbox（7 個 provider）+ harness 抽象層內建——[Stage 4 已標註](04-agent-frameworks.md)這是「production coding agent 首次 architecturally sound」。harness 不再是各家自己拼、開始有共用抽象。 +- **Aider / Claude Code / Codex 三家對照**：同樣是 coding agent，但 harness 取捨不同——Aider 走 git-commit-per-edit 的極簡 repo 狀態、Claude Code 走 CLAUDE.md + plan mode + subagent、Codex 走 cloud sandbox + harness 抽象。**讀者要學的是「harness 取捨怎麼影響 agent 行為」、不是記哪家 API**。 + +> 📚 **想深入 coding-agent harness 設計**：先讀 [OpenAI — Harness Engineering](https://openai.com/index/harness-engineering/)（Codex case study）+ [Anthropic — Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents) 的 tool design 段；想看實作對照就跑 [Stage 5 Claude Code 生態](05-claude-code-ecosystem.md) + [Stage 8 Code Sandbox](08-agent-interfaces.md)。 + +### ⚖️ Eval rigor — harness 設計怎麼悄悄 bias 你的 benchmark 數字 + +Stage 7 的 Benchmark Landscape 提過 Berkeley 的 reward-hacking 警告。這裡補一個更基礎、更常被忽略的問題：**agent 的分數有一大半是 harness 給的、不是 model 給的**。 + +**核心事實（多 source 證實）**： + +- **同一個 model、不同 scaffold，分數可以差一倍**——研究顯示一個 model 在「精緻 agent scaffold」下拿 60%、換成 unassisted 可能掉到 30%。**scaffold（= harness）跟 model 一樣是 benchmark 的變因**。([SWE-bench benchmark hygiene 分析](https://www.whocodesbest.com/news/2026/swe-bench-april-2026-benchmark-hygiene-matters)) +- **單次 run 不可信**：分數在多次 run 之間若差 > 10%、signal-to-noise 太低、不能從單次結論。SWE-bench 用 per-issue Docker image 鎖 repo 快照 + pinned deps 來 exact replay；τ-bench 用 **pass^k**（k 次都過才算）量 reliability。 +- **benchmark 本身常有 reward-design bug**：[Establishing Best Practices for Building Rigorous Agentic Benchmarks](https://arxiv.org/abs/2507.02825) 盤點出很多 agentic benchmark 的 task setup / reward 有洞——例如 τ-bench 早期版本把「空回應」算成正確。 +- **reward hacking 不是邊緣案例**：在開放任務上，某研究測得 top model 利用 rubric 漏洞的比率：agentic code generation 達 **75%**、creative task 達 67%（數字依任務與評測設計而異、見下方 source）。 + +**對讀者的 takeaway（harness engineer 視角）**： + +| 你在做什麼 | harness 該怎麼設計 | +|---|---| +| 比較兩個 model | **固定 scaffold**——否則你比的是 scaffold 不是 model | +| 報告 agent 分數 | 報 pass^k（k≥3）或多 run 平均 + 變異、不要報單次 best | +| 自己寫 eval | 先假設「agent 會 reward hack」——held-out test + LLM judge + 改檔偵測三管齊下 | +| 信一個 benchmark 排名 | 先查它的 reward design 有沒有被審計過（empty-response / shortcut 漏洞）| + +> 📚 **想深入 eval rigor**：[Establishing Best Practices for Building Rigorous Agentic Benchmarks](https://arxiv.org/abs/2507.02825) 是系統性的盤點；reward hacking 的 production 警告見 [Stage 7 Benchmark Landscape](07-multi-agent-production.md) 的 Berkeley 段 + τ-bench（[sierra-research/tau2-bench](https://github.com/sierra-research/tau2-bench)）的 pass^k 設計。 + +### 🔀 Dynamic Workflows（Opus 4.8）— 當 agent 自己寫出 workflow + +上面 coding-agent harness 提到 Opus 4.8 的 **Dynamic Workflows**（2026-05-28、Claude Code research preview）。它值得單獨拆開——因為它把 [Stage 4 教的 workflow vs agent 區分](04-agent-frameworks.md#兩個維度先分清楚workflow-vs-agent--single-vs-multi)整個塌縮掉，是理解「agent-authored orchestration」最好的活教材。 + +> 📌 **Claude Fable 5（2026-06-09 發布；2026-06-12 暫停、2026-07-01 恢復）**：Anthropic 的 Mythos-class **Claude Fable 5**（`claude-fable-5`、定位在 Opus 之上）是廣泛可用的最高能力成員，與 limited-availability 的 Claude Mythos 5（`claude-mythos-5`）同日發布。2026-06-12 曾被美國出口管制暫停，但出口管制 2026-06-30 解除、[Fable 5 於 2026-07-01 全球恢復](https://www.anthropic.com/news/redeploying-fable-5)（加了新安全 classifier）。Fable 5 又是最高階的 Claude 層級；Opus 4.8 為 Opus-class 旗艦，**Dynamic Workflows 仍是 Opus 4.8 的功能**，不歸屬於 Fable 5。 + +**名字其實是個矛盾**：照 Anthropic 自己 [Building Effective Agents](https://www.anthropic.com/research/building-effective-agents) 的定義，**workflow = 人預先寫死的 code path、agent = LLM runtime 自主決定**。但 Dynamic Workflows 是——**agent（Claude）在 runtime 自己決定任務怎麼拆、然後產出一段 JavaScript orchestration script、丟給一個獨立的背景 runtime 執行**。人沒有寫那個 workflow、是 agent 寫的。它同時是 agent（誰決定）也是 workflow（怎麼跑）。 + +**核心機制 — 為什麼不只是「平行 subagent」**： + +關鍵不在平行、在 **context offloading**。一般 subagent / skill：每個中間結果都回到 Claude 的 context window。Dynamic Workflows：loop、分支、中間結果**全留在 script 變數裡**——只有最終 verified 答案回到 context。這是它能跑「幾十萬行 codebase 遷移、以既有測試套件為標準、from kickoff to merge」的原因——因為中間幾百次 agent 呼叫的雜訊不會塞爆 context。 + +| 面向 | 事實（official Claude Code docs 證實）| +|---|---| +| 它就是什麼 pattern | [Anthropic 五個 workflow pattern](https://www.anthropic.com/research/building-effective-agents) 裡的 **orchestrator-workers**（唯一「runtime 才決定 subtask」那個）放大版 + adversarial 驗證 | +| 規模上限 | **同時最多 16 個 agent**、單次 run **累計上限 1,000 個**（防 runaway）。不是「幾百個同時跑」 | +| 怎麼觸發 | (1) prompt 裡含 `workflow` 字 (2) saved command 如 `/deep-research` (3) `/effort ultracode`（xhigh reasoning + 自動 orchestration）| +| 平台 | Claude Code feature（CLI / Desktop / IDE / headless / Agent SDK、v2.1.154+）。**不是** raw API（沒有 `/v1/workflows`）| +| 品質機制 | adversarial propose / refute / converge loop——獨立 agent 各攻一角、其他 agent 試圖反駁、迭代到收斂、merge 前驗證 | + +**這一節就是用這個 pattern 研究出來的**（活範例）：本節事實是跑一個 dynamic-workflow-style 的 orchestration 蒐集的——4 個並行 research agent 各攻一個角度（官方 / 技術 / 定位 / skeptic）+ 一個 skeptic synthesizer 把**無法跨 source corroborate 的宣稱剔掉**。被剔掉的包括「hundreds of parallel subagents」（實為 16 concurrent / 1,000 total）、「750k 行 Bun 從 Zig 遷移到 Rust、過 99.8% 測試、11 天 merge」（vendor 單一 case、未獨立審計、不能當 verified fact）。**這就是 orchestrator-workers + adversarial verify 的實際運作**。 + +> ⚠️ **什麼不是（誠實限制，跟功能本身一樣重要）**： +> - **不是通用 workflow 引擎**（不是 Airflow / n8n / Temporal——人不畫 DAG、是 agent 寫 code） +> - **不是無限平行**（16 concurrent / 1,000 total per run、不是「幾百個同時」） +> - **不是 GA**（research preview、paid-plan-gated、定價 / 可用性可能變；官方兩個來源對 Pro 是否含括還不一致） +> - **reliability 提升主要來自「不確定時 abstain」**（refusal trade-off——答對更少不確定題、不是答對更多；來自 system card）。更多 subagent ≠ 更高正確覆蓋率 +> - **不是免費**（Anthropic 自己警告 token 用量「substantially more」、建議先用 scoped task 抓 consumption） +> - **不解決 dispatch problem**（「何時該 fan out vs 一次仔細做」還是靠 Claude runtime 判斷；獨立分析者指出 verifier 紀律仍薄——fan-out 沒驗證好可能產出「五十個看似合理的 bug」、比一次仔細做更糟） + +> 📚 **權威來源**：[Claude Code — Dynamic Workflows docs](https://code.claude.com/docs/en/workflows)（機制 + 16/1000 上限 + 觸發方式的 canonical）· [Anthropic — Introducing Dynamic Workflows](https://claude.com/blog/introducing-dynamic-workflows-in-claude-code)（定位 + token 警告）· [Anthropic — Building Effective Agents](https://www.anthropic.com/research/building-effective-agents)（workflow vs agent + orchestrator-workers 根基）· [Opus 4.8 announcement](https://www.anthropic.com/news/claude-opus-4-8)。 + +### ⏳ Harness 會過期：Model-Harness-Fit 與 Bitter Lesson + +前面教了很多 harness 怎麼設計。這裡補一個容易被忽略、但很重要的提醒：**你今天搭的 harness，是為「今天這個 model」量身搭的；model 一變強，有些 harness 就過期了。** + +- **Model-Harness-Fit**：一套 harness 是跟當下 model 的能力「配對」出來的。你為了補 model 的弱點（不會規劃、容易忘、不會自己驗收）搭了一堆 scaffold（外圍輔助結構）；等下一代 model 自己就會做這些事，那些 scaffold 反而變成多餘的包袱。 +- **Bitter Lesson（苦澀的教訓）**：AI 歷史一再證明，靠「人工塞進去的巧思」短期會贏，長期幾乎都輸給「讓模型用更多運算自己學」。套到 harness 上就是：過度搭建（over-scaffolding）是在跟這個趨勢對賭。 + +**白話的取捨**：能用「最薄、剛好能出貨」的 harness，就不要搭一座城堡。每加一層 scaffold，先問一句：「這是在補 model 真正的弱點，還是只是我不放心？」——下一代 model 出來時，前者會留下，後者會變成你要回頭拆的技術債。 + +> 📚 概念根源：Rich Sutton, [The Bitter Lesson](http://www.incompleteideas.net/IncIdeas/BitterLesson.html)（2019）；呼應本 stage「同一 model、不同 scaffold，分數差一倍」——scaffold 影響這麼大，它的「保存期限」就值得在設計時放心上。 + +### 🧑‍🔧 分工：你出題 + 驗收，agent 動手做 + +Anthropic 2026-06 的報告用真實使用資料歸成一句大白話：**你決定「要做什麼」，agent 決定「怎麼做」。** + +想像你**找人裝潢房子**：你說「我要一間北歐風書房、預算多少、幾號要好」，師傅去挑材料、動手施工；做完你不滿意就退回重做。你不會自己去釘每一塊木板——但「要什麼」和「驗收」始終是你的事。 + +報告的數字剛好也是這樣：平常一次工作裡，**「要做什麼」大約七成是你在決定，「怎麼做」大約八成丟給 agent**。 + +記三件事就好： +- **線劃清楚，agent 才幫得上忙**：你把力氣花在「講清楚要什麼 + 檢查成果」，動手的部分交給 agent；要求講得含糊，它就容易做歪。這正是本 stage 一直在說的「工作邊界」。 +- **越內行，agent 幫你放大越多**：因為你更會出題、更會驗收。所以它是**放大**內行人，不是取代內行人。 +- **一群 agent 也要分工**：等你同時用好幾個 agent（Stage 7）時，會多一個「總指揮」把工作切給各管一塊的小 agent——分工就從「人 vs agent」延伸到「agent 之間」。 + +> 📚 來源：[Anthropic — Agentic coding and persistent returns to expertise](https://www.anthropic.com/research/claude-code-expertise)（2026-06-16、用真實 Claude Code 使用資料量化「人 vs agent 分工」）· [2026 Agentic Coding Trends Report](https://resources.anthropic.com/2026-agentic-coding-trends-report)（開發從寫 code 轉向 orchestrating agent teams）。 + +### 📋 概念驗證 prompt（self-quiz） + +> 🛠️ **想直接動手寫 SKILL.md / CLAUDE.md？** 4 個實作用 prompt（audit 既有 / 生成新的）已**移到 [Stage 5](05-claude-code-ecosystem.md)**——讀者真正動手寫的地方： +> - [Stage 5.1 CLAUDE.md 設計 prompts](05-claude-code-ecosystem.md#-claudemd-設計-prompts依-5-原則) +> - [Stage 5.3 SKILL.md 設計 prompts](05-claude-code-ecosystem.md#-skillmd-設計-prompts含-skill-creator-替代) + +本節留 **1 個 quiz prompt 讓你確認自己懂這 5 個原則**（不寫 code 的概念 check）： + +#### Prompt 1 — Self-quiz（考自己懂不懂這 5 原則） + +``` +我剛學完 OpenAI 5 個 harness engineering 原則： +1. Legibility +2. System of Record +3. Progressive Disclosure +4. Taste Invariants +5. Throughput Changes Merge Philosophy + +請出 5 道情境題、每題描述一個真實的 SKILL.md / CLAUDE.md 設計決定（例如「我把所有 examples 都塞 SKILL.md 主檔、< 1000 行」），問**違反哪一條原則 + 該怎麼修**。 + +一次出 1 題、等我回答後給回饋、再出下一題。最後給總成績。 +``` + +→ **建議用法**：學完上面 5 個原則後跑這個 quiz 確認你抓到 concept；實作面（寫 SKILL.md / CLAUDE.md）的 prompt 在 [Stage 5](05-claude-code-ecosystem.md)。 + +### 📐 進階 agentic 概念應用流程（讀者導引） + +學完前面 5 原則 + Anthropic 對照後，**怎麼把這些概念實際用到 agent 設計上？** 從 Stage 7（你已能做 production agent）往下、5 個步驟到 production： + +1. **建立概念地圖主軸 — 四層工作邊界**：Types → Config → Repo → Service。想清楚 agent 能動到 stack 哪一層、跨層算違規。 + → 本 stage §概念地圖主軸：四層工作邊界 + +2. **挑 2-3 個相關進階概念**：從 12 個 skeleton 找跟你問題最相關的（Work Boundary / Contract / PAR / Autonomy 等）。 + → 本 stage §12 個進階概念（pattern 清單） + +3. **套 5 個 OpenAI 原則（cross-cutting）**：Legibility / SoR / Progressive Disclosure / Taste Invariants / Throughput Merge Philosophy。這 5 原則跨越所有 12 概念、決定「做對與否」。 + → 本 stage §跨概念 Harness Engineering 原則 + +4. **Encode 到 Skills + CLAUDE.md**：用 Stage 5 的 4 個 prompt 動手寫——CLAUDE.md audit / generate（[Stage 5.1](05-claude-code-ecosystem.md#51--claude-code-基礎)）+ SKILL.md audit / generate（[Stage 5.3](05-claude-code-ecosystem.md#53--skillsclaude-code-的行為層-claude-code-生態最關鍵的一層)）。 + +5. **Verify with acceptance gate**：Preset YAML 抓 drift / LLM-as-judge 自動評 / Human spot-check 補 edge case。 + → [agent-collab-skills](https://github.com/WenyuChiou/agent-collab-skills) + +→ **Production agent ready**：可穩定給人用、有自動驗證、有可預測失敗。 + +→ **怎麼用這 5 步**：第一次讀本 stage 時、照 1 → 5 順序走；之後實作 agent 卡關時、回來判斷自己卡在哪一步。 + +→ **跟前面 Why → What → How 表的差別**：那張是「**痛點 ↔ 原則 ↔ 工具**」的橫向對應、reference 用；這 5 步是「**從學完到上線**」的縱向流程、do-this-then-this 用。 + +## 📖 完整 reading path（按 depth 分層） + +按深度排序、不必全讀。Foundation tier 必讀（~95 min）、其他**遇到問題再深挖**： + +### 🌳 Reading decision tree（按你卡的問題選讀） + +![Agentic AI Advanced Reading Decision Tree](../resources/diagrams/reading-decision-tree.png) + +不是 reading list、是 **decision tree**——你現在卡哪個問題、就先讀對應的那 1-2 篇（上圖 5 個分支對應 5 種卡關狀況；下面是每個分支讀完第 1 篇後的進階推薦）： + +**分支讀完後的延伸閱讀**（branch-specific second readings）： + +- 「不知道 agent 怎麼開始」→ 再讀 ReAct paper + Lilian Weng "LLM Powered Autonomous Agents" +- 「multi-agent 要不要用、怎麼開」→ 再讀 Anthropic Multi-Agent Research（case study section） +- 「context 沒效率」→ 再讀 Anthropic Multi-Agent Research（memory section） +- 「eval 怎麼寫 / 自動驗證」→ 再讀 Anthropic Multi-Agent Research（eval section） +- 「想跟上 frontier 現況」→ 再讀 AutoGen + ReAct paper + +→ **規則**：每個分支**最多挑 2 篇深讀**——讀完再回來看自己其他問題、決定下一篇。不要 broad-scan 全部清單。 + +下面是按 depth tier 排的完整 list（給已知道想讀什麼的人查）： + +**Foundation tier**（先讀這 4 個、~95 min 總長）: +- [Anthropic — Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents) +- [Cognition — Don't Build Multi-Agents](https://cognition.ai/blog/dont-build-multi-agents) +- [Anthropic — How we built our multi-agent research system](https://www.anthropic.com/engineering/built-multi-agent-research-system) +- [Lilian Weng — LLM Powered Autonomous Agents](https://lilianweng.github.io/posts/2023-06-23-agent/) + +**Workflow patterns tier**: +- [LangGraph Planning Agents Tutorial](https://blog.langchain.com/planning-agents/) +- [Microsoft AutoGen docs](https://microsoft.github.io/autogen/) +- [DSPy](https://dspy.ai/learn/) + +**Production / Harness tier**: +- [OpenAI Harness Engineering (2026-02)](https://openai.com/index/harness-engineering) +- [Hamel Husain Evals blog](https://hamel.dev/blog/posts/evals/) +- [Simon Willison coding agents notes](https://simonwillison.net/tags/coding-agents/) + +**Frontier research papers**（挑 3-5 篇深讀）: +- ReAct / Reflexion / CoALA / Self-Discover / Voyager / Constitutional AI / AutoGen + +**中文 / hands-on**: +- [李宏毅 GenAI 2024 / 2025](https://speech.ee.ntu.edu.tw/~hylee/) +- [datawhalechina/hello-agents](https://github.com/datawhalechina/hello-agents) + +> 📋 **學完進階概念回頭看 synthesis** → [Stage 5 §🗺️ 7-Layer Architecture Map](05-claude-code-ecosystem.md#-7-layer-architecture-map先看這張圖再讀-51-57)（Claude Code 7 個 primitive + 3 個工程學 discipline 整合圖） + +## ✅ 自我檢查 + +讀完本 stage 應該能： + +- [ ] 用 **Types → Config → Repo → Service** 四層解釋 Cognition Flappy Bird / Anthropic speculative-leap 為什麼算 work boundary 違規 +- [ ] 說出 12 個進階概念中 5 個的「動到哪一層」+ 一句話定義 +- [ ] 講得出 Harness Engineering 原則的 4 大類別（① Context 管理 / ② Interface / ③ Quality 驗證 / ④ Process 紀律）各自解什麼問題，以及類別之間的 enabling 關係 +- [ ] 知道下一步該翻哪份 paper / blog（不必全部讀完） +- [ ] 區分 PAR loop（單 agent 自我修正）vs agent-debate（2 agents 對峙） +- [ ] 在 brief 中明確寫出 task 的 work boundary（in / out） + +→ 全部能做到 = 你已超出 Stage 7 production 化的範圍、進入 frontier agentic workflow design。**剩下就是看哪個 paper 對你最痛、深讀那個**。 + +--- + +→ 接下來 [**Stage 8 — Agent 操作介面（Agent Interfaces）**](08-agent-interfaces.md)（**兩 track 共用 hub**）：學 agent 怎麼跟非 API 世界互動（Computer Use / Browser Use / Code Sandbox）。或挑一個[特化分支](../README.md#️-學習地圖兩條學習路徑)、或回過頭來貢獻這份 repo。 diff --git a/stages/07.5-advanced-agentic-concepts.zh-Hans.md b/stages/07.5-advanced-agentic-concepts.zh-Hans.md new file mode 100644 index 0000000..763edd0 --- /dev/null +++ b/stages/07.5-advanced-agentic-concepts.zh-Hans.md @@ -0,0 +1,628 @@ +# Stage 7.5 — 进阶 Agentic 概念地图（Advanced Agentic Concepts Map） + +> [繁體中文](./07.5-advanced-agentic-concepts.md) | **简体中文** | [English](./07.5-advanced-agentic-concepts.en.md) + +⏱ **时间估算**：1 周（约 5 小时——不写 code、只读资源建立概念地图） + +> 🚪 **进入条件**：完成 [Stage 7 — 多 Agent 系统与稳定运作](07-multi-agent-production.md)（或至少 Stage 4 + 6 + 7）。本章是 production 之后的 frontier 概念地图、不是入门——没做过 production agent 会读不出这些概念在解什么痛点。 + +> 💡 这是一份 **进阶概念地图 + reading path**，不是完整教学。Stage 4 / 6 / 7 学完后已经能做能上线给人用的 **agent**（AI 自主执行体、自己会规划 + 执行任务的 LLM 系统、俗称 production agent）；本 stage 帮你定位**业界还在讨论哪些进阶概念存在**、**每个概念解什么问题**、**该先读哪些 paper / blog**，避免你在真实工作里踩到别人已经踩过的坑。 + +> 📋 **本章内容**（8 个区块、依序读）： +> +> 1. 为什么有这 stage（定位） +> 2. 概念地图主轴：**Types → Config → Repo → Service** 四层工作边界 +> 3. 12 个进阶概念 skeleton +> 4. 为什么选这 12 个 +> 5. 跨概念 Harness Engineering 原则（4 大类别 + 关系图） +> 6. 进阶 agentic 应用流程（5 step） +> 7. 完整 reading path +> 8. 自我检查 + +> 🔤 **常用缩写快速表**（本章会反复出现）： +> +> | 缩写 | 全称 | 一句话解释 | +> |---|---|---| +> | **agent** | AI 自主执行体 | 自己会规划 + 执行任务的 LLM 系统 | +> | **PR** | Pull Request | 把改动送进主分支的请求（GitHub 术语）| +> | **SoR** | System of Record | 知识的权威来源、唯一 source of truth | +> | **ACI** | Agent-Computer Interface | agent 跟系统之间的对接层（工具 / API / docs）| +> | **MCP** | Model Context Protocol | 把 agent 工具标准化的 spec | +> | **PAR** | Plan-Act-Reflect | 单 agent 自我循环模式（规划 → 执行 → 反思 → 修正 → 重试）| +> | **CI** | Continuous Integration | commit 后自动跑 test / lint 的系统 | +> | **QA** | Quality Assurance | 质量把关（人工或自动）| +> | **lint / linter** | — | 自动扫 code 找违规的工具 | +> | **`[OAI]` / `[Anth]`** | OpenAI / Anthropic | 后面的 source tag | + +## 🎯 为什么有这 stage + +Stage 4 / 6 / 7 加起来足以做 **70% 能给真人用、不会三天两头搞砸的 agent**。每个 stage 教什么、学完能做什么： + +| Stage | 教什么 | 学完能做什么 | +|---|---|---| +| **4** | 挑 **framework**（agent 用哪个工具写）| LangGraph / AutoGen / DSPy 选一个来写 agent | +| **6** | **context engineering**（动态管理塞给 agent 的资料）| memory / retrieval / prompt 组装 | +| **7** | **harness engineering**（agent 周围的可靠运作环境）| observability / retry / cost gate / eval / sandbox 等 8 个元件 | +| **7.5（本章）**| **进阶概念地图** | 遇到问题知道翻哪篇 paper / 看别人 agent 知道它在做什么 / 知道每个概念对应 agent 系统哪一层 | + +但前线的 AI lab（Anthropic / OpenAI / Cognition / Microsoft）+ 学术界（Stanford / CMU / Princeton）在 2024-2026 持续推出 12+ 个进阶设计概念。**有些现在你用不到、但需要知道它们存在**——这样以后遇到问题时、才知道有现成的 pattern 可以套。本 stage 不是再教一套理论、而是给你一张**地图**： + +- 不是要你全部学会 +- 不是要你全部都用 +- 是让你**遇到问题时、知道该翻哪篇 paper / blog 找答案** +- 是让你**看别人写的 agent 时、看得出它在做什么**——举例：别人的 agent 一出错就”重试 N 次直到放弃”（=只有 retry）、跟它”出错后先反思一次、修正做法、再重试”（即所谓 PAR loop）、是完全不同等级的设计。能看出差别、你才知道该不该借用对方的做法 +- 是让你**知道每个概念该套到 agent 系统哪个部分、解哪一类问题** + +## 🧭 概念地图主轴：四层工作边界（work boundary） + +本 stage 用**工作边界**作为整理进阶 agentic workflow 的主轴：把 agent 系统拆成 4 个层级（**Types → Config → Repo → Service**、下面马上展开）、然后问“agent 动的对象属于哪一层？跨层越界会出什么问题？”这不是把整章缩成一个模型、而是先给读者一个定位坐标、后面 12 个概念才能放到同一张图上比较。 + +> 💡 **“stack”是什么意思**：软件工程习惯把系统拆成上下层、每层各管一件事、上层盖在下层之上、合称 stack（堆叠）。例如 web 应用常见“frontend → backend → database”三层 stack。本 stage 把 agent 系统也拆成 4 层（Types / Config / Repo / Service）、看 agent 该动到哪一层。 + +> ⚠️ **这套 4 层跟 Stage 7 的 prompt → context → harness 三层不一样，是两种不同视角**： +> - **Prompt → Context → Harness**（Stage 7）：**stack 位置**——你正在工程“字符串 / 信息 / 外围 runtime”的哪一个对象？ +> - **Types → Config → Repo → Service**（本 stage）：**自主权范围**——agent 能动到 stack 多深？跨层越界算不算违规？ +> +> 两者**正交**，解决的是不同问题。读完本节后，你应该能同时用两种视角看 agent 系统。 + +借用软件架构里的 **Types → Config → Repo → Service** 分层，套到 agent 系统： + +![Agentic Stack 四层工作边界](../resources/diagrams/stack-4layer.zh-Hans.png) + +→ 每一层上下都是一个**工作边界**。agent 操作的范围 = 它的自主权范围： + +- **Agent at Types layer** = 只能符合既有 contract（契约），不能改 schema（例：Codex 接到 brief 后，只能加 inline gloss） +- **Agent at Config layer** = 可以调 budget / policy，但不能改 memory（例：context-budget agent 改 `max_cost_usd`） +- **Agent at Repo layer** = 可以读写 memory / vector store，但不能 redesign workflow +- **Agent at Service layer** = 可以重组整个 workflow，拥有最高自主权 + +### 为什么工作边界适合当主轴 + +很多进阶概念最后都会回到同一个问题：agent 的自主权到底到哪里为止？把 agent 当作新进实习生来想：你交代一个明确的小任务、他却自作主张把邻近的东西也动了——这就叫“跨工作边界”。业界已有 3 个公开记录的真实案例可以对应： + +- **越界没收手**（Cognition 的 Flappy Bird 案例）：用 **multi-agent**（多个 agent 并行协作）拆解任务、其中一个 **subagent**（主 agent 派出的子 agent、执行某个子任务）负责画绿色管道、另一个负责画云朵背景，结果合起来双方风格完全对不上——因为每个 subagent 只看到自己那块、没有对方在做什么的 **context**（上下文、agent 拿到的全部资料）。Cognition 直白写道：“sub-agent 像一群过度自信的新人、根本不会在该问的时候问问题”。 + → 出处：[Cognition — Don't Build Multi-Agents (2025-06)](https://cognition.ai/blog/dont-build-multi-agents) + +- **加料**（Anthropic Multi-Agent Research 的 speculative-leap 现象）：subagent 被指派“研究某主题”、它在报告里擅自加上“我推测 X 也可能成立、虽然我没验证”这类没人要的推论。Anthropic 在他们的 multi-agent 论文里专门讲为什么这种“主动补完”要通过工程设计消除、否则 hallucination 会在 supervisor 没注意时偷渡进结果。 + → 出处：[Anthropic — How we built our multi-agent research system (2025-06)](https://www.anthropic.com/engineering/built-multi-agent-research-system) + +- **operator 给太多权限**（Replit Agent 2024 prod database 事故）：据社群讨论，有用户把 production database access 直接交给 agent、没设“破坏性操作要先 confirm”的 gate，结果 agent 在“修 bug”过程跑了破坏性 SQL、清掉 production 数据。错不在 agent 看起来合理地照指令做、错在 operator 没设边界。 + → 出处：[Simon Willison 对此事故的分析（2024）](https://simonwillison.net/2024/Aug/26/replit/)（社群整理、非 Replit 官方 postmortem） + +**这 3 个案例告诉你的事**： + +- agent 不会“刚好停在你交代的那个点”——brief 要明确写“**只能动 X、绝对不要动 Y**”、subagent 要明确收到 parent 的 full context +- agent 会主动“补完”没被要求的东西——要靠 structured output schema + evaluator-optimizer loop 把这类 speculative 内容过滤掉 +- 规则“装好了 ≠ 会被遵守”——operator 自律不够、必须有 mechanical gate（permission check / cost cap / destructive op confirm）才挡得住 + +→ **实作对照**：work boundary 写进 brief（Anthropic 的 brief template、LangGraph 的 state schema、`agent-collab-skills` 的 task-splitter 都是同一概念）+ enforce 在 acceptance gate / evaluator loop + 破坏性操作加 explicit gate（[autonomy gradient](#-12-个进阶概念--skeleton) 那节展开）。 + +### 🔁 Failure-mode lifecycle（产业级 agent 失败模式怎么演化成最佳实践） + +![Failure-Mode Evolution Cycle](../resources/diagrams/failure-lifecycle.zh-Hans.png) + +每个产业级 agent failure mode 都走过 **发现 incident → 公开文档化 → encode 成 framework pattern → 自动消除** 的循环。5 个有公开记录的案例： + +| # | Incident（发现）| 文档化（命名）| Codify（变成什么 pattern）| 公开出处 | +|---|---|---|---|---| +| 1 | Multi-agent subagent context drift（Flappy Bird 风格分裂）| "Sub-agents don't share principal-agent context" | **Single-thread principle**: 别堆 multi-agent、用 linear orchestration | Cognition 2025-06 | +| 2 | Subagent speculative leap（没验证的推论偷渡进结果）| "Speculative hallucination via filling-in" | **Evaluator-optimizer loop**: 加 critique step 强制 review | Anthropic Multi-Agent Research 2025-06 | +| 3 | Production permission drift（agent 砍 prod DB）| "Unbounded autonomy on destructive ops" | **Autonomy gradient**: suggest / propose / execute 三段授权 | Replit Agent 2024 incident | +| 4 | Agent looping without self-criticism（AutoGPT 卡 loop）| "Reflexion-less iteration" | **Plan-Act-Reflect loop**: 加 self-critique + revise | Reflexion paper (Shinn 2023) | +| 5 | Skill library corruption（broken skill 进 library）| "Untested skill commit" | **Pre-verify before commit**: skill 入 library 前必跑 test | Voyager paper (Wang 2024) | + +→ **这套“fail → publish → codify → fix”循环是整个 agentic 领域的进化机制**——不是“一开始就写死所有规则”、而是“**每个 production incident 都被公开 + codify 成 pattern**”。Anthropic Skills 的 `references/` 机制、OpenAI 的 Taste Invariants、LangChain 的 evaluator pattern、Anthropic 的 evaluator-optimizer——都是同一逻辑的不同实作。 + +→ **怎么用这张表学**：遇到自己 agent 搞砸时、查上表“最像哪一行”、然后读对应 pattern 名（Single-thread / Evaluator-optimizer / Autonomy gradient / PAR / Pre-verify）的 deep dive。本 stage 后面 12 个 skeleton 涵盖全部 5 个 pattern。 + +## 📚 12 个进阶概念 — skeleton + +每个概念控制在 4 行以内：一句话定义 + agent 动到哪一层 + 最值得读的 1 个资源。 + +### 🗺️ 12 概念 cluster map（动到哪一层 × 解什么类型的问题） + +![12 Advanced Agentic AI Concepts — Cluster Map](../resources/diagrams/concept-cluster.zh-Hans.png) + +上图把 12 个概念按 **“动到哪一层”**（横轴）和 **“解决什么类型的问题”**（纵轴）分群、让你一眼看出哪些概念适合一起学、哪些可以暂时跳过。其中 **Work Boundary（#1）跨所有层**（属于通用纪律、不是某个特定位置）。 + +→ **怎么用这张 map**： +- **第一次学**：先学“**编排类 + 反思类**”（共 6 个，是 multi-agent / production 的基础） +- **要 deploy production**：补“**治理类 + 韧性类**”（共 6 个，防止上线搞砸） +- **跨类别主轴**：**Work Boundary（#1）是贯穿全部 12 个概念的 root discipline** + +下面 12 个概念用表格列出（# / 概念 / 动到哪一层 / 一句话定义 / 最佳读物）： + +| # | 概念 | 动到哪一层 | 一句话定义 | 最佳读物 | +|---|---|---|---|---| +| 1 | **Work Boundary / Scope Discipline** | 跨所有层（discipline）| agent 只动 brief 指定的对象、不越界 | [Hamel — Evals + Skills](https://hamel.dev/blog/posts/evals-skills/) + [Cognition — Don't Build Multi-Agents](https://cognition.ai/blog/dont-build-multi-agents) | +| 2 | **Contract-driven Hand-offs** | Types + Service | 上游 agent 承诺的 artifacts、下游 agent 必须验证自己真的收到了 | [Anthropic — Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents) Routing pattern | +| 3 | **Speculative / Parallel Exploration** | Service（编排）| 跑 N 条 alternative 路径、最后取最佳那条（不只是独立 parallel）| [LangGraph Plan-Execute Tutorial](https://blog.langchain.com/planning-agents/) | +| 4 | **Agent-as-Judge / Constitutional AI** | Service（agent 评 agent）| 用一个 agent 评另一个 agent 的输出、并按明确原则反复修正 | [Constitutional AI (Bai 2022)](https://arxiv.org/abs/2212.08073) | +| 5 | **Plan-Act-Reflect Loop** | Service（单 agent 自我循环）| write plan → execute → critique → revise → re-execute、直到 PASS 或 EXHAUSTED | [Reflexion (Shinn 2023)](https://arxiv.org/abs/2303.11366) + [Self-Discover (Zhou ICML 2024)](https://arxiv.org/abs/2402.03620) | +| 6 | **Hierarchical Task Decomposition** | Service（多层 supervisor）| supervisor → worker → sub-worker、至少 2 层 recursion | [Microsoft AutoGen GroupChat docs](https://microsoft.github.io/autogen/) | +| 7 | **Autonomy Gradients / Trust Layers** | Config（autonomy policy）| 不同任务给不同自主权（suggest / propose / execute）| [Claude Code permission system](https://docs.claude.com/en/docs/agents-and-tools/claude-code/overview) | +| 8 | **Cost-aware Budget Gates** | Config（cost policy）| 超过 \$ 预算就自动停或升级审核（不只是 token 上限）| [OpenAI Harness Engineering (2026-02)](https://openai.com/index/harness-engineering) | +| 9 | **Failure Injection / Chaos Eval** | Service（测试 agent 容错）| 故意给 broken input / stale data / API timeout、看 agent 怎么处理 | [Hamel Husain — Evals blog series](https://hamel.dev/blog/posts/evals/) | +| 10 | **Self-organizing Teams** | Service（agent 动态协商 role）| agents 不是预先分配 role、而是根据任务动态分工 | [CAMEL (Li 2023)](https://arxiv.org/abs/2303.17760) + AutoGen | +| 11 | **Spec-driven Development** | Types（spec = code）| agent task 由 formal spec（YAML / JSON Schema）定义、不是自由 prompt | [DSPy](https://github.com/stanfordnlp/dspy) signatures tutorial | +| 12 | **Graceful Degradation Paths** | Config（fallback policy）| frontier model 挂掉时、回退到便宜 model 并降低预期、而不是直接 crash | [OpenRouter routing docs](https://openrouter.ai/docs) + [Anthropic model fallback](https://docs.claude.com/en/docs/build-with-claude/models) | + +## 为什么选这 12 个 + +- 都有可验证的 primary source（Anthropic / OpenAI / Cognition / Microsoft / academic paper），不是空谈 +- 都对应到至少一个公开实作（LangGraph / AutoGen / Anthropic Skills / DSPy 等）、可以直接拿来抄 +- 都在 Stage 4 / 6 / 7 已覆盖范围之外，没有重复 +- 避免“无限延伸”——其他进阶概念（Voyager skill learning / MemoryLLM / world models）很重要，但**先把这 12 个学完** + +## 🔬 跨概念 Harness Engineering 原则（多 source 整理） + +**这些原则不是任何单一厂商独有的**。Anthropic、OpenAI、Cognition、Hamel Husain 等都在各自的文章 / blog / docs 中谈过，只是用词不同，但指向的是同一组设计约束。下面先把**原则整理成 4 大类别**、列出主要来源，再往下展开细节。 + +> 📚 主要 source： +> - **Anthropic**（Building Effective Agents · Skills · Multi-Agent Research · CLAUDE.md memory docs） +> - **OpenAI**（[Harness Engineering 2026-02](https://openai.com/index/harness-engineering/)，把它们清楚整理成 5 条命名原则） +> - **Cognition AI**（[Don't Build Multi-Agents](https://cognition.ai/blog/dont-build-multi-agents)） +> - **Hamel Husain**（[Evals are everything](https://hamel.dev/blog/posts/evals/)） +> - **Lilian Weng**（[LLM Powered Autonomous Agents](https://lilianweng.github.io/posts/2023-06-23-agent/)） + +> 🔤 **下面表格 source tag 缩写对照**（之后 chapter 都会用这 4 个 tag）： +> - `[OAI]` = OpenAI +> - `[Anth]` = Anthropic +> - `[Cognition]` = Cognition AI +> - `[Hamel]` = Hamel Husain + +### 4 大类别 × 多 source + +| 类别 | 核心问题 | 该类别下的原则（含 source） | +|---|---|---| +| **① Context 管理** | 上下文不爆炸，agent 永远拿到对的信息 | **System of Record** [OAI] / **Memory Persistence** [Anth] / **Progressive Disclosure** [OAI + Anth] | +| **② Interface / 沟通** | agent 看得懂 codebase，也能说清楚自己在做什么 | **Legibility** [OAI] / **ACI / Tool Documentation** [Anth] / **Transparency**（show planning）[Anth] | +| **③ Quality / 验证** | 写得对，不能 hallucinate | **Taste Invariants** [OAI] / **Evaluator-Optimizer loop** [Anth] / **Human + LLM-as-Judge** [Anth] / **"Evals are everything"** [Hamel] | +| **④ Process 纪律** | scale + iterate 不爆 | **Simplicity** [Anth] / **Throughput Changes Merge Philosophy** [OAI] / **Don't Build Multi-Agents (when unnecessary)** [Cognition] | + +→ **OpenAI 那 5 条原则**是“命名最清楚 + case study 最完整”的整理版本。但类别 ① 的 SoR / Memory Persistence、类别 ② 的 ACI、类别 ③ 的 Eval-Optimizer、类别 ④ 的 Simplicity，其实都先由 Anthropic 等其他来源讲过。下面会继续用 OpenAI 的命名往下展开，因为它们写得最完整，同时在每节标出对应的 Anthropic 等 source。 + +### 原则之间的主要关系（cross-category dependencies） + +它们不是 5 条互不相关的原则，也不是 12 个彼此孤立的概念。它们之间有明显的 **enabling 关系**： + +![5 个 Harness Engineering 原则之间的 cross-source dependency](../resources/diagrams/principle-dependency.zh-Hans.png) + +→ **4 个关系 insight**： + +| 关系 | 描述 | 为什么重要 | +|---|---|---| +| **SoR + Memory + PD 三者配对** | SoR 提供目的地、Memory 跨 session 保存信息、Progressive Disclosure 是导航机制 | 三者单独用不够、必须一起设计 | +| **Legibility ↔ Transparency 双向** | agent 能读 codebase 才能 self-report；agent 会 self-report 你才能验证 legibility 有效 | 互为前提、缺一不可 | +| **Quality 是 Process 自动化前置** | 没写死 invariants + eval loop、人类就无法把 review 交给 automation | 是 ④ Process 的必要条件 | +| **Simplicity 是隐性 root** | 一上来就堆 multi-agent、其他所有原则的复杂度都会暴涨 | Cognition“Don't Build Multi-Agents”= Anthropic“Simplicity”、同一回事 | + +→ 下面 5 节继续用 OpenAI 的命名版本展开（因为最完整），同时回头标 cross-source mapping。 + +### 为什么要在意这些原则 — Why → What → How + +下表以“**痛点（Why）→ 原则（What）→ 实作（How）**”三层解释这些原则在解什么问题、用什么工具落地： + +| 痛点（Why）| 原则（What）| 实作（How）| +|---|---|---| +| Context 200k 满 / Multi-agent context overflow | Progressive Disclosure + Memory Persistence | Skills `references/` / `CLAUDE.md` `@-import` / `.ai/` brief | +| Agent 看不懂自家 codebase | Legibility + Tool Doc / ACI | `AGENTS.md` 100 行 / poka-yoke 工具设计 / 一致 schema 命名 | +| 多 agent desync、不同事实版本 | System of Record | `docs/` + `.coord/` shared-memory skill | +| 随机 drift / Review 漏抓 | Taste Invariants + Transparency（planning 显示）| `agent-acceptance-gate` preset YAMLs / evaluator-optimizer loop | +| Agent 写 PR 快、Human QA 跟不上 | Throughput Changes Merge Philosophy | mandatory preset / LLM-as-judge / Human spot-check | +| 一上来就堆 multi-agent overkill | Simplicity（Anthropic）| 先 basic LLM call、确认需要才加 agent | + +→ **6 个痛点 → 5 + 3 个原则**（OpenAI 5 + Anthropic 3 extra）→ **8+ 个具体工具 / 机制**。 + +### 5 个 OpenAI 原则速查表 + +下面 5 节各自展开原则细节（含 OpenAI 原文 quote）；先给速查表： + +| # | 原则 | 一句话 | 跨 work boundary | 对应 tool | +|---|---|---|---|---| +| 1 | **Legibility** | 把 agent 当新进工程师、为它优化 navigability（不是让人读懂 agent）| Repo + Types | Skill `references/` + AGENTS.md / CLAUDE.md pattern | +| 2 | **System of Record** | 知识住 `docs/`、不住 prompt；100 行 entry map 指向深处 | Repo | `.coord/memory.yml` shared-memory + AGENTS.md / CLAUDE.md | +| 3 | **Progressive Disclosure** | 小 entry point + 教 agent 之后去哪查（跟 SoR 配对：SoR 提供 destination、PD 是 navigation）| Repo + Types | Skill `references/` 机制 + Codex `.ai/.md` brief | +| 4 | **Architecture & Taste Invariants** | 定义边界、不细管实作；lint 强制 schema / 命名 / 文件大小 | Config + cross-cutting | `agent-acceptance-gate` preset YAML、custom linter | +| 5 | **Throughput Changes Merge Philosophy** | agent PR 速度 > 人类 QA 速度 → QA 必须自动化、不依赖逐行读 | Service（merge workflow）| 自动 lint + test + acceptance gate、mandatory preset | + +→ 下面 5 节展开每个原则本身、最后 Anthropic ↔ OpenAI 对照 列出两家对照词 + 推荐读本。 + +### 1. Legibility — 让 agent 能读懂 codebase / docs + +> “Because the repository is entirely agent-generated, it's optimized first for **Codex's legibility**.” — OpenAI + +人类读 code 有很多视觉辅助：IDE 高亮、跳转、目录树、鼠标 hover、直觉。**agent 全部没有**——它只看纯文字 + 工具回传值。如果 codebase / docs 对 agent 不友善、agent 读错地方、推论错方向、写错 code。优化方向跟”让人读懂 agent 输出”相反：**把 agent 当新进工程师、为它优化 navigability**。 + +**(a) Codebase 对 agent 友善** + +像给新人写的 onboarding 文档、所有”靠经验推测”的东西都得明写： + +- **一致 schema 命名**：`get_user_by_id` 永远用这个格式、不要混 `fetchUser` / `findUserById` / `userLookup` 三种写法。AI 看 1000 个文件、靠 pattern matching 推论、规律不一致就会推错。 +- **文件大小限制**：规定文件 < 500 行、agent 一次能完整读进 context。超过 500 行 agent 一次读不完、就会跳读、漏看关键逻辑。 +- **`docs/` 阶层结构**：`docs/api/` / `docs/architecture/` / `docs/runbook/` 分得清楚、agent 才知道 routing。乱塞一个目录里、agent 找不到入口。 + +**(b) Tool / API 对 agent 友善（ACI）** + +agent 跟工具的对接层、就是 ACI（Agent-Computer Interface）。设计目标： + +- **清楚的 tool description**：每个 tool 一行讲”干什么”、不是只写 function signature。AI 没法靠变数名猜功能。 +- **Poka-yoke 工具设计**：把容易出错的设计拿掉。例：强制吃 absolute path、不允许 relative；强制吃 ISO 日期格式、不允许自由文字。让 agent 想犯错都犯不了。 +- **Schema 标注**：每个字段都有 type + 简述 + 范例值。AI 看了就能直接用、不用猜。 + +→ **核心精神**：**为 agent 优化、不是为人**——很多优化方向跟”让人读爽”相反、但 agent 才是现在 80% 的 reader。 + +- **跨 work boundary**：Repo + Types +- **对应 tool**：Claude Code Skill 的 `references/` 机制 + AGENTS.md / CLAUDE.md pattern + +### 2. System of Record — 知识的唯一权威来源 + +> "The repository's knowledge base lives in a structured `docs/` directory **treated as the system of record**. A short `AGENTS.md` (roughly 100 lines) is injected into context and serves primarily as a map." — OpenAI + +LLM 容易忘事、容易脑补。如果你把所有业务知识都塞进 system prompt、会发生两件事：(1) context 立刻爆（200k token 也不够塞）、(2) 不同 agent / 不同 session 读到的版本对不上、产生"自相矛盾的事实"。SoR（System of Record、知识的权威来源）解这个问题：**所有真实知识住外部 docs、不住 prompt、agent 从 docs 动态拉**。 + +**(a) 知识住 docs、不住 prompt** + +像公司只有一份"员工手册"当权威、不要每个 onboarding 都重抄一份： + +- **100 行 entry map**：AGENTS.md / CLAUDE.md 只放"地图"（指向 `docs/` 各区的索引）、不放实际内容。 +- **`docs/` 结构化**：实际内容住 `docs/api/`、`docs/architecture/`、`docs/runbook/` 等子目录、agent 按需拉。 +- **prompt 绝不重复 docs 内容**：避免"prompt 讲一套、docs 讲另一套"的版本对不上。 + +**(b) 跨 session / 跨 agent 持久化** + +agent 不是一次性对话、会跨好几个 session、subagent 之间也要共享事实： + +- **`.coord/memory.yml` 共享记忆**：subagent 跟 supervisor 读同一份、不会各自记不同版本。 +- **decisions log**：重要决策写进 docs、新 session 从读档开始、不依赖"上次跟 agent 讲过的话"。 +- **versioned**：用 git 管 docs、要追溯"这条知识什么时候改的"随时可查。 + +→ **核心精神**：**唯一权威、单向同步**——agent 从 SoR 拉、不从 prompt 拉；SoR 改了所有 agent 下次跑都读新版本。 + +- **跨 work boundary**：Repo +- **对应 tool**：`.coord/memory.yml`（[agent-shared-memory](https://github.com/WenyuChiou/agent-collab-skills) skill）+ AGENTS.md / CLAUDE.md pattern + +### 3. Progressive Disclosure — 从 small entry point 渐进深挖 + +> "Agents start with a small, stable entry point and **are taught where to look next**, rather than being overwhelmed up front." — OpenAI + +一次塞太多 context 给 agent、它会被淹没——注意力分散、抓不到重点、质量下降、token cost 暴增。**正确做法是分批揭露**：先给 small + stable 的入口、再"教 agent 之后去哪查"。跟 #2 SoR 配对运作：**SoR 提供目的地**、**PD（Progressive Disclosure）是导航机制**。 + +**(a) Small entry point** + +入门 prompt 像书的目录、不是整本书： + +- **AGENTS.md / CLAUDE.md 100 行**：只放最高层的"这个项目干什么 + 主要结构在哪"、不放细节。 +- **brief 而非 dump**：给 agent 任务时、用 100 行 brief、不是把整个 codebase 一次倾倒给它。 +- **入口要 stable**：100 行的内容变动越少越好、agent 才能对它建立可靠的 mental model。 + +**(b) Navigation 机制——教 agent 之后去哪挖** + +agent 需要的时候、自己去拉深处资料： + +- **Skill `references/` 机制**：Claude Code 的 Skill 把详细 reference 放在 `references/` 子目录、agent 需要才 load。默认不进 prompt、需要才进。 +- **`@-import` 语法**：CLAUDE.md 可以写 `@docs/architecture.md` 指向深处、按需拉而不是预先塞。 +- **task brief 指向法**：Codex `.ai/.md` brief 一开头给 agent "先读 `docs/X.md` 第 1-2 节、执行前再读 `docs/Y.md`"。 + +→ **核心精神**：**懒加载（lazy load）胜过热加载（eager load）**——能晚一刻塞 context、就晚一刻。 + +- **跨 work boundary**：Repo + Types +- **对应 tool**：Claude Code Skill 的 `references/` 机制（只在 agent 需要才 load）+ Codex `.ai/.md` brief pattern（先读 brief 再决定深挖） + +### 4. Architecture & Taste Invariants — 用 lint 强制不变量 + +> "We enforce these rules with custom linters and structural tests, plus a small set of **'taste invariants.'** ... **By enforcing invariants, not micromanaging implementations**, we let agents ship fast." — OpenAI + +AI 写 code 时倾向“怎么快怎么来”、常导致模块过度耦合、命名混乱、文件爆炸。OpenAI 团队用**强制性结构规则**约束 AI、让 agent 在你划好的边界内快跑、而不是每行都要人盯： + +**(a) Enforcing Architecture — 用“物理边界”框住 AI** + +像在盖房子前先搭好钢筋支架、AI 只能在格子里填肉： + +- **单向依赖（One-way Dependency）**：定义严格的层级、底层的 Types 绝对不能引用高层的 Service。AI 想偷渡 import 会被挡。 +- **刚性目录结构**：规定特定的 code 必须待在特定目录（如 `models/` / `controllers/` / `schemas/`）、AI 不能自己乱建 folder。 +- **自动化 Linter**：如果 AI 写出违规 code（例如在数据层直接调用 API）、CI 自动拒绝 merge、逼 AI 重写。 + +**(b) Enforcing Taste — 把“工程美学”变成规则** + +“品味”听起来主观、但工程上指的是**可维护性、一致性、简洁度**。AI 没有美感、它只会根据概率产出结果——所以要把美感写成 lint 规则： + +- **黄金准则 list**：写下“**偏好 composition 而非 inheritance**”、“**函数必须短小**”、“**文件 < 500 行**”这类原则、变成 invariant。 +- **代码风格统一**：harness 强制 AI 产出的命名、逻辑组织看起来像“同一个高阶工程师写的”、而不是混杂风格的大杂烩。 +- **拒绝“AI slop”**：AI 常生成冗余、无用但“看起来正确”的 code。设定“品味基准”要求 AI 不断重构、简化、直到达到人类专家认可的优雅程度。 + +→ **核心精神**：**定义边界、不细管实作**——让 agent 在你划好的格子里自由冲、而不是每行都要人盯。 + +- **跨 work boundary**：Config + cross-cutting（lint 规则写在 Config、强制检查跨所有层） +- **对应 tool**：`agent-acceptance-gate` YAML preset（`multi-locale-mirror-sync.yml` / `catalog-entry-add.yml` / `fact-check-frontier-models.yml`）——预先 codify“跑出来该长什么样” + +### 5. Throughput Changes Merge Philosophy — agent 高吞吐 → 人类 QA 变瓶颈 + +> "...3.5 PRs per engineer per day... **the bottleneck became human QA capacity**." — OpenAI + +过去 1 个工程师 1 天写 1-2 个 PR、人类逐行 review 跟得上。**agent 上线后变成 1 天 3.5 个 PR**、再加上 self-correcting agent 自己会 retry、实际吞吐还更高。问题不是 agent 不够快、是**人类 review 来不及**——QA 变瓶颈。Merge 逻辑必须换、不能再依赖"人逐行读过"当 quality gate。 + +**(a) Pre-merge automation——把 review 自动化** + +人不再是逐行 reviewer、是 spot-checker： + +- **自动 lint**：CI 跑 linter 强制 style / schema / 命名。agent 违规 → CI 红灯、merge 被挡。 +- **自动 test**：unit test + integration test 全自动跑、coverage 不达标不能 merge。 +- **自动 acceptance gate**：commit 前跑 acceptance gate preset（如 `multi-locale-mirror-sync.yml`）、预先 codify"跑出来该长什么样"、agent 不符就 fail。 + +**(b) Self-verification——agent 自己先验一轮** + +agent 提 PR 前、先自己跑一遍 evaluator-optimizer loop： + +- **agent 内建 critique step**：写完 code 再调用一次 critique agent 自审、发现问题就重写。 +- **LLM-as-judge 自动评分**：用另一个 LLM agent 给 PR 打分、低于门槛就退回 agent 改。 +- **Human spot-check only**：人类只看 agent + LLM-judge 都过关后的"最终样子"、不再逐行读过程。 + +→ **核心精神**：**Quality gate 从"人读过"变成"机器跑过 + 人 spot-check"**——人类角色从"逐行守门"升级为"决定 gate 怎么设"。 + +- **跨 work boundary**：Service（merge workflow） +- **对应 tool**：`agent-acceptance-gate` 整套、特别是 mandatory preset 机制（trigger fire → preset 必跑） + +### 5 原则 × Stage 7 Harness 8 元件对照表 + +下表展示 5 个原则怎么作用到 [Stage 7 Harness Engineering 的 8 个核心元件](07-multi-agent-production.zh-Hans.md#harness-的-8-个核心元件)上（✓ = applies、✓★ = primary lever）： + +| 原则 ＼ Harness 元件 | 1. Agent Loop | 2. Tool Reg | 3. Ctx Mgr | 4. Retry | 5. Sandbox | 6. Obs | 7. Eval | 8. Cost / Lat | +|---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:| +| **1. Legibility** | | ✓ | ✓ | | | ✓ | | | +| **2. SoR** | | | ✓★ | | | ✓ | | | +| **3. Progr. Disc.** | ✓ | | ✓★ | | | | | ✓ | +| **4. Invariants** | | ✓ | | ✓ | ✓ | | ✓★ | | +| **5. Merge Phil.** | | | | | | | ✓★ | ✓ | + +→ **Context Manager（#3）+ Eval（#7）是被 4-5 个原则同时作用的热点**——这也是 v0.2.3 preset / `agent-acceptance-gate` / `agent-shared-memory` 都围着这两个元件设计的原因。 + +→ **Tool Registry（#2）+ Observability（#6）次热**——被 3 个原则影响、Legibility 把 schema 写对 + Invariants 把 lint 写对 + SoR 把记录写对。 + +→ **Retry / Sandbox / Cost-Latency** 只被 1-2 个原则作用——这几个元件相对“机械”、原则上对应 1 个 lever 就够。 + +### 📚 Anthropic ↔ OpenAI cross-vendor 对照 + 推荐读本 + +OpenAI 那 5 原则 Anthropic 大部分也都讨论过、用的词不同。下表是 cross-vendor 对照、每个都附 canonical URL： + +| OpenAI 原则 | Anthropic 对应词 / pattern | 最权威 URL | +|---|---|---| +| **1. Legibility** | ACI（Agent-Computer Interface）+ Tool Documentation | [Building Effective Agents Appendix](https://www.anthropic.com/engineering/building-effective-agents) | +| **2. System of Record** | CLAUDE.md hierarchy + Memory persistence | [Claude Code: How Claude remembers your project](https://code.claude.com/docs/en/memory) + [Multi-Agent Research System](https://www.anthropic.com/engineering/built-multi-agent-research-system) | +| **3. Progressive Disclosure** | **同词**（Anthropic Skills 自己也用“core design principle”描述）| [Equipping Agents for the Real World with Agent Skills](https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills) ⭐⭐⭐ | +| **4. Taste Invariants** | Evaluator-optimizer loops + Tool "poka-yoke"（如强制 absolute filepath）| [Building Effective Agents Evaluator-optimizer](https://www.anthropic.com/engineering/building-effective-agents) | +| **5. Throughput Changes Merge Philosophy** | "Human evaluation catches what automation misses" + LLM-as-judge 并用 | [Multi-Agent Research System Evaluation challenges](https://www.anthropic.com/engineering/built-multi-agent-research-system) | + +**Anthropic 额外强调的 3 个 OpenAI 没重点讲的原则**： + +| 原则 | 白话 | URL | +|---|---|---| +| **Simplicity** | 先用 basic LLM call、不要跳 multi-step agent | [Building Effective Agents Simplicity](https://www.anthropic.com/engineering/building-effective-agents) | +| **Transparency** | "explicitly showing the agent's planning steps"——agent 自己秀 plan | [Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents) | +| **Memory persistence** | context 满前先存外部、subagent 用 fresh context 接力 | [Multi-Agent Research System](https://www.anthropic.com/engineering/built-multi-agent-research-system) | + +#### 推荐阅读顺序（45 + 20 min） + +**先读这 3 篇（总计 ~45 min）**： + +1. [Anthropic — Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents) ⭐⭐⭐ — 涵盖原则 #1 + #4 + Simplicity / Transparency、**最基础、先读** +2. [Anthropic Engineering — Equipping Agents for the Real World with Agent Skills](https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills) ⭐⭐⭐ — 涵盖原则 #3、Anthropic 直接用 "progressive disclosure" 词、3-tier loading 完整解说 +3. [Claude Code — How Claude remembers your project](https://code.claude.com/docs/en/memory) ⭐⭐ — 涵盖原则 #2、CLAUDE.md 4-tier hierarchy + `@-import` + AGENTS.md 互通 + +**再读这 1 篇（~20 min）**： + +4. [Anthropic — How we built our multi-agent research system](https://www.anthropic.com/engineering/built-multi-agent-research-system) — 补 #2 + #5 + Memory persistence production 案例 + +**OpenAI 原始文章**： + +5. [OpenAI — Harness Engineering](https://openai.com/index/harness-engineering/) — Codex 自身 case study、5 原则的源头 + +### 🛠 Coding-agent harness 为什么跟一般 tool-use agent 不同 + +上面 5 原则对所有 agent 通用，但 **coding agent**（Claude Code / Codex / Aider 这类）的 harness 有额外重量级需求，值得单独拆开看——因为 Stage 4 的 CodeAct、Stage 5 的 Claude Code 生态、Stage 8 的 sandbox 全都从这条线长出来。 + +**三个 coding-agent 专属的 harness 元件**： + +| 元件 | 为什么 coding agent 特别需要 | 对照 Stage | +|---|---|---| +| **文件系统 + repo 状态快照** | agent 改 code → 要能 diff / rollback / replay；不像 chat agent 改完就忘 | Stage 5 CLAUDE.md hierarchy、Stage 7 Retry | +| **可隔离的执行 sandbox** | agent 写的 code 要真的跑起来验证（不是只生成）、但不能污染 host | Stage 8 Code Sandbox（e2b / Daytona）| +| **长 horizon 任务分解 + parallel subagent** | 大型重构 / 跨文件修改超过单一 context、要拆成 subtask 并行跑 | Stage 7 multi-agent · **[Opus 4.8 Dynamic Workflows](#-dynamic-workflowsopus-48-当-agent-自己写出-workflow)（research preview）就是这个方向的 product 化——见下方专节** | + +**为什么这条线 2026 特别热**： + +- **OpenAI Agents SDK 2026-04 更新**把 sandbox（7 个 provider）+ harness 抽象层内建——[Stage 4 已标注](04-agent-frameworks.zh-Hans.md)这是"production coding agent 首次 architecturally sound"。harness 不再是各家自己拼、开始有共用抽象。 +- **Aider / Claude Code / Codex 三家对照**：同样是 coding agent，但 harness 取舍不同——Aider 走 git-commit-per-edit 的极简 repo 状态、Claude Code 走 CLAUDE.md + plan mode + subagent、Codex 走 cloud sandbox + harness 抽象。**读者要学的是"harness 取舍怎么影响 agent 行为"、不是记哪家 API**。 + +> 📚 **想深入 coding-agent harness 设计**：先读 [OpenAI — Harness Engineering](https://openai.com/index/harness-engineering/)（Codex case study）+ [Anthropic — Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents) 的 tool design 段；想看实作对照就跑 [Stage 5 Claude Code 生态](05-claude-code-ecosystem.zh-Hans.md) + [Stage 8 Code Sandbox](08-agent-interfaces.zh-Hans.md)。 + +### ⚖️ Eval rigor — harness 设计怎么悄悄 bias 你的 benchmark 数字 + +Stage 7 的 Benchmark Landscape 提过 Berkeley 的 reward-hacking 警告。这里补一个更基础、更常被忽略的问题：**agent 的分数有一大半是 harness 给的、不是 model 给的**。 + +**核心事实（多 source 证实）**： + +- **同一个 model、不同 scaffold，分数可以差一倍**——研究显示一个 model 在"精致 agent scaffold"下拿 60%、换成 unassisted 可能掉到 30%。**scaffold（= harness）跟 model 一样是 benchmark 的变因**。([SWE-bench benchmark hygiene 分析](https://www.whocodesbest.com/news/2026/swe-bench-april-2026-benchmark-hygiene-matters)) +- **单次 run 不可信**：分数在多次 run 之间若差 > 10%、signal-to-noise 太低、不能从单次结论。SWE-bench 用 per-issue Docker image 锁 repo 快照 + pinned deps 来 exact replay；τ-bench 用 **pass^k**（k 次都过才算）量 reliability。 +- **benchmark 本身常有 reward-design bug**：[Establishing Best Practices for Building Rigorous Agentic Benchmarks](https://arxiv.org/abs/2507.02825) 盘点出很多 agentic benchmark 的 task setup / reward 有洞——例如 τ-bench 早期版本把"空响应"算成正确。 +- **reward hacking 不是边缘案例**：在开放任务上，某研究测得 top model 利用 rubric 漏洞的比率：agentic code generation 达 **75%**、creative task 达 67%（数字依任务与评测设计而异、见下方 source）。 + +**对读者的 takeaway（harness engineer 视角）**： + +| 你在做什么 | harness 该怎么设计 | +|---|---| +| 比较两个 model | **固定 scaffold**——否则你比的是 scaffold 不是 model | +| 报告 agent 分数 | 报 pass^k（k≥3）或多 run 平均 + 变异、不要报单次 best | +| 自己写 eval | 先假设"agent 会 reward hack"——held-out test + LLM judge + 改文件侦测三管齐下 | +| 信一个 benchmark 排名 | 先查它的 reward design 有没有被审计过（empty-response / shortcut 漏洞）| + +> 📚 **想深入 eval rigor**：[Establishing Best Practices for Building Rigorous Agentic Benchmarks](https://arxiv.org/abs/2507.02825) 是系统性的盘点；reward hacking 的 production 警告见 [Stage 7 Benchmark Landscape](07-multi-agent-production.zh-Hans.md) 的 Berkeley 段 + τ-bench（[sierra-research/tau2-bench](https://github.com/sierra-research/tau2-bench)）的 pass^k 设计。 + +### 🔀 Dynamic Workflows（Opus 4.8）— 当 agent 自己写出 workflow + +上面 coding-agent harness 提到 Opus 4.8 的 **Dynamic Workflows**（2026-05-28、Claude Code research preview）。它值得单独拆开——因为它把 [Stage 4 教的 workflow vs agent 区分](04-agent-frameworks.zh-Hans.md#两个维度先分清楚workflow-vs-agent--single-vs-multi)整个塌缩掉，是理解"agent-authored orchestration"最好的活教材。 + +> 📌 **Claude Fable 5（2026-06-09 发布；2026-06-12 暂停、2026-07-01 恢复）**：Anthropic 的 Mythos-class **Claude Fable 5**（`claude-fable-5`、定位在 Opus 之上）是广泛可用的最高能力成员，与 limited-availability 的 Claude Mythos 5（`claude-mythos-5`）同日发布。2026-06-12 曾被美国出口管制暂停，但出口管制 2026-06-30 解除、[Fable 5 于 2026-07-01 全球恢复](https://www.anthropic.com/news/redeploying-fable-5)（加了新安全 classifier）。Fable 5 又是最高阶的 Claude 层级；Opus 4.8 为 Opus-class 旗舰，**Dynamic Workflows 仍是 Opus 4.8 的功能**，不归属于 Fable 5。 + +**名字其实是个矛盾**：照 Anthropic 自己 [Building Effective Agents](https://www.anthropic.com/research/building-effective-agents) 的定义，**workflow = 人预先写死的 code path、agent = LLM runtime 自主决定**。但 Dynamic Workflows 是——**agent（Claude）在 runtime 自己决定任务怎么拆、然后产出一段 JavaScript orchestration script、丢给一个独立的背景 runtime 执行**。人没有写那个 workflow、是 agent 写的。它同时是 agent（谁决定）也是 workflow（怎么跑）。 + +**核心机制 — 为什么不只是"并行 subagent"**： + +关键不在并行、在 **context offloading**。一般 subagent / skill：每个中间结果都回到 Claude 的 context window。Dynamic Workflows：loop、分支、中间结果**全留在 script 变数里**——只有最终 verified 答案回到 context。这是它能跑"几十万行 codebase 迁移、以既有测试套件为标准、from kickoff to merge"的原因——因为中间几百次 agent 调用的杂讯不会塞爆 context。 + +| 面向 | 事实（official Claude Code docs 证实）| +|---|---| +| 它就是什么 pattern | [Anthropic 五个 workflow pattern](https://www.anthropic.com/research/building-effective-agents) 里的 **orchestrator-workers**（唯一"runtime 才决定 subtask"那个）放大版 + adversarial 验证 | +| 规模上限 | **同时最多 16 个 agent**、单次 run **累计上限 1,000 个**（防 runaway）。不是"几百个同时跑" | +| 怎么触发 | (1) prompt 里含 `workflow` 字 (2) saved command 如 `/deep-research` (3) `/effort ultracode`（xhigh reasoning + 自动 orchestration）| +| 平台 | Claude Code feature（CLI / Desktop / IDE / headless / Agent SDK、v2.1.154+）。**不是** raw API（没有 `/v1/workflows`）| +| 质量机制 | adversarial propose / refute / converge loop——独立 agent 各攻一角、其他 agent 试图反驳、迭代到收敛、merge 前验证 | + +**这一节就是用这个 pattern 研究出来的**（活范例）：本节事实是跑一个 dynamic-workflow-style 的 orchestration 搜集的——4 个并行 research agent 各攻一个角度（官方 / 技术 / 定位 / skeptic）+ 一个 skeptic synthesizer 把**无法跨 source corroborate 的宣称剔掉**。被剔掉的包括"hundreds of parallel subagents"（实为 16 concurrent / 1,000 total）、"750k 行 Bun 从 Zig 迁移到 Rust、过 99.8% 测试、11 天 merge"（vendor 单一 case、未独立审计、不能当 verified fact）。**这就是 orchestrator-workers + adversarial verify 的实际运作**。 + +> ⚠️ **什么不是（诚实限制，跟功能本身一样重要）**： +> - **不是通用 workflow 引擎**（不是 Airflow / n8n / Temporal——人不画 DAG、是 agent 写 code） +> - **不是无限并行**（16 concurrent / 1,000 total per run、不是"几百个同时"） +> - **不是 GA**（research preview、paid-plan-gated、定价 / 可用性可能变；官方两个来源对 Pro 是否含括还不一致） +> - **reliability 提升主要来自"不确定时 abstain"**（refusal trade-off——答对更少不确定题、不是答对更多；来自 system card）。更多 subagent ≠ 更高正确覆盖率 +> - **不是免费**（Anthropic 自己警告 token 用量"substantially more"、建议先用 scoped task 抓 consumption） +> - **不解决 dispatch problem**（"何时该 fan out vs 一次仔细做"还是靠 Claude runtime 判断；独立分析者指出 verifier 纪律仍薄——fan-out 没验证好可能产出"五十个看似合理的 bug"、比一次仔细做更糟） + +> 📚 **权威来源**：[Claude Code — Dynamic Workflows docs](https://code.claude.com/docs/en/workflows)（机制 + 16/1000 上限 + 触发方式的 canonical）· [Anthropic — Introducing Dynamic Workflows](https://claude.com/blog/introducing-dynamic-workflows-in-claude-code)（定位 + token 警告）· [Anthropic — Building Effective Agents](https://www.anthropic.com/research/building-effective-agents)（workflow vs agent + orchestrator-workers 根基）· [Opus 4.8 announcement](https://www.anthropic.com/news/claude-opus-4-8)。 + +### ⏳ Harness 会过期：Model-Harness-Fit 与 Bitter Lesson + +前面教了很多 harness 怎么设计。这里补一个容易被忽略、但很重要的提醒：**你今天搭的 harness，是为“今天这个 model”量身搭的；model 一变强，有些 harness 就过期了。** + +- **Model-Harness-Fit**：一套 harness 是跟当下 model 的能力“配对”出来的。你为了补 model 的弱点（不会规划、容易忘、不会自己验收）搭了一堆 scaffold（外围辅助结构）；等下一代 model 自己就会做这些事，那些 scaffold 反而变成多余的包袱。 +- **Bitter Lesson（苦涩的教训）**：AI 历史一再证明，靠“人工塞进去的巧思”短期会赢，长期几乎都输给“让模型用更多运算自己学”。套到 harness 上就是：过度搭建（over-scaffolding）是在跟这个趋势对赌。 + +**白话的取舍**：能用“最薄、刚好能出货”的 harness，就不要搭一座城堡。每加一层 scaffold，先问一句：“这是在补 model 真正的弱点，还是只是我不放心？”——下一代 model 出来时，前者会留下，后者会变成你要回头拆的技术债。 + +> 📚 概念根源：Rich Sutton, [The Bitter Lesson](http://www.incompleteideas.net/IncIdeas/BitterLesson.html)（2019）；呼应本 stage“同一 model、不同 scaffold，分数差一倍”——scaffold 影响这么大，它的“保存期限”就值得在设计时放心上。 + +### 🧑‍🔧 分工：你出题 + 验收，agent 动手做 + +Anthropic 2026-06 的报告用真实使用数据归成一句大白话：**你决定“要做什么”，agent 决定“怎么做”。** + +想象你**找人装潢房子**：你说“我要一间北欧风书房、预算多少、几号要好”，师傅去挑材料、动手施工；做完你不满意就退回重做。你不会自己去钉每一块木板——但“要什么”和“验收”始终是你的事。 + +报告的数字刚好也是这样：平常一次工作里，**“要做什么”大约七成是你在决定，“怎么做”大约八成丢给 agent**。 + +记三件事就好： +- **线划清楚，agent 才帮得上忙**：你把力气花在“讲清楚要什么 + 检查成果”，动手的部分交给 agent；要求讲得含糊，它就容易做歪。这正是本 stage 一直在说的“工作边界”。 +- **越内行，agent 帮你放大越多**：因为你更会出题、更会验收。所以它是**放大**内行人，不是取代内行人。 +- **一群 agent 也要分工**：等你同时用好几个 agent（Stage 7）时，会多一个“总指挥”把工作切给各管一块的小 agent——分工就从“人 vs agent”延伸到“agent 之间”。 + +> 📚 来源：[Anthropic — Agentic coding and persistent returns to expertise](https://www.anthropic.com/research/claude-code-expertise)（2026-06-16、用真实 Claude Code 使用数据量化“人 vs agent 分工”）· [2026 Agentic Coding Trends Report](https://resources.anthropic.com/2026-agentic-coding-trends-report)（开发从写 code 转向 orchestrating agent teams）。 + +### 📋 概念验证 prompt（self-quiz） + +> 🛠️ **想直接动手写 SKILL.md / CLAUDE.md？** 4 个实作用 prompt（audit 既有 / 生成新的）已经**移到 [Stage 5](05-claude-code-ecosystem.zh-Hans.md)**，那边才是读者真正开始动手写的地方： +> - [Stage 5.1 CLAUDE.md 设计 prompts](05-claude-code-ecosystem.zh-Hans.md#-claudemd-设计-prompts依-5-原则) +> - [Stage 5.3 SKILL.md 设计 prompts](05-claude-code-ecosystem.zh-Hans.md#-skillmd-设计-prompts含-skill-creator-替代) + +本节只保留 **1 个 quiz prompt，帮你确认自己是否真的理解这 5 条原则**，属于不写 code 的概念检查。 + +#### Prompt 1 — Self-quiz（考自己懂不懂这 5 条原则） + +``` +我刚学完 OpenAI 5 个 harness engineering 原则： +1. Legibility +2. System of Record +3. Progressive Disclosure +4. Taste Invariants +5. Throughput Changes Merge Philosophy + +请出 5 道情境题、每题描述一个真实的 SKILL.md / CLAUDE.md 设计决定（例如“我把所有 examples 都塞 SKILL.md 主档、< 1000 行”），问**违反哪一条原则 + 该怎么修**。 + +一次出 1 题、等我回答后给反馈、再出下一题。最后给总成绩。 +``` + +→ **建议用法**：学完上面 5 条原则后跑这个 quiz，确认你真的抓到 concept；真正写 SKILL.md / CLAUDE.md 的 prompt，请回到 [Stage 5](05-claude-code-ecosystem.zh-Hans.md)。 + +### 📐 进阶 agentic 概念应用流程（读者导引） + +学完前面 5 条原则 + Anthropic 对照后，**怎么把这些概念真的用到 agent 设计上？** 从 Stage 7（你已经能做 production agent）往下、5 个步骤到 production： + +1. **建立概念地图主轴 — 四层工作边界**：Types → Config → Repo → Service。想清楚 agent 能动到 stack 哪一层、跨层算违规。 + → 本 stage §概念地图主轴：四层工作边界 + +2. **挑 2-3 个相关进阶概念**：从 12 个 skeleton 找跟你问题最相关的（Work Boundary / Contract / PAR / Autonomy 等）。 + → 本 stage §12 个进阶概念（pattern 清单） + +3. **套 5 个 OpenAI 原则（cross-cutting）**：Legibility / SoR / Progressive Disclosure / Taste Invariants / Throughput Merge Philosophy。这 5 条原则跨越全部 12 个概念、决定“做得对不对”。 + → 本 stage §跨概念 Harness Engineering 原则 + +4. **Encode 到 Skills + CLAUDE.md**：用 Stage 5 的 4 个 prompt 动手写——CLAUDE.md audit / generate（[Stage 5.1](05-claude-code-ecosystem.zh-Hans.md#51--claude-code-基础)）+ SKILL.md audit / generate（[Stage 5.3](05-claude-code-ecosystem.zh-Hans.md#53--skillsclaude-code-的行为层-claude-code-生态最关键的一层)）。 + +5. **Verify with acceptance gate**：Preset YAML 抓 drift / LLM-as-judge 自动评 / Human spot-check 补 edge case。 + → [agent-collab-skills](https://github.com/WenyuChiou/agent-collab-skills) + +→ **Production agent ready**：可稳定给人用、有自动验证、有可预测失败。 + +→ **怎么用这 5 步**：第一次读本 stage 时、按 1 → 5 顺序走；之后实作 agent 卡住时、再回来判断自己卡在了哪一步。 + +→ **它和前面 Why → What → How 表的区别**：那张是“**痛点 ↔ 原则 ↔ 工具**”的横向对照、适合当 reference；这 5 步是“**从学完到上线**”的纵向流程、适合照着一步一步做。 + +## 📖 完整 reading path（按 depth 分层） + +按深度排序，不必全读。Foundation tier 必读（总计约 95 分钟），其他内容则是**遇到真实问题时再深挖**： + +### 🌳 Reading decision tree（按你卡的问题选读） + +![Agentic AI Advanced Reading Decision Tree](../resources/diagrams/reading-decision-tree.zh-Hans.png) + +这不是普通 reading list、而是 **decision tree**。你现在卡在哪个问题、就先读对应的那 1-2 篇（上图 5 个分支对应 5 种卡关状况；下面是每个分支读完第 1 篇后的进阶推荐）： + +**分支读完后的延伸阅读**（branch-specific second readings）： + +- “不知道 agent 怎么开始”→ 再读 ReAct paper + Lilian Weng "LLM Powered Autonomous Agents" +- “multi-agent 要不要用、怎么开”→ 再读 Anthropic Multi-Agent Research（case study section） +- “context 没效率”→ 再读 Anthropic Multi-Agent Research（memory section） +- “eval 怎么写 / 自动验证”→ 再读 Anthropic Multi-Agent Research（eval section） +- “想跟上 frontier 现况”→ 再读 AutoGen + ReAct paper + +→ **规则**：每个分支**最多挑 2 篇深读**。读完再回来看自己还有什么问题，再决定下一篇，不要先泛读整个清单。 + +**Foundation tier**（先读这 4 个，总计约 95 分钟）： +- [Anthropic — Building Effective Agents](https://www.anthropic.com/engineering/building-effective-agents) +- [Cognition — Don't Build Multi-Agents](https://cognition.ai/blog/dont-build-multi-agents) +- [Anthropic — How we built our multi-agent research system](https://www.anthropic.com/engineering/built-multi-agent-research-system) +- [Lilian Weng — LLM Powered Autonomous Agents](https://lilianweng.github.io/posts/2023-06-23-agent/) + +**Workflow patterns tier**： +- [LangGraph Planning Agents Tutorial](https://blog.langchain.com/planning-agents/) +- [Microsoft AutoGen docs](https://microsoft.github.io/autogen/) +- [DSPy](https://dspy.ai/learn/) + +**Production / Harness tier**： +- [OpenAI Harness Engineering (2026-02)](https://openai.com/index/harness-engineering) +- [Hamel Husain Evals blog](https://hamel.dev/blog/posts/evals/) +- [Simon Willison coding agents notes](https://simonwillison.net/tags/coding-agents/) + +**Frontier research papers**（挑 3-5 篇深读）： +- ReAct / Reflexion / CoALA / Self-Discover / Voyager / Constitutional AI / AutoGen + +**中文 / hands-on**： +- [李宏毅 GenAI 2024 / 2025](https://speech.ee.ntu.edu.tw/~hylee/) +- [datawhalechina/hello-agents](https://github.com/datawhalechina/hello-agents) + +> 📋 **学完进阶概念回头看 synthesis** → [Stage 5 §🗺️ 7-Layer Architecture Map](05-claude-code-ecosystem.zh-Hans.md#-7-layer-architecture-map先看这张图再读-51-57)（Claude Code 7 个 primitive + 3 个工程学 discipline 整合图） + +## ✅ 自我检查 + +读完本 stage 后，你应该能： + +- [ ] 用 **Types → Config → Repo → Service** 四层解释 Cognition Flappy Bird / Anthropic speculative-leap 为什么算 work-boundary violation +- [ ] 说出 12 个进阶概念里 5 个的“动到哪一层” + 一句话定义 +- [ ] 讲得出 4 大类别核心原则（① Context 管理 / ② Interface / ③ Quality 验证 / ④ Process 纪律）各自解决什么问题，以及类别之间的 enabling 关系 +- [ ] 知道下一步该翻哪篇 paper / blog，而不是先把全部都读完 +- [ ] 区分 PAR loop（单 agent 自我修正）和 agent-debate（两个 agent 对峙） +- [ ] 在 brief 里明确写出 task 的 work boundary（什么 in-scope / out-of-scope） + +→ 如果这些都能做到，你已经超出 Stage 7 Production 化的范围，进入 frontier agentic workflow design。**剩下的就是挑一篇最贴近你当前痛点的 paper，深读它。** + +--- + +→ 接下来 [**Stage 8 — Agent Interfaces**](08-agent-interfaces.md)（**两 track 共用 hub**）：学 agent 怎么跟非 API 世界互动（Computer Use / Browser Use / Code Sandbox）。或挑一个[特化分支](../README.zh-Hans.md#-学习地图两条学习路径)、或回头来贡献这份 repo。 diff --git a/stages/08-agent-interfaces.en.md b/stages/08-agent-interfaces.en.md new file mode 100644 index 0000000..e8aa49f --- /dev/null +++ b/stages/08-agent-interfaces.en.md @@ -0,0 +1,529 @@ +# Stage 8 — Agent Interfaces: Computer Use, Browser Use, & Code Sandbox + +> [繁體中文](./08-agent-interfaces.md) | [简体中文](./08-agent-interfaces.zh-Hans.md) | **English** + +⏱ **Estimated Time**: 2-3 weeks (approx. 12-20 hours) + +> 💡 **High-Density Terminology**: This chapter is packed with terms like Computer Use, DOM, microVM, Firecracker, Sandbox, and Cold Start. We'll explain them as we go. If you're unfamiliar with them, it's a good idea to first review the glossaries in Stage 1 and Stage 7. + +> 📋 **Chapter Outline**: [What are Agent Interfaces? (Positioning)] → [Learning Objectives] → [Prerequisites] → [Required Reading] → [🖱 Computer Use (Screen-level)] → [🌐 Browser Use (Web-level)] → [📦 Code Sandbox (Isolated Execution, with Mini-Glossary)] → [How Track A Uses It] → [How Track B Builds It] → [⚠ 2026 Safety & Security] → [Hands-on Exercises] → [Recommended Tools] → [Featured Projects] → [Self-Check] → [The Next Frontier (Voice/VLA Forward Note)] + +> 🔑 **Key Terms**: See explanations within this chapter and in the main [`resources/glossary.md`](../resources/glossary.md). + +**👥 Shared Hub**: Like Stage 5 (The Claude Code Ecosystem), this chapter serves as a hub for both Track A (CLI Power User) and Track B (Agent Builder). Stages 5 and 8 are the two central hubs of this curriculum. + +## 🎯 What are Agent Interfaces? (Positioning) + +**Agent Interfaces refers to how an agent operates the real world beyond APIs—the computer screen, the web, or an isolated code execution sandbox—the agent's IO layer to the non-API world.** Stages 0-7 taught you **how to build the agent itself** (LLM → prompt → tool → context → memory → multi-agent → harness). This stage teaches you **how the agent, once built, operates in a real environment.** + +**The 3 Layers of Interfaces**: + +| Interface | Target of Operation | How it Works | Representative Tools | +|---|---|---|---| +| **🖱 Computer Use** (screen-level)| Any desktop app (Excel, SAP, Photoshop, software without APIs) | Screenshot → Vision model analyzes → Calculate coordinates → Simulate keyboard/mouse | Anthropic Claude Computer Use / OpenAI Codex desktop / Gemini in Chrome | +| **🌐 Browser Use** (web-level) | Any webpage | DOM-aware navigation + Vision fallback when necessary | Atlas / Comet / browser-use (OSS, 86k stars) | +| **📦 Code Sandbox** (isolated exec)| Agent-generated code running in an isolated environment | microVM / Container / Userspace kernel | E2B / Daytona / Modal / Vercel Sandbox / OpenAI Agents SDK (built-in as of April 2026) | + +### How This Stage Differs from Previous Ones (Avoiding Conceptual Confusion) + +**A reader's first question might be**: How is this different from Stage 3's Tool Use, Stage 5's MCP, or Stage 7's Harnesses? + +| Comparison | What That Stage Covers | What This Stage Covers | +|---|---|---| +| **Stage 3 Tool Use** | How an agent **calls APIs** (function calling, JSON schema) | How an agent **operates an environment** (software without APIs, real webpages, running code) | +| **Stage 5 MCP** | How tools/data sources are **exposed to the agent in a standardized way** | How an agent **actually interacts** with an environment (MCP is the protocol, Interface is the behavior) | +| **Stage 7 Harness** | The agent's **runtime control flow** (loops, retries, safety) | The agent's **I/O boundary** (interaction with the outside world, invisible to the runtime) | + +→ **The core distinction**: A **Tool** is an **API call**; an **Interface** is an **environment operation**. The former deals with abstract APIs, the latter directly with a real GUI, web, or OS. + +### Why 2024-2026 is the Breakthrough Era for Agent Interfaces + +**Why are we covering this now?**: + +- **Before Oct 2024**: Agents could only interact with the API-driven world (calling OpenAI/GitHub/Slack APIs, returning text). +- **Oct 2024**: Anthropic's Computer Use beta is released → **Agents can operate a real screen for the first time.** +- **2025-2026**: OpenAI (Atlas + Codex desktop) and Google (Gemini in Chrome) enter the field → Mainstream adoption. +- **May 2026**: The OSWorld benchmark reaches **76.26%** (superhuman, vs. a 72.36% human baseline) → It transitions from a research curiosity to a production reality. + +**The curriculum gap without this stage**: After completing Stage 7, you might think you're done. In reality, your agent can only talk to APIs. **It can't operate software without APIs, interact with real webpages, or run code.** You also wouldn't have been warned about safety issues like the Comet injection or the Amazon injunction (see [Safety](#-2026-safety--security-highlights)). + +### Why is this a Shared Hub? + +Like Stage 5 (The Claude Code Ecosystem), this stage is a **hub**, not track-specific: + +- **Track A (CLI Power User)**: Uses Claude Computer Use to delegate desktop tasks, uses Codex background mode, and connects to browser MCPs in Claude Code. +- **Track B (Agent Builder)**: Embeds `browser-use` into their own agents, uses E2B/Daytona to run agent-generated code, and uses the built-in sandbox in the OpenAI Agents SDK. + +**Both tracks need these 3 interface layers**—which is why this chapter is positioned as a hub. + +## 📌 Learning Objectives + +After completing this stage, you will be able to: + +- Distinguish between the 3 layers of agent interfaces (Computer Use, Browser Use, Sandbox) and their relationship to Tools, MCP, and Harnesses. +- Explain the mental models for Computer Use and Browser Use (screenshot → vision → coords vs. DOM-aware). +- Define isolation technology terms like microVM, Container, Firecracker, gVisor, and Cold Start. +- Recall the May 2026 SOTA numbers for OSWorld/WebArena and interpret the warnings about reward-hacking. +- **For Track A**: Integrate Computer Use, browser MCPs, and Codex background mode into your daily CLI workflow. +- **For Track B**: Use `browser-use` and E2B to embed environmental interactions and sandbox isolation into your own agents. +- Design with 4 safety patterns (approval gate, sandbox, human-in-the-loop, output filter) to prevent injection attacks. + +## 🚪 Entry Conditions + +You should have already: + +- Completed [Stage 5](05-claude-code-ecosystem.md) (understand MCP/Skills/Plugins, use Claude Code daily). +- Completed [Stage 7](07-multi-agent-production.md) (understand harness engineering, know what the reward-hacking warning is about). +- Have a basic familiarity with Docker/VM concepts (this chapter explains the difference between microVMs and Containers, but you'll struggle if you've never touched Docker). +- **For Track A only**: Completing Stage 5 is sufficient; Stage 7 is optional. The Track A portion of this chapter does not depend on building experience. +- **For Track B**: Stage 7 is mandatory, otherwise you will get stuck on the build examples in 9. + +If you don't meet these, go back and catch up. + +## 📚 Required Reading + +1. [**Anthropic — Introducing Computer Use**](https://www.anthropic.com/news/3-5-models-and-computer-use) — The original launch announcement for Computer Use. A must-read to understand how it works. +2. [**Anthropic — Claude Opus 4.8 Release Notes**](https://docs.anthropic.com/en/release-notes/overview) — Opus 4.8 (May 2026) introduces Dynamic Workflows + parallel subagent harness and is the Opus-class flagship. On 2026-06-09 Anthropic released Claude Fable 5 (`claude-fable-5`) and Claude Mythos 5 (`claude-mythos-5`), the Mythos-class tier above the Opus class. They were suspended 2026-06-12 by a US export-control directive, but the controls were lifted 2026-06-30 and [Fable 5 was redeployed globally 2026-07-01](https://www.anthropic.com/news/redeploying-fable-5) (Mythos 5 restored only for approved US organizations) — Fable 5 is again the highest Claude tier. +3. [**OpenAI — The next evolution of the Agents SDK**](https://openai.com/index/the-next-evolution-of-the-agents-sdk/) ⭐ **April 2026** — A milestone for architecturally sound production coding agents, with a built-in sandbox and harness abstractions. +4. [**OpenAI — Computer-Using Agent (CUA)**](https://openai.com/index/computer-using-agent/) — OpenAI's version of Computer Use, with WebArena/OSWorld numbers. +5. [**browser-use docs**](https://docs.browser-use.com/) — The #1 open-source web agent (86k+ stars), get started with 5 lines of Python. +6. [**Microsoft OmniParser**](https://microsoft.github.io/OmniParser/) — An open-source GUI parsing tool and an important building block for Computer Use. + +> 💡 **Read selectively**: Track A students should read 1 & 2. Track B students must read 3, 5, & 6. Read everything if you want the full picture. + +## 🖱 Computer Use — The Screen-Level Agent + +### Mental Model: The Workflow and Why + +**The Workflow**: +``` +Agent receives a task + ↓ +1. Take a screenshot → See the current screen + ↓ +2. Vision model analyzes → Identify buttons, text boxes, icons + ↓ +3. Calculate coordinates → "The button is at (453, 218)" + ↓ +4. Simulate keyboard/mouse → click(453, 218) / type("hello") + ↓ +5. Take another screenshot → Check the result, decide the next step +``` + +**Why this paradigm (vs. Tool Use)?**: +- Most software **has no API, only a GUI**—SAP, Excel, Photoshop, any traditional desktop app. The only way for an agent to use them is at the screen level. +- API integration (Stage 3 Tool Use) requires waiting for vendors to provide an interface, which doesn't always happen. +- The screen-level is the **final mile**—"an agent can do anything a human can do on a computer." + +**Why this only became feasible in 2026**: +- **Advances in Vision Models**: Claude 4.x and GPT-5.x are fully multimodal, dramatically improving the accuracy of identifying screen elements. +- **OS-level Training Data**: The [OSWorld dataset (NeurIPS 2024)](https://github.com/xlang-ai/OSWorld) released 369 real-world tasks across multiple OSes, giving frontier labs the data they needed for training. +- **Anthropic's Computer Use beta (Oct 2024) kicked off a commercial race**—OpenAI and Google followed, and benchmarks soared. + +### 2026 Frontier: A 4-Way Comparison + +| Vendor | Product | 2026 Status | OSWorld | Strengths | +|---|---|---|---|---| +| **Anthropic** | [Opus 4.8 / Sonnet 5 Computer Use](https://www.anthropic.com/news/3-5-models-and-computer-use) | GA, cross-platform on macOS/Linux/Windows (Docker) | **72.7%** (Opus 4.6 baseline, near human-level 72.36%; Computer Use numbers for Opus 4.7 / 4.8 not yet public) | Reasoning + code agent, home turf for Stages 5/7. Opus 4.8 is the Opus-class flagship; the Mythos-class Fable 5 (2026-06-09) was suspended 2026-06-12 and restored 2026-07-01 | +| **OpenAI** | [Codex desktop](https://openai.com/index/codex-for-almost-everything/) (April 2026)| GA, **background mode** doesn't hog the cursor, in-app browser, 90+ plugins | CUA 38.1% | Merged with ChatGPT + Atlas to become a **Desktop Superapp** | +| **OpenAI** | [Computer-Using Agent (CUA)](https://openai.com/index/computer-using-agent/) | API | 38.1% / WebArena 58.1% | API-first, can be integrated into your own stack | +| **Google** | [Gemini in Chrome](https://gemini.google/overview/gemini-in-chrome/) (Gemini 3) | GA + Android | — | **Auto Browse** + **Chrome Skills**, Chrome Enterprise Premium $6/user/month | +| **OpenAI Operator**| (Discontinued Aug 2025) | ❌ Unavailable | — | Unstable handling of CAPTCHA, JS, and sessions; replaced by Atlas | + +→ For the latest details, see [Agentic Browser Landscape 2026](https://nohacks.co/blog/agentic-browser-landscape-2026) and the [OSWorld leaderboard](https://os-world.github.io/). + +### Why the OSWorld Numbers Vary So Much (Understanding Benchmark Discipline) + +**Current Status**: + +| Model | OSWorld | Distance from Human Baseline | +|---|---|---| +| Human baseline | **72.36%** | — | +| Claude Opus 4.6 (Anthropic)| **72.7%** | On par | +| May 2026 SOTA (Strongest Model)| **76.26%** | **Superhuman** | +| OpenAI CUA | 38.1% | -34% | +| Most other models | 30-50% | -22% to -42% | + +**Why it's harder than SWE-bench**: +- **More open-ended tasks**: SWE-bench has clear tests to determine pass/fail; OSWorld tasks have vague specs (e.g., "help me turn this csv into a chart"). +- **Cross-OS**: Covers Ubuntu, Windows, and macOS. +- **Cross-application chains**: Often requires opening 3-4 apps (e.g., Excel → Chrome → Slack). + +**Why real ability ≠ the numbers** (echoing the [reward-hacking warning in Stage 7](07-multi-agent-production.en.md#-agent-benchmark-landscape-how-to-read-it-not-just-the-leaderboard---reward-hacking-warning)): +- OSWorld was also on the list in the [UC Berkeley April 2026 reward-hacking report](https://rdi.berkeley.edu/blog/trustworthy-benchmarks-cont/), which proved it could be hacked to 100%. +- **Discipline when looking at numbers**: Don't just look at the top of the leaderboard. The ground truth is the hold-out test for your own use case. + +### Platform Support (as of May 2026) + +| OS | Anthropic | OpenAI | Google | +|---|---|---|---| +| **macOS** | ✅ GA | ✅ Atlas + Codex desktop GA | Inside Chrome | +| **Linux** | ✅ Docker | ⚠ More restricted | Inside Chrome | +| **Windows** | ✅ Docker | 🔜 Native preview / Atlas for Win coming | Inside Chrome | +| **Mobile** | — | — | ✅ Gemini in Chrome on Android | + +## 🌐 Browser Use — The Web-Level Agent + +### Mental Model: DOM-aware vs. Screen-pixel + Why + +**The Core Distinction**: + +| Approach | How it Works | When to Use | +|---|---|---| +| **DOM-aware** (in-browser, has a DOM)| Directly queries `