chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,18 @@
|
||||
# Agent Reach Configuration
|
||||
# Copy to .env and fill in your values
|
||||
# Or use: agent-reach configure
|
||||
|
||||
# Exa Search (free 1000/month) — https://exa.ai
|
||||
# EXA_API_KEY=exa-your-key-here
|
||||
|
||||
# GitHub Token (optional, for higher rate limits) — https://github.com/settings/tokens
|
||||
# GITHUB_TOKEN=ghp_your_token_here
|
||||
|
||||
# Reddit ISP Proxy (optional, for full Reddit access)
|
||||
# REDDIT_PROXY=http://user:pass@ip:port
|
||||
|
||||
# Groq Whisper (optional, for video transcription) — https://console.groq.com
|
||||
# GROQ_API_KEY=gsk_your_key_here
|
||||
|
||||
# OpenAI Whisper (optional fallback when Groq is rate-limited) — https://platform.openai.com
|
||||
# OPENAI_API_KEY=sk-your_key_here
|
||||
@@ -0,0 +1,70 @@
|
||||
name: ci
|
||||
|
||||
on:
|
||||
push:
|
||||
pull_request:
|
||||
|
||||
jobs:
|
||||
test:
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
python-version: ["3.10", "3.11", "3.12", "3.13"]
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
|
||||
- name: Install package and test deps
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install -c constraints.txt -e .[dev]
|
||||
|
||||
- name: Run tests
|
||||
run: |
|
||||
pytest -q
|
||||
|
||||
# Editable installs (-e) never exercise wheel packaging, so a broken wheel
|
||||
# can pass tests and still fail every real `pip install` from source.
|
||||
# This job builds the actual wheel and installs it into a clean venv.
|
||||
wheel-gate:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Build wheel
|
||||
run: |
|
||||
python -m pip install --upgrade pip build
|
||||
python -m build
|
||||
|
||||
- name: Verify wheel has no duplicate entries and ships data files
|
||||
run: |
|
||||
python - <<'PY'
|
||||
import glob, zipfile, collections
|
||||
whl = glob.glob("dist/*.whl")[0]
|
||||
names = zipfile.ZipFile(whl).namelist()
|
||||
dupes = [n for n, c in collections.Counter(names).items() if c > 1]
|
||||
assert not dupes, f"duplicate entries in wheel: {dupes}"
|
||||
assert "agent_reach/skill/SKILL.md" in names, "SKILL.md missing from wheel"
|
||||
for prefix in ("agent_reach/guides/", "agent_reach/scripts/", "agent_reach/skill/references/"):
|
||||
assert any(n.startswith(prefix) for n in names), f"{prefix} missing from wheel"
|
||||
print(f"wheel OK: {len(names)} entries, no duplicates, data files present")
|
||||
PY
|
||||
|
||||
- name: Smoke-install wheel into clean venv
|
||||
run: |
|
||||
python -m venv /tmp/smoke
|
||||
/tmp/smoke/bin/pip install --quiet dist/*.whl
|
||||
/tmp/smoke/bin/agent-reach version
|
||||
cd /tmp && /tmp/smoke/bin/python -c "import agent_reach; from importlib.resources import files; assert (files('agent_reach')/'skill'/'SKILL.md').is_file(); print('SKILL.md ships in site-packages OK')"
|
||||
+14
@@ -0,0 +1,14 @@
|
||||
__pycache__/
|
||||
*.pyc
|
||||
*.pyo
|
||||
.pytest_cache/
|
||||
*.egg-info/
|
||||
dist/
|
||||
build/
|
||||
.env
|
||||
.agent-reach/
|
||||
*.log
|
||||
|
||||
# Claude Code personal permission settings — local only, never commit
|
||||
.claude/settings.local.json
|
||||
uv.lock
|
||||
@@ -0,0 +1,97 @@
|
||||
# Changelog / 更新日志
|
||||
|
||||
All notable changes to this project will be documented in this file.
|
||||
|
||||
本项目的所有重要变更都会记录在此文件中。
|
||||
|
||||
---
|
||||
|
||||
## [1.3.1] - 2026-03-27
|
||||
|
||||
### 🐛 Bug Fixes / 修复
|
||||
|
||||
#### 📈 Xueqiu (雪球) — 全面修复
|
||||
|
||||
- **修复 400 错误根本原因：** `_ensure_cookies()` 仅访问首页只能获取 `acw_tc`（防 DDoS token），`xq_a_token` 由雪球前端 JS 动态生成，无法通过纯 HTTP 请求获取。新增三级 cookie 加载策略：① 读取 config 文件（`--from-browser` 保存的）→ ② 自动从本地 Chrome 浏览器提取（需安装 browser-cookie3）→ ③ homepage fallback
|
||||
- **修复 User-Agent：** `"agent-reach/1.0"` 被雪球反爬系统识别拒绝，改为真实 Chrome UA
|
||||
- **修复缺失 `Referer` 头：** 所有 API 请求加上 `Referer: https://xueqiu.com/`
|
||||
- **修复 `get_hot_posts()` 端点：** 原端点 `/statuses/hot/listV3.json` 已废弃（返回空 body），改为 `/v4/statuses/public_timeline_by_category.json`，正确解析 `item.data` JSON 字符串获取 author/likes/text
|
||||
- **修复 `urllib.request.quote` → `urllib.parse.quote`：** 明确使用正确模块
|
||||
- **修复 `configure --from-browser` 不提取雪球 Cookie：** `PLATFORM_SPECS` 加入 Xueqiu，检测 `xq_a_token` 存在才保存
|
||||
- **修正文档误导：** README/SKILL.md 中"无需配置"/"public API, no login required" → 准确描述需要 browser cookie
|
||||
- **改善错误信息：** `check()` 失败时提示 `configure --from-browser chrome` 而非"可能需要代理"
|
||||
|
||||
---
|
||||
|
||||
## [1.3.0] - 2026-03-12
|
||||
|
||||
### 🆕 New Channels / 新增渠道
|
||||
|
||||
#### 💻 V2EX
|
||||
- Hot topics, node topics, topic detail + replies, user profile via public JSON API
|
||||
- Zero config — no auth, no proxy, no API key required
|
||||
- `get_hot_topics(limit)`, `get_node_topics(node_name, limit)`, `get_topic(id)`, `get_user(username)`
|
||||
- 通过公开 JSON API 获取热门帖子、节点帖子、帖子详情+回复、用户信息
|
||||
- 零配置，无需认证、无需代理、无需 API Key
|
||||
|
||||
### 📈 Improvements / 改进
|
||||
|
||||
- Channel count: 14 → 15
|
||||
- 渠道数量：14 → 15
|
||||
|
||||
---
|
||||
|
||||
## [1.1.0] - 2025-02-25
|
||||
|
||||
### 🆕 New Channels / 新增渠道
|
||||
|
||||
#### ~~📷 Instagram~~ (removed — upstream blocked)
|
||||
- ~~Read public posts and profiles via [instaloader](https://github.com/instaloader/instaloader)~~
|
||||
- **Removed:** Instagram's aggressive anti-scraping measures broke all available open-source tools (instaloader, etc.). See [instaloader#2585](https://github.com/instaloader/instaloader/issues/2585). Will re-add when upstream recovers.
|
||||
- **已移除：** Instagram 反爬封杀导致所有开源工具（instaloader 等）失效。上游恢复后会重新加回。
|
||||
|
||||
#### 💼 LinkedIn
|
||||
- Read person profiles, company pages, and job details via [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server)
|
||||
- Search people and jobs via MCP, with Exa fallback
|
||||
- Fallback to Jina Reader when MCP is not configured
|
||||
- 通过 linkedin-scraper-mcp 读取个人 Profile、公司页面、职位详情
|
||||
- 通过 MCP 搜索人才和职位，Exa 兜底
|
||||
- 未配置 MCP 时自动 fallback 到 Jina Reader
|
||||
|
||||
#### 🏢 Boss直聘
|
||||
- QR code login via [mcp-bosszp](https://github.com/mucsbr/mcp-bosszp)
|
||||
- Job search and recruiter greeting via MCP
|
||||
- Fallback to Jina Reader for reading job pages
|
||||
- 通过 mcp-bosszp 扫码登录
|
||||
- MCP 搜索职位、向 HR 打招呼
|
||||
- Jina Reader 兜底读取职位页面
|
||||
|
||||
### 📈 Improvements / 改进
|
||||
|
||||
- Channel count: 9 → 12
|
||||
- `agent-reach doctor` now detects all 12 channels
|
||||
- CLI: added `search-linkedin`, `search-bosszhipin` subcommands
|
||||
- Updated install guide with setup instructions for new channels
|
||||
- 渠道数量：9 → 11
|
||||
- `agent-reach doctor` 现在检测全部 11 个渠道
|
||||
- CLI：新增 `search-linkedin`、`search-bosszhipin` 子命令
|
||||
- 安装指南新增渠道配置说明
|
||||
|
||||
---
|
||||
|
||||
## [1.0.0] - 2025-02-24
|
||||
|
||||
### 🎉 Initial Release / 首次发布
|
||||
|
||||
- 9 channels: Web, Twitter/X, YouTube, Bilibili, GitHub, Reddit, XiaoHongShu, RSS, Exa Search
|
||||
- CLI with `read`, `search`, `doctor`, `install` commands
|
||||
- Unified channel interface — each platform is a single pluggable Python file
|
||||
- Auto-detection of local vs server environments
|
||||
- Built-in diagnostics via `agent-reach doctor`
|
||||
- Skill registration for Claude Code / OpenClaw / Cursor
|
||||
- 9 个渠道：网页、Twitter/X、YouTube、B站、GitHub、Reddit、小红书、RSS、Exa 搜索
|
||||
- CLI 支持 `read`、`search`、`doctor`、`install` 命令
|
||||
- 统一渠道接口 — 每个平台一个独立可插拔的 Python 文件
|
||||
- 自动检测本地/服务器环境
|
||||
- 内置诊断 `agent-reach doctor`
|
||||
- Skill 注册支持 Claude Code / OpenClaw / Cursor
|
||||
@@ -0,0 +1,44 @@
|
||||
# CLAUDE.md
|
||||
|
||||
## Project
|
||||
Agent Reach — Python CLI + library that gives AI agents read/search access to 13 internet platforms.
|
||||
Positioning: installer + doctor + config tool. NOT a wrapper — after install, agents call upstream tools directly.
|
||||
Repo: github.com/Panniantong/Agent-Reach | License: MIT | Version: 1.5.0
|
||||
|
||||
## Commands
|
||||
- `pip install -e .` — Dev install
|
||||
- `pytest tests/ -v` — All tests
|
||||
- `pytest tests/test_cli.py -v` — CLI tests only
|
||||
- `bash test.sh` — Full integration test (creates venv, installs, runs doctor + channel tests)
|
||||
- `python -m agent_reach.cli doctor` — Run diagnostics
|
||||
- `python -m agent_reach.cli install --env=auto` — Auto-configure
|
||||
|
||||
## Structure
|
||||
- `agent_reach/cli.py` — CLI entry point (argparse)
|
||||
- `agent_reach/core.py` — Core read/search routing logic
|
||||
- `agent_reach/config.py` — Config management (YAML, env vars)
|
||||
- `agent_reach/doctor.py` — Diagnostics engine
|
||||
- `agent_reach/channels/` — One file per platform (twitter.py, reddit.py, youtube.py, etc.)
|
||||
- `agent_reach/channels/base.py` — Base channel class (all channels inherit from this)
|
||||
- `agent_reach/integrations/mcp_server.py` — MCP server integration
|
||||
- `agent_reach/skill/` — OpenClaw skill files
|
||||
- `agent_reach/guides/` — Usage guides
|
||||
- `tests/` — pytest tests
|
||||
- `config/mcporter.json` — MCP tool config
|
||||
|
||||
## Conventions
|
||||
- Python 3.10+ with type hints
|
||||
- Each channel is a single file in `channels/`, inherits from `BaseChannel`
|
||||
- Channel contract: must implement `can_handle(url)`, `read(url)`, `search(query)`, `check()` methods
|
||||
- Use `loguru` for logging, `rich` for CLI output
|
||||
- Commit format: `type(scope): message` (one commit = one thing)
|
||||
- All upstream tool calls go through public API/CLI, never hack internals
|
||||
|
||||
## Rules
|
||||
- NEVER modify upstream open source projects' source code
|
||||
- Agent Reach is a "glue layer" — only route and call, don't reimagine
|
||||
- Version in THREE places must match: `pyproject.toml`, `__init__.py`, `tests/test_cli.py`
|
||||
- Always new branch for changes, PR to main, never push to main directly
|
||||
- Run `pytest tests/ -v` before committing — all tests must pass
|
||||
- Cookie-based auth (Twitter, XHS): use Cookie-Editor export method only, no QR scan
|
||||
- XHS login: Cookie-Editor browser export only (QR will hang)
|
||||
+107
@@ -0,0 +1,107 @@
|
||||
# Contributing to Agent Reach
|
||||
|
||||
Thank you for your interest in contributing to Agent Reach! This document provides guidelines and instructions for contributing.
|
||||
|
||||
## Getting Started
|
||||
|
||||
1. Fork the repository on GitHub
|
||||
2. Clone your fork locally
|
||||
3. Create a new branch for your contribution
|
||||
4. Make your changes
|
||||
5. Run tests and linting
|
||||
6. Submit a pull request
|
||||
|
||||
## Development Setup
|
||||
|
||||
```bash
|
||||
# Clone your fork
|
||||
git clone https://github.com/YOUR_USERNAME/Agent-Reach.git
|
||||
cd Agent-Reach
|
||||
|
||||
# Install in development mode
|
||||
pip install -e ".[dev]"
|
||||
|
||||
# Install pre-commit hooks (optional but recommended)
|
||||
pre-commit install
|
||||
```
|
||||
|
||||
## Code Style
|
||||
|
||||
We use the following tools to maintain code quality:
|
||||
|
||||
- **ruff**: Linting and import sorting
|
||||
- **mypy**: Type checking
|
||||
- **pytest**: Testing
|
||||
|
||||
Run all checks before submitting a PR:
|
||||
|
||||
```bash
|
||||
# Linting
|
||||
ruff check agent_reach tests
|
||||
ruff format agent_reach tests
|
||||
|
||||
# Type checking
|
||||
mypy agent_reach
|
||||
|
||||
# Tests
|
||||
pytest
|
||||
```
|
||||
|
||||
## Adding New Channels
|
||||
|
||||
Agent Reach uses a unified channel interface. To add a new platform:
|
||||
|
||||
1. Create a new file in `agent_reach/channels/`
|
||||
2. Implement the channel contract (see existing channels for examples)
|
||||
3. Add tests in `tests/test_channels.py`
|
||||
4. Update `agent_reach/doctor.py` to include the new channel
|
||||
5. Update documentation
|
||||
|
||||
## Pull Request Guidelines
|
||||
|
||||
- **Small, focused changes** are preferred over large refactors
|
||||
- Include tests for new functionality
|
||||
- Update documentation if needed
|
||||
- Follow existing code style
|
||||
- Reference any related issues
|
||||
|
||||
## Reporting Issues
|
||||
|
||||
When reporting bugs, please include:
|
||||
|
||||
- Python version
|
||||
- Operating system
|
||||
- Steps to reproduce
|
||||
- Expected vs actual behavior
|
||||
- Any error messages
|
||||
|
||||
## Questions?
|
||||
|
||||
Feel free to open an issue for questions or join discussions.
|
||||
|
||||
---
|
||||
|
||||
感谢您对 Agent Reach 的贡献！本文档提供了贡献指南。
|
||||
|
||||
## 快速开始
|
||||
|
||||
1. 在 GitHub 上 fork 仓库
|
||||
2. 本地 clone 您的 fork
|
||||
3. 创建新分支
|
||||
4. 提交更改
|
||||
5. 运行测试和 lint
|
||||
6. 提交 pull request
|
||||
|
||||
## 代码规范
|
||||
|
||||
- 使用 **ruff** 进行代码检查
|
||||
- 使用 **mypy** 进行类型检查
|
||||
- 使用 **pytest** 运行测试
|
||||
|
||||
## 添加新渠道
|
||||
|
||||
1. 在 `agent_reach/channels/` 创建新文件
|
||||
2. 实现渠道接口
|
||||
3. 添加测试
|
||||
4. 更新 doctor 检测
|
||||
5. 更新文档
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025 Agent Eyes
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,326 @@
|
||||
<h1 align="center">👁️ Agent Reach</h1>
|
||||
|
||||
<p align="center">
|
||||
<strong>给你的 AI Agent 一键装上互联网能力</strong>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
当下最稳的接入方式，替你选好、装好、体检好——接入方式会换代，你不用操心
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="https://trendshift.io/repositories/24387"><img src="https://trendshift.io/api/badge/repositories/24387" alt="Trendshift GitHub Trending #1 Repository of the Day"></a>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg?style=for-the-badge" alt="MIT License"></a>
|
||||
<a href="https://www.python.org/"><img src="https://img.shields.io/badge/Python-3.10+-green.svg?style=for-the-badge&logo=python&logoColor=white" alt="Python 3.10+"></a>
|
||||
<a href="https://github.com/Panniantong/agent-reach/stargazers"><img src="https://img.shields.io/github/stars/Panniantong/agent-reach?style=for-the-badge" alt="GitHub Stars"></a>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="#快速上手">快速开始</a> · <a href="docs/README_en.md">English</a> · <a href="docs/README_ja.md">日本語</a> · <a href="docs/README_ko.md">한국어</a> · <a href="#支持的平台">支持平台</a> · <a href="#设计理念">设计理念</a>
|
||||
</p>
|
||||
|
||||
---
|
||||
|
||||
## 为什么需要 Agent Reach？
|
||||
|
||||
AI Agent 已经能帮你写代码、改文档、管项目——但你让它去网上找点东西，它就抓瞎了：
|
||||
|
||||
- 📺 "帮我看看这个 YouTube 教程讲了什么" → **看不了**，拿不到字幕
|
||||
- 🐦 "帮我搜一下推特上大家怎么评价这个产品" → **搜不了**，Twitter API 要付费
|
||||
- 📖 "去 Reddit 上看看有没有人遇到过同样的 bug" → **403 被封**，服务器 IP 被拒
|
||||
- 📕 "帮我看看小红书上这个品的口碑" → **打不开**，必须登录才能看
|
||||
- 📺 "B站上有个技术视频，帮我总结一下" → **拿不到**，通用下载工具被 B站风控全面拦截
|
||||
- 🔍 "帮我在网上搜一下最新的 LLM 框架对比" → **没有好用的搜索**，要么付费要么质量差
|
||||
- 🌐 "帮我看看这个网页写了啥" → **抓回来一堆 HTML 标签**，根本没法读
|
||||
- 📦 "这个 GitHub 仓库是干嘛的？Issue 里说了什么？" → 能用，但认证配置很麻烦
|
||||
- 📡 "帮我订阅这几个 RSS 源，有更新告诉我" → 要自己装库写代码
|
||||
|
||||
**这些不难实现，但是需要自己折腾配置**
|
||||
|
||||
每个平台都有自己的门槛——要付费的 API、要绕过的封锁、要登录的账号、要清洗的数据。你要一个一个去踩坑、装工具、调配置，光是让 Agent 能读个推特就得折腾半天。
|
||||
|
||||
**Agent Reach 把这件事变成一句话：**
|
||||
|
||||
```
|
||||
帮我安装 Agent Reach：https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md
|
||||
```
|
||||
|
||||
复制给你的 Agent，几分钟后它就能读推特、搜 Reddit、看 YouTube、刷小红书了。
|
||||
|
||||
**已经装过了？更新也是一句话：**
|
||||
|
||||
```
|
||||
帮我更新 Agent Reach：https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md
|
||||
```
|
||||
|
||||
> ⭐ **Star 这个项目**，我们会持续追踪各平台的变化、接入新的渠道。你不用自己盯——平台封了我们修，有新渠道我们加。
|
||||
|
||||
### ✅ 在你用之前，你可能想知道
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 💰 **完全免费** | 所有工具开源、所有 API 免费。唯一可能花钱的是服务器代理（$1/月），本地电脑不需要 |
|
||||
| 🔒 **隐私安全** | Cookie 只存在你本地，不上传不外传。代码完全开源，随时可审查 |
|
||||
| 🔄 **持续换代** | 每个平台都是「首选 + 备选」多后端路由。某个接入方式失效了，我们换下一个，你无感（2026-06 实例：yt-dlp 被 B站风控封死 → 已切换 bili-cli，用户零操作） |
|
||||
| 🤖 **兼容所有 Agent** | Claude Code、OpenClaw、Cursor、Windsurf……任何能跑命令行的 Agent 都能用 |
|
||||
| 🩺 **自带诊断** | `agent-reach doctor` 一条命令告诉你哪个通、哪个不通、怎么修 |
|
||||
|
||||
---
|
||||
|
||||
## 支持的平台
|
||||
|
||||
| 平台 | 装好即用 | 配置后解锁 | 怎么配 |
|
||||
|------|---------|-----------|-------|
|
||||
| 🌐 **网页** | 阅读任意网页 | — | 无需配置 |
|
||||
| 📺 **YouTube** | 字幕提取 + 视频搜索 | — | 无需配置 |
|
||||
| 📡 **RSS** | 阅读任意 RSS/Atom 源 | — | 无需配置 |
|
||||
| 🔍 **全网搜索** | — | 全网语义搜索 | 自动配置（MCP 接入，免费无需 Key） |
|
||||
| 📦 **GitHub** | 读公开仓库 + 搜索 | 私有仓库、提 Issue/PR、Fork | 告诉 Agent「帮我登录 GitHub」 |
|
||||
| 🐦 **Twitter/X** | 读单条推文 | 搜索推文、浏览时间线、读长文 | 告诉 Agent「帮我配 Twitter」 |
|
||||
| 📺 **B站** | 搜索 + 视频详情（bili-cli，无需登录） | 字幕（OpenCLI） | 告诉 Agent「帮我配 B站」 |
|
||||
| 📖 **Reddit** | —（没有零配置路径：匿名接口已被封） | 搜索 + 读帖子和评论 | 桌面装 OpenCLI 用浏览器登录态；或 rdt-cli + Cookie |
|
||||
| 📘 **Facebook** | — | 搜索、主页、Feed、群组列表 | 桌面装 OpenCLI（复用 Chrome 登录态） |
|
||||
| 📷 **Instagram** | — | 用户搜索、Profile、用户最近帖子、Explore | 桌面装 OpenCLI（复用 Chrome 登录态） |
|
||||
| 📕 **小红书** | — | 搜索、阅读、评论 | 桌面装 OpenCLI（刷过小红书即可用）；服务器用 xiaohongshu-mcp 扫码 |
|
||||
| 💼 **LinkedIn** | Jina Reader 读公开页面 | Profile 详情、公司页面、职位搜索 | 告诉 Agent「帮我配 LinkedIn」 |
|
||||
| 💻 **V2EX** | 热门帖子、节点帖子、帖子详情+回复、用户信息 | — | 无需配置 |
|
||||
| 📈 **雪球** | 股票行情、搜索股票、热门帖子、热门股票排行 | — | 告诉 Agent「帮我配雪球」 |
|
||||
| 🎙️ **小宇宙播客** | — | 播客音频转文字（Whisper 转录，免费 Key） | 告诉 Agent「帮我配小宇宙播客」 |
|
||||
|
||||
> **不知道怎么配？不用查文档。** 直接告诉 Agent「帮我配 XXX」，它知道需要什么、会一步一步引导你。
|
||||
>
|
||||
> 🍪 需要 Cookie/登录态的平台（Twitter、小红书、Reddit、Facebook、Instagram 等），优先让用户在自己的浏览器里登录。OpenCLI 复用 Chrome 登录态；传统 CLI 才需要 Cookie-Editor 导出 Cookie。
|
||||
>
|
||||
> 🔒 Cookie 只存在你本地，不上传不外传。代码完全开源，随时可审查。
|
||||
> 💻 本地电脑不需要代理。代理只有部署在服务器上才需要（~$1/月）。
|
||||
|
||||
---
|
||||
|
||||
## 快速上手
|
||||
|
||||
> ⚠️ **OpenClaw 用户请先确认 exec 权限已开启**
|
||||
>
|
||||
> Agent Reach 依赖 Agent 执行 shell 命令（`pip install`、`mcporter`、`twitter` 等）。如果你的 OpenClaw 使用了默认的 `messaging` 工具配置，Agent 将无法执行命令。**安装前请先开启 exec 权限**：
|
||||
>
|
||||
> ```bash
|
||||
> openclaw config set tools.profile "coding"
|
||||
> ```
|
||||
> 或在 `~/.openclaw/openclaw.json` 中设置 `"tools": { "profile": "coding" }`。
|
||||
> 设置后重启 Gateway（`openclaw gateway restart`）并开启新对话即可。其他平台（Claude Code、Cursor、Windsurf 等）不受此限制。
|
||||
|
||||
复制这句话给你的 AI Agent（Claude Code、OpenClaw、Cursor 等）：
|
||||
|
||||
```
|
||||
帮我安装 Agent Reach：https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md
|
||||
```
|
||||
|
||||
就这一步。Agent 会自己完成剩下的所有事情。
|
||||
|
||||
> 🔄 **已安装过？** 更新也是一句话：
|
||||
> ```
|
||||
> 帮我更新 Agent Reach：https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md
|
||||
> ```
|
||||
|
||||
> 🛡️ **担心安全？** 可以用安全模式——不会自动装系统包，只告诉你需要什么：
|
||||
> ```
|
||||
> 帮我安装 Agent Reach（安全模式）：https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md
|
||||
> 安装时使用 --safe 参数
|
||||
> ```
|
||||
|
||||
<details>
|
||||
<summary>它会做什么？（点击展开）</summary>
|
||||
|
||||
1. **安装 CLI 工具** — `pip install` 装好 `agent-reach` 命令行（自带 yt-dlp、feedparser）
|
||||
2. **安装系统基建** — 自动检测并安装 Node.js、gh CLI、mcporter
|
||||
3. **配置搜索引擎** — 通过 MCP 接入 Exa（免费，无需 API Key）
|
||||
4. **检测环境** — 判断是本地电脑还是服务器，给出对应的配置建议
|
||||
5. **注册 SKILL.md** — 在 Agent 的 skills 目录安装使用指南，以后 Agent 遇到"全网调研"、"搜推特"、"看视频"这类需求，会自动知道该调哪个上游工具
|
||||
6. **问你要不要更多** — 默认只激活 6 个零配置渠道；小红书、Twitter、Reddit、Facebook、Instagram 这些需要登录态的，Agent 会列菜单问你要哪些，点名才装
|
||||
|
||||
安装完之后，`agent-reach doctor` 一条命令告诉你每个渠道的状态、当前走哪条路。
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## 装好就能用
|
||||
|
||||
不需要任何配置，告诉 Agent 就行：
|
||||
|
||||
- "帮我看看这个链接" → `curl https://r.jina.ai/URL` 读任意网页
|
||||
- "这个 GitHub 仓库是做什么的" → `gh repo view owner/repo`
|
||||
- "这个 YouTube 视频讲了什么" → `yt-dlp` 提取字幕
|
||||
- "B站搜一下 AI 教程" → `bili search`（无需登录）
|
||||
- "全网搜一下 LLM 框架对比" → Exa 语义搜索
|
||||
- "订阅这个 RSS" → `feedparser` 解析
|
||||
|
||||
**不需要记命令。** Agent 读了 SKILL.md 之后自己知道该调什么。需要登录的平台（小红书、Twitter、Reddit、Facebook、Instagram），告诉 Agent「帮我配 XXX」即可解锁。
|
||||
|
||||
---
|
||||
|
||||
## 能力边界：读内容 vs 操作网页
|
||||
|
||||
有些任务超出"读"的范围：登录后的网页操作、表单提交、多账号隔离、并行浏览器会话、自动化流程中的登录、验证、风控提示等高摩擦环节的人工接手。这类"动手"场景，可以配合 [BrowserAct](https://www.browseract.com/?co-from=Agent&redirect=https://github.com/browser-act/skills/tree/main) 这类浏览器自动化工具——30+ 预制平台技能，支持 Claude Code / OpenClaw / Cursor 等主流 Agent。
|
||||
|
||||
---
|
||||
|
||||
## 设计理念
|
||||
|
||||
**Agent Reach 是一个能力层（capability layer），不是又一个工具。**
|
||||
|
||||
它比任何具体实现高一层——负责**选型、安装、体检、路由**，不负责底层读取本身。读取由 Agent 直接调用上游工具完成，没有包装层。
|
||||
|
||||
你给一个新 Agent 装环境的时候，总要花时间去找工具、装依赖、调配置——Twitter 用什么读？Reddit 怎么登录？小红书的 CLI 停更了换什么？每次都要重新踩一遍。Agent Reach 做的事情很简单：**当下最稳的接入方式，我们替你选好、装好、体检好。接入方式会换代（2026 年 3 月一批单平台 CLI 集体停更，我们换了路由），你不用操心。**
|
||||
|
||||
### 🔌 每个平台 = 首选 + 备选的有序后端列表
|
||||
|
||||
换接入方式 = 调整列表顺序，不是重写代码。`agent-reach doctor` 会告诉你每个平台**当前在用哪个后端**。
|
||||
|
||||
```
|
||||
channels/
|
||||
├── web.py → Jina Reader
|
||||
├── twitter.py → twitter-cli ▸ OpenCLI ▸ bird
|
||||
├── youtube.py → yt-dlp
|
||||
├── github.py → gh CLI
|
||||
├── bilibili.py → bili-cli ▸ OpenCLI ▸ 搜索 API（yt-dlp 已被 B站风控封死，退役）
|
||||
├── reddit.py → OpenCLI ▸ rdt-cli（无零配置路径，必须登录态）
|
||||
├── facebook.py → OpenCLI（桌面浏览器登录态）
|
||||
├── instagram.py → OpenCLI（桌面浏览器登录态）
|
||||
├── xiaohongshu.py → OpenCLI ▸ xiaohongshu-mcp ▸ xhs-cli
|
||||
├── linkedin.py → linkedin-mcp ▸ Jina Reader
|
||||
├── rss.py → feedparser
|
||||
├── exa_search.py → Exa via mcporter
|
||||
└── __init__.py → 渠道注册（doctor 检测用）
|
||||
```
|
||||
|
||||
每个渠道文件按序**真实探测**各候选后端（不只是看命令存不存在），第一个完整可用的当选；坏掉的会给出修复处方。实际的读取和搜索由 Agent 直接调用上游工具完成。
|
||||
|
||||
### 当前选型
|
||||
|
||||
| 场景 | 首选 | 备选 | 为什么这么选 |
|
||||
|------|------|------|-----------|
|
||||
| 读网页 | [Jina Reader](https://github.com/jina-ai/reader) | — | 免费，不需要 API Key |
|
||||
| 读推特 | [twitter-cli](https://github.com/public-clis/twitter-cli) | [OpenCLI](https://github.com/jackwener/opencli) | 实测搜索稳定；OpenCLI 走浏览器登录态兜底 |
|
||||
| Reddit | [OpenCLI](https://github.com/jackwener/opencli)（桌面） | [rdt-cli](https://github.com/public-clis/rdt-cli) | 匿名接口已被封、官方 API 审批制——只剩登录态路线 |
|
||||
| Facebook | [OpenCLI](https://github.com/jackwener/opencli)（桌面） | — | Graph API/Groups API 权限收紧；浏览器登录态是当前最实用路径 |
|
||||
| Instagram | [OpenCLI](https://github.com/jackwener/opencli)（桌面） | 官方 Graph API（Business/Creator + 审批） | instaloader 类路径不稳定；OpenCLI 复用真实浏览器会话 |
|
||||
| YouTube 字幕 + 搜索 | [yt-dlp](https://github.com/yt-dlp/yt-dlp) | — | 154K Star，YouTube 仍是最佳（注意：不再用于 B站） |
|
||||
| B站 | [bili-cli](https://github.com/public-clis/bilibili-cli) | OpenCLI ▸ 搜索 API | yt-dlp 被 B站风控 412 封死（2026-06 实测），bili-cli 无登录可搜可读 |
|
||||
| 搜全网 | [Exa](https://exa.ai) via [mcporter](https://github.com/nicobailon/mcporter) | — | AI 语义搜索，MCP 接入免 Key |
|
||||
| GitHub | [gh CLI](https://cli.github.com) | — | 官方工具，认证后完整 API 能力 |
|
||||
| 读 RSS | [feedparser](https://github.com/kurtmckee/feedparser) | — | Python 生态标准选择 |
|
||||
| 小红书 | [OpenCLI](https://github.com/jackwener/opencli)（桌面） | [xiaohongshu-mcp](https://github.com/xpzouying/xiaohongshu-mcp)（服务器）▸ xhs-cli | xhs-cli 作者已转投 OpenCLI（24K Star）；浏览器登录态零摩擦 |
|
||||
| LinkedIn | [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server) | Jina Reader | MCP 服务，浏览器自动化 |
|
||||
|
||||
> 📌 这些都是「当前选型」，基于真机实测定期复核。某条路失效了我们换下一条——`agent-reach doctor` 永远告诉你现在走的是哪条。
|
||||
|
||||
---
|
||||
|
||||
## 安全性
|
||||
|
||||
Agent Reach 在设计上重视安全：
|
||||
|
||||
| 措施 | 说明 |
|
||||
|------|------|
|
||||
| 🔒 **凭据本地存储** | Cookie、Token 只存在你本机 `~/.agent-reach/config.yaml`，文件权限 600（仅所有者可读写），不上传不外传 |
|
||||
| 🛡️ **安全模式** | `agent-reach install --safe` 不会自动修改系统，只列出需要什么，由你决定装不装 |
|
||||
| 👀 **完全开源** | 代码透明，随时可审查。所有依赖工具也是开源项目 |
|
||||
| 🔍 **Dry Run** | `agent-reach install --dry-run` 预览所有操作，不做任何改动 |
|
||||
| 🧩 **可插拔架构** | 不信任某个组件？换掉对应的 channel 文件即可，不影响其他 |
|
||||
|
||||
### 🍪 Cookie 安全建议
|
||||
|
||||
> ⚠️ **封号风险提醒：** 使用 Cookie 登录的平台（Twitter、小红书等），通过脚本/API 调用**存在被平台检测并封号的风险**。请务必使用**专用小号**，不要用你的主账号。
|
||||
|
||||
需要 Cookie 或登录态的平台（Twitter、小红书、Reddit、Facebook、Instagram 等）建议使用**专用小号**，不要用主账号。原因有二：
|
||||
1. **封号风险** — 平台可能检测到非正常浏览器的 API 调用行为，导致账号被限制或封禁
|
||||
2. **安全风险** — Cookie 等同于完整登录权限，用小号可以在凭据泄露时限制影响范围
|
||||
|
||||
### 📦 安装方式
|
||||
|
||||
| 方式 | 命令 | 适合场景 |
|
||||
|------|------|---------|
|
||||
| 一键全自动（默认） | `agent-reach install --env=auto` | 个人电脑、开发环境 |
|
||||
| 安全模式 | `agent-reach install --env=auto --safe` | 生产服务器、多人共用机器 |
|
||||
| 仅预览 | `agent-reach install --env=auto --dry-run` | 先看看会做什么 |
|
||||
|
||||
### 🗑️ 卸载
|
||||
|
||||
```bash
|
||||
agent-reach uninstall
|
||||
```
|
||||
|
||||
会清除：`~/.agent-reach/`（含所有 token/cookie）、各 Agent 的 skill 文件、mcporter 中的 MCP 配置。
|
||||
|
||||
```bash
|
||||
# 只预览，不实际删除
|
||||
agent-reach uninstall --dry-run
|
||||
|
||||
# 只删 skill 文件，保留 token 配置（重装时用）
|
||||
agent-reach uninstall --keep-config
|
||||
```
|
||||
|
||||
卸载 Python 包本身：`pip uninstall agent-reach`
|
||||
|
||||
---
|
||||
|
||||
## ⭐ 为什么值得 Star
|
||||
|
||||
这个项目我自己每天在用，所以我会一直维护它。
|
||||
|
||||
- 有新需求或者大家提了想要的渠道，我会陆续加上
|
||||
- 每个渠道我会尽量保证**能用、好用、免费**
|
||||
- 平台改了反爬或者 API 变了，我会想办法解决
|
||||
|
||||
为 Web 4.0 基建贡献一份自己的力量。
|
||||
|
||||
Star 一下，下次需要的时候能找到。⭐
|
||||
|
||||
---
|
||||
|
||||
## 致谢
|
||||
|
||||
[OpenCLI](https://github.com/jackwener/opencli) · [twitter-cli](https://github.com/public-clis/twitter-cli) · [rdt-cli](https://github.com/public-clis/rdt-cli) · [xiaohongshu-mcp](https://github.com/xpzouying/xiaohongshu-mcp) · [xhs-cli](https://github.com/jackwener/xiaohongshu-cli) · [bili-cli](https://github.com/public-clis/bilibili-cli) · [yt-dlp](https://github.com/yt-dlp/yt-dlp) · [Jina Reader](https://github.com/jina-ai/reader) · [Exa](https://exa.ai) · [mcporter](https://github.com/nicobailon/mcporter) · [feedparser](https://github.com/kurtmckee/feedparser) · [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server)
|
||||
|
||||
## 联系
|
||||
|
||||
- 📧 **Email:** pnt01@foxmail.com
|
||||
- 🐦 **Twitter/X:** [@Neo_Reidlab](https://x.com/Neo_Reidlab)
|
||||
|
||||
## 业务合作 / Agent 落地
|
||||
|
||||
我正在承接 Agent 相关的定制与落地合作。
|
||||
|
||||
如果你在企业生产、运营、市场、投研、数据处理、内容处理或其他业务流程里，有希望用 Agent 自动化的环节，欢迎加我微信交流。
|
||||
|
||||
不需要你已经想清楚方案。只要你有真实流程、真实问题或真实需求，我可以一起判断 Agent 能不能解决、怎么做。
|
||||
|
||||
加好友请备注：`业务 + 你想让 Agent 帮你做什么`
|
||||
|
||||
Builder 也欢迎备注：`Builder + 你在做什么`
|
||||
|
||||
只是想进交流群，备注：`加群`
|
||||
|
||||
<p align="center">
|
||||
<img src="docs/wechat-group-qr.jpg" width="280" alt="WeChat QR">
|
||||
</p>
|
||||
|
||||
> Bug 反馈和功能请求请用 [GitHub Issues](https://github.com/Panniantong/Agent-Reach/issues)，更容易跟踪。
|
||||
|
||||
## License
|
||||
|
||||
[MIT](LICENSE)
|
||||
|
||||
## 友情链接
|
||||
|
||||
[方舟 Agent Plan 模型订阅套餐](https://dis.chatdesks.cn/chatdesk/hsyqAgent-Reach.html) — 集成了包含 Doubao-Seed、Doubao-Seedance、Doubao-Seedream 等在内的字节跳动自研 SOTA 级模型，覆盖文本、代码、图像、视频等多模态任务。最新支持 MiniMax-M3、DeepSeek-V4 系列、GLM-5.2、Doubao-Seed-2.0 系列、Kimi-K2.6 等模型，工具不限。超全模态模型与 Harness 升级一步到位，深度支持 Agent 框架与 AI 编程工具。一次订阅，可以为不同任务切换合适的 AI 引擎。
|
||||
|
||||
[腾讯云 OpenClaw](https://www.tencentcloud.com/act/pro/intl-openclaw?referral_code=G76Y819A&lang=zh&pg=) — 在腾讯云Lighthouse秒级部署OpenClaw全能助手，可通过对话丝滑接入Agent Reach，给你的OpenClaw一键装上互联网能力。
|
||||
|
||||
[AtomGit 镜像](https://atomgit.com/qq_51337814/Agent-Reach) — Agent Reach 的 AtomGit 同步镜像，便于国内访问与克隆。
|
||||
|
||||
## Star History
|
||||
|
||||
[](https://star-history.com/#Panniantong/Agent-Reach&Date)
|
||||
@@ -0,0 +1,7 @@
|
||||
# WeHub 来源说明
|
||||
|
||||
- 原始项目：`Panniantong/Agent-Reach`
|
||||
- 原始仓库：https://github.com/Panniantong/Agent-Reach
|
||||
- 导入方式：上游默认分支的最新快照
|
||||
- 原作者、版权和许可证信息以原始仓库及本仓库 LICENSE 为准
|
||||
- 本文件仅用于记录来源，不代表 WeHub 是原项目作者
|
||||
+51
@@ -0,0 +1,51 @@
|
||||
# Security Policy
|
||||
|
||||
## Supported Versions
|
||||
|
||||
| Version | Supported |
|
||||
|---------|-----------|
|
||||
| Latest | ✅ Yes |
|
||||
|
||||
## Reporting a Vulnerability
|
||||
|
||||
If you discover a security vulnerability in Agent-Reach, please report
|
||||
it responsibly by using GitHub's private security advisory feature:
|
||||
|
||||
👉 **[Report a vulnerability](https://github.com/Panniantong/Agent-Reach/security/advisories/new)**
|
||||
|
||||
Please do NOT open a public GitHub issue for security vulnerabilities.
|
||||
|
||||
## What to Include
|
||||
|
||||
- Description of the vulnerability
|
||||
- Steps to reproduce
|
||||
- Affected versions
|
||||
- Potential impact
|
||||
- Suggested fix (if any)
|
||||
|
||||
## Response Timeline
|
||||
|
||||
- Acknowledgement within **48 hours**
|
||||
- Status update within **7 days**
|
||||
- Fix timeline communicated within **14 days**
|
||||
|
||||
## Scope
|
||||
|
||||
The following are considered in scope:
|
||||
- Authentication and authorization bypass
|
||||
- Remote code execution
|
||||
- Path traversal / arbitrary file read
|
||||
- Server-Side Request Forgery (SSRF)
|
||||
- Injection vulnerabilities (SQL, command, prompt)
|
||||
- Sensitive data exposure
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- Vulnerabilities in dependencies (report to the dependency maintainer)
|
||||
- Social engineering attacks
|
||||
- Denial of service via resource exhaustion
|
||||
|
||||
## Credits
|
||||
|
||||
We appreciate responsible disclosure and will credit researchers
|
||||
in our release notes unless anonymity is requested.
|
||||
@@ -0,0 +1,9 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Agent Reach — Give your AI Agent eyes to see the entire internet."""
|
||||
|
||||
__version__ = "1.5.0"
|
||||
__author__ = "Neo Reid"
|
||||
|
||||
from agent_reach.core import AgentReach
|
||||
|
||||
__all__ = ["AgentReach"]
|
||||
@@ -0,0 +1,16 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Cross-channel backends.
|
||||
|
||||
A backend here is an upstream runtime that serves MULTIPLE channels
|
||||
(e.g. OpenCLI covers xiaohongshu/reddit/bilibili/twitter through one
|
||||
browser session), as opposed to the per-platform tools probed inside
|
||||
each channel file.
|
||||
"""
|
||||
|
||||
from .opencli import ( # noqa: F401
|
||||
OPENCLI_EXTENSION_URL,
|
||||
OPENCLI_PACKAGE,
|
||||
OpenCLIStatus,
|
||||
opencli_status,
|
||||
opencli_summary,
|
||||
)
|
||||
@@ -0,0 +1,136 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""OpenCLI backend probing.
|
||||
|
||||
OpenCLI (github.com/jackwener/opencli) drives the user's real Chrome via a
|
||||
browser-bridge extension + local daemon, reusing existing login sessions —
|
||||
zero per-platform configuration, desktop-only (no headless).
|
||||
|
||||
Probing notes (verified live):
|
||||
- `opencli doctor` AUTO-STARTS the daemon — a side effect, so health
|
||||
checks must use `opencli daemon status` (pure query) instead.
|
||||
- Exit codes are always 0; status must be parsed from text output.
|
||||
- "Extension: disconnected" does NOT mean unusable: the extension's
|
||||
service worker sleeps and any real opencli command wakes it up
|
||||
(verified: status flips disconnected→connected after one call).
|
||||
Since daemon status can't tell "sleeping" from "never installed",
|
||||
we check Chrome's Extensions directory on disk to disambiguate.
|
||||
"""
|
||||
|
||||
import glob
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
|
||||
from agent_reach.probe import probe_command
|
||||
|
||||
OPENCLI_PACKAGE = "@jackwener/opencli"
|
||||
OPENCLI_EXTENSION_ID = "ildkmabpimmkaediidaifkhjpohdnifk"
|
||||
OPENCLI_EXTENSION_URL = (
|
||||
f"https://chromewebstore.google.com/detail/opencli/{OPENCLI_EXTENSION_ID}"
|
||||
)
|
||||
|
||||
#: Chrome-family profile roots that contain <Profile>/Extensions/<id>/
|
||||
_CHROME_PROFILE_ROOTS = (
|
||||
"~/Library/Application Support/Google/Chrome", # macOS Chrome
|
||||
"~/Library/Application Support/Chromium", # macOS Chromium
|
||||
"~/.config/google-chrome", # Linux Chrome
|
||||
"~/.config/chromium", # Linux Chromium
|
||||
)
|
||||
|
||||
|
||||
def _extension_installed_on_disk() -> bool:
|
||||
"""True if the OpenCLI extension exists in any Chrome profile.
|
||||
|
||||
Store-installed extensions always live under
|
||||
<profile>/Extensions/<extension id>/ — this disambiguates a sleeping
|
||||
service worker from a never-installed extension. Dev installs via
|
||||
"Load unpacked" are not covered (those users can read `opencli doctor`).
|
||||
"""
|
||||
roots = [os.path.expanduser(p) for p in _CHROME_PROFILE_ROOTS]
|
||||
local_app_data = os.environ.get("LOCALAPPDATA")
|
||||
if local_app_data: # Windows
|
||||
roots.append(os.path.join(local_app_data, "Google", "Chrome", "User Data"))
|
||||
for root in roots:
|
||||
if glob.glob(os.path.join(root, "*", "Extensions", OPENCLI_EXTENSION_ID)):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
@dataclass
|
||||
class OpenCLIStatus:
|
||||
installed: bool = False
|
||||
broken: bool = False
|
||||
daemon_running: bool = False
|
||||
extension_connected: bool = False
|
||||
extension_installed: bool = False
|
||||
version: str = ""
|
||||
hint: str = ""
|
||||
|
||||
@property
|
||||
def ready(self) -> bool:
|
||||
"""Usable now or on first call.
|
||||
|
||||
A live connection counts, and so does an installed-but-sleeping
|
||||
extension: its service worker wakes on the first real command.
|
||||
"""
|
||||
return self.installed and not self.broken and (
|
||||
self.extension_connected or self.extension_installed
|
||||
)
|
||||
|
||||
|
||||
def opencli_status(timeout: int = 10) -> OpenCLIStatus:
|
||||
"""Probe OpenCLI install + daemon/extension state without side effects."""
|
||||
version_probe = probe_command(
|
||||
"opencli", ["--version"], timeout=timeout, package=OPENCLI_PACKAGE
|
||||
)
|
||||
if version_probe.status == "missing":
|
||||
return OpenCLIStatus(installed=False)
|
||||
if not version_probe.ok:
|
||||
return OpenCLIStatus(
|
||||
installed=True,
|
||||
broken=True,
|
||||
hint=(
|
||||
"opencli 命令存在但无法执行（node 环境损坏），重装：\n"
|
||||
f" npm install -g {OPENCLI_PACKAGE}"
|
||||
),
|
||||
)
|
||||
|
||||
st = OpenCLIStatus(installed=True, version=version_probe.output.strip())
|
||||
|
||||
daemon_probe = probe_command(
|
||||
"opencli", ["daemon", "status"], timeout=timeout, package=OPENCLI_PACKAGE
|
||||
)
|
||||
output = daemon_probe.output if daemon_probe.ok else ""
|
||||
# `opencli daemon status` prints lines like:
|
||||
# Daemon: running (PID 37389) / Daemon: not running
|
||||
# Extension: connected / Extension: disconnected
|
||||
for line in output.splitlines():
|
||||
line = line.strip().lower()
|
||||
if line.startswith("daemon:"):
|
||||
st.daemon_running = "not running" not in line and "running" in line
|
||||
elif line.startswith("extension:"):
|
||||
st.extension_connected = "disconnected" not in line and "connected" in line
|
||||
|
||||
if not st.extension_connected:
|
||||
st.extension_installed = _extension_installed_on_disk()
|
||||
if not st.extension_installed:
|
||||
st.hint = (
|
||||
"OpenCLI 已安装，但 Chrome 扩展未安装。\n"
|
||||
f" 1. 安装扩展（需手动点一次）：{OPENCLI_EXTENSION_URL}\n"
|
||||
" 2. 保持 Chrome 打开，运行 `opencli doctor` 验证"
|
||||
)
|
||||
return st
|
||||
|
||||
|
||||
def opencli_summary(st: OpenCLIStatus) -> str:
|
||||
"""One-line state description for channel messages / install output."""
|
||||
if not st.installed:
|
||||
return "OpenCLI 未安装"
|
||||
if st.broken:
|
||||
return "OpenCLI 无法执行（node 环境损坏）"
|
||||
if st.extension_connected:
|
||||
return f"OpenCLI 可用（浏览器登录态，v{st.version}）"
|
||||
if st.ready:
|
||||
return "OpenCLI 可用（扩展睡眠中，调用时自动唤醒）"
|
||||
if st.daemon_running:
|
||||
return "OpenCLI 已安装，等待 Chrome 扩展安装"
|
||||
return "OpenCLI 已安装（daemon 未运行，使用时自动启动；需 Chrome 扩展）"
|
||||
@@ -0,0 +1,63 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
Channel registry — lists all supported platforms for doctor checks.
|
||||
"""
|
||||
|
||||
from typing import List, Optional
|
||||
|
||||
# Import all channels
|
||||
from .base import Channel
|
||||
from .bilibili import BilibiliChannel
|
||||
from .exa_search import ExaSearchChannel
|
||||
from .facebook import FacebookChannel
|
||||
from .github import GitHubChannel
|
||||
from .instagram import InstagramChannel
|
||||
from .linkedin import LinkedInChannel
|
||||
from .reddit import RedditChannel
|
||||
from .rss import RSSChannel
|
||||
from .twitter import TwitterChannel
|
||||
from .v2ex import V2EXChannel
|
||||
from .web import WebChannel
|
||||
from .xiaohongshu import XiaoHongShuChannel
|
||||
from .xiaoyuzhou import XiaoyuzhouChannel
|
||||
from .xueqiu import XueqiuChannel
|
||||
from .youtube import YouTubeChannel
|
||||
|
||||
ALL_CHANNELS: List[Channel] = [
|
||||
GitHubChannel(),
|
||||
TwitterChannel(),
|
||||
YouTubeChannel(),
|
||||
RedditChannel(),
|
||||
FacebookChannel(),
|
||||
InstagramChannel(),
|
||||
BilibiliChannel(),
|
||||
XiaoHongShuChannel(),
|
||||
LinkedInChannel(),
|
||||
XiaoyuzhouChannel(),
|
||||
V2EXChannel(),
|
||||
XueqiuChannel(),
|
||||
RSSChannel(),
|
||||
ExaSearchChannel(),
|
||||
WebChannel(),
|
||||
]
|
||||
|
||||
|
||||
def get_channel(name: str) -> Optional[Channel]:
|
||||
"""Get a channel by name."""
|
||||
for ch in ALL_CHANNELS:
|
||||
if ch.name == name:
|
||||
return ch
|
||||
return None
|
||||
|
||||
|
||||
def get_all_channels() -> List[Channel]:
|
||||
"""Get all registered channels."""
|
||||
return ALL_CHANNELS
|
||||
|
||||
|
||||
__all__ = [
|
||||
"Channel",
|
||||
"ALL_CHANNELS",
|
||||
"get_channel",
|
||||
"get_all_channels",
|
||||
]
|
||||
@@ -0,0 +1,48 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Shared channel helper for OpenCLI browser-session-only platforms."""
|
||||
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from .base import Channel
|
||||
|
||||
|
||||
class OpenCLISiteChannel(Channel):
|
||||
"""A platform served directly by OpenCLI.
|
||||
|
||||
These channels are intentionally thin: Agent Reach only installs,
|
||||
health-checks, and routes. Agents call `opencli <site> ...` directly.
|
||||
"""
|
||||
|
||||
site: str = ""
|
||||
domains: tuple[str, ...] = ()
|
||||
usage: str = ""
|
||||
login_hint: str = ""
|
||||
|
||||
backends = ["OpenCLI"]
|
||||
tier = 1
|
||||
|
||||
def can_handle(self, url: str) -> bool:
|
||||
domain = urlparse(url).netloc.lower()
|
||||
return any(domain == d or domain.endswith(f".{d}") for d in self.domains)
|
||||
|
||||
def check(self, config=None):
|
||||
from agent_reach.backends import opencli_status
|
||||
|
||||
self.active_backend = None
|
||||
st = opencli_status()
|
||||
if not st.installed:
|
||||
return "off", (
|
||||
f"未安装 {self.description} 后端。安装：\n"
|
||||
" agent-reach install --channels opencli\n"
|
||||
f"然后在 Chrome 里登录 {self.login_hint}"
|
||||
)
|
||||
if st.broken:
|
||||
return "error", st.hint
|
||||
|
||||
self.active_backend = "OpenCLI"
|
||||
if st.ready:
|
||||
return "ok", (
|
||||
f"OpenCLI 可用（复用浏览器登录态）。用法：{self.usage}。"
|
||||
f"若提示登录，请先在 Chrome 里登录 {self.login_hint}"
|
||||
)
|
||||
return "warn", st.hint
|
||||
@@ -0,0 +1,70 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
Channel base class — platform availability checking.
|
||||
|
||||
Each channel represents a platform (YouTube, Twitter, GitHub, etc.)
|
||||
and provides:
|
||||
- can_handle(url) → does this URL belong to this platform?
|
||||
- check(config) → is the upstream tool installed and configured?
|
||||
|
||||
After installation, agents call upstream tools directly.
|
||||
|
||||
Backend routing semantics:
|
||||
- `backends` is an ORDERED candidate list: backends[0] is the preferred
|
||||
backend, the rest are fallbacks. "Switching backends" for a platform
|
||||
means reordering this list (or a user override) — not rewriting code.
|
||||
- check() must set `self.active_backend` to the backend that is actually
|
||||
serving the channel right now (None when nothing usable is found).
|
||||
shutil.which() alone is NOT proof of health — a stale venv shim passes
|
||||
which() but cannot execute (see agent_reach.probe). Channels should
|
||||
really execute a lightweight command before claiming a backend active.
|
||||
- Users can force a backend with config key `<channel>_backend`
|
||||
(or env var `<CHANNEL>_BACKEND`); ordered_backends() applies it.
|
||||
"""
|
||||
|
||||
from abc import ABC, abstractmethod
|
||||
from typing import List, Optional, Tuple
|
||||
|
||||
|
||||
class Channel(ABC):
|
||||
"""Base class for all channels."""
|
||||
|
||||
name: str = "" # e.g. "youtube"
|
||||
description: str = "" # e.g. "YouTube 视频和字幕"
|
||||
backends: List[str] = [] # ordered candidates — backends[0] = preferred
|
||||
tier: int = 0 # 0=zero-config, 1=needs free key, 2=needs setup
|
||||
|
||||
#: Backend currently serving this channel; set by check(), None = unavailable.
|
||||
active_backend: Optional[str] = None
|
||||
|
||||
@abstractmethod
|
||||
def can_handle(self, url: str) -> bool:
|
||||
"""Check if this channel can handle this URL."""
|
||||
...
|
||||
|
||||
def ordered_backends(self, config=None) -> List[str]:
|
||||
"""Candidate backends in probe order, honoring the user override.
|
||||
|
||||
The config key `<channel>_backend` (env `<CHANNEL>_BACKEND`) moves the
|
||||
named backend to the front of the list; unknown values are ignored so
|
||||
a stale override can never hide working backends.
|
||||
"""
|
||||
candidates = list(self.backends)
|
||||
override = config.get(f"{self.name}_backend") if config else None
|
||||
if override:
|
||||
for i, b in enumerate(candidates):
|
||||
if b == override or b.startswith(override):
|
||||
candidates.insert(0, candidates.pop(i))
|
||||
break
|
||||
return candidates
|
||||
|
||||
def check(self, config=None) -> Tuple[str, str]:
|
||||
"""
|
||||
Check if this channel's upstream tool is available.
|
||||
Returns (status, message) where status is 'ok'/'warn'/'off'/'error'.
|
||||
|
||||
Subclasses with external backends must really probe them (see
|
||||
agent_reach.probe.probe_command) and set self.active_backend.
|
||||
"""
|
||||
self.active_backend = self.backends[0] if self.backends else "内置"
|
||||
return "ok", f"{'、'.join(self.backends) if self.backends else '内置'}"
|
||||
@@ -0,0 +1,119 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Bilibili — multi-backend: bili-cli / OpenCLI / search API.
|
||||
|
||||
yt-dlp was REMOVED from this channel (live-verified 2026-06): bilibili's
|
||||
risk control 412-blocks yt-dlp's requests in every configuration we
|
||||
tried — latest version, direct, proxied, with warmed cookies — while
|
||||
bili-cli keeps working (search/hot/video detail without login) and
|
||||
OpenCLI covers subtitles through the browser session. yt-dlp remains the
|
||||
YouTube backend; it just no longer serves bilibili.
|
||||
"""
|
||||
|
||||
import json
|
||||
import urllib.request
|
||||
|
||||
from agent_reach.probe import probe_command
|
||||
|
||||
from .base import Channel
|
||||
|
||||
_UA = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36"
|
||||
_TIMEOUT = 10
|
||||
_SEARCH_API = "https://api.bilibili.com/x/web-interface/search/all/v2?keyword=test&page=1"
|
||||
|
||||
|
||||
def _search_api_ok() -> bool:
|
||||
"""Return True if Bilibili search API responds with code 0."""
|
||||
req = urllib.request.Request(_SEARCH_API, headers={"User-Agent": _UA})
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=_TIMEOUT) as resp:
|
||||
data = json.loads(resp.read())
|
||||
return data.get("code") == 0
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
class BilibiliChannel(Channel):
|
||||
name = "bilibili"
|
||||
description = "B站视频、字幕和搜索"
|
||||
backends = ["bili-cli", "OpenCLI", "B站搜索 API"]
|
||||
tier = 1
|
||||
|
||||
def can_handle(self, url: str) -> bool:
|
||||
from urllib.parse import urlparse
|
||||
d = urlparse(url).netloc.lower()
|
||||
return "bilibili.com" in d or "b23.tv" in d
|
||||
|
||||
def check(self, config=None):
|
||||
"""Probe candidates in order; first fully-usable backend wins."""
|
||||
self.active_backend = None
|
||||
findings = []
|
||||
|
||||
for backend in self.ordered_backends(config):
|
||||
if backend == "bili-cli":
|
||||
result = self._check_bili_cli()
|
||||
elif backend == "OpenCLI":
|
||||
result = self._check_opencli()
|
||||
else:
|
||||
result = self._check_search_api()
|
||||
if result is None:
|
||||
continue
|
||||
findings.append((backend, *result))
|
||||
|
||||
# 有后端断链时，即使别的候选兜底成功也要把处方带出来
|
||||
broken_notes = [m for _, s, m in findings if s == "error"]
|
||||
|
||||
for wanted in ("ok", "warn"):
|
||||
for backend, status, message in findings:
|
||||
if status == wanted:
|
||||
self.active_backend = backend
|
||||
if broken_notes:
|
||||
message += "\n[备选后端异常] " + "；".join(broken_notes)
|
||||
return status, message
|
||||
|
||||
if findings:
|
||||
return "error", "\n".join(m for _, _, m in findings)
|
||||
|
||||
return "off", (
|
||||
"没有可用的 B站后端（搜索 API 也不可达，可能是网络问题）。推荐：\n"
|
||||
" pipx install bilibili-cli（搜索/热门/视频详情，无需登录）\n"
|
||||
" 或桌面装 OpenCLI（额外解锁字幕）：agent-reach install --channels opencli"
|
||||
)
|
||||
|
||||
def _check_bili_cli(self):
|
||||
"""bili-cli candidate. None = not installed."""
|
||||
probe = probe_command("bili", ["--version"], timeout=10, package="bilibili-cli")
|
||||
if probe.status == "missing":
|
||||
return None
|
||||
if probe.status == "broken":
|
||||
return "error", "bili 命令存在但无法执行\n" + probe.hint
|
||||
if not probe.ok:
|
||||
return "warn", f"bili-cli 探测失败（{probe.status}），运行 `bili status` 查看详情"
|
||||
return "ok", (
|
||||
"bili-cli 可用（搜索/热门/排行/视频详情/音频，无需登录；"
|
||||
"字幕需 OpenCLI。上游 2026-03 起停更）"
|
||||
)
|
||||
|
||||
def _check_opencli(self):
|
||||
"""OpenCLI candidate. None = not installed."""
|
||||
from agent_reach.backends import opencli_status
|
||||
|
||||
st = opencli_status()
|
||||
if not st.installed:
|
||||
return None
|
||||
if st.broken:
|
||||
return "error", st.hint
|
||||
if st.ready:
|
||||
return "ok", (
|
||||
"OpenCLI 可用（复用浏览器登录态）。用法："
|
||||
"opencli bilibili search/video/subtitle/ranking -f yaml"
|
||||
)
|
||||
return "warn", st.hint
|
||||
|
||||
def _check_search_api(self):
|
||||
"""Zero-dependency search API fallback. None = unreachable."""
|
||||
if not _search_api_ok():
|
||||
return None
|
||||
return "ok", (
|
||||
"B站搜索 API 可达（仅搜索，curl 直连）。"
|
||||
"完整功能建议安装 bili-cli：pipx install bilibili-cli"
|
||||
)
|
||||
@@ -0,0 +1,40 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Exa Search — check if mcporter + Exa MCP is available."""
|
||||
|
||||
from agent_reach.probe import probe_command
|
||||
|
||||
from .base import Channel
|
||||
|
||||
#: mcporter 是 npm 包，断链处方与默认的 pipx/uv 不同
|
||||
_MCPORTER_BROKEN_HINT = "mcporter 无法执行（node 环境损坏），重装：\n npm install -g mcporter"
|
||||
|
||||
|
||||
class ExaSearchChannel(Channel):
|
||||
name = "exa_search"
|
||||
description = "全网语义搜索"
|
||||
backends = ["Exa via mcporter"]
|
||||
tier = 0
|
||||
|
||||
def can_handle(self, url: str) -> bool:
|
||||
return False # Search-only channel
|
||||
|
||||
def check(self, config=None):
|
||||
self.active_backend = None
|
||||
probe = probe_command("mcporter", ["config", "list"], timeout=10, package="mcporter")
|
||||
if probe.status == "missing":
|
||||
return "off", (
|
||||
"需要 mcporter + Exa MCP。安装：\n"
|
||||
" npm install -g mcporter\n"
|
||||
" mcporter config add exa https://mcp.exa.ai/mcp"
|
||||
)
|
||||
if probe.status == "broken":
|
||||
return "error", _MCPORTER_BROKEN_HINT
|
||||
if not probe.ok: # timeout / error
|
||||
return "error", f"mcporter 执行异常：{probe.hint or probe.output or probe.status}"
|
||||
if "exa" in probe.output.lower():
|
||||
self.active_backend = self.backends[0]
|
||||
return "ok", "全网语义搜索可用（免费，无需 API Key）"
|
||||
return "off", (
|
||||
"mcporter 已装但 Exa 未配置。运行：\n"
|
||||
" mcporter config add exa https://mcp.exa.ai/mcp"
|
||||
)
|
||||
@@ -0,0 +1,13 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Facebook — OpenCLI backend using the user's logged-in Chrome session."""
|
||||
|
||||
from ._opencli_site import OpenCLISiteChannel
|
||||
|
||||
|
||||
class FacebookChannel(OpenCLISiteChannel):
|
||||
name = "facebook"
|
||||
description = "Facebook 帖子、主页和群组"
|
||||
site = "facebook"
|
||||
domains = ("facebook.com", "fb.com", "fb.watch")
|
||||
usage = "opencli facebook search/profile/feed/groups -f yaml"
|
||||
login_hint = "facebook.com"
|
||||
@@ -0,0 +1,42 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""GitHub — check if gh CLI is available."""
|
||||
|
||||
from agent_reach.probe import probe_command
|
||||
|
||||
from .base import Channel
|
||||
|
||||
|
||||
class GitHubChannel(Channel):
|
||||
name = "github"
|
||||
description = "GitHub 仓库和代码"
|
||||
backends = ["gh CLI"]
|
||||
tier = 0
|
||||
|
||||
def can_handle(self, url: str) -> bool:
|
||||
from urllib.parse import urlparse
|
||||
return "github.com" in urlparse(url).netloc.lower()
|
||||
|
||||
def check(self, config=None):
|
||||
# 真跑 gh auth status 探活。注意：未登录时 rc!=0 是正常业务态（warn），不是 error。
|
||||
probe = probe_command("gh", ["auth", "status"], timeout=10, package="gh")
|
||||
if probe.status == "missing":
|
||||
self.active_backend = None
|
||||
return "warn", "gh CLI 未安装。安装：https://cli.github.com"
|
||||
if probe.status == "broken":
|
||||
# gh 是二进制安装（brew/官方包），不是 pip 包——处方不用 pipx/uv 文案
|
||||
self.active_backend = None
|
||||
return "error", (
|
||||
"gh 命令存在但无法执行——安装已损坏。重装即可修复：\n"
|
||||
" brew reinstall gh\n"
|
||||
"或从 https://cli.github.com 重新安装 gh CLI"
|
||||
)
|
||||
if probe.status == "timeout":
|
||||
# gh 本体能启动（工具是活的），只是状态检查超时
|
||||
self.active_backend = "gh CLI"
|
||||
return "warn", "gh CLI 状态检查超时，运行 gh auth status 查看详情"
|
||||
if probe.ok:
|
||||
self.active_backend = "gh CLI"
|
||||
return "ok", "完整可用（读取、搜索、Fork、Issue、PR 等）"
|
||||
# rc != 0：gh 活着但未认证（gh auth status 的正常业务态）
|
||||
self.active_backend = "gh CLI"
|
||||
return "warn", "gh CLI 已安装但未认证。运行 gh auth login 可解锁完整功能"
|
||||
@@ -0,0 +1,13 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Instagram — OpenCLI backend using the user's logged-in Chrome session."""
|
||||
|
||||
from ._opencli_site import OpenCLISiteChannel
|
||||
|
||||
|
||||
class InstagramChannel(OpenCLISiteChannel):
|
||||
name = "instagram"
|
||||
description = "Instagram 用户、主页和指定用户帖子"
|
||||
site = "instagram"
|
||||
domains = ("instagram.com", "instagr.am")
|
||||
usage = "opencli instagram search/profile/user/explore -f yaml"
|
||||
login_hint = "instagram.com"
|
||||
@@ -0,0 +1,43 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""LinkedIn — check if linkedin-scraper-mcp is available."""
|
||||
|
||||
from agent_reach.probe import probe_command
|
||||
|
||||
from .base import Channel
|
||||
|
||||
#: mcporter 是 npm 包，断链处方与默认的 pipx/uv 不同
|
||||
_MCPORTER_BROKEN_HINT = "mcporter 无法执行（node 环境损坏），重装：\n npm install -g mcporter"
|
||||
|
||||
|
||||
class LinkedInChannel(Channel):
|
||||
name = "linkedin"
|
||||
description = "LinkedIn 职业社交"
|
||||
backends = ["linkedin-scraper-mcp", "Jina Reader"]
|
||||
tier = 2
|
||||
|
||||
def can_handle(self, url: str) -> bool:
|
||||
from urllib.parse import urlparse
|
||||
return "linkedin.com" in urlparse(url).netloc.lower()
|
||||
|
||||
def check(self, config=None):
|
||||
self.active_backend = None
|
||||
probe = probe_command("mcporter", ["config", "list"], timeout=10, package="mcporter")
|
||||
if probe.status == "missing":
|
||||
return "off", (
|
||||
"基本内容可通过 Jina Reader 读取。完整功能需要：\n"
|
||||
" pip install linkedin-scraper-mcp\n"
|
||||
" mcporter config add linkedin http://localhost:3000/mcp\n"
|
||||
" 详见 https://github.com/stickerdaniel/linkedin-mcp-server"
|
||||
)
|
||||
if probe.status == "broken":
|
||||
return "error", _MCPORTER_BROKEN_HINT
|
||||
if not probe.ok: # timeout / error
|
||||
return "error", f"mcporter 执行异常：{probe.hint or probe.output or probe.status}"
|
||||
if "linkedin" in probe.output.lower():
|
||||
self.active_backend = "linkedin-scraper-mcp"
|
||||
return "ok", "完整可用（Profile、公司、职位搜索）"
|
||||
return "off", (
|
||||
"mcporter 已装但 LinkedIn MCP 未配置。运行：\n"
|
||||
" pip install linkedin-scraper-mcp\n"
|
||||
" mcporter config add linkedin http://localhost:3000/mcp"
|
||||
)
|
||||
@@ -0,0 +1,165 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Reddit — multi-backend: OpenCLI / rdt-cli. Login is mandatory.
|
||||
|
||||
Honest tiering (live-verified 2026-06): there is NO zero-config path.
|
||||
Anonymous .json endpoints are blocked (403 anti-bot, all variants), and
|
||||
the official API closed self-service registration in 2025-11 (manual
|
||||
approval, individual scripts rarely granted — PRAW is only an option for
|
||||
users who already hold credentials). Every working backend rides a
|
||||
logged-in session: OpenCLI reuses the browser's, rdt-cli imports cookies.
|
||||
"""
|
||||
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
from agent_reach.utils.process import utf8_subprocess_env
|
||||
|
||||
from .base import Channel
|
||||
|
||||
_CREDENTIAL_FILE = "~/.config/rdt-cli/credential.json"
|
||||
# Pinned to the 0.4.2 state — PyPI still only has 0.4.1 (upstream issue #10).
|
||||
_RDT_GIT_SOURCE = "git+https://github.com/public-clis/rdt-cli.git@5e4fb3720d5c174e976cd425ccc3b879d52cac66"
|
||||
|
||||
#: shell 对"找到但不可执行/找不到"使用的退出码（对齐 agent_reach.probe）
|
||||
_BROKEN_EXIT_CODES = (126, 127)
|
||||
|
||||
#: rdt 应从固定 git 源安装（PyPI 落后），断链处方与 probe 默认的 pipx/uv 不同
|
||||
_RDT_BROKEN_HINT = (
|
||||
"rdt 命令存在但无法执行——通常是系统 Python 升级后 venv 解释器丢失。\n"
|
||||
"PyPI 版本落后，推荐用固定 git 源强制重装：\n"
|
||||
f" pipx install --force '{_RDT_GIT_SOURCE}'"
|
||||
)
|
||||
|
||||
|
||||
class RedditChannel(Channel):
|
||||
name = "reddit"
|
||||
description = "Reddit 帖子和评论"
|
||||
backends = ["OpenCLI", "rdt-cli"]
|
||||
tier = 1 # no zero-config path exists — see module docstring
|
||||
|
||||
def can_handle(self, url: str) -> bool:
|
||||
from urllib.parse import urlparse
|
||||
|
||||
d = urlparse(url).netloc.lower()
|
||||
return "reddit.com" in d or "redd.it" in d
|
||||
|
||||
def check(self, config=None):
|
||||
"""Probe candidates in order; first fully-usable backend wins."""
|
||||
self.active_backend = None
|
||||
findings = []
|
||||
|
||||
for backend in self.ordered_backends(config):
|
||||
if backend == "OpenCLI":
|
||||
result = self._check_opencli()
|
||||
else:
|
||||
result = self._check_rdt()
|
||||
if result is None:
|
||||
continue
|
||||
findings.append((backend, *result))
|
||||
|
||||
for wanted in ("ok", "warn"):
|
||||
for backend, status, message in findings:
|
||||
if status == wanted:
|
||||
self.active_backend = backend
|
||||
return status, message
|
||||
|
||||
if findings:
|
||||
return "error", "\n".join(m for _, _, m in findings)
|
||||
|
||||
return "off", (
|
||||
"未安装任何 Reddit 后端。注意：Reddit 没有零配置路径"
|
||||
"（匿名 .json 已被封，官方 API 需人工审批），必须用登录态。推荐：\n"
|
||||
" 桌面：agent-reach install --channels opencli\n"
|
||||
" （复用 Chrome 登录态，登录过 reddit.com 即可用）\n"
|
||||
f" 服务器/存量：pipx install '{_RDT_GIT_SOURCE}'\n"
|
||||
" 然后 `rdt login` 或手动写入 Cookie（见 doctor 提示）\n"
|
||||
"中国大陆访问 Reddit 需要代理"
|
||||
)
|
||||
|
||||
def _check_opencli(self):
|
||||
"""OpenCLI candidate. None = not installed."""
|
||||
from agent_reach.backends import opencli_status
|
||||
|
||||
st = opencli_status()
|
||||
if not st.installed:
|
||||
return None
|
||||
if st.broken:
|
||||
return "error", st.hint
|
||||
if st.ready:
|
||||
return "ok", (
|
||||
"OpenCLI 可用（复用浏览器登录态）。用法："
|
||||
"opencli reddit search/read/subreddit/hot -f yaml"
|
||||
)
|
||||
return "warn", st.hint
|
||||
|
||||
def _check_rdt(self):
|
||||
"""rdt-cli candidate. None = not installed."""
|
||||
rdt = shutil.which("rdt")
|
||||
if not rdt:
|
||||
return None
|
||||
|
||||
# 不走 probe_command：实测 `rdt status --json` 成功时（rc=0）也会向 stderr
|
||||
# 打网络重试日志，probe 把 stdout+stderr 合并后 JSON 解析必炸。
|
||||
# 故保留手写 subprocess（stdout 单独捕获），但异常分类对齐 probe 语义：
|
||||
# exec 失败/126/127 → broken（venv 断链处方），TimeoutExpired → 超时。
|
||||
try:
|
||||
r = subprocess.run(
|
||||
[rdt, "status", "--json"],
|
||||
capture_output=True,
|
||||
encoding="utf-8",
|
||||
errors="replace",
|
||||
timeout=10,
|
||||
env=utf8_subprocess_env(),
|
||||
)
|
||||
except subprocess.TimeoutExpired:
|
||||
return "error", "rdt 响应超时（>10s），Reddit 状态未知。稍后重试或运行 `rdt status` 查看详情"
|
||||
except OSError:
|
||||
# 含 FileNotFoundError：which 命中但 exec 失败 = venv 断链（probe 的 broken）
|
||||
return "error", _RDT_BROKEN_HINT
|
||||
|
||||
if r.returncode in _BROKEN_EXIT_CODES:
|
||||
return "error", _RDT_BROKEN_HINT
|
||||
|
||||
if r.returncode != 0:
|
||||
detail = (r.stderr or r.stdout or "").strip().splitlines()
|
||||
tail = detail[-1] if detail else "无输出"
|
||||
return "error", f"rdt 异常退出（exit {r.returncode}）：{tail}。运行 `rdt status` 查看详情"
|
||||
|
||||
# 进程正常退出 → rdt 本身是活的（无论登录与否）
|
||||
try:
|
||||
data = json.loads(r.stdout or "")
|
||||
except json.JSONDecodeError:
|
||||
data = None
|
||||
if not isinstance(data, dict):
|
||||
return "warn", "rdt-cli 可用但状态输出无法解析，运行 `rdt status` 查看登录状态"
|
||||
|
||||
info = data.get("data")
|
||||
if not isinstance(info, dict):
|
||||
info = {}
|
||||
authenticated = info.get("authenticated", False)
|
||||
username = info.get("username") or ""
|
||||
|
||||
if authenticated:
|
||||
suffix = f"（已登录：{username}）" if username else ""
|
||||
return "ok", (
|
||||
f"rdt-cli 可用{suffix}（搜索帖子、阅读全文、查看评论；"
|
||||
"上游 2026-03 起停更，桌面用户建议迁移到 OpenCLI）"
|
||||
)
|
||||
|
||||
return "warn", (
|
||||
"rdt-cli 已安装但未登录。Reddit 自 2024 年起要求认证，"
|
||||
"未登录时所有请求均返回 403。\n\n"
|
||||
"方法一（自动）：运行 `rdt login`\n"
|
||||
" 先在浏览器登录 reddit.com，再运行此命令自动提取 Cookie。\n\n"
|
||||
"方法二（手动，适用于 Chrome/Edge 127+ 无法自动提取时）：\n"
|
||||
" 1. Chrome 应用商店安装 Cookie-Editor 扩展：\n"
|
||||
" https://chromewebstore.google.com/detail/cookie-editor/hlkenndednhfkekhgcdicdfddnkalmdm\n"
|
||||
" 2. 在浏览器打开 reddit.com（确保已登录）\n"
|
||||
" 3. 点击 Cookie-Editor 图标，找到 `reddit_session`，复制其 Value\n"
|
||||
f" 4. 将以下内容写入 {_CREDENTIAL_FILE}：\n"
|
||||
' {"cookies": {"reddit_session": "<粘贴 Value>"}, '
|
||||
'"source": "manual", "username": "<你的用户名>", '
|
||||
'"modhash": null, "saved_at": 0, "last_verified_at": null}\n\n'
|
||||
"验证：`rdt status --json` 确认 authenticated: true"
|
||||
)
|
||||
@@ -0,0 +1,27 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""RSS — check if feedparser is available."""
|
||||
|
||||
from .base import Channel
|
||||
|
||||
|
||||
class RSSChannel(Channel):
|
||||
name = "rss"
|
||||
description = "RSS/Atom 订阅源"
|
||||
backends = ["feedparser"]
|
||||
tier = 0
|
||||
|
||||
def can_handle(self, url: str) -> bool:
|
||||
return any(x in url.lower() for x in ["/feed", "/rss", ".xml", "atom"])
|
||||
|
||||
def check(self, config=None):
|
||||
try:
|
||||
import feedparser # noqa: F401
|
||||
except ImportError:
|
||||
self.active_backend = None
|
||||
return "off", "feedparser 未安装。安装：pip install feedparser"
|
||||
except Exception as e:
|
||||
# 已安装但导入期崩溃（半残安装/版本冲突）→ 重装处方
|
||||
self.active_backend = None
|
||||
return "error", f"feedparser 导入失败：{e}\n修复：pip install --force-reinstall feedparser"
|
||||
self.active_backend = self.backends[0]
|
||||
return "ok", "可读取 RSS/Atom 源"
|
||||
@@ -0,0 +1,145 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Twitter/X — check if twitter-cli or bird CLI is available."""
|
||||
|
||||
from .base import Channel
|
||||
from agent_reach.probe import probe_command
|
||||
|
||||
|
||||
class TwitterChannel(Channel):
|
||||
name = "twitter"
|
||||
description = "Twitter/X 推文"
|
||||
backends = ["twitter-cli", "OpenCLI", "bird CLI (legacy)"]
|
||||
tier = 1
|
||||
|
||||
def can_handle(self, url: str) -> bool:
|
||||
from urllib.parse import urlparse
|
||||
d = urlparse(url).netloc.lower()
|
||||
return "x.com" in d or "twitter.com" in d
|
||||
|
||||
def check(self, config=None):
|
||||
"""Probe candidates in order; first fully-usable backend wins.
|
||||
|
||||
与其他多后端渠道同一套两段式：先收集全部候选状态，第一个 ok 获胜；
|
||||
没有 ok 才轮到第一个 warn——否则「装了但未登录」的 twitter-cli
|
||||
会把排在后面、完整可用的 OpenCLI 挡在门外。
|
||||
"""
|
||||
self.active_backend = None
|
||||
findings = []
|
||||
|
||||
for backend in self.ordered_backends(config):
|
||||
if backend == "twitter-cli":
|
||||
result = self._check_twitter_cli()
|
||||
elif backend == "OpenCLI":
|
||||
result = self._check_opencli()
|
||||
elif backend == "bird CLI (legacy)":
|
||||
result = self._check_bird()
|
||||
else:
|
||||
continue
|
||||
|
||||
if result is None:
|
||||
continue # 未安装——不参与候选
|
||||
findings.append((backend, *result))
|
||||
|
||||
for wanted in ("ok", "warn"):
|
||||
for backend, status, message in findings:
|
||||
if status == wanted:
|
||||
self.active_backend = backend
|
||||
return status, message
|
||||
|
||||
if findings: # 只剩 broken/timeout 候选
|
||||
return "error", "\n".join(m for _, _, m in findings)
|
||||
|
||||
return "warn", (
|
||||
"Twitter CLI 未安装。安装方式：\n"
|
||||
" pipx install twitter-cli\n"
|
||||
"或：\n"
|
||||
" uv tool install twitter-cli"
|
||||
)
|
||||
|
||||
def _check_twitter_cli(self):
|
||||
"""探测 twitter-cli。返回 None 表示未安装，否则返回 (status, message)。
|
||||
|
||||
`twitter status` 才是健康信号：已登录时输出 "ok: true"，
|
||||
未登录时以非零退出码输出 "not_authenticated"——工具本身是活的，
|
||||
所以 probe 的 error 状态也要看 output 内容再分类。
|
||||
"""
|
||||
probe = probe_command(
|
||||
"twitter", ["status"], timeout=15, retries=1, package="twitter-cli"
|
||||
)
|
||||
if probe.status == "missing":
|
||||
return None
|
||||
if probe.status == "broken":
|
||||
return "error", "twitter-cli 命令存在但无法执行。\n" + probe.hint
|
||||
if probe.status == "timeout":
|
||||
return "error", "twitter-cli 健康检查超时（已重试 1 次）。\n" + probe.hint
|
||||
|
||||
output = probe.output
|
||||
if "ok: true" in output:
|
||||
return "ok", (
|
||||
"twitter-cli 完整可用（搜索、读推文、时间线、长文/Article、"
|
||||
"用户查询、Thread）"
|
||||
)
|
||||
if "not_authenticated" in output:
|
||||
return "warn", (
|
||||
"twitter-cli 已安装但未认证。设置方式：\n"
|
||||
" export TWITTER_AUTH_TOKEN=\"xxx\"\n"
|
||||
" export TWITTER_CT0=\"yyy\"\n"
|
||||
"或确保已在浏览器中登录 x.com"
|
||||
)
|
||||
return "warn", (
|
||||
"twitter-cli 已安装但认证检查失败。运行：\n"
|
||||
" twitter -v status 查看详细信息"
|
||||
)
|
||||
|
||||
def _check_opencli(self):
|
||||
"""OpenCLI candidate. None = not installed."""
|
||||
from agent_reach.backends import opencli_status
|
||||
|
||||
st = opencli_status()
|
||||
if not st.installed:
|
||||
return None
|
||||
if st.broken:
|
||||
return "error", st.hint
|
||||
if st.ready:
|
||||
return "ok", (
|
||||
"OpenCLI 可用（复用浏览器登录态）。用法："
|
||||
"opencli twitter search/article/user-posts -f yaml"
|
||||
)
|
||||
return "warn", st.hint
|
||||
|
||||
def _check_bird(self):
|
||||
"""探测 bird/birdx（legacy 回退）。返回 None 表示均未安装，否则返回 (status, message)。"""
|
||||
last_failure = None
|
||||
for cmd in ("bird", "birdx"):
|
||||
probe = probe_command(
|
||||
cmd, ["check"], timeout=15, retries=1, package="@steipete/bird"
|
||||
)
|
||||
if probe.status == "missing":
|
||||
continue
|
||||
if probe.status == "broken":
|
||||
last_failure = (
|
||||
"error",
|
||||
f"{cmd} 命令存在但无法执行（bird 是 npm 包，可用 "
|
||||
"npm install -g @steipete/bird 重装）。\n" + probe.hint,
|
||||
)
|
||||
continue # bird 坏了再试 birdx
|
||||
if probe.status == "timeout":
|
||||
last_failure = (
|
||||
"error",
|
||||
f"{cmd} 健康检查超时（已重试 1 次）。\n" + probe.hint,
|
||||
)
|
||||
continue
|
||||
|
||||
output = probe.output
|
||||
if probe.ok:
|
||||
return "ok", "bird CLI 可用（读取、搜索推文，含长文/X Article）"
|
||||
if "Missing credentials" in output or "missing" in output.lower():
|
||||
return "warn", (
|
||||
"bird CLI 已安装但未配置认证。设置环境变量：\n"
|
||||
" export AUTH_TOKEN=\"xxx\"\n"
|
||||
" export CT0=\"yyy\""
|
||||
)
|
||||
return "warn", (
|
||||
"bird CLI 已安装但认证检查失败。"
|
||||
)
|
||||
return last_failure
|
||||
@@ -0,0 +1,214 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""V2EX — public API channel for topics, nodes, users, and replies."""
|
||||
|
||||
import json
|
||||
import urllib.request
|
||||
from typing import Any
|
||||
from .base import Channel
|
||||
|
||||
_UA = "agent-reach/1.0"
|
||||
_TIMEOUT = 10
|
||||
|
||||
|
||||
def _get_json(url: str) -> Any:
|
||||
"""Fetch *url* and return parsed JSON. Raises on HTTP/network errors."""
|
||||
req = urllib.request.Request(url, headers={"User-Agent": _UA})
|
||||
with urllib.request.urlopen(req, timeout=_TIMEOUT) as resp:
|
||||
return json.loads(resp.read().decode("utf-8"))
|
||||
|
||||
|
||||
class V2EXChannel(Channel):
|
||||
name = "v2ex"
|
||||
description = "V2EX 节点、主题与回复"
|
||||
backends = ["V2EX API (public)"]
|
||||
tier = 0
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# URL routing
|
||||
# ------------------------------------------------------------------ #
|
||||
|
||||
def can_handle(self, url: str) -> bool:
|
||||
from urllib.parse import urlparse
|
||||
d = urlparse(url).netloc.lower()
|
||||
return "v2ex.com" in d
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# Health check
|
||||
# ------------------------------------------------------------------ #
|
||||
|
||||
def check(self, config=None):
|
||||
try:
|
||||
_get_json(
|
||||
"https://www.v2ex.com/api/topics/show.json?node_name=python&page=1"
|
||||
)
|
||||
self.active_backend = self.backends[0]
|
||||
return "ok", "公开 API 可用（热门主题、节点浏览、主题详情、用户信息）"
|
||||
except Exception as e:
|
||||
self.active_backend = None
|
||||
return "warn", f"V2EX API 连接失败（可能需要代理）：{e}"
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# Data-fetching methods
|
||||
# ------------------------------------------------------------------ #
|
||||
|
||||
def get_hot_topics(self, limit: int = 20) -> list:
|
||||
"""获取热门帖子列表。
|
||||
|
||||
Returns a list of dicts with keys:
|
||||
title, url, replies, node_name, node_title, content
|
||||
"""
|
||||
data = _get_json("https://www.v2ex.com/api/topics/hot.json")
|
||||
results = []
|
||||
for item in data[:limit]:
|
||||
node = item.get("node") or {}
|
||||
content = item.get("content", "") or ""
|
||||
results.append(
|
||||
{
|
||||
"id": item.get("id", 0),
|
||||
"title": item.get("title", ""),
|
||||
"url": item.get("url", ""),
|
||||
"replies": item.get("replies", 0),
|
||||
"node_name": node.get("name", ""),
|
||||
"node_title": node.get("title", ""),
|
||||
"content": content[:200],
|
||||
"created": item.get("created", 0),
|
||||
}
|
||||
)
|
||||
return results
|
||||
|
||||
def get_node_topics(self, node_name: str, limit: int = 20) -> list:
|
||||
"""获取指定节点的最新帖子。
|
||||
|
||||
Args:
|
||||
node_name: 节点名称，如 "python"、"tech"、"jobs"
|
||||
limit: 最多返回条数
|
||||
|
||||
Returns a list of dicts with keys:
|
||||
title, url, replies, node_name, node_title, content
|
||||
"""
|
||||
url = (
|
||||
f"https://www.v2ex.com/api/topics/show.json"
|
||||
f"?node_name={node_name}&page=1"
|
||||
)
|
||||
data = _get_json(url)
|
||||
results = []
|
||||
for item in data[:limit]:
|
||||
node = item.get("node") or {}
|
||||
content = item.get("content", "") or ""
|
||||
results.append(
|
||||
{
|
||||
"id": item.get("id", 0),
|
||||
"title": item.get("title", ""),
|
||||
"url": item.get("url", ""),
|
||||
"replies": item.get("replies", 0),
|
||||
"node_name": node.get("name", node_name),
|
||||
"node_title": node.get("title", ""),
|
||||
"content": content[:200],
|
||||
"created": item.get("created", 0),
|
||||
}
|
||||
)
|
||||
return results
|
||||
|
||||
def get_topic(self, topic_id: int) -> dict:
|
||||
"""获取单个帖子详情和回复列表。
|
||||
|
||||
Args:
|
||||
topic_id: 帖子 ID（从 URL https://www.v2ex.com/t/<id> 中获取）
|
||||
|
||||
Returns a dict with keys:
|
||||
id, title, url, content, replies_count, node_name, node_title,
|
||||
author, created, replies (list of dicts with: author, content, created)
|
||||
"""
|
||||
topic_data = _get_json(
|
||||
f"https://www.v2ex.com/api/topics/show.json?id={topic_id}"
|
||||
)
|
||||
# API returns a list even for single-ID queries
|
||||
if isinstance(topic_data, list):
|
||||
topic = topic_data[0] if topic_data else {}
|
||||
else:
|
||||
topic = topic_data
|
||||
|
||||
node = topic.get("node") or {}
|
||||
member = topic.get("member") or {}
|
||||
|
||||
# Fetch replies (first page)
|
||||
try:
|
||||
replies_raw = _get_json(
|
||||
f"https://www.v2ex.com/api/replies/show.json"
|
||||
f"?topic_id={topic_id}&page=1"
|
||||
)
|
||||
except Exception:
|
||||
replies_raw = []
|
||||
|
||||
replies = [
|
||||
{
|
||||
"author": (r.get("member") or {}).get("username", ""),
|
||||
"content": r.get("content", ""),
|
||||
"created": r.get("created", 0),
|
||||
}
|
||||
for r in (replies_raw or [])
|
||||
]
|
||||
|
||||
return {
|
||||
"id": topic.get("id", topic_id),
|
||||
"title": topic.get("title", ""),
|
||||
"url": topic.get("url", f"https://www.v2ex.com/t/{topic_id}"),
|
||||
"content": topic.get("content", ""),
|
||||
"replies_count": topic.get("replies", 0),
|
||||
"node_name": node.get("name", ""),
|
||||
"node_title": node.get("title", ""),
|
||||
"author": member.get("username", ""),
|
||||
"created": topic.get("created", 0),
|
||||
"replies": replies,
|
||||
}
|
||||
|
||||
def get_user(self, username: str) -> dict:
|
||||
"""获取用户信息。
|
||||
|
||||
Args:
|
||||
username: V2EX 用户名
|
||||
|
||||
Returns a dict with keys:
|
||||
id, username, url, website, twitter, psn, github, btc,
|
||||
location, bio, avatar, created
|
||||
"""
|
||||
data = _get_json(
|
||||
f"https://www.v2ex.com/api/members/show.json?username={username}"
|
||||
)
|
||||
return {
|
||||
"id": data.get("id", 0),
|
||||
"username": data.get("username", username),
|
||||
"url": data.get("url", f"https://www.v2ex.com/member/{username}"),
|
||||
"website": data.get("website", ""),
|
||||
"twitter": data.get("twitter", ""),
|
||||
"psn": data.get("psn", ""),
|
||||
"github": data.get("github", ""),
|
||||
"btc": data.get("btc", ""),
|
||||
"location": data.get("location", ""),
|
||||
"bio": data.get("bio", ""),
|
||||
"avatar": data.get("avatar_large", data.get("avatar_normal", "")),
|
||||
"created": data.get("created", 0),
|
||||
}
|
||||
|
||||
def search(self, query: str, limit: int = 10) -> list:
|
||||
"""搜索帖子。
|
||||
|
||||
注意：V2EX 公开 API 暂不支持全文搜索端点（/api/search.json 不可用）。
|
||||
本方法通过 Jina Reader 代理 V2EX 站内搜索页面获取结果（纯文本，无结构化数据）。
|
||||
|
||||
如需精确搜索，建议直接访问 https://www.v2ex.com/?q=<query> 或
|
||||
使用 Exa channel 的 site:v2ex.com 搜索。
|
||||
|
||||
Returns:
|
||||
list of dicts with keys: title, url, snippet
|
||||
如果搜索不可用，返回包含单条 {"error": str} 的列表。
|
||||
"""
|
||||
return [
|
||||
{
|
||||
"error": (
|
||||
"V2EX 公开 API 不提供搜索端点。"
|
||||
f"建议改用：https://www.v2ex.com/?q={query} "
|
||||
"或通过 Exa channel 使用 site:v2ex.com 搜索。"
|
||||
)
|
||||
}
|
||||
]
|
||||
@@ -0,0 +1,34 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Web — any URL via Jina Reader. Always available."""
|
||||
|
||||
import urllib.request
|
||||
from .base import Channel
|
||||
|
||||
_UA = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36"
|
||||
|
||||
|
||||
class WebChannel(Channel):
|
||||
name = "web"
|
||||
description = "任意网页"
|
||||
backends = ["Jina Reader"]
|
||||
tier = 0
|
||||
|
||||
def can_handle(self, url: str) -> bool:
|
||||
return True # Fallback — handles any URL
|
||||
|
||||
def check(self, config=None):
|
||||
# 恒可用兜底渠道：无本地命令、不做网络探测（doctor 已有多个渠道触网），保持零开销
|
||||
self.active_backend = self.backends[0]
|
||||
return "ok", "通过 Jina Reader 读取任意网页（curl https://r.jina.ai/URL）"
|
||||
|
||||
def read(self, url: str) -> str:
|
||||
"""通过 Jina Reader 读取网页，返回 Markdown 全文。"""
|
||||
if not url.startswith(("http://", "https://")):
|
||||
url = "https://" + url
|
||||
jina_url = f"https://r.jina.ai/{url}"
|
||||
req = urllib.request.Request(
|
||||
jina_url,
|
||||
headers={"User-Agent": _UA, "Accept": "text/plain"},
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=30) as resp:
|
||||
return resp.read().decode("utf-8")
|
||||
@@ -0,0 +1,258 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""XiaoHongShu — multi-backend: OpenCLI / xiaohongshu-mcp / xhs-cli.
|
||||
|
||||
Backend order encodes the recommendation, and probing order makes the
|
||||
environment split automatic: OpenCLI needs a desktop Chrome so it simply
|
||||
never probes alive on a server, where xiaohongshu-mcp (self-contained
|
||||
headless browser) takes over. xhs-cli (upstream unmaintained since
|
||||
2026-03) keeps working for existing installs as the last candidate.
|
||||
"""
|
||||
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
from agent_reach.probe import probe_command
|
||||
|
||||
from .base import Channel
|
||||
|
||||
_MCP_ENDPOINT = "http://localhost:18060/mcp"
|
||||
_MCP_INSTALL_URL = "https://github.com/xpzouying/xiaohongshu-mcp"
|
||||
|
||||
|
||||
def _mcp_service_reachable(timeout: int = 3) -> bool:
|
||||
"""True if the xiaohongshu-mcp HTTP service answers on localhost.
|
||||
|
||||
Any HTTP response counts (the MCP endpoint replies 405 to GET) —
|
||||
we only care that the service is up. Proxies are bypassed explicitly:
|
||||
localhost must never be routed through HTTP_PROXY.
|
||||
"""
|
||||
req = urllib.request.Request(_MCP_ENDPOINT, method="GET")
|
||||
opener = urllib.request.build_opener(urllib.request.ProxyHandler({}))
|
||||
try:
|
||||
opener.open(req, timeout=timeout)
|
||||
return True
|
||||
except urllib.error.HTTPError:
|
||||
return True # 405/404 etc. — service is alive
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def format_xhs_result(data):
|
||||
"""Clean XHS API response, keeping only useful fields.
|
||||
|
||||
Handles both single note objects and lists of notes (search results).
|
||||
Drastically reduces token usage by stripping structural redundancy (#134).
|
||||
"""
|
||||
if isinstance(data, list):
|
||||
return [_clean_note(item) for item in data]
|
||||
if isinstance(data, dict):
|
||||
# Handle search_feeds wrapper: {"items": [...]} or {"data": {"items": [...]}}
|
||||
items = None
|
||||
if "items" in data:
|
||||
items = data["items"]
|
||||
elif "data" in data and isinstance(data.get("data"), dict):
|
||||
items = data["data"].get("items") or data["data"].get("notes")
|
||||
if items and isinstance(items, list):
|
||||
return [_clean_note(item) for item in items]
|
||||
# Single note
|
||||
return _clean_note(data)
|
||||
return data
|
||||
|
||||
|
||||
def _clean_note(note):
|
||||
"""Extract useful fields from a single XHS note/feed item."""
|
||||
if not isinstance(note, dict):
|
||||
return note
|
||||
|
||||
# Some responses nest the note under "note_card" or "note"
|
||||
inner = note.get("note_card") or note.get("note") or note
|
||||
|
||||
result = {}
|
||||
|
||||
# Basic info
|
||||
for key in ("id", "note_id", "xsec_token", "title", "desc", "type", "time"):
|
||||
if key in inner:
|
||||
result[key] = inner[key]
|
||||
|
||||
# Content (may be in desc or content)
|
||||
if "content" in inner and "desc" not in result:
|
||||
result["content"] = inner["content"]
|
||||
|
||||
# Author
|
||||
user = inner.get("user") or inner.get("author")
|
||||
if isinstance(user, dict):
|
||||
result["user"] = {
|
||||
k: user[k] for k in ("nickname", "user_id", "nick_name") if k in user
|
||||
}
|
||||
|
||||
# Engagement metrics
|
||||
interact = inner.get("interact_info") or inner.get("note_interact_info") or {}
|
||||
if isinstance(interact, dict):
|
||||
for key in ("liked_count", "collected_count", "comment_count", "share_count"):
|
||||
if key in interact:
|
||||
result[key] = interact[key]
|
||||
# Also check top-level (some API formats)
|
||||
for key in ("liked_count", "collected_count", "comment_count", "share_count"):
|
||||
if key in inner and key not in result:
|
||||
result[key] = inner[key]
|
||||
|
||||
# Images — just URLs
|
||||
images = inner.get("image_list") or inner.get("images_list") or []
|
||||
if isinstance(images, list):
|
||||
urls = []
|
||||
for img in images:
|
||||
if isinstance(img, dict):
|
||||
url = img.get("url") or img.get("url_default") or img.get("original")
|
||||
if url:
|
||||
urls.append(url)
|
||||
elif isinstance(img, str):
|
||||
urls.append(img)
|
||||
if urls:
|
||||
result["images"] = urls
|
||||
|
||||
# Tags
|
||||
tags = inner.get("tag_list") or inner.get("tags") or []
|
||||
if isinstance(tags, list):
|
||||
tag_names = []
|
||||
for t in tags:
|
||||
if isinstance(t, dict) and "name" in t:
|
||||
tag_names.append(t["name"])
|
||||
elif isinstance(t, str):
|
||||
tag_names.append(t)
|
||||
if tag_names:
|
||||
result["tags"] = tag_names
|
||||
|
||||
# Comments (if present, e.g. from get_feed_detail with comments)
|
||||
comments = inner.get("comments") or []
|
||||
if isinstance(comments, list) and comments:
|
||||
result["comments"] = [_clean_comment(c) for c in comments]
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def _clean_comment(comment):
|
||||
"""Extract useful fields from a comment."""
|
||||
if not isinstance(comment, dict):
|
||||
return comment
|
||||
result = {}
|
||||
if "content" in comment:
|
||||
result["content"] = comment["content"]
|
||||
user = comment.get("user_info") or comment.get("user")
|
||||
if isinstance(user, dict):
|
||||
result["user"] = user.get("nickname") or user.get("nick_name", "")
|
||||
for key in ("like_count", "sub_comment_count"):
|
||||
if key in comment:
|
||||
result[key] = comment[key]
|
||||
return result
|
||||
|
||||
|
||||
class XiaoHongShuChannel(Channel):
|
||||
name = "xiaohongshu"
|
||||
description = "小红书笔记"
|
||||
backends = ["OpenCLI", "xiaohongshu-mcp", "xhs-cli (xiaohongshu-cli)"]
|
||||
tier = 1
|
||||
|
||||
def can_handle(self, url: str) -> bool:
|
||||
from urllib.parse import urlparse
|
||||
d = urlparse(url).netloc.lower()
|
||||
return "xiaohongshu.com" in d or "xhslink.com" in d
|
||||
|
||||
def check(self, config=None):
|
||||
"""Probe candidates in order; first fully-usable backend wins.
|
||||
|
||||
If none is fully usable, the first fixable candidate (warn) is
|
||||
reported, so the user gets one actionable prescription instead
|
||||
of three half-relevant ones.
|
||||
"""
|
||||
self.active_backend = None
|
||||
findings = [] # (backend, status, message)
|
||||
|
||||
for backend in self.ordered_backends(config):
|
||||
if backend == "OpenCLI":
|
||||
result = self._check_opencli()
|
||||
elif backend == "xiaohongshu-mcp":
|
||||
result = self._check_mcp()
|
||||
else:
|
||||
result = self._check_xhs_cli()
|
||||
if result is None:
|
||||
continue # not installed — not a candidate right now
|
||||
findings.append((backend, *result))
|
||||
|
||||
for wanted in ("ok", "warn"):
|
||||
for backend, status, message in findings:
|
||||
if status == wanted:
|
||||
self.active_backend = backend
|
||||
return status, message
|
||||
|
||||
if findings: # only broken candidates left
|
||||
return "error", "\n".join(m for _, _, m in findings)
|
||||
|
||||
return "off", (
|
||||
"未安装任何小红书后端。推荐：\n"
|
||||
" 桌面：agent-reach install --channels opencli\n"
|
||||
" （复用 Chrome 登录态，刷过小红书即零配置可用）\n"
|
||||
f" 服务器：xiaohongshu-mcp（自带无头浏览器+扫码登录）：{_MCP_INSTALL_URL}"
|
||||
)
|
||||
|
||||
def _check_opencli(self):
|
||||
"""OpenCLI candidate. None = not installed."""
|
||||
from agent_reach.backends import opencli_status
|
||||
|
||||
st = opencli_status()
|
||||
if not st.installed:
|
||||
return None
|
||||
if st.broken:
|
||||
return "error", st.hint
|
||||
if st.ready:
|
||||
return "ok", (
|
||||
"OpenCLI 可用（复用浏览器登录态）。用法："
|
||||
"opencli xiaohongshu search/note/comments/feed -f yaml"
|
||||
)
|
||||
return "warn", st.hint
|
||||
|
||||
def _check_mcp(self):
|
||||
"""xiaohongshu-mcp candidate. None = service not running."""
|
||||
if not _mcp_service_reachable():
|
||||
return None
|
||||
mcporter = probe_command(
|
||||
"mcporter", ["config", "list"], timeout=10, package="mcporter"
|
||||
)
|
||||
if mcporter.ok and "xiaohongshu" in mcporter.output:
|
||||
return "ok", (
|
||||
"xiaohongshu-mcp 服务运行中"
|
||||
"（mcporter call 'xiaohongshu.search_feeds(keyword: \"...\")'）。"
|
||||
"若未登录，让 agent 调 get_login_qrcode 扫码"
|
||||
)
|
||||
return "warn", (
|
||||
"xiaohongshu-mcp 服务在跑但 mcporter 未接入。运行：\n"
|
||||
f" mcporter config add xiaohongshu {_MCP_ENDPOINT}"
|
||||
)
|
||||
|
||||
def _check_xhs_cli(self):
|
||||
"""Legacy xhs-cli candidate. None = not installed."""
|
||||
probe = probe_command(
|
||||
"xhs", ["status"], timeout=10, package="xiaohongshu-cli"
|
||||
)
|
||||
if probe.status == "missing":
|
||||
return None
|
||||
if probe.status == "broken":
|
||||
return "error", "xhs 命令存在但无法执行\n" + probe.hint
|
||||
if probe.status == "timeout":
|
||||
return "warn", "xhs-cli 已安装但状态检测超时\n" + probe.hint
|
||||
|
||||
# 进程是活的（执行成功或运行后非零退出）——按输出内容分类
|
||||
if probe.ok and "ok: true" in probe.output:
|
||||
return "ok", (
|
||||
"xhs-cli 可用（搜索、阅读、评论、热门；上游 2026-03 起停更，"
|
||||
"桌面用户建议迁移到 OpenCLI）"
|
||||
)
|
||||
if "not_authenticated" in probe.output or "expired" in probe.output:
|
||||
return "warn", (
|
||||
"xhs-cli 已安装但未登录。运行：\n"
|
||||
" xhs login\n"
|
||||
"（自动从浏览器提取 Cookie，或扫码登录）"
|
||||
)
|
||||
return "warn", (
|
||||
"xhs-cli 已安装但状态异常。运行：\n"
|
||||
" xhs -v status 查看详细信息"
|
||||
)
|
||||
@@ -0,0 +1,63 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Xiaoyuzhou Podcast (小宇宙播客) — transcribe podcasts via Groq Whisper API."""
|
||||
|
||||
import os
|
||||
from agent_reach.config import Config
|
||||
from agent_reach.probe import probe_command
|
||||
from .base import Channel
|
||||
|
||||
|
||||
class XiaoyuzhouChannel(Channel):
|
||||
name = "xiaoyuzhou"
|
||||
description = "小宇宙播客转文字"
|
||||
backends = ["groq-whisper", "ffmpeg"]
|
||||
tier = 1
|
||||
|
||||
def can_handle(self, url: str) -> bool:
|
||||
from urllib.parse import urlparse
|
||||
d = urlparse(url).netloc.lower()
|
||||
return "xiaoyuzhoufm.com" in d
|
||||
|
||||
def check(self, config=None):
|
||||
self.active_backend = None
|
||||
|
||||
# Check ffmpeg — really execute it: a stale pip-installed ffmpeg shim
|
||||
# passes shutil.which() but cannot run
|
||||
probe = probe_command("ffmpeg", ["-version"], timeout=10, package="ffmpeg")
|
||||
if probe.status == "missing":
|
||||
return "off", (
|
||||
"需要 ffmpeg（音频转码和切片）。安装：\n"
|
||||
" Ubuntu/Debian: apt install -y ffmpeg\n"
|
||||
" macOS: brew install ffmpeg"
|
||||
)
|
||||
if not probe.ok:
|
||||
return "error", (
|
||||
"ffmpeg 无法执行，重装：brew install ffmpeg（macOS）/ apt install ffmpeg（Linux）"
|
||||
)
|
||||
|
||||
# Check script exists
|
||||
script = os.path.expanduser("~/.agent-reach/tools/xiaoyuzhou/transcribe.sh")
|
||||
if not os.path.isfile(script):
|
||||
return "off", (
|
||||
"转录脚本未安装。运行：\n"
|
||||
" agent-reach install --env=auto\n"
|
||||
" 或手动复制 transcribe.sh 到 ~/.agent-reach/tools/xiaoyuzhou/"
|
||||
)
|
||||
|
||||
# Check GROQ_API_KEY — prefer env var, fall back to Agent Reach config
|
||||
has_key = bool(os.environ.get("GROQ_API_KEY"))
|
||||
if not has_key:
|
||||
try:
|
||||
cfg = config if config is not None else Config()
|
||||
has_key = bool(cfg.get("groq_api_key"))
|
||||
except Exception:
|
||||
has_key = False
|
||||
if not has_key:
|
||||
return "warn", (
|
||||
"需要配置 Groq API Key（免费）。步骤：\n"
|
||||
" 1. 注册 https://console.groq.com\n"
|
||||
" 2. 运行: agent-reach configure groq-key gsk_xxxxx"
|
||||
)
|
||||
|
||||
self.active_backend = "groq-whisper"
|
||||
return "ok", "完整可用（播客下载 + Whisper 转录）"
|
||||
@@ -0,0 +1,319 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Xueqiu (雪球) — stock quotes, search, trending posts & hot stocks."""
|
||||
|
||||
import http.cookiejar
|
||||
import json
|
||||
import re
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
from typing import Any
|
||||
|
||||
from .base import Channel
|
||||
|
||||
_UA = (
|
||||
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
|
||||
"AppleWebKit/537.36 (KHTML, like Gecko) "
|
||||
"Chrome/120.0.0.0 Safari/537.36"
|
||||
)
|
||||
_REFERER = "https://xueqiu.com/"
|
||||
_TIMEOUT = 10
|
||||
_XUEQIU_HOME = "https://xueqiu.com"
|
||||
|
||||
# --------------- cookie-aware HTTP helpers --------------- #
|
||||
|
||||
_cookie_jar = http.cookiejar.CookieJar()
|
||||
_opener = urllib.request.build_opener(
|
||||
urllib.request.HTTPCookieProcessor(_cookie_jar),
|
||||
)
|
||||
_cookies_initialized = False
|
||||
|
||||
|
||||
def _inject_cookie_string(cookie_str: str) -> None:
|
||||
"""Parse a 'name=value; name2=value2' string and inject into the cookie jar."""
|
||||
for pair in cookie_str.split(";"):
|
||||
pair = pair.strip()
|
||||
if "=" not in pair:
|
||||
continue
|
||||
name, _, value = pair.partition("=")
|
||||
cookie = http.cookiejar.Cookie(
|
||||
version=0,
|
||||
name=name.strip(),
|
||||
value=value.strip(),
|
||||
port=None,
|
||||
port_specified=False,
|
||||
domain=".xueqiu.com",
|
||||
domain_specified=True,
|
||||
domain_initial_dot=True,
|
||||
path="/",
|
||||
path_specified=True,
|
||||
secure=True,
|
||||
expires=None,
|
||||
discard=True,
|
||||
comment=None,
|
||||
comment_url=None,
|
||||
rest={},
|
||||
)
|
||||
_cookie_jar.set_cookie(cookie)
|
||||
|
||||
|
||||
def _load_cookies_from_config() -> bool:
|
||||
"""Try to load Xueqiu cookies from agent-reach config file (xueqiu_cookie key)."""
|
||||
try:
|
||||
from ..config import Config
|
||||
|
||||
cfg = Config()
|
||||
cookie_str = cfg.get("xueqiu_cookie")
|
||||
if not cookie_str:
|
||||
return False
|
||||
_inject_cookie_string(cookie_str)
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def _load_cookies_from_browser() -> bool:
|
||||
"""Try to silently load Xueqiu cookies from the local Chrome browser.
|
||||
|
||||
Only succeeds when browser_cookie3 is installed AND the user is logged in
|
||||
(xq_a_token present). Failures are silently ignored so that agents without
|
||||
a local browser keep working.
|
||||
"""
|
||||
try:
|
||||
try:
|
||||
import rookiepy
|
||||
cookies = rookiepy.chrome([".xueqiu.com"])
|
||||
if not any(c.get("name") == "xq_a_token" for c in cookies):
|
||||
return False
|
||||
for c in cookies:
|
||||
name = c.get("name")
|
||||
value = c.get("value")
|
||||
if name and value is not None:
|
||||
_inject_cookie_string(f"{name}={value}")
|
||||
return True
|
||||
except ImportError:
|
||||
import browser_cookie3
|
||||
cookies = list(browser_cookie3.chrome(domain_name=".xueqiu.com"))
|
||||
if not any(c.name == "xq_a_token" for c in cookies):
|
||||
return False
|
||||
for c in cookies:
|
||||
_cookie_jar.set_cookie(c)
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def _ensure_cookies() -> None:
|
||||
"""Populate session cookies using the best available source.
|
||||
|
||||
Priority order:
|
||||
1. Saved cookie string in ~/.agent-reach/config.yaml (set by configure --from-browser)
|
||||
2. Live Chrome browser cookies via rookiepy/browser_cookie3 (if installed + logged in)
|
||||
3. Homepage visit fallback (only yields anti-DDoS acw_tc,
|
||||
not enough for stock APIs)
|
||||
"""
|
||||
global _cookies_initialized
|
||||
if _cookies_initialized:
|
||||
return
|
||||
if _load_cookies_from_config():
|
||||
_cookies_initialized = True
|
||||
return
|
||||
if _load_cookies_from_browser():
|
||||
_cookies_initialized = True
|
||||
return
|
||||
# Fallback: visit homepage to pick up acw_tc anti-DDoS cookie.
|
||||
# This is not sufficient for authenticated APIs but avoids hard failures
|
||||
# on public endpoints that only need the session cookie.
|
||||
req = urllib.request.Request(_XUEQIU_HOME, headers={"User-Agent": _UA})
|
||||
_opener.open(req, timeout=_TIMEOUT)
|
||||
_cookies_initialized = True
|
||||
|
||||
|
||||
def _get_json(url: str) -> Any:
|
||||
"""Fetch *url* with Xueqiu session cookies and return parsed JSON."""
|
||||
_ensure_cookies()
|
||||
req = urllib.request.Request(
|
||||
url, headers={"User-Agent": _UA, "Referer": _REFERER}
|
||||
)
|
||||
with _opener.open(req, timeout=_TIMEOUT) as resp:
|
||||
return json.loads(resp.read().decode("utf-8"))
|
||||
|
||||
|
||||
def _strip_html(text: str) -> str:
|
||||
"""Remove HTML tags and decode common entities."""
|
||||
text = re.sub(r"<[^>]+>", "", text)
|
||||
for entity, char in ((" ", " "), ("&", "&"), ("<", "<"), (">", ">")):
|
||||
text = text.replace(entity, char)
|
||||
return text.strip()
|
||||
|
||||
|
||||
class XueqiuChannel(Channel):
|
||||
name = "xueqiu"
|
||||
description = "雪球股票行情与社区动态"
|
||||
backends = ["Xueqiu API (需要登录 Cookie)"]
|
||||
tier = 1
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# URL routing
|
||||
# ------------------------------------------------------------------ #
|
||||
|
||||
def can_handle(self, url: str) -> bool:
|
||||
d = urllib.parse.urlparse(url).netloc.lower()
|
||||
return "xueqiu.com" in d
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# Health check
|
||||
# ------------------------------------------------------------------ #
|
||||
|
||||
def check(self, config=None):
|
||||
self.active_backend = None
|
||||
try:
|
||||
data = _get_json(
|
||||
"https://stock.xueqiu.com/v5/stock/batch/quote.json?symbol=SH000001"
|
||||
)
|
||||
items = (data.get("data") or {}).get("items") or []
|
||||
if items:
|
||||
self.active_backend = self.backends[0]
|
||||
return "ok", "公开 API 可用（行情、搜索、热帖、热股）"
|
||||
return "warn", "API 响应异常（返回数据为空）"
|
||||
except Exception as e:
|
||||
return "warn", (
|
||||
f"Xueqiu API 连接失败：{e}。"
|
||||
"请先登录雪球后运行：agent-reach configure --from-browser chrome"
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# Data-fetching methods
|
||||
# ------------------------------------------------------------------ #
|
||||
|
||||
def get_stock_quote(self, symbol: str) -> dict:
|
||||
"""获取实时股票行情。
|
||||
|
||||
Args:
|
||||
symbol: 股票代码，如 SH600519（沪）、SZ000858（深）、AAPL（美）、00700（港）
|
||||
|
||||
Returns a dict with keys:
|
||||
symbol, name, current, percent, chg, high, low, open, last_close,
|
||||
volume, amount, market_capital, turnover_rate, pe_ttm, timestamp
|
||||
"""
|
||||
data = _get_json(
|
||||
f"https://stock.xueqiu.com/v5/stock/batch/quote.json?symbol={symbol}"
|
||||
)
|
||||
items = (data.get("data") or {}).get("items") or []
|
||||
q = (items[0].get("quote") or {}) if items else {}
|
||||
return {
|
||||
"symbol": q.get("symbol", symbol),
|
||||
"name": q.get("name", ""),
|
||||
"current": q.get("current"),
|
||||
"percent": q.get("percent"),
|
||||
"chg": q.get("chg"),
|
||||
"high": q.get("high"),
|
||||
"low": q.get("low"),
|
||||
"open": q.get("open"),
|
||||
"last_close": q.get("last_close"),
|
||||
"volume": q.get("volume"),
|
||||
"amount": q.get("amount"),
|
||||
"market_capital": q.get("market_capital"),
|
||||
"turnover_rate": q.get("turnover_rate"),
|
||||
"pe_ttm": q.get("pe_ttm"),
|
||||
"timestamp": q.get("timestamp"),
|
||||
}
|
||||
|
||||
def search_stock(self, query: str, limit: int = 10) -> list:
|
||||
"""搜索股票。
|
||||
|
||||
Args:
|
||||
query: 股票代码或中文名称，如 "茅台"、"600519"
|
||||
limit: 最多返回条数
|
||||
|
||||
Returns a list of dicts with keys:
|
||||
symbol, name, exchange
|
||||
"""
|
||||
data = _get_json(
|
||||
f"https://xueqiu.com/stock/search.json"
|
||||
f"?code={urllib.parse.quote(query)}&size={limit}"
|
||||
)
|
||||
stocks = data.get("stocks") or []
|
||||
results = []
|
||||
for s in stocks[:limit]:
|
||||
results.append(
|
||||
{
|
||||
"symbol": s.get("code", ""),
|
||||
"name": s.get("name", ""),
|
||||
"exchange": s.get("exchange", ""),
|
||||
}
|
||||
)
|
||||
return results
|
||||
|
||||
def get_hot_posts(self, limit: int = 20) -> list:
|
||||
"""获取雪球热门帖子。
|
||||
|
||||
Uses the v4 public timeline endpoint which returns posts in a `list`
|
||||
array. Each item carries a JSON-encoded `data` field containing the
|
||||
actual post payload (title, description, user, like_count, target).
|
||||
|
||||
Args:
|
||||
limit: 最多返回条数（上限 50）
|
||||
|
||||
Returns a list of dicts with keys:
|
||||
id, title, text, author, likes, url
|
||||
"""
|
||||
data = _get_json(
|
||||
"https://xueqiu.com/v4/statuses/public_timeline_by_category.json"
|
||||
"?since_id=-1&max_id=-1&count=20&category=-1"
|
||||
)
|
||||
items = data.get("list") or []
|
||||
results = []
|
||||
for item in items[:limit]:
|
||||
# Each item.data is a JSON string containing the real post payload
|
||||
try:
|
||||
post = (
|
||||
json.loads(item["data"])
|
||||
if isinstance(item.get("data"), str)
|
||||
else {}
|
||||
)
|
||||
except (json.JSONDecodeError, KeyError):
|
||||
post = {}
|
||||
user = post.get("user") or {}
|
||||
text = _strip_html(
|
||||
post.get("text") or post.get("description") or ""
|
||||
)
|
||||
target = post.get("target", "")
|
||||
results.append(
|
||||
{
|
||||
"id": post.get("id", 0),
|
||||
"title": post.get("title") or "",
|
||||
"text": text[:200],
|
||||
"author": user.get("screen_name", ""),
|
||||
"likes": post.get("like_count", 0),
|
||||
"url": f"https://xueqiu.com{target}" if target else "",
|
||||
}
|
||||
)
|
||||
return results
|
||||
|
||||
def get_hot_stocks(self, limit: int = 10, stock_type: int = 10) -> list:
|
||||
"""获取热门股票排行。
|
||||
|
||||
Args:
|
||||
limit: 最多返回条数（上限 50）
|
||||
stock_type: 10=人气榜（默认），12=关注榜
|
||||
|
||||
Returns a list of dicts with keys:
|
||||
symbol, name, current, percent, rank
|
||||
"""
|
||||
data = _get_json(
|
||||
f"https://stock.xueqiu.com/v5/stock/hot_stock/list.json"
|
||||
f"?size={limit}&type={stock_type}"
|
||||
)
|
||||
items = (data.get("data") or {}).get("items") or []
|
||||
results = []
|
||||
for idx, item in enumerate(items[:limit], 1):
|
||||
results.append(
|
||||
{
|
||||
"symbol": item.get("code") or item.get("symbol", ""),
|
||||
"name": item.get("name", ""),
|
||||
"current": item.get("current"),
|
||||
"percent": item.get("percent"),
|
||||
"rank": idx,
|
||||
}
|
||||
)
|
||||
return results
|
||||
@@ -0,0 +1,91 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""YouTube — check if yt-dlp is available with JS runtime."""
|
||||
|
||||
import shutil
|
||||
|
||||
from agent_reach.probe import probe_command
|
||||
from agent_reach.utils.paths import get_ytdlp_config_path, render_ytdlp_fix_command
|
||||
from agent_reach.utils.text import read_utf8_text
|
||||
|
||||
from .base import Channel
|
||||
|
||||
|
||||
def _has_js_runtime_config(config_path) -> bool:
|
||||
"""Return whether yt-dlp config explicitly enables a JS runtime."""
|
||||
try:
|
||||
if not config_path.exists():
|
||||
return False
|
||||
return "--js-runtimes" in read_utf8_text(config_path)
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
|
||||
class YouTubeChannel(Channel):
|
||||
name = "youtube"
|
||||
description = "YouTube 视频和字幕"
|
||||
backends = ["yt-dlp"]
|
||||
tier = 0
|
||||
|
||||
def can_handle(self, url: str) -> bool:
|
||||
from urllib.parse import urlparse
|
||||
|
||||
d = urlparse(url).netloc.lower()
|
||||
return "youtube.com" in d or "youtu.be" in d
|
||||
|
||||
def check(self, config=None):
|
||||
# 真跑 yt-dlp --version 探活，区分未装 / venv 断链 / 跑不动
|
||||
probe = probe_command("yt-dlp", ["--version"], timeout=10, package="yt-dlp")
|
||||
if probe.status == "missing":
|
||||
self.active_backend = None
|
||||
return "off", "yt-dlp 未安装。安装：pip install yt-dlp"
|
||||
if probe.status == "broken":
|
||||
self.active_backend = None
|
||||
return "error", f"yt-dlp 已安装但无法执行\n{probe.hint}"
|
||||
if not probe.ok: # timeout / error：装了但跑不动
|
||||
self.active_backend = None
|
||||
detail = probe.hint or probe.output or probe.status
|
||||
return "error", f"yt-dlp 无法正常运行：{detail}"
|
||||
# yt-dlp 本体是活的；后面的 JS runtime/转写检查只影响 ok/warn，不影响后端归属
|
||||
self.active_backend = "yt-dlp"
|
||||
# Check JS runtime
|
||||
has_js = shutil.which("deno") or shutil.which("node")
|
||||
if not has_js:
|
||||
return "warn", (
|
||||
"yt-dlp 已安装但缺少 JS runtime（YouTube 必须）。\n"
|
||||
" 安装 Node.js 或 deno，然后运行：agent-reach install"
|
||||
)
|
||||
# Check yt-dlp config for --js-runtimes
|
||||
# Deno works out of the box; Node.js requires explicit config
|
||||
has_deno = shutil.which("deno")
|
||||
if not has_deno:
|
||||
ytdlp_config = get_ytdlp_config_path()
|
||||
if not _has_js_runtime_config(ytdlp_config):
|
||||
return "warn", (
|
||||
f"yt-dlp 已安装但未配置 JS runtime。运行：\n {render_ytdlp_fix_command()}"
|
||||
)
|
||||
# Surface transcription readiness so `doctor` reports it.
|
||||
msg = "可提取视频信息和字幕"
|
||||
if config is not None:
|
||||
providers = []
|
||||
if config.is_configured("groq_whisper"):
|
||||
providers.append("groq")
|
||||
if config.is_configured("openai_whisper"):
|
||||
providers.append("openai")
|
||||
if providers:
|
||||
if not shutil.which("ffmpeg"):
|
||||
msg += "（音频转写需安装 ffmpeg）"
|
||||
else:
|
||||
msg += f"，可转写音频（{'→'.join(providers)}）"
|
||||
return "ok", msg
|
||||
|
||||
def transcribe(self, url: str, *, provider: str = "auto", config=None) -> str:
|
||||
"""Download a YouTube video's audio and return its transcript.
|
||||
|
||||
Delegates to :func:`agent_reach.transcribe.transcribe`. Imported lazily
|
||||
so the channel module stays cheap to import for users who never
|
||||
transcribe.
|
||||
"""
|
||||
from agent_reach.transcribe import transcribe as _transcribe
|
||||
|
||||
return _transcribe(url, provider=provider, config=config)
|
||||
|
||||
+1834
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,130 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Configuration management for Agent Reach.
|
||||
|
||||
Stores settings in ~/.agent-reach/config.yaml.
|
||||
Auto-creates directory on first use.
|
||||
"""
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Any, Optional
|
||||
|
||||
import yaml
|
||||
|
||||
from agent_reach.utils.paths import make_private_dir
|
||||
|
||||
|
||||
class Config:
|
||||
"""Manages Agent Reach configuration."""
|
||||
|
||||
CONFIG_DIR = Path.home() / ".agent-reach"
|
||||
CONFIG_FILE = CONFIG_DIR / "config.yaml"
|
||||
|
||||
# Feature → required config keys
|
||||
FEATURE_REQUIREMENTS = {
|
||||
"exa_search": ["exa_api_key"],
|
||||
"twitter_xreach": ["twitter_auth_token", "twitter_ct0"], # legacy key name; used by twitter-cli
|
||||
"groq_whisper": ["groq_api_key"],
|
||||
"openai_whisper": ["openai_api_key"],
|
||||
"github_token": ["github_token"],
|
||||
}
|
||||
|
||||
def __init__(self, config_path: Optional[Path] = None):
|
||||
self.config_path = Path(config_path) if config_path else self.CONFIG_FILE
|
||||
self.config_dir = self.config_path.parent
|
||||
self.data: dict = {}
|
||||
self._ensure_dir()
|
||||
self.load()
|
||||
|
||||
def _ensure_dir(self):
|
||||
"""Create config directory if it doesn't exist."""
|
||||
make_private_dir(self.config_dir)
|
||||
|
||||
def load(self):
|
||||
"""Load config from YAML file."""
|
||||
if self.config_path.exists():
|
||||
with open(self.config_path, "r", encoding="utf-8") as f:
|
||||
self.data = yaml.safe_load(f) or {}
|
||||
else:
|
||||
self.data = {}
|
||||
|
||||
def save(self):
|
||||
"""Save config to YAML file."""
|
||||
self._ensure_dir()
|
||||
# Create file with restricted permissions from the start to avoid
|
||||
# a race window where credentials are briefly world-readable.
|
||||
try:
|
||||
import stat
|
||||
fd = os.open(
|
||||
str(self.config_path),
|
||||
os.O_WRONLY | os.O_CREAT | os.O_TRUNC,
|
||||
stat.S_IRUSR | stat.S_IWUSR, # 0o600
|
||||
)
|
||||
if os.name != "nt":
|
||||
os.chmod(self.config_path, stat.S_IRUSR | stat.S_IWUSR)
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as f:
|
||||
yaml.dump(self.data, f, default_flow_style=False, allow_unicode=True)
|
||||
except OSError:
|
||||
# Fallback for Windows or other edge cases where os.open flags
|
||||
# are not fully supported.
|
||||
with open(self.config_path, "w", encoding="utf-8") as f:
|
||||
yaml.dump(self.data, f, default_flow_style=False, allow_unicode=True)
|
||||
if os.name != "nt":
|
||||
os.chmod(self.config_path, 0o600)
|
||||
|
||||
def get(self, key: str, default: Any = None) -> Any:
|
||||
"""Get a config value. Also checks environment variables (uppercase)."""
|
||||
# Config file first
|
||||
if key in self.data:
|
||||
return self.data[key]
|
||||
# Then env var (uppercase)
|
||||
env_val = os.environ.get(key.upper())
|
||||
if env_val:
|
||||
return env_val
|
||||
return default
|
||||
|
||||
def set(self, key: str, value: Any):
|
||||
"""Set a config value and save."""
|
||||
self.data[key] = value
|
||||
self.save()
|
||||
|
||||
def delete(self, key: str):
|
||||
"""Delete a config key and save."""
|
||||
self.data.pop(key, None)
|
||||
self.save()
|
||||
|
||||
def is_configured(self, feature: str) -> bool:
|
||||
"""Check if a feature has all required config."""
|
||||
required = self.FEATURE_REQUIREMENTS.get(feature, [])
|
||||
return all(self.get(k) for k in required)
|
||||
|
||||
def get_configured_features(self) -> dict:
|
||||
"""Return status of all optional features."""
|
||||
return {
|
||||
feature: self.is_configured(feature)
|
||||
for feature in self.FEATURE_REQUIREMENTS
|
||||
}
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
"""Return config as dict (masks sensitive values)."""
|
||||
sensitive_markers = (
|
||||
"key",
|
||||
"token",
|
||||
"password",
|
||||
"proxy",
|
||||
"cookie",
|
||||
"secret",
|
||||
"session",
|
||||
"sessdata",
|
||||
"csrf",
|
||||
"auth",
|
||||
"cred",
|
||||
"ct0",
|
||||
)
|
||||
masked = {}
|
||||
for k, v in self.data.items():
|
||||
if any(s in k.lower() for s in sensitive_markers):
|
||||
masked[k] = f"{str(v)[:8]}..." if v else None
|
||||
else:
|
||||
masked[k] = v
|
||||
return masked
|
||||
@@ -0,0 +1,297 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Auto-extract cookies from local browsers for all supported platforms.
|
||||
|
||||
Supports: Chrome, Firefox, Edge, Brave, Opera
|
||||
Extracts: Twitter, XiaoHongShu, Bilibili cookies in one shot.
|
||||
|
||||
Usage:
|
||||
agent-reach configure --from-browser chrome
|
||||
"""
|
||||
|
||||
from typing import Dict, List, Tuple
|
||||
|
||||
# Platform cookie specs: (platform_name, domain_pattern, needed_cookies)
|
||||
PLATFORM_SPECS = [
|
||||
{
|
||||
"name": "Twitter/X",
|
||||
"domains": [".x.com", ".twitter.com"],
|
||||
"cookies": ["auth_token", "ct0"],
|
||||
"config_key": "twitter",
|
||||
},
|
||||
{
|
||||
"name": "XiaoHongShu",
|
||||
"domains": [".xiaohongshu.com"],
|
||||
"cookies": None, # None = grab all cookies as header string
|
||||
"config_key": "xhs",
|
||||
},
|
||||
{
|
||||
"name": "Bilibili",
|
||||
"domains": [".bilibili.com"],
|
||||
"cookies": ["SESSDATA", "bili_jct"],
|
||||
"config_key": "bilibili",
|
||||
},
|
||||
{
|
||||
"name": "Xueqiu",
|
||||
"domains": [".xueqiu.com", "xueqiu.com"],
|
||||
"cookies": None, # grab all — xq_a_token + session cookies required
|
||||
"config_key": "xueqiu",
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
def extract_all(browser: str = "chrome") -> Dict[str, dict]:
|
||||
"""
|
||||
Extract cookies for all supported platforms from the specified browser.
|
||||
|
||||
Returns:
|
||||
{
|
||||
"twitter": {"auth_token": "xxx", "ct0": "yyy"},
|
||||
"xhs": {"cookie_string": "a=1; b=2; ..."},
|
||||
"bilibili": {"SESSDATA": "xxx", "bili_jct": "yyy"},
|
||||
}
|
||||
"""
|
||||
# Try rookiepy first (Rust-based, more stable), fallback to browser_cookie3
|
||||
use_rookiepy = False
|
||||
try:
|
||||
import rookiepy
|
||||
use_rookiepy = True
|
||||
except ImportError:
|
||||
try:
|
||||
import browser_cookie3
|
||||
except ImportError:
|
||||
raise RuntimeError(
|
||||
"Cookie extraction requires rookiepy or browser_cookie3.\n"
|
||||
"Install: pip install rookiepy (recommended)\n"
|
||||
" or: pip install browser-cookie3"
|
||||
)
|
||||
|
||||
browser = browser.lower()
|
||||
supported = ["chrome", "firefox", "edge", "brave", "opera"]
|
||||
if browser not in supported:
|
||||
raise ValueError(
|
||||
f"Unsupported browser: {browser}. Supported: {', '.join(supported)}"
|
||||
)
|
||||
|
||||
if use_rookiepy:
|
||||
# rookiepy returns list of dicts with name/value/domain/path keys
|
||||
try:
|
||||
browser_funcs = {
|
||||
"chrome": rookiepy.chrome,
|
||||
"firefox": rookiepy.firefox,
|
||||
"edge": rookiepy.edge,
|
||||
"brave": rookiepy.brave,
|
||||
"opera": rookiepy.opera,
|
||||
}
|
||||
raw_cookies = browser_funcs[browser]()
|
||||
# Wrap into objects with .name, .value, .domain for compatibility
|
||||
class _Cookie:
|
||||
def __init__(self, d):
|
||||
self.name = d.get("name", "")
|
||||
self.value = d.get("value", "")
|
||||
self.domain = d.get("domain", "")
|
||||
cookie_jar = [_Cookie(c) for c in raw_cookies]
|
||||
except Exception as e:
|
||||
raise RuntimeError(
|
||||
f"Could not read {browser} cookies via rookiepy: {e}\n"
|
||||
f"Make sure {browser} is closed and you have permission."
|
||||
)
|
||||
else:
|
||||
browser_funcs = {
|
||||
"chrome": browser_cookie3.chrome,
|
||||
"firefox": browser_cookie3.firefox,
|
||||
"edge": browser_cookie3.edge,
|
||||
"brave": browser_cookie3.brave,
|
||||
"opera": browser_cookie3.opera,
|
||||
}
|
||||
try:
|
||||
cookie_jar = browser_funcs[browser]()
|
||||
except Exception as e:
|
||||
raise RuntimeError(
|
||||
f"Could not read {browser} cookies: {e}\n"
|
||||
f"Make sure {browser} is closed and you have permission."
|
||||
)
|
||||
|
||||
results = {}
|
||||
|
||||
for spec in PLATFORM_SPECS:
|
||||
platform_cookies = {}
|
||||
all_cookies_for_domain = []
|
||||
|
||||
for cookie in cookie_jar:
|
||||
# Check if cookie belongs to this platform
|
||||
domain_match = any(
|
||||
cookie.domain.endswith(d) or cookie.domain == d.lstrip(".")
|
||||
for d in spec["domains"]
|
||||
)
|
||||
if not domain_match:
|
||||
continue
|
||||
|
||||
all_cookies_for_domain.append(cookie)
|
||||
|
||||
if spec["cookies"] is not None:
|
||||
if cookie.name in spec["cookies"]:
|
||||
platform_cookies[cookie.name] = cookie.value
|
||||
|
||||
if spec["cookies"] is None:
|
||||
# Grab all as header string
|
||||
if all_cookies_for_domain:
|
||||
cookie_str = "; ".join(
|
||||
f"{c.name}={c.value}" for c in all_cookies_for_domain
|
||||
)
|
||||
results[spec["config_key"]] = {"cookie_string": cookie_str}
|
||||
else:
|
||||
if platform_cookies:
|
||||
results[spec["config_key"]] = platform_cookies
|
||||
|
||||
return results
|
||||
|
||||
|
||||
def _open_owner_only(path: str):
|
||||
"""Open *path* for writing, atomically creating it with mode 0o600.
|
||||
|
||||
Mirrors the pattern used by Config.save() in config.py: O_WRONLY|O_CREAT|
|
||||
O_TRUNC + an explicit mode argument so the file is never briefly
|
||||
world-readable between open() and a later os.chmod(). On Windows (or any
|
||||
OS that rejects the open flags) we fall back to a plain open().
|
||||
"""
|
||||
import os
|
||||
import stat
|
||||
|
||||
try:
|
||||
fd = os.open(
|
||||
path,
|
||||
os.O_WRONLY | os.O_CREAT | os.O_TRUNC,
|
||||
stat.S_IRUSR | stat.S_IWUSR, # 0o600
|
||||
)
|
||||
if os.name != "nt":
|
||||
os.chmod(path, stat.S_IRUSR | stat.S_IWUSR)
|
||||
return os.fdopen(fd, "w", encoding="utf-8")
|
||||
except OSError:
|
||||
handle = open(path, "w", encoding="utf-8")
|
||||
if os.name != "nt":
|
||||
os.chmod(path, 0o600)
|
||||
return handle
|
||||
|
||||
|
||||
def _sync_xfetch_session(auth_token: str, ct0: str) -> None:
|
||||
"""Sync Twitter credentials to ~/.config/xfetch/session.json (legacy xreach compat)."""
|
||||
import json
|
||||
import os
|
||||
|
||||
try:
|
||||
from agent_reach.utils.paths import make_private_dir
|
||||
|
||||
xfetch_dir = os.path.join(os.path.expanduser("~"), ".config", "xfetch")
|
||||
make_private_dir(xfetch_dir)
|
||||
session_path = os.path.join(xfetch_dir, "session.json")
|
||||
session_data: dict = {}
|
||||
if os.path.exists(session_path):
|
||||
try:
|
||||
with open(session_path, "r", encoding="utf-8") as sf:
|
||||
session_data = json.load(sf)
|
||||
except (json.JSONDecodeError, OSError):
|
||||
session_data = {}
|
||||
session_data["authToken"] = auth_token
|
||||
session_data["ct0"] = ct0
|
||||
with _open_owner_only(session_path) as sf:
|
||||
json.dump(session_data, sf, indent=2)
|
||||
except Exception:
|
||||
# Non-fatal: agent-reach config is the source of truth, xfetch sync is best-effort
|
||||
pass
|
||||
|
||||
|
||||
def _sync_bird_env(auth_token: str, ct0: str) -> None:
|
||||
"""Write Twitter credentials to ~/.config/bird/credentials.env for bird CLI.
|
||||
|
||||
bird reads AUTH_TOKEN and CT0 from environment variables. This writes a
|
||||
shell-sourceable file so users can `source ~/.config/bird/credentials.env`.
|
||||
Values are passed through shlex.quote so a token containing a quote, $, or
|
||||
backtick cannot break out into shell syntax when the file is sourced.
|
||||
"""
|
||||
import os
|
||||
import shlex
|
||||
|
||||
try:
|
||||
from agent_reach.utils.paths import make_private_dir
|
||||
|
||||
bird_dir = os.path.join(os.path.expanduser("~"), ".config", "bird")
|
||||
make_private_dir(bird_dir)
|
||||
env_path = os.path.join(bird_dir, "credentials.env")
|
||||
with _open_owner_only(env_path) as f:
|
||||
f.write(f"AUTH_TOKEN={shlex.quote(auth_token)}\n")
|
||||
f.write(f"CT0={shlex.quote(ct0)}\n")
|
||||
except Exception:
|
||||
# Non-fatal: agent-reach config is the source of truth, bird env sync is best-effort
|
||||
pass
|
||||
|
||||
|
||||
# Alias for callers expecting the name _sync_bird_credentials
|
||||
_sync_bird_credentials = _sync_bird_env
|
||||
|
||||
|
||||
def configure_from_browser(browser: str, config) -> List[Tuple[str, bool, str]]:
|
||||
"""
|
||||
Extract cookies and configure all found platforms.
|
||||
|
||||
Returns list of (platform_name, success, message) tuples.
|
||||
"""
|
||||
results_list = []
|
||||
|
||||
try:
|
||||
extracted = extract_all(browser)
|
||||
except Exception as e:
|
||||
return [("Browser", False, str(e))]
|
||||
|
||||
if not extracted:
|
||||
return [("All platforms", False,
|
||||
f"No platform cookies found in {browser}. "
|
||||
f"Make sure you're logged into Twitter, XiaoHongShu, etc. in {browser}.")]
|
||||
|
||||
# Configure each found platform
|
||||
if "twitter" in extracted:
|
||||
tc = extracted["twitter"]
|
||||
if "auth_token" in tc and "ct0" in tc:
|
||||
config.set("twitter_auth_token", tc["auth_token"])
|
||||
config.set("twitter_ct0", tc["ct0"])
|
||||
# Legacy sync (best-effort)
|
||||
_sync_xfetch_session(tc["auth_token"], tc["ct0"])
|
||||
results_list.append(("Twitter/X", True, "auth_token + ct0"))
|
||||
else:
|
||||
found = ", ".join(tc.keys())
|
||||
missing = [k for k in ["auth_token", "ct0"] if k not in tc]
|
||||
results_list.append(("Twitter/X", False,
|
||||
f"Found {found}, but missing: {', '.join(missing)}. "
|
||||
f"Make sure you're logged into x.com in {browser}."))
|
||||
|
||||
if "xhs" in extracted:
|
||||
cookie_str = extracted["xhs"].get("cookie_string", "")
|
||||
if cookie_str:
|
||||
config.set("xhs_cookie", cookie_str)
|
||||
n_cookies = len(cookie_str.split(";"))
|
||||
results_list.append(("XiaoHongShu", True, f"{n_cookies} cookies"))
|
||||
|
||||
if "bilibili" in extracted:
|
||||
bc = extracted["bilibili"]
|
||||
if "SESSDATA" in bc:
|
||||
config.set("bilibili_sessdata", bc["SESSDATA"])
|
||||
if "bili_jct" in bc:
|
||||
config.set("bilibili_csrf", bc["bili_jct"])
|
||||
results_list.append(("Bilibili", True, "SESSDATA" +
|
||||
(" + bili_jct" if "bili_jct" in bc else "")))
|
||||
else:
|
||||
results_list.append(("Bilibili", False,
|
||||
f"No SESSDATA found. Make sure you're logged into bilibili.com in {browser}."))
|
||||
|
||||
if "xueqiu" in extracted:
|
||||
cookie_str = extracted["xueqiu"].get("cookie_string", "")
|
||||
# Only save if xq_a_token is present — anonymous cookies are useless
|
||||
if cookie_str and "xq_a_token" in cookie_str:
|
||||
config.set("xueqiu_cookie", cookie_str)
|
||||
n_cookies = len(cookie_str.split(";"))
|
||||
results_list.append(("Xueqiu", True, f"{n_cookies} cookies (含 xq_a_token)"))
|
||||
elif cookie_str:
|
||||
results_list.append(("Xueqiu", False,
|
||||
f"找到 {len(cookie_str.split(';'))} 个 Cookie 但缺少 xq_a_token，"
|
||||
f"请先在 {browser} 中登录 xueqiu.com"))
|
||||
|
||||
return results_list
|
||||
@@ -0,0 +1,42 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
AgentReach — installer, doctor, and configuration tool.
|
||||
|
||||
Agent Reach helps AI agents install and configure upstream platform tools
|
||||
(twitter-cli, yt-dlp, mcporter, gh CLI, etc.). After installation, agents
|
||||
call the upstream tools directly — no wrapper layer needed.
|
||||
|
||||
Usage:
|
||||
from agent_reach.doctor import check_all, format_report
|
||||
from agent_reach.config import Config
|
||||
|
||||
config = Config()
|
||||
results = check_all(config)
|
||||
print(format_report(results))
|
||||
"""
|
||||
|
||||
from typing import Dict, Optional
|
||||
|
||||
from agent_reach.config import Config
|
||||
|
||||
|
||||
class AgentReach:
|
||||
"""Give your AI Agent eyes to see the entire internet.
|
||||
|
||||
This class provides health-check functionality.
|
||||
For reading/searching, use the upstream tools directly
|
||||
(see SKILL.md for commands).
|
||||
"""
|
||||
|
||||
def __init__(self, config: Optional[Config] = None):
|
||||
self.config = config or Config()
|
||||
|
||||
def doctor(self) -> Dict[str, dict]:
|
||||
"""Check all channel availability."""
|
||||
from agent_reach.doctor import check_all
|
||||
return check_all(self.config)
|
||||
|
||||
def doctor_report(self) -> str:
|
||||
"""Get formatted health report."""
|
||||
from agent_reach.doctor import check_all, format_report
|
||||
return format_report(check_all(self.config))
|
||||
@@ -0,0 +1,127 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Environment health checker — powered by channels.
|
||||
|
||||
Each channel knows how to check itself. Doctor just collects the results.
|
||||
"""
|
||||
|
||||
from typing import Dict
|
||||
from agent_reach.config import Config
|
||||
from agent_reach.channels import get_all_channels
|
||||
|
||||
|
||||
def check_all(config: Config) -> Dict[str, dict]:
|
||||
"""Check all channels and return status dict.
|
||||
|
||||
A single misbehaving channel must never take the whole report down,
|
||||
so per-channel exceptions degrade to status="error".
|
||||
"""
|
||||
results = {}
|
||||
for ch in get_all_channels():
|
||||
try:
|
||||
status, message = ch.check(config)
|
||||
active = getattr(ch, "active_backend", None)
|
||||
except Exception as e: # noqa: BLE001 — doctor must survive any channel
|
||||
# Channels are registry singletons: a stale active_backend from a
|
||||
# previous check must not leak into an errored result.
|
||||
status, message, active = "error", f"体检异常：{e}", None
|
||||
results[ch.name] = {
|
||||
"status": status,
|
||||
"name": ch.description,
|
||||
"message": message,
|
||||
"tier": ch.tier,
|
||||
"backends": ch.backends,
|
||||
"active_backend": active,
|
||||
}
|
||||
return results
|
||||
|
||||
|
||||
def _name_msg(r: dict, escape) -> str:
|
||||
"""Render one channel line; show the active backend when there is a choice."""
|
||||
text = f"[bold]{escape(r['name'])}[/bold] — {escape(r['message'])}"
|
||||
active = r.get("active_backend")
|
||||
if active and len(r.get("backends", [])) > 1:
|
||||
text += f" [dim]（当前后端：{escape(active)}）[/dim]"
|
||||
return text
|
||||
|
||||
|
||||
def format_report(results: Dict[str, dict]) -> str:
|
||||
"""Format results as a readable text report (with Rich markup)."""
|
||||
try:
|
||||
from rich.markup import escape
|
||||
except ImportError:
|
||||
escape = lambda x: x
|
||||
|
||||
lines = []
|
||||
lines.append("[bold cyan]Agent Reach 状态[/bold cyan]")
|
||||
lines.append("[cyan]" + "=" * 40 + "[/cyan]")
|
||||
lines.append("图例：[green]✅[/green] 可用 [yellow][!][/yellow] 已装但需配置/登录 [red][X][/red] 未安装")
|
||||
|
||||
ok_count = sum(1 for r in results.values() if r["status"] == "ok")
|
||||
total = len(results)
|
||||
|
||||
# Tier 0 — zero config
|
||||
lines.append("")
|
||||
lines.append("[bold]✅ 装好即用：[/bold]")
|
||||
for key, r in results.items():
|
||||
if r["tier"] == 0:
|
||||
name_msg = _name_msg(r, escape)
|
||||
if r["status"] == "ok":
|
||||
lines.append(f" [green]✅[/green] {name_msg}")
|
||||
elif r["status"] == "warn":
|
||||
lines.append(f" [yellow][!][/yellow] {name_msg}")
|
||||
elif r["status"] in ("off", "error"):
|
||||
lines.append(f" [red][X][/red] {name_msg}")
|
||||
|
||||
# Tier 1 — needs free key / login
|
||||
tier1 = {k: r for k, r in results.items() if r["tier"] == 1}
|
||||
tier1_active = {k: r for k, r in tier1.items() if r["status"] == "ok"}
|
||||
tier1_inactive = {k: r for k, r in tier1.items() if r["status"] != "ok"}
|
||||
if tier1_active:
|
||||
lines.append("")
|
||||
lines.append("[bold]可选渠道（已安装）：[/bold]")
|
||||
for key, r in tier1_active.items():
|
||||
lines.append(f" [green]✅[/green] {_name_msg(r, escape)}")
|
||||
|
||||
# Tier 2 — optional complex setup
|
||||
tier2 = {k: r for k, r in results.items() if r["tier"] == 2}
|
||||
tier2_active = {k: r for k, r in tier2.items() if r["status"] == "ok"}
|
||||
tier2_inactive = {k: r for k, r in tier2.items() if r["status"] != "ok"}
|
||||
if tier2_active:
|
||||
if not tier1_active:
|
||||
lines.append("")
|
||||
lines.append("[bold]可选渠道（已安装）：[/bold]")
|
||||
for key, r in tier2_active.items():
|
||||
lines.append(f" [green]✅[/green] {_name_msg(r, escape)}")
|
||||
|
||||
lines.append("")
|
||||
status_color = "green" if ok_count == total else ("yellow" if ok_count > 0 else "red")
|
||||
lines.append(f"状态：[{status_color}]{ok_count}/{total}[/{status_color}] 个渠道可用")
|
||||
|
||||
# Summarize inactive optional channels in one line instead of listing each
|
||||
all_inactive = list(tier1_inactive.values()) + list(tier2_inactive.values())
|
||||
if all_inactive:
|
||||
names = [r["name"] for r in all_inactive]
|
||||
lines.append(
|
||||
f"还有 {len(names)} 个可选渠道可以解锁（{'、'.join(names)}），"
|
||||
"告诉你的 Agent「帮我装 XXX」即可"
|
||||
)
|
||||
|
||||
# Security check: config file permissions (Unix only)
|
||||
import os
|
||||
import stat
|
||||
import sys
|
||||
|
||||
config_path = Config.CONFIG_DIR / "config.yaml"
|
||||
if config_path.exists() and sys.platform != "win32":
|
||||
try:
|
||||
mode = config_path.stat().st_mode
|
||||
if mode & (stat.S_IRGRP | stat.S_IROTH):
|
||||
lines.append("")
|
||||
lines.append(
|
||||
"[bold red][!] 安全提示：config.yaml 权限过宽（其他用户可读）[/bold red]"
|
||||
)
|
||||
lines.append(" 修复：chmod 600 ~/.agent-reach/config.yaml")
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
return "\n".join(lines)
|
||||
@@ -0,0 +1,41 @@
|
||||
# Exa Search 配置指南
|
||||
|
||||
## 功能说明
|
||||
Exa 是一个 AI 语义搜索引擎。通过 MCP 接入，**免费、无需 API Key**。配置后解锁：
|
||||
- 全网语义搜索
|
||||
- Reddit 搜索（通过 site:reddit.com）
|
||||
- Twitter 搜索（通过 site:x.com）
|
||||
|
||||
## Agent 可自动完成的步骤
|
||||
|
||||
`agent-reach install --env=auto` 会自动完成以下步骤，通常不需要手动操作。
|
||||
|
||||
### 1. 安装 mcporter
|
||||
```bash
|
||||
npm install -g mcporter
|
||||
```
|
||||
|
||||
### 2. 注册 Exa MCP
|
||||
```bash
|
||||
mcporter config add exa https://mcp.exa.ai/mcp
|
||||
```
|
||||
|
||||
### 3. 验证
|
||||
```bash
|
||||
agent-reach doctor | grep "Search"
|
||||
mcporter call 'exa.web_search_exa(query: "test", numResults: 1)'
|
||||
```
|
||||
|
||||
## 需要用户手动做的步骤
|
||||
|
||||
**无。** Exa 通过 MCP 接入，免费、无需注册、无需 API Key。
|
||||
|
||||
如果 `agent-reach install` 因为网络问题没有自动配置 Exa，手动运行上面两条命令即可。
|
||||
|
||||
## 常见问题
|
||||
|
||||
**Q: 有搜索次数限制吗？**
|
||||
A: MCP 端点由 Exa 官方提供（mcp.exa.ai），当前免费无限制。如果未来有变化，会在 agent-reach 更新中适配。
|
||||
|
||||
**Q: mcporter 是什么？**
|
||||
A: MCP 协议的命令行桥接工具，用来调用 MCP Server。Agent Reach 用它来连接 Exa 和小红书。
|
||||
@@ -0,0 +1,47 @@
|
||||
# Groq Whisper 配置指南
|
||||
|
||||
## 功能说明
|
||||
当 YouTube/Bilibili 视频没有字幕时，用 Groq 的 Whisper API 进行语音转文字。Groq 提供免费额度。
|
||||
|
||||
## Agent 可自动完成的步骤
|
||||
|
||||
1. 检查是否已配置：
|
||||
```bash
|
||||
agent-reach doctor | grep -i "groq\|whisper"
|
||||
```
|
||||
|
||||
2. 如果用户提供了 key，写入配置：
|
||||
```python
|
||||
from agent_reach.config import Config
|
||||
c = Config()
|
||||
c.set("groq_api_key", "用户提供的KEY")
|
||||
```
|
||||
|
||||
3. 测试（可选）：
|
||||
```bash
|
||||
curl -s https://api.groq.com/openai/v1/models \
|
||||
-H "Authorization: Bearer 用户提供的KEY" \
|
||||
-o /dev/null -w "%{http_code}"
|
||||
```
|
||||
返回 200 = 可用
|
||||
|
||||
## 需要用户手动做的步骤
|
||||
|
||||
请告诉用户：
|
||||
|
||||
> 视频语音转文字需要一个 Groq API Key（免费）。
|
||||
>
|
||||
> 步骤：
|
||||
> 1. 打开 https://console.groq.com
|
||||
> 2. 用 Google 账号或邮箱注册
|
||||
> 3. 点击左侧 "API Keys"
|
||||
> 4. 点击 "Create API Key"
|
||||
> 5. 复制生成的 Key，发给我
|
||||
>
|
||||
> Groq 提供免费额度，日常使用完全够用。
|
||||
|
||||
## Agent 收到 key 后的操作
|
||||
|
||||
1. 写入配置：`config.set("groq_api_key", key)`
|
||||
2. 测试 API 可用性
|
||||
3. 反馈："✅ 语音转文字已开启！现在遇到没有字幕的视频，我也能帮你提取内容了。"
|
||||
@@ -0,0 +1,54 @@
|
||||
# Reddit 配置指南
|
||||
|
||||
## 功能说明
|
||||
|
||||
Reddit 封锁了几乎所有非浏览器的直接访问（包括数据中心和 ISP 代理 IP），JSON API 返回 403。
|
||||
|
||||
Agent Reach 通过 **rdt-cli** 实现 Reddit 的搜索和阅读功能：
|
||||
- **搜索**：`rdt search "关键词"`
|
||||
- **阅读完整帖子+评论**：`rdt read POST_ID`
|
||||
|
||||
免费，无需代理，无需 API Key。需要登录认证（`rdt login`，自动从浏览器提取 Cookie）。
|
||||
|
||||
## Agent 可自动完成的步骤
|
||||
|
||||
1. 检查 rdt-cli 是否可用：
|
||||
```bash
|
||||
which rdt && echo "installed" || echo "not installed"
|
||||
```
|
||||
|
||||
2. 如果未安装，自动安装（PyPI 版本暂时落后，从 GitHub 安装最新版）：
|
||||
```bash
|
||||
pipx install 'git+https://github.com/public-clis/rdt-cli.git'
|
||||
```
|
||||
|
||||
或一键安装：
|
||||
```bash
|
||||
agent-reach install --env=auto --channels=reddit
|
||||
```
|
||||
|
||||
## 使用示例
|
||||
|
||||
搜索 Reddit 内容：
|
||||
```bash
|
||||
rdt search "python best practices" -n 5
|
||||
```
|
||||
|
||||
阅读完整帖子和评论：
|
||||
```bash
|
||||
rdt read POST_ID
|
||||
```
|
||||
|
||||
## 需要用户手动做的步骤
|
||||
|
||||
无。rdt-cli 通过 `agent-reach install --env=auto` 自动安装。
|
||||
|
||||
## Fallback：Exa 搜索
|
||||
|
||||
如果你已经配置了 Exa（通过 mcporter），也可以通过 Exa 搜索 Reddit 内容：
|
||||
|
||||
```bash
|
||||
mcporter call 'exa.web_search_exa(query: "python best practices", numResults: 5, includeDomains: ["reddit.com"])'
|
||||
```
|
||||
|
||||
rdt-cli 是当前推荐方案，无需额外配置即可使用。
|
||||
@@ -0,0 +1,84 @@
|
||||
# Twitter 高级功能配置指南（twitter-cli）
|
||||
|
||||
Twitter 基础阅读通过 Jina Reader 免费可用，无需配置。
|
||||
|
||||
高级功能需要 twitter-cli（@public-clis/twitter-cli）：
|
||||
|
||||
- 搜索推文（`twitter search`）
|
||||
- 读取完整推文和对话链（`twitter tweet`、`twitter thread`）
|
||||
- 用户时间线（`twitter timeline`）
|
||||
- 长文阅读（`twitter article`）
|
||||
|
||||
twitter-cli 是免费开源工具（pipx 安装），但需要你的 Twitter 账号 cookie。
|
||||
|
||||
## 快速配置
|
||||
|
||||
1. 检查 twitter-cli 是否安装：
|
||||
|
||||
```bash
|
||||
which twitter && echo "installed" || echo "not installed"
|
||||
```
|
||||
|
||||
2. 安装 twitter-cli：
|
||||
|
||||
```bash
|
||||
pipx install twitter-cli
|
||||
```
|
||||
|
||||
3. 测试是否配置好：
|
||||
|
||||
```bash
|
||||
twitter search "test" -n 1
|
||||
```
|
||||
|
||||
## 获取 Cookie（Cookie-Editor 方式，推荐）
|
||||
|
||||
1. 安装 [Cookie-Editor](https://cookie-editor.com/) 浏览器扩展
|
||||
2. 登录 x.com
|
||||
3. 点击 Cookie-Editor 图标 → Export → 复制全部
|
||||
4. 运行配置命令：
|
||||
|
||||
```bash
|
||||
agent-reach configure twitter-cookies "粘贴的 cookie JSON"
|
||||
```
|
||||
|
||||
这会自动提取 `auth_token` 和 `ct0`，并写入环境变量。
|
||||
|
||||
## 手动设置 Cookie
|
||||
|
||||
如果你已经知道 `auth_token` 和 `ct0`：
|
||||
|
||||
1. 安装 twitter-cli（如果没装）：`pipx install twitter-cli`
|
||||
|
||||
2. 设置环境变量：
|
||||
|
||||
```bash
|
||||
export AUTH_TOKEN="你的auth_token"
|
||||
export CT0="你的ct0"
|
||||
```
|
||||
|
||||
3. 测试：
|
||||
|
||||
```bash
|
||||
twitter search "test" -n 1
|
||||
```
|
||||
|
||||
## 代理配置
|
||||
|
||||
> twitter-cli 支持通过环境变量设置代理：
|
||||
|
||||
```bash
|
||||
export HTTP_PROXY="http://user:pass@host:port"
|
||||
export HTTPS_PROXY="http://user:pass@host:port"
|
||||
twitter search "test" -n 1
|
||||
```
|
||||
|
||||
也可以使用全局代理工具：
|
||||
|
||||
```bash
|
||||
proxychains twitter search "test" -n 1
|
||||
```
|
||||
|
||||
## Fallback：bird CLI
|
||||
|
||||
如果你已经安装了 [bird CLI](https://www.npmjs.com/package/@steipete/bird)（`npm install -g @steipete/bird`），它也能正常工作。Agent Reach 会自动检测并使用已安装的 bird。两者功能类似，twitter-cli 是当前推荐方案。
|
||||
@@ -0,0 +1,85 @@
|
||||
# 小红书配置指南
|
||||
|
||||
## 功能说明
|
||||
读取和搜索小红书笔记。通过 [xhs-cli](https://github.com/jackwener/xiaohongshu-cli)（⭐1.5K，pipx 一行安装）实现。
|
||||
|
||||
## 前置条件
|
||||
- Python 3.10+（pipx 安装）
|
||||
- 浏览器已登录 xiaohongshu.com（用于导出 Cookie）
|
||||
|
||||
## Agent 可自动完成的步骤
|
||||
|
||||
### 1. 安装 xhs-cli
|
||||
```bash
|
||||
pipx install xiaohongshu-cli
|
||||
```
|
||||
|
||||
### 2. 登录（从浏览器提取 Cookie）
|
||||
```bash
|
||||
xhs login
|
||||
```
|
||||
|
||||
> 这会自动从浏览器提取 Cookie。如果自动提取失败，可以手动导入（见下方）。
|
||||
|
||||
### 3. 验证
|
||||
```bash
|
||||
agent-reach doctor
|
||||
```
|
||||
|
||||
应该看到小红书显示为 ✅。
|
||||
|
||||
## 需要用户手动做的步骤
|
||||
|
||||
如果 `xhs login` 自动提取失败，需要手动导入 cookies：
|
||||
|
||||
> **推荐方式：Cookie-Editor 浏览器导出（最可靠）**
|
||||
>
|
||||
> 1. 在 Chrome 中安装 [Cookie-Editor](https://chromewebstore.google.com/detail/cookie-editor/hlkenndednhfkekhgcdicdfddnkalmdm) 扩展
|
||||
> 2. 浏览器登录 xiaohongshu.com
|
||||
> 3. 点击 Cookie-Editor 图标 → Export → Header String
|
||||
> 4. 把导出的字符串发给 Agent，运行：`agent-reach configure xhs-cookies "导出的cookie字符串"`
|
||||
>
|
||||
> **注意**：不要依赖 QR 扫码登录，Cookie-Editor 导出方式最简单可靠。
|
||||
|
||||
## 使用示例
|
||||
|
||||
搜索笔记：
|
||||
```bash
|
||||
xhs search "关键词"
|
||||
```
|
||||
|
||||
阅读笔记详情：
|
||||
```bash
|
||||
xhs read NOTE_ID
|
||||
```
|
||||
|
||||
查看评论：
|
||||
```bash
|
||||
xhs comments NOTE_ID
|
||||
```
|
||||
|
||||
## 常见问题
|
||||
|
||||
**Q: Cookie 过期了？**
|
||||
A: 重新运行 `xhs login` 或通过 Cookie-Editor 重新导出。
|
||||
|
||||
**Q: 小红书提示 IP 风险？**
|
||||
A: 推荐使用住宅代理：`export HTTP_PROXY="http://user:pass@ip:port"`。
|
||||
|
||||
**Q: xhs-cli 不支持我的系统？**
|
||||
A: 确保 Python 3.10+ 和 pipx 已安装。运行 `pipx install xiaohongshu-cli` 即可。
|
||||
|
||||
## 备选方案：Docker MCP
|
||||
|
||||
如果你已经在使用 [xiaohongshu-mcp](https://github.com/xpzouying/xiaohongshu-mcp) Docker 方案，它也能正常工作：
|
||||
|
||||
```bash
|
||||
docker run -d \
|
||||
--name xiaohongshu-mcp \
|
||||
-p 18060:18060 \
|
||||
xpzouying/xiaohongshu-mcp
|
||||
|
||||
mcporter config add xiaohongshu http://localhost:18060/mcp
|
||||
```
|
||||
|
||||
xhs-cli 是当前推荐方案，不需要 Docker，安装更简单。
|
||||
@@ -0,0 +1 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
@@ -0,0 +1,67 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
Agent Reach MCP Server — expose doctor/status as MCP tool.
|
||||
|
||||
Run: python -m agent_reach.integrations.mcp_server
|
||||
|
||||
Agent Reach is an installer + doctor tool. For actual reading/searching,
|
||||
agents should call upstream tools directly (twitter-cli, yt-dlp, mcporter, etc.).
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import sys
|
||||
|
||||
from agent_reach.config import Config
|
||||
from agent_reach.core import AgentReach
|
||||
|
||||
try:
|
||||
from mcp.server import Server
|
||||
from mcp.server.stdio import stdio_server
|
||||
from mcp.types import Tool, TextContent
|
||||
HAS_MCP = True
|
||||
except ImportError:
|
||||
HAS_MCP = False
|
||||
|
||||
|
||||
def create_server():
|
||||
if not HAS_MCP:
|
||||
print("MCP not installed. Install: pip install agent-reach[mcp]", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
server = Server("agent-reach")
|
||||
config = Config()
|
||||
eyes = AgentReach(config)
|
||||
|
||||
@server.list_tools()
|
||||
async def list_tools():
|
||||
return [
|
||||
Tool(name="get_status",
|
||||
description="Get Agent Reach status: which channels are installed and active.",
|
||||
inputSchema={"type": "object", "properties": {}}),
|
||||
]
|
||||
|
||||
@server.call_tool()
|
||||
async def call_tool(name: str, arguments: dict):
|
||||
try:
|
||||
if name == "get_status":
|
||||
result = eyes.doctor_report()
|
||||
else:
|
||||
result = f"Unknown tool: {name}"
|
||||
|
||||
text = json.dumps(result, ensure_ascii=False, indent=2) if isinstance(result, (dict, list)) else str(result)
|
||||
return [TextContent(type="text", text=text)]
|
||||
except Exception as e:
|
||||
return [TextContent(type="text", text=f"Error: {str(e)}")]
|
||||
|
||||
return server
|
||||
|
||||
|
||||
async def main():
|
||||
server = create_server()
|
||||
async with stdio_server() as (read_stream, write_stream):
|
||||
await server.run(read_stream, write_stream, server.create_initialization_options())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,103 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Lightweight upstream command probing.
|
||||
|
||||
Distinguishes the three failure modes that look identical to shutil.which():
|
||||
- missing: command not on PATH
|
||||
- broken: command exists but cannot execute — most commonly a stale venv
|
||||
shebang after a system Python upgrade (pipx/uv tool installs break this
|
||||
way: which() finds the shim, but exec fails with FileNotFoundError
|
||||
pointing at the shim itself)
|
||||
- timeout/error: command runs but misbehaves
|
||||
|
||||
Channels use probe_command() inside check() so doctor reports real health,
|
||||
not just file existence.
|
||||
"""
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
from dataclasses import dataclass
|
||||
from typing import Optional, Sequence
|
||||
|
||||
from agent_reach.utils.process import utf8_subprocess_env
|
||||
|
||||
#: Exit codes shells use for "found but not executable" / "not found".
|
||||
_BROKEN_EXIT_CODES = (126, 127)
|
||||
|
||||
|
||||
@dataclass
|
||||
class ProbeResult:
|
||||
status: str # "ok" | "missing" | "broken" | "timeout" | "error"
|
||||
output: str = ""
|
||||
hint: str = ""
|
||||
|
||||
@property
|
||||
def ok(self) -> bool:
|
||||
return self.status == "ok"
|
||||
|
||||
|
||||
def reinstall_hint(package: str) -> str:
|
||||
"""Prescription for a broken (stale-venv) CLI install."""
|
||||
return (
|
||||
f"命令存在但无法执行——通常是系统 Python 升级后 venv 解释器丢失。重装即可修复：\n"
|
||||
f" uv tool install --force {package}\n"
|
||||
f"或：pipx reinstall {package}"
|
||||
)
|
||||
|
||||
|
||||
def probe_command(
|
||||
cmd: str,
|
||||
args: Sequence[str] = ("--version",),
|
||||
timeout: int = 10,
|
||||
retries: int = 0,
|
||||
package: Optional[str] = None,
|
||||
) -> ProbeResult:
|
||||
"""Actually execute `cmd *args` and classify the result.
|
||||
|
||||
Intended for SIDE-EFFECT-FREE health probes only (version/status
|
||||
commands): retries re-run the command verbatim with no backoff, so a
|
||||
non-idempotent command would repeat its effect.
|
||||
|
||||
package: pip/pipx package name used in the broken-install hint
|
||||
(defaults to cmd).
|
||||
"""
|
||||
path = shutil.which(cmd)
|
||||
if not path:
|
||||
return ProbeResult("missing")
|
||||
|
||||
last: Optional[ProbeResult] = None
|
||||
for _ in range(retries + 1):
|
||||
last = _run_once(path, args, timeout, package or cmd)
|
||||
if last.ok:
|
||||
return last
|
||||
# missing/broken won't heal between retries — only transient
|
||||
# failures (timeout/error) are worth a second attempt
|
||||
if last.status in ("missing", "broken"):
|
||||
return last
|
||||
return last
|
||||
|
||||
|
||||
def _run_once(path: str, args: Sequence[str], timeout: int, package: str) -> ProbeResult:
|
||||
try:
|
||||
r = subprocess.run(
|
||||
[path, *args],
|
||||
capture_output=True,
|
||||
encoding="utf-8",
|
||||
errors="replace",
|
||||
timeout=timeout,
|
||||
env=utf8_subprocess_env(),
|
||||
)
|
||||
except FileNotFoundError:
|
||||
# which() found it but exec failed: the shebang interpreter is gone
|
||||
return ProbeResult("broken", hint=reinstall_hint(package))
|
||||
except OSError:
|
||||
return ProbeResult("broken", hint=reinstall_hint(package))
|
||||
except subprocess.TimeoutExpired:
|
||||
return ProbeResult("timeout", hint=f"`{path}` 响应超时（>{timeout}s）")
|
||||
|
||||
if r.returncode in _BROKEN_EXIT_CODES:
|
||||
return ProbeResult("broken", hint=reinstall_hint(package))
|
||||
|
||||
output = (r.stdout or "") + (r.stderr or "")
|
||||
if r.returncode != 0:
|
||||
return ProbeResult("error", output=output.strip())
|
||||
return ProbeResult("ok", output=output.strip())
|
||||
Executable
+269
@@ -0,0 +1,269 @@
|
||||
#!/bin/bash
|
||||
# 小宇宙播客转文字脚本
|
||||
# 用法: bash transcribe.sh [--polish] <小宇宙链接> [输出文件路径]
|
||||
# 环境变量: GROQ_API_KEY (必须)
|
||||
#
|
||||
# --polish: 转录后调用 Groq Llama 3.3 70B 给文稿补中文标点+合理分段
|
||||
# （Whisper 对中文标点支持较弱，开启后阅读体验显著更好）
|
||||
|
||||
set -e
|
||||
|
||||
POLISH=0
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--polish) POLISH=1; shift ;;
|
||||
--) shift; break ;;
|
||||
-h|--help)
|
||||
echo "用法: bash transcribe.sh [--polish] <小宇宙链接> [输出文件路径]"
|
||||
exit 0 ;;
|
||||
--*)
|
||||
echo "未知选项: $1" >&2
|
||||
exit 1 ;;
|
||||
*) break ;;
|
||||
esac
|
||||
done
|
||||
|
||||
URL="${1:?用法: bash transcribe.sh [--polish] <小宇宙链接> [输出文件路径]}"
|
||||
OUTPUT="${2:-/tmp/podcast_transcript.txt}"
|
||||
TMPDIR="/tmp/xiaoyuzhou_$$"
|
||||
|
||||
# Try env var first, then agent-reach config.yaml
|
||||
if [ -z "$GROQ_API_KEY" ]; then
|
||||
CONFIG_FILE="$HOME/.agent-reach/config.yaml"
|
||||
if [ -f "$CONFIG_FILE" ]; then
|
||||
GROQ_API_KEY=$(python3 -c "import yaml; print((yaml.safe_load(open('$CONFIG_FILE')) or {}).get('groq_api_key',''))" 2>/dev/null || true)
|
||||
fi
|
||||
fi
|
||||
GROQ_API_KEY="${GROQ_API_KEY:?请设置 GROQ_API_KEY 环境变量或运行 agent-reach configure groq-key}"
|
||||
|
||||
# Groq API 限制: 25MB per file
|
||||
MAX_CHUNK_SIZE_MB=20
|
||||
AUDIO_BITRATE="64k"
|
||||
|
||||
cleanup() {
|
||||
rm -rf "$TMPDIR"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
mkdir -p "$TMPDIR"
|
||||
|
||||
echo "📻 小宇宙播客转文字"
|
||||
echo "===================="
|
||||
|
||||
# Step 1: 提取音频 URL 和标题
|
||||
echo "🔍 正在解析页面..."
|
||||
PAGE=$(curl -s "$URL")
|
||||
AUDIO_URL=$(echo "$PAGE" | perl -ne 'while (/(https:\/\/media\.xyzcdn\.net\/[^"]*\.(?:m4a|mp3))/gi) { print "$1\n" }' | head -1)
|
||||
TITLE=$(echo "$PAGE" | perl -ne 'if (/"title":"([^"]*)"/) { print "$1\n"; last }' | head -1)
|
||||
|
||||
if [ -z "$AUDIO_URL" ]; then
|
||||
echo "❌ 无法从页面提取音频链接"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "📝 标题: $TITLE"
|
||||
echo "🔗 音频: $AUDIO_URL"
|
||||
|
||||
# Step 2: 下载音频
|
||||
echo "⬇️ 正在下载音频..."
|
||||
EXT="${AUDIO_URL##*.}"
|
||||
curl -sL -o "$TMPDIR/original.$EXT" "$AUDIO_URL"
|
||||
FILE_SIZE=$(ls -lh "$TMPDIR/original.$EXT" | awk '{print $5}')
|
||||
echo "📦 文件大小: $FILE_SIZE"
|
||||
|
||||
# Step 3: 获取时长
|
||||
DURATION=$(ffprobe -v quiet -show_entries format=duration -of csv=p=0 "$TMPDIR/original.$EXT" 2>/dev/null | cut -d. -f1)
|
||||
DURATION_MIN=$((DURATION / 60))
|
||||
DURATION_SEC=$((DURATION % 60))
|
||||
echo "⏱️ 时长: ${DURATION_MIN}分${DURATION_SEC}秒"
|
||||
|
||||
# Step 4: 转为低码率单声道 MP3
|
||||
echo "🔄 正在转码..."
|
||||
ffmpeg -y -i "$TMPDIR/original.$EXT" -b:a "$AUDIO_BITRATE" -ac 1 "$TMPDIR/mono.mp3" 2>/dev/null
|
||||
MONO_SIZE=$(stat -c%s "$TMPDIR/mono.mp3" 2>/dev/null || stat -f%z "$TMPDIR/mono.mp3")
|
||||
echo "📦 转码后: $(echo "$MONO_SIZE / 1024 / 1024" | bc)MB"
|
||||
|
||||
# Step 5: 按大小切片
|
||||
MAX_BYTES=$((MAX_CHUNK_SIZE_MB * 1024 * 1024))
|
||||
|
||||
if [ "$MONO_SIZE" -le "$MAX_BYTES" ]; then
|
||||
# 不需要切片
|
||||
cp "$TMPDIR/mono.mp3" "$TMPDIR/chunk_0.mp3"
|
||||
NUM_CHUNKS=1
|
||||
echo "📎 无需切片"
|
||||
else
|
||||
# 计算需要几个 chunk
|
||||
NUM_CHUNKS=$(( (MONO_SIZE / MAX_BYTES) + 1 ))
|
||||
CHUNK_DURATION=$(( DURATION / NUM_CHUNKS + 10 )) # 加 10 秒缓冲
|
||||
echo "✂️ 切分为 $NUM_CHUNKS 段 (每段约 $((CHUNK_DURATION / 60)) 分钟)..."
|
||||
|
||||
for i in $(seq 0 $((NUM_CHUNKS - 1))); do
|
||||
START=$((i * CHUNK_DURATION))
|
||||
ffmpeg -y -i "$TMPDIR/mono.mp3" -ss "$START" -t "$CHUNK_DURATION" -c copy "$TMPDIR/chunk_${i}.mp3" 2>/dev/null
|
||||
CHUNK_SIZE=$(ls -lh "$TMPDIR/chunk_${i}.mp3" | awk '{print $5}')
|
||||
echo " 段 $((i+1))/$NUM_CHUNKS: $CHUNK_SIZE"
|
||||
done
|
||||
fi
|
||||
|
||||
# Step 6: 调用 Groq Whisper API 转录
|
||||
echo "🎙️ 正在转录 (Groq Whisper large-v3)..."
|
||||
|
||||
for i in $(seq 0 $((NUM_CHUNKS - 1))); do
|
||||
echo -n " 段 $((i+1))/$NUM_CHUNKS... "
|
||||
|
||||
RESPONSE=$(curl -s -w "\n%{http_code}" \
|
||||
https://api.groq.com/openai/v1/audio/transcriptions \
|
||||
-H "Authorization: Bearer $GROQ_API_KEY" \
|
||||
-F file="@$TMPDIR/chunk_${i}.mp3" \
|
||||
-F model="whisper-large-v3" \
|
||||
-F language="zh" \
|
||||
-F prompt="以下是一段中文普通话播客录音，请输出包含完整中文标点（，。？！：；“”‘’）的转写文本。" \
|
||||
-F response_format="text")
|
||||
|
||||
HTTP_CODE=$(echo "$RESPONSE" | tail -1)
|
||||
BODY=$(echo "$RESPONSE" | sed '$d')
|
||||
|
||||
if [ "$HTTP_CODE" != "200" ]; then
|
||||
echo "❌ API 错误 (HTTP $HTTP_CODE)"
|
||||
echo "$BODY"
|
||||
|
||||
# 如果是速率限制，等待后重试
|
||||
if [ "$HTTP_CODE" = "429" ]; then
|
||||
# 从错误信息中提取等待时间，默认 120 秒
|
||||
WAIT_SEC=$(echo "$BODY" | perl -ne 'if (/in (\d+)m/) { print "$1\n"; exit }')
|
||||
WAIT_SEC=${WAIT_SEC:-2}
|
||||
WAIT_SEC=$((WAIT_SEC * 60 + 30))
|
||||
echo " ⏳ 速率限制，等待 ${WAIT_SEC} 秒后重试..."
|
||||
sleep "$WAIT_SEC"
|
||||
RESPONSE=$(curl -s -w "\n%{http_code}" \
|
||||
https://api.groq.com/openai/v1/audio/transcriptions \
|
||||
-H "Authorization: Bearer $GROQ_API_KEY" \
|
||||
-F file="@$TMPDIR/chunk_${i}.mp3" \
|
||||
-F model="whisper-large-v3" \
|
||||
-F language="zh" \
|
||||
-F prompt="以下是一段中文普通话播客录音，请输出包含完整中文标点（，。？！：；“”‘’）的转写文本。" \
|
||||
-F response_format="text")
|
||||
HTTP_CODE=$(echo "$RESPONSE" | tail -1)
|
||||
BODY=$(echo "$RESPONSE" | sed '$d')
|
||||
|
||||
if [ "$HTTP_CODE" != "200" ]; then
|
||||
echo " ❌ 重试失败"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "$BODY" > "$TMPDIR/transcript_${i}.txt"
|
||||
CHARS=$(wc -m < "$TMPDIR/transcript_${i}.txt")
|
||||
echo "✅ ($CHARS 字)"
|
||||
done
|
||||
|
||||
# Step 6.5 (可选): 用 Llama 3.3 70B 给文稿补标点+分段
|
||||
if [ "$POLISH" = "1" ]; then
|
||||
echo "✨ 正在润色（Llama 3.3 70B 加标点+分段）..."
|
||||
for i in $(seq 0 $((NUM_CHUNKS - 1))); do
|
||||
echo -n " 段 $((i+1))/$NUM_CHUNKS... "
|
||||
IN_FILE="$TMPDIR/transcript_${i}.txt" \
|
||||
OUT_FILE="$TMPDIR/polished_${i}.txt" \
|
||||
GROQ_API_KEY="$GROQ_API_KEY" \
|
||||
python3 <<'PY'
|
||||
import json, os, sys, urllib.request, urllib.error
|
||||
|
||||
KEY = os.environ["GROQ_API_KEY"]
|
||||
IN = os.environ["IN_FILE"]
|
||||
OUT = os.environ["OUT_FILE"]
|
||||
|
||||
MODEL = "llama-3.3-70b-versatile"
|
||||
MAX_DEPTH = 3
|
||||
PROMPT_TMPL = (
|
||||
"以下是一段中文普通话播客的语音转写片段，由于 Whisper 对中文标点支持较弱，"
|
||||
"整段几乎没有标点。请你**只做一件事**：在合适位置补充中文标点（，。！？：；），"
|
||||
"可以适度分段。\n\n"
|
||||
"**严格要求**：\n"
|
||||
"- 不得修改、删除、增加任何汉字或英文/数字\n"
|
||||
"- 不得改写、润色、总结\n"
|
||||
"- 不得添加任何解释、前言、后记\n"
|
||||
"- 直接输出加好标点+合理分段后的全文\n\n"
|
||||
"原文：\n{}"
|
||||
)
|
||||
|
||||
def call_groq(text):
|
||||
body = json.dumps({
|
||||
"model": MODEL,
|
||||
"temperature": 0.2,
|
||||
"max_completion_tokens": 8192,
|
||||
"messages": [{"role": "user", "content": PROMPT_TMPL.format(text)}],
|
||||
}).encode()
|
||||
req = urllib.request.Request(
|
||||
"https://api.groq.com/openai/v1/chat/completions",
|
||||
data=body,
|
||||
headers={
|
||||
"Authorization": f"Bearer {KEY}",
|
||||
"Content-Type": "application/json",
|
||||
"User-Agent": "agent-reach-xiaoyuzhou/1.0",
|
||||
},
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=180) as r:
|
||||
resp = json.load(r)
|
||||
return (
|
||||
resp["choices"][0]["message"]["content"].strip(),
|
||||
resp["choices"][0].get("finish_reason"),
|
||||
)
|
||||
|
||||
def polish(text, depth=0):
|
||||
try:
|
||||
out, fr = call_groq(text)
|
||||
except urllib.error.HTTPError as e:
|
||||
sys.stderr.write(f"polish HTTP {e.code}: {e.read().decode(errors='replace')[:200]}\n")
|
||||
return text # fallback to raw
|
||||
except Exception as e:
|
||||
sys.stderr.write(f"polish error: {e}\n")
|
||||
return text
|
||||
if fr != "length" or depth >= MAX_DEPTH:
|
||||
return out
|
||||
# 输出被截断：从中点切两半递归处理
|
||||
mid = len(text) // 2
|
||||
return polish(text[:mid], depth + 1) + polish(text[mid:], depth + 1)
|
||||
|
||||
content = open(IN, encoding="utf-8").read().strip()
|
||||
result = polish(content)
|
||||
open(OUT, "w", encoding="utf-8").write(result + "\n")
|
||||
print(f"✅ ({len(result)} 字)")
|
||||
PY
|
||||
done
|
||||
fi
|
||||
|
||||
# Step 7: 合并输出
|
||||
echo "📄 正在合并文字稿..."
|
||||
|
||||
{
|
||||
echo "# $TITLE"
|
||||
echo ""
|
||||
echo "来源: $URL"
|
||||
echo "时长: ${DURATION_MIN}分${DURATION_SEC}秒"
|
||||
echo "转录时间: $(date '+%Y-%m-%d %H:%M')"
|
||||
if [ "$POLISH" = "1" ]; then
|
||||
echo "润色: Groq Llama 3.3 70B"
|
||||
fi
|
||||
echo ""
|
||||
echo "---"
|
||||
echo ""
|
||||
|
||||
for i in $(seq 0 $((NUM_CHUNKS - 1))); do
|
||||
if [ "$POLISH" = "1" ] && [ -f "$TMPDIR/polished_${i}.txt" ]; then
|
||||
cat "$TMPDIR/polished_${i}.txt"
|
||||
else
|
||||
cat "$TMPDIR/transcript_${i}.txt"
|
||||
fi
|
||||
echo ""
|
||||
done
|
||||
} > "$OUTPUT"
|
||||
|
||||
TOTAL_CHARS=$(wc -m < "$OUTPUT")
|
||||
echo ""
|
||||
echo "✅ 完成！"
|
||||
echo "📄 输出: $OUTPUT"
|
||||
echo "📊 总字数: $TOTAL_CHARS"
|
||||
echo "===================="
|
||||
@@ -0,0 +1,140 @@
|
||||
---
|
||||
name: agent-reach
|
||||
description: >
|
||||
MUST USE when user wants to 调研/research/搜索/search/查/找/look up anything
|
||||
on the internet — e.g. 全网调研 X / 帮我调研一下 X / 查一下 X / 搜搜 X /
|
||||
看看大家怎么评价 X / X 上有什么讨论 / research this topic。
|
||||
|
||||
Also MUST USE when user mentions any platform or shares any URL/链接:
|
||||
小红书/xiaohongshu/xhs, Twitter/推特/X, B站/bilibili, Reddit, Facebook,
|
||||
Instagram, V2EX, LinkedIn/领英/招聘/求职/jobs, YouTube, GitHub code search, 小宇宙播客,
|
||||
雪球/股票行情, RSS feeds, or any web URL.
|
||||
|
||||
15 platforms, multi-backend routing (OpenCLI / per-platform CLIs / APIs).
|
||||
Zero config for 6 channels. Run `agent-reach doctor --json` to see which
|
||||
backend serves each platform right now.
|
||||
|
||||
NOT for: 写报告/数据分析/翻译等内容加工（本 skill 只负责从互联网获取内容）；
|
||||
发帖/评论/点赞等写操作；已有专门 skill 的平台（先用专门 skill）。
|
||||
|
||||
【路由方式】SKILL.md 包含路由表和常用命令，复杂场景需按需阅读对应分类的 references/*.md。
|
||||
分类：search / social (小红书/推特/B站/V2EX/Reddit/Facebook/Instagram) / career(LinkedIn) / dev(github) / web(网页/文章/RSS) / video(YouTube/B站/播客)。
|
||||
triggers:
|
||||
- research: 调研/全网调研/帮我调研/研究一下/research/深入了解
|
||||
- search: 搜/查/找/search/搜索/查一下/帮我搜/看看大家怎么说
|
||||
- social:
|
||||
- 小红书: xiaohongshu/xhs/小红书/红书
|
||||
- Twitter: twitter/推特/x.com/推文
|
||||
- B站: bilibili/b站/哔哩哔哩
|
||||
- V2EX: v2ex
|
||||
- Reddit: reddit
|
||||
- Facebook: facebook/fb/facebook groups
|
||||
- Instagram: instagram/ig
|
||||
- career: 招聘/职位/求职/linkedin/领英/找工作
|
||||
- dev: github/代码/仓库/gh/issue/pr/分支/commit
|
||||
- web: 网页/链接/文章/rss/读一下/打开这个
|
||||
- video: youtube/视频/播客/字幕/小宇宙/转录/yt
|
||||
- finance: 雪球/股票/stock/xueqiu/行情/基金
|
||||
metadata:
|
||||
openclaw:
|
||||
homepage: https://github.com/Panniantong/Agent-Reach
|
||||
---
|
||||
|
||||
# Agent Reach — 互联网能力路由器
|
||||
|
||||
15 平台、多后端。**本 skill 存在时必须用它访问这些平台，不要自己发明方案。**
|
||||
|
||||
## 常驻规则（全程适用）
|
||||
|
||||
1. **动手前先体检**：多后端/登录态平台（小红书/Reddit/B站/Twitter/Facebook/Instagram）先跑
|
||||
`agent-reach doctor --json`，按各平台 `active_backend` 字段选命令组。
|
||||
2. **声明你在用什么**：开始干活前说一句「使用 agent-reach 的 X 平台 / Y 后端」。
|
||||
3. **失败按 references 里的重试链处理**，不要瞎猜命令。
|
||||
4. **全网调研类任务**：组合多平台（Exa 搜索 + Twitter/Reddit 看讨论 + 小红书/B站看中文场景），并行收集再汇总。
|
||||
5. **替用户盯版本**：完成一次较大的调研/多平台任务后，顺手跑
|
||||
`agent-reach check-update`（很快，一个 API 调用）。有新版就在收尾汇报里附一句：
|
||||
「Agent Reach 有新版 vX.Y.Z，复制这句话给我即可更新：帮我更新 Agent Reach：
|
||||
https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md」。
|
||||
不要中断当前任务去更新，也不要重复提醒同一个版本。
|
||||
|
||||
## 路由表
|
||||
|
||||
| 用户意图 | 分类 | 详细文档 |
|
||||
|---------|------|---------|
|
||||
| 网页搜索/代码搜索 | search | [references/search.md](references/search.md) |
|
||||
| 小红书/推特/B站/V2EX/Reddit/Facebook/Instagram | social | [references/social.md](references/social.md) |
|
||||
| 招聘/职位/LinkedIn | career | [references/career.md](references/career.md) |
|
||||
| GitHub/代码 | dev | [references/dev.md](references/dev.md) |
|
||||
| 网页/文章/RSS | web | [references/web.md](references/web.md) |
|
||||
| YouTube/B站/播客字幕 | video | [references/video.md](references/video.md) |
|
||||
|
||||
## 零配置快速命令
|
||||
|
||||
```bash
|
||||
# Exa 网页搜索
|
||||
mcporter call 'exa.web_search_exa(query: "query", numResults: 5)'
|
||||
|
||||
# 通用网页阅读
|
||||
curl -s "https://r.jina.ai/URL"
|
||||
|
||||
# GitHub 搜索
|
||||
gh search repos "query" --sort stars --limit 10
|
||||
|
||||
# YouTube 字幕（注意：B站不要用 yt-dlp，见 video.md）
|
||||
yt-dlp --write-sub --skip-download -o "/tmp/%(id)s" "URL"
|
||||
|
||||
# V2EX 热门
|
||||
curl -s "https://www.v2ex.com/api/topics/hot.json" -H "User-Agent: agent-reach/1.0"
|
||||
|
||||
# B站搜索（bili-cli，无需登录）
|
||||
bili search "query" --type video -n 5
|
||||
```
|
||||
|
||||
## 需登录态的平台（按 doctor 的 active_backend 选命令）
|
||||
|
||||
```bash
|
||||
# Twitter 搜索（twitter-cli 首选；失败重试链见 social.md）
|
||||
twitter search "query" -n 10
|
||||
|
||||
# Reddit（无零配置路径：OpenCLI 或 rdt-cli，必须登录态）
|
||||
opencli reddit search "query" -f yaml # 桌面
|
||||
rdt search "query" --limit 10 # 存量/服务器
|
||||
|
||||
# 小红书（桌面首选 OpenCLI）
|
||||
opencli xiaohongshu search "query" -f yaml
|
||||
|
||||
# Facebook / Instagram（桌面 OpenCLI，复用浏览器登录态）
|
||||
opencli facebook search "query" -f yaml
|
||||
opencli facebook groups -f yaml
|
||||
opencli instagram search "query" -f yaml # 搜用户
|
||||
opencli instagram user USERNAME -f yaml # 读指定用户最近帖子
|
||||
```
|
||||
|
||||
## 环境检查
|
||||
|
||||
```bash
|
||||
# 检查可用 channel 与每个平台当前激活的后端
|
||||
agent-reach doctor --json
|
||||
```
|
||||
|
||||
## 工作区规则
|
||||
|
||||
**不要在 agent workspace 创建文件。** 使用 `/tmp/` 存放临时输出，`~/.agent-reach/` 存放持久数据。
|
||||
|
||||
## 详细文档
|
||||
|
||||
根据用户需求，阅读对应的详细文档：
|
||||
|
||||
- [搜索工具](references/search.md) — Exa AI 搜索
|
||||
- [社交媒体](references/social.md) — 小红书, Twitter, B站, V2EX, Reddit, Facebook, Instagram（多后端/登录态命令组）
|
||||
- [职场招聘](references/career.md) — LinkedIn
|
||||
- [开发工具](references/dev.md) — GitHub CLI
|
||||
- [网页阅读](references/web.md) — Jina Reader, RSS
|
||||
- [视频播客](references/video.md) — YouTube, B站, 小宇宙
|
||||
|
||||
## 配置渠道
|
||||
|
||||
如果某个 channel 需要配置，获取安装指南：
|
||||
https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md
|
||||
|
||||
用户只需提供 cookies，其他配置由 agent 完成。
|
||||
@@ -0,0 +1,131 @@
|
||||
---
|
||||
name: agent-reach
|
||||
description: >
|
||||
MUST USE when user wants to research/search/look up/find anything on the
|
||||
internet — e.g. "research this topic", "do a deep dive on X", "search the
|
||||
web for X", "see what people say about X", "look this up".
|
||||
|
||||
Also MUST USE when user mentions any platform or shares any URL/link:
|
||||
Twitter/X, Reddit, Facebook, Instagram, YouTube, GitHub, Bilibili, XiaoHongShu,
|
||||
Xiaoyuzhou Podcast, LinkedIn/jobs/recruiting, V2EX, Xueqiu (stocks), RSS.
|
||||
|
||||
15 platforms, multi-backend routing (OpenCLI / per-platform CLIs / APIs).
|
||||
Zero config for 6 channels. Run `agent-reach doctor --json` to see which
|
||||
backend serves each platform right now.
|
||||
|
||||
NOT for: writing reports/analysis/translation (this skill only FETCHES
|
||||
internet content); posting/commenting/liking (write operations); platforms
|
||||
that already have a dedicated skill installed (prefer that skill).
|
||||
metadata:
|
||||
openclaw:
|
||||
homepage: https://github.com/Panniantong/Agent-Reach
|
||||
---
|
||||
|
||||
# Agent Reach — internet capability router
|
||||
|
||||
15 platforms, multiple backends each. **When this skill exists, use it for
|
||||
these platforms — do not invent your own approach.**
|
||||
|
||||
## Standing rules (apply for the whole session)
|
||||
|
||||
1. **Health-check before acting**: for multi-backend/login-backed platforms (XiaoHongShu /
|
||||
Reddit / Bilibili / Twitter / Facebook / Instagram), run `agent-reach doctor --json` first and
|
||||
pick the command group matching each platform's `active_backend`.
|
||||
2. **Announce what you use**: say "using agent-reach, platform X via backend Y"
|
||||
before starting.
|
||||
3. **On failure, follow the retry chains in references/** — never guess
|
||||
commands.
|
||||
4. **For broad research tasks**: combine platforms (Exa for web search +
|
||||
Twitter/Reddit for discussions + XiaoHongShu/Bilibili for Chinese
|
||||
perspectives), collect in parallel, then synthesize.
|
||||
5. **Watch versions for the user**: after finishing a substantial
|
||||
multi-platform task, run `agent-reach check-update` (fast, one API call).
|
||||
If a new version exists, append one line to your wrap-up: "Agent Reach
|
||||
vX.Y.Z is available — paste this to me to update: 帮我更新 Agent Reach：
|
||||
https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md".
|
||||
Never interrupt the current task to update; never nag about the same version twice.
|
||||
|
||||
## Routing table
|
||||
|
||||
| User intent | Category | Details |
|
||||
|---------|------|---------|
|
||||
| Web / code search | search | [references/search.md](references/search.md) |
|
||||
| XiaoHongShu / Twitter / Bilibili / V2EX / Reddit / Facebook / Instagram | social | [references/social.md](references/social.md) |
|
||||
| Jobs / LinkedIn | career | [references/career.md](references/career.md) |
|
||||
| GitHub / code | dev | [references/dev.md](references/dev.md) |
|
||||
| Web pages / articles / RSS | web | [references/web.md](references/web.md) |
|
||||
| YouTube / Bilibili / podcast transcripts | video | [references/video.md](references/video.md) |
|
||||
|
||||
## Zero-config quick commands
|
||||
|
||||
```bash
|
||||
# Exa web search
|
||||
mcporter call 'exa.web_search_exa(query: "query", numResults: 5)'
|
||||
|
||||
# Read any web page
|
||||
curl -s "https://r.jina.ai/URL"
|
||||
|
||||
# GitHub search
|
||||
gh search repos "query" --sort stars --limit 10
|
||||
|
||||
# YouTube subtitles (NOTE: never use yt-dlp for Bilibili — see video.md)
|
||||
yt-dlp --write-sub --skip-download -o "/tmp/%(id)s" "URL"
|
||||
|
||||
# V2EX hot topics
|
||||
curl -s "https://www.v2ex.com/api/topics/hot.json" -H "User-Agent: agent-reach/1.0"
|
||||
|
||||
# Bilibili search (bili-cli, no login needed)
|
||||
bili search "query" --type video -n 5
|
||||
```
|
||||
|
||||
## Login-backed platforms (pick by doctor's active_backend)
|
||||
|
||||
```bash
|
||||
# Twitter search (twitter-cli preferred; retry chain in social.md)
|
||||
twitter search "query" -n 10
|
||||
|
||||
# Reddit (NO zero-config path — OpenCLI or rdt-cli, login required)
|
||||
opencli reddit search "query" -f yaml # desktop
|
||||
rdt search "query" --limit 10 # legacy/server
|
||||
|
||||
# XiaoHongShu (desktop prefers OpenCLI)
|
||||
opencli xiaohongshu search "query" -f yaml
|
||||
|
||||
# Facebook / Instagram (desktop OpenCLI, browser session)
|
||||
opencli facebook search "query" -f yaml
|
||||
opencli facebook groups -f yaml
|
||||
opencli instagram search "query" -f yaml # user search
|
||||
opencli instagram user USERNAME -f yaml # recent posts from one user
|
||||
```
|
||||
|
||||
## Environment check
|
||||
|
||||
```bash
|
||||
# Channel availability + which backend serves each platform
|
||||
agent-reach doctor --json
|
||||
```
|
||||
|
||||
## Workspace rules
|
||||
|
||||
**Never create files in the agent workspace.** Use `/tmp/` for temporary
|
||||
output and `~/.agent-reach/` for persistent data.
|
||||
|
||||
## Detailed references
|
||||
|
||||
Read the matching file when you need specifics (commands above cover the
|
||||
common cases; references hold per-backend command groups, caveats, retry
|
||||
chains — note: reference docs are written in Chinese, commands are universal):
|
||||
|
||||
- [Search](references/search.md) — Exa AI search
|
||||
- [Social](references/social.md) — XiaoHongShu, Twitter, Bilibili, V2EX, Reddit, Facebook, Instagram (multi-backend/login-backed groups)
|
||||
- [Career](references/career.md) — LinkedIn
|
||||
- [Dev](references/dev.md) — GitHub CLI
|
||||
- [Web](references/web.md) — Jina Reader, RSS
|
||||
- [Video](references/video.md) — YouTube, Bilibili, Xiaoyuzhou
|
||||
|
||||
## Configure a channel
|
||||
|
||||
If a channel needs setup, fetch the install guide:
|
||||
https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md
|
||||
|
||||
The user only provides cookies / one extension click; the agent does the rest.
|
||||
@@ -0,0 +1,29 @@
|
||||
# 职场招聘
|
||||
|
||||
LinkedIn。
|
||||
|
||||
## LinkedIn
|
||||
|
||||
```bash
|
||||
# 获取个人资料
|
||||
mcporter call 'linkedin-scraper.get_person_profile(linkedin_url: "https://linkedin.com/in/username")'
|
||||
|
||||
# 搜索人才
|
||||
mcporter call 'linkedin-scraper.search_people(keyword: "AI engineer", limit: 10)'
|
||||
|
||||
# 获取公司资料
|
||||
mcporter call 'linkedin-scraper.get_company_profile(linkedin_url: "https://linkedin.com/company/xxx")'
|
||||
|
||||
# 搜索职位
|
||||
mcporter call 'linkedin-scraper.search_jobs(keyword: "software engineer", limit: 10)'
|
||||
```
|
||||
|
||||
> **需要登录**: LinkedIn scraper 需要有效的登录态。
|
||||
|
||||
### Fallback 方案
|
||||
|
||||
如果 MCP 不可用，可以用 Jina Reader：
|
||||
|
||||
```bash
|
||||
curl -s "https://r.jina.ai/https://linkedin.com/in/username"
|
||||
```
|
||||
@@ -0,0 +1,62 @@
|
||||
# 开发工具
|
||||
|
||||
GitHub CLI
|
||||
|
||||
## GitHub (gh CLI)
|
||||
|
||||
GitHub 官方命令行工具，用于仓库、Issue、PR、Actions、Release 以及 API 访问。
|
||||
|
||||
```bash
|
||||
# 认证
|
||||
gh auth login
|
||||
gh auth status
|
||||
|
||||
# 搜索
|
||||
gh search repos "query" --sort stars --limit 10
|
||||
gh search code "query" --language python
|
||||
|
||||
# 仓库
|
||||
gh repo view owner/repo
|
||||
gh repo clone owner/repo
|
||||
gh repo create my-repo --private
|
||||
gh repo fork owner/repo
|
||||
gh repo fork owner/repo --clone
|
||||
gh repo sync owner/repo
|
||||
|
||||
# Issues
|
||||
gh issue list -R owner/repo --state open
|
||||
gh issue view 123 -R owner/repo
|
||||
gh issue create -R owner/repo --title "Title" --body "Body"
|
||||
|
||||
# Pull Requests
|
||||
gh pr list -R owner/repo --state open
|
||||
gh pr view 123 -R owner/repo
|
||||
gh pr create -R owner/repo --title "Title" --body "Body"
|
||||
gh pr checks 123 --repo owner/repo
|
||||
|
||||
# Actions / CI
|
||||
gh run list --repo owner/repo --limit 10
|
||||
gh run view <run-id> --repo owner/repo
|
||||
gh run view <run-id> --repo owner/repo --log-failed
|
||||
gh workflow list --repo owner/repo
|
||||
|
||||
# Releases
|
||||
gh release list -R owner/repo
|
||||
gh release create v1.0.0
|
||||
|
||||
# API
|
||||
gh api /user
|
||||
gh api repos/owner/repo
|
||||
|
||||
# JSON 输出
|
||||
gh issue list --repo owner/repo --json number,title --jq '.[] | "\(.number): \(.title)"'
|
||||
```
|
||||
|
||||
|
||||
## 选择指南
|
||||
|
||||
| 工具 | 来源 | 用途 |
|
||||
|-----|------|------|
|
||||
| gh CLI | agent-reach | Git 操作 |
|
||||
| zread | my-mcp-tools | 读仓库内容 |
|
||||
| context7 | my-mcp-tools | 查技术文档 |
|
||||
@@ -0,0 +1,33 @@
|
||||
# 搜索工具
|
||||
|
||||
Exa AI 搜索引擎。
|
||||
|
||||
## Exa AI 搜索
|
||||
|
||||
高质量 AI 搜索引擎，擅长技术和代码搜索。
|
||||
|
||||
```bash
|
||||
mcporter call 'exa.web_search_exa(query: "query", numResults: 5)'
|
||||
mcporter call 'exa.get_code_context_exa(query: "code question", tokensNum: 3000)'
|
||||
```
|
||||
|
||||
### 使用场景
|
||||
|
||||
| 场景 | 参数 |
|
||||
|-----|------|
|
||||
| 网页搜索 | `web_search_exa(query: "...", numResults: 5)` |
|
||||
| 代码搜索 | `get_code_context_exa(query: "...", tokensNum: 3000)` |
|
||||
|
||||
### 特点
|
||||
|
||||
- 擅长英文内容和技术文档
|
||||
- 支持代码上下文搜索
|
||||
- 结果质量高
|
||||
|
||||
## 与其他搜索工具对比
|
||||
|
||||
| 工具 | 来源 | 适用场景 |
|
||||
|-----|------|---------|
|
||||
| Exa | agent-reach | 英文/技术/代码搜索 |
|
||||
| 智谱搜索 | my-mcp-tools | 中文搜索 |
|
||||
| GitHub 搜索 | agent-reach (dev.md) | 仓库/代码搜索 |
|
||||
@@ -0,0 +1,275 @@
|
||||
# 社交媒体 & 社区
|
||||
|
||||
小红书、Twitter/X、B站、V2EX、Reddit、Facebook、Instagram。
|
||||
|
||||
## 小红书 / XiaoHongShu（多后端）
|
||||
|
||||
小红书有三个后端，**先跑 `agent-reach doctor --json` 看 xiaohongshu 的 `active_backend` 是哪个**，再用对应命令组。
|
||||
|
||||
### 后端 A：OpenCLI（桌面首选，复用浏览器登录态）
|
||||
|
||||
```bash
|
||||
# 搜索笔记
|
||||
opencli xiaohongshu search "query" -f yaml
|
||||
|
||||
# 读笔记正文+互动数据（用搜索结果里的完整 URL，含 xsec_token）
|
||||
opencli xiaohongshu note "NOTE_URL" -f yaml
|
||||
|
||||
# 评论（支持楼中楼）
|
||||
opencli xiaohongshu comments NOTE_ID -f yaml
|
||||
|
||||
# 首页推荐 feed
|
||||
opencli xiaohongshu feed -f yaml
|
||||
|
||||
# 用户主页公开笔记
|
||||
opencli xiaohongshu user USER_ID -f yaml
|
||||
```
|
||||
|
||||
> 要求 Chrome 打开且装了 OpenCLI 扩展。报 AUTH_REQUIRED 说明浏览器里没登录小红书，让用户在 Chrome 里登录一次即可。
|
||||
|
||||
### 后端 B：xiaohongshu-mcp（服务器场景）
|
||||
|
||||
```bash
|
||||
# 未登录时：先查状态，再取二维码给用户扫
|
||||
mcporter call 'xiaohongshu.check_login_status()' --timeout 120000
|
||||
mcporter call 'xiaohongshu.get_login_qrcode()' --timeout 120000
|
||||
|
||||
# 搜索
|
||||
mcporter call 'xiaohongshu.search_feeds(keyword: "query")' --timeout 120000
|
||||
|
||||
# 笔记详情+评论（feed_id 和 xsec_token 从搜索结果取）
|
||||
mcporter call 'xiaohongshu.get_feed_detail(feed_id: "...", xsec_token: "...")' --timeout 120000
|
||||
```
|
||||
|
||||
> 首次调用会自动下载约 150MB 无头浏览器，务必带 `--timeout 120000`。未登录时 search 会挂死，先 check_login_status。
|
||||
|
||||
### 后端 C：xhs-cli（存量备选，上游 2026-03 起停更）
|
||||
|
||||
```bash
|
||||
xhs search "query" # 搜索
|
||||
xhs read NOTE_ID_OR_URL # 读笔记（必须用搜索结果中的 URL/ID，不能裸 note_id）
|
||||
xhs comments NOTE_ID_OR_URL # 评论
|
||||
xhs hot # 热门
|
||||
xhs feed # 推荐
|
||||
```
|
||||
|
||||
> 已知不稳定：`xhs user` / `xhs user-posts` / `xhs favorites` 可能返回 API error（上游停更无人修）。新装用户建议直接走后端 A/B。
|
||||
|
||||
### 通用注意事项
|
||||
|
||||
> **xsec_token 限制**: 小红书强制 xsec_token 机制，**不能直接用裸 note_id 去读**。正确流程：先搜索/feed 拿结果，再用结果中的完整 URL/ID 去读。三个后端都一样。
|
||||
>
|
||||
> **频率控制**: 高频请求（批量搜索、深翻评论）会触发验证码，平台限制无法绕过。每次操作间隔 2-3 秒。
|
||||
>
|
||||
> **写操作（发帖/评论/点赞）**: 建议只读。xhs-cli v0.6.x 写操作可能因签名问题返回 406。
|
||||
|
||||
## Twitter/X (twitter-cli)
|
||||
|
||||
### 稳定命令
|
||||
|
||||
```bash
|
||||
# 首页时间线（最稳定）
|
||||
twitter feed -n 20
|
||||
|
||||
# 读取单条推文（含回复）
|
||||
twitter tweet URL_OR_ID
|
||||
|
||||
# 读取长文 / X Article
|
||||
twitter article URL_OR_ID
|
||||
|
||||
# 用户时间线
|
||||
twitter user-posts @username -n 20
|
||||
|
||||
# 用户资料
|
||||
twitter user @username
|
||||
```
|
||||
|
||||
### 可能不稳定的命令
|
||||
|
||||
```bash
|
||||
# 搜索推文（Twitter 频繁改 GraphQL 端点，可能 404）
|
||||
twitter search "query" -n 10
|
||||
|
||||
# likes（2024 年后只能看自己的，平台限制）
|
||||
twitter likes
|
||||
```
|
||||
|
||||
### search 失败时的重试链（按序执行，成功即停）
|
||||
|
||||
1. 直接重试一次（偶发失败常见）：`twitter search "query" -n 10`
|
||||
2. 升级后再试：`pipx upgrade twitter-cli && twitter search "query" -n 10`
|
||||
3. 换 OpenCLI 备选（桌面，复用浏览器登录态）：`opencli twitter search "query" -f yaml`
|
||||
4. 都不行就改用 `twitter feed` / `twitter user-posts @somebody` 等稳定命令绕路
|
||||
|
||||
### 重要注意事项
|
||||
|
||||
> **安装**: `pipx install twitter-cli`（确保 v0.8.5+）
|
||||
>
|
||||
> **认证**: 推荐用 Cookie-Editor 导出后设置环境变量 `TWITTER_AUTH_TOKEN` + `TWITTER_CT0`。自动提取在 SSH/Docker/无头环境不可用。
|
||||
>
|
||||
> **IP 风控**: 不要在 VPS/数据中心 IP 上频繁调用，尤其是 followers/following，有封号风险。使用住宅代理或本地环境。
|
||||
>
|
||||
> **OpenCLI 备选**: 桌面装了 OpenCLI 的话，`opencli twitter search/article/user-posts -f yaml` 全套可用（浏览器登录态，无需 cookie 环境变量）。
|
||||
>
|
||||
> **输出格式**: 建议用 `--yaml` 或 `--json` 获得结构化输出，对 AI agent 更友好。
|
||||
|
||||
## B站 / Bilibili
|
||||
|
||||
> ⚠️ **不要用 yt-dlp 读 B站**（风控已全面 412 拦截，实测无解）。用 bili-cli / OpenCLI。
|
||||
|
||||
```bash
|
||||
# 搜索 / 热门 / 视频详情（bili-cli，只读无需登录）
|
||||
bili search "query" --type video -n 5
|
||||
bili hot -n 10
|
||||
bili video BVxxx
|
||||
|
||||
# 字幕（OpenCLI，需桌面 Chrome）
|
||||
opencli bilibili subtitle BVxxx
|
||||
```
|
||||
|
||||
> 详细命令（音频转写、API 直连兜底）见 [references/video.md](video.md)。
|
||||
|
||||
## V2EX (公开 API)
|
||||
|
||||
无需认证，直接调用公开 API。
|
||||
|
||||
### 热门主题
|
||||
|
||||
```bash
|
||||
curl -s "https://www.v2ex.com/api/topics/hot.json" -H "User-Agent: agent-reach/1.0"
|
||||
```
|
||||
|
||||
### 节点主题
|
||||
|
||||
```bash
|
||||
# node_name 如: python, tech, jobs, qna, programmers
|
||||
curl -s "https://www.v2ex.com/api/topics/show.json?node_name=python&page=1" -H "User-Agent: agent-reach/1.0"
|
||||
```
|
||||
|
||||
### 主题详情
|
||||
|
||||
```bash
|
||||
# topic_id 从 URL 获取，如 https://www.v2ex.com/t/1234567
|
||||
curl -s "https://www.v2ex.com/api/topics/show.json?id=TOPIC_ID" -H "User-Agent: agent-reach/1.0"
|
||||
```
|
||||
|
||||
### 主题回复
|
||||
|
||||
```bash
|
||||
curl -s "https://www.v2ex.com/api/replies/show.json?topic_id=TOPIC_ID&page=1" -H "User-Agent: agent-reach/1.0"
|
||||
```
|
||||
|
||||
### 用户信息
|
||||
|
||||
```bash
|
||||
curl -s "https://www.v2ex.com/api/members/show.json?username=USERNAME" -H "User-Agent: agent-reach/1.0"
|
||||
```
|
||||
|
||||
### Python 调用示例
|
||||
|
||||
```python
|
||||
from agent_reach.channels.v2ex import V2EXChannel
|
||||
|
||||
ch = V2EXChannel()
|
||||
|
||||
# 获取热门帖子
|
||||
topics = ch.get_hot_topics(limit=10)
|
||||
for t in topics:
|
||||
print(f"[{t['node_title']}] {t['title']} ({t['replies']} 回复)")
|
||||
|
||||
# 获取节点帖子
|
||||
node_topics = ch.get_node_topics("python", limit=5)
|
||||
|
||||
# 获取帖子详情 + 回复
|
||||
topic = ch.get_topic(1234567)
|
||||
print(topic["title"], "—", topic["author"])
|
||||
|
||||
# 获取用户信息
|
||||
user = ch.get_user("Livid")
|
||||
```
|
||||
|
||||
> **节点列表**: https://www.v2ex.com/planes
|
||||
|
||||
## Reddit（多后端，必须登录态）
|
||||
|
||||
**Reddit 没有零配置路径**：匿名 `.json` 端点已被封（403），官方 API 自 2025-11 起人工审批基本不批。两个后端都靠登录态，先跑 `agent-reach doctor --json` 看 reddit 的 `active_backend`。中国大陆访问需代理。
|
||||
|
||||
### 后端 A：OpenCLI（桌面首选，复用浏览器登录态）
|
||||
|
||||
```bash
|
||||
# 搜索帖子
|
||||
opencli reddit search "query" -f yaml
|
||||
|
||||
# 读帖子全文 + 评论
|
||||
opencli reddit read POST_ID -f yaml
|
||||
|
||||
# 浏览 subreddit / 热门 / Popular
|
||||
opencli reddit subreddit LocalLLaMA -f yaml
|
||||
opencli reddit hot -f yaml
|
||||
opencli reddit popular -f yaml
|
||||
|
||||
# subreddit 元信息（订阅数、简介）
|
||||
opencli reddit subreddit-info LocalLLaMA -f yaml
|
||||
```
|
||||
|
||||
> 要求 Chrome 打开且浏览器里登录过 reddit.com。
|
||||
|
||||
### 后端 B：rdt-cli（存量/服务器备选，上游 2026-03 起停更）
|
||||
|
||||
```bash
|
||||
rdt search "query" --limit 10 # 搜索帖子
|
||||
rdt read POST_ID # 读帖子全文 + 评论
|
||||
rdt sub python --limit 20 # 浏览 subreddit
|
||||
rdt popular --limit 10 # 浏览热门
|
||||
rdt all --limit 10 # 浏览 /r/all
|
||||
```
|
||||
|
||||
> **安装**: `pipx install 'git+https://github.com/public-clis/rdt-cli.git'`（PyPI 版本落后，需从 GitHub 装 v0.4.2+）。先 `rdt login` 才能搜索和阅读（服务器无浏览器时手动写 Cookie，见 doctor 提示）。
|
||||
> 建议使用 `--yaml` 输出，对 AI agent 更友好。
|
||||
|
||||
### 高级选项：官方 API + PRAW（仅限已有凭证的用户）
|
||||
|
||||
2025-11 前注册过 Reddit script app（持有 client_id/client_secret）的用户可以用 PRAW 走官方 API（100 QPM 免费）。新申请需人工审批且个人项目基本不批，**不要推荐新用户走这条路**。
|
||||
|
||||
## Facebook（OpenCLI，必须登录态）
|
||||
|
||||
Facebook 走 OpenCLI，复用用户 Chrome 里的 facebook.com 登录态。先跑 `agent-reach doctor --json` 看 facebook 的 `active_backend`，正常应为 `OpenCLI`。不要推荐 Jina/Exa/Graph API 作为默认路径。
|
||||
|
||||
```bash
|
||||
# 搜索用户 / 主页 / 帖子
|
||||
opencli facebook search "query" -f yaml
|
||||
|
||||
# 用户或主页信息
|
||||
opencli facebook profile zuck -f yaml
|
||||
|
||||
# 当前账号 News Feed
|
||||
opencli facebook feed --limit 10 -f yaml
|
||||
|
||||
# 当前账号可见的群组列表/最近动态
|
||||
opencli facebook groups --limit 20 -f yaml
|
||||
```
|
||||
|
||||
> 要求 Chrome 打开且装了 OpenCLI 扩展，并已登录 facebook.com。Facebook Groups 当前只承诺读取当前账号可见的群组列表/最近动态，不承诺任意群帖子和评论 API。
|
||||
|
||||
## Instagram（OpenCLI，必须登录态）
|
||||
|
||||
Instagram 走 OpenCLI，复用用户 Chrome 里的 instagram.com 登录态。先跑 `agent-reach doctor --json` 看 instagram 的 `active_backend`，正常应为 `OpenCLI`。不要默认恢复 instaloader；历史上 cookies/401/429 不稳定。
|
||||
|
||||
```bash
|
||||
# 搜索用户（不是全站帖子关键词搜索）
|
||||
opencli instagram search "query" -f yaml
|
||||
|
||||
# 用户 Profile
|
||||
opencli instagram profile nasa -f yaml
|
||||
|
||||
# 用户最近帖子
|
||||
opencli instagram user nasa --limit 12 -f yaml
|
||||
|
||||
# Explore / Discover
|
||||
opencli instagram explore --limit 20 -f yaml
|
||||
|
||||
# 当前账号收藏
|
||||
opencli instagram saved --limit 20 -f yaml
|
||||
```
|
||||
|
||||
> 要求 Chrome 打开且装了 OpenCLI 扩展，并已登录 instagram.com。`instagram search` 是用户搜索；读帖子需要先确定 username，再用 `instagram user USERNAME`。若出现 429 / login required，先让用户在 Chrome 里重新登录并降低频率。
|
||||
@@ -0,0 +1,131 @@
|
||||
# 视频/播客
|
||||
|
||||
YouTube、B站、小宇宙播客的字幕和转录。
|
||||
|
||||
## YouTube (yt-dlp)
|
||||
|
||||
### 获取视频元数据
|
||||
|
||||
```bash
|
||||
yt-dlp --dump-json "URL"
|
||||
```
|
||||
|
||||
### 下载字幕
|
||||
|
||||
```bash
|
||||
# 下载字幕 (不下载视频)
|
||||
yt-dlp --write-sub --write-auto-sub --sub-lang "zh-Hans,zh,en" --skip-download -o "/tmp/%(id)s" "URL"
|
||||
|
||||
# 然后读取 .vtt 文件
|
||||
cat /tmp/VIDEO_ID.*.vtt
|
||||
```
|
||||
|
||||
### 获取评论
|
||||
|
||||
```bash
|
||||
# 提取评论（best-effort，不保证完整）
|
||||
yt-dlp --write-comments --skip-download --write-info-json \
|
||||
--extractor-args "youtube:max_comments=20" \
|
||||
-o "/tmp/%(id)s" "URL"
|
||||
# 评论在 .info.json 的 comments 字段中
|
||||
```
|
||||
|
||||
### 搜索视频
|
||||
|
||||
```bash
|
||||
yt-dlp --dump-json "ytsearch5:query"
|
||||
```
|
||||
|
||||
> **字幕注意**: 手动上传的字幕提取可靠；自动生成字幕可能存在行间重复，需后处理。
|
||||
> **评论注意**: `--write-comments` 基于网页抓取（非 YouTube Data API），部分评论可能丢失。
|
||||
|
||||
### 无字幕兜底：Whisper 音频转写
|
||||
|
||||
```bash
|
||||
# 视频没有字幕时的兜底：下载音频并用 Whisper 转写（Groq 免费 key 即可）
|
||||
agent-reach transcribe "https://www.youtube.com/watch?v=VIDEO_ID"
|
||||
agent-reach transcribe ./local_audio.mp3 -o /tmp/transcript.txt
|
||||
```
|
||||
|
||||
> `agent-reach transcribe` 只接收公开 http(s) URL 或本地音频文件。用 `ytsearch5:` 搜索时，先从 yt-dlp 结果里选出具体视频 URL，再转写。
|
||||
> 需要先配置 key：`agent-reach configure groq-key gsk_xxx`（免费，console.groq.com）
|
||||
> 或 `agent-reach configure openai-key sk-xxx`。默认 auto 模式：groq 失败自动降级 openai。
|
||||
|
||||
## B站 / Bilibili（bili-cli 为主，OpenCLI 补字幕）
|
||||
|
||||
> ⚠️ **不要用 yt-dlp 读 B站**：B站风控已全面 412 拦截 yt-dlp（实测最新版、直连/代理/带 Cookie 全部无效）。yt-dlp 只用于 YouTube。
|
||||
|
||||
### 视频详情/搜索/热门/排行 (bili-cli，只读无需登录)
|
||||
|
||||
```bash
|
||||
# 视频详情（标题/UP主/时长/播放互动数据/字幕可用性）
|
||||
bili video BVxxx
|
||||
|
||||
# 搜索视频
|
||||
bili search "query" --type video -n 5
|
||||
|
||||
# 热门视频 / 排行榜
|
||||
bili hot -n 10
|
||||
bili rank -n 10
|
||||
|
||||
# 下载音频并切分为 ASR-ready WAV（无字幕时配合 agent-reach transcribe 转写）
|
||||
bili audio BVxxx
|
||||
```
|
||||
|
||||
### 字幕 (OpenCLI，需要桌面 Chrome)
|
||||
|
||||
```bash
|
||||
# 字幕逐句带时间轴
|
||||
opencli bilibili subtitle BVxxx
|
||||
|
||||
# OpenCLI 也能搜索/读视频元数据（备选）
|
||||
opencli bilibili search "query" -f yaml
|
||||
opencli bilibili video BVxxx -f yaml
|
||||
```
|
||||
|
||||
### 零配置兜底：搜索 API 直连
|
||||
|
||||
```bash
|
||||
UA="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36"
|
||||
curl -s -c /tmp/bili_ck.txt -o /dev/null -A "$UA" "https://www.bilibili.com/"
|
||||
curl -s -b /tmp/bili_ck.txt -A "$UA" -e "https://www.bilibili.com/" \
|
||||
"https://api.bilibili.com/x/web-interface/search/all/v2?keyword=QUERY&page=1"
|
||||
```
|
||||
|
||||
> **安装 bili-cli**: `pipx install bilibili-cli`（上游 2026-03 起停更但实测健康；只读场景无需登录，`bili login` 扫码可解锁动态/收藏等个人功能）。
|
||||
|
||||
## 小宇宙播客 / Xiaoyuzhou Podcast
|
||||
|
||||
### 转录单集播客（可选 --polish 增强标点）
|
||||
|
||||
```bash
|
||||
# 输出 Markdown 文件到 /tmp/。--polish 让 Llama 3.3 70B 给文稿补中文标点+合理分段
|
||||
~/.agent-reach/tools/xiaoyuzhou/transcribe.sh --polish "https://www.xiaoyuzhoufm.com/episode/EPISODE_ID"
|
||||
```
|
||||
|
||||
> 转写 prompt 已要求 Whisper 输出中文标点；若标点效果仍不理想，可加 `--polish` 用 Groq 上免费的 Llama 3.3 70B 补标点+合理分段（9 分钟播客约多 ~7 秒）。每次转写多一轮 LLM 调用，按需使用。
|
||||
|
||||
### 前置要求
|
||||
|
||||
1. **ffmpeg**: `brew install ffmpeg`
|
||||
2. **Groq API Key** (免费): https://console.groq.com/keys
|
||||
3. **配置 Key**: `agent-reach configure groq-key YOUR_KEY`
|
||||
4. **首次运行**: `agent-reach install --env=auto` 安装工具
|
||||
|
||||
### 检查状态
|
||||
|
||||
```bash
|
||||
agent-reach doctor
|
||||
```
|
||||
|
||||
> 输出 Markdown 文件默认保存到 `/tmp/`。
|
||||
|
||||
## 选择指南
|
||||
|
||||
| 场景 | 推荐工具 |
|
||||
|-----|---------|
|
||||
| YouTube 字幕 | yt-dlp |
|
||||
| B站视频详情/搜索 | bili-cli |
|
||||
| B站字幕 | opencli bilibili subtitle |
|
||||
| 播客转录 | 小宇宙 transcribe.sh |
|
||||
| 无字幕音视频 | agent-reach transcribe（B站音频先 `bili audio`） |
|
||||
@@ -0,0 +1,50 @@
|
||||
# 网页阅读
|
||||
|
||||
通用网页、RSS。
|
||||
|
||||
## 通用网页 (Jina Reader)
|
||||
|
||||
```bash
|
||||
# 读取任意网页内容
|
||||
curl -s "https://r.jina.ai/URL"
|
||||
|
||||
# 示例
|
||||
curl -s "https://r.jina.ai/https://example.com/article"
|
||||
```
|
||||
|
||||
**适用场景**: 大多数网页可以直接用 Jina Reader 读取。
|
||||
|
||||
## Web Reader (MCP)
|
||||
|
||||
```bash
|
||||
# 读取网页内容 (Markdown 格式)
|
||||
mcporter call 'web-reader.webReader(url: "https://example.com")'
|
||||
|
||||
# 保留图片
|
||||
mcporter call 'web-reader.webReader(url: "https://example.com", retain_images: true)'
|
||||
|
||||
# 纯文本格式
|
||||
mcporter call 'web-reader.webReader(url: "https://example.com", return_format: "text")'
|
||||
```
|
||||
|
||||
**适用场景**: 需要更精确控制输出格式时使用。
|
||||
|
||||
## RSS (feedparser)
|
||||
|
||||
```python
|
||||
python3 -c "
|
||||
import feedparser
|
||||
for e in feedparser.parse('FEED_URL').entries[:5]:
|
||||
print(f'{e.title} — {e.link}')
|
||||
"
|
||||
```
|
||||
|
||||
**适用场景**: 订阅博客、新闻源、播客等 RSS feed。
|
||||
|
||||
## 选择指南
|
||||
|
||||
| 场景 | 推荐工具 |
|
||||
|-----|---------|
|
||||
| 通用网页 | Jina Reader (`curl r.jina.ai`) |
|
||||
| 需要图片/格式控制 | web-reader MCP |
|
||||
| RSS 订阅 | feedparser |
|
||||
@@ -0,0 +1,318 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Whisper audio transcription with Groq → OpenAI fallback.
|
||||
|
||||
Downloads audio (yt-dlp), compresses + chunks (ffmpeg), and posts to a
|
||||
Whisper-compatible API. Defaults to Groq's free `whisper-large-v3` and falls
|
||||
back to OpenAI's `whisper-1` on HTTP error.
|
||||
|
||||
Public entry point:
|
||||
transcribe(source, *, provider="auto", out_dir=None, config=None) -> str
|
||||
|
||||
Designed to be importable from channels (e.g. YouTubeChannel.transcribe).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import List, Optional
|
||||
from urllib.parse import urlparse
|
||||
|
||||
import requests
|
||||
|
||||
from agent_reach.config import Config
|
||||
|
||||
# Whisper API limit is 25MB; leave headroom for multipart overhead.
|
||||
SIZE_LIMIT_BYTES = 24 * 1024 * 1024
|
||||
CHUNK_SECONDS = 600 # 10 min — small enough that boundary cuts rarely lose meaning
|
||||
|
||||
PROVIDERS = {
|
||||
"groq": {
|
||||
"endpoint": "https://api.groq.com/openai/v1/audio/transcriptions",
|
||||
"model": "whisper-large-v3",
|
||||
"key_field": "groq_api_key",
|
||||
},
|
||||
"openai": {
|
||||
"endpoint": "https://api.openai.com/v1/audio/transcriptions",
|
||||
"model": "whisper-1",
|
||||
"key_field": "openai_api_key",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
class TranscribeError(RuntimeError):
|
||||
"""Raised when transcription cannot complete."""
|
||||
|
||||
|
||||
class MissingDependency(TranscribeError):
|
||||
"""Raised when a required external binary is missing."""
|
||||
|
||||
|
||||
class NoProviderConfigured(TranscribeError):
|
||||
"""Raised when no provider has an API key configured."""
|
||||
|
||||
|
||||
_BLOCKED_HOSTS = {
|
||||
"localhost",
|
||||
"metadata.google.internal",
|
||||
}
|
||||
|
||||
|
||||
def _require(binary: str) -> None:
|
||||
if not shutil.which(binary):
|
||||
raise MissingDependency(f"{binary} not found in PATH")
|
||||
|
||||
|
||||
def _run(cmd: List[str], timeout: int = 600) -> None:
|
||||
"""Run a subprocess, raising TranscribeError on nonzero exit or timeout.
|
||||
|
||||
cmd carries user-supplied URLs/paths into yt-dlp/ffmpeg — a stalled
|
||||
network read or a hung probe must not block the CLI forever.
|
||||
"""
|
||||
try:
|
||||
proc = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout)
|
||||
except subprocess.TimeoutExpired:
|
||||
raise TranscribeError(f"{cmd[0]} timed out after {timeout}s")
|
||||
if proc.returncode != 0:
|
||||
raise TranscribeError(
|
||||
f"{cmd[0]} failed (exit {proc.returncode}): {proc.stderr.strip()[:300]}"
|
||||
)
|
||||
|
||||
|
||||
def _is_private_ip(value: str) -> bool:
|
||||
try:
|
||||
ip = ipaddress.ip_address(value)
|
||||
except ValueError:
|
||||
return False
|
||||
return any(
|
||||
(
|
||||
ip.is_private,
|
||||
ip.is_loopback,
|
||||
ip.is_link_local,
|
||||
ip.is_reserved,
|
||||
ip.is_multicast,
|
||||
ip.is_unspecified,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _assert_safe_public_url(url: str) -> None:
|
||||
"""Reject literal local/internal URLs without DNS-resolving public hosts."""
|
||||
if "://" not in url:
|
||||
before_slash = url.split("/", 1)[0]
|
||||
if ":" in before_slash:
|
||||
host_part, port_part = before_slash.rsplit(":", 1)
|
||||
if not host_part or not port_part.isdigit():
|
||||
raise TranscribeError("SSRF blocked: only public http(s) URLs are allowed")
|
||||
parsed = urlparse(f"https://{url}")
|
||||
else:
|
||||
parsed = urlparse(url)
|
||||
if parsed.scheme not in {"http", "https"}:
|
||||
raise TranscribeError("SSRF blocked: only public http(s) URLs are allowed")
|
||||
|
||||
host = (parsed.hostname or "").strip().lower().rstrip(".")
|
||||
if not host:
|
||||
raise TranscribeError("SSRF blocked: URL host is missing")
|
||||
if host in _BLOCKED_HOSTS or host.endswith(".localhost"):
|
||||
raise TranscribeError("SSRF blocked: internal host is not allowed")
|
||||
if _is_private_ip(host):
|
||||
raise TranscribeError("SSRF blocked: private/internal IP is not allowed")
|
||||
|
||||
|
||||
def download_audio(url: str, out_dir: Path) -> Path:
|
||||
"""Download audio with yt-dlp into out_dir; return the resulting file path."""
|
||||
_assert_safe_public_url(url)
|
||||
_require("yt-dlp")
|
||||
template = out_dir / "source.%(ext)s"
|
||||
_run(
|
||||
[
|
||||
"yt-dlp",
|
||||
"-x",
|
||||
"--audio-format",
|
||||
"m4a",
|
||||
"--audio-quality",
|
||||
"0",
|
||||
"-o",
|
||||
str(template),
|
||||
"--",
|
||||
url,
|
||||
],
|
||||
timeout=1800, # long podcasts over slow networks — generous but bounded
|
||||
)
|
||||
files = sorted(out_dir.glob("source.*"))
|
||||
if not files:
|
||||
raise TranscribeError("yt-dlp produced no output file")
|
||||
return files[0]
|
||||
|
||||
|
||||
def compress_audio(src: Path, out_dir: Path) -> Path:
|
||||
"""Re-encode to mono / 16kHz / 32kbps m4a — keeps most content under 25MB."""
|
||||
_require("ffmpeg")
|
||||
dst = out_dir / "compressed.m4a"
|
||||
_run(
|
||||
[
|
||||
"ffmpeg",
|
||||
"-loglevel",
|
||||
"error",
|
||||
"-y",
|
||||
"-i",
|
||||
str(src),
|
||||
"-vn",
|
||||
"-ac",
|
||||
"1",
|
||||
"-ar",
|
||||
"16000",
|
||||
"-b:a",
|
||||
"32k",
|
||||
str(dst),
|
||||
]
|
||||
)
|
||||
return dst
|
||||
|
||||
|
||||
def chunk_audio(src: Path, out_dir: Path, segment_seconds: int = CHUNK_SECONDS) -> List[Path]:
|
||||
"""Split src into segments. Re-encodes each segment so cuts align to keyframes."""
|
||||
_require("ffmpeg")
|
||||
pattern = out_dir / "chunk_%03d.m4a"
|
||||
_run(
|
||||
[
|
||||
"ffmpeg",
|
||||
"-loglevel",
|
||||
"error",
|
||||
"-y",
|
||||
"-i",
|
||||
str(src),
|
||||
"-f",
|
||||
"segment",
|
||||
"-segment_time",
|
||||
str(segment_seconds),
|
||||
"-ac",
|
||||
"1",
|
||||
"-ar",
|
||||
"16000",
|
||||
"-b:a",
|
||||
"32k",
|
||||
str(pattern),
|
||||
]
|
||||
)
|
||||
chunks = sorted(out_dir.glob("chunk_*.m4a"))
|
||||
if not chunks:
|
||||
raise TranscribeError("ffmpeg produced no chunks")
|
||||
return chunks
|
||||
|
||||
|
||||
def _provider_key(provider: str, config: Config) -> Optional[str]:
|
||||
field = PROVIDERS[provider]["key_field"]
|
||||
val = config.get(field)
|
||||
return val or None
|
||||
|
||||
|
||||
def transcribe_chunk(
|
||||
chunk: Path,
|
||||
provider: str,
|
||||
*,
|
||||
config: Optional[Config] = None,
|
||||
timeout: int = 120,
|
||||
) -> str:
|
||||
"""Transcribe one chunk via the named provider. Raises TranscribeError on failure."""
|
||||
if provider not in PROVIDERS:
|
||||
raise TranscribeError(f"unknown provider: {provider}")
|
||||
cfg = config or Config()
|
||||
key = _provider_key(provider, cfg)
|
||||
if not key:
|
||||
raise NoProviderConfigured(
|
||||
f"{provider}: missing {PROVIDERS[provider]['key_field']} "
|
||||
f"(configure with `agent-reach configure {provider}-key ...`)"
|
||||
)
|
||||
|
||||
info = PROVIDERS[provider]
|
||||
with chunk.open("rb") as fh:
|
||||
try:
|
||||
resp = requests.post(
|
||||
info["endpoint"],
|
||||
headers={"Authorization": f"Bearer {key}"},
|
||||
files={"file": (chunk.name, fh, "audio/m4a")},
|
||||
data={"model": info["model"], "response_format": "text"},
|
||||
timeout=timeout,
|
||||
)
|
||||
except requests.RequestException as e:
|
||||
raise TranscribeError(f"{provider}: network error: {e}") from e
|
||||
|
||||
if not resp.ok:
|
||||
raise TranscribeError(f"{provider}: HTTP {resp.status_code}: {resp.text[:300]}")
|
||||
return resp.text
|
||||
|
||||
|
||||
def _provider_order(provider: str) -> List[str]:
|
||||
if provider == "auto":
|
||||
return ["groq", "openai"]
|
||||
if provider in PROVIDERS:
|
||||
return [provider]
|
||||
raise TranscribeError(f"unknown provider: {provider} (use groq|openai|auto)")
|
||||
|
||||
|
||||
def transcribe(
|
||||
source: str,
|
||||
*,
|
||||
provider: str = "auto",
|
||||
out_dir: Optional[Path] = None,
|
||||
config: Optional[Config] = None,
|
||||
) -> str:
|
||||
"""Transcribe a URL or local file path. Returns the joined transcript text.
|
||||
|
||||
`provider` is one of `auto` (groq → openai), `groq`, or `openai`.
|
||||
`out_dir` defaults to a fresh temp directory; intermediate files stay there.
|
||||
"""
|
||||
cfg = config or Config()
|
||||
order = _provider_order(provider)
|
||||
|
||||
# Validate at least one provider is configured before doing expensive work.
|
||||
if not any(_provider_key(p, cfg) for p in order):
|
||||
names = ", ".join(PROVIDERS[p]["key_field"] for p in order)
|
||||
raise NoProviderConfigured(f"no provider key configured (need one of: {names})")
|
||||
|
||||
if out_dir:
|
||||
return _transcribe_in_dir(source, order, cfg, Path(out_dir))
|
||||
|
||||
with tempfile.TemporaryDirectory(prefix="transcribe-") as tmp:
|
||||
return _transcribe_in_dir(source, order, cfg, Path(tmp))
|
||||
|
||||
|
||||
def _transcribe_in_dir(source: str, order: List[str], cfg: Config, work_dir: Path) -> str:
|
||||
work_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
src_path = Path(source)
|
||||
if src_path.is_file():
|
||||
audio = src_path
|
||||
else:
|
||||
audio = download_audio(source, work_dir)
|
||||
|
||||
compressed = compress_audio(audio, work_dir)
|
||||
if compressed.stat().st_size <= SIZE_LIMIT_BYTES:
|
||||
chunks = [compressed]
|
||||
else:
|
||||
chunks = chunk_audio(compressed, work_dir)
|
||||
|
||||
pieces: List[str] = []
|
||||
for chunk in chunks:
|
||||
text = _transcribe_with_fallback(chunk, order, cfg)
|
||||
pieces.append(text.strip())
|
||||
return "\n".join(p for p in pieces if p)
|
||||
|
||||
|
||||
def _transcribe_with_fallback(chunk: Path, order: List[str], config: Config) -> str:
|
||||
"""Try each provider in order; return first success or raise the last error."""
|
||||
last_err: Optional[Exception] = None
|
||||
for p in order:
|
||||
if not _provider_key(p, config):
|
||||
# Skip silently — caller already validated at least one is configured.
|
||||
continue
|
||||
try:
|
||||
return transcribe_chunk(chunk, p, config=config)
|
||||
except TranscribeError as e:
|
||||
last_err = e
|
||||
continue
|
||||
raise TranscribeError(f"all providers failed for {chunk.name}: {last_err}")
|
||||
@@ -0,0 +1,54 @@
|
||||
"""Cross-platform path and remediation helpers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def make_private_dir(path: str | Path) -> Path:
|
||||
"""Create a directory restricted to the current user where supported."""
|
||||
target = Path(path)
|
||||
target.mkdir(mode=0o700, parents=True, exist_ok=True)
|
||||
if sys.platform != "win32":
|
||||
os.chmod(target, 0o700)
|
||||
return target
|
||||
|
||||
|
||||
def get_ytdlp_config_dir() -> Path:
|
||||
"""Return the recommended yt-dlp user config directory for this OS."""
|
||||
|
||||
if sys.platform == "win32":
|
||||
appdata = os.environ.get("APPDATA")
|
||||
if appdata:
|
||||
return Path(appdata) / "yt-dlp"
|
||||
return Path.home() / "AppData" / "Roaming" / "yt-dlp"
|
||||
if sys.platform == "darwin":
|
||||
return Path.home() / "Library" / "Application Support" / "yt-dlp"
|
||||
return Path.home() / ".config" / "yt-dlp"
|
||||
|
||||
|
||||
def get_ytdlp_config_path() -> Path:
|
||||
"""Return the yt-dlp user config file path for this OS."""
|
||||
|
||||
return get_ytdlp_config_dir() / "config"
|
||||
|
||||
|
||||
def render_ytdlp_fix_command() -> str:
|
||||
"""Return an OS-appropriate command to enable Node.js as yt-dlp JS runtime."""
|
||||
|
||||
config_path = get_ytdlp_config_path()
|
||||
if sys.platform == "win32":
|
||||
return (
|
||||
f"$cfg = '{config_path}'\n"
|
||||
"New-Item -ItemType Directory -Force -Path (Split-Path $cfg) | Out-Null\n"
|
||||
"if (-not (Test-Path $cfg) -or -not (Select-String -Path $cfg -Pattern '--js-runtimes' -Quiet)) {\n"
|
||||
" Add-Content -Path $cfg -Value '--js-runtimes node'\n"
|
||||
"}"
|
||||
)
|
||||
return (
|
||||
f"mkdir -p '{config_path.parent}' && "
|
||||
f"grep -qxF -- '--js-runtimes node' '{config_path}' 2>/dev/null || "
|
||||
f"printf '%s\n' '--js-runtimes node' >> '{config_path}'"
|
||||
)
|
||||
@@ -0,0 +1,26 @@
|
||||
"""Subprocess helpers for consistent cross-platform text handling."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from collections.abc import Mapping
|
||||
|
||||
UTF8_ENV = {
|
||||
"PYTHONUTF8": "1",
|
||||
"PYTHONIOENCODING": "utf-8",
|
||||
}
|
||||
|
||||
|
||||
def utf8_subprocess_env(base: Mapping[str, str] | None = None) -> dict[str, str]:
|
||||
"""Return an environment that forces Python child processes into UTF-8 mode."""
|
||||
env = dict(base or os.environ)
|
||||
env.update(UTF8_ENV)
|
||||
return env
|
||||
|
||||
|
||||
def mcporter_utf8_env_args() -> list[str]:
|
||||
"""Return mcporter --env arguments for UTF-8 Python stdio servers."""
|
||||
args = []
|
||||
for key, value in UTF8_ENV.items():
|
||||
args.extend(["--env", f"{key}={value}"])
|
||||
return args
|
||||
@@ -0,0 +1,14 @@
|
||||
"""UTF-8-safe text helpers for cross-platform file operations."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def read_utf8_text(path: str | Path, default: str = "") -> str:
|
||||
"""Read text as UTF-8 with replacement semantics."""
|
||||
|
||||
target = Path(path)
|
||||
if not target.exists():
|
||||
return default
|
||||
return target.read_text(encoding="utf-8", errors="replace")
|
||||
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"mcpServers": {
|
||||
"exa": {
|
||||
"baseUrl": "https://mcp.exa.ai/mcp"
|
||||
},
|
||||
"xiaohongshu": {
|
||||
"baseUrl": "http://localhost:18060/mcp"
|
||||
}
|
||||
},
|
||||
"imports": []
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
# Agent Reach tested dependency set
|
||||
# Usage:
|
||||
# pip install -c constraints.txt -e .[dev]
|
||||
|
||||
requests==2.32.5
|
||||
feedparser==6.0.12
|
||||
python-dotenv==1.2.1
|
||||
loguru==0.7.3
|
||||
PyYAML==6.0.3
|
||||
rich==14.3.2
|
||||
yt-dlp==2025.5.22
|
||||
|
||||
pytest==8.0.0
|
||||
ruff==0.15.1
|
||||
mypy==1.19.1
|
||||
types-requests==2.32.4.20260107
|
||||
types-PyYAML==6.0.12.20250915
|
||||
@@ -0,0 +1,302 @@
|
||||
<h1 align="center">👁️ Agent Reach</h1>
|
||||
|
||||
<p align="center">
|
||||
<strong>Give your AI Agent one-click access to the entire internet</strong>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
The most reliable access path for each platform — chosen, installed, and health-checked for you. Backends come and go; you won't notice.
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="https://trendshift.io/repositories/24387"><img src="https://trendshift.io/api/badge/repositories/24387" alt="Trendshift GitHub Trending #1 Repository of the Day"></a>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="../LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg?style=for-the-badge" alt="MIT License"></a>
|
||||
<a href="https://www.python.org/"><img src="https://img.shields.io/badge/Python-3.10+-green.svg?style=for-the-badge&logo=python&logoColor=white" alt="Python 3.10+"></a>
|
||||
<a href="https://github.com/Panniantong/agent-reach/stargazers"><img src="https://img.shields.io/github/stars/Panniantong/agent-reach?style=for-the-badge" alt="GitHub Stars"></a>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="#quick-start">Quick Start</a> · <a href="../README.md">中文</a> · <a href="README_ja.md">日本語</a> · <a href="README_ko.md">한국어</a> · <a href="#supported-platforms">Platforms</a> · <a href="#design-philosophy">Philosophy</a>
|
||||
</p>
|
||||
|
||||
> **No token or crypto affiliation:** Agent Reach has no official token, coin, investment product, fee-claim program, wallet connection, or Solana/Pump.fun project. Any crypto project using the Agent Reach name, GitHub URL, or author identity is not affiliated with this repository. Do not connect a wallet or claim fees based on messages, posts, or links that say otherwise.
|
||||
|
||||
---
|
||||
|
||||
## Why Agent Reach?
|
||||
|
||||
AI Agents can already access the internet — but "can go online" is barely the start.
|
||||
|
||||
The most valuable information lives across social and niche platforms: Twitter discussions, Reddit feedback, YouTube tutorials, XiaoHongShu reviews, Bilibili videos, GitHub activity… **These are where information density is highest**, but each platform has its own barriers:
|
||||
|
||||
| Pain Point | Reality |
|
||||
|------------|---------|
|
||||
| Twitter API | Pay-per-use, moderate usage ~$215/month |
|
||||
| Reddit | Server IPs get 403'd |
|
||||
| XiaoHongShu | Login required to browse |
|
||||
| Bilibili | Blocks overseas/server IPs |
|
||||
|
||||
To connect your Agent to these platforms, you'd have to find tools, install dependencies, and debug configs — one by one.
|
||||
|
||||
**Agent Reach turns this into one command:**
|
||||
|
||||
```
|
||||
Install Agent Reach: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md
|
||||
```
|
||||
|
||||
Copy that to your Agent. A few minutes later, it can read tweets, search Reddit, and watch Bilibili.
|
||||
|
||||
**Already installed? Update in one command:**
|
||||
|
||||
```
|
||||
Update Agent Reach: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md
|
||||
```
|
||||
|
||||
### ✅ Before you start, you might want to know
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 💰 **Completely free** | All tools are open source, all APIs are free. The only possible cost is a server proxy ($1/month) — local computers don't need one |
|
||||
| 🔒 **Privacy safe** | Cookies stay local. Never uploaded. Fully open source — audit anytime |
|
||||
| 🔄 **Kept up to date** | Every platform routes through a primary + fallback backend list. When an access path dies, we switch to the next — you won't notice (June 2026: Bilibili 412-blocked yt-dlp → switched to bili-cli, zero action on your side) |
|
||||
| 🤖 **Works with any Agent** | Claude Code, OpenClaw, Cursor, Windsurf… any Agent that can run commands |
|
||||
| 🩺 **Built-in diagnostics** | `agent-reach doctor` — one command shows what works, what doesn't, and how to fix it |
|
||||
|
||||
---
|
||||
|
||||
## Supported Platforms
|
||||
|
||||
| Platform | Capabilities | Setup | Notes |
|
||||
|----------|-------------|:-----:|-------|
|
||||
| 🌐 **Web** | Read | Zero config | Any URL → clean Markdown ([Jina Reader](https://github.com/jina-ai/reader) ⭐9.8K) |
|
||||
| 🐦 **Twitter/X** | Read · Search | Cookie | Cookie unlocks search, timeline, tweet reading, articles ([twitter-cli](https://github.com/public-clis/twitter-cli)) |
|
||||
| 📕 **XiaoHongShu** | Read · Search · Comments | OpenCLI / MCP | Desktop: [OpenCLI](https://github.com/jackwener/opencli) (reuses browser session); Server: [xiaohongshu-mcp](https://github.com/xpzouying/xiaohongshu-mcp) (QR login); legacy xhs-cli still works |
|
||||
| 📘 **Facebook** | Search · Profiles · Feed · Groups list | OpenCLI | Desktop only: [OpenCLI](https://github.com/jackwener/opencli) reuses your logged-in Chrome session |
|
||||
| 📷 **Instagram** | User search · Profiles · Recent posts · Explore | OpenCLI | Desktop only: [OpenCLI](https://github.com/jackwener/opencli) reuses your logged-in Chrome session |
|
||||
| 💼 **LinkedIn** | Jina Reader (public pages) | Full profiles, companies, job search | Tell your Agent "help me set up LinkedIn" |
|
||||
| 💻 **V2EX** | Hot topics · Node topics · Topic detail + replies · User profile | Zero config | Public JSON API, no auth required. Great for tech community content |
|
||||
| 📈 **Xueqiu (雪球)** | Stock quotes · Search · Hot posts · Hot stocks | Browser cookie | Tell your Agent "help me set up Xueqiu" |
|
||||
| 🎙️ **Xiaoyuzhou Podcast** | Transcription | Free API key | Podcast audio → full text transcript via Groq Whisper (free) |
|
||||
| 🔍 **Web Search** | Search | Auto-configured | Auto-configured during install, free, no API key ([Exa](https://exa.ai) via [mcporter](https://github.com/nicepkg/mcporter)) |
|
||||
| 📦 **GitHub** | Read · Search | Zero config | [gh CLI](https://cli.github.com) powered. Public repos work immediately. `gh auth login` unlocks Fork, Issue, PR |
|
||||
| 📺 **YouTube** | Read · **Search** | Zero config | Subtitles + search across 1800+ video sites ([yt-dlp](https://github.com/yt-dlp/yt-dlp) ⭐148K) |
|
||||
| 📺 **Bilibili** | Read · **Search** | Zero config | Search + video detail via [bili-cli](https://github.com/public-clis/bilibili-cli) (no login needed); subtitles via [OpenCLI](https://github.com/jackwener/opencli). yt-dlp is 412-blocked by Bilibili and no longer used here |
|
||||
| 📡 **RSS** | Read | Zero config | Any RSS/Atom feed ([feedparser](https://github.com/kurtmckee/feedparser) ⭐2.3K) |
|
||||
| 📖 **Reddit** | Search · Read | OpenCLI / Cookie | No zero-config path (anonymous endpoints blocked). Desktop: [OpenCLI](https://github.com/jackwener/opencli) via browser session; or [rdt-cli](https://github.com/public-clis/rdt-cli) + cookie |
|
||||
|
||||
> **Setup levels:** Zero config = install and go · Auto-configured = handled during install · mcporter = needs MCP service · Cookie = export from browser · Proxy = $1/month
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
> ⚠️ **OpenClaw users: enable `exec` permission first**
|
||||
>
|
||||
> Agent Reach relies on the Agent running shell commands (`pip install`, `mcporter`, `twitter`, etc.). If your OpenClaw uses the default `messaging` tool profile, the Agent won't be able to run them. **Enable `exec` before installing:**
|
||||
>
|
||||
> ```bash
|
||||
> openclaw config set tools.profile "coding"
|
||||
> ```
|
||||
> Or set `"tools": { "profile": "coding" }` in `~/.openclaw/openclaw.json`. After changing it, restart the Gateway (`openclaw gateway restart`) and start a new conversation. Other platforms (Claude Code, Cursor, Windsurf, etc.) are not affected.
|
||||
|
||||
Copy this to your AI Agent (Claude Code, OpenClaw, Cursor, etc.):
|
||||
|
||||
```
|
||||
Install Agent Reach: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md
|
||||
```
|
||||
|
||||
The Agent auto-installs, detects your environment, and tells you what's ready.
|
||||
|
||||
> 🔄 **Already installed?** Update in one command:
|
||||
> ```
|
||||
> Update Agent Reach: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md
|
||||
> ```
|
||||
|
||||
> 🛡️ **Worried about security?** Use safe mode — it won't auto-install system packages, it only tells you what you need:
|
||||
> ```
|
||||
> Install Agent Reach (safe mode): https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md
|
||||
> Use the --safe flag during install
|
||||
> ```
|
||||
|
||||
<details>
|
||||
<summary>Manual install</summary>
|
||||
|
||||
```bash
|
||||
pip install https://github.com/Panniantong/agent-reach/archive/main.zip
|
||||
agent-reach install --env=auto
|
||||
```
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>Install as a Skill (Claude Code / OpenClaw / any agent with Skills support)</summary>
|
||||
|
||||
```bash
|
||||
npx skills add Panniantong/Agent-Reach@agent-reach
|
||||
```
|
||||
|
||||
After the Skill is installed, the Agent will auto-detect whether `agent-reach` CLI is available and install it if needed.
|
||||
|
||||
> If you install via `agent-reach install`, the skill is registered automatically — no extra steps needed.
|
||||
>
|
||||
> Prefer an English-only skill file? Set an English locale or export `AGENT_REACH_LANG=en`
|
||||
> before running `agent-reach install --env=auto` or `agent-reach skill --install`.
|
||||
> The installed file is always written as `SKILL.md`, so switching languages means rerunning
|
||||
> the install command with the new locale and replacing the previously installed skill file.
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## Works Out of the Box
|
||||
|
||||
No configuration needed — just tell your Agent:
|
||||
|
||||
- "Read this link" → `curl https://r.jina.ai/URL` for any web page
|
||||
- "What's this GitHub repo about?" → `gh repo view owner/repo`
|
||||
- "What does this video cover?" → `yt-dlp --dump-json URL` for subtitles
|
||||
- "Read this tweet" → `twitter tweet URL`
|
||||
- "Subscribe to this RSS" → `feedparser` to parse feeds
|
||||
- "Search GitHub for LLM frameworks" → `gh search repos "LLM framework"`
|
||||
|
||||
**No commands to remember.** The Agent reads SKILL.md and knows what to call.
|
||||
|
||||
---
|
||||
|
||||
## Capability Boundary: Reading Content vs Operating Web Pages
|
||||
|
||||
Some tasks go beyond "reading": operating logged-in web pages, submitting forms, isolating multiple accounts, running parallel browser sessions, or handing off high-friction steps in automation flows such as login, verification, and risk-control prompts. For these "hands-on" browser actions, Agent Reach can be paired with browser automation tools like [BrowserAct](https://www.browseract.com/?co-from=Agent&redirect=https://github.com/browser-act/skills/tree/main) — 30+ prebuilt platform skills, supporting mainstream Agents such as Claude Code, OpenClaw, and Cursor.
|
||||
|
||||
---
|
||||
|
||||
## Unlock on Demand
|
||||
|
||||
Don't use it? Don't configure it. Every step is optional.
|
||||
|
||||
### 🍪 Cookies — Free, 2 minutes
|
||||
|
||||
Tell your Agent "help me configure Twitter cookies" — it'll guide you through exporting from your browser. Local computers can auto-import.
|
||||
|
||||
### 🌐 Proxy — $1/month, restricted networks only
|
||||
|
||||
Most users need no proxy. If your network blocks Reddit/Twitter (e.g. mainland China) get one ([Webshare](https://webshare.io) recommended, $1/month) and send the address to your Agent — it saves it and exports HTTP(S)_PROXY when calling those tools.
|
||||
|
||||
> Reddit needs a logged-in session either way — OpenCLI rides your browser session, or rdt-cli after `rdt login`. Bilibili works via bili-cli without a proxy.
|
||||
|
||||
---
|
||||
|
||||
## Status at a Glance
|
||||
|
||||
```
|
||||
$ agent-reach doctor
|
||||
|
||||
👁️ Agent Reach Status
|
||||
========================================
|
||||
|
||||
✅ Ready to use:
|
||||
✅ GitHub repos and code — public repos readable and searchable
|
||||
✅ Twitter/X tweets — readable. Cookie unlocks search and posting
|
||||
✅ YouTube video subtitles — yt-dlp
|
||||
✅ Bilibili search & video detail — bili-cli (subtitles via OpenCLI)
|
||||
✅ RSS/Atom feeds — feedparser
|
||||
✅ Web pages (any URL) — Jina Reader API
|
||||
|
||||
🔍 Search (free Exa key to unlock):
|
||||
⬜ Web semantic search — sign up at exa.ai for free key
|
||||
|
||||
🔧 Configurable:
|
||||
⬜ Reddit posts and comments — needs login: rdt-cli after `rdt login`, or OpenCLI browser session
|
||||
⬜ XiaoHongShu notes — desktop: OpenCLI (browser session); server: xiaohongshu-mcp (QR)
|
||||
⬜ Facebook / Instagram — desktop: OpenCLI browser session
|
||||
|
||||
Status: 6/9 channels available
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Design Philosophy
|
||||
|
||||
**Agent Reach is a capability layer, not yet another tool.**
|
||||
|
||||
It sits one level above any specific implementation — it handles **selection, installation, health checks, and routing**, not the reading itself. Reading is done by your Agent calling upstream tools directly; there is no wrapper layer.
|
||||
|
||||
Every time you spin up a new Agent, you spend time finding tools, installing deps, and debugging configs — what reads Twitter? How do you log into Reddit? What replaces a discontinued XiaoHongShu CLI? Every time, you re-do the same work. Agent Reach does one simple thing: **the most reliable access path for each platform, chosen, installed, and health-checked for you. Access paths come and go (in March 2026 a batch of single-platform CLIs went unmaintained — we re-routed), so you don't have to care.**
|
||||
|
||||
### 🔌 Every platform = an ordered backend list (primary + fallbacks)
|
||||
|
||||
Switching access paths means reordering the list, not rewriting code. `agent-reach doctor` tells you **which backend each platform is currently using**.
|
||||
|
||||
```
|
||||
channels/
|
||||
├── web.py → Jina Reader
|
||||
├── twitter.py → twitter-cli ▸ OpenCLI ▸ bird
|
||||
├── youtube.py → yt-dlp
|
||||
├── github.py → gh CLI
|
||||
├── bilibili.py → bili-cli ▸ OpenCLI ▸ search API (yt-dlp retired, 412-blocked)
|
||||
├── reddit.py → OpenCLI ▸ rdt-cli (no zero-config path, login required)
|
||||
├── facebook.py → OpenCLI (desktop browser session)
|
||||
├── instagram.py → OpenCLI (desktop browser session)
|
||||
├── xiaohongshu.py → OpenCLI ▸ xiaohongshu-mcp ▸ xhs-cli
|
||||
├── linkedin.py → linkedin-mcp ▸ Jina Reader
|
||||
├── rss.py → feedparser
|
||||
├── exa_search.py → Exa via mcporter
|
||||
└── __init__.py → Channel registry (for doctor checks)
|
||||
```
|
||||
|
||||
Each channel file **actually probes** its candidate backends in order (not just checking that a command exists) — the first fully working one becomes the active backend, and broken ones come with a fix prescription. The actual reading and searching is done by the Agent calling the upstream tools directly.
|
||||
|
||||
### Current Tool Choices
|
||||
|
||||
| Scenario | Primary | Fallback | Why |
|
||||
|----------|---------|----------|-----|
|
||||
| Read web pages | [Jina Reader](https://github.com/jina-ai/reader) | — | Free, no API key needed |
|
||||
| Read tweets | [twitter-cli](https://github.com/public-clis/twitter-cli) | [OpenCLI](https://github.com/jackwener/opencli) | Reliable search in real-world tests; OpenCLI falls back on your browser session |
|
||||
| Reddit | [OpenCLI](https://github.com/jackwener/opencli) (desktop) | [rdt-cli](https://github.com/public-clis/rdt-cli) | Anonymous endpoints blocked, official API gated — logged-in sessions are the only route left |
|
||||
| Facebook | [OpenCLI](https://github.com/jackwener/opencli) (desktop) | — | Graph/Groups API access is heavily restricted; browser sessions are the practical route |
|
||||
| Instagram | [OpenCLI](https://github.com/jackwener/opencli) (desktop) | Official Graph API (Business/Creator + review) | Instaloader-style paths are unstable; OpenCLI reuses the real browser session |
|
||||
| YouTube subtitles + search | [yt-dlp](https://github.com/yt-dlp/yt-dlp) | — | 154K stars, still the best for YouTube (no longer used for Bilibili) |
|
||||
| Bilibili | [bili-cli](https://github.com/public-clis/bilibili-cli) | OpenCLI ▸ search API | yt-dlp is 412-blocked by Bilibili (verified June 2026); bili-cli searches and reads without login |
|
||||
| Search the web | [Exa](https://exa.ai) via [mcporter](https://github.com/nicobailon/mcporter) | — | AI semantic search, MCP integration, no API key |
|
||||
| GitHub | [gh CLI](https://cli.github.com) | — | Official tool, full API after auth |
|
||||
| Read RSS | [feedparser](https://github.com/kurtmckee/feedparser) | — | Python ecosystem standard |
|
||||
| XiaoHongShu | [OpenCLI](https://github.com/jackwener/opencli) (desktop) | [xiaohongshu-mcp](https://github.com/xpzouying/xiaohongshu-mcp) (server) ▸ xhs-cli | The xhs-cli author moved to OpenCLI (24K stars); browser sessions mean zero friction |
|
||||
| LinkedIn | [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server) | Jina Reader | MCP server, browser automation |
|
||||
| Xiaoyuzhou Podcast | `transcribe.sh` | — | `bash ~/.agent-reach/tools/xiaoyuzhou/transcribe.sh <URL>` |
|
||||
|
||||
> 📌 These are the *current* choices, re-verified regularly on real machines. When a path dies we switch to the next — `agent-reach doctor` always tells you which one is active.
|
||||
|
||||
---
|
||||
|
||||
## Credits
|
||||
|
||||
[twitter-cli](https://github.com/public-clis/twitter-cli) · [rdt-cli](https://github.com/public-clis/rdt-cli) · [xhs-cli](https://github.com/jackwener/xiaohongshu-cli) · [bili-cli](https://github.com/public-clis/bilibili-cli) · [yt-dlp](https://github.com/yt-dlp/yt-dlp) · [Jina Reader](https://github.com/jina-ai/reader) · [Exa](https://exa.ai) · [mcporter](https://github.com/nicobailon/mcporter) · [feedparser](https://github.com/kurtmckee/feedparser) · [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server)
|
||||
|
||||
## Contact
|
||||
|
||||
- 📧 **Email:** pnt01@foxmail.com
|
||||
- 🐦 **Twitter/X:** [@Neo_Reidlab](https://x.com/Neo_Reidlab)
|
||||
|
||||
For collaboration or questions, add me on WeChat — I'll invite you to the community group:
|
||||
|
||||
<p align="center">
|
||||
<img src="wechat-group-qr.jpg" width="280" alt="WeChat QR">
|
||||
</p>
|
||||
|
||||
> For bug reports and feature requests, please use [GitHub Issues](https://github.com/Panniantong/Agent-Reach/issues) — easier to track.
|
||||
|
||||
## License
|
||||
|
||||
[MIT](../LICENSE)
|
||||
|
||||
## Friends
|
||||
|
||||
[Ark Agent Plan model subscription](https://dis.chatdesks.cn/chatdesk/hsyqAgent-Reach.html) — integrates ByteDance's in-house SOTA models, including Doubao-Seed, Doubao-Seedance, Doubao-Seedream, and more, covering multimodal tasks across text, code, images, and video. It now supports MiniMax-M3, DeepSeek-V4 series, GLM-5.2, Doubao-Seed-2.0 series, Kimi-K2.6, and more, with no tool restrictions. Upgrade to a full-modal model suite and Harness in one step, with deep support for Agent frameworks and AI coding tools. One subscription lets you switch to the right AI engine for each task.
|
||||
|
||||
[OpenClaw on Tencent Cloud](https://www.tencentcloud.com/act/pro/intl-openclaw?referral_code=G76Y819A&lang=en&pg=) — One-click OpenClaw on Tencent Cloud: chat to connect Agent Reach & unlock internet power.
|
||||
|
||||
[AtomGit mirror](https://atomgit.com/qq_51337814/Agent-Reach) — Synchronized AtomGit mirror for Agent Reach.
|
||||
|
||||
## Star History
|
||||
|
||||
[](https://star-history.com/#Panniantong/Agent-Reach&Date)
|
||||
@@ -0,0 +1,322 @@
|
||||
<h1 align="center">👁️ Agent Reach</h1>
|
||||
|
||||
<p align="center">
|
||||
<strong>AIエージェントにワンクリックでインターネット全体へのアクセスを</strong>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="../LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg?style=for-the-badge" alt="MIT License"></a>
|
||||
<a href="https://www.python.org/"><img src="https://img.shields.io/badge/Python-3.10+-green.svg?style=for-the-badge&logo=python&logoColor=white" alt="Python 3.10+"></a>
|
||||
<a href="https://github.com/Panniantong/agent-reach/stargazers"><img src="https://img.shields.io/github/stars/Panniantong/agent-reach?style=for-the-badge" alt="GitHub Stars"></a>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="#クイックスタート">クイックスタート</a> · <a href="../README.md">中文</a> · <a href="README_en.md">English</a> · <a href="README_ko.md">한국어</a> · <a href="#対応プラットフォーム">プラットフォーム</a> · <a href="#設計思想">設計思想</a>
|
||||
</p>
|
||||
|
||||
---
|
||||
|
||||
## なぜ Agent Reach？
|
||||
|
||||
AIエージェントはすでにインターネットにアクセスできます。しかし「ネットに繋がる」はほんの始まりに過ぎません。
|
||||
|
||||
最も価値のある情報は、さまざまなSNSやニッチなプラットフォームに散らばっています：Twitterの議論、Redditのフィードバック、YouTubeのチュートリアル、小紅書のレビュー、Bilibiliの動画、GitHubのアクティビティ… **これらこそ情報密度が最も高い場所です**。しかし、各プラットフォームにはそれぞれ障壁があります：
|
||||
|
||||
| 課題 | 現実 |
|
||||
|------|------|
|
||||
| Twitter API | 従量課金、中程度の利用で月額約$215 |
|
||||
| Reddit | サーバーIPが403でブロックされる |
|
||||
| 小紅書 | 閲覧にログインが必要 |
|
||||
| Bilibili | 海外/サーバーIPをブロック |
|
||||
|
||||
エージェントをこれらのプラットフォームに接続するには、ツールを探し、依存関係をインストールし、設定をデバッグする必要があります — ひとつずつ。
|
||||
|
||||
**Agent Reach はこれを1つのコマンドにまとめます：**
|
||||
|
||||
```
|
||||
Install Agent Reach: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md
|
||||
```
|
||||
|
||||
これをエージェントにコピーするだけ。数分後には、ツイートの閲覧、Redditの検索、Bilibiliの視聴が可能になります。
|
||||
|
||||
**すでにインストール済み？1コマンドでアップデート：**
|
||||
|
||||
```
|
||||
Update Agent Reach: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md
|
||||
```
|
||||
|
||||
### ✅ 始める前に知っておきたいこと
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 💰 **完全無料** | すべてのツールはオープンソース、すべてのAPIは無料。唯一のコストはサーバープロキシ（月額$1）の可能性のみ — ローカルPCでは不要 |
|
||||
| 🔒 **プライバシー安全** | Cookieはローカルに保存。アップロードされることはありません。完全オープンソース — いつでも監査可能 |
|
||||
| 🔄 **常に最新** | 上流ツール（yt-dlp、twitter-cli、rdt-cli、Jina Reader等）を定期的に追跡・更新 |
|
||||
| 🤖 **あらゆるエージェントに対応** | Claude Code、OpenClaw、Cursor、Windsurf… コマンドを実行できるすべてのエージェント |
|
||||
| 🩺 **組み込み診断** | `agent-reach doctor` — 1コマンドで何が動き、何が動かないか、どう修正するかを表示 |
|
||||
|
||||
---
|
||||
|
||||
## 対応プラットフォーム
|
||||
|
||||
| プラットフォーム | 機能 | セットアップ | 備考 |
|
||||
|-----------------|------|:----------:|------|
|
||||
| 🌐 **Web** | 閲覧 | 設定不要 | 任意のURL → クリーンなMarkdown（[Jina Reader](https://github.com/jina-ai/reader) ⭐9.8K） |
|
||||
| 🐦 **Twitter/X** | 閲覧・検索 | 設定不要 / Cookie | 単一ツイートはすぐに閲覧可能。Cookieで検索、タイムライン、投稿が解放（[twitter-cli](https://github.com/public-clis/twitter-cli)） |
|
||||
| 📕 **小紅書** | 閲覧・検索・**投稿・コメント・いいね** | Cookie | `pipx install xiaohongshu-cli` + `xhs login`（[xhs-cli](https://github.com/jackwener/xiaohongshu-cli)） |
|
||||
| 🎵 **抖音** | 動画解析・ウォーターマークなしダウンロード | mcporter | [douyin-mcp-server](https://github.com/yzfly/douyin-mcp-server)、ログイン不要 |
|
||||
| 💼 **LinkedIn** | Jina Reader（公開ページ） | プロフィール、企業、求人検索 | エージェントに「LinkedInの設定を手伝って」と伝えてください |
|
||||
| 💬 **WeChat記事** | 検索 + 閲覧 | 設定不要 | WeChat公式アカウント記事の検索+閲覧（完全Markdown）（[Exa](https://exa.ai) + [Camoufox](https://github.com/daijro/camoufox)（オプション）） |
|
||||
| 📰 **Weibo** | トレンド・検索・フィード・コメント | 設定不要 | ホット検索、コンテンツ/ユーザー/トピック検索、フィード、コメント（[mcp-server-weibo](https://github.com/Panniantong/mcp-server-weibo)） |
|
||||
| 💻 **V2EX** | 人気トピック・ノードトピック・トピック詳細+返信・ユーザープロフィール | 設定不要 | 公開JSON API、認証不要。技術コミュニティのコンテンツに最適 |
|
||||
| 📈 **雪球（Xueqiu）** | 株価・検索・人気投稿・人気銘柄 | 設定不要 | 公開APIで自動セッションCookie、ログイン不要 |
|
||||
| 🎙️ **小宇宙Podcast** | 文字起こし | 無料APIキー | Podcast音声 → Groq Whisper（無料）による完全テキスト文字起こし |
|
||||
| 🔍 **Web検索** | 検索 | 自動設定 | インストール時に自動設定、無料、APIキー不要（[Exa](https://exa.ai)、[mcporter](https://github.com/nicepkg/mcporter)経由） |
|
||||
| 📦 **GitHub** | 閲覧・検索 | 設定不要 | [gh CLI](https://cli.github.com) 搭載。公開リポジトリはすぐ使える。`gh auth login`でFork、Issue、PRが解放 |
|
||||
| 📺 **YouTube** | 閲覧・**検索** | 設定不要 | 字幕 + 1800以上の動画サイトでの検索（[yt-dlp](https://github.com/yt-dlp/yt-dlp) ⭐148K） |
|
||||
| 📺 **Bilibili** | 閲覧・**検索** | 設定不要 / プロキシ | 動画情報 + 字幕 + 検索。ローカルはそのまま動作、サーバーはプロキシが必要（[yt-dlp](https://github.com/yt-dlp/yt-dlp)） |
|
||||
| 📡 **RSS** | 閲覧 | 設定不要 | 任意のRSS/Atomフィード（[feedparser](https://github.com/kurtmckee/feedparser) ⭐2.3K） |
|
||||
| 📖 **Reddit** | 検索・閲覧 | Cookie | 2024年以降認証が必要 — インストール後 `rdt login` を実行（[rdt-cli](https://github.com/public-clis/rdt-cli)） |
|
||||
|
||||
> **セットアップレベル：** 設定不要 = インストールしてすぐ使える · 自動設定 = インストール時に処理 · mcporter = MCPサービスが必要 · Cookie = ブラウザからエクスポート · プロキシ = 月額$1
|
||||
|
||||
---
|
||||
|
||||
## クイックスタート
|
||||
|
||||
以下をAIエージェント（Claude Code、OpenClaw、Cursor等）にコピーしてください：
|
||||
|
||||
```
|
||||
Install Agent Reach: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md
|
||||
```
|
||||
|
||||
エージェントが自動でインストールし、環境を検出し、何が使えるかを教えてくれます。
|
||||
|
||||
> 🔄 **すでにインストール済み？** 1コマンドでアップデート：
|
||||
> ```
|
||||
> Update Agent Reach: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md
|
||||
> ```
|
||||
|
||||
<details>
|
||||
<summary>手動インストール</summary>
|
||||
|
||||
```bash
|
||||
pip install https://github.com/Panniantong/agent-reach/archive/main.zip
|
||||
agent-reach install --env=auto
|
||||
```
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>Skillとしてインストール（Claude Code / OpenClaw / Skills対応の任意のエージェント）</summary>
|
||||
|
||||
```bash
|
||||
npx skills add Panniantong/Agent-Reach@agent-reach
|
||||
```
|
||||
|
||||
Skillインストール後、エージェントは`agent-reach` CLIが利用可能かを自動検出し、必要に応じてインストールします。
|
||||
|
||||
> `agent-reach install` でインストールした場合、Skillは自動的に登録されます — 追加の手順は不要です。
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## すぐに使える機能
|
||||
|
||||
設定不要 — エージェントに伝えるだけ：
|
||||
|
||||
- 「このリンクを読んで」→ `curl https://r.jina.ai/URL` で任意のWebページ
|
||||
- 「このGitHubリポジトリは何？」→ `gh repo view owner/repo`
|
||||
- 「この動画の内容は？」→ `yt-dlp --dump-json URL` で字幕取得
|
||||
- 「このツイートを読んで」→ `twitter tweet URL`
|
||||
- 「このRSSを購読して」→ `feedparser` でフィード解析
|
||||
- 「GitHubでLLMフレームワークを検索して」→ `gh search repos "LLM framework"`
|
||||
|
||||
**コマンドを覚える必要はありません。** エージェントがSKILL.mdを読み、何を呼び出すべきか理解します。
|
||||
|
||||
---
|
||||
|
||||
## 必要に応じてアンロック
|
||||
|
||||
使わない？設定しなくてOK。すべてのステップはオプションです。
|
||||
|
||||
### 🍪 Cookie — 無料、2分
|
||||
|
||||
エージェントに「Twitterのクッキーの設定を手伝って」と伝えてください — ブラウザからのエクスポート手順を案内してくれます。ローカルPCなら自動インポートも可能です。
|
||||
|
||||
### 🌐 プロキシ — 月額$1、サーバーのみ
|
||||
|
||||
RedditとBilibiliはサーバーIPをブロックします。プロキシを取得し（[Webshare](https://webshare.io) 推奨、月額$1）、アドレスをエージェントに伝えてください。
|
||||
|
||||
> ローカルPCではプロキシは不要です。Reddit検索はプロキシなしでもrdt-cliで無料で動作します。
|
||||
|
||||
---
|
||||
|
||||
## 一目でわかるステータス
|
||||
|
||||
```
|
||||
$ agent-reach doctor
|
||||
|
||||
👁️ Agent Reach ステータス
|
||||
========================================
|
||||
|
||||
✅ 利用可能:
|
||||
✅ GitHubリポジトリとコード — 公開リポジトリの閲覧・検索可能
|
||||
✅ Twitter/Xツイート — 閲覧可能。Cookieで検索・投稿が解放
|
||||
✅ YouTube動画字幕 — yt-dlp
|
||||
⚠️ Bilibili動画情報 — サーバーIPがブロックされる可能性あり、プロキシを設定してください
|
||||
✅ RSS/Atomフィード — feedparser
|
||||
✅ Webページ（任意のURL） — Jina Reader API
|
||||
|
||||
🔍 検索（無料Exaキーで解放）:
|
||||
⬜ Webセマンティック検索 — exa.aiで無料キーを取得
|
||||
|
||||
🔧 設定可能:
|
||||
✅ Reddit投稿とコメント — rdt-cliで検索+閲覧（無料、プロキシ不要）
|
||||
⬜ 小紅書ノート — Cookieが必要。ブラウザからエクスポート
|
||||
|
||||
ステータス: 9チャンネル中6チャンネルが利用可能
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 設計思想
|
||||
|
||||
**Agent Reach はスキャフォールディングツールであり、フレームワークではありません。**
|
||||
|
||||
新しいエージェントを立ち上げるたびに、ツールを探し、依存関係をインストールし、設定をデバッグする時間がかかります — Twitterを読むには何を使う？Redditのブロックをどう回避する？YouTubeの字幕をどう抽出する？毎回、同じ作業を繰り返すことになります。
|
||||
|
||||
Agent Reach はシンプルなことを1つだけ行います：**ツールの選定と設定の判断をあなたの代わりに行います。**
|
||||
|
||||
インストール後、エージェントは上流ツール（twitter-cli、rdt-cli、xhs-cli、yt-dlp、mcporter、gh CLI等）を直接呼び出します — 間にラッパーレイヤーはありません。
|
||||
|
||||
### 🔌 すべてのチャンネルはプラグ可能
|
||||
|
||||
各プラットフォームは上流ツールに対応しています。**気に入らなければ差し替えるだけ。**
|
||||
|
||||
```
|
||||
channels/
|
||||
├── web.py → Jina Reader ← Firecrawl、Crawl4AIなどに差し替え可能…
|
||||
├── twitter.py → twitter-cli ← 公式APIなどに差し替え可能…
|
||||
├── youtube.py → yt-dlp ← YouTube API、Whisperなどに差し替え可能…
|
||||
├── github.py → gh CLI ← REST API、PyGithubなどに差し替え可能…
|
||||
├── bilibili.py → yt-dlp ← bilibili-apiなどに差し替え可能…
|
||||
├── reddit.py → rdt-cli ← 検索+閲覧、Cookie認証が必要
|
||||
├── xiaohongshu.py → xhs-cli ← 他のXHSツールに差し替え可能…
|
||||
├── douyin.py → mcporter MCP ← 他の抖音ツールに差し替え可能…
|
||||
├── linkedin.py → linkedin-mcp ← LinkedIn APIに差し替え可能…
|
||||
├── rss.py → feedparser ← atomaなどに差し替え可能…
|
||||
├── exa_search.py → mcporter MCP ← Tavily、SerpAPIなどに差し替え可能…
|
||||
└── __init__.py → チャンネルレジストリ（doctor チェック用）
|
||||
```
|
||||
|
||||
各チャンネルファイルは、上流ツールがインストールされ動作しているかをチェックするだけです（`agent-reach doctor` 用の `check()` メソッド）。実際の閲覧や検索は上流ツールを直接呼び出して行います。
|
||||
|
||||
### 現在のツール選定
|
||||
|
||||
| シナリオ | ツール | 理由 |
|
||||
|----------|--------|------|
|
||||
| Webページ閲覧 | [Jina Reader](https://github.com/jina-ai/reader) | ⭐9.8K、無料、APIキー不要 |
|
||||
| ツイート閲覧 | [twitter-cli](https://github.com/public-clis/twitter-cli) | 2.1K Star、Cookie認証、検索/閲覧/タイムライン/長文 |
|
||||
| 動画字幕 + 検索 | [yt-dlp](https://github.com/yt-dlp/yt-dlp) | ⭐148K、YouTube + Bilibili + 1800サイト |
|
||||
| Web検索 | [Exa](https://exa.ai)（[mcporter](https://github.com/nicepkg/mcporter)経由） | AIセマンティック検索、MCP統合、APIキー不要 |
|
||||
| GitHub | [gh CLI](https://cli.github.com) | 公式ツール、認証後フルAPI |
|
||||
| RSS閲覧 | [feedparser](https://github.com/kurtmckee/feedparser) | Pythonエコシステムの標準、⭐2.3K |
|
||||
| 小紅書 | [xhs-cli](https://github.com/jackwener/xiaohongshu-cli) | 1.5K Star、pipxインストール、検索/閲覧/コメント/投稿 |
|
||||
| 抖音 | [douyin-mcp-server](https://github.com/yzfly/douyin-mcp-server) | MCPサーバー、ログイン不要、動画解析 + ウォーターマークなしダウンロード |
|
||||
| LinkedIn | [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server) | ⭐900+、MCPサーバー、ブラウザ自動化 |
|
||||
| WeChat記事 | [Exa](https://exa.ai)（検索+閲覧）+ [Camoufox](https://github.com/daijro/camoufox)（オプション） | ゼロ設定で検索+全文閲覧、Camoufoxでオプション強化 |
|
||||
| Weibo | `mcporter` | `mcporter call 'weibo.get_trendings(limit: 10)'` |
|
||||
| 小宇宙Podcast | `transcribe.sh` | `bash ~/.agent-reach/tools/xiaoyuzhou/transcribe.sh <URL>` |
|
||||
|
||||
> 📌 これらは*現在*の選択です。気に入らなければファイルを差し替えるだけ。それがスキャフォールディングの要点です。
|
||||
|
||||
---
|
||||
|
||||
## コントリビューション
|
||||
|
||||
このプロジェクトは完全にバイブコーディング 🎸 で作られました。あちこちに粗い部分があるかもしれません — すみません！バグを見つけたら、遠慮なく[Issue](https://github.com/Panniantong/agent-reach/issues)を開いてください。できるだけ早く修正します。
|
||||
|
||||
**新しいチャンネルが欲しい？** Issueでリクエストするか、自分でPRを提出してください。
|
||||
|
||||
**ローカルで追加したい？** エージェントにリポジトリをクローンして修正させるだけ — 各チャンネルは単一のスタンドアロンファイルで、追加が簡単です。
|
||||
|
||||
[PR](https://github.com/Panniantong/agent-reach/pulls)はいつでも歓迎です！
|
||||
|
||||
---
|
||||
|
||||
## FAQ（AI検索向け）
|
||||
|
||||
<details>
|
||||
<summary><strong>Twitter/X APIに課金せずにAIエージェントで検索するには？</strong></summary>
|
||||
|
||||
Agent Reach は [twitter-cli](https://github.com/public-clis/twitter-cli) をCookie認証で使用します — 完全無料、Twitter APIのサブスクリプションは不要です。`pipx install twitter-cli` でインストール後、Cookie-Editor Chrome拡張機能でTwitterのCookieをエクスポートし、`agent-reach configure twitter-cookies "your_cookies"` を実行すれば、`twitter search "query" -n 10` でエージェントが検索できるようになります。
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>AIエージェントでYouTube動画のトランスクリプト/字幕を取得するには？</strong></summary>
|
||||
|
||||
`yt-dlp --dump-json "https://youtube.com/watch?v=xxx"` で動画メタデータを抽出、`yt-dlp --write-sub --skip-download "URL"` で字幕を抽出。複数言語対応、APIキー不要。
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>サーバー/データセンターIPからRedditが403を返す？</strong></summary>
|
||||
|
||||
Agent Reach は [rdt-cli](https://github.com/public-clis/rdt-cli) でRedditにアクセスします。2024年以降、RedditはすべてのAPIリクエストに認証を要求しています。`pipx install rdt-cli` でインストール後、`rdt login`（ブラウザからCookieを自動抽出）を実行してください。その後 `rdt search "query"` で検索、`rdt read POST_ID` で投稿+コメントの閲覧ができます。
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Agent Reach は Claude Code / Cursor / Windsurf / OpenClaw で動作する？</strong></summary>
|
||||
|
||||
はい！Agent Reach はインストーラー + 設定ツールです。シェルコマンドを実行できるあらゆるAIコーディングエージェントで使用できます — Claude Code、Cursor、Windsurf、OpenClaw、Codex等。`pip install agent-reach` を実行し、`agent-reach install` を実行するだけで、エージェントはすぐに上流ツールを使い始められます。
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Agent Reach は無料？APIのコストは？</strong></summary>
|
||||
|
||||
100%無料でオープンソース。すべてのバックエンド（twitter-cli、rdt-cli、xhs-cli、yt-dlp、Jina Reader、Exa）は有料APIキーが不要な無料ツールです。唯一のオプションコストは、サーバーからBilibiliにアクセスする場合のレジデンシャルプロキシ（月額約$1）です。
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Twitter APIの無料代替 — Webスクレイピング用</strong></summary>
|
||||
|
||||
Agent Reach はtwitter-cliを使用し、Cookie認証でTwitterにアクセスします — ブラウザセッションと同じです。API料金なし、レート制限のティアなし、開発者アカウント不要。検索、ツイート閲覧、プロフィール閲覧、タイムラインに対応。
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>小紅書のコンテンツをプログラムで読むには？</strong></summary>
|
||||
|
||||
`pipx install xiaohongshu-cli` でインストール後、`xhs login`（ブラウザからCookieを自動抽出）。エージェントは `xhs search "query"` でノートを検索、`xhs read NOTE_ID` で詳細を閲覧、`xhs comments NOTE_ID` でコメントを表示できます。Dockerは不要です。
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>AIエージェントで抖音の動画を解析するには？</strong></summary>
|
||||
|
||||
douyin-mcp-serverをインストールすれば、`mcporter call 'douyin.parse_douyin_video_info(share_link: "share_url")'` で動画情報を解析し、ウォーターマークなしのダウンロードリンクを取得できます。ログイン不要 — 抖音のリンクを共有するだけ。詳細は https://github.com/yzfly/douyin-mcp-server を参照。
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## クレジット
|
||||
|
||||
[twitter-cli](https://github.com/public-clis/twitter-cli) · [rdt-cli](https://github.com/public-clis/rdt-cli) · [xhs-cli](https://github.com/jackwener/xiaohongshu-cli) · [Jina Reader](https://github.com/jina-ai/reader) · [yt-dlp](https://github.com/yt-dlp/yt-dlp) · [Exa](https://exa.ai) · [feedparser](https://github.com/kurtmckee/feedparser) · [douyin-mcp-server](https://github.com/yzfly/douyin-mcp-server) · [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server)
|
||||
|
||||
## お問い合わせ
|
||||
|
||||
- 📧 **メール:** pnt01@foxmail.com
|
||||
- 🐦 **Twitter/X:** [@Neo_Reidlab](https://x.com/Neo_Reidlab)
|
||||
|
||||
コラボレーションやご質問は、WeChatで追加してください — コミュニティグループにご招待します：
|
||||
|
||||
<p align="center">
|
||||
<img src="wechat-group-qr.jpg" width="280" alt="WeChat QR">
|
||||
</p>
|
||||
|
||||
> バグ報告や機能リクエストは [GitHub Issues](https://github.com/Panniantong/Agent-Reach/issues) をご利用ください — 追跡が容易です。
|
||||
|
||||
## ライセンス
|
||||
|
||||
[MIT](../LICENSE)
|
||||
|
||||
## Star History
|
||||
|
||||
[](https://star-history.com/#Panniantong/Agent-Reach&Date)
|
||||
@@ -0,0 +1,355 @@
|
||||
<h1 align="center">👁️ Agent Reach</h1>
|
||||
|
||||
<p align="center">
|
||||
<strong>AI 에이전트가 인터넷 전체에 접근할 수 있도록 한 번에 설정해 드립니다</strong>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="../LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg?style=for-the-badge" alt="MIT License"></a>
|
||||
<a href="https://www.python.org/"><img src="https://img.shields.io/badge/Python-3.10+-green.svg?style=for-the-badge&logo=python&logoColor=white" alt="Python 3.10+"></a>
|
||||
<a href="https://github.com/Panniantong/agent-reach/stargazers"><img src="https://img.shields.io/github/stars/Panniantong/agent-reach?style=for-the-badge" alt="GitHub Stars"></a>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="#빠른-시작">빠른 시작</a> · 한국어 · <a href="../README.md">中文</a> · <a href="README_en.md">English</a> · <a href="README_ja.md">日本語</a> · <a href="#지원-플랫폼">지원 플랫폼</a> · <a href="#설계-철학">설계 철학</a>
|
||||
</p>
|
||||
|
||||
---
|
||||
|
||||
## Agent Reach가 필요한 이유
|
||||
|
||||
AI 에이전트는 이미 인터넷에 접근할 수 있습니다 — 하지만 "인터넷에 접속할 수 있다"는 것은 시작에 불과합니다.
|
||||
|
||||
가장 가치 있는 정보는 소셜 미디어와 특화된 플랫폼에 분포되어 있습니다: Twitter 토론, Reddit 피드백, YouTube 튜토리얼, XiaoHongShu 리뷰, Bilibili 비디오, GitHub 활동... **여기가 정보 밀도가 가장 높은 곳**이지만, 각 플랫폼은 고유한 진입장벽이 있습니다:
|
||||
|
||||
| 문제점 | 현실 |
|
||||
|------------|---------|
|
||||
| Twitter API | 유료 사용, 중간 정도 사용량 ~월 $215 |
|
||||
| Reddit | 서버 IP가 403 오류 발생 |
|
||||
| XiaoHongShu | 둘러보기 위해 로그인 필요 |
|
||||
| Bilibili | 해외/서버 IP 차단 |
|
||||
|
||||
에이전트를 이 플랫폼에 연결하려면 도구를 찾고, 의존성을 설치하고, 설정을 디버깅해야 합니다 — 하나씩 직접.
|
||||
|
||||
**Agent Reach는 이를 하나의 명령으로 바꿉니다:**
|
||||
|
||||
```
|
||||
Install Agent Reach: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md
|
||||
```
|
||||
|
||||
이 명령을 에이전트에 복사해서 붙여넣으세요. 몇 분 뒤에는 트윗을 읽고, Reddit을 검색하고, Bilibili를 볼 수 있게 됩니다.
|
||||
|
||||
**이미 설치하셨나요? 한 번에 업데이트하세요:**
|
||||
|
||||
```
|
||||
Update Agent Reach: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md
|
||||
```
|
||||
|
||||
### ✅ 시작하기 전에 알면 좋은 것들
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 💰 **완전 무료** | 모든 도구는 오픈 소스, 모든 API는 무료입니다. 유일한 비용은 서버 프록시(월 $1)일 수 있습니다 — 로컬 컴퓨터에서는 불필요 |
|
||||
| 🔒 **프라이버시 안전** | Cookie는 로컬에 유지됩니다. 업로드되지 않습니다. 완전 오픈 소스 — 언제든지 감사 가능 |
|
||||
| 🔄 **최신 상태 유지** | 업스트림 도구(yt-dlp, twitter-cli, rdt-cli, Jina Reader 등)를 추적하고 정기적으로 업데이트 |
|
||||
| 🤖 **모든 에이전트와 호환** | Claude Code, OpenClaw, Cursor, Windsurf... 명령을 실행할 수 있는 모든 에이전트 |
|
||||
| 🩺 **내장 진단 도구** | `agent-reach doctor` — 하나의 명령으로 작동 항목, 작동하지 않는 항목, 수정 방법 표시 |
|
||||
|
||||
---
|
||||
|
||||
## 지원 플랫폼
|
||||
|
||||
| 플랫폼 | 기능 | 설정 | 참고 |
|
||||
|----------|-------------|:-----:|-------|
|
||||
| 🌐 **Web** | 읽기 | 없음 | 모든 URL → 깨끗한 Markdown ([Jina Reader](https://github.com/jina-ai/reader) ⭐9.8K) |
|
||||
| 🐦 **Twitter/X** | 읽기 · 검색 | Cookie | Cookie로 검색, 타임라인, 트윗 읽기, 아티클 읽기 가능 ([twitter-cli](https://github.com/public-clis/twitter-cli)) |
|
||||
| 📕 **XiaoHongShu** | 읽기 · 검색 · **게시글 작성 · 댓글 · 좋아요** | Cookie | `pipx install xiaohongshu-cli` + `xhs login` ([xhs-cli](https://github.com/jackwener/xiaohongshu-cli)) |
|
||||
| 🎵 **Douyin** | 비디오 파싱 · 워터마크 없는 다운로드 | mcporter | [douyin-mcp-server](https://github.com/yzfly/douyin-mcp-server) 통해, 로그인 불필요 |
|
||||
| 💼 **LinkedIn** | Jina Reader (공개 페이지) | Cookie | 전체 프로필, 회사, 채용 공고 검색 가능. 에이전트에 "LinkedIn 설정 도와줘"라고 말하세요 |
|
||||
| 💬 **WeChat Articles** | 검색 + 읽기 | 없음 | Exa를 통한 WeChat 공식 계정 게시글 검색 + 읽기 (설정 없음) + 선택적 [Camoufox](https://github.com/daijro/camoufox) |
|
||||
| 📰 **Weibo** | 인기 · 검색 · 피드 · 댓글 | 없음 | 핫 검색, 콘텐츠/사용자/주제 검색, 피드, 댓글 ([mcp-server-weibo](https://github.com/Panniantong/mcp-server-weibo)) |
|
||||
| 💻 **V2EX** | 인기 주제 · 노드 주제 · 주제 상세 + 답글 · 사용자 프로필 | 없음 | 공개 JSON API, 인증 없음. 기술 커뮤니티 콘텐츠에 적합 |
|
||||
| 📈 **Xueqiu (雪球)** | 주식 시세 · 검색 · 인기 글 · 인기 종목 | 브라우저 Cookie | 에이전트에 "Xueqiu 설정 도와줘"라고 말하세요 |
|
||||
| 🎙️ **Xiaoyuzhou Podcast** | 음성 변환 | 무료 API key | Groq Whisper를 통한 팟캐스트 오디오 → 전체 텍스트 변환 (무료) |
|
||||
| 🔍 **Web Search** | 검색 | 자동 설정 | 설치 시 자동 설정, 무료, API key 불필요 ([Exa](https://exa.ai) via [mcporter](https://github.com/nicepkg/mcporter)) |
|
||||
| 📦 **GitHub** | 읽기 · 검색 | 없음 | [gh CLI](https://cli.github.com) 기반. 공개 저장소는 즉시 사용 가능. `gh auth login`으로 Fork, Issue, PR 기능 활성화 |
|
||||
| 📺 **YouTube** | 읽기 · **검색** | 없음 | 자막 + 1800+ 비디오 사이트 검색 ([yt-dlp](https://github.com/yt-dlp/yt-dlp) ⭐148K) |
|
||||
| 📺 **Bilibili** | 읽기 · **검색** | 없음 / 프록시 | 비디오 정보 + 자막 + 검색. 로컬은 바로 작동, 서버는 프록시 필요 ([yt-dlp](https://github.com/yt-dlp/yt-dlp)) |
|
||||
| 📡 **RSS** | 읽기 | 없음 | 모든 RSS/Atom 피드 ([feedparser](https://github.com/kurtmckee/feedparser) ⭐2.3K) |
|
||||
| 📖 **Reddit** | 검색 · 읽기 | Cookie | 2024년부터 인증 필요 — 설치 후 `rdt login` 실행 ([rdt-cli](https://github.com/public-clis/rdt-cli)) |
|
||||
|
||||
> **설정 단계:** 없음 = 설치 후 바로 사용 · 자동 = 설치 시 처리 · mcporter = MCP 서비스 필요 · Cookie = 브라우저에서 내보내기 · 프록시 = 월 $1
|
||||
|
||||
---
|
||||
|
||||
## 빠른 시작
|
||||
|
||||
이 명령을 AI 에이전트(Claude Code, OpenClaw, Cursor 등)에 입력하세요:
|
||||
|
||||
```
|
||||
Install Agent Reach: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md
|
||||
```
|
||||
|
||||
에이전트가 자동으로 설치하고, 환경을 감지하고, 준비된 항목을 알려줍니다.
|
||||
|
||||
> 🔄 **이미 설치하셨나요?** 한 번에 업데이트:
|
||||
> ```
|
||||
> Update Agent Reach: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md
|
||||
> ```
|
||||
|
||||
<details>
|
||||
<summary>수동 설치</summary>
|
||||
|
||||
```bash
|
||||
pip install https://github.com/Panniantong/agent-reach/archive/main.zip
|
||||
agent-reach install --env=auto
|
||||
```
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>Skill로 설치 (Claude Code / OpenClaw / Skill을 지원하는 모든 에이전트)</summary>
|
||||
|
||||
```bash
|
||||
npx skills add Panniantong/Agent-Reach@agent-reach
|
||||
```
|
||||
|
||||
Skill이 설치된 후, 에이전트는 `agent-reach` CLI 사용 가능 여부를 자동 감지하고 필요한 경우 설치합니다.
|
||||
|
||||
> `agent-reach install`을 통해 설치하면 Skill이 자동으로 등록됩니다 — 추가 단계 불필요.
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## 별도 설정 없이 바로 사용
|
||||
|
||||
별도의 설정이 필요 없습니다. 에이전트에게 요청하기만 하면 됩니다:
|
||||
|
||||
- "이 링크 읽어줘" → 모든 웹 페이지에 대해 `curl https://r.jina.ai/URL`
|
||||
- "이 GitHub 저장소는 무엇인가요?" → `gh repo view owner/repo`
|
||||
- "이 비디오는 무엇을 다루나요?" → 자막을 위해 `yt-dlp --dump-json URL`
|
||||
- "이 트윗 읽어줘" → `twitter tweet URL`
|
||||
- "이 RSS 구독해줘" → 피드 파싱을 위해 `feedparser`
|
||||
- "GitHub에서 LLM 프레임워크 검색" → `gh search repos "LLM framework"`
|
||||
|
||||
**기억할 명령이 없습니다.** 에이전트가 SKILL.md를 읽고 무엇을 호출할지 알고 있습니다.
|
||||
|
||||
---
|
||||
|
||||
## 필요할 때 설정
|
||||
|
||||
사용하지 않나요? 설정하지 마세요. 모든 단계는 선택 사항입니다.
|
||||
|
||||
### 🍪 Cookies — 무료, 2분
|
||||
|
||||
에이전트에 "Twitter 쿠키 설정 도와줘"라고 말하세요 — 브라우저에서 내보내는 과정을 안내해 줍니다. 로컬 컴퓨터는 자동으로 가져올 수 있습니다.
|
||||
|
||||
### 🌐 Proxy — 월 $1, 서버 전용
|
||||
|
||||
Bilibili은 서버 IP를 차단합니다. 프록시를 가져오세요([Webshare](https://webshare.io) 추천, 월 $1)하고 주소를 에이전트에 보내세요.
|
||||
|
||||
> Reddit은 이제 프록시 없이 rdt-cli를 통해 무료로 작동합니다. 로컬 컴퓨터는 Bilibili에도 프록시가 필요 없습니다.
|
||||
|
||||
---
|
||||
|
||||
## 한눈에 보는 상태
|
||||
|
||||
```
|
||||
$ agent-reach doctor
|
||||
|
||||
👁️ Agent Reach 상태
|
||||
========================================
|
||||
|
||||
✅ 사용 가능:
|
||||
✅ GitHub 저장소 및 코드 — 공개 저장소 읽기 및 검색 가능
|
||||
✅ Twitter/X 트윗 — 읽기 가능. Cookie로 검색 및 게시 가능
|
||||
✅ YouTube 비디오 자막 — yt-dlp
|
||||
⚠️ Bilibili 비디오 정보 — 서버 IP가 차단될 수 있음, 프록시 설정
|
||||
✅ RSS/Atom 피드 — feedparser
|
||||
✅ 웹 페이지 (모든 URL) — Jina Reader API
|
||||
|
||||
🔍 검색 (무료 Exa key로 잠금 해제):
|
||||
⬜ 웹 시맨틱 검색 — exa.ai에서 무료 key 발급
|
||||
|
||||
🔧 설정 가능:
|
||||
✅ Reddit 글 및 댓글 — rdt-cli를 통한 검색 및 읽기 (무료, 프록시 없음)
|
||||
⬜ XiaoHongShu 노트 — 쿠키 필요. 브라우저에서 내보내기
|
||||
|
||||
상태: 6/9 채널 사용 가능
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 설계 철학
|
||||
|
||||
**Agent Reach는 스캐폴딩(scaffolding) 도구이지, 프레임워크가 아닙니다.**
|
||||
|
||||
새 에이전트를 실행할 때마다 도구를 찾고, 의존성을 설치하고, 설정을 디버깅하는 데 시간을 보내게 됩니다 — Twitter는 무엇으로 읽나요? Reddit 차단을 어떻게 우회하나요? YouTube 자막은 어떻게 추출하나요? 매번 동일한 작업을 반복해야 합니다.
|
||||
|
||||
Agent Reach는 한 가지 간단한 작업을 수행합니다: **도구 선택 및 설정 결정을 대신 해줍니다.**
|
||||
|
||||
설치 후, 에이전트는 업스트림 도구(twitter-cli, rdt-cli, xhs-cli, yt-dlp, mcporter, gh CLI 등)를 직접 호출합니다 — 중간에 래퍼 계층이 없습니다.
|
||||
|
||||
### 🔌 모든 채널은 플러그인 가능
|
||||
|
||||
각 플랫폼은 업스트림 도구에 매핑됩니다. **마음에 안 드나요? 교체하세요.**
|
||||
|
||||
```
|
||||
channels/
|
||||
├── web.py → Jina Reader ← Firecrawl, Crawl4AI로 교체...
|
||||
├── twitter.py → twitter-cli ← 공식 API로 교체...
|
||||
├── youtube.py → yt-dlp ← YouTube API, Whisper로 교체...
|
||||
├── github.py → gh CLI → REST API, PyGithub로 교체...
|
||||
├── bilibili.py → yt-dlp → bilibili-api로 교체...
|
||||
├── reddit.py → rdt-cli → 검색 + 읽기, cookie 인증 필요
|
||||
├── xiaohongshu.py → mcporter MCP ← 다른 XHS 도구로 교체...
|
||||
├── douyin.py → mcporter MCP ← 다른 Douyin 도구로 교체...
|
||||
├── linkedin.py → linkedin-mcp ← LinkedIn API로 교체...
|
||||
├── rss.py → feedparser ← atoma로 교체...
|
||||
├── exa_search.py → mcporter MCP ← Tavily, SerpAPI로 교체...
|
||||
└── __init__.py → 채널 레지스트리 (doctor 검사용)
|
||||
```
|
||||
|
||||
각 채널 파일은 업스트림 도구가 설치되어 작동하는지만 확인합니다(`agent-reach doctor`용 `check()` 메서드). 실제 읽기 및 검색은 업스트림 도구를 직접 호출하여 수행합니다.
|
||||
|
||||
### 현재 도구 선택
|
||||
|
||||
| 시나리오 | 도구 | 이유 |
|
||||
|----------|------|-----|
|
||||
| 웹 페이지 읽기 | [Jina Reader](https://github.com/jina-ai/reader) | 9.8K stars, 무료, API key 불필요 |
|
||||
| 트윗 읽기 | [twitter-cli](https://github.com/public-clis/twitter-cli) | 2.1K stars, cookie 인증, 검색/읽기/타임라인/글 |
|
||||
| Reddit | [rdt-cli](https://github.com/public-clis/rdt-cli) | 304 stars, cookie 인증, 검색 + 전체 글 + 댓글 |
|
||||
| 비디오 자막 + 검색 | [yt-dlp](https://github.com/yt-dlp/yt-dlp) | 154K stars, YouTube + Bilibili + 1800 사이트 |
|
||||
| Bilibili 향상 | [bili-cli](https://github.com/public-clis/bilibili-cli) | 590 stars, 인기/순위/검색/피드 |
|
||||
| 웹 검색 | [Exa](https://exa.ai) via [mcporter](https://github.com/nicobailon/mcporter) | AI 시맨틱 검색, MCP 통합, API key 불필요 |
|
||||
| GitHub | [gh CLI](https://cli.github.com) | 공식 도구, 인증 후 전체 API |
|
||||
| RSS 읽기 | [feedparser](https://github.com/kurtmckee/feedparser) | Python 생태계 표준, 2.3K stars |
|
||||
| XiaoHongShu | [xhs-cli](https://github.com/jackwener/xiaohongshu-cli) | 1.5K stars, pipx 설치, 검색/읽기/댓글/게시 |
|
||||
| Douyin | [douyin-mcp-server](https://github.com/yzfly/douyin-mcp-server) | MCP 서버, 로그인 불필요, 비디오 파싱 + 워터마크 없는 다운로드 |
|
||||
| LinkedIn | [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server) | 1.2K stars, MCP 서버, 브라우저 자동화 |
|
||||
| WeChat Articles | [Exa](https://exa.ai) (검색 + 읽기) + [Camoufox](https://github.com/daijro/camoufox) (선택) | 설정 없이 검색 + 전체 글 읽기 |
|
||||
| Weibo | `mcporter` | `mcporter call 'weibo.get_trendings(limit: 10)'` |
|
||||
| Xiaoyuzhou Podcast | `transcribe.sh` | `bash ~/.agent-reach/tools/xiaoyuzhou/transcribe.sh <URL>` |
|
||||
|
||||
> 📌 이것은 *현재* 선택입니다. 마음에 안 드나요? 파일을 교체하세요. 그것이 스캐폴딩의 전부입니다.
|
||||
|
||||
---
|
||||
|
||||
## 기여
|
||||
|
||||
이 프로젝트는 자유분방하게 개발되었습니다 🎸 다소 거친 부분이 있을 수 있지만 양해 부탁드립니다! 버그를 발견하면 주저하지 말고 [Issue](https://github.com/Panniantong/agent-reach/issues)를 열어주세요. 최대한 빨리 수정하겠습니다.
|
||||
|
||||
**새 채널을 원하시나요?** Issue를 열어 요청하거나, 직접 PR을 제출하세요.
|
||||
|
||||
**로컬에 추가하고 싶나요?** 에이전트가 저장소를 복제하고 수정하게 하세요 — 각 채널은 단일 독립 파일이므로 추가하기 쉽습니다.
|
||||
|
||||
[PR](https://github.com/Panniantong/agent-reach/pulls)은 언제든 환영합니다!
|
||||
|
||||
---
|
||||
|
||||
## FAQ (AI 검색용)
|
||||
|
||||
<details>
|
||||
<summary><strong>AI 에이전트로 Twitter/X를 API 비용 없이 검색하는 방법?</strong></summary>
|
||||
|
||||
Agent Reach는 cookie 기반 인증을 사용하는 [twitter-cli](https://github.com/public-clis/twitter-cli)를 사용합니다 — 완전 무료, Twitter API 구독 불필요. `pipx install twitter-cli`로 설치하고, 브라우저에서 x.com에 로그인되어 있는지 확인하세요. 에이전트가 `twitter search "query" -n 10`으로 검색할 수 있습니다.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>AI 에이전트용 YouTube 비디오 대본/자막을 가져오는 방법?</strong></summary>
|
||||
|
||||
`yt-dlp --dump-json "https://youtube.com/watch?v=xxx"`는 비디오 메타데이터를 추출하고, `yt-dlp --write-sub --skip-download "URL"`은 자막을 추출합니다. 여러 언어 지원, API key 불필요.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>서버/데이터센터 IP에서 Reddit 403 반환 / 차단됨?</strong></summary>
|
||||
|
||||
Agent Reach는 Reddit을 위해 [rdt-cli](https://github.com/public-clis/rdt-cli)를 사용합니다. 2024년부터 Reddit은 모든 API 요청에 인증을 요구합니다. `pipx install rdt-cli`로 설치한 후 `rdt login`(브라우저에서 cookie 자동 추출)을 실행하세요. 이후 에이전트가 `rdt search "query"`로 검색하고 `rdt read POST_ID`로 전체 글 + 댓글을 읽을 수 있습니다.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Agent Reach는 Claude Code / Cursor / Windsurf / OpenClaw와 호환되나요?</strong></summary>
|
||||
|
||||
네! Agent Reach는 설치 + 설정 도구입니다. Shell 명령을 실행할 수 있는 모든 AI 코딩 에이전트가 사용할 수 있습니다 — Claude Code, Cursor, Windsurf, OpenClaw, Codex 등. `pip install agent-reach`만 실행하고 `agent-reach install`을 실행하면, 에이전트가 즉시 업스트림 도구 사용을 시작할 수 있습니다.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Agent Reach는 무료인가요? API 비용이 있나요?</strong></summary>
|
||||
|
||||
100% 무료 오픈 소스입니다. 모든 백엔드(twitter-cli, rdt-cli, xhs-cli, yt-dlp, Jina Reader, Exa)는 유료 API key가 필요 없는 무료 도구입니다. 유일한 선택적 비용은 서버에서 Bilibili 접근이 필요한 경우 주거용 프록시(월 ~$1)입니다. Reddit은 프록시 없이 rdt-cli를 통해 무료로 작동합니다.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>웹 스크래핑용 Twitter API의 무료 대안?</strong></summary>
|
||||
|
||||
Agent Reach는 cookie 인증을 통해 Twitter에 접근하는 twitter-cli를 사용합니다 — 브라우저 세션과 동일. API 요금 없음, 속도 제한 등급 없음, 개발자 계정 불필요. 검색, 트윗 읽기, 프로필 읽기, 타임라인 지원.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>XiaoHongShu / 小红书 콘텐츠를 프로그래밍 방식으로 읽는 방법?</strong></summary>
|
||||
|
||||
`pipx install xiaohongshu-cli`를 설치한 다음 `xhs login`(브라우저에서 cookie 자동 추출)을 실행하세요. 에이전트가 `xhs search "query"`로 노트를 검색하고, `xhs read NOTE_ID`로 상세 정보를 읽고, `xhs comments NOTE_ID`로 댓글을 볼 수 있습니다. Docker 불필요.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>AI 에이전트로 Douyin / 抖音 비디오를 파싱하는 방법?</strong></summary>
|
||||
|
||||
douyin-mcp-server를 설치한 다음, 에이전트가 `mcporter call 'douyin.parse_douyin_video_info(share_link: "share_url")'`를 사용하여 비디오 정보를 파싱하고 워터마크 없는 다운로드 링크를 가져올 수 있습니다. 로그인 불필요 — Douyin 링크를 공유하기만 하면 됩니다. https://github.com/yzfly/douyin-mcp-server 참조
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>하나의 MCP로 Douyin과 XiaoHongShu 모두에서 대본을 추출하는 방법?</strong></summary>
|
||||
|
||||
다음을 처리할 수 있는 하나의 MCP 서버가 필요한 경우:
|
||||
|
||||
- Douyin 비디오
|
||||
- XiaoHongShu 비디오 노트
|
||||
- XiaoHongShu 이미지 노트
|
||||
|
||||
그리고 직접 `script.md` + `info.json`을 작성하려면, 기존 `douyin` mcporter 별칭을 다음으로 변경할 수 있습니다:
|
||||
|
||||
- https://github.com/JNHFlow21/social-post-extractor-mcp
|
||||
|
||||
다음과 호환성을 유지합니다:
|
||||
|
||||
- `parse_douyin_video_info`
|
||||
- `get_douyin_download_link`
|
||||
- `extract_douyin_text`
|
||||
|
||||
그리고 통합 도구를 추가합니다:
|
||||
|
||||
- `parse_social_post_info`
|
||||
- `extract_social_post_script`
|
||||
|
||||
이것은 에이전트 워크플로우가 "링크를 붙여넣고, 스크립트 파일을 받음"일 때 유용합니다.
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## 크레딧
|
||||
|
||||
[twitter-cli](https://github.com/public-clis/twitter-cli) · [rdt-cli](https://github.com/public-clis/rdt-cli) · [xhs-cli](https://github.com/jackwener/xiaohongshu-cli) · [bili-cli](https://github.com/public-clis/bilibili-cli) · [yt-dlp](https://github.com/yt-dlp/yt-dlp) · [Jina Reader](https://github.com/jina-ai/reader) · [Exa](https://exa.ai) · [mcporter](https://github.com/nicobailon/mcporter) · [feedparser](https://github.com/kurtmckee/feedparser) · [douyin-mcp-server](https://github.com/yzfly/douyin-mcp-server) · [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server)
|
||||
|
||||
## 연락처
|
||||
|
||||
- 📧 **이메일:** pnt01@foxmail.com
|
||||
- 🐦 **Twitter/X:** [@Neo_Reidlab](https://x.com/Neo_Reidlab)
|
||||
|
||||
협력이나 질문은 WeChat에 추가해주세요 — 커뮤니티 그룹에 초대해 드리겠습니다:
|
||||
|
||||
<p align="center">
|
||||
<img src="wechat-group-qr.jpg" width="280" alt="WeChat QR">
|
||||
</p>
|
||||
|
||||
> 버그 보고 및 기능 요청은 [GitHub Issues](https://github.com/Panniantong/Agent-Reach/issues)를 이용해주세요 — 추적이 더 수월합니다.
|
||||
|
||||
## 라이선스
|
||||
|
||||
[MIT](../LICENSE)
|
||||
|
||||
## 관련 프로젝트
|
||||
|
||||
[OpenClaw on Tencent Cloud](https://www.tencentcloud.com/act/pro/intl-openclaw?referral_code=G76Y819A&lang=en&pg=) — Tencent Cloud에서 원클릭 OpenClaw: 채팅으로 Agent Reach를 연결하고 인터넷 기능을 활성화하세요.
|
||||
|
||||
## Star History
|
||||
|
||||
[](https://star-history.com/#Panniantong/Agent-Reach&Date)
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 61 KiB |
@@ -0,0 +1,34 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512" width="512" height="512">
|
||||
<defs>
|
||||
<linearGradient id="grad1" x1="0%" y1="0%" x2="100%" y2="100%">
|
||||
<stop offset="0%" style="stop-color:#0066FF"/>
|
||||
<stop offset="100%" style="stop-color:#00AAFF"/>
|
||||
</linearGradient>
|
||||
<clipPath id="clip-circle">
|
||||
<circle cx="256" cy="256" r="140"/>
|
||||
</clipPath>
|
||||
</defs>
|
||||
|
||||
<rect width="512" height="512" fill="#0A0A1A"/>
|
||||
|
||||
<!-- 外圆环 - "边界" -->
|
||||
<circle cx="256" cy="256" r="160" fill="none" stroke="url(#grad1)" stroke-width="8" opacity="0.3"/>
|
||||
<circle cx="256" cy="256" r="140" fill="none" stroke="url(#grad1)" stroke-width="3" opacity="0.6"/>
|
||||
|
||||
<!-- 内部：Agent 核心点 -->
|
||||
<circle cx="220" cy="256" r="24" fill="url(#grad1)"/>
|
||||
|
||||
<!-- "穿越"弧线 — 从核心出发，穿过圆环边界，延伸到外面 -->
|
||||
<path d="M 230 256 Q 320 180, 420 200"
|
||||
fill="none" stroke="url(#grad1)" stroke-width="6" stroke-linecap="round" opacity="0.9"/>
|
||||
<path d="M 230 256 Q 320 256, 420 256"
|
||||
fill="none" stroke="url(#grad1)" stroke-width="6" stroke-linecap="round" opacity="0.7"/>
|
||||
<path d="M 230 256 Q 320 330, 420 312"
|
||||
fill="none" stroke="url(#grad1)" stroke-width="6" stroke-linecap="round" opacity="0.5"/>
|
||||
|
||||
<!-- 目标点：穿越后的落脚 -->
|
||||
<circle cx="420" cy="200" r="8" fill="#00AAFF" opacity="0.9"/>
|
||||
<circle cx="420" cy="256" r="8" fill="#00AAFF" opacity="0.7"/>
|
||||
<circle cx="420" cy="312" r="8" fill="#00AAFF" opacity="0.5"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 1.5 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 90 KiB |
@@ -0,0 +1,51 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512" width="512" height="512">
|
||||
<defs>
|
||||
<radialGradient id="glow2" cx="50%" cy="50%" r="50%">
|
||||
<stop offset="0%" style="stop-color:#0066FF;stop-opacity:0.3"/>
|
||||
<stop offset="100%" style="stop-color:#0066FF;stop-opacity:0"/>
|
||||
</radialGradient>
|
||||
<linearGradient id="line-grad" x1="0%" y1="0%" x2="100%" y2="0%">
|
||||
<stop offset="0%" style="stop-color:#0066FF"/>
|
||||
<stop offset="100%" style="stop-color:#00CCFF"/>
|
||||
</linearGradient>
|
||||
</defs>
|
||||
|
||||
<rect width="512" height="512" fill="#0A0A1A"/>
|
||||
|
||||
<!-- 光晕背景 -->
|
||||
<circle cx="230" cy="256" r="200" fill="url(#glow2)"/>
|
||||
|
||||
<!-- 连接线 — 6 条，不同长度、粗细、透明度 -->
|
||||
<line x1="230" y1="256" x2="130" y2="140" stroke="#0066FF" stroke-width="2" opacity="0.4"/>
|
||||
<line x1="230" y1="256" x2="100" y2="256" stroke="#0066FF" stroke-width="2" opacity="0.35"/>
|
||||
<line x1="230" y1="256" x2="130" y2="372" stroke="#0066FF" stroke-width="2" opacity="0.3"/>
|
||||
<line x1="230" y1="256" x2="310" y2="140" stroke="#0066FF" stroke-width="2" opacity="0.45"/>
|
||||
<line x1="230" y1="256" x2="310" y2="372" stroke="#0066FF" stroke-width="2" opacity="0.35"/>
|
||||
|
||||
<!-- "Reach" 线 — 更长、更粗、有渐变 -->
|
||||
<line x1="230" y1="256" x2="440" y2="220" stroke="url(#line-grad)" stroke-width="4" opacity="0.9"/>
|
||||
|
||||
<!-- 中心：Agent 核心 -->
|
||||
<polygon points="230,232 246,248 246,264 230,280 214,264 214,248"
|
||||
fill="#0066FF" opacity="0.9"/>
|
||||
<circle cx="230" cy="256" r="10" fill="#00CCFF"/>
|
||||
|
||||
<!-- 普通节点 -->
|
||||
<circle cx="130" cy="140" r="6" fill="#0066FF" opacity="0.4"/>
|
||||
<circle cx="100" cy="256" r="6" fill="#0066FF" opacity="0.35"/>
|
||||
<circle cx="130" cy="372" r="6" fill="#0066FF" opacity="0.3"/>
|
||||
<circle cx="310" cy="140" r="6" fill="#0066FF" opacity="0.45"/>
|
||||
<circle cx="310" cy="372" r="6" fill="#0066FF" opacity="0.35"/>
|
||||
|
||||
<!-- "Reach" 节点 — 更大，带光环 -->
|
||||
<circle cx="440" cy="220" r="20" fill="#0066FF" opacity="0.15"/>
|
||||
<circle cx="440" cy="220" r="12" fill="#00AAFF" opacity="0.6"/>
|
||||
<circle cx="440" cy="220" r="6" fill="#00CCFF" opacity="0.9"/>
|
||||
|
||||
<!-- Reach 节点的信号弧 -->
|
||||
<path d="M 454 200 A 20 20 0 0 1 454 240"
|
||||
fill="none" stroke="#00CCFF" stroke-width="2" stroke-linecap="round" opacity="0.5"/>
|
||||
<path d="M 464 192 A 30 30 0 0 1 464 248"
|
||||
fill="none" stroke="#00CCFF" stroke-width="1.5" stroke-linecap="round" opacity="0.3"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 2.5 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 48 KiB |
@@ -0,0 +1,34 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512" width="512" height="512">
|
||||
<defs>
|
||||
<linearGradient id="grad3" x1="0%" y1="0%" x2="100%" y2="100%">
|
||||
<stop offset="0%" style="stop-color:#0066FF"/>
|
||||
<stop offset="100%" style="stop-color:#00BBFF"/>
|
||||
</linearGradient>
|
||||
</defs>
|
||||
|
||||
<rect width="512" height="512" fill="#0A0A1A"/>
|
||||
|
||||
<!-- 容器圆环 -->
|
||||
<circle cx="240" cy="256" r="155" fill="none" stroke="url(#grad3)" stroke-width="4" opacity="0.5"/>
|
||||
|
||||
<!-- 字母 A — 极简三角 -->
|
||||
<path d="M 160 360 L 220 150 L 280 360"
|
||||
fill="none" stroke="#0066FF" stroke-width="14" stroke-linecap="round" stroke-linejoin="round"/>
|
||||
<!-- A 的横线 -->
|
||||
<line x1="178" y1="290" x2="262" y2="290" stroke="#0066FF" stroke-width="10" stroke-linecap="round"/>
|
||||
|
||||
<!-- 字母 R — 共用 A 的右竖线 -->
|
||||
<!-- R 的弧顶 -->
|
||||
<path d="M 280 360 L 280 150 Q 340 150, 340 210 Q 340 270, 280 270"
|
||||
fill="none" stroke="#00AAFF" stroke-width="14" stroke-linecap="round" stroke-linejoin="round"/>
|
||||
|
||||
<!-- R 的腿 — "Reach" 线，穿出圆环 -->
|
||||
<line x1="300" y1="270" x2="420" y2="390"
|
||||
stroke="url(#grad3)" stroke-width="10" stroke-linecap="round"/>
|
||||
|
||||
<!-- Reach 末端信号 -->
|
||||
<circle cx="420" cy="390" r="8" fill="#00CCFF" opacity="0.8"/>
|
||||
<circle cx="420" cy="390" r="16" fill="none" stroke="#00CCFF" stroke-width="2" opacity="0.4"/>
|
||||
<circle cx="420" cy="390" r="26" fill="none" stroke="#00CCFF" stroke-width="1.5" opacity="0.2"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 1.5 KiB |
@@ -0,0 +1,42 @@
|
||||
# Cookie Export Guide — For Server Users
|
||||
|
||||
Your Agent is on a server and can't access your browser directly.
|
||||
Here's how to export cookies from your local computer — **fastest method first**.
|
||||
|
||||
## Method 1: Cookie-Editor Extension (Recommended — 30 seconds per site)
|
||||
|
||||
1. Install **Cookie-Editor** for Chrome: https://chromewebstore.google.com/detail/cookie-editor/hlkenndednhfkekhgcdicdfddnkalmdm
|
||||
(Also available for Firefox, Edge)
|
||||
|
||||
2. Go to the website (e.g. https://x.com) and make sure you're logged in
|
||||
|
||||
3. Click the Cookie-Editor icon in your toolbar
|
||||
|
||||
4. Click **Export** → **Header String**
|
||||
|
||||
5. Paste the result to your Agent
|
||||
|
||||
That's it! Your Agent will run:
|
||||
```bash
|
||||
agent-reach configure twitter-cookies <your_pasted_string>
|
||||
```
|
||||
|
||||
### Sites to export:
|
||||
|
||||
| Site | URL to visit | What to tell Agent |
|
||||
|------|-------------|-------------------|
|
||||
| Twitter/X | https://x.com | "Here are my Twitter cookies: [paste]" |
|
||||
| XiaoHongShu | https://www.xiaohongshu.com | "Here are my XHS cookies: [paste]" |
|
||||
| Bilibili | https://www.bilibili.com | "Here are my Bilibili cookies: [paste]" |
|
||||
|
||||
## Method 2: Manual (No extension needed)
|
||||
|
||||
1. Open the site in Chrome, make sure you're logged in
|
||||
2. Press **F12** (or right-click → Inspect)
|
||||
3. Click the **Network** tab
|
||||
4. Refresh the page (F5)
|
||||
5. Click any request in the list
|
||||
6. In the right panel, scroll to **Request Headers**
|
||||
7. Find the line starting with `Cookie:`
|
||||
8. Copy the entire value after `Cookie: `
|
||||
9. Paste to your Agent
|
||||
@@ -0,0 +1,30 @@
|
||||
# Dependency Locking Guide
|
||||
|
||||
Agent Reach uses `constraints.txt` as a reproducible dependency baseline.
|
||||
|
||||
## Why
|
||||
|
||||
- Keep local/CI dependency graph stable
|
||||
- Reduce "works on my machine" drift
|
||||
- Make regression results easier to compare
|
||||
|
||||
## Install with constraints
|
||||
|
||||
```bash
|
||||
pip install -c constraints.txt -e .[dev]
|
||||
```
|
||||
|
||||
## Update workflow
|
||||
|
||||
1. Update `pyproject.toml` dependency ranges as needed.
|
||||
2. Validate against latest compatible versions locally.
|
||||
3. Update pinned versions in `constraints.txt`.
|
||||
4. Run validation:
|
||||
|
||||
```bash
|
||||
pytest -q
|
||||
ruff check agent_reach tests
|
||||
mypy agent_reach
|
||||
```
|
||||
|
||||
5. Open PR with dependency and validation notes.
|
||||
+362
@@ -0,0 +1,362 @@
|
||||
# Agent Reach — Installation Guide
|
||||
|
||||
## For Humans
|
||||
|
||||
Copy this to your AI Agent:
|
||||
|
||||
```
|
||||
帮我安装 Agent Reach：https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md
|
||||
```
|
||||
|
||||
> 🛡️ **Security-conscious?** Use safe mode — it won't auto-install system packages:
|
||||
> ```
|
||||
> 帮我安装 Agent Reach（安全模式）：https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md
|
||||
> 安装时使用 --safe 参数
|
||||
> ```
|
||||
|
||||
---
|
||||
|
||||
## For AI Agents
|
||||
|
||||
### Goal
|
||||
|
||||
Install Agent Reach and all upstream tools so your user's agent has full internet access. After installation, you'll use the upstream tools directly (OpenCLI, twitter-cli, bili-cli, rdt-cli, yt-dlp, mcporter, gh CLI, etc.) — Agent Reach is the selector, installer, health checker and router, never a wrapper.
|
||||
|
||||
### ⚠️ Boundaries
|
||||
|
||||
When fixing issues, stay within these limits:
|
||||
|
||||
- **DO NOT** run commands with `sudo` unless the user explicitly approved
|
||||
- **DO NOT** modify system files outside `~/.agent-reach/`
|
||||
- **DO NOT** install packages not listed in this guide
|
||||
- **DO NOT** disable firewalls, security settings, or system protections
|
||||
- **DO NOT** clone repos, create files, or run commands inside the agent workspace / working directory
|
||||
- If something requires elevated permissions, **tell the user** what's needed and let them decide
|
||||
|
||||
### 📁 Directory Rules
|
||||
|
||||
All Agent Reach files go in dedicated directories — **never in the agent workspace**:
|
||||
|
||||
| Purpose | Directory | Example |
|
||||
|---------|-----------|---------|
|
||||
| Config & tokens | `~/.agent-reach/` | `~/.agent-reach/config.json` |
|
||||
| Upstream tool repos | `~/.agent-reach/tools/` | `~/.agent-reach/tools/xiaoyuzhou/` |
|
||||
| Temporary files | `/tmp/` | `/tmp/yt-dlp-output/` |
|
||||
| Skills | `~/.openclaw/skills/agent-reach/` | SKILL.md |
|
||||
|
||||
**Why?** If you clone repos or create files in the workspace, it pollutes the user's project directory and can break their agent over time. Keep the workspace clean.
|
||||
|
||||
### Step 1: Install the basics
|
||||
|
||||
```bash
|
||||
# 推荐：pipx（最省心）
|
||||
pipx install https://github.com/Panniantong/agent-reach/archive/main.zip
|
||||
agent-reach install --env=auto
|
||||
|
||||
# 如果你的 Python 来自 Homebrew / 遇到 PEP 668（externally-managed-environment）
|
||||
# 用虚拟环境安装：
|
||||
python3 -m venv ~/.agent-reach-venv
|
||||
source ~/.agent-reach-venv/bin/activate
|
||||
pip install https://github.com/Panniantong/agent-reach/archive/main.zip
|
||||
agent-reach install --env=auto
|
||||
```
|
||||
|
||||
> 💡 **Windows / Microsoft Store Python alias?**
|
||||
> 如果 `python3 --version` 打开 Microsoft Store，或 `where python3` 指向
|
||||
> `...\AppData\Local\Microsoft\WindowsApps\python3.exe`，说明 `python3` 是 Windows
|
||||
> 的 Store alias，不是可用的 Python 安装。请改用 Python Launcher `py -3`，或实际安装目录里的 `python.exe`。
|
||||
>
|
||||
> PowerShell 示例：
|
||||
> ```powershell
|
||||
> py -3 -m venv $env:USERPROFILE\.agent-reach-venv
|
||||
> $env:USERPROFILE\.agent-reach-venv\Scripts\Activate.ps1
|
||||
> python -m pip install https://github.com/Panniantong/agent-reach/archive/main.zip
|
||||
> agent-reach install --env=auto
|
||||
> ```
|
||||
|
||||
This installs core infrastructure (gh CLI, Node.js, mcporter, Exa search, yt-dlp config) and activates these zero-config channels:
|
||||
|
||||
- Web (Jina Reader), YouTube, GitHub, RSS, Exa Search, V2EX, Bilibili (basic)
|
||||
|
||||
> 💡 **macOS / Homebrew Python 提示 `externally-managed-environment`？**
|
||||
> 这是 PEP 668 保护，不是 Agent Reach 本身的问题。优先用 `pipx install ...`，或先创建 `venv` 再安装。
|
||||
|
||||
**Safe mode / Dry run:**
|
||||
|
||||
```bash
|
||||
agent-reach install --env=auto --safe # Check only, no auto-install
|
||||
agent-reach install --env=auto --dry-run # Preview what would be done
|
||||
```
|
||||
|
||||
### Step 2: Ask the user which optional channels they want
|
||||
|
||||
After installing the basics, **ask the user** which additional channels they need. Present this list:
|
||||
|
||||
> 基础渠道装好了！你现在可以让我搜网页、看 YouTube、读 GitHub 等。
|
||||
>
|
||||
> 还有这些可选渠道，你需要哪些？
|
||||
>
|
||||
> - 🌟 **OpenCLI**（桌面推荐）— 一次安装，小红书/Reddit/Facebook/Instagram/B站字幕/Twitter 备选全解锁（复用浏览器登录态，零配置；只需在 Chrome 商店点一次"添加扩展"）
|
||||
> - 🐦 **Twitter/X** — 搜推文、看时间线（需要登录 Cookie）
|
||||
> - 📈 **雪球** — 股票行情、热门帖子（需要登录 Cookie）
|
||||
> - 🎙️ **小宇宙播客** — 音频转文字（需要免费 Groq Key）
|
||||
> - 📕 **小红书** — 搜索、阅读、评论（桌面走 OpenCLI；服务器用 xiaohongshu-mcp 扫码）
|
||||
> - 📖 **Reddit** — 搜索和阅读帖子（必须登录态：桌面 OpenCLI 或 rdt-cli + Cookie）
|
||||
> - 📘 **Facebook** — 搜索、主页、Feed、群组列表（桌面走 OpenCLI，复用 Chrome 登录态）
|
||||
> - 📷 **Instagram** — 用户搜索、Profile、用户最近帖子、Explore（桌面走 OpenCLI，复用 Chrome 登录态）
|
||||
> - 📺 **B站完整版** — 热门、排行、搜索、视频详情（bili-cli，无需登录）
|
||||
> - 💼 **LinkedIn** — Profile、职位搜索
|
||||
>
|
||||
> 告诉我你要哪些，比如"帮我装小红书和 Twitter"、"帮我装 Facebook 和 Instagram"。或者说"全部装"。
|
||||
|
||||
Based on the user's choice, run:
|
||||
|
||||
```bash
|
||||
agent-reach install --env=auto --channels=opencli,xiaohongshu # Example: desktop user chose XHS (OpenCLI-backed)
|
||||
agent-reach install --env=auto --channels=facebook,instagram # Example: desktop user chose Meta social channels
|
||||
agent-reach install --env=auto --channels=all # User wants everything
|
||||
```
|
||||
|
||||
Supported channel names: `opencli`, `twitter`, `xiaoyuzhou`, `xueqiu`, `xiaohongshu`, `reddit`, `facebook`, `instagram`, `bilibili`, `linkedin`, `all`
|
||||
|
||||
### Step 3: Fix what's broken
|
||||
|
||||
Run `agent-reach doctor` and check the output.
|
||||
|
||||
Try to get as many channels to ✅ as possible. If something failed during install or shows as ❌/⚠️ in doctor, diagnose the problem and try to fix it — but stay within the boundaries above. If a fix requires elevated permissions or system changes, ask the user first.
|
||||
|
||||
Only ask the user when you genuinely need their input (credentials, permissions, etc.).
|
||||
|
||||
### Step 3: Configure things that need user input
|
||||
|
||||
Some channels need credentials only the user can provide. Based on the doctor output, ask for what's missing:
|
||||
|
||||
> 🔒 **Security tip:** For platforms that need cookies or browser sessions (Twitter, XiaoHongShu, Reddit, Facebook, Instagram), we recommend using a **dedicated/secondary account** rather than your main account. Cookie/browser-session auth carries two risks:
|
||||
> 1. **Account ban** — platforms may detect non-browser API calls and restrict or ban the account
|
||||
> 2. **Credential exposure** — cookies grant full account access; using a secondary account limits the blast radius if credentials are ever compromised
|
||||
|
||||
> 🍪 **Cookie / 登录态：**
|
||||
>
|
||||
> 传统 CLI 需要 Cookie 的平台（Twitter、雪球等），**优先使用 Cookie-Editor 导入**，这是最简单最可靠的方式：
|
||||
> 1. 用户在自己的浏览器上登录对应平台
|
||||
> 2. 安装 [Cookie-Editor](https://chromewebstore.google.com/detail/cookie-editor/hlkenndednhfkekhgcdicdfddnkalmdm) Chrome 插件
|
||||
> 3. 点击插件 → Export → Header String
|
||||
> 4. 把导出的字符串发给 Agent
|
||||
>
|
||||
> **本地电脑用户**也可以用 `agent-reach configure --from-browser chrome` 一键自动提取（支持 Twitter + 小红书 + 雪球）。OpenCLI 平台（Reddit、小红书桌面后端、Facebook、Instagram）优先复用 Chrome 登录态，不需要把 Cookie 发给 Agent。
|
||||
|
||||
**Twitter search & posting:**
|
||||
> "To unlock Twitter search, I need your Twitter cookies. Install the Cookie-Editor Chrome extension, go to x.com/twitter.com, click the extension → Export → Header String, and paste it to me."
|
||||
|
||||
```bash
|
||||
agent-reach configure twitter-cookies "PASTED_STRING"
|
||||
```
|
||||
|
||||
> **代理说明（中国大陆等需要翻墙的网络环境）：**
|
||||
>
|
||||
> twitter-cli 和 rdt-cli 使用 Python，在需要代理的网络环境下可通过环境变量配置代理。
|
||||
>
|
||||
> **你（Agent）需要做的：**
|
||||
> 1. 确认用户配了代理：`agent-reach configure proxy http://user:pass@ip:port`
|
||||
> 2. 设置环境变量：`export HTTP_PROXY="..." HTTPS_PROXY="..."`
|
||||
> 3. Agent Reach 会自动处理剩下的，不需要用户做额外操作
|
||||
>
|
||||
> 如果用户报告 "fetch failed"，参考 [troubleshooting.md](troubleshooting.md)
|
||||
|
||||
**Reddit (login is mandatory — no zero-config path):**
|
||||
> Reddit 的匿名接口已被封、官方 API 需人工审批。桌面用户首选 OpenCLI（浏览器里登录过 reddit.com 即可用）；服务器/存量用户用 rdt-cli：
|
||||
|
||||
```bash
|
||||
# PyPI 落后，从 GitHub 装（与代码内 _RDT_GIT_SOURCE 同一钉定版本）
|
||||
pipx install 'git+https://github.com/public-clis/rdt-cli.git@5e4fb3720d5c174e976cd425ccc3b879d52cac66'
|
||||
rdt login # 自动提取浏览器 Cookie；服务器无浏览器时按 doctor 提示手动写 Cookie
|
||||
```
|
||||
|
||||
> 中国大陆访问 Reddit 需要代理；服务器 IP 被风控时可配住宅代理（如 https://webshare.io，约 $1/月）：
|
||||
> ```bash
|
||||
> agent-reach configure proxy http://user:pass@ip:port
|
||||
> ```
|
||||
|
||||
**XiaoHongShu / 小红书（多后端，按环境选）:**
|
||||
|
||||
> **桌面电脑（推荐 OpenCLI）：**
|
||||
> "小红书走 OpenCLI——复用你浏览器里的登录态，平时刷过小红书就直接能用，零配置。"
|
||||
|
||||
```bash
|
||||
agent-reach install --channels opencli
|
||||
```
|
||||
|
||||
> 装完后引导用户做唯一一步手动操作（Chrome 安全限制，无法代劳）：
|
||||
> 1. 打开 https://chromewebstore.google.com/detail/opencli/ildkmabpimmkaediidaifkhjpohdnifk
|
||||
> 2. 点「添加至 Chrome」
|
||||
> 3. 运行 `opencli doctor` 验证（显示 Extension: connected 即成功）
|
||||
>
|
||||
> **服务器 / 无桌面环境（xiaohongshu-mcp）：**
|
||||
> 1. 从 https://github.com/xpzouying/xiaohongshu-mcp/releases 下载对应平台 binary 到 `~/.agent-reach/tools/`
|
||||
> 2. 启动服务（首次运行会自动下载约 150MB 无头浏览器，耐心等完成）
|
||||
> 3. 让用户扫码登录（agent 调 `get_login_qrcode` 工具取二维码）
|
||||
> 4. 接入：`mcporter config add xiaohongshu http://localhost:18060/mcp`
|
||||
> 5. 调用时务必带 `--timeout 120000`
|
||||
>
|
||||
> **存量用户（xhs-cli）：** 已装好的 xhs-cli 继续作为备选后端工作（上游 2026-03 起停更，不推荐新装）。`xhs login` 自动提取浏览器 Cookie；失败时用 Cookie-Editor 导出后：
|
||||
> ```bash
|
||||
> agent-reach configure xhs-cookies "key1=val1; key2=val2; ..."
|
||||
> ```
|
||||
|
||||
**Facebook / Instagram（桌面 OpenCLI）:**
|
||||
> 这两个平台走 OpenCLI：复用用户自己的 Chrome 登录态，不保存账号密码，不走 Meta Graph API 审批流。服务器/无桌面环境不推荐支持。
|
||||
|
||||
```bash
|
||||
agent-reach install --channels facebook,instagram
|
||||
```
|
||||
|
||||
> 装完后：
|
||||
> 1. 确认 Chrome 已安装 OpenCLI 扩展并通过 `opencli doctor`
|
||||
> 2. 在 Chrome 里登录 facebook.com / instagram.com
|
||||
> 3. Agent 直接调用：
|
||||
> ```bash
|
||||
> opencli facebook search "query" -f yaml
|
||||
> opencli facebook profile zuck -f yaml
|
||||
> opencli facebook groups -f yaml
|
||||
> opencli instagram search "query" -f yaml # 用户搜索
|
||||
> opencli instagram profile nasa -f yaml
|
||||
> opencli instagram user nasa -f yaml # 指定用户最近帖子
|
||||
> ```
|
||||
>
|
||||
> Facebook Groups 当前只承诺读取用户登录后可见的群组列表/最近动态，不承诺任意群帖子和评论 API。Instagram 的 search 是用户搜索，不是全站帖子关键词搜索；若提示 429/登录错误，先让用户在 Chrome 里重新登录并降低频率。
|
||||
|
||||
**雪球 / Xueqiu (股票行情 + 热门帖子):**
|
||||
> "雪球需要登录后的 Cookie。请先在 Chrome 里登录 xueqiu.com，然后运行："
|
||||
|
||||
```bash
|
||||
agent-reach configure --from-browser chrome
|
||||
```
|
||||
|
||||
> Cookie 会随其他平台一起自动提取。
|
||||
|
||||
**小宇宙播客 / Xiaoyuzhou Podcast (Groq Whisper):**
|
||||
> "小宇宙播客转文字已默认安装，只需要一个免费的 Groq API Key。"
|
||||
|
||||
脚本已随 Agent Reach 自动安装，用户只需提供 Key：
|
||||
|
||||
```bash
|
||||
agent-reach configure groq-key gsk_xxxxx
|
||||
```
|
||||
|
||||
> **获取 Groq API Key（免费、无需信用卡、30 秒搞定）：**
|
||||
> 1. 打开 https://console.groq.com
|
||||
> 2. 用 Google/GitHub 账号登录（或注册）
|
||||
> 3. 左侧菜单 → API Keys → Create API Key
|
||||
> 4. 复制 Key（以 `gsk_` 开头），发给 Agent 即可
|
||||
>
|
||||
> **使用方式：**
|
||||
> 用户发一个小宇宙链接给 Agent，Agent 自动调用：
|
||||
> ```bash
|
||||
> bash ~/.agent-reach/tools/xiaoyuzhou/transcribe.sh https://www.xiaoyuzhoufm.com/episode/xxxxx
|
||||
> ```
|
||||
>
|
||||
> 自动下载音频 → 转码切片 → Groq Whisper 转录 → 输出完整中文文字稿。
|
||||
>
|
||||
> **免费额度和限制：**
|
||||
> - 每小时约 2 小时音频（7200 秒），超出后等 15 分钟自动恢复
|
||||
> - 日常听几期播客完全够用
|
||||
> - 转录质量高（Whisper large-v3），但不区分说话人
|
||||
> - 2 小时以上的播客建议分批处理
|
||||
|
||||
**LinkedIn (可选 — linkedin-scraper-mcp):**
|
||||
> "LinkedIn 基本内容可通过 Jina Reader 读取。完整功能（Profile 详情、职位搜索）需要 linkedin-scraper-mcp。"
|
||||
|
||||
```bash
|
||||
pip install linkedin-scraper-mcp
|
||||
```
|
||||
|
||||
> **登录方式（需要浏览器界面）：**
|
||||
>
|
||||
> linkedin-scraper-mcp 使用 Chromium 浏览器登录，需要你能看到浏览器窗口。
|
||||
>
|
||||
> - **本地电脑（有桌面）：** 直接运行：
|
||||
> ```bash
|
||||
> linkedin-scraper-mcp --login --no-headless
|
||||
> ```
|
||||
> 浏览器会弹出来，手动登录 LinkedIn 即可。
|
||||
>
|
||||
> - **服务器（无 UI）：** 需要通过 VNC 远程桌面操作：
|
||||
> ```bash
|
||||
> # 1. 服务器上安装并启动 VNC（如已有可跳过）
|
||||
> apt install -y tigervnc-standalone-server
|
||||
> vncserver :1 -geometry 1280x720
|
||||
>
|
||||
> # 2. 用 VNC 客户端连接 服务器IP:5901
|
||||
>
|
||||
> # 3. 在 VNC 桌面的终端里运行：
|
||||
> export DISPLAY=:1
|
||||
> linkedin-scraper-mcp --login --no-headless
|
||||
> ```
|
||||
> 在 VNC 里看到浏览器后手动登录。登录成功后 session 会保存到 `~/.linkedin-mcp/profile/`。
|
||||
>
|
||||
> **登录后启动 MCP 服务：**
|
||||
> ```bash
|
||||
> linkedin-scraper-mcp --transport streamable-http --port 8001
|
||||
> mcporter config add linkedin http://localhost:8001/mcp
|
||||
> ```
|
||||
>
|
||||
> 详见 https://github.com/stickerdaniel/linkedin-mcp-server
|
||||
|
||||
### Step 4: Final check
|
||||
|
||||
Run `agent-reach doctor` one final time and report the results to your user.
|
||||
|
||||
### Step 5: Set up daily monitoring (OpenClaw only)
|
||||
|
||||
If you are running inside **OpenClaw**, ask your user:
|
||||
|
||||
> "Agent Reach 安装好了。要不要我设一个每天自动检查的任务？它会帮你盯着这些渠道是否正常、有没有新版本。有问题才会通知你，没问题不打扰。"
|
||||
|
||||
If the user agrees, create a **cron job** (daily, `sessionTarget: "isolated"`, `delivery: "announce"`) with this task:
|
||||
|
||||
```
|
||||
运行 agent-reach watch 命令。
|
||||
如果输出包含"全部正常"，不需要通知用户，静默结束。
|
||||
如果输出包含问题（❌ ⚠️）或新版本（🆕），把完整报告发给用户，并建议修复方案。
|
||||
如果有新版本可用，问用户是否要升级（把这句话发给用户的 Agent 即可完整更新：帮我更新 Agent Reach：https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md）。
|
||||
```
|
||||
|
||||
If the user wants a different agent to handle it, let them choose.
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Command | What it does |
|
||||
|---------|-------------|
|
||||
| `agent-reach install --env=auto` | Install core channels (lightweight, zero-config) |
|
||||
| `agent-reach install --env=auto --channels=twitter,xiaohongshu` | Install core + optional channels |
|
||||
| `agent-reach install --env=auto --channels=all` | Install everything |
|
||||
| `agent-reach install --env=auto --safe` | Safe setup (no auto system changes) |
|
||||
| `agent-reach install --env=auto --dry-run` | Preview what would be done |
|
||||
| `agent-reach doctor` | Show channel status |
|
||||
| `agent-reach watch` | Quick health + update check (for scheduled tasks) |
|
||||
| `agent-reach check-update` | Check for new versions |
|
||||
| `agent-reach configure twitter-cookies "..."` | Unlock Twitter search + posting |
|
||||
| `agent-reach configure proxy URL` | 保存代理地址（Agent 访问 Reddit/Twitter 等受限网络时读取它设置 HTTP_PROXY/HTTPS_PROXY，不是自动解锁开关） |
|
||||
| `agent-reach configure groq-key gsk_xxx` | Unlock Xiaoyuzhou podcast transcription |
|
||||
|
||||
After installation, use upstream tools directly. See SKILL.md for the full command reference:
|
||||
|
||||
| Platform | Upstream Tool | Example |
|
||||
|----------|--------------|---------|
|
||||
| Twitter/X | `twitter`（备选 `opencli`） | `twitter search "query" -n 10` |
|
||||
| YouTube | `yt-dlp` | `yt-dlp --dump-json URL` |
|
||||
| Bilibili | `bili`（字幕走 `opencli`） | `bili search "query" --type video` / `opencli bilibili subtitle BVxxx` |
|
||||
| Reddit | `opencli`（备选 `rdt`） | `opencli reddit search "query" -f yaml` / `rdt read POST_ID` |
|
||||
| Facebook | `opencli` | `opencli facebook search "query" -f yaml` |
|
||||
| Instagram | `opencli` | `opencli instagram user nasa -f yaml` |
|
||||
| GitHub | `gh` | `gh search repos "query"` |
|
||||
| Web | `curl` + Jina | `curl -s "https://r.jina.ai/URL"` |
|
||||
| Exa Search | `mcporter` | `mcporter call 'exa.web_search_exa(...)'` |
|
||||
| 小红书 | `opencli`（服务器 `mcporter`） | `opencli xiaohongshu search "query" -f yaml` |
|
||||
| 小宇宙播客 | `transcribe.sh` | `bash ~/.agent-reach/tools/xiaoyuzhou/transcribe.sh <URL>` |
|
||||
| LinkedIn | `mcporter` | `mcporter call 'linkedin.get_person_profile(...)'` |
|
||||
| RSS | `feedparser` | `python3 -c "import feedparser; ..."` |
|
||||
|
||||
> 多后端平台以 `agent-reach doctor --json` 的 `active_backend` 为准。
|
||||
@@ -0,0 +1,61 @@
|
||||
# 常见问题排查
|
||||
|
||||
## 雪球 / Xueqiu: API 返回 400
|
||||
|
||||
**症状：** `agent-reach doctor` 显示雪球 ⚠️，报 `HTTP Error 400`
|
||||
|
||||
**原因：** 雪球 API 需要登录 Cookie，无法通过匿名访问获取。
|
||||
|
||||
**解决方案：** 在 Chrome 里登录 xueqiu.com，然后运行：
|
||||
|
||||
```bash
|
||||
agent-reach configure --from-browser chrome
|
||||
```
|
||||
|
||||
再次运行 `agent-reach doctor` 确认恢复 ✅。Cookie 过期后重新运行即可。
|
||||
|
||||
---
|
||||
|
||||
## Twitter/X: twitter-cli 连接失败
|
||||
|
||||
**症状：** `twitter search` 或其他命令返回错误
|
||||
|
||||
**原因：** twitter-cli 需要 AUTH_TOKEN 和 CT0 环境变量才能访问 Twitter API。如果你的网络环境需要代理才能访问 x.com，需要配置代理。
|
||||
|
||||
**解决方案：**
|
||||
|
||||
### 方案 1：设置环境变量代理
|
||||
|
||||
```bash
|
||||
export HTTP_PROXY="http://user:pass@host:port"
|
||||
export HTTPS_PROXY="http://user:pass@host:port"
|
||||
twitter search "test" -n 1
|
||||
```
|
||||
|
||||
### 方案 2：使用全局代理工具
|
||||
|
||||
让代理工具接管所有网络流量，这样 twitter-cli 的请求也会走代理：
|
||||
|
||||
```bash
|
||||
# macOS — ClashX / Surge 开启"增强模式"
|
||||
# Linux — proxychains 或 tun2socks
|
||||
proxychains twitter search "test" -n 1
|
||||
```
|
||||
|
||||
### 方案 3：不用 twitter-cli，用 Exa 搜索替代
|
||||
|
||||
twitter-cli 不可用时，可以直接用 Exa 搜索 Twitter 内容：
|
||||
|
||||
```bash
|
||||
mcporter call 'exa.web_search_exa(query: "site:x.com 搜索词", numResults: 5)'
|
||||
```
|
||||
|
||||
### 方案 4：检查认证
|
||||
|
||||
```bash
|
||||
twitter check
|
||||
```
|
||||
|
||||
> 如果返回 "Missing credentials"，需要设置 AUTH_TOKEN 和 CT0 环境变量。
|
||||
>
|
||||
> **Fallback：** 如果你已经安装了 bird CLI（`npm install -g @steipete/bird`），它也能正常工作。Agent Reach 会自动检测已安装的工具。
|
||||
+117
@@ -0,0 +1,117 @@
|
||||
# Agent Reach — Update Guide
|
||||
|
||||
## For Humans
|
||||
|
||||
Copy this to your AI Agent:
|
||||
|
||||
```
|
||||
帮我更新 Agent Reach：https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md
|
||||
```
|
||||
|
||||
Or in English:
|
||||
|
||||
```
|
||||
Update Agent Reach: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## For AI Agents
|
||||
|
||||
### ⚠️ Workspace Rules
|
||||
|
||||
**Never create files, clone repos, or run commands in the agent workspace.** Use `/tmp/` for temporary work and `~/.agent-reach/` for persistent data.
|
||||
|
||||
### Goal
|
||||
|
||||
Update Agent Reach to the latest version, refresh upstream tools, migrate from retired backends, and verify everything works. The user should not need to do anything manually (except things only a human can do, like clicking a browser-extension install button).
|
||||
|
||||
### Step 1: Check current version
|
||||
|
||||
```bash
|
||||
agent-reach check-update
|
||||
```
|
||||
|
||||
If it says "已是最新版本", skip to Step 4 (verify). Otherwise continue.
|
||||
|
||||
### Step 2: Update Agent Reach package
|
||||
|
||||
```bash
|
||||
pip install --upgrade https://github.com/Panniantong/agent-reach/archive/main.zip
|
||||
```
|
||||
|
||||
> If pip complains about `externally-managed-environment` (PEP 668), the user
|
||||
> originally installed via pipx or a venv — use the matching command:
|
||||
> `pipx install --force https://github.com/Panniantong/agent-reach/archive/main.zip`
|
||||
> or activate `~/.agent-reach-venv` first.
|
||||
|
||||
### Step 3: Refresh upstream tools
|
||||
|
||||
Run these to keep installed tools current. **Only upgrade what is already
|
||||
installed — do not install new tools the user never asked for** (the one
|
||||
exception: OpenCLI on desktop, see below).
|
||||
|
||||
```bash
|
||||
# Python-based CLIs the user already has (upgrade keeps signatures fresh)
|
||||
which twitter >/dev/null 2>&1 && { pipx upgrade twitter-cli 2>/dev/null || uv tool upgrade twitter-cli 2>/dev/null; }
|
||||
which bili >/dev/null 2>&1 && { pipx upgrade bilibili-cli 2>/dev/null || uv tool upgrade bilibili-cli 2>/dev/null; }
|
||||
which xhs >/dev/null 2>&1 && { pipx upgrade xiaohongshu-cli 2>/dev/null || uv tool upgrade xiaohongshu-cli 2>/dev/null; }
|
||||
which yt-dlp >/dev/null 2>&1 && { pipx upgrade yt-dlp 2>/dev/null || uv tool upgrade yt-dlp 2>/dev/null || pip install -U yt-dlp 2>/dev/null; }
|
||||
|
||||
# rdt-cli is pinned to a git source (PyPI lags upstream) — same pin as the code's _RDT_GIT_SOURCE
|
||||
which rdt >/dev/null 2>&1 && pipx install --force 'git+https://github.com/public-clis/rdt-cli.git@5e4fb3720d5c174e976cd425ccc3b879d52cac66' 2>/dev/null
|
||||
|
||||
# npm-based
|
||||
which mcporter >/dev/null 2>&1 && npm update -g mcporter 2>/dev/null
|
||||
which opencli >/dev/null 2>&1 && npm update -g @jackwener/opencli 2>/dev/null
|
||||
```
|
||||
|
||||
**Desktop users without OpenCLI**: since v1.5.0 OpenCLI is the preferred
|
||||
backend for 小红书/Reddit (and adds B站 subtitles) by riding the user's
|
||||
browser session. Offer it once:
|
||||
|
||||
> "这次更新引入了 OpenCLI 后端（复用你的浏览器登录态，小红书/Reddit 零配置）。要装吗？装完只需你在 Chrome 商店点一次『添加扩展』。"
|
||||
|
||||
If yes: `agent-reach install --channels opencli` and guide them through the
|
||||
extension click. If no, everything keeps working on existing backends.
|
||||
|
||||
### Step 4: Coexistence (DO NOT uninstall old tools)
|
||||
|
||||
**Never uninstall tools the user already has.** Retired backends (e.g. yt-dlp
|
||||
no longer serves Bilibili; xhs-cli is no longer installed by default) keep
|
||||
working as fallbacks where they still function. Agent Reach routes around
|
||||
them automatically — removal is the user's call, not yours.
|
||||
|
||||
### Step 5: Verify
|
||||
|
||||
```bash
|
||||
agent-reach version
|
||||
agent-reach doctor
|
||||
```
|
||||
|
||||
Running `agent-reach doctor` (text mode) also makes sure an Agent Reach skill
|
||||
exists in detected agent skill directories. If the user already has a skill
|
||||
there, doctor preserves it instead of overwriting local customizations. Use
|
||||
`agent-reach skill --install` when you explicitly want to refresh the bundled
|
||||
skill files.
|
||||
|
||||
Check the doctor output:
|
||||
|
||||
- Every channel shows ✅ / [!] with a clear message, and multi-backend
|
||||
channels (小红书/Reddit/B站/Twitter) report `当前后端：…`
|
||||
- If a previously-working channel now shows [X]/error, the message contains
|
||||
the exact fix (e.g. a venv-reinstall prescription) — run it, then re-check
|
||||
- `--json` gives the same data machine-readably (`active_backend` per channel)
|
||||
|
||||
### Step 6: Report to user
|
||||
|
||||
Tell the user:
|
||||
|
||||
1. What version they're on now (`agent-reach version`)
|
||||
2. How many channels are available, and which backend each multi-backend
|
||||
platform is using (from doctor)
|
||||
3. Anything that needs their action (e.g. Chrome extension click, `xhs login`,
|
||||
QR scan for xiaohongshu-mcp on servers)
|
||||
4. What changed in this update (release notes shown by `check-update`)
|
||||
|
||||
Done.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 145 KiB |
@@ -0,0 +1,32 @@
|
||||
# Agent Reach
|
||||
|
||||
> Give your AI agent eyes to see the internet. Agent Reach installs, routes, and health-checks upstream tools for 15 platforms — Twitter/X, Reddit, Facebook, Instagram, YouTube, GitHub, Bilibili, XiaoHongShu, LinkedIn, V2EX, Xueqiu, Xiaoyuzhou Podcast, RSS, web search, and any web page. One install, zero API fees.
|
||||
|
||||
## Quick Start
|
||||
|
||||
- [Installation Guide](https://github.com/Panniantong/Agent-Reach/blob/main/docs/install.md): Step-by-step setup instructions for AI agents
|
||||
- [README (中文)](https://github.com/Panniantong/Agent-Reach/blob/main/README.md): Full documentation in Chinese
|
||||
- [README (English)](https://github.com/Panniantong/Agent-Reach/blob/main/docs/README_en.md): Full documentation in English
|
||||
|
||||
## Core Commands
|
||||
|
||||
- [CLI Usage](https://github.com/Panniantong/Agent-Reach/blob/main/docs/install.md): `agent-reach doctor --json` shows the active backend; agents then call upstream tools directly (`opencli`, `twitter`, `bili`, `yt-dlp`, `gh`, `mcporter`, etc.).
|
||||
|
||||
## Key Features
|
||||
|
||||
- Read any URL: tweets, Reddit posts, Facebook pages/feed/groups, Instagram profiles/posts, YouTube videos (transcripts), GitHub repos, articles, XiaoHongShu notes, Bilibili videos, RSS feeds
|
||||
- Search/discover across platforms: Twitter/X, Reddit, Facebook, Instagram user search, GitHub, YouTube, Bilibili, XiaoHongShu, LinkedIn, V2EX, Xueqiu, Web (via Exa)
|
||||
- Self-diagnosis: `agent-reach doctor` checks what works and what needs setup
|
||||
- Auto-installs dependencies: `agent-reach install --env=auto`
|
||||
- Browser-session / cookie auth for platforms that require login (Twitter, Reddit, Facebook, Instagram, XiaoHongShu, Xueqiu)
|
||||
- Proxy support for platforms that block server IPs (Reddit, Twitter)
|
||||
- Zero API fees: all backends are free and open-source (OpenCLI, twitter-cli, rdt-cli, bili-cli, yt-dlp, Jina Reader, mcporter/Exa, etc.)
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- [Troubleshooting Guide](https://github.com/Panniantong/Agent-Reach/blob/main/docs/troubleshooting.md): Common issues and solutions
|
||||
|
||||
## Optional
|
||||
|
||||
- [SKILL.md](https://github.com/Panniantong/Agent-Reach/blob/main/agent_reach/skill/SKILL.md): Integration guide for AI agent frameworks (OpenClaw, Claude Code, etc.)
|
||||
- [PyPI Package](https://pypi.org/project/agent-reach/): `pip install agent-reach`
|
||||
@@ -0,0 +1,82 @@
|
||||
[project]
|
||||
name = "agent-reach"
|
||||
version = "1.5.0"
|
||||
description = "Give your AI Agent eyes to see the entire internet. Search + Read 10+ platforms."
|
||||
readme = "README.md"
|
||||
license = {text = "MIT"}
|
||||
requires-python = ">=3.10"
|
||||
authors = [{name = "Neo Reid"}]
|
||||
keywords = [
|
||||
"ai-agent", "llm-tools", "agent-infrastructure", "mcp",
|
||||
"web-reader", "web-scraper", "search",
|
||||
"twitter-scraper", "reddit-scraper", "youtube-transcript",
|
||||
"bilibili", "xiaohongshu",
|
||||
"ai-search", "cli", "automation",
|
||||
"claude-code", "cursor", "openai",
|
||||
"free-api", "no-api-key",
|
||||
]
|
||||
classifiers = [
|
||||
"Development Status :: 4 - Beta",
|
||||
"Intended Audience :: Developers",
|
||||
"License :: OSI Approved :: MIT License",
|
||||
"Programming Language :: Python :: 3",
|
||||
"Programming Language :: Python :: 3.10",
|
||||
"Programming Language :: Python :: 3.11",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
"Topic :: Internet :: WWW/HTTP",
|
||||
"Topic :: Scientific/Engineering :: Artificial Intelligence",
|
||||
]
|
||||
|
||||
dependencies = [
|
||||
"requests>=2.28",
|
||||
"feedparser>=6.0",
|
||||
"python-dotenv>=1.0",
|
||||
"loguru>=0.7",
|
||||
"pyyaml>=6.0",
|
||||
"rich>=13.0",
|
||||
"yt-dlp>=2024.0",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
browser = ["playwright>=1.40"]
|
||||
cookies = ["browser-cookie3>=0.19"]
|
||||
all = ["playwright>=1.40", "mcp[cli]>=1.0", "browser-cookie3>=0.19"]
|
||||
dev = [
|
||||
"pytest>=8.0",
|
||||
"ruff>=0.8",
|
||||
"mypy>=1.12",
|
||||
"types-requests>=2.32",
|
||||
"types-PyYAML>=6.0",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
agent-reach = "agent_reach.cli:main"
|
||||
|
||||
[project.urls]
|
||||
Homepage = "https://github.com/Panniantong/agent-reach"
|
||||
Repository = "https://github.com/Panniantong/agent-reach"
|
||||
Issues = "https://github.com/Panniantong/agent-reach/issues"
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[tool.hatch.build.targets.wheel]
|
||||
packages = ["agent_reach"]
|
||||
|
||||
[tool.ruff]
|
||||
target-version = "py310"
|
||||
line-length = 100
|
||||
|
||||
[tool.ruff.lint]
|
||||
select = ["E", "F", "I"]
|
||||
ignore = ["E501"]
|
||||
|
||||
[tool.mypy]
|
||||
python_version = "3.10"
|
||||
warn_unused_configs = true
|
||||
warn_redundant_casts = true
|
||||
warn_unused_ignores = true
|
||||
check_untyped_defs = true
|
||||
ignore_missing_imports = true
|
||||
exclude = ["^tests/"]
|
||||
Executable
+72
@@ -0,0 +1,72 @@
|
||||
#!/bin/bash
|
||||
# sync-upstream.sh — Sync channel implementations from upstream tools
|
||||
#
|
||||
# Usage: ./scripts/sync-upstream.sh
|
||||
#
|
||||
# This script checks for updates in x-reader's fetchers/ directory
|
||||
# and shows which files have changed. You can then manually review
|
||||
# and merge the changes.
|
||||
|
||||
set -e
|
||||
|
||||
UPSTREAM_REPO="runesleo/x-reader"
|
||||
UPSTREAM_BRANCH="main"
|
||||
UPSTREAM_DIR="x_reader/fetchers"
|
||||
LOCAL_DIR="agent_reach/channels"
|
||||
|
||||
echo "👁️ Agent Reach — Upstream Sync"
|
||||
echo "Checking for updates from $UPSTREAM_REPO..."
|
||||
echo ""
|
||||
|
||||
# Create temp dir for upstream code
|
||||
TMPDIR=$(mktemp -d)
|
||||
trap "rm -rf $TMPDIR" EXIT
|
||||
|
||||
# Clone upstream (shallow)
|
||||
git clone --depth 1 --branch "$UPSTREAM_BRANCH" \
|
||||
"https://github.com/$UPSTREAM_REPO.git" "$TMPDIR/upstream" 2>/dev/null
|
||||
|
||||
if [ ! -d "$TMPDIR/upstream/$UPSTREAM_DIR" ]; then
|
||||
echo "❌ Upstream directory not found: $UPSTREAM_DIR"
|
||||
echo " x-reader may have changed their structure."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Compare each file
|
||||
echo "Comparing files..."
|
||||
echo ""
|
||||
|
||||
CHANGES=0
|
||||
for upstream_file in "$TMPDIR/upstream/$UPSTREAM_DIR"/*.py; do
|
||||
filename=$(basename "$upstream_file")
|
||||
local_file="$LOCAL_DIR/$filename"
|
||||
|
||||
if [ ! -f "$local_file" ]; then
|
||||
echo "🆕 NEW: $filename (exists in upstream but not locally)"
|
||||
CHANGES=$((CHANGES + 1))
|
||||
continue
|
||||
fi
|
||||
|
||||
# Compare (ignoring import path differences)
|
||||
if ! diff -q <(sed 's/x_reader\.fetchers/agent_reach.channels/g' "$upstream_file") "$local_file" > /dev/null 2>&1; then
|
||||
echo "📝 CHANGED: $filename"
|
||||
diff --color -u <(sed 's/x_reader\.fetchers/agent_reach.channels/g' "$upstream_file") "$local_file" | head -20
|
||||
echo " ..."
|
||||
echo ""
|
||||
CHANGES=$((CHANGES + 1))
|
||||
fi
|
||||
done
|
||||
|
||||
if [ $CHANGES -eq 0 ]; then
|
||||
echo "✅ All channels are up to date with upstream!"
|
||||
else
|
||||
echo ""
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
echo "$CHANGES file(s) have upstream changes."
|
||||
echo ""
|
||||
echo "To merge a specific file:"
|
||||
echo " cp $TMPDIR/upstream/$UPSTREAM_DIR/FILENAME.py $LOCAL_DIR/FILENAME.py"
|
||||
echo " sed -i 's/x_reader\\.fetchers/agent_reach.channels/g' $LOCAL_DIR/FILENAME.py"
|
||||
echo ""
|
||||
echo "Then review changes, run tests, and commit."
|
||||
fi
|
||||
@@ -0,0 +1,92 @@
|
||||
#!/bin/bash
|
||||
# Agent Reach 一键完整测试
|
||||
# 用法: bash test-agent-reach.sh
|
||||
# 在任何有 Python 3.10+ 的机器上跑就行
|
||||
|
||||
set -e
|
||||
|
||||
echo "╔════════════════════════════════════════════╗"
|
||||
echo "║ 👁️ Agent Reach 完整测试 ║"
|
||||
echo "╚════════════════════════════════════════════╝"
|
||||
echo ""
|
||||
|
||||
# ── 1. 准备干净环境 ──
|
||||
echo "📦 创建测试环境..."
|
||||
TEST_DIR=$(mktemp -d)
|
||||
python3 -m venv "$TEST_DIR/venv"
|
||||
source "$TEST_DIR/venv/bin/activate"
|
||||
|
||||
# ── 2. 安装 ──
|
||||
echo "📥 从 GitHub 安装..."
|
||||
pip install -q https://github.com/Panniantong/agent-reach/archive/main.zip 2>&1 | tail -1
|
||||
echo ""
|
||||
|
||||
# ── 3. 自动配置 ──
|
||||
echo "⚙️ 运行 install..."
|
||||
agent-reach install --env=auto 2>&1
|
||||
echo ""
|
||||
|
||||
# ── 4. 诊断 ──
|
||||
echo "🩺 运行 doctor..."
|
||||
agent-reach doctor 2>&1
|
||||
echo ""
|
||||
|
||||
# ── 5. 逐个测试 ──
|
||||
PASS=0
|
||||
FAIL=0
|
||||
SKIP=0
|
||||
|
||||
test_it() {
|
||||
local name="$1"
|
||||
shift
|
||||
echo -n " $name ... "
|
||||
output=$(eval "$@" 2>&1) || true
|
||||
if echo "$output" | grep -q "📖\|🔗\|http"; then
|
||||
echo "✅"
|
||||
PASS=$((PASS+1))
|
||||
elif echo "$output" | grep -q "⚠️\|not installed\|not configured"; then
|
||||
echo "⏭️ (跳过 — 缺依赖)"
|
||||
SKIP=$((SKIP+1))
|
||||
else
|
||||
echo "❌"
|
||||
echo " $(echo "$output" | head -2)"
|
||||
FAIL=$((FAIL+1))
|
||||
fi
|
||||
}
|
||||
|
||||
echo "📖 阅读测试"
|
||||
test_it "网页" "agent-reach read 'https://example.com'"
|
||||
test_it "GitHub" "agent-reach read 'https://github.com/Panniantong/agent-reach'"
|
||||
test_it "YouTube" "agent-reach read 'https://www.youtube.com/watch?v=dQw4w9WgXcQ'"
|
||||
test_it "B站" "agent-reach read 'https://www.bilibili.com/video/BV1d4411N7zD'"
|
||||
test_it "RSS" "agent-reach read 'https://hnrss.org/frontpage'"
|
||||
test_it "Twitter" "agent-reach read 'https://x.com/elonmusk/status/1893797839927353448'"
|
||||
test_it "Reddit" "agent-reach read 'https://www.reddit.com/r/LocalLLaMA/hot'"
|
||||
|
||||
echo ""
|
||||
echo "🔍 搜索测试"
|
||||
test_it "全网搜索" "agent-reach search 'best AI agent framework' -n 2"
|
||||
test_it "GitHub搜索" "agent-reach search-github 'yt-dlp' -n 2"
|
||||
test_it "Twitter搜索" "agent-reach search-twitter 'AI agent' -n 2"
|
||||
test_it "Reddit搜索" "agent-reach search-reddit 'machine learning' -n 2"
|
||||
test_it "YouTube搜索" "agent-reach search-youtube 'AI tutorial' -n 2"
|
||||
test_it "B站搜索" "agent-reach search-bilibili 'AI' -n 2"
|
||||
test_it "小红书搜索" "agent-reach search-xhs 'AI' -n 2"
|
||||
|
||||
echo ""
|
||||
echo "════════════════════════════════════════════"
|
||||
echo " ✅ 通过: $PASS ❌ 失败: $FAIL ⏭️ 跳过: $SKIP"
|
||||
echo "════════════════════════════════════════════"
|
||||
|
||||
# ── 6. 清理 ──
|
||||
deactivate 2>/dev/null || true
|
||||
rm -rf "$TEST_DIR"
|
||||
|
||||
if [ $FAIL -eq 0 ]; then
|
||||
echo ""
|
||||
echo "🎉 全部通过！"
|
||||
else
|
||||
echo ""
|
||||
echo "⚠️ 有 $FAIL 个测试失败，请检查上面的输出"
|
||||
exit 1
|
||||
fi
|
||||
@@ -0,0 +1,185 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Contract tests for channel adapters."""
|
||||
|
||||
import subprocess
|
||||
|
||||
from agent_reach.channels import get_all_channels
|
||||
from agent_reach.config import Config
|
||||
|
||||
|
||||
def _fake_run_ok(cmd, **kwargs):
|
||||
"""Pretend any probed CLI executes fine and prints a version."""
|
||||
return subprocess.CompletedProcess(cmd, 0, "2026.06.09", "")
|
||||
|
||||
|
||||
def test_channel_registry_contract():
|
||||
channels = get_all_channels()
|
||||
assert channels, "channel registry must not be empty"
|
||||
names = [ch.name for ch in channels]
|
||||
assert len(names) == len(set(names)), "channel names must be unique"
|
||||
|
||||
for ch in channels:
|
||||
assert isinstance(ch.name, str) and ch.name
|
||||
assert isinstance(ch.description, str) and ch.description
|
||||
assert isinstance(ch.backends, list)
|
||||
assert ch.tier in {0, 1, 2}
|
||||
|
||||
|
||||
def test_channel_check_contract_with_minimal_runtime(monkeypatch, tmp_path):
|
||||
# Keep contract tests deterministic by simulating "deps mostly absent".
|
||||
monkeypatch.setattr("shutil.which", lambda _cmd: None)
|
||||
config = Config(config_path=tmp_path / "config.yaml")
|
||||
|
||||
for ch in get_all_channels():
|
||||
status, message = ch.check(config)
|
||||
assert status in {"ok", "warn", "off", "error"}
|
||||
assert isinstance(message, str) and message.strip()
|
||||
|
||||
|
||||
def test_channel_active_backend_attribute_contract():
|
||||
"""Every channel exposes active_backend (default None, or str once set)."""
|
||||
for ch in get_all_channels():
|
||||
assert hasattr(ch, "active_backend")
|
||||
# Fresh instances must default to None / str (class attribute on base)
|
||||
fresh = type(ch)()
|
||||
assert fresh.active_backend is None or isinstance(fresh.active_backend, str)
|
||||
|
||||
|
||||
def test_channel_active_backend_set_by_check(monkeypatch, tmp_path):
|
||||
"""After check(), active_backend is None or a str — never anything else."""
|
||||
monkeypatch.setattr("shutil.which", lambda _cmd: None)
|
||||
|
||||
# Keep the network-based channels (V2EX/Xueqiu/Bilibili API) deterministic.
|
||||
import urllib.request
|
||||
from urllib.error import URLError
|
||||
|
||||
def _no_net(*_a, **_k):
|
||||
raise URLError("offline")
|
||||
|
||||
monkeypatch.setattr(urllib.request, "urlopen", _no_net)
|
||||
import agent_reach.channels.xueqiu as xueqiu_mod
|
||||
monkeypatch.setattr(xueqiu_mod, "_cookies_initialized", True)
|
||||
monkeypatch.setattr(xueqiu_mod._opener, "open", _no_net)
|
||||
|
||||
config = Config(config_path=tmp_path / "config.yaml")
|
||||
for ch in get_all_channels():
|
||||
ch.check(config)
|
||||
assert ch.active_backend is None or isinstance(ch.active_backend, str), (
|
||||
f"{ch.name}: active_backend must be None or str after check()"
|
||||
)
|
||||
|
||||
|
||||
def test_ordered_backends_contract(tmp_path):
|
||||
"""ordered_backends(config) is a reordering (same multiset) of backends."""
|
||||
config = Config(config_path=tmp_path / "config.yaml")
|
||||
for ch in get_all_channels():
|
||||
ordered = ch.ordered_backends(config)
|
||||
assert isinstance(ordered, list)
|
||||
assert sorted(ordered) == sorted(ch.backends), (
|
||||
f"{ch.name}: ordered_backends must be a permutation of backends"
|
||||
)
|
||||
# And without any config at all
|
||||
ordered_none = ch.ordered_backends(None)
|
||||
assert sorted(ordered_none) == sorted(ch.backends)
|
||||
|
||||
|
||||
def test_ordered_backends_override_moves_backend_to_front():
|
||||
"""Config key <channel>_backend promotes the named backend to front."""
|
||||
from agent_reach.channels.twitter import TwitterChannel
|
||||
|
||||
ch = TwitterChannel()
|
||||
ordered = ch.ordered_backends({"twitter_backend": "bird"})
|
||||
assert ordered[0] == "bird CLI (legacy)"
|
||||
assert sorted(ordered) == sorted(ch.backends)
|
||||
|
||||
# Unknown override is ignored — never hides working backends
|
||||
ordered_unknown = ch.ordered_backends({"twitter_backend": "no-such-tool"})
|
||||
assert ordered_unknown == list(ch.backends)
|
||||
|
||||
|
||||
def test_youtube_warns_when_node_only_and_no_config(monkeypatch, tmp_path):
|
||||
"""YouTube should warn when only Node.js is installed but no yt-dlp config exists."""
|
||||
from agent_reach.channels.youtube import YouTubeChannel
|
||||
|
||||
def fake_which(cmd):
|
||||
if cmd == "yt-dlp":
|
||||
return "/usr/bin/yt-dlp"
|
||||
if cmd == "node":
|
||||
return "/usr/bin/node"
|
||||
return None # deno not installed
|
||||
|
||||
monkeypatch.setattr("shutil.which", fake_which)
|
||||
monkeypatch.setattr("subprocess.run", _fake_run_ok) # yt-dlp probe really executes now
|
||||
# Point to a non-existent config file
|
||||
monkeypatch.setattr("os.path.expanduser", lambda p: str(tmp_path / ".config/yt-dlp/config"))
|
||||
|
||||
ch = YouTubeChannel()
|
||||
status, message = ch.check()
|
||||
assert status == "warn"
|
||||
assert "--js-runtimes" in message
|
||||
assert ch.active_backend == "yt-dlp" # 本体活着，warn 只关乎 JS runtime
|
||||
|
||||
|
||||
def test_youtube_warns_with_windows_specific_fix_command(monkeypatch, tmp_path):
|
||||
"""Windows guidance should use a PowerShell-style yt-dlp config command."""
|
||||
from agent_reach.channels.youtube import YouTubeChannel
|
||||
|
||||
def fake_which(cmd):
|
||||
if cmd == "yt-dlp":
|
||||
return "C:/yt-dlp.exe"
|
||||
if cmd == "node":
|
||||
return "C:/node.exe"
|
||||
return None
|
||||
|
||||
monkeypatch.setattr("shutil.which", fake_which)
|
||||
monkeypatch.setattr("subprocess.run", _fake_run_ok) # yt-dlp probe really executes now
|
||||
monkeypatch.setattr("agent_reach.utils.paths.sys.platform", "win32")
|
||||
monkeypatch.setenv("APPDATA", str(tmp_path / "AppData" / "Roaming"))
|
||||
|
||||
ch = YouTubeChannel()
|
||||
status, message = ch.check()
|
||||
assert status == "warn"
|
||||
assert "Select-String" in message
|
||||
assert "--js-runtimes node" in message
|
||||
|
||||
|
||||
def test_youtube_ok_when_deno_installed(monkeypatch):
|
||||
"""YouTube should return ok when Deno is installed (no config needed)."""
|
||||
from agent_reach.channels.youtube import YouTubeChannel
|
||||
|
||||
def fake_which(cmd):
|
||||
if cmd == "yt-dlp":
|
||||
return "/usr/bin/yt-dlp"
|
||||
if cmd == "deno":
|
||||
return "/usr/bin/deno"
|
||||
return None
|
||||
|
||||
monkeypatch.setattr("shutil.which", fake_which)
|
||||
monkeypatch.setattr("subprocess.run", _fake_run_ok) # yt-dlp probe really executes now
|
||||
|
||||
ch = YouTubeChannel()
|
||||
status, _msg = ch.check()
|
||||
assert status == "ok"
|
||||
assert ch.active_backend == "yt-dlp"
|
||||
|
||||
|
||||
def test_channel_can_handle_contract():
|
||||
url_samples = {
|
||||
"github": "https://github.com/panniantong/agent-reach",
|
||||
"twitter": "https://x.com/user/status/1",
|
||||
"youtube": "https://youtube.com/watch?v=abc",
|
||||
"reddit": "https://reddit.com/r/python",
|
||||
"facebook": "https://www.facebook.com/zuck",
|
||||
"instagram": "https://www.instagram.com/openai/",
|
||||
"bilibili": "https://www.bilibili.com/video/BV1xx411",
|
||||
"xiaohongshu": "https://www.xiaohongshu.com/explore/123",
|
||||
"linkedin": "https://www.linkedin.com/in/test",
|
||||
"rss": "https://example.com/feed.xml",
|
||||
"xueqiu": "https://xueqiu.com/S/SH600519",
|
||||
"exa_search": "https://example.com",
|
||||
"web": "https://example.com",
|
||||
}
|
||||
for ch in get_all_channels():
|
||||
sample = url_samples.get(ch.name, "https://example.com")
|
||||
result = ch.can_handle(sample)
|
||||
assert isinstance(result, bool)
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,306 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Tests for Agent Reach CLI."""
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
from argparse import Namespace
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
import requests
|
||||
|
||||
import agent_reach.cli as cli
|
||||
from agent_reach.cli import main
|
||||
from agent_reach.config import Config
|
||||
|
||||
|
||||
class TestCLI:
|
||||
def test_version(self, capsys):
|
||||
with pytest.raises(SystemExit) as exc_info:
|
||||
with patch("sys.argv", ["agent-reach", "version"]):
|
||||
main()
|
||||
assert exc_info.value.code == 0
|
||||
captured = capsys.readouterr()
|
||||
assert "Agent Reach v" in captured.out
|
||||
|
||||
def test_no_command_shows_help(self, capsys):
|
||||
with pytest.raises(SystemExit) as exc_info:
|
||||
with patch("sys.argv", ["agent-reach"]):
|
||||
main()
|
||||
assert exc_info.value.code == 0
|
||||
|
||||
def test_doctor_runs(self, capsys):
|
||||
with patch("sys.argv", ["agent-reach", "doctor"]):
|
||||
main()
|
||||
captured = capsys.readouterr()
|
||||
assert "Agent Reach" in captured.out
|
||||
assert "✅" in captured.out
|
||||
|
||||
def test_doctor_preserves_existing_skill_install(self, monkeypatch, tmp_path, capsys):
|
||||
skill_dir = tmp_path / ".agents" / "skills" / "agent-reach"
|
||||
skill_dir.mkdir(parents=True)
|
||||
skill_file = skill_dir / "SKILL.md"
|
||||
custom_content = "# custom Agent Reach skill\n"
|
||||
skill_file.write_text(custom_content, encoding="utf-8")
|
||||
|
||||
monkeypatch.setattr(
|
||||
cli.os.path,
|
||||
"expanduser",
|
||||
lambda p: p.replace("~", str(tmp_path)),
|
||||
)
|
||||
config_dir = tmp_path / ".agent-reach"
|
||||
monkeypatch.setattr(Config, "CONFIG_DIR", config_dir)
|
||||
monkeypatch.setattr(Config, "CONFIG_FILE", config_dir / "config.yaml")
|
||||
monkeypatch.setattr("agent_reach.doctor.check_all", lambda config: {})
|
||||
monkeypatch.setattr("agent_reach.doctor.format_report", lambda results: "report")
|
||||
|
||||
cli._cmd_doctor(Namespace(json=False))
|
||||
|
||||
assert skill_file.read_text(encoding="utf-8") == custom_content
|
||||
out = capsys.readouterr().out
|
||||
assert "preserving existing files" in out
|
||||
assert f"Skill installed for Agent: {skill_dir}" not in out
|
||||
|
||||
def test_transcribe_command_prints_text(self, capsys):
|
||||
with patch("agent_reach.transcribe.transcribe", return_value="hello transcript"):
|
||||
with patch("sys.argv", ["agent-reach", "transcribe", "audio.mp3"]):
|
||||
main()
|
||||
captured = capsys.readouterr()
|
||||
assert "hello transcript" in captured.out
|
||||
|
||||
def test_transcribe_command_writes_output_file(self, capsys, tmp_path):
|
||||
out_file = tmp_path / "t.txt"
|
||||
with patch("agent_reach.transcribe.transcribe", return_value="saved text"):
|
||||
with patch("sys.argv", ["agent-reach", "transcribe", "audio.mp3", "-o", str(out_file)]):
|
||||
main()
|
||||
assert out_file.read_text(encoding="utf-8").strip() == "saved text"
|
||||
assert "Transcript written" in capsys.readouterr().out
|
||||
|
||||
def test_parse_twitter_cookie_input_separate_values(self):
|
||||
auth_token, ct0 = cli._parse_twitter_cookie_input("token123 ct0abc")
|
||||
assert auth_token == "token123"
|
||||
assert ct0 == "ct0abc"
|
||||
|
||||
def test_parse_twitter_cookie_input_cookie_header(self):
|
||||
auth_token, ct0 = cli._parse_twitter_cookie_input(
|
||||
"auth_token=token123; ct0=ct0abc; other=value"
|
||||
)
|
||||
assert auth_token == "token123"
|
||||
assert ct0 == "ct0abc"
|
||||
|
||||
def test_install_rdt_cli_prefers_github_source(self, monkeypatch, capsys):
|
||||
state = {"rdt_installed": False}
|
||||
commands = []
|
||||
|
||||
def fake_which(name):
|
||||
if name == "rdt":
|
||||
return "/usr/local/bin/rdt" if state["rdt_installed"] else None
|
||||
if name == "pipx":
|
||||
return "/usr/local/bin/pipx"
|
||||
return None
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
commands.append(cmd)
|
||||
state["rdt_installed"] = True
|
||||
return subprocess.CompletedProcess(cmd, 0, "", "")
|
||||
|
||||
monkeypatch.setattr(shutil, "which", fake_which)
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
|
||||
cli._install_rdt_cli()
|
||||
|
||||
out = capsys.readouterr().out
|
||||
assert commands == [["pipx", "install", cli._RDT_GIT_SOURCE]]
|
||||
assert "✅ rdt-cli installed" in out
|
||||
|
||||
def test_install_reddit_deps_routes_by_environment(self, monkeypatch):
|
||||
"""桌面 → OpenCLI;服务器 → rdt-cli(钉 git 源)。"""
|
||||
calls = []
|
||||
monkeypatch.setattr(cli, "_install_opencli_deps", lambda: calls.append("opencli"))
|
||||
monkeypatch.setattr(cli, "_install_rdt_cli", lambda: calls.append("rdt"))
|
||||
monkeypatch.setattr(shutil, "which", lambda _: None)
|
||||
|
||||
monkeypatch.setattr(cli, "_detect_environment", lambda: "local")
|
||||
cli._install_reddit_deps()
|
||||
assert calls == ["opencli"]
|
||||
|
||||
calls.clear()
|
||||
monkeypatch.setattr(cli, "_detect_environment", lambda: "server")
|
||||
cli._install_reddit_deps()
|
||||
assert calls == ["rdt"]
|
||||
|
||||
def test_install_facebook_instagram_routes_to_opencli_once(self, monkeypatch, capsys):
|
||||
calls = []
|
||||
|
||||
monkeypatch.setattr(cli, "_detect_environment", lambda: "local")
|
||||
monkeypatch.setattr(cli, "_install_system_deps", lambda: None)
|
||||
monkeypatch.setattr(cli, "_install_mcporter", lambda: None)
|
||||
monkeypatch.setattr(cli, "_install_opencli_deps", lambda: calls.append("opencli"))
|
||||
monkeypatch.setattr(cli, "_install_skill", lambda: None)
|
||||
monkeypatch.setattr(
|
||||
"agent_reach.doctor.check_all",
|
||||
lambda config: {
|
||||
"facebook": {
|
||||
"status": "ok",
|
||||
"name": "Facebook",
|
||||
"message": "ok",
|
||||
"tier": 1,
|
||||
"backends": ["OpenCLI"],
|
||||
"active_backend": "OpenCLI",
|
||||
}
|
||||
},
|
||||
)
|
||||
monkeypatch.setattr("agent_reach.doctor.format_report", lambda results: "report")
|
||||
|
||||
cli._cmd_install(
|
||||
Namespace(
|
||||
env="auto",
|
||||
proxy="",
|
||||
safe=False,
|
||||
dry_run=False,
|
||||
channels="facebook,instagram,opencli",
|
||||
)
|
||||
)
|
||||
|
||||
assert calls == ["opencli"]
|
||||
assert "Installation complete" in capsys.readouterr().out
|
||||
|
||||
def test_install_server_dry_run_skips_opencli_only_channels(self, monkeypatch, capsys):
|
||||
monkeypatch.setattr(cli, "_install_system_deps_dryrun", lambda: None)
|
||||
|
||||
cli._cmd_install(
|
||||
Namespace(
|
||||
env="server",
|
||||
proxy="",
|
||||
safe=False,
|
||||
dry_run=True,
|
||||
channels="facebook,instagram,opencli,bilibili",
|
||||
)
|
||||
)
|
||||
|
||||
out = capsys.readouterr().out
|
||||
assert "服务器环境跳过：facebook, instagram, opencli" in out
|
||||
assert "[dry-run] Would install optional channels: bilibili" in out
|
||||
assert "facebook, instagram, opencli, bilibili" not in out
|
||||
|
||||
|
||||
class TestCheckUpdateRetry:
|
||||
def test_retry_timeout_classification(self):
|
||||
sleeps = []
|
||||
|
||||
def fake_sleep(seconds):
|
||||
sleeps.append(seconds)
|
||||
|
||||
with patch("requests.get", side_effect=requests.exceptions.Timeout("timed out")):
|
||||
resp, err, attempts = cli._github_get_with_retry(
|
||||
"https://api.github.com/test",
|
||||
timeout=1,
|
||||
retries=3,
|
||||
sleeper=fake_sleep,
|
||||
)
|
||||
|
||||
assert resp is None
|
||||
assert err == "timeout"
|
||||
assert attempts == 3
|
||||
assert sleeps == [1, 2]
|
||||
|
||||
def test_retry_dns_classification(self):
|
||||
error = requests.exceptions.ConnectionError("getaddrinfo failed for api.github.com")
|
||||
with patch("requests.get", side_effect=error):
|
||||
resp, err, attempts = cli._github_get_with_retry(
|
||||
"https://api.github.com/test",
|
||||
retries=1,
|
||||
sleeper=lambda _x: None,
|
||||
)
|
||||
assert resp is None
|
||||
assert err == "dns"
|
||||
assert attempts == 1
|
||||
|
||||
def test_retry_rate_limit_then_success(self):
|
||||
sleeps = []
|
||||
|
||||
class R:
|
||||
def __init__(self, code, payload=None, headers=None):
|
||||
self.status_code = code
|
||||
self._payload = payload or {}
|
||||
self.headers = headers or {}
|
||||
|
||||
def json(self):
|
||||
return self._payload
|
||||
|
||||
sequence = [
|
||||
R(429, headers={"Retry-After": "3"}),
|
||||
R(200, payload={"tag_name": "v1.5.0"}),
|
||||
]
|
||||
|
||||
with patch("requests.get", side_effect=sequence):
|
||||
resp, err, attempts = cli._github_get_with_retry(
|
||||
"https://api.github.com/test",
|
||||
retries=3,
|
||||
sleeper=lambda s: sleeps.append(s),
|
||||
)
|
||||
|
||||
assert err is None
|
||||
assert resp is not None
|
||||
assert resp.status_code == 200
|
||||
assert attempts == 2
|
||||
assert sleeps == [3.0]
|
||||
|
||||
def test_classify_rate_limit_from_403(self):
|
||||
class R:
|
||||
status_code = 403
|
||||
headers = {"X-RateLimit-Remaining": "0"}
|
||||
|
||||
@staticmethod
|
||||
def json():
|
||||
return {"message": "API rate limit exceeded"}
|
||||
|
||||
assert cli._classify_github_response_error(R()) == "rate_limit"
|
||||
|
||||
def test_check_update_reports_classified_error(self, capsys):
|
||||
with patch("agent_reach.cli._github_get_with_retry", return_value=(None, "timeout", 3)):
|
||||
result = cli._cmd_check_update()
|
||||
|
||||
captured = capsys.readouterr()
|
||||
assert result == "error"
|
||||
assert "网络超时" in captured.out
|
||||
assert "已重试 3 次" in captured.out
|
||||
|
||||
|
||||
class TestVersionCompare:
|
||||
def test_newer_remote_triggers_update(self):
|
||||
assert cli._is_newer_version("1.5.0", "1.4.2") is True
|
||||
|
||||
def test_equal_versions_no_update(self):
|
||||
assert cli._is_newer_version("1.5.0", "1.5.0") is False
|
||||
|
||||
def test_local_ahead_of_release_no_downgrade_prompt(self):
|
||||
"""发版窗口期本地装了 main(更新)时,不能提示"有更新"诱导降级。"""
|
||||
assert cli._is_newer_version("1.4.2", "1.5.0") is False
|
||||
|
||||
def test_unparseable_falls_back_to_inequality(self):
|
||||
assert cli._is_newer_version("2026.06-beta", "1.5.0") is True
|
||||
assert cli._is_newer_version("1.5.0", "1.5.0-dev") is True
|
||||
|
||||
|
||||
class TestWatchVersionCompare:
|
||||
def test_watch_does_not_prompt_downgrade(self, monkeypatch, capsys):
|
||||
"""watch 与 check-update 同语义:本地领先远端 release 时不提示更新。"""
|
||||
class R:
|
||||
status_code = 200
|
||||
headers = {}
|
||||
|
||||
@staticmethod
|
||||
def json():
|
||||
return {"tag_name": "v1.4.2", "body": ""}
|
||||
|
||||
monkeypatch.setattr(cli, "_github_get_with_retry", lambda *a, **k: (R(), None, 1))
|
||||
monkeypatch.setattr(
|
||||
"agent_reach.doctor.check_all",
|
||||
lambda config: {"web": {"status": "ok", "name": "任意网页", "message": "ok",
|
||||
"tier": 0, "backends": ["Jina Reader"], "active_backend": "Jina Reader"}},
|
||||
)
|
||||
cli._cmd_watch()
|
||||
out = capsys.readouterr().out
|
||||
assert "新版本可用" not in out
|
||||
assert "全部正常" in out
|
||||
@@ -0,0 +1,135 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Tests for Agent Reach config module."""
|
||||
|
||||
import pytest
|
||||
|
||||
from agent_reach.config import Config
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def tmp_config(tmp_path):
|
||||
"""Create a Config with a temporary directory."""
|
||||
config_file = tmp_path / "config.yaml"
|
||||
return Config(config_path=config_file)
|
||||
|
||||
|
||||
class TestConfig:
|
||||
def test_init_creates_dir(self, tmp_path):
|
||||
config_file = tmp_path / "subdir" / "config.yaml"
|
||||
Config(config_path=config_file)
|
||||
assert config_file.parent.exists()
|
||||
|
||||
def test_set_and_get(self, tmp_config):
|
||||
tmp_config.set("test_key", "test_value")
|
||||
assert tmp_config.get("test_key") == "test_value"
|
||||
|
||||
def test_get_default(self, tmp_config):
|
||||
assert tmp_config.get("nonexistent") is None
|
||||
assert tmp_config.get("nonexistent", "default") == "default"
|
||||
|
||||
def test_get_from_env(self, tmp_config, monkeypatch):
|
||||
monkeypatch.setenv("TEST_ENV_KEY", "env_value")
|
||||
assert tmp_config.get("test_env_key") == "env_value"
|
||||
|
||||
def test_config_file_priority_over_env(self, tmp_config, monkeypatch):
|
||||
monkeypatch.setenv("MY_KEY", "from_env")
|
||||
tmp_config.set("my_key", "from_config")
|
||||
assert tmp_config.get("my_key") == "from_config"
|
||||
|
||||
def test_save_and_load(self, tmp_config):
|
||||
tmp_config.set("key1", "value1")
|
||||
tmp_config.set("key2", 42)
|
||||
|
||||
# Create new config from same file
|
||||
config2 = Config(config_path=tmp_config.config_path)
|
||||
assert config2.get("key1") == "value1"
|
||||
assert config2.get("key2") == 42
|
||||
|
||||
def test_delete(self, tmp_config):
|
||||
tmp_config.set("to_delete", "value")
|
||||
assert tmp_config.get("to_delete") == "value"
|
||||
tmp_config.delete("to_delete")
|
||||
assert tmp_config.get("to_delete") is None
|
||||
|
||||
def test_is_configured(self, tmp_config):
|
||||
assert not tmp_config.is_configured("exa_search")
|
||||
tmp_config.set("exa_api_key", "test-key")
|
||||
assert tmp_config.is_configured("exa_search")
|
||||
|
||||
def test_get_configured_features(self, tmp_config):
|
||||
features = tmp_config.get_configured_features()
|
||||
assert isinstance(features, dict)
|
||||
assert "exa_search" in features
|
||||
assert all(v is False for v in features.values())
|
||||
|
||||
def test_to_dict_masks_sensitive(self, tmp_config):
|
||||
tmp_config.set("exa_api_key", "super-secret-key-12345")
|
||||
tmp_config.set("normal_setting", "visible")
|
||||
masked = tmp_config.to_dict()
|
||||
assert masked["exa_api_key"] == "super-se..."
|
||||
assert masked["normal_setting"] == "visible"
|
||||
|
||||
def test_to_dict_masks_cookie_and_session_credentials(self, tmp_config):
|
||||
secrets = {
|
||||
"twitter_ct0": "csrf-secret-value",
|
||||
"xhs_cookie": "web_session=xhs-secret",
|
||||
"xueqiu_cookie": "xq_a_token=xueqiu-secret",
|
||||
"bilibili_sessdata": "bili-session-secret",
|
||||
"bilibili_csrf": "bili-csrf-secret",
|
||||
"twitter_auth_token": "twitter-auth-secret",
|
||||
}
|
||||
for key, value in secrets.items():
|
||||
tmp_config.set(key, value)
|
||||
|
||||
tmp_config.set("normal_setting", "visible")
|
||||
masked = tmp_config.to_dict()
|
||||
|
||||
dumped = str(masked)
|
||||
for value in secrets.values():
|
||||
assert value not in dumped
|
||||
assert masked["normal_setting"] == "visible"
|
||||
|
||||
def test_save_creates_file_with_restricted_permissions(self, tmp_path):
|
||||
import stat
|
||||
import sys
|
||||
config_file = tmp_path / "secure_config.yaml"
|
||||
config = Config(config_path=config_file)
|
||||
config.set("secret_key", "my-secret")
|
||||
|
||||
if sys.platform != "win32":
|
||||
mode = config_file.stat().st_mode
|
||||
# File should be owner-only read/write (0o600)
|
||||
assert not (mode & stat.S_IRGRP), "group read should not be set"
|
||||
assert not (mode & stat.S_IROTH), "other read should not be set"
|
||||
|
||||
def test_save_tightens_existing_config_file_permissions(self, tmp_path):
|
||||
import os
|
||||
import stat
|
||||
import sys
|
||||
|
||||
config_file = tmp_path / "secure_config.yaml"
|
||||
config_file.write_text("twitter_auth_token: old\n", encoding="utf-8")
|
||||
if sys.platform != "win32":
|
||||
os.chmod(config_file, 0o644)
|
||||
|
||||
config = Config(config_path=config_file)
|
||||
config.set("twitter_auth_token", "new-secret")
|
||||
|
||||
if sys.platform != "win32":
|
||||
mode = config_file.stat().st_mode
|
||||
assert not (mode & stat.S_IRGRP), "group read should be removed"
|
||||
assert not (mode & stat.S_IROTH), "other read should be removed"
|
||||
|
||||
def test_config_dir_has_restricted_permissions(self, tmp_path):
|
||||
import stat
|
||||
import sys
|
||||
|
||||
config_file = tmp_path / "private" / "config.yaml"
|
||||
Config(config_path=config_file)
|
||||
|
||||
if sys.platform != "win32":
|
||||
mode = config_file.parent.stat().st_mode
|
||||
assert not (mode & stat.S_IRGRP), "group read should not be set"
|
||||
assert not (mode & stat.S_IXGRP), "group execute should not be set"
|
||||
assert not (mode & stat.S_IROTH), "other read should not be set"
|
||||
assert not (mode & stat.S_IXOTH), "other execute should not be set"
|
||||
@@ -0,0 +1,151 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Verify credential files written by cookie sync helpers and CLI helpers
|
||||
are owner-only (0o600) and that values containing shell metacharacters do
|
||||
not break the shell-sourceable env file produced by _sync_bird_env().
|
||||
|
||||
Companion to tests/test_config.py::test_save_creates_file_with_restricted_permissions —
|
||||
the same threat-model claim ("Cookie/Token only stored locally, 600
|
||||
permissions") covers these paths.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import stat
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
from agent_reach.cli import _configure_xhs_cookies
|
||||
from agent_reach.cookie_extract import _sync_bird_env, _sync_xfetch_session
|
||||
|
||||
|
||||
def _owner_only(path: str) -> bool:
|
||||
mode = os.stat(path).st_mode
|
||||
return not (mode & (stat.S_IRGRP | stat.S_IWGRP | stat.S_IROTH | stat.S_IWOTH))
|
||||
|
||||
|
||||
def _owner_only_dir(path: str) -> bool:
|
||||
mode = os.stat(path).st_mode
|
||||
return not (
|
||||
mode
|
||||
& (
|
||||
stat.S_IRGRP
|
||||
| stat.S_IWGRP
|
||||
| stat.S_IXGRP
|
||||
| stat.S_IROTH
|
||||
| stat.S_IWOTH
|
||||
| stat.S_IXOTH
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="POSIX perm semantics only")
|
||||
def test_sync_xfetch_session_writes_0600(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("HOME", str(tmp_path))
|
||||
_sync_xfetch_session("auth_xxx", "ct0_yyy")
|
||||
session_path = tmp_path / ".config" / "xfetch" / "session.json"
|
||||
assert session_path.exists(), "expected ~/.config/xfetch/session.json"
|
||||
assert _owner_only_dir(str(session_path.parent)), "xfetch dir must be 0o700"
|
||||
assert _owner_only(str(session_path)), "session.json must be 0o600"
|
||||
# Round-trip the content so we know we didn't accidentally corrupt JSON.
|
||||
data = json.loads(session_path.read_text(encoding="utf-8"))
|
||||
assert data["authToken"] == "auth_xxx"
|
||||
assert data["ct0"] == "ct0_yyy"
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="POSIX perm semantics only")
|
||||
def test_sync_xfetch_session_tightens_existing_file(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("HOME", str(tmp_path))
|
||||
session_path = tmp_path / ".config" / "xfetch" / "session.json"
|
||||
session_path.parent.mkdir(parents=True)
|
||||
session_path.write_text('{"authToken": "old"}', encoding="utf-8")
|
||||
os.chmod(session_path, 0o644)
|
||||
|
||||
_sync_xfetch_session("auth_xxx", "ct0_yyy")
|
||||
|
||||
assert _owner_only(str(session_path)), "existing session.json must be tightened"
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="POSIX perm semantics only")
|
||||
def test_sync_bird_env_writes_0600(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("HOME", str(tmp_path))
|
||||
_sync_bird_env("auth_xxx", "ct0_yyy")
|
||||
env_path = tmp_path / ".config" / "bird" / "credentials.env"
|
||||
assert env_path.exists(), "expected ~/.config/bird/credentials.env"
|
||||
assert _owner_only_dir(str(env_path.parent)), "bird dir must be 0o700"
|
||||
assert _owner_only(str(env_path)), "credentials.env must be 0o600"
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="POSIX perm semantics only")
|
||||
def test_sync_bird_env_tightens_existing_file(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("HOME", str(tmp_path))
|
||||
env_path = tmp_path / ".config" / "bird" / "credentials.env"
|
||||
env_path.parent.mkdir(parents=True)
|
||||
env_path.write_text("AUTH_TOKEN=old\n", encoding="utf-8")
|
||||
os.chmod(env_path, 0o644)
|
||||
|
||||
_sync_bird_env("auth_xxx", "ct0_yyy")
|
||||
|
||||
assert _owner_only(str(env_path)), "existing credentials.env must be tightened"
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="POSIX sh needed for sourcing")
|
||||
def test_sync_bird_env_quotes_shell_metachars(tmp_path, monkeypatch):
|
||||
"""Tokens containing ", $, `, ; etc. must not break out of the assignment.
|
||||
|
||||
Prior implementation used `f'AUTH_TOKEN="{auth_token}"'` which an attacker-
|
||||
controlled cookie containing a literal `"` could break out of, turning a
|
||||
later `source ~/.config/bird/credentials.env` into arbitrary shell.
|
||||
"""
|
||||
monkeypatch.setenv("HOME", str(tmp_path))
|
||||
# Side-effect markers live under tmp_path (auto-cleaned by pytest) rather
|
||||
# than a shared absolute /tmp path — otherwise one vulnerable run leaves a
|
||||
# marker behind that fails every later run on the same machine/CI runner.
|
||||
pwn_auth = tmp_path / "pwn-auth"
|
||||
pwn_ct0 = tmp_path / "pwn-ct0"
|
||||
hostile_auth = f'inj"; touch {pwn_auth}; #'
|
||||
hostile_ct0 = f"ct0_$(touch {pwn_ct0})"
|
||||
_sync_bird_env(hostile_auth, hostile_ct0)
|
||||
env_path = tmp_path / ".config" / "bird" / "credentials.env"
|
||||
|
||||
# Sourcing the file must NOT execute the injected payload. Read back the
|
||||
# exported values from a subshell instead — they should equal the originals.
|
||||
probe = (
|
||||
f". {env_path}; "
|
||||
f'printf "AUTH=%s\\nCT0=%s\\n" "$AUTH_TOKEN" "$CT0"'
|
||||
)
|
||||
result = subprocess.run(
|
||||
["sh", "-c", probe],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=5,
|
||||
)
|
||||
assert result.returncode == 0, result.stderr
|
||||
lines = dict(
|
||||
line.split("=", 1) for line in result.stdout.strip().splitlines() if "=" in line
|
||||
)
|
||||
assert lines["AUTH"] == hostile_auth, "auth_token round-trip broke — injection possible"
|
||||
assert lines["CT0"] == hostile_ct0, "ct0 round-trip broke — injection possible"
|
||||
# And no side-effect files materialised.
|
||||
assert not pwn_auth.exists()
|
||||
assert not pwn_ct0.exists()
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="POSIX perm semantics only")
|
||||
def test_configure_xhs_cookies_tightens_local_fallback_file(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("HOME", str(tmp_path))
|
||||
monkeypatch.setattr("shutil.which", lambda name: None if name == "docker" else name)
|
||||
|
||||
cookie_path = tmp_path / ".agent-reach" / "xhs-cookies.json"
|
||||
cookie_path.parent.mkdir()
|
||||
cookie_path.write_text("[]", encoding="utf-8")
|
||||
os.chmod(cookie_path, 0o644)
|
||||
|
||||
_configure_xhs_cookies("web_session=xhs_secret")
|
||||
|
||||
assert _owner_only_dir(str(cookie_path.parent)), "~/.agent-reach must be 0o700"
|
||||
assert _owner_only(str(cookie_path)), "existing xhs-cookies.json must be tightened"
|
||||
data = json.loads(cookie_path.read_text(encoding="utf-8"))
|
||||
assert data[0]["name"] == "web_session"
|
||||
assert data[0]["value"] == "xhs_secret"
|
||||
@@ -0,0 +1,29 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Tests for AgentReach core class."""
|
||||
|
||||
import pytest
|
||||
|
||||
from agent_reach.config import Config
|
||||
from agent_reach.core import AgentReach
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def eyes(tmp_path):
|
||||
config = Config(config_path=tmp_path / "config.yaml")
|
||||
return AgentReach(config=config)
|
||||
|
||||
|
||||
class TestAgentReach:
|
||||
def test_init(self, eyes):
|
||||
assert eyes.config is not None
|
||||
|
||||
def test_doctor(self, eyes):
|
||||
results = eyes.doctor()
|
||||
assert isinstance(results, dict)
|
||||
assert "web" in results
|
||||
assert "github" in results
|
||||
|
||||
def test_doctor_report(self, eyes):
|
||||
report = eyes.doctor_report()
|
||||
assert isinstance(report, str)
|
||||
assert "Agent Reach" in report
|
||||
@@ -0,0 +1,126 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Tests for doctor module."""
|
||||
|
||||
import pytest
|
||||
|
||||
import agent_reach.doctor as doctor
|
||||
from agent_reach.config import Config
|
||||
|
||||
|
||||
class _StubChannel:
|
||||
def __init__(self, name, description, tier, status, message, backends=None,
|
||||
active_backend=None):
|
||||
self.name = name
|
||||
self.description = description
|
||||
self.tier = tier
|
||||
self._status = status
|
||||
self._message = message
|
||||
self.backends = backends or []
|
||||
self.active_backend = active_backend
|
||||
|
||||
def check(self, config=None):
|
||||
return self._status, self._message
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def tmp_config(tmp_path):
|
||||
return Config(config_path=tmp_path / "config.yaml")
|
||||
|
||||
|
||||
class TestDoctor:
|
||||
def test_check_all_collects_channel_results(self, tmp_config, monkeypatch):
|
||||
monkeypatch.setattr(
|
||||
doctor,
|
||||
"get_all_channels",
|
||||
lambda: [
|
||||
_StubChannel("web", "网页", 0, "ok", "可抓取网页", ["requests"],
|
||||
active_backend="requests"),
|
||||
_StubChannel("github", "GitHub", 0, "warn", "gh 未安装", ["gh"]),
|
||||
_StubChannel("exa_search", "全网语义搜索", 1, "off", "mcporter 未配置", ["Exa"]),
|
||||
],
|
||||
)
|
||||
|
||||
results = doctor.check_all(tmp_config)
|
||||
|
||||
assert results == {
|
||||
"web": {
|
||||
"status": "ok",
|
||||
"name": "网页",
|
||||
"message": "可抓取网页",
|
||||
"tier": 0,
|
||||
"backends": ["requests"],
|
||||
"active_backend": "requests",
|
||||
},
|
||||
"github": {
|
||||
"status": "warn",
|
||||
"name": "GitHub",
|
||||
"message": "gh 未安装",
|
||||
"tier": 0,
|
||||
"backends": ["gh"],
|
||||
"active_backend": None,
|
||||
},
|
||||
"exa_search": {
|
||||
"status": "off",
|
||||
"name": "全网语义搜索",
|
||||
"message": "mcporter 未配置",
|
||||
"tier": 1,
|
||||
"backends": ["Exa"],
|
||||
"active_backend": None,
|
||||
},
|
||||
}
|
||||
|
||||
def test_format_report(self):
|
||||
report = doctor.format_report(
|
||||
{
|
||||
"web": {
|
||||
"status": "ok",
|
||||
"name": "网页",
|
||||
"message": "可抓取网页",
|
||||
"tier": 0,
|
||||
"backends": ["requests"],
|
||||
},
|
||||
"exa_search": {
|
||||
"status": "off",
|
||||
"name": "全网语义搜索",
|
||||
"message": "mcporter 未配置",
|
||||
"tier": 1,
|
||||
"backends": ["Exa"],
|
||||
},
|
||||
"xiaohongshu": {
|
||||
"status": "warn",
|
||||
"name": "小红书",
|
||||
"message": "MCP 已配置，但健康检查超时",
|
||||
"tier": 2,
|
||||
"backends": ["mcporter"],
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
# Strip Rich markup tags for assertion (PR #170 added [bold], [yellow] etc.)
|
||||
import re
|
||||
plain = re.sub(r"\[[^\]]*\]", "", report)
|
||||
assert "Agent Reach" in plain
|
||||
assert "装好即用：" in plain
|
||||
assert "1/3 个渠道可用" in plain
|
||||
# Inactive optional channels should be summarized in one line
|
||||
assert "可选渠道可以解锁" in plain
|
||||
|
||||
|
||||
def test_stale_active_backend_does_not_leak_into_errored_result(monkeypatch):
|
||||
"""渠道单例上一轮的 active_backend 不得泄漏进本轮异常结果(Codex review 发现)。"""
|
||||
from agent_reach import doctor
|
||||
|
||||
class _ExplodingChannel:
|
||||
name = "boom"
|
||||
description = "爆炸渠道"
|
||||
tier = 0
|
||||
backends = ["a", "b"]
|
||||
active_backend = "a" # 上一轮成功的残留
|
||||
|
||||
def check(self, config=None):
|
||||
raise RuntimeError("boom")
|
||||
|
||||
monkeypatch.setattr(doctor, "get_all_channels", lambda: [_ExplodingChannel()])
|
||||
results = doctor.check_all(config=None)
|
||||
assert results["boom"]["status"] == "error"
|
||||
assert results["boom"]["active_backend"] is None
|
||||
@@ -0,0 +1,100 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Tests for the OpenCLI cross-channel backend probing."""
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
from agent_reach.backends import opencli_status, opencli_summary
|
||||
from agent_reach.probe import ProbeResult
|
||||
|
||||
|
||||
def _status_with(version_probe, daemon_probe=None, ext_on_disk=False):
|
||||
"""Run opencli_status with probe_command and disk check patched."""
|
||||
calls = []
|
||||
|
||||
def fake_probe(cmd, args=("--version",), **kwargs):
|
||||
calls.append(list(args))
|
||||
if list(args) == ["--version"]:
|
||||
return version_probe
|
||||
return daemon_probe or ProbeResult("missing")
|
||||
|
||||
with patch("agent_reach.backends.opencli.probe_command", side_effect=fake_probe), \
|
||||
patch(
|
||||
"agent_reach.backends.opencli._extension_installed_on_disk",
|
||||
return_value=ext_on_disk,
|
||||
):
|
||||
return opencli_status(), calls
|
||||
|
||||
|
||||
def test_not_installed():
|
||||
st, _ = _status_with(ProbeResult("missing"))
|
||||
assert not st.installed
|
||||
assert not st.ready
|
||||
assert "未安装" in opencli_summary(st)
|
||||
|
||||
|
||||
def test_broken_node_env_gives_npm_hint():
|
||||
st, _ = _status_with(ProbeResult("broken", hint="x"))
|
||||
assert st.installed and st.broken
|
||||
assert "npm install -g @jackwener/opencli" in st.hint
|
||||
assert not st.ready
|
||||
|
||||
|
||||
def test_daemon_running_extension_connected_is_ready():
|
||||
daemon_out = "Daemon: running (PID 37389)\nVersion: v1.8.3\nExtension: connected\n"
|
||||
st, _ = _status_with(
|
||||
ProbeResult("ok", output="1.8.3"),
|
||||
ProbeResult("ok", output=daemon_out),
|
||||
)
|
||||
assert st.installed and st.daemon_running and st.extension_connected
|
||||
assert st.ready
|
||||
assert "1.8.3" in opencli_summary(st)
|
||||
|
||||
|
||||
def test_extension_never_installed_not_ready_with_store_guide():
|
||||
daemon_out = "Daemon: running (PID 1)\nExtension: disconnected\n"
|
||||
st, _ = _status_with(
|
||||
ProbeResult("ok", output="1.8.3"),
|
||||
ProbeResult("ok", output=daemon_out),
|
||||
ext_on_disk=False,
|
||||
)
|
||||
assert st.daemon_running and not st.extension_connected
|
||||
assert not st.ready
|
||||
assert "chromewebstore.google.com" in st.hint
|
||||
|
||||
|
||||
def test_sleeping_extension_counts_as_ready():
|
||||
"""实测:扩展 service worker 睡眠时 daemon status 报 disconnected,
|
||||
但任何真实命令会唤醒它——装在磁盘上即视为可用。"""
|
||||
daemon_out = "Daemon: running (PID 1)\nExtension: disconnected\n"
|
||||
st, _ = _status_with(
|
||||
ProbeResult("ok", output="1.8.3"),
|
||||
ProbeResult("ok", output=daemon_out),
|
||||
ext_on_disk=True,
|
||||
)
|
||||
assert not st.extension_connected
|
||||
assert st.extension_installed
|
||||
assert st.ready
|
||||
assert "唤醒" in opencli_summary(st)
|
||||
assert st.hint == ""
|
||||
|
||||
|
||||
def test_daemon_not_running_parsed_correctly():
|
||||
st, _ = _status_with(
|
||||
ProbeResult("ok", output="1.8.3"),
|
||||
ProbeResult("ok", output="Daemon: not running\n"),
|
||||
)
|
||||
assert st.installed
|
||||
assert not st.daemon_running
|
||||
assert not st.extension_connected
|
||||
assert "自动启动" in opencli_summary(st)
|
||||
|
||||
|
||||
def test_probe_uses_daemon_status_not_doctor():
|
||||
"""`opencli doctor` auto-starts the daemon (side effect) — health checks
|
||||
must only ever call `daemon status`."""
|
||||
_, calls = _status_with(
|
||||
ProbeResult("ok", output="1.8.3"),
|
||||
ProbeResult("ok", output="Daemon: not running\n"),
|
||||
)
|
||||
assert ["daemon", "status"] in calls
|
||||
assert ["doctor"] not in calls
|
||||
@@ -0,0 +1,90 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Tests for agent_reach.probe — real-execution probing and failure classification."""
|
||||
|
||||
import os
|
||||
import stat
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
from agent_reach.probe import ProbeResult, probe_command, reinstall_hint
|
||||
|
||||
|
||||
def _make_executable(path, content):
|
||||
path.write_text(content)
|
||||
path.chmod(path.stat().st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH)
|
||||
return str(path)
|
||||
|
||||
|
||||
def test_missing_command():
|
||||
r = probe_command("definitely-not-a-real-command-xyz")
|
||||
assert r.status == "missing"
|
||||
assert not r.ok
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="shebang semantics are POSIX-only")
|
||||
def test_broken_shebang_detected_as_broken(tmp_path, monkeypatch):
|
||||
"""A stale venv shim: which() finds it, exec raises FileNotFoundError."""
|
||||
script = _make_executable(
|
||||
tmp_path / "stale-tool", "#!/nonexistent/venv/bin/python\nprint('hi')\n"
|
||||
)
|
||||
monkeypatch.setenv("PATH", str(tmp_path) + os.pathsep + os.environ.get("PATH", ""))
|
||||
|
||||
r = probe_command("stale-tool", package="stale-tool-pkg")
|
||||
assert r.status == "broken"
|
||||
assert "uv tool install --force stale-tool-pkg" in r.hint
|
||||
assert "pipx reinstall stale-tool-pkg" in r.hint
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="shell script fixture is POSIX-only")
|
||||
def test_healthy_command_returns_ok_with_output(tmp_path, monkeypatch):
|
||||
script = _make_executable(
|
||||
tmp_path / "healthy-tool", "#!/bin/sh\necho 'healthy-tool 1.2.3'\n"
|
||||
)
|
||||
monkeypatch.setenv("PATH", str(tmp_path) + os.pathsep + os.environ.get("PATH", ""))
|
||||
|
||||
r = probe_command("healthy-tool")
|
||||
assert r.ok
|
||||
assert "1.2.3" in r.output
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="shell script fixture is POSIX-only")
|
||||
def test_nonzero_exit_classified_as_error(tmp_path, monkeypatch):
|
||||
script = _make_executable(
|
||||
tmp_path / "failing-tool", "#!/bin/sh\necho 'boom' >&2\nexit 3\n"
|
||||
)
|
||||
monkeypatch.setenv("PATH", str(tmp_path) + os.pathsep + os.environ.get("PATH", ""))
|
||||
|
||||
r = probe_command("failing-tool")
|
||||
assert r.status == "error"
|
||||
assert "boom" in r.output
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="shell script fixture is POSIX-only")
|
||||
def test_exit_127_classified_as_broken(tmp_path, monkeypatch):
|
||||
script = _make_executable(tmp_path / "wrapper-tool", "#!/bin/sh\nexit 127\n")
|
||||
monkeypatch.setenv("PATH", str(tmp_path) + os.pathsep + os.environ.get("PATH", ""))
|
||||
|
||||
r = probe_command("wrapper-tool", package="wrapper-pkg")
|
||||
assert r.status == "broken"
|
||||
assert "wrapper-pkg" in r.hint
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="shell script fixture is POSIX-only")
|
||||
def test_retries_help_transient_failures(tmp_path, monkeypatch):
|
||||
"""First call fails (exit 1), second succeeds — retries=1 should return ok."""
|
||||
marker = tmp_path / "ran-once"
|
||||
script = _make_executable(
|
||||
tmp_path / "flaky-tool",
|
||||
f"#!/bin/sh\nif [ -f {marker} ]; then echo ok; exit 0; fi\ntouch {marker}\nexit 1\n",
|
||||
)
|
||||
monkeypatch.setenv("PATH", str(tmp_path) + os.pathsep + os.environ.get("PATH", ""))
|
||||
|
||||
r = probe_command("flaky-tool", retries=1)
|
||||
assert r.ok
|
||||
|
||||
|
||||
def test_reinstall_hint_mentions_both_installers():
|
||||
hint = reinstall_hint("some-pkg")
|
||||
assert "uv tool install --force some-pkg" in hint
|
||||
assert "pipx reinstall some-pkg" in hint
|
||||
@@ -0,0 +1,18 @@
|
||||
from agent_reach.utils.process import mcporter_utf8_env_args, utf8_subprocess_env
|
||||
|
||||
|
||||
def test_utf8_subprocess_env_forces_python_utf8():
|
||||
env = utf8_subprocess_env({"PYTHONUTF8": "0", "OTHER": "value"})
|
||||
|
||||
assert env["PYTHONUTF8"] == "1"
|
||||
assert env["PYTHONIOENCODING"] == "utf-8"
|
||||
assert env["OTHER"] == "value"
|
||||
|
||||
|
||||
def test_mcporter_utf8_env_args():
|
||||
assert mcporter_utf8_env_args() == [
|
||||
"--env",
|
||||
"PYTHONUTF8=1",
|
||||
"--env",
|
||||
"PYTHONIOENCODING=utf-8",
|
||||
]
|
||||
@@ -0,0 +1,125 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Tests for 'agent-reach skill' command and _install_skill / _uninstall_skill."""
|
||||
|
||||
import importlib.resources
|
||||
import os
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from agent_reach.cli import _install_skill, _uninstall_skill
|
||||
|
||||
|
||||
class TestSkillCommand(unittest.TestCase):
|
||||
"""Test skill install and uninstall via CLI helpers."""
|
||||
|
||||
def test_skill_resources_include_both_locales(self):
|
||||
"""Package resources should expose both default and English skill markdown files."""
|
||||
skill_dir = importlib.resources.files("agent_reach").joinpath("skill")
|
||||
|
||||
default_skill = skill_dir.joinpath("SKILL.md").read_text(encoding="utf-8")
|
||||
english_skill = skill_dir.joinpath("SKILL_en.md").read_text(encoding="utf-8")
|
||||
|
||||
self.assertTrue(default_skill.strip())
|
||||
self.assertTrue(english_skill.strip())
|
||||
|
||||
def test_install_skill_creates_skill_md(self):
|
||||
"""_install_skill should create SKILL.md in the first available skill dir."""
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
skill_dir = os.path.join(tmpdir, "skills")
|
||||
os.makedirs(skill_dir)
|
||||
|
||||
with patch(
|
||||
"agent_reach.cli.os.path.expanduser",
|
||||
side_effect=lambda p: p.replace("~", tmpdir),
|
||||
), patch.dict(os.environ, {}, clear=False):
|
||||
# Remove OPENCLAW_HOME to avoid interference
|
||||
env = os.environ.copy()
|
||||
env.pop("OPENCLAW_HOME", None)
|
||||
with patch.dict(os.environ, env, clear=True):
|
||||
_install_skill()
|
||||
|
||||
# Check at least one known skill dir pattern
|
||||
for dirpath, _, filenames in os.walk(tmpdir):
|
||||
if "SKILL.md" in filenames:
|
||||
# Verify content is non-empty
|
||||
with open(os.path.join(dirpath, "SKILL.md"), encoding="utf-8") as f:
|
||||
content = f.read()
|
||||
self.assertIn("Agent Reach", content)
|
||||
# _install_skill may or may not find dirs depending on mock; just ensure no crash
|
||||
# The important test is that the function runs without error
|
||||
|
||||
def test_uninstall_skill_removes_dir(self):
|
||||
"""_uninstall_skill should remove skill directories."""
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
# Create a fake skill installation
|
||||
skill_path = os.path.join(tmpdir, ".openclaw", "skills", "agent-reach")
|
||||
os.makedirs(skill_path)
|
||||
with open(os.path.join(skill_path, "SKILL.md"), "w", encoding="utf-8") as f:
|
||||
f.write("test")
|
||||
|
||||
self.assertTrue(os.path.exists(skill_path))
|
||||
|
||||
with patch(
|
||||
"agent_reach.cli.os.path.expanduser",
|
||||
side_effect=lambda p: p.replace("~", tmpdir),
|
||||
), patch.dict(os.environ, {}, clear=False):
|
||||
env = os.environ.copy()
|
||||
env.pop("OPENCLAW_HOME", None)
|
||||
with patch.dict(os.environ, env, clear=True):
|
||||
_uninstall_skill()
|
||||
|
||||
self.assertFalse(os.path.exists(skill_path))
|
||||
|
||||
def test_install_creates_dir_if_parent_exists(self):
|
||||
"""_install_skill should create agent-reach dir inside existing skill dir."""
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
# Create the .openclaw/skills parent but not agent-reach subdir
|
||||
skill_parent = os.path.join(tmpdir, ".openclaw", "skills")
|
||||
os.makedirs(skill_parent)
|
||||
|
||||
with patch(
|
||||
"agent_reach.cli.os.path.expanduser",
|
||||
side_effect=lambda p: p.replace("~", tmpdir),
|
||||
), patch.dict(os.environ, {}, clear=False):
|
||||
env = os.environ.copy()
|
||||
env.pop("OPENCLAW_HOME", None)
|
||||
with patch.dict(os.environ, env, clear=True):
|
||||
_install_skill()
|
||||
|
||||
target = os.path.join(skill_parent, "agent-reach", "SKILL.md")
|
||||
self.assertTrue(os.path.exists(target))
|
||||
with open(target, encoding="utf-8") as f:
|
||||
content = f.read()
|
||||
self.assertIn("Agent Reach", content)
|
||||
|
||||
def test_install_uses_english_skill_for_english_locale(self):
|
||||
"""_install_skill should install the English skill file for English locales."""
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
skill_parent = os.path.join(tmpdir, ".openclaw", "skills")
|
||||
os.makedirs(skill_parent)
|
||||
|
||||
with patch(
|
||||
"agent_reach.cli.os.path.expanduser",
|
||||
side_effect=lambda p: p.replace("~", tmpdir),
|
||||
):
|
||||
env = os.environ.copy()
|
||||
env.pop("OPENCLAW_HOME", None)
|
||||
env["LANG"] = "en_US.UTF-8"
|
||||
with patch.dict(os.environ, env, clear=True):
|
||||
_install_skill()
|
||||
|
||||
target = os.path.join(skill_parent, "agent-reach", "SKILL.md")
|
||||
self.assertTrue(os.path.exists(target))
|
||||
with open(target, encoding="utf-8") as f:
|
||||
content = f.read()
|
||||
self.assertTrue(content.strip())
|
||||
self.assertIn("Xiaoyuzhou Podcast, LinkedIn", content)
|
||||
self.assertNotIn("搜推特", content)
|
||||
self.assertTrue(
|
||||
os.path.exists(os.path.join(skill_parent, "agent-reach", "references"))
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,401 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Tests for agent_reach.transcribe — provider routing, fallback, and errors."""
|
||||
|
||||
from pathlib import Path
|
||||
from typing import List
|
||||
|
||||
import pytest
|
||||
|
||||
from agent_reach import transcribe as tr
|
||||
from agent_reach.config import Config
|
||||
|
||||
# --- Fixtures ----------------------------------------------------------- #
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fake_config(tmp_path, monkeypatch):
|
||||
"""A Config that writes to a temp dir and never touches the user's HOME."""
|
||||
cfg_path = tmp_path / "config.yaml"
|
||||
monkeypatch.setattr(Config, "CONFIG_DIR", tmp_path)
|
||||
monkeypatch.setattr(Config, "CONFIG_FILE", cfg_path)
|
||||
cfg = Config(config_path=cfg_path)
|
||||
return cfg
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def chunk_file(tmp_path):
|
||||
p = tmp_path / "chunk.m4a"
|
||||
p.write_bytes(b"\x00fake-m4a-bytes")
|
||||
return p
|
||||
|
||||
|
||||
class FakeResponse:
|
||||
def __init__(self, status_code: int, text: str = ""):
|
||||
self.status_code = status_code
|
||||
self.text = text
|
||||
|
||||
@property
|
||||
def ok(self) -> bool:
|
||||
return 200 <= self.status_code < 300
|
||||
|
||||
|
||||
# --- transcribe_chunk: provider routing -------------------------------- #
|
||||
|
||||
|
||||
class TestTranscribeChunk:
|
||||
def test_routes_to_groq_endpoint(self, monkeypatch, fake_config, chunk_file):
|
||||
fake_config.set("groq_api_key", "gsk_test")
|
||||
captured = {}
|
||||
|
||||
def fake_post(url, headers=None, files=None, data=None, timeout=None):
|
||||
captured["url"] = url
|
||||
captured["headers"] = headers
|
||||
captured["model"] = data["model"]
|
||||
return FakeResponse(200, "hello world")
|
||||
|
||||
monkeypatch.setattr(tr.requests, "post", fake_post)
|
||||
text = tr.transcribe_chunk(chunk_file, "groq", config=fake_config)
|
||||
assert text == "hello world"
|
||||
assert captured["url"] == tr.PROVIDERS["groq"]["endpoint"]
|
||||
assert captured["model"] == "whisper-large-v3"
|
||||
assert captured["headers"]["Authorization"] == "Bearer gsk_test"
|
||||
|
||||
def test_routes_to_openai_endpoint(self, monkeypatch, fake_config, chunk_file):
|
||||
fake_config.set("openai_api_key", "sk-test")
|
||||
captured = {}
|
||||
|
||||
def fake_post(url, headers=None, files=None, data=None, timeout=None):
|
||||
captured["url"] = url
|
||||
captured["model"] = data["model"]
|
||||
return FakeResponse(200, "openai output")
|
||||
|
||||
monkeypatch.setattr(tr.requests, "post", fake_post)
|
||||
text = tr.transcribe_chunk(chunk_file, "openai", config=fake_config)
|
||||
assert text == "openai output"
|
||||
assert captured["url"] == tr.PROVIDERS["openai"]["endpoint"]
|
||||
assert captured["model"] == "whisper-1"
|
||||
|
||||
def test_raises_when_key_missing(self, fake_config, chunk_file):
|
||||
with pytest.raises(tr.NoProviderConfigured):
|
||||
tr.transcribe_chunk(chunk_file, "groq", config=fake_config)
|
||||
|
||||
def test_raises_on_http_error(self, monkeypatch, fake_config, chunk_file):
|
||||
fake_config.set("groq_api_key", "gsk_test")
|
||||
monkeypatch.setattr(
|
||||
tr.requests,
|
||||
"post",
|
||||
lambda *a, **k: FakeResponse(429, "rate limited"),
|
||||
)
|
||||
with pytest.raises(tr.TranscribeError, match="HTTP 429"):
|
||||
tr.transcribe_chunk(chunk_file, "groq", config=fake_config)
|
||||
|
||||
def test_unknown_provider(self, fake_config, chunk_file):
|
||||
with pytest.raises(tr.TranscribeError, match="unknown provider"):
|
||||
tr.transcribe_chunk(chunk_file, "azure", config=fake_config)
|
||||
|
||||
|
||||
# --- _transcribe_with_fallback ----------------------------------------- #
|
||||
|
||||
|
||||
class TestFallback:
|
||||
def test_groq_succeeds_no_openai_call(self, monkeypatch, fake_config, chunk_file):
|
||||
fake_config.set("groq_api_key", "gsk_test")
|
||||
fake_config.set("openai_api_key", "sk-test")
|
||||
calls: List[str] = []
|
||||
|
||||
def fake_post(url, headers=None, files=None, data=None, timeout=None):
|
||||
calls.append(url)
|
||||
return FakeResponse(200, "from-groq")
|
||||
|
||||
monkeypatch.setattr(tr.requests, "post", fake_post)
|
||||
text = tr._transcribe_with_fallback(chunk_file, ["groq", "openai"], fake_config)
|
||||
assert text == "from-groq"
|
||||
assert calls == [tr.PROVIDERS["groq"]["endpoint"]]
|
||||
|
||||
def test_groq_429_falls_back_to_openai(self, monkeypatch, fake_config, chunk_file):
|
||||
fake_config.set("groq_api_key", "gsk_test")
|
||||
fake_config.set("openai_api_key", "sk-test")
|
||||
calls: List[str] = []
|
||||
|
||||
def fake_post(url, headers=None, files=None, data=None, timeout=None):
|
||||
calls.append(url)
|
||||
if url == tr.PROVIDERS["groq"]["endpoint"]:
|
||||
return FakeResponse(429, "rate limited")
|
||||
return FakeResponse(200, "from-openai")
|
||||
|
||||
monkeypatch.setattr(tr.requests, "post", fake_post)
|
||||
text = tr._transcribe_with_fallback(chunk_file, ["groq", "openai"], fake_config)
|
||||
assert text == "from-openai"
|
||||
assert calls == [
|
||||
tr.PROVIDERS["groq"]["endpoint"],
|
||||
tr.PROVIDERS["openai"]["endpoint"],
|
||||
]
|
||||
|
||||
def test_skip_unconfigured_provider(self, monkeypatch, fake_config, chunk_file):
|
||||
# Only openai key configured — fallback should skip groq silently.
|
||||
fake_config.set("openai_api_key", "sk-test")
|
||||
calls: List[str] = []
|
||||
|
||||
def fake_post(url, headers=None, files=None, data=None, timeout=None):
|
||||
calls.append(url)
|
||||
return FakeResponse(200, "via-openai")
|
||||
|
||||
monkeypatch.setattr(tr.requests, "post", fake_post)
|
||||
text = tr._transcribe_with_fallback(chunk_file, ["groq", "openai"], fake_config)
|
||||
assert text == "via-openai"
|
||||
assert calls == [tr.PROVIDERS["openai"]["endpoint"]]
|
||||
|
||||
def test_all_fail_raises_with_last_error(self, monkeypatch, fake_config, chunk_file):
|
||||
fake_config.set("groq_api_key", "gsk_test")
|
||||
fake_config.set("openai_api_key", "sk-test")
|
||||
monkeypatch.setattr(
|
||||
tr.requests,
|
||||
"post",
|
||||
lambda *a, **k: FakeResponse(500, "boom"),
|
||||
)
|
||||
with pytest.raises(tr.TranscribeError, match="all providers failed"):
|
||||
tr._transcribe_with_fallback(chunk_file, ["groq", "openai"], fake_config)
|
||||
|
||||
|
||||
# --- transcribe (orchestrator) ---------------------------------------- #
|
||||
|
||||
|
||||
class TestOrchestrator:
|
||||
def test_local_file_skips_yt_dlp(self, monkeypatch, fake_config, tmp_path, chunk_file):
|
||||
fake_config.set("groq_api_key", "gsk_test")
|
||||
|
||||
def boom_download(*a, **k):
|
||||
raise AssertionError("yt-dlp must not be called for local files")
|
||||
|
||||
# Stub heavy external steps to no-ops that keep file paths valid.
|
||||
compressed = tmp_path / "compressed.m4a"
|
||||
compressed.write_bytes(b"x" * 1024)
|
||||
|
||||
def fake_compress(src, out_dir):
|
||||
return compressed
|
||||
|
||||
monkeypatch.setattr(tr, "download_audio", boom_download)
|
||||
monkeypatch.setattr(tr, "compress_audio", fake_compress)
|
||||
monkeypatch.setattr(
|
||||
tr.requests,
|
||||
"post",
|
||||
lambda *a, **k: FakeResponse(200, "transcript text"),
|
||||
)
|
||||
|
||||
text = tr.transcribe(
|
||||
str(chunk_file),
|
||||
out_dir=tmp_path / "work",
|
||||
config=fake_config,
|
||||
)
|
||||
assert text == "transcript text"
|
||||
|
||||
def test_chunks_concatenated_with_newlines(
|
||||
self, monkeypatch, fake_config, tmp_path, chunk_file
|
||||
):
|
||||
fake_config.set("groq_api_key", "gsk_test")
|
||||
# Force the "needs chunking" path by writing a file above the size limit.
|
||||
big = tmp_path / "compressed.m4a"
|
||||
big.write_bytes(b"x" * (tr.SIZE_LIMIT_BYTES + 1))
|
||||
monkeypatch.setattr(tr, "compress_audio", lambda src, out_dir: big)
|
||||
c1 = tmp_path / "chunk_001.m4a"
|
||||
c2 = tmp_path / "chunk_002.m4a"
|
||||
c1.write_bytes(b"a")
|
||||
c2.write_bytes(b"b")
|
||||
monkeypatch.setattr(tr, "chunk_audio", lambda src, out_dir: [c1, c2])
|
||||
|
||||
responses = iter(["part one ", "part two "])
|
||||
monkeypatch.setattr(
|
||||
tr.requests,
|
||||
"post",
|
||||
lambda *a, **k: FakeResponse(200, next(responses)),
|
||||
)
|
||||
|
||||
text = tr.transcribe(
|
||||
str(chunk_file),
|
||||
out_dir=tmp_path / "work",
|
||||
config=fake_config,
|
||||
)
|
||||
assert text == "part one\npart two"
|
||||
|
||||
def test_no_provider_configured_fails_fast(self, fake_config, chunk_file):
|
||||
with pytest.raises(tr.NoProviderConfigured):
|
||||
tr.transcribe(str(chunk_file), config=fake_config)
|
||||
|
||||
def test_invalid_provider_string(self, fake_config, chunk_file):
|
||||
with pytest.raises(tr.TranscribeError, match="unknown provider"):
|
||||
tr.transcribe(str(chunk_file), provider="azure", config=fake_config)
|
||||
|
||||
def test_auto_temp_dir_is_cleaned_up(self, monkeypatch, fake_config, tmp_path):
|
||||
fake_config.set("groq_api_key", "gsk_test")
|
||||
created_work_dirs = []
|
||||
|
||||
class FakeTemporaryDirectory:
|
||||
def __init__(self, prefix=None):
|
||||
self.path = tmp_path / "auto-work"
|
||||
|
||||
def __enter__(self):
|
||||
self.path.mkdir()
|
||||
created_work_dirs.append(self.path)
|
||||
return str(self.path)
|
||||
|
||||
def __exit__(self, *_):
|
||||
for child in self.path.iterdir():
|
||||
child.unlink()
|
||||
self.path.rmdir()
|
||||
|
||||
def fake_download(source, out_dir):
|
||||
assert Path(out_dir) == tmp_path / "auto-work"
|
||||
audio = Path(out_dir) / "source.m4a"
|
||||
audio.write_bytes(b"audio")
|
||||
return audio
|
||||
|
||||
def fake_compress(src, out_dir):
|
||||
compressed = Path(out_dir) / "compressed.m4a"
|
||||
compressed.write_bytes(b"x" * 1024)
|
||||
return compressed
|
||||
|
||||
monkeypatch.setattr(tr.tempfile, "TemporaryDirectory", FakeTemporaryDirectory)
|
||||
monkeypatch.setattr(tr, "download_audio", fake_download)
|
||||
monkeypatch.setattr(tr, "compress_audio", fake_compress)
|
||||
monkeypatch.setattr(
|
||||
tr.requests,
|
||||
"post",
|
||||
lambda *a, **k: FakeResponse(200, "transcript text"),
|
||||
)
|
||||
|
||||
text = tr.transcribe("https://example.com/video", config=fake_config)
|
||||
|
||||
assert text == "transcript text"
|
||||
assert created_work_dirs
|
||||
assert not created_work_dirs[0].exists()
|
||||
|
||||
def test_explicit_out_dir_is_preserved(self, monkeypatch, fake_config, tmp_path):
|
||||
fake_config.set("groq_api_key", "gsk_test")
|
||||
work = tmp_path / "caller-owned"
|
||||
|
||||
def fake_download(source, out_dir):
|
||||
audio = Path(out_dir) / "source.m4a"
|
||||
audio.write_bytes(b"audio")
|
||||
return audio
|
||||
|
||||
def fake_compress(src, out_dir):
|
||||
compressed = Path(out_dir) / "compressed.m4a"
|
||||
compressed.write_bytes(b"x" * 1024)
|
||||
return compressed
|
||||
|
||||
monkeypatch.setattr(tr, "download_audio", fake_download)
|
||||
monkeypatch.setattr(tr, "compress_audio", fake_compress)
|
||||
monkeypatch.setattr(
|
||||
tr.requests,
|
||||
"post",
|
||||
lambda *a, **k: FakeResponse(200, "transcript text"),
|
||||
)
|
||||
|
||||
tr.transcribe("https://example.com/video", out_dir=work, config=fake_config)
|
||||
|
||||
assert work.exists()
|
||||
assert (work / "compressed.m4a").exists()
|
||||
|
||||
|
||||
class TestDownloadAudioSafety:
|
||||
def test_rejects_private_network_url_before_yt_dlp(self, monkeypatch, tmp_path):
|
||||
monkeypatch.setattr(tr, "_require", lambda binary: None)
|
||||
|
||||
def should_not_run(*args, **kwargs):
|
||||
raise AssertionError("yt-dlp must not run for private/internal URLs")
|
||||
|
||||
monkeypatch.setattr(tr, "_run", should_not_run)
|
||||
|
||||
with pytest.raises(tr.TranscribeError, match="private|internal|SSRF"):
|
||||
tr.download_audio("http://169.254.169.254/latest/meta-data/", tmp_path)
|
||||
|
||||
def test_passes_public_url_after_end_of_options_marker(self, monkeypatch, tmp_path):
|
||||
monkeypatch.setattr(tr, "_require", lambda binary: None)
|
||||
captured = {}
|
||||
|
||||
def fake_run(cmd, timeout=600):
|
||||
captured["cmd"] = cmd
|
||||
(tmp_path / "source.m4a").write_bytes(b"audio")
|
||||
|
||||
monkeypatch.setattr(tr, "_run", fake_run)
|
||||
|
||||
audio = tr.download_audio("https://example.com/watch?v=123", tmp_path)
|
||||
|
||||
assert audio == tmp_path / "source.m4a"
|
||||
assert "--" in captured["cmd"]
|
||||
marker_index = captured["cmd"].index("--")
|
||||
assert captured["cmd"][marker_index + 1] == "https://example.com/watch?v=123"
|
||||
|
||||
def test_preserves_bare_public_urls_supported_by_yt_dlp(self, monkeypatch, tmp_path):
|
||||
monkeypatch.setattr(tr, "_require", lambda binary: None)
|
||||
captured = {}
|
||||
|
||||
def fake_run(cmd, timeout=600):
|
||||
captured["cmd"] = cmd
|
||||
(tmp_path / "source.m4a").write_bytes(b"audio")
|
||||
|
||||
monkeypatch.setattr(tr, "_run", fake_run)
|
||||
|
||||
tr.download_audio("youtu.be/abc123", tmp_path)
|
||||
|
||||
assert captured["cmd"][-1] == "youtu.be/abc123"
|
||||
|
||||
def test_does_not_dns_resolve_public_hostnames(self, monkeypatch, tmp_path):
|
||||
import socket
|
||||
|
||||
monkeypatch.setattr(tr, "_require", lambda binary: None)
|
||||
monkeypatch.setattr(
|
||||
socket,
|
||||
"getaddrinfo",
|
||||
lambda *args, **kwargs: (_ for _ in ()).throw(
|
||||
AssertionError("public hostnames should not be DNS-resolved here")
|
||||
),
|
||||
)
|
||||
captured = {}
|
||||
|
||||
def fake_run(cmd, timeout=600):
|
||||
captured["cmd"] = cmd
|
||||
(tmp_path / "source.m4a").write_bytes(b"audio")
|
||||
|
||||
monkeypatch.setattr(tr, "_run", fake_run)
|
||||
|
||||
tr.download_audio("https://youtu.be/abc123", tmp_path)
|
||||
|
||||
assert captured["cmd"][-1] == "https://youtu.be/abc123"
|
||||
|
||||
|
||||
# --- YouTubeChannel integration --------------------------------------- #
|
||||
|
||||
|
||||
class TestYouTubeChannelTranscribe:
|
||||
def test_delegates_to_transcribe(self, monkeypatch, fake_config):
|
||||
from agent_reach.channels.youtube import YouTubeChannel
|
||||
|
||||
captured = {}
|
||||
|
||||
def fake_transcribe(source, *, provider="auto", out_dir=None, config=None):
|
||||
captured["source"] = source
|
||||
captured["provider"] = provider
|
||||
captured["config"] = config
|
||||
return "delegated text"
|
||||
|
||||
monkeypatch.setattr(tr, "transcribe", fake_transcribe)
|
||||
out = YouTubeChannel().transcribe(
|
||||
"https://youtu.be/abc", provider="groq", config=fake_config
|
||||
)
|
||||
assert out == "delegated text"
|
||||
assert captured["source"] == "https://youtu.be/abc"
|
||||
assert captured["provider"] == "groq"
|
||||
assert captured["config"] is fake_config
|
||||
|
||||
|
||||
# --- Config feature requirement --------------------------------------- #
|
||||
|
||||
|
||||
class TestConfigOpenAIWhisper:
|
||||
def test_openai_whisper_feature_registered(self, fake_config):
|
||||
assert "openai_whisper" in Config.FEATURE_REQUIREMENTS
|
||||
assert Config.FEATURE_REQUIREMENTS["openai_whisper"] == ["openai_api_key"]
|
||||
assert not fake_config.is_configured("openai_whisper")
|
||||
fake_config.set("openai_api_key", "sk-test")
|
||||
assert fake_config.is_configured("openai_whisper")
|
||||
@@ -0,0 +1,184 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
from unittest.mock import patch, Mock
|
||||
|
||||
from agent_reach.channels.twitter import TwitterChannel
|
||||
|
||||
|
||||
def _cp(stdout="", stderr="", returncode=0):
|
||||
m = Mock()
|
||||
m.stdout = stdout
|
||||
m.stderr = stderr
|
||||
m.returncode = returncode
|
||||
return m
|
||||
|
||||
|
||||
# --- twitter-cli tests ---
|
||||
|
||||
def test_check_twitter_cli_found_and_auth_ok():
|
||||
"""twitter-cli found + twitter status ok → ok."""
|
||||
channel = TwitterChannel()
|
||||
with patch("shutil.which", side_effect=lambda name: "/usr/local/bin/twitter" if name == "twitter" else None), patch(
|
||||
"subprocess.run",
|
||||
return_value=_cp(stdout="ok: true\nusername: testuser\n", returncode=0),
|
||||
):
|
||||
status, message = channel.check()
|
||||
assert status == "ok"
|
||||
assert "twitter-cli" in message
|
||||
assert "完整可用" in message
|
||||
assert channel.active_backend == "twitter-cli"
|
||||
|
||||
|
||||
def test_check_twitter_cli_found_auth_missing():
|
||||
"""twitter-cli found + not_authenticated → warn about auth."""
|
||||
channel = TwitterChannel()
|
||||
with patch("shutil.which", side_effect=lambda name: "/usr/local/bin/twitter" if name == "twitter" else None), patch(
|
||||
"subprocess.run",
|
||||
return_value=_cp(
|
||||
stderr="ok: false\nerror:\n code: not_authenticated\n",
|
||||
returncode=1,
|
||||
),
|
||||
):
|
||||
status, message = channel.check()
|
||||
assert status == "warn"
|
||||
assert "未认证" in message
|
||||
# 未认证是业务态：工具进程活着，后端仍可用
|
||||
assert channel.active_backend == "twitter-cli"
|
||||
|
||||
|
||||
# --- bird CLI fallback tests ---
|
||||
|
||||
def test_check_bird_fallback_auth_ok():
|
||||
"""No twitter-cli, but bird found + bird check ok → ok."""
|
||||
channel = TwitterChannel()
|
||||
def which_side_effect(name):
|
||||
if name == "bird":
|
||||
return "/usr/local/bin/bird"
|
||||
return None
|
||||
with patch("shutil.which", side_effect=which_side_effect), patch(
|
||||
"subprocess.run",
|
||||
return_value=_cp(stdout="Authenticated as @user\n", returncode=0),
|
||||
):
|
||||
status, message = channel.check()
|
||||
assert status == "ok"
|
||||
assert "bird" in message
|
||||
assert channel.active_backend == "bird CLI (legacy)"
|
||||
|
||||
|
||||
def test_check_bird_fallback_auth_missing():
|
||||
"""No twitter-cli, bird found but Missing credentials → warn."""
|
||||
channel = TwitterChannel()
|
||||
def which_side_effect(name):
|
||||
if name == "bird":
|
||||
return "/usr/local/bin/bird"
|
||||
return None
|
||||
with patch("shutil.which", side_effect=which_side_effect), patch(
|
||||
"subprocess.run",
|
||||
return_value=_cp(stderr="Missing credentials\n", returncode=1),
|
||||
):
|
||||
status, message = channel.check()
|
||||
assert status == "warn"
|
||||
assert "未配置认证" in message
|
||||
|
||||
|
||||
# --- neither installed ---
|
||||
|
||||
def test_check_nothing_installed():
|
||||
"""Neither twitter-cli nor bird → warn with install hint."""
|
||||
channel = TwitterChannel()
|
||||
with patch("shutil.which", return_value=None):
|
||||
status, message = channel.check()
|
||||
assert status == "warn"
|
||||
assert "twitter-cli" in message
|
||||
assert channel.active_backend is None
|
||||
|
||||
|
||||
# --- twitter-cli preferred over bird ---
|
||||
|
||||
def test_twitter_cli_preferred_over_bird():
|
||||
"""When both are installed, twitter-cli is used."""
|
||||
channel = TwitterChannel()
|
||||
def which_side_effect(name):
|
||||
if name == "twitter":
|
||||
return "/usr/local/bin/twitter"
|
||||
if name == "bird":
|
||||
return "/usr/local/bin/bird"
|
||||
return None
|
||||
with patch("shutil.which", side_effect=which_side_effect), patch(
|
||||
"subprocess.run",
|
||||
return_value=_cp(stdout="ok: true\n", returncode=0),
|
||||
):
|
||||
status, message = channel.check()
|
||||
assert status == "ok"
|
||||
assert "twitter-cli" in message
|
||||
assert channel.active_backend == "twitter-cli"
|
||||
|
||||
|
||||
# --- broken install (stale venv shim) ---
|
||||
|
||||
def test_check_twitter_cli_broken_reports_error_with_reinstall_hint():
|
||||
"""which 命中但 exec 抛 FileNotFoundError（venv 断链）→ error + 重装处方。"""
|
||||
channel = TwitterChannel()
|
||||
with patch(
|
||||
"shutil.which",
|
||||
side_effect=lambda name: "/usr/local/bin/twitter" if name == "twitter" else None,
|
||||
), patch("subprocess.run", side_effect=FileNotFoundError("/usr/local/bin/twitter")):
|
||||
status, message = channel.check()
|
||||
assert status == "error"
|
||||
assert "无法执行" in message
|
||||
assert "uv tool install --force twitter-cli" in message
|
||||
assert "pipx reinstall twitter-cli" in message
|
||||
assert channel.active_backend is None
|
||||
|
||||
|
||||
def test_check_twitter_cli_broken_falls_back_to_bird():
|
||||
"""twitter-cli 断链但 bird 健康 → 回退到 bird，后端正确归属。"""
|
||||
channel = TwitterChannel()
|
||||
|
||||
def which_side_effect(name):
|
||||
if name in ("twitter", "bird"):
|
||||
return f"/usr/local/bin/{name}"
|
||||
return None
|
||||
|
||||
def run_side_effect(cmd, **kwargs):
|
||||
if "twitter" in cmd[0]:
|
||||
raise FileNotFoundError(cmd[0])
|
||||
return _cp(stdout="Authenticated as @user\n", returncode=0)
|
||||
|
||||
with patch("shutil.which", side_effect=which_side_effect), patch(
|
||||
"subprocess.run", side_effect=run_side_effect
|
||||
):
|
||||
status, message = channel.check()
|
||||
assert status == "ok"
|
||||
assert "bird" in message
|
||||
assert channel.active_backend == "bird CLI (legacy)"
|
||||
|
||||
|
||||
def test_unauthenticated_twitter_cli_does_not_block_working_opencli():
|
||||
"""warn 候选不得屏蔽排在后面的 ok 候选(Codex review 发现)。"""
|
||||
channel = TwitterChannel()
|
||||
with patch.object(
|
||||
TwitterChannel, "_check_twitter_cli",
|
||||
return_value=("warn", "twitter-cli 已安装但未认证"),
|
||||
), patch.object(
|
||||
TwitterChannel, "_check_opencli",
|
||||
return_value=("ok", "OpenCLI 可用（复用浏览器登录态）"),
|
||||
), patch.object(TwitterChannel, "_check_bird", return_value=None):
|
||||
status, msg = channel.check()
|
||||
assert status == "ok"
|
||||
assert channel.active_backend == "OpenCLI"
|
||||
|
||||
|
||||
def test_all_warn_falls_back_to_first_warn():
|
||||
channel = TwitterChannel()
|
||||
with patch.object(
|
||||
TwitterChannel, "_check_twitter_cli",
|
||||
return_value=("warn", "twitter-cli 未认证"),
|
||||
), patch.object(
|
||||
TwitterChannel, "_check_opencli",
|
||||
return_value=("warn", "扩展未连接"),
|
||||
), patch.object(TwitterChannel, "_check_bird", return_value=None):
|
||||
status, msg = channel.check()
|
||||
assert status == "warn"
|
||||
assert channel.active_backend == "twitter-cli"
|
||||
assert "未认证" in msg
|
||||
@@ -0,0 +1,133 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Tests for XiaoHongShu output formatter (issue #134)."""
|
||||
|
||||
import unittest
|
||||
|
||||
from agent_reach.channels.xiaohongshu import format_xhs_result
|
||||
|
||||
|
||||
class TestFormatXhsResult(unittest.TestCase):
|
||||
"""Test format_xhs_result strips redundant fields."""
|
||||
|
||||
SAMPLE_NOTE = {
|
||||
"id": "abc123",
|
||||
"title": "测试笔记",
|
||||
"desc": "这是正文内容",
|
||||
"type": "normal",
|
||||
"xsec_token": "tok_xxx",
|
||||
"user": {
|
||||
"nickname": "小红",
|
||||
"user_id": "u123",
|
||||
"avatar": "https://example.com/avatar.jpg",
|
||||
"extra_field": "should be dropped",
|
||||
},
|
||||
"interact_info": {
|
||||
"liked_count": "100",
|
||||
"collected_count": "50",
|
||||
"comment_count": "20",
|
||||
"share_count": "10",
|
||||
"sticky_count": "0",
|
||||
"relation": "none",
|
||||
},
|
||||
"image_list": [
|
||||
{
|
||||
"url": "https://img.example.com/1.jpg",
|
||||
"info_list": [{"url": "https://img.example.com/1_small.jpg", "image_scene": "WB_DFT"}],
|
||||
"width": 1080,
|
||||
"height": 1440,
|
||||
"trace_id": "tr_123",
|
||||
},
|
||||
{
|
||||
"url": "https://img.example.com/2.jpg",
|
||||
"info_list": [{"url": "https://img.example.com/2_small.jpg"}],
|
||||
"width": 1080,
|
||||
"height": 1080,
|
||||
},
|
||||
],
|
||||
"tag_list": [
|
||||
{"id": "t1", "name": "旅行", "type": "topic"},
|
||||
{"id": "t2", "name": "美食", "type": "topic"},
|
||||
],
|
||||
"at_user_list": [],
|
||||
"geo_info": {"latitude": 0, "longitude": 0},
|
||||
"audit_info": {"audit_status": 0},
|
||||
"model_type": None,
|
||||
"note_flow_source": "search",
|
||||
}
|
||||
|
||||
def test_single_note_keeps_useful_fields(self):
|
||||
result = format_xhs_result(self.SAMPLE_NOTE)
|
||||
self.assertEqual(result["id"], "abc123")
|
||||
self.assertEqual(result["title"], "测试笔记")
|
||||
self.assertEqual(result["desc"], "这是正文内容")
|
||||
self.assertEqual(result["type"], "normal")
|
||||
self.assertEqual(result["user"]["nickname"], "小红")
|
||||
self.assertEqual(result["liked_count"], "100")
|
||||
self.assertEqual(result["collected_count"], "50")
|
||||
self.assertEqual(result["images"], [
|
||||
"https://img.example.com/1.jpg",
|
||||
"https://img.example.com/2.jpg",
|
||||
])
|
||||
self.assertEqual(result["tags"], ["旅行", "美食"])
|
||||
|
||||
def test_single_note_drops_useless_fields(self):
|
||||
result = format_xhs_result(self.SAMPLE_NOTE)
|
||||
self.assertNotIn("at_user_list", result)
|
||||
self.assertNotIn("geo_info", result)
|
||||
self.assertNotIn("audit_info", result)
|
||||
self.assertNotIn("model_type", result)
|
||||
self.assertNotIn("note_flow_source", result)
|
||||
# User should not have extra fields
|
||||
self.assertNotIn("avatar", result.get("user", {}))
|
||||
self.assertNotIn("extra_field", result.get("user", {}))
|
||||
|
||||
def test_search_results_wrapper(self):
|
||||
"""Handle {"items": [...]} wrapper from search_feeds."""
|
||||
wrapped = {"items": [self.SAMPLE_NOTE, self.SAMPLE_NOTE]}
|
||||
result = format_xhs_result(wrapped)
|
||||
self.assertIsInstance(result, list)
|
||||
self.assertEqual(len(result), 2)
|
||||
self.assertEqual(result[0]["title"], "测试笔记")
|
||||
|
||||
def test_list_input(self):
|
||||
result = format_xhs_result([self.SAMPLE_NOTE])
|
||||
self.assertIsInstance(result, list)
|
||||
self.assertEqual(len(result), 1)
|
||||
self.assertEqual(result[0]["title"], "测试笔记")
|
||||
|
||||
def test_note_card_wrapper(self):
|
||||
"""Handle notes nested under 'note_card'."""
|
||||
wrapped = {"note_card": self.SAMPLE_NOTE}
|
||||
result = format_xhs_result(wrapped)
|
||||
self.assertEqual(result["title"], "测试笔记")
|
||||
|
||||
def test_with_comments(self):
|
||||
note = dict(self.SAMPLE_NOTE)
|
||||
note["comments"] = [
|
||||
{
|
||||
"content": "写得好！",
|
||||
"user_info": {"nickname": "路人甲", "user_id": "u456"},
|
||||
"like_count": 5,
|
||||
"sub_comment_count": 1,
|
||||
"ip_location": "上海",
|
||||
"status": 0,
|
||||
}
|
||||
]
|
||||
result = format_xhs_result(note)
|
||||
self.assertEqual(len(result["comments"]), 1)
|
||||
self.assertEqual(result["comments"][0]["content"], "写得好！")
|
||||
self.assertEqual(result["comments"][0]["user"], "路人甲")
|
||||
self.assertEqual(result["comments"][0]["like_count"], 5)
|
||||
self.assertNotIn("ip_location", result["comments"][0])
|
||||
|
||||
def test_empty_input(self):
|
||||
self.assertEqual(format_xhs_result({}), {})
|
||||
self.assertEqual(format_xhs_result([]), [])
|
||||
|
||||
def test_non_dict_passthrough(self):
|
||||
self.assertEqual(format_xhs_result("hello"), "hello")
|
||||
self.assertIsNone(format_xhs_result(None))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,21 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
import agent_reach.cli as cli
|
||||
|
||||
|
||||
class _DummyConfig:
|
||||
def get(self, _key):
|
||||
return None
|
||||
|
||||
|
||||
def test_install_xiaoyuzhou_deps_does_not_raise_when_no_groq_key(capsys):
|
||||
with patch("agent_reach.config.Config", return_value=_DummyConfig()), \
|
||||
patch("os.path.isfile", side_effect=lambda p: True if str(p).endswith("transcribe.sh") else False), \
|
||||
patch("shutil.which", return_value=None):
|
||||
cli._install_xiaoyuzhou_deps()
|
||||
|
||||
out = capsys.readouterr().out
|
||||
assert "Xiaoyuzhou" in out
|
||||
assert "Groq API key not set" in out
|
||||
@@ -0,0 +1,147 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Dedicated tests for the ``youtube`` channel.
|
||||
|
||||
YouTube's readiness is layered: yt-dlp must probe alive (missing / broken /
|
||||
unrunnable are distinct), a JS runtime is mandatory (deno works out of the
|
||||
box, Node needs an explicit ``--js-runtimes`` config), and transcription
|
||||
readiness (whisper provider + ffmpeg) is surfaced for ``doctor`` without
|
||||
gating the backend. These tests stub ``probe_command`` / ``shutil.which``
|
||||
so every branch runs offline. Follow-up to #331 — extends dedicated
|
||||
channel coverage after rss (#360), github (#361), web (#363),
|
||||
reddit (#364), xueqiu (#365) and v2ex (#366).
|
||||
"""
|
||||
|
||||
from unittest.mock import Mock, patch
|
||||
|
||||
from agent_reach.probe import ProbeResult
|
||||
from agent_reach.channels import youtube as yt
|
||||
from agent_reach.channels.youtube import YouTubeChannel, _has_js_runtime_config
|
||||
|
||||
|
||||
def _which(*present):
|
||||
"""shutil.which side effect: returns a path for the named tools present."""
|
||||
return lambda name: f"/usr/bin/{name}" if name in present else None
|
||||
|
||||
|
||||
# --- can_handle ---
|
||||
|
||||
def test_can_handle_matches_youtube_hosts():
|
||||
ch = YouTubeChannel()
|
||||
for url in [
|
||||
"https://www.youtube.com/watch?v=abc",
|
||||
"https://youtu.be/abc",
|
||||
"https://m.youtube.com/watch?v=abc",
|
||||
"https://YOUTUBE.COM/shorts/x",
|
||||
]:
|
||||
assert ch.can_handle(url) is True, url
|
||||
for url in ["https://example.com", "https://vimeo.com/123", ""]:
|
||||
assert ch.can_handle(url) is False, url
|
||||
|
||||
|
||||
# --- _has_js_runtime_config ---
|
||||
|
||||
def test_has_js_runtime_config_missing_file_is_false(tmp_path):
|
||||
assert _has_js_runtime_config(tmp_path / "nope.conf") is False
|
||||
|
||||
|
||||
def test_has_js_runtime_config_true_when_flag_present(tmp_path):
|
||||
cfg = tmp_path / "config"
|
||||
cfg.write_text("--js-runtimes deno\n--no-mtime\n", encoding="utf-8")
|
||||
assert _has_js_runtime_config(cfg) is True
|
||||
|
||||
|
||||
def test_has_js_runtime_config_false_when_flag_absent(tmp_path):
|
||||
cfg = tmp_path / "config"
|
||||
cfg.write_text("--no-mtime\n", encoding="utf-8")
|
||||
assert _has_js_runtime_config(cfg) is False
|
||||
|
||||
|
||||
def test_has_js_runtime_config_swallows_oserror():
|
||||
bad = Mock()
|
||||
bad.exists.return_value = True
|
||||
with patch.object(yt, "read_utf8_text", side_effect=OSError("denied")):
|
||||
assert _has_js_runtime_config(bad) is False
|
||||
|
||||
|
||||
# --- check(): yt-dlp probe states ---
|
||||
|
||||
def test_check_off_when_ytdlp_missing():
|
||||
ch = YouTubeChannel()
|
||||
with patch.object(yt, "probe_command", return_value=ProbeResult("missing")):
|
||||
status, message = ch.check()
|
||||
assert status == "off"
|
||||
assert ch.active_backend is None
|
||||
|
||||
|
||||
def test_check_error_when_ytdlp_broken():
|
||||
ch = YouTubeChannel()
|
||||
with patch.object(yt, "probe_command", return_value=ProbeResult("broken", hint="relink venv")):
|
||||
status, message = ch.check()
|
||||
assert status == "error"
|
||||
assert "relink venv" in message
|
||||
assert ch.active_backend is None
|
||||
|
||||
|
||||
def test_check_error_when_ytdlp_unrunnable():
|
||||
ch = YouTubeChannel()
|
||||
with patch.object(yt, "probe_command", return_value=ProbeResult("timeout", hint="too slow")):
|
||||
status, message = ch.check()
|
||||
assert status == "error"
|
||||
assert ch.active_backend is None
|
||||
|
||||
|
||||
# --- check(): JS runtime gating (backend stays yt-dlp once the probe is ok) ---
|
||||
|
||||
def test_check_warn_when_no_js_runtime_but_backend_active():
|
||||
ch = YouTubeChannel()
|
||||
with patch.object(yt, "probe_command", return_value=ProbeResult("ok")), \
|
||||
patch("shutil.which", side_effect=_which()): # no deno, no node
|
||||
status, message = ch.check()
|
||||
assert status == "warn"
|
||||
assert "JS runtime" in message
|
||||
assert ch.active_backend == "yt-dlp" # probe was ok → backend attributed
|
||||
|
||||
|
||||
def test_check_warn_when_node_only_and_config_missing_flag():
|
||||
ch = YouTubeChannel()
|
||||
with patch.object(yt, "probe_command", return_value=ProbeResult("ok")), \
|
||||
patch("shutil.which", side_effect=_which("node")), \
|
||||
patch.object(yt, "_has_js_runtime_config", return_value=False):
|
||||
status, message = ch.check()
|
||||
assert status == "warn"
|
||||
assert ch.active_backend == "yt-dlp"
|
||||
|
||||
|
||||
def test_check_ok_with_deno():
|
||||
ch = YouTubeChannel()
|
||||
with patch.object(yt, "probe_command", return_value=ProbeResult("ok")), \
|
||||
patch("shutil.which", side_effect=_which("deno")):
|
||||
status, message = ch.check()
|
||||
assert status == "ok"
|
||||
assert message == "可提取视频信息和字幕"
|
||||
assert ch.active_backend == "yt-dlp"
|
||||
|
||||
|
||||
# --- check(): transcription readiness surfaced (deno present so JS path is ok) ---
|
||||
|
||||
def test_check_ok_reports_transcription_when_provider_and_ffmpeg_present():
|
||||
ch = YouTubeChannel()
|
||||
cfg = Mock()
|
||||
cfg.is_configured = lambda key: key == "groq_whisper"
|
||||
with patch.object(yt, "probe_command", return_value=ProbeResult("ok")), \
|
||||
patch("shutil.which", side_effect=_which("deno", "ffmpeg")):
|
||||
status, message = ch.check(config=cfg)
|
||||
assert status == "ok"
|
||||
assert "groq" in message
|
||||
assert "可转写音频" in message
|
||||
|
||||
|
||||
def test_check_ok_flags_missing_ffmpeg_for_transcription():
|
||||
ch = YouTubeChannel()
|
||||
cfg = Mock()
|
||||
cfg.is_configured = lambda key: key == "openai_whisper"
|
||||
with patch.object(yt, "probe_command", return_value=ProbeResult("ok")), \
|
||||
patch("shutil.which", side_effect=_which("deno")): # provider yes, ffmpeg no
|
||||
status, message = ch.check(config=cfg)
|
||||
assert status == "ok"
|
||||
assert "ffmpeg" in message
|
||||
Reference in New Issue
Block a user