chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,127 @@
|
||||
# Adapter: whisper（视频/音频转录）
|
||||
|
||||
被 `/cheat-learn-from` 在 Way b（用户提供视频文件，让工具转录）时调用。
|
||||
|
||||
> **优先 Way a**（用户直接粘 script 文本——简单 + 准确）。Way b（whisper）只在用户**找不到 script 只有视频**时用。
|
||||
|
||||
---
|
||||
|
||||
## 这个 adapter 是干嘛的
|
||||
|
||||
把 mp4 / mov / mp3 等媒体文件转成文字 transcript，让 Claude 能读对标账号的稿子。
|
||||
|
||||
抖音 / B站 / YouTube 大多数视频**没有官方字幕**——拿稿子绕不开 ASR（语音转录）。这是为什么本 adapter 存在。
|
||||
|
||||
---
|
||||
|
||||
## 安装（一次性）
|
||||
|
||||
### 选项 A：whisper-cpp（**推荐**——快、轻、纯 C++）
|
||||
|
||||
Mac M 系列芯片上一条 3 分钟视频转录 30-60 秒。
|
||||
|
||||
```bash
|
||||
# 1. 装 whisper-cpp
|
||||
brew install whisper-cpp
|
||||
|
||||
# 2. 装 ffmpeg（whisper-cpp 依赖，从视频里抽音频）
|
||||
brew install ffmpeg
|
||||
|
||||
# 3. 下载模型（中文推荐 medium 或 large-v3，准确度够 + 速度还行）
|
||||
# whisper-cpp 第一次运行会自动下载，或手动：
|
||||
mkdir -p ~/.whisper-cpp/models
|
||||
cd ~/.whisper-cpp/models
|
||||
# medium 模型 (~1.5GB)
|
||||
curl -L -O https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-medium.bin
|
||||
```
|
||||
|
||||
### 选项 B：openai-whisper（Python 版，更慢但有 API 兼容性）
|
||||
|
||||
```bash
|
||||
pip install openai-whisper
|
||||
brew install ffmpeg
|
||||
|
||||
# 模型自动下载
|
||||
```
|
||||
|
||||
### 选项 C：用云端 API（不需要本地模型）
|
||||
|
||||
`/cheat-learn-from` 暂不直接支持云端 API——如果你有 OpenAI / Azure / 阿里云的 ASR API key，可以自己改 `run.sh` 走云端。
|
||||
|
||||
---
|
||||
|
||||
## 用法
|
||||
|
||||
cheat-learn-from 自动调用，你不需要手动跑。但如果想手动测试：
|
||||
|
||||
```bash
|
||||
# 转录单个视频
|
||||
bash run.sh <video_path> <output_dir>
|
||||
|
||||
# 例：
|
||||
bash run.sh ~/Desktop/对标账号/某视频.mp4 ~/my-channel/samples/对标账号/abc123/
|
||||
# → 输出 ~/my-channel/samples/对标账号/abc123/transcript.md
|
||||
```
|
||||
|
||||
## 输出格式
|
||||
|
||||
`transcript.md`：
|
||||
|
||||
```markdown
|
||||
# Transcript: <video filename>
|
||||
|
||||
**Source**: <video file path>
|
||||
**Transcribed at**: <ISO timestamp>
|
||||
**Engine**: whisper-cpp medium / openai-whisper large / etc.
|
||||
**Duration**: <video length>
|
||||
|
||||
---
|
||||
|
||||
[纯文本转录，按段落分（不是字幕格式）]
|
||||
```
|
||||
|
||||
> 注意 whisper 输出的字幕是按 **句子** 分行的（每句换行 + 时间戳）。
|
||||
> run.sh 会去掉时间戳 + 把短句合并成段落，让 Claude 读起来像稿子，不是字幕表。
|
||||
|
||||
## 失败模式
|
||||
|
||||
| 症状 | 原因 | 处理 |
|
||||
|---|---|---|
|
||||
| `whisper-cpp: command not found` | 没装 | 跑 `brew install whisper-cpp` |
|
||||
| `ffmpeg: command not found` | 没装 ffmpeg | 跑 `brew install ffmpeg` |
|
||||
| 转录乱码 / 大量错字 | 视频是英文但用了中文模型，反之亦然 | 改 `run.sh` 里 `--language` 参数 |
|
||||
| 转录慢（>10 分钟） | 用了 large 模型 + 没有 GPU/M-chip 加速 | 换 medium 模型 |
|
||||
| Disk full | 模型文件大（large-v3 ~3GB） | 用 medium（~1.5GB）够用 |
|
||||
|
||||
## 稳定性等级
|
||||
|
||||
★★★★ — whisper 是开源标准 ASR，不会突然失效。模型更新自由，pin 版本无虞。
|
||||
|
||||
## 风险提示
|
||||
|
||||
- **TOS**：你转录**自己下载的对标账号视频**用于个人学习参考是合理使用；**不要**把转录结果再发布
|
||||
- **隐私**：whisper 全部本地运行，不传任何数据到云端
|
||||
|
||||
## 文件清单
|
||||
|
||||
```
|
||||
adapters/script-extraction/whisper/
|
||||
├── README.md # 本文件
|
||||
└── run.sh # cheat-learn-from 调用的 wrapper
|
||||
```
|
||||
|
||||
## 与其他 adapter 的关系
|
||||
|
||||
- 同 `adapters/perf-data/douyin-session/`、`adapters/trend-sources/*` 一样，是 cheat-on-content 的可选 adapter
|
||||
- 只在 `/cheat-learn-from --way b` 时调用——Way a（粘文本）不需要
|
||||
|
||||
## 用户自己下载视频的说明
|
||||
|
||||
工具**不直接抓视频**——避免 TOS 风险 + 反爬维护成本。建议用：
|
||||
|
||||
- **抖音**：第三方下载器 / 抖音 PC 版 → 复制视频链接 → 粘进下载器
|
||||
- **B站**：[BBDown](https://github.com/nilaoda/BBDown) / [you-get](https://github.com/soimort/you-get)
|
||||
- **YouTube**：[yt-dlp](https://github.com/yt-dlp/yt-dlp)（最强大）
|
||||
- **小红书**：[xhs-downloader](https://github.com/JoeanAmier/XHS-Downloader)
|
||||
|
||||
下载后扔到 `samples/<benchmark-name>/<video-id>/source.mp4` 即可——cheat-learn-from 会自动找到。
|
||||
Executable
+133
@@ -0,0 +1,133 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# whisper adapter wrapper
|
||||
#
|
||||
# Called by /cheat-learn-from when user provides video file (Way b).
|
||||
# Transcribes video → transcript.md (paragraph format, no timestamps).
|
||||
#
|
||||
# Usage:
|
||||
# bash run.sh <video_path> <output_dir> [--lang <code>] [--model <name>]
|
||||
#
|
||||
# Defaults:
|
||||
# --lang zh
|
||||
# --model medium (whisper-cpp) or medium (openai-whisper)
|
||||
#
|
||||
# Output: writes transcript.md INTO output_dir.
|
||||
# Exit codes:
|
||||
# 0 = success
|
||||
# 1 = whisper not installed
|
||||
# 2 = ffmpeg not installed
|
||||
# 3 = video file not found / unreadable
|
||||
# 4 = transcription failed
|
||||
|
||||
set -uo pipefail
|
||||
|
||||
VIDEO="${1:-}"
|
||||
OUTPUT_DIR="${2:-}"
|
||||
LANG="zh"
|
||||
MODEL="medium"
|
||||
|
||||
# Parse optional flags
|
||||
shift 2 2>/dev/null || true
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--lang) LANG="$2"; shift 2 ;;
|
||||
--model) MODEL="$2"; shift 2 ;;
|
||||
*) echo "Unknown flag: $1" >&2; exit 4 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [[ -z "$VIDEO" || -z "$OUTPUT_DIR" ]]; then
|
||||
echo "Usage: bash run.sh <video_path> <output_dir> [--lang zh|en|...] [--model tiny|base|small|medium|large-v3]" >&2
|
||||
exit 4
|
||||
fi
|
||||
|
||||
if [[ ! -f "$VIDEO" ]]; then
|
||||
echo "❌ Video file not found: $VIDEO" >&2
|
||||
exit 3
|
||||
fi
|
||||
|
||||
mkdir -p "$OUTPUT_DIR"
|
||||
|
||||
# Detect available whisper engine
|
||||
ENGINE=""
|
||||
if command -v whisper-cpp >/dev/null 2>&1; then
|
||||
ENGINE="whisper-cpp"
|
||||
elif command -v whisper >/dev/null 2>&1; then
|
||||
ENGINE="openai-whisper"
|
||||
else
|
||||
cat >&2 <<EOF
|
||||
❌ Neither whisper-cpp nor openai-whisper installed.
|
||||
|
||||
Install one:
|
||||
Option A (recommended, fast): brew install whisper-cpp
|
||||
Option B (Python, slower): pip install openai-whisper
|
||||
|
||||
Then re-run /cheat-learn-from.
|
||||
|
||||
See adapters/script-extraction/whisper/README.md for details.
|
||||
EOF
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if ! command -v ffmpeg >/dev/null 2>&1; then
|
||||
echo "❌ ffmpeg not installed. Run: brew install ffmpeg" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
echo "[whisper] engine: $ENGINE | model: $MODEL | lang: $LANG"
|
||||
echo "[whisper] transcribing: $VIDEO"
|
||||
|
||||
TMP_OUT=$(mktemp -d)
|
||||
trap 'rm -rf "$TMP_OUT"' EXIT
|
||||
|
||||
# Transcribe — get raw text output
|
||||
if [[ "$ENGINE" == "whisper-cpp" ]]; then
|
||||
# whisper-cpp needs WAV input, convert via ffmpeg
|
||||
AUDIO="$TMP_OUT/audio.wav"
|
||||
ffmpeg -y -loglevel error -i "$VIDEO" -ar 16000 -ac 1 -f wav "$AUDIO" 2>&1 || {
|
||||
echo "❌ ffmpeg failed to extract audio" >&2; exit 4;
|
||||
}
|
||||
whisper-cpp -m "$HOME/.whisper-cpp/models/ggml-${MODEL}.bin" -l "$LANG" -otxt -of "$TMP_OUT/out" "$AUDIO" >/dev/null 2>&1 || {
|
||||
echo "❌ whisper-cpp failed (model file might be missing — check ~/.whisper-cpp/models/)" >&2; exit 4;
|
||||
}
|
||||
RAW_TXT="$TMP_OUT/out.txt"
|
||||
else
|
||||
# openai-whisper
|
||||
whisper "$VIDEO" --language "$LANG" --model "$MODEL" --output_format txt --output_dir "$TMP_OUT" >/dev/null 2>&1 || {
|
||||
echo "❌ openai-whisper failed" >&2; exit 4;
|
||||
}
|
||||
# openai-whisper names output as <video-basename>.txt
|
||||
BASENAME=$(basename "$VIDEO" | sed 's/\.[^.]*$//')
|
||||
RAW_TXT="$TMP_OUT/${BASENAME}.txt"
|
||||
fi
|
||||
|
||||
if [[ ! -f "$RAW_TXT" ]]; then
|
||||
echo "❌ No transcript produced" >&2
|
||||
exit 4
|
||||
fi
|
||||
|
||||
# Get video metadata for header
|
||||
DURATION=$(ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1:nokey=1 "$VIDEO" 2>/dev/null | awk '{printf "%d:%02d", $1/60, $1%60}')
|
||||
[[ -z "$DURATION" ]] && DURATION="unknown"
|
||||
|
||||
# Build output transcript.md
|
||||
TRANSCRIPT_OUT="$OUTPUT_DIR/transcript.md"
|
||||
{
|
||||
echo "# Transcript: $(basename "$VIDEO")"
|
||||
echo ""
|
||||
echo "**Source**: $VIDEO"
|
||||
echo "**Transcribed at**: $(date -u +"%Y-%m-%dT%H:%M:%SZ")"
|
||||
echo "**Engine**: $ENGINE / $MODEL"
|
||||
echo "**Language**: $LANG"
|
||||
echo "**Duration**: $DURATION"
|
||||
echo ""
|
||||
echo "---"
|
||||
echo ""
|
||||
# Raw text — whisper outputs one sentence per line; merge into paragraphs
|
||||
# Heuristic: collapse to single paragraph (Claude can re-paragraph if needed)
|
||||
awk 'BEGIN{ORS=""} {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $0); if($0!=""){print $0; if(NR%5==0)print "\n\n"; else print " "}} END{print "\n"}' "$RAW_TXT"
|
||||
} > "$TRANSCRIPT_OUT"
|
||||
|
||||
echo "✅ transcript.md written → $TRANSCRIPT_OUT"
|
||||
exit 0
|
||||
Reference in New Issue
Block a user