chore: import upstream snapshot with attribution

2026-07-13 13:25:10 +08:00
commit c397331b1e
3684 changed files with 990993 additions and 0 deletions
@@ -0,0 +1,41 @@
+# Reproducing the FunASR-vs-whisper.cpp benchmark
+
+Scripts and method behind [`../BENCHMARKS.md`](../BENCHMARKS.md).
+
+## Metric (authoritative FunASR口径)
+- **micro-CER** = `Σ edit_distance / Σ reference_chars` over all files (not a per-file mean).
+- **normalize_zh**: `re.sub(r'[^\w一-鿿]', '', text).upper()` — drop punctuation/whitespace,
+  keep word chars + CJK, upper-case. (SenseVoice meta tags `<|...|>` are stripped first.)
+- **RTF** = `Σ compute_time / Σ audio_duration`, model-load time excluded.
+
+`compute_cer.py` implements exactly this:
+```bash
+python compute_cer.py --refs testset.json --hyp_dir <hyps>/ [--time_file <times>.txt]
+```
+`testset.json` is a list of `{"id"/"key", "ref", "duration"}`; `<hyps>/{key}.txt` are
+the transcripts; `<times>.txt` has `key compute_seconds` per line.
+
+## Producing hypotheses
+
+FunASR (this runtime), per clip:
+```bash
+# SenseVoice / Paraformer: ids -> detok
+build/bin/llama-funasr-sensevoice -m sensevoice-small.gguf -a $k.wav > $k.ids
+python ../sensevoice/detok.py <model>/chn_jpn_yue_eng_ko_spectok.bpe.model $k.ids > $k.txt
+build/bin/llama-funasr-paraformer -m paraformer.gguf -a $k.wav > $k.ids
+python ../paraformer/detok_paraformer.py <model>/tokens.json $k.ids > $k.txt
+# Fun-ASR-Nano: text directly
+build/bin/llama-funasr-cli --enc funasr-encoder.gguf -m qwen3-0.6b-q8_0.gguf -a $k.wav --chunk 15 > $k.txt
+```
+Compute time is on each tool's stderr (`encode … s` / `enc … dec … s`).
+
+whisper.cpp, per clip (forced Chinese, no timestamps):
+```bash
+whisper-cli -m models/ggml-<size>.bin -l zh -nt -t 8 $k.wav > $k.txt   # 2>stderr has "total time"/"load time"
+```
+RTF compute time = `(total − load) ms`.
+
+## Notes
+- Run all systems with the **same thread count** (here `-t 8` / 8 threads) for a fair RTF.
+- Whisper does its own internal 30 s windowing; the FunASR segmentation口径 is documented
+  in `BENCHMARKS.md` (see the methodology/caveats sections).