chore: import upstream snapshot with attribution

2026-07-13 13:25:10 +08:00
commit c397331b1e
3684 changed files with 990993 additions and 0 deletions
@@ -0,0 +1,130 @@
+# Give Your AI Agent Ears: FunASR as a Drop-in Speech Backend
+
+**TL;DR**: `funasr-server` turns FunASR into an OpenAI-compatible `/v1/audio/transcriptions` endpoint. Agent frameworks such as LangChain, AutoGen, CrewAI, Dify, and MCP-based assistants can use it by changing the base URL.
+
+---
+
+## The Problem
+
+Every voice-enabled AI agent needs speech-to-text. Most developers default to:
+
+- **OpenAI Whisper API** - convenient, but paid per minute and sends audio to a hosted service
+- **Local Whisper** - self-hosted, but slower and does not include speaker diarization by default
+- **Google/Azure STT** - mature, but adds vendor lock-in and service-specific authentication
+
+What if you could get **170x realtime speed**, **50+ languages**, **speaker diarization**, **emotion detection**, and **private deployment** while keeping OpenAI SDK compatibility?
+
+## The Solution: FunASR + OpenAI-Compatible Server
+
+```bash
+pip install funasr fastapi uvicorn python-multipart
+funasr-server --model sensevoice --device cuda --port 8000
+```
+
+That is it. You now have a local speech API at `http://localhost:8000/v1`.
+
+## Verify It in 60 Seconds
+
+In another terminal, use the bundled smoke test:
+
+```bash
+git clone https://github.com/modelscope/FunASR
+cd FunASR/examples/openai_api
+bash smoke_test.sh
+# Cross-platform alternative:
+python smoke_test.py
+```
+
+Or run the equivalent commands manually:
+
+```bash
+curl -L https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/BAC009S0764W0121.wav -o sample.wav
+curl http://localhost:8000/v1/audio/transcriptions \
+  -F file=@sample.wav \
+  -F model=sensevoice \
+  -F response_format=verbose_json
+```
+
+The response includes `text`; with `verbose_json`, supported models can also return segment-level metadata.
+
+## Use with Any Agent Framework
+
+### OpenAI SDK
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
+result = client.audio.transcriptions.create(
+    model="sensevoice",
+    file=open("user_voice.wav", "rb"),
+)
+print(result.text)
+```
+
+### LangChain
+
+```python
+# Just override the base_url in your audio chain.
+transcription = openai_client.audio.transcriptions.create(
+    model="sensevoice",
+    file=audio_file,
+)
+agent.invoke({"input": transcription.text})
+```
+
+### MCP (Claude, Cursor, Windsurf)
+
+```json
+{
+  "mcpServers": {
+    "funasr": {
+      "command": "python",
+      "args": ["funasr_mcp.py"]
+    }
+  }
+}
+```
+
+Now your AI assistant can transcribe local audio files while keeping the audio inside your environment.
+
+## Why FunASR Over Whisper?
+
+| | FunASR (SenseVoice) | Whisper large-v3 |
+|---|---|---|
+| Speed | **170x** realtime | 13x realtime |
+| Architecture | Non-autoregressive (parallel) | Autoregressive (sequential) |
+| Speaker ID | Built-in | Needs pyannote + HF token |
+| Emotion | Detects happy/sad/angry | No |
+| CPU viable | 17x realtime on CPU | Impractical |
+| Cost | Free (MIT) | $0.006/min (API) |
+| Deployment | Self-hosted API server | Local model or hosted API |
+
+## Available Models
+
+| Model | Best For | Speed |
+|-------|----------|-------|
+| `sensevoice` | General purpose, emotion | 170x GPU / 17x CPU |
+| `paraformer` | Chinese production | 120x GPU / 15x CPU |
+| `paraformer-en` | English production | 120x GPU / 15x CPU |
+| `fun-asr-nano` | 31 languages, LLM-based | 17x GPU |
+
+## Get Started
+
+```bash
+pip install funasr fastapi uvicorn python-multipart
+funasr-server --model sensevoice --device cuda
+```
+
+Then point your agent's audio transcription client to `http://localhost:8000/v1`.
+
+---
+
+**Links:**
+
+- GitHub: https://github.com/modelscope/FunASR
+- OpenAI API example: https://github.com/modelscope/FunASR/tree/main/examples/openai_api
+- Agent integration: https://modelscope.github.io/FunASR/agent.html
+- Benchmark: https://modelscope.github.io/FunASR/benchmark.html
+- Live demo: https://huggingface.co/spaces/FunAudioLLM/Fun-ASR-Nano-GPU-Debug
+- PyPI: `pip install funasr`