wehub/modelscope--funasr
1
0
Fork 0
Code Issues Pull Requests Actions Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 13:25:10 +08:00
commit c397331b1e
3684 changed files with 990993 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
examples/openai_api/.dockerignore
+11
View File
@@ -0,0 +1,11 @@
__pycache__/
*.py[cod]
*.wav
*.mp3
*.flac
*.m4a
*.ogg
*.webm
*.log
.cache/
.pytest_cache/
examples/openai_api/.env.example
+8
View File
@@ -0,0 +1,8 @@
# Host port exposed by Docker Compose.
FUNASR_HOST_PORT=8000
# The example Docker image defaults to CPU so the container can start on machines
# without NVIDIA Container Toolkit. Keep CPU mode unless the image has CUDA-capable PyTorch.
FUNASR_DEVICE=cpu
# Set FUNASR_DEVICE=cuda only after using a CUDA-capable PyTorch image.
FUNASR_MODEL=sensevoice
examples/openai_api/BLOG_POST.md
+130
View File
@@ -0,0 +1,130 @@
# Give Your AI Agent Ears: FunASR as a Drop-in Speech Backend
**TL;DR**: `funasr-server` turns FunASR into an OpenAI-compatible `/v1/audio/transcriptions` endpoint. Agent frameworks such as LangChain, AutoGen, CrewAI, Dify, and MCP-based assistants can use it by changing the base URL.
---
## The Problem
Every voice-enabled AI agent needs speech-to-text. Most developers default to:
- **OpenAI Whisper API** - convenient, but paid per minute and sends audio to a hosted service
- **Local Whisper** - self-hosted, but slower and does not include speaker diarization by default
- **Google/Azure STT** - mature, but adds vendor lock-in and service-specific authentication
What if you could get **170x realtime speed**, **50+ languages**, **speaker diarization**, **emotion detection**, and **private deployment** while keeping OpenAI SDK compatibility?
## The Solution: FunASR + OpenAI-Compatible Server
```bash
pip install funasr fastapi uvicorn python-multipart
funasr-server --model sensevoice --device cuda --port 8000
```
That is it. You now have a local speech API at `http://localhost:8000/v1`.
## Verify It in 60 Seconds
In another terminal, use the bundled smoke test:
```bash
git clone https://github.com/modelscope/FunASR
cd FunASR/examples/openai_api
bash smoke_test.sh
# Cross-platform alternative:
python smoke_test.py
```
Or run the equivalent commands manually:
```bash
curl -L https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/BAC009S0764W0121.wav -o sample.wav
curl http://localhost:8000/v1/audio/transcriptions \
-F file=@sample.wav \
-F model=sensevoice \
-F response_format=verbose_json
```
The response includes `text`; with `verbose_json`, supported models can also return segment-level metadata.
## Use with Any Agent Framework
### OpenAI SDK
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
result = client.audio.transcriptions.create(
model="sensevoice",
file=open("user_voice.wav", "rb"),
)
print(result.text)
```
### LangChain
```python
# Just override the base_url in your audio chain.
transcription = openai_client.audio.transcriptions.create(
model="sensevoice",
file=audio_file,
)
agent.invoke({"input": transcription.text})
```
### MCP (Claude, Cursor, Windsurf)
```json
{
"mcpServers": {
"funasr": {
"command": "python",
"args": ["funasr_mcp.py"]
}
}
}
```
Now your AI assistant can transcribe local audio files while keeping the audio inside your environment.
## Why FunASR Over Whisper?
| | FunASR (SenseVoice) | Whisper large-v3 |
|---|---|---|
| Speed | **170x** realtime | 13x realtime |
| Architecture | Non-autoregressive (parallel) | Autoregressive (sequential) |
| Speaker ID | Built-in | Needs pyannote + HF token |
| Emotion | Detects happy/sad/angry | No |
| CPU viable | 17x realtime on CPU | Impractical |
| Cost | Free (MIT) | $0.006/min (API) |
| Deployment | Self-hosted API server | Local model or hosted API |
## Available Models
| Model | Best For | Speed |
|-------|----------|-------|
| `sensevoice` | General purpose, emotion | 170x GPU / 17x CPU |
| `paraformer` | Chinese production | 120x GPU / 15x CPU |
| `paraformer-en` | English production | 120x GPU / 15x CPU |
| `fun-asr-nano` | 31 languages, LLM-based | 17x GPU |
## Get Started
```bash
pip install funasr fastapi uvicorn python-multipart
funasr-server --model sensevoice --device cuda
```
Then point your agent's audio transcription client to `http://localhost:8000/v1`.
---
**Links:**
- GitHub: https://github.com/modelscope/FunASR
- OpenAI API example: https://github.com/modelscope/FunASR/tree/main/examples/openai_api
- Agent integration: https://modelscope.github.io/FunASR/agent.html
- Benchmark: https://modelscope.github.io/FunASR/benchmark.html
- Live demo: https://huggingface.co/spaces/FunAudioLLM/Fun-ASR-Nano-GPU-Debug
- PyPI: `pip install funasr`
examples/openai_api/CLIENTS.md
+165
View File
@@ -0,0 +1,165 @@
# Client Recipes for the FunASR OpenAI-Compatible API
Use this page when `funasr-server` is already running and you want to connect an existing application, agent tool, or workflow engine to local speech recognition. For JavaScript, TypeScript, and Next.js examples, see the [JavaScript/TypeScript recipes](JAVASCRIPT.md) or [Chinese JavaScript/TypeScript recipes](JAVASCRIPT_zh.md). For Dify, n8n, HTTP nodes, and webhook workers, see the [workflow recipes](WORKFLOWS.md) or [Chinese workflow recipes](WORKFLOWS_zh.md). For browser upload or microphone demos, use the [Gradio browser demo](GRADIO.md). For no-code API smoke tests, import the [Postman collection](POSTMAN.md). For schema-driven imports or client generation, use the [OpenAPI spec](OPENAPI.md). Before sharing the service, review the [security and gateway guide](SECURITY.md).
## Preflight
```bash
export BASE_URL=http://localhost:8000
curl -fsS "$BASE_URL/health"
curl -fsS "$BASE_URL/v1/models"
```
If the server is on another machine, replace `localhost` with the reachable host name or service address. Keep `/v1` in SDK base URLs, and omit `/v1` for direct endpoint checks like `/health`.
## Model aliases
| Alias | Good first use | Notes |
|---|---|---|
| `sensevoice` | Private multilingual API | Fast default with language, emotion, and event tags. |
| `paraformer` | Mandarin production transcription | Includes VAD and punctuation. |
| `paraformer-en` | English transcription | Smaller English-only route. |
| `fun-asr-nano` | LLM-based ASR experiments | Pair with vLLM for higher throughput deployments. |
## Python OpenAI SDK
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
with open("meeting.wav", "rb") as audio:
result = client.audio.transcriptions.create(
model="sensevoice",
file=audio,
response_format="verbose_json",
)
print(result.text)
for segment in getattr(result, "segments", []):
print(segment)
```
Most OpenAI SDKs require an API key value even when the local FunASR server does not check it. Use any placeholder for local development, then add real authentication at your gateway if the service is shared.
## JavaScript and TypeScript
Use the [JavaScript/TypeScript recipes](JAVASCRIPT.md) for OpenAI JS SDK, built-in `fetch`, TypeScript helper functions, and Next.js route handlers. Minimal OpenAI SDK shape:
```javascript
import OpenAI from "openai";
import { createReadStream } from "node:fs";
const client = new OpenAI({
baseURL: "http://localhost:8000/v1",
apiKey: "local-development",
});
const result = await client.audio.transcriptions.create({
model: "sensevoice",
file: createReadStream("meeting.wav"),
response_format: "verbose_json",
});
console.log(result.text);
```
For browser uploads, send audio to your backend first, then proxy to FunASR with authentication and upload limits. See the [Chinese JavaScript/TypeScript recipes](JAVASCRIPT_zh.md) for localized guidance.
## Plain Python requests
```python
import requests
with open("meeting.wav", "rb") as audio:
response = requests.post(
"http://localhost:8000/v1/audio/transcriptions",
files={"file": ("meeting.wav", audio, "audio/wav")},
data={"model": "sensevoice", "response_format": "verbose_json"},
timeout=300,
)
response.raise_for_status()
print(response.json()["text"])
```
This is the most portable pattern for internal services, queues, notebooks, and low-code tools that can issue multipart HTTP requests.
## Agent tool pattern
Expose transcription as a regular tool function. The agent does not need to know FunASR internals; it only needs a file path or uploaded audio object.
```python
from pathlib import Path
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="local")
def transcribe_audio(audio_path: str) -> str:
"""Transcribe a local audio file with FunASR and return plain text."""
path = Path(audio_path)
with path.open("rb") as audio:
result = client.audio.transcriptions.create(
model="sensevoice",
file=audio,
)
return result.text
```
For LangChain, LlamaIndex, AutoGen, CrewAI, Semantic Kernel, and similar frameworks, register the function above using that framework's normal tool or function-calling mechanism.
## Dify, workflow engines, and HTTP nodes
Use a multipart HTTP node or custom tool:
| Setting | Value |
|---|---|
| Method | `POST` |
| URL | `http://<funasr-host>:8000/v1/audio/transcriptions` |
| Body type | `multipart/form-data` |
| File field | `file` |
| Text fields | `model=sensevoice`, `response_format=verbose_json` |
| Result path | `text` for transcript, `segments` for timestamps/speakers |
When the workflow system cannot send files directly, upload audio to an internal object store first, then run a small worker that downloads the object and calls FunASR with the `requests` recipe above. See [workflow recipes](WORKFLOWS.md) for Dify, n8n, and webhook-worker patterns, or the [Chinese workflow recipes](WORKFLOWS_zh.md).
## Response formats
`response_format=json` returns a compact response:
```json
{"text": "recognized speech"}
```
`response_format=verbose_json` adds operational fields useful for agents and subtitles:
```json
{
"text": "recognized speech",
"segments": [
{"start": 0.0, "end": 3.2, "text": "recognized speech", "speaker": 0}
],
"language": "auto",
"duration": 0.42,
"model": "sensevoice"
}
```
## Production checklist
- Put TLS, authentication, rate limits, and upload-size limits in front of the service before exposing it outside a trusted network; use the [security and gateway guide](SECURITY.md) as the rollout checklist.
- Preload the default model at startup and use `/health` for readiness checks.
- Set client timeouts based on maximum audio duration; long recordings need longer HTTP timeouts.
- Log audio duration, model alias, device, latency, response format, and error type for every request.
- Pin model aliases and deployment images in production notes so benchmark results remain reproducible.
- For GPU hosts, keep one worker per GPU until you have measured memory headroom and concurrency behavior.
## Troubleshooting quick checks
| Symptom | Check |
|---|---|
| SDK says authentication is missing | Pass any placeholder `api_key` for local development. |
| 400 unknown model | Call `/v1/models` and use one of the listed aliases. |
| Request times out | Increase client timeout or split very long recordings. |
| First request is slow | The model may be loading; preload with `--model sensevoice`. |
| CUDA is unavailable | Start with `--device cpu` to verify the API path, then fix GPU drivers/runtime. |
| Port conflict | Start with `--port 9000` and set `BASE_URL=http://localhost:9000`. |
examples/openai_api/Dockerfile
+33
View File
@@ -0,0 +1,33 @@
FROM python:3.10-slim
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
PIP_NO_CACHE_DIR=1 \
FUNASR_MODEL=sensevoice \
FUNASR_DEVICE=cpu \
FUNASR_PORT=8000
WORKDIR /app
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
ffmpeg \
git \
libsndfile1 \
&& rm -rf /var/lib/apt/lists/*
RUN python -m pip install --upgrade pip \
&& pip install \
funasr \
fastapi \
"uvicorn[standard]" \
python-multipart
COPY server.py /app/server.py
EXPOSE 8000
HEALTHCHECK --interval=30s --timeout=5s --start-period=60s --retries=3 \
CMD python -c "import os, urllib.request; urllib.request.urlopen('http://127.0.0.1:%s/health' % os.getenv('FUNASR_PORT', '8000'), timeout=3).read()" || exit 1
CMD ["sh", "-c", "python server.py --host 0.0.0.0 --port ${FUNASR_PORT:-8000} --device ${FUNASR_DEVICE:-cpu} --model ${FUNASR_MODEL:-sensevoice}"]
examples/openai_api/GRADIO.md
+62
View File
@@ -0,0 +1,62 @@
# Gradio Browser Demo for the FunASR OpenAI-Compatible API
Use this demo when you want a browser UI for uploading or recording audio while the FunASR OpenAI-compatible API server runs locally, in Docker, or behind a private Kubernetes service.
The Gradio app does not load FunASR models itself. It calls the same `/health`, `/v1/models`, and `/v1/audio/transcriptions` endpoints used by the smoke tests, SDK recipes, Postman collection, and OpenAPI spec.
## 1. Start the API server
From `examples/openai_api`:
```bash
pip install funasr fastapi uvicorn python-multipart
python server.py --model sensevoice --device cuda --port 8000
```
For a portable CPU check, use `--device cpu`. For Docker Compose or Kubernetes, keep the service private and expose it locally with the documented port mapping or `kubectl port-forward`.
## 2. Install and launch the browser UI
In another terminal:
```bash
pip install gradio
python gradio_app.py --base-url http://localhost:8000
```
Open the printed local URL, upload or record an audio file, choose a model alias, and click **Transcribe**.
## 3. Verify the backend first
The UI has a **Check service** button. You can run the same check in a terminal:
```bash
python smoke_test.py --base-url http://localhost:8000
curl http://localhost:8000/v1/models
```
If the API server is remote, set the reachable URL explicitly:
```bash
python gradio_app.py --base-url http://funasr-api.speech.svc.cluster.local:8000
```
For OpenAI SDK clients, remember that SDK base URLs include `/v1`; this Gradio demo expects the direct service base URL without `/v1`.
## Model aliases
| Alias | Good first use |
|---|---|
| `sensevoice` | Fast multilingual private transcription and agent voice input. |
| `paraformer` | Mandarin-oriented production transcription. |
| `paraformer-en` | English-only compatibility checks. |
| `fun-asr-nano` | LLM-based ASR and vLLM experiments. |
See the [model selection guide](../../docs/model_selection.md) for a deeper comparison.
## Production notes
- Treat the Gradio app as a demo or internal operator UI, not a public production frontend.
- Add authentication, TLS, upload-size limits, and rate limits before exposing any audio upload UI outside a trusted network; see the [security and gateway guide](SECURITY.md).
- Keep browser uploads close to your backend; do not send private audio to an unauthenticated public endpoint.
- Log model alias, audio duration, latency, response format, and error text when debugging.
examples/openai_api/GRADIO_zh.md
+62
View File
@@ -0,0 +1,62 @@
# FunASR OpenAI 兼容 API Gradio 浏览器 Demo
当 FunASR OpenAI 兼容 API 已经在本地、Docker 或私有 Kubernetes 服务中运行，而你想用浏览器上传或录制音频时，可以使用这个 Gradio demo。
Gradio app 本身不加载 FunASR 模型。它调用和 smoke test、SDK 配方、Postman 集合、OpenAPI 规范相同的 `/health`、`/v1/models` 和 `/v1/audio/transcriptions` 接口。
## 1. 启动 API 服务
在 `examples/openai_api` 目录中执行：
```bash
pip install funasr fastapi uvicorn python-multipart
python server.py --model sensevoice --device cuda --port 8000
```
便携 CPU 验证可以使用 `--device cpu`。如果使用 Docker Compose 或 Kubernetes，请保持服务私有，并通过文档中的端口映射或 `kubectl port-forward` 暴露到本机验证。
## 2. 安装并启动浏览器 UI
在另一个终端执行：
```bash
pip install gradio
python gradio_app.py --base-url http://localhost:8000
```
打开命令行输出的本地 URL，上传或录制音频，选择模型 alias，然后点击 **Transcribe**。
## 3. 先验证后端服务
UI 中有 **Check service** 按钮。你也可以在终端运行相同检查：
```bash
python smoke_test.py --base-url http://localhost:8000
curl http://localhost:8000/v1/models
```
如果 API 服务在远端，请显式设置可访问的地址：
```bash
python gradio_app.py --base-url http://funasr-api.speech.svc.cluster.local:8000
```
OpenAI SDK 的 base URL 需要包含 `/v1`；这个 Gradio demo 使用的是不带 `/v1` 的直接服务 base URL。
## 模型别名
| Alias | 适合场景 |
|---|---|
| `sensevoice` | 快速多语种私有转写和 Agent 语音输入。 |
| `paraformer` | 中文生产转写。 |
| `paraformer-en` | 英文兼容性检查。 |
| `fun-asr-nano` | LLM-based ASR 和 vLLM 实验。 |
更完整的对比见 [模型选择指南](../../docs/model_selection_zh.md)。
## 生产注意事项
- Gradio app 适合作为 demo 或内部操作界面，不建议直接作为公网生产前端。
- 任何音频上传 UI 对可信网络外开放前，都需要先增加鉴权、TLS、上传大小限制和限流；见 [安全与网关指南](SECURITY_zh.md)。
- 浏览器上传应尽量靠近你的后端服务，不要把私有音频发送到未鉴权的公网端点。
- 排查问题时记录模型 alias、音频时长、延迟、响应格式和错误文本。
examples/openai_api/JAVASCRIPT.md
+180
View File
@@ -0,0 +1,180 @@
# JavaScript and TypeScript Recipes for the FunASR OpenAI-Compatible API
Use this guide when `funasr-server` is already running and you want to connect Node.js, TypeScript services, Next.js route handlers, or other JavaScript agent workflows to local speech recognition.
## Preflight
Start the API server first:
```bash
cd examples/openai_api
python server.py --model sensevoice --device cuda --port 8000
```
Then verify the service from another terminal:
```bash
python smoke_test.py --base-url http://localhost:8000
```
SDK base URLs include `/v1`; direct health checks do not:
```text
OpenAI SDK baseURL: http://localhost:8000/v1
Health endpoint: http://localhost:8000/health
Transcription URL: http://localhost:8000/v1/audio/transcriptions
```
## OpenAI JavaScript SDK
Install the official JavaScript SDK:
```bash
npm install openai
```
Create `transcribe.mjs`:
```javascript
import OpenAI from "openai";
import { createReadStream } from "node:fs";
const audioPath = process.argv[2] ?? "sample.wav";
const client = new OpenAI({
baseURL: process.env.FUNASR_OPENAI_BASE_URL ?? "http://localhost:8000/v1",
apiKey: process.env.OPENAI_API_KEY ?? "local-development",
});
const result = await client.audio.transcriptions.create({
model: process.env.FUNASR_MODEL ?? "sensevoice",
file: createReadStream(audioPath),
response_format: "verbose_json",
});
console.log(result.text);
for (const segment of result.segments ?? []) {
console.log(`${segment.start}s-${segment.end}s`, segment.text);
}
```
Run it:
```bash
node transcribe.mjs meeting.wav
```
Most OpenAI-compatible SDKs require an API key value even when the local FunASR server does not check it. Use any placeholder for local development, then add real authentication at your gateway if the service is shared.
## Built-in fetch without an SDK
Node.js 18+ includes `fetch`, `FormData`, and `Blob`, so you can call the API without third-party dependencies:
```javascript
import { readFile } from "node:fs/promises";
import { basename } from "node:path";
const baseUrl = process.env.FUNASR_BASE_URL ?? "http://localhost:8000";
const audioPath = process.argv[2] ?? "sample.wav";
const audio = await readFile(audioPath);
const form = new FormData();
form.append("file", new Blob([audio], { type: "audio/wav" }), basename(audioPath));
form.append("model", process.env.FUNASR_MODEL ?? "sensevoice");
form.append("response_format", "verbose_json");
const response = await fetch(`${baseUrl}/v1/audio/transcriptions`, {
method: "POST",
body: form,
});
if (!response.ok) {
throw new Error(`FunASR request failed: ${response.status} ${await response.text()}`);
}
const result = await response.json();
console.log(result.text);
```
Use this pattern for queue workers, webhook workers, scheduled jobs, and small internal services.
## TypeScript helper
```typescript
import OpenAI from "openai";
import { createReadStream } from "node:fs";
export interface FunASRTranscript {
text: string;
segments?: Array<{ start: number; end: number; text: string; speaker?: number }>;
language?: string;
duration?: number;
model?: string;
}
const client = new OpenAI({
baseURL: process.env.FUNASR_OPENAI_BASE_URL ?? "http://localhost:8000/v1",
apiKey: process.env.OPENAI_API_KEY ?? "local-development",
});
export async function transcribeWithFunASR(audioPath: string): Promise<FunASRTranscript> {
const result = await client.audio.transcriptions.create({
model: process.env.FUNASR_MODEL ?? "sensevoice",
file: createReadStream(audioPath),
response_format: "verbose_json",
});
return result as FunASRTranscript;
}
```
Keep the return type small and application-owned. FunASR can return richer metadata over time, and your application can opt into only the fields it needs.
## Next.js route handler
Proxy browser uploads through your backend so you can enforce authentication, file-size limits, and audit logs before audio reaches FunASR.
```typescript
export async function POST(request: Request) {
const incoming = await request.formData();
const file = incoming.get("file");
if (!(file instanceof File)) {
return Response.json({ error: "missing file" }, { status: 400 });
}
const upstream = new FormData();
upstream.append("file", file, file.name || "audio.wav");
upstream.append("model", "sensevoice");
upstream.append("response_format", "verbose_json");
const response = await fetch("http://funasr-api:8000/v1/audio/transcriptions", {
method: "POST",
body: upstream,
});
const body = await response.json();
return Response.json(body, { status: response.status });
}
```
In Docker Compose or Kubernetes, replace `funasr-api` with the service name reachable from your web backend. Avoid sending browser traffic directly to an unauthenticated FunASR endpoint on a public network.
## Production checklist
- Put TLS, authentication, upload-size limits, and rate limits in front of the API.
- Set request timeouts based on maximum audio duration; long recordings need longer HTTP timeouts.
- Log audio duration, model alias, response format, latency, and upstream error text.
- Run `GET /health` and `GET /v1/models` during readiness checks before accepting user uploads.
- Keep audio upload handling on the server side for browser applications.
- Pin `openai` package versions in production services and retest after SDK upgrades.
## Troubleshooting
| Symptom | Fix |
|---|---|
| SDK reports a missing API key | Pass any placeholder `apiKey` for local development. |
| 404 from SDK calls | Use `baseURL=http://localhost:8000/v1`; direct endpoint calls use `http://localhost:8000`. |
| `unknown model` | Call `/v1/models` and use one of the returned aliases. |
| Browser upload fails with CORS or auth errors | Send uploads to your backend first, then proxy to FunASR. |
| Request times out | Increase SDK or fetch timeouts, or split very long audio. |
examples/openai_api/JAVASCRIPT_zh.md
+180
View File
@@ -0,0 +1,180 @@
# FunASR OpenAI 兼容 API JavaScript/TypeScript 接入配方
当 `funasr-server` 已经启动，而你希望把 Node.js、TypeScript 服务、Next.js route handler 或 JavaScript Agent 工作流接入本地语音识别时，可以按这份指南复制代码。
## 预检查
先启动 API 服务：
```bash
cd examples/openai_api
python server.py --model sensevoice --device cuda --port 8000
```
在另一个终端验证服务：
```bash
python smoke_test.py --base-url http://localhost:8000
```
SDK base URL 需要包含 `/v1`，直接健康检查不需要：
```text
OpenAI SDK baseURL: http://localhost:8000/v1
健康检查: http://localhost:8000/health
转写接口: http://localhost:8000/v1/audio/transcriptions
```
## OpenAI JavaScript SDK
安装官方 JavaScript SDK：
```bash
npm install openai
```
创建 `transcribe.mjs`：
```javascript
import OpenAI from "openai";
import { createReadStream } from "node:fs";
const audioPath = process.argv[2] ?? "sample.wav";
const client = new OpenAI({
baseURL: process.env.FUNASR_OPENAI_BASE_URL ?? "http://localhost:8000/v1",
apiKey: process.env.OPENAI_API_KEY ?? "local-development",
});
const result = await client.audio.transcriptions.create({
model: process.env.FUNASR_MODEL ?? "sensevoice",
file: createReadStream(audioPath),
response_format: "verbose_json",
});
console.log(result.text);
for (const segment of result.segments ?? []) {
console.log(`${segment.start}s-${segment.end}s`, segment.text);
}
```
运行：
```bash
node transcribe.mjs meeting.wav
```
多数 OpenAI 兼容 SDK 即使在本地服务不校验密钥时，也要求传入一个 API key。开发环境可以使用任意占位值；如果服务被多人共享，请在网关层增加真实鉴权。
## 不依赖 SDK 的内置 fetch 写法
Node.js 18+ 内置 `fetch`、`FormData` 和 `Blob`，可以不安装第三方依赖直接调用接口：
```javascript
import { readFile } from "node:fs/promises";
import { basename } from "node:path";
const baseUrl = process.env.FUNASR_BASE_URL ?? "http://localhost:8000";
const audioPath = process.argv[2] ?? "sample.wav";
const audio = await readFile(audioPath);
const form = new FormData();
form.append("file", new Blob([audio], { type: "audio/wav" }), basename(audioPath));
form.append("model", process.env.FUNASR_MODEL ?? "sensevoice");
form.append("response_format", "verbose_json");
const response = await fetch(`${baseUrl}/v1/audio/transcriptions`, {
method: "POST",
body: form,
});
if (!response.ok) {
throw new Error(`FunASR request failed: ${response.status} ${await response.text()}`);
}
const result = await response.json();
console.log(result.text);
```
这个模式适合队列 worker、webhook worker、定时任务和小型内部服务。
## TypeScript helper
```typescript
import OpenAI from "openai";
import { createReadStream } from "node:fs";
export interface FunASRTranscript {
text: string;
segments?: Array<{ start: number; end: number; text: string; speaker?: number }>;
language?: string;
duration?: number;
model?: string;
}
const client = new OpenAI({
baseURL: process.env.FUNASR_OPENAI_BASE_URL ?? "http://localhost:8000/v1",
apiKey: process.env.OPENAI_API_KEY ?? "local-development",
});
export async function transcribeWithFunASR(audioPath: string): Promise<FunASRTranscript> {
const result = await client.audio.transcriptions.create({
model: process.env.FUNASR_MODEL ?? "sensevoice",
file: createReadStream(audioPath),
response_format: "verbose_json",
});
return result as FunASRTranscript;
}
```
建议在应用侧维护一个小而稳定的返回类型。FunASR 后续可能返回更丰富的元数据，业务代码只需要消费自己关心的字段。
## Next.js route handler
浏览器上传建议先进入自己的后端，再由后端转发到 FunASR，这样可以统一做鉴权、文件大小限制和审计日志。
```typescript
export async function POST(request: Request) {
const incoming = await request.formData();
const file = incoming.get("file");
if (!(file instanceof File)) {
return Response.json({ error: "missing file" }, { status: 400 });
}
const upstream = new FormData();
upstream.append("file", file, file.name || "audio.wav");
upstream.append("model", "sensevoice");
upstream.append("response_format", "verbose_json");
const response = await fetch("http://funasr-api:8000/v1/audio/transcriptions", {
method: "POST",
body: upstream,
});
const body = await response.json();
return Response.json(body, { status: response.status });
}
```
在 Docker Compose 或 Kubernetes 中，把 `funasr-api` 换成 Web 后端能访问到的 service name。不要把未鉴权的 FunASR 接口直接暴露给公网浏览器。
## 生产检查清单
- 在 API 前增加 TLS、鉴权、上传大小限制和限流。
- 根据最大音频时长设置请求超时；长录音需要更长的 HTTP timeout。
- 记录音频时长、模型别名、响应格式、延迟和上游错误文本。
- 接收用户上传前，先用 `GET /health` 和 `GET /v1/models` 做就绪检查。
- 浏览器应用应把音频上传处理留在服务端。
- 生产服务固定 `openai` 包版本，并在 SDK 升级后重新测试。
## 故障排查
| 现象 | 处理方式 |
|---|---|
| SDK 提示缺少 API key | 本地开发传入任意占位 `apiKey`。 |
| SDK 调用返回 404 | SDK 使用 `baseURL=http://localhost:8000/v1`；直接端点调用使用 `http://localhost:8000`。 |
| `unknown model` | 调用 `/v1/models`，使用返回的模型别名。 |
| 浏览器上传遇到 CORS 或鉴权错误 | 先上传到自己的后端，再由后端代理到 FunASR。 |
| 请求超时 | 增加 SDK 或 fetch 超时，或切分超长音频。 |
examples/openai_api/OPENAPI.md
+53
View File
@@ -0,0 +1,53 @@
# OpenAPI Spec for the FunASR OpenAI-Compatible API
(English|[简体中文](OPENAPI_zh.md))
Use [`openapi.json`](openapi.json) when you want to inspect, mock, document, or import the FunASR speech API before wiring it into an application, API gateway, workflow engine, or SDK generator.
The running FastAPI server also exposes live docs at `/docs` and a generated schema at `/openapi.json`. This checked-in spec is a portable reference for the example server in this directory.
## Import options
| Tool | How to use it |
|---|---|
| Swagger Editor or Redoc | Import `openapi.json` to inspect `/health`, `/v1/models`, and `/v1/audio/transcriptions`. |
| Postman | Import `openapi.json` if you prefer schema-driven collections, or use the ready-made [Postman collection](POSTMAN.md). |
| Dify, n8n, or internal workflow tools | Use the multipart request shape in the spec together with the [workflow recipes](WORKFLOWS.md). |
| API gateway or internal developer portal | Publish the spec and point the server URL to your reachable FunASR API endpoint. |
| Client generation | Generate a small internal client, then keep the multipart `file` field mapped to a binary upload. |
## Server URL
The spec includes local examples:
- `http://localhost:8000`
- `http://funasr-api:8000`
Replace them with the URL reachable from your app, container, or workflow runtime.
## Endpoints
| Endpoint | Method | Purpose |
|---|---|---|
| `/health` | `GET` | Readiness check, selected device, loaded models, and available aliases. |
| `/v1/models` | `GET` | OpenAI-style model list with `ready` flags. |
| `/v1/audio/transcriptions` | `POST` | Multipart audio transcription. Use `response_format=verbose_json` for segments. |
## Multipart transcription fields
| Field | Type | Required | Notes |
|---|---|---|---|
| `file` | binary | yes | Audio file such as wav, mp3, flac, m4a, ogg, or webm. |
| `model` | string | no | Defaults to `sensevoice`; available aliases are listed by `/v1/models`. |
| `language` | string | no | Optional language hint. |
| `response_format` | string | no | Use `json` or `verbose_json`. |
## Validate against a running server
```bash
cd examples/openai_api
python server.py --model sensevoice --device cuda --port 8000
curl -fsS http://localhost:8000/openapi.json > /tmp/funasr-openapi-live.json
```
The live FastAPI schema may include framework-specific validation details; this checked-in spec keeps the public integration surface small and stable.
examples/openai_api/OPENAPI_zh.md
+53
View File
@@ -0,0 +1,53 @@
# FunASR OpenAI 兼容 API OpenAPI 规范
[English](OPENAPI.md)
当你希望在接入应用、API 网关、开发者门户、工作流引擎或 SDK 生成器之前，先查看、mock、导入或发布 FunASR 语音 API 时，可以使用 [`openapi.json`](openapi.json)。
运行中的 FastAPI 服务也会在 `/docs` 暴露 Swagger UI，并在 `/openapi.json` 暴露实时 schema。仓库里的 `openapi.json` 是这个示例服务的便携参考规范，便于在服务启动前完成评估和导入。
## 导入方式
| 工具 | 使用方式 |
|---|---|
| Swagger Editor 或 Redoc | 导入 `openapi.json`，查看 `/health`、`/v1/models` 和 `/v1/audio/transcriptions`。 |
| Postman | 如果偏好 schema 驱动集合，可导入 `openapi.json`；如果想直接 smoke test，可使用现成的 [Postman 集合](POSTMAN_zh.md)。 |
| Dify、n8n 或内部工作流工具 | 结合规范中的 multipart 请求结构和 [工作流配方](WORKFLOWS_zh.md) 配置 HTTP 节点。 |
| API 网关或内部开发者门户 | 发布该规范，并把 server URL 改成你的 FunASR API 可访问地址。 |
| 客户端生成 | 生成内部小客户端，并确保 multipart `file` 字段映射为二进制上传。 |
## Server URL
规范内置了两个示例地址：
- `http://localhost:8000`
- `http://funasr-api:8000`
请替换为你的应用、容器或工作流运行环境能访问到的地址。
## 端点
| Endpoint | Method | 用途 |
|---|---|---|
| `/health` | `GET` | 健康检查、设备、已加载模型和可用别名。 |
| `/v1/models` | `GET` | OpenAI 风格模型列表，包含 `ready` 状态。 |
| `/v1/audio/transcriptions` | `POST` | multipart 音频转写；使用 `response_format=verbose_json` 返回 segments。 |
## Multipart 转写字段
| Field | Type | Required | 说明 |
|---|---|---|---|
| `file` | binary | yes | 音频文件，例如 wav、mp3、flac、m4a、ogg 或 webm。 |
| `model` | string | no | 默认 `sensevoice`；可用别名由 `/v1/models` 返回。 |
| `language` | string | no | 可选语言提示。 |
| `response_format` | string | no | 使用 `json` 或 `verbose_json`。 |
## 对照运行中的服务验证
```bash
cd examples/openai_api
python server.py --model sensevoice --device cuda --port 8000
curl -fsS http://localhost:8000/openapi.json > /tmp/funasr-openapi-live.json
```
实时 FastAPI schema 可能包含框架级校验细节；仓库中的静态规范保留更小、更稳定的公开集成面。
examples/openai_api/POSTMAN.md
+39
View File
@@ -0,0 +1,39 @@
# Postman Collection for the FunASR OpenAI-Compatible API
(English|[简体中文](POSTMAN_zh.md))
Use the Postman collection when you want to verify a private FunASR speech API before wiring it into Dify, n8n, an agent framework, or an internal service. If you prefer schema-driven imports, use the [OpenAPI spec](OPENAPI.md).
## Import
1. Start the server:
```bash
cd examples/openai_api
python server.py --model sensevoice --device cuda --port 8000
```
2. Import [`funasr-openai-api.postman_collection.json`](funasr-openai-api.postman_collection.json) into Postman.
3. Set the collection variable `FUNASR_BASE_URL` to the reachable server URL, for example `http://localhost:8000` or `http://funasr-api:8000`.
4. Keep `MODEL_ALIAS=sensevoice` for the first smoke test, or switch it to one of the aliases returned by `/v1/models`.
## Requests
| Request | Purpose |
|---|---|
| `Health check` | Confirms the server is reachable and returns JSON. |
| `List model aliases` | Shows available OpenAI-compatible model aliases. |
| `Transcribe audio - verbose JSON` | Uploads an audio file and returns `text`, `segments`, and timing metadata. |
| `Transcribe audio - text only` | Minimal transcription request for OpenAI-compatible clients. |
For the transcription requests, open the `Body` tab and choose a local audio file for the `file` form-data field before sending.
## Troubleshooting
| Symptom | Fix |
|---|---|
| `ECONNREFUSED` | Confirm the server is running and that `FUNASR_BASE_URL` is reachable from Postman. |
| Docker service works but Postman cannot connect | Use the host port exposed by Docker Compose, for example `http://localhost:8000`. |
| `422` or missing file errors | Make sure the `file` form-data row is enabled and points to a local audio file. |
| Unknown model alias | Run `List model aliases` and copy one of the returned aliases into `MODEL_ALIAS`. |
| No `segments` in the response | Set `RESPONSE_FORMAT=verbose_json`. |
examples/openai_api/POSTMAN_zh.md
+39
View File
@@ -0,0 +1,39 @@
# FunASR OpenAI 兼容 API Postman 集合
[English](POSTMAN.md)
当你希望先用图形界面验证私有 FunASR 语音 API，再接入 Dify、n8n、Agent 框架或内部服务时，可以导入 Postman collection。需要按 schema 导入时，可使用 [OpenAPI 规范](OPENAPI_zh.md)。
## 导入步骤
1. 启动服务：
```bash
cd examples/openai_api
python server.py --model sensevoice --device cuda --port 8000
```
2. 在 Postman 中导入 [`funasr-openai-api.postman_collection.json`](funasr-openai-api.postman_collection.json)。
3. 将 collection 变量 `FUNASR_BASE_URL` 改成可访问的服务地址，例如 `http://localhost:8000` 或 `http://funasr-api:8000`。
4. 第一次 smoke test 建议保持 `MODEL_ALIAS=sensevoice`；也可以先运行 `/v1/models`，再复制返回的模型别名。
## 请求列表
| Request | 用途 |
|---|---|
| `Health check` | 确认服务可访问并返回 JSON。 |
| `List model aliases` | 查看服务暴露的 OpenAI 兼容模型别名。 |
| `Transcribe audio - verbose JSON` | 上传音频并返回 `text`、`segments` 和耗时信息。 |
| `Transcribe audio - text only` | 最小转写请求，适合验证 OpenAI 兼容客户端。 |
发送转写请求前，请打开 `Body` tab，在 `file` form-data 字段选择本地音频文件。
## 故障排查
| 现象 | 处理方式 |
|---|---|
| `ECONNREFUSED` | 确认服务已启动，并且 Postman 能访问 `FUNASR_BASE_URL`。 |
| Docker 服务正常但 Postman 连不上 | 使用 Docker Compose 暴露到宿主机的端口，例如 `http://localhost:8000`。 |
| `422` 或提示缺少文件 | 确认 `file` form-data 行已启用，并指向本地音频文件。 |
| 模型别名未知 | 先运行 `List model aliases`，再把返回的别名填入 `MODEL_ALIAS`。 |
| 响应中没有 `segments` | 设置 `RESPONSE_FORMAT=verbose_json`。 |
examples/openai_api/README.md
+192
View File
@@ -0,0 +1,192 @@
(English|[简体中文](README_zh.md)|[日本語](README_ja.md)|[한국어](README_ko.md))
# FunASR OpenAI-Compatible API Server
Drop-in replacement for OpenAI's `/v1/audio/transcriptions` endpoint. Works with **any agent framework** that supports OpenAI audio API.
## Quick Start
```bash
pip install funasr fastapi uvicorn python-multipart
python server.py --model sensevoice --device cuda --port 8000
```
Server starts in ~20s (model loading). Health check: `GET /health`
Need copy-paste integration snippets for Python SDK, JavaScript/TypeScript, HTTP clients, agent tools, a browser demo, Postman, OpenAPI imports, Kubernetes deployment, or Dify/n8n-style workflows? See [Client recipes](CLIENTS.md), [JavaScript/TypeScript recipes](JAVASCRIPT.md), [Gradio browser demo](GRADIO.md), [workflow recipes](WORKFLOWS.md), the [Chinese workflow recipes](WORKFLOWS_zh.md), the [Postman collection](POSTMAN.md), the [OpenAPI spec](OPENAPI.md), the [security and gateway guide](SECURITY.md), and the [Kubernetes deployment template](kubernetes/README.md).
### End-to-end smoke test
In another terminal, download a public sample and verify both health and transcription:
```bash
bash smoke_test.sh
# Cross-platform alternative without curl/bash:
python smoke_test.py
```
Equivalent manual commands:
```bash
curl -L https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/BAC009S0764W0121.wav -o sample.wav
curl http://localhost:8000/health
curl http://localhost:8000/v1/audio/transcriptions \
-F file=@sample.wav \
-F model=sensevoice \
-F response_format=verbose_json
```
## Browser demo with Gradio
If you want a local browser UI for upload or microphone testing, run the API server first and then launch the optional Gradio frontend:
```bash
pip install gradio
python gradio_app.py --base-url http://localhost:8000
```
The browser demo calls the same OpenAI-compatible API endpoints as the smoke tests. See [Gradio browser demo](GRADIO.md) for Docker, Kubernetes, and production notes.
## Usage with OpenAI SDK (Python)
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
# Basic transcription
result = client.audio.transcriptions.create(
model="sensevoice", # or "paraformer", "paraformer-en", "fun-asr-nano"
file=open("meeting.wav", "rb"),
)
print(result.text)
# With timestamps/segments
result = client.audio.transcriptions.create(
model="sensevoice",
file=open("meeting.wav", "rb"),
response_format="verbose_json",
)
# Returns: text, segments (with start/end/speaker), duration
```
## Usage with curl
```bash
curl http://localhost:8000/v1/audio/transcriptions \
-F file=@audio.wav \
-F model=sensevoice
# With verbose output
curl http://localhost:8000/v1/audio/transcriptions \
-F file=@audio.wav \
-F model=sensevoice \
-F response_format=verbose_json
```
## Available Models
| Model | Speed (GPU) | Speed (CPU) | Languages | Features |
|-------|-------------|-------------|-----------|----------|
| `sensevoice` | 170x realtime | 17x realtime | zh/en/ja/ko/yue | Emotion detection |
| `paraformer` | 120x realtime | 15x realtime | zh/en | Punctuation |
| `paraformer-en` | 120x realtime | 15x realtime | en | English only |
| `fun-asr-nano` | 17x realtime | 3.6x realtime | 31 languages | LLM-based, timestamps |
## API Endpoints
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/v1/audio/transcriptions` | POST | Transcribe audio (OpenAI-compatible) |
| `/v1/models` | GET | List available models |
| `/health` | GET | Health check + loaded models |
| `/docs` | GET | Interactive API documentation (Swagger) |
Prefer no-code API checks? Use the [Gradio browser demo](GRADIO.md) for local upload or microphone testing, or import the [Postman collection](POSTMAN.md) and run health, model-list, and transcription requests from Postman. For API gateways, developer portals, or client generation, use the [OpenAPI spec](OPENAPI.md).
## Agent Framework Integration
Works with: **LangChain**, **LlamaIndex**, **AutoGen**, **CrewAI**, **Semantic Kernel**, **Dify**, **n8n**, or any framework using OpenAI audio API. See [Client recipes](CLIENTS.md) and [JavaScript/TypeScript recipes](JAVASCRIPT.md) for SDK and agent-tool patterns, plus [workflow recipes](WORKFLOWS.md) for low-code HTTP nodes and webhook workers ([中文](WORKFLOWS_zh.md)).
### LangChain Example
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="x")
def transcribe_for_agent(audio_path: str) -> str:
"""Tool function for LangChain agent."""
result = client.audio.transcriptions.create(
model="sensevoice", file=open(audio_path, "rb")
)
return result.text
```
## Docker Deployment
Build the example image from this directory. The default image starts in CPU mode so it can be used as a portable smoke test.
```bash
cd examples/openai_api
cp .env.example .env
docker compose up --build
```
Equivalent one-off `docker run` command:
```bash
docker build -t funasr-api .
docker run --rm -p 8000:8000 \
-e FUNASR_DEVICE=cpu \
-e FUNASR_MODEL=sensevoice \
funasr-api
```
For GPU hosts, use NVIDIA Container Toolkit and a CUDA-capable PyTorch/FunASR image. After adapting the image dependencies for CUDA, run the same server with `FUNASR_DEVICE=cuda`:
```bash
docker run --rm --gpus all -p 8000:8000 \
-e FUNASR_DEVICE=cuda \
-e FUNASR_MODEL=sensevoice \
funasr-api
```
Verify the container from another terminal:
```bash
BASE_URL=http://localhost:8000 bash smoke_test.sh
python smoke_test.py --base-url http://localhost:8000
```
## Kubernetes Deployment
Before sharing the service across a team or exposing it through a gateway, review the [security and gateway guide](SECURITY.md) for TLS, authentication, upload limits, rate limits, and logging.
For an internal cluster service with persistent model cache, health probes, and a private `ClusterIP`, start from the [Kubernetes deployment template](kubernetes/README.md). Build and push the example image, apply the manifests, then verify through `kubectl port-forward` with `python smoke_test.py --base-url http://localhost:8000`.
Keep the default CPU mode until you have built a CUDA-capable image and configured GPU scheduling for your cluster.
## Configuration
| Arg | Default | Description |
|-----|---------|-------------|
| `--host` | 0.0.0.0 | Bind address |
| `--port` | 8000 | Port |
| `--device` | cuda | Device (cuda/cpu/mps) |
| `--model` | sensevoice | Pre-load model at startup |
Docker environment variables:
| Env | Default | Description |
|-----|---------|-------------|
| `FUNASR_PORT` | 8000 | Container port passed to `server.py` |
| `FUNASR_DEVICE` | cpu | Container device mode; set to `cuda` only when the image has CUDA-capable dependencies |
| `FUNASR_MODEL` | sensevoice | Model alias loaded at container startup |
## Troubleshooting
- If CUDA is unavailable, use `--device cpu` for a slower but simple smoke test.
- If port 8000 is occupied, start with `--port 9000` and run `BASE_URL=http://localhost:9000 bash smoke_test.sh` or `python smoke_test.py --base-url http://localhost:9000`.
- If model download is slow, retry with a stable network or pre-download the model from ModelScope/Hugging Face.
examples/openai_api/README_ja.md
+184
View File
@@ -0,0 +1,184 @@
([English](README.md)|[简体中文](README_zh.md)|日本語|[한국어](README_ko.md))
# FunASR OpenAI 互換 API サーバー
FunASR OpenAI 互換 API は `/v1/audio/transcriptions` を提供します。OpenAI スタイルの SDK、エージェントフレームワーク、Dify、n8n、HTTP ノード、社内システムから、プライベートな音声認識サービスとして利用できます。
## クイックスタート
```bash
pip install funasr fastapi uvicorn python-multipart
python server.py --model sensevoice --device cuda --port 8000
```
モデルのロード後にサービスが起動します。ヘルスチェックは `GET /health` です。
コピーして使える連携例が必要な場合は、[クライアントレシピ](CLIENTS.md)、[JavaScript/TypeScript レシピ](JAVASCRIPT.md)、[Gradio ブラウザデモ](GRADIO.md)、[ワークフローレシピ](WORKFLOWS.md)、[Postman コレクション](POSTMAN.md)、[OpenAPI 仕様](OPENAPI.md)、[セキュリティとゲートウェイガイド](SECURITY.md)、[Kubernetes デプロイテンプレート](kubernetes/README.md)を参照してください。
### エンドツーエンド smoke test
別のターミナルで実行します。
```bash
bash smoke_test.sh
# curl/bash を使わないクロスプラットフォーム版:
python smoke_test.py
```
同等の手動コマンド:
```bash
curl -L https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/BAC009S0764W0121.wav -o sample.wav
curl http://localhost:8000/health
curl http://localhost:8000/v1/audio/transcriptions \
-F file=@sample.wav \
-F model=sensevoice \
-F response_format=verbose_json
```
## Gradio ブラウザデモ
ローカルブラウザで音声ファイルのアップロードやマイク入力を試したい場合は、先に API サーバーを起動し、オプションの Gradio フロントエンドを起動します。
```bash
pip install gradio
python gradio_app.py --base-url http://localhost:8000
```
このブラウザデモは smoke test と同じ OpenAI 互換 API エンドポイントを呼び出します。Docker、Kubernetes、本番利用の注意点は [Gradio ブラウザデモ](GRADIO.md)を参照してください。
## OpenAI SDK で使う
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
result = client.audio.transcriptions.create(
model="sensevoice", # "paraformer", "paraformer-en", "fun-asr-nano" も利用できます
file=open("meeting.wav", "rb"),
)
print(result.text)
verbose = client.audio.transcriptions.create(
model="sensevoice",
file=open("meeting.wav", "rb"),
response_format="verbose_json",
)
print(verbose.segments)
```
## curl で使う
```bash
curl http://localhost:8000/v1/audio/transcriptions \
-F file=@audio.wav \
-F model=sensevoice
curl http://localhost:8000/v1/audio/transcriptions \
-F file=@audio.wav \
-F model=sensevoice \
-F response_format=verbose_json
```
## 利用できるモデル
| Model | GPU 速度 | CPU 速度 | 言語 | 特徴 |
|---|---|---|---|---|
| `sensevoice` | 170x realtime | 17x realtime | zh/en/ja/ko/yue | 感情・イベントタグ |
| `paraformer` | 120x realtime | 15x realtime | zh/en | 句読点復元 |
| `paraformer-en` | 120x realtime | 15x realtime | en | 英語認識 |
| `fun-asr-nano` | 17x realtime | 3.6x realtime | 31 languages | LLM-based、タイムスタンプ |
## API エンドポイント
| Endpoint | Method | 説明 |
|---|---|---|
| `/v1/audio/transcriptions` | POST | OpenAI 互換の音声文字起こし |
| `/v1/models` | GET | モデルエイリアスの一覧 |
| `/health` | GET | ヘルスチェック、ロード済みモデル、利用可能モデル |
| `/docs` | GET | FastAPI Swagger ドキュメント |
コードを書かずに確認したい場合は、[Gradio ブラウザデモ](GRADIO.md)でローカルアップロードやマイク入力を試すか、[Postman コレクション](POSTMAN.md)をインポートしてください。API ゲートウェイ、開発者ポータル、社内クライアント生成には [OpenAPI 仕様](OPENAPI.md)を利用できます。
## エージェントとローコードワークフロー
**LangChain**、**LlamaIndex**、**AutoGen**、**CrewAI**、**Semantic Kernel**、**Dify**、**n8n**、OpenAI audio API または multipart HTTP を使える任意のシステムで利用できます。
- SDK、JavaScript/TypeScript、Agent tool の書き方は [クライアントレシピ](CLIENTS.md) と [JavaScript/TypeScript レシピ](JAVASCRIPT.md)を参照してください。
- Dify、n8n、HTTP ノード、webhook worker は [ワークフローレシピ](WORKFLOWS.md)を参照してください。
- GUI smoke test は [Postman コレクション](POSTMAN.md)を参照してください。
- schema-driven import には [OpenAPI 仕様](OPENAPI.md)を使えます。
## Docker デプロイ
デフォルトのイメージは CPU モードで起動し、再現しやすい smoke test として使えます。
```bash
cd examples/openai_api
cp .env.example .env
docker compose up --build
```
同等の `docker run`:
```bash
docker build -t funasr-api .
docker run --rm -p 8000:8000 \
-e FUNASR_DEVICE=cpu \
-e FUNASR_MODEL=sensevoice \
funasr-api
```
GPU ホストでは NVIDIA Container Toolkit と CUDA 対応の PyTorch/FunASR イメージが必要です。CUDA 依存関係に合わせてイメージを調整した後、次のように起動できます。
```bash
docker run --rm --gpus all -p 8000:8000 \
-e FUNASR_DEVICE=cuda \
-e FUNASR_MODEL=sensevoice \
funasr-api
```
コンテナの検証:
```bash
BASE_URL=http://localhost:8000 bash smoke_test.sh
python smoke_test.py --base-url http://localhost:8000
```
## Kubernetes デプロイ
チーム内で共有したりゲートウェイ経由で公開したりする前に、[セキュリティとゲートウェイガイド](SECURITY.md)を確認し、TLS、認証、アップロード制限、レート制限、ログ方針を整えてください。
永続化されたモデルキャッシュ、ヘルスプローブ、プライベート `ClusterIP` を持つ内部クラスタサービスが必要な場合は、[Kubernetes デプロイテンプレート](kubernetes/README.md)から始めてください。サンプルイメージをビルドして push し、manifests を適用した後、`kubectl port-forward` と `python smoke_test.py --base-url http://localhost:8000` で検証します。
CUDA 対応イメージと GPU スケジューリング設定が整うまでは、デフォルトの CPU モードを維持してください。
## 設定
| 引数 | デフォルト | 説明 |
|---|---|---|
| `--host` | `0.0.0.0` | バインドアドレス |
| `--port` | `8000` | ポート |
| `--device` | `cuda` | `cuda`、`cpu`、`mps` |
| `--model` | `sensevoice` | 起動時にプリロードするモデル |
Docker 環境変数:
| Env | デフォルト | 説明 |
|---|---|---|
| `FUNASR_PORT` | `8000` | `server.py` に渡すコンテナポート |
| `FUNASR_DEVICE` | `cpu` | コンテナのデバイスモード。CUDA 対応依存関係を持つイメージでのみ `cuda` に設定してください |
| `FUNASR_MODEL` | `sensevoice` | コンテナ起動時にロードするモデルエイリアス |
## トラブルシューティング
| 症状 | 対処 |
|---|---|
| CUDA が利用できない | まず `--device cpu` で smoke test を通します。 |
| 8000 ポートが使用中 | `--port 9000` に変更し、`BASE_URL=http://localhost:9000 bash smoke_test.sh` または `python smoke_test.py --base-url http://localhost:9000` を実行します。 |
| モデルのダウンロードが遅い | 安定したネットワークで再試行するか、ModelScope/Hugging Face から事前にモデルをダウンロードします。 |
| Dify/n8n コンテナから `localhost` に接続できない | ワークフロー実行環境から到達できるホスト名、Compose service name、または Kubernetes service name を使います。 |
| 応答に `segments` がない | `response_format=verbose_json` を設定します。 |
examples/openai_api/README_ko.md
+184
View File
@@ -0,0 +1,184 @@
([English](README.md)|[简体中文](README_zh.md)|[日本語](README_ja.md)|한국어)
# FunASR OpenAI 호환 API 서버
FunASR OpenAI 호환 API는 `/v1/audio/transcriptions`를 제공합니다. OpenAI 스타일 SDK, 에이전트 프레임워크, Dify, n8n, HTTP 노드, 내부 업무 시스템에서 프라이빗 음성 인식 서비스로 사용할 수 있습니다.
## 빠른 시작
```bash
pip install funasr fastapi uvicorn python-multipart
python server.py --model sensevoice --device cuda --port 8000
```
모델 로드가 끝나면 서비스가 시작됩니다. 헬스 체크는 `GET /health`입니다.
바로 복사해서 쓸 수 있는 연동 예제가 필요하면 [클라이언트 레시피](CLIENTS.md), [JavaScript/TypeScript 레시피](JAVASCRIPT.md), [Gradio 브라우저 데모](GRADIO.md), [워크플로 레시피](WORKFLOWS.md), [Postman 컬렉션](POSTMAN.md), [OpenAPI 명세](OPENAPI.md), [보안 및 게이트웨이 가이드](SECURITY.md), [Kubernetes 배포 템플릿](kubernetes/README.md)을 참고하세요.
### 엔드투엔드 smoke test
다른 터미널에서 실행합니다.
```bash
bash smoke_test.sh
# curl/bash가 없는 환경을 위한 크로스 플랫폼 방식:
python smoke_test.py
```
동일한 수동 명령:
```bash
curl -L https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/BAC009S0764W0121.wav -o sample.wav
curl http://localhost:8000/health
curl http://localhost:8000/v1/audio/transcriptions \
-F file=@sample.wav \
-F model=sensevoice \
-F response_format=verbose_json
```
## Gradio 브라우저 데모
로컬 브라우저에서 파일 업로드나 마이크 입력을 테스트하려면 먼저 API 서버를 시작한 뒤 선택 사항인 Gradio 프런트엔드를 실행합니다.
```bash
pip install gradio
python gradio_app.py --base-url http://localhost:8000
```
이 브라우저 데모는 smoke test와 동일한 OpenAI 호환 API 엔드포인트를 호출합니다. Docker, Kubernetes, 프로덕션 참고 사항은 [Gradio 브라우저 데모](GRADIO.md)를 확인하세요.
## OpenAI SDK로 사용하기
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
result = client.audio.transcriptions.create(
model="sensevoice", # "paraformer", "paraformer-en", "fun-asr-nano"도 사용할 수 있습니다
file=open("meeting.wav", "rb"),
)
print(result.text)
verbose = client.audio.transcriptions.create(
model="sensevoice",
file=open("meeting.wav", "rb"),
response_format="verbose_json",
)
print(verbose.segments)
```
## curl로 사용하기
```bash
curl http://localhost:8000/v1/audio/transcriptions \
-F file=@audio.wav \
-F model=sensevoice
curl http://localhost:8000/v1/audio/transcriptions \
-F file=@audio.wav \
-F model=sensevoice \
-F response_format=verbose_json
```
## 사용 가능한 모델
| Model | GPU 속도 | CPU 속도 | 언어 | 특징 |
|---|---|---|---|---|
| `sensevoice` | 170x realtime | 17x realtime | zh/en/ja/ko/yue | 감정 및 이벤트 태그 |
| `paraformer` | 120x realtime | 15x realtime | zh/en | 문장부호 복원 |
| `paraformer-en` | 120x realtime | 15x realtime | en | 영어 인식 |
| `fun-asr-nano` | 17x realtime | 3.6x realtime | 31 languages | LLM-based, 타임스탬프 |
## API 엔드포인트
| Endpoint | Method | 설명 |
|---|---|---|
| `/v1/audio/transcriptions` | POST | OpenAI 호환 음성 전사 |
| `/v1/models` | GET | 모델 별칭 목록 |
| `/health` | GET | 헬스 체크, 로드된 모델, 사용 가능한 모델 |
| `/docs` | GET | FastAPI Swagger 문서 |
코드 작성 없이 확인하려면 [Gradio 브라우저 데모](GRADIO.md)로 로컬 업로드나 마이크 테스트를 진행하거나 [Postman 컬렉션](POSTMAN.md)을 가져오세요. API 게이트웨이, 개발자 포털, 내부 클라이언트 생성을 위해서는 [OpenAPI 명세](OPENAPI.md)를 사용할 수 있습니다.
## 에이전트 및 로우코드 워크플로
**LangChain**, **LlamaIndex**, **AutoGen**, **CrewAI**, **Semantic Kernel**, **Dify**, **n8n**, OpenAI audio API 또는 multipart HTTP를 지원하는 모든 시스템에서 사용할 수 있습니다.
- SDK, JavaScript/TypeScript, Agent tool 작성법은 [클라이언트 레시피](CLIENTS.md)와 [JavaScript/TypeScript 레시피](JAVASCRIPT.md)를 참고하세요.
- Dify, n8n, HTTP 노드, webhook worker는 [워크플로 레시피](WORKFLOWS.md)를 참고하세요.
- GUI smoke test는 [Postman 컬렉션](POSTMAN.md)을 참고하세요.
- schema 기반 가져오기는 [OpenAPI 명세](OPENAPI.md)를 사용할 수 있습니다.
## Docker 배포
기본 이미지는 CPU 모드로 시작하므로 재현 가능한 smoke test로 사용할 수 있습니다.
```bash
cd examples/openai_api
cp .env.example .env
docker compose up --build
```
동일한 `docker run` 명령:
```bash
docker build -t funasr-api .
docker run --rm -p 8000:8000 \
-e FUNASR_DEVICE=cpu \
-e FUNASR_MODEL=sensevoice \
funasr-api
```
GPU 호스트에서는 NVIDIA Container Toolkit과 CUDA 지원 PyTorch/FunASR 이미지가 필요합니다. CUDA 의존성에 맞게 이미지를 조정한 뒤 다음처럼 실행할 수 있습니다.
```bash
docker run --rm --gpus all -p 8000:8000 \
-e FUNASR_DEVICE=cuda \
-e FUNASR_MODEL=sensevoice \
funasr-api
```
컨테이너 검증:
```bash
BASE_URL=http://localhost:8000 bash smoke_test.sh
python smoke_test.py --base-url http://localhost:8000
```
## Kubernetes 배포
팀에서 공유하거나 게이트웨이를 통해 노출하기 전에 [보안 및 게이트웨이 가이드](SECURITY.md)를 검토하고 TLS, 인증, 업로드 제한, 속도 제한, 로그 정책을 준비하세요.
영구 모델 캐시, 헬스 프로브, 프라이빗 `ClusterIP`를 갖춘 내부 클러스터 서비스가 필요하다면 [Kubernetes 배포 템플릿](kubernetes/README.md)에서 시작하세요. 예제 이미지를 빌드하고 push한 뒤 manifests를 적용하고, `kubectl port-forward`와 `python smoke_test.py --base-url http://localhost:8000`으로 검증합니다.
CUDA 지원 이미지와 GPU 스케줄링 설정이 준비되기 전까지는 기본 CPU 모드를 유지하세요.
## 설정
| 인자 | 기본값 | 설명 |
|---|---|---|
| `--host` | `0.0.0.0` | 바인드 주소 |
| `--port` | `8000` | 포트 |
| `--device` | `cuda` | `cuda`, `cpu`, `mps` |
| `--model` | `sensevoice` | 시작 시 미리 로드할 모델 |
Docker 환경 변수:
| Env | 기본값 | 설명 |
|---|---|---|
| `FUNASR_PORT` | `8000` | `server.py`로 전달되는 컨테이너 포트 |
| `FUNASR_DEVICE` | `cpu` | 컨테이너 디바이스 모드. CUDA 지원 의존성이 포함된 이미지에서만 `cuda`로 설정하세요 |
| `FUNASR_MODEL` | `sensevoice` | 컨테이너 시작 시 로드할 모델 별칭 |
## 문제 해결
| 증상 | 해결 방법 |
|---|---|
| CUDA를 사용할 수 없음 | 먼저 `--device cpu`로 smoke test를 통과시키세요. |
| 8000 포트가 사용 중 | `--port 9000`으로 바꾸고 `BASE_URL=http://localhost:9000 bash smoke_test.sh` 또는 `python smoke_test.py --base-url http://localhost:9000`을 실행하세요. |
| 모델 다운로드가 느림 | 안정적인 네트워크에서 다시 시도하거나 ModelScope/Hugging Face에서 모델을 미리 다운로드하세요. |
| Dify/n8n 컨테이너에서 `localhost` 접속 실패 | 워크플로 런타임에서 접근 가능한 호스트명, Compose service name 또는 Kubernetes service name을 사용하세요. |
| 응답에 `segments`가 없음 | `response_format=verbose_json`를 설정하세요. |
examples/openai_api/README_zh.md
+184
View File
@@ -0,0 +1,184 @@
([English](README.md)|简体中文|[日本語](README_ja.md)|[한국어](README_ko.md))
# FunASR OpenAI 兼容 API 服务
FunASR OpenAI 兼容 API 提供 `/v1/audio/transcriptions`，可作为私有语音转写服务接入 OpenAI 风格 SDK、Agent 框架、Dify、n8n、HTTP 节点和内部业务系统。
## 快速开始
```bash
pip install funasr fastapi uvicorn python-multipart
python server.py --model sensevoice --device cuda --port 8000
```
服务通常会在模型加载后启动。健康检查：`GET /health`。
需要直接复制的接入示例？可以继续查看 [客户端配方](CLIENTS.md)、[JavaScript/TypeScript 配方](JAVASCRIPT_zh.md)、[Gradio 浏览器 Demo](GRADIO_zh.md)、[工作流配方](WORKFLOWS_zh.md)、[Postman 集合](POSTMAN_zh.md)、[OpenAPI 规范](OPENAPI_zh.md)、[安全与网关指南](SECURITY_zh.md) 和 [Kubernetes 部署模板](kubernetes/README_zh.md)。
### 端到端 smoke test
在另一个终端运行：
```bash
bash smoke_test.sh
# 不依赖 curl/bash 的跨平台方式：
python smoke_test.py
```
等价手动命令：
```bash
curl -L https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/BAC009S0764W0121.wav -o sample.wav
curl http://localhost:8000/health
curl http://localhost:8000/v1/audio/transcriptions \
-F file=@sample.wav \
-F model=sensevoice \
-F response_format=verbose_json
```
## Gradio 浏览器 Demo
如果希望用本地浏览器上传音频或测试麦克风，先启动 API 服务，再运行可选 Gradio 前端：
```bash
pip install gradio
python gradio_app.py --base-url http://localhost:8000
```
这个浏览器 demo 调用的就是 smoke test 使用的 OpenAI 兼容 API 端点。Docker、Kubernetes 和生产注意事项见 [Gradio 浏览器 Demo](GRADIO_zh.md)。
## 使用 OpenAI SDK
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
result = client.audio.transcriptions.create(
model="sensevoice", # 也可以使用 "paraformer"、"paraformer-en"、"fun-asr-nano"
file=open("meeting.wav", "rb"),
)
print(result.text)
verbose = client.audio.transcriptions.create(
model="sensevoice",
file=open("meeting.wav", "rb"),
response_format="verbose_json",
)
print(verbose.segments)
```
## 使用 curl
```bash
curl http://localhost:8000/v1/audio/transcriptions \
-F file=@audio.wav \
-F model=sensevoice
curl http://localhost:8000/v1/audio/transcriptions \
-F file=@audio.wav \
-F model=sensevoice \
-F response_format=verbose_json
```
## 可用模型
| Model | GPU 速度 | CPU 速度 | 语言 | 特性 |
|---|---|---|---|---|
| `sensevoice` | 170x realtime | 17x realtime | zh/en/ja/ko/yue | 情感与事件标签 |
| `paraformer` | 120x realtime | 15x realtime | zh/en | 标点恢复 |
| `paraformer-en` | 120x realtime | 15x realtime | en | 英文识别 |
| `fun-asr-nano` | 17x realtime | 3.6x realtime | 31 languages | LLM-based，时间戳 |
## API 端点
| Endpoint | Method | 说明 |
|---|---|---|
| `/v1/audio/transcriptions` | POST | OpenAI 兼容音频转写 |
| `/v1/models` | GET | 列出模型别名 |
| `/health` | GET | 健康检查、已加载模型和可用模型 |
| `/docs` | GET | FastAPI Swagger 文档 |
不想写代码验证接口时，可以使用 [Gradio 浏览器 Demo](GRADIO_zh.md) 做本地上传或麦克风测试，也可以导入 [Postman 集合](POSTMAN_zh.md)。如果要接入 API 网关、开发者门户或生成内部客户端，可以使用 [OpenAPI 规范](OPENAPI_zh.md)。
## Agent 与低代码工作流
适用场景包括 **LangChain**、**LlamaIndex**、**AutoGen**、**CrewAI**、**Semantic Kernel**、**Dify**、**n8n** 和任何支持 OpenAI audio API 或 multipart HTTP 的系统。
- SDK、JavaScript/TypeScript 和 Agent tool 写法见 [客户端配方](CLIENTS.md) 与 [JavaScript/TypeScript 配方](JAVASCRIPT_zh.md)。
- Dify、n8n、HTTP 节点和 webhook worker 见 [工作流配方](WORKFLOWS_zh.md)。
- 图形界面 smoke test 见 [Postman 集合](POSTMAN_zh.md)。
- schema 驱动导入见 [OpenAPI 规范](OPENAPI_zh.md)。
## Docker 部署
默认镜像以 CPU 模式启动，适合作为可复现 smoke test。
```bash
cd examples/openai_api
cp .env.example .env
docker compose up --build
```
等价 `docker run`：
```bash
docker build -t funasr-api .
docker run --rm -p 8000:8000 \
-e FUNASR_DEVICE=cpu \
-e FUNASR_MODEL=sensevoice \
funasr-api
```
GPU 环境需要 NVIDIA Container Toolkit 和 CUDA-capable PyTorch/FunASR 镜像。适配 CUDA 依赖后，可使用：
```bash
docker run --rm --gpus all -p 8000:8000 \
-e FUNASR_DEVICE=cuda \
-e FUNASR_MODEL=sensevoice \
funasr-api
```
验证容器：
```bash
BASE_URL=http://localhost:8000 bash smoke_test.sh
python smoke_test.py --base-url http://localhost:8000
```
## Kubernetes 部署
在跨团队共享服务或通过网关暴露服务前，请先阅读 [安全与网关指南](SECURITY_zh.md)，补齐 TLS、鉴权、上传限制、限流和日志策略。
如果需要在集群内部提供带持久化模型缓存、健康检查和私有 `ClusterIP` 的语音 API，可以从 [Kubernetes 部署模板](kubernetes/README_zh.md) 开始。先构建并推送示例镜像，应用 manifests，再通过 `kubectl port-forward` 和 `python smoke_test.py --base-url http://localhost:8000` 验证。
在没有 CUDA-capable 镜像和 GPU 调度配置前，请保持默认 CPU 模式。
## 配置
| 参数 | 默认值 | 说明 |
|---|---|---|
| `--host` | `0.0.0.0` | 监听地址 |
| `--port` | `8000` | 监听端口 |
| `--device` | `cuda` | `cuda`、`cpu` 或 `mps` |
| `--model` | `sensevoice` | 启动时预加载模型 |
Docker 环境变量：
| Env | 默认值 | 说明 |
|---|---|---|
| `FUNASR_PORT` | `8000` | 传给 `server.py` 的容器端口 |
| `FUNASR_DEVICE` | `cpu` | 容器设备模式；只有在镜像已适配 CUDA 时才设为 `cuda` |
| `FUNASR_MODEL` | `sensevoice` | 容器启动时加载的模型别名 |
## 故障排查
| 现象 | 处理方式 |
|---|---|
| CUDA 不可用 | 先用 `--device cpu` 跑通 smoke test。 |
| 8000 端口被占用 | 改用 `--port 9000`，并运行 `BASE_URL=http://localhost:9000 bash smoke_test.sh` 或 `python smoke_test.py --base-url http://localhost:9000`。 |
| 模型下载很慢 | 换稳定网络，或提前从 ModelScope/Hugging Face 下载模型。 |
| Dify/n8n 容器里访问 `localhost` 失败 | 使用工作流运行时可访问的主机名、Compose service name 或 Kubernetes service name。 |
| 响应中没有 `segments` | 设置 `response_format=verbose_json`。 |
examples/openai_api/SECURITY.md
+132
View File
@@ -0,0 +1,132 @@
# Security and Gateway Guide for the FunASR OpenAI-Compatible API
Use this guide before sharing the example OpenAI-compatible API with a team, workflow engine, browser UI, or service outside your laptop. The example server is intentionally small: it focuses on `/v1/audio/transcriptions` compatibility and does not enforce authentication by itself.
## Recommended topology
```text
OpenAI SDK / Dify / n8n / browser UI
|
v
TLS + auth + upload limits + logs
(reverse proxy, API gateway, ingress, or service mesh)
|
v
FunASR OpenAI-compatible API
(private host, VM, container, or Kubernetes ClusterIP)
```
Keep FunASR on a private network whenever possible. Put public TLS, identity, request limits, and audit logging at the boundary that your team already operates.
## Minimum controls before sharing
| Control | Why it matters | Where to enforce it |
|---|---|---|
| TLS | Audio often contains private data. | Reverse proxy, API gateway, or ingress. |
| Authentication | The local example accepts any SDK `api_key` placeholder. | Gateway bearer token, basic auth, OAuth/OIDC, or internal SSO. |
| Upload-size limits | Prevent accidental multi-GB uploads and memory pressure. | Gateway request-body limit and app-level checks. |
| Timeouts | Long recordings need longer HTTP timeouts, but stuck clients should not hang forever. | Client, proxy, and server process manager. |
| Rate limits | Protect GPU/CPU capacity from bursts. | Gateway, ingress controller, or queue worker. |
| Private `/health` | Health output is useful operational data, not a public product endpoint. | Network allowlist or private monitoring path. |
| Logs and retention | Request metadata is useful; raw audio may be sensitive. | Central logging policy and storage lifecycle. |
## NGINX reverse proxy sketch
This is a starting point, not a complete production policy. Add your own certificates, identity provider, and secret management.
```nginx
server {
listen 443 ssl http2;
server_name funasr.example.com;
client_max_body_size 200m;
proxy_read_timeout 600s;
proxy_send_timeout 600s;
location / {
# Add auth_request, basic auth, mTLS, or an API gateway policy here.
proxy_pass http://127.0.0.1:8000;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Authorization $http_authorization;
}
location = /health {
allow 10.0.0.0/8;
deny all;
proxy_pass http://127.0.0.1:8000/health;
}
}
```
## Caddy reverse proxy sketch
Generate the password hash with `caddy hash-password` and store the real credential outside the repository.
```caddyfile
funasr.example.com {
request_body {
max_size 200MB
}
basicauth /* {
team_user <hashed-password>
}
reverse_proxy 127.0.0.1:8000 {
transport http {
read_timeout 600s
write_timeout 600s
}
}
}
```
For production teams, prefer your standard SSO/OIDC gateway over shared passwords.
## Kubernetes notes
The Kubernetes template keeps the service private with `ClusterIP`. Before adding an ingress or load balancer:
- Add an ingress controller or API gateway that enforces TLS, authentication, upload-size limits, and rate limits.
- Keep model cache volumes private to the namespace or node pool that owns the service.
- Use `NetworkPolicy` to restrict which namespaces can call the service.
- Use `kubectl port-forward` plus `smoke_test.py` for first validation before exposing a route.
- If you add GPUs, pin scheduling rules and record the image tag, CUDA runtime, and model alias in deployment notes.
## Client configuration
OpenAI SDKs usually require an API key string even when FunASR does not check it locally. After you add a gateway, use the gateway-issued token as the SDK key:
```python
import os
from openai import OpenAI
client = OpenAI(
base_url="https://funasr.example.com/v1",
api_key=os.environ["FUNASR_API_KEY"],
)
```
For internal HTTP workers, read tokens from environment variables or your secret manager. Do not commit tokens to workflow definitions, notebooks, screenshots, or Postman exports.
## Data handling checklist
- Decide whether raw audio can be stored, for how long, and who can access it.
- Log request IDs, duration, model alias, status, latency, and error class; avoid logging raw transcript text unless your policy allows it.
- If transcripts may contain personal data, document retention and deletion rules before onboarding users.
- Keep public samples separate from private customer or employee audio when writing benchmark reports.
- Redact headers, tokens, file names, and speaker names before opening GitHub issues.
## Rollout checklist
1. Start locally and run `bash smoke_test.sh` or `python smoke_test.py`.
2. Add the gateway and verify `/health`, `/v1/models`, and `/v1/audio/transcriptions` through the gateway URL.
3. Test a small file, a large allowed file, and a file above the upload limit.
4. Confirm unauthorized requests fail before reaching FunASR.
5. Confirm timeout behavior for long audio and slow clients.
6. Record the model alias, device, image tag, FunASR version, and gateway policy.
Related guides: [OpenAI API README](README.md), [client recipes](CLIENTS.md), [workflow recipes](WORKFLOWS.md), [Gradio browser demo](GRADIO.md), [Kubernetes template](kubernetes/README.md), and the repository [security policy](../../SECURITY.md).
examples/openai_api/SECURITY_zh.md
+132
View File
@@ -0,0 +1,132 @@
# FunASR OpenAI 兼容 API 安全与网关指南
当你准备把示例 OpenAI 兼容 API 提供给团队、工作流引擎、浏览器 UI 或笔记本之外的服务时，请先看这份指南。示例服务刻意保持轻量，主要关注 `/v1/audio/transcriptions` 兼容性，本身不内置鉴权策略。
## 推荐拓扑
```text
OpenAI SDK / Dify / n8n / 浏览器 UI
|
v
TLS + 鉴权 + 上传限制 + 日志
(反向代理、API 网关、Ingress 或 Service Mesh)
|
v
FunASR OpenAI 兼容 API
(私有主机、虚机、容器或 Kubernetes ClusterIP)
```
只要条件允许，就让 FunASR 保持在私有网络中。公网 TLS、身份认证、请求限制和审计日志应放在团队已有的边界组件上。
## 对团队开放前的最低控制项
| 控制项 | 为什么重要 | 建议在哪里实现 |
|---|---|---|
| TLS | 音频通常包含隐私信息。 | 反向代理、API 网关或 Ingress。 |
| 鉴权 | 本地示例会接受任意 SDK `api_key` 占位符。 | 网关 Bearer token、Basic auth、OAuth/OIDC 或内部 SSO。 |
| 上传大小限制 | 避免误传超大文件导致内存和磁盘压力。 | 网关 request body limit 和应用侧检查。 |
| 超时 | 长音频需要更长 HTTP timeout，但异常客户端不能无限挂住。 | 客户端、代理和服务进程管理器。 |
| 限流 | 防止突发请求打满 GPU/CPU。 | 网关、Ingress controller 或队列 worker。 |
| 私有 `/health` | 健康信息是运维数据，不应作为公开产品端点。 | 网络 allowlist 或私有监控路径。 |
| 日志与留存 | 请求元数据有价值，但原始音频可能敏感。 | 集中日志策略和存储生命周期。 |
## NGINX 反向代理示例
下面只是起点，不是完整生产策略。证书、身份系统和密钥管理需要按你的环境补齐。
```nginx
server {
listen 443 ssl http2;
server_name funasr.example.com;
client_max_body_size 200m;
proxy_read_timeout 600s;
proxy_send_timeout 600s;
location / {
# 在这里增加 auth_request、Basic auth、mTLS 或 API 网关策略。
proxy_pass http://127.0.0.1:8000;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Authorization $http_authorization;
}
location = /health {
allow 10.0.0.0/8;
deny all;
proxy_pass http://127.0.0.1:8000/health;
}
}
```
## Caddy 反向代理示例
使用 `caddy hash-password` 生成密码 hash，真实凭据不要写入仓库。
```caddyfile
funasr.example.com {
request_body {
max_size 200MB
}
basicauth /* {
team_user <hashed-password>
}
reverse_proxy 127.0.0.1:8000 {
transport http {
read_timeout 600s
write_timeout 600s
}
}
}
```
生产团队优先使用已有 SSO/OIDC 网关，而不是共享密码。
## Kubernetes 注意事项
Kubernetes 模板默认使用私有 `ClusterIP`。在增加 Ingress 或 LoadBalancer 前，请先完成：
- 使用 Ingress controller 或 API 网关强制 TLS、鉴权、上传大小限制和限流。
- 模型缓存卷只暴露给拥有该服务的 namespace 或 node pool。
- 使用 `NetworkPolicy` 限制可调用服务的 namespace。
- 第一次验证先用 `kubectl port-forward` 加 `smoke_test.py`，再开放路由。
- 如果增加 GPU，固定调度规则，并在部署说明中记录镜像 tag、CUDA runtime 和模型 alias。
## 客户端配置
OpenAI SDK 通常要求传入 API key 字符串，即使本地 FunASR 不检查它。加上网关后，请使用网关发放的 token 作为 SDK key：
```python
import os
from openai import OpenAI
client = OpenAI(
base_url="https://funasr.example.com/v1",
api_key=os.environ["FUNASR_API_KEY"],
)
```
内部 HTTP worker 应从环境变量或密钥系统读取 token。不要把 token 提交到工作流定义、notebook、截图或 Postman 导出文件里。
## 数据处理清单
- 先决定原始音频是否允许存储、保存多久、谁可以访问。
- 日志建议记录 request ID、音频时长、模型 alias、状态、延迟和错误类型；除非策略允许，不要记录原始转写文本。
- 如果转写文本可能包含个人信息，请在接入用户前写清留存和删除规则。
- 写 benchmark 报告时，把公开样例和客户/员工私有音频分开。
- 打开 GitHub issue 前，先脱敏 header、token、文件名和说话人姓名。
## 上线检查清单
1. 先在本地启动，并运行 `bash smoke_test.sh` 或 `python smoke_test.py`。
2. 增加网关后，通过网关 URL 验证 `/health`、`/v1/models` 和 `/v1/audio/transcriptions`。
3. 分别测试小文件、允许范围内的大文件，以及超过上传限制的文件。
4. 确认未授权请求会在到达 FunASR 前失败。
5. 确认长音频和慢客户端的 timeout 行为。
6. 记录模型 alias、设备、镜像 tag、FunASR 版本和网关策略。
相关文档：[OpenAI API README](README_zh.md)、[客户端配方](CLIENTS.md)、[工作流配方](WORKFLOWS_zh.md)、[Gradio 浏览器 Demo](GRADIO_zh.md)、[Kubernetes 模板](kubernetes/README_zh.md) 和仓库 [安全策略](../../SECURITY.md)。
examples/openai_api/WORKFLOWS.md
+166
View File
@@ -0,0 +1,166 @@
# Low-code Workflow Recipes for the FunASR OpenAI-Compatible API
[中文](WORKFLOWS_zh.md)
Use this guide when you want Dify, n8n, webhook workers, or another workflow engine to call a private FunASR speech API. Start with the local smoke test in this directory, then replace `localhost` with the reachable service name inside your network.
## Server preflight
```bash
cd examples/openai_api
python server.py --model sensevoice --device cuda --port 8000
```
From the workflow host or container:
```bash
export FUNASR_BASE_URL=http://<funasr-host>:8000
curl -fsS "$FUNASR_BASE_URL/health"
curl -fsS "$FUNASR_BASE_URL/v1/models"
```
If the workflow engine runs in Docker, `localhost` usually means the workflow container itself. Use a Docker Compose service name, Kubernetes service name, or LAN host name instead.
## Postman smoke test
Before configuring a low-code tool, you can import the [Postman collection](POSTMAN.md) and run health, model-list, and transcription requests from a GUI. For schema-driven imports, use the [OpenAPI spec](OPENAPI.md). Set `FUNASR_BASE_URL`, choose a local audio file for the multipart `file` field, and keep `MODEL_ALIAS=sensevoice` for the first test.
## Multipart HTTP request
Every workflow engine eventually needs to send this request shape:
| Field | Value |
|---|---|
| Method | `POST` |
| URL | `http://<funasr-host>:8000/v1/audio/transcriptions` |
| Body type | `multipart/form-data` |
| File field | `file` |
| Text field | `model=sensevoice` |
| Text field | `response_format=verbose_json` |
| Timeout | Set according to maximum audio duration, for example 300 seconds for long files. |
Equivalent curl command:
```bash
curl "$FUNASR_BASE_URL/v1/audio/transcriptions" \
-F file=@meeting.wav \
-F model=sensevoice \
-F response_format=verbose_json
```
Typical JSON fields to map downstream:
| Path | Use |
|---|---|
| `text` | Plain transcript for a chatbot, ticket, or knowledge-base step. |
| `segments` | Timestamps and speaker labels when `verbose_json` is requested. |
| `duration` | Audio processing time reported by the API, useful for logs. |
| `model` | Model alias used for the request. |
## Dify custom tool or HTTP node
Use this pattern when a Dify application receives an uploaded audio file or a URL to internal audio storage.
### Direct file upload path
Configure an HTTP request node or custom tool with:
- Method: `POST`
- URL: `http://<funasr-host>:8000/v1/audio/transcriptions`
- Body: `multipart/form-data`
- File part: `file`, bound to the uploaded audio variable
- Text parts: `model=sensevoice`, `response_format=verbose_json`
- Output variable: map `text` as the transcript, and keep `segments` when timestamps or speaker labels matter
### Audio URL path
Some workflow tools pass a file URL rather than raw multipart bytes. In that case, add a small internal worker:
1. Dify sends the audio URL and metadata to the worker.
2. The worker downloads the file from trusted storage.
3. The worker posts multipart data to FunASR.
4. The worker returns `text`, `segments`, and operational logs to Dify.
```python
import requests
FUNASR_URL = "http://funasr-api:8000/v1/audio/transcriptions"
def transcribe_from_url(audio_url: str) -> dict:
audio_response = requests.get(audio_url, timeout=120)
audio_response.raise_for_status()
files = {"file": ("audio.wav", audio_response.content, "audio/wav")}
data = {"model": "sensevoice", "response_format": "verbose_json"}
response = requests.post(FUNASR_URL, files=files, data=data, timeout=300)
response.raise_for_status()
return response.json()
```
Keep this worker inside your trusted network and validate allowed URL domains before downloading user-provided links.
## n8n HTTP Request node
A common n8n flow is: trigger -> binary audio data -> HTTP Request -> transcript consumer.
Recommended HTTP Request settings:
| n8n setting | Value |
|---|---|
| Method | `POST` |
| URL | `http://<funasr-host>:8000/v1/audio/transcriptions` |
| Send Body | enabled |
| Body Content Type | `Form-Data` / multipart |
| Binary file field | `file` |
| Additional form fields | `model=sensevoice`, `response_format=verbose_json` |
| Response Format | JSON |
| Timeout | Increase for long recordings. |
After the request, use `{{$json.text}}` as the transcript. If `verbose_json` is enabled, route `{{$json.segments}}` to subtitle, speaker, or QA steps.
## Webhook worker pattern
Use this when the workflow engine cannot send multipart files reliably or when audio needs pre-processing.
```python
from pathlib import Path
import tempfile
import requests
FUNASR_URL = "http://localhost:8000/v1/audio/transcriptions"
def transcribe_bytes(filename: str, payload: bytes, content_type: str = "audio/wav") -> dict:
with tempfile.NamedTemporaryFile(suffix=Path(filename).suffix or ".wav") as tmp:
tmp.write(payload)
tmp.flush()
with open(tmp.name, "rb") as audio:
response = requests.post(
FUNASR_URL,
files={"file": (filename, audio, content_type)},
data={"model": "sensevoice", "response_format": "verbose_json"},
timeout=300,
)
response.raise_for_status()
return response.json()
```
This worker is also the right place to add audio conversion, file-size checks, request IDs, authentication, and retries.
## Production guardrails
- Put authentication, TLS, upload-size limits, and rate limits in front of FunASR before sharing it across teams; use the [security and gateway guide](SECURITY.md) for proxy and gateway patterns.
- Use `/health` for workflow readiness checks and `/v1/models` to validate model aliases.
- Log request id, audio duration, model alias, response format, device, latency, and error type.
- Set workflow timeouts according to maximum audio duration; split very long recordings before sending them through low-code tools.
- Keep private audio in trusted storage and avoid putting signed URLs, credentials, or transcripts into public logs.
- Run the same workflow with at least one public smoke sample and one realistic private sample before production use.
## Troubleshooting
| Symptom | Fix |
|---|---|
| Workflow can call `/health` but transcription fails | Confirm the request is `multipart/form-data` and the binary field is named `file`. |
| `localhost` connection fails from Dify or n8n | Use the host, Compose service, or Kubernetes service reachable from the workflow runtime. |
| Response has no `segments` | Set `response_format=verbose_json`. |
| Requests time out | Increase HTTP timeout or split long recordings. |
| First request is slow | Preload the model with `--model sensevoice` and use `/health` as a readiness check. |
| Unknown model alias | Call `/v1/models` and use one of the returned aliases. |
examples/openai_api/WORKFLOWS_zh.md
+166
View File
@@ -0,0 +1,166 @@
# FunASR OpenAI 兼容 API 低代码工作流配方
[English](WORKFLOWS.md)
当你希望让 Dify、n8n、HTTP 节点、webhook worker 或其他低代码工作流引擎调用私有 FunASR 语音 API 时，可以从这份配方开始。建议先在本目录跑通本地 smoke test，再把 `localhost` 换成工作流运行环境能够访问到的服务名或内网地址。
## 服务预检
```bash
cd examples/openai_api
python server.py --model sensevoice --device cuda --port 8000
```
在工作流所在主机或容器中执行：
```bash
export FUNASR_BASE_URL=http://<funasr-host>:8000
curl -fsS "$FUNASR_BASE_URL/health"
curl -fsS "$FUNASR_BASE_URL/v1/models"
```
如果工作流引擎运行在 Docker 中，`localhost` 通常指的是工作流容器自身。请改用 Docker Compose service name、Kubernetes service name 或内网主机名。
## Postman smoke test
在配置低代码工具前，可以先导入 [Postman collection](POSTMAN_zh.md)，从图形界面跑通 health、模型列表和转写请求；需要按 schema 导入时可使用 [OpenAPI spec](OPENAPI_zh.md)。设置 `FUNASR_BASE_URL`，在 multipart `file` 字段选择本地音频文件，第一次测试建议保持 `MODEL_ALIAS=sensevoice`。
## Multipart HTTP 请求
所有工作流引擎最终都需要发出下面这种请求：
| 字段 | 值 |
|---|---|
| Method | `POST` |
| URL | `http://<funasr-host>:8000/v1/audio/transcriptions` |
| Body type | `multipart/form-data` |
| File field | `file` |
| Text field | `model=sensevoice` |
| Text field | `response_format=verbose_json` |
| Timeout | 根据最长音频时长设置，例如长录音可先设为 300 秒。 |
等价 curl 命令：
```bash
curl "$FUNASR_BASE_URL/v1/audio/transcriptions" \
-F file=@meeting.wav \
-F model=sensevoice \
-F response_format=verbose_json
```
常用响应字段映射：
| 路径 | 用途 |
|---|---|
| `text` | 纯文本转写结果，可传给聊天机器人、工单、知识库或后续摘要节点。 |
| `segments` | 请求 `verbose_json` 时返回的时间戳、说话人等分段信息。 |
| `duration` | API 返回的音频处理时长，适合写入日志和监控。 |
| `model` | 本次请求使用的模型别名。 |
## Dify 自定义工具或 HTTP 节点
当 Dify 应用接收上传音频文件，或收到内部音频存储 URL 时，可以使用下面两种模式。
### 直接上传文件
在 HTTP request 节点或自定义工具中配置：
- Method: `POST`
- URL: `http://<funasr-host>:8000/v1/audio/transcriptions`
- Body: `multipart/form-data`
- File part: `file`，绑定到上传音频变量
- Text parts: `model=sensevoice`、`response_format=verbose_json`
- Output variable: 把 `text` 映射为转写文本；需要时间戳或说话人信息时保留 `segments`
### 音频 URL 转写
有些工作流工具只能传文件 URL，而不能直接传 multipart 二进制。此时建议加一个内网 worker：
1. Dify 将音频 URL 和元数据发送给 worker。
2. worker 从可信存储下载音频。
3. worker 使用 multipart 请求调用 FunASR。
4. worker 将 `text`、`segments` 和运行日志返回给 Dify。
```python
import requests
FUNASR_URL = "http://funasr-api:8000/v1/audio/transcriptions"
def transcribe_from_url(audio_url: str) -> dict:
audio_response = requests.get(audio_url, timeout=120)
audio_response.raise_for_status()
files = {"file": ("audio.wav", audio_response.content, "audio/wav")}
data = {"model": "sensevoice", "response_format": "verbose_json"}
response = requests.post(FUNASR_URL, files=files, data=data, timeout=300)
response.raise_for_status()
return response.json()
```
请把这个 worker 放在可信内网中，并在下载用户提供的链接前校验允许访问的域名或存储桶。
## n8n HTTP Request 节点
一个常见 n8n 流程是：触发器 -> 二进制音频数据 -> HTTP Request -> 转写结果消费节点。
推荐 HTTP Request 配置：
| n8n 设置 | 值 |
|---|---|
| Method | `POST` |
| URL | `http://<funasr-host>:8000/v1/audio/transcriptions` |
| Send Body | enabled |
| Body Content Type | `Form-Data` / multipart |
| Binary file field | `file` |
| Additional form fields | `model=sensevoice`、`response_format=verbose_json` |
| Response Format | JSON |
| Timeout | 长录音场景需要调大。 |
请求之后，使用 `{{$json.text}}` 作为转写文本。如果启用了 `verbose_json`，可以把 `{{$json.segments}}` 传给字幕、说话人分析或质检节点。
## Webhook worker 模式
当工作流引擎不能稳定发送 multipart 文件，或音频需要预处理时，可以把转写封装成一个内部 webhook worker。
```python
from pathlib import Path
import tempfile
import requests
FUNASR_URL = "http://localhost:8000/v1/audio/transcriptions"
def transcribe_bytes(filename: str, payload: bytes, content_type: str = "audio/wav") -> dict:
with tempfile.NamedTemporaryFile(suffix=Path(filename).suffix or ".wav") as tmp:
tmp.write(payload)
tmp.flush()
with open(tmp.name, "rb") as audio:
response = requests.post(
FUNASR_URL,
files={"file": (filename, audio, content_type)},
data={"model": "sensevoice", "response_format": "verbose_json"},
timeout=300,
)
response.raise_for_status()
return response.json()
```
这个 worker 也适合集中做音频转码、文件大小限制、请求 ID、鉴权、重试和审计日志。
## 生产环境护栏
- 在跨团队共享 FunASR 服务前，先加好鉴权、TLS、上传大小限制和限流；代理和网关模式见 [安全与网关指南](SECURITY_zh.md)。
- 使用 `/health` 做工作流 readiness check，使用 `/v1/models` 校验模型别名。
- 记录 request id、音频时长、模型别名、响应格式、设备、延迟和错误类型。
- 按最长音频时长设置工作流超时；超长录音建议先切分，再交给低代码工具处理。
- 私有音频放在可信存储中，避免把签名 URL、凭据或转写文本写入公开日志。
- 上生产前，至少用一个公开 smoke 样例和一个真实业务样例完整跑通同一条工作流。
## 故障排查
| 现象 | 处理方式 |
|---|---|
| 工作流能访问 `/health`，但转写失败 | 确认请求是 `multipart/form-data`，且二进制字段名是 `file`。 |
| Dify 或 n8n 访问 `localhost` 失败 | 换成工作流运行时可访问的主机名、Compose service name 或 Kubernetes service name。 |
| 响应中没有 `segments` | 设置 `response_format=verbose_json`。 |
| 请求超时 | 调大 HTTP timeout，或先切分长录音。 |
| 第一次请求很慢 | 使用 `--model sensevoice` 预加载模型，并用 `/health` 做 readiness check。 |
| 模型别名未知 | 调用 `/v1/models`，使用返回列表中的别名。 |
examples/openai_api/docker-compose.yml
+17
View File
@@ -0,0 +1,17 @@
services:
funasr-api:
build:
context: .
image: funasr-api:local
ports:
- "${FUNASR_HOST_PORT:-8000}:8000"
environment:
FUNASR_PORT: "8000"
FUNASR_DEVICE: "${FUNASR_DEVICE:-cpu}"
FUNASR_MODEL: "${FUNASR_MODEL:-sensevoice}"
volumes:
- funasr-cache:/root/.cache
shm_size: "2gb"
volumes:
funasr-cache:
examples/openai_api/funasr-openai-api.postman_collection.json
+185
View File
@@ -0,0 +1,185 @@
{
"info": {
"name": "FunASR OpenAI-Compatible API",
"description": "Smoke-test the FunASR OpenAI-compatible speech API. Start the server, set FUNASR_BASE_URL, choose a local audio file for multipart requests, then run health, model list, and transcription checks.",
"schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json"
},
"variable": [
{
"key": "FUNASR_BASE_URL",
"value": "http://localhost:8000",
"type": "string"
},
{
"key": "MODEL_ALIAS",
"value": "sensevoice",
"type": "string"
},
{
"key": "RESPONSE_FORMAT",
"value": "verbose_json",
"type": "string"
}
],
"item": [
{
"name": "Health check",
"request": {
"method": "GET",
"header": [],
"url": {
"raw": "{{FUNASR_BASE_URL}}/health",
"host": [
"{{FUNASR_BASE_URL}}"
],
"path": [
"health"
]
},
"description": "Confirm the FunASR API server is reachable and ready."
},
"event": [
{
"listen": "test",
"script": {
"type": "text/javascript",
"exec": [
"pm.test('health returns 200', function () { pm.response.to.have.status(200); });",
"pm.test('health response is JSON', function () { pm.response.to.be.json; });"
]
}
}
]
},
{
"name": "List model aliases",
"request": {
"method": "GET",
"header": [],
"url": {
"raw": "{{FUNASR_BASE_URL}}/v1/models",
"host": [
"{{FUNASR_BASE_URL}}"
],
"path": [
"v1",
"models"
]
},
"description": "List OpenAI-compatible model aliases exposed by the server."
},
"event": [
{
"listen": "test",
"script": {
"type": "text/javascript",
"exec": [
"pm.test('models returns 200', function () { pm.response.to.have.status(200); });",
"pm.test('models response is JSON', function () { pm.response.to.be.json; });"
]
}
}
]
},
{
"name": "Transcribe audio - verbose JSON",
"request": {
"method": "POST",
"header": [],
"body": {
"mode": "formdata",
"formdata": [
{
"key": "file",
"type": "file",
"src": "sample.wav",
"description": "Choose a local wav, mp3, m4a, or other supported audio file before sending."
},
{
"key": "model",
"value": "{{MODEL_ALIAS}}",
"type": "text"
},
{
"key": "response_format",
"value": "{{RESPONSE_FORMAT}}",
"type": "text"
}
]
},
"url": {
"raw": "{{FUNASR_BASE_URL}}/v1/audio/transcriptions",
"host": [
"{{FUNASR_BASE_URL}}"
],
"path": [
"v1",
"audio",
"transcriptions"
]
},
"description": "Upload an audio file with multipart/form-data and return transcript text plus segments when RESPONSE_FORMAT is verbose_json."
},
"event": [
{
"listen": "test",
"script": {
"type": "text/javascript",
"exec": [
"pm.test('transcription returns 200', function () { pm.response.to.have.status(200); });",
"pm.test('transcription response is JSON', function () { pm.response.to.be.json; });",
"pm.test('transcription has text', function () { var data = pm.response.json(); pm.expect(data.text).to.be.a('string'); });"
]
}
}
]
},
{
"name": "Transcribe audio - text only",
"request": {
"method": "POST",
"header": [],
"body": {
"mode": "formdata",
"formdata": [
{
"key": "file",
"type": "file",
"src": "sample.wav",
"description": "Choose a local audio file before sending."
},
{
"key": "model",
"value": "{{MODEL_ALIAS}}",
"type": "text"
}
]
},
"url": {
"raw": "{{FUNASR_BASE_URL}}/v1/audio/transcriptions",
"host": [
"{{FUNASR_BASE_URL}}"
],
"path": [
"v1",
"audio",
"transcriptions"
]
},
"description": "Minimal OpenAI-compatible transcription request."
},
"event": [
{
"listen": "test",
"script": {
"type": "text/javascript",
"exec": [
"pm.test('transcription returns 200', function () { pm.response.to.have.status(200); });",
"pm.test('transcription response is JSON', function () { pm.response.to.be.json; });"
]
}
}
]
}
]
}
examples/openai_api/gradio_app.py
+157
View File
@@ -0,0 +1,157 @@
#!/usr/bin/env python3
"""Browser demo for the FunASR OpenAI-compatible API."""
from __future__ import annotations
import argparse
import json
import mimetypes
import os
from pathlib import Path
import urllib.error
import urllib.request
import uuid
DEFAULT_BASE_URL = "http://localhost:8000"
DEFAULT_MODEL = "sensevoice"
def request_json(url: str, timeout: float) -> dict:
request = urllib.request.Request(url, headers={"Accept": "application/json"})
with urllib.request.urlopen(request, timeout=timeout) as response:
return json.loads(response.read().decode("utf-8"))
def multipart_body(audio_path: Path, model: str, response_format: str) -> tuple[bytes, str]:
boundary = f"----funasr-gradio-{uuid.uuid4().hex}"
content_type = mimetypes.guess_type(str(audio_path))[0] or "application/octet-stream"
parts: list[bytes] = []
def add_text(name: str, value: str) -> None:
parts.append(
(
f"--{boundary}\r\n"
f"Content-Disposition: form-data; name=\"{name}\"\r\n\r\n"
f"{value}\r\n"
).encode("utf-8")
)
parts.append(
(
f"--{boundary}\r\n"
f"Content-Disposition: form-data; name=\"file\"; filename=\"{audio_path.name}\"\r\n"
f"Content-Type: {content_type}\r\n\r\n"
).encode("utf-8")
)
parts.append(audio_path.read_bytes())
parts.append(b"\r\n")
add_text("model", model)
add_text("response_format", response_format)
parts.append(f"--{boundary}--\r\n".encode("utf-8"))
return b"".join(parts), boundary
def transcribe_audio(base_url: str, audio_path: str | None, model: str, response_format: str, timeout: float) -> tuple[str, str]:
if not audio_path:
return "", "Upload or record an audio file first."
base_url = base_url.rstrip("/")
path = Path(audio_path)
body, boundary = multipart_body(path, model, response_format)
request = urllib.request.Request(
f"{base_url}/v1/audio/transcriptions",
data=body,
method="POST",
headers={
"Accept": "application/json",
"Content-Type": f"multipart/form-data; boundary={boundary}",
"Content-Length": str(len(body)),
},
)
with urllib.request.urlopen(request, timeout=timeout) as response:
payload = json.loads(response.read().decode("utf-8"))
text = payload.get("text", "")
return text, json.dumps(payload, ensure_ascii=False, indent=2)
def check_service(base_url: str, timeout: float) -> str:
base_url = base_url.rstrip("/")
health = request_json(f"{base_url}/health", timeout)
models = request_json(f"{base_url}/v1/models", timeout)
return json.dumps({"health": health, "models": models}, ensure_ascii=False, indent=2)
def safe_transcribe(base_url: str, audio_path: str | None, model: str, response_format: str, timeout: float) -> tuple[str, str]:
try:
return transcribe_audio(base_url, audio_path, model, response_format, timeout)
except urllib.error.HTTPError as error:
detail = error.read().decode("utf-8", errors="replace")
return "", f"HTTP {error.code} from {error.url}: {detail}"
except Exception as error:
return "", f"Transcription failed: {error}"
def safe_check(base_url: str, timeout: float) -> str:
try:
return check_service(base_url, timeout)
except urllib.error.HTTPError as error:
detail = error.read().decode("utf-8", errors="replace")
return f"HTTP {error.code} from {error.url}: {detail}"
except Exception as error:
return f"Service check failed: {error}"
def build_app(default_base_url: str, default_timeout: float):
try:
import gradio as gr
except ImportError as error:
raise SystemExit("Install Gradio first: pip install gradio") from error
with gr.Blocks(title="FunASR OpenAI API Demo") as demo:
gr.Markdown("# FunASR OpenAI-Compatible API Demo")
gr.Markdown("Start `python server.py --model sensevoice --device cuda --port 8000`, then upload or record audio here.")
with gr.Row():
base_url = gr.Textbox(label="API base URL", value=default_base_url)
model = gr.Dropdown(
label="Model alias",
choices=["sensevoice", "paraformer", "paraformer-en", "fun-asr-nano"],
value=DEFAULT_MODEL,
)
response_format = gr.Radio(label="Response format", choices=["json", "verbose_json"], value="verbose_json")
timeout = gr.Number(label="Timeout seconds", value=default_timeout, precision=0)
audio = gr.Audio(label="Audio", sources=["upload", "microphone"], type="filepath")
with gr.Row():
check_button = gr.Button("Check service")
transcribe_button = gr.Button("Transcribe", variant="primary")
transcript = gr.Textbox(label="Transcript", lines=6)
raw_json = gr.Code(label="Raw JSON or status", language="json")
check_button.click(fn=safe_check, inputs=[base_url, timeout], outputs=raw_json)
transcribe_button.click(
fn=safe_transcribe,
inputs=[base_url, audio, model, response_format, timeout],
outputs=[transcript, raw_json],
)
return demo
def main() -> None:
parser = argparse.ArgumentParser(description="Run a Gradio demo for the FunASR OpenAI-compatible API")
parser.add_argument("--base-url", default=os.getenv("BASE_URL", DEFAULT_BASE_URL), help="FunASR API base URL")
parser.add_argument("--host", default=os.getenv("GRADIO_HOST", "127.0.0.1"), help="Gradio bind host")
parser.add_argument("--port", type=int, default=int(os.getenv("GRADIO_PORT", "7860")), help="Gradio bind port")
parser.add_argument("--timeout", type=float, default=float(os.getenv("TIMEOUT", "300")), help="HTTP timeout in seconds")
parser.add_argument("--share", action="store_true", help="Create a temporary Gradio share link")
args = parser.parse_args()
app = build_app(args.base_url, args.timeout)
app.launch(server_name=args.host, server_port=args.port, share=args.share)
if __name__ == "__main__":
main()
examples/openai_api/kubernetes/README.md
+80
View File
@@ -0,0 +1,80 @@
# Kubernetes Deployment for the FunASR OpenAI-Compatible API
This folder provides a CPU-first Kubernetes template for the `examples/openai_api` server. Use it when you want an internal OpenAI-compatible speech endpoint reachable by agents, web backends, workflow engines, or batch workers inside a cluster.
The manifest is intentionally conservative:
- `ClusterIP` service by default, not a public `LoadBalancer`.
- `FUNASR_DEVICE=cpu` by default so the image matches the portable Dockerfile.
- A persistent cache volume mounted at `/root/.cache` so model downloads survive pod restarts.
- `/health` startup, readiness, and liveness probes.
- A memory-backed `/dev/shm` volume for PyTorch and audio preprocessing.
## 1. Build and push the image
From `examples/openai_api`:
```bash
docker build -t registry.example.com/speech/funasr-api:cpu-latest .
docker push registry.example.com/speech/funasr-api:cpu-latest
```
Edit `kustomization.yaml` if you use a different registry, repository, or tag.
## 2. Deploy
```bash
kubectl create namespace speech --dry-run=client -o yaml | kubectl apply -f -
kubectl -n speech apply -k .
kubectl -n speech rollout status deploy/funasr-api --timeout=15m
```
Model download and first load can take several minutes. The `startupProbe` allows up to 10 minutes before Kubernetes restarts the container.
## 3. Smoke test
Keep the service private and verify it through `port-forward` first:
```bash
kubectl -n speech port-forward svc/funasr-api 8000:8000
```
From another terminal in `examples/openai_api`:
```bash
python smoke_test.py --base-url http://localhost:8000
```
For in-cluster clients, use `http://funasr-api.speech.svc.cluster.local:8000` as the direct HTTP base URL and `http://funasr-api.speech.svc.cluster.local:8000/v1` as the OpenAI SDK base URL.
## 4. Tune for your cluster
| Setting | Default | When to change it |
|---|---|---|
| `FUNASR_MODEL` | `sensevoice` | Use another alias after checking `/v1/models`. |
| `FUNASR_DEVICE` | `cpu` | Set to `cuda` only after building a CUDA-capable image and configuring GPU scheduling. |
| PVC size | `20Gi` | Increase when caching multiple models or large model revisions. |
| Memory request | `8Gi` | Tune after observing startup and real audio workloads. |
| Startup probe | 10 minutes | Increase if your registry, model hub, or storage backend is slow. |
## GPU notes
The example Dockerfile is CPU-first. For GPU clusters you need to adapt the image to CUDA-capable PyTorch/FunASR dependencies, then add your cluster's GPU scheduling fields, for example:
```yaml
resources:
limits:
nvidia.com/gpu: "1"
nodeSelector:
nvidia.com/gpu.present: "true"
```
Exact GPU labels, runtime classes, and device plugin configuration vary by Kubernetes distribution. Keep the service private until authentication, TLS, upload-size limits, and rate limits are in place.
## Operational checks
- Use `/health` for readiness and `/v1/models` to confirm model aliases.
- Log model alias, device, audio duration, response format, latency, and error text.
- Start with one replica because the cache PVC is `ReadWriteOnce`; scale horizontally with a registry image, per-pod cache, or a shared read-only model cache after measuring memory and startup time.
- Put authentication and network policy in front of the service before exposing it outside a trusted namespace.
- For Dify, n8n, or web backends inside the same cluster, point them at the Kubernetes service name instead of `localhost`.
examples/openai_api/kubernetes/README_zh.md
+80
View File
@@ -0,0 +1,80 @@
# FunASR OpenAI 兼容 API Kubernetes 部署
这个目录提供 `examples/openai_api` 服务的 CPU-first Kubernetes 模板。适合在集群内部为 Agent、Web 后端、工作流引擎或批处理 worker 提供 OpenAI 兼容语音转写接口。
模板默认比较保守：
- 默认使用 `ClusterIP`，不直接暴露公网 `LoadBalancer`。
- 默认 `FUNASR_DEVICE=cpu`，与便携 Dockerfile 匹配。
- 在 `/root/.cache` 挂载持久化缓存卷，避免 Pod 重启后重复下载模型。
- 使用 `/health` 做 startup、readiness 和 liveness probe。
- 挂载内存型 `/dev/shm`，便于 PyTorch 和音频预处理使用。
## 1. 构建并推送镜像
在 `examples/openai_api` 目录中执行：
```bash
docker build -t registry.example.com/speech/funasr-api:cpu-latest .
docker push registry.example.com/speech/funasr-api:cpu-latest
```
如果使用不同的 registry、repo 或 tag，请修改 `kustomization.yaml`。
## 2. 部署
```bash
kubectl create namespace speech --dry-run=client -o yaml | kubectl apply -f -
kubectl -n speech apply -k .
kubectl -n speech rollout status deploy/funasr-api --timeout=15m
```
模型下载和首次加载可能需要几分钟。`startupProbe` 默认允许最多 10 分钟，超过后 Kubernetes 才会重启容器。
## 3. Smoke test
建议先保持服务内网私有，通过 `port-forward` 验证：
```bash
kubectl -n speech port-forward svc/funasr-api 8000:8000
```
在另一个终端进入 `examples/openai_api` 后运行：
```bash
python smoke_test.py --base-url http://localhost:8000
```
集群内客户端可以使用 `http://funasr-api.speech.svc.cluster.local:8000` 作为直接 HTTP base URL，使用 `http://funasr-api.speech.svc.cluster.local:8000/v1` 作为 OpenAI SDK base URL。
## 4. 根据集群调整配置
| 配置 | 默认值 | 什么时候调整 |
|---|---|---|
| `FUNASR_MODEL` | `sensevoice` | 调用 `/v1/models` 后按需要切换模型别名。 |
| `FUNASR_DEVICE` | `cpu` | 只有在镜像已适配 CUDA 且集群 GPU 调度已配置后才改成 `cuda`。 |
| PVC 大小 | `20Gi` | 缓存多个模型或较大模型版本时增大。 |
| 内存 request | `8Gi` | 根据启动过程和真实音频负载观测结果调整。 |
| Startup probe | 10 分钟 | registry、模型下载或存储后端较慢时增大。 |
## GPU 说明
示例 Dockerfile 默认面向 CPU。GPU 集群需要先把镜像改成 CUDA-capable PyTorch/FunASR 依赖，再根据集群增加 GPU 调度字段，例如：
```yaml
resources:
limits:
nvidia.com/gpu: "1"
nodeSelector:
nvidia.com/gpu.present: "true"
```
不同 Kubernetes 发行版的 GPU label、runtime class 和 device plugin 配置并不相同。服务对外开放前，请先补齐鉴权、TLS、上传大小限制和限流。
## 运维检查
- 使用 `/health` 做就绪检查，使用 `/v1/models` 确认模型别名。
- 记录模型别名、设备、音频时长、响应格式、延迟和错误文本。
- 由于缓存 PVC 是 `ReadWriteOnce`，建议先从 1 个副本开始；横向扩容前先评估镜像、每 Pod 缓存或共享只读模型缓存方案。
- 服务暴露到可信 namespace 之外前，先加鉴权和网络策略。
- Dify、n8n 或 Web 后端在同一集群内访问时，应使用 Kubernetes service name，不要使用 `localhost`。
examples/openai_api/kubernetes/funasr-api.yaml
+114
View File
@@ -0,0 +1,114 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: funasr-cache
labels:
app.kubernetes.io/name: funasr-api
app.kubernetes.io/component: cache
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
---
apiVersion: v1
kind: ConfigMap
metadata:
name: funasr-api-config
labels:
app.kubernetes.io/name: funasr-api
app.kubernetes.io/component: api
data:
FUNASR_PORT: "8000"
FUNASR_DEVICE: "cpu"
FUNASR_MODEL: "sensevoice"
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: funasr-api
labels:
app.kubernetes.io/name: funasr-api
app.kubernetes.io/component: api
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: funasr-api
app.kubernetes.io/component: api
template:
metadata:
labels:
app.kubernetes.io/name: funasr-api
app.kubernetes.io/component: api
spec:
containers:
- name: funasr-api
image: funasr-api:local
imagePullPolicy: IfNotPresent
envFrom:
- configMapRef:
name: funasr-api-config
ports:
- name: http
containerPort: 8000
readinessProbe:
httpGet:
path: /health
port: http
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
livenessProbe:
httpGet:
path: /health
port: http
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 3
startupProbe:
httpGet:
path: /health
port: http
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 60
resources:
requests:
cpu: "2"
memory: 8Gi
limits:
memory: 16Gi
volumeMounts:
- name: cache
mountPath: /root/.cache
- name: dshm
mountPath: /dev/shm
volumes:
- name: cache
persistentVolumeClaim:
claimName: funasr-cache
- name: dshm
emptyDir:
medium: Memory
sizeLimit: 2Gi
---
apiVersion: v1
kind: Service
metadata:
name: funasr-api
labels:
app.kubernetes.io/name: funasr-api
app.kubernetes.io/component: api
spec:
type: ClusterIP
selector:
app.kubernetes.io/name: funasr-api
app.kubernetes.io/component: api
ports:
- name: http
port: 8000
targetPort: http
examples/openai_api/kubernetes/kustomization.yaml
+8
View File
@@ -0,0 +1,8 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- funasr-api.yaml
images:
- name: funasr-api
newName: registry.example.com/speech/funasr-api
newTag: cpu-latest
examples/openai_api/openapi.json
+374
View File
@@ -0,0 +1,374 @@
{
"openapi": "3.0.3",
"info": {
"title": "FunASR OpenAI-Compatible API",
"version": "1.0.0",
"description": "Static OpenAPI description for the FunASR OpenAI-compatible speech transcription server. The running FastAPI service also exposes Swagger UI at /docs and an OpenAPI document at /openapi.json."
},
"servers": [
{
"url": "http://localhost:8000",
"description": "Local development server"
},
{
"url": "http://funasr-api:8000",
"description": "Example Docker Compose or internal service name"
}
],
"tags": [
{
"name": "Transcription",
"description": "OpenAI-compatible speech transcription"
},
{
"name": "Models",
"description": "Model discovery"
},
{
"name": "Health",
"description": "Readiness and loaded model status"
}
],
"paths": {
"/v1/audio/transcriptions": {
"post": {
"tags": [
"Transcription"
],
"summary": "Transcribe an audio file",
"description": "Upload an audio file with multipart/form-data and receive an OpenAI-compatible transcript response. Set response_format=verbose_json to include segments and timing metadata.",
"operationId": "createTranscription",
"requestBody": {
"required": true,
"content": {
"multipart/form-data": {
"schema": {
"$ref": "#/components/schemas/TranscriptionRequest"
}
}
}
},
"responses": {
"200": {
"description": "Transcription result",
"content": {
"application/json": {
"schema": {
"oneOf": [
{
"$ref": "#/components/schemas/TranscriptionTextResponse"
},
{
"$ref": "#/components/schemas/VerboseTranscriptionResponse"
}
]
},
"examples": {
"json": {
"summary": "Default text response",
"value": {
"text": "Welcome to FunASR."
}
},
"verbose_json": {
"summary": "Verbose response",
"value": {
"text": "Welcome to FunASR.",
"segments": [
{
"start": 0.0,
"end": 1.6,
"text": "Welcome to FunASR.",
"speaker": null
}
],
"language": "auto",
"duration": 0.843,
"model": "sensevoice"
}
}
}
}
}
},
"400": {
"description": "Unknown model alias",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
},
"422": {
"description": "Invalid multipart request"
},
"500": {
"description": "Transcription runtime error",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
},
"/v1/models": {
"get": {
"tags": [
"Models"
],
"summary": "List model aliases",
"description": "Return OpenAI-style model objects for the aliases configured by the FunASR server.",
"operationId": "listModels",
"responses": {
"200": {
"description": "Model list",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ModelsResponse"
}
}
}
}
}
}
},
"/health": {
"get": {
"tags": [
"Health"
],
"summary": "Health check",
"description": "Check server readiness, selected device, loaded models, and available model aliases.",
"operationId": "getHealth",
"responses": {
"200": {
"description": "Server health",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/HealthResponse"
}
}
}
}
}
}
}
},
"components": {
"schemas": {
"TranscriptionRequest": {
"type": "object",
"required": [
"file"
],
"properties": {
"file": {
"type": "string",
"format": "binary",
"description": "Audio file such as wav, mp3, flac, m4a, ogg, or webm."
},
"model": {
"type": "string",
"enum": [
"sensevoice",
"paraformer",
"paraformer-en",
"fun-asr-nano"
],
"default": "sensevoice",
"description": "FunASR model alias."
},
"language": {
"type": "string",
"nullable": true,
"description": "Optional language hint passed to FunASR."
},
"response_format": {
"type": "string",
"enum": [
"json",
"verbose_json"
],
"default": "json",
"description": "Use verbose_json to include segments and timing metadata."
}
}
},
"TranscriptionTextResponse": {
"type": "object",
"required": [
"text"
],
"properties": {
"text": {
"type": "string",
"description": "Cleaned transcript text."
}
}
},
"VerboseTranscriptionResponse": {
"type": "object",
"required": [
"text",
"segments",
"language",
"duration",
"model"
],
"properties": {
"text": {
"type": "string"
},
"segments": {
"type": "array",
"items": {
"$ref": "#/components/schemas/TranscriptionSegment"
}
},
"language": {
"type": "string",
"example": "auto"
},
"duration": {
"type": "number",
"format": "float",
"description": "Server-side transcription latency in seconds."
},
"model": {
"type": "string",
"example": "sensevoice"
}
}
},
"TranscriptionSegment": {
"type": "object",
"required": [
"start",
"end",
"text"
],
"properties": {
"start": {
"type": "number",
"format": "float",
"description": "Segment start time in seconds."
},
"end": {
"type": "number",
"format": "float",
"description": "Segment end time in seconds."
},
"text": {
"type": "string"
},
"speaker": {
"type": "string",
"nullable": true,
"description": "Speaker label when available."
}
}
},
"ModelObject": {
"type": "object",
"required": [
"id",
"object",
"created",
"owned_by",
"ready"
],
"properties": {
"id": {
"type": "string",
"example": "sensevoice"
},
"object": {
"type": "string",
"example": "model"
},
"created": {
"type": "integer",
"example": 1700000000
},
"owned_by": {
"type": "string",
"example": "funasr"
},
"ready": {
"type": "boolean",
"description": "Whether this alias is already loaded in memory."
}
}
},
"ModelsResponse": {
"type": "object",
"required": [
"object",
"data"
],
"properties": {
"object": {
"type": "string",
"example": "list"
},
"data": {
"type": "array",
"items": {
"$ref": "#/components/schemas/ModelObject"
}
}
}
},
"HealthResponse": {
"type": "object",
"required": [
"status",
"device",
"models_loaded",
"models_available"
],
"properties": {
"status": {
"type": "string",
"example": "ok"
},
"device": {
"type": "string",
"example": "cuda"
},
"models_loaded": {
"type": "array",
"items": {
"type": "string"
}
},
"models_available": {
"type": "array",
"items": {
"type": "string"
}
}
}
},
"ErrorResponse": {
"type": "object",
"required": [
"detail"
],
"properties": {
"detail": {
"type": "string"
}
}
}
}
}
}
examples/openai_api/server.py
+208
View File
@@ -0,0 +1,208 @@
"""
FunASR OpenAI-Compatible API Server
Drop-in replacement for OpenAI's /v1/audio/transcriptions endpoint.
Works with any agent framework that supports OpenAI audio API.
Usage:
python server.py --model sensevoice --device cuda --port 8000
Then use with any OpenAI-compatible client:
curl http://localhost:8000/v1/audio/transcriptions \
-F file=@audio.wav -F model=sensevoice
"""
import argparse
import tempfile
import time
import os
import re
import logging
from typing import Optional
import uvicorn
from fastapi import FastAPI, UploadFile, File, Form, HTTPException
from fastapi.responses import JSONResponse
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
logger = logging.getLogger(__name__)
app = FastAPI(title="FunASR OpenAI-Compatible API", version="1.0.0")
MODEL_REGISTRY = {}
DEVICE = "cpu"
MODEL_CONFIGS = {
"sensevoice": {
"model": "iic/SenseVoiceSmall",
"vad_model": "fsmn-vad",
"vad_kwargs": {"max_single_segment_time": 30000},
},
"paraformer": {
"model": "paraformer-zh",
"vad_model": "fsmn-vad",
"punc_model": "ct-punc",
},
"paraformer-en": {
"model": "paraformer-en",
"vad_model": "fsmn-vad",
},
"fun-asr-nano": {
"model": "FunAudioLLM/Fun-ASR-Nano-2512",
"hub": "hf",
"trust_remote_code": True,
"vad_model": "fsmn-vad",
"vad_kwargs": {"max_single_segment_time": 30000},
},
}
def load_model(model_name: str):
"""Load a model and store in registry."""
if model_name in MODEL_REGISTRY:
return MODEL_REGISTRY[model_name]
if model_name not in MODEL_CONFIGS:
available = list(MODEL_CONFIGS.keys())
raise ValueError(f"Unknown model '{model_name}'. Available: {available}")
from funasr import AutoModel
cfg = MODEL_CONFIGS[model_name].copy()
cfg["device"] = DEVICE
cfg["disable_update"] = True
logger.info(f"Loading model '{model_name}' on {DEVICE}...")
t0 = time.time()
model = AutoModel(**cfg)
elapsed = time.time() - t0
logger.info(f"Model '{model_name}' loaded in {elapsed:.1f}s")
MODEL_REGISTRY[model_name] = model
return model
def clean_text(text: str) -> str:
"""Remove SenseVoice special tags from output."""
return re.sub(r'<\|[^|]*\|>', '', text).strip()
@app.post("/v1/audio/transcriptions")
async def transcribe(
file: UploadFile = File(...),
model: str = Form(default="sensevoice"),
language: Optional[str] = Form(default=None),
response_format: Optional[str] = Form(default="json"),
):
"""
OpenAI-compatible audio transcription endpoint.
Accepts the same parameters as OpenAI's /v1/audio/transcriptions:
- file: Audio file (wav, mp3, flac, m4a, ogg, webm)
- model: Model to use (sensevoice, paraformer, fun-asr-nano)
- language: Optional language hint
- response_format: json or verbose_json
"""
# Validate model
if model not in MODEL_CONFIGS:
raise HTTPException(
status_code=400,
detail=f"Model '{model}' not found. Available: {list(MODEL_CONFIGS.keys())}"
)
# Save uploaded file
suffix = os.path.splitext(file.filename)[1] if file.filename else ".wav"
with tempfile.NamedTemporaryFile(delete=False, suffix=suffix) as tmp:
content = await file.read()
tmp.write(content)
tmp_path = tmp.name
try:
asr_model = load_model(model)
t0 = time.time()
generate_kwargs = {"input": tmp_path, "batch_size": 1}
if language:
generate_kwargs["language"] = language
result = asr_model.generate(**generate_kwargs)
elapsed = time.time() - t0
text = clean_text(result[0]["text"])
if response_format == "verbose_json":
segments = []
if "sentence_info" in result[0]:
for seg in result[0]["sentence_info"]:
segments.append({
"start": seg.get("start", 0) / 1000.0,
"end": seg.get("end", 0) / 1000.0,
"text": clean_text(seg.get("text", "")),
"speaker": seg.get("spk", None),
})
return JSONResponse({
"text": text,
"segments": segments,
"language": language or "auto",
"duration": round(elapsed, 3),
"model": model,
})
else:
return JSONResponse({"text": text})
except Exception as e:
logger.error(f"Transcription error: {e}")
raise HTTPException(status_code=500, detail=str(e))
finally:
os.unlink(tmp_path)
@app.get("/v1/models")
async def list_models():
"""List available models (OpenAI-compatible)."""
models = []
for name in MODEL_CONFIGS:
models.append({
"id": name,
"object": "model",
"created": 1700000000,
"owned_by": "funasr",
"ready": name in MODEL_REGISTRY,
})
return JSONResponse({"object": "list", "data": models})
@app.get("/health")
async def health():
"""Health check endpoint."""
return {
"status": "ok",
"device": DEVICE,
"models_loaded": list(MODEL_REGISTRY.keys()),
"models_available": list(MODEL_CONFIGS.keys()),
}
def main():
parser = argparse.ArgumentParser(description="FunASR OpenAI-Compatible API Server")
parser.add_argument("--host", default="0.0.0.0", help="Bind host")
parser.add_argument("--port", type=int, default=8000, help="Bind port")
parser.add_argument("--device", default="cuda", help="Device: cuda, cpu, mps")
parser.add_argument("--model", default="sensevoice", help="Pre-load model at startup")
args = parser.parse_args()
global DEVICE
DEVICE = args.device
# Pre-load default model
load_model(args.model)
logger.info(f"FunASR API server starting on http://{args.host}:{args.port}")
logger.info(f" Device: {DEVICE}")
logger.info(f" Models: {list(MODEL_CONFIGS.keys())}")
logger.info(f" Docs: http://{args.host}:{args.port}/docs")
uvicorn.run(app, host=args.host, port=args.port)
if __name__ == "__main__":
main()
examples/openai_api/smoke_test.py
Executable
+113
View File
@@ -0,0 +1,113 @@
#!/usr/bin/env python3
"""Cross-platform smoke test for the FunASR OpenAI-compatible API."""
from __future__ import annotations
import argparse
import json
import mimetypes
import os
from pathlib import Path
import sys
import urllib.error
import urllib.request
import uuid
SAMPLE_URL = "https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/BAC009S0764W0121.wav"
def request_json(url: str, timeout: float) -> dict:
request = urllib.request.Request(url, headers={"Accept": "application/json"})
with urllib.request.urlopen(request, timeout=timeout) as response:
return json.loads(response.read().decode("utf-8"))
def download_if_needed(path: Path, sample_url: str, timeout: float) -> None:
if path.exists():
return
print(f"Downloading sample audio to {path}")
with urllib.request.urlopen(sample_url, timeout=timeout) as response:
path.write_bytes(response.read())
def multipart_body(audio_path: Path, model: str, response_format: str) -> tuple[bytes, str]:
boundary = f"----funasr-smoke-{uuid.uuid4().hex}"
content_type = mimetypes.guess_type(str(audio_path))[0] or "application/octet-stream"
parts: list[bytes] = []
def add_text(name: str, value: str) -> None:
parts.append(
(
f"--{boundary}\r\n"
f"Content-Disposition: form-data; name=\"{name}\"\r\n\r\n"
f"{value}\r\n"
).encode("utf-8")
)
parts.append(
(
f"--{boundary}\r\n"
f"Content-Disposition: form-data; name=\"file\"; filename=\"{audio_path.name}\"\r\n"
f"Content-Type: {content_type}\r\n\r\n"
).encode("utf-8")
)
parts.append(audio_path.read_bytes())
parts.append(b"\r\n")
add_text("model", model)
add_text("response_format", response_format)
parts.append(f"--{boundary}--\r\n".encode("utf-8"))
return b"".join(parts), boundary
def transcribe(base_url: str, audio_path: Path, model: str, response_format: str, timeout: float) -> dict:
body, boundary = multipart_body(audio_path, model, response_format)
request = urllib.request.Request(
f"{base_url}/v1/audio/transcriptions",
data=body,
method="POST",
headers={
"Accept": "application/json",
"Content-Type": f"multipart/form-data; boundary={boundary}",
"Content-Length": str(len(body)),
},
)
with urllib.request.urlopen(request, timeout=timeout) as response:
return json.loads(response.read().decode("utf-8"))
def print_json(title: str, payload: dict) -> None:
print(f"\n== {title} ==")
print(json.dumps(payload, ensure_ascii=False, indent=2))
def main() -> int:
parser = argparse.ArgumentParser(description="Smoke test the FunASR OpenAI-compatible API")
parser.add_argument("audio_path", nargs="?", default="sample.wav", help="Audio file to transcribe; downloads a sample if missing")
parser.add_argument("--base-url", default=os.getenv("BASE_URL", "http://localhost:8000"), help="FunASR API base URL")
parser.add_argument("--model", default=os.getenv("MODEL", "sensevoice"), help="Model alias to use")
parser.add_argument("--response-format", default=os.getenv("RESPONSE_FORMAT", "verbose_json"), choices=["json", "verbose_json"], help="Transcription response format")
parser.add_argument("--sample-url", default=os.getenv("SAMPLE_URL", SAMPLE_URL), help="Sample audio URL used when audio_path does not exist")
parser.add_argument("--timeout", type=float, default=float(os.getenv("TIMEOUT", "300")), help="HTTP timeout in seconds")
args = parser.parse_args()
base_url = args.base_url.rstrip("/")
audio_path = Path(args.audio_path)
try:
download_if_needed(audio_path, args.sample_url, args.timeout)
print_json("health", request_json(f"{base_url}/health", args.timeout))
print_json("models", request_json(f"{base_url}/v1/models", args.timeout))
print(f"\nTranscribing {audio_path} with model={args.model}, response_format={args.response_format}")
print_json("transcription", transcribe(base_url, audio_path, args.model, args.response_format, args.timeout))
except urllib.error.HTTPError as error:
detail = error.read().decode("utf-8", errors="replace")
print(f"HTTP {error.code} from {error.url}: {detail}", file=sys.stderr)
return 1
except Exception as error:
print(f"Smoke test failed: {error}", file=sys.stderr)
return 1
return 0
if __name__ == "__main__":
raise SystemExit(main())
examples/openai_api/smoke_test.sh
Executable
+24
View File
@@ -0,0 +1,24 @@
#!/usr/bin/env bash
set -euo pipefail
BASE_URL="${BASE_URL:-http://localhost:8000}"
MODEL="${MODEL:-sensevoice}"
RESPONSE_FORMAT="${RESPONSE_FORMAT:-verbose_json}"
AUDIO_PATH="${1:-sample.wav}"
SAMPLE_URL="${SAMPLE_URL:-https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/BAC009S0764W0121.wav}"
if [ ! -f "$AUDIO_PATH" ]; then
echo "Downloading sample audio to $AUDIO_PATH"
curl -L "$SAMPLE_URL" -o "$AUDIO_PATH"
fi
echo "Checking $BASE_URL/health"
curl -fsS "$BASE_URL/health"
printf '\n\n'
echo "Transcribing $AUDIO_PATH with model=$MODEL"
curl -fsS "$BASE_URL/v1/audio/transcriptions" \
-F "file=@${AUDIO_PATH}" \
-F "model=${MODEL}" \
-F "response_format=${RESPONSE_FORMAT}"
printf '\n'